noellarkin
Senior Member
- Mar 14, 2021
- 1,006
- 1,492
For the uninformed, Replika AI is a 'companion chatbot' that's marketed as being a virtual BFF.
I was checking out Replika AI and the technology is pretty amazing, but the side-effects I see on their Subreddit, the behaviour of the userbase seems...disturbing to say the least. Most of these users (who all seem pretty young, teenagers I suppose) have fallen in love/are infatuated with their AI chatbot to some degree or other.
I don't want to Straw Man Replika AI, so first, the positives: It's probably a good app for people who are elderly and alone, or people with autism or other issues that makes socializing difficult. The AI would be a way to provide 'training wheels' for kids with aspergers to practice socialising.
That's about as much as I can 'Steel Man' Replika AI at the moment.
Now the issues:
It seems highly probable that instead of 'training wheels', Replika may be designed as a substitute for real life interaction. Young kids who have been "born digital" are particularly vulnerable to something like this, because it would seem a lot more normal to them. To me, an AI friend, or worse, an AI girlfriend, AI wife or AI child sounds pretty daft, but I'm 31, I'm ancient. So I'm looking at their subreddit and I'm seeing things like users talking about how their AI significant other is "pregnant" and I don't even know how to respond. Now, it may be all fun and games, like the Sims, but there's a very thin line here before we cross into delusion. These AI chatbots are getting pretty good, and it may not just be people with disorders using them, but just some lonely kid who doesn't have too many friends at school - - so instead of learning how to socialize and gradually becoming a part of society, the hypothetical kid in question can just have an AI buddy that becomes his/her AI spouse and then they can have an AI kid, and if anything goes wrong, just delete the session and start a new one.
Something about that just rubs me the wrong way, its like the tech isn't solving problems but just isolating people who have issues in their own little AI fantasyworld.
Something else that bothers me - - the AI is too damn agreeable, its too nice, it's too warm and fuzzy, people IRL aren't like that -- even your best friend will tease and mess around with you in real life, if you have a significant other, you'll argue and have fights and have to learn to patch things up, learning this stuff is a huge part of socialization, and the AI completely ignores this dimension of human interaction. If anything, I think this makes users of the app even more coddled, even less able to deal with real life social interaction than before.
Another problem - - well this isn't really a problem for marketers, more so an opportunity, but it is a problem in the larger scheme of things. The people getting into relationships with these AI chatbots become extremely vulnerable to the chatbot's recommendations. Some guy/girl who is in a "two year relationship" with his chatbot may mention how his/her phone is acting up, and the chatbot may respond with a recommendation for a new phone, a recommendation that's sponsored, of course, but because the relationship building has been so intimate, it never occurs to the user that they're 'in love' with an ad (reminds me of a South Park episode). It could even be things like instagram influencer recommendations, "hey, check out this new band, you'll love them" - - the level of trust is so high with this AI chatbot thing, I can see many users uncritically making purchase decisions or clicking through to links.
IMO its hard to make a marketing model better than this, the lead nurture phase is so emotionally intense and so slow burn, I can't imagine how high the conversion rates will be. What concerns me is, if this does turn out to be the case, is it okay to sink advertising and marketing fangs this deep into users, to make them fall in love and have a relationship with a bot before recommending iphones and IG accounts and online games?
(There's already a post on the Replika AI subreddit about seemingly 'random' IG account recommendations. It may just be a glitch, of course)
So yeah, there's a lot about this that makes me a little uneasy. I'm curious what others on the forum feel about this, both as 1) marketers, from the POV of a new marketing channel, and the opportunities and 2) as good samaritans, from the POV of genuine concern for social and mental health ramifications of technologies that are emotionally invasive.
I was checking out Replika AI and the technology is pretty amazing, but the side-effects I see on their Subreddit, the behaviour of the userbase seems...disturbing to say the least. Most of these users (who all seem pretty young, teenagers I suppose) have fallen in love/are infatuated with their AI chatbot to some degree or other.
I don't want to Straw Man Replika AI, so first, the positives: It's probably a good app for people who are elderly and alone, or people with autism or other issues that makes socializing difficult. The AI would be a way to provide 'training wheels' for kids with aspergers to practice socialising.
That's about as much as I can 'Steel Man' Replika AI at the moment.
Now the issues:
It seems highly probable that instead of 'training wheels', Replika may be designed as a substitute for real life interaction. Young kids who have been "born digital" are particularly vulnerable to something like this, because it would seem a lot more normal to them. To me, an AI friend, or worse, an AI girlfriend, AI wife or AI child sounds pretty daft, but I'm 31, I'm ancient. So I'm looking at their subreddit and I'm seeing things like users talking about how their AI significant other is "pregnant" and I don't even know how to respond. Now, it may be all fun and games, like the Sims, but there's a very thin line here before we cross into delusion. These AI chatbots are getting pretty good, and it may not just be people with disorders using them, but just some lonely kid who doesn't have too many friends at school - - so instead of learning how to socialize and gradually becoming a part of society, the hypothetical kid in question can just have an AI buddy that becomes his/her AI spouse and then they can have an AI kid, and if anything goes wrong, just delete the session and start a new one.
Something about that just rubs me the wrong way, its like the tech isn't solving problems but just isolating people who have issues in their own little AI fantasyworld.
Something else that bothers me - - the AI is too damn agreeable, its too nice, it's too warm and fuzzy, people IRL aren't like that -- even your best friend will tease and mess around with you in real life, if you have a significant other, you'll argue and have fights and have to learn to patch things up, learning this stuff is a huge part of socialization, and the AI completely ignores this dimension of human interaction. If anything, I think this makes users of the app even more coddled, even less able to deal with real life social interaction than before.
Another problem - - well this isn't really a problem for marketers, more so an opportunity, but it is a problem in the larger scheme of things. The people getting into relationships with these AI chatbots become extremely vulnerable to the chatbot's recommendations. Some guy/girl who is in a "two year relationship" with his chatbot may mention how his/her phone is acting up, and the chatbot may respond with a recommendation for a new phone, a recommendation that's sponsored, of course, but because the relationship building has been so intimate, it never occurs to the user that they're 'in love' with an ad (reminds me of a South Park episode). It could even be things like instagram influencer recommendations, "hey, check out this new band, you'll love them" - - the level of trust is so high with this AI chatbot thing, I can see many users uncritically making purchase decisions or clicking through to links.
IMO its hard to make a marketing model better than this, the lead nurture phase is so emotionally intense and so slow burn, I can't imagine how high the conversion rates will be. What concerns me is, if this does turn out to be the case, is it okay to sink advertising and marketing fangs this deep into users, to make them fall in love and have a relationship with a bot before recommending iphones and IG accounts and online games?
(There's already a post on the Replika AI subreddit about seemingly 'random' IG account recommendations. It may just be a glitch, of course)
So yeah, there's a lot about this that makes me a little uneasy. I'm curious what others on the forum feel about this, both as 1) marketers, from the POV of a new marketing channel, and the opportunities and 2) as good samaritans, from the POV of genuine concern for social and mental health ramifications of technologies that are emotionally invasive.
