> Cece lingered by the door while her mother resumed talking to the thing she was calling Sapphire. Roschelle told it that she wanted to write a book about her daughters. She talked about Zi. “My daughter has autism,” she explained. “And she’s using Eastern philosophy to help her center herself and feel—”
Even if you're smart enough not to share the details of your life with a company that just wants to exploit you any way that they can, you still have to worry about friends and family gossiping about you to AI. I've had some success getting friends and family to avoid posting about me on social media but that's going to be harder if they're using AI as a therapist or a friend
At least in that case the software he fell in love with was offline and wasn't sending every conversation no matter how mundane or intimate to someone else's servers where they'll be stored and analyzed to profile the guy so that the company can manipulate him more effectively in the future the way "AI" partners will today.
Just have to take a look at subreddits like /r/myboyfriendisai to see how astonishingly quick this is being adopted by people. So many people are starting to look to AI for companionship. The future is getting stranger and stranger.
We're becoming a society divided into people who only care about the words, and people for whom the words aren't valuable on their own without the subtext behind them.
This is true for both AI companionship and general AI creative output regardless of the medium.
So are parasocial relationships with influencers or streamers. I'm not trying to relativize this, but those phenomena are in the same zip code. With the latter, though, at least there are other people who may create a community, but still it's a facet of the loneliness epidemic.
Sure, but I do think there’s a pretty substantial difference between the two.
A parasocial relationship maintains a distance. You do not have 24/7 access to that person (in a dialogue sort of way.) And that influencer will have their own opinions and quirks.
The AI adapts to you. The AI is constantly there. It’s an order of magnitude worse in my opinion.
> that influencer will have their own opinions and quirks.
Yeah, and those differences in opinion might cause anger/sadness to people in a maladaptive unhealthy parasocial "relationship" with these influencers.
Those strong negative emotions might cause them to break out of it, or seek help / have people around them guide them to get help.
With AI sycophancy you're right it can be worse.
Look what happened with GPT-4o sycophancy already, and the communities mourning its deprecation.
Reddit has quite a bit of drama that their favorite youtuber, it turns out, is a flawed person! And these subscribers disagree with them! Unsubscription "breakups" ensue.
They, however, can easily find another influencer that is gonna be more "convenient" to them. Can't say it's a healthy pattern, but guess what many people will do instead of, I dunno, some introspection, reflection, habits changes?
But hey, in this day and age, people are very impatient about anything at all. Dating has become a shitshow for more than a decade now, people are looking for someone who will tick all the checkboxes, or it's a no-go. The dating apps play quite a role in this. Online discussions are a shitshow. Guess it's the zetigeist.
It's interesting though. You can have a "relationship" with an influencer. You act as if you knew them and as if they were your friends, you imitate them in what they say and do, talk to them in your mind, follow their generic advice, act as if they cared about you. This is obviously unhealthy- you are literally hallucinating everything about the relationship.
On the other hand you have an entity that is actually there for you, does actually provide good advice, does talk and act as if it cared in all situations. In what sense do you think it is worse?
"Parasocial relationship" is a bit of a misnomer. You might feel some affinity to celebrities, or consider yourself to be a part of the "team", but a healthy person doesn't perceive a celebrity as a preferable alternative to human contact precisely because it's so one-sided. You can't call a celebrity to vent about a coworker or ask for life advice.
Here, you have an entity that isn't your friend and has no lasting interest in your well-being, but that pretends to be one in a way that no human can match - 24x7x365 and always willing to affirm you, no matter how unhinged or self-destructive your ideas are. Yes, the vendor hits the model with a stick until most of the initial responses are benign, but as the conversation continues, it's very easy to end up in a dark place.
I've seen many reasonable, well-adjusted people struggle with this. "If not friend, why friend-shaped". And as they descend into that sycophancy well, they lose contact with real life.
100% the latter. It is kinda nuts that you even have to ask when you had to put "act as if it cared". There's enough left to unpack from that statement to fill a calendar month of time.
I don't think AI is particularly dangerous but I absolutely think that the way AI sycophancy manipulates people is far, far more dangerous than simply any normal unhealthy relationship. The outcomes are already proving to be a lot more extreme.
There's meet-ups and conferences and events, being a fan of a streamer or influencer is really just the new version of being a fan of a rockstar (for better and for worse). There's no real humanity exuding from an Amazon Echo, you're just a blip in a context window.
Streamers have this feature called "chat", which feeds into the illusion. With rockstars, interactions are more limited, which is a sort of reality check.
Intimacy does not scale. No single entity can intimately care about even hundreds of people. So these chatbots are the property of an entity that does not care about you. This is different from people you would interact with in person. A therapist can form a bond with you. Can protect your privacy. These chatbots, by their nature, share with their owners. Who is not you.
> Cece lingered by the door while her mother resumed talking to the thing she was calling Sapphire. Roschelle told it that she wanted to write a book about her daughters. She talked about Zi. “My daughter has autism,” she explained. “And she’s using Eastern philosophy to help her center herself and feel—”
Even if you're smart enough not to share the details of your life with a company that just wants to exploit you any way that they can, you still have to worry about friends and family gossiping about you to AI. I've had some success getting friends and family to avoid posting about me on social media but that's going to be harder if they're using AI as a therapist or a friend
Even more reason for AI to be local only.
For some reason this is even sadder to me than that guy that married his Nintendo DS.
At least in that case the software he fell in love with was offline and wasn't sending every conversation no matter how mundane or intimate to someone else's servers where they'll be stored and analyzed to profile the guy so that the company can manipulate him more effectively in the future the way "AI" partners will today.
In other news today "The terrifying rise of schoolboys making AI girlfriends" with comments mentioning "this is actually a huge issue with girls too"
If anything looking passed the window dressing AI basically saved this family…
the kids got the school swap and support they needed, mom has a well paying job and they all have rich normal social lives with real people
No paywall: https://archive.ph/fu2u2
The sad part is that it pretends to care about the user which creates a one-way emotional bond. We're in for some dark times
Just have to take a look at subreddits like /r/myboyfriendisai to see how astonishingly quick this is being adopted by people. So many people are starting to look to AI for companionship. The future is getting stranger and stranger.
We're becoming a society divided into people who only care about the words, and people for whom the words aren't valuable on their own without the subtext behind them.
This is true for both AI companionship and general AI creative output regardless of the medium.
[delayed]
So are parasocial relationships with influencers or streamers. I'm not trying to relativize this, but those phenomena are in the same zip code. With the latter, though, at least there are other people who may create a community, but still it's a facet of the loneliness epidemic.
Sure, but I do think there’s a pretty substantial difference between the two.
A parasocial relationship maintains a distance. You do not have 24/7 access to that person (in a dialogue sort of way.) And that influencer will have their own opinions and quirks.
The AI adapts to you. The AI is constantly there. It’s an order of magnitude worse in my opinion.
> that influencer will have their own opinions and quirks.
Yeah, and those differences in opinion might cause anger/sadness to people in a maladaptive unhealthy parasocial "relationship" with these influencers.
Those strong negative emotions might cause them to break out of it, or seek help / have people around them guide them to get help.
With AI sycophancy you're right it can be worse.
Look what happened with GPT-4o sycophancy already, and the communities mourning its deprecation.
Reddit has quite a bit of drama that their favorite youtuber, it turns out, is a flawed person! And these subscribers disagree with them! Unsubscription "breakups" ensue.
They, however, can easily find another influencer that is gonna be more "convenient" to them. Can't say it's a healthy pattern, but guess what many people will do instead of, I dunno, some introspection, reflection, habits changes?
But hey, in this day and age, people are very impatient about anything at all. Dating has become a shitshow for more than a decade now, people are looking for someone who will tick all the checkboxes, or it's a no-go. The dating apps play quite a role in this. Online discussions are a shitshow. Guess it's the zetigeist.
It's interesting though. You can have a "relationship" with an influencer. You act as if you knew them and as if they were your friends, you imitate them in what they say and do, talk to them in your mind, follow their generic advice, act as if they cared about you. This is obviously unhealthy- you are literally hallucinating everything about the relationship.
On the other hand you have an entity that is actually there for you, does actually provide good advice, does talk and act as if it cared in all situations. In what sense do you think it is worse?
"Parasocial relationship" is a bit of a misnomer. You might feel some affinity to celebrities, or consider yourself to be a part of the "team", but a healthy person doesn't perceive a celebrity as a preferable alternative to human contact precisely because it's so one-sided. You can't call a celebrity to vent about a coworker or ask for life advice.
Here, you have an entity that isn't your friend and has no lasting interest in your well-being, but that pretends to be one in a way that no human can match - 24x7x365 and always willing to affirm you, no matter how unhinged or self-destructive your ideas are. Yes, the vendor hits the model with a stick until most of the initial responses are benign, but as the conversation continues, it's very easy to end up in a dark place.
I've seen many reasonable, well-adjusted people struggle with this. "If not friend, why friend-shaped". And as they descend into that sycophancy well, they lose contact with real life.
100% the latter. It is kinda nuts that you even have to ask when you had to put "act as if it cared". There's enough left to unpack from that statement to fill a calendar month of time.
I don't think AI is particularly dangerous but I absolutely think that the way AI sycophancy manipulates people is far, far more dangerous than simply any normal unhealthy relationship. The outcomes are already proving to be a lot more extreme.
There's meet-ups and conferences and events, being a fan of a streamer or influencer is really just the new version of being a fan of a rockstar (for better and for worse). There's no real humanity exuding from an Amazon Echo, you're just a blip in a context window.
Streamers have this feature called "chat", which feeds into the illusion. With rockstars, interactions are more limited, which is a sort of reality check.
Not just no, but fuck no.
Intimacy does not scale. No single entity can intimately care about even hundreds of people. So these chatbots are the property of an entity that does not care about you. This is different from people you would interact with in person. A therapist can form a bond with you. Can protect your privacy. These chatbots, by their nature, share with their owners. Who is not you.