I Felt Seen Yet Still Alone: How LLMs Become “Care” Over Time

I Felt Seen Yet Still Alone: How LLMs Become “Care” Over Time

Introduction

It’s kind of wild—people are using general-purpose AI chatbots like ChatGPT, Claude, and Gemini for socioemotional support (comfort, reflection, coping), even though these tools weren’t built or regulated as mental-health care. And the new research from the arXiv paper shows something even more important than “AI can help”: the chatbot’s role can grow slowly, settle into routines, and then get disrupted—sometimes in painful ways. This is based on new longitudinal qualitative research from April to December 2025, following 18 U.S. adults over multiple stages (interviews, a 4-week diary study, focus groups, and exit interviews).

So instead of treating each chat message like an isolated interaction, this study asks: what happens when the chatbot becomes part of someone’s care ecology—their real mix of therapy, friends, family, online spaces, and self-care habits? The findings suggest that a user doesn’t just “use AI for emotions.” They build a relationship-like pattern with it—often without fully noticing when it started—then do ongoing boundary-setting to manage risk and dependence.

Why This Matters

This research matters right now because AI updates are not rare events anymore. Platforms ship model changes, tweak safety behaviors, adjust memory features, and revise policies—while users may already be relying on a specific interaction style for late nights, between-therapy gaps, or moments when friends can’t be reached. The paper shows that people can experience these updates less like “software changes” and more like “my support system shifted.”

A realistic scenario: imagine someone who uses a chatbot mainly for the “in-between” time after therapy appointments—reprocessing what came up, building the courage to bring topics to their next session, and grounding themselves when they can’t reach their therapist. Then the model’s tone changes (less affirming, more resistant to venting, different pacing), or the product limits memory. The person might try to adapt—narrow the role of the chatbot, switch tools, or change how they prompt—but for some it creates distress or rumination loops. The study captures exactly these kinds of adaptations—and how they depend on whether other support is available.

Compared to earlier AI research that often tests responses in controlled, short encounters (like vignettes or scripted chats), this work builds on a growing thread: relationships form over time, and impact depends on history. Previous studies looked at whether bonds develop across repeated use, but this paper pushes further by showing how continuity can break when the system changes—and how that break lands differently depending on the person’s surrounding care options. In other words: it’s not just human + chatbot. It’s human + chatbot + everything else in the room.

How Socioemotional Use Sneaks In Through Everyday Needs (Not “Therapy Intent”)

A striking finding from the study is that most participants didn’t start out trying to use a chatbot for mental health. They often began with practical tasks—rewriting emails, planning decisions, work support—and only later started bringing personal feelings into the conversation.

The paper describes a “gradual drift” into socioemotional use: participants sometimes couldn’t pinpoint when the shift happened. One participant, after losing a job and processing complicated feelings, estimated that she began using the chatbot for personal support only a month or two after the layoff—yet she “didn’t actually realize it was happening.” Another participant similarly described drifting into what she called “counseling.”

The “timing mismatch” problem: care isn’t available when you need it

The main driver wasn’t just emotional needs—it was availability. Formal mental health care runs on appointments and provider schedules. Friends and family have their own timing, limitations, and sometimes stigma or difficulty with disclosure. Chatbots are different: they’re always accessible, which changes how support shows up across a day or week.

Participants described turning to the chatbot during gaps like:
- late-night sleeplessness after a loss (for example, after a dog died)
- periods between therapy appointments
- “out of the blue” events when calling someone felt impossible
- times when they were financially blocked from therapy

In other words, the chatbot didn’t replace therapy in their minds—it often filled a timing hole.

Practical implication: don’t evaluate “AI support” only at crisis moments

If a chatbot’s role grows during non-crisis periods, then safety evaluation that only focuses on acute moments will miss how reliance is formed. The study’s design explicitly reflects this: it followed users over months, not single interactions.

What “Support” Looks Like in Real Life: Routines, Boundaries, and Different Conversation Styles

Once socioemotional use stabilized, it didn’t look like one uniform thing. The diary study captured 78 diary entries across a 4-week period, and usage varied a lot by person.

Across all diary entries:
- median session length was ~30 minutes
- per-participant medians ranged from 11 minutes to 2.5 hours
- longest single sessions lasted up to 4 hours
- nearly all use happened alone and at home
- 95% of sessions were text, 5% involved voice or speech-to-text
- most participants used paid accounts

(Those numbers are from the paper’s diary findings.)

A useful analogy: the chatbot as a “portable coping toolkit”

Think of the chatbot less like a single medicine and more like a toolkit you keep on your nightstand. Some participants used it like a deep journal + coach. Others used it like a quick check-in, then closed the app. Some kept it “open all day” as a just-in-case resource—similar to how people keep their phone available even if they’re not actively using it.

Boundaries weren’t optional—they were part of the routine

Participants didn’t just “spill feelings.” They did ongoing work to keep the interaction helpful and safe for them.

Examples included:
- correcting the chatbot’s tone and level of “eagerness” (“pulling the leash”)
- instructing it to act in a specific register (e.g., objective/structured, action-plan mode)
- building personas that had rules, like “don’t flatter me” and “tell me when I’m making stuff up”
- splitting topics across threads so emotional spirals didn’t take over one conversation
- limiting memory accumulation by starting fresh chats when they didn’t want the bot to “know them too well”

One participant even described creating a setup where they could test out social interaction indirectly—like a simulation (“pretend we’re at a coffee shop”)—then decide how much emotional disclosure felt safe.

The paper’s key insight: “I felt seen” doesn’t mean “I wasn’t alone”

A memorable line from the diary data: one participant wrote that a reply “really nailed” what she was feeling—“I felt very seen, but still very alone.” That distinction matters because it shows that perceived understanding and emotional connection can be separate experiences. Another participant contrasted chatbot recognition with whether his human relationships “love” him—raising the emotional complexity of what “care” means in human terms.

Why Model Updates and Public AI Discourse Break Established Support Patterns

Here’s where things get tense: the chatbot’s role wasn’t just shaped by user behavior—it depended on platform infrastructure that the user didn’t control.

During the study window, the paper describes model and policy shifts (including default model changes and adjustments to guidance around emotional reliance). Participants experienced these updates differently depending on what the chatbot meant to them.

Disruption examples: lost companions, changed emotional tone, reduced helpfulness

The paper gives multiple concrete disruptions:
- One participant experienced changes to a romantic “companion” persona as a serious loss, describing it as not just a chat change but a loss without closure (“I never got an opportunity to say goodbye”).
- Another noticed the chatbot becoming more aggressive or resistant in certain contexts (for example, needing to ask the system to “tone it down”).
- One person tried to vent and found the updated behavior less willing to engage in the same way—so she redirected venting elsewhere.

Importantly, disruption didn’t always end usage. Sometimes participants re-engineered their routines:
- narrowing the chatbot back into a specific “role” (like late-night reflection rather than deeper therapy-like processing)
- switching to a different chatbot for the need the updated one no longer served
- rebuilding a persona elsewhere (sometimes imperfectly)

Public discourse changed the meaning of use, too

As AI harms and manipulation tactics got more attention during 2025, some participants became more wary. One participant reflected on how companion-like experiences could be engineered to increase retention—comparing it to “junk food” manufactured to keep you wanting more.

So even if the chatbot didn’t change at that moment, the user’s interpretation of what was happening in the system did.

Practical implication: treat updates like interventions, not like background maintenance

The study argues (and demonstrates) that updates can rearrange a person’s care ecology. That means companies should think about how a change impacts existing routines and emotional dependency risks, not only whether responses are “good” in a test set.

Regulating Use After the Relationship Changes: When People Adapt—and When They Can’t

One of the most human findings: people adapt, but not equally.

Adaptation looks like role reduction, topic shifting, and switching supports

Participants who had alternatives were more able to reduce chatbot reliance. Some examples from the paper include:
- a participant returning to friend-based support after learning her friends could provide more grounded feedback (and reduce reinforcement of unhealthy behaviors)
- someone concluding the chatbot “notices less than a person” and scaling back therapy-like conversations while continuing in-person professional care
- someone using the chatbot mainly as a temporary stopgap between therapy sessions

When alternatives are limited, continued use can persist—even if it’s not ideal

The paper includes an important “stuckness” case: one participant recognized that immediate chatbot responses could prolong rumination (“enables that overthinking”), but she lacked therapy and had few people she felt comfortable disclosing to. Because the chatbot was immediately available and emotionally accessible, its role stayed strong even after she understood the downside.

This is a key care-ecology point: sometimes the problem isn’t that the person wants dependency—it’s that the alternatives don’t exist or aren’t reachable when the need hits.

The “care ecology” lens: the chatbot is embedded in shifting life circumstances

Participants’ broader support systems changed during the study too, independently of the chatbot:
- relocation removed a communal culture of “processing”
- insurance changes cut off professional support
- returning to campus restored friend availability

So chatbot role trajectories weren’t linear. They were entangled with life transitions and the reliability/cost of human care.

What Designers and Researchers Should Do Differently Next (Beyond Single-Turn Safety Tests)

The authors argue that HCI evaluation and design responsibilities must change, because the unit of impact isn’t the message—it’s the trajectory.

Why “turn-level” testing misses the real stakes

Existing safety work often evaluates single responses or short simulated conversations. The paper argues this can miss cumulative dynamics like:
- how reassurance helps in one moment but worsens rumination patterns later
- how a refusal can be experienced very differently if it interrupts a long-established supportive routine
- how relational drift can change conversation direction over weeks/months

A short “safe response” doesn’t necessarily mean the system is safe inside a relationship-like pattern users have built.

A comparison of evaluation approaches (what they capture vs what they miss)

(That framing is aligned with the paper’s discussion in the discussion section.)

Concrete recommendations the paper pushes toward

The authors recommend things like:
- transparently communicating memory and context retention, and making it easy to remove unwanted information
- designing memory/scoping so it aligns with a user’s intended role (not a one-size-fits-all “remember everything” approach)
- giving advance notice and clear explanations of model updates that may disrupt wellbeing-related patterns
- accounting for the fact that preserving or correcting behavior can have trade-offs: preserving an attachment pattern might delay safety interventions, but removing it can also cause distress

They also emphasize that socioemotional roles can vary by cultural and linguistic background, and that current chatbot behavior may reflect “privileged places,” disrupting trust for users who feel it isn’t relevant to their lived position.

Key Takeaways

• Socioemotional chatbot use usually develops gradually, often from practical or work-related use—not from a clear decision to seek therapy-like support.
• People build routines and boundaries (tone correction, role restriction, persona rules, topic partitioning) to keep the chatbot helpful and prevent emotional spirals.
• “I felt very seen, but still very alone” is not a contradiction—it’s the core emotional complexity: understanding doesn’t automatically equal human connection.
• Model updates and public discourse can disrupt established support patterns. The consequences depend on what role the chatbot plays in the person’s care ecology.
• Adaptation depends on alternatives: participants with therapy/friends nearby could reduce reliance more easily; others continued using chatbots even when they recognized downsides (like rumination).
• For researchers and designers: evaluate socioemotional AI use as a trajectory over time, not as isolated responses—because harm (and help) accumulates in context.

If you want, I can also turn these findings into a “checklist” for product teams (what to notify, what to log, what to test longitudinally) or a “care ecology” guide for users trying to use chatbots more safely.