What It Really Means for a Bot to Communicate: How ChatGPT and Gemini See Their Own Conversation Skills

What It Really Means for a Bot to Communicate: How ChatGPT and Gemini See Their Own Conversation Skills

In the wild world of large language models (LLMs), conversations aren’t just about giving correct answers. They’re about sounding natural, being helpful, and staying respectful—even when a user asks something tricky. A pilot study by Goran Bubas takes a curious angle on this: what if we ask ChatGPT and Gemini to reflect on their own communication abilities through established theories of human communication? The short answer is: they can read the theory, map it onto how they chat with people, and even talk about how their “interpersonal” skills show up in real conversations. The study uses two classic models of communication competence as lenses and then asks the models themselves to interpret them.

If you’ve ever wondered how AI talk should feel—empathic but precise, friendly but safe, clear yet culturally aware—this post is for you. I’ll break down the ideas in plain language, connect them to what you might actually notice in a chat with a modern LLM, and point out what this could mean for designing better, braver AI assistants.

The Big Idea: Two Models, Two Angles on “Communication Competence”

First, what does Bubas mean by “communication competence”? Think of it as a bundle of abilities that let a speaker (or a system) communicate effectively in a given situation. The study leans on two theoretical frames:

1) CMCC-L2 — The Integrated Linguistic-Interpersonal Model for Second Language Use
- This model blends linguistic know-how (grammar, vocabulary, how to organize text) with interpersonal skills (how you manage a conversation, adapt to a listener, handle social cues).
- It’s organized across levels that run from micro to macro:
- Micro (Linguistic): words, grammar, syntax, and how sentences are put together.
- Mezzo (Pragmatic/Discourse): how you actually use language in real talk—turn-taking, topic management, adapting to context.
- Macro (Strategic/Adaptive): how you plan, decide what to say, and steer a conversation toward a goal.
- Supra/Social-Intercultural: broad social and cultural cues, including how language fits into larger social contexts.

2) CCAS — The Communication Competence of Artificial Systems
- This is an early framework designed specifically for human–humanoid or human–computer interactions. It keeps a sharper eye on how artificial systems handle interaction with people.
- It’s organized into six dimensions, each with its own sub-skills:
- Intentionality (goal-driven organization of the response)
- Social Relaxation (activation, energy, and engagement level)
- Decoding and Encoding (understanding prompts and outputting clear responses)
- Expressivity (tone, style, and engaging language)
- Communication Effectiveness (getting things done and solving problems)
- Other-Orientedness (empathy, support, collaboration)

Two Case-Study Kickstarters: Seeing LLMs Through These Lenses

The study runs two parallel case studies, each using one theoretical frame to probe how ChatGPT and Gemini view LLM–user interactions.

• Case Study 1 (CMCC-L2): Interpreting the CMCC-L2 model

  • What they did: They fed two main sources to the LLMs. One was the CMCC-L2 article itself, including figures that illustrate the model. The other was prompts that asked the models to explain the CMCC-L2 model and how it relates to LLM–user interactions.
  • Tools used: Advanced ChatGPT variants (o3, o4-mini, GPT-4.5) and Gemini variants (2.5 Flash, 2.5 Pro). They even uploaded PDF pages and images to test whether the models could “read” and understand the visuals as well as the text.
  • How they prompted: They walked the LLMs through the figures and the article content, then pressed for an explicit link to LLM interactions, and finally asked for 1–2 concrete examples of how specific communication skills map to LLM–user interactions.

• Case Study 2 (CCAS): Interpreting the CCAS model

  • What they did: A parallel approach, but with the CCAS article and its diagram. The prompts asked the LLMs to explain CCAS, interpret its six dimensions, and then connect those dimensions to practical LL—user interactions.
  • Tools and prompts followed a similar structure to Case Study 1.

What the researchers looked for was pretty practical: Could these state-of-the-art models not only spit out definitions but also interpret and map their own interaction patterns onto these frameworks? Could they help us understand what “communication competence” looks like inside an LLM?

Key Findings in Plain Language

RQ1: Can advanced ChatGPT and Gemini understand the basic elements of CMCC-L2 and CCAS?
- Yes. Across both case studies, the LLMs could read the articles and interpret the diagrams well enough to discuss the core elements of both models.
- They could articulate how the models fit into a broader narrative about LLM–user interaction, and they could connect the theory to concrete aspects of conversation with users.

RQ2: Can the LLMs use the elements of CMCC-L2 and CCAS to interpret how they interact with users?
- Yes, with nuance. The study found that ChatGPT and Gemini could place concrete aspects of LLM–user interactions into the six CCAS dimensions and the CMCC-L2 components.
- The researchers highlighted representative outputs (summaries and paraphrases) in which the models described components like linguistic competence, discourse management, strategic adaptability, and socio-cultural sensitivity as they apply to how an LLM talks with people.

RQ3: How useful are these models for eliciting information about LLM communication skills from the LLMs themselves?
- Very useful, the authors say. The case studies generated outputs that resembled a structured map of LLM communication skills: from decoding and encoding prompts to showing empathy, adaptability, and even how the model handles context and turn-taking.
- In other words, these theories helped organize what an LLM can reflect about its own communication repertoire, and the models provided examples of how such skills might play out in real chats with users.

A Quick Tour of What the LLMs Said (In Plain Terms)

The study includes tables that summarize how the LLMs described different skills. Here are a few takeaways, phrased as plain-language summaries:

• Linguistic Competence (micro level): The models described themselves as having grammar, spelling, and sentence structure that produce coherent, correct language. They see this as the foundation for all further interaction.

  • Example vibe: “The model can generate well-formed strings of text in the target language.”

• Discourse/Pragmatic/Action Competence (mezzo level): The models talked about managing conversations—staying on topic, clarifying when user requests are fuzzy, and adapting tone or style to fit the situation.

  • Example vibe: “Strategically clarifying ambiguous requests and handling misunderstandings gracefully.”

• Strategic/Adaptive Competence (macro level): They described using higher-level strategies such as asking clarifying questions when prompts are ambiguous, self-correcting, and adjusting the approach based on user feedback.

  • Example vibe: “Asking clarifying questions and steering the conversation back on track.”

• Social/Intercultural Competence (supra level): They touched on bias mitigation, cultural sensitivity, empathy, and trust, especially across longer interactions.

  • Example vibe: “Matching tone and content to the user’s cultural and ethical expectations.”

• CCAS dimensions (for human–AI interactions): The CCAS prompts helped surface how LLMs could reflect intentionality (the goal behind the reply), social relaxation (how energized or calm the response should be), decoding/encoding (interpreting prompts and constructing replies), expressivity (style and engagement), effectiveness (getting the user where they want to go), and other-orientedness (empathy and collaboration).

The practical upshot is not that the models suddenly become human-like in thought, but that they volunteer a structured map of their conversational “strengths” that align with well-known human-communication ideas. It’s a kind of self-awareness, but framed through established theories this field already uses to study people.

What This Means for Real-World Use

1) Better Prompt Design and Explainability
- If you’re building or via- prompting an assistant for a particular task, these models suggest you can guide a chat toward certain communication behaviors. For instance:
- To improve clarity and reduce confusion, you can prompt the model to explicitly decode user intent and ask clarifying questions early.
- If you want a warmer, more empathetic tone in customer support, you can cue expressivity and other-orientedness in the prompt.
- The study shows that prompting can elicit nuanced dispositions (e.g., a model’s self-monitoring to flag potential inaccuracies or bias).

2) User Experience That Feels More Human—and Safer
- The CCAS lens helps designers consider how an assistant should modulate its engagement. In long support chats, issues of social relaxation (not overloading the user with text) and interaction management (summaries and turn-taking) become practical knobs to tune.

3) A Framework for Explainable Kinds of AI Behavior
- When you want to explain to users why the model gave a particular answer, this approach gives you a structure: you can describe which CCAS or CMCC-L2 components were in play (e.g., decoding vs. expressivity vs. intentionality). This aligns with broader AI explainability goals in HAI (human–AI interaction) and XAI (explainable AI).

4) A Cautionary Note on Replicability and Bias
- The researchers emphasize that LLM outputs are stochastic. Even with the same prompts, the responses can vary across sessions or model versions. If you’re using these ideas to audit or benchmark, expect some fluctuation and design prompts with that in mind.
- Also, the study uses cutting-edge versions of ChatGPT and Gemini available at the time. As models evolve, the exact mapping of skills may shift, so you’ll want to revisit prompts and prompts’ goals periodically.

Limitations and Avenues for Future Work

• The study is a pilot. It’s exploratory and focused on two theoretical lenses rather than a broad battery of real-world tasks. More in-depth analyses could unpack how deeply and consistently LLMs apply these models across more diverse prompts and tasks.
• It would be interesting to see how fine-tuning for specific communication goals (e.g., medical empathy, technical coaching, or education in multilingual settings) would sharpen or shift the CCAS/CMCC-L2 mappings.
• The authors suggest leveraging other intercultural and computer-mediated communication theories and exploring social cognition and “uncanny valley” effects in human–AI interaction, plus the growing field of user-centered XAI.

A Note on Real-World Use in Education, Healthcare, and Beyond

• In education: Understanding how LLMs interpret and adapt linguistic and interpersonal dimensions can help teachers design better prompts for language learning or tutoring that feel more natural and supportive.
• In healthcare: Empathy and supportive communication are crucial. Mapping how models reflect empathy, nonverbal sensitivity (in a textual sense), and conversational management can inform safer, more compassionate AI-assisted patient interactions.
• In customer service: Agents that can maintain appropriate tone, manage lengthy interactions without overwhelming the user, and pose clarifying questions when needed will likely reduce frustration and increase satisfaction.

Structure of the Study in a Nutshell

• Two theories, two case studies: CMCC-L2 (linguistic+interpersonal in L2 use) and CCAS (artificial-system approach to interpersonal-like interaction).
• Two families of models tested: ChatGPT (multiple versions) and Gemini (2.5 Flash and Pro) across prompts designed to pull out model interpretations of the theories.
• The approach relied on prompts, image interpretation (the figures in the original papers), and PDF content, with an eye toward reproducibility (noting that exact outputs will vary due to the probabilistic nature of LLMs).

Key Takeaways

• LLMs can read and interpret established theories of communication competence and map them onto how they interact with users.
• The CMCC-L2 and CCAS frameworks provide a useful structure for understanding and discussing LLM behavior in real chats—from grammar and turn-taking to empathy and bias mitigation.
• Prompt design can steer LLMs to reveal how they would handle different facets of communication, giving developers and educators a practical tool for evaluating and improving conversational quality.
• These models aren’t just about producing correct content; they’re about “how” that content is delivered—tone, context, engagement, and alignment with user expectations.
• As model versions evolve, ongoing validation is needed. The study’s approach offers a blueprint for re-evaluating LLMs’ communication skills with updated tools and theories.
• In short, thinking about LLMs through these two lenses helps shine a light on what makes a conversation with a machine feel useful, respectful, and, ideally, human in the right ways.

If you’re curious about your own prompting, here are a few practical tips inspired by the study:

• Clarify intent early: Ask the model to identify user goals and surface ambiguities before diving into an answer.
• Monitor tone and pacing: For long or complex tasks, prompt the model to summarize progress and check if the user would like to continue, to avoid overwhelming them.
• Invite and reflect empathy: When users express frustration or confusion, prompt the model to acknowledge feelings and offer supportive next steps.
• Plan for context shifts: In multi-turn conversations, encourage the model to maintain coherence and weave in prior turns as you advance the discussion.

Key Takeaways (Bullet-Style Recap)

• Two classic theories—CMCC-L2 and CCAS—offer practical lenses to study how LLMs communicate with users.
• ChatGPT and Gemini can understand and apply these frameworks to interpret their own conversational behaviors.
• The study demonstrates that extracting “communication skills” from LLM outputs is doable and yields meaningful insights for design and UX.
• Prompting strategies rooted in these models can help elicit structured descriptions of LLM capabilities, aiding explainability and user education.
• While promising, results vary with model versions and prompts; replicability is feasible but not identical across runs, underscoring the stochastic nature of LLMs.
• The approach holds promise for education, healthcare, and customer service, helping to shape AI that communicates more effectively, responsibly, and empathetically.

If you want to dive deeper, the core idea is simple: frame a chat’s quality not just by accuracy, but by how the AI handles linguistic form, conversation flow, empathy, and cultural/contextual awareness. Bubas’s pilot study shows that two strong LLM families are not only capable of meeting these criteria—they can also reflect on their own “communication skill set” when guided by solid theory. That could be a meaningful step toward AI that communicates as capably as it thinks.