Student-ChatGPT Trace Dashboard for EFL Writing Teachers: PAD

Student-ChatGPT Trace Dashboard for EFL Writing Teachers: PAD

Introduction

If you’ve ever taught (or supported teaching) English writing with ChatGPT in the loop, you already know the annoying part: students’ AI help is happening in messy, hard-to-scan ways. It’s not just “did they use it?”—it’s how they used it, when they used it, and whether it helped them revise meaningfully or simply replaced thinking.

A new research effort from the original paper introduces a teacher-focused tool called the Prompt Analytics Dashboard (PAD). The big idea is straightforward but powerful: teachers shouldn’t have to manually hunt through chat logs and draft histories. PAD makes student–ChatGPT interactions visible and interpretable—so instructors can spot patterns, understand what’s going on, and intervene at the right moment in EFL writing classes.

This work is based on new research described in the paper, and it’s built through two iterative co-design sessions with six EFL instructors. Rather than treating AI traces as “surveillance data,” the researchers designed the dashboard around teacher needs: quick overview first, deeper evidence on demand, and careful explanations to avoid creating a climate of over-monitoring.

Why This Matters

This research lands at the exact moment when many EFL programs are stuck in a frustrating cycle: schools want to allow AI for feedback and brainstorming, but teachers don’t have time to interpret the resulting interaction data. PAD matters because it pushes learning analytics beyond “scores” and into actionable visibility. Most AI-in-education tools either (1) generate feedback directly or (2) report performance outcomes. PAD instead helps you answer the pedagogical question: Was the student’s AI use aligned with learning goals—and did it lead to better writing decisions?

Here’s a realistic scenario you could apply today: imagine a university writing course where students submit weekly essays. A teacher notices that one cohort’s final drafts are improving, but another cohort’s writing looks strangely “flat” or repetitive. With PAD-style visibility, the teacher could quickly compare patterns: which students are asking for full paragraph generation without revision attempts, which are using the model to negotiate meaning or evaluate drafts, and which students are producing high-effort revision history after AI prompts. That means interventions become targeted—less “everyone, stop using ChatGPT” and more “you’re using it in a way that’s currently not strengthening your revision skills.”

Compared to earlier AI research in education, this paper builds on a key shift: from “AI is available” to “AI interactions can be analyzed in a teacher-readable way.” Prior work often focuses on how LLMs can support learning or on risks like plagiarism and bias. PAD’s distinctive contribution is translating raw prompt-response traces into a compact, teacher-usable set of signals—so instructors can make instructional choices without becoming data analysts. And importantly, misuse signals are treated as teacher-defined patterns that may pose pedagogical risks, not automatic proof of misconduct.

How PAD Turns Chat Logs into Teacher-Readable Signals

PAD is designed for EFL writing education, where student drafts and revisions tell a story—but ChatGPT interactions add an extra layer to that story. The researchers built PAD around the reality that teachers need two things at the same time:
1. A quick, high-level sense of what’s happening across the class, and
2. A way to zoom in to specific evidence when something looks off or promising.

A useful analogy is airplane dashboards. A pilot doesn’t stare at individual engine readings all day. Instead, they watch a few critical indicators and alarms, then zoom into the underlying system only when needed. PAD follows a similar “overview → evidence” logic so teachers aren’t buried in chat transcript scrolling.

What PAD connects in one place

The dashboard is integrated into a broader system where students:
- write weekly essays through a student-side application, and
- interact with ChatGPT while drafting and revising,
- with essays and chat logs then analyzed by LLM-based components.

So PAD isn’t just “a chat viewer.” It’s a pipeline that links prompt activity to essay revision behavior, and then organizes that into teacher-facing views.

The Co-Design Findings: What Teachers Actually Wanted

The backbone of PAD isn’t just technical design—it’s centered on how instructors work under real constraints. In the first phase, the team gathered insight from six university EFL instructors using ChatGPT in writing courses. They used surveys and interviews, and they also observed how teachers reviewed drafts and chat logs to judge learning effectiveness.

From this phase, the researchers distilled practical design goals:
1. Quick overview of student–ChatGPT interactions
2. Identifying common patterns and undesirable usage
3. Supporting in-depth analysis of editing and chat logs
4. Customizing prompt instructions for a personalized learning environment

In the second phase, teachers used a prototype PAD during co-design and gave feedback via semi-structured interviews. Thematic analysis produced four implementation implications:
1. Simplifying analytic interpretations on general statistics (so teachers don’t need to “decode” charts)
2. Improving browsing efficiency for log review
3. Enhancing targeted feedback ability for instructors
4. Allowing flexible, teacher-driven customization of what the dashboard emphasizes

This matters because many dashboards fail when they assume teachers want more information rather than better interpretations. PAD is explicitly built to respect teacher time and cognition.

The Prompt Analytics Dashboard UI: Overview, Filters, and Drill-Down Evidence

PAD’s interface is built around a clear interaction pattern: “overview–zoom–filter–details-on-demand.” If you’ve ever tried to teach with limited time, you’ll recognize why this is essential: teachers need patterns first, but they also need proof before they make judgment calls.

A practical walkthrough of the three main interface views

PAD includes the following structural components (based on the described prototype):

Population pool + narrowing controls

Teachers can choose the “population pool” to analyze—like selecting specific classes, students, and weeks. This is a subtle but important design choice: it turns “the whole class” from an overwhelming blob into a manageable slice.

Overview view: weekly signals at a glance

PAD shows an overview chart with weekly x-axis fixed, and includes things like:
- chat count over time,
- the last saved essay score graded by AI,
- and the misuse count.

This is where teachers can spot timing issues fast—like “this week we see a sudden jump in prompt activity but no corresponding revision improvement,” or “this cohort’s misuse-like patterns are spiking.”

Tabs for patterns, objective filtering, and teacher messaging

PAD’s overview area includes tabs that support:
- chatting patterns,
- additional filtering based on learning objectives,
- and the ability to deliver messages to students.

That last part—teacher messaging—is a big deal for real classroom use. Visibility without the ability to act can still leave teachers feeling stuck.

Drill-down view: evidence for revision histories and chat content

Teachers can track essay changes by moving sliders. And the dashboard can display the full chat history, with:
- search,
- and tags per prompt so teachers can quickly locate relevant interactions without manually scanning every line.

The taxonomy behind the scenes: what PAD labels and why

Under the UI, PAD builds a compact trace taxonomy distilled from co-design sessions. The paper describes it as three core categories:
1. Misuse signals
2. Goal-alignment cues
3. Revision effort

A crucial nuance: misuse does not mean definitive academic misconduct. It refers to teacher-defined patterns of AI use that may raise pedagogical risks and therefore warrant closer attention. That framing matters ethically and instructionally—it helps prevent the dashboard from becoming a “gotcha tool.”

How PAD Detects Patterns: Misuse, Alignment, and Scoring Components

PAD’s interpretation pipeline relies on LLM-based components analyzing both student essays and chat logs. The architecture described in the paper includes three main LLM-driven pieces:
1. A GPT-4-based component that detects potential misuse signals
2. An automated essay scoring model (referenced as Han & Yoo, 2023 in the system)
3. A GPT-4 classifier that categorizes prompts by learning objectives

Misuse signals: not “wrong,” but “risky patterns”

The GPT-4 misuse component looks for things like:
- requests for generation without revision, and
- chat patterns such as negotiations or asking for essay evaluation.

The educational logic is: if students are repeatedly asking the model to generate content without actually revising and integrating that content into their own draft process, then the learning benefit may shrink. PAD flags these patterns so the teacher can intervene with coaching on revision strategies and goal-aligned use of AI.

Goal-alignment cues: connecting prompts to learning objectives

Another GPT-4 component classifies prompts according to learning objectives. The value here is that teachers don’t just want “activity monitoring.” They want to know whether prompts match what the course is trying to teach—like argument structure, vocabulary usage, coherence, or other objective-driven skills.

Essay scoring: pairing interaction visibility with writing outcomes

PAD also shows AI-graded essay scores (at least the last saved score in the overview). That helps teachers interpret whether changes in prompt behavior correspond to changes in writing performance and revision outcomes.

Comparison: what each analytic component contributes

(These components are described as part of PAD’s architecture in the paper.)

And yes—PAD is still careful about interpretation: the “misuse” output is positioned as a prompt for closer checking rather than a final verdict about misconduct.

What Teachers Reported After Using PAD

Design claims matter, but teacher feedback is the acid test. The paper reports that instructors found PAD helpful for two major reasons:

1. Reduced scanning burden
   Instead of manually digging through chat logs and draft histories, teachers could use summarized signals and filters.

2. Clearer timing for interventions
   By seeing patterns week-by-week and drilling down into evidence, teachers could decide when to talk to students, what to ask about, and which behaviors needed attention.

This is consistent with the co-design findings: teachers wanted interpretability, efficiency, and personalized feedback. PAD’s overview-and-evidence layout is basically an answer to those needs.

Design Choice That Keeps PAD from Feeling Like Over-Surveillance

One subtle but important element is the paper’s emphasis on micro-explanations attached to summarized signals. The system is meant to reduce “over-surveillance” feelings by avoiding opaque analytics. If a dashboard shows counts without explaining what they mean or why a signal is raised, teachers (and students) can interpret it as a monitoring tool rather than a learning support tool.

PAD’s approach tries to keep the framing educational: it summarizes potential risks and alignment, and then gives teachers evidence snippets to check, contextualize, and respond.

In other words, it supports a teaching relationship more than a policing relationship—at least by design.

Where PAD Could Go Next in Real Classrooms

The paper closes with future work directions—scalability, real-time feedback, and adaptive analytics. Those aren’t just technical wishes; they connect directly to day-to-day classroom realities.

Scalability

If a dashboard only works for small classes, teachers can’t depend on it. Scaling matters if schools want consistent AI integration across multiple writing sections.

Real-time feedback

Right now, weekly essay workflows are natural, but real-time signals would let teachers intervene earlier—like during a drafting phase—when students are still deciding how to use AI.

Adaptive analytics

Adaptive analytics could mean the system learns from teacher preferences. For example, one instructor might care more about revision effort signals, while another prioritizes alignment to specific objectives in that week’s writing module.

If implemented responsibly, these steps could turn PAD from a “post-hoc investigation tool” into a continuous teaching companion.

Key Takeaways

• PAD (Prompt Analytics Dashboard) is a teacher-centered system that visualizes student–ChatGPT interactions in EFL writing classes, alongside essay revision history.
• The work was built through two iterative co-design sessions with six EFL instructors, resulting in a dashboard that prioritizes overview + efficient drill-down rather than endless transcript browsing.
• PAD uses a compact trace taxonomy with three categories: misuse signals, goal-alignment cues, and revision effort—and importantly, misuse is not an automatic misconduct verdict.
• The dashboard includes three interface views: an overview with weekly trends, filters for learning objectives, and a drill-down mode showing chat history and revision evidence on demand.
• The underlying analytics pipeline combines GPT-4-based misuse detection and objective classification, plus an existing AI essay scoring model, linking interaction patterns to writing outcomes.
• Teachers reported reduced scanning burden and clearer timing for interventions, making it easier to provide targeted, pedagogical feedback.
• The system’s use of micro-explanations aims to keep the tool educational rather than overly surveillant—supporting responsible, teacher-guided AI use.

If you’re an EFL teacher, instructional designer, or learning analytics person, this paper is a strong example of the “missing middle” problem: not whether AI can help students, but whether teachers can interpret AI-assisted learning well enough to guide it. PAD is basically a step toward that capability.