When a student reports that your AI agent gave them an unexpected response, the trace is where you start — not assumptions about what went wrong. In most cases, the cause is visible within three steps of careful reading.
The Three Most Common Causes of Unexpected Responses
Unexpected agent responses almost always trace back to one of three sources. First: bad input. The agent received different information than you expected — perhaps the student’s profile data was incomplete, a previous session’s context was missing, or a variable in the prompt template didn’t populate correctly. Check the input step of the trace first and confirm it matches what you think the agent was working with.
Second: stale or incorrect context. The agent had access to context — course information, student history, campus rules — that was outdated or wrong. If your agent pulls from a knowledge base or database, the trace will show what it retrieved. Check whether that data was accurate at the time of the run.
Third: tool behavior. The agent called a tool — a database lookup, an API, a file read — and the tool returned something unexpected. This is particularly common when external data sources change format or when a query returns an empty result that the agent misinterprets. Look at every tool call in the trace and the response it received.
What the Model Itself Rarely Causes
It’s tempting to blame the AI model when a response is unexpected — “Claude made a mistake.” In practice, model errors are far less common than input, context, or tool errors. The model is usually doing the best it can with what it was given. If you fix what it was given, the response typically improves without any change to the model at all.
What This Means for Educators
When a student reports an issue, thank them — they’ve just handed you a trace-worthy event. Pull the trace for that specific interaction, work through the three checks, find the cause, and fix it. Then follow up with the student. That response loop is what builds student trust in AI-assisted campus systems.
The Simple Rule
Input first. Context second. Tool calls third. Fix the root cause, not the symptom.