Debugging agents: when something goes wrong

Most blog posts about AI agents focus on the happy path. Set up, run, save time, profit.

Reality: agents fail. Sometimes spectacularly, sometimes subtly. The teams that succeed with agents are the ones that get good at debugging when things go wrong.

This post covers our framework for debugging agent issues — both the systematic process and the specific failure modes.

The general framework

When something seems wrong with an agent, work through these questions in order:

1. Is the agent actually doing what it's configured to do?

Check the audit ledger. What did the agent see? What did it decide? What did it execute?

This is the first question because often the "agent problem" is actually a "configuration mismatch" — the agent is doing exactly what it's configured to do, and the configuration is wrong.

2. Is the output quality below expectation?

If the agent is doing what it's configured to do but the quality is wrong, the issue is likely: → Prompt needs refinement → Model not well-suited to the task → Context windows insufficient → Knowledge base out of date

3. Is the agent failing at the system level?

If the agent is throwing errors or timing out, the issue is likely: → Connector authentication broken → External service rate limiting → Resource exhaustion → Network/connectivity issues

4. Is the failure novel or recurring?

One-off failures usually resolve. Recurring failures need root cause analysis.

5. What changed?

If the agent was working and now isn't, something changed. External API. Configuration. Model. Input pattern. Find what.

The framework: start with the ledger, narrow down, find the change, fix.

Specific failure modes and their debugging signatures

We covered failure modes in [post 179]. Here are the debugging signatures for each:

Failure mode: Silent quality drift.

Signature: gradual decline in output ratings over time. Most outputs still acceptable. Pattern only visible across many samples.

Debug: pull 100 outputs from last week, 100 from 3 months ago. Compare quality ratings. Identify where drift started.

Fix: usually requires prompt refinement, context update, or knowledge base refresh.

Failure mode: Confident wrongness.

Signature: agent's confidence score is high, but output is incorrect on close inspection.

Debug: sample high-confidence outputs and verify accuracy. If high-confidence false positive rate is elevated, the confidence threshold is miscalibrated.

Fix: lower confidence threshold for autonomous action. Add verification steps for important claims.

Failure mode: Cascading misclassification.

Signature: downstream outputs look wrong, but each step "seems right" given its inputs.

Debug: trace backward through the audit ledger. Find the step where the wrong path got taken.

Fix: usually requires classification step refinement or addition of validation between steps.

Failure mode: Configuration drift.

Signature: agent that worked starts producing odd outputs after external system changes.

Debug: check what external systems the agent depends on. What changed? New fields, deprecated endpoints, schema changes.

Fix: update connector configuration, refresh references, adjust prompts.

Failure mode: The cost surprise (cloud-LLM platforms).

Signature: bill spikes unexpectedly.

Debug: identify which agent had volume spike. Was it expected? Is the volume legitimate?

Fix: add throttling, optimize prompts for token efficiency, consider migrating to local-first.

Failure mode: Feature creep.

Signature: one agent does many things, all of them mediocrely.

Debug: review the agent's actual workflow. How many distinct functions does it serve?

Fix: split into multiple specialized agents.

Failure mode: Unreviewed exception.

Signature: agent's fallback path is firing more than expected.

Debug: query audit ledger for fallback executions. Read what triggered each.

Fix: usually the main path needs refinement to handle previously-unseen cases.

Failure mode: Missed context.

Signature: agent's output is technically correct but inappropriate given broader context.

Debug: review what context the agent had access to. What broader context exists outside what the agent sees?

Fix: add context to the agent's input, OR add review gate for high-stakes outputs.

Failure mode: Privacy leak.

Signature: PII appearing in outputs where it shouldn't.

Debug: trace which data sources feed into outputs. Where's the leakage path?

Fix: add redaction at ingestion, restrict data sources, configure PII detection.

Failure mode: Just worse than before.

Signature: customers/colleagues prefer the pre-agent process.

Debug: this isn't really a debug — it's a question. Does the agent fit the work?

Fix: sometimes the answer is to retire the agent.

Tools for debugging in Avery NXR specifically

The platform provides debugging capabilities:

Audit ledger query. Filter executions by date, agent, outcome, confidence level. Find specific failures.

Step-level execution traces. For each execution, see what each step did. Identify which step failed or produced bad output.

Sample outputs. Pull recent outputs for any agent. Quality-rate them. Find patterns.

Connector health dashboard. See which connectors are healthy, which have auth issues, which have rate limit issues.

Confidence distribution. See how confidence levels are distributed across recent outputs. Identify miscalibration.

Re-run capability. Re-run a specific execution with the same inputs to test fixes.

These tools mean debugging Avery NXR agents is grep-and-investigate, not magic.

The "agent debugging session" pattern

Effective debugging sessions follow a pattern:

→ Reproduce the issue. Can you trigger it again? If not, gather more samples until you can.

→ Isolate the failing step. Which specific step in the workflow is wrong?

→ Examine the inputs to that step. Does the agent have what it needs?

→ Examine the output of that step. Is it what you'd expect?

→ Hypothesize the cause. What could explain the discrepancy?

→ Test the hypothesis. Make a change, re-run, verify.

→ If fix works, deploy. If not, refine hypothesis.

This pattern is unromantic. It's also how real debugging happens.

When NOT to debug

Some failures should result in retirement, not debugging:

→ Agent has been drifting for months. Better to retire and rebuild than to patch.

→ Underlying workflow changed. If the business process is now different, the old agent may not fit. Retire.

→ Cost of debugging exceeds value of fixing. Some agents weren't earning their keep. Don't spend hours debugging marginal value.

→ Maintainer left. Sometimes the knowledge of why the agent was configured a certain way is lost. Better to start over with current understanding.

Knowing when to retire is part of operational maturity.

Common debugging mistakes

Mistake 1: Blaming the model.

Often the issue is configuration, not the model. Try refining your prompt or context before switching models.

Mistake 2: Not checking the audit ledger first.

The audit ledger is your debugging tool. Use it before guessing.

Mistake 3: Trying to debug under pressure.

If something is on fire, stabilize first (drop the agent down the trust ladder, switch to manual handling). Debug after pressure is off.

Mistake 4: Not documenting fixes.

When you fix something, write down what you learned. Future you (or your teammate) will benefit.

Mistake 5: Working in isolation.

Some debugging benefits from a second perspective. Ask a colleague to look. Often they see things you've missed.

The debugging culture we recommend

For teams running multiple agents, build debugging norms:

→ Weekly debugging time. Schedule 30-60 min/week for one person to debug across all agents.

→ Document patterns. When you see a failure mode, write it up. Build internal knowledge.

→ Share learnings. Across teams, share what debugging patterns work.

→ Improve templates from debug findings. When you find that a template needs better defaults, update the template.

→ Train new team members. Debugging is a learnable skill. Train people explicitly.

The teams that do this well have agents that survive year 1 at higher rates than the 41% average we covered in [post 215].

What we do in Avery NXR specifically to make debugging better

Product features that exist for debugging:

→ Comprehensive audit ledger with rich querying → Step-level execution traces → Confidence scoring on outputs → Connector health monitoring → Re-run capability for testing → Configuration versioning (track changes over time) → Sandbox mode for testing changes

These features exist because we know debugging matters. They're not flashy. They're necessary.

What we're building next for debugging

Features in active development:

→ Automated regression detection (alert when output quality drifts) → Better visualization of multi-step workflows → Faster sample output review interface → Suggested fixes based on common patterns → Bias monitoring dashboards

We'll keep investing here. Debugging tooling determines whether agents are durable.

The principle

Agents fail. Get good at debugging.

The teams that get value from agents long-term aren't the ones whose agents never fail. They're the ones who fail, debug, fix, and improve faster than the failures accumulate.

Build the muscle. Use the tools. Document the patterns. Train the team.

This is unsexy work. It's also where durable AI operations live.

→ avery.software — Free Desktop tier. The platform built for teams who debug agents systematically.