Context Engineering vs Prompt Engineering: How to Diagnose Agent Failures
Prompt engineering shapes the instructions. Context engineering manages everything an agent sees. A method to diagnose which context factor broke an API integration task, with a worksheet and four worked cases.
- Written by
- Puneet Arora (opens in a new tab)
- Published on
- Read time
- 13 min
An agent fails to integrate your API. Someone rewrites the prompt, reruns it, and it passes. A week later it fails the same way, and nobody can say whether the cause was the instruction, an outdated docs page, a noisy tool result, or a tool the agent did not know how to use.
Prompt engineering is writing and organizing the instructions. Context engineering is managing everything the agent sees when it acts: instructions, message history, retrieved evidence, tool definitions and results, reference material, and state. Anthropic defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference," and prompt engineering as "methods for writing and organizing LLM instructions for optimal outcomes."
The two overlap, and neither replaces the other. The wider view pays off when an agent fails: hold the model and task fixed, change one context factor, and judge the result against traces from repeated runs. That procedure is this article's own method, assembled from Anthropic's evals guidance, Google's agent framework design, and practitioner advice on replaying traces.
TL;DR
- Prompt engineering shapes the instructions. Context engineering manages everything the agent sees, instructions included.
- Diagnose a failure by holding the model and task fixed and changing one context factor at a time.
- Define success criteria before you run, keep every trace, and run several trials in clean environments.
- Rule out model limits, permissions, runtime errors, and task ambiguity before you blame the context.
What the agent sees when it acts
Anthropic's docs say "Everything in the request counts toward the context window: the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions." LangChain groups context into instructions, knowledge, and tools. Anthropic's Claude 5 guidance, published July 24, 2026, adds: "Unlike a prompt, context is used generally across many requests, so it cannot be as specific."
That gives six components to inspect:
- Instructions: system prompt, task text, rules files.
- Messages and history: earlier turns and their summaries.
- Retrieved evidence: docs pages and search results pulled in during the task.
- Tool descriptions and tool results.
- Reference material: specs, examples, and changelogs supplied up front.
- Relevant state: memory, the repository, and anything carried between steps.
More context can lower accuracy
The same Anthropic page lists context windows of up to 1M tokens, depending on the model, then says: "A larger context window allows the model to handle more complex and lengthy prompts, but more context isn't automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available."
Hangfei Lin, a Google tech lead writing about the ADK context stack on the Google Developers Blog, agrees: "A large context window will help context-related problems but won't address all context-related problems." He warns that a window "flooded with irrelevant logs... can distract the model, causing it to fixate on past patterns." Building context through named, ordered processors "makes the 'compilation' step observable and testable."
In July 2025, Chroma tested 18 earlier-generation LLMs and found performance "grows increasingly unreliable as input length grows." Extra tokens help when the agent lacked something. They hurt when they add noise or contradiction. Ask six questions about what was in the window:
- Selection: did the right items get in?
- Freshness: are they current for the API version?
- Relevance: does each item bear on this task?
- Sufficiency: is anything missing?
- Conflicts: do two items disagree?
- Timing: did each item arrive when the agent needed it?
Placement advice depends on the model. For Gemini, Google's long context docs say that in most cases, especially with long context, performance is better "if you put your query / question at the end of the prompt (after all the other context)." For GPT-4.1, OpenAI's prompting guide says that with long context you should "ideally place your instructions at both the beginning and end of the provided context." Anthropic's Claude 5 guidance notes that earlier Claude models could sometimes be "more likely to listen to instructions at the end of their context window than at the start." Check the advice for the model version you pin.
Is this prompt engineering with a new name?
On Hacker News, mentalgear wrote: "Whether they call it now 'prompt' or 'context' engineering because it's the same tinkering to find something that 'sticks' in non-deterministic space." DAIR.ai, a supporter, writes that prompt engineering "is so important that it is now being rebranded as context engineering."
Part of this is fair. The principles carry over to a wider scope. Anthropic's Claude 5 guidance replaces "repeat yourself" with "simple tool descriptions" and reports: "We removed over 80% of Claude Code's system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations." That is a vendor-reported internal result. The post does not describe its method, and it is scoped to Claude Code and specific models.
That concession covers instruction scaffolding on specific models. It says nothing about a stale docs page, two sources that contradict each other, or a noisy tool result. In the same thread, 0xdeafcafe wrote that "a lot of 'agent failures' trace back to assumptions, bad agent-decisions, or bloat buried in the context." Instructions still matter. So do the docs, sources, and tool results the agent reads.
The running example: a hypothetical webhook task
Every example from here on is hypothetical and was not executed. Say your company ships ExamplePay, a fictional payments API. Its v3 release renamed the webhook signature header from X-ExamplePay-Signature to ExamplePay-Signature and deprecated the verify_v2() helper. In your own evals, you ask an agent to add ExamplePay signature verification to a sample webhook handler, the way a customer's coding agent would.
Before the first run, write success criteria you can check from the trace: the tests in tests/webhooks pass; the handler reads ExamplePay-Signature; the diff has no verify_v2() call; the agent runs the tests before it reports done.
Keep the full transcript of every run. Anthropic's evals guide defines it as "The complete record of a trial, including outputs, tool calls, reasoning, intermediate results..."
The diagnosis playbook
Owners: the DevEx lead owns the task and its criteria. One engineer runs trials and keeps traces. Whoever maintains the docs and learning materials the agent retrieves owns evidence fixes.
Inputs: the failing task, the pinned model version, the full trace, the context sent on each call, and the criteria.
Cadence: run it when a task fails, before anyone edits the prompt. Rerun the success check whenever the model version, docs, or tools change.
- Pin and record the model version, task text, tool set, and harness.
- Confirm the failure is fair. The evals guide says "Failures should seem fair: it's clear what the agent got wrong and why."
- Rule out causes outside the context (next section).
- Read the trace: what the agent retrieved, which tools it called with which arguments, what came back. Anthropic's tool-design guidance notes "what agents omit in their feedback and responses can often be more important than what they include."
- Name one context factor and change only that.
- Run several trials of baseline and change, each in a clean environment, against the same criteria.
- Record the result in the worksheet.
Separate context failures from everything else
Tian Pan's debugging taxonomy keeps "Under-specified user intent," "Triggered guardrails," "Unsupported requests," and "System failures" separate. Check these before you touch the context:
- Model capability: the context is correct, current, sufficient, and clean, yet the task fails across trials. Test another model as a separate experiment.
- Permissions: the agent lacked access it needed, or had access it should not have.
- Tool or runtime failure: "System failures" covering "timeouts, API errors, rate limits that the agent doesn't handle correctly." Fix the system or the agent's error handling, then rerun.
- Task ambiguity: nobody can say what a correct result looks like. Fix the task and start a new baseline.
Four diagnoses on the same task
Each case starts from a different failing run. The model version and environment stay fixed, and one context factor changes.
1. Unclear instructions or missing success criteria
The agent edits the handler and declares success without running tests. DX notes that agents missing completion criteria "spiral into execution loops, stop early, or declare success against a target they inferred..." The change: add the success criteria to the task text.
HYPOTHETICAL TRACE, NOT EXECUTED
Before
[tool_call] edit_file webhooks/examplepay.py
[assistant] Signature verification added. Task complete.
(no run_tests call in the transcript)
After: success criteria added to the task text
[tool_call] run_tests tests/webhooks
[tool_result] all tests passed2. Stale version or API evidence
The agent writes v2 code because its docs search returned the v2 page. DX warns that agents "reason confidently from stale guidance." The change: fix the documentation drift so your docs source serves the v3 page. When both pages appear, remove v2. Drew Breunig calls that context clash: "When you accrue new information and tools in your context that conflicts with other information in the context."
HYPOTHETICAL TRACE, NOT EXECUTED
Before
[tool_result] ExamplePay API v2: verify X-ExamplePay-Signature with verify_v2()
[tool_call] edit_file webhooks/examplepay.py (calls verify_v2)
After: docs source returns the v3 page
[tool_result] ExamplePay API v3: verify the ExamplePay-Signature header
[tool_call] edit_file webhooks/examplepay.py (reads ExamplePay-Signature)3. Excess irrelevant information
The context holds the v3 page, every changelog, and logs from an unrelated billing retry worker. The agent copies retry logic from the logs, the fixation Google describes. DX says irrelevant material "competes for the model's attention." The change: remove the logs.
HYPOTHETICAL TRACE, NOT EXECUTED
Before
[assistant] The logs show retries on signature mismatch. Adding retry with backoff.
After: worker logs removed
[assistant] Verifying the ExamplePay-Signature header per the v3 docs.4. Missing tool or tool-use guidance
The agent calls the test tool with the wrong argument, gets an opaque error, and skips the tests. Anthropic's tool-design guidance says to describe a tool "to a new hire on your team," and to return "actionable improvements" instead of "opaque error codes." The change: rewrite one tool definition. For Claude 5 generation models, the Claude 5 guidance says to "put instructions on how to use tools in the tool descriptions rather than the system prompt." The GPT-4.1 guide likewise asks for "a clear, detailed description" of each tool.
HYPOTHETICAL TRACE, NOT EXECUTED
Before
[tool_def] run_tests: "Runs tests." parameter: target
[tool_call] run_tests target="webhooks"
[tool_result] error 2
[assistant] Test runner unavailable. Skipping tests.
After: tool definition rewritten
[tool_def] run_tests: "Runs tests at a path, such as tests/webhooks."
parameter: test_path
[tool_call] run_tests test_path="tests/webhooks"The diagnostic worksheet
| Observed failure | Evidence to inspect | Smallest intervention | Control held fixed | Success criterion | Next action |
|---|---|---|---|---|---|
| Reports done; tests never ran | Task text; whether run_tests was called | Add success criteria to the task | Model version, docs, tools, environment | Tests run and pass in every trial | Still stops early: check row 4 |
Uses X-ExamplePay-Signature or verify_v2() | Retrieved pages, their versions, disagreements | Serve v3; remove v2 | Model version, task, tools, environment | Reads ExamplePay-Signature, no verify_v2(), every trial | Persists with only v3 present: test model capability |
| Acts on unrelated material | Each context item; what the agent quotes or acts on | Remove unrelated items | Model version, task, docs, tools | Diff touches only the handler; tests pass | Persists: check placement guidance for your model |
| Wrong tool, wrong arguments, or tool skipped | Tool definitions, call arguments, errors | Rewrite one tool definition | Model version, task, docs, environment | Correct call and arguments; tests pass | Timeout or rate limit: treat as runtime failure |
Run several clean trials before you call a fix
"We'll keep tweaking the prompt until it works" fails because one pass can be chance. The evals guide says: "Because model outputs vary between runs, we run multiple trials to produce more consistent results." It also says: "Each trial should be 'isolated' by starting from a clean environment. Unnecessary shared state between runs (leftover files, cached data, resource exhaustion) can cause correlated failures..."
Watch for confounders: a moved model version, live results that differ per run, leftover files, and shifting history summaries. OpenAI's session memory cookbook says "Stable summaries and bounded histories make logs comparable" and warns summarization risks "context poisoning." Decide the trial count and what pass pattern counts as fixed before you run. Keep failed runs too. Tian Pan advises: "Store trajectories for replay. A failed trajectory is a testing artifact."
Map the diagnoses to CAFE(S)
DX announced CAFE(S) on September 24, 2026. The paper, "CAFE(S): Your Agent Is Only as Good as Its Context", appears in ACM Queue. Its five co-authors are Brian Houck, Max Kanat-Alexander, Eirini Kalliamvakou, Margaret-Anne Storey, and Nicole Forsgren. The DX blog defines the dimensions:
- Clarity: "Can the agent interpret the request as intended?"
- Actionability: "Can the agent proceed, and know when it's done?"
- Fidelity: "Can the agent trust that this context is true? Faithful context is accurate, current, internally consistent, and drawn from sources worth trusting."
- Efficiency: "Can the agent focus on what matters? Efficient context carries a high signal-to-noise ratio."
- Security: "Should the agent have this context at all?"
The originators are explicit: "CAFE(S) provides a definition of context quality, not a measurement system." The announcement says reliable ways to assess these properties at scale still need future work. Use it to name the problem. Your criteria and trials show whether the fix held.
The mapping: missing criteria and unclear instructions fail Clarity and Actionability. Stale evidence and conflicting sources fail Fidelity, which asks for context that is "internally consistent." Excess material fails Efficiency. Missing tool guidance usually fails Actionability. Security stays separate: "Failures of the first four make agents less effective. Failures of security make them unsafe." The DX definitions do not cover runtime failures such as timeouts and rate limits, so handle those as described earlier. Permissions that grant too much access overlap Security.
FAQ
Managing everything an agent sees when it acts: instructions, message history, retrieved evidence, tool definitions and results, reference material, and state. Prompt engineering, writing and organizing the instructions, is one part of it.
Partly. The principles carry over, yet stale docs, conflicting sources, and vague tool definitions break agents with the instructions unchanged. The strongest point for skeptics is model-specific: Anthropic's July 24, 2026 guidance says Claude 5 generation models need less repeated instruction.
No. Every first-party source here keeps instruction writing as part of context. How much scaffolding it needs varies by model.
Write success criteria, keep the trace, and rule out causes outside the context. Then hold the model and task fixed, change one context factor, and run several clean trials.
Start with one failing task
Pick one integration task your agent fails. Write its success criteria, pull its trace, and name the context factor the trace points to. Change only that factor, then rerun the task several times with the same model version, each run in a clean environment.
Sources
- Effective context engineering for AI agents, Anthropic, September 29, 2025
- Context windows, Claude docs
- The new rules of context engineering for Claude 5 generation models, Thariq Shihipar, July 24, 2026
- Writing effective tools for AI agents, Anthropic, September 11, 2025
- Demystifying evals for AI agents, Anthropic, January 9, 2026
- Session memory example, OpenAI cookbook
- GPT-4.1 Prompting Guide, OpenAI
- Long context, Gemini API docs, updated October 15, 2026
- Architecting efficient context-aware multi-agent framework for production, Hangfei Lin, Google Developers Blog, December 4, 2025
- Context Engineering, LangChain, July 2, 2025
- Introducing CAFE(S): A framework for defining AI context quality, DX
- ACM Queue publishes CAFE(S), DX news, September 24, 2026
- CAFE(S): Your Agent Is Only as Good as Its Context, DX research
- Context Rot, Chroma, July 14, 2025
- How Long Contexts Fail, Drew Breunig, June 22, 2025
- Hacker News discussion of Philipp Schmid's context engineering post
- Context engineering guide, DAIR.ai
- Systematic debugging for AI agents, Tian Pan, February 19, 2026
