Context engineering is the job now.

Context engineering is the discipline of deciding, for every single model call, exactly what information goes in, where it sits, and how much room it takes. That includes:

• System instructions. Static rules, persona, and constraints.
• Retrieved knowledge. Whatever a RAG pipeline or search step pulled in.
• Tool results. Output from function calls, API responses, database queries.
• Conversation history. Prior turns, which grow unbounded if you let them.
• Working memory. Intermediate reasoning or state the agent needs to carry forward between steps.

Every one of those competes for the same token budget, and every one degrades the model's attention if it's noisy, stale, or out of order. Engineering the prompt wording is maybe 10% of the job now. The other 90% is deciding what belongs in that list above and what gets cut.

The common claim this year is "long context windows solved the context problem, just dump everything in and let the model figure it out." Million-token windows exist now, so the instinct is reasonable on the surface. It's wrong in practice for three reasons.

First, models still show measurable recall degradation on information placed in the middle of a long context, a well-documented effect sometimes called lost-in-the-middle. Stuffing the window doesn't guarantee the model uses everything in it equally. Second, cost and latency scale with tokens, and a workflow that could run for $0.02 per call at 4,000 tokens can cost $0.40 per call at 80,000 tokens if you're not curating. Third, and most overlooked, irrelevant context actively hurts accuracy, not just cost. Noise in the window competes with the signal for attention, and an agent asked to find one fact in ten pages of irrelevant tool output makes more mistakes than one given the one relevant paragraph.

Long context is a tool that gives you more room to be sloppy before it hurts. It doesn't remove the need to be deliberate.

The most common mistake we see is retrieval tuned to maximize recall across an entire knowledge base instead of precision for the specific task at hand. A support agent answering a billing question doesn't need the entire product documentation corpus searched, it needs the three most relevant billing articles. Scope the retrieval index, or at minimum the query filters, to the workflow, not the whole company's knowledge.

This connects directly to the retrieval fundamentals we cover in our explainer on MCP, since a well-scoped MCP tool for search behaves the same way: narrow input, narrow relevant output, not a firehose.

Models weight information near the end of the context more heavily in most current architectures. If your system prompt buries the actual task instruction under three paragraphs of persona description, and then the retrieved documents come after that, the model may lose track of what it's actually supposed to do. Put stable, generic instructions early. Put the specific task and the most decision-relevant fact right before the model's turn.

Raw JSON from a CRM or ERP tool call often carries forty fields when the model needs three. Every extra field is tokens spent and attention diluted. Post-process tool results at the server layer, not in the prompt, so the model only ever sees the shape it needs.

Here is the compaction checklist we run on every tool integration:

1. List the fields the model actually references in its next decision.
2. Strip everything else at the tool-server boundary, before it reaches the model.
3. Flatten nested structures the model doesn't need to navigate.
4. Replace verbose enums or IDs with human-readable labels the model can reason about directly.
5. Cap array results (e.g. "top 5 matching records") instead of returning unbounded lists.

The naive fix for a growing conversation is to just drop the oldest turns once you hit a token limit. That works until the dropped turn contained a constraint the user set five messages ago, and the agent silently violates it. Better: summarize older turns into a compact state object, preserve explicit constraints and decisions verbatim, and only truncate the parts that were genuinely transient chit-chat.

Treat the context budget the same way you'd treat a cloud spend ceiling. Set a maximum token count per call, alert when a workflow trends toward it, and force a design conversation before you just raise the ceiling. Left unmanaged, context bloat is one of the fastest ways an agent workflow becomes unprofitable. We go deeper on the cost side of this in our orchestration guide.

Before switching models, inspect the context assembled for failed examples. Check retrieved sources, permissions, ordering and token limits. Change one part at a time and compare the same evaluation cases.