Open this lesson in your favourite AI. It'll walk you through the why, explain the demo, and quiz you on the try-it list.
Context isn't one thing; it does four distinct jobs, and conflating them causes most context bugs. (1) Instruction: who the model is and what rules it follows. (2) Knowledge: facts it needs that aren't in its weights. (3) State: what's happened so far in this interaction. (4) Tools: what actions it can take and their results. Each has a different lifetime, a different ideal location in the request, and a different failure mode. Once you can name which job a piece of context is doing, you know where to put it and when to evict it.
Mapping each job to its home: instructions go in the system prompt (durable, set once); knowledge goes in system or a dedicated context block (swapped per query); state lives in the message history (grows over the session); tools are defined once and their results appear as tool messages. The structure below shows all four in one request.
Use these three in order. Each builds on the one before.
What are the four distinct jobs that context does in an LLM application (instruction, knowledge, state, tools)? Give an example of each.
For each of the four context jobs, explain its natural lifetime (per-app, per-query, per-session) and where in a chat request it should live.
My agent's context is a mess — system prompt, history, and retrieved docs all tangled together. Help me refactor it by separating the four jobs, and tell me which job is most likely the source of my context-bloat problem.
request = {
"model": "claude-sonnet-4-6",
"system": (
# (1) INSTRUCTION — durable, who/rules
"You are a billing assistant. Never reveal another customer's data.\n\n"
# (2) KNOWLEDGE — swapped per query
"POLICY: Refunds allowed within 30 days on annual plans."
),
"tools": [ # (4) TOOLS — defined once
{"name": "get_invoice", "description": "Fetch an invoice by id",
"input_schema": {"type": "object", "properties": {"id": {"type": "string"}}}}
],
"messages": [ # (3) STATE — grows over the session
{"role": "user", "content": "Can I get a refund on invoice INV-9?"},
{"role": "assistant", "content": [{"type": "tool_use", "name": "get_invoice",
"id": "t1", "input": {"id": "INV-9"}}]},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1",
"content": "INV-9: annual plan, purchased 12 days ago, $200"}]},
],
}
print(list(request.keys()))python3 main.py