Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
Distributed tracing was designed for systems where the same input produces the same call graph, the shape of a request is knowable in advance, cost is roughly constant per request, and success is a status code. An agent violates all four. The same question produces a different number of tool calls each time, so no two traces are comparable by shape. Cost varies by an order of magnitude between two identical requests. And most importantly, the thing that goes wrong is usually not an error — it is a correct-looking answer that is wrong, which returns 200, takes a normal amount of time, and satisfies every check you have. Recognising which assumptions break tells you exactly what has to be added.
Two traces of the same user question, side by side. Everything ordinary telemetry measures is fine in both, and one of them is a failure. That is the gap this course closes.
# ── The same question, two runs, minutes apart ────────────────────
#
# RUN A — trace 4bf92f35... 1,840ms status OK
# invoke_agent research-assistant 1,840ms
# ├─ chat gpt-4o 410ms in 812 out 96
# ├─ execute_tool search_docs 180ms
# ├─ chat gpt-4o 520ms in 2,140 out 210
# └─ chat gpt-4o 730ms in 2,410 out 380
# answer: correct, cites 2 sources
# cost: $0.0181
#
# RUN B — trace 8ac9de11... 1,910ms status OK
# invoke_agent research-assistant 1,910ms
# ├─ chat gpt-4o 430ms in 812 out 88
# ├─ execute_tool search_docs 190ms
# ├─ execute_tool search_docs 175ms <- searched twice
# ├─ chat gpt-4o 540ms in 3,890 out 240
# └─ chat gpt-4o 575ms in 4,110 out 410
# answer: CONFIDENTLY WRONG. Cites a source that does not say that.
# cost: $0.0297 (+64%)
#
# Every ordinary signal says these are the same:
# latency 1,840 vs 1,910ms within noise
# status OK vs OK
# error rate 0 vs 0
# HTTP code 200 vs 200
# One of them is a production incident.
# ── The four assumptions, and what replaces each ─────────────────
#
# 1. DETERMINISM: same input -> same call graph
# Broken by: sampling temperature, model updates, tool ordering.
# Consequence: you cannot diff two traces by shape, and "compare
# against the normal trace" is not a thing.
# Replace with: record the SHAPE as data (step count, tool
# sequence, token counts) so you can compare distributions
# instead of individual traces. -> modules 2 and 4
#
# 2. FIXED SHAPE: a known set of spans per request
# Broken by: loops that run until the model decides to stop.
# Consequence: p99 span count matters as much as p99 latency, and
# a runaway loop is a cost incident with no error.
# Replace with: step count and depth as first-class metrics, with
# a hard ceiling. -> modules 4 and 10
#
# 3. BOUNDED COST: roughly constant per request
# Broken by: token counts that vary 5x on the same question.
# Consequence: your unit economics are a distribution, not a
# number, and the tail is what bankrupts you.
# Replace with: cost derived per span from token attributes, with
# exemplars to the expensive traces. -> module 9
#
# 4. BINARY SUCCESS: it worked or it threw
# Broken by: fluent, plausible, wrong. THE hard one.
# Consequence: your error rate can be zero during a total quality
# outage. No amount of conventional instrumentation catches it.
# Replace with: a quality SLI sampled from production, and a
# burn-rate alert on it. -> module 11
# ── What stays exactly the same ──────────────────────────────────
# Everything in the engineering track's telemetry course still
# applies: span kinds, context propagation, cardinality discipline,
# the Collector. An agent is a distributed system with an unusual
# failure model, not a different kind of software. This course adds
# the four things above and assumes the rest.python3 main.py