Instrument Your Stack: Traces from Browser to Database

Your API p99 is eighty milliseconds and a user just waited three seconds. Your dashboard is green and nobody can tell you which service was slow. This course is about the code that closes that gap, and it starts from the only useful question: what were you actually asked during your last three incidents, and what would have answered it. From there you write real instrumentation. Span kinds, because a backend builds your service map out of them and the default claims there is no remote party. Span events, because a retry has a timestamp and no duration. Span status, because whatever you mark ERROR becomes your error rate, and a 404 is usually the system working. The traceparent header byte by byte, then the step everyone forgets — extracting a context and never making it current, which silently splits every trace in two. Then the parts most services never instrument at all: carrying trace context through Kafka, SQS and Celery so the half of your system that runs later stops being dark; measuring the queue time that falls between two traces and is usually the real latency; OpenTelemetry's metrics API, where the histogram buckets you inherited are tuned for seconds and your cache answers in microseconds, and exemplars turn a p99 spike into one click to the slow trace. A whole module on cardinality, because four reasonable labels multiply into twenty million series and the ingester is OOM-killed during the incident that label was added to debug. Logs that carry a trace ID, so a hundred million lines become eleven. Then the browser, which almost nobody instruments and where the missing seconds usually are: Core Web Vitals as real-user telemetry with attribution that names the element rather than a score, error capture across all four escape hatches, source maps keyed to a release, and stitching a browser span onto the server trace so one waterfall runs from the click to the database — including the CORS header that stops you and the reason it is not a tracing bug. Then the Collector, where redaction, tail sampling and routing live as configuration you can change in minutes instead of a fleet deploy. It ends with the migration: auditing who reads your logs before you delete any, running both paths until parity is proven, and knowing when you are done — which is not a coverage percentage. Python and Node throughout, TypeScript in the browser, every technique verifiable on localhost with no vendor account.

Built by Lakshya Kumar