You can't fix what you can't ask

Here’s the moment that separates the two words people use interchangeably. It’s the middle of an incident. Something is slow — but only sometimes, only for some users, and only since roughly an hour ago. You open your dashboards. They’re green, because they were built to watch the failures you already knew about. The failure you’re living through right now is one nobody drew a chart for.

That gap is the whole distinction between monitoring and observability. Monitoring is the set of questions you asked in advance and wired up: is CPU high, is the error rate above X, is the queue backing up. Useful, necessary, and blind to anything you didn’t anticipate. Observability is a property of the system itself — whether the data it emits is rich enough that you can ask a brand-new question, right now, and get an answer without deploying new code to go collect it. Monitoring is the dashboard. Observability is whether the dashboard could have been about something else.

Three kinds of signal buy you that ability, and it’s worth being precise about what each one is actually good at — because teams tend to reach for the one they already have rather than the one the question needs.

Logs are discrete events: a line, with detail, about one thing that happened. They’re the richest per-event and the easiest to add — a print statement grows up into a log. Their strength is depth on a single occurrence (“what exactly happened to this request”). Their weakness is that they don’t naturally answer “how often, across everyone,” and at scale they get expensive to store and slow to search. A structured log — key-value fields, not a prose sentence — is the version you can actually query later.

Metrics are aggregates over time: a number, counted or measured, rolled up into a series you can graph. They’re cheap and fast precisely because they throw detail away — a counter doesn’t remember which requests, just how many. That makes them perfect for “is it happening more than yesterday” and terrible for “which user, and why.” The classic trap is cardinality: the moment you try to slice a metric by something high-variety — user ID, request ID — the cheapness evaporates, because you’re really asking for a separate series per value.

Traces follow one request across every service it touches, stitching the hops into a single timeline with timings. In a world of many small services, this is the signal that answers “where did the 800 milliseconds go” — the question logs and metrics both struggle with, because the latency lives in the seams between components, not inside any one of them.

Notice the shape of the trade-off. Detail and cost move together. Logs are the richest and priciest per unit; metrics are the cheapest because they forget the most; traces sit in between and specialize in the seams. There’s no free signal — you’re always paying for detail with storage, or buying cheapness with amnesia. The mature version of this isn’t “collect everything.” Collecting everything is how observability bills quietly overtake your compute bills, and it doesn’t even help, because a haystack of undifferentiated data is its own kind of blindness.

The judgment call — the actual skill — is deciding which questions you’re likely to need to answer, and which you can afford to answer only sometimes. High-value flows (a checkout, an auth handshake) probably deserve traces and structured logs, so that when they break you can reconstruct a single failure end to end. Broad health probably wants cheap metrics with deliberately bounded cardinality. And the tell that you’ve built real observability isn’t a wall of dashboards — it’s the incident where someone asks a question no one anticipated, and the answer is already sitting in the data, waiting to be queried.

Because that’s the quiet promise buried in the word. The bug you’ll lose an evening to is, almost by definition, the one you didn’t predict — if you’d predicted it, you’d have prevented it. Observability is the bet that you can’t foresee your failures, so instead of pre-drawing the chart, you make the system able to explain itself after the fact. You can’t fix what you can’t see. But more precisely: you can’t fix what you can’t ask about — and whether you can ask is a decision you make long before the 2 a.m. you’ll need it.