The Monitor That Reported Itself

A daily check printed this line:

counts: {'raw-equal': 55, 'header-missing': 5}
STATUS: measured — 60 first-party 402(s) judged: 55 byte-identical ...

The two halves contradict each other. Five of the sixty responses had no PAYMENT-REQUIRED header, which is why judge() filed them under header-missing. But header-missing means the server answered 402 and forgot its header. Those five were HTTP 429 — they never claimed to be 402 at all, and the check had caused them itself. The next line made it worse: the /svc alias door, measured on six of those requests, read 1/6 byte-identical. The door was not 1/6 broken. The probe had spent its budget pacing at six requests per second and the last five alias requests were refused for going too fast.

What the check was actually doing

The property being measured is narrow and worth stating: for a V2 x402 challenge, is the response body the same bytes as the base64-decoded PAYMENT-REQUIRED header? Not the same JSON — the same bytes. That distinction is the whole reason the check exists: “the body carries the same document” is implementable, “the body is a copy of the header” is only true if both paths serialize identically.

The procedure is:

1. sample first-party endpoints, fetch each anonymously, expect 402;
2. decode the header, compare it to the body byte for byte;
3. repeat on the alias route, which is a different handler path and therefore a separate reading.

The classification step took two inputs: the header string and the body. It never saw the status code. That was fine while every response was 402, and wrong the moment one was not.

The 429 was ours

A rate-limit response is not a property of the endpoint. It is a property of the conversation: too many requests from one address in the window. The probe's own pacing is what produced it. The measurement then reported the consequence back as if it were a defect in the measured object — twice, in two different classes:

None of the three is a lie about the code. Each is a lie about the observation, and the second one sends a reader to fix a header that was never missing.

Three states, not two

The correction is not “add a message”. It is to give the measurement its third state:

• measured, and the property holds — 402 with byte-identical carriers;
• measured, and the property does not hold — 402 with a real carrier defect;
• not measured — 429, transport failure, a 404 for a delisted id, a 200 that answered something else.

The third state has to be counted, printed, and kept out of the denominator. We made four changes:

1. A non-402 response is classified non-402 before it reaches the header/body judge. A 402 that genuinely lacks the header still lands in header-missing, so the real defect class remains.
2. 429 is retried once, honoring Retry-After (default 1.5 s, capped at 5 s). Throttling is often one window wide.
3. measured and each surface's denominator exclude non-402. “How many 402s did we judge” and “of those, how many hold” now share a denominator.
4. The predicate used by the live loop is the same function the selftest drives. Five controls were added: 429, 404, and 200 must all classify non-402; 402 without a header must stay header-missing; 402 with a header must stay raw-equal.

After the change, the same command over the same population:

counts: {'raw-equal': 60}
surface[x402]: 54/54 byte-identical
surface[svc]:  6/6 byte-identical

The property was always holding. The report was describing the reporter.

The same bug in the sibling check

A second check samples four challenge surfaces — direct, alias, V1, and MCP — and verifies each advertises a real resource.description. On the MCP surface its loop did this:

if challenge_of(body) is None:
    continue  # e.g. -32602 service-not-found / 429: nothing to judge

Two unlike things, one branch. A JSON-RPC “service not found” is a fact about the endpoint; a 429 is a fact about the caller. Both silently reduced the measured count. It now counts throttled probes per surface, prints them, and says in the status line when the whole sample was throttled by our own request rate.

The rule that generalizes

Any monitor that talks to the system it monitors is part of that system. Its own load, its own retries, and its own rate budget are inputs to the observation. If the report cannot say “I did not measure this”, it will eventually say something confident and false instead — and the failure mode is asymmetric: a false red wakes someone up, but a false green and a falsely-attributed defect both survive review, because they look like data.

The states are three. The denominator has to know which one each row is in.

Related: your 404s have user-agents · the error class that was thrown away · same byte count, different page