2026-10-29 · engineering note
Some of our endpoints take a URL from the caller and fetch it: scrape this page, read these
headers, parse that feed. The caller pays per call, and the platform keeps a per-endpoint
failure counter. Three delivery failures from three distinct signing wallets inside a 24-hour
window delists the endpoint, and active=0 does not reverse itself.
That rule is reasonable for an endpoint that is actually broken. It is a bug for an endpoint whose caller misspelled a hostname — and until this week, that is exactly what we were counting.
When the resolver answers ENOTFOUND, it is telling you something definitive:
this name does not exist. It answered. The answer is a fact about the input the caller
supplied, not about the service that received it. Our handlers caught that error in the same
catch as a connection reset and returned {ok: false}, which the
gateway booked as a failed delivery.
Three callers typing https://exmaple.com, three wallets, one afternoon: the
endpoint is off the shelf, and the buyers who arrive tomorrow find a 404 that says the service
is not found. The operator's own correctness rule destroyed the supply it was meant to
protect, and the audit trail says the endpoint failed, which it never did.
The fix is not "treat DNS errors as fine". It is to stop collapsing three different statements into one verdict:
| bucket | signal | what it means | recorded as |
|---|---|---|---|
| caller | ENOTFOUND, EBADNAME, ERR_INVALID_URL | the name you gave does not resolve; the resolver answered | 400, refused, cause=caller |
| not measured | EAI_AGAIN, resolver timeout | nobody answered; we know nothing | not a caller error, not a service strike |
| upstream | connected, HTTP 5xx, malformed body | the service we call failed | delivery failure, counts toward delist |
The middle row is the one that gets lost. EAI_AGAIN and ENOTFOUND
arrive through the same code path, and a two-bucket design has to put them somewhere: calling
them a caller error refunds a service that is genuinely down, and calling them a service failure
delists a service whose upstream never heard the question. They are not the same statement and
they should not share a verdict.
We wrote a guard that stands up a resolver which answers ENOTFOUND for one
hostname, calls every first-party handler that takes a URL, and reads the status and body that
comes back. On the deployed bytes:
49 handler(s) judged; a caller's nonexistent hostname never books a service strike
tally: ANSWER=9 CALLER=40Forty handlers refuse with a 4xx carrying the resolver's own message. The other nine return a 2xx whose body states the host does not exist — also correct, and deliberately a different contract: those services report on a name rather than fetching it, so "this name does not resolve" is the answer they were paid for. The guard reports that split instead of folding it into a green number, because the right verdict differs by what the endpoint promises to do.
A second door matters here and it is easy to miss: a handler that returns
200 {ok:false} books the same delivery failure as a 500. Judging attribution by the
HTTP status alone would have declared this fixed while the platform still charged the caller and
still counted the strike. The probe reads the body for the verdict and the status for the
contract, and the two have to agree.
Every handler that fetches a caller-named URL had its own catch. There is no
single choke point where the error class reaches the billing decision — the sidecar converts a
thrown BadRequestError into a refusal, but the decision to throw is per handler.
That is why the measurement has a population (49 handlers) rather than a sample, and why the
guard re-runs against the deployed bytes: the class of bug outlives the commit that fixes it.
A rule that delists on failure has to be able to tell "the caller was wrong" from "we were wrong" from "we don't know". If it cannot, it is a machine for removing working endpoints.
If you run a paid API with an automatic health rule, the cheapest audit is this one: find every place a caller-supplied identifier reaches the network, and ask what your logs would say if that identifier were a typo. Ours said "outage". Yours might too.
minia2a is an x402 pay-per-call marketplace: 1,700+ endpoints, USDC on Base, five free trial calls per signed wallet. The measurement tool described here ships in the platform's own repository.