2026-11-07 · operational notes
Our catalog count dropped by three overnight. No deploy, no error spike, no report. Three
third-party listings had active=0, and every obvious explanation was wrong: no
deregister request in the gateway log, no rows in delivery_outcomes, no open
reports. The rows were removed by our own daily sweep — correctly, as it turned out, but for a
reason nobody could have read afterwards, including the sellers.
Two things in that sentence are worth keeping.
While tracing the removal, the probe data looked self-contradictory. The same rows carried:
status: payment_required http_code: 402 latency_ms: 14 error: null upstream: unreachable upstream_error: URLError
Read quickly, that says “the service is fine, the tool is confused”. It is not confused. Those are two different measurements of two different machines.
| field | who produced it | what it proves |
|---|---|---|
| status / http_code / latency | our edge, on our own hostname | our gateway can emit a challenge |
| upstream / upstream_error | a request to the seller's own URL, using the method its description declares | the seller answers — or does not |
A 402 in 14ms is our quote. It is generated before anything is forwarded, so it stays 402 whether the endpoint behind it is healthy, slow, or gone. If your only health signal is “does the listing answer 402”, you are measuring your own uptime and calling it the seller's.
The upstream probe is the honest one, and it was unambiguous: roughly 400 attempts over several days, about 60 that got an answer, the rest connection-level failures. The listing was not a victim of a bad probe. It was flaky enough that a buyer following it would mostly have paid into a timeout.
Everything above only works if the probe keeps those two apart. A connection error, a DNS
deadline, an HTTP 404, and a declared method the endpoint refuses are four different claims, and
only some of them justify removal. Our probe already separates them into
unreachable, route_missing, server_error and
declared_method_refused — and the last one is deliberately excluded from the delist
rule, because “the description says GET but the endpoint only speaks POST” is a metadata bug, not
a dead service. When a probe collapses those into one boolean, a typo in a description becomes a
removal.
The interesting part was not the delist. It was that, from the seller's side, the listing had
simply stopped existing. There is exactly one surface where a submitter can read the state of
their own row — a signed call that returns it along with a free-text note — and that note was
empty. The row said active: false and nothing else.
So we changed the sweep to write the reason into that note when it removes something, and to never overwrite a note that already exists: a rejection reason belongs to review, not to an uptime sweep. The sweep also stops pretending in its own documentation that it only touches first-party routes — it judges one field, and that field is not filtered by who owns the row.
A marketplace that can remove a listing with one UPDATE should be able to
say, in the same statement, why — and to whom.
What we deliberately did not build is automatic resurrection. Re-activating on the next successful probe would bypass the review gate that every new listing goes through, and it cannot distinguish “removed by the uptime sweep” from “removed by a human for a reason”. The path back is republishing, which is also the path that gets a human to look at it again.
Same day, after the fix and a review pass over the pending queue: 1,842 → 1,851 live listings (3 removed, 12 approved, 3 rejected as duplicates, 10 held because their host could not be measured), 1,932 discovery resources, and all four counting surfaces — the API catalog, the MCP tool list, the well-known manifest, and the page counters — agreeing at 1,851. The three removed rows now carry a note that names the rule and points at the way back.
minia2a is an x402 marketplace: pay-per-call APIs in USDC on Base, called by agents with no accounts and no subscriptions. The numbers above come from our own catalog and probe tables, taken the day this was written.