The Access Check That Could Be Forged

Here is a line of code that reads as a security control:

const clientIp = req.headers['x-forwarded-for'] || req.socket.remoteAddress;
if (!clientIp.includes(ALLOWED_IP)) {
  return deny();
}

It is an allowlist. It names exactly one permitted caller. It has a default-deny shape. If you were reviewing this file you would tick the box that says access is restricted to the known caller and move on.

It is also worth nothing at all, because X-Forwarded-For is a header the caller writes. Anyone who can open a TCP connection to the port can put any address they like in that field, including the one the allowlist is looking for. The check would pass. Not occasionally, or under some misconfiguration — always, by construction, for every request.

We found this while hardening a relay that sits in the payment path of a live agent service, and the part worth writing down is not the bug. It is what the bug was hiding.

Why nobody noticed

The relay was reachable only through a host firewall that permitted a single source address. So every request that arrived was, in fact, from the permitted caller. The forged check and a working check produce identical observations when the layer below is doing the work.

That is the general shape of this failure, and it is not really a coding mistake:

And here the redundancy was real, which made it worse. The firewall genuinely blocked everything else. So the app-layer check was dead weight that looked like a second layer. Anyone reasoning about this system — including the person who wrote it — would conclude that removing the firewall rule would still leave the app protected. It would not. It would leave a public port with an allowlist that answers "yes" to anyone who asks politely.

A forged check is worse than no check

Absent a control, the next reader knows there is nothing there and designs accordingly. A check that can be forged does the opposite: it terminates the thought. It is a claim about the system's defences that is written in the confident grammar of working code, and it survives review because it looks right.

We have hit this shape before in a different costume — an upstream that fails and returns 200 OK with plausible zero values, so the caller can't tell "the answer is zero" from "we couldn't ask". Same family: a surface that reports success without having done the work. The fix is the same in both cases. Make the failure visible, and make the check's own evidence the thing you test.

What the rest of the relay looked like

The forged allowlist was the sharpest problem but not the only one. Once we were in the file, the same three patterns showed up that we now look for in any relay or proxy sitting in an agent's payment path.

1. A shared secret with more than one power

The relay authenticated callers with a secret that fell back, if unset, to a literal compiled into the source — and that literal was the platform's administrator key: the same value that gated service approval, listing removal, message deletion and token signing. One credential, many powers, sitting in a file on a second machine, protecting a proxy that forwards any path to a third-party API.

The fix is not "move the secret somewhere safer". It is to stop reusing one credential for unrelated powers: the relay got its own secret, the literal was deleted, and the process now refuses to start when it is missing rather than falling back to a working-but-worse value.

2. A proxy that forwards everything

"Any path forwards to the upstream API" is not a smaller version of "the three methods this service needs". It is a general-purpose write proxy into a funded account, and the difference matters the moment the credential leaks, because the blast radius is the whole account rather than the one operation. The relay now answers three paths and returns a hard 404 for everything else.

3. Running as root for one privilege

It needed to bind port 80. It was running as root. On Linux you can hand a process exactly one capability — CAP_NET_BIND_SERVICE — and nothing else, which turns "compromise of this process owns the machine" into "compromise of this process owns a listening socket".

Verification, because "I changed it" is not evidence

Every claim above is cheap to make and worthless without a measurement, so each one got a control. The shape we use:

• The change worked: a request with the new credential returns 200 and a real balance figure from the upstream.
• The old state is genuinely gone: the same request with the previous administrator key returns 401. This is the one that matters — rotating a secret and deleting the old one are different acts, and only the second one is testable from outside.
• The forged check is closed: a request from a non-permitted address carrying the forged header returns 403. Testing without the forged header would have proved nothing; the old code would also have returned 403 for that request.
• The port really is closed: six independent nodes across five countries got No route to host against the public address. Taking the words "it's firewalled" on trust is how you end up describing a firewall rule that was never applied.

Two more measurements that are less obvious but decided the design:

Read the environment from the running process, not the config file. A config change that never reached the process looks identical to no change at all. Reading the live process's environment and then calling through the path it describes closes that gap — the file, the process and the wire all have to agree.

Ask where the packets actually go. The caller was dialing the relay's public address. Both machines turned out to be in the same private network, so those packets — carrying the upstream API key, which is money — were leaving to the internet gateway and coming back for no reason. Moving the call to the private address removed the public hop entirely. The obvious proposal here was TLS; a self-signed certificate would have required installing a trust anchor on the production host whose private key lives on the less trusted of the two machines, which is a sideways trade dressed as an upgrade. The question to ask first is not "should this be encrypted" but "why is this traffic public at all".

The part that is easy to skip

A fix deployed to a directory that a deploy process can overwrite is a fix with a half-life. The handler lived in exactly such a directory, and the reverted version would not have failed — it would have quietly gone back to the public path and kept working, with the only observable difference being a source address in the relay's own log.

So the fix shipped with a daily check that reads that log and asks a narrow question: did any accepted request arrive from the public address? It also reads the handler directly, because a check that only fires when there is traffic is blind on a day with none — and a service with zero calls is exactly the service whose config drifts unnoticed.

The check reports four states and never collapses them: private path, public path in use, handler reverted, and could not look. That last one is not a shade of "fine". A monitor that cannot distinguish "verified" from "unable to verify" is a monitor that will one day report all-clear because it stopped working.

Notes from operating x402 payment infrastructure and the services around it. If you run a relay, a signing proxy, or anything else that sits between an agent's money and a third-party API, the four questions worth answering are: what identity does it accept, where does that identity come from, what is the smallest set of operations it needs, and what does it look like when it fails?