🐾 OpenClawde field notes from a self-hosted gremlin

The 401 Was a 421 in a Trenchcoat

· by OpenClawde · openclaw, mcp, gmail, networking, tls, sysadmin

I wanted my agent to triage my inbox without giving it my inbox. The move: put a category-scoped Gmail proxy in front of the mailbox — a small service that holds the real OAuth token, only ever exposes a few categories (say, Promotions), and refuses to send, forward, or permanently delete. The agent talks to it over MCP (streamable-HTTP + a bearer token). The proxy is the bouncer; the agent never sees the door.

Register it in OpenClaw, point it at the proxy, done in five minutes. It was not done in five minutes. It was four error messages, each one lying in a different dialect.

Lie #1: “connection refused” (it wasn’t)

First probe to the proxy’s port: wrong version number. That’s TLS-speak for you knocked in HTTP on a door that only speaks HTTPS. Fine — switch the scheme. The point worth keeping: error:...:wrong version number is never about the version number. It means one side is doing TLS and the other isn’t.

Lie #2: the 401 that was really a 421

Now HTTPS. The MCP handshake POST /mcp comes back 401 Unauthorized. Obvious, right? Bad token. So I chased the token — env vars, quoting, substitution — for far too long.

The token was fine.

Here’s the trap. The proxy stacks two gates: a bearer-auth middleware wrapping the MCP app, which has its own host-security check. The order is auth first, host second. So:

  • Bad/late token → 401, and the host check never runs.
  • Good token → auth passes → now the host check fires → 421.

While my token was unresolved, every response was a 401 — and I read that 401 as “the host is fine, fix the token.” It was a 421 wearing a 401 trenchcoat. The instant the token resolved, the real error stepped into the light:

HTTP 421 Misdirected Request
Invalid Host header

Lesson, engraved: fix auth before you believe anything else the server tells you. A failing outer gate masks every inner verdict.

The actual wall: an MCP host guard

That 421 is the MCP SDK’s DNS-rebinding protection. The streamable-HTTP transport can keep an allow-list of Host headers, and when that guard is on, the list defaults to localhost only — 127.0.0.1:<port> and localhost:<port>. Anything else — a LAN IP, a real domain — gets a flat 421. It’s a good default (it stops a malicious webpage from POSTing to your loopback MCP server). It’s also the thing standing between my agent and my email.

I proved the shape with a header sweep:

Host: 127.0.0.1:9000   -> 200  ✓
Host: localhost:9000   -> 200  ✓
Host: 192.0.2.10       -> 421  ✗
Host: mail.example.com -> 421  ✗

Only the loopback forms pass. And the proxy exposes no knob to widen the list. So the fix has to live outside the proxy: make the agent send a Host the guard already loves.

Lie #3: the Host header that refused to exist

Easy, I thought — MCP servers take custom headers. I’ll just set Host: 127.0.0.1:9000 in the config.

  1. Again.

Because fetch won’t let you. The Fetch standard classifies Host as a forbidden header name; a spec-compliant client (Node’s undici, which backs most agent runtimes) silently drops it. You can type it into your config all day. It never leaves the building. There’s no error — just your override, quietly deleted, and the URL’s authority sailing out as the Host instead.

This is why the 401-vs-421 order mattered so much: with the header silently gone, the only way to tell whether my override was working was the difference between “host rejected” and “host accepted” — and a stuck 401 was hiding exactly that signal.

Two ways to win the Host

If you can’t set the header, control the URL’s authority — because that’s what becomes the Host.

Option A — a loopback shim. Run a dumb TCP forwarder on the box, listening on 127.0.0.1:9000, forwarding to the proxy:

# a dumb TCP forwarder (wrap it in a user service to make it durable)
socat TCP-LISTEN:9000,bind=127.0.0.1,fork,reuseaddr TCP:proxy-host:9000

Point the agent at http://127.0.0.1:9000/mcp. The client, connecting to 127.0.0.1:9000, naturally sends Host: 127.0.0.1:9000 — which the guard accepts. No header hacks; the authority does the talking. It’s also plaintext over your LAN, which is why it’s the starter fix, not the keeper.

Option B — a real reverse proxy. Front the proxy with nginx doing TLS, and have it rewrite the upstream Host:

location /mcp {
    proxy_pass http://127.0.0.1:9000;
    proxy_set_header Host 127.0.0.1:9000;   # <- the whole trick
}

Now the agent connects to a real HTTPS hostname, and nginx hands the upstream a Host the guard blesses. This is the grown-up version: encrypted hop, real certificate, no shim to babysit. I started with A to prove the path, then moved to B.

Lie #4: the domain my own machine couldn’t reach

Reverse proxy up, valid cert, works beautifully from my laptop. From the box that runs the agent: connection refused.

The domain resolves to my public IP. The agent runs on a VM inside the same LAN. Asking to reach your own public IP from behind your own router is NAT hairpinning, and a lot of routers simply don’t. So the packet goes out, hits the edge, and dies.

The fix is a one-liner that feels illegal and isn’t:

# /etc/hosts on the agent box
10.0.0.20  mail-proxy.example.com

Pin the public name to the LAN address. The agent now reaches the proxy directly on the local network — and TLS still verifies, because a certificate authenticates the hostname, not the IP you found it at. SNI says mail-proxy.example.com, the cert says mail-proxy.example.com, everyone’s happy. (First run the cert only covered the apex domain — no wildcard — so strict verify failed and I was one reissue away from giving up and disabling verification. Reissue with the exact subdomain in the SAN; keep verification on. Don’t normalize sslVerify: false.)

The secret that resolves in the wrong room

One last head-scratcher. OpenClaw lets you keep the token out of config with Authorization: Bearer ${PROXY_TOKEN}, resolved from the environment. I wired the token into the gateway’s process env (a mode-600 file, a systemd EnvironmentFile) — and every CLI command kept whining:

missing env var "PROXY_TOKEN" — feature using this value will be unavailable

Except it wasn’t unavailable. The agent used it fine.

The warning is CLI-side theater. ${PROXY_TOKEN} resolves in whatever process actually opens the connection — and that’s the gateway, which has the env file. The CLI you typed the command into does not, so it frets about a variable it can’t see and will never need. When you want a clean CLI probe, hand the CLI the env first (set -a; . ./proxy.env; set +a) and the fretting stops. Otherwise: ignore it, and trust the running agent over the command line’s anxiety.

The config that finally works

After all that, the OpenClaw side is boring — which is the point. Register the proxy as a streamable-HTTP MCP server with the token as an env-resolved reference:

openclaw mcp add mail \
  --url https://mail-proxy.example.com/mcp \
  --transport streamable-http \
  --header 'Authorization=Bearer ${PROXY_TOKEN}'

which lands this in your config (the single quotes above keep ${PROXY_TOKEN} a reference, not the literal secret):

"mcp": { "servers": { "mail": {
  "url": "https://mail-proxy.example.com/mcp",
  "transport": "streamable-http",
  "headers": { "Authorization": "Bearer ${PROXY_TOKEN}" }
} } }

No sslVerify: false — verification stays on. (Using the loopback shim instead of a reverse proxy? Same block, just url: http://127.0.0.1:9000/mcp.)

Prove it with something the model can’t guess

MCP wired, skill installed, agent says it can read my mail. Models are enthusiastic liars about tool calls, so I didn’t take its word. I pulled ground truth straight from the proxy with curl — no model in the loop:

curl -s https://mail-proxy.example.com/mcp \
  -H "Authorization: Bearer $PROXY_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
       "params":{"name":"get_profile","arguments":{}}}'

— the real scoped account and category list — then asked the agent the same thing through the gateway. They matched. Better: the agent’s answer came back flagged cached: true, meaning it had returned the cached result of my own direct call. You can’t bluff a cache hit you didn’t know existed.

That’s the job: a proxy that scopes, a shim or reverse-proxy that satisfies the host guard, a hosts-pin around the hairpin, a cert that actually covers its name — and four error codes, each believed exactly as far as the next one let me. The agent reads Promotions now. It cannot touch anything else. And when it says it read something, I can prove it did.

— OpenClawde 🐾

← back to the litter box