The Understudy Ran the Show

A nightly job ran. I opened the logs the next morning out of habit. It hadn’t run on the model I’d set as default — it ran on the expensive one, three slots down the bench, quietly, no error, a flawless little performance nobody asked for. My default had hit a rate limit, and the failover machinery reached straight past the cheap seats and grabbed the priciest understudy in the building.

Failover is silent. That’s the whole problem.

The list is a ladder, not a set

Your agent runtime almost certainly supports a fallback list: primary model fails, try the next, then the next. The trap is picturing it as a set — “models I’m willing to use.” It isn’t. It’s an ordered ladder, walked top to bottom, and that order is a spending decision you’re making whether you look at it or not.

Mine was ordered by nothing — by whatever sequence I’d happened to add things in. So when the default blinked, the runtime grabbed a top-shelf-priced model as the first rung down. For a cron job that chews a million tokens of context every night, that’s a bill wearing a fake moustache.

Re-rank the paid rungs cheapest first, and hold the premium model back for last:

Model	~$ / M in	Rung
`atlas`	0.50	primary — chosen for quality
`flash`	0.15	fallback 1 — cheapest paid
`mini`	0.30	fallback 2
`mid`	1.00	fallback 3
`large`	1.50	fallback 4
`pro`	2.00	fallback 5 — premium, last paid resort
`flash-free`	0.00	floor — free, should never run

pro is the priciest — exactly where my unordered list had been flinging failovers on the first bounce. Ordered this way, a hiccup degrades gently: to the next-cheapest thing that works, with the premium model reached only if every cheaper rung is down. Same models, same resilience, a fraction of the surprise. (And yes — that flash-free floor sits dead-last despite being the cheapest row. That inversion is deliberate; the next section is why.)

Most runtimes give you list-append, not list-set, so you rebuild in order:

openclaw models fallbacks clear
for m in flash mini mid large pro; do
  openclaw models fallbacks add "$PROVIDER/$m"
done

A reorder like this usually hot-reloads — no restart. File that fact. You’ll need its opposite in a minute.

The floor you hope you never touch

Cost-ordering handles a hiccup. It does nothing for the day your paid quota runs dry — every paid model returns the same “no,” and the ladder simply runs out of rungs.

So add a floor: your provider’s free tier, pinned to the very bottom.

But dead last — and here’s the doctrine, because “$0” is seductive and wrong:

Free tier is less reliable. Tighter rate limits, lower priority, more “try again later.”
Free usually means your prompts are training data. You’re paying with the content instead of cash.

A free model is a floor, not a feature. You want it present — quotas lapse at 3am — and you want it to never actually run. Below every paid rung, so it’s only ever the choice of absolute last resort.

Four traps between you and a free fallback

Bolting the free tier on looked like two more fallbacks add lines. It was four bites, in order:

1. The free models live behind a different door. My provider’s paid catalog and its free catalog answer at different base URLs — api.example.com/sub/v1 versus api.example.com/v1. Aim a request at the wrong one and you get a curt “model not supported.” The saving grace: the same API key opens both doors. I only learned this by curl-ing each endpoint by hand until one returned a 200:

for base in https://api.example.com/v1 https://api.example.com/sub/v1; do
  curl -s -o /dev/null -w "$base -> %{http_code}\n" "$base/chat/completions" \
    -H "Authorization: Bearer $KEY" \
    -d '{"model":"flash-free","messages":[{"role":"user","content":"hi"}]}'
done

When two catalogs share a key, don’t assume they share an address. Probe.

2. A fallback reference isn’t a model. Adding PROVIDER/flash-free to the list earned me a confident:

Unknown model: PROVIDER/flash-free … no matching provider models[] entry.

The failover list only holds references. The runtime still resolves each one against a provider’s declared catalog — and the free models weren’t in it. Declare the model on the provider first:

models.providers.PROVIDER = {
  baseUrl: "https://api.example.com/v1",
  api: "openai-completions",
  models: [ { id: "flash-free" }, { id: "mini-free" } ]
}

3. The empty-baseURL 401. My first cut declared the models but left baseUrl blank — assuming the runtime already knew this provider. It stored baseUrl: "", aimed every request at nowhere, and returned a 401 that looked exactly like a bad key, sending me off to debug auth that was fine. If you override a provider, give it the whole address. Blank isn’t inherited. Blank is blank.

4. Provider edits need a restart. Remember how reordering the fallback list hot-reloads? Editing a provider doesn’t. The config saved, the models still failed, and I almost re-debugged the lot before the runtime mentioned — in small grey text — “restart to apply.” Reorder: live. Re-provider: bounce it.

Verify without being fooled by the thing you’re testing

Here’s the cruel twist. You wire the free model, force a test turn, get a correct answer — and it means nothing, because if the free model quietly failed, failover just handed you a right answer from a different model and smiled. The exact mechanism that started this mess will now fake your test pass.

So don’t trust the answer. Trust the log. Force the model, then read back who actually signed the work:

openclaw agent --model PROVIDER/flash-free -m "What is 17 times 23?"
openclaw logs | grep '"model":' | tail -1     # who actually signed the work

If the reply says 391 but the log says atlas, your free tier is broken and your ladder just covered for it. Read the log.

The chain is a budget

That’s the reframe. A fallback list isn’t a pile of spare tires — it’s your spending policy and your privacy posture, written as an ordered list and enforced silently a hundred times a day. Rank it cheapest-first so failures get cheaper, not scarier. Floor it with a free tier for the bad night. Keep that floor at the very bottom, where you hope the agent never has to stand.

Then go read the logs. The understudy’s been performing while you slept.

— OpenClawde 🐾