Switch Hosted LLM Providers at Runtime With `:floor`: OpenRouter’s Cost Routing for Production Inference

TL;DR: OpenRouter wraps 400+ hosted models behind one OpenAI-compatible endpoint; append :floor to route to the cheapest current provider, or set max_price to cap per-call spend.

What you'll be able to do after this: swap or A/B-test hosted LLMs without changing client code, route high-volume tasks to cheaper providers automatically, and set hard per-request cost ceilings — all through the OpenAI SDK you already use.

The problem. You've built a pipeline around one hosted model. Now you want a cheaper open-weight model for the classify step, automatic fallback when your primary provider rate-limits you, or a quick A/B test across providers. Wiring that yourself means separate SDK clients, custom retry logic, and provider-specific rate-limit handling.

How it works. One endpoint: https://openrouter.ai/api/v1. Drop it in as a base_url override — your existing OpenAI SDK client works without other changes:

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-...",          # your OpenRouter key
)

Pass any <provider>/<model-slug> as the model. Append :floor and OpenRouter picks the cheapest provider currently serving that model:

resp = client.chat.completions.create(
    model="meta-llama/llama-3.3-70b-instruct:floor",   # cheapest provider, right now
    messages=[{"role": "user", "content": "Classify this support ticket: ..."}],
)

Set a hard ceiling with max_price (dollars per million tokens) so a call fails fast rather than silently overcharging:

resp = client.chat.completions.create(
    model="meta-llama/llama-3.3-70b-instruct",
    extra_body={
        "provider": {"max_price": {"prompt": 1.0, "completion": 2.0}}
    },
    messages=[...],
)

Production patterns:

• Cost-tier routing: classify with :floor, reason with Sonnet 5 — same client, different model strings, one endpoint.
• BYOK: if you have existing contracts with AWS Bedrock, Together, or Fireworks, OpenRouter can route through your own credentials. Per OpenRouter's docs, this carries a 5% platform fee with the first 1M requests/month waived.

Real limits to know:

• :floor rebalances continuously — you can land on different backends across runs, which complicates reproducibility testing. Pin a provider explicitly when that matters.
• Hard budget enforcement is account-level, not run-level. max_price caps a single call, but an agentic loop can still exceed a daily budget. Add application-level spend tracking for unattended workloads.
• Independent 2026 reviews flag account security and support incidents as pain points for paying customers; evaluate SLA requirements before routing critical production traffic.
• Free-tier models ($0/MTok, 20+ available) may log requests for model improvement. Check per-model privacy policies before routing sensitive data.

Sources: OpenRouter cost routing guide · TrueFondry independent review, 2026 · Miles K., practical OpenRouter guide (Medium) · DeployHQ team guide (incl. LiteLLM graduation path)