GitHub Models Is Gone: Three Zero-Cost Inference Alternatives to Switch to Today

GitHub Models' inference API went offline today (July 30) — here are three OpenAI-SDK-compatible replacements starting at $0, with a migration snippet you can drop in right now.

The GitHub Models playground, model catalog, and models.inference.ai.azure.com endpoint are all offline as of today. If any of your code or CI scripts called that endpoint, those calls are failing now.

Three replacements, ordered by setup friction:

1. Groq — fastest, best free tier

Groq's free tier gives you 14,400 tokens/minute on Llama 3.1 8B, Llama 3.3 70B, Gemma2 9B, and others — at ~750 tokens/second on 70B, it's faster than most paid APIs. The free tier is generous enough for prototyping and light CI use.

2. OpenRouter — broadest model selection

OpenRouter aggregates 500+ models from every major provider. Many are available free (labeled :free in the model ID), including Llama, Mistral, Qwen, and Gemma variants. One API key routes to whichever provider is fastest and cheapest.

3. Ollama — zero cost, private, no rate limits

ollama pull llama3.1:8b
ollama serve   # http://localhost:11434/v1

No rate limits, no data leaves your machine.

Migration snippet (swap the base_url and api_key; the rest of your code stays the same):

# Before (GitHub Models)
client = openai.OpenAI(
    base_url="https://models.inference.ai.azure.com",
    api_key=os.environ["GITHUB_TOKEN"],
)

# After — Groq
client = openai.OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key=os.environ["GROQ_API_KEY"],
)

# After — OpenRouter
client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)

# After — Ollama (local)
client = openai.OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required field, ignored by Ollama
)

Model names differ across providers — update the model= argument to match your chosen provider's model list. Everything else (messages format, streaming, response parsing) is identical.

Sources: GitHub Models retirement — GitHub Changelog · Groq free tier · OpenRouter free models · Ollama