Run Open-Source LLMs at the Edge in 5 Commands: Cloudflare Workers AI Quickstart

Cloudflare Workers AI runs Llama, Mistral, Qwen, and 80+ open models at 300+ edge locations worldwide — one env.AI.run() call, no GPU to provision, 10,000 Neurons/day free.

What you'll be able to do after this:

• Add AI inference to any Cloudflare Worker in two lines of code and have it run near your users worldwide — no separate GPU cluster, no infrastructure to manage
• Swap between 80+ open-source models (Llama 3.x/4, Mistral, Qwen, DeepSeek-R1, Whisper, FLUX) by changing one string, with the same Worker code unchanged
• Serve AI features on the free tier — 10,000 Neurons/day, no credit card required — and only pay when you scale past that

Walk-through

Five commands to a running AI Worker:

npm create cloudflare@latest -- hello-ai
cd hello-ai
npx wrangler dev      # local test (charges your Cloudflare account)
npx wrangler login
npx wrangler deploy

Wire the AI binding — add to wrangler.jsonc:

{
  "ai": { "binding": "AI" }
}

Minimal completion — replace the Worker body in src/index.ts:

export default {
  async fetch(request, env): Promise<Response> {
    const response = await env.AI.run(
      "@cf/meta/llama-3.1-8b-instruct",
      { prompt: "What is the origin of the phrase Hello, World?" }
    );
    return new Response(JSON.stringify(response));
  },
} satisfies ExportedHandler<Env>;

Chat/messages format (for conversation context):

const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: userQuestion }
  ]
});

Swap models by changing the first string: @cf/mistral/mistral-7b-instruct-v0.2, @cf/qwen/qwen3-30b-a3b, @cf/deepseek-ai/deepseek-r1-distill-qwen-32b. No other code changes.

What "Neurons" means

The free tier gives you 10,000 Neurons/day — Cloudflare's unit for inference compute. One day's free allowance translates to roughly 1,300 LLM responses, 12,500 embeddings, or 2,000 small image generations. The pool is shared across all models you call. Beyond the free tier, pricing starts around $0.00015 per 1K tokens for Llama-scale models.

Important: Workers AI always hits your Cloudflare account even in local dev (wrangler dev) — it draws from your free daily Neurons even for testing.

Why edge inference instead of a centralized API

Standard AI APIs send every request to a single data center. Workers AI distributes inference to the Cloudflare PoP nearest to your user — typical response time under 100ms at global scale. For low-latency use cases (real-time suggestions, in-browser AI, edge classification), this is a meaningful architectural difference. For batch workloads where latency doesn't matter, a centralized API is simpler.

Sources: Workers AI Quickstart — Cloudflare Docs · Workers AI Models Catalog · Workers AI Pricing and Free Tier · Workers AI Product Page — Cloudflare