Beyond `ollama run`: Customize Any Local LLM's Behavior with an Ollama Modelfile

Ollama's Modelfile is its Dockerfile equivalent: define a system prompt and parameters once, build a named model, and invoke it anywhere — CLI or API — with no per-session flags or copy-pasted prompts.

What you'll be able to do after this:

• Create named local model variants with baked-in system prompts (e.g., a "code-reviewer" that always focuses on security and performance)
• Set temperature, context window, and stop sequences once in a file rather than on every API call
• Run any of the 45,000+ GGUF checkpoints on HuggingFace Hub directly through Ollama, pinned to the right quantization level

What a Modelfile is

A Modelfile is a plain text file that configures how Ollama loads and runs a model. It works like a Dockerfile: FROM sets the base model, then PARAMETER, SYSTEM, TEMPLATE, and ADAPTER instructions tune its behavior. You create a named model from it once, and that configuration is permanent — no flags needed at runtime.

Write your first Modelfile

FROM llama3.1:8b

PARAMETER temperature 0.2
PARAMETER num_ctx 32768

SYSTEM """
You are a code reviewer focused on security vulnerabilities, performance bottlenecks, and maintainability. Explain the risk or cost of each issue before suggesting a fix. Skip style suggestions unless they affect readability.
"""

Build the model and run it:

ollama create code-reviewer -f ./Modelfile
ollama run code-reviewer "Review this function: ..."

Every session with code-reviewer opens with that system prompt and those parameters — no extra configuration in your calling code.

Pull any HuggingFace Hub GGUF

HuggingFace Hub hosts 45,000+ GGUF checkpoints. Reference them directly in your Modelfile or run them inline:

FROM hf.co/bartowski/Llama-3.1-8B-Instruct-GGUF:Q4_K_M

PARAMETER temperature 0.1
SYSTEM "You are a concise technical assistant."

Which quantization to pick:

Format	VRAM (7B model)	Accuracy retained	Best for
Q4_K_M	~4 GB	~98%	Default — the sweet spot
Q8_0	~8 GB	~99.5%	When you have the VRAM
F16	~14 GB	100%	Highest fidelity, fine-tuning

Q4_K_M is almost always the right starting point. You will not notice the difference from full precision on instruction-following tasks.

Use the Modelfile-configured model in code

Once created, your named model works in the Ollama REST API exactly like any other model:

import openai

client = openai.OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required field, ignored
)

resp = client.chat.completions.create(
    model="code-reviewer",   # your Modelfile model
    messages=[{"role": "user", "content": code_snippet}],
)

The system prompt and parameters defined in the Modelfile apply automatically. Your application carries no configuration — it just calls the model by name.

Sources: Ollama Modelfile reference — GitHub · GGUF & Modelfile: Power User's Guide — DEV Community · Ollama + HuggingFace Hub — HuggingFace Docs · Ollama Tutorial for Beginners 2026 — YouTube