Three Things That Break When You Bump Claude Sonnet 5's Model ID: Tokenizer, Thinking, and Sampling Params

Chris Harper

3 min read

Jul 8, 2026 · 04:17 UTC

Workflow

Claude Code

Best Practices

LLM

TL;DR: Swapping claude-sonnet-4-6 for claude-sonnet-5 silently breaks three things — adaptive thinking fires on every call, temperature/top_p/top_k now return 400, and the new tokenizer makes the same prompt cost ~30% more tokens.

If your team just bumped the model ID and moved on, read this. Sonnet 5 bills itself as a drop-in upgrade and mostly is — but it has three behavior changes that will break running code and inflate costs if you don't address them.

1. The new tokenizer: budget everything 30% higher

Claude Sonnet 5 uses a new tokenizer. The same input text produces approximately 30% more tokens than on Sonnet 4.6. Per-token pricing is unchanged ($2/$10 at introductory rates; $3/$15 after August 31), but 30% more tokens means 30% higher cost for the same request.

Practical impact:

max_tokens limits sized close to your expected output length may now truncate. Revisit any hardcoded limits.
Context window capacity in text terms is lower: the same 1M-token window holds less text than on Sonnet 4.6.
Token counting — don't reuse counts measured against Sonnet 4.6; recount with the token counting API against the new model.
Cost estimates for existing workloads are wrong. Recount and rebudget before you flip the switch in production.

import anthropic
client = anthropic.Anthropic()

# Recount your prompt under the new tokenizer
response = client.messages.count_tokens(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": your_prompt}],
)
print(f"Sonnet 5 tokens: {response.input_tokens}")

2. Adaptive thinking is on by default

On Sonnet 4.6, requests without a thinking field ran without thinking. On Sonnet 5, the same requests run with adaptive thinking enabled — Claude decides whether to think before responding, and thinking tokens count toward your max_tokens limit.

To disable explicitly (match Sonnet 4.6 behavior):

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    thinking={"type": "disabled"},   # explicitly off
    messages=[{"role": "user", "content": prompt}]
)

For most agentic workloads this is better — Sonnet 5 thinks only when the task warrants it. But for high-throughput, low-complexity classification or extraction tasks where you want deterministic, fast responses, disable thinking and adjust max_tokens to leave no headroom for thinking tokens.

3. Temperature, top_p, and top_k now return 400

Setting any of these to a non-default value returns a 400 error — enforced at the API level.

# This returns 400 on Sonnet 5:
client.messages.create(
    model="claude-sonnet-5",
    temperature=0.7,   # 400 error
    max_tokens=1024,
    messages=[...]
)

# Remove the parameter entirely:
client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[...]    # default behavior is correct
)

Search your codebase for temperature=, top_p=, and top_k= on any Sonnet call and remove them. Move style/tone guidance into the system prompt instead.

Also: manual extended thinking is removed

# Not supported (400 on Sonnet 5):
thinking = {"type": "enabled", "budget_tokens": 32000}

# Use adaptive thinking instead:
thinking = {"type": "adaptive"}   # on by default; omit or pass this explicitly

Migration checklist

Grep temperature=, top_p=, top_k= — remove all non-default values from Sonnet calls
Grep budget_tokens — migrate to {"type": "adaptive"}
Recount prompts with token counting API against claude-sonnet-5
Revisit all max_tokens limits; add headroom for thinking tokens
Update cost estimates: same text → ~30% more tokens, pricing unchanged per token