Trace Every Claude Call Without Changing Your Code: W&B Weave for LLM Observability

W&B Weave auto-patches the Anthropic Python SDK at weave.init() — every messages.create call becomes a traced record with inputs, outputs, token counts, cost, and latency, visible in the W&B dashboard with no additional code.

What you'll be able to do after this:

• Capture every Anthropic SDK call as a structured trace — prompt, response, token counts, cost, and latency — with a single weave.init() call and zero changes to existing API calls
• Trace your own application logic with the @weave.op() decorator, building a hierarchical call tree that shows exactly where your app's time and tokens go
• Run a scored evaluation across a test dataset using LLM-as-judge scorers, with a comparison dashboard generated automatically in the W&B UI

---

Langfuse (covered previously) uses callbacks and context managers. W&B Weave uses Python decorators and auto-patches the Anthropic SDK at import time. If you're already using W&B for ML training runs, Weave plugs into the same workspace so your model experiments and LLM app evals live side by side.

Step 1: Install and auto-trace the Anthropic SDK

pip install weave anthropic

import weave
import anthropic

weave.init("my-claude-app")  # free tier; creates a project at wandb.ai

client = anthropic.Anthropic()

# Weave auto-patches the Anthropic SDK — every call below is traced automatically
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)
print(response.content[0].text)

Run this, then open wandb.ai/<username>/my-claude-app — you'll see the full prompt, response, input/output tokens, cost, and latency for every call, with no changes to your calling code.

Step 2: Trace your application code

The @weave.op() decorator marks a function for tracing. Every call logs its arguments and return value, and nests inside any SDK calls the function makes:

@weave.op()
def answer_question(question: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        messages=[{"role": "user", "content": question}]
    )
    return response.content[0].text

Step 3: Run a scored evaluation

@weave.op()
def correctness_scorer(model_output: str, target: str) -> dict:
    judge = client.messages.create(
        model="claude-haiku-4-5-20251001",  # cheap model as judge
        max_tokens=64,
        messages=[{
            "role": "user",
            "content": f"Does '{model_output}' correctly address '{target}'? Reply 'yes' or 'no' only."
        }]
    )
    return {"correct": 1 if "yes" in judge.content[0].text.lower() else 0}

evaluation = weave.Evaluation(
    dataset=[
        {"question": "Capital of France?", "target": "Paris"},
        {"question": "Capital of Germany?", "target": "Berlin"},
        {"question": "Capital of Japan?", "target": "Tokyo"},
    ],
    scorers=[correctness_scorer],
)

import asyncio
results = asyncio.run(evaluation.evaluate(answer_question))
print(results)  # {"correct": {"mean": 1.0}}

Weave runs every example, applies your scorers, and writes a comparison dashboard in the W&B UI showing per-example pass/fail, aggregate scores, and the full trace tree for each call.

Sources: Anthropic integration — W&B Weave docs · Build an evaluation · W&B Weave: Confidently iterate on LLM-powered applications — YouTube