Your LLM App Is a Black Box Without This: Langfuse Traces Every Call in 5 Lines

TL;DR: Add Langfuse to any LLM app with one import swap and get a searchable trace of every prompt, response, token cost, and latency — open-source, self-hostable, works with LangChain and the Anthropic SDK.

What you'll be able to do after this: trace why a specific request returned wrong output; compare latency and token cost across model versions; catch regressions when you update a prompt or switch models.

Three things to know going in:

• Langfuse is open-source (Apache 2), self-hostable with Docker Compose in about 15 minutes, and has a managed cloud option for getting started
• Traces are async — events queue locally and flush in batches, so adding tracing doesn't add latency to your app's response time
• Beyond traces, you can attach evaluation scores to any generation and run batch evals across prompt versions — the trace store doubles as your evaluation dataset

Set it up:

pip install langfuse
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_BASE_URL=https://cloud.langfuse.com  # or your self-hosted URL

For the OpenAI SDK — one import swap, zero other changes:

from langfuse.openai import openai  # replaces: from openai import OpenAI

completion = openai.chat.completions.create(
    name="classify-support-ticket",   # shows as the trace name in the UI
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a support ticket classifier."},
        {"role": "user", "content": ticket_body},
    ],
)

Every call now appears in the Langfuse dashboard with its latency, token count, cost estimate, and the exact prompt-response pair. "Why did this ticket get classified wrong?" becomes a 10-second search.

For the Anthropic SDK — use @observe:

from langfuse.decorators import observe, langfuse_context
from anthropic import Anthropic

client = Anthropic()

@observe(as_type="generation")
def classify(ticket: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-5-20260628",
        max_tokens=256,
        messages=[{"role": "user", "content": ticket}],
    )
    langfuse_context.update_current_observation(
        usage={"input": response.usage.input_tokens, "output": response.usage.output_tokens}
    )
    return response.content[0].text

Call langfuse.flush() at the end of any short-lived script — without it, buffered traces may not upload before the process exits.

Real limits: The managed cloud tier has trace volume and retention limits; self-hosting removes both but adds infrastructure. The flush() call is easy to forget in scripts. And the import-swap approach for OpenAI captures calls automatically but doesn't capture calls made through LangChain, LlamaIndex, or custom wrappers — for those, use the callback integrations or the @observe decorator on your pipeline functions.

Sources: Get Started with Tracing — Langfuse docs · Complete Langfuse Observability and Evaluation Pipeline — MarkTechPost · Get Started with Langfuse — YouTube · Langfuse 2026 Guide — QASkills