Debug Your LLM Agent by Watching Every Decision: LangSmith Tracing in 15 Minutes

TL;DR: Set LANGSMITH_TRACING=true + your API key, and every model call, tool use, and decision your agent makes appears as a searchable, timestamped trace — two env vars and zero code changes if you're on LangChain/LangGraph.

What you'll be able to do after this:

• See every LLM call, tool invocation, and chain step your agent takes, with full inputs, outputs, and latency
• Click into any failing trace and pinpoint exactly which step produced the wrong output — no print statements, no guessing
• Track token counts and estimated cost per trace so you can identify expensive or slow steps and optimize them

Resource: LangSmith Tutorial: Observability and Tracing for AI Agents (YouTube, April 2026) — walks through full setup from account creation to reading production traces.

Quick setup

# 1. Create a free account at smith.langchain.com
#    Settings → API Keys → Create API Key

export LANGSMITH_API_KEY="ls__..."
export LANGSMITH_TRACING=true
export LANGSMITH_PROJECT="my-agent"   # organizes traces by project

If you're using LangChain or LangGraph, that's all — every model call and tool use is traced automatically. For raw Anthropic or OpenAI calls, add @traceable:

from langsmith import traceable

@traceable
def call_model(prompt: str) -> str:
    response = anthropic.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

@traceable(run_type="tool")
def search_docs(query: str) -> list[str]:
    return vector_store.similarity_search(query)

Each @traceable function becomes a span. Nested calls create nested spans — the full call tree captures automatically.

What you see in the UI

Click any trace to inspect: inputs and outputs for each LLM call, which tools fired and with what arguments, total latency broken down by step, and token counts with estimated cost. The "Playground" button opens any trace step for interactive rerunning with edited inputs — the fastest way to iterate on a broken prompt.

Production tips

• Add LANGSMITH_SAMPLING_RATE=0.1 to trace only 10% of requests once the setup is validated
• Use LANGSMITH_PROJECT to separate dev, staging, and prod traces
• Subagents do not inherit parent tracing by default — enable it explicitly per subagent if you need cross-agent traces

Sources: LangSmith Tutorial: Observability and Tracing for AI Agents (YouTube, April 2026) · LangSmith Tracing Quickstart — LangChain Docs · How to Trace and Monitor AI Agents with LangSmith — FreeCodeCamp