See Inside Every LLM Call: Add Open-Source Observability to Any AI App With Arize Phoenix

TL;DR: Arize Phoenix gives you W&B Weave-style LLM tracing + evaluations but self-hosted and open-source — two pip installs, one .instrument() call, and every LLM call in your app becomes a searchable trace.

What you'll be able to do after this:

• Run a local Phoenix server that captures distributed traces from any LLM app — OpenAI, Anthropic, LangChain, LlamaIndex, or raw API calls — with a single auto-instrumentation call
• Open the Phoenix UI at http://localhost:6006 and inspect every span: latency, token counts, prompts sent, completions received, RAG retrievals, and tool calls in one view
• Run built-in LLM-as-judge evaluators (hallucination, faithfulness, relevance) against your captured traces to catch regressions before users see them

Why Phoenix over W&B Weave or LangSmith?

All three solve the same core problem: you can't debug what you can't see. Phoenix's differentiator is self-hosted + open-source: your traces never leave your infrastructure. This matters for HIPAA, SOC 2, and any project where sending LLM inputs/outputs to a SaaS is a non-starter. By 2026 it has 9,000+ GitHub stars and native support for OpenInference — the OpenTelemetry-based standard for LLM trace semantics.

Walk-through

Install:

pip install arize-phoenix arize-phoenix-otel \
    openinference-instrumentation-anthropic \
    openinference-instrumentation-openai \
    openinference-instrumentation-langchain

Launch Phoenix locally:

import phoenix as px

session = px.launch_app()   # starts UI at http://localhost:6006
print(session.url)           # open this in your browser

Or run it as a persistent server (survives script restarts):

docker run -p 6006:6006 -p 4317:4317 arizephoenix/phoenix:latest

Register the tracer and instrument your stack:

from phoenix.otel import register
from openinference.instrumentation.anthropic import AnthropicInstrumentor
from openinference.instrumentation.langchain import LangChainInstrumentor

# Point traces at your Phoenix instance
tracer_provider = register(
    project_name="my-rag-app",
    endpoint="http://localhost:6006/v1/traces",
)

# One call per framework — no code changes elsewhere
AnthropicInstrumentor().instrument(tracer_provider=tracer_provider)
LangChainInstrumentor().instrument(tracer_provider=tracer_provider)

Now run your app normally — every anthropic.messages.create() call, every LangChain chain invocation, and every retrieval step is captured automatically.

Run evaluations on captured traces:

import phoenix as px
from phoenix.evals import HallucinationEvaluator, RelevanceEvaluator, run_evals
from phoenix.evals import AnthropicModel

client = px.Client()
traces_df = client.get_spans_dataframe(project_name="my-rag-app")

eval_model = AnthropicModel(model="claude-sonnet-5")
results = run_evals(
    dataframe=traces_df,
    evaluators=[HallucinationEvaluator(eval_model), RelevanceEvaluator(eval_model)],
    provide_explanation=True,
)
px.log_evaluations(*results, project_name="my-rag-app")

Evaluation results appear as annotations on each trace in the UI — you can sort and filter by hallucination score, compare runs, and catch regressions before they reach production.

What you see in the UI

Each trace shows the full call tree: the top-level query → the retrieval span (with the chunks fetched) → the LLM span (with the exact prompt + completion + token counts). You can click into any span to see latency, attributes, and error details. The experiments tab lets you A/B compare prompts or models by replaying a dataset through two different app versions.

Sources: Arize Phoenix docs · GitHub: Arize-ai/phoenix (9k+ stars) · OpenInference instrumentation packages — arize-ai.github.io · Arize Phoenix LLM observability guide 2026