Guide

Observability Setup

Configuration steps for the three most common LLM tracing stacks: OpenTelemetry (vendor-neutral), LangSmith (best for LangChain / LangGraph), and Langfuse (open source, self-hostable). Pick one as your primary backend — you can always fan out later via the OTel collector.

For the bigger picture on what to instrument and why, see Production Deployment & Observability.

Choosing a backend

OpenTelemetry

Vendor-neutral protocol. Use when you already have an APM (Datadog, Honeycomb, Grafana Tempo, Phoenix) or want to keep options open.

LangSmith

Zero-config for LangChain / LangGraph. Best when you also want datasets, evals, and prompt management in the same UI.

Langfuse

Open source, self-hostable, OTel-native. Best when you need data residency or want to own the storage.

1. OpenTelemetry + OpenInference

OpenInference adds LLM-specific span attributes (model, prompt, tokens, cost) on top of OTel. Ship to any OTLP-compatible backend.

Install

bash

pip install \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp \
  openinference-instrumentation-openai \
  openinference-instrumentation-langchain

Environment

bash

# OTLP collector or vendor endpoint
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.your-vendor.com"
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer <TOKEN>"
export OTEL_SERVICE_NAME="checkout-agent"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=prod,service.version=1.4.0"

Bootstrap

python

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
from openinference.instrumentation.langchain import LangChainInstrumentor

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

OpenAIInstrumentor().instrument()
LangChainInstrumentor().instrument()

Verify: run one request, then check your backend for a span named ChatOpenAI with llm.model_name and llm.token_count.total attributes.

2. LangSmith

Auto-traces every LangChain / LangGraph call with no code changes — just env vars. For non-LangChain code, wrap functions with @traceable.

Install

bash

pip install langsmith
# JS: npm i langsmith

Environment

bash

export LANGSMITH_TRACING="true"
export LANGSMITH_API_KEY="lsv2_..."
export LANGSMITH_PROJECT="prod-checkout-agent"
# Self-hosted / EU region:
# export LANGSMITH_ENDPOINT="https://eu.api.smith.langchain.com"

Trace custom code

python

from langsmith import traceable
from openai import OpenAI
from langsmith.wrappers import wrap_openai

client = wrap_openai(OpenAI())  # auto-traces OpenAI SDK

@traceable(run_type="chain", metadata={"version": "v3", "tenant": "acme"})
def answer(question: str) -> str:
    r = client.chat.completions.create(
        model="gpt-4o-2024-08-06",
        messages=[{"role": "user", "content": question}],
    )
    return r.choices[0].message.content

Verify: open the LangSmith project — the run should appear within a few seconds with full input/output, token counts, and latency.

3. Langfuse

Use Langfuse Cloud or self-host with Docker Compose. The Python / JS SDKs send traces over HTTP; OTel ingestion is also supported.

Install

bash

pip install langfuse
# JS: npm i langfuse

Environment

bash

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"  # or your self-hosted URL

Trace OpenAI & LangChain

python

# Drop-in OpenAI wrapper
from langfuse.openai import openai

r = openai.chat.completions.create(
    model="gpt-4o-2024-08-06",
    messages=[{"role": "user", "content": "Hello"}],
    name="greeting",            # span name in Langfuse
    metadata={"tenant": "acme"},
)

# LangChain callback handler
from langfuse.callback import CallbackHandler
handler = CallbackHandler()
chain.invoke({"input": "..."}, config={"callbacks": [handler]})

Self-host (Docker)

bash

git clone https://github.com/langfuse/langfuse.git
cd langfuse
docker compose up -d
# UI: http://localhost:3000

Cross-cutting practices

Propagate trace IDs from the frontend (traceparent header) so a user click and the model call share one trace.
Redact PII in a span processor before export — emails, tokens, payment data.
Sample smartly — 100% of errors and slow requests, e.g. 10% of healthy traffic.
Tag every run with environment, service.version, prompt.version, tenant.
One backend, fanned out — send to the OTel collector and let it route to LangSmith / Langfuse / Datadog in parallel.

Verification checklist

A test request produces a trace within 10 seconds.
Spans include model name, prompt, completion, token counts, and latency.
Errors surface as failed spans with stack traces.
Trace IDs appear in app logs for cross-correlation.
PII redaction rules verified against a known-bad payload.