Skip to content
All tags

#llm-observability

3 posts
ai deep-dive

Arize Phoenix: Turning Traces into Datasets, Experiments, and Evaluators

Phoenix is an MIT-licensed open-source LLM observability and evaluation platform. It collects traces with OpenTelemetry and OpenInference, turns production failures into versioned datasets, compares prompt, model, or RAG changes in experiments, then writes code, human, and LLM evaluator scores back as annotations. It is not Arize AX, and self-hosting defaults require security work.

#arize-phoenix#llm-observability#evaluation#opentelemetry#openinference#ai-agent
ai deep-dive

Helicone Deep Dive: LLM Gateway, Request Tracing, and Cost Analytics

Helicone is an open-source LLM gateway and observability platform: requests sent through its compatible endpoint automatically capture model, latency, tokens, cost, and custom properties, while managed credits or BYOK enable routing and fallbacks.

#helicone#llm-observability#ai-gateway#tracing#llmops#open-source
ai deep-dive

LangSmith Deep Dive: From Agent Traces to Offline and Online Evaluation

LangSmith structures LLM applications as projects, traces, runs, and threads, then uses datasets, evaluators, and experiments to turn production failures into offline regression tests. It observes any LLM application and does not require LangChain.

#langsmith#llm-observability#tracing#evaluation#llmops#ai-agent