LLM observability
Software that traces, logs and monitors calls to AI models and agents in production.
Includes: LLM tracing, prompt management and cost and latency monitoring. Not: general application monitoring (Observability and monitoring).
For example: Langfuse, LangSmith, Helicone.
12 listed products
| Product | What it says it does |
|---|---|
| W&B Weave | W&B Weave is an observability and evaluation platform that helps you track, evaluate, and improve your agents and LLM applications. |
| LangSmith | LangSmith helps you and your team develop and evaluate language models and intelligent agents. |
| OpenLIT | OpenTelemetry-native agent observability, evals, guardrails, prompt management, and cost and GPU monitoring |
| AgentOps | The leading developer platform for building AI agents and LLM apps. |
| Braintrust | The active observability platform for agents |
| Helicone | to route, debug, and analyze their applications. |
| Laminar | Laminar is an open-source agent observability platform. |
| Langfuse | Trace, evaluate, and improve AI agents with one open platform. |
| Opik | Debug, evaluate, and monitor your LLM applications, RAG systems, |
| Pydantic Logfire | Full-stack observability and evaluation for AI apps |
| Traceloop OpenLLMetry | Open-source Observability for LLMs with OpenTelemetry |
| Arize Phoenix | AI Observability and Evaluation |
12 of these are read from their own pages. A product appears where AirStack's category list places it; 7,865 of 13,386 listed products carry no category yet.