Langfuse

Langfuse is an open source observability platform specifically designed for LLM-powered applications. As AI applications move from prototype to production, understanding why they behave as they do becomes critical, and traditional application monitoring tools are not equipped to handle the specific challenges of language model applications.

The core capability is tracing. Langfuse wraps LLM API calls and captures the complete input-output-metadata for each request, storing it in a structured format that allows debugging, analysis, and optimisation. When an AI application produces an unexpected or poor-quality response, traces allow developers to see exactly what prompt was sent, which model was used, what the response was, how much it cost, and how long it took.

Evaluation features allow scoring LLM outputs either manually (human annotation), through LLM-based evaluation (asking a judge model to rate responses), or through custom scoring functions. Systematically evaluating output quality across many samples is essential for AI application quality assurance that goes beyond testing individual examples.

Datasets for testing allow capturing representative inputs, running the application against them, and comparing results across different prompt versions or model changes. This enables regression testing for AI applications where quality can change subtly when prompts or models are updated.

The open source nature allows self-hosting for complete data control, which is important for teams whose LLM applications process sensitive data. The cloud version is available for teams that prefer managed infrastructure.

For teams building serious LLM applications in production, Langfuse or a similar observability tool is essential infrastructure. LangSmith from LangChain is the primary competitor.