Most LLM evaluations treat the application as an end-to-end black box.
However, LLM applications require evaluations and tracing at the component level, because the problem could be anywhere inside: in the retriever, tool call, or the LLM itself.
In DeepEval, this can be done in just 3 lines of code:
- Trace individual LLM components (tools, retrievers, generators) using the @observe decorator.
- Attach different metrics to each part.
- Get a visual breakdown of what works and what doesn't.
That's it.
No need to refactor existing code.
See the example below for an RAG application.
DeepEval is completely open-source (14k+ stars), and it's easy to deploy independently so that your data stays with you.
GitHub
••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore