Ragas

Ragas is a Python library for the systematic evaluation of applications built on large language models, in particular retrieval-augmented generation (RAG) pipelines. Its goal is to replace informal “vibe checks” with repeatable evaluation loops.

It combines LLM-driven metrics, such as faithfulness, answer relevance or context precision and recall, with an experiments-first approach: changes are made, evaluations are run, results are compared and the application is improved iteratively. Ragas can also generate test datasets and integrates with common LLM frameworks and observability tools.

Ragas is free and open source under the Apache 2.0 licence.