We lifted 6 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| explodinggradients/ragas | pwc_unofficial | 4 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| cacher | Ran | explodinggradients/ragas/src/ragas/cache.py code served (permissive licence) · get_code("8187466746dd8d3f") |
| new_group | Ran | explodinggradients/ragas/src/ragas/callbacks.py code served (permissive licence) · get_code("1125bcb828fbea16") |
| parse_run_traces | Ran | explodinggradients/ragas/src/ragas/callbacks.py code served (permissive licence) · get_code("b510616f4989a5ae") |
| run | Ran | explodinggradients/ragas/src/ragas/async_utils.py code served (permissive licence) · get_code("b9d42011ea2f4abc") |
| as_completed | Not yet run | explodinggradients/ragas/src/ragas/async_utils.py code served (permissive licence) · get_code("0b25e64377d6f17d") |
| run_async_tasks | Not yet run | explodinggradients/ragas/src/ragas/async_utils.py code served (permissive licence) · get_code("bb58276206a9b053") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We introduce Ragas (Retrieval Augmented Generation Assessment), a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines. RAG systems are composed of a retrieval and an LLM based generation module, and provide LLMs with knowledge from a reference textual database, which enables them to act as a natural language layer between a user and textual databases, reducing the risk of hallucinations. Evaluating RAG architectures is, however, challenging because there are several dimensions to consider: the ability of the retrieval system to identify relevant and focused context passages, the ability of the LLM to exploit such passages in a faithful way, or the quality of the generation itself. With Ragas, we put forward a suite of metrics which can be used to evaluate these different dimensions \textit{without having to rely on ground truth human annotations}. We posit that such a framework can crucially contribute to faster evaluation cycles of RAG architectures, which is especially important given the fast adoption of LLMs.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2309.15217")
get_code_for_paper("2309.15217")
have("2309.15217")
Connect an agent — have() is free.