io.github.HZYAI/ragscoreai

RAGScore

Verified · yesterday

Generate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.

Install

claude mcp add ragscore -- uvx ragscore

Requires: OPENAI_API_KEY, ANTHROPIC_API_KEY

Our take

RAG evaluation in two commands: generate QA pairs from your documents, fire them at your RAG endpoint, and get accuracy scores with per-question failure diagnosis. Works with any LLM including local Ollama, which keeps evaluation private, and the notebook API with built-in plots makes it genuinely quick to audit a pipeline. Young project from a new org — the methodology is LLM-as-judge with all the caveats that implies, so read the scored failures rather than trusting the single headline number.

reviewed by hand · 2026-07-05

Something wrong or dead here? Report it

Verification record

last verified
yesterday
github stars
22
last commit
38 days ago
archived
no
license
Apache-2.0
in registry since
2026-05-19

github.com/HZYAI/RagScore