Hacker GitHub

← All repos

colingfly/sparse-eval

View on GitHub ↗ ★ 6 Python

Reliability evaluation for AI agents, APIs, and pipelines. LLM-judged scoring, schema validation, latency profiling, failure mining, and root cause analysis

3 mentions on Hacker News by 1 person: 1 story, 2 comments · first 2026-03-13 · latest 2026-03-17

Mentions per year 3 mentions · busiest year 2026 (3)
20082026
AllStoriesComments NewestTopOldest
  1. If helpful, I put together a small tool (Cane Eval) to structure this kind of eval loop: github.com/colingfly/cane-eval Still early, but would love feedback if anyone is working on similar problems.
    comment by colinfly · 2026-03-17 · in What broke when I tried to evaluate an AI agent in production
  2. …cane-eval rca tests.yaml --threshold 60 cane-eval run tests.yaml --mine --export dpo GitHub: https://github.com/colingfly/cane-eval MIT licensed, pure Python, uses the Anthropic API. Happy to answer questions about the approach.
    comment by colinfly · 2026-03-13 · in Open-source LLM-as-judge eval suite with root cause analysis and failure mining