ed-is-ai/featherbench
View on GitHub ↗ ★ 4 Python
A featherweight framework for building your own LLM benchmarks
8 mentions on Hacker News by 1 person: 0 stories, 8 comments · first 2026-08-07 · latest 2026-08-31
20082026
-
…the evidence + code to run this + what you have todo to adapt it for your own use cases is all here https://github.com/ed-is-ai/featherbench Encourage everyone to eval like the devil
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
…code for running the evals, evals are all shared. Anyone can run this to verify it for themselves https://github.com/ed-is-ai/featherbench
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
…The results all there, and code if you want to run the evals (or make your own - it's very easy) https://github.com/ed-is-ai/featherbench You might have read about Ox Alpha aka glm5.3-flash, well I updated to include that
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
…above. And you can add your own tests if this isn't enough for you... Policy of total transparency https://github.com/ed-is-ai/featherbench
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
…are here for anyone who wants to delve, see if they can reproduce. That is the point of discourse https://github.com/ed-is-ai/featherbench
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
https://github.com/ed-is-ai/featherbench
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
pcwelder - https://github.com/ed-is-ai/featherbench all the data is here if you care to look But thanks for the feedback
comment by ed-is-ai · 2026-08-31 · in GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost -
…python, it's shared on GitHub in-case anyone wants to have a go at building their own Eval suite: https://github.com/ed-is-ai/featherbench . It's designed to be easy to integrate and adapt to any python project where you just want some…
comment by ed-is-ai · 2026-08-07 · in Building the Ed-O-Meter: Notes on Writing My Own LLM Benchmark