vectara/hallucination-leaderboard
View on GitHub ↗ ★ 3,323 Python
Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents
26 mentions on Hacker News by 11 people: 3 stories, 23 comments · first 2023-11-06 · latest 2025-12-18
20082026
-
…is now on Vectara hallucination leaderboard, and rated at 13.5% grounded hallucination rate. https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2025-12-18 · in Gemini 3 Flash: Frontier intelligence built for speed -
…hallucination rate, similar to qwen3-next-80b-a3b-thinking (9.3%) but of course it is much smaller. https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2025-12-17 · in Nvidia Nemotron 3 Family of Models -
…leaderboard: it scores at 10.9% hallucination rate, better than Gemini-3, GPT-5.1-high or Grok-4. https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2025-11-25 · in Claude Opus 4.5 -
…"This evaluates how often an LLM introduces hallucinations when summarizing a document." https://github.com/vectara/hallucination-leaderboard If the figures on this leaderboard are to be trusted, many frontier and near-frontier models are…
comment by nopinsight · 2025-10-22 · in AI assistants misrepresent news content 45% of the time -
…be missing Gemni Pro (non-experimental). [1] https://huggingface.co/spaces/vectara/leaderboard [2] https://github.com/vectara/hallucination-leaderboard/tree/ma...
comment by SubiculumCode · 2025-09-09 · in Hallucination Risk Calculator -
…seen them add spurious details or anything, which is in line with most benchmarks I've seen ( https://github.com/vectara/hallucination-leaderboard ), and they always execute on such simple text-transformation commands first-time, and…
comment by logicprog · 2025-08-13 · in Training language models to be warm and empathetic makes them less reliable -
…Research, and ask it to write a report about a topic that you're intimately familiar with. [0] - https://github.com/vectara/hallucination-leaderboard
comment by hnhn34 · 2025-05-11 · in Update turns Google Gemini into a prude, breaking apps for trauma survivors -
…[2] for more data on hallucinations. [1]: https://openai.com/index/o3-o4-mini-system-card/ [2]: https://github.com/vectara/hallucination-leaderboard/
comment by Nauxuron · 2025-05-11 · in In 2025, venture capital can't pretend everything is fine any more -
…have liked (TL;DR slightly higher than Llama3). Check the numbers on the hallucination leaderboard: https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2025-04-08 · in The Llama 4 herd -
…it also maintained a very low hallucination rate of 1.1% on Vectara Hallucination Leaderboard: https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2025-03-26 · in Gemini 2.5 -
This is why metrics and leaderboards like these are so important (but under reported on): https://github.com/vectara/hallucination-leaderboard https://www.kaggle.com/facts-leaderboard Google Gemni models seem to lead...hopefully the…
comment by SubiculumCode · 2025-02-26 · in The Deep Research problem -
…does extremely well on the Hallucination Evaluation Leaderboard, at 1.3% hallucination rate https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2024-12-11 · in Gemini 2.0: our new AI model for the agentic era -
Check out Granite 3.0 on the hallucination leaderboard: https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2024-10-24 · in IBM Granite 3.0: open enterprise models -
Great release. Models just added to Hallucination Leaderboard: https://github.com/vectara/hallucination-leaderboard . TL;DR: * 90B-Vision: 4.3% hallucination rate * 11B-Vision: 5.5% hallucination rate
comment by ofermend · 2024-09-27 · in Llama 3.2: Revolutionizing edge AI and vision with open, customizable models -
…issues that may not have been apparent during development. [1] Vectara hallucination leaderboard https://github.com/vectara/hallucination-leaderboard [2] On epistemic black holes: How self-sealing belief systems develop and evolve".…
comment by fsiefken · 2024-09-01 · in AI is growing faster than companies can secure it, warn industry leaders -
…and some other metrics internally to focus prompt engineering, model selection, and model training: https://github.com/vectara/hallucination-leaderboard
comment by eskibars · 2024-06-28 · in CriticGPT: Finding GPT-4's mistakes with GPT-4 -
This model is great. Jumped immediately to 2nd place on HHEM leaderboard: https://github.com/vectara/hallucination-leaderboard
comment by ofermend · 2024-04-25 · in Snowflake Arctic Instruct (128x3B MoE), largest open source model -
Llama 2 (and variants). Has the lowest hallucination rate ( https://github.com/vectara/hallucination-leaderboard ), and its open source and so we know what went into it, and the community can improve it
comment by simonhughes22 · 2023-11-22 · in Ask HN: Best Alternatives to OpenAI ChatGPT? -
…( https://venturebeat.com/ai/google-bard-fails-to-deliver-on-i... .) and prone to hallucination ( https://github.com/vectara/hallucination-leaderboard ).
comment by simonhughes22 · 2023-11-22 · in Inflection-2: the next step up -
You can view the responses here in the linked csv file: https://github.com/vectara/hallucination-leaderboard
comment by simonhughes22 · 2023-11-06 · in Show HN: Open-source model and scorecard for measuring hallucinations in LLMs -
…today https://huggingface.co/vectara/hallucination_evaluation_mode... (along with this leaderboard: https://github.com/vectara/hallucination-leaderboard ). GPT-4 was already in the lead. Looking forward to seeing GPT4-Turbo there soon.
comment by ofermend · 2023-11-06 · in New models and developer products -
…HEM) today https://huggingface.co/vectara/hallucination_evaluation_mode... , with a leaderboard: https://github.com/vectara/hallucination-leaderboard GPT-4 was already in the lead. Looking forward to seeing GPT4-Turbo there soon.
comment by ofermend · 2023-11-06 · in GPTs: Custom versions of ChatGPT