cpldcpu/MisguidedAttention
View on GitHub ↗ ★ 489 Python
A collection of prompts to challenge the reasoning abilities of large language models in presence of misguiding information
8 mentions on Hacker News by 7 people: 1 story, 7 comments · first 2024-11-10 · latest 2026-04-23
20082026
-
…foot. At the same time, I can tell you from my experience that a lot of the Misguided attention ( https://github.com/cpldcpu/MisguidedAttention ) type of stump questions work at a much lower rate with newer models. Progress is being made,…
comment by curioussquirrel · 2026-04-23 · in Ask HN: Is the ongoing AI research driving LLM models to be better? -
…original problems of this nature, I defer to a 20:80 ratio of genuine improvement to recall. [0] https://github.com/cpldcpu/MisguidedAttention
comment by pona-a · 2025-02-01 · in OpenAI O3-Mini -
There is also a curated benchmark just for those famous problems slightly variated: https://github.com/cpldcpu/MisguidedAttention/tree/main/eval
comment by WiSaGaN · 2025-01-01 · in 30% drop in O1-preview accuracy when Putnam problems are slightly variated -
…really hard to make it perform that other, similar but not quite the same task. In a similar vein - https://github.com/cpldcpu/MisguidedAttention/tree/main/eval...
comment by NitpickLawyer · 2024-12-31 · in Deepseek: The quiet giant leading China’s AI race -
…astronaut is IN China. This has been a sobering reminder to me as discussion of AGI heats up. [1] https://github.com/cpldcpu/MisguidedAttention
comment by botro · 2024-12-20 · in OpenAI O3 breakthrough high score on ARC-AGI-PUB -
…how they make use of them. There's also a bunch of interesting examples along the same lines here: https://github.com/cpldcpu/MisguidedAttention . Although I should note that even top OpenAI models have troubles with much of this stuff.…
comment by int_19h · 2024-12-04 · in Amazon Nova -
…to describe a piece of code, you'll only get the rephrasing of what the code already says. - [0] https://github.com/cpldcpu/MisguidedAttention
comment by pona-a · 2024-12-01 · in The Opposite of Documentation is Superstition (2020)