suzgunmirac/BIG-Bench-Hard

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

3 mentions on Hacker News by 3 people: 0 stories, 3 comments · first 2023-03-22 · latest 2023-11-23

Mentions per year 3 mentions · busiest year 2023 (3)

20082026

All Stories Comments Newest Top Oldest

…recall which. Here is another from BBH [1]: >>> (-1 + 2 + 9 * 5) = ... (-1 + 2 + 9 * 5) = 43 [1] https://github.com/suzgunmirac/BIG-Bench-Hard/tree/main
comment by sorokod · 2023-11-23 · in Exponentially faster language modelling
…category except SAT-Math. We definitely need harder benchmarks. So far, there's BIG-Bench Hard https://github.com/suzgunmirac/BIG-Bench-Hard and just published, Advanced Reasoning Benchmark https://arb.duckai.org/
comment by lhl · 2023-07-27 · in LLaMA2 Chat 70B outperformed ChatGPT
…models seem to be lagging for reasoning tasks vs OpenAI ones at the moment - see the graphs at https://github.com/suzgunmirac/BIG-Bench-Hard
comment by letitgo12345 · 2023-03-22 · in Bard is much worse at puzzle solving than ChatGPT