simonw/pelican-bicycle
View on GitHub ↗ ★ 218 Shell
LLM benchmark: Generate an SVG of a pelican riding a bicycle
8 mentions on Hacker News by 7 people: 0 stories, 8 comments · first 2024-11-28 · latest 2026-09-06
20082026
-
The [Pelican Benchmark]( https://github.com/simonw/pelican-bicycle ) is in the LLM's training data and probably not a useful indicator of improving LLM capabilities any more. For the past two years, I've run the Little Dorrit Editor…
comment by svcrunch · 2026-09-06 · in GPT-6 Astra soundly defeats Fable 5.1 on recognizing handwritten corrections -
…therefore you can tune a model as much as possible. One of real benchmarks is drawing of pelican - https://github.com/simonw/pelican-bicycle - Simon Willison made it for his llms’ tests. If you want really find out a model that works for…
comment by sixtyj · 2026-06-18 · in Local Qwen isn't a worse Opus, it's a different tool -
…animation. I thought it would be funny to include the pelicans on bicycles as well (you know, https://github.com/simonw/pelican-bicycle ). The linked animation is Opus 4.7 and it did a great job. It even added a little jump! Do you think…
comment by prathje · 2026-04-29 -
https://github.com/simonw/pelican-bicycle/blob/main/README.m...
comment by qingcharles · 2025-04-13 · in Nice things with SVG -
More recently, `gemini-exp-1206` did quite well [1]. [1]: https://github.com/simonw/pelican-bicycle/blob/main/README.m...
comment by danielcorin · 2024-12-16 · in Pelicans on a bicycle -
…https://simonwillison.net/2024/Dec/6/gemini-exp-1206/ Claude 3.5 Sonnet is in second place: https://github.com/simonw/pelican-bicycle?tab=readme-ov-file...
comment by simonw · 2024-12-16 · in Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning -
…that is I thought this model did very well, especially compared to others that I've tried: https://github.com/simonw/pelican-bicycle?tab=readme-ov-file...
comment by simonw · 2024-11-28 · in QwQ: Alibaba's O1-like reasoning LLM -
For comparison, this is what other models produce: https://github.com/simonw/pelican-bicycle/blob/main/README.m...
comment by tethys · 2024-11-28 · in QwQ: Alibaba's O1-like reasoning LLM