spencermountain/wtf_wikipedia
View on GitHub ↗ ★ 858 JavaScript
a pretty-committed wikipedia markup parser
4 mentions on Hacker News by 3 people: 0 stories, 4 comments · first 2018-09-25 · latest 2023-03-25
20082026
-
…we even need structured data in the post-AI age?) Mediawiki is notorious for being hard to parse: * https://github.com/spencermountain/wtf_wikipedia#ok-first- - why it's hard * https://techblog.wikimedia.org/2022/04/26/what-it-takes-to-p...…
comment by rjh29 · 2023-03-25 · in Experimental library for scraping websites using OpenAI's GPT API -
Parsing wikitext is insanely hard. Using HTML is a good compromise. See also: * https://github.com/spencermountain/wtf_wikipedia#ok-first- * https://osr.cs.fau.de/wp-content/uploads/2017/09/wikitext-pa... *…
comment by rjh29 · 2023-03-12 · in Show HN: Terminal Based Wikipedia -
…rendered HTML, on the other hand, is designed for humans, and so what you see is what you get. [1]: https://github.com/spencermountain/wtf_wikipedia
comment by cldellow · 2021-08-19 · in Data-Mining Wikipedia for Fun and Profit -
The formats used to store data are an incredible mess. https://github.com/spencermountain/wtf_wikipedia has a take on it.
comment by davidy123 · 2018-09-25 · in Amazon donates $1M to Wikimedia