soskek/bookcorpus
View on GitHub ↗ ★ 865 Python
Crawl BookCorpus
4 mentions on Hacker News by 4 people: 1 story, 3 comments · first 2020-10-25 · latest 2024-05-28
20082026
-
…dataset is just "all of bibliotik": https://www.twitter.com/theshawwn/status/1320282149329784833 https://github.com/soskek/bookcorpus/issues/27 And we also apparently know for a fact that this was included in the datasets used to train…
comment by Intralexical · 2024-05-28 · in The Internet Archive is under a DDoS attack -
…and looks like there have been successful efforts to create a similar dataset called bookcorpus: https://github.com/soskek/bookcorpus/issues/27 ). The discussion on that github issue suggests bookcorpus is very similar to "books2", which…
comment by nickvincent · 2021-03-01 · in On the Dangers of Stochastic Parrots [pdf] -
…of running it. Here are some notes on what I did with this script, in case you find it helpful: https://github.com/soskek/bookcorpus/issues/27 Happy reading! Please let me know any other questions you might have. By the way, dmca.mp4 is a…
comment by sillysaurusx · 2020-10-25 · in Books in .txt format for AI training purposes