joaoventura/WikiCorpusExtractor
View on GitHub ↗ ★ 24 Python
Extracts text from WikiMedia XML Dump files
6 mentions on Hacker News by 1 person: 0 stories, 6 comments · first 2017-01-21 · latest 2017-11-15
20082026
-
…and I used to build my corpora from wikipedia documents. Here's a tool that I've built to do it: https://github.com/joaoventura/WikiCorpusExtractor
comment by jventura · 2017-11-15 · in A Guide to Natural Language Processing -
…a bit about Wikipedia dumps datasets, here's a tool that I've created to build textual corpora: https://github.com/joaoventura/WikiCorpusExtractor [0] - http://www.sciencedirect.com/science/article/pii/S1877050912... [1] -…
comment by jventura · 2017-09-12 · in What every software engineer should know about search -
…from wikipedia's XML dump files. Here's the link to it, may someone need to do something similar: https://github.com/joaoventura/WikiCorpusExtractor
comment by jventura · 2017-03-27 · in How to Extract Data from Wikipedia and Wikidata -
…that is the reason I recommend statistical approaches, as they tend to be language independent. [0] https://github.com/joaoventura/WikiCorpusExtractor/blob/mast...
comment by jventura · 2017-02-20 · in Ask HN: Needs advice on learning NLP -
…in the wild, but if you need to roll your own from wikipedia texts you can use this tool I did: https://github.com/joaoventura/WikiCorpusExtractor From this experiment, and depending if you like statistics or not, you can play a bit with…
comment by jventura · 2017-02-20 · in Ask HN: Needs advice on learning NLP -
…idea on how statistics is used on text-mining. [0] https://en.wikipedia.org/wiki/Tf%E2%80%93idf [1] https://github.com/joaoventura/WikiCorpusExtractor
comment by jventura · 2017-01-21 · in How to Get into Natural Language Processing