dw/csvmonkey
View on GitHub ↗ ★ 102 C++
Header-only vectorized, lazy-decoding, zero-copy CSV file parser
6 mentions on Hacker News by 3 people: 0 stories, 6 comments · first 2017-06-12 · latest 2020-10-10
20082026
-
…not possible to waste a whole machine on it. I never finished packaging and writing tests, but https://github.com/dw/csvmonkey hits 1.9gb/sec on certain data sets with one thread, and that implementation is not even representative of the…
comment by dmw_ng · 2020-10-10 · in Loading CSV File at the Speed Limit of the NVMe Storage -
I can't speak for this project, but my own for CSV files ( https://github.com/dw/csvmonkey ) provides a high level interface that allows the tokenized data to be manipulated in-place without full decoding. The interface exported in Python…
comment by _wmd · 2019-02-21 · in Simdjson – Parsing Gigabytes of JSON per Second -
…the transfer. The format is so simple that it can approach almost free to parse (see e.g. my own https://github.com/dw/csvmonkey ). CSV is also distinguished in that regular home users with spreadsheet programs can usually do most things a…
comment by _wmd · 2018-12-25 · in SQLite as an Application File Format (2014) -
Check out csvmonkey for a really fast C++ csv parser: https://github.com/dw/csvmonkey
comment by burntsushi · 2018-09-09 · in XSV: A fast CSV command-line toolkit written in Rust -
…no clue how to disable it, but it's default behaviour. Parsing isn't actually a tough problem – https://github.com/dw/csvmonkey is a project of mine, it manages almost 2GB/sec throughput _per thread_ on a decade old Xeon
comment by _wmd · 2018-05-24 · in Command-line Tools can be 235x Faster than a Hadoop Cluster (2014) -
…a claim" or inventing a heuristic, you can test this trivially: $ git clone https://....:x-oauth-basic@github.com/dw/csvmonkey.git Cloning into 'csvmonkey'... remote: Counting objects: 340, done. remote: Compressing objects: 100% (27/27),…
comment by _wmd · 2017-06-12 · in Don't publicly expose .git (2015)