Hacker GitHub

← All repos

dbro/csvquote

View on GitHub ↗ ★ 447 C

Enables common unix utilities like cut, awk, wc, head to work correctly with csv data containing delimiters and newlines

25 mentions on Hacker News by 9 people: 2 stories, 23 comments · first 2013-09-19 · latest 2025-03-26

Mentions per year 25 mentions · busiest year 2014 (6)
20082026
AllStoriesComments NewestTopOldest
  1. Forgive me for promoting this that I wrote: csvquote: https://github.com/dbro/csvquote Especially for use with existing shell text processing tools, eg. cut, sort, wc, etc.
    comment by dbro · 2025-03-26 · in A love letter to the CSV format
  2. ▲ 3
    Csvquote – smart and simple CSV processing on the command line
    story by kamaraju · 2024-09-27 · 0 comments
  3. ▲ 9
    csvquote – smart and simple CSV processing on the command line
    story by pabs3 · 2023-07-01 · 0 comments
  4. …does. It does nothing else, just this so that programs like awk, sed, cut, etc. can work properly. https://github.com/dbro/csvquote
    comment by dbro · 2023-06-29 · in The Awk Programming Language, Second Edition
  5. …command line data tools. If you like using awk, sed, cut, tr, etc. then it may be useful to you. < https://github.com/dbro/csvquote > Using it with the first example command from this article would be csvquote file.csv | awk -F, '{print…
    comment by dbro · 2023-06-28 · in Using command line to process CSV files (2022)
  6. …contains delimiters and/or newlines in quoted fields, you might want to pipe it through csvquote. https://github.com/dbro/csvquote
    comment by SoftTalker · 2022-09-21 · in Q – Run SQL Directly on CSV or TSV Files
  7. …that doesn't support it. For example: csvquote test.csv | awk '{print $1, $2}' | csvquote -u [1] https://github.com/dbro/csvquote
    comment by jeroenjanssens · 2022-05-12 · in Modernizing AWK, a 45-year old language, by adding CSV support
  8. …| csvquote -u > output.csv Also works for other text processing tools like cut, sed, sort, etc. [1] https://github.com/dbro/csvquote
    comment by dbro · 2021-10-01 · in Understanding Awk
  9. …worked for me, I'm not sure but I think it was this: http://lorance.freeshell.org/csv/ There's also https://github.com/dbro/csvquote which is more unix-like in philosophy: it sits in a pipeline, and only handles transforming the CVS data…
    comment by throwawayboise · 2021-09-07 · in Awk: The Power and Promise of a 40-Year-Old Language
  10. Good idea! Looks similar to something I wrote called csvquote https://github.com/dbro/csvquote , which enables awk and other command line text tools to work with CSV data that contains embedded commas and newlines.
    comment by dbro · 2020-10-31 · in Awk: `Begin { ` Part 1
  11. …this only works if your csv files don't contain new lines. However if they do, I recommend using https://github.com/dbro/csvquote to circumvent the issue. Yesterday I used awk as a QA tool. I had to subtract a sum of values in the last…
    comment by afiodorov · 2020-05-20 · in The State of the AWK
  12. To simplify working with CSV data using command line tools, I wrote csvquote ( https://github.com/dbro/csvquote ). There are some examples on that page that show how it works with awk, cut, sed, etc.
    comment by dbro · 2020-01-21 · in Why Learn Awk? (2016)
  13. While not exactly what you asked for, I wrote something similar called csvquote ( https://github.com/dbro/csvquote ) which transforms "typical" CSV or TSV data to use the ASCII characters for field separators and record separators, and…
    comment by dbro · 2019-05-20 · in Show HN: UXY – adding structure to Unix tools
  14. You might want to check out https://github.com/dbro/csvquote which helps awl and other text tools handle csv files which have quoted strings as values.
    comment by dbro · 2018-08-18 · in GoAWK: an AWK interpreter written in Go
  15. Regarding quoting support, check out https://github.com/dbro/csvquote
    comment by ggrothendieck · 2015-08-16 · in Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
  16. …use an adapter to allow existing tools to work around CSV's quirky quoting methods. csvquote ( https://github.com/dbro/csvquote ) enables the regular UNIX command line text toolset (like cut, wc, awk, etc.) to work properly with CSV data.
    comment by dbro · 2015-02-22 · in XSV – A fast CSV toolkit in Rust
  17. yes! https://github.com/dbro/csvquote csvquote allows UNIX tools to work properly with quoted fields that contain delimiters inside the data. It is a simple translation tool that temporarily replaces the special characters occurring inside…
    comment by dbro · 2014-08-27 · in Useful Unix commands for exploring data
  18. …version of tools like grep, we can make the data simple for these tools to understand. That's what https://github.com/dbro/csvquote does. It can be run in a pipeline before the grep stage, and allow grep/cut/awk/... to work with…
    comment by dbro · 2014-05-28 · in CSVKit: CSV utilities that includes csvsql, csvgrep, csvstat, and more
  19. …for exchanging data between different organizations that are producing and consuming the data. https://github.com/dbro/csvquote is a small and fast script that can replace ambiguous separators (commas and newlines, for example) inside…
    comment by dbro · 2014-05-26 · in So You Want to Write Your Own CSV code
  20. that's exactly what https://github.com/dbro/csvquote does for commas and newlines both.
    comment by dbro · 2014-03-27 · in ASCII Delimited Text – Not CSV or TAB delimited text
  21. …and am happy to hear that other people find it useful. It is indeed a simple state machine (see https://github.com/dbro/csvquote/blob/master/csvquote.c ), and it translates CSV/TSV files into files which follow the spirit of what's…
    comment by dbro · 2014-03-27 · in ASCII Delimited Text – Not CSV or TAB delimited text
  22. Related: This tool: https://github.com/dbro/csvquote will convert all the record/field separators (such as tabs/newlines for TSV) into non-printing characters and then in the end reverse it. Example: csvquote foobar.csv | cut -d ',' -f 5 |…
    comment by susi22 · 2014-03-26 · in ASCII Delimited Text – Not CSV or TAB delimited text
  23. This is a great list, but IMO lacks the most powerful (but unfortunately unpopular) one: https://github.com/dbro/csvquote Apply it first, then do the normal processing with GNU coreutils and you'll cover most use cases.
    comment by susi22 · 2013-11-13 · in CSVfix is a tool for manipulating CSV data
  24. This also really helpful (pity it's relatively unknown): https://github.com/dbro/csvquote And: http://en.wikipedia.org/wiki/GNU_Core_Utilities (section "Text utilities") http://directory.fsf.org/wiki/Textutils Run "$ info coreutils"
    comment by susi22 · 2013-09-19 · in Command-line tools for data science
  25. …the linux command line spirit "Do one thing, and do it well" (it does two) - I much prefer this: https://github.com/dbro/csvquote
    comment by susi22 · 2013-09-19 · in Cutcsv: Unix cut for CSV files