Hacker GitHub

← All repos

benhoyt/countwords

View on GitHub ↗ ★ 307 Rust archived

Playing with counting word frequencies (and performance) in various languages.

17 mentions on Hacker News by 10 people: 0 stories, 17 comments · first 2021-03-15 · latest 2024-06-01

Mentions per year 17 mentions · busiest year 2021 (10)
20082026
AllStoriesComments NewestTopOldest
  1. …write to the pointer and not re-hash the word and write that to the map, which can be expensive. https://github.com/benhoyt/countwords/blob/master/optimized....
    comment by jamamp · 2024-06-01 · in Go: Sentinel errors and errors.Is() slow your code down by 3000%
  2. Why not read the source code? :-) I wrote comments explaining things: https://github.com/benhoyt/countwords/blob/8553c8f600c40a462...
    comment by burntsushi · 2022-07-25 · in Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
  3. …is an implementation detail. I think you might benefit from reading the "bonus" Rust submission: https://github.com/benhoyt/countwords/blob/8553c8f600c40a462... IMO, Ben kind of glossed over the bonus submission. But I personally think it…
    comment by burntsushi · 2022-07-25 · in Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
  4. https://github.com/benhoyt/countwords/blob/master/simple.lis... From Hoyt's repository itself.
    comment by Jtsummers · 2022-07-24 · in Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
  5. …more complex ones including pointer arithmetic, tricky manual memory management, and so on. [1] https://github.com/benhoyt/countwords/blob/c66dd01d868aa83dc... [2] https://github.com/benhoyt/countwords/blob/c66dd01d868aa83dc...
    comment by benhoyt · 2022-07-24 · in Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
  6. …decided he was tired of maintaining and decided to archive the project (which fair enough!). [0]: https://github.com/benhoyt/countwords/pull/115
    comment by mustache_kimono · 2022-07-24 · in Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
  7. There is a gap between languages at the moment imo. Ruby is just ruthlessly productive. Compare https://github.com/benhoyt/countwords/blob/master/simple.rb with https://github.com/benhoyt/countwords/blob/master/simple.go Hoping new age…
    comment by bestinterest · 2022-04-25 · in Sorbet and 100% cov makes Ruby refactoring possible
  8. …being just as sloppy now as you were in your comments. You're probably using the wrong input. See https://github.com/benhoyt/countwords/blob/aa45dcfd7eff177c8... and https://github.com/benhoyt/countwords/blob/aa45dcfd7eff177c8...
    comment by burntsushi · 2021-03-16 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  9. The source code of the trie program I wrote is here: https://github.com/benhoyt/countwords/blob/master/rust/optim... The comments explain why I think it's slower. The TL;DR is that using a trie requires more memory accesses than a hash…
    comment by burntsushi · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  10. Checkout Crystal lang one (Ruby but fast), https://github.com/benhoyt/countwords/pull/21/files seems to keep readability intact and committer is saying it outperforms Go which is interesting.
    comment by bestinterest · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  11. Yeah, I noticed that too: https://github.com/benhoyt/countwords/blob/9d81d13711e56c250... The constraints do say that it is okay to assume lines are reasonable length. But yes, if you were making GNU wordfreq, that might not be something…
    comment by burntsushi · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  12. …they are included as a baseline for reading a file and for tokenising strings into words. See https://github.com/benhoyt/countwords/blob/eb2a8adf21c895907...
    comment by rtb · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  13. The grep version searches the input for "foobar" [1]. Hence, it doesn't solve the problem. [1] https://github.com/benhoyt/countwords/blob/eb2a8adf21c895907...
    comment by fmos · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  14. That file repeated 10 times, even: https://github.com/benhoyt/countwords/blob/master/test.sh
    comment by duckerude · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  15. On a 4.3MB file: https://github.com/benhoyt/countwords/blob/master/kjvbible.t...
    comment by woliveirajr · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  16. …for a memory access per byte. I did something similar in the 'optimized-trie' program for Rust: https://github.com/benhoyt/countwords/blob/master/rust/optim... --- It ended up being slower. But, it is very much impacted by cache effects.…
    comment by burntsushi · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
  17. Thanks for that - fixed. Repo link is: https://github.com/benhoyt/countwords
    comment by benhoyt · 2021-03-15 · in Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust