BurntSushi/memchr on Hacker News

…1-3 bytes (what's supported by memchr) [1]: https://github.com/Dr-Emann/memchr_n#performance [2]: https://github.com/BurntSushi/memchr/pull/241

comment by Dr_Emann · 2026-09-24 · in Fearless SIMD v1.0

It's not quite that dire. The `memchr` crate uses abstraction to limit the code duplication: https://github.com/BurntSushi/memchr/blob/1230fc5c638a4d922f... That is, the `memchr` crate has a `Vector` trait that is generic over the vector…

comment by burntsushi · 2025-11-06 · in The state of SIMD in Rust in 2025

…(1.00x) memmem/subtitles/never/huge-zh-john-watson 47.6 GB/s (1.00x) 34.0 GB/s (1.40x) See also: https://github.com/BurntSushi/memchr/discussions/159 [1]: https://github.com/BurntSushi/rebar [2]: https://github.com/BurntSushi/memchr

comment by burntsushi · 2025-09-23 · in Processing Strings 109x Faster Than Nvidia on H100

…But aarch64 lacks such an instruction, so you need to do other shenanigans for the best codegen: https://github.com/BurntSushi/memchr/blob/ceef3c921b5685847e...

comment by burntsushi · 2025-08-11 · in Faster substring search with SIMD in Zig

…ripgrep will pluck `Sherlock` out of the regex and search that. The actual implementation is here: https://github.com/BurntSushi/memchr?tab=readme-ov-file#algo... The main difference with the algorithm presented here is that instead of…

comment by burntsushi · 2025-06-14 · in SIMD-friendly algorithms for substring searching (2016)

…has a specialized routine for counting occurrences of bytes (which ripgrep uses for line counting): https://github.com/BurntSushi/memchr/blob/746182171d2e886006... Because it's faster to do that than it is to reuse the generalized `memchr`…

comment by burntsushi · 2025-03-11 · in Show HN: Krep a High-Performance String Search Utility Written in C

…of the approach here, ripgrep is already using a pretty sophisticated substring search algorithm: https://github.com/BurntSushi/memchr?tab=readme-ov-file#algo... And it uses memory maps (sometimes, when it thinks it will be fast, but it…

comment by burntsushi · 2025-03-11 · in Show HN: Krep a High-Performance String Search Utility Written in C

…up to ripgrep's, you can plug your implementation into the `memchr` crate's benchmark harness: https://github.com/BurntSushi/memchr It uses rebar: https://github.com/BurntSushi/rebar

comment by burntsushi · 2024-05-21 · in Regular expression matching can be simple and fast (2007)

…crate does. It uses `vshrn` just like in your links. And vpmaxq before even bothering with vshrn: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf...

comment by burntsushi · 2023-12-26 · in Ask HN: What's the fastest programming language with a large standard library?

…for why std doesn't make as much use of SIMD as it probably should (the core/alloc/std split). [1]: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf... [2]: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf... [3]:…

comment by burntsushi · 2023-12-26 · in Ask HN: What's the fastest programming language with a large standard library?

…[4]: https://github.com/dotnet/runtime/pull/88394#issuecomment-16... [5]: https://github.com/BurntSushi/memchr#why-is-the-standard-lib... [6]: https://github.com/rust-lang/rfcs/pull/3469 [7]: https://github.com/BurntSushi/rebar#summ

comment by burntsushi · 2023-12-26 · in Ask HN: What's the fastest programming language with a large standard library?

…And this isn't a trick either. That really is the scaling behavior of the implementation. See: https://github.com/BurntSushi/memchr/blob/ce7b8e606410f6c81a... > If an algorithm is defined as being unscalable (fixed input), what sense does…

comment by burntsushi · 2023-12-04 · in Rust-Cache

…crate utilizes information from the vector step to limit where and when it calls `memcmp`]( https://github.com/BurntSushi/memchr/blob/46620054ff25b16d22... ). Your code might do well in cases where matches are very rare. I took a quick…

comment by burntsushi · 2023-08-29 · in Stringzilla: Fastest string sort, search, split, and shuffle using SIMD

…as movemask. I tested the PR and it has a significant speedup over the non-vectorized version. [1] https://github.com/BurntSushi/memchr/pull/114

comment by celeritascelery · 2023-07-06 · in Regex engine internals as a library

…operations such as movemask. This is discussed a bit more here (including the bit about movemask): https://github.com/BurntSushi/memchr/issues/76

comment by burntsushi · 2023-07-05 · in Regex engine internals as a library

…it was never the thing most commonly used). What ripgrep uses today is succinctly described here: https://github.com/BurntSushi/memchr#algorithms-used (But it has always eschewed algorithms like Boyer-Moore in favor of more heuristic-y…

comment by burntsushi · 2023-05-18 · in Sneller Regex vs Ripgrep

…and then only call them when the requisite CPU features are enabled is enough. In a nutshell: https://github.com/BurntSushi/memchr/blob/8037d11b4357b0f07b...

comment by burntsushi · 2023-05-02 · in SIMD with Zig

…algorithms depending on needle and haystack lengths and possibly also contents of the needle. E.g. https://github.com/BurntSushi/memchr#user-content-algorithms...

comment by the8472 · 2022-12-16 · in Checking for the absence of a string, naive AVX-512 edition

…analogous to what Hyperscan does). You really want blocks of bytes, not one-byte-at-a-time. [1]: https://github.com/BurntSushi/memchr/blob/8037d11b4357b0f07b...

comment by burntsushi · 2022-09-08 · in A byte string library for Rust

…* Define a large suite of benchmarks, and ensure that I test the non-vector implementation: https://github.com/BurntSushi/memchr/tree/master/bench * Have a thorough testing strategy, including hand written tests, exhaustive tests, property…

comment by burntsushi · 2021-05-16 · in Verifying Vectorized Rust Revisited

…too. The main loop is indeed quite large, but before that, it does unaligned loads of 32 bytes: https://github.com/BurntSushi/rust-memchr/blob/427fdc384007d...

comment by burntsushi · 2021-04-23 · in The Boyer-Moore Fast String Searching Algorithm

…touch all bytes at that point? Leaning heavily on simd instructions? Yes. That's what memchr does: https://github.com/BurntSushi/rust-memchr/blob/427fdc384007d... > And I realize newlines is only to track line counts. Am I wrong that that…

comment by burntsushi · 2021-04-23 · in The Boyer-Moore Fast String Searching Algorithm

…My sibling comment linked to my blog post on ripgrep, which uses Rust's regex engine. ;-) [1] - https://github.com/BurntSushi/rust-memchr/blob/master/src/li...

comment by burntsushi · 2017-01-21 · in How quickly can you remove spaces from a string?