intel/hyperscan on Hacker News

…at a fast regex library [1] and take a subset of those tests or benchmarks and apply to match. [1] https://github.com/intel/hyperscan/tree/master That could be a fun comparison. As someone mentioned, regular expressions are extremely fast,…

comment by sgt · 2026-02-26 · in Regex is dead. We replaced it

…for small sets, are substantially similar to the algorithms used in Hyperscan (truffle and shufti). https://github.com/intel/hyperscan Having something hard coded for spaces can be much faster, especially since 5 of the 6 characters are a…

comment by ncruces · 2025-08-17 · in Counting Words at SIMD Speed

Maybe even better examples would be < https://github.com/intel/hyperscan >, < https://github.com/simdutf/simdutf >, and my own < https://github.com/ashvardanian/StringZilla > :)

comment by ashvardanian · 2025-07-05 · in The messy reality of SIMD (vector) functions

…solution. My first recommendation will be to look at HyperScan. It has remarkable throughput: https://github.com/intel/hyperscan I’ve only avoided it in the tutorial, as I want to keep the build system lean. I wouldn’t be surprised if it’s…

comment by ashvardanian · 2025-04-18 · in Less Slow C++

…but of course given who it's from it may be tied to a single brand of CPU. github seems to be https://github.com/intel/hyperscan

comment by jepler · 2024-03-07 · in Ask HN: Regex on a File or Stream

…[2] https://github.com/BurntSushi/aho-corasick/discussions/136 [3] https://github.com/intel/hyperscan [4] https://github.com/OISF/suricata [5] https://github.com/VectorCamp/vectorscan

comment by dalf · 2024-03-04 · in Aho-Corasick Algorithm

…not all WAFs have be structured like ModSecurity. They could for example be based on something like https://github.com/intel/hyperscan and perf is at a very different level.

comment by allanrbo · 2023-11-13 · in Discouraging the use of web application firewalls

…in minor point releases. See https://abi-laboratory.pro/index.php?view=changelog&l=glibc&... and https://github.com/intel/hyperscan/issues/359 .

comment by burntsushi · 2023-08-27 · in Be careful of the examples you use. They stick

…`git push` request/response cycle. Under the hood it’s using Intel’s hyperscan as the regex engine. https://github.com/intel/hyperscan

comment by greysteil · 2023-05-11 · in GitHub push protection is free for all public repositories

…[2]: https://github.com/onekey-sec/unblob/blob/main/unblob/proces... [3]: https://github.com/intel/hyperscan

comment by kissgyorgy · 2023-01-19 · in Show HN: Unblob – extraction suite for 30+ file formats

…using make -j instead of cmake -build to speed it up. Decent number of compiler warnings too. [1] https://github.com/intel/hyperscan

comment by jandrese · 2022-09-12 · in Intel Hyperscan is a high-performance multiple regex matching library

Intel has an implementation of this technique here as well: https://github.com/intel/hyperscan

comment by wooosh · 2022-09-05 · in High speed Unicode routines using SIMD

It was an Intel project. https://github.com/intel/hyperscan/

comment by unreal37 · 2021-06-07 · in Sorry everybody, I failed with you

I built (or at least "designed most of, built parts of") Hyperscan: https://github.com/intel/hyperscan - a fast multiple regular expression matcher. It was not originally open sourced; it was a commercial product for some time. I wasn't…

comment by glangdale · 2021-05-16 · in Ask HN: What are some tools / libraries you built yourself?

…backtrackers (like modern JIT'd libpcre) may outrace them due to having optimizations. Hyperscan ( https://github.com/intel/hyperscan ) a project I worked on for many years avoids the potential DoS attacks on backtrackers or the ones RE2…

comment by glangdale · 2021-03-22 · in Regexploit: DoS-Able Regular Expressions

…certainty of your position are why you are a troll. [1] - https://sr.ht/~pierrenn/ripgrep/ [2] - https://github.com/intel/hyperscan

comment by burntsushi · 2021-02-09 · in Rust Foundation: Hello, World

…And yet... it concocted a good chunk of a PSHUFB based pattern matcher (effectively "shufti" from https://github.com/intel/hyperscan ) that I invented in 2009. If I could push further on this and scale from 4 instructions handled in about…

comment by glangdale · 2020-11-24 · in Superoptimizer – A Look at the Smallest Program [pdf] (1987)

…) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan ( https://github.com/intel/hyperscan ) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that…

comment by glangdale · 2020-08-06 · in What's wrong with Intel, and how to fix it: Former principal engineer unloads

…only on the side that makes their stuff look good. Back when I worked on the Hyperscan project ( https://github.com/intel/hyperscan ), we had an API call that allocated scratch memory (necessary due to Hyperscan's rather constrained…

comment by glangdale · 2020-07-06 · in A Journey building a fast JSON parser and full JSONPath, Oj for Go

…often" over lots of input is performance critical. This was a key driver behind writing Hyperscan ( https://github.com/intel/hyperscan ) which I used to work on when it was primarily used for network security (thousands or tens of…

comment by glangdale · 2020-06-16 · in How the most popular Chrome extensions affect browser performance

…variations. As somehow who designed an implementation of \b for an NFA-based regex matcher ( https://github.com/intel/hyperscan ) I always have an appreciation for the Scunthorpe problem; we did see quite a number of scam/spam/bad-word…

comment by glangdale · 2020-05-10 · in Scunthorpe Sans – a self-censoring font

…( http://www.naturalsemi.com/ ). I tracked these guys because we were doing automata in s/w (the https://github.com/intel/hyperscan project at Intel). There were some aspects of their methodology I didn't like (notably, they tended to run…

comment by glangdale · 2020-03-29 · in ComputeDRAM: In-Memory Compute Using Off-the-Shelf DRAMs (2019) [pdf]

…reading such a big file are likely to dominate in practice. We implemented some SIMD approaches in https://github.com/intel/hyperscan that would probably work pretty well (effectively a 2-char search followed by a quick confirm) for this…

comment by glangdale · 2020-03-01 · in A journey to searching Have I Been Pwned database in 49μs

iirc, Github uses (used?) my old project ( https://github.com/intel/hyperscan ) at Intel. It's probably faster than the alternatives, although if you want to support all types of regex you'll need to use Hyperscan as a prefilter for a…

comment by glangdale · 2020-02-24 · in Show HN: Search code in GitHub repos using regular expressions

…backtracking world. We never got around to implementing arbitrary lookarounds in Hyperscan ( https://github.com/intel/hyperscan ) but if we had done something in the automata world to handle lookaround, lookbehinds are way easier than…

comment by glangdale · 2020-01-31 · in Show HN: Regex Cheatsheet

…The computations data∊nl data∊sp will use vectorised search methods similar to Intel's hyperscan ( https://github.com/intel/hyperscan ). The character lookups here are changed to be lookups from a 256-bit table, or two SSE registers, and…

comment by mlochbaum · 2019-10-24 · in Beating C with Dyalog APL

…wrote a generalization of the Levenshtein distance algorithm for regular expressions in Hyperscan ( https://github.com/intel/hyperscan ), which allows you to do Levenstein distance matching over regular expressions. It is not necessarily a…

comment by glangdale · 2019-08-23 · in Levenshtein Distance

But if you want faster, (but more complicated): https://github.com/intel/hyperscan

comment by Quarrel · 2019-06-29 · in Regular Expression Matching Can Be Simple and Fast (2007)

…- 2006! - we were well aware that it wasn't really a new thing. 12 years of work on Hyperscan ( https://github.com/intel/hyperscan ), 1 acquisition and 1 open source release later - as well as multiple other projects doing similar things…

comment by glangdale · 2019-02-08 · in Show HN: Nevod is easier and faster than RegExp

…in 2006, sold a company to Intel based around in in 2013 and open-sourced the resulting library ( https://github.com/intel/hyperscan ) in 2015. All multi-pattern systems get a huge improvement over one-at-a-time regex. It would be…

comment by glangdale · 2019-02-08 · in Show HN: Nevod is easier and faster than RegExp

https://github.com/intel/hyperscan/ implements this along multiple regex matching and a pcre pattern compatible engine (chimera).

comment by mgliwka · 2018-09-04 · in Levenshtein Automata (2010)

…Got a decent if not ginormous exit too, which was nice. Also got to IMO change the industry - Hyperscan (github.com/intel/hyperscan) is used in a lot of places. This nice run has come to an end for me and my team and it's made me realize…

comment by glangdale · 2018-06-05 · in Ask HN: Are you working on interesting technical problems?

▲ 4

High-performance regular expression matching library

story by handpickednames · 2018-04-28 · 0 comments

…string searching mechanism can outrun memory bandwidth. Or you could go use my old project ( https://github.com/intel/hyperscan ) and then you could look for regexes too. :-)

comment by glangdale · 2018-04-28 · in How long does it take to linear search 200MB in memory?

…advantage of this whether they are given a pure literal case or where regexes have literal factors: https://github.com/intel/hyperscan being a project I've worked on for a little while.

comment by glangdale · 2017-12-12 · in Regex was taking 5 days to run. So I built a tool that did it in 15 minutes

…much the first decade of my 'proper' career working on ideas that went into our regex matcher (now at github.com/intel/hyperscan fwiw). During some of the more intense periods I could barely take a shower or daydream without having to grab…

comment by glangdale · 2017-10-25 · in I have no side code projects to show you

The same Hyperscan as here? https://github.com/01org/hyperscan Project homepage suggests this is an Intel project.

comment by bsg75 · 2017-08-10 · in A rising sentiment that IBM’s Watson can’t deliver on its promises

…some code here that does this, but it's not the easiest to understand (this is the run-time only): https://github.com/01org/hyperscan/blob/master/src/nfa/shuft... The idea is that you represent a character class by intersecting two PSHUFB…

comment by glangdale · 2017-07-04 · in Pruning spaces from strings quickly on ARM processors

Also see the definition of 'block' in https://github.com/01org/hyperscan/blob/master/src/nfa/shuft...

comment by glangdale · 2017-05-26 · in Writing a Fast JSON Parser

…ub.com/rust-lang/regex/blob/master/src/simd_acce... if interested or look up the Hyperscan project at https://github.com/01org/hyperscan for a few different versions of this trick.

comment by glangdale · 2017-05-26 · in Writing a Fast JSON Parser

Full disclosure - I work on the Hyperscan project at Intel github.com/01org/hyperscan Regular expression implementation is fun and many interesting things have been done in this area. We are partial to the work of Gonzalo Navarro (in terms…

comment by glangdale · 2017-03-21 · in The 1960's elegance behind Go's regexp

…years, did the 'startup' in this area, now acquired by Intel, open sourced the product (ob plug: https://github.com/01org/hyperscan ) and I would never claim that we are anything near 'solving' regex. More like "mitigating" or…

comment by glangdale · 2016-12-11 · in Ask HN: What problem in your industry is a potential startup?

Hi, author of Hyperscan ( https://github.com/01org/hyperscan ) here. I strongly suspect we don't support enough of this: > many of which used zero-width assertions that required non-trivial transformations and pre- and post-processing of…

comment by glangdale · 2016-11-29 · in SeaHash: A fast, portable hash function in Rust

…in that particular use case (but uses finite automata, so you may miss some things from PCRE): https://github.com/01org/hyperscan )

comment by burntsushi · 2016-11-29 · in SeaHash: A fast, portable hash function in Rust

This is really, really good. We work on regex ourselves (github.com/01org/hyperscan) and I will definitely point people towards this when they are trying to figure regex out. The only regret I have is that many of the interesting regexes…

comment by glangdale · 2016-11-15 · in Regex 101

Speaking as the leader of the Hyperscan project ( https://github.com/01org/hyperscan ), I'd say you might not be the only project feeling a bit neglected in the performance comparison here. It's nice to be mentioned - and even called out…

comment by glangdale · 2016-09-28 · in Ripgrep – A new command line search tool

…techniques from our library get used in other projects as one of the points of open sourcing it ( https://github.com/01org/hyperscan ) was to share how we do things with the community.

comment by glangdale · 2016-09-26 · in Ripgrep – A new command line search tool

…team) have spent a lot of time staring at regular expressions over the years (shameless plug: https://github.com/01org/hyperscan ). I think a better format for regex is long overdue, but this isn't it. It's way too verbose (other…

comment by glangdale · 2016-08-30 · in SRL – Simple Regex Language

…top. It's quite fast! [1] - https://github.com/rust-lang-nursery/regex/blob/master/src/s... [2] - https://github.com/01org/hyperscan

comment by burntsushi · 2016-06-11 · in Why Rust for Low-Level Linux Programming?