June 2017 - Hyperscan

by Barr, Matthew

Hi all, We have just released Hyperscan 4.5.1 on GitHub. This is a bugfix release. After we released 4.5 last week, a few things were discovered that required fixes. >From the change log for this release: [4.5.1] 2017-06-16 - Bugfix for issue #56: workaround for gcc-4.8 C++11 defect. - Bugfix for literal matching table generation, reversing a regression in performance for some literal matching cases. - Bugfixes for hsbench, related to multicore benchmarking, portability fixes for FreeBSD, and clarifying output results. - CMake: removed a duplicate else branch that causes very recent (v3.9) builds of CMake to fail. Regards, Matt.

3 years, 1 month

1
0
0 / 0

Is the feature "Grouping patterns with different pattern-ids and scan submission particular pattern-ids set" available

by Rajendra Prasad Palnaty

Hi All, I have evaluated one of the hardware PME. It allows a set of pattern-ids can be grouped,.can be submitted to single PME instance and allows packet payload submission with that id. Example: HTTP Request Header Patterns /User-Agent/m set=1 subset=1 id=123 /Host/m set=1 subset=1 id=124 HTTP Response Header Patterns (set=2). /Conent-Length/m set=2 subset=1 id=201 /200 OK HTTP/m set=2 subset=1 id=201 We could submit packet data with set and subset onto PME.(i.e HTTP Request Header will be submitted with set=1 subset=1 and HTTP Response header with set=2 and subset=1). Scan happens only specific set and subset that is submitted. Is this kind of feature available with hyperscan? Please respond. Thanks Rajendra

3 years, 1 month

2
1
0 / 0

Do we have grouping of patterns(expressions) and scanning on specific group?

by Rajendra Prasad Palnaty

Hi All, Do we have grouping of patterns? (i.e pushing patterns with setid or group id and data scanning only on specific set) Example: Let's have HTTP(1K patterns), FTP(500 patterns), SMTP(400 patterns) and PDF file patterns(2k patterns) and we want make use of single instance of hyperscan. If we submit HTTP data to hyperscan it should check only HTTP patterns(1K). Thanks Rajendra

3 years, 1 month

1
0
0 / 0

Suricata with Hyperscan

by Alexis Fredes Hadad

Hello! I am trying to add Hyperscan to my Suricata installed on Ubuntu 16.04 LTS on a vmware virtual machine. I followed the tutorial of https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Hyperscan and when I ask for suricata --build-info|grep Hyperscan it says "support no". Any idea? Thanks! Alexis

3 years, 1 month

3
9
0 / 0

Is there any limitation for number of patterns in hyperscan?

by Rajendra Prasad Palnaty

Hi All, I am new bee to hyperscan,. Is there any limit for pushing patterns in hyperscan? Will it be possible to group patterns (e.g HTTP patterns set, FTP Patterns set etc,.) on an single instance of hyperscan? Thanks Prasad

3 years, 1 month

1
0
0 / 0

Hyperscan v4.5

by Barr, Matthew

Hi All, We have just released Hyperscan v4.5.0 on Github. The changelog is quite long for this release. In addition to two new features - approximate matching, and support for AVX-512 instructions - there has been a significant improvement to pattern compilation times in this release, which leads to reduced compile times for many large pattern sets. The changelog for this release is: ## [4.5.0] 2017-06-09 - New API feature: approximate matching using the "edit distance" extended parameter. This allows the user to request all matches that are a given edit distance from an exact match for a pattern. - Initial support for Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512), disabled by default. To enable it, pass `-DBUILD_AVX512=1` to `cmake`. - Major compile time improvements in many subsystems, reducing compile time significantly for many large pattern sets. - Internal reworking of literal matchers to operate on literals of at most eight characters, with subsequent confirmation done in the Rose interpreter. This reduces complexity and bytecode size and improves performance for many pattern sets. - Improve performance of the FDR literal matcher front end. - Improve bucket assignment and other heuristics governing the FDR literal matcher. - Improve optimisation passes that take advantage of extended parameter constraints (`min_offset`, etc). - Introduce further lookaround specialisations to improve scanning performance. - Optimise Rose interpreter construction to reduce the length of programs generated in some situations. - Remove the old "Rose" pattern decomposition analysis pass in favour of the new "Violet" pass introduced in Hyperscan 4.3.0. - In streaming mode, allow exhaustion (where the stream can no longer produce matchers) to be detected in more situations, improving scanning performance. - Improve parsing of control verbs (such as `(*UTF8)`) that can only occur at the beginning of the pattern. Combinations of supported verbs in any order are now permitted. - Update version of PCRE used by testing tools as a syntax and semantic reference to PCRE 8.40. - Tuning support for Intel(R) microarchitecture code names Skylake, Skylake Server, Goldmont. - CMake: when building a native build with a version of GCC that doesn't recognise the host compiler, tune for the microarch selected by `-march=native`. - CMake: don't fail if SQLite (which is only required to build the `hsbench` tool) is not present. - CMake: detect libc++ directly and use that to inform the Boost version requirement. - Bugfix for issue #51: make the fat runtime build wrapper less fragile. - Bugfix for issues #46, #52: use `sqlite3_errmsg()` to allow SQLite 3.6.x to be used. Thanks to @EaseTheWorld for the PR. Regards, Matt.

3 years, 1 month

1
0
0 / 0

What is the maximum limit of expression count allowed in hyperscan

by Rajendra Prasad Palnaty

Hi All, I am trying to evaluate hyperscan tool (hsbench), I tried to push more than 6K (10K,20K etc,.) expressions onto hyperscan using hsbench,. I didnt received any error, but expression count it is showing as 6000 (6k). Is there any limit on expression count. Thanks

3 years, 1 month

2
1
0 / 0

2020

2019

2018

2017

2016

2015

Hyperscan June 2017