Hi Rajendra, What you describe is what we call “setwise matching”. This is not currently a supported feature of Hyperscan. This would be one of our most frequently requested features, however, and we have often thought about it. Do note that many h/w systems that do allow this grouping are doing it as they have a limited number of PME instances available. So they can’t just throw some additional pattern matching databases at the problem. The workarounds to not having setwise matching are really only semi-satisfactory. Suppose as a running example, you have pattern subsets A, B and C. 1. One approach is to simply compile A, B and C separately and submit data to each of them as needed. This approach has a few strengths – if the common case is that you only want a low number of sets, and some of the sets are very small (i.e. 1-10 patterns) then this won’t be a big performance problem. However, there are some nuisances – the separate calls will return their matches in order but if you want to consider an overall ordering of the matches, you will have to merge-sort the matches from A/B/C together. 2. Another approach is to dump A, B and C into a single ‘union’ database and ignore the matches you don’t care about. This can be OK if the overheads are not large, but can be a problem if there are some very expensive sets in the mix. 3. Finally, you can specialize common cases (sort of a hybrid of approaches 1 and 2). If you always know the subset relationships, you can build a ‘union’ database for your most common protocol (HTTP + TCP + IP, say) which will be tuned for the hot cases, and you could use approach 1 to handle less common or important protocols (‘finger’ plus TCP plus IP). We’re not 100% happy with forcing people to use these workarounds. One of our biggest questions is what a setwise API should look like. Should set membership be adjustable dynamically? For example, can we start by matching A+B, then on a call back (or at a stream write boundary), turn off set B and turn on set C? Or should it be a fixed subset that can’t change for the duration of a block or stream? The other open question is what the expectations of performance and overheads are – is setwise matching expected to be at parity of performance with, say, specializing for every possible set? Opinions and design ideas welcome. If we add this feature, it won’t be in the short term and will take considerable design to get right. Regards, Geoff. From: Hyperscan [mailto:hyperscan-bounces@lists.01.org] On Behalf Of Rajendra Prasad Palnaty Sent: Tuesday, June 13, 2017 3:56 AM To: hyperscan(a)lists.01.org Subject: [Hyperscan] Is the feature "Grouping patterns with different pattern-ids and scan submission particular pattern-ids set" available Hi All, I have evaluated one of the hardware PME. It allows a set of pattern-ids can be grouped,.can be submitted to single PME instance and allows packet payload submission with that id. Example: HTTP Request Header Patterns /User-Agent/m set=1 subset=1 id=123 /Host/m set=1 subset=1 id=124 HTTP Response Header Patterns (set=2). /Conent-Length/m set=2 subset=1 id=201 /200 OK HTTP/m set=2 subset=1 id=201 We could submit packet data with set and subset onto PME.(i.e HTTP Request Header will be submitted with set=1 subset=1 and HTTP Response header with set=2 and subset=1). Scan happens only specific set and subset that is submitted. Is this kind of feature available with hyperscan? Please respond. Thanks Rajendra