The question is about how to use hyperscan database.
I have compile some original databases, and next I want reconstruct them on different
memory locations (say, NUMA nodes). To save time of compiling patterns, I just follow
steps as below:
compile original databases A, B, and get their size NA, NB (by hs_database_size)
allocate enough memory (>=(NA+NB)) at new memory locations; the start addresses are
memcpy A to A’, and B to B’
allocate scratch for A’ and B’
These steps go well, but I have some problems when calling hs_scan:
when link to debug-compiled hyperscan library, an assertion fails (a value is not 64 bits
I compile 4 databases and 3 of them work well, but one raises segment fault
After long time debugging, I think maybe I shouldn’t memcpy database simply like above.
Do I have to call hs_serialize_XXX to serialize original databases and then deserialize
them on new memory locations?
Thanks for your reply,
Show replies by thread
Hyperscan databases cannot be copied with memcpy() or similar. There are some structures
in the database that must be aligned correctly - like values that will be used directly by
SIMD instructions - and when the database is created in hs_compile_XXX or
hs_deserialize_database() adjustments are performed for alignment.
Unfortunately this is a little slower than a straight memcpy(), but the alignment is
So in your example, you should serialize the first compiled DB and then either use
hs_deserialize_database() to create new copies, or use hs_deserialize_database_at() if you
need to use a specific memory location. These are documented here:
New copies of the scratch space can be created using hs_clone_scratch().