Hyperscan 5.2.0 performance with Snort 3.0
by mmutahir＠whizzsystems.com
Hi
I am experimenting in improving the performance of Snort by integrating the Hyperscan library into snort. However I am unable to see any significant performance improvements by using Hyperscan. The article at https://software.intel.com/en-us/articles/hyperscan-and-snort-integration shows that the performance improvement is approximately six times in terms of Packets/second processed. But the snort version is 2.9.8.2 and Hyperscan version is 4.3.1.
I followed the method on installing snort on my debian system using the following article http://sublimerobots.com/2018/06/installing-snort-3-b245-in-ubuntu/, the only difference being in the snort and hyperscan versions being used
The following is the output of Snort -V, which shows that hyperscan is integrated.
# snort -V
,,_ -*> Snort++ <*-
o" )~ Version 3.0.0 (Build 247) from 2.9.11
'''' By Martin Roesch & The Snort Team
http://snort.org/contact#team
Copyright (C) 2014-2018 Cisco and/or its affiliates. All rights reserved.
Copyright (C) 1998-2013 Sourcefire, Inc., et al.
Using DAQ version 2.2.2
Using LuaJIT version 2.1.0-beta3
Using OpenSSL 1.1.1d 10 Sep 2019
Using libpcap version 1.8.1
Using PCRE version 8.39 2016-06-14
Using ZLIB version 1.2.11
Using Hyperscan version 5.2.0 2019-11-21
Using LZMA version 5.2.4
# uname -a
Linux rhino 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2 (2019-08-28) x86_64 GNU/Linux
To enable hyperscan I edited the snort.lua in the default community rules provided by snort website and added the following lines just before the ips entry
search_engine = { search_method = 'hyperscan',
split_any_any = true }
The following is my command that I give for starting snort. I ran it for 30 seconds and on enp4s0 I was replaying the bigflows.pcap file using colasoft packet player in Windows 7.
snort -Q -i enp4s0:enp5s0 -A none --daq afpacket -z 4 -c /opt/snort/etc/snort/snort.lua -R /opt/snort//etc/snort/rules/snort3-community.rules
But there was no change in the performance numbers after running snort.
Can any one guide me if
1. Hyperscan is being even used by snort or not, using the methodology above.
2. How can we get the performance numbers using Snort 3.0 and Hyperscan 5.2.0
The log of the snort command
--------------------------------------------------
o")~ Snort++ 3.0.0-247
--------------------------------------------------
Loading /opt/snort/etc/snort/snort.lua:
ssh
pop
binder
stream_tcp
gtp_inspect
dce_http_proxy
stream_icmp
normalizer
ftp_server
stream_udp
search_engine
dce_smb
ips
modbus
rpc_decode
latency
wizard
appid
file_id
ftp_data
smtp
back_orifice
port_scan
dce_http_server
dce_tcp
telnet
ssl
sip
classifications
http2_inspect
http_inspect
stream_user
stream_ip
dnp3
ftp_client
stream
references
arp_spoof
dns
dce_udp
imap
stream_file
Finished /opt/snort/etc/snort/snort.lua.
Loading rules:
Loading /opt/snort//etc/snort/rules/snort3-community.rules:
Finished /opt/snort//etc/snort/rules/snort3-community.rules.
Finished rules.
--------------------------------------------------
rule counts
total rules loaded: 829
text rules: 829
option chains: 829
chain headers: 46
--------------------------------------------------
port rule counts
tcp udp icmp ip
any 63 3 0 0
src 124 3 0 0
dst 539 98 0 0
both 0 1 0 0
total 726 105 0 0
--------------------------------------------------
flowbits
defined: 20
not checked: 11
not set: 3
--------------------------------------------------
service rule counts - tcp to-srv to-cli
dns: 1 0
ftp: 7 2
ftp-data: 0 8
http: 485 92
imap: 0 8
irc: 4 1
netbios-ssn: 15 1
pop3: 0 8
smtp: 16 0
ssl: 14 31
telnet: 1 0
total: 543 151
--------------------------------------------------
service rule counts - udp to-srv to-cli
dns: 88 2
http: 4 0
total: 92 2
--------------------------------------------------
fast pattern port groups src dst any
packet: 12 24 2
--------------------------------------------------
fast pattern service groups to-srv to-cli
packet: 10 6
key: 1 0
header: 1 4
body: 1 0
file: 2 4
--------------------------------------------------
search engine
instances: 64
patterns: 1135
--------------------------------------------------
afpacket DAQ configured to inline.
Commencing packet processing
++ [0] enp4s0:enp5s0
++ [1] enp4s0:enp5s0
++ [2] enp4s0:enp5s0
++ [3] enp4s0:enp5s0
^C** caught int signal
== stopping
-- [3] enp4s0:enp5s0
-- [1] enp4s0:enp5s0
-- [0] enp4s0:enp5s0
-- [2] enp4s0:enp5s0
--------------------------------------------------
Packet Statistics
--------------------------------------------------
daq
received: 639395
analyzed: 470637
dropped: 1161079
outstanding: 168758
allow: 443828
block: 3572
replace: 11295
whitelist: 11942
idle: 4
rx_bytes: 198625867
--------------------------------------------------
codec
total: 470637 (100.000%)
discards: 1165 ( 0.248%)
eth: 470637 (100.000%)
icmp4: 13367 ( 2.840%)
icmp4_ip: 1124 ( 0.239%)
icmp6: 24 ( 0.005%)
igmp: 178 ( 0.038%)
ipv4: 470227 ( 99.913%)
ipv6: 410 ( 0.087%)
ipv6_hop_opts: 24 ( 0.005%)
tcp: 359828 ( 76.456%)
udp: 96097 ( 20.418%)
--------------------------------------------------
Module Statistics
--------------------------------------------------
detection
analyzed: 470637
raw_searches: 420589
cooked_searches: 12652
pkt_searches: 433241
key_searches: 1426
header_searches: 2666
body_searches: 6
file_searches: 2184
--------------------------------------------------
search_engine
max_queued: 16
total_inserts: 12200
total_unique: 12200
non_qualified_events: 12200
searched_bytes: 367147806
--------------------------------------------------
latency
total_packets: 483530
total_usecs: 19857972
max_usecs: 12853
packet_timeouts: 20
total_rule_evals: 12200
--------------------------------------------------
host_tracker
service_adds: 1577
--------------------------------------------------
host_cache
lru_cache_adds: 151
lru_cache_find_hits: 1426
lru_cache_find_misses: 151
--------------------------------------------------
tcp
bad_tcp4_checksum: 22
--------------------------------------------------
appid
packets: 469472
processed_packets: 469341
ignored_packets: 131
total_sessions: 13089
appid_unknown: 4757
--------------------------------------------------
back_orifice
packets: 89779
--------------------------------------------------
binder
packets: 12925
inspects: 12925
--------------------------------------------------
dce_tcp
pdus: 15
binds: 3
bind_acks: 3
requests: 3
responses: 3
auth3s: 3
tcp_sessions: 3
tcp_packets: 15
max_concurrent_sessions: 1
--------------------------------------------------
dns
packets: 5974
requests: 3314
responses: 2656
--------------------------------------------------
file_id
total_files: 1093
total_file_data: 4640477
--------------------------------------------------
http_inspect
flows: 1164
scans: 8137
reassembles: 12348
inspections: 7740
requests: 1429
responses: 1247
get_requests: 1398
post_requests: 31
request_bodies: 13
chunked: 122
uri_normalizations: 229
uri_path: 10
uri_coding: 4
max_concurrent_sessions: 179
--------------------------------------------------
normalizer
ip4_opts: 178
tcp_options: 6081
test_tcp_trim_win: 116541
tcp_ts_nop: 5180
test_tcp_ips_data: 2
tcp_block: 4031
--------------------------------------------------
port_scan
packets: 469494
--------------------------------------------------
ssh
packets: 5
max_concurrent_sessions: 1
--------------------------------------------------
ssl
packets: 810
decoded: 810
client_hello: 145
server_hello: 120
certificate: 53
server_done: 235
client_key_exchange: 52
server_key_exchange: 13
change_cipher: 206
client_application: 173
server_application: 131
alert: 3
unrecognized_records: 144
handshakes_completed: 37
sessions_ignored: 39
max_concurrent_sessions: 66
--------------------------------------------------
stream
ip_flows: 158
icmp_flows: 212
tcp_flows: 11352
udp_flows: 1203
--------------------------------------------------
stream_icmp
sessions: 212
max: 59
created: 212
released: 212
--------------------------------------------------
stream_ip
sessions: 158
max: 43
created: 158
released: 158
--------------------------------------------------
stream_tcp
sessions: 11352
max: 2904
created: 11352
released: 9265
instantiated: 10278
setups: 11352
restarts: 1399
resyns: 5
discards: 122178
events: 3639
syn_trackers: 3967
syn_ack_trackers: 607
data_trackers: 4696
segs_queued: 31639
segs_released: 31639
segs_used: 16924
rebuilt_packets: 12884
rebuilt_buffers: 94
rebuilt_bytes: 14102519
overlaps: 5
gaps: 110
client_cleanups: 1677
server_cleanups: 1887
memory: 4393
initializing: 5
established: 62
closing: 10
syns: 19393
syn_acks: 7730
resets: 209
fins: 22678
--------------------------------------------------
stream_udp
sessions: 1203
max: 330
created: 1203
released: 1203
--------------------------------------------------
telnet
total_packets: 12
max_concurrent_sessions: 2
--------------------------------------------------
wizard
tcp_scans: 7601
tcp_hits: 1509
udp_scans: 83922
udp_hits: 68
--------------------------------------------------
Appid dynamic stats:
unknown_app: flows: 1454, clients: 1198, users: 0, payloads 0, misc: 0
--------------------------------------------------
File Statistics
--------------------------------------------------
file type stats (files)
Type Download Upload
GZ( 33) 61 3
GIF( 63) 272 0
PNG( 69) 67 0
JPEG( 70) 199 0
ICO(149) 9 0
JPEG(157) 12 0
Total 620 3
--------------------------------------------------
file type stats (bytes)
Type Download Upload
GZ( 33) 100875 1182
GIF( 63) 81758 0
PNG( 69) 344442 0
JPEG( 70) 2539929 0
ICO(149) 10350 0
JPEG(157) 428068 0
Total 3505422 1182
--------------------------------------------------
file signature stats
Type Download Upload
GZ( 33) 61 3
GIF( 63) 272 0
PNG( 69) 67 0
JPEG( 70) 186 0
ICO(149) 9 0
JPEG(157) 12 0
Total 607 3
--------------------------------------------------
Summary Statistics
--------------------------------------------------
process
signals: 1
--------------------------------------------------
timing
runtime: 00:00:33
seconds: 33.504975
packets: 639395
pkts/sec: 19375
o")~ Snort exiting
7 months, 3 weeks
- 3
- 6
[DISCUSS] hyperscan support ARM
by bo zhaobo
Hi hyperscan team,
I'm an newbee for hyperscan project. I'm so excited to have a conversation
with you.
We have a plan to make hyperscan to support ARM64 function. And we will
propose a series of PRs to make this happen, including hardware platform
logical judgement code ,ARM NEON instruction set support and etc.. We won't
propose intrusive changes to existing code. Now the detailed design are
still uncertain, just a draft. Hope community can take part in the detailed
feature design at the beginning.
But before the whole work begins, we want to know community attitude about
this. We hope the kind feedback from your side.
Thanks very much.
BR
ZhaoBo
[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=s...>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=s...>
19/11/06
下午05:30:01
8 months, 3 weeks
- 1
- 2
- ← Newer
- 1
- Older →