December 2020 - LKP - Ml01.01.Org

[btrfs] e7a8dd2d95: fio.write_iops -98.3% regression

by kernel test robot

Greeting, FYI, we noticed a -98.3% regression of fio.write_iops due to commit: commit: e7a8dd2d9537a7ec5aeadb002fc4934be1a396eb ("[PATCH v3] btrfs: Make btrfs_direct_write atomic with respect to inode_lock") url: https://github.com/0day-ci/linux/commits/Goldwyn-Rodrigues/btrfs-Make-btr... base: https://git.kernel.org/cgit/linux/kernel/git/kdave/linux.git for-next in testcase: fio-basic on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 112G memory with following parameters: runtime: 300s disk: 1HDD fs: btrfs nr_task: 100% test_size: 128G rw: write bs: 4k ioengine: sync direct: direct cpufreq_governor: performance ucode: 0x42e test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user. test-url: https://github.com/axboe/fio If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= bs/compiler/cpufreq_governor/direct/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/ucode: 4k/gcc-9/performance/direct/1HDD/btrfs/sync/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/write/lkp-ivb-2ep1/128G/fio-basic/0x42e commit: 97847e0652 ("Merge branch 'for-next-next-v5.10-20201211' into for-next-20201211") e7a8dd2d95 ("btrfs: Make btrfs_direct_write atomic with respect to inode_lock") 97847e06525b51ea e7a8dd2d9537a7ec5aeadb002fc ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 0:4 -4% 0:4 perf-profile.children.cycles-pp.error_return 0:4 5% 0:4 perf-profile.children.cycles-pp.error_entry 0:4 5% 0:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 0.02 ± 3% +0.1 0.12 ± 13% fio.latency_1000ms% 0.01 -0.0 0.00 fio.latency_1000us% 0.96 ± 13% -0.8 0.19 ±101% fio.latency_100ms% 31.35 -31.4 0.00 fio.latency_10ms% 8.28 ± 2% -8.3 0.00 fio.latency_20ms% 0.04 ± 29% -0.0 0.02 ±100% fio.latency_250ms% 0.01 -0.0 0.00 fio.latency_2ms% 54.35 -54.3 0.00 fio.latency_4ms% 0.02 ± 10% +99.6 99.66 fio.latency_500ms% 0.01 -0.0 0.00 fio.latency_500us% 4.97 -5.0 0.00 fio.latency_50ms% 0.01 -0.0 0.00 fio.latency_750us% 17301324 -93.9% 1062134 ± 2% fio.time.file_system_outputs 316.50 ± 45% +583.7% 2164 ± 14% fio.time.involuntary_context_switches 23.75 -63.2% 8.75 ± 4% fio.time.percent_of_cpu_this_job_got 62.17 -58.4% 25.87 ± 2% fio.time.system_time 10.95 ± 2% -83.7% 1.78 ± 9% fio.time.user_time 2194671 -93.0% 153935 fio.time.voluntary_context_switches 2162665 -98.3% 36777 fio.workload 28.16 -98.3% 0.48 fio.write_bw_MBps 12156928 +3384.6% 4.236e+08 fio.write_clat_90%_us 23855104 ± 2% +1715.4% 4.331e+08 fio.write_clat_95%_us 50724864 ± 4% +797.2% 4.551e+08 fio.write_clat_99%_us 6657995 +5784.2% 3.918e+08 fio.write_clat_mean_us 15787433 ± 2% +110.1% 33164741 ± 9% fio.write_clat_stddev 7208 -98.3% 122.53 fio.write_iops 10477 +39.4% 14609 uptime.idle 18.03 ± 15% -25.7% 13.39 boot-time.dhcp 1017 ± 11% -23.4% 779.50 boot-time.idle 64.67 +45.5% 94.08 iostat.cpu.idle 32.22 ± 2% -92.9% 2.28 ± 3% iostat.cpu.iowait 460106 ± 33% +56.7% 720959 ± 15% numa-numastat.node0.local_node 461015 ± 33% +61.1% 742788 ± 15% numa-numastat.node0.numa_hit 598719 ± 25% -43.3% 339612 ± 34% numa-numastat.node1.local_node 639421 ± 24% -43.4% 361729 ± 31% numa-numastat.node1.numa_hit 61123 ± 15% -82.4% 10785 meminfo.Active(anon) 10906 ± 2% +490.4% 64393 ± 15% meminfo.Active(file) 1678 -85.8% 238.00 ± 14% meminfo.Dirty 31700 +351.3% 143073 ± 5% meminfo.Inactive(file) 283671 -52.2% 135570 meminfo.SUnreclaim 359419 -41.1% 211861 meminfo.Slab 64.47 +29.6 94.08 mpstat.cpu.all.idle% 32.42 ± 2% -30.1 2.29 ± 3% mpstat.cpu.all.iowait% 2.04 ± 9% +1.0 3.08 ± 12% mpstat.cpu.all.irq% 0.23 ± 6% -0.0 0.19 ± 2% mpstat.cpu.all.soft% 0.66 ± 2% -0.4 0.25 ± 3% mpstat.cpu.all.sys% 0.17 -0.1 0.11 ± 2% mpstat.cpu.all.usr% 64.25 +45.1% 93.25 vmstat.cpu.id 31.75 ± 2% -95.3% 1.50 ± 33% vmstat.cpu.wa 28682 -89.7% 2962 ± 2% vmstat.io.bo 46.50 -98.4% 0.75 ± 57% vmstat.procs.b 38969 -92.5% 2923 vmstat.system.cs 100537 -4.5% 96061 vmstat.system.in 1.306e+09 ± 7% -99.1% 11707061 ± 18% cpuidle.C1.time 5821744 ± 7% -96.9% 182330 ± 10% cpuidle.C1.usage 6.258e+09 ± 9% -59.8% 2.514e+09 ± 9% cpuidle.C1E.time 18768783 ± 4% -29.4% 13255968 ± 26% cpuidle.C1E.usage 6.397e+09 ± 9% +77.6% 1.136e+10 ± 2% cpuidle.C6.time 9966359 ± 7% +63.3% 16279745 ± 22% cpuidle.C6.usage 5156525 ± 14% -99.2% 40443 ± 13% cpuidle.POLL.time 3217284 ± 17% -99.4% 19436 ± 12% cpuidle.POLL.usage 12395 ± 7% +201.0% 37310 ± 23% numa-meminfo.node0.Active 10314 ± 6% +235.7% 34621 ± 21% numa-meminfo.node0.Active(file) 1636 -91.5% 138.75 ± 16% numa-meminfo.node0.Dirty 24039 ± 54% +264.0% 87496 ± 4% numa-meminfo.node0.Inactive(file) 219811 ± 4% -68.3% 69676 ± 4% numa-meminfo.node0.SUnreclaim 258497 ± 4% -57.4% 110004 ± 6% numa-meminfo.node0.Slab 59798 ± 14% -36.6% 37897 ± 6% numa-meminfo.node1.Active 59211 ± 15% -86.3% 8123 ± 15% numa-meminfo.node1.Active(anon) 587.00 ± 72% +4972.2% 29774 ± 11% numa-meminfo.node1.Active(file) 183687 ± 20% -31.4% 126043 ± 15% numa-meminfo.node1.AnonPages.max 7665 ±171% +624.8% 55557 ± 9% numa-meminfo.node1.Inactive(file) 139520 ±120% -81.7% 25573 ± 4% sched_debug.cfs_rq:/.load.avg 50.14 ± 19% -21.8% 39.22 ± 7% sched_debug.cfs_rq:/.load_avg.avg 0.11 ± 6% -22.2% 0.09 ± 7% sched_debug.cfs_rq:/.nr_running.avg 0.29 ± 5% -15.0% 0.25 sched_debug.cfs_rq:/.nr_running.stddev 122.82 ± 4% -16.2% 102.93 ± 4% sched_debug.cfs_rq:/.runnable_avg.avg 122.46 ± 4% -16.5% 102.27 ± 5% sched_debug.cfs_rq:/.util_avg.avg 101814 ± 11% +277.7% 384526 ± 27% sched_debug.cpu.avg_idle.min 220884 ± 5% -37.6% 137900 ± 13% sched_debug.cpu.avg_idle.stddev 1.10 ± 4% +17.5% 1.30 ± 3% sched_debug.cpu.clock.stddev 1669 ± 4% -63.3% 612.98 ± 6% sched_debug.cpu.clock_task.stddev 5275 ± 6% +10.7% 5839 sched_debug.cpu.curr->pid.max 106300 ± 10% -89.3% 11398 sched_debug.cpu.nr_switches.avg 1209385 ± 11% -97.4% 31808 ± 14% sched_debug.cpu.nr_switches.max 192228 ± 11% -97.3% 5222 ± 18% sched_debug.cpu.nr_switches.stddev -251.12 -86.0% -35.25 sched_debug.cpu.nr_uninterruptible.min 40.50 ± 14% -64.2% 14.48 ± 24% sched_debug.cpu.nr_uninterruptible.stddev 2577 ± 6% +235.7% 8653 ± 21% numa-vmstat.node0.nr_active_file 10194 ± 42% +216.4% 32253 ± 5% numa-vmstat.node0.nr_dirtied 409.50 -91.6% 34.50 ± 15% numa-vmstat.node0.nr_dirty 6009 ± 54% +264.0% 21873 ± 4% numa-vmstat.node0.nr_inactive_file 54951 ± 4% -68.3% 17418 ± 4% numa-vmstat.node0.nr_slab_unreclaimable 9778 ± 43% +229.6% 32228 ± 5% numa-vmstat.node0.nr_written 2577 ± 6% +235.7% 8653 ± 21% numa-vmstat.node0.nr_zone_active_file 6009 ± 54% +264.0% 21873 ± 4% numa-vmstat.node0.nr_zone_inactive_file 414.75 ± 2% -92.5% 31.00 ± 13% numa-vmstat.node0.nr_zone_write_pending 14804 ± 15% -86.3% 2027 ± 15% numa-vmstat.node1.nr_active_anon 146.75 ± 72% +4970.4% 7440 ± 10% numa-vmstat.node1.nr_active_file 2882 ±147% +676.1% 22371 ± 5% numa-vmstat.node1.nr_dirtied 1915 ±171% +625.1% 13888 ± 9% numa-vmstat.node1.nr_inactive_file 2871 ±147% +678.4% 22351 ± 5% numa-vmstat.node1.nr_written 14804 ± 15% -86.3% 2027 ± 15% numa-vmstat.node1.nr_zone_active_anon 146.75 ± 72% +4970.4% 7440 ± 10% numa-vmstat.node1.nr_zone_active_file 1915 ±171% +625.1% 13888 ± 9% numa-vmstat.node1.nr_zone_inactive_file 15380 ± 15% -82.4% 2703 proc-vmstat.nr_active_anon 2725 ± 2% +490.4% 16094 ± 16% proc-vmstat.nr_active_file 17429 +471.3% 99580 ± 2% proc-vmstat.nr_dirtied 416.75 -86.0% 58.25 ± 13% proc-vmstat.nr_dirty 567440 +4.9% 595244 proc-vmstat.nr_file_pages 7926 +351.0% 35749 ± 5% proc-vmstat.nr_inactive_file 12783 +1.7% 13005 proc-vmstat.nr_kernel_stack 297875 -4.5% 284331 proc-vmstat.nr_shmem 70910 -52.2% 33895 proc-vmstat.nr_slab_unreclaimable 16905 +488.8% 99538 ± 2% proc-vmstat.nr_written 15380 ± 15% -82.4% 2703 proc-vmstat.nr_zone_active_anon 2725 ± 2% +490.4% 16094 ± 16% proc-vmstat.nr_zone_active_file 7926 +351.0% 35749 ± 5% proc-vmstat.nr_zone_inactive_file 424.75 -87.9% 51.50 ± 9% proc-vmstat.nr_zone_write_pending 42826 ± 14% -40.5% 25476 ± 3% proc-vmstat.pgactivate 1239967 -3.7% 1194026 proc-vmstat.pgalloc_normal 8740013 -89.7% 904228 ± 2% proc-vmstat.pgpgout 53472 ± 3% +5.6% 56482 proc-vmstat.pgreuse 158.00 ± 12% +348.6% 708.75 ± 2% slabinfo.biovec-128.active_objs 158.00 ± 12% +348.6% 708.75 ± 2% slabinfo.biovec-128.num_objs 374.25 ± 8% +258.8% 1342 ± 2% slabinfo.biovec-64.active_objs 374.25 ± 8% +258.8% 1342 ± 2% slabinfo.biovec-64.num_objs 163.50 ± 13% +227.8% 536.00 ± 7% slabinfo.biovec-max.active_objs 204.75 ± 13% +168.7% 550.25 ± 7% slabinfo.biovec-max.num_objs 1297 ± 14% -34.9% 845.25 ± 6% slabinfo.btrfs_delayed_tree_ref.active_objs 1297 ± 14% -34.9% 845.25 ± 6% slabinfo.btrfs_delayed_tree_ref.num_objs 2006 ± 13% +59.9% 3209 slabinfo.btrfs_extent_buffer.active_objs 2006 ± 13% +59.9% 3209 slabinfo.btrfs_extent_buffer.num_objs 336.00 ± 23% -31.2% 231.00 ± 15% slabinfo.kmalloc-rcl-192.active_objs 336.00 ± 23% -31.2% 231.00 ± 15% slabinfo.kmalloc-rcl-192.num_objs 1079929 -99.6% 4015 slabinfo.pid_namespace.active_objs 19284 -99.6% 71.25 slabinfo.pid_namespace.active_slabs 1079934 -99.6% 4015 slabinfo.pid_namespace.num_objs 19284 -99.6% 71.25 slabinfo.pid_namespace.num_slabs 681.25 ± 10% +233.8% 2274 ± 12% slabinfo.skbuff_fclone_cache.active_objs 681.25 ± 10% +233.8% 2274 ± 12% slabinfo.skbuff_fclone_cache.num_objs 705679 ± 2% -96.8% 22805 softirqs.BLOCK 56594 ± 12% -36.3% 36023 ± 19% softirqs.CPU0.RCU 61813 ± 12% -41.2% 36360 ± 13% softirqs.CPU1.RCU 61170 ± 12% -41.9% 35511 ± 4% softirqs.CPU10.RCU 59336 ± 10% -37.7% 36972 ± 5% softirqs.CPU11.RCU 58606 ± 14% -36.7% 37114 ± 17% softirqs.CPU2.RCU 54177 ± 12% -40.1% 32471 ± 12% softirqs.CPU24.RCU 55834 ± 11% -36.9% 35230 ± 10% softirqs.CPU25.RCU 53216 ± 9% -31.9% 36262 ± 15% softirqs.CPU26.RCU 53744 ± 12% -35.7% 34533 ± 16% softirqs.CPU27.RCU 51611 ± 8% -33.8% 34181 ± 13% softirqs.CPU28.RCU 53726 ± 7% -33.5% 35754 ± 9% softirqs.CPU29.RCU 61015 ± 14% -41.3% 35804 ± 18% softirqs.CPU3.RCU 54715 ± 12% -37.5% 34188 ± 3% softirqs.CPU30.RCU 51435 ± 10% -33.5% 34224 ± 7% softirqs.CPU31.RCU 52697 ± 8% -38.8% 32238 ± 5% softirqs.CPU32.RCU 27579 ± 7% +49.8% 41304 ± 3% softirqs.CPU32.SCHED 48705 ± 9% -32.2% 33031 ± 4% softirqs.CPU33.RCU 48234 ± 8% -29.0% 34223 ± 3% softirqs.CPU34.RCU 49281 ± 10% -29.7% 34654 ± 3% softirqs.CPU35.RCU 54023 ± 8% -35.2% 34993 ± 19% softirqs.CPU4.RCU 55222 ± 26% -34.4% 36234 ± 19% softirqs.CPU40.RCU 59824 ± 8% -37.8% 37225 ± 14% softirqs.CPU5.RCU 62723 ± 13% -41.6% 36615 ± 9% softirqs.CPU6.RCU 56745 ± 13% -33.6% 37668 ± 11% softirqs.CPU7.RCU 685836 -98.4% 10922 ± 2% softirqs.CPU8.BLOCK 63989 ± 9% -41.2% 37647 ± 5% softirqs.CPU8.RCU 693875 -96.6% 23342 softirqs.CPU8.TASKLET 60701 ± 10% -41.2% 35709 ± 9% softirqs.CPU9.RCU 2511282 ± 18% -31.4% 1721969 ± 10% softirqs.RCU 694098 -96.6% 23565 softirqs.TASKLET 2089 ± 62% -80.3% 410.75 ± 91% interrupts.41:PCI-MSI.2621447-edge.eth0-TxRx-6 1385835 -96.8% 44728 interrupts.73:PCI-MSI.2097152-edge.isci-msix 774.25 ± 60% -87.9% 94.00 ± 35% interrupts.CPU1.RES:Rescheduling_interrupts 1167 ± 14% -85.9% 164.50 ± 63% interrupts.CPU10.RES:Rescheduling_interrupts 1041 ± 18% -82.2% 185.25 ± 48% interrupts.CPU11.RES:Rescheduling_interrupts 188.00 ± 23% +32.0% 248.25 ± 26% interrupts.CPU14.NMI:Non-maskable_interrupts 188.00 ± 23% +32.0% 248.25 ± 26% interrupts.CPU14.PMI:Performance_monitoring_interrupts 113.75 ± 29% -44.2% 63.50 ± 15% interrupts.CPU16.RES:Rescheduling_interrupts 470.50 ± 38% -83.7% 76.50 ± 39% interrupts.CPU2.RES:Rescheduling_interrupts 880.75 ± 24% -87.1% 113.75 ± 54% interrupts.CPU24.RES:Rescheduling_interrupts 320.50 ± 27% -37.6% 200.00 ± 28% interrupts.CPU25.NMI:Non-maskable_interrupts 320.50 ± 27% -37.6% 200.00 ± 28% interrupts.CPU25.PMI:Performance_monitoring_interrupts 640.00 ± 28% -85.5% 92.50 ± 59% interrupts.CPU25.RES:Rescheduling_interrupts 572.00 ± 30% -85.7% 81.75 ± 42% interrupts.CPU26.RES:Rescheduling_interrupts 582.25 ± 25% -85.1% 86.50 ± 40% interrupts.CPU27.RES:Rescheduling_interrupts 474.00 ± 20% -82.4% 83.25 ± 24% interrupts.CPU28.RES:Rescheduling_interrupts 375.00 ± 13% -74.7% 95.00 ± 29% interrupts.CPU29.RES:Rescheduling_interrupts 527.50 ± 46% -82.2% 93.75 ± 46% interrupts.CPU3.RES:Rescheduling_interrupts 949.00 ± 10% +56.0% 1480 ± 37% interrupts.CPU30.CAL:Function_call_interrupts 447.75 ± 30% -70.2% 133.50 ± 48% interrupts.CPU30.RES:Rescheduling_interrupts 892.25 ± 2% +26.9% 1132 ± 15% interrupts.CPU31.CAL:Function_call_interrupts 2089 ± 62% -80.3% 410.75 ± 91% interrupts.CPU32.41:PCI-MSI.2621447-edge.eth0-TxRx-6 968.00 ± 10% +94.1% 1878 ± 59% interrupts.CPU32.CAL:Function_call_interrupts 826.50 ± 28% -71.7% 233.75 ± 31% interrupts.CPU32.NMI:Non-maskable_interrupts 826.50 ± 28% -71.7% 233.75 ± 31% interrupts.CPU32.PMI:Performance_monitoring_interrupts 656.25 ± 15% -75.5% 160.75 ± 64% interrupts.CPU32.RES:Rescheduling_interrupts 558.25 ± 32% -75.0% 139.50 ± 55% interrupts.CPU33.RES:Rescheduling_interrupts 447.25 ± 16% -61.8% 171.00 ± 47% interrupts.CPU34.RES:Rescheduling_interrupts 504.50 ± 33% -81.9% 91.25 ± 32% interrupts.CPU35.RES:Rescheduling_interrupts 301.50 ± 40% -77.9% 66.75 ± 62% interrupts.CPU4.RES:Rescheduling_interrupts 315.25 ± 43% -65.9% 107.50 ± 81% interrupts.CPU5.RES:Rescheduling_interrupts 491.75 ± 57% -86.3% 67.25 ± 35% interrupts.CPU6.RES:Rescheduling_interrupts 1385835 -96.8% 44728 interrupts.CPU8.73:PCI-MSI.2097152-edge.isci-msix 916.25 ± 9% +26.0% 1154 ± 18% interrupts.CPU8.CAL:Function_call_interrupts 827.25 ± 38% -75.0% 206.50 ± 39% interrupts.CPU8.NMI:Non-maskable_interrupts 827.25 ± 38% -75.0% 206.50 ± 39% interrupts.CPU8.PMI:Performance_monitoring_interrupts 906.25 ± 2% +53.9% 1395 ± 47% interrupts.CPU9.CAL:Function_call_interrupts 1107 ± 10% -87.6% 137.50 ± 49% interrupts.CPU9.RES:Rescheduling_interrupts 13410 ± 5% -16.0% 11263 ± 7% interrupts.NMI:Non-maskable_interrupts 13410 ± 5% -16.0% 11263 ± 7% interrupts.PMI:Performance_monitoring_interrupts 15515 ± 16% -69.0% 4813 ± 14% interrupts.RES:Rescheduling_interrupts 0.02 ± 49% +181.5% 0.05 ± 72% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.01 ± 8% +35.1% 0.01 ± 8% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.03 ± 59% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.01 ± 14% +287.0% 0.02 ± 16% perf-sched.sch_delay.avg.ms.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 0.01 ± 57% -58.8% 0.00 ± 14% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.00 ±118% +350.0% 0.02 ± 68% perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.__btrfs_tree_read_lock.btrfs_read_lock_root_node.btrfs_search_slot 0.00 ±100% +412.5% 0.01 ± 21% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot 0.01 ± 49% +86.5% 0.02 ± 8% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.00 ± 57% +733.3% 0.01 ± 18% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.02 ± 9% +37.7% 0.02 ± 10% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 0.01 ± 7% +73.1% 0.01 ± 3% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 0.03 ± 34% +72.1% 0.06 ± 18% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.03 ± 59% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.03 ±110% +197.4% 0.08 ± 32% perf-sched.sch_delay.max.ms.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.filemap_fdatawait_range 0.01 ±110% +274.4% 0.04 ± 64% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.__btrfs_tree_read_lock.btrfs_read_lock_root_node.btrfs_search_slot 0.03 ± 60% +91.0% 0.06 ± 10% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.02 ± 31% +130.6% 0.04 ± 12% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.04 ± 18% +26.8% 0.05 ± 7% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 0.01 ± 11% +125.0% 0.01 ± 3% perf-sched.total_sch_delay.average.ms 10.12 ± 21% +1307.5% 142.39 perf-sched.total_wait_and_delay.average.ms 172750 ± 17% -91.4% 14933 ± 2% perf-sched.total_wait_and_delay.count.ms 5885 ± 25% +56.9% 9234 ± 11% perf-sched.total_wait_and_delay.max.ms 10.11 ± 21% +1308.2% 142.38 perf-sched.total_wait_time.average.ms 5885 ± 25% +56.9% 9234 ± 11% perf-sched.total_wait_time.max.ms 8.01 ± 22% +1181.9% 102.62 ± 62% perf-sched.wait_and_delay.avg.ms.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 1.02 ±173% +314.1% 4.24 ± 3% perf-sched.wait_and_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 232.13 ± 18% +103.1% 471.43 ± 36% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 4.54 ± 6% -8.7% 4.14 ± 4% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 393.96 ± 14% +23.3% 485.81 ± 5% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 4.09 ± 26% +6387.8% 265.12 ± 2% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 61985 ± 18% -98.0% 1211 ± 2% perf-sched.wait_and_delay.count.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 2.00 ±173% +4850.0% 99.00 ± 18% perf-sched.wait_and_delay.count.io_schedule.wait_on_page_bit.__filemap_fdatawait_range.filemap_fdatawait_range 83.50 ± 16% -71.0% 24.25 ± 6% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 1547 ± 7% +16.4% 1801 ± 3% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 102506 ± 17% -97.7% 2395 ± 7% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 943.35 ± 8% -81.1% 178.11 ± 56% perf-sched.wait_and_delay.max.ms.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 1.26 ±173% +304.5% 5.11 perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork 1589 ± 52% +190.9% 4624 ± 25% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 0.00 ±173% +1.2e+05% 5.20 ± 73% perf-sched.wait_time.avg.ms.btrfs_commit_transaction.transaction_kthread.kthread.ret_from_fork 3.92 ± 11% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 8.00 ± 22% +1182.5% 102.60 ± 62% perf-sched.wait_time.avg.ms.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 0.28 ±155% +24627.1% 69.30 ±106% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot 232.13 ± 18% +103.1% 471.42 ± 36% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 4.53 ± 6% -8.9% 4.13 ± 4% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 393.96 ± 14% +23.3% 485.80 ± 5% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 4.08 ± 26% +6398.3% 265.11 ± 2% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 0.00 ±173% +4e+05% 17.17 ±107% perf-sched.wait_time.max.ms.btrfs_commit_transaction.transaction_kthread.kthread.ret_from_fork 3.92 ± 11% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 943.35 ± 8% -81.1% 178.04 ± 56% perf-sched.wait_time.max.ms.io_schedule.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 6.02 ±172% +2236.8% 140.75 ± 90% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot 0.01 ± 86% +7423.8% 0.79 ±167% perf-sched.wait_time.max.ms.schedule_timeout.io_schedule_timeout.wait_for_completion_io.write_all_supers 1589 ± 52% +190.9% 4624 ± 25% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork 44.03 +20.3% 52.96 ± 12% perf-stat.i.MPKI 2.589e+08 -36.7% 1.639e+08 perf-stat.i.branch-instructions 5.01 +6.7 11.71 ± 7% perf-stat.i.branch-miss-rate% 12714258 +31.8% 16758440 ± 7% perf-stat.i.branch-misses 9.35 ± 2% +14.7 24.09 ± 3% perf-stat.i.cache-miss-rate% 4375264 +81.1% 7923317 ± 18% perf-stat.i.cache-misses 48162798 -32.2% 32638947 ± 14% perf-stat.i.cache-references 39282 -92.6% 2916 perf-stat.i.context-switches 4.10 +83.9% 7.54 ± 6% perf-stat.i.cpi 68.61 ± 2% +64.8% 113.06 perf-stat.i.cpu-migrations 1061 ± 3% -42.2% 613.59 ± 8% perf-stat.i.cycles-between-cache-misses 1.23 ± 2% -0.2 1.06 ± 8% perf-stat.i.dTLB-load-miss-rate% 3992778 ± 2% -47.7% 2089367 ± 11% perf-stat.i.dTLB-load-misses 3.459e+08 -35.4% 2.234e+08 ± 2% perf-stat.i.dTLB-loads 0.17 ± 2% +0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate% 70.32 +21.5 91.83 perf-stat.i.iTLB-load-miss-rate% 1985999 ± 2% -39.6% 1199619 ± 2% perf-stat.i.iTLB-load-misses 854405 -87.4% 107547 ± 14% perf-stat.i.iTLB-loads 1.254e+09 -36.9% 7.911e+08 perf-stat.i.instructions 0.26 -41.9% 0.15 ± 6% perf-stat.i.ipc 1.34 ± 7% -58.0% 0.56 ± 31% perf-stat.i.metric.K/sec 19.36 -25.9% 14.35 ± 6% perf-stat.i.metric.M/sec 2893 -2.0% 2835 perf-stat.i.minor-faults 197998 ± 19% -32.9% 132808 ± 3% perf-stat.i.node-load-misses 228908 ± 17% -31.8% 156142 ± 3% perf-stat.i.node-loads 236295 ± 7% -17.5% 194871 ± 6% perf-stat.i.node-stores 3820 -1.7% 3754 perf-stat.i.page-faults 4.91 +5.3 10.21 ± 7% perf-stat.overall.branch-miss-rate% 9.09 ± 2% +15.1 24.16 ± 3% perf-stat.overall.cache-miss-rate% 3.65 +65.6% 6.04 ± 7% perf-stat.overall.cpi 1045 ± 3% -41.3% 613.83 ± 8% perf-stat.overall.cycles-between-cache-misses 1.14 ± 2% -0.2 0.92 ± 8% perf-stat.overall.dTLB-load-miss-rate% 0.17 ± 3% +0.0 0.19 ± 4% perf-stat.overall.dTLB-store-miss-rate% 69.91 +21.9 91.81 perf-stat.overall.iTLB-load-miss-rate% 0.27 -39.3% 0.17 ± 6% perf-stat.overall.ipc 175094 +3617.1% 6508495 perf-stat.overall.path-length 2.583e+08 -36.7% 1.636e+08 perf-stat.ps.branch-instructions 12678701 +31.8% 16711512 ± 7% perf-stat.ps.branch-misses 4361806 +81.1% 7898239 ± 18% perf-stat.ps.cache-misses 48005640 -32.2% 32532931 ± 14% perf-stat.ps.cache-references 39151 -92.6% 2907 perf-stat.ps.context-switches 68.42 ± 2% +64.7% 112.70 perf-stat.ps.cpu-migrations 3979673 ± 2% -47.7% 2082707 ± 11% perf-stat.ps.dTLB-load-misses 3.449e+08 -35.4% 2.229e+08 ± 2% perf-stat.ps.dTLB-loads 1979429 ± 2% -39.6% 1195674 ± 2% perf-stat.ps.iTLB-load-misses 851561 -87.4% 107209 ± 14% perf-stat.ps.iTLB-loads 1.251e+09 -36.9% 7.896e+08 perf-stat.ps.instructions 2884 -2.0% 2827 perf-stat.ps.minor-faults 197548 ± 19% -33.0% 132439 ± 3% perf-stat.ps.node-load-misses 228425 ± 17% -31.8% 155760 ± 3% perf-stat.ps.node-loads 236208 ± 7% -17.3% 195231 ± 6% perf-stat.ps.node-stores 3.787e+11 -36.8% 2.393e+11 perf-stat.total.instructions 10.21 ± 20% -9.9 0.36 ±100% perf-profile.calltrace.cycles-pp.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 8.66 ± 23% -8.5 0.17 ±173% perf-profile.calltrace.cycles-pp.iomap_apply.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write.vfs_write 11.16 ± 19% -7.3 3.87 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 11.02 ± 19% -7.2 3.84 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.77 ± 19% -7.2 3.59 ± 6% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.72 ± 19% -7.1 3.59 ± 6% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.54 ± 19% -6.9 3.59 ± 6% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.51 ± 20% -6.9 3.59 ± 6% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 6.28 ± 31% -6.1 0.14 ±173% perf-profile.calltrace.cycles-pp.btrfs_dio_iomap_begin.iomap_apply.__iomap_dio_rw.btrfs_file_write_iter.new_sync_write 6.48 ± 12% -4.6 1.87 ± 23% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 6.63 ± 12% -4.5 2.11 ± 22% perf-profile.calltrace.cycles-pp.ret_from_fork 6.63 ± 12% -4.5 2.11 ± 22% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 5.90 ± 11% -4.1 1.82 ± 23% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.91 ± 8% +0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 1.16 ± 12% +0.3 1.43 ± 6% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.64 ± 9% +0.3 0.93 ± 8% perf-profile.calltrace.cycles-pp.update_blocked_averages.run_rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack 0.99 ± 7% +0.3 1.32 ± 12% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.65 ± 10% +0.3 0.98 ± 11% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu 0.26 ±100% +0.4 0.68 ± 14% perf-profile.calltrace.cycles-pp.cpuidle_governor_latency_req.menu_select.do_idle.cpu_startup_entry.start_secondary 1.29 ± 6% +0.4 1.71 ± 14% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 1.56 ± 9% +0.4 1.99 ± 3% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.75 ± 10% +0.5 2.27 ± 5% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.67 ± 65% +0.5 1.19 ± 8% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_irq_work.asm_sysvec_irq_work.cpuidle_enter_state.cpuidle_enter 0.67 ± 65% +0.5 1.19 ± 8% perf-profile.calltrace.cycles-pp.asm_sysvec_irq_work.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.67 ± 65% +0.5 1.19 ± 8% perf-profile.calltrace.cycles-pp.sysvec_irq_work.asm_sysvec_irq_work.cpuidle_enter_state.cpuidle_enter.do_idle 0.67 ± 65% +0.5 1.19 ± 8% perf-profile.calltrace.cycles-pp.__sysvec_irq_work.asm_call_sysvec_on_stack.sysvec_irq_work.asm_sysvec_irq_work.cpuidle_enter_state 0.00 +0.6 0.60 ± 6% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.wait_log_commit.btrfs_sync_log.btrfs_sync_file 0.00 +0.6 0.60 ± 7% perf-profile.calltrace.cycles-pp.timerqueue_del.__remove_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 0.14 ±173% +0.6 0.74 ± 7% perf-profile.calltrace.cycles-pp.__remove_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.00 +0.6 0.62 ± 13% perf-profile.calltrace.cycles-pp.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.00 +0.6 0.62 ± 5% perf-profile.calltrace.cycles-pp.__mutex_lock.wait_log_commit.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter 0.80 ± 58% +0.6 1.45 ± 17% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 0.80 ± 58% +0.6 1.45 ± 17% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 0.15 ±173% +0.7 0.83 ± 12% perf-profile.calltrace.cycles-pp.rcu_sched_clock_irq.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.00 +0.7 0.72 ± 5% perf-profile.calltrace.cycles-pp.wait_log_commit.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter.new_sync_write 0.00 +0.7 0.73 ± 20% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node 0.00 +0.8 0.77 ± 18% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot 0.00 +0.8 0.77 ± 18% perf-profile.calltrace.cycles-pp.btrfs_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.btrfs_log_inode.btrfs_log_inode_parent 0.00 +0.8 0.77 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.btrfs_log_inode 0.00 +0.8 0.77 ± 18% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items 2.60 ± 14% +0.9 3.48 ± 10% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.9 0.90 ± 5% perf-profile.calltrace.cycles-pp.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter.new_sync_write.vfs_write 2.73 ± 7% +1.1 3.81 ± 4% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 2.92 ± 7% +1.2 4.08 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 4.36 ± 15% +1.2 5.60 ± 9% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 1.03 ± 13% +1.2 2.28 ± 33% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel 1.05 ± 13% +1.3 2.31 ± 33% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify 3.44 ± 8% +1.3 4.72 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 1.15 ± 15% +1.4 2.54 ± 32% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify 1.15 ± 15% +1.4 2.54 ± 32% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64_no_verify 1.15 ± 15% +1.4 2.54 ± 32% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64_no_verify 0.00 +1.7 1.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe 0.00 +1.7 1.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file 0.00 +1.9 1.87 ± 18% perf-profile.calltrace.cycles-pp.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file.btrfs_file_write_iter 0.00 +2.0 2.02 ± 17% perf-profile.calltrace.cycles-pp.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file.btrfs_file_write_iter.new_sync_write 0.00 +2.0 2.03 ± 17% perf-profile.calltrace.cycles-pp.btrfs_log_dentry_safe.btrfs_sync_file.btrfs_file_write_iter.new_sync_write.vfs_write 5.73 ± 7% +2.3 8.03 ± 3% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 8.28 ± 10% +2.5 10.75 ± 4% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 8.66 ± 9% +2.7 11.31 ± 4% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 8.79 ± 9% +2.7 11.50 ± 4% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +3.0 3.00 ± 11% perf-profile.calltrace.cycles-pp.btrfs_sync_file.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 13.50 ± 9% +4.3 17.84 ± 7% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 15.39 ± 9% +4.7 20.10 ± 7% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 51.90 +8.8 60.72 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 72.26 ± 2% +9.9 82.17 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 71.31 ± 2% +9.9 81.25 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 79.02 ± 3% +10.8 89.79 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 79.10 ± 3% +10.8 89.91 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 79.10 ± 3% +10.8 89.91 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 80.25 ± 2% +12.2 92.45 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 10.22 ± 20% -9.6 0.58 ± 26% perf-profile.children.cycles-pp.__iomap_dio_rw 8.66 ± 23% -8.2 0.45 ± 32% perf-profile.children.cycles-pp.iomap_apply 10.79 ± 19% -7.2 3.62 ± 6% perf-profile.children.cycles-pp.ksys_write 12.01 ± 17% -7.1 4.86 ± 8% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 10.74 ± 19% -7.1 3.62 ± 6% perf-profile.children.cycles-pp.vfs_write 11.77 ± 17% -7.0 4.73 ± 8% perf-profile.children.cycles-pp.do_syscall_64 10.56 ± 19% -6.9 3.62 ± 6% perf-profile.children.cycles-pp.new_sync_write 10.52 ± 20% -6.9 3.59 ± 6% perf-profile.children.cycles-pp.btrfs_file_write_iter 6.28 ± 31% -5.9 0.38 ± 36% perf-profile.children.cycles-pp.btrfs_dio_iomap_begin 6.48 ± 12% -4.6 1.87 ± 23% perf-profile.children.cycles-pp.worker_thread 6.63 ± 12% -4.5 2.11 ± 22% perf-profile.children.cycles-pp.kthread 6.63 ± 12% -4.5 2.12 ± 22% perf-profile.children.cycles-pp.ret_from_fork 5.91 ± 11% -4.1 1.82 ± 23% perf-profile.children.cycles-pp.process_one_work 3.45 ± 24% -3.3 0.14 ± 55% perf-profile.children.cycles-pp.btrfs_work_helper 3.43 ± 24% -3.3 0.14 ± 55% perf-profile.children.cycles-pp.btrfs_finish_ordered_io 2.77 ± 43% -2.7 0.04 ±106% perf-profile.children.cycles-pp.asm_common_interrupt 2.77 ± 43% -2.7 0.04 ±106% perf-profile.children.cycles-pp.common_interrupt 2.33 ± 8% -2.3 0.04 ±107% perf-profile.children.cycles-pp.can_nocow_extent 2.21 ± 12% -2.1 0.06 ± 67% perf-profile.children.cycles-pp.iomap_dio_bio_actor 2.01 ± 42% -1.9 0.08 ± 17% perf-profile.children.cycles-pp.blk_done_softirq 1.95 ± 42% -1.9 0.07 ± 17% perf-profile.children.cycles-pp.scsi_io_completion 1.95 ± 42% -1.9 0.07 ± 17% perf-profile.children.cycles-pp.scsi_end_request 2.13 ± 94% -1.9 0.27 ± 49% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_space 2.17 ± 12% -1.8 0.37 ± 27% perf-profile.children.cycles-pp.__schedule 2.51 ± 96% -1.7 0.77 ± 25% perf-profile.children.cycles-pp._raw_spin_lock 2.00 ±100% -1.7 0.26 ± 51% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 1.92 ±104% -1.6 0.29 ± 47% perf-profile.children.cycles-pp.__reserve_bytes 1.68 ± 13% -1.6 0.05 ± 62% perf-profile.children.cycles-pp.iomap_dio_submit_bio 1.90 ±105% -1.6 0.28 ± 48% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 1.66 ± 12% -1.6 0.05 ± 62% perf-profile.children.cycles-pp.btrfs_submit_direct 1.43 ± 41% -1.4 0.04 ±100% perf-profile.children.cycles-pp.blk_update_request 1.35 ± 41% -1.3 0.03 ±100% perf-profile.children.cycles-pp.btrfs_end_bio 1.50 ± 13% -1.3 0.24 ± 17% perf-profile.children.cycles-pp.schedule 1.11 ± 12% -1.1 0.04 ±100% perf-profile.children.cycles-pp.btrfs_map_bio 0.97 ± 18% -0.9 0.03 ±100% perf-profile.children.cycles-pp.io_schedule 0.98 ± 32% -0.9 0.09 ± 16% perf-profile.children.cycles-pp.__blk_mq_run_hw_queue 0.97 ± 32% -0.9 0.09 ± 16% perf-profile.children.cycles-pp.blk_mq_sched_dispatch_requests 0.96 ± 33% -0.9 0.09 ± 16% perf-profile.children.cycles-pp.__blk_mq_sched_dispatch_requests 0.92 ± 33% -0.8 0.09 ± 21% perf-profile.children.cycles-pp.__blk_mq_do_dispatch_sched 0.89 ± 4% -0.8 0.05 ± 61% perf-profile.children.cycles-pp.read_block_for_search 0.94 ± 32% -0.8 0.15 ± 39% perf-profile.children.cycles-pp.try_to_wake_up 0.85 ± 16% -0.7 0.16 ± 67% perf-profile.children.cycles-pp.btrfs_csum_file_blocks 0.69 ± 34% -0.6 0.06 ± 17% perf-profile.children.cycles-pp.blk_mq_dispatch_rq_list 0.73 ± 9% -0.6 0.14 ± 47% perf-profile.children.cycles-pp.schedule_idle 0.65 ± 33% -0.6 0.06 ± 17% perf-profile.children.cycles-pp.scsi_queue_rq 0.65 ± 30% -0.5 0.12 ± 33% perf-profile.children.cycles-pp.pick_next_task_fair 0.56 ± 10% -0.5 0.09 ± 13% perf-profile.children.cycles-pp.kmem_cache_alloc 0.42 ± 32% -0.4 0.04 ± 58% perf-profile.children.cycles-pp.sas_queuecommand 0.39 ± 33% -0.4 0.03 ±100% perf-profile.children.cycles-pp.__ata_scsi_queuecmd 0.43 ± 8% -0.4 0.07 ± 22% perf-profile.children.cycles-pp.dequeue_task_fair 0.43 ± 18% -0.3 0.09 ± 28% perf-profile.children.cycles-pp.blk_finish_plug 0.42 ± 19% -0.3 0.09 ± 28% perf-profile.children.cycles-pp.blk_flush_plug_list 0.42 ± 18% -0.3 0.09 ± 28% perf-profile.children.cycles-pp.blk_mq_flush_plug_list 0.40 ± 22% -0.3 0.09 ± 28% perf-profile.children.cycles-pp.blk_mq_sched_insert_requests 0.38 ± 25% -0.3 0.08 ± 30% perf-profile.children.cycles-pp.ttwu_do_activate 0.37 ± 40% -0.3 0.07 ± 20% perf-profile.children.cycles-pp.newidle_balance 0.34 ± 26% -0.3 0.05 ± 60% perf-profile.children.cycles-pp.kmem_cache_free 0.35 ± 6% -0.3 0.07 ± 21% perf-profile.children.cycles-pp.dequeue_entity 0.33 ± 22% -0.2 0.08 ± 13% perf-profile.children.cycles-pp.enqueue_task_fair 0.36 ± 24% -0.2 0.16 ± 9% perf-profile.children.cycles-pp.update_load_avg 0.26 ± 23% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.enqueue_entity 0.29 ± 22% -0.2 0.12 ± 81% perf-profile.children.cycles-pp.btrfs_lookup_csum 0.18 ± 20% -0.2 0.03 ±100% perf-profile.children.cycles-pp.set_next_entity 0.12 ± 22% -0.1 0.03 ±100% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.21 ± 9% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.12 ± 15% -0.1 0.05 ± 67% perf-profile.children.cycles-pp.__wake_up_common_lock 0.11 ± 20% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.__update_load_avg_se 0.15 ± 20% -0.1 0.08 ± 13% perf-profile.children.cycles-pp.__blk_mq_delay_run_hw_queue 0.79 ± 4% -0.1 0.73 ± 4% perf-profile.children.cycles-pp.read_tsc 0.12 ± 23% -0.0 0.08 ± 30% perf-profile.children.cycles-pp.read 0.07 ± 7% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.path_openat 0.09 ± 16% +0.0 0.12 ± 13% perf-profile.children.cycles-pp.rcu_nmi_exit 0.07 ± 7% +0.0 0.10 ± 12% perf-profile.children.cycles-pp.do_filp_open 0.07 ± 19% +0.0 0.11 ± 23% perf-profile.children.cycles-pp.rcu_segcblist_ready_cbs 0.08 ± 14% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.tick_program_event 0.01 ±173% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.run_timer_softirq 0.05 ± 62% +0.1 0.10 ± 18% perf-profile.children.cycles-pp.perf_pmu_disable 0.01 ±173% +0.1 0.07 ± 26% perf-profile.children.cycles-pp.cpuidle_get_cpu_driver 0.03 ±100% +0.1 0.09 ± 23% perf-profile.children.cycles-pp.tick_nohz_idle_retain_tick 0.21 ± 10% +0.1 0.27 ± 9% perf-profile.children.cycles-pp.menu_reflect 0.03 ±102% +0.1 0.09 ± 24% perf-profile.children.cycles-pp.rcu_is_cpu_rrupt_from_idle 0.00 +0.1 0.07 ± 23% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush 0.00 +0.1 0.07 ± 23% perf-profile.children.cycles-pp.btrfs_alloc_tree_block 0.19 ± 12% +0.1 0.26 ± 11% perf-profile.children.cycles-pp.rb_next 0.16 ± 14% +0.1 0.23 ± 10% perf-profile.children.cycles-pp.irqentry_exit 0.07 ± 11% +0.1 0.14 ± 25% perf-profile.children.cycles-pp.rcu_needs_cpu 0.03 ±102% +0.1 0.10 ± 30% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.01 ±173% +0.1 0.09 ± 45% perf-profile.children.cycles-pp.leave_mm 0.14 ± 19% +0.1 0.22 ± 16% perf-profile.children.cycles-pp.pm_qos_read_value 0.00 +0.1 0.08 ± 37% perf-profile.children.cycles-pp.cpu_latency_qos_limit 0.22 ± 18% +0.1 0.30 ± 14% perf-profile.children.cycles-pp.get_cpu_device 0.02 ±173% +0.1 0.10 ± 35% perf-profile.children.cycles-pp.sched_idle_set_state 0.07 ± 69% +0.1 0.15 ± 19% perf-profile.children.cycles-pp.hrtimer_forward 0.14 ± 10% +0.1 0.23 ± 14% perf-profile.children.cycles-pp.perf_event_task_tick 0.10 ± 15% +0.1 0.19 ± 9% perf-profile.children.cycles-pp.x86_pmu_disable 0.08 ± 67% +0.1 0.17 ± 15% perf-profile.children.cycles-pp.cpuidle_reflect 0.00 +0.1 0.09 ± 27% perf-profile.children.cycles-pp.check_leaf 0.07 ± 58% +0.1 0.17 ± 26% perf-profile.children.cycles-pp.run_local_timers 0.00 +0.1 0.10 ± 27% perf-profile.children.cycles-pp.btrfs_cow_block 0.00 +0.1 0.10 ± 27% perf-profile.children.cycles-pp.__btrfs_cow_block 0.00 +0.1 0.10 ± 25% perf-profile.children.cycles-pp.btree_csum_one_bio 0.17 ± 14% +0.1 0.27 ± 33% perf-profile.children.cycles-pp.__hrtimer_get_next_event 0.00 +0.1 0.10 ± 35% perf-profile.children.cycles-pp.write_one_eb 0.00 +0.1 0.10 ± 27% perf-profile.children.cycles-pp.btrfs_submit_metadata_bio 0.43 ± 13% +0.1 0.53 ± 5% perf-profile.children.cycles-pp.update_irq_load_avg 0.10 ± 18% +0.1 0.21 ± 14% perf-profile.children.cycles-pp.cpuidle_not_available 0.00 +0.1 0.11 ± 23% perf-profile.children.cycles-pp.submit_one_bio 0.16 ± 35% +0.1 0.27 ± 25% perf-profile.children.cycles-pp.wait_consider_task 0.16 ± 19% +0.1 0.28 ± 24% perf-profile.children.cycles-pp.tick_check_oneshot_broadcast_this_cpu 0.18 ± 32% +0.1 0.30 ± 19% perf-profile.children.cycles-pp.irqtime_account_process_tick 0.00 +0.1 0.12 ± 23% perf-profile.children.cycles-pp.btrfs_write_marked_extents 0.00 +0.1 0.12 ± 23% perf-profile.children.cycles-pp.btree_write_cache_pages 0.00 +0.1 0.13 ± 17% perf-profile.children.cycles-pp.do_writepages 0.00 +0.1 0.14 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 1.08 ± 11% +0.2 1.23 ± 7% perf-profile.children.cycles-pp.io_serial_in 0.39 ± 10% +0.2 0.55 ± 3% perf-profile.children.cycles-pp.arch_scale_freq_tick 0.00 +0.2 0.17 ± 55% perf-profile.children.cycles-pp.btrfs_log_changed_extents 0.42 ± 5% +0.2 0.63 ± 6% perf-profile.children.cycles-pp.timerqueue_del 0.32 ± 14% +0.2 0.54 ± 11% perf-profile.children.cycles-pp.timerqueue_add 0.72 ± 11% +0.2 0.95 ± 8% perf-profile.children.cycles-pp.update_blocked_averages 0.49 ± 8% +0.2 0.72 ± 13% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.52 ± 6% +0.2 0.76 ± 7% perf-profile.children.cycles-pp.__remove_hrtimer 0.38 ± 16% +0.3 0.64 ± 13% perf-profile.children.cycles-pp.enqueue_hrtimer 0.96 ± 7% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.68 ± 11% +0.3 1.00 ± 11% perf-profile.children.cycles-pp.run_rebalance_domains 0.53 ± 11% +0.3 0.86 ± 13% perf-profile.children.cycles-pp.rcu_sched_clock_irq 1.22 ± 10% +0.3 1.56 ± 6% perf-profile.children.cycles-pp.tick_nohz_next_event 1.03 ± 6% +0.4 1.39 ± 13% perf-profile.children.cycles-pp.tick_irq_enter 1.38 ± 5% +0.4 1.78 ± 14% perf-profile.children.cycles-pp.irq_enter_rcu 1.64 ± 9% +0.4 2.08 ± 3% perf-profile.children.cycles-pp.scheduler_tick 1.81 ± 11% +0.6 2.36 ± 5% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.23 ± 18% +0.6 0.83 ± 36% perf-profile.children.cycles-pp.__btrfs_tree_read_lock 0.00 +0.7 0.72 ± 5% perf-profile.children.cycles-pp.wait_log_commit 0.06 ± 74% +0.8 0.81 ± 37% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.00 +0.8 0.76 ± 3% perf-profile.children.cycles-pp.__mutex_lock 0.03 ±100% +0.8 0.79 ± 14% perf-profile.children.cycles-pp.__btrfs_tree_lock 0.03 ±100% +0.8 0.79 ± 14% perf-profile.children.cycles-pp.btrfs_lock_root_node 0.00 +0.8 0.79 ± 14% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.00 +0.9 0.90 ± 5% perf-profile.children.cycles-pp.btrfs_sync_log 2.83 ± 6% +1.1 3.93 ± 3% perf-profile.children.cycles-pp.update_process_times 3.01 ± 6% +1.2 4.20 ± 4% perf-profile.children.cycles-pp.tick_sched_handle 3.54 ± 7% +1.3 4.85 ± 3% perf-profile.children.cycles-pp.tick_sched_timer 4.44 ± 14% +1.3 5.78 ± 10% perf-profile.children.cycles-pp.menu_select 1.15 ± 15% +1.4 2.54 ± 32% perf-profile.children.cycles-pp.start_kernel 0.08 ± 78% +1.5 1.60 ± 20% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.00 +1.7 1.71 ± 19% perf-profile.children.cycles-pp.btrfs_insert_empty_items 0.00 +1.9 1.87 ± 18% perf-profile.children.cycles-pp.btrfs_log_inode 0.00 +2.0 2.02 ± 17% perf-profile.children.cycles-pp.btrfs_log_inode_parent 0.00 +2.0 2.03 ± 17% perf-profile.children.cycles-pp.btrfs_log_dentry_safe 0.04 ±115% +2.2 2.23 ± 14% perf-profile.children.cycles-pp.osq_lock 5.92 ± 6% +2.3 8.26 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues 8.53 ± 9% +2.5 11.05 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt 8.91 ± 9% +2.7 11.63 ± 5% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.00 +3.0 3.00 ± 11% perf-profile.children.cycles-pp.btrfs_sync_file 13.89 ± 9% +4.2 18.09 ± 6% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 15.08 ± 9% +4.4 19.51 ± 6% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 51.91 +8.8 60.72 ± 2% perf-profile.children.cycles-pp.intel_idle 79.10 ± 3% +10.8 89.91 perf-profile.children.cycles-pp.start_secondary 73.22 ± 2% +11.2 84.38 perf-profile.children.cycles-pp.cpuidle_enter_state 73.32 ± 2% +11.2 84.49 perf-profile.children.cycles-pp.cpuidle_enter 80.25 ± 2% +12.2 92.45 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 80.25 ± 2% +12.2 92.45 perf-profile.children.cycles-pp.cpu_startup_entry 80.23 ± 2% +12.2 92.45 perf-profile.children.cycles-pp.do_idle 0.97 ± 2% -0.5 0.44 ± 19% perf-profile.self.cycles-pp._raw_spin_lock 0.36 ± 16% -0.3 0.05 ± 58% perf-profile.self.cycles-pp.__schedule 0.35 ± 8% -0.3 0.07 ± 13% perf-profile.self.cycles-pp.kmem_cache_alloc 0.12 ± 22% -0.1 0.03 ±100% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.15 ± 25% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.update_load_avg 0.11 ± 20% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__update_load_avg_se 0.18 ± 14% -0.1 0.12 ± 32% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.08 ± 17% +0.0 0.10 ± 8% perf-profile.self.cycles-pp.cpuidle_enter 0.07 ± 15% +0.0 0.10 ± 21% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit 0.07 ± 15% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.rcu_nmi_exit 0.01 ±173% +0.0 0.06 ± 9% perf-profile.self.cycles-pp.arch_cpu_idle_enter 0.04 ± 58% +0.0 0.08 ± 33% perf-profile.self.cycles-pp.perf_pmu_disable 0.10 ± 18% +0.0 0.14 ± 18% perf-profile.self.cycles-pp.irq_enter_rcu 0.12 ± 6% +0.0 0.17 ± 10% perf-profile.self.cycles-pp.menu_reflect 0.08 ± 14% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.tick_program_event 0.03 ±100% +0.0 0.07 ± 27% perf-profile.self.cycles-pp.rcu_is_cpu_rrupt_from_idle 0.15 ± 18% +0.0 0.20 ± 7% perf-profile.self.cycles-pp.asm_call_sysvec_on_stack 0.06 ± 14% +0.1 0.11 ± 23% perf-profile.self.cycles-pp.rcu_segcblist_ready_cbs 0.01 ±173% +0.1 0.07 ± 26% perf-profile.self.cycles-pp.cpuidle_get_cpu_driver 0.20 ± 17% +0.1 0.27 ± 10% perf-profile.self.cycles-pp.get_next_timer_interrupt 0.10 ± 23% +0.1 0.17 ± 25% perf-profile.self.cycles-pp.__hrtimer_get_next_event 0.03 ±102% +0.1 0.10 ± 30% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.01 ±173% +0.1 0.08 ± 37% perf-profile.self.cycles-pp.sched_idle_set_state 0.22 ± 17% +0.1 0.29 ± 12% perf-profile.self.cycles-pp.scheduler_tick 0.17 ± 14% +0.1 0.24 ± 14% perf-profile.self.cycles-pp.rb_next 0.00 +0.1 0.07 ± 24% perf-profile.self.cycles-pp.tick_nohz_idle_retain_tick 0.13 ± 15% +0.1 0.21 ± 7% perf-profile.self.cycles-pp.irq_exit_rcu 0.00 +0.1 0.08 ± 37% perf-profile.self.cycles-pp.cpu_latency_qos_limit 0.22 ± 18% +0.1 0.30 ± 14% perf-profile.self.cycles-pp.get_cpu_device 0.10 ± 30% +0.1 0.19 ± 21% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length 0.14 ± 10% +0.1 0.23 ± 14% perf-profile.self.cycles-pp.perf_event_task_tick 0.11 ± 26% +0.1 0.20 ± 12% perf-profile.self.cycles-pp.clockevents_program_event 0.09 ± 20% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.x86_pmu_disable 0.05 ±102% +0.1 0.14 ± 16% perf-profile.self.cycles-pp.hrtimer_forward 0.10 ± 18% +0.1 0.20 ± 18% perf-profile.self.cycles-pp.cpuidle_not_available 0.05 ± 60% +0.1 0.14 ± 23% perf-profile.self.cycles-pp.run_local_timers 0.24 ± 8% +0.1 0.35 ± 19% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.16 ± 18% +0.1 0.28 ± 24% perf-profile.self.cycles-pp.tick_check_oneshot_broadcast_this_cpu 0.17 ± 26% +0.1 0.30 ± 21% perf-profile.self.cycles-pp.irqtime_account_process_tick 0.40 ± 7% +0.1 0.55 ± 17% perf-profile.self.cycles-pp.tick_nohz_next_event 0.24 ± 10% +0.1 0.39 ± 6% perf-profile.self.cycles-pp.timerqueue_add 0.39 ± 10% +0.2 0.55 ± 3% perf-profile.self.cycles-pp.arch_scale_freq_tick 0.32 ± 12% +0.2 0.49 ± 9% perf-profile.self.cycles-pp.__hrtimer_run_queues 0.28 ± 13% +0.2 0.45 ± 9% perf-profile.self.cycles-pp.update_blocked_averages 0.40 ± 14% +0.2 0.65 ± 19% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.98 ± 10% +0.3 1.23 ± 7% perf-profile.self.cycles-pp.io_serial_in 0.04 ±113% +2.2 2.21 ± 14% perf-profile.self.cycles-pp.osq_lock 51.91 +8.8 60.71 ± 2% perf-profile.self.cycles-pp.intel_idle fio.write_bw_MBps 30 +----------------------------------------------------------------------+ |..+..+..+.+. +..+. : +. +..+..+ +..+..+..+..+..+.+..+..+..| 25 |-+ : : : : | | : : : : | | : : : : | 20 |-+ : : : : | | : : : : | 15 |-+ : : : : | | : : : : | 10 |-+ : : : : | | : : : : | | :: :: | 5 |-+ : : | | : : | 0 +----------------------------------------------------------------------+ fio.write_iops 8000 +--------------------------------------------------------------------+ |..+..+.+..+..+..+.+..+ +.+..+..+..+ +..+..+.+..+..+..+.+.. .| 7000 |-+ : : : : +. | 6000 |-+ : : : : | | : : : : | 5000 |-+ : : : : | | : : : : | 4000 |-+ : : : : | | : : : : | 3000 |-+ : : : : | 2000 |-+ : : : : | | : : :: | 1000 |-+ : : | | : : | 0 +--------------------------------------------------------------------+ fio.write_clat_mean_us 4.5e+08 +-----------------------------------------------------------------+ | O | 4e+08 |-+O O O O O O O O O O O O O O O O O O | 3.5e+08 |-+ | | | 3e+08 |-+ | 2.5e+08 |-+ | | | 2e+08 |-+ | 1.5e+08 |-+ | | | 1e+08 |-+ | 5e+07 |-+ | | | 0 +-----------------------------------------------------------------+ fio.write_clat_stddev 4.5e+07 +-----------------------------------------------------------------+ | O O O | 4e+07 |-+ O O | 3.5e+07 |-+ O O O O O O | | O | 3e+07 |-+O O O O O O O | 2.5e+07 |-+ | | | 2e+07 |-+ | 1.5e+07 |..+.+..+..+.+..+.+..+ +..+..+.+..+ +..+..+.+..+.+..+..+.+..| | : : : : | 1e+07 |-+ : : : : | 5e+06 |-+ : : : : | | :: :: | 0 +-----------------------------------------------------------------+ fio.write_clat_90__us 4.5e+08 +-----------------------------------------------------------------+ | O O O O O O O O O O O O O | 4e+08 |-+ | 3.5e+08 |-+ | | | 3e+08 |-+ | 2.5e+08 |-+ | | | 2e+08 |-+ | 1.5e+08 |-+ | | | 1e+08 |-+ | 5e+07 |-+ | | | 0 +-----------------------------------------------------------------+ fio.write_clat_95__us 5e+08 +-----------------------------------------------------------------+ 4.5e+08 |-+ O O | | O O O O O O O O O O O O O O O O O | 4e+08 |-+ | 3.5e+08 |-+ | | | 3e+08 |-+ | 2.5e+08 |-+ | 2e+08 |-+ | | | 1.5e+08 |-+ | 1e+08 |-+ | | | 5e+07 |..+.+..+..+.+..+.+..+.. .+..+..+.+..+.. .+..+..+.+..+.+..+..+.+..| 0 +-----------------------------------------------------------------+ fio.write_clat_99__us 5e+08 +-----------------------------------------------------------------+ 4.5e+08 |-+O O O O O O O O O O O O O O O O O | | O O | 4e+08 |-+ | 3.5e+08 |-+ | | | 3e+08 |-+ | 2.5e+08 |-+ | 2e+08 |-+ | | | 1.5e+08 |-+ | 1e+08 |-+ | | .+..| 5e+07 |..+.+..+..+.+..+.+..+.. .+..+..+.+..+.. .+..+..+.+..+.+..+..+ | 0 +-----------------------------------------------------------------+ fio.latency_500ms_ 100 +---------------------------------------------------------------------+ 90 |-+ | | | 80 |-+ | 70 |-+ | | | 60 |-+ | 50 |-+ | 40 |-+ | | | 30 |-+ | 20 |-+ | | | 10 |-+ | 0 +---------------------------------------------------------------------+ fio.workload 2.5e+06 +-----------------------------------------------------------------+ | .+.. | |..+.+..+..+.+..+.+..+ +. +.+..+ +..+..+.+..+.+..+..+.+..| 2e+06 |-+ : : : : | | : : : : | | : : : : | 1.5e+06 |-+ : : : : | | : : : : | 1e+06 |-+ : : : : | | : : : : | | : : : : | 500000 |-+ : : : : | | : : | | : : | 0 +-----------------------------------------------------------------+ fio.time.user_time 16 +----------------------------------------------------------------------+ | : : | 14 |-+ : : | 12 |-+ +.. +.. : : | | +. .. +.. .+ .. + : +.. .+.. .+.. .+..+.. | 10 |.. .. + +. : + + : +. +. + +..| | +..+ : : : : | 8 |-+ : : : : | | : : : : | 6 |-+ : : : : | 4 |-+ : : : : | | : : : : | 2 |-+O O O O :: O O O :: O | | O O O : O O : O O O O | 0 +----------------------------------------------------------------------+ fio.time.system_time 70 +----------------------------------------------------------------------+ | .+.. +..+.. + .+..+.. | 60 |.+ +..+.+..+..+..+..+ : +.. ..: +..+..+..+..+..+ +..| | : : + : : | 50 |-+ : : : : | | : : : : | 40 |-+ : : : : | | : : : : | 30 |-+ : : : : | | O O : : O O :O:O O O O O | 20 |-+O O O O : : O : : | | O :: O O : : | 10 |-+ : : | | : : | 0 +----------------------------------------------------------------------+ fio.time.percent_of_cpu_this_job_got 30 +----------------------------------------------------------------------+ | | 25 |-+ .+.. .+ | |..+..+..+.+..+..+..+..+ +. +..+. : +..+..+..+..+..+.+..+..+..| | : : : : | 20 |-+ : : : : | | : : : : | 15 |-+ : : : : | | : : : : | 10 |-+ : : : : | | O O : : O O :O:O O O O O | | O O O O O :O:O O O : : | 5 |-+ :: : | | : : | 0 +----------------------------------------------------------------------+ fio.time.voluntary_context_switches 2.5e+06 +-----------------------------------------------------------------+ |..+.+..+.. .+..+. .+ .+.. .+ +..+..+.+..+.+.. | | + +. : +. +.+. : : +..+.+..| 2e+06 |-+ : : : : | | : : : : | | : : : : | 1.5e+06 |-+ : : : : | | : : : : | 1e+06 |-+ : : : : | | : : : : | | : : : : | 500000 |-+ : : :: | | : : | | O O O O O O O : O O O O O : O O O O O | 0 +-----------------------------------------------------------------+ fio.time.involuntary_context_switches 3000 +--------------------------------------------------------------------+ | | 2500 |-+ O O O O | | O O O O | | O O O | 2000 |-+ O O O O | | O O O O | 1500 |-+ | | | 1000 |-+ | | | | +.. +.. | 500 |-.+..+.+.. +. .. + .+..| |. +..+..+.+..+.. .. +..+..+..+. .+ + +..+..+.+. | 0 +--------------------------------------------------------------------+ fio.time.file_system_outputs 1.8e+07 +-----------------------------------------------------------------+ | + +. : : +.+. : : +..| 1.6e+07 |-+ : : : : | 1.4e+07 |-+ : : : : | | : : : : | 1.2e+07 |-+ : : : : | 1e+07 |-+ : : : : | | : : : : | 8e+06 |-+ : : : : | 6e+06 |-+ : : : : | | : : : : | 4e+06 |-+ :: :: | 2e+06 |-+ : : | | O O O O O O O : O O O O O : O O O O O | 0 +-----------------------------------------------------------------+ perf-sched.total_wait_time.average.ms 160 +---------------------------------------------------------------------+ | O O | 140 |-+ O O O O O O | 120 |-+ O O O O O | | O O O O O | 100 |-+ O | | | 80 |-+ | | | 60 |-+ | 40 |-+ | | | 20 |-+ | |..+..+.+..+..+..+..+.+.. .+..+..+.+..+.. .+..+.+..+..+..+..+.+..+..| 0 +---------------------------------------------------------------------+ perf-sched.total_wait_and_delay.count.ms 250000 +------------------------------------------------------------------+ | | | .+.. .+..+.. .+ +.+.. +..+ .+.. | 200000 |..+.+. + + : : .. : .+. .+..+ +.. | | : : + : +. +. + :| | : : : : : :| 150000 |-+ : : : : : : | | : : : : + | 100000 |-+ : : : : | | : : : : | | : : : : | 50000 |-+ : : :: | | :: :: | | O O O O O O O : O O O O O : O O O O O | 0 +------------------------------------------------------------------+ perf-sched.total_wait_and_delay.average.ms 160 +---------------------------------------------------------------------+ | O O | 140 |-+ O O O O O O | 120 |-+ O O O O O | | O O O O O | 100 |-+ O | | | 80 |-+ | | | 60 |-+ | 40 |-+ | | | 20 |-+ | |..+..+.+..+..+..+..+.+.. .+..+..+.+..+.. .+..+.+..+..+..+..+.+..+..| 0 +---------------------------------------------------------------------+ perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 300 +---------------------------------------------------------------------+ | O O O O O O | 250 |-+O O O O O O O O O O O O O | | | | | 200 |-+ | | | 150 |-+ | | | 100 |-+ | | | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 140000 +------------------------------------------------------------------+ | +.+.. | 120000 |-.+.+..+..+.+..+.. .+ : .+..+ .+. .+..+.+.. | |. + : : +. : +. +. +.. :| 100000 |-+ : : : : + :| | : : : : : : | 80000 |-+ : : : : : : | | : : : : + | 60000 |-+ : : : : | | : : : : | 40000 |-+ : : : : | | : : :: | 20000 |-+ : :: | | : : | 0 +------------------------------------------------------------------+ perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 300 +---------------------------------------------------------------------+ | O O O O O O | 250 |-+O O O O O O O O O O O O O | | | | | 200 |-+ | | | 150 |-+ | | | 100 |-+ | | | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ fio.latency_2ms_ 0.01 +-------------------------------------------------------------------+ | : : : : | | : : : : | 0.008 |-+ : : : : | | : : : : | | : : : : | 0.006 |-+ : : : : | | : : : : | 0.004 |-+ : : : : | | : : : : | | : : : : | 0.002 |-+ :: : : | | : : | | : : | 0 +-------------------------------------------------------------------+ fio.latency_4ms_ 60 +----------------------------------------------------------------------+ |..+..+..+.+..+..+..+..+ +..+..+..+..+ +..+..+..+..+..+.+..+..+..| 50 |-+ : : : : | | : : : : | | : : : : | 40 |-+ : : : : | | : : : : | 30 |-+ : : : : | | : : : : | 20 |-+ : : : : | | : : : : | | :: :: | 10 |-+ : : | | : : | 0 +----------------------------------------------------------------------+ fio.latency_10ms_ 35 +----------------------------------------------------------------------+ | .+..+..+.+..+..+..+..+ +..+.. .+ .+.+..+.. .| 30 |.+ : : +..+. : +..+..+..+..+. +. | | : : : : | 25 |-+ : : : : | | : : : : | 20 |-+ : : : : | | : : : : | 15 |-+ : : : : | | : : : : | 10 |-+ : : : : | | :: : : | 5 |-+ : : | | : : | 0 +----------------------------------------------------------------------+ fio.latency_20ms_ 9 +-----------------------------------------------------------------------+ |. .+.. .+.. .+..+ +.. .+.+..+ +. +. +.+..+..+.. .| 8 |-+ +. + +. : : +. : : +. | 7 |-+ : : : : | | : : : : | 6 |-+ : : : : | 5 |-+ : : : : | | : : : : | 4 |-+ : : : : | 3 |-+ : : : : | | : : : : | 2 |-+ : : : : | 1 |-+ : : | | : : | 0 +-----------------------------------------------------------------------+ fio.latency_50ms_ 6 +-----------------------------------------------------------------------+ | +.. .+.. | 5 |..+..+..+..+.+..+..+.. +..+.. .+..+ : +. +..+.+..+.. .+..| | + : + : : +. | | : : : : | 4 |-+ : : : : | | : : : : | 3 |-+ : : : : | | : : : : | 2 |-+ : : : : | | : : : : | | : : : : | 1 |-+ : : : | | : : | 0 +-----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

9 months

1
0
0 / 0

[locking/rwsem] 617f3ef951: unixbench.score -21.2% regression

by kernel test robot

Greeting, FYI, we noticed a -21.2% regression of unixbench.score due to commit: commit: 617f3ef95177840c77f59c2aec1029d27d5547d6 ("locking/rwsem: Remove reader optimistic spinning") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: unixbench on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory with following parameters: runtime: 300s nr_task: 30% test: shell8 cpufreq_governor: performance ucode: 0xde test-description: UnixBench is the original BYTE UNIX benchmark suite aims to test performance of Unix-like system. test-url: https://github.com/kdlucas/byte-unixbench If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/30%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/shell8/unixbench/0xde commit: 1a728dff85 ("locking/rwsem: Enable reader optimistic lock stealing") 617f3ef951 ("locking/rwsem: Remove reader optimistic spinning") 1a728dff855a318b 617f3ef95177840c77f59c2aec1 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 39:4 -992% :4 perf-profile.calltrace.cycles-pp.error_entry 25:4 -635% :4 perf-profile.children.cycles-pp.error_entry %stddev %change %stddev \ | \ 21807 ± 3% -21.2% 17186 unixbench.score 1287072 ± 3% -38.7% 788414 unixbench.time.involuntary_context_switches 37161 ± 4% +31.3% 48798 unixbench.time.major_page_faults 1.047e+08 ± 3% -21.1% 82610985 unixbench.time.minor_page_faults 1341 -27.1% 978.00 unixbench.time.percent_of_cpu_this_job_got 370.87 -33.3% 247.55 unixbench.time.system_time 490.05 -23.3% 376.03 unixbench.time.user_time 3083520 ± 3% +59.7% 4924900 unixbench.time.voluntary_context_switches 824314 ± 3% -21.2% 649654 unixbench.workload 0.03 ± 27% -51.9% 0.02 ± 59% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 385.15 ± 2% +62.5% 625.72 uptime.idle 17.03 -1.8% 16.73 boot-time.boot 11.01 -1.6% 10.83 boot-time.dhcp 214.12 ± 3% -3.1% 207.49 boot-time.idle 13.72 ± 4% +23.5 37.24 mpstat.cpu.all.idle% 1.06 -0.1 0.94 mpstat.cpu.all.irq% 49.32 ± 2% -11.8 37.53 mpstat.cpu.all.sys% 35.24 ± 2% -11.6 23.68 mpstat.cpu.all.usr% 15.50 ± 3% +145.2% 38.00 vmstat.cpu.id 49.00 ± 2% -22.4% 38.00 vmstat.cpu.sy 33.75 ± 2% -33.3% 22.50 ± 2% vmstat.cpu.us 21.75 ± 3% -33.3% 14.50 ± 3% vmstat.procs.r 97370 ± 3% +56.4% 152258 vmstat.system.cs 37589 -2.1% 36804 vmstat.system.in 11861 ± 9% -18.0% 9730 slabinfo.filp.active_objs 13242 ± 8% -15.5% 11184 slabinfo.filp.num_objs 14731 ± 7% -9.5% 13325 ± 5% slabinfo.kmalloc-8.active_objs 14731 ± 7% -9.5% 13325 ± 5% slabinfo.kmalloc-8.num_objs 5545 ± 2% -13.8% 4780 ± 4% slabinfo.pid.active_objs 5563 ± 2% -13.8% 4793 ± 4% slabinfo.pid.num_objs 5822 ± 14% -40.4% 3468 ± 5% slabinfo.task_delay_info.active_objs 5825 ± 14% -40.5% 3468 ± 5% slabinfo.task_delay_info.num_objs 32104492 ± 3% +303.3% 1.295e+08 ± 11% cpuidle.C1.time 882330 ± 5% +131.5% 2042656 ± 10% cpuidle.C1.usage 21965263 ± 3% +340.5% 96762398 ± 14% cpuidle.C1E.time 442911 ± 2% +211.3% 1378866 ± 14% cpuidle.C1E.usage 6511399 ± 4% +606.6% 46010023 ± 13% cpuidle.C3.time 66498 ± 5% +559.3% 438404 ± 9% cpuidle.C3.usage 101453 ± 18% +139.9% 243422 ± 7% cpuidle.C6.usage 11612925 ± 66% +228.1% 38105483 ± 21% cpuidle.C8.time 12182 ± 65% +220.7% 39070 ± 21% cpuidle.C8.usage 173670 ± 30% +166.5% 462831 ± 7% cpuidle.POLL.time 17441 ± 18% +195.6% 51559 ± 6% cpuidle.POLL.usage 59652 +0.6% 60011 proc-vmstat.nr_inactive_anon 5840 +1.4% 5922 proc-vmstat.nr_kernel_stack 1404 +3.1% 1447 proc-vmstat.nr_page_table_pages 13255 -1.7% 13026 proc-vmstat.nr_slab_reclaimable 14861 -6.9% 13835 ± 4% proc-vmstat.nr_slab_unreclaimable 59652 +0.6% 60011 proc-vmstat.nr_zone_inactive_anon 70705198 ± 3% -21.1% 55762717 proc-vmstat.numa_hit 70705198 ± 3% -21.1% 55762717 proc-vmstat.numa_local 116793 ± 3% -32.1% 79354 proc-vmstat.pgactivate 74803470 ± 3% -20.7% 59317396 proc-vmstat.pgalloc_normal 1.049e+08 ± 3% -21.1% 82849276 proc-vmstat.pgfault 74776100 ± 3% -20.7% 59288922 proc-vmstat.pgfree 5338118 ± 3% -24.5% 4030373 proc-vmstat.pgreuse 3413 ± 2% -19.6% 2745 proc-vmstat.thp_fault_alloc 1466090 ± 3% -21.2% 1155570 proc-vmstat.unevictable_pgs_culled 382612 ± 60% -66.4% 128559 ± 7% sched_debug.cfs_rq:/.load.max 274.29 ± 53% -48.2% 142.09 ± 15% sched_debug.cfs_rq:/.load_avg.avg 452211 -25.5% 336912 sched_debug.cfs_rq:/.min_vruntime.avg 489002 -22.6% 378258 ± 2% sched_debug.cfs_rq:/.min_vruntime.max 440439 -26.4% 324253 sched_debug.cfs_rq:/.min_vruntime.min 210358 ± 8% -21.6% 164917 ± 9% sched_debug.cpu.avg_idle.avg 666992 ± 9% -14.4% 571102 ± 3% sched_debug.cpu.avg_idle.max 183734 ± 2% -10.3% 164859 sched_debug.cpu.avg_idle.stddev 0.54 ± 19% -29.1% 0.38 ± 15% sched_debug.cpu.clock.stddev 199240 ± 3% +53.0% 304861 sched_debug.cpu.nr_switches.avg 215766 ± 3% +48.7% 320802 sched_debug.cpu.nr_switches.max 186135 ± 3% +55.3% 289040 sched_debug.cpu.nr_switches.min 82.12 ± 12% +247.2% 285.12 ± 11% sched_debug.cpu.nr_uninterruptible.max -122.88 +312.6% -507.00 sched_debug.cpu.nr_uninterruptible.min 51.87 ± 13% +263.6% 188.58 ± 8% sched_debug.cpu.nr_uninterruptible.stddev 242600 ± 5% -33.6% 161055 softirqs.CPU0.RCU 15144 ± 2% +14.4% 17328 ± 3% softirqs.CPU0.SCHED 14371 +13.8% 16354 ± 3% softirqs.CPU1.SCHED 14064 ± 2% +10.1% 15480 softirqs.CPU10.SCHED 241990 ± 4% -33.1% 161882 softirqs.CPU11.RCU 13889 +12.7% 15650 softirqs.CPU11.SCHED 239184 ± 7% -33.4% 159310 softirqs.CPU12.RCU 13988 ± 3% +12.3% 15711 softirqs.CPU13.SCHED 13744 ± 2% +14.1% 15688 ± 2% softirqs.CPU14.SCHED 14104 +12.8% 15912 ± 2% softirqs.CPU2.SCHED 240729 ± 4% -32.5% 162396 softirqs.CPU3.RCU 14143 ± 2% +13.6% 16064 ± 3% softirqs.CPU3.SCHED 240581 ± 5% -31.9% 163890 softirqs.CPU4.RCU 13936 +10.5% 15402 softirqs.CPU5.SCHED 13761 ± 3% +12.9% 15532 softirqs.CPU6.SCHED 14059 +11.8% 15724 ± 2% softirqs.CPU7.SCHED 241270 ± 5% -34.5% 157931 ± 2% softirqs.CPU8.RCU 13695 +11.8% 15314 ± 2% softirqs.CPU8.SCHED 225021 +12.1% 252248 softirqs.SCHED 14.09 ± 73% -14.1 0.00 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.09 ± 73% -14.1 0.00 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 10.52 ± 60% -10.5 0.00 perf-profile.calltrace.cycles-pp.setlocale 9.13 ±104% -9.1 0.00 perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit 8.43 ± 62% -8.4 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale 8.43 ± 62% -8.4 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale 8.43 ± 62% -8.4 0.00 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.setlocale 8.43 ± 62% -8.4 0.00 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 8.43 ± 62% -8.4 0.00 perf-profile.calltrace.cycles-pp.__open64_nocancel.setlocale 6.35 ±101% -6.3 0.00 perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit 4.86 ±102% -4.9 0.00 perf-profile.calltrace.cycles-pp.terminate_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 6.35 ±101% -3.6 2.78 ±173% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group 6.35 ±101% -3.6 2.78 ±173% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.get_signal 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.get_signal.arch_do_signal 7.98 ± 60% -3.0 5.00 ±173% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 6.35 ±101% -0.8 5.55 ±173% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.35 ±101% -0.8 5.55 ±173% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.35 ±101% -0.8 5.55 ±173% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.09 ± 73% -14.1 0.00 perf-profile.children.cycles-pp.do_filp_open 14.09 ± 73% -14.1 0.00 perf-profile.children.cycles-pp.do_sys_open 14.09 ± 73% -14.1 0.00 perf-profile.children.cycles-pp.do_sys_openat2 14.09 ± 73% -14.1 0.00 perf-profile.children.cycles-pp.path_openat 10.52 ± 60% -10.5 0.00 perf-profile.children.cycles-pp.setlocale 9.13 ±104% -9.1 0.00 perf-profile.children.cycles-pp.free_pgtables 8.43 ± 62% -8.4 0.00 perf-profile.children.cycles-pp.__open64_nocancel 6.35 ±101% -6.3 0.00 perf-profile.children.cycles-pp.sync_regs 6.35 ±101% -6.3 0.00 perf-profile.children.cycles-pp.unlink_file_vma 4.86 ±102% -4.9 0.00 perf-profile.children.cycles-pp.terminate_walk 7.98 ± 60% -3.0 5.00 ±173% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 7.98 ± 60% -3.0 5.00 ±173% perf-profile.children.cycles-pp.arch_do_signal 7.98 ± 60% -3.0 5.00 ±173% perf-profile.children.cycles-pp.get_signal 7.98 ± 60% -3.0 5.00 ±173% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 6.35 ±101% -0.8 5.55 ±173% perf-profile.children.cycles-pp.__x64_sys_exit_group 5.65 ±106% -0.7 5.00 ±173% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 5.65 ±106% -0.1 5.55 ±173% perf-profile.children.cycles-pp.handle_mm_fault 6.35 ±101% -6.3 0.00 perf-profile.self.cycles-pp.sync_regs 31246 ± 5% +200.7% 93965 interrupts.CAL:Function_call_interrupts 2091 ± 11% +179.5% 5844 interrupts.CPU0.CAL:Function_call_interrupts 26624 ± 2% -20.9% 21057 interrupts.CPU0.RES:Rescheduling_interrupts 500.25 ± 7% +378.2% 2392 interrupts.CPU0.TLB:TLB_shootdowns 1931 ± 4% +201.7% 5827 interrupts.CPU1.CAL:Function_call_interrupts 26939 ± 4% -22.2% 20957 interrupts.CPU1.RES:Rescheduling_interrupts 499.25 ± 5% +374.8% 2370 ± 4% interrupts.CPU1.TLB:TLB_shootdowns 1967 ± 6% +195.8% 5820 interrupts.CPU10.CAL:Function_call_interrupts 26951 ± 2% -19.9% 21594 interrupts.CPU10.RES:Rescheduling_interrupts 546.25 +336.4% 2384 ± 4% interrupts.CPU10.TLB:TLB_shootdowns 1927 ± 3% +211.0% 5994 interrupts.CPU11.CAL:Function_call_interrupts 26619 ± 2% -18.7% 21630 interrupts.CPU11.RES:Rescheduling_interrupts 514.25 ± 6% +361.8% 2374 ± 2% interrupts.CPU11.TLB:TLB_shootdowns 1887 ± 9% +206.9% 5791 ± 4% interrupts.CPU12.CAL:Function_call_interrupts 26377 ± 4% -19.5% 21237 ± 2% interrupts.CPU12.RES:Rescheduling_interrupts 496.00 ± 3% +372.8% 2345 ± 2% interrupts.CPU12.TLB:TLB_shootdowns 1912 ± 7% +208.6% 5899 ± 2% interrupts.CPU13.CAL:Function_call_interrupts 27187 ± 4% -20.1% 21734 ± 2% interrupts.CPU13.RES:Rescheduling_interrupts 500.50 ± 5% +380.5% 2405 ± 3% interrupts.CPU13.TLB:TLB_shootdowns 1891 ± 7% +215.1% 5960 ± 2% interrupts.CPU14.CAL:Function_call_interrupts 26046 ± 4% -17.0% 21627 interrupts.CPU14.RES:Rescheduling_interrupts 506.25 ± 9% +367.9% 2368 ± 2% interrupts.CPU14.TLB:TLB_shootdowns 1960 ± 4% +195.8% 5798 interrupts.CPU15.CAL:Function_call_interrupts 26567 ± 4% -19.7% 21342 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 536.50 ± 3% +350.6% 2417 interrupts.CPU15.TLB:TLB_shootdowns 2001 ± 4% +190.3% 5811 ± 2% interrupts.CPU2.CAL:Function_call_interrupts 27172 ± 3% -21.5% 21338 interrupts.CPU2.RES:Rescheduling_interrupts 539.00 ± 4% +332.7% 2332 ± 3% interrupts.CPU2.TLB:TLB_shootdowns 1950 ± 6% +198.1% 5814 interrupts.CPU3.CAL:Function_call_interrupts 26579 ± 2% -19.2% 21479 interrupts.CPU3.RES:Rescheduling_interrupts 504.75 ± 8% +369.0% 2367 ± 4% interrupts.CPU3.TLB:TLB_shootdowns 1965 ± 10% +199.8% 5891 interrupts.CPU4.CAL:Function_call_interrupts 26811 ± 3% -19.8% 21512 interrupts.CPU4.RES:Rescheduling_interrupts 515.75 ± 5% +358.1% 2362 ± 2% interrupts.CPU4.TLB:TLB_shootdowns 1935 ± 7% +206.3% 5926 interrupts.CPU5.CAL:Function_call_interrupts 26997 ± 3% -20.4% 21486 interrupts.CPU5.RES:Rescheduling_interrupts 531.25 ± 3% +348.6% 2383 interrupts.CPU5.TLB:TLB_shootdowns 1904 ± 5% +214.0% 5979 interrupts.CPU6.CAL:Function_call_interrupts 26702 ± 4% -20.1% 21341 interrupts.CPU6.RES:Rescheduling_interrupts 505.25 ± 3% +354.6% 2296 ± 2% interrupts.CPU6.TLB:TLB_shootdowns 1933 ± 4% +205.3% 5901 interrupts.CPU7.CAL:Function_call_interrupts 26505 ± 3% -19.6% 21302 interrupts.CPU7.RES:Rescheduling_interrupts 494.00 ± 4% +371.1% 2327 ± 3% interrupts.CPU7.TLB:TLB_shootdowns 2003 ± 5% +190.5% 5818 interrupts.CPU8.CAL:Function_call_interrupts 26799 ± 3% -20.2% 21376 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts 519.00 ± 4% +364.7% 2411 ± 4% interrupts.CPU8.TLB:TLB_shootdowns 1986 ± 10% +196.5% 5888 ± 2% interrupts.CPU9.CAL:Function_call_interrupts 26994 ± 3% -20.6% 21439 interrupts.CPU9.RES:Rescheduling_interrupts 526.00 ± 6% +351.8% 2376 interrupts.CPU9.TLB:TLB_shootdowns 427874 ± 3% -20.0% 342458 interrupts.RES:Rescheduling_interrupts 8234 ± 2% +360.4% 37914 ± 2% interrupts.TLB:TLB_shootdowns 47.20 +6.4% 50.22 perf-stat.i.MPKI 9.902e+09 ± 3% -19.6% 7.965e+09 perf-stat.i.branch-instructions 2.24 +0.1 2.32 perf-stat.i.branch-miss-rate% 2.143e+08 ± 3% -16.4% 1.791e+08 perf-stat.i.branch-misses 99172014 ± 2% -15.6% 83714922 perf-stat.i.cache-misses 2.363e+09 ± 3% -14.4% 2.022e+09 perf-stat.i.cache-references 101957 ± 3% +56.1% 159180 perf-stat.i.context-switches 1.20 ± 3% -5.8% 1.13 perf-stat.i.cpi 5.571e+10 -23.7% 4.249e+10 perf-stat.i.cpu-cycles 15111 ± 3% +116.2% 32667 perf-stat.i.cpu-migrations 610.84 ± 2% -10.2% 548.56 perf-stat.i.cycles-between-cache-misses 1.234e+10 ± 3% -19.7% 9.908e+09 perf-stat.i.dTLB-loads 0.05 +0.0 0.05 perf-stat.i.dTLB-store-miss-rate% 3568011 ± 3% -14.8% 3038208 perf-stat.i.dTLB-store-misses 7.127e+09 ± 3% -19.9% 5.706e+09 perf-stat.i.dTLB-stores 57.81 -2.0 55.78 perf-stat.i.iTLB-load-miss-rate% 10339942 ± 3% -16.8% 8603048 perf-stat.i.iTLB-load-misses 7643641 ± 3% -7.7% 7057126 perf-stat.i.iTLB-loads 4.833e+10 ± 3% -19.6% 3.887e+10 perf-stat.i.instructions 5183 -1.9% 5085 perf-stat.i.instructions-per-iTLB-miss 0.85 ± 4% +5.6% 0.90 perf-stat.i.ipc 577.76 ± 4% +32.0% 762.53 perf-stat.i.major-faults 3.48 -23.7% 2.66 perf-stat.i.metric.GHz 0.55 ± 44% -77.5% 0.12 ± 30% perf-stat.i.metric.K/sec 1986 ± 3% -19.3% 1602 perf-stat.i.metric.M/sec 1599403 ± 3% -20.7% 1268462 perf-stat.i.minor-faults 5321084 ± 3% -15.9% 4474662 perf-stat.i.node-loads 32016465 ± 3% -17.1% 26551337 perf-stat.i.node-stores 1599980 ± 3% -20.7% 1269225 perf-stat.i.page-faults 48.88 +6.4% 52.03 perf-stat.overall.MPKI 2.16 +0.1 2.25 perf-stat.overall.branch-miss-rate% 4.20 -0.1 4.14 perf-stat.overall.cache-miss-rate% 1.15 ± 4% -5.3% 1.09 perf-stat.overall.cpi 562.37 ± 3% -9.7% 507.65 perf-stat.overall.cycles-between-cache-misses 0.05 +0.0 0.05 perf-stat.overall.dTLB-store-miss-rate% 57.50 -2.6 54.94 perf-stat.overall.iTLB-load-miss-rate% 4674 -3.3% 4518 perf-stat.overall.instructions-per-iTLB-miss 0.87 ± 4% +5.4% 0.91 perf-stat.overall.ipc 9.742e+09 ± 3% -19.6% 7.836e+09 perf-stat.ps.branch-instructions 2.108e+08 ± 3% -16.5% 1.762e+08 perf-stat.ps.branch-misses 97569500 ± 2% -15.6% 82348192 perf-stat.ps.cache-misses 2.324e+09 ± 3% -14.4% 1.989e+09 perf-stat.ps.cache-references 100305 ± 3% +56.1% 156566 perf-stat.ps.context-switches 5.481e+10 -23.7% 4.18e+10 perf-stat.ps.cpu-cycles 14866 ± 3% +116.1% 32130 perf-stat.ps.cpu-migrations 1.214e+10 ± 3% -19.7% 9.746e+09 perf-stat.ps.dTLB-loads 3510146 ± 3% -14.9% 2988346 perf-stat.ps.dTLB-store-misses 7.012e+09 ± 3% -20.0% 5.612e+09 perf-stat.ps.dTLB-stores 10172369 ± 3% -16.8% 8461956 perf-stat.ps.iTLB-load-misses 7519764 ± 3% -7.7% 6941264 perf-stat.ps.iTLB-loads 4.755e+10 ± 3% -19.6% 3.823e+10 perf-stat.ps.instructions 568.39 ± 4% +32.0% 750.01 perf-stat.ps.major-faults 1573444 ± 3% -20.7% 1247622 perf-stat.ps.minor-faults 5235200 ± 3% -15.9% 4401366 perf-stat.ps.node-loads 31497428 ± 3% -17.1% 26115335 perf-stat.ps.node-stores 1574012 ± 3% -20.7% 1248372 perf-stat.ps.page-faults 3.111e+12 ± 3% -20.5% 2.473e+12 perf-stat.total.instructions unixbench.time.user_time 500 +---------------------------------------------------------------------+ |. +..+..+ +..+..+.+..+..+.+..+..+.+..+..+.+..+..+..+.+..+..+.+..| 480 |-+ | | | 460 |-+ | | | 440 |-+ | | | 420 |-+ | | | 400 |-+ | | | 380 |-+ O O O O | | O O O O O O O O O O O O O O | 360 +---------------------------------------------------------------------+ unixbench.time.system_time 380 +---------------------------------------------------------------------+ |..+.+..+..+.+..+..+..+.+..+. +. +.+..+..+.+..+..+..+.+. +.+..| 360 |-+ | | | 340 |-+ | | | 320 |-+ | | | 300 |-+ | | | 280 |-+ | | | 260 |-+ O | | O O O O O O O O O O O O O O O O O | 240 +---------------------------------------------------------------------+ unixbench.time.percent_of_cpu_this_job_got 1400 +--------------------------------------------------------------------+ |..+.+..+..+.+.. .+.. .+.+..+..+.+..+.+..+..+.+..+..+. | 1350 |-+ +..+ +. +..+..+.+..| 1300 |-+ | | | 1250 |-+ | 1200 |-+ | | | 1150 |-+ | 1100 |-+ | | | 1050 |-+ | 1000 |-+ | | O O O O O O O O O O O O O O O O O O | 950 +--------------------------------------------------------------------+ unixbench.time.minor_page_faults 1.1e+08 +----------------------------------------------------------------+ |..+.+..+.+..+.+..+.+..+.+ + +..+.+..+.+..+.+ + +..+.| 1.05e+08 |-+ : :: : : :: : | | : : : : : : : : | 1e+08 |-+ : : : : : : : : | | + + + + | 9.5e+07 |-+ | | | 9e+07 |-+ | | | 8.5e+07 |-+ | | O O O O O O O O O O O O O O O | 8e+07 |-+ | | | 7.5e+07 +----------------------------------------------------------------+ unixbench.time.voluntary_context_switches 5e+06 +-----------------------------------------------------------------+ | O O O O O O O O O O O | | O O | 4.5e+06 |-+ O | | | | | 4e+06 |-+ | | | 3.5e+06 |-+ | | | | .+. .+.+..+.+..+. +.. +.+..+.+..+.+.. + +.+..| 3e+06 |.+ +..+.+. +.. + .. +. .. + .. | | + + + + | | | 2.5e+06 +-----------------------------------------------------------------+ unixbench.time.involuntary_context_switches 1.4e+06 +-----------------------------------------------------------------+ |..+.+..+.+..+.+.. +.+.. +.. +.+..+.+..+.+..+ + + | 1.3e+06 |-+ .. + .. + .. + .. + ..| | +.+ + + + + + | 1.2e+06 |-+ | | | 1.1e+06 |-+ | | | 1e+06 |-+ | | | 900000 |-+ | | | 800000 |-+O O O O O O O O O O O O O O | | O O O O | 700000 +-----------------------------------------------------------------+ unixbench.score 23000 +-------------------------------------------------------------------+ |..+.+..+.+..+..+.+..+..+.+ + +.+..+.+..+..+.+ + +.+..| 22000 |-+ + : : : : : : : | 21000 |-+ + : : : : : : : | | + : : : : :: | 20000 |-+ + + + | | | 19000 |-+ | | | 18000 |-+ | 17000 |-+O O O O O O O O O O O O | | O O O | 16000 |-+ | | O O O | 15000 +-------------------------------------------------------------------+ unixbench.workload 900000 +------------------------------------------------------------------+ | | 850000 |..+.+..+.+..+.+..+..+.+..+ + +..+.+..+.+..+..+ + +.+..| | : + + : : + : + | 800000 |-+ : + + : : + : + | | + + + + | 750000 |-+ | | | 700000 |-+ | | | 650000 |-+O O O O O O O O O O O O O O O | | | 600000 |-+ O O O | | | 550000 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

9 months

2
1
0 / 0

[KVM] fdd90b978b: WARNING:suspicious_RCU_usage

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: fdd90b978b4e0dbbdf0f50f6dd01569a569ad614 ("[PATCH v2 15/16] KVM: x86: declare Xen HVM shared info capability and add test case") url: https://github.com/0day-ci/linux/commits/David-Woodhouse/KVM-Add-Xen-hype... base: https://git.kernel.org/cgit/linux/kernel/git/mst/vhost.git linux-next in testcase: kernel-selftests version: kernel-selftests-x86_64-b5a583fb-1_20201015 with following parameters: group: kvm ucode: 0xe2 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: 8 threads Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> kern :warn : [ 86.138096] WARNING: suspicious RCU usage kern :warn : [ 86.142286] 5.10.0-rc5-gfdd90b978b4e #1 Tainted: G I kern :warn : [ 86.148879] ----------------------------- kern :warn : [ 86.153110] include/linux/kvm_host.h:633 suspicious rcu_dereference_check() usage! kern :warn : [ 86.160992] other info that might help us debug this: kern :warn : [ 86.169413] rcu_scheduler_active = 2, debug_locks = 1 kern :warn : [ 86.176231] no locks held by xen_shinfo_test/10611. kern :warn : [ 86.181328] stack backtrace: kern :warn : [ 86.185946] CPU: 2 PID: 10611 Comm: xen_shinfo_test Tainted: G I 5.10.0-rc5-gfdd90b978b4e #1 kern :warn : [ 86.196092] Hardware name: /NUC6i7KYB, BIOS KYSKLi70.86A.0041.2016.0817.1130 08/17/2016 kern :warn : [ 86.204517] Call Trace: kern :warn : [ 86.207122] dump_stack+0x8d/0xb5 kern :warn : [ 86.210615] kvm_map_gfn+0xbb/0xc0 kern :warn : [ 86.214127] kvm_xen_map_guest_page+0x5c/0x100 kern :warn : [ 86.218787] kvm_xen_hvm_set_attr+0xf8/0x1c0 kern :warn : [ 86.223275] kvm_arch_vm_ioctl+0xae9/0xc80 kern :warn : [ 86.227549] ? preempt_count_sub+0xa1/0xe0 kern :warn : [ 86.231777] ? _raw_spin_unlock+0x29/0x40 kern :warn : [ 86.235954] ? do_fault+0x22d/0x4e0 kern :warn : [ 86.239627] kvm_vm_ioctl+0x313/0xd80 kern :warn : [ 86.243481] ? find_held_lock+0x2b/0x80 kern :warn : [ 86.247500] __x64_sys_ioctl+0x83/0xc0 kern :warn : [ 86.251445] do_syscall_64+0x33/0x40 kern :warn : [ 86.255125] entry_SYSCALL_64_after_hwframe+0x44/0xa9 kern :warn : [ 86.260398] RIP: 0033:0x7f63eac65427 kern :warn : [ 86.264172] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48 kern :warn : [ 86.283684] RSP: 002b:00007ffcafb0b078 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 kern :warn : [ 86.291524] RAX: ffffffffffffffda RBX: 000000004028aec8 RCX: 00007f63eac65427 kern :warn : [ 86.298957] RDX: 00007ffcafb0b0e0 RSI: 000000004028aec8 RDI: 0000000000000006 kern :warn : [ 86.306361] RBP: 000000000040a7d0 R08: 000000000040be70 R09: 000000004028aec8 kern :warn : [ 86.313753] R10: 0000000000000001 R11: 0000000000000246 R12: 00000000004017e0 kern :warn : [ 86.321286] R13: 00007ffcafb0b2c0 R14: 0000000000000000 R15: 0000000000000000 user :notice: [ 94.700635] gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ix86_64 -Iinclude/x86_64 -I.. -pthread -no-pie x86_64/cr4_cpuid_sync_test.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-fdd90b978b4e0dbbdf0f50f6dd01569a569ad614/tools/testing/selftests/kvm/libkvm.a -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-fdd90b978b4e0dbbdf0f50f6dd01569a569ad614/tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test user :notice: [ 110.220295] gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ix86_64 -Iinclude/x86_64 -I.. -pthread -no-pie x86_64/evmcs_test.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-fdd90b978b4e0dbbdf0f50f6dd01569a569ad614/tools/testing/selftests/kvm/libkvm.a -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-fdd90b978b4e0dbbdf0f50f6dd01569a569ad614/tools/testing/selftests/kvm/x86_64/evmcs_test To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

9 months, 1 week

2
1
0 / 0

[printk] 698d74a93f: WARNING:at_kernel/rcu/tree_plugin.h:#rcu_note_context_switch

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 698d74a93f2e1047141df80bcdaa215f69555c6b ("printk: add pr_flush()") https://git.kernel.org/cgit/linux/kernel/git/rt/linux-rt-devel.git linux-5.10.y-rt-rebase in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------------------+------------+------------+ | | 232e3aeb81 | 698d74a93f | +--------------------------------------------------------------+------------+------------+ | boot_successes | 4 | 2 | | boot_failures | 0 | 6 | | WARNING:at_kernel/rcu/tree_plugin.h:#rcu_note_context_switch | 0 | 6 | | RIP:rcu_note_context_switch | 0 | 6 | +--------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 58.773663] WARNING: CPU: 0 PID: 126 at kernel/rcu/tree_plugin.h:297 rcu_note_context_switch+0x82/0x600 [ 58.775771] Modules linked in: sr_mod cdrom ppdev joydev ata_generic parport_pc floppy pata_acpi parport bochs_drm drm_vram_helper drm_ttm_helper ttm ata_piix libata i2c_piix4 [ 58.779412] CPU: 0 PID: 126 Comm: lkp-setup-rootf Not tainted 5.10.1-00101-g698d74a93f2e #1 [ 58.781308] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 58.783233] RIP: 0010:rcu_note_context_switch+0x82/0x600 [ 58.784509] Code: f6 74 0f 65 8b 05 9e 2a 23 6c 85 c0 0f 84 3b 01 00 00 45 84 e4 75 15 65 48 8b 04 25 00 7f 01 00 8b 88 9c 07 00 00 85 c9 7e 02 <0f> 0b 65 48 8b 04 25 00 7f 01 00 8b 90 9c 07 00 00 85 d2 7e 0e 41 [ 58.789021] RSP: 0018:ffffad130073bc30 EFLAGS: 00010002 [ 58.790214] RAX: ffff8d6f17218000 RBX: ffff8d6fb7c2a780 RCX: 0000000000000001 [ 58.792037] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff951d3177 [ 58.793706] RBP: ffffad130073bcc8 R08: 0000000000000001 R09: 0000000000000001 [ 58.795437] R10: ffffffff972b15e0 R11: 0000000000000001 R12: 0000000000000000 [ 58.797082] R13: ffff8d6f17218000 R14: 0000000000000000 R15: ffff8d6fb7c29958 [ 58.798684] FS: 00007fc12c59e700(0000) GS:ffff8d6fb7c00000(0000) knlGS:0000000000000000 [ 58.804602] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 58.805881] CR2: 00007fc12bf502f0 CR3: 00000001ac284000 CR4: 00000000000406f0 [ 58.807306] Call Trace: [ 58.807986] __schedule+0xe0/0xb60 [ 58.808854] ? mark_held_locks+0x49/0x80 [ 58.809853] schedule+0x4e/0x100 [ 58.810661] schedule_timeout+0x1c3/0x3a0 [ 58.811626] ? __next_timer_interrupt+0x100/0x100 [ 58.812692] ? msleep+0x25/0x60 [ 58.813408] msleep+0x39/0x60 [ 58.814111] pr_flush+0x13b/0x160 [ 58.814932] kmsg_dump+0x75/0x200 [ 58.815982] ? wake_up_klogd+0x4f/0x80 [ 58.817015] ? vprintk_emit+0x38/0x60 [ 58.817912] ? printk+0x62/0x79 [ 58.818656] ? emergency_restart+0xf/0x20 [ 58.819533] emergency_restart+0xf/0x20 [ 58.820468] __handle_sysrq.cold+0x51/0x138 [ 58.821514] write_sysrq_trigger+0x3f/0x60 [ 58.822546] proc_reg_write+0x53/0x80 [ 58.823724] vfs_write+0xee/0x3c0 [ 58.824544] ksys_write+0x68/0xe0 [ 58.825370] do_syscall_64+0x33/0x40 [ 58.826278] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 58.827565] RIP: 0033:0x7fc12bc7e100 [ 58.828505] Code: 73 01 c3 48 8b 0d 40 0d 2d 00 31 d2 48 29 c2 64 89 11 48 83 c8 ff eb ea 90 90 83 3d dd 70 2d 00 00 75 10 b8 01 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 de b0 01 00 48 89 04 24 [ 58.833215] RSP: 002b:00007fff429087a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 58.835302] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fc12bc7e100 [ 58.837209] RDX: 0000000000000002 RSI: 0000000001093808 RDI: 0000000000000001 [ 58.839443] RBP: 0000000001093808 R08: 000000000000000a R09: 00007fc12c59e700 [ 58.841536] R10: 00007fff42908618 R11: 0000000000000246 R12: 00007fc12bf50260 [ 58.843121] R13: 0000000000000002 R14: 0000000000000000 R15: 00000000011234c8 [ 58.844886] irq event stamp: 43466 [ 58.845746] hardirqs last enabled at (43465): [<ffffffff9498176f>] _raw_spin_unlock_irqrestore+0x4f/0x60 [ 58.847866] hardirqs last disabled at (43466): [<ffffffff9497a148>] __schedule+0x6e8/0xb60 [ 58.849630] softirqs last enabled at (43388): [<ffffffff94c0034e>] __do_softirq+0x34e/0x49c [ 58.851457] softirqs last disabled at (43379): [<ffffffff94a010f2>] asm_call_irq_on_stack+0x12/0x20 [ 58.853970] ---[ end trace 240f55e0d36d2f4e ]--- To reproduce: # build kernel cd linux cp config-5.10.1-00101-g698d74a93f2e .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

[dma] adc430f226: WARNING:at_kernel/sched/core.c:#__might_sleep

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: adc430f226136c02a382fe6dff29a6382059ce0c ("dma-buf: system_heap: Add deferred freeing to the system heap") url: https://github.com/0day-ci/linux/commits/John-Stultz/dma-buf-heaps-Add-de... in testcase: locktorture version: with following parameters: runtime: 300s test: cpuhotplug test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors. test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------+------------+------------+ | | 51e229fb2d | adc430f226 | +-------------------------------------------------------+------------+------------+ | boot_successes | 19 | 0 | | boot_failures | 9 | 4 | | invoked_oom-killer:gfp_mask=0x | 1 | | | Mem-Info | 1 | | | EIP:__put_user_nocheck_4 | 1 | | | kernel_BUG_at_kernel/sched/core.c | 8 | 4 | | invalid_opcode:#[##] | 8 | 4 | | EIP:sched_cpu_dying | 8 | 4 | | Kernel_panic-not_syncing:Fatal_exception | 8 | 4 | | WARNING:possible_circular_locking_dependency_detected | 8 | 4 | | WARNING:at_kernel/sched/core.c:#__might_sleep | 0 | 4 | | EIP:__might_sleep | 0 | 4 | +-------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 107.814818] WARNING: CPU: 0 PID: 130 at kernel/sched/core.c:7858 __might_sleep+0x6c/0x80 [ 107.816258] mtdoops: mtd device (mtddev=name/number) must be supplied [ 107.818504] Modules linked in: [ 107.818520] CPU: 0 PID: 130 Comm: dmabuf-deferred Not tainted 5.10.0-next-20201217-00003-gadc430f22613 #1 [ 107.818536] EIP: __might_sleep+0x6c/0x80 [ 107.821708] L440GX flash mapping: failed to find PIIX4 ISA bridge, cannot continue [ 107.823070] Code: 32 db 01 8b 73 0c 8b 83 e0 17 00 00 89 44 24 0c 8b 9b e0 17 00 00 89 5c 24 08 89 74 24 04 c7 04 24 38 92 cb da e8 c9 a5 b5 00 <0f> 0b 8b 4d ec 8b 55 f0 8b 45 f4 eb b0 8d b4 26 00 00 00 00 55 89 [ 107.827562] SBC-GXx flash: IO:0x258-0x259 MEM:0xdc000-0xdffff [ 107.828850] EAX: 00000069 EBX: d96a0ed1 ECX: 00000000 EDX: 00000000 [ 107.836825] scx200_docflash: NatSemi SCx200 DOCCS Flash Driver [ 107.839849] ESI: 00000001 EDI: c11e4200 EBP: c11b3eec ESP: c11b3ec8 [ 107.839857] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010296 [ 107.839867] CR0: 80050033 CR2: 00000000 CR3: 1b540000 CR4: 00040690 [ 107.839889] Call Trace: [ 107.842656] slram: not enough parameters. [ 107.844946] ? prepare_to_wait_event+0x51/0x140 [ 107.855521] mtd mtd0: GPIO lookup for consumer wp [ 107.855658] ? prepare_to_wait_event+0x51/0x140 [ 107.856828] mtd mtd0: using lookup tables for GPIO lookup [ 107.856842] mtd mtd0: No GPIO consumer wp found [ 107.858676] __mutex_lock+0x21/0x820 [ 107.858693] ? lockdep_hardirqs_on_prepare+0xda/0x180 [ 107.858706] ? _raw_spin_unlock_irqrestore+0x35/0x40 [ 107.874413] ? prepare_to_wait_event+0x7d/0x140 [ 107.876397] ? trace_hardirqs_on+0x22/0xe0 [ 107.878110] ? prepare_to_wait_event+0x7d/0x140 [ 107.880095] mutex_lock_nested+0x20/0x40 [ 107.881791] ? deferred_free_thread+0x75/0xc0 [ 107.883801] deferred_free_thread+0x75/0xc0 [ 107.885686] ? do_wait_intr_irq+0xa0/0xa0 [ 107.887498] kthread+0x107/0x120 [ 107.889019] ? freelist_shrink_count+0x40/0x40 [ 107.890980] ? kthread_create_worker_on_cpu+0x20/0x20 [ 107.893128] ret_from_fork+0x19/0x24 [ 107.894780] irq event stamp: 339 [ 107.896256] hardirqs last enabled at (349): [<d96bfb75>] console_unlock+0x3f5/0x5a0 [ 107.899642] hardirqs last disabled at (358): [<d96bfb66>] console_unlock+0x3e6/0x5a0 [ 107.903036] softirqs last enabled at (336): [<da2d79d5>] __do_softirq+0x295/0x435 [ 107.906066] softirqs last disabled at (331): [<d96232bd>] do_softirq_own_stack+0x1d/0x40 [ 107.908613] random: get_random_bytes called from init_oops_id+0x37/0x40 with crng_init=0 [ 107.908624] ---[ end trace 4d9142d8a53ac308 ]--- To reproduce: # build kernel cd linux cp config-5.10.0-next-20201217-00003-gadc430f22613 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

[kernel/watchdog_hld.c] 6e37d53a67: BUG:sleeping_function_called_from_invalid_context_at_include/linux/sched/mm.h

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 6e37d53a67753bcb12a0b9102cac85d98f8a0453 ("[PATCH] kernel/watchdog_hld.c: Fix access percpu in preemptible context") url: https://github.com/0day-ci/linux/commits/Lecopzer-Chen/kernel-watchdog_hl... base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git accefff5b547a9a1d959c7e76ad539bf2480e78b in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------------------------------------------------------------------------------+------------+------------+ | | accefff5b5 | 6e37d53a67 | +--------------------------------------------------------------------------------------------------------------------------+------------+------------+ | boot_successes | 18 | 0 | | boot_failures | 2 | 35 | | Kernel_panic-not_syncing:VFS:Unable_to_mount_root_fs_on_unknown-block(#,#) | 2 | | | BUG:sleeping_function_called_from_invalid_context_at_include/linux/sched/mm.h | 0 | 33 | | kernel_BUG_at_kernel/sched/core.c | 0 | 1 | | invalid_opcode:#[##] | 0 | 1 | | RIP:sched_cpu_dying | 0 | 1 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 1 | | BUG:kernel_failed_in_early-boot_stage,last_printk:LKP:HOSTNAME_vm-snb-#,MAC#:#:#:#:#:#,kernel##,serial_console/dev/ttyS0 | 0 | 2 | +--------------------------------------------------------------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 0.468064] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:196 [ 0.468516] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper/0 [ 0.469509] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.10.0-10896-g6e37d53a6775 #1 [ 0.470503] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 0.470503] Call Trace: [ 0.470503] dump_stack+0x6d/0x88 [ 0.470503] ___might_sleep.cold+0x94/0xa2 [ 0.470503] kmem_cache_alloc_trace+0x3ca/0x460 [ 0.470503] ? hardlockup_detector_event_create+0xa0/0xa0 [ 0.470503] perf_event_alloc+0x51/0xc00 [ 0.470503] ? dynamic_debug_init+0x159/0x1a3 [ 0.470503] perf_event_create_kernel_counter+0x2d/0x140 [ 0.470503] hardlockup_detector_event_create+0x47/0xa0 [ 0.470503] hardlockup_detector_perf_init+0xc/0x40 [ 0.470503] lockup_detector_init+0x51/0x88 [ 0.470503] kernel_init_freeable+0x104/0x255 [ 0.470503] ? rest_init+0xd0/0xd0 [ 0.470503] kernel_init+0xa/0x110 [ 0.470503] ret_from_fork+0x22/0x30 [ 0.470526] NMI watchdog: Perf NMI watchdog permanently disabled [ 0.471647] smp: Bringing up secondary CPUs ... [ 0.472699] x86: Booting SMP configuration: [ 0.473511] .... node #0, CPUs: #1 [ 0.138070] kvm-clock: cpu 1, msr 337b041, secondary cpu clock [ 0.138070] masked ExtINT on CPU#1 [ 0.138070] smpboot: CPU 1 Converting physical 0 to logical die 1 [ 0.507566] kvm-guest: stealtime: cpu 1, msr 23fd18540 [ 0.508605] smp: Brought up 1 node, 2 CPUs [ 0.509515] smpboot: Max logical packages: 2 [ 0.510509] smpboot: Total of 2 processors activated (10774.03 BogoMIPS) [ 0.512579] devtmpfs: initialized [ 0.513583] x86/mm: Memory block size: 128MB [ 0.516951] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.517516] futex hash table entries: 512 (order: 3, 32768 bytes, linear) [ 0.519552] pinctrl core: initialized pinctrl subsystem [ 0.520772] NET: Registered protocol family 16 [ 0.521876] audit: initializing netlink subsys (disabled) [ 0.523545] audit: type=2000 audit(1608248685.973:1): state=initialized audit_enabled=0 res=1 [ 0.523736] thermal_sys: Registered thermal governor 'fair_share' [ 0.524508] thermal_sys: Registered thermal governor 'bang_bang' [ 0.525508] thermal_sys: Registered thermal governor 'step_wise' [ 0.526508] thermal_sys: Registered thermal governor 'user_space' [ 0.527537] cpuidle: using governor menu [ 0.529821] ACPI: bus type PCI registered [ 0.530508] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.532208] PCI: Using configuration type 1 for base access [ 0.535587] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.538626] ACPI: Added _OSI(Module Device) [ 0.539509] ACPI: Added _OSI(Processor Device) [ 0.540483] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.540511] ACPI: Added _OSI(Processor Aggregator Device) [ 0.541523] ACPI: Added _OSI(Linux-Dell-Video) [ 0.542548] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 0.543518] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 0.546159] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.547852] ACPI: Interpreter enabled [ 0.548531] ACPI: (supports S0 S3 S4 S5) [ 0.549476] ACPI: Using IOAPIC for interrupt routing [ 0.549545] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.550690] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.556249] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.556518] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.557536] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.559088] acpiphp: Slot [3] registered [ 0.559544] acpiphp: Slot [4] registered [ 0.560469] acpiphp: Slot [5] registered [ 0.560541] acpiphp: Slot [6] registered [ 0.561424] acpiphp: Slot [7] registered [ 0.561544] acpiphp: Slot [8] registered [ 0.562546] acpiphp: Slot [9] registered [ 0.563462] acpiphp: Slot [10] registered [ 0.563543] acpiphp: Slot [11] registered [ 0.564546] acpiphp: Slot [12] registered [ 0.565481] acpiphp: Slot [13] registered [ 0.565545] acpiphp: Slot [14] registered [ 0.566516] acpiphp: Slot [15] registered [ 0.567492] acpiphp: Slot [16] registered [ 0.567544] acpiphp: Slot [17] registered [ 0.568528] acpiphp: Slot [18] registered [ 0.569487] acpiphp: Slot [19] registered [ 0.569551] acpiphp: Slot [20] registered [ 0.570547] acpiphp: Slot [21] registered [ 0.571532] acpiphp: Slot [22] registered [ 0.572489] acpiphp: Slot [23] registered [ 0.572544] acpiphp: Slot [24] registered [ 0.573485] acpiphp: Slot [25] registered [ 0.573550] acpiphp: Slot [26] registered [ 0.574545] acpiphp: Slot [27] registered [ 0.575545] acpiphp: Slot [28] registered [ 0.576546] acpiphp: Slot [29] registered [ 0.577548] acpiphp: Slot [30] registered [ 0.578497] acpiphp: Slot [31] registered [ 0.578531] PCI host bridge to bus 0000:00 [ 0.579510] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.580509] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.581509] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.582511] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.583510] pci_bus 0000:00: root bus resource [mem 0x240000000-0x2bfffffff window] [ 0.584511] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.585577] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.587080] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 To reproduce: # build kernel cd linux cp config-5.10.0-10896-g6e37d53a6775 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Oliver Sang

9 months, 1 week

2
2
0 / 0

[x86/kprobes] 413d31338f: kernel_BUG_at_mm/vmalloc.c

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 413d31338fca081133c508bcabc57854765f43df ("[PATCH 1/1] x86/kprobes: Do not decode opcode in resume_execution()") url: https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/x86-kprobes-Cla... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 238c91115cd05c71447ea071624a4c9fe661f970 in testcase: trinity version: trinity-static-i386-x86_64-f93256fb_2019-08-28 with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------+------------+------------+ | | 238c91115c | 413d31338f | +------------------------------------------+------------+------------+ | boot_successes | 8 | 0 | | boot_failures | 0 | 8 | | kernel_BUG_at_mm/vmalloc.c | 0 | 8 | | invalid_opcode:#[##] | 0 | 8 | | EIP:vmalloc_to_page | 0 | 8 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 8 | +------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 8.468076] kernel BUG at mm/vmalloc.c:364! [ 8.468257] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 8.468344] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.9.0-rc7-00002-g413d31338fca08 #2 [ 8.468344] EIP: vmalloc_to_page+0x27/0x107 [ 8.468344] Code: 89 c8 c3 55 89 c2 89 c1 c1 e9 1e 89 e5 57 56 53 53 a1 64 06 cb c1 89 55 f0 8d 1c c8 89 d0 e8 9d ff ff ff 85 c0 8b 55 f0 75 02 <0f> 0b 8b 03 8b 4b 04 89 c6 81 e6 18 0e 00 00 74 02 0f 0b 89 c3 81 [ 8.468344] EAX: 00000000 EBX: c1f6a000 ECX: 00000000 EDX: 00000000 [ 8.468344] ESI: d087bda5 EDI: d087bda5 EBP: d087bce0 ESP: d087bcd0 [ 8.468344] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010246 [ 8.468344] CR0: 80050033 CR2: 00000000 CR3: 01f6a000 CR4: 000406b0 [ 8.468344] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 [ 8.468344] DR6: fffe0ff0 DR7: 00000400 [ 8.468344] Call Trace: [ 8.468344] __text_poke+0x4c/0x28f [ 8.468344] ? insn_get_displacement+0x1b/0xce [ 8.468344] ? cmp_ex_sort+0x17/0x17 [ 8.468344] text_poke+0x35/0x3a [ 8.468344] arch_copy_kprobe+0x12b/0x148 [ 8.468344] arch_prepare_kprobe+0x48/0x6a [ 8.468344] register_kprobe+0x27f/0x374 [ 8.468344] __register_trace_kprobe+0x6e/0x73 [ 8.468344] register_trace_kprobe+0x140/0x282 [ 8.468344] ? traceprobe_set_print_fmt+0x40/0x58 [ 8.468344] trace_kprobe_create+0x33e/0x3c3 [ 8.468344] ? __kmalloc+0xc1/0xcd [ 8.468344] ? argv_split+0x5b/0xb9 [ 8.468344] ? trace_kprobe_create+0x3c3/0x3c3 [ 8.468344] create_or_delete_trace_kprobe+0x1b/0x27 [ 8.468344] trace_run_command+0x48/0x6b [ 8.468344] trace_kprobe_run_command+0xf/0x11 [ 8.468344] dynevent_create+0xb/0xd [ 8.468344] kprobe_event_gen_test_init+0x82/0x1e2 [ 8.468344] ? kprobe_event_cmd_init+0x13/0x13 [ 8.468344] ? init_trace_printk+0xf/0xf [ 8.468344] do_one_initcall+0xaa/0x1d4 [ 8.468344] ? rdinit_setup+0x21/0x21 [ 8.468344] ? trace_initcall_level+0x72/0x76 [ 8.468344] do_initcalls+0xa8/0xc8 [ 8.468344] kernel_init_freeable+0x94/0xb9 [ 8.468344] ? rest_init+0xce/0xce [ 8.468344] kernel_init+0x8/0xe4 [ 8.468344] ret_from_fork+0x1c/0x28 [ 8.468344] Modules linked in: [ 8.476471] ---[ end trace 543d3449d7fd5d3a ]--- To reproduce: # build kernel cd linux cp config-5.9.0-rc7-00002-g413d31338fca08 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Oliver Sang

9 months, 1 week

2
1
0 / 0

[kernfs] 37746795a6: stress-ng.spawn.ops_per_sec 7.1% improvement

by kernel test robot

Greeting, FYI, we noticed a 7.1% improvement of stress-ng.spawn.ops_per_sec to commit: commit: 37746795a6fa6be42e847f0a6f5ac54d3d273eec ("[PATCH 2/2] kernfs: remove mutex in kernfs_dop_revalidate") url: https://github.com/0day-ci/linux/commits/Fox-Chen/kernfs-speed-up-concurr... base: https://git.kernel.org/cgit/linux/kernel/git/gregkh/driver-core.git 33c0c9bdf7a59051a654cd98b7d2b48ce0080967 in testcase: stress-ng on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory with following parameters: nr_threads: 100% disk: 1HDD testtime: 30s class: exec_spawn cpufreq_governor: performance ucode: 0x5003003 Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: exec_spawn/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/stress-ng/30s/0x5003003 commit: d680236464 ("kernfs: replace the mutex in kernfs_iop_permission with a rwlock") 37746795a6 ("kernfs: remove mutex in kernfs_dop_revalidate") d68023646451afd9 37746795a6fa6be42e847f0a6f5 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 5:5 0% 5:5 stress-ng.spawn.pass 2:5 4% 2:5 perf-profile.children.cycles-pp.error_entry 0:5 1% 0:5 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 493582 +7.1% 528714 stress-ng.spawn.ops 16452 +7.1% 17623 stress-ng.spawn.ops_per_sec 32.33 +0.1% 32.36 stress-ng.time.elapsed_time 32.33 +0.1% 32.36 stress-ng.time.elapsed_time.max 2.00 +0.0% 2.00 stress-ng.time.exit_status 658333 +3.5% 681600 stress-ng.time.involuntary_context_switches 283022 +9.0% 308406 stress-ng.time.major_page_faults 6745 -1.2% 6665 stress-ng.time.maximum_resident_set_size 89500911 +7.1% 95880935 stress-ng.time.minor_page_faults 4096 +0.0% 4096 stress-ng.time.page_size 8052 -0.7% 7998 stress-ng.time.percent_of_cpu_this_job_got 2375 -2.0% 2328 stress-ng.time.system_time 228.10 +14.1% 260.33 ± 11% stress-ng.time.user_time 2063596 +19.6% 2468759 ± 4% stress-ng.time.voluntary_context_switches 57.35 +0.8% 57.80 uptime.boot 2342 ± 3% +5.3% 2467 ± 3% uptime.idle 21.95 +0.6% 22.08 boot-time.boot 15.06 +1.2% 15.23 boot-time.dhcp 1626 +5.2% 1711 ± 4% boot-time.idle 0.86 +5.0% 0.91 ± 9% boot-time.smp_boot 20.74 ± 6% +1.8% 21.12 ± 4% iostat.cpu.idle 0.00 ± 6% +333.7% 0.01 ±140% iostat.cpu.iowait 71.91 -1.8% 70.61 ± 2% iostat.cpu.system 7.34 +12.5% 8.26 ± 10% iostat.cpu.user 16.56 ± 7% +0.4 16.91 ± 3% mpstat.cpu.all.idle% 0.00 ±199% +0.0 0.01 ±188% mpstat.cpu.all.iowait% 1.14 +0.0 1.17 mpstat.cpu.all.irq% 0.85 ± 3% +0.1 0.97 ± 5% mpstat.cpu.all.soft% 73.75 -1.5 72.26 mpstat.cpu.all.sys% 7.71 +1.0 8.68 ± 10% mpstat.cpu.all.usr% 13533153 +78.1% 24104024 ± 60% cpuidle.C1.time 348159 ± 2% +55.8% 542319 ± 28% cpuidle.C1.usage 4.654e+08 ± 11% -11.7% 4.11e+08 ± 46% cpuidle.C1E.time 1382970 ± 7% -3.6% 1332529 ± 31% cpuidle.C1E.usage 84828815 ± 23% +81.5% 1.539e+08 ± 95% cpuidle.C6.time 145559 ± 18% +57.0% 228510 ± 74% cpuidle.C6.usage 521838 ± 9% +79.2% 935160 ± 16% cpuidle.POLL.time 173621 ± 10% +85.3% 321728 ± 18% cpuidle.POLL.usage 0.00 -100.0% 0.00 numa-numastat.node0.interleave_hit 29619941 ± 8% +14.2% 33837582 ± 6% numa-numastat.node0.local_node 29647151 ± 7% +14.3% 33879700 ± 6% numa-numastat.node0.numa_hit 27211 ±125% +54.7% 42099 ± 89% numa-numastat.node0.other_node 0.00 -100.0% 0.00 numa-numastat.node1.interleave_hit 30592614 ± 6% +0.5% 30741986 ± 8% numa-numastat.node1.local_node 30655936 ± 6% +0.4% 30783157 ± 8% numa-numastat.node1.numa_hit 63307 ± 45% -35.0% 41163 ± 84% numa-numastat.node1.other_node 20.00 ± 6% +3.0% 20.60 ± 3% vmstat.cpu.id 71.40 -2.0% 70.00 vmstat.cpu.sy 7.00 +5.7% 7.40 ± 10% vmstat.cpu.us 0.00 -100.0% 0.00 vmstat.io.bi 0.00 -100.0% 0.00 vmstat.io.bo 1256109 +0.9% 1267283 vmstat.memory.cache 1.936e+08 -0.0% 1.936e+08 vmstat.memory.free 0.00 +4e+101% 0.40 ±200% vmstat.procs.b 82.20 -1.2% 81.20 vmstat.procs.r 117814 ± 2% +20.0% 141395 ± 4% vmstat.system.cs 239630 +1.1% 242302 vmstat.system.in 32.33 +0.1% 32.36 time.elapsed_time 32.33 +0.1% 32.36 time.elapsed_time.max 2.00 +0.0% 2.00 time.exit_status 658333 +3.5% 681600 time.involuntary_context_switches 283022 +9.0% 308406 time.major_page_faults 6745 -1.2% 6665 time.maximum_resident_set_size 89500911 +7.1% 95880935 time.minor_page_faults 4096 +0.0% 4096 time.page_size 8052 -0.7% 7998 time.percent_of_cpu_this_job_got 2375 -2.0% 2328 time.system_time 228.10 +14.1% 260.33 ± 11% time.user_time 2063596 +19.6% 2468759 ± 4% time.voluntary_context_switches 57492 ± 5% +7.7% 61902 ± 9% meminfo.Active 57492 ± 5% +7.7% 61902 ± 9% meminfo.Active(anon) 32863 ± 10% +8.9% 35801 ± 12% meminfo.AnonHugePages 321698 +1.1% 325077 meminfo.AnonPages 1160948 +0.9% 1171968 meminfo.Cached 197632 +0.0% 197632 meminfo.CmaFree 204800 +0.0% 204800 meminfo.CmaTotal 98353700 +0.0% 98353700 meminfo.CommitLimit 5425201 -0.4% 5401901 meminfo.Committed_AS 1.89e+08 +0.1% 1.892e+08 meminfo.DirectMap1G 12385942 ± 3% -1.7% 12176322 ± 4% meminfo.DirectMap2M 711682 ± 6% -0.0% 711587 ± 6% meminfo.DirectMap4k 2048 +0.0% 2048 meminfo.Hugepagesize 452644 +2.4% 463321 ± 2% meminfo.Inactive 452644 +2.4% 463321 ± 2% meminfo.Inactive(anon) 83283 +1.6% 84618 meminfo.KReclaimable 21139 ± 2% +2.3% 21620 meminfo.KernelStack 114807 ± 4% +0.3% 115131 ± 4% meminfo.Mapped 1.928e+08 -0.0% 1.928e+08 meminfo.MemAvailable 1.936e+08 -0.0% 1.936e+08 meminfo.MemFree 1.967e+08 +0.0% 1.967e+08 meminfo.MemTotal 3084866 +0.7% 3105963 meminfo.Memused 20.40 ± 2% +2.0% 20.80 meminfo.Mlocked 16462 ± 5% +6.5% 17537 ± 11% meminfo.PageTables 47865 +0.0% 47868 meminfo.Percpu 83283 +1.6% 84618 meminfo.SReclaimable 216074 +2.0% 220498 meminfo.SUnreclaim 188121 ± 2% +5.9% 199137 ± 2% meminfo.Shmem 299358 +1.9% 305117 meminfo.Slab 972837 +0.0% 972840 meminfo.Unevictable 3.436e+10 +0.0% 3.436e+10 meminfo.VmallocTotal 245154 +0.3% 246008 meminfo.VmallocUsed 107302 +0.4% 107744 meminfo.max_used_kB 2806 ± 25% -26.8% 2052 ± 31% numa-meminfo.node0.Active 2806 ± 25% -26.8% 2052 ± 31% numa-meminfo.node0.Active(anon) 18061 ± 30% -8.3% 16557 ± 68% numa-meminfo.node0.AnonHugePages 139516 ± 40% +1.1% 141057 ± 53% numa-meminfo.node0.AnonPages 194343 ± 41% +6.2% 206444 ± 44% numa-meminfo.node0.AnonPages.max 512231 -1.0% 507232 ± 2% numa-meminfo.node0.FilePages 146001 ± 37% +1.4% 148068 ± 49% numa-meminfo.node0.Inactive 146001 ± 37% +1.4% 148068 ± 49% numa-meminfo.node0.Inactive(anon) 46192 ± 11% -4.3% 44208 ± 5% numa-meminfo.node0.KReclaimable 11483 ± 10% +0.1% 11492 ± 6% numa-meminfo.node0.KernelStack 42293 ± 6% +3.2% 43629 ± 6% numa-meminfo.node0.Mapped 96166177 +0.1% 96252121 numa-meminfo.node0.MemFree 97661364 +0.0% 97661364 numa-meminfo.node0.MemTotal 1495185 ± 7% -5.7% 1409241 ± 9% numa-meminfo.node0.MemUsed 10.00 ± 46% -6.0% 9.40 ± 55% numa-meminfo.node0.Mlocked 9356 ± 42% +4.6% 9786 ± 24% numa-meminfo.node0.PageTables 46192 ± 11% -4.3% 44208 ± 5% numa-meminfo.node0.SReclaimable 116911 ± 8% -3.8% 112488 ± 6% numa-meminfo.node0.SUnreclaim 9165 ± 44% -6.7% 8550 ± 47% numa-meminfo.node0.Shmem 163103 ± 9% -3.9% 156697 ± 4% numa-meminfo.node0.Slab 503071 -0.9% 498687 ± 2% numa-meminfo.node0.Unevictable 54326 ± 4% +10.5% 60044 ± 9% numa-meminfo.node1.Active 54326 ± 4% +10.5% 60044 ± 9% numa-meminfo.node1.Active(anon) 14838 ± 49% +30.3% 19337 ± 43% numa-meminfo.node1.AnonHugePages 182538 ± 30% +1.3% 184944 ± 38% numa-meminfo.node1.AnonPages 368061 ± 14% +2.0% 375459 ± 19% numa-meminfo.node1.AnonPages.max 647698 +2.8% 665753 ± 2% numa-meminfo.node1.FilePages 306345 ± 18% +3.4% 316859 ± 21% numa-meminfo.node1.Inactive 306345 ± 18% +3.4% 316859 ± 21% numa-meminfo.node1.Inactive(anon) 37231 ± 15% +9.0% 40572 ± 6% numa-meminfo.node1.KReclaimable 9798 ± 11% +2.7% 10065 ± 7% numa-meminfo.node1.KernelStack 72101 ± 9% +1.0% 72804 ± 9% numa-meminfo.node1.Mapped 97456255 -0.1% 97344817 numa-meminfo.node1.MemFree 99046036 +0.0% 99046036 numa-meminfo.node1.MemTotal 1589779 ± 7% +7.0% 1701218 ± 7% numa-meminfo.node1.MemUsed 10.00 ± 42% +8.0% 10.80 ± 46% numa-meminfo.node1.Mlocked 7050 ± 48% +12.6% 7941 ± 37% numa-meminfo.node1.PageTables 37231 ± 15% +9.0% 40572 ± 6% numa-meminfo.node1.SReclaimable 101025 ± 7% +9.7% 110844 ± 7% numa-meminfo.node1.SUnreclaim 177937 ± 4% +7.7% 191605 ± 4% numa-meminfo.node1.Shmem 138256 ± 9% +9.5% 151417 ± 5% numa-meminfo.node1.Slab 469765 +0.9% 474151 ± 2% numa-meminfo.node1.Unevictable 707.60 ± 26% -28.1% 509.00 ± 32% numa-vmstat.node0.nr_active_anon 35024 ± 40% +0.7% 35273 ± 53% numa-vmstat.node0.nr_anon_pages 8.40 ± 34% -9.5% 7.60 ± 71% numa-vmstat.node0.nr_anon_transparent_hugepages 128053 -1.0% 126803 ± 2% numa-vmstat.node0.nr_file_pages 24041336 +0.1% 24062572 numa-vmstat.node0.nr_free_pages 36675 ± 37% +1.2% 37106 ± 49% numa-vmstat.node0.nr_inactive_anon 11536 ± 10% -0.3% 11499 ± 6% numa-vmstat.node0.nr_kernel_stack 10780 ± 6% +2.8% 11086 ± 6% numa-vmstat.node0.nr_mapped 2.00 ± 63% -10.0% 1.80 ± 81% numa-vmstat.node0.nr_mlock 2377 ± 41% +3.6% 2464 ± 23% numa-vmstat.node0.nr_page_table_pages 2287 ± 44% -6.7% 2133 ± 47% numa-vmstat.node0.nr_shmem 11548 ± 11% -4.3% 11052 ± 5% numa-vmstat.node0.nr_slab_reclaimable 29442 ± 9% -3.0% 28551 ± 6% numa-vmstat.node0.nr_slab_unreclaimable 125767 -0.9% 124671 ± 2% numa-vmstat.node0.nr_unevictable 707.60 ± 26% -28.1% 509.00 ± 32% numa-vmstat.node0.nr_zone_active_anon 36673 ± 37% +1.2% 37104 ± 49% numa-vmstat.node0.nr_zone_inactive_anon 125767 -0.9% 124671 ± 2% numa-vmstat.node0.nr_zone_unevictable 13913349 ± 9% +15.6% 16082258 ± 7% numa-vmstat.node0.numa_hit 147975 -0.2% 147609 numa-vmstat.node0.numa_interleave 13885197 ± 9% +15.3% 16014881 ± 8% numa-vmstat.node0.numa_local 28205 ±117% +139.1% 67439 ± 79% numa-vmstat.node0.numa_other 13663 ± 4% +10.4% 15084 ± 9% numa-vmstat.node1.nr_active_anon 45968 ± 30% +1.0% 46420 ± 38% numa-vmstat.node1.nr_anon_pages 6.80 ± 53% +29.4% 8.80 ± 46% numa-vmstat.node1.nr_anon_transparent_hugepages 162202 +2.8% 166816 ± 2% numa-vmstat.node1.nr_file_pages 49408 +0.0% 49408 numa-vmstat.node1.nr_free_cma 24363325 -0.1% 24335148 numa-vmstat.node1.nr_free_pages 77056 ± 18% +3.4% 79702 ± 21% numa-vmstat.node1.nr_inactive_anon 2.00 ±200% -100.0% 0.00 numa-vmstat.node1.nr_isolated_anon 9828 ± 11% +2.9% 10113 ± 7% numa-vmstat.node1.nr_kernel_stack 18317 ± 8% +0.9% 18479 ± 9% numa-vmstat.node1.nr_mapped 2.00 ± 63% +10.0% 2.20 ± 66% numa-vmstat.node1.nr_mlock 1739 ± 49% +16.0% 2017 ± 36% numa-vmstat.node1.nr_page_table_pages 44762 ± 4% +7.9% 48279 ± 4% numa-vmstat.node1.nr_shmem 9381 ± 15% +8.7% 10198 ± 6% numa-vmstat.node1.nr_slab_reclaimable 25395 ± 6% +10.8% 28147 ± 7% numa-vmstat.node1.nr_slab_unreclaimable 117441 +0.9% 118537 ± 2% numa-vmstat.node1.nr_unevictable 13663 ± 4% +10.4% 15084 ± 9% numa-vmstat.node1.nr_zone_active_anon 77055 ± 18% +3.4% 79703 ± 21% numa-vmstat.node1.nr_zone_inactive_anon 117441 +0.9% 118537 ± 2% numa-vmstat.node1.nr_zone_unevictable 14877170 ± 8% +1.3% 15074255 ± 10% numa-vmstat.node1.numa_hit 147642 +0.2% 148007 numa-vmstat.node1.numa_interleave 14664461 ± 8% +1.7% 14907749 ± 10% numa-vmstat.node1.numa_local 212740 ± 13% -21.8% 166413 ± 30% numa-vmstat.node1.numa_other 0.00 -100.0% 0.00 proc-vmstat.compact_isolated 14437 ± 5% +7.5% 15522 ± 9% proc-vmstat.nr_active_anon 80719 +0.9% 81439 proc-vmstat.nr_anon_pages 15.60 ± 11% +9.0% 17.00 ± 12% proc-vmstat.nr_anon_transparent_hugepages 4811596 -0.0% 4811016 proc-vmstat.nr_dirty_background_threshold 9634959 -0.0% 9633797 proc-vmstat.nr_dirty_threshold 290438 +1.0% 293219 proc-vmstat.nr_file_pages 49408 +0.0% 49408 proc-vmstat.nr_free_cma 48404974 -0.0% 48399157 proc-vmstat.nr_free_pages 113570 +2.3% 116153 ± 2% proc-vmstat.nr_inactive_anon 2.00 ±200% -100.0% 0.00 proc-vmstat.nr_isolated_anon 21262 ± 2% +1.4% 21562 proc-vmstat.nr_kernel_stack 29141 ± 4% +0.3% 29219 ± 4% proc-vmstat.nr_mapped 5.00 +0.0% 5.00 proc-vmstat.nr_mlock 4080 ± 3% +8.6% 4432 ± 10% proc-vmstat.nr_page_table_pages 47231 ± 2% +5.9% 50012 ± 2% proc-vmstat.nr_shmem 20862 +1.6% 21195 proc-vmstat.nr_slab_reclaimable 54404 +2.8% 55944 proc-vmstat.nr_slab_unreclaimable 243209 +0.0% 243209 proc-vmstat.nr_unevictable 14437 ± 5% +7.5% 15522 ± 9% proc-vmstat.nr_zone_active_anon 113570 +2.3% 116154 ± 2% proc-vmstat.nr_zone_inactive_anon 243209 +0.0% 243209 proc-vmstat.nr_zone_unevictable 10025 ± 29% -37.3% 6284 ±121% proc-vmstat.numa_hint_faults 4564 ± 50% -32.8% 3065 ±148% proc-vmstat.numa_hint_faults_local 59972641 +7.1% 64258388 proc-vmstat.numa_hit 15.60 ± 97% -9.0% 14.20 ± 81% proc-vmstat.numa_huge_pte_updates 0.00 -100.0% 0.00 proc-vmstat.numa_interleave 59882112 +7.2% 64175117 proc-vmstat.numa_local 90528 ± 8% -8.0% 83271 ± 8% proc-vmstat.numa_other 8106 ± 56% -61.0% 3161 ±115% proc-vmstat.numa_pages_migrated 33480 ± 37% -25.8% 24831 ± 74% proc-vmstat.numa_pte_updates 7938 ± 11% +9.7% 8706 ± 4% proc-vmstat.pgactivate 0.00 -100.0% 0.00 proc-vmstat.pgalloc_dma 0.00 -100.0% 0.00 proc-vmstat.pgalloc_dma32 61541296 +7.1% 65900818 proc-vmstat.pgalloc_normal 90175093 +7.0% 96521304 proc-vmstat.pgfault 61212150 +7.1% 65565925 proc-vmstat.pgfree 8106 ± 56% -61.0% 3161 ±115% proc-vmstat.pgmigrate_success 0.00 -100.0% 0.00 proc-vmstat.pgpgin 0.00 -100.0% 0.00 proc-vmstat.pgpgout 9401 ± 2% +0.8% 9480 ± 4% proc-vmstat.pgreuse 21.80 ± 14% +10.1% 24.00 ± 20% proc-vmstat.thp_collapse_alloc 12.00 +1.7% 12.20 ± 3% proc-vmstat.thp_fault_alloc 185841 +6.8% 198421 proc-vmstat.thp_split_pmd 1.60 ±101% -62.5% 0.60 ± 81% proc-vmstat.thp_zero_page_alloc 24.60 -1.6% 24.20 proc-vmstat.unevictable_pgs_culled 7.60 ± 6% +0.0% 7.60 ± 6% proc-vmstat.unevictable_pgs_mlocked 5.20 ± 65% +0.0% 5.20 ± 65% proc-vmstat.unevictable_pgs_munlocked 3.40 ± 57% -11.8% 3.00 ± 59% proc-vmstat.unevictable_pgs_rescued 9.80 +7.5% 10.53 perf-stat.i.MPKI 1.787e+10 +0.5% 1.796e+10 perf-stat.i.branch-instructions 1.98 ± 2% +0.1 2.07 perf-stat.i.branch-miss-rate% 3.304e+08 +6.3% 3.513e+08 perf-stat.i.branch-misses 18.20 ± 3% +0.4 18.59 ± 4% perf-stat.i.cache-miss-rate% 1.575e+08 +10.8% 1.745e+08 perf-stat.i.cache-misses 9.07e+08 +8.0% 9.797e+08 perf-stat.i.cache-references 125799 ± 2% +20.2% 151216 ± 4% perf-stat.i.context-switches 2.46 +0.0% 2.46 perf-stat.i.cpi 96030 -0.0% 96027 perf-stat.i.cpu-clock 2.255e+11 -0.1% 2.254e+11 perf-stat.i.cpu-cycles 13008 +17.8% 15319 ± 3% perf-stat.i.cpu-migrations 1463 ± 2% -9.4% 1327 ± 2% perf-stat.i.cycles-between-cache-misses 0.08 ± 3% +0.0 0.09 ± 6% perf-stat.i.dTLB-load-miss-rate% 20730209 ± 4% +5.8% 21924944 ± 7% perf-stat.i.dTLB-load-misses 2.37e+10 +0.4% 2.379e+10 perf-stat.i.dTLB-loads 0.12 -0.0 0.11 perf-stat.i.dTLB-store-miss-rate% 8331280 +6.5% 8873162 perf-stat.i.dTLB-store-misses 6.682e+09 +6.8% 7.139e+09 perf-stat.i.dTLB-stores 61.43 +0.1 61.57 perf-stat.i.iTLB-load-miss-rate% 20115197 +8.1% 21749805 ± 4% perf-stat.i.iTLB-load-misses 12220026 +7.7% 13160915 perf-stat.i.iTLB-loads 8.852e+10 +0.4% 8.891e+10 perf-stat.i.instructions 4352 -6.9% 4054 ± 5% perf-stat.i.instructions-per-iTLB-miss 0.43 -0.3% 0.43 ± 3% perf-stat.i.ipc 8482 +8.6% 9215 perf-stat.i.major-faults 2.36 -0.1% 2.35 perf-stat.i.metric.GHz 0.29 ± 6% +5.6% 0.31 ± 6% perf-stat.i.metric.K/sec 514.77 +1.4% 522.01 perf-stat.i.metric.M/sec 2671826 +7.1% 2860632 perf-stat.i.minor-faults 87.67 -0.0 87.63 perf-stat.i.node-load-miss-rate% 56003720 +10.9% 62097081 ± 2% perf-stat.i.node-load-misses 6916914 ± 2% +10.1% 7614425 perf-stat.i.node-loads 83.57 +0.3 83.84 perf-stat.i.node-store-miss-rate% 23407883 +12.1% 26229322 ± 2% perf-stat.i.node-store-misses 3909193 +9.8% 4292145 perf-stat.i.node-stores 2680309 +7.1% 2869848 perf-stat.i.page-faults 96030 -0.0% 96027 perf-stat.i.task-clock 10.26 +7.6% 11.03 perf-stat.overall.MPKI 1.85 +0.1 1.96 perf-stat.overall.branch-miss-rate% 17.34 +0.4 17.77 perf-stat.overall.cache-miss-rate% 2.55 -0.5% 2.53 perf-stat.overall.cpi 1432 -9.7% 1292 perf-stat.overall.cycles-between-cache-misses 0.09 ± 3% +0.0 0.09 ± 8% perf-stat.overall.dTLB-load-miss-rate% 0.12 -0.0 0.12 perf-stat.overall.dTLB-store-miss-rate% 62.20 +0.1 62.26 perf-stat.overall.iTLB-load-miss-rate% 4397 -6.9% 4095 ± 5% perf-stat.overall.instructions-per-iTLB-miss 0.39 +0.5% 0.39 perf-stat.overall.ipc 88.99 +0.1 89.06 perf-stat.overall.node-load-miss-rate% 85.65 +0.3 85.90 perf-stat.overall.node-store-miss-rate% 1.73e+10 +0.6% 1.741e+10 perf-stat.ps.branch-instructions 3.2e+08 +6.4% 3.405e+08 perf-stat.ps.branch-misses 1.524e+08 +10.8% 1.689e+08 perf-stat.ps.cache-misses 8.792e+08 +8.1% 9.502e+08 perf-stat.ps.cache-references 122004 ± 2% +20.2% 146655 ± 4% perf-stat.ps.context-switches 93026 +0.0% 93070 perf-stat.ps.cpu-clock 2.183e+11 +0.0% 2.183e+11 perf-stat.ps.cpu-cycles 12615 +17.7% 14850 ± 3% perf-stat.ps.cpu-migrations 20082827 ± 4% +5.9% 21263864 ± 7% perf-stat.ps.dTLB-load-misses 2.295e+10 +0.5% 2.306e+10 perf-stat.ps.dTLB-loads 8076104 +6.6% 8606781 perf-stat.ps.dTLB-store-misses 6.477e+09 +6.9% 6.925e+09 perf-stat.ps.dTLB-stores 19492405 +8.2% 21089430 ± 4% perf-stat.ps.iTLB-load-misses 11847526 +7.8% 12767337 perf-stat.ps.iTLB-loads 8.572e+10 +0.5% 8.615e+10 perf-stat.ps.instructions 8236 +8.7% 8952 perf-stat.ps.major-faults 2590182 +7.1% 2774966 perf-stat.ps.minor-faults 54165096 +10.9% 60090441 ± 2% perf-stat.ps.node-load-misses 6703050 ± 2% +10.2% 7383643 perf-stat.ps.node-loads 22644577 +12.1% 25385485 ± 2% perf-stat.ps.node-store-misses 3794981 +9.8% 4166539 perf-stat.ps.node-stores 2598418 +7.1% 2783918 perf-stat.ps.page-faults 93026 +0.0% 93070 perf-stat.ps.task-clock 2.802e+12 +1.7% 2.851e+12 ± 2% perf-stat.total.instructions 153.91 ±122% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.avg 10292 ±123% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.max 0.00 +0.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.min 1247 ±122% -100.0% 0.00 sched_debug.cfs_rq:/.MIN_vruntime.stddev 8232 ± 94% -56.9% 3551 ± 26% sched_debug.cfs_rq:/.load.avg 251690 ±158% -82.2% 44870 ± 30% sched_debug.cfs_rq:/.load.max 35413 ±139% -74.5% 9039 ± 20% sched_debug.cfs_rq:/.load.stddev 400.94 ±159% -86.2% 55.43 ± 45% sched_debug.cfs_rq:/.load_avg.avg 13377 ±181% -91.8% 1101 ± 35% sched_debug.cfs_rq:/.load_avg.max 2149 ±176% -91.3% 187.72 ± 36% sched_debug.cfs_rq:/.load_avg.stddev 153.91 ±122% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.avg 10292 ±123% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.max 0.00 +0.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.min 1247 ±122% -100.0% 0.00 sched_debug.cfs_rq:/.max_vruntime.stddev 26788 ± 24% +12.5% 30136 ± 18% sched_debug.cfs_rq:/.min_vruntime.avg 43679 ± 22% +9.7% 47922 ± 16% sched_debug.cfs_rq:/.min_vruntime.max 15761 ± 28% +20.7% 19018 ± 28% sched_debug.cfs_rq:/.min_vruntime.min 4192 ± 14% -4.7% 3994 ± 13% sched_debug.cfs_rq:/.min_vruntime.stddev 0.19 ± 17% -21.2% 0.15 ± 15% sched_debug.cfs_rq:/.nr_running.avg 1.00 +0.0% 1.00 sched_debug.cfs_rq:/.nr_running.max 0.39 ± 6% -8.9% 0.36 ± 6% sched_debug.cfs_rq:/.nr_running.stddev 31.80 ± 66% -42.9% 18.14 ±106% sched_debug.cfs_rq:/.removed.load_avg.avg 1021 -41.8% 594.40 ± 81% sched_debug.cfs_rq:/.removed.load_avg.max 165.58 ± 33% -40.5% 98.46 ± 88% sched_debug.cfs_rq:/.removed.load_avg.stddev 12.13 ± 51% -39.0% 7.40 ±116% sched_debug.cfs_rq:/.removed.runnable_avg.avg 500.00 ± 8% -47.1% 264.60 ± 86% sched_debug.cfs_rq:/.removed.runnable_avg.max 69.38 ± 21% -42.4% 39.94 ± 96% sched_debug.cfs_rq:/.removed.runnable_avg.stddev 12.12 ± 51% -38.9% 7.40 ±116% sched_debug.cfs_rq:/.removed.util_avg.avg 500.00 ± 8% -47.1% 264.60 ± 86% sched_debug.cfs_rq:/.removed.util_avg.max 69.37 ± 21% -42.4% 39.94 ± 96% sched_debug.cfs_rq:/.removed.util_avg.stddev 399.13 ± 9% -0.1% 398.68 ± 15% sched_debug.cfs_rq:/.runnable_avg.avg 1431 ± 8% -4.6% 1366 ± 25% sched_debug.cfs_rq:/.runnable_avg.max 360.11 ± 7% -2.8% 350.14 ± 17% sched_debug.cfs_rq:/.runnable_avg.stddev -2403 -1.6% -2364 sched_debug.cfs_rq:/.spread0.avg 14529 ± 45% +6.2% 15424 ± 44% sched_debug.cfs_rq:/.spread0.max -13483 +0.5% -13546 sched_debug.cfs_rq:/.spread0.min 4196 ± 15% -4.8% 3994 ± 13% sched_debug.cfs_rq:/.spread0.stddev 395.71 ± 9% +0.1% 396.03 ± 15% sched_debug.cfs_rq:/.util_avg.avg 1431 ± 8% -4.6% 1365 ± 25% sched_debug.cfs_rq:/.util_avg.max 358.05 ± 7% -2.4% 349.60 ± 18% sched_debug.cfs_rq:/.util_avg.stddev 47.36 ± 25% -15.1% 40.23 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.avg 1025 +0.0% 1025 sched_debug.cfs_rq:/.util_est_enqueued.max 163.55 ± 11% -4.0% 156.95 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev 728663 ± 4% +2.1% 743737 ± 3% sched_debug.cpu.avg_idle.avg 1150618 ± 26% +24.0% 1427019 ± 26% sched_debug.cpu.avg_idle.max 2768 ± 26% -1.7% 2721 ± 41% sched_debug.cpu.avg_idle.min 323425 ± 3% -3.6% 311816 ± 5% sched_debug.cpu.avg_idle.stddev 23663 +1.2% 23951 ± 2% sched_debug.cpu.clock.avg 23669 +1.2% 23957 ± 2% sched_debug.cpu.clock.max 23657 +1.2% 23943 ± 2% sched_debug.cpu.clock.min 3.73 ± 35% +6.7% 3.98 ± 21% sched_debug.cpu.clock.stddev 23535 +1.2% 23821 ± 2% sched_debug.cpu.clock_task.avg 23653 +1.2% 23941 ± 2% sched_debug.cpu.clock_task.max 18831 ± 2% +0.0% 18836 ± 3% sched_debug.cpu.clock_task.min 559.17 ± 5% +5.5% 590.03 ± 3% sched_debug.cpu.clock_task.stddev 285.57 ± 10% -14.6% 243.91 ± 16% sched_debug.cpu.curr->pid.avg 2455 +0.4% 2464 sched_debug.cpu.curr->pid.max 744.56 ± 5% -7.2% 691.31 ± 8% sched_debug.cpu.curr->pid.stddev 508978 -0.3% 507211 sched_debug.cpu.max_idle_balance_cost.avg 967370 ± 10% -14.7% 825183 ± 10% sched_debug.cpu.max_idle_balance_cost.max 500000 +0.0% 500000 sched_debug.cpu.max_idle_balance_cost.min 60399 ± 28% -32.1% 40998 ± 41% sched_debug.cpu.max_idle_balance_cost.stddev 4294 +0.0% 4294 sched_debug.cpu.next_balance.avg 4294 +0.0% 4294 sched_debug.cpu.next_balance.max 4294 +0.0% 4294 sched_debug.cpu.next_balance.min 0.00 ± 35% -15.6% 0.00 ± 11% sched_debug.cpu.next_balance.stddev 0.14 ± 10% -15.4% 0.11 ± 15% sched_debug.cpu.nr_running.avg 1.00 +0.0% 1.00 sched_debug.cpu.nr_running.max 0.34 ± 4% -7.1% 0.32 ± 7% sched_debug.cpu.nr_running.stddev 1503 -1.1% 1488 sched_debug.cpu.nr_switches.avg 7983 ± 15% +7.0% 8542 ± 17% sched_debug.cpu.nr_switches.max 418.20 ± 12% +9.0% 456.00 ± 17% sched_debug.cpu.nr_switches.min 1139 ± 9% +4.1% 1185 ± 12% sched_debug.cpu.nr_switches.stddev 0.01 ± 89% +60.0% 0.02 ± 63% sched_debug.cpu.nr_uninterruptible.avg 40.40 ± 21% -18.8% 32.80 ± 30% sched_debug.cpu.nr_uninterruptible.max -18.60 -18.3% -15.20 sched_debug.cpu.nr_uninterruptible.min 7.42 ± 5% -7.6% 6.85 ± 15% sched_debug.cpu.nr_uninterruptible.stddev 23660 +1.2% 23947 ± 2% sched_debug.cpu_clk 996147 +0.0% 996147 sched_debug.dl_rq:.dl_bw->bw.avg 996147 +0.0% 996147 sched_debug.dl_rq:.dl_bw->bw.max 996147 +0.0% 996147 sched_debug.dl_rq:.dl_bw->bw.min 4.295e+09 +0.0% 4.295e+09 sched_debug.jiffies 23164 +1.2% 23436 ± 2% sched_debug.ktime 950.00 +0.0% 950.00 sched_debug.rt_rq:/.rt_runtime.avg 950.00 +0.0% 950.00 sched_debug.rt_rq:/.rt_runtime.max 950.00 +0.0% 950.00 sched_debug.rt_rq:/.rt_runtime.min 24018 +1.1% 24289 ± 2% sched_debug.sched_clk 1.00 +0.0% 1.00 sched_debug.sched_clock_stable() 4139835 +0.0% 4139835 sched_debug.sysctl_sched.sysctl_sched_features 24.00 +0.0% 24.00 sched_debug.sysctl_sched.sysctl_sched_latency 3.00 +0.0% 3.00 sched_debug.sysctl_sched.sysctl_sched_min_granularity 1.00 +0.0% 1.00 sched_debug.sysctl_sched.sysctl_sched_tunable_scaling 4.00 +0.0% 4.00 sched_debug.sysctl_sched.sysctl_sched_wakeup_granularity 0.00 ±200% +316.7% 0.01 ±200% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.01 ± 81% -39.3% 0.00 ±125% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.00 ±200% +750.0% 0.00 ±118% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 0.00 ±122% +173.7% 0.01 ±200% perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 0.48 ±124% -52.4% 0.23 ±165% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 ±126% -26.7% 0.00 ±123% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 +6e+98% 0.00 ±200% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 0.01 ± 24% +356.4% 0.04 ± 84% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.00 ±109% +118.2% 0.00 ± 79% perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +8e+98% 0.00 ±200% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.seq_read_iter.seq_read 0.00 ± 50% +300.0% 0.01 ± 83% perf-sched.sch_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 0.30 ±155% +589.2% 2.05 ±117% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.01 ± 23% +243.9% 0.03 ± 84% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.02 ± 14% +6.1% 0.02 ± 55% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.01 ± 84% -61.4% 0.00 ±200% perf-sched.sch_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 0.01 ± 51% +41.5% 0.01 ± 74% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 0.00 ± 14% +105.9% 0.01 ± 73% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.00 ± 11% +13.6% 0.01 perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 0.15 ± 84% +94.4% 0.28 ± 62% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 0.00 ±200% +316.7% 0.01 ±200% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.01 ± 81% -43.3% 0.00 ±125% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.00 ±200% +550.0% 0.01 ±144% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 0.01 ±124% +265.4% 0.02 ±200% perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 3.96 ±121% -38.8% 2.42 ±174% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 ±131% +5.0% 0.00 ±124% perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 +6e+98% 0.00 ±200% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 0.01 ± 24% +356.4% 0.04 ± 84% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.30 ±174% +214.1% 0.93 ±124% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +8e+98% 0.00 ±200% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.mutex_lock.seq_read_iter.seq_read 0.00 ± 50% +281.2% 0.01 ± 82% perf-sched.sch_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork 3.33 ±168% +76.4% 5.87 ±100% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.01 ± 23% +243.9% 0.03 ± 84% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.02 ± 35% -11.2% 0.02 ± 64% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.01 ± 84% -61.4% 0.00 ±200% perf-sched.sch_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 0.01 ± 53% +17.5% 0.01 ± 70% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 0.01 ± 25% +218.3% 0.04 ±116% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.03 ± 97% +1.2% 0.03 ± 87% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 3.44 +12.9% 3.88 ± 24% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork 0.02 ± 43% +21.9% 0.03 ± 42% perf-sched.total_sch_delay.average.ms 7.58 ± 67% +15.1% 8.73 ± 49% perf-sched.total_sch_delay.max.ms 4.33 ± 71% -54.5% 1.97 ±111% perf-sched.total_wait_and_delay.average.ms 626.20 ± 22% +1.6% 636.20 ± 16% perf-sched.total_wait_and_delay.count.ms 420.37 ± 60% -51.7% 202.97 ± 91% perf-sched.total_wait_and_delay.max.ms 4.31 ± 72% -54.9% 1.94 ±112% perf-sched.total_wait_time.average.ms 419.67 ± 60% -51.6% 202.95 ± 91% perf-sched.total_wait_time.max.ms 0.28 ±200% +107.4% 0.57 ±123% perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.28 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 0.66 ± 86% -28.5% 0.47 ± 78% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +1.6e+102% 1.58 ±200% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.get_signal.arch_do_signal 0.02 ±133% -40.5% 0.01 ± 98% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.03 ± 53% +39.1% 0.04 ± 24% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 1.58 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__fput.task_work_run.do_exit 1.62 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.47 ±199% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_release_kernel.perf_release 0.00 +1.3e+102% 1.26 ±200% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.00 +4.8e+100% 0.05 ±200% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 3.81 ± 50% -14.8% 3.25 ± 50% perf-sched.wait_and_delay.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 25.73 ± 54% -21.5% 20.20 ±119% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.12 ±124% -84.9% 0.02 ±200% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 1.61 ±123% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 84.12 ±126% -40.1% 50.40 ±200% perf-sched.wait_and_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 8.41 ± 48% +9.0% 9.17 ± 59% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.01 ±128% +36.5% 0.02 ±141% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 20.28 ± 66% -15.3% 17.19 ±112% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 0.40 ±200% +100.0% 0.80 ±122% perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.40 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read 9.40 ± 26% +4.3% 9.80 ± 11% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +2e+101% 0.20 ±200% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.get_signal.arch_do_signal 1.20 ±133% +116.7% 2.60 ± 52% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.20 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 271.40 ± 61% +23.1% 334.20 ± 34% perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read 0.20 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__fput.task_work_run.do_exit 0.20 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.40 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.mutex_lock.perf_event_release_kernel.perf_release 0.00 +2e+101% 0.20 ±200% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy 0.40 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 96.00 +0.2% 96.20 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 5.40 ± 66% -22.2% 4.20 ± 64% perf-sched.wait_and_delay.count.rcu_gp_kthread.kthread.ret_from_fork 13.20 ± 34% -45.5% 7.20 ± 91% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.20 ±200% -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 2.40 ± 81% -66.7% 0.80 ±200% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.40 ±122% -100.0% 0.00 perf-sched.wait_and_delay.count.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 1.80 ± 88% -55.6% 0.80 ±200% perf-sched.wait_and_delay.count.schedule_timeout.kcompactd.kthread.ret_from_fork 65.20 ± 43% -35.0% 42.40 ± 69% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 97.00 +1.6% 98.60 perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork 54.00 ± 60% -44.4% 30.00 ± 95% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 0.55 ±200% +107.6% 1.14 ±123% perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.55 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 4.68 ± 92% -28.8% 3.34 ±115% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +1.6e+102% 1.58 ±200% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.get_signal.arch_do_signal 0.03 ±124% -20.8% 0.03 ±104% perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.89 ± 75% +58.5% 1.42 ± 60% perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 1.58 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__fput.task_work_run.do_exit 1.62 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.95 ±199% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_release_kernel.perf_release 0.00 +1.3e+102% 1.26 ±200% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.00 +4.7e+102% 4.69 ±200% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 3.99 ± 50% -10.1% 3.59 ± 51% perf-sched.wait_and_delay.max.ms.rcu_gp_kthread.kthread.ret_from_fork 316.97 ± 51% -54.0% 145.80 ±112% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.00 ±200% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.47 ±129% -87.1% 0.06 ±200% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 1.61 ±123% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 201.99 ±122% -50.1% 100.81 ±200% perf-sched.wait_and_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 98.58 ± 55% -15.0% 83.84 ± 77% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.83 ±189% +48.5% 1.23 ±193% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 361.04 ± 77% -67.5% 117.45 ± 74% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork 0.81 ± 81% -30.0% 0.57 ±123% perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.54 ±122% -56.0% 0.24 ±199% perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 0.18 ± 52% +37.2% 0.24 ± 51% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +1.6e+102% 1.58 ±200% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.get_signal.arch_do_signal 0.01 ±143% -44.6% 0.01 ±117% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 +1.1e+100% 0.01 ±200% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 0.02 ± 51% +30.1% 0.03 ± 20% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 1.58 ±200% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__fput.task_work_run.do_exit 1.62 ±200% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.00 +1.8e+99% 0.00 ±200% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.step_into.walk_component 0.94 ±122% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_release_kernel.perf_release 0.00 +1.5e+100% 0.02 ±200% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.seq_read_iter.seq_read 0.00 +1.3e+102% 1.26 ±200% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy 0.00 ±200% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.00 +4.8e+100% 0.05 ±200% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 1.24 ±200% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.task_work_run.do_exit.do_group_exit 3.81 ± 50% -15.0% 3.24 ± 50% perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 25.44 ± 54% -28.6% 18.16 ±133% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.11 ±131% -87.7% 0.01 ±200% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 2.06 ± 88% -70.8% 0.60 ±200% perf-sched.wait_time.avg.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 84.11 ±126% -40.1% 50.40 ±199% perf-sched.wait_time.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 8.41 ± 49% +9.0% 9.16 ± 59% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.01 ±200% +51.2% 0.01 ±200% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 20.13 ± 67% -16.1% 16.90 ±114% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 1.62 ± 81% -30.0% 1.14 ±123% perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 1.08 ±122% -56.0% 0.48 ±199% perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 1.38 ± 68% -1.1% 1.37 ± 69% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +1.6e+102% 1.58 ±200% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.get_signal.arch_do_signal 0.03 ±125% -23.8% 0.03 ±116% perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.00 +1.1e+100% 0.01 ±200% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 0.89 ± 75% +4.2% 0.93 ± 49% perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 1.58 ±200% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__fput.task_work_run.do_exit 1.62 ±200% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.00 +1.8e+99% 0.00 ±200% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.step_into.walk_component 1.88 ±122% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_release_kernel.perf_release 0.00 +1.5e+100% 0.02 ±200% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.mutex_lock.seq_read_iter.seq_read 0.00 +1.3e+102% 1.26 ±200% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy 0.00 ±200% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.00 +4.7e+102% 4.69 ±200% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 1.24 ±200% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.task_work_run.do_exit.do_group_exit 3.99 ± 50% -10.2% 3.58 ± 51% perf-sched.wait_time.max.ms.rcu_gp_kthread.kthread.ret_from_fork 316.96 ± 51% -54.8% 143.21 ±116% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.46 ±131% -87.7% 0.06 ±200% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 2.06 ± 88% -70.8% 0.60 ±200% perf-sched.wait_time.max.ms.schedule_timeout.__skb_wait_for_more_packets.unix_dgram_recvmsg.__sys_recvfrom 201.99 ±122% -50.1% 100.80 ±199% perf-sched.wait_time.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 98.58 ± 55% -15.0% 83.83 ± 77% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.79 ±199% +50.3% 1.19 ±199% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork 359.66 ± 78% -67.4% 117.28 ± 75% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork 194313 +0.0% 194331 slabinfo.Acpi-Operand.active_objs 3470 +0.0% 3470 slabinfo.Acpi-Operand.active_slabs 194326 +0.0% 194331 slabinfo.Acpi-Operand.num_objs 3470 +0.0% 3470 slabinfo.Acpi-Operand.num_slabs 1292 ± 14% +9.0% 1409 ± 6% slabinfo.Acpi-Parse.active_objs 17.00 ± 15% +9.4% 18.60 ± 7% slabinfo.Acpi-Parse.active_slabs 1292 ± 14% +9.0% 1409 ± 6% slabinfo.Acpi-Parse.num_objs 17.00 ± 15% +9.4% 18.60 ± 7% slabinfo.Acpi-Parse.num_slabs 4821 +2.4% 4938 slabinfo.Acpi-State.active_objs 94.00 +2.8% 96.60 slabinfo.Acpi-State.active_slabs 4821 +2.4% 4938 slabinfo.Acpi-State.num_objs 94.00 +2.8% 96.60 slabinfo.Acpi-State.num_slabs 1938 ± 7% +1.2% 1961 ± 7% slabinfo.PING.active_objs 59.80 ± 7% +1.3% 60.60 ± 7% slabinfo.PING.active_slabs 1938 ± 7% +1.2% 1961 ± 7% slabinfo.PING.num_objs 59.80 ± 7% +1.3% 60.60 ± 7% slabinfo.PING.num_slabs 192.00 +0.0% 192.00 slabinfo.RAW.active_objs 6.00 +0.0% 6.00 slabinfo.RAW.active_slabs 192.00 +0.0% 192.00 slabinfo.RAW.num_objs 6.00 +0.0% 6.00 slabinfo.RAW.num_slabs 119.60 ± 10% -4.3% 114.40 ± 11% slabinfo.RAWv6.active_objs 4.60 ± 10% -4.3% 4.40 ± 11% slabinfo.RAWv6.active_slabs 119.60 ± 10% -4.3% 114.40 ± 11% slabinfo.RAWv6.num_objs 4.60 ± 10% -4.3% 4.40 ± 11% slabinfo.RAWv6.num_slabs 42.00 +0.0% 42.00 slabinfo.TCP.active_objs 3.00 +0.0% 3.00 slabinfo.TCP.active_slabs 42.00 +0.0% 42.00 slabinfo.TCP.num_objs 3.00 +0.0% 3.00 slabinfo.TCP.num_slabs 26.00 +0.0% 26.00 slabinfo.TCPv6.active_objs 2.00 +0.0% 2.00 slabinfo.TCPv6.active_slabs 26.00 +0.0% 26.00 slabinfo.TCPv6.num_objs 2.00 +0.0% 2.00 slabinfo.TCPv6.num_slabs 96.00 ± 15% +10.0% 105.60 ± 11% slabinfo.UDPv6.active_objs 4.00 ± 15% +10.0% 4.40 ± 11% slabinfo.UDPv6.active_slabs 96.00 ± 15% +10.0% 105.60 ± 11% slabinfo.UDPv6.num_objs 4.00 ± 15% +10.0% 4.40 ± 11% slabinfo.UDPv6.num_slabs 32670 ± 3% +0.7% 32896 slabinfo.anon_vma.active_objs 710.40 ± 3% +0.6% 714.80 slabinfo.anon_vma.active_slabs 32693 ± 3% +0.6% 32896 slabinfo.anon_vma.num_objs 710.40 ± 3% +0.6% 714.80 slabinfo.anon_vma.num_slabs 69646 ± 3% -3.5% 67204 slabinfo.anon_vma_chain.active_objs 1089 ± 3% -3.5% 1051 slabinfo.anon_vma_chain.active_slabs 69738 ± 3% -3.5% 67316 slabinfo.anon_vma_chain.num_objs 1089 ± 3% -3.5% 1051 slabinfo.anon_vma_chain.num_slabs 319.80 ± 14% +4.9% 335.40 ± 9% slabinfo.bdev_cache.active_objs 8.20 ± 14% +4.9% 8.60 ± 9% slabinfo.bdev_cache.active_slabs 319.80 ± 14% +4.9% 335.40 ± 9% slabinfo.bdev_cache.num_objs 8.20 ± 14% +4.9% 8.60 ± 9% slabinfo.bdev_cache.num_slabs 89.60 ± 21% -17.9% 73.60 ± 37% slabinfo.biovec-128.active_objs 5.60 ± 21% -17.9% 4.60 ± 37% slabinfo.biovec-128.active_slabs 89.60 ± 21% -17.9% 73.60 ± 37% slabinfo.biovec-128.num_objs 5.60 ± 21% -17.9% 4.60 ± 37% slabinfo.biovec-128.num_slabs 281.60 ± 16% -13.6% 243.20 ± 19% slabinfo.biovec-64.active_objs 8.80 ± 16% -13.6% 7.60 ± 19% slabinfo.biovec-64.active_slabs 281.60 ± 16% -13.6% 243.20 ± 19% slabinfo.biovec-64.num_objs 8.80 ± 16% -13.6% 7.60 ± 19% slabinfo.biovec-64.num_slabs 73.60 ± 4% +2.2% 75.20 ± 8% slabinfo.biovec-max.active_objs 9.20 ± 4% +2.2% 9.40 ± 8% slabinfo.biovec-max.active_slabs 73.60 ± 4% +2.2% 75.20 ± 8% slabinfo.biovec-max.num_objs 9.20 ± 4% +2.2% 9.40 ± 8% slabinfo.biovec-max.num_slabs 85.80 ± 18% +27.3% 109.20 ± 26% slabinfo.buffer_head.active_objs 2.20 ± 18% +27.3% 2.80 ± 26% slabinfo.buffer_head.active_slabs 85.80 ± 18% +27.3% 109.20 ± 26% slabinfo.buffer_head.num_objs 2.20 ± 18% +27.3% 2.80 ± 26% slabinfo.buffer_head.num_slabs 28228 ± 8% +1.2% 28562 ± 7% slabinfo.cred_jar.active_objs 672.80 ± 8% +1.2% 681.00 ± 7% slabinfo.cred_jar.active_slabs 28275 ± 8% +1.2% 28619 ± 7% slabinfo.cred_jar.num_objs 672.80 ± 8% +1.2% 681.00 ± 7% slabinfo.cred_jar.num_slabs 42.00 +0.0% 42.00 slabinfo.dax_cache.active_objs 1.00 +0.0% 1.00 slabinfo.dax_cache.active_slabs 42.00 +0.0% 42.00 slabinfo.dax_cache.num_objs 1.00 +0.0% 1.00 slabinfo.dax_cache.num_slabs 113561 +1.7% 115520 slabinfo.dentry.active_objs 2709 +1.7% 2755 slabinfo.dentry.active_slabs 113824 +1.7% 115756 slabinfo.dentry.num_objs 2709 +1.7% 2755 slabinfo.dentry.num_slabs 30.00 +0.0% 30.00 slabinfo.dmaengine-unmap-128.active_objs 1.00 +0.0% 1.00 slabinfo.dmaengine-unmap-128.active_slabs 30.00 +0.0% 30.00 slabinfo.dmaengine-unmap-128.num_objs 1.00 +0.0% 1.00 slabinfo.dmaengine-unmap-128.num_slabs 1365 ± 10% -3.1% 1322 ± 8% slabinfo.dmaengine-unmap-16.active_objs 32.00 ± 11% -3.1% 31.00 ± 8% slabinfo.dmaengine-unmap-16.active_slabs 1365 ± 10% -3.1% 1322 ± 8% slabinfo.dmaengine-unmap-16.num_objs 32.00 ± 11% -3.1% 31.00 ± 8% slabinfo.dmaengine-unmap-16.num_slabs 15.00 +0.0% 15.00 slabinfo.dmaengine-unmap-256.active_objs 1.00 +0.0% 1.00 slabinfo.dmaengine-unmap-256.active_slabs 15.00 +0.0% 15.00 slabinfo.dmaengine-unmap-256.num_objs 1.00 +0.0% 1.00 slabinfo.dmaengine-unmap-256.num_slabs 5150 ± 6% +3.3% 5320 slabinfo.eventpoll_pwq.active_objs 91.80 ± 6% +3.5% 95.00 slabinfo.eventpoll_pwq.active_slabs 5150 ± 6% +3.3% 5320 slabinfo.eventpoll_pwq.num_objs 91.80 ± 6% +3.5% 95.00 slabinfo.eventpoll_pwq.num_slabs 1071 ± 8% +18.9% 1273 ± 22% slabinfo.file_lock_cache.active_objs 28.60 ± 8% +18.9% 34.00 ± 22% slabinfo.file_lock_cache.active_slabs 1071 ± 8% +18.9% 1273 ± 22% slabinfo.file_lock_cache.num_objs 28.60 ± 8% +18.9% 34.00 ± 22% slabinfo.file_lock_cache.num_slabs 4624 +1.2% 4679 slabinfo.files_cache.active_objs 100.20 +1.0% 101.20 slabinfo.files_cache.active_slabs 4624 +1.2% 4679 slabinfo.files_cache.num_objs 100.20 +1.0% 101.20 slabinfo.files_cache.num_slabs 37183 ± 6% +1.9% 37891 ± 3% slabinfo.filp.active_objs 1184 ± 5% +1.4% 1200 ± 3% slabinfo.filp.active_slabs 37922 ± 5% +1.4% 38441 ± 3% slabinfo.filp.num_objs 1184 ± 5% +1.4% 1200 ± 3% slabinfo.filp.num_slabs 2647 ± 7% -10.0% 2382 ± 8% slabinfo.fsnotify_mark_connector.active_objs 20.00 ± 7% -10.0% 18.00 ± 9% slabinfo.fsnotify_mark_connector.active_slabs 2647 ± 7% -10.0% 2382 ± 8% slabinfo.fsnotify_mark_connector.num_objs 20.00 ± 7% -10.0% 18.00 ± 9% slabinfo.fsnotify_mark_connector.num_slabs 31994 +0.3% 32096 slabinfo.ftrace_event_field.active_objs 376.40 +0.3% 377.60 slabinfo.ftrace_event_field.active_slabs 31994 +0.3% 32096 slabinfo.ftrace_event_field.num_objs 376.40 +0.3% 377.60 slabinfo.ftrace_event_field.num_slabs 104.00 +0.0% 104.00 slabinfo.hugetlbfs_inode_cache.active_objs 2.00 +0.0% 2.00 slabinfo.hugetlbfs_inode_cache.active_slabs 104.00 +0.0% 104.00 slabinfo.hugetlbfs_inode_cache.num_objs 2.00 +0.0% 2.00 slabinfo.hugetlbfs_inode_cache.num_slabs 65803 +0.8% 66321 slabinfo.inode_cache.active_objs 1218 +0.8% 1228 slabinfo.inode_cache.active_slabs 65815 +0.8% 66333 slabinfo.inode_cache.num_objs 1218 +0.8% 1228 slabinfo.inode_cache.num_slabs 44.00 +0.0% 44.00 slabinfo.ip6-frags.active_objs 1.00 +0.0% 1.00 slabinfo.ip6-frags.active_slabs 44.00 +0.0% 44.00 slabinfo.ip6-frags.num_objs 1.00 +0.0% 1.00 slabinfo.ip6-frags.num_slabs 85629 +0.1% 85725 slabinfo.kernfs_node_cache.active_objs 1426 +0.1% 1428 slabinfo.kernfs_node_cache.active_slabs 85629 +0.1% 85725 slabinfo.kernfs_node_cache.num_objs 1426 +0.1% 1428 slabinfo.kernfs_node_cache.num_slabs 3280 +0.1% 3282 slabinfo.khugepaged_mm_slot.active_objs 90.60 +0.0% 90.60 slabinfo.khugepaged_mm_slot.active_slabs 3280 +0.1% 3282 slabinfo.khugepaged_mm_slot.num_objs 90.60 +0.0% 90.60 slabinfo.khugepaged_mm_slot.num_slabs 5213 +0.5% 5240 slabinfo.kmalloc-128.active_objs 163.80 +0.6% 164.80 slabinfo.kmalloc-128.active_slabs 5263 +0.4% 5284 slabinfo.kmalloc-128.num_objs 163.80 +0.6% 164.80 slabinfo.kmalloc-128.num_slabs 35019 +0.7% 35276 slabinfo.kmalloc-16.active_objs 136.60 +0.9% 137.80 slabinfo.kmalloc-16.active_slabs 35019 +0.7% 35276 slabinfo.kmalloc-16.num_objs 136.60 +0.9% 137.80 slabinfo.kmalloc-16.num_slabs 5520 +0.4% 5541 slabinfo.kmalloc-192.active_objs 131.80 +0.2% 132.00 slabinfo.kmalloc-192.active_slabs 5551 +0.2% 5559 slabinfo.kmalloc-192.num_objs 131.80 +0.2% 132.00 slabinfo.kmalloc-192.num_slabs 6442 ± 2% -2.9% 6257 ± 2% slabinfo.kmalloc-1k.active_objs 205.00 ± 2% -3.1% 198.60 ± 2% slabinfo.kmalloc-1k.active_slabs 6578 ± 2% -3.1% 6376 ± 2% slabinfo.kmalloc-1k.num_objs 205.00 ± 2% -3.1% 198.60 ± 2% slabinfo.kmalloc-1k.num_slabs 9743 ± 3% +2.3% 9963 ± 3% slabinfo.kmalloc-256.active_objs 304.80 ± 3% +2.4% 312.00 ± 3% slabinfo.kmalloc-256.active_slabs 9774 ± 3% +2.2% 9989 ± 3% slabinfo.kmalloc-256.num_objs 304.80 ± 3% +2.4% 312.00 ± 3% slabinfo.kmalloc-256.num_slabs 7757 ± 3% -0.1% 7750 ± 2% slabinfo.kmalloc-2k.active_objs 484.80 ± 3% +0.0% 484.80 ± 2% slabinfo.kmalloc-2k.active_slabs 7767 ± 3% -0.1% 7762 ± 2% slabinfo.kmalloc-2k.num_objs 484.80 ± 3% +0.0% 484.80 ± 2% slabinfo.kmalloc-2k.num_slabs 73804 ± 2% +0.5% 74176 slabinfo.kmalloc-32.active_objs 576.20 ± 2% +0.6% 579.40 slabinfo.kmalloc-32.active_slabs 73819 ± 2% +0.5% 74190 slabinfo.kmalloc-32.num_objs 576.20 ± 2% +0.6% 579.40 slabinfo.kmalloc-32.num_slabs 1936 +0.7% 1950 slabinfo.kmalloc-4k.active_objs 243.80 +0.2% 244.40 slabinfo.kmalloc-4k.active_slabs 1955 +0.2% 1960 slabinfo.kmalloc-4k.num_objs 243.80 +0.2% 244.40 slabinfo.kmalloc-4k.num_slabs 16168 ± 2% +0.8% 16298 slabinfo.kmalloc-512.active_objs 507.40 ± 2% +1.0% 512.40 slabinfo.kmalloc-512.active_slabs 16248 ± 2% +1.0% 16407 slabinfo.kmalloc-512.num_objs 507.40 ± 2% +1.0% 512.40 slabinfo.kmalloc-512.num_slabs 48426 +0.3% 48555 slabinfo.kmalloc-64.active_objs 757.80 +0.1% 758.80 slabinfo.kmalloc-64.active_slabs 48528 +0.2% 48602 slabinfo.kmalloc-64.num_objs 757.80 +0.1% 758.80 slabinfo.kmalloc-64.num_slabs 56519 +1.0% 57078 slabinfo.kmalloc-8.active_objs 110.80 +1.1% 112.00 slabinfo.kmalloc-8.active_slabs 56967 +1.0% 57551 slabinfo.kmalloc-8.num_objs 110.80 +1.1% 112.00 slabinfo.kmalloc-8.num_slabs 816.60 +0.0% 817.00 slabinfo.kmalloc-8k.active_objs 204.00 +0.0% 204.00 slabinfo.kmalloc-8k.active_slabs 816.80 +0.0% 817.00 slabinfo.kmalloc-8k.num_objs 204.00 +0.0% 204.00 slabinfo.kmalloc-8k.num_slabs 7449 ± 3% -1.2% 7361 ± 2% slabinfo.kmalloc-96.active_objs 178.40 ± 3% -0.7% 177.20 slabinfo.kmalloc-96.active_slabs 7515 ± 3% -0.6% 7469 slabinfo.kmalloc-96.num_objs 178.40 ± 3% -0.7% 177.20 slabinfo.kmalloc-96.num_slabs 428.80 ± 27% +23.9% 531.20 ± 18% slabinfo.kmalloc-rcl-128.active_objs 13.40 ± 27% +23.9% 16.60 ± 18% slabinfo.kmalloc-rcl-128.active_slabs 428.80 ± 27% +23.9% 531.20 ± 18% slabinfo.kmalloc-rcl-128.num_objs 13.40 ± 27% +23.9% 16.60 ± 18% slabinfo.kmalloc-rcl-128.num_slabs 42.00 +0.0% 42.00 slabinfo.kmalloc-rcl-192.active_objs 1.00 +0.0% 1.00 slabinfo.kmalloc-rcl-192.active_slabs 42.00 +0.0% 42.00 slabinfo.kmalloc-rcl-192.num_objs 1.00 +0.0% 1.00 slabinfo.kmalloc-rcl-192.num_slabs 1460 ± 8% +3.7% 1514 ± 9% slabinfo.kmalloc-rcl-512.active_objs 44.80 ± 8% +4.5% 46.80 ± 9% slabinfo.kmalloc-rcl-512.active_slabs 1460 ± 8% +3.7% 1514 ± 9% slabinfo.kmalloc-rcl-512.num_objs 44.80 ± 8% +4.5% 46.80 ± 9% slabinfo.kmalloc-rcl-512.num_slabs 4075 ± 3% +3.9% 4235 ± 5% slabinfo.kmalloc-rcl-64.active_objs 63.00 ± 3% +4.1% 65.60 ± 5% slabinfo.kmalloc-rcl-64.active_slabs 4075 ± 3% +3.9% 4235 ± 5% slabinfo.kmalloc-rcl-64.num_objs 63.00 ± 3% +4.1% 65.60 ± 5% slabinfo.kmalloc-rcl-64.num_slabs 1946 ± 12% +3.7% 2019 ± 14% slabinfo.kmalloc-rcl-96.active_objs 46.00 ± 12% +2.6% 47.20 ± 15% slabinfo.kmalloc-rcl-96.active_slabs 1946 ± 12% +3.7% 2019 ± 14% slabinfo.kmalloc-rcl-96.num_objs 46.00 ± 12% +2.6% 47.20 ± 15% slabinfo.kmalloc-rcl-96.num_slabs 377.60 ± 14% +3.4% 390.40 ± 6% slabinfo.kmem_cache.active_objs 11.80 ± 14% +3.4% 12.20 ± 6% slabinfo.kmem_cache.active_slabs 377.60 ± 14% +3.4% 390.40 ± 6% slabinfo.kmem_cache.num_objs 11.80 ± 14% +3.4% 12.20 ± 6% slabinfo.kmem_cache.num_slabs 793.00 ± 11% +3.5% 820.60 ± 6% slabinfo.kmem_cache_node.active_objs 13.00 ± 10% +3.1% 13.40 ± 5% slabinfo.kmem_cache_node.active_slabs 832.00 ± 10% +3.1% 857.60 ± 5% slabinfo.kmem_cache_node.num_objs 13.00 ± 10% +3.1% 13.40 ± 5% slabinfo.kmem_cache_node.num_slabs 18201 +0.1% 18228 slabinfo.lsm_file_cache.active_objs 106.40 +0.4% 106.80 slabinfo.lsm_file_cache.active_slabs 18201 +0.1% 18228 slabinfo.lsm_file_cache.num_objs 106.40 +0.4% 106.80 slabinfo.lsm_file_cache.num_slabs 3128 +0.4% 3140 slabinfo.mm_struct.active_objs 103.60 +0.6% 104.20 slabinfo.mm_struct.active_slabs 3128 +0.4% 3140 slabinfo.mm_struct.num_objs 103.60 +0.6% 104.20 slabinfo.mm_struct.num_slabs 809.20 ± 14% +5.2% 851.60 ± 4% slabinfo.mnt_cache.active_objs 15.00 ± 15% +5.3% 15.80 ± 4% slabinfo.mnt_cache.active_slabs 809.20 ± 14% +5.2% 851.60 ± 4% slabinfo.mnt_cache.num_objs 15.00 ± 15% +5.3% 15.80 ± 4% slabinfo.mnt_cache.num_slabs 34.00 +0.0% 34.00 slabinfo.mqueue_inode_cache.active_objs 1.00 +0.0% 1.00 slabinfo.mqueue_inode_cache.active_slabs 34.00 +0.0% 34.00 slabinfo.mqueue_inode_cache.num_objs 1.00 +0.0% 1.00 slabinfo.mqueue_inode_cache.num_slabs 780.00 +0.4% 783.20 slabinfo.names_cache.active_objs 97.20 +0.2% 97.40 slabinfo.names_cache.active_slabs 780.00 +0.4% 783.20 slabinfo.names_cache.num_objs 97.20 +0.2% 97.40 slabinfo.names_cache.num_slabs 36.00 +0.0% 36.00 slabinfo.nfs_read_data.active_objs 1.00 +0.0% 1.00 slabinfo.nfs_read_data.active_slabs 36.00 +0.0% 36.00 slabinfo.nfs_read_data.num_objs 1.00 +0.0% 1.00 slabinfo.nfs_read_data.num_slabs 173.60 ± 14% -7.1% 161.20 ± 18% slabinfo.numa_policy.active_objs 2.80 ± 14% -7.1% 2.60 ± 18% slabinfo.numa_policy.active_slabs 173.60 ± 14% -7.1% 161.20 ± 18% slabinfo.numa_policy.num_objs 2.80 ± 14% -7.1% 2.60 ± 18% slabinfo.numa_policy.num_slabs 9042 ± 4% +3.3% 9339 slabinfo.pde_opener.active_objs 88.20 ± 4% +3.2% 91.00 slabinfo.pde_opener.active_slabs 9042 ± 4% +3.3% 9339 slabinfo.pde_opener.num_objs 88.20 ± 4% +3.2% 91.00 slabinfo.pde_opener.num_slabs 9807 ± 12% +2.8% 10077 ± 10% slabinfo.pid.active_objs 306.00 ± 12% +2.7% 314.40 ± 10% slabinfo.pid.active_slabs 9807 ± 12% +2.8% 10078 ± 10% slabinfo.pid.num_objs 306.00 ± 12% +2.7% 314.40 ± 10% slabinfo.pid.num_slabs 1219 ± 7% +1.4% 1236 ± 8% slabinfo.pool_workqueue.active_objs 38.20 ± 6% -1.0% 37.80 ± 8% slabinfo.pool_workqueue.active_slabs 1238 ± 7% -0.1% 1236 ± 8% slabinfo.pool_workqueue.num_objs 38.20 ± 6% -1.0% 37.80 ± 8% slabinfo.pool_workqueue.num_slabs 2906 ± 2% +2.6% 2982 slabinfo.proc_dir_entry.active_objs 69.20 ± 2% +2.6% 71.00 slabinfo.proc_dir_entry.active_slabs 2906 ± 2% +2.6% 2982 slabinfo.proc_dir_entry.num_objs 69.20 ± 2% +2.6% 71.00 slabinfo.proc_dir_entry.num_slabs 13418 +6.0% 14228 ± 3% slabinfo.proc_inode_cache.active_objs 283.00 +5.6% 298.80 ± 3% slabinfo.proc_inode_cache.active_slabs 13605 +5.6% 14370 ± 3% slabinfo.proc_inode_cache.num_objs 283.00 +5.6% 298.80 ± 3% slabinfo.proc_inode_cache.num_slabs 16537 +0.7% 16645 slabinfo.radix_tree_node.active_objs 294.60 +0.7% 296.80 slabinfo.radix_tree_node.active_slabs 16537 +0.7% 16645 slabinfo.radix_tree_node.num_objs 294.60 +0.7% 296.80 slabinfo.radix_tree_node.num_slabs 32.00 +0.0% 32.00 slabinfo.request_queue.active_objs 2.00 +0.0% 2.00 slabinfo.request_queue.active_slabs 32.00 +0.0% 32.00 slabinfo.request_queue.num_objs 2.00 +0.0% 2.00 slabinfo.request_queue.num_slabs 51.00 +0.0% 51.00 slabinfo.rpc_inode_cache.active_objs 1.00 +0.0% 1.00 slabinfo.rpc_inode_cache.active_slabs 51.00 +0.0% 51.00 slabinfo.rpc_inode_cache.num_objs 1.00 +0.0% 1.00 slabinfo.rpc_inode_cache.num_slabs 640.00 +0.0% 640.00 slabinfo.scsi_sense_cache.active_objs 20.00 +0.0% 20.00 slabinfo.scsi_sense_cache.active_slabs 640.00 +0.0% 640.00 slabinfo.scsi_sense_cache.num_objs 20.00 +0.0% 20.00 slabinfo.scsi_sense_cache.num_slabs 3264 +0.0% 3264 slabinfo.seq_file.active_objs 96.00 +0.0% 96.00 slabinfo.seq_file.active_slabs 3264 +0.0% 3264 slabinfo.seq_file.num_objs 96.00 +0.0% 96.00 slabinfo.seq_file.num_slabs 5432 ± 4% +1.4% 5510 ± 2% slabinfo.shmem_inode_cache.active_objs 118.00 ± 4% +1.5% 119.80 ± 2% slabinfo.shmem_inode_cache.active_slabs 5432 ± 4% +1.4% 5510 ± 2% slabinfo.shmem_inode_cache.num_objs 118.00 ± 4% +1.5% 119.80 ± 2% slabinfo.shmem_inode_cache.num_slabs 2575 ± 2% -1.2% 2543 slabinfo.sighand_cache.active_objs 171.60 ± 2% -0.1% 171.40 slabinfo.sighand_cache.active_slabs 2580 ± 2% -0.1% 2579 ± 2% slabinfo.sighand_cache.num_objs 171.60 ± 2% -0.1% 171.40 slabinfo.sighand_cache.num_slabs 8042 ± 9% +1.3% 8146 ± 7% slabinfo.signal_cache.active_objs 289.00 ± 9% +1.1% 292.20 ± 7% slabinfo.signal_cache.active_slabs 8100 ± 9% +1.1% 8191 ± 7% slabinfo.signal_cache.num_objs 289.00 ± 9% +1.1% 292.20 ± 7% slabinfo.signal_cache.num_slabs 349.20 ± 5% -1.8% 342.80 ± 10% slabinfo.skbuff_ext_cache.active_objs 15.20 ± 2% +0.0% 15.20 ± 2% slabinfo.skbuff_ext_cache.active_slabs 486.40 ± 2% +1.2% 492.00 ± 2% slabinfo.skbuff_ext_cache.num_objs 15.20 ± 2% +0.0% 15.20 ± 2% slabinfo.skbuff_ext_cache.num_slabs 256.00 ± 15% -2.5% 249.60 ± 18% slabinfo.skbuff_fclone_cache.active_objs 8.00 ± 15% -2.5% 7.80 ± 18% slabinfo.skbuff_fclone_cache.active_slabs 256.00 ± 15% -2.5% 249.60 ± 18% slabinfo.skbuff_fclone_cache.num_objs 8.00 ± 15% -2.5% 7.80 ± 18% slabinfo.skbuff_fclone_cache.num_slabs 4729 ± 9% +1.6% 4806 ± 4% slabinfo.skbuff_head_cache.active_objs 149.20 ± 8% +0.7% 150.20 ± 4% slabinfo.skbuff_head_cache.active_slabs 4774 ± 8% +0.7% 4806 ± 4% slabinfo.skbuff_head_cache.num_objs 149.20 ± 8% +0.7% 150.20 ± 4% slabinfo.skbuff_head_cache.num_slabs 3017 ± 6% +0.7% 3039 ± 4% slabinfo.sock_inode_cache.active_objs 76.80 ± 6% +0.8% 77.40 ± 4% slabinfo.sock_inode_cache.active_slabs 3017 ± 6% +0.7% 3039 ± 4% slabinfo.sock_inode_cache.num_objs 76.80 ± 6% +0.8% 77.40 ± 4% slabinfo.sock_inode_cache.num_slabs 9633 ± 13% +3.5% 9974 ± 13% slabinfo.task_delay_info.active_objs 188.20 ± 13% +3.5% 194.80 ± 13% slabinfo.task_delay_info.active_slabs 9633 ± 13% +3.5% 9974 ± 13% slabinfo.task_delay_info.num_objs 188.20 ± 13% +3.5% 194.80 ± 13% slabinfo.task_delay_info.num_slabs 1146 ± 9% +2.8% 1178 ± 6% slabinfo.task_group.active_objs 24.20 ± 10% +2.5% 24.80 ± 6% slabinfo.task_group.active_slabs 1146 ± 9% +2.8% 1178 ± 6% slabinfo.task_group.num_objs 24.20 ± 10% +2.5% 24.80 ± 6% slabinfo.task_group.num_slabs 2295 ± 8% +2.8% 2358 ± 3% slabinfo.task_struct.active_objs 2298 ± 8% +2.8% 2361 ± 3% slabinfo.task_struct.active_slabs 2298 ± 8% +2.8% 2361 ± 3% slabinfo.task_struct.num_objs 2298 ± 8% +2.8% 2361 ± 3% slabinfo.task_struct.num_slabs 43.00 +0.0% 43.00 slabinfo.taskstats.active_objs 0.00 -100.0% 0.00 slabinfo.taskstats.active_slabs 43.00 +0.0% 43.00 slabinfo.taskstats.num_objs 0.00 -100.0% 0.00 slabinfo.taskstats.num_slabs 3512 ± 2% +0.0% 3512 ± 2% slabinfo.trace_event_file.active_objs 75.60 ± 2% +0.0% 75.60 ± 2% slabinfo.trace_event_file.active_slabs 3512 ± 2% +0.0% 3512 ± 2% slabinfo.trace_event_file.num_objs 75.60 ± 2% +0.0% 75.60 ± 2% slabinfo.trace_event_file.num_slabs 33.00 +0.0% 33.00 slabinfo.tw_sock_TCP.active_objs 1.00 +0.0% 1.00 slabinfo.tw_sock_TCP.active_slabs 33.00 +0.0% 33.00 slabinfo.tw_sock_TCP.num_objs 1.00 +0.0% 1.00 slabinfo.tw_sock_TCP.num_slabs 59051 +0.6% 59420 slabinfo.vm_area_struct.active_objs 1478 +0.6% 1487 slabinfo.vm_area_struct.active_slabs 59148 +0.6% 59510 slabinfo.vm_area_struct.num_objs 1478 +0.6% 1487 slabinfo.vm_area_struct.num_slabs 20826 +1.8% 21209 ± 4% slabinfo.vmap_area.active_objs 368.00 +2.0% 375.20 ± 3% slabinfo.vmap_area.active_slabs 23589 +1.9% 24044 ± 3% slabinfo.vmap_area.num_objs 368.00 +2.0% 375.20 ± 3% slabinfo.vmap_area.num_slabs 42.00 +0.0% 42.00 slabinfo.xfrm_state.active_objs 1.00 +0.0% 1.00 slabinfo.xfrm_state.active_slabs 42.00 +0.0% 42.00 slabinfo.xfrm_state.num_objs 1.00 +0.0% 1.00 slabinfo.xfrm_state.num_slabs 72.00 +0.0% 72.00 slabinfo.xfs_btree_cur.active_objs 2.00 +0.0% 2.00 slabinfo.xfs_btree_cur.active_slabs 72.00 +0.0% 72.00 slabinfo.xfs_btree_cur.num_objs 2.00 +0.0% 2.00 slabinfo.xfs_btree_cur.num_slabs 201.60 ± 8% -4.2% 193.20 ± 10% slabinfo.xfs_buf.active_objs 4.80 ± 8% -4.2% 4.60 ± 10% slabinfo.xfs_buf.active_slabs 201.60 ± 8% -4.2% 193.20 ± 10% slabinfo.xfs_buf.num_objs 4.80 ± 8% -4.2% 4.60 ± 10% slabinfo.xfs_buf.num_slabs 38.00 +0.0% 38.00 slabinfo.xfs_efi_item.active_objs 1.00 +0.0% 1.00 slabinfo.xfs_efi_item.active_slabs 38.00 +0.0% 38.00 slabinfo.xfs_efi_item.num_objs 1.00 +0.0% 1.00 slabinfo.xfs_efi_item.num_slabs 96.00 +0.0% 96.00 slabinfo.xfs_inode.active_objs 3.00 +0.0% 3.00 slabinfo.xfs_inode.active_slabs 96.00 +0.0% 96.00 slabinfo.xfs_inode.num_objs 3.00 +0.0% 3.00 slabinfo.xfs_inode.num_slabs 454.60 -2.1% 445.20 ± 3% softirqs.BLOCK 2.80 ±200% +0.0% 2.80 ±200% softirqs.CPU0.BLOCK 1.00 +0.0% 1.00 softirqs.CPU0.HI 0.20 ±200% +600.0% 1.40 ±166% softirqs.CPU0.NET_RX 0.00 +6e+101% 0.60 ±200% softirqs.CPU0.NET_TX 13781 ± 10% -1.4% 13584 ± 14% softirqs.CPU0.RCU 7448 ± 6% +7.8% 8028 ± 7% softirqs.CPU0.SCHED 113.40 -0.7% 112.60 softirqs.CPU0.TASKLET 1240 ± 36% -20.2% 990.00 ± 51% softirqs.CPU0.TIMER 0.00 +7.2e+103% 72.00 ±200% softirqs.CPU1.BLOCK 0.60 ± 81% +2666.7% 16.60 ±185% softirqs.CPU1.NET_RX 0.20 ±200% +200.0% 0.60 ±200% softirqs.CPU1.NET_TX 9946 ± 4% +7.2% 10665 ± 3% softirqs.CPU1.RCU 6409 ± 4% +13.2% 7256 ± 9% softirqs.CPU1.SCHED 0.00 -100.0% 0.00 softirqs.CPU1.TASKLET 78.40 ± 17% +287.5% 303.80 ±130% softirqs.CPU1.TIMER 0.20 ±200% -100.0% 0.00 softirqs.CPU10.BLOCK 1.60 ± 30% -25.0% 1.20 ± 33% softirqs.CPU10.HI 23.20 ±164% -98.3% 0.40 ±200% softirqs.CPU10.NET_RX 1.20 ±161% -100.0% 0.00 softirqs.CPU10.NET_TX 9744 ± 5% +4.1% 10147 ± 7% softirqs.CPU10.RCU 5452 ± 5% +6.3% 5794 ± 3% softirqs.CPU10.SCHED 54.60 +2.6% 56.00 softirqs.CPU10.TASKLET 89.20 ± 32% -32.7% 60.00 ± 10% softirqs.CPU10.TIMER 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU11.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU11.NET_TX 9644 ± 3% +9.7% 10579 ± 4% softirqs.CPU11.RCU 5556 ± 4% +5.1% 5840 ± 3% softirqs.CPU11.SCHED 0.00 +4e+101% 0.40 ±200% softirqs.CPU11.TASKLET 71.40 ± 26% +67.5% 119.60 ±100% softirqs.CPU11.TIMER 0.00 +2.6e+102% 2.60 ±181% softirqs.CPU12.BLOCK 0.40 ±122% +900.0% 4.00 ±142% softirqs.CPU12.NET_RX 0.40 ±122% +150.0% 1.00 ±109% softirqs.CPU12.NET_TX 9710 ± 4% +4.8% 10174 ± 2% softirqs.CPU12.RCU 5495 ± 2% +6.5% 5854 ± 2% softirqs.CPU12.SCHED 0.40 ±200% -100.0% 0.00 softirqs.CPU12.TASKLET 44.20 ± 5% +30.3% 57.60 ± 33% softirqs.CPU12.TIMER 2.60 ±181% -100.0% 0.00 softirqs.CPU13.BLOCK 27.80 ±168% -96.4% 1.00 ± 89% softirqs.CPU13.NET_RX 0.60 ± 81% -33.3% 0.40 ±200% softirqs.CPU13.NET_TX 9659 ± 5% +8.3% 10458 ± 3% softirqs.CPU13.RCU 5546 ± 2% +5.7% 5864 ± 2% softirqs.CPU13.SCHED 1.60 ± 50% +0.0% 1.60 ± 50% softirqs.CPU13.TASKLET 57.40 ± 20% -13.2% 49.80 ± 15% softirqs.CPU13.TIMER 4.80 ±200% -95.8% 0.20 ±200% softirqs.CPU14.BLOCK 1.60 ±170% -87.5% 0.20 ±200% softirqs.CPU14.NET_RX 0.40 ±122% -50.0% 0.20 ±200% softirqs.CPU14.NET_TX 9555 ± 4% +5.6% 10093 ± 6% softirqs.CPU14.RCU 5573 ± 3% +4.8% 5838 softirqs.CPU14.SCHED 2.00 +0.0% 2.00 softirqs.CPU14.TASKLET 59.60 ± 30% +5.4% 62.80 ± 20% softirqs.CPU14.TIMER 1.60 ±200% -100.0% 0.00 softirqs.CPU15.BLOCK 0.60 ± 81% -66.7% 0.20 ±200% softirqs.CPU15.NET_RX 0.40 ±122% -50.0% 0.20 ±200% softirqs.CPU15.NET_TX 9979 ± 4% +4.3% 10410 ± 3% softirqs.CPU15.RCU 5584 ± 3% +5.0% 5863 ± 2% softirqs.CPU15.SCHED 2.20 ± 18% -9.1% 2.00 softirqs.CPU15.TASKLET 66.40 ± 30% -9.9% 59.80 ± 24% softirqs.CPU15.TIMER 0.20 ±200% +1200.0% 2.60 ±118% softirqs.CPU16.NET_RX 9828 ± 4% +6.4% 10461 ± 3% softirqs.CPU16.RCU 5513 +5.3% 5805 ± 2% softirqs.CPU16.SCHED 1.60 ± 50% +0.0% 1.60 ± 50% softirqs.CPU16.TASKLET 100.00 ± 43% -37.6% 62.40 ± 22% softirqs.CPU16.TIMER 57.60 ±200% +37.2% 79.00 ±185% softirqs.CPU17.BLOCK 0.80 ±145% -50.0% 0.40 ±122% softirqs.CPU17.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU17.NET_TX 9556 ± 7% +11.8% 10686 ± 2% softirqs.CPU17.RCU 5434 ± 4% +9.0% 5924 ± 5% softirqs.CPU17.SCHED 1.60 ± 50% -50.0% 0.80 ±122% softirqs.CPU17.TASKLET 84.20 ± 34% -16.9% 70.00 ± 44% softirqs.CPU17.TIMER 1.20 ±200% +4033.3% 49.60 ±192% softirqs.CPU18.BLOCK 26.60 ±196% -95.5% 1.20 ±133% softirqs.CPU18.NET_RX 0.60 ±133% +0.0% 0.60 ±133% softirqs.CPU18.NET_TX 9709 ± 5% +10.1% 10689 ± 4% softirqs.CPU18.RCU 5506 ± 2% +6.5% 5863 ± 4% softirqs.CPU18.SCHED 0.80 ±122% -50.0% 0.40 ±200% softirqs.CPU18.TASKLET 89.40 ± 72% -5.8% 84.20 ± 22% softirqs.CPU18.TIMER 170.20 ± 99% -74.6% 43.20 ±187% softirqs.CPU19.BLOCK 1.80 ±173% -55.6% 0.80 ±145% softirqs.CPU19.NET_RX 0.40 ±122% -100.0% 0.00 softirqs.CPU19.NET_TX 9859 ± 6% +9.3% 10771 ± 2% softirqs.CPU19.RCU 5236 ± 5% +11.4% 5835 ± 4% softirqs.CPU19.SCHED 91.80 ± 39% -35.5% 59.20 ± 31% softirqs.CPU19.TIMER 3.60 ±200% -55.6% 1.60 ±200% softirqs.CPU2.BLOCK 33.60 ±197% -98.8% 0.40 ±122% softirqs.CPU2.NET_RX 0.60 ± 81% -66.7% 0.20 ±200% softirqs.CPU2.NET_TX 9703 ± 5% +8.5% 10529 ± 5% softirqs.CPU2.RCU 5536 ± 10% +11.4% 6166 ± 4% softirqs.CPU2.SCHED 0.60 ±133% +0.0% 0.60 ±200% softirqs.CPU2.TASKLET 159.40 ± 57% +17.8% 187.80 ± 73% softirqs.CPU2.TIMER 2.40 ±200% +1508.3% 38.60 ±200% softirqs.CPU20.BLOCK 0.40 ±122% +6900.0% 28.00 ±200% softirqs.CPU20.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU20.NET_TX 10238 ± 9% +5.8% 10836 ± 3% softirqs.CPU20.RCU 5617 ± 2% +6.2% 5967 ± 2% softirqs.CPU20.SCHED 96.20 ± 83% -6.4% 90.00 ± 43% softirqs.CPU20.TIMER 2.60 ±200% -92.3% 0.20 ±200% softirqs.CPU21.BLOCK 2.20 ±155% -45.5% 1.20 ± 81% softirqs.CPU21.NET_RX 0.40 ±122% -100.0% 0.00 softirqs.CPU21.NET_TX 9959 ± 6% +7.5% 10705 ± 2% softirqs.CPU21.RCU 5495 +8.7% 5976 ± 4% softirqs.CPU21.SCHED 1.80 ±200% -100.0% 0.00 softirqs.CPU21.TASKLET 62.20 ± 22% +41.2% 87.80 ± 43% softirqs.CPU21.TIMER 0.00 +5.6e+102% 5.60 ±200% softirqs.CPU22.BLOCK 0.40 ±200% +200.0% 1.20 ±122% softirqs.CPU22.NET_RX 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU22.NET_TX 9915 ± 8% +7.6% 10673 ± 2% softirqs.CPU22.RCU 5420 ± 3% +4.8% 5678 ± 8% softirqs.CPU22.SCHED 0.20 ±200% -100.0% 0.00 softirqs.CPU22.TASKLET 52.00 ± 11% +262.7% 188.60 ± 56% softirqs.CPU22.TIMER 2.40 ±200% -100.0% 0.00 softirqs.CPU23.BLOCK 0.60 ± 81% +33.3% 0.80 ± 93% softirqs.CPU23.NET_RX 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU23.NET_TX 9918 ± 6% +5.8% 10491 ± 2% softirqs.CPU23.RCU 5471 +7.1% 5857 ± 4% softirqs.CPU23.SCHED 0.60 ±133% +800.0% 5.40 ±200% softirqs.CPU23.TASKLET 68.40 ± 8% +19.9% 82.00 ± 39% softirqs.CPU23.TIMER 0.00 +4e+101% 0.40 ±200% softirqs.CPU24.BLOCK 0.40 ±122% +0.0% 0.40 ±200% softirqs.CPU24.NET_RX 0.00 -100.0% 0.00 softirqs.CPU24.NET_TX 10028 ± 5% +7.8% 10813 ± 14% softirqs.CPU24.RCU 5435 ± 8% +9.5% 5950 ± 4% softirqs.CPU24.SCHED 321.60 ±131% +74.9% 562.60 ± 90% softirqs.CPU24.TIMER 0.00 +6.4e+102% 6.40 ±200% softirqs.CPU25.BLOCK 10338 ± 6% -1.5% 10179 ± 6% softirqs.CPU25.RCU 5717 ± 9% +1.7% 5816 softirqs.CPU25.SCHED 544.40 ±101% -79.8% 110.00 ± 24% softirqs.CPU25.TIMER 0.40 ±200% +1200.0% 5.20 ±200% softirqs.CPU26.BLOCK 3.00 ±122% -93.3% 0.20 ±200% softirqs.CPU26.NET_RX 0.40 ±122% -100.0% 0.00 softirqs.CPU26.NET_TX 9837 ± 4% +2.2% 10057 ± 5% softirqs.CPU26.RCU 5230 ± 4% +8.4% 5672 ± 2% softirqs.CPU26.SCHED 2.00 +0.0% 2.00 softirqs.CPU26.TASKLET 89.00 ± 13% +26.7% 112.80 ± 24% softirqs.CPU26.TIMER 19.00 ±200% -72.6% 5.20 ±200% softirqs.CPU27.BLOCK 0.20 ±200% +600.0% 1.40 ±166% softirqs.CPU27.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU27.NET_TX 9807 ± 3% +0.5% 9861 ± 5% softirqs.CPU27.RCU 5296 +10.4% 5845 softirqs.CPU27.SCHED 2.20 ± 18% -9.1% 2.00 softirqs.CPU27.TASKLET 99.80 ± 17% +1.0% 100.80 ± 28% softirqs.CPU27.TIMER 0.00 +6e+101% 0.60 ±200% softirqs.CPU28.BLOCK 33.20 ±200% -94.6% 1.80 ±200% softirqs.CPU28.NET_RX 0.00 +1.2e+102% 1.20 ±200% softirqs.CPU28.NET_TX 9607 ± 7% +3.1% 9903 ± 4% softirqs.CPU28.RCU 5286 ± 2% +10.9% 5859 ± 2% softirqs.CPU28.SCHED 2.00 -20.0% 1.60 ± 50% softirqs.CPU28.TASKLET 112.00 ± 39% -13.9% 96.40 ± 24% softirqs.CPU28.TIMER 0.00 +4e+101% 0.40 ±200% softirqs.CPU29.BLOCK 2.60 ±181% -15.4% 2.20 ±178% softirqs.CPU29.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU29.NET_TX 9718 ± 4% +0.6% 9774 ± 5% softirqs.CPU29.RCU 5346 ± 2% +8.3% 5790 ± 2% softirqs.CPU29.SCHED 1.60 ± 50% +0.0% 1.60 ± 50% softirqs.CPU29.TASKLET 111.80 ± 39% -0.9% 110.80 ± 21% softirqs.CPU29.TIMER 1.20 ±200% -100.0% 0.00 softirqs.CPU3.BLOCK 1.40 ± 34% -85.7% 0.20 ±200% softirqs.CPU3.NET_RX 0.40 ±122% -100.0% 0.00 softirqs.CPU3.NET_TX 10506 ± 16% -1.3% 10375 ± 3% softirqs.CPU3.RCU 5598 ± 3% +5.9% 5928 ± 4% softirqs.CPU3.SCHED 0.80 ±145% -100.0% 0.00 softirqs.CPU3.TASKLET 335.60 ±119% -78.3% 72.80 ± 32% softirqs.CPU3.TIMER 0.00 +2.3e+103% 22.80 ±146% softirqs.CPU30.BLOCK 0.00 +3.6e+102% 3.60 ±122% softirqs.CPU30.NET_RX 0.00 +2.4e+102% 2.40 ±122% softirqs.CPU30.NET_TX 9928 ± 6% +1.1% 10034 ± 4% softirqs.CPU30.RCU 5311 +9.8% 5834 ± 2% softirqs.CPU30.SCHED 1.80 ± 22% -33.3% 1.20 ± 81% softirqs.CPU30.TASKLET 83.20 ± 29% +44.0% 119.80 ± 53% softirqs.CPU30.TIMER 0.00 +1.5e+103% 14.60 ±127% softirqs.CPU31.BLOCK 0.00 +8e+101% 0.80 ±200% softirqs.CPU31.NET_RX 10070 ± 6% -2.7% 9794 ± 5% softirqs.CPU31.RCU 5350 ± 3% +9.4% 5851 softirqs.CPU31.SCHED 1.80 ± 22% +22.2% 2.20 ± 18% softirqs.CPU31.TASKLET 80.80 ± 18% +20.8% 97.60 ± 28% softirqs.CPU31.TIMER 93.00 ±200% -97.8% 2.00 ±126% softirqs.CPU32.NET_RX 0.00 +4e+101% 0.40 ±200% softirqs.CPU32.NET_TX 9527 ± 4% +0.5% 9571 ± 7% softirqs.CPU32.RCU 5308 +9.9% 5832 softirqs.CPU32.SCHED 3.80 ± 94% -21.1% 3.00 ± 66% softirqs.CPU32.TASKLET 82.20 ± 21% +10.9% 91.20 ± 24% softirqs.CPU32.TIMER 1.80 ±200% -100.0% 0.00 softirqs.CPU33.BLOCK 9541 ± 4% +1.1% 9645 ± 4% softirqs.CPU33.RCU 5395 ± 2% +7.7% 5810 ± 2% softirqs.CPU33.SCHED 2.00 +0.0% 2.00 softirqs.CPU33.TASKLET 87.20 ± 20% -5.0% 82.80 ± 28% softirqs.CPU33.TIMER 0.00 +6e+102% 6.00 ±191% softirqs.CPU34.BLOCK 0.80 ±200% -100.0% 0.00 softirqs.CPU34.NET_RX 0.00 -100.0% 0.00 softirqs.CPU34.NET_TX 9528 ± 4% +2.8% 9791 ± 10% softirqs.CPU34.RCU 5292 ± 2% +8.6% 5745 ± 2% softirqs.CPU34.SCHED 0.40 ±200% +450.0% 2.20 ± 18% softirqs.CPU34.TASKLET 71.00 ± 6% +38.6% 98.40 ± 36% softirqs.CPU34.TIMER 0.80 ±200% -100.0% 0.00 softirqs.CPU35.NET_RX 9437 ± 3% -0.1% 9426 ± 6% softirqs.CPU35.RCU 5240 +9.6% 5742 ± 3% softirqs.CPU35.SCHED 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU35.TASKLET 75.40 ± 14% +8.0% 81.40 ± 22% softirqs.CPU35.TIMER 0.00 +9.2e+102% 9.20 ±200% softirqs.CPU36.BLOCK 0.00 +8.8e+102% 8.80 ±163% softirqs.CPU36.NET_RX 9729 ± 5% -2.6% 9479 ± 6% softirqs.CPU36.RCU 5286 ± 2% +9.9% 5810 softirqs.CPU36.SCHED 102.40 ± 20% -27.9% 73.80 ± 11% softirqs.CPU36.TIMER 0.20 ±200% +200.0% 0.60 ±200% softirqs.CPU37.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU37.NET_TX 9514 ± 3% +2.3% 9736 ± 5% softirqs.CPU37.RCU 5313 +11.3% 5913 softirqs.CPU37.SCHED 96.60 ± 31% -31.5% 66.20 ± 11% softirqs.CPU37.TIMER 1.00 ±154% -100.0% 0.00 softirqs.CPU38.NET_RX 9429 ± 4% -0.3% 9406 ± 6% softirqs.CPU38.RCU 5297 +8.7% 5758 softirqs.CPU38.SCHED 83.80 ± 14% -0.2% 83.60 ± 12% softirqs.CPU38.TIMER 3.20 ±184% -100.0% 0.00 softirqs.CPU39.BLOCK 1.20 ±200% -100.0% 0.00 softirqs.CPU39.NET_RX 9430 ± 3% -0.0% 9426 ± 5% softirqs.CPU39.RCU 5317 ± 3% +9.5% 5824 ± 2% softirqs.CPU39.SCHED 2.60 ±200% -30.8% 1.80 ±200% softirqs.CPU39.TASKLET 99.40 ± 20% +16.7% 116.00 ± 69% softirqs.CPU39.TIMER 2.40 ±133% -50.0% 1.20 ±161% softirqs.CPU4.NET_RX 0.80 ±200% -100.0% 0.00 softirqs.CPU4.NET_TX 9491 ± 4% +6.5% 10113 ± 6% softirqs.CPU4.RCU 5526 ± 2% +5.4% 5825 ± 4% softirqs.CPU4.SCHED 6.60 ±200% -87.9% 0.80 ±122% softirqs.CPU4.TASKLET 79.80 ± 33% +3.5% 82.60 ± 47% softirqs.CPU4.TIMER 2.20 ±142% -36.4% 1.40 ±166% softirqs.CPU40.NET_RX 0.00 -100.0% 0.00 softirqs.CPU40.NET_TX 9638 ± 7% -2.5% 9400 ± 5% softirqs.CPU40.RCU 5193 ± 3% +8.2% 5619 ± 7% softirqs.CPU40.SCHED 0.00 +4.4e+102% 4.40 ±200% softirqs.CPU40.TASKLET 71.80 ± 10% +122.3% 159.60 ±100% softirqs.CPU40.TIMER 0.00 +8.6e+102% 8.60 ±200% softirqs.CPU41.BLOCK 0.00 +5.6e+103% 55.80 ±167% softirqs.CPU41.NET_RX 9401 ± 4% +0.5% 9448 ± 4% softirqs.CPU41.RCU 5270 +12.1% 5906 ± 3% softirqs.CPU41.SCHED 1.00 ±200% -100.0% 0.00 softirqs.CPU41.TASKLET 84.80 ± 6% +23.6% 104.80 ± 43% softirqs.CPU41.TIMER 9544 ± 4% -0.7% 9478 ± 6% softirqs.CPU42.RCU 5346 +7.4% 5744 softirqs.CPU42.SCHED 0.80 ±200% -100.0% 0.00 softirqs.CPU42.TASKLET 87.20 ± 19% -20.0% 69.80 ± 14% softirqs.CPU42.TIMER 7.00 ±200% -100.0% 0.00 softirqs.CPU43.BLOCK 0.00 +1.2e+103% 11.80 ±191% softirqs.CPU43.NET_RX 9456 ± 3% +0.1% 9464 ± 6% softirqs.CPU43.RCU 5286 +8.2% 5721 ± 2% softirqs.CPU43.SCHED 89.20 ± 25% -6.1% 83.80 ± 17% softirqs.CPU43.TIMER 0.00 +5e+102% 5.00 ±200% softirqs.CPU44.BLOCK 0.20 ±200% +900.0% 2.00 ±176% softirqs.CPU44.NET_RX 0.00 +1e+102% 1.00 ±200% softirqs.CPU44.NET_TX 9496 ± 4% +1.3% 9620 ± 7% softirqs.CPU44.RCU 5340 ± 2% +5.8% 5652 ± 3% softirqs.CPU44.SCHED 1.00 ±200% -40.0% 0.60 ±133% softirqs.CPU44.TASKLET 80.00 ± 24% +22.5% 98.00 ± 43% softirqs.CPU44.TIMER 9507 ± 4% +2.8% 9776 ± 10% softirqs.CPU45.RCU 5363 +9.1% 5853 ± 2% softirqs.CPU45.SCHED 8.60 ±200% -100.0% 0.00 softirqs.CPU45.TASKLET 104.80 ± 43% -1.7% 103.00 ± 23% softirqs.CPU45.TIMER 8.60 ±200% -100.0% 0.00 softirqs.CPU46.BLOCK 0.00 +8e+101% 0.80 ±200% softirqs.CPU46.NET_RX 9406 ± 4% +5.2% 9895 ± 7% softirqs.CPU46.RCU 5306 ± 2% +13.9% 6043 ± 7% softirqs.CPU46.SCHED 1.40 ±200% -85.7% 0.20 ±200% softirqs.CPU46.TASKLET 97.40 ± 21% +192.2% 284.60 ±147% softirqs.CPU46.TIMER 0.40 ±200% +0.0% 0.40 ±200% softirqs.CPU47.BLOCK 9396 ± 4% +0.9% 9480 ± 5% softirqs.CPU47.RCU 4820 ± 4% +11.5% 5374 ± 6% softirqs.CPU47.SCHED 0.00 -100.0% 0.00 softirqs.CPU47.TASKLET 90.40 ± 17% -4.0% 86.80 ± 25% softirqs.CPU47.TIMER 0.00 +2.4e+102% 2.40 ±200% softirqs.CPU48.BLOCK 1.40 ± 57% -71.4% 0.40 ±200% softirqs.CPU48.NET_RX 1.00 ± 63% -100.0% 0.00 softirqs.CPU48.NET_TX 9293 ± 3% +10.6% 10278 ± 4% softirqs.CPU48.RCU 5274 ± 2% +7.4% 5665 ± 3% softirqs.CPU48.SCHED 9.80 ±129% -59.2% 4.00 ±187% softirqs.CPU48.TASKLET 57.60 ± 24% +140.6% 138.60 ± 43% softirqs.CPU48.TIMER 2.80 ±200% -42.9% 1.60 ±200% softirqs.CPU49.BLOCK 3.20 ±170% -50.0% 1.60 ±115% softirqs.CPU49.NET_RX 1.20 ±161% -50.0% 0.60 ±200% softirqs.CPU49.NET_TX 9653 ± 6% +7.2% 10350 ± 6% softirqs.CPU49.RCU 5536 ± 2% +7.4% 5944 ± 2% softirqs.CPU49.SCHED 3.60 ±200% -100.0% 0.00 softirqs.CPU49.TASKLET 111.20 ± 48% +1.3% 112.60 ± 89% softirqs.CPU49.TIMER 0.20 ±200% +400.0% 1.00 ±154% softirqs.CPU5.NET_RX 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU5.NET_TX 9907 ± 6% +5.1% 10409 ± 4% softirqs.CPU5.RCU 5804 ± 7% +1.5% 5889 ± 2% softirqs.CPU5.SCHED 434.60 ± 92% -84.3% 68.20 ± 31% softirqs.CPU5.TIMER 1.20 ±200% -100.0% 0.00 softirqs.CPU50.BLOCK 0.40 ±122% +700.0% 3.20 ±110% softirqs.CPU50.NET_RX 0.00 +1e+102% 1.00 ±154% softirqs.CPU50.NET_TX 9479 ± 3% +10.7% 10491 ± 4% softirqs.CPU50.RCU 5523 ± 5% +10.5% 6102 ± 4% softirqs.CPU50.SCHED 0.40 ±200% -100.0% 0.00 softirqs.CPU50.TASKLET 131.80 ±102% +110.6% 277.60 ±130% softirqs.CPU50.TIMER 2.00 ±154% +180.0% 5.60 ±200% softirqs.CPU51.BLOCK 1.20 ± 97% +100.0% 2.40 ±159% softirqs.CPU51.NET_RX 0.00 +4e+101% 0.40 ±122% softirqs.CPU51.NET_TX 9832 ± 7% +3.0% 10129 ± 4% softirqs.CPU51.RCU 5580 ± 2% +2.6% 5723 ± 6% softirqs.CPU51.SCHED 0.60 ±133% +2000.0% 12.60 ±150% softirqs.CPU51.TASKLET 118.80 ± 54% -38.2% 73.40 ± 34% softirqs.CPU51.TIMER 1.60 ±200% +0.0% 1.60 ±200% softirqs.CPU52.BLOCK 1.40 ±166% -85.7% 0.20 ±200% softirqs.CPU52.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU52.NET_TX 9434 ± 4% +7.7% 10163 ± 4% softirqs.CPU52.RCU 5609 +7.6% 6033 ± 3% softirqs.CPU52.SCHED 0.00 -100.0% 0.00 softirqs.CPU52.TASKLET 91.20 ± 56% +9.9% 100.20 ± 46% softirqs.CPU52.TIMER 5.20 ±200% -96.2% 0.20 ±200% softirqs.CPU53.BLOCK 0.80 ±122% +1125.0% 9.80 ±200% softirqs.CPU53.NET_RX 0.60 ±133% -66.7% 0.20 ±200% softirqs.CPU53.NET_TX 9428 ± 5% +8.2% 10202 ± 4% softirqs.CPU53.RCU 5340 ± 5% +11.7% 5962 ± 2% softirqs.CPU53.SCHED 0.40 ±200% -100.0% 0.00 softirqs.CPU53.TASKLET 62.60 ± 30% +24.9% 78.20 ± 41% softirqs.CPU53.TIMER 2.00 ±154% +0.0% 2.00 ±151% softirqs.CPU54.BLOCK 26.00 ±186% -98.5% 0.40 ±122% softirqs.CPU54.NET_RX 0.40 ±122% -100.0% 0.00 softirqs.CPU54.NET_TX 9565 ± 4% +7.8% 10313 ± 3% softirqs.CPU54.RCU 5593 +6.3% 5943 ± 3% softirqs.CPU54.SCHED 0.20 ±200% -100.0% 0.00 softirqs.CPU54.TASKLET 69.20 ± 23% +37.3% 95.00 ± 66% softirqs.CPU54.TIMER 1.60 ±200% -12.5% 1.40 ±200% softirqs.CPU55.BLOCK 1.60 ±170% +50.0% 2.40 ±140% softirqs.CPU55.NET_RX 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU55.NET_TX 9414 ± 3% +7.3% 10104 ± 3% softirqs.CPU55.RCU 5480 ± 3% +8.6% 5950 ± 4% softirqs.CPU55.SCHED 0.60 ±133% -100.0% 0.00 softirqs.CPU55.TASKLET 85.20 ± 60% +67.6% 142.80 ± 89% softirqs.CPU55.TIMER 2.80 ±200% -7.1% 2.60 ±181% softirqs.CPU56.BLOCK 1.80 ±150% -44.4% 1.00 ±154% softirqs.CPU56.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU56.NET_TX 9380 ± 6% +12.3% 10538 ± 5% softirqs.CPU56.RCU 5584 ± 9% +7.4% 5999 ± 4% softirqs.CPU56.SCHED 0.00 -100.0% 0.00 softirqs.CPU56.TASKLET 335.20 ±119% -80.1% 66.60 ± 37% softirqs.CPU56.TIMER 1.60 ±200% -100.0% 0.00 softirqs.CPU57.BLOCK 0.60 ±133% +15366.7% 92.80 ±200% softirqs.CPU57.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU57.NET_TX 9278 ± 4% +8.2% 10042 ± 3% softirqs.CPU57.RCU 5451 ± 2% +7.6% 5868 ± 4% softirqs.CPU57.SCHED 54.00 ± 27% +3.7% 56.00 ± 21% softirqs.CPU57.TIMER 2.80 ±200% +100.0% 5.60 ±200% softirqs.CPU58.BLOCK 0.00 +1.6e+102% 1.60 ± 84% softirqs.CPU58.NET_RX 0.00 -100.0% 0.00 softirqs.CPU58.NET_TX 9662 ± 6% +7.1% 10348 ± 5% softirqs.CPU58.RCU 5524 ± 5% +6.8% 5900 ± 3% softirqs.CPU58.SCHED 109.60 ±116% -52.0% 52.60 ± 20% softirqs.CPU58.TIMER 2.80 ±200% +128.6% 6.40 ±145% softirqs.CPU59.BLOCK 1.20 ±161% +1283.3% 16.60 ±194% softirqs.CPU59.NET_RX 0.40 ±122% +0.0% 0.40 ±122% softirqs.CPU59.NET_TX 9339 ± 5% +12.1% 10472 ± 7% softirqs.CPU59.RCU 5539 ± 3% +10.9% 6140 ± 5% softirqs.CPU59.SCHED 147.40 ±125% +94.4% 286.60 ±153% softirqs.CPU59.TIMER 1.60 ±200% -100.0% 0.00 softirqs.CPU6.BLOCK 0.40 ±122% +400.0% 2.00 ±176% softirqs.CPU6.NET_RX 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU6.NET_TX 9706 ± 4% +9.7% 10652 ± 6% softirqs.CPU6.RCU 5463 ± 2% +7.3% 5864 ± 3% softirqs.CPU6.SCHED 0.20 ±200% -100.0% 0.00 softirqs.CPU6.TASKLET 109.80 ± 51% -6.9% 102.20 ± 59% softirqs.CPU6.TIMER 3.60 ±101% -77.8% 0.80 ±122% softirqs.CPU60.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU60.NET_TX 9683 ± 5% +3.5% 10024 ± 5% softirqs.CPU60.RCU 5478 ± 2% +5.4% 5773 ± 3% softirqs.CPU60.SCHED 1.20 ±133% -100.0% 0.00 softirqs.CPU60.TASKLET 109.20 ± 30% -38.8% 66.80 ± 38% softirqs.CPU60.TIMER 8.60 ±149% +0.0% 8.60 ±188% softirqs.CPU61.NET_RX 0.80 ± 93% -75.0% 0.20 ±200% softirqs.CPU61.NET_TX 9695 ± 3% +14.5% 11097 ± 16% softirqs.CPU61.RCU 5562 ± 2% +7.9% 6001 ± 2% softirqs.CPU61.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU61.TASKLET 67.60 ± 36% +374.6% 320.80 ±142% softirqs.CPU61.TIMER 61.40 ±193% -96.1% 2.40 ±200% softirqs.CPU62.BLOCK 11.60 ±195% -93.1% 0.80 ±145% softirqs.CPU62.NET_RX 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU62.NET_TX 9448 ± 7% +9.1% 10306 ± 3% softirqs.CPU62.RCU 5412 ± 4% +10.4% 5973 ± 5% softirqs.CPU62.SCHED 54.00 ± 13% +358.9% 247.80 ±100% softirqs.CPU62.TIMER 1.20 ±200% +133.3% 2.80 ±200% softirqs.CPU63.BLOCK 0.60 ±200% -66.7% 0.20 ±200% softirqs.CPU63.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU63.NET_TX 9591 ± 4% +6.1% 10176 ± 2% softirqs.CPU63.RCU 5503 +8.4% 5968 ± 4% softirqs.CPU63.SCHED 0.00 +6e+101% 0.60 ±133% softirqs.CPU63.TASKLET 62.20 ± 14% +46.0% 90.80 ± 79% softirqs.CPU63.TIMER 0.20 ±200% -100.0% 0.00 softirqs.CPU64.BLOCK 5.80 ±107% -93.1% 0.40 ±122% softirqs.CPU64.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU64.NET_TX 9906 ± 6% +8.4% 10740 ± 5% softirqs.CPU64.RCU 5714 +1.8% 5815 ± 3% softirqs.CPU64.SCHED 1.20 ±200% -66.7% 0.40 ±200% softirqs.CPU64.TASKLET 233.20 ± 62% -62.8% 86.80 ± 42% softirqs.CPU64.TIMER 0.00 +1.6e+102% 1.60 ±200% softirqs.CPU65.BLOCK 19.80 ±200% -98.0% 0.40 ±122% softirqs.CPU65.NET_RX 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU65.NET_TX 9616 ± 4% +7.1% 10299 ± 3% softirqs.CPU65.RCU 5418 ± 2% +8.5% 5882 ± 2% softirqs.CPU65.SCHED 0.00 +8e+101% 0.80 ±122% softirqs.CPU65.TASKLET 92.40 ± 35% -32.0% 62.80 ± 38% softirqs.CPU65.TIMER 1.60 ±200% +175.0% 4.40 ±200% softirqs.CPU66.BLOCK 1.20 ±161% -100.0% 0.00 softirqs.CPU66.NET_RX 0.40 ±200% -100.0% 0.00 softirqs.CPU66.NET_TX 9924 ± 5% +5.6% 10481 ± 3% softirqs.CPU66.RCU 5617 ± 2% +5.6% 5930 ± 2% softirqs.CPU66.SCHED 218.00 ± 87% -61.0% 85.00 ± 62% softirqs.CPU66.TIMER 1.60 ±200% +37.5% 2.20 ±142% softirqs.CPU67.BLOCK 0.60 ± 81% +166.7% 1.60 ±115% softirqs.CPU67.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU67.NET_TX 9823 ± 5% +7.5% 10557 ± 4% softirqs.CPU67.RCU 5500 ± 2% +8.8% 5986 ± 4% softirqs.CPU67.SCHED 108.00 ± 40% -44.1% 60.40 ± 20% softirqs.CPU67.TIMER 5.00 ±162% -92.0% 0.40 ±200% softirqs.CPU68.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU68.NET_TX 9634 ± 4% +7.2% 10331 ± 4% softirqs.CPU68.RCU 5543 ± 2% +6.5% 5905 ± 2% softirqs.CPU68.SCHED 7.20 ±179% -100.0% 0.00 softirqs.CPU68.TASKLET 67.80 ± 35% -10.9% 60.40 ± 16% softirqs.CPU68.TIMER 2.40 ±200% -100.0% 0.00 softirqs.CPU69.BLOCK 1.80 ±173% -88.9% 0.20 ±200% softirqs.CPU69.NET_RX 0.60 ±200% -66.7% 0.20 ±200% softirqs.CPU69.NET_TX 9984 ± 8% +4.0% 10379 ± 2% softirqs.CPU69.RCU 5674 +3.4% 5869 ± 3% softirqs.CPU69.SCHED 2.00 ±200% -100.0% 0.00 softirqs.CPU69.TASKLET 113.60 ± 50% -13.2% 98.60 ± 61% softirqs.CPU69.TIMER 0.40 ±122% +400.0% 2.00 ±176% softirqs.CPU7.NET_RX 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU7.NET_TX 9727 ± 4% +6.2% 10327 ± 4% softirqs.CPU7.RCU 5479 ± 2% +9.1% 5978 ± 3% softirqs.CPU7.SCHED 0.40 ±200% +50.0% 0.60 ±133% softirqs.CPU7.TASKLET 91.20 ± 60% -0.7% 90.60 ± 77% softirqs.CPU7.TIMER 0.00 +2.4e+102% 2.40 ±200% softirqs.CPU70.BLOCK 0.20 ±200% +100.0% 0.40 ±122% softirqs.CPU70.NET_RX 9660 ± 5% +6.8% 10316 ± 3% softirqs.CPU70.RCU 5526 ± 2% +6.2% 5867 ± 2% softirqs.CPU70.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU70.TASKLET 64.60 ± 25% -19.2% 52.20 ± 21% softirqs.CPU70.TIMER 1.60 ±200% -100.0% 0.00 softirqs.CPU71.BLOCK 0.00 +1.2e+102% 1.20 ±122% softirqs.CPU71.NET_RX 9651 ± 5% +10.2% 10633 ± 5% softirqs.CPU71.RCU 5454 ± 2% +6.8% 5825 ± 4% softirqs.CPU71.SCHED 0.20 ±200% +0.0% 0.20 ±200% softirqs.CPU71.TASKLET 60.40 ± 13% +13.9% 68.80 ± 20% softirqs.CPU71.TIMER 0.00 +4e+101% 0.40 ±200% softirqs.CPU72.BLOCK 0.00 +1e+102% 1.00 ±200% softirqs.CPU72.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU72.NET_TX 9735 ± 5% -0.2% 9714 ± 6% softirqs.CPU72.RCU 5243 +9.2% 5725 softirqs.CPU72.SCHED 96.60 ± 15% -13.0% 84.00 ± 18% softirqs.CPU72.TIMER 11.80 ±134% -98.3% 0.20 ±200% softirqs.CPU73.NET_RX 9651 ± 4% +1.1% 9759 ± 6% softirqs.CPU73.RCU 5352 ± 2% +8.0% 5781 softirqs.CPU73.SCHED 89.60 ± 14% +3.3% 92.60 ± 24% softirqs.CPU73.TIMER 94.00 ±198% -100.0% 0.00 softirqs.CPU74.NET_RX 9880 ± 8% -0.7% 9815 ± 8% softirqs.CPU74.RCU 5295 ± 2% +8.5% 5745 ± 4% softirqs.CPU74.SCHED 0.00 +4.4e+102% 4.40 ±178% softirqs.CPU74.TASKLET 80.40 ± 17% +13.2% 91.00 ± 20% softirqs.CPU74.TIMER 0.20 ±200% -100.0% 0.00 softirqs.CPU75.NET_RX 9652 ± 5% +1.3% 9781 ± 4% softirqs.CPU75.RCU 5264 ± 3% +10.8% 5835 ± 2% softirqs.CPU75.SCHED 0.60 ±200% +766.7% 5.20 ±200% softirqs.CPU75.TASKLET 94.00 ± 17% +20.6% 113.40 ± 41% softirqs.CPU75.TIMER 0.00 +2.2e+102% 2.20 ±200% softirqs.CPU76.BLOCK 9634 ± 5% +1.0% 9729 ± 5% softirqs.CPU76.RCU 5318 +8.4% 5763 softirqs.CPU76.SCHED 87.20 ± 11% -6.0% 82.00 ± 8% softirqs.CPU76.TIMER 0.00 +9.8e+102% 9.80 ±189% softirqs.CPU77.BLOCK 0.20 ±200% -100.0% 0.00 softirqs.CPU77.NET_RX 9781 ± 6% -1.4% 9641 ± 7% softirqs.CPU77.RCU 5328 ± 2% +7.8% 5744 softirqs.CPU77.SCHED 76.60 ± 21% +26.6% 97.00 ± 50% softirqs.CPU77.TIMER 22.40 ±141% -100.0% 0.00 softirqs.CPU78.BLOCK 0.00 +1.6e+102% 1.60 ±145% softirqs.CPU78.NET_RX 9664 ± 4% +0.7% 9732 ± 5% softirqs.CPU78.RCU 5344 +9.4% 5845 softirqs.CPU78.SCHED 0.00 +6e+101% 0.60 ±200% softirqs.CPU78.TASKLET 72.80 ± 14% +69.8% 123.60 ± 30% softirqs.CPU78.TIMER 9721 ± 4% -0.6% 9662 ± 6% softirqs.CPU79.RCU 5329 ± 2% +9.6% 5841 softirqs.CPU79.SCHED 71.60 ± 9% +46.6% 105.00 ± 28% softirqs.CPU79.TIMER 0.20 ±200% -100.0% 0.00 softirqs.CPU8.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU8.NET_TX 9679 ± 5% +9.0% 10552 ± 3% softirqs.CPU8.RCU 5501 +12.0% 6160 ± 3% softirqs.CPU8.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU8.TASKLET 67.40 ± 28% +238.0% 227.80 ±100% softirqs.CPU8.TIMER 0.20 ±200% +200.0% 0.60 ±200% softirqs.CPU80.NET_RX 0.00 -100.0% 0.00 softirqs.CPU80.NET_TX 9310 ± 4% +0.3% 9343 ± 7% softirqs.CPU80.RCU 5291 ± 2% +8.2% 5723 ± 4% softirqs.CPU80.SCHED 74.80 ± 14% -7.2% 69.40 ± 18% softirqs.CPU80.TIMER 0.40 ±200% -100.0% 0.00 softirqs.CPU81.BLOCK 0.00 +6e+101% 0.60 ±200% softirqs.CPU81.NET_RX 0.00 +2e+101% 0.20 ±200% softirqs.CPU81.NET_TX 9268 ± 2% +1.2% 9380 ± 6% softirqs.CPU81.RCU 5320 ± 2% +8.4% 5769 ± 2% softirqs.CPU81.SCHED 0.00 +3.8e+102% 3.80 ±200% softirqs.CPU81.TASKLET 75.80 ± 17% +12.7% 85.40 ± 13% softirqs.CPU81.TIMER 0.40 ±200% -50.0% 0.20 ±200% softirqs.CPU82.BLOCK 17.00 ±148% -100.0% 0.00 softirqs.CPU82.NET_RX 9336 ± 3% -2.4% 9113 ± 7% softirqs.CPU82.RCU 5322 ± 4% +6.5% 5667 ± 4% softirqs.CPU82.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU82.TASKLET 80.20 ± 22% +10.7% 88.80 ± 31% softirqs.CPU82.TIMER 3.80 ±200% -84.2% 0.60 ±200% softirqs.CPU83.BLOCK 0.00 +3.3e+103% 33.20 ±194% softirqs.CPU83.NET_RX 9158 ± 5% +0.8% 9228 ± 7% softirqs.CPU83.RCU 5255 +7.8% 5663 ± 4% softirqs.CPU83.SCHED 0.00 +2e+102% 2.00 ±154% softirqs.CPU83.TASKLET 104.40 ± 45% -21.1% 82.40 ± 17% softirqs.CPU83.TIMER 9337 ± 4% +0.4% 9372 ± 5% softirqs.CPU84.RCU 5273 +8.6% 5727 softirqs.CPU84.SCHED 0.00 +1.6e+102% 1.60 ±200% softirqs.CPU84.TASKLET 76.40 ± 16% -0.5% 76.00 ± 32% softirqs.CPU84.TIMER 0.40 ±200% -100.0% 0.00 softirqs.CPU85.BLOCK 9324 ± 3% +1.2% 9439 ± 6% softirqs.CPU85.RCU 5274 ± 2% +11.2% 5866 ± 2% softirqs.CPU85.SCHED 0.00 +3.4e+102% 3.40 ±200% softirqs.CPU85.TASKLET 81.00 ± 31% +48.1% 120.00 ± 27% softirqs.CPU85.TIMER 5.60 ±200% -89.3% 0.60 ±200% softirqs.CPU86.BLOCK 0.20 ±200% +3600.0% 7.40 ±200% softirqs.CPU86.NET_RX 9558 ± 6% -1.6% 9406 ± 5% softirqs.CPU86.RCU 5336 ± 2% +9.1% 5823 ± 2% softirqs.CPU86.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU86.TASKLET 80.80 ± 34% +3.5% 83.60 ± 47% softirqs.CPU86.TIMER 1.60 ±200% +2737.5% 45.40 ±192% softirqs.CPU87.NET_RX 1.00 ±200% -100.0% 0.00 softirqs.CPU87.NET_TX 9425 ± 4% +0.1% 9437 ± 4% softirqs.CPU87.RCU 5302 +7.9% 5719 ± 8% softirqs.CPU87.SCHED 0.00 +2e+101% 0.20 ±200% softirqs.CPU87.TASKLET 85.20 ± 23% -9.2% 77.40 ± 17% softirqs.CPU87.TIMER 13.60 ±167% -100.0% 0.00 softirqs.CPU88.NET_RX 0.60 ±200% -100.0% 0.00 softirqs.CPU88.NET_TX 9096 ± 2% +2.5% 9324 ± 4% softirqs.CPU88.RCU 5246 ± 2% +10.2% 5783 softirqs.CPU88.SCHED 109.00 ± 48% -20.2% 87.00 ± 16% softirqs.CPU88.TIMER 5.00 ±190% +20.0% 6.00 ±200% softirqs.CPU89.BLOCK 9277 ± 8% +0.6% 9337 ± 5% softirqs.CPU89.RCU 5186 ± 2% +10.6% 5738 softirqs.CPU89.SCHED 88.60 ± 31% +2.7% 91.00 ± 56% softirqs.CPU89.TIMER 0.20 ±200% -100.0% 0.00 softirqs.CPU9.NET_RX 0.00 -100.0% 0.00 softirqs.CPU9.NET_TX 9533 ± 5% +8.9% 10386 ± 3% softirqs.CPU9.RCU 5530 +9.5% 6053 ± 5% softirqs.CPU9.SCHED 1.60 ± 50% +37.5% 2.20 ± 18% softirqs.CPU9.TASKLET 115.40 ± 43% -48.9% 59.00 ± 43% softirqs.CPU9.TIMER 10.00 ±190% -100.0% 0.00 softirqs.CPU90.BLOCK 0.20 ±200% -100.0% 0.00 softirqs.CPU90.NET_RX 0.20 ±200% -100.0% 0.00 softirqs.CPU90.NET_TX 9347 ± 4% +1.8% 9519 ± 6% softirqs.CPU90.RCU 5344 +8.0% 5772 softirqs.CPU90.SCHED 93.40 ± 24% -11.6% 82.60 ± 7% softirqs.CPU90.TIMER 1.60 ±200% -37.5% 1.00 ±200% softirqs.CPU91.NET_RX 9591 ± 7% -2.5% 9349 ± 5% softirqs.CPU91.RCU 5235 ± 4% +9.7% 5745 softirqs.CPU91.SCHED 84.40 ± 23% -10.0% 76.00 ± 15% softirqs.CPU91.TIMER 10.40 ±160% -100.0% 0.00 softirqs.CPU92.BLOCK 20.00 ±200% -100.0% 0.00 softirqs.CPU92.NET_RX 9322 ± 4% -0.7% 9257 ± 5% softirqs.CPU92.RCU 5371 ± 2% +7.7% 5785 softirqs.CPU92.SCHED 0.00 -100.0% 0.00 softirqs.CPU92.TASKLET 74.20 ± 17% -3.8% 71.40 ± 22% softirqs.CPU92.TIMER 4.20 ±200% -100.0% 0.00 softirqs.CPU93.BLOCK 0.80 ±122% -100.0% 0.00 softirqs.CPU93.NET_RX 9278 ± 4% +1.6% 9430 ± 7% softirqs.CPU93.RCU 5311 ± 2% +9.3% 5802 ± 3% softirqs.CPU93.SCHED 0.60 ±200% -66.7% 0.20 ±200% softirqs.CPU93.TASKLET 89.80 ± 39% -6.0% 84.40 ± 33% softirqs.CPU93.TIMER 9269 ± 4% -0.3% 9246 ± 5% softirqs.CPU94.RCU 5281 ± 2% +9.9% 5804 ± 2% softirqs.CPU94.SCHED 76.40 ± 38% +30.6% 99.80 ± 17% softirqs.CPU94.TIMER 0.40 ±200% -100.0% 0.00 softirqs.CPU95.BLOCK 1.00 ±200% -100.0% 0.00 softirqs.CPU95.NET_TX 7830 ± 4% -0.2% 7813 ± 6% softirqs.CPU95.RCU 4581 +7.2% 4912 softirqs.CPU95.SCHED 70.00 ± 19% +9.7% 76.80 ± 37% softirqs.CPU95.TIMER 2.60 ± 18% -15.4% 2.20 ± 18% softirqs.HI 530.80 ± 64% -23.9% 404.00 ± 87% softirqs.NET_RX 19.20 ± 12% -8.3% 17.60 ± 12% softirqs.NET_TX 926446 +3.9% 962877 softirqs.RCU 521434 +8.1% 563728 softirqs.SCHED 254.00 +0.2% 254.40 softirqs.TASKLET 11253 +0.1% 11260 softirqs.TIMER 8.85 ± 5% -8.8 0.00 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 9.54 ± 5% -8.7 0.80 ± 3% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 9.55 ± 5% -8.7 0.81 ± 3% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.40 ± 5% -8.5 0.93 ± 4% perf-profile.calltrace.cycles-pp.__libc_start_main 6.21 ± 5% -6.2 0.00 perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 6.14 ± 5% -6.1 0.00 perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2 6.12 ± 5% -6.1 0.00 perf-profile.calltrace.cycles-pp.lookup_fast.walk_component.link_path_walk.path_openat.do_filp_open 6.09 ± 5% -6.1 0.00 perf-profile.calltrace.cycles-pp.kernfs_dop_revalidate.lookup_fast.walk_component.link_path_walk.path_openat 6.04 ± 5% -6.0 0.00 perf-profile.calltrace.cycles-pp.__mutex_lock.kernfs_dop_revalidate.lookup_fast.walk_component.link_path_walk 5.88 ± 6% -5.9 0.00 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.kernfs_dop_revalidate.lookup_fast.walk_component 5.08 ± 6% -5.1 0.00 perf-profile.calltrace.cycles-pp.__open64_nocancel.__sysconf.__libc_start_main 5.08 ± 6% -5.1 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__sysconf.__libc_start_main 5.08 ± 6% -5.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__sysconf.__libc_start_main 5.08 ± 6% -5.1 0.00 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__sysconf 5.47 ± 5% -4.8 0.64 ± 4% perf-profile.calltrace.cycles-pp.__sysconf.__libc_start_main 3.78 ± 5% -3.8 0.00 perf-profile.calltrace.cycles-pp.__open64_nocancel.__libc_start_main 3.77 ± 5% -3.8 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__libc_start_main 3.77 ± 5% -3.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__libc_start_main 3.77 ± 5% -3.8 0.00 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel.__libc_start_main 35.51 ± 5% -3.4 32.08 ± 11% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.kernfs_fop_readdir.iterate_dir.__x64_sys_getdents64 36.35 ± 4% -3.2 33.10 ± 11% perf-profile.calltrace.cycles-pp.__mutex_lock.kernfs_fop_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64 37.45 ± 4% -3.2 34.27 ± 10% perf-profile.calltrace.cycles-pp.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 37.42 ± 4% -3.2 34.24 ± 10% perf-profile.calltrace.cycles-pp.kernfs_fop_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 37.44 ± 4% -3.2 34.27 ± 10% perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.47 ± 6% -2.5 0.00 perf-profile.calltrace.cycles-pp.lookup_fast.path_openat.do_filp_open.do_sys_openat2.do_sys_open 2.46 ± 6% -2.5 0.00 perf-profile.calltrace.cycles-pp.kernfs_dop_revalidate.lookup_fast.path_openat.do_filp_open.do_sys_openat2 2.44 ± 6% -2.4 0.00 perf-profile.calltrace.cycles-pp.__mutex_lock.kernfs_dop_revalidate.lookup_fast.path_openat.do_filp_open 2.38 ± 7% -2.4 0.00 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.kernfs_dop_revalidate.lookup_fast.path_openat 0.52 ± 3% -0.2 0.33 ± 81% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.osq_lock.__mutex_lock.kernfs_fop_readdir.iterate_dir 0.31 ± 81% +0.0 0.32 ± 81% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.osq_lock.__mutex_lock.kernfs_fop_readdir 0.60 ± 3% +0.1 0.66 ± 6% perf-profile.calltrace.cycles-pp.elf_map.load_elf_interp.load_elf_binary.exec_binprm.bprm_execve 0.57 ± 2% +0.1 0.62 ± 2% perf-profile.calltrace.cycles-pp._dl_addr 0.89 +0.1 0.95 perf-profile.calltrace.cycles-pp.vma_interval_tree_insert.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey 0.60 ± 2% +0.1 0.66 ± 2% perf-profile.calltrace.cycles-pp.perf_event_mmap.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 0.78 ± 3% +0.1 0.86 ± 5% perf-profile.calltrace.cycles-pp.load_elf_interp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common 0.69 ± 2% +0.1 0.76 ± 4% perf-profile.calltrace.cycles-pp.vma_interval_tree_insert.vma_link.mmap_region.do_mmap.vm_mmap_pgoff 0.76 ± 2% +0.1 0.83 ± 3% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap 0.95 +0.1 1.03 ± 2% perf-profile.calltrace.cycles-pp.vma_interval_tree_insert.__vma_adjust.__split_vma.__do_munmap.mmap_region 0.83 ± 2% +0.1 0.92 ± 3% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.61 ± 4% +0.1 0.70 ± 5% perf-profile.calltrace.cycles-pp.kmem_cache_free.remove_vma.exit_mmap.mmput.do_exit 0.83 ± 2% +0.1 0.93 ± 2% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.58 ± 2% +0.1 0.68 ± 3% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.__split_vma 0.00 +0.1 0.10 ±200% perf-profile.calltrace.cycles-pp.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 0.70 ± 4% +0.1 0.81 ± 5% perf-profile.calltrace.cycles-pp.remove_vma.exit_mmap.mmput.do_exit.do_group_exit 0.64 ± 8% +0.1 0.75 ± 11% perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary 0.65 ± 7% +0.1 0.76 ± 11% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm 0.65 ± 2% +0.1 0.77 ± 2% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.67 ± 7% +0.1 0.79 ± 10% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.elf_map.load_elf_binary.exec_binprm.bprm_execve 0.64 ± 14% +0.1 0.77 ± 14% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region 0.99 ± 2% +0.1 1.11 ± 3% perf-profile.calltrace.cycles-pp.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 0.67 ± 13% +0.1 0.80 ± 15% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu 0.68 ± 13% +0.1 0.82 ± 14% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap 0.97 ± 2% +0.1 1.11 ± 2% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.97 ± 2% +0.1 1.11 ± 2% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.84 ± 12% +0.1 0.99 ± 13% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap 0.86 ± 12% +0.2 1.01 ± 12% perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap.mmap_region 0.86 ± 12% +0.2 1.01 ± 12% perf-profile.calltrace.cycles-pp.lru_add_drain.unmap_region.__do_munmap.mmap_region.do_mmap 0.93 ± 8% +0.2 1.08 ± 11% perf-profile.calltrace.cycles-pp.elf_map.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common 0.00 +0.2 0.16 ±200% perf-profile.calltrace.cycles-pp.irqentry_exit_to_user_mode.asm_exc_page_fault 0.61 ± 3% +0.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.__vma_adjust.vma_merge.mmap_region.do_mmap.vm_mmap_pgoff 0.66 ± 3% +0.2 0.84 ± 4% perf-profile.calltrace.cycles-pp.vma_merge.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 1.94 ± 2% +0.2 2.12 ± 3% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput 2.04 ± 2% +0.2 2.24 ± 3% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit 0.41 ± 50% +0.2 0.61 ± 3% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_optimistic_spin.rwsem_down_write_slowpath.vma_link.mmap_region 2.15 ± 2% +0.2 2.35 ± 3% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit 1.63 ± 3% +0.2 1.84 ± 4% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 0.00 +0.2 0.20 ±122% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.vma_merge 0.52 ± 51% +0.2 0.74 ± 15% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain 2.27 ± 11% +0.2 2.49 ± 7% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.41 ± 11% +0.2 2.66 ± 7% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.40 ± 11% +0.3 2.65 ± 7% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 1.73 ± 7% +0.3 2.00 ± 7% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput 0.21 ±122% +0.3 0.49 ± 51% perf-profile.calltrace.cycles-pp.vma_link.mmap_region.do_mmap.vm_mmap_pgoff.elf_map 2.02 ± 6% +0.3 2.31 ± 6% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit 2.02 ± 6% +0.3 2.31 ± 6% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit 2.55 ± 3% +0.3 2.86 ± 5% perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve 2.59 ± 3% +0.3 2.91 ± 5% perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64 1.01 ± 21% +0.3 1.32 ± 4% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_optimistic_spin.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables 0.00 +0.3 0.32 ±200% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 2.69 ± 10% +0.3 3.01 ± 7% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 2.66 ± 11% +0.3 2.98 ± 7% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.66 ± 11% +0.3 2.98 ± 7% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.66 ± 11% +0.3 2.98 ± 7% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 2.90 ± 3% +0.3 3.24 ± 5% perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.21 ± 3% +0.4 3.59 ± 5% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.23 ± 3% +0.4 3.60 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.25 ± 3% +0.4 3.65 ± 3% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 0.10 ±200% +0.4 0.53 ± 5% perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.mmput.do_exit 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.kernfs_fop_readdir.iterate_dir.__x64_sys_getdents64 0.00 +0.6 0.58 ± 5% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.vma_merge.mmap_region 0.00 +0.6 0.59 ± 6% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__vma_adjust.vma_merge.mmap_region.do_mmap 5.05 ± 3% +0.6 5.69 ± 3% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 5.37 ± 3% +0.7 6.05 ± 3% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 5.84 ± 2% +0.7 6.58 ± 3% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 5.89 ± 2% +0.7 6.63 ± 3% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault 2.04 ± 5% +0.9 2.92 ± 10% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup 2.07 ± 5% +0.9 2.96 ± 10% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey 6.01 ± 2% +0.9 6.94 ± 6% perf-profile.calltrace.cycles-pp.asm_exc_page_fault 3.70 ± 3% +1.0 4.74 ± 7% perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect 4.12 ± 3% +1.1 5.21 ± 6% perf-profile.calltrace.cycles-pp.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64 4.60 ± 3% +1.1 5.75 ± 6% perf-profile.calltrace.cycles-pp.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.75 ± 3% +1.2 5.91 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.75 ± 3% +1.2 5.91 ± 6% perf-profile.calltrace.cycles-pp.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.53 ± 8% +1.2 3.73 ± 9% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.vma_link.mmap_region 3.35 ± 6% +1.3 4.64 ± 7% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap 3.40 ± 6% +1.3 4.69 ± 7% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap.mmap_region 3.14 ± 7% +1.3 4.47 ± 8% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 3.18 ± 7% +1.3 4.52 ± 8% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap.vm_mmap_pgoff 3.46 ± 6% +1.4 4.86 ± 9% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 3.49 ± 6% +1.4 4.91 ± 9% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region.__do_munmap 5.09 ± 5% +1.5 6.54 ± 6% perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.__do_munmap.mmap_region.do_mmap 3.72 ± 6% +1.5 5.18 ± 8% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.unmap_region.__do_munmap.mmap_region 3.83 ± 6% +1.5 5.32 ± 8% perf-profile.calltrace.cycles-pp.free_pgtables.unmap_region.__do_munmap.mmap_region.do_mmap 4.35 ± 5% +1.5 5.84 ± 7% perf-profile.calltrace.cycles-pp.vma_link.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 5.51 ± 4% +1.5 7.03 ± 5% perf-profile.calltrace.cycles-pp.__split_vma.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff 2.90 ± 6% +1.5 4.44 ± 11% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 2.95 ± 6% +1.6 4.51 ± 11% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap.mmput 3.63 ± 5% +1.6 5.26 ± 10% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit 4.91 ± 6% +1.7 6.57 ± 7% perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff 4.34 ± 5% +1.7 6.08 ± 9% perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit 4.24 ± 7% +2.0 6.21 ± 9% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.__vma_adjust.__split_vma 9.35 ± 4% +2.4 11.71 ± 5% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 9.33 ± 4% +2.4 11.69 ± 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group 9.89 ± 4% +2.4 12.30 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.89 ± 4% +2.4 12.30 ± 5% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.89 ± 4% +2.4 12.30 ± 5% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.02 ± 7% +2.7 7.69 ± 11% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables 10.92 ± 5% +3.3 14.18 ± 6% perf-profile.calltrace.cycles-pp.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 17.23 ± 5% +5.1 22.33 ± 6% perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64 17.51 ± 5% +5.1 22.64 ± 6% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.71 ± 5% +5.1 22.86 ± 6% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.76 ± 5% +5.2 22.92 ± 6% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 76.30 +6.3 82.63 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 76.45 +6.7 83.11 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 44.84 ± 5% -11.7 33.11 ± 11% perf-profile.children.cycles-pp.__mutex_lock 8.56 ± 6% -8.6 0.00 perf-profile.children.cycles-pp.kernfs_dop_revalidate 8.86 ± 5% -8.6 0.31 perf-profile.children.cycles-pp.__open64_nocancel 8.72 ± 6% -8.5 0.18 ± 4% perf-profile.children.cycles-pp.lookup_fast 9.67 ± 5% -8.5 1.19 ± 3% perf-profile.children.cycles-pp.path_openat 9.40 ± 5% -8.5 0.93 ± 4% perf-profile.children.cycles-pp.__libc_start_main 9.68 ± 5% -8.5 1.21 ± 2% perf-profile.children.cycles-pp.do_sys_openat2 9.67 ± 5% -8.5 1.20 ± 3% perf-profile.children.cycles-pp.do_filp_open 9.68 ± 5% -8.5 1.22 ± 2% perf-profile.children.cycles-pp.do_sys_open 6.32 ± 5% -6.1 0.23 ± 6% perf-profile.children.cycles-pp.link_path_walk 6.25 ± 5% -6.1 0.16 ± 5% perf-profile.children.cycles-pp.walk_component 56.38 ± 2% -5.6 50.73 ± 3% perf-profile.children.cycles-pp.osq_lock 5.48 ± 5% -4.8 0.65 ± 4% perf-profile.children.cycles-pp.__sysconf 37.45 ± 4% -3.2 34.27 ± 10% perf-profile.children.cycles-pp.__x64_sys_getdents64 37.42 ± 4% -3.2 34.24 ± 10% perf-profile.children.cycles-pp.kernfs_fop_readdir 37.44 ± 4% -3.2 34.27 ± 10% perf-profile.children.cycles-pp.iterate_dir 85.86 -2.2 83.66 perf-profile.children.cycles-pp.do_syscall_64 86.15 -1.8 84.32 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.31 ± 12% -0.0 0.27 ± 13% perf-profile.children.cycles-pp.vprintk_emit 0.28 ± 16% -0.0 0.25 ± 14% perf-profile.children.cycles-pp.write 0.28 ± 16% -0.0 0.25 ± 14% perf-profile.children.cycles-pp.devkmsg_write.cold 0.28 ± 16% -0.0 0.25 ± 14% perf-profile.children.cycles-pp.devkmsg_emit 0.29 ± 15% -0.0 0.26 ± 12% perf-profile.children.cycles-pp.console_unlock 0.08 ± 9% -0.0 0.05 ± 53% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.21 ± 15% -0.0 0.18 ± 13% perf-profile.children.cycles-pp.wait_for_xmitr 0.20 ± 16% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.serial8250_console_putchar 0.22 ± 15% -0.0 0.20 ± 12% perf-profile.children.cycles-pp.serial8250_console_write 0.21 ± 16% -0.0 0.19 ± 13% perf-profile.children.cycles-pp.uart_console_write 0.02 ±123% -0.0 0.00 perf-profile.children.cycles-pp.delay_tsc 0.14 ± 16% -0.0 0.12 ± 20% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work 0.32 ± 15% -0.0 0.30 ± 10% perf-profile.children.cycles-pp.ksys_write 0.32 ± 15% -0.0 0.30 ± 10% perf-profile.children.cycles-pp.vfs_write 0.32 ± 15% -0.0 0.30 ± 9% perf-profile.children.cycles-pp.new_sync_write 0.17 ± 15% -0.0 0.16 ± 14% perf-profile.children.cycles-pp.process_one_work 0.16 ± 17% -0.0 0.15 ± 13% perf-profile.children.cycles-pp.io_serial_in 0.19 ± 16% -0.0 0.17 ± 11% perf-profile.children.cycles-pp.worker_thread 0.07 ± 18% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.lf 0.07 ± 18% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.con_scroll 0.07 ± 18% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.fbcon_scroll 0.07 ± 18% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.fbcon_redraw 0.13 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.mutex_unlock 0.01 ±200% -0.0 0.00 perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.01 ±200% -0.0 0.00 perf-profile.children.cycles-pp.start_kernel 0.07 ± 18% -0.0 0.06 ± 10% perf-profile.children.cycles-pp.vt_console_print 0.07 ± 14% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.fbcon_putcs 0.07 ± 14% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.bit_putcs 0.28 ± 16% -0.0 0.27 ± 15% perf-profile.children.cycles-pp.kthread 0.05 ± 52% -0.0 0.04 ± 51% perf-profile.children.cycles-pp.drm_fb_helper_sys_imageblit 0.05 ± 52% -0.0 0.04 ± 51% perf-profile.children.cycles-pp.sys_imageblit 0.05 -0.0 0.04 ± 50% perf-profile.children.cycles-pp.map_vdso 0.08 ± 4% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.27 -0.0 0.26 ± 3% perf-profile.children.cycles-pp.mutex_lock 0.09 ± 25% -0.0 0.09 ± 31% perf-profile.children.cycles-pp.ktime_get 0.09 ± 25% -0.0 0.08 ± 40% perf-profile.children.cycles-pp.clockevents_program_event 0.06 ± 6% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.brk 0.06 ± 6% -0.0 0.06 ± 10% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.05 ± 9% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.__x64_sys_brk 0.15 ± 5% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.filldir64 0.27 ± 9% +0.0 0.27 ± 7% perf-profile.children.cycles-pp.memcpy_erms 0.16 ± 4% +0.0 0.16 ± 4% perf-profile.children.cycles-pp.step_into 0.13 ± 4% +0.0 0.13 ± 4% perf-profile.children.cycles-pp.dput 0.09 ± 5% +0.0 0.09 ± 9% perf-profile.children.cycles-pp.strlen 0.06 ± 7% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.05 ± 7% +0.0 0.05 ± 7% perf-profile.children.cycles-pp.strncpy_from_user 0.01 ±200% +0.0 0.01 ±200% perf-profile.children.cycles-pp.__update_load_avg_se 0.01 ±200% +0.0 0.01 ±200% perf-profile.children.cycles-pp.terminate_walk 0.12 ± 10% +0.0 0.13 ± 10% perf-profile.children.cycles-pp.newidle_balance 0.10 ± 14% +0.0 0.10 ± 19% perf-profile.children.cycles-pp.find_busiest_group 0.10 ± 13% +0.0 0.10 ± 16% perf-profile.children.cycles-pp.update_sd_lb_stats 0.09 ± 7% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.update_curr 0.08 ± 9% +0.0 0.08 perf-profile.children.cycles-pp.getname_flags 0.08 ± 5% +0.0 0.08 ± 11% perf-profile.children.cycles-pp.inode_permission 0.07 ± 5% +0.0 0.07 perf-profile.children.cycles-pp.do_task_dead 0.06 ± 7% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.schedule_timeout 0.06 ± 6% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.rcu_all_qs 0.05 +0.0 0.05 ± 7% perf-profile.children.cycles-pp.clock_gettime 0.05 +0.0 0.05 ± 7% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size 0.05 +0.0 0.05 ± 7% perf-profile.children.cycles-pp.__lock_page_killable 0.09 ± 5% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.do_brk_flags 0.09 ± 5% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.kernfs_put 0.07 ± 10% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.do_sigaction 0.05 ± 7% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.touch_atime 0.06 ± 8% +0.0 0.06 perf-profile.children.cycles-pp.acct_collect 0.05 +0.0 0.05 ± 9% perf-profile.children.cycles-pp.vmacache_update 0.13 ± 10% +0.0 0.13 ± 14% perf-profile.children.cycles-pp.load_balance 0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.free_unref_page_list 0.09 +0.0 0.09 ± 5% perf-profile.children.cycles-pp.setproctitle_init 0.07 +0.0 0.07 ± 6% perf-profile.children.cycles-pp.mm_init 0.06 ± 6% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.update_rq_clock 0.07 ± 7% +0.0 0.07 perf-profile.children.cycles-pp.swake_up_locked 0.06 ± 7% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.pgd_alloc 0.07 ± 5% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.move_page_tables 0.06 ± 8% +0.0 0.06 perf-profile.children.cycles-pp.create_elf_tables 0.05 ± 7% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.__fxstat64 0.06 +0.0 0.06 ± 7% perf-profile.children.cycles-pp.memset_erms 0.06 +0.0 0.06 ± 7% perf-profile.children.cycles-pp.page_get_link 0.05 +0.0 0.05 ± 9% perf-profile.children.cycles-pp.__libc_csu_init 0.65 ± 11% +0.0 0.66 ± 9% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.14 ± 4% +0.0 0.15 ± 3% perf-profile.children.cycles-pp._cond_resched 0.11 ± 3% +0.0 0.12 ± 8% perf-profile.children.cycles-pp.pick_link 0.10 ± 4% +0.0 0.11 ± 5% perf-profile.children.cycles-pp.kernfs_dir_pos 0.08 ± 6% +0.0 0.08 ± 4% perf-profile.children.cycles-pp.mm_release 0.07 +0.0 0.08 ± 6% perf-profile.children.cycles-pp.ima_file_mmap 0.07 ± 6% +0.0 0.08 ± 11% perf-profile.children.cycles-pp.open_exec 0.05 ± 7% +0.0 0.06 ± 6% perf-profile.children.cycles-pp.__kernel_read 0.53 ± 2% +0.0 0.53 ± 2% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.17 ± 7% +0.0 0.18 ± 7% perf-profile.children.cycles-pp.pick_next_task_fair 0.09 ± 18% +0.0 0.10 ± 18% perf-profile.children.cycles-pp.smpboot_thread_fn 0.09 ± 4% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.__mod_lruvec_state 0.10 ± 6% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.__wake_up_common 0.09 ± 4% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.unmap_single_vma 0.08 ± 9% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.fput_many 0.08 ± 13% +0.0 0.08 ± 14% perf-profile.children.cycles-pp.xas_start 0.07 ± 5% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.complete 0.06 ± 7% +0.0 0.07 ± 9% perf-profile.children.cycles-pp.wait_for_completion_killable 0.06 ± 12% +0.0 0.07 perf-profile.children.cycles-pp.sched_move_task 0.06 ± 12% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.aes_gcm_dec_256_avx512 0.07 ± 7% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.get_nprocs_conf 0.04 ± 51% +0.0 0.05 perf-profile.children.cycles-pp._copy_from_user 0.64 ± 12% +0.0 0.65 ± 9% perf-profile.children.cycles-pp.hrtimer_interrupt 0.08 ± 5% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_trace 0.08 ± 4% +0.0 0.09 perf-profile.children.cycles-pp._exit 0.07 ± 5% +0.0 0.08 ± 7% perf-profile.children.cycles-pp.vm_normal_page 0.07 ± 5% +0.0 0.08 perf-profile.children.cycles-pp.__mod_node_page_state 0.08 ± 6% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.aa_get_task_label 0.08 ± 6% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.07 +0.0 0.08 ± 9% perf-profile.children.cycles-pp.__slab_free 0.05 +0.0 0.06 ± 6% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.05 +0.0 0.06 ± 6% perf-profile.children.cycles-pp.down_write_killable 0.09 ± 4% +0.0 0.10 ± 5% perf-profile.children.cycles-pp.do_notify_parent 0.08 +0.0 0.09 ± 4% perf-profile.children.cycles-pp._copy_to_user 0.08 ± 7% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.put_cred_rcu 0.07 ± 7% +0.0 0.07 ± 6% perf-profile.children.cycles-pp._find_next_bit 0.07 +0.0 0.08 ± 5% perf-profile.children.cycles-pp.unlazy_walk 0.07 ± 7% +0.0 0.07 ± 10% perf-profile.children.cycles-pp.filename_lookup 0.06 ± 12% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.vfs_fstat 0.07 ± 7% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.06 ± 7% +0.0 0.07 ± 10% perf-profile.children.cycles-pp.path_lookupat 0.06 ± 6% +0.0 0.07 perf-profile.children.cycles-pp.__legitimize_path 0.06 ± 7% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.__entry_text_start 0.06 ± 7% +0.0 0.07 ± 10% perf-profile.children.cycles-pp.dequeue_entity 0.06 +0.0 0.07 ± 5% perf-profile.children.cycles-pp.copy_creds 0.35 ± 14% +0.0 0.36 ± 12% perf-profile.children.cycles-pp.ret_from_fork 0.44 ± 11% +0.0 0.45 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.34 ± 13% +0.0 0.35 ± 7% perf-profile.children.cycles-pp.tick_sched_timer 0.11 ± 3% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.__d_lookup_rcu 0.10 +0.0 0.11 ± 5% perf-profile.children.cycles-pp.apparmor_task_getsecid 0.08 ± 7% +0.0 0.09 perf-profile.children.cycles-pp.__wake_up_common_lock 0.08 ± 5% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.security_file_alloc 0.07 ± 5% +0.0 0.08 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.07 ± 9% +0.0 0.08 ± 7% perf-profile.children.cycles-pp.ttwu_do_activate 0.07 ± 6% +0.0 0.08 ± 9% perf-profile.children.cycles-pp.__exp_finite 0.05 ± 7% +0.0 0.06 ± 6% perf-profile.children.cycles-pp.schedule_tail 0.05 ± 9% +0.0 0.06 ± 12% perf-profile.children.cycles-pp.security_cred_free 0.03 ± 81% +0.0 0.04 ± 50% perf-profile.children.cycles-pp.__irqentry_text_end 0.03 ± 81% +0.0 0.04 ± 50% perf-profile.children.cycles-pp.free_pgd_range 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp.update_blocked_averages 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp._raw_spin_trylock 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp.sched_ttwu_pending 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp.__kmalloc 0.00 +0.0 0.01 ±200% perf-profile.children.cycles-pp.down_read_trylock 0.04 ± 50% +0.0 0.05 perf-profile.children.cycles-pp.__tlb_remove_page_size 0.31 ± 13% +0.0 0.32 ± 7% perf-profile.children.cycles-pp.tick_sched_handle 0.30 ± 14% +0.0 0.31 ± 6% perf-profile.children.cycles-pp.update_process_times 0.09 ± 8% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair 0.09 ± 4% +0.0 0.10 ± 7% perf-profile.children.cycles-pp.stress_matrix_xy_add._GLOBAL___stress_matrix_info.ifunc 0.07 ± 20% +0.0 0.08 ± 15% perf-profile.children.cycles-pp.run_ksoftirqd 0.06 ± 7% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.apparmor_file_alloc_security 0.06 ± 7% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.enqueue_entity 0.06 ± 6% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.get_nprocs 0.06 +0.0 0.07 perf-profile.children.cycles-pp.lockref_put_or_lock 0.17 ± 2% +0.0 0.19 ± 2% perf-profile.children.cycles-pp.do_dentry_open 0.14 ± 11% +0.0 0.15 ± 8% perf-profile.children.cycles-pp.task_tick_fair 0.14 ± 3% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.ptep_clear_flush 0.11 ± 9% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.xas_find 0.11 ± 6% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.alloc_bprm 0.10 ± 4% +0.0 0.12 ± 4% perf-profile.children.cycles-pp.security_task_getsecid 0.10 +0.0 0.11 ± 6% perf-profile.children.cycles-pp.mark_page_accessed 0.07 ± 5% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.do_wp_page 0.07 +0.0 0.08 ± 4% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics 0.07 ± 7% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.__unlock_page_memcg 0.07 +0.0 0.08 ± 9% perf-profile.children.cycles-pp.__mmdrop 0.04 ± 50% +0.0 0.05 ± 7% perf-profile.children.cycles-pp.select_idle_sibling 0.04 ± 50% +0.0 0.05 ± 7% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.04 ± 50% +0.0 0.05 ± 9% perf-profile.children.cycles-pp.copy_page_to_iter 0.04 ± 50% +0.0 0.05 ± 7% perf-profile.children.cycles-pp.up_read 0.17 ± 2% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.prepend_path 0.14 ± 5% +0.0 0.15 ± 5% perf-profile.children.cycles-pp.vm_unmapped_area 0.13 ± 3% +0.0 0.14 ± 5% perf-profile.children.cycles-pp.do_open_execat 0.10 ± 4% +0.0 0.11 perf-profile.children.cycles-pp.prepend_name 0.08 ± 5% +0.0 0.10 ± 5% perf-profile.children.cycles-pp.perf_output_copy 0.07 ± 12% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.get_available_features 0.06 +0.0 0.07 ± 10% perf-profile.children.cycles-pp.__free_pages_ok 0.18 ± 4% +0.0 0.19 ± 3% perf-profile.children.cycles-pp.generic_file_buffered_read 0.17 ± 2% +0.0 0.18 ± 3% perf-profile.children.cycles-pp.begin_new_exec 0.14 ± 5% +0.0 0.16 ± 4% perf-profile.children.cycles-pp.update_load_avg 0.15 ± 4% +0.0 0.17 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.11 ± 4% +0.0 0.12 ± 5% perf-profile.children.cycles-pp.__pmd_alloc 0.11 ± 4% +0.0 0.12 ± 7% perf-profile.children.cycles-pp._init 0.07 ± 6% +0.0 0.09 ± 8% perf-profile.children.cycles-pp.wmemchr 0.07 ± 5% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.common_file_perm 0.08 ± 4% +0.0 0.10 ± 8% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.08 ± 5% +0.0 0.10 ± 4% perf-profile.children.cycles-pp._start 0.21 ± 11% +0.0 0.22 ± 7% perf-profile.children.cycles-pp.scheduler_tick 0.18 ± 6% +0.0 0.19 ± 5% perf-profile.children.cycles-pp.sched_exec 0.18 ± 5% +0.0 0.20 ± 2% perf-profile.children.cycles-pp.kernfs_get 0.15 ± 4% +0.0 0.17 ± 2% perf-profile.children.cycles-pp.new_sync_read 0.13 ± 3% +0.0 0.15 ± 5% perf-profile.children.cycles-pp.__list_del_entry_valid 0.09 ± 5% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.memmove 0.10 ± 5% +0.0 0.11 perf-profile.children.cycles-pp.security_mmap_file 0.09 ± 7% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.__do_sys_newfstat 0.09 ± 5% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.stress_cpu_bitops._GLOBAL___stress_cpu_info.ifunc 0.08 ± 5% +0.0 0.10 ± 7% perf-profile.children.cycles-pp.dequeue_task_fair 0.17 ± 6% +0.0 0.19 ± 3% perf-profile.children.cycles-pp.rmqueue 0.14 ± 4% +0.0 0.16 ± 7% perf-profile.children.cycles-pp.__libc_pthread_init 0.11 ± 8% +0.0 0.13 ± 7% perf-profile.children.cycles-pp.prepare_creds 0.10 ± 7% +0.0 0.12 perf-profile.children.cycles-pp.__put_task_struct 0.09 ± 4% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.cosf32 0.06 ± 6% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.alloc_vmap_area 0.19 ± 5% +0.0 0.20 ± 3% perf-profile.children.cycles-pp.vma_interval_tree_augment_rotate 0.18 ± 4% +0.0 0.20 ± 4% perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown 0.16 ± 2% +0.0 0.18 ± 3% perf-profile.children.cycles-pp.native_flush_tlb_one_user 0.17 ± 4% +0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_rt_sigaction 0.11 ± 7% +0.0 0.13 ± 8% perf-profile.children.cycles-pp.do_faccessat 0.12 ± 6% +0.0 0.13 ± 8% perf-profile.children.cycles-pp.__getrlimit 0.08 ± 5% +0.0 0.10 ± 6% perf-profile.children.cycles-pp.__pud_alloc 0.09 ± 8% +0.0 0.10 ± 9% perf-profile.children.cycles-pp.__pthread_initialize_minimal_internal 0.09 ± 4% +0.0 0.10 ± 9% perf-profile.children.cycles-pp.__cxa_atexit 0.15 ± 4% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.__get_free_pages 0.14 ± 5% +0.0 0.16 ± 5% perf-profile.children.cycles-pp._fini 0.08 ± 10% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.stress_vm_flip._GLOBAL___stress_vm_info.ifunc 0.00 +0.0 0.02 ±200% perf-profile.children.cycles-pp.rebalance_domains 0.26 ± 3% +0.0 0.27 ± 2% perf-profile.children.cycles-pp.___might_sleep 0.14 ± 10% +0.0 0.16 ± 6% perf-profile.children.cycles-pp.xas_load 0.13 ± 5% +0.0 0.15 ± 5% perf-profile.children.cycles-pp.__pte_alloc 0.13 +0.0 0.15 ± 7% perf-profile.children.cycles-pp.release_task 0.12 ± 5% +0.0 0.14 ± 2% perf-profile.children.cycles-pp.change_p4d_range 0.12 ± 6% +0.0 0.14 ± 8% perf-profile.children.cycles-pp.rb_next 0.27 ± 3% +0.0 0.29 ± 4% perf-profile.children.cycles-pp.pagecache_get_page 0.22 +0.0 0.24 ± 4% perf-profile.children.cycles-pp.__vma_link_rb 0.17 ± 4% +0.0 0.19 ± 2% perf-profile.children.cycles-pp.__might_sleep 0.09 ± 5% +0.0 0.11 ± 7% perf-profile.children.cycles-pp.__kmalloc_node 0.04 ± 50% +0.0 0.06 ± 12% perf-profile.children.cycles-pp.generic_file_mmap 0.03 ± 81% +0.0 0.05 perf-profile.children.cycles-pp.arch_dup_task_struct 0.03 ± 81% +0.0 0.05 perf-profile.children.cycles-pp.PageHuge 0.03 ± 81% +0.0 0.05 perf-profile.children.cycles-pp.__might_fault 0.02 ±122% +0.0 0.04 ± 50% perf-profile.children.cycles-pp.io_schedule 0.00 +0.0 0.02 ±122% perf-profile.children.cycles-pp.__install_special_mapping 0.00 +0.0 0.02 ±122% perf-profile.children.cycles-pp.cpumask_any_but 0.00 +0.0 0.02 ±122% perf-profile.children.cycles-pp.__vma_rb_erase 0.00 +0.0 0.02 ±122% perf-profile.children.cycles-pp.update_cfs_group 0.00 +0.0 0.02 ±122% perf-profile.children.cycles-pp.apparmor_file_open 0.06 ± 7% +0.0 0.08 ± 9% perf-profile.children.cycles-pp.__get_vm_area_node 0.01 ±200% +0.0 0.03 ± 81% perf-profile.children.cycles-pp.__register_atfork 0.01 ±200% +0.0 0.03 ± 81% perf-profile.children.cycles-pp.cosf32x 0.28 +0.0 0.30 ± 4% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.26 ± 4% +0.0 0.28 ± 3% perf-profile.children.cycles-pp.clear_page_erms 0.18 ± 2% +0.0 0.20 ± 3% perf-profile.children.cycles-pp.flush_tlb_func_common 0.16 ± 3% +0.0 0.18 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 0.18 ± 4% +0.0 0.20 ± 3% perf-profile.children.cycles-pp.perf_event_mmap_output 0.16 ± 4% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.vmacache_find 0.15 ± 3% +0.0 0.17 ± 6% perf-profile.children.cycles-pp.wait_task_zombie 0.13 ± 5% +0.0 0.16 ± 5% perf-profile.children.cycles-pp.anon_vma_clone 0.02 ±122% +0.0 0.04 ± 50% perf-profile.children.cycles-pp.security_prepare_creds 0.08 ± 4% +0.0 0.10 ± 7% perf-profile.children.cycles-pp.__vmalloc_node_range 0.26 ± 3% +0.0 0.28 ± 4% perf-profile.children.cycles-pp.find_get_entry 0.24 ± 4% +0.0 0.26 ± 4% perf-profile.children.cycles-pp.get_unmapped_area 0.15 +0.0 0.17 ± 4% perf-profile.children.cycles-pp.page_add_new_anon_rmap 0.17 ± 4% +0.0 0.19 ± 6% perf-profile.children.cycles-pp.get_arg_page 0.15 ± 5% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.seq_read_iter 0.11 ± 15% +0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.03 ± 81% +0.0 0.05 ± 9% perf-profile.children.cycles-pp.queued_read_lock_slowpath 0.26 ± 5% +0.0 0.28 ± 5% perf-profile.children.cycles-pp.filemap_fault 0.26 ± 4% +0.0 0.28 ± 2% perf-profile.children.cycles-pp.flush_tlb_mm_range 0.28 ± 3% +0.0 0.31 ± 3% perf-profile.children.cycles-pp.prep_new_page 0.15 ± 5% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.seq_read 0.10 ± 4% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.kernfs_fop_release 0.08 ± 4% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.seq_release 0.26 ± 2% +0.0 0.29 perf-profile.children.cycles-pp.__rb_erase_color 0.35 ± 3% +0.0 0.38 ± 3% perf-profile.children.cycles-pp.sync_regs 0.23 +0.0 0.26 ± 3% perf-profile.children.cycles-pp.wake_up_new_task 0.15 ± 6% +0.0 0.18 ± 12% perf-profile.children.cycles-pp.time 0.16 ± 3% +0.0 0.19 ± 5% perf-profile.children.cycles-pp.__close_nocancel 0.19 ± 7% +0.0 0.21 ± 4% perf-profile.children.cycles-pp.__put_anon_vma 0.28 +0.0 0.31 perf-profile.children.cycles-pp.d_path 0.22 ± 3% +0.0 0.25 ± 5% perf-profile.children.cycles-pp.__perf_sw_event 0.21 ± 5% +0.0 0.24 ± 3% perf-profile.children.cycles-pp.setup_arg_pages 0.16 ± 4% +0.0 0.19 ± 6% perf-profile.children.cycles-pp.__get_user_pages_remote 0.15 ± 5% +0.0 0.18 ± 4% perf-profile.children.cycles-pp.copy_string_kernel 0.16 ± 4% +0.0 0.19 ± 5% perf-profile.children.cycles-pp.__read_nocancel 0.02 ±122% +0.0 0.05 perf-profile.children.cycles-pp.vm_brk_flags 0.02 ±122% +0.0 0.05 perf-profile.children.cycles-pp.security_file_open 0.02 ±123% +0.0 0.05 ± 7% perf-profile.children.cycles-pp.atime_needs_update 0.00 +0.0 0.03 ± 81% perf-profile.children.cycles-pp.__perf_event__output_id_sample 0.00 +0.0 0.03 ± 81% perf-profile.children.cycles-pp.__perf_event_header__init_id 0.00 +0.0 0.03 ± 81% perf-profile.children.cycles-pp.cgroup_throttle_swaprate 0.00 +0.0 0.03 ± 81% perf-profile.children.cycles-pp.complete_walk 0.23 ± 3% +0.0 0.26 ± 2% perf-profile.children.cycles-pp.change_protection 0.16 ± 5% +0.0 0.19 ± 6% perf-profile.children.cycles-pp.__get_user_pages 0.19 ± 6% +0.0 0.22 ± 2% perf-profile.children.cycles-pp.shift_arg_pages 0.27 ± 3% +0.0 0.30 ± 2% perf-profile.children.cycles-pp.unlock_page 0.01 ±200% +0.0 0.04 ± 51% perf-profile.children.cycles-pp.__generic_file_write_iter 0.01 ±200% +0.0 0.04 ± 51% perf-profile.children.cycles-pp.generic_perform_write 0.01 ±200% +0.0 0.05 ± 52% perf-profile.children.cycles-pp.generic_file_write_iter 0.02 ±123% +0.0 0.06 ± 53% perf-profile.children.cycles-pp.__libc_write 0.37 ± 4% +0.0 0.40 ± 4% perf-profile.children.cycles-pp.__do_fault 0.24 ± 4% +0.0 0.27 ± 5% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.18 ± 4% +0.0 0.22 ± 3% perf-profile.children.cycles-pp.kfree 0.18 ± 4% +0.0 0.21 ± 3% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm 0.32 ± 3% +0.0 0.35 ± 3% perf-profile.children.cycles-pp.find_idlest_group 0.37 ± 4% +0.0 0.40 ± 3% perf-profile.children.cycles-pp.schedule 0.36 ± 2% +0.0 0.39 ± 2% perf-profile.children.cycles-pp.do_wait 0.31 ± 3% +0.0 0.35 ± 5% perf-profile.children.cycles-pp.__alloc_file 0.01 ±200% +0.0 0.05 perf-profile.children.cycles-pp.stress_vecmath._GLOBAL___stress_vecmath_info.ifunc 0.00 +0.0 0.04 ± 50% perf-profile.children.cycles-pp.vma_gap_callbacks_rotate 0.00 +0.0 0.04 ± 50% perf-profile.children.cycles-pp.sysfs_kf_seq_show 0.00 +0.0 0.04 ± 50% perf-profile.children.cycles-pp.ima_file_check 0.51 +0.0 0.55 ± 3% perf-profile.children.cycles-pp.__rb_insert_augmented 0.36 ± 3% +0.0 0.40 ± 3% perf-profile.children.cycles-pp.perf_iterate_sb 0.35 ± 2% +0.0 0.39 ± 3% perf-profile.children.cycles-pp.vfs_read 0.31 ± 3% +0.0 0.35 ± 3% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.43 ± 3% +0.0 0.48 ± 2% perf-profile.children.cycles-pp.alloc_pages_vma 0.35 ± 4% +0.0 0.39 ± 6% perf-profile.children.cycles-pp.finish_fault 0.21 ± 2% +0.0 0.25 ± 4% perf-profile.children.cycles-pp.put_task_stack 0.11 ± 10% +0.0 0.15 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.34 ± 4% +0.0 0.38 ± 5% perf-profile.children.cycles-pp.osq_unlock 0.37 ± 2% +0.0 0.41 ± 2% perf-profile.children.cycles-pp.kernel_wait4 0.42 ± 3% +0.0 0.46 ± 4% perf-profile.children.cycles-pp.select_task_rq_fair 0.33 +0.0 0.37 ± 3% perf-profile.children.cycles-pp.task_work_run 0.36 ± 2% +0.0 0.40 ± 3% perf-profile.children.cycles-pp.ksys_read 0.37 ± 2% +0.0 0.42 ± 2% perf-profile.children.cycles-pp.__do_sys_wait4 0.31 ± 3% +0.0 0.35 ± 4% perf-profile.children.cycles-pp.alloc_empty_file 0.01 ±200% +0.0 0.05 ± 9% perf-profile.children.cycles-pp.__x64_sys_close 0.54 ± 2% +0.0 0.58 ± 2% perf-profile.children.cycles-pp.copy_page 0.53 ± 3% +0.0 0.58 ± 4% perf-profile.children.cycles-pp.native_irq_return_iret 0.36 ± 3% +0.0 0.41 perf-profile.children.cycles-pp._raw_spin_lock 0.31 ± 2% +0.0 0.35 ± 3% perf-profile.children.cycles-pp.__clear_user 0.01 ±200% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.filp_close 0.00 +0.0 0.05 ± 52% perf-profile.children.cycles-pp.mem_cgroup_from_task 0.34 ± 2% +0.0 0.38 ± 5% perf-profile.children.cycles-pp.try_charge 0.30 ± 2% +0.0 0.34 ± 3% perf-profile.children.cycles-pp.__fput 0.24 +0.0 0.29 ± 3% perf-profile.children.cycles-pp.__count_memcg_events 0.49 ± 3% +0.0 0.54 ± 3% perf-profile.children.cycles-pp.get_page_from_freelist 0.37 ± 3% +0.0 0.42 ± 2% perf-profile.children.cycles-pp.page_add_file_rmap 0.29 ± 3% +0.0 0.34 ± 3% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list 0.27 ± 3% +0.0 0.32 ± 3% perf-profile.children.cycles-pp.uncharge_batch 0.27 ± 2% +0.0 0.32 ± 5% perf-profile.children.cycles-pp.lock_page_memcg 0.36 ± 3% +0.0 0.41 ± 3% perf-profile.children.cycles-pp.__cpu_indicator_init 0.30 ± 6% +0.0 0.35 ± 8% perf-profile.children.cycles-pp.lru_cache_add 0.27 ± 3% +0.0 0.32 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 0.36 ± 7% +0.1 0.41 ± 7% perf-profile.children.cycles-pp.__x64_sys_munmap 0.24 ± 2% +0.1 0.29 ± 4% perf-profile.children.cycles-pp.__memcg_kmem_uncharge_page 0.48 ± 2% +0.1 0.53 perf-profile.children.cycles-pp.find_vma 0.42 ± 2% +0.1 0.47 ± 3% perf-profile.children.cycles-pp.pte_alloc_one 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.queued_write_lock_slowpath 0.18 ± 3% +0.1 0.24 ± 5% perf-profile.children.cycles-pp.schedule_idle 0.57 ± 2% +0.1 0.63 ± 2% perf-profile.children.cycles-pp._dl_addr 0.34 ± 2% +0.1 0.41 ± 4% perf-profile.children.cycles-pp.__mod_memcg_state 0.42 ± 2% +0.1 0.49 ± 5% perf-profile.children.cycles-pp.__anon_vma_prepare 0.33 ± 2% +0.1 0.40 ± 5% perf-profile.children.cycles-pp.finish_task_switch 0.78 ± 3% +0.1 0.86 ± 5% perf-profile.children.cycles-pp.load_elf_interp 0.50 ± 5% +0.1 0.57 ± 5% perf-profile.children.cycles-pp.unlink_anon_vmas 0.12 ± 11% +0.1 0.19 ± 10% perf-profile.children.cycles-pp.rwsem_wake 0.57 ± 2% +0.1 0.65 ± 3% perf-profile.children.cycles-pp.mem_cgroup_charge 0.93 ± 2% +0.1 1.01 ± 3% perf-profile.children.cycles-pp.vma_interval_tree_remove 0.38 ± 2% +0.1 0.46 ± 3% perf-profile.children.cycles-pp.propagate_protected_usage 0.48 ± 8% +0.1 0.56 ± 9% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.81 ± 2% +0.1 0.90 ± 3% perf-profile.children.cycles-pp.page_remove_rmap 0.54 ± 2% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.52 ± 5% +0.1 0.60 ± 4% perf-profile.children.cycles-pp.vm_area_alloc 0.62 ± 4% +0.1 0.71 ± 4% perf-profile.children.cycles-pp.vm_area_dup 0.54 ± 3% +0.1 0.63 ± 5% perf-profile.children.cycles-pp.up_write 0.63 ± 3% +0.1 0.72 ± 3% perf-profile.children.cycles-pp.__schedule 0.70 ± 2% +0.1 0.79 ± 3% perf-profile.children.cycles-pp.do_anonymous_page 0.60 +0.1 0.70 ± 3% perf-profile.children.cycles-pp.down_write 0.51 ± 2% +0.1 0.60 ± 4% perf-profile.children.cycles-pp.__memcg_kmem_charge_page 0.95 ± 2% +0.1 1.05 perf-profile.children.cycles-pp.perf_event_mmap 0.57 ± 3% +0.1 0.68 ± 2% perf-profile.children.cycles-pp.obj_cgroup_charge 0.82 ± 5% +0.1 0.93 ± 7% perf-profile.children.cycles-pp.__vm_munmap 0.66 ± 2% +0.1 0.77 ± 2% perf-profile.children.cycles-pp.copy_process 1.03 ± 6% +0.1 1.15 ± 2% perf-profile.children.cycles-pp.rcu_do_batch 0.92 ± 3% +0.1 1.03 ± 4% perf-profile.children.cycles-pp.alloc_set_pte 1.05 ± 6% +0.1 1.17 ± 2% perf-profile.children.cycles-pp.rcu_core 1.05 ± 2% +0.1 1.18 ± 3% perf-profile.children.cycles-pp.__alloc_pages_nodemask 1.10 ± 2% +0.1 1.23 ± 3% perf-profile.children.cycles-pp.wp_page_copy 0.83 ± 2% +0.1 0.96 ± 2% perf-profile.children.cycles-pp.__memcg_kmem_uncharge 1.68 ± 4% +0.1 1.82 ± 5% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 1.04 ± 5% +0.1 1.18 ± 3% perf-profile.children.cycles-pp.do_softirq_own_stack 0.73 +0.1 0.87 ± 4% perf-profile.children.cycles-pp.__memcg_kmem_charge 0.97 +0.1 1.12 ± 2% perf-profile.children.cycles-pp.__do_sys_clone 0.97 +0.1 1.12 ± 2% perf-profile.children.cycles-pp.kernel_clone 1.06 ± 5% +0.1 1.21 ± 4% perf-profile.children.cycles-pp.irq_exit_rcu 1.74 ± 4% +0.2 1.89 ± 5% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 1.11 ± 5% +0.2 1.26 ± 3% perf-profile.children.cycles-pp.__softirqentry_text_start 1.80 ± 5% +0.2 1.96 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.78 +0.2 0.94 ± 3% perf-profile.children.cycles-pp.page_counter_try_charge 1.06 ± 4% +0.2 1.23 ± 5% perf-profile.children.cycles-pp.remove_vma 0.68 ± 3% +0.2 0.86 ± 4% perf-profile.children.cycles-pp.vma_merge 1.19 ± 3% +0.2 1.38 ± 3% perf-profile.children.cycles-pp.drain_obj_stock 2.07 ± 2% +0.2 2.27 ± 3% perf-profile.children.cycles-pp.zap_pte_range 1.15 ± 11% +0.2 1.35 ± 12% perf-profile.children.cycles-pp.lru_add_drain_cpu 1.16 ± 11% +0.2 1.36 ± 12% perf-profile.children.cycles-pp.lru_add_drain 1.53 ± 5% +0.2 1.74 ± 8% perf-profile.children.cycles-pp.elf_map 2.23 ± 2% +0.2 2.44 ± 2% perf-profile.children.cycles-pp.unmap_page_range 0.00 +0.2 0.22 ±200% perf-profile.children.cycles-pp.irqentry_exit_to_user_mode 2.29 ± 11% +0.2 2.51 ± 7% perf-profile.children.cycles-pp.intel_idle 2.36 ± 2% +0.2 2.58 ± 2% perf-profile.children.cycles-pp.unmap_vmas 1.43 ± 3% +0.2 1.65 ± 3% perf-profile.children.cycles-pp.refill_obj_stock 1.45 ± 2% +0.2 1.69 ± 3% perf-profile.children.cycles-pp.page_counter_cancel 1.41 ± 10% +0.2 1.65 ± 11% perf-profile.children.cycles-pp.pagevec_lru_move_fn 2.82 +0.2 3.07 ± 2% perf-profile.children.cycles-pp.vma_interval_tree_insert 2.43 ± 11% +0.2 2.68 ± 7% perf-profile.children.cycles-pp.cpuidle_enter 2.43 ± 11% +0.2 2.68 ± 7% perf-profile.children.cycles-pp.cpuidle_enter_state 1.51 ± 2% +0.2 1.75 ± 3% perf-profile.children.cycles-pp.page_counter_uncharge 1.79 ± 5% +0.3 2.08 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc 1.94 ± 6% +0.3 2.24 ± 7% perf-profile.children.cycles-pp.release_pages 2.20 ± 6% +0.3 2.51 ± 6% perf-profile.children.cycles-pp.tlb_finish_mmu 2.55 ± 3% +0.3 2.87 ± 5% perf-profile.children.cycles-pp.load_elf_binary 2.18 ± 6% +0.3 2.49 ± 6% perf-profile.children.cycles-pp.tlb_flush_mmu 2.60 ± 3% +0.3 2.91 ± 5% perf-profile.children.cycles-pp.exec_binprm 2.69 ± 10% +0.3 3.01 ± 7% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 2.69 ± 10% +0.3 3.01 ± 7% perf-profile.children.cycles-pp.cpu_startup_entry 2.69 ± 10% +0.3 3.01 ± 7% perf-profile.children.cycles-pp.do_idle 2.66 ± 11% +0.3 2.98 ± 7% perf-profile.children.cycles-pp.start_secondary 2.30 ± 5% +0.3 2.63 ± 4% perf-profile.children.cycles-pp.kmem_cache_free 2.91 ± 3% +0.3 3.24 ± 5% perf-profile.children.cycles-pp.bprm_execve 1.72 ± 13% +0.4 2.07 ± 14% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.25 ± 3% +0.4 0.60 ±105% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 1.80 ± 13% +0.4 2.16 ± 13% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 3.23 ± 3% +0.4 3.61 ± 5% perf-profile.children.cycles-pp.__x64_sys_execve 3.22 ± 3% +0.4 3.60 ± 5% perf-profile.children.cycles-pp.do_execveat_common 3.16 ± 3% +0.4 3.57 ± 5% perf-profile.children.cycles-pp.filemap_map_pages 2.80 ± 2% +0.5 3.31 ± 4% perf-profile.children.cycles-pp.rwsem_spin_on_owner 5.00 ± 3% +0.6 5.63 ± 4% perf-profile.children.cycles-pp.do_fault 7.50 ± 3% +1.0 8.47 ± 3% perf-profile.children.cycles-pp.__handle_mm_fault 7.92 ± 3% +1.0 8.95 ± 3% perf-profile.children.cycles-pp.handle_mm_fault 8.45 ± 2% +1.1 9.53 ± 3% perf-profile.children.cycles-pp.do_user_addr_fault 8.50 ± 2% +1.1 9.59 ± 3% perf-profile.children.cycles-pp.exc_page_fault 4.61 ± 3% +1.1 5.75 ± 6% perf-profile.children.cycles-pp.mprotect_fixup 4.75 ± 3% +1.2 5.91 ± 6% perf-profile.children.cycles-pp.do_mprotect_pkey 4.75 ± 3% +1.2 5.91 ± 6% perf-profile.children.cycles-pp.__x64_sys_mprotect 8.83 ± 2% +1.3 10.17 ± 6% perf-profile.children.cycles-pp.asm_exc_page_fault 5.12 ± 5% +1.6 6.72 ± 7% perf-profile.children.cycles-pp.vma_link 5.40 ± 6% +1.7 7.14 ± 6% perf-profile.children.cycles-pp.unmap_region 9.34 ± 4% +2.4 11.70 ± 5% perf-profile.children.cycles-pp.exit_mmap 9.35 ± 4% +2.4 11.72 ± 5% perf-profile.children.cycles-pp.mmput 9.96 ± 4% +2.4 12.39 ± 5% perf-profile.children.cycles-pp.do_exit 9.97 ± 4% +2.4 12.39 ± 5% perf-profile.children.cycles-pp.do_group_exit 9.97 ± 4% +2.4 12.39 ± 5% perf-profile.children.cycles-pp.__x64_sys_exit_group 9.86 ± 4% +2.6 12.49 ± 6% perf-profile.children.cycles-pp.__split_vma 9.62 ± 4% +2.7 12.29 ± 6% perf-profile.children.cycles-pp.__vma_adjust 7.50 ± 5% +3.1 10.62 ± 9% perf-profile.children.cycles-pp.unlink_file_vma 8.34 ± 5% +3.3 11.61 ± 8% perf-profile.children.cycles-pp.free_pgtables 11.75 ± 5% +3.4 15.11 ± 6% perf-profile.children.cycles-pp.__do_munmap 17.77 ± 5% +5.2 22.92 ± 6% perf-profile.children.cycles-pp.ksys_mmap_pgoff 18.24 ± 5% +5.2 23.47 ± 6% perf-profile.children.cycles-pp.mmap_region 18.54 ± 5% +5.3 23.80 ± 6% perf-profile.children.cycles-pp.do_mmap 18.78 ± 5% +5.3 24.07 ± 6% perf-profile.children.cycles-pp.vm_mmap_pgoff 15.93 ± 6% +6.7 22.63 ± 9% perf-profile.children.cycles-pp.rwsem_optimistic_spin 16.16 ± 6% +6.7 22.91 ± 8% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 55.52 ± 2% -5.6 49.88 ± 3% perf-profile.self.cycles-pp.osq_lock 0.37 -0.0 0.34 ± 3% perf-profile.self.cycles-pp.__mutex_lock 0.02 ±123% -0.0 0.00 perf-profile.self.cycles-pp.delay_tsc 0.16 ± 17% -0.0 0.15 ± 13% perf-profile.self.cycles-pp.io_serial_in 0.13 ± 3% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.mutex_unlock 0.05 ± 52% -0.0 0.04 ± 51% perf-profile.self.cycles-pp.sys_imageblit 0.27 ± 9% -0.0 0.26 ± 8% perf-profile.self.cycles-pp.memcpy_erms 0.23 ± 3% -0.0 0.23 ± 3% perf-profile.self.cycles-pp.mutex_lock 0.08 ± 30% -0.0 0.08 ± 35% perf-profile.self.cycles-pp.ktime_get 0.09 ± 11% -0.0 0.08 ± 14% perf-profile.self.cycles-pp.update_sd_lb_stats 0.07 ± 5% -0.0 0.07 perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.08 ± 5% -0.0 0.08 ± 9% perf-profile.self.cycles-pp.strlen 0.09 ± 7% +0.0 0.09 perf-profile.self.cycles-pp.filldir64 0.06 ± 10% +0.0 0.06 ± 10% perf-profile.self.cycles-pp.do_mmap 0.05 +0.0 0.05 perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.02 ±122% +0.0 0.02 ±122% perf-profile.self.cycles-pp.PageHuge 0.08 ± 5% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.kernfs_put 0.06 ± 6% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.prepend_path 0.06 ± 6% +0.0 0.06 perf-profile.self.cycles-pp.memset_erms 0.05 +0.0 0.05 ± 7% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size 0.05 +0.0 0.05 ± 9% perf-profile.self.cycles-pp._cond_resched 0.10 ± 6% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.kernfs_fop_readdir 0.07 ± 6% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.__alloc_pages_nodemask 0.06 ± 6% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.fput_many 0.06 ± 6% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.do_dentry_open 0.06 ± 6% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.__list_add_valid 0.06 ± 6% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.__entry_text_start 0.11 ± 4% +0.0 0.11 ± 6% perf-profile.self.cycles-pp.unmap_page_range 0.07 ± 5% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.rmqueue 0.07 ± 5% +0.0 0.08 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.07 ± 6% +0.0 0.08 perf-profile.self.cycles-pp.__do_munmap 0.05 +0.0 0.06 ± 8% perf-profile.self.cycles-pp.acct_collect 0.52 +0.0 0.52 ± 2% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.09 ± 4% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.unmap_single_vma 0.07 ± 10% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.filemap_fault 0.07 ± 9% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.__mod_node_page_state 0.06 ± 10% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.vm_normal_page 0.06 ± 12% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.apparmor_file_alloc_security 0.06 ± 6% +0.0 0.07 ± 17% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.06 ± 6% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.06 ± 7% +0.0 0.07 perf-profile.self.cycles-pp.get_nprocs_conf 0.05 +0.0 0.06 ± 8% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.11 ± 4% +0.0 0.11 ± 7% perf-profile.self.cycles-pp.obj_cgroup_charge 0.11 ± 3% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.__d_lookup_rcu 0.08 ± 5% +0.0 0.09 ± 8% perf-profile.self.cycles-pp.mark_page_accessed 0.06 ± 6% +0.0 0.07 ± 9% perf-profile.self.cycles-pp.__fput 0.07 ± 5% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.__slab_free 0.06 ± 7% +0.0 0.07 ± 5% perf-profile.self.cycles-pp._find_next_bit 0.06 ± 12% +0.0 0.07 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.05 ± 9% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.update_load_avg 0.07 ± 6% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.aa_get_task_label 0.07 ± 5% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.vma_merge 0.05 ± 9% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.do_fault 0.05 ± 9% +0.0 0.06 ± 12% perf-profile.self.cycles-pp.asm_exc_page_fault 0.06 ± 8% +0.0 0.07 ± 12% perf-profile.self.cycles-pp.wp_page_copy 0.00 +0.0 0.01 ±200% perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.0 0.01 ±200% perf-profile.self.cycles-pp.__cpu_indicator_init 0.00 +0.0 0.01 ±200% perf-profile.self.cycles-pp.__vma_rb_erase 0.04 ± 50% +0.0 0.05 perf-profile.self.cycles-pp.vmacache_update 0.13 ± 4% +0.0 0.14 ± 4% perf-profile.self.cycles-pp.___perf_sw_event 0.12 ± 3% +0.0 0.13 ± 3% perf-profile.self.cycles-pp.vm_area_dup 0.07 ± 20% +0.0 0.08 ± 14% perf-profile.self.cycles-pp.xas_start 0.07 ± 7% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.xas_load 0.06 ± 7% +0.0 0.07 ± 6% perf-profile.self.cycles-pp.__unlock_page_memcg 0.06 +0.0 0.07 perf-profile.self.cycles-pp.common_file_perm 0.06 ± 6% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.lockref_put_or_lock 0.14 ± 2% +0.0 0.15 ± 4% perf-profile.self.cycles-pp.handle_mm_fault 0.10 ± 3% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__split_vma 0.11 +0.0 0.12 ± 3% perf-profile.self.cycles-pp.do_user_addr_fault 0.10 ± 4% +0.0 0.11 ± 5% perf-profile.self.cycles-pp.mem_cgroup_charge 0.06 ± 8% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.page_counter_uncharge 0.17 ± 3% +0.0 0.18 ± 5% perf-profile.self.cycles-pp.try_charge 0.10 ± 5% +0.0 0.11 ± 3% perf-profile.self.cycles-pp.d_path 0.10 ± 4% +0.0 0.11 perf-profile.self.cycles-pp.change_p4d_range 0.10 ± 4% +0.0 0.11 perf-profile.self.cycles-pp.prepend_name 0.14 ± 5% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.perf_iterate_sb 0.07 ± 6% +0.0 0.09 ± 8% perf-profile.self.cycles-pp.__perf_sw_event 0.07 ± 7% +0.0 0.08 perf-profile.self.cycles-pp.do_wp_page 0.20 ± 3% +0.0 0.21 ± 3% perf-profile.self.cycles-pp.find_get_entry 0.18 ± 5% +0.0 0.19 ± 3% perf-profile.self.cycles-pp.kernfs_get 0.16 ± 2% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_flush_tlb_one_user 0.13 ± 5% +0.0 0.15 ± 3% perf-profile.self.cycles-pp.vm_unmapped_area 0.15 ± 2% +0.0 0.16 ± 4% perf-profile.self.cycles-pp.__vma_adjust 0.15 ± 2% +0.0 0.16 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.10 ± 4% +0.0 0.12 ± 5% perf-profile.self.cycles-pp.kfree 0.27 ± 2% +0.0 0.29 ± 5% perf-profile.self.cycles-pp.free_pages_and_swap_cache 0.13 ± 5% +0.0 0.15 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid 0.13 ± 3% +0.0 0.14 ± 5% perf-profile.self.cycles-pp.__pagevec_lru_add_fn 0.12 ± 3% +0.0 0.14 ± 5% perf-profile.self.cycles-pp.rb_next 0.16 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.perf_event_mmap 0.20 ± 4% +0.0 0.21 ± 2% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.15 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.16 ± 3% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.vmacache_find 0.14 ± 5% +0.0 0.16 ± 4% perf-profile.self.cycles-pp.alloc_set_pte 0.03 ± 81% +0.0 0.05 perf-profile.self.cycles-pp.kmem_cache_alloc_trace 0.02 ±122% +0.0 0.04 ± 50% perf-profile.self.cycles-pp.__irqentry_text_end 0.02 ±122% +0.0 0.04 ± 50% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.00 +0.0 0.02 ±122% perf-profile.self.cycles-pp.remove_vma 0.00 +0.0 0.02 ±122% perf-profile.self.cycles-pp.update_cfs_group 0.01 ±200% +0.0 0.03 ± 81% perf-profile.self.cycles-pp.perf_event_mmap_output 0.01 ±200% +0.0 0.03 ± 81% perf-profile.self.cycles-pp.update_rq_clock 0.18 ± 6% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.vma_interval_tree_augment_rotate 0.14 ± 9% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.drain_obj_stock 0.03 ± 81% +0.0 0.05 ± 7% perf-profile.self.cycles-pp.__schedule 0.25 ± 2% +0.0 0.27 ± 2% perf-profile.self.cycles-pp.clear_page_erms 0.21 ± 3% +0.0 0.23 perf-profile.self.cycles-pp.page_add_file_rmap 0.22 ± 3% +0.0 0.24 ± 3% perf-profile.self.cycles-pp.__vma_link_rb 0.03 ± 81% +0.0 0.05 ± 9% perf-profile.self.cycles-pp.update_curr 0.23 ± 4% +0.0 0.26 perf-profile.self.cycles-pp.__rb_erase_color 0.24 ± 2% +0.0 0.26 ± 3% perf-profile.self.cycles-pp.___might_sleep 0.27 ± 2% +0.0 0.30 ± 3% perf-profile.self.cycles-pp.find_vma 0.19 ± 8% +0.0 0.21 ± 7% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.16 ± 2% +0.0 0.19 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.35 ± 2% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.sync_regs 0.30 ± 3% +0.0 0.33 perf-profile.self.cycles-pp._raw_spin_lock 0.28 ± 3% +0.0 0.31 ± 3% perf-profile.self.cycles-pp.find_idlest_group 0.02 ±122% +0.0 0.05 perf-profile.self.cycles-pp.exc_page_fault 0.02 ±122% +0.0 0.05 perf-profile.self.cycles-pp.unlink_anon_vmas 0.02 ±122% +0.0 0.05 perf-profile.self.cycles-pp.rcu_all_qs 0.02 ±122% +0.0 0.05 perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.00 +0.0 0.03 ± 81% perf-profile.self.cycles-pp.vma_gap_callbacks_rotate 0.27 ± 5% +0.0 0.30 ± 2% perf-profile.self.cycles-pp.mmap_region 0.26 ± 3% +0.0 0.29 ± 2% perf-profile.self.cycles-pp.unlock_page 0.23 ± 8% +0.0 0.26 ± 6% perf-profile.self.cycles-pp.refill_obj_stock 0.17 ± 4% +0.0 0.20 perf-profile.self.cycles-pp.get_mem_cgroup_from_mm 0.00 +0.0 0.04 ± 83% perf-profile.self.cycles-pp.mem_cgroup_from_task 0.48 +0.0 0.52 ± 3% perf-profile.self.cycles-pp.__rb_insert_augmented 0.00 +0.0 0.04 ± 50% perf-profile.self.cycles-pp.link_path_walk 0.00 +0.0 0.04 ± 50% perf-profile.self.cycles-pp.page_add_new_anon_rmap 0.24 +0.0 0.28 ± 3% perf-profile.self.cycles-pp.__count_memcg_events 0.33 ± 4% +0.0 0.37 ± 5% perf-profile.self.cycles-pp.osq_unlock 0.01 ±200% +0.0 0.05 ± 7% perf-profile.self.cycles-pp.up_read 0.52 ± 3% +0.0 0.57 ± 2% perf-profile.self.cycles-pp.copy_page 0.26 ± 2% +0.0 0.30 ± 5% perf-profile.self.cycles-pp.lock_page_memcg 0.53 ± 2% +0.0 0.58 ± 4% perf-profile.self.cycles-pp.native_irq_return_iret 0.55 ± 2% +0.1 0.61 ± 2% perf-profile.self.cycles-pp._dl_addr 0.59 ± 3% +0.1 0.64 ± 3% perf-profile.self.cycles-pp.page_remove_rmap 0.41 ± 4% +0.1 0.47 ± 4% perf-profile.self.cycles-pp.__handle_mm_fault 0.33 ± 3% +0.1 0.40 ± 4% perf-profile.self.cycles-pp.__mod_memcg_state 0.63 ± 2% +0.1 0.70 ± 2% perf-profile.self.cycles-pp.release_pages 0.44 ± 8% +0.1 0.52 ± 10% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.53 ± 3% +0.1 0.62 ± 5% perf-profile.self.cycles-pp.up_write 0.91 ± 2% +0.1 0.99 ± 3% perf-profile.self.cycles-pp.vma_interval_tree_remove 0.37 ± 2% +0.1 0.45 ± 4% perf-profile.self.cycles-pp.propagate_protected_usage 0.51 +0.1 0.60 ± 3% perf-profile.self.cycles-pp.down_write 0.98 ± 2% +0.1 1.07 ± 2% perf-profile.self.cycles-pp.zap_pte_range 0.31 ± 8% +0.1 0.41 ± 9% perf-profile.self.cycles-pp.rwsem_optimistic_spin 0.69 ± 5% +0.1 0.79 ± 5% perf-profile.self.cycles-pp.kmem_cache_alloc 0.68 +0.1 0.81 ± 3% perf-profile.self.cycles-pp.page_counter_try_charge 0.94 ± 8% +0.1 1.08 ± 8% perf-profile.self.cycles-pp.kmem_cache_free 1.16 ± 2% +0.2 1.34 ± 3% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +0.2 0.21 ±200% perf-profile.self.cycles-pp.irqentry_exit_to_user_mode 2.29 ± 11% +0.2 2.51 ± 7% perf-profile.self.cycles-pp.intel_idle 2.77 +0.2 3.01 ± 2% perf-profile.self.cycles-pp.vma_interval_tree_insert 2.17 ± 4% +0.3 2.45 ± 6% perf-profile.self.cycles-pp.filemap_map_pages 0.00 +0.3 0.33 ±200% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 1.72 ± 13% +0.4 2.07 ± 14% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.75 +0.5 3.25 ± 4% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.00 -100.0% 0.00 interrupts.0:IO-APIC.2-edge.timer 0.20 ±200% +300.0% 0.80 ±122% interrupts.165:PCI-MSI.31983617-edge.i40e-eth1-TxRx-0 0.00 +3.1e+103% 31.00 ±182% interrupts.166:PCI-MSI.31983618-edge.i40e-eth1-TxRx-1 66.60 ±199% -100.0% 0.00 interrupts.167:PCI-MSI.31983619-edge.i40e-eth1-TxRx-2 2.20 ±158% -18.2% 1.80 ±200% interrupts.169:PCI-MSI.31983621-edge.i40e-eth1-TxRx-4 0.00 +4e+101% 0.40 ±200% interrupts.170:PCI-MSI.31983622-edge.i40e-eth1-TxRx-5 0.40 ±200% -50.0% 0.20 ±200% interrupts.171:PCI-MSI.31983623-edge.i40e-eth1-TxRx-6 0.20 ±200% -100.0% 0.00 interrupts.172:PCI-MSI.31983624-edge.i40e-eth1-TxRx-7 0.00 +4e+101% 0.40 ±200% interrupts.173:PCI-MSI.31983625-edge.i40e-eth1-TxRx-8 0.00 -100.0% 0.00 interrupts.174:PCI-MSI.31983626-edge.i40e-eth1-TxRx-9 26.40 ±196% -97.0% 0.80 ±200% interrupts.175:PCI-MSI.31983627-edge.i40e-eth1-TxRx-10 0.00 -100.0% 0.00 interrupts.177:PCI-MSI.31983629-edge.i40e-eth1-TxRx-12 42.00 ±200% -100.0% 0.00 interrupts.178:PCI-MSI.31983630-edge.i40e-eth1-TxRx-13 1.80 ±200% -100.0% 0.00 interrupts.179:PCI-MSI.31983631-edge.i40e-eth1-TxRx-14 0.40 ±200% -100.0% 0.00 interrupts.180:PCI-MSI.31983632-edge.i40e-eth1-TxRx-15 0.00 +4.8e+102% 4.80 ±125% interrupts.181:PCI-MSI.31983633-edge.i40e-eth1-TxRx-16 0.00 -100.0% 0.00 interrupts.182:PCI-MSI.31983634-edge.i40e-eth1-TxRx-17 22.00 ±200% -93.6% 1.40 ±139% interrupts.183:PCI-MSI.31983635-edge.i40e-eth1-TxRx-18 1.40 ±200% -100.0% 0.00 interrupts.184:PCI-MSI.31983636-edge.i40e-eth1-TxRx-19 0.00 -100.0% 0.00 interrupts.185:PCI-MSI.31983637-edge.i40e-eth1-TxRx-20 3.40 ±200% -100.0% 0.00 interrupts.186:PCI-MSI.31983638-edge.i40e-eth1-TxRx-21 0.60 ±133% +200.0% 1.80 ±200% interrupts.187:PCI-MSI.31983639-edge.i40e-eth1-TxRx-22 0.00 -100.0% 0.00 interrupts.188:PCI-MSI.31983640-edge.i40e-eth1-TxRx-23 0.60 ±200% +66.7% 1.00 ±200% interrupts.189:PCI-MSI.31983641-edge.i40e-eth1-TxRx-24 2.00 ±154% -70.0% 0.60 ±200% interrupts.191:PCI-MSI.31983643-edge.i40e-eth1-TxRx-26 0.80 ±200% +300.0% 3.20 ±170% interrupts.192:PCI-MSI.31983644-edge.i40e-eth1-TxRx-27 66.80 ±200% -99.4% 0.40 ±200% interrupts.193:PCI-MSI.31983645-edge.i40e-eth1-TxRx-28 1.20 ±133% -100.0% 0.00 interrupts.194:PCI-MSI.31983646-edge.i40e-eth1-TxRx-29 0.00 +8e+101% 0.80 ±122% interrupts.195:PCI-MSI.31983647-edge.i40e-eth1-TxRx-30 0.00 +2e+102% 2.00 ±200% interrupts.196:PCI-MSI.31983648-edge.i40e-eth1-TxRx-31 186.20 ±199% -99.0% 1.80 ±133% interrupts.197:PCI-MSI.31983649-edge.i40e-eth1-TxRx-32 1.80 ±200% -100.0% 0.00 interrupts.199:PCI-MSI.31983651-edge.i40e-eth1-TxRx-34 1.40 ±200% -100.0% 0.00 interrupts.200:PCI-MSI.31983652-edge.i40e-eth1-TxRx-35 0.40 ±200% +3900.0% 16.00 ±165% interrupts.201:PCI-MSI.31983653-edge.i40e-eth1-TxRx-36 0.20 ±200% +200.0% 0.60 ±200% interrupts.202:PCI-MSI.31983654-edge.i40e-eth1-TxRx-37 1.80 ±133% -77.8% 0.40 ±200% interrupts.203:PCI-MSI.31983655-edge.i40e-eth1-TxRx-38 0.80 ±200% -100.0% 0.00 interrupts.204:PCI-MSI.31983656-edge.i40e-eth1-TxRx-39 3.40 ±110% -35.3% 2.20 ±129% interrupts.205:PCI-MSI.31983657-edge.i40e-eth1-TxRx-40 0.00 +2.2e+103% 21.80 ±195% interrupts.206:PCI-MSI.31983658-edge.i40e-eth1-TxRx-41 0.00 +2.4e+103% 24.00 ±189% interrupts.208:PCI-MSI.31983660-edge.i40e-eth1-TxRx-43 0.00 +4e+101% 0.40 ±200% interrupts.209:PCI-MSI.31983661-edge.i40e-eth1-TxRx-44 0.20 ±200% +900.0% 2.00 ±200% interrupts.211:PCI-MSI.31983663-edge.i40e-eth1-TxRx-46 0.00 +2e+101% 0.20 ±200% interrupts.213:PCI-MSI.31983665-edge.i40e-eth1-TxRx-48 0.40 ±200% +150.0% 1.00 ±200% interrupts.214:PCI-MSI.31983666-edge.i40e-eth1-TxRx-49 0.00 +2.4e+102% 2.40 ±161% interrupts.215:PCI-MSI.31983667-edge.i40e-eth1-TxRx-50 0.80 ±200% -100.0% 0.00 interrupts.216:PCI-MSI.31983668-edge.i40e-eth1-TxRx-51 1.80 ±200% -88.9% 0.20 ±200% interrupts.217:PCI-MSI.31983669-edge.i40e-eth1-TxRx-52 0.00 +1.6e+103% 15.60 ±199% interrupts.218:PCI-MSI.31983670-edge.i40e-eth1-TxRx-53 50.80 ±191% -100.0% 0.00 interrupts.219:PCI-MSI.31983671-edge.i40e-eth1-TxRx-54 1.80 ±200% +66.7% 3.00 ±200% interrupts.220:PCI-MSI.31983672-edge.i40e-eth1-TxRx-55 1.80 ±123% -22.2% 1.40 ±200% interrupts.221:PCI-MSI.31983673-edge.i40e-eth1-TxRx-56 0.40 ±200% +46400.0% 186.00 ±200% interrupts.222:PCI-MSI.31983674-edge.i40e-eth1-TxRx-57 0.00 +2.6e+102% 2.60 ±136% interrupts.223:PCI-MSI.31983675-edge.i40e-eth1-TxRx-58 0.80 ±200% +975.0% 8.60 ±200% interrupts.224:PCI-MSI.31983676-edge.i40e-eth1-TxRx-59 5.20 ±118% -80.8% 1.00 ±200% interrupts.225:PCI-MSI.31983677-edge.i40e-eth1-TxRx-60 10.80 ±200% -37.0% 6.80 ±185% interrupts.226:PCI-MSI.31983678-edge.i40e-eth1-TxRx-61 9.60 ±200% -85.4% 1.40 ±200% interrupts.227:PCI-MSI.31983679-edge.i40e-eth1-TxRx-62 1.40 ±200% -100.0% 0.00 interrupts.228:PCI-MSI.31983680-edge.i40e-eth1-TxRx-63 4.40 ±123% -100.0% 0.00 interrupts.229:PCI-MSI.31983681-edge.i40e-eth1-TxRx-64 40.00 ±200% -100.0% 0.00 interrupts.230:PCI-MSI.31983682-edge.i40e-eth1-TxRx-65 0.00 -100.0% 0.00 interrupts.231:PCI-MSI.31983683-edge.i40e-eth1-TxRx-66 0.00 +1.8e+102% 1.80 ±200% interrupts.232:PCI-MSI.31983684-edge.i40e-eth1-TxRx-67 9.80 ±171% -100.0% 0.00 interrupts.233:PCI-MSI.31983685-edge.i40e-eth1-TxRx-68 1.20 ±133% -100.0% 0.00 interrupts.234:PCI-MSI.31983686-edge.i40e-eth1-TxRx-69 0.00 +1.4e+102% 1.40 ±200% interrupts.236:PCI-MSI.31983688-edge.i40e-eth1-TxRx-71 0.00 +1.2e+102% 1.20 ±200% interrupts.237:PCI-MSI.31983689-edge.i40e-eth1-TxRx-72 19.60 ±160% -100.0% 0.00 interrupts.238:PCI-MSI.31983690-edge.i40e-eth1-TxRx-73 188.40 ±198% -100.0% 0.00 interrupts.239:PCI-MSI.31983691-edge.i40e-eth1-TxRx-74 0.00 -100.0% 0.00 interrupts.240:PCI-MSI.31983692-edge.i40e-eth1-TxRx-75 0.00 -100.0% 0.00 interrupts.242:PCI-MSI.31983694-edge.i40e-eth1-TxRx-77 0.00 +3.6e+102% 3.60 ±141% interrupts.243:PCI-MSI.31983695-edge.i40e-eth1-TxRx-78 0.00 +4e+101% 0.40 ±200% interrupts.245:PCI-MSI.31983697-edge.i40e-eth1-TxRx-80 0.00 +6e+101% 0.60 ±200% interrupts.246:PCI-MSI.31983698-edge.i40e-eth1-TxRx-81 26.80 ±196% -100.0% 0.00 interrupts.247:PCI-MSI.31983699-edge.i40e-eth1-TxRx-82 0.00 +6.6e+103% 66.00 ±194% interrupts.248:PCI-MSI.31983700-edge.i40e-eth1-TxRx-83 0.00 +1e+103% 10.40 ±200% interrupts.251:PCI-MSI.31983703-edge.i40e-eth1-TxRx-86 0.40 ±200% +15500.0% 62.40 ±189% interrupts.252:PCI-MSI.31983704-edge.i40e-eth1-TxRx-87 22.00 ±200% -100.0% 0.00 interrupts.253:PCI-MSI.31983705-edge.i40e-eth1-TxRx-88 0.00 -100.0% 0.00 interrupts.255:PCI-MSI.31983707-edge.i40e-eth1-TxRx-90 2.80 ±200% -42.9% 1.60 ±200% interrupts.256:PCI-MSI.31983708-edge.i40e-eth1-TxRx-91 16.20 ±193% -100.0% 0.00 interrupts.257:PCI-MSI.31983709-edge.i40e-eth1-TxRx-92 1.60 ±145% -100.0% 0.00 interrupts.258:PCI-MSI.31983710-edge.i40e-eth1-TxRx-93 0.00 -100.0% 0.00 interrupts.293:PCI-MSI.327680-edge.xhci_hcd 0.00 -100.0% 0.00 interrupts.295:PCI-MSI.65536-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.297:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.298:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.298:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.299:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.299:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.300:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.300:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.301:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.301:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.302:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.302:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.303:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.303:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.304:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.305:PCI-MSI.376832-edge.ahci[0000:00:17.0] 0.00 -100.0% 0.00 interrupts.305:PCI-MSI.67174400-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.306:PCI-MSI.67174400-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.307:PCI-MSI.67174400-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.307:PCI-MSI.67176448-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.308:PCI-MSI.67176448-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.308:PCI-MSI.67178496-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.309:PCI-MSI.67176448-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.309:PCI-MSI.67178496-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.309:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.310:PCI-MSI.67178496-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.310:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.310:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.311:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.311:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.311:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.312:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.312:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.312:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.313:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.313:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.313:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.314:PCI-MSI.376832-edge.ahci[0000:00:17.0] 0.00 -100.0% 0.00 interrupts.314:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.314:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.315:PCI-MSI.376832-edge.ahci[0000:00:17.0] 0.00 -100.0% 0.00 interrupts.315:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.3:IO-APIC.3-edge 28.60 ± 82% +25.2% 35.80 ± 51% interrupts.4:IO-APIC.4-edge.ttyS0 0.00 -100.0% 0.00 interrupts.8:IO-APIC.8-edge.rtc0 0.00 -100.0% 0.00 interrupts.9:IO-APIC.9-fasteoi.acpi 430443 ± 2% +18.5% 509902 ± 3% interrupts.CAL:Function_call_interrupts 0.00 -100.0% 0.00 interrupts.CPU0.0:IO-APIC.2-edge.timer 0.00 +6e+101% 0.60 ±133% interrupts.CPU0.165:PCI-MSI.31983617-edge.i40e-eth1-TxRx-0 0.20 ±200% -100.0% 0.00 interrupts.CPU0.202:PCI-MSI.31983654-edge.i40e-eth1-TxRx-37 0.00 -100.0% 0.00 interrupts.CPU0.295:PCI-MSI.65536-edge.ioat-msix 4518 ± 10% +23.8% 5593 ± 10% interrupts.CPU0.CAL:Function_call_interrupts 9.00 ± 17% +22.2% 11.00 ± 18% interrupts.CPU0.IWI:IRQ_work_interrupts 68000 +0.3% 68188 interrupts.CPU0.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU0.MCP:Machine_check_polls 5704 ± 32% +5.1% 5997 ± 33% interrupts.CPU0.NMI:Non-maskable_interrupts 5704 ± 32% +5.1% 5997 ± 33% interrupts.CPU0.PMI:Performance_monitoring_interrupts 2404 ± 8% +30.6% 3141 ± 8% interrupts.CPU0.RES:Rescheduling_interrupts 0.00 -100.0% 0.00 interrupts.CPU0.RTR:APIC_ICR_read_retries 191.60 ± 20% +22.9% 235.40 ± 27% interrupts.CPU0.TLB:TLB_shootdowns 0.00 +3.1e+103% 30.60 ±185% interrupts.CPU1.166:PCI-MSI.31983618-edge.i40e-eth1-TxRx-1 0.00 +2e+101% 0.20 ±200% interrupts.CPU1.203:PCI-MSI.31983655-edge.i40e-eth1-TxRx-38 0.00 -100.0% 0.00 interrupts.CPU1.251:PCI-MSI.31983703-edge.i40e-eth1-TxRx-86 0.00 -100.0% 0.00 interrupts.CPU1.315:PCI-MSI.376832-edge.ahci[0000:00:17.0] 0.00 -100.0% 0.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 5235 ± 8% +17.8% 6165 ± 11% interrupts.CPU1.CAL:Function_call_interrupts 9.40 ± 14% +12.8% 10.60 ± 14% interrupts.CPU1.IWI:IRQ_work_interrupts 68104 -0.0% 68097 interrupts.CPU1.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU1.MCP:Machine_check_polls 7072 ± 5% -4.4% 6763 ± 25% interrupts.CPU1.NMI:Non-maskable_interrupts 7072 ± 5% -4.4% 6763 ± 25% interrupts.CPU1.PMI:Performance_monitoring_interrupts 2715 ± 10% +20.1% 3259 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts 163.40 ± 12% +60.7% 262.60 ± 17% interrupts.CPU1.TLB:TLB_shootdowns 26.40 ±196% -97.0% 0.80 ±200% interrupts.CPU10.175:PCI-MSI.31983627-edge.i40e-eth1-TxRx-10 0.00 -100.0% 0.00 interrupts.CPU10.293:PCI-MSI.327680-edge.xhci_hcd 0.00 -100.0% 0.00 interrupts.CPU10.298:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU10.299:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU10.299:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU10.301:PCI-MSI.75776-edge.ioat-msix 4360 ± 6% +17.3% 5117 ± 9% interrupts.CPU10.CAL:Function_call_interrupts 9.40 ± 15% +12.8% 10.60 ± 12% interrupts.CPU10.IWI:IRQ_work_interrupts 68050 +0.5% 68364 interrupts.CPU10.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU10.MCP:Machine_check_polls 7140 ± 5% -6.2% 6696 ± 24% interrupts.CPU10.NMI:Non-maskable_interrupts 7140 ± 5% -6.2% 6696 ± 24% interrupts.CPU10.PMI:Performance_monitoring_interrupts 2476 ± 10% +30.4% 3229 ± 12% interrupts.CPU10.RES:Rescheduling_interrupts 220.20 ± 16% +25.1% 275.40 ± 27% interrupts.CPU10.TLB:TLB_shootdowns 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU11.165:PCI-MSI.31983617-edge.i40e-eth1-TxRx-0 0.00 +2e+101% 0.20 ±200% interrupts.CPU11.213:PCI-MSI.31983665-edge.i40e-eth1-TxRx-48 0.00 -100.0% 0.00 interrupts.CPU11.300:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU11.8:IO-APIC.8-edge.rtc0 4351 ± 5% +20.7% 5254 ± 2% interrupts.CPU11.CAL:Function_call_interrupts 9.00 ± 12% +17.8% 10.60 ± 12% interrupts.CPU11.IWI:IRQ_work_interrupts 68094 +0.2% 68224 interrupts.CPU11.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU11.MCP:Machine_check_polls 7088 ± 5% -5.2% 6718 ± 23% interrupts.CPU11.NMI:Non-maskable_interrupts 7088 ± 5% -5.2% 6718 ± 23% interrupts.CPU11.PMI:Performance_monitoring_interrupts 2514 ± 9% +27.9% 3216 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts 169.60 ± 21% +77.2% 300.60 ± 11% interrupts.CPU11.TLB:TLB_shootdowns 0.00 +4e+101% 0.40 ±122% interrupts.CPU12.166:PCI-MSI.31983618-edge.i40e-eth1-TxRx-1 0.00 -100.0% 0.00 interrupts.CPU12.177:PCI-MSI.31983629-edge.i40e-eth1-TxRx-12 0.00 +2e+101% 0.20 ±200% interrupts.CPU12.214:PCI-MSI.31983666-edge.i40e-eth1-TxRx-49 0.00 -100.0% 0.00 interrupts.CPU12.298:PCI-MSI.67584-edge.ioat-msix 4241 ± 5% +22.7% 5204 ± 2% interrupts.CPU12.CAL:Function_call_interrupts 9.00 ± 12% +28.9% 11.60 ± 21% interrupts.CPU12.IWI:IRQ_work_interrupts 68107 +0.3% 68293 interrupts.CPU12.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU12.MCP:Machine_check_polls 7045 ± 5% -4.2% 6751 ± 24% interrupts.CPU12.NMI:Non-maskable_interrupts 7045 ± 5% -4.2% 6751 ± 24% interrupts.CPU12.PMI:Performance_monitoring_interrupts 2507 ± 10% +28.3% 3216 ± 8% interrupts.CPU12.RES:Rescheduling_interrupts 193.80 ± 20% +64.7% 319.20 ± 17% interrupts.CPU12.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU13.167:PCI-MSI.31983619-edge.i40e-eth1-TxRx-2 42.00 ±200% -100.0% 0.00 interrupts.CPU13.178:PCI-MSI.31983630-edge.i40e-eth1-TxRx-13 0.00 -100.0% 0.00 interrupts.CPU13.215:PCI-MSI.31983667-edge.i40e-eth1-TxRx-50 0.00 -100.0% 0.00 interrupts.CPU13.299:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU13.299:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU13.300:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU13.300:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU13.301:PCI-MSI.75776-edge.ioat-msix 0.00 +7.6e+102% 7.60 ±200% interrupts.CPU13.4:IO-APIC.4-edge.ttyS0 4414 ± 3% +23.3% 5443 ± 3% interrupts.CPU13.CAL:Function_call_interrupts 9.20 ± 15% +19.6% 11.00 ± 11% interrupts.CPU13.IWI:IRQ_work_interrupts 68067 +0.5% 68405 interrupts.CPU13.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU13.MCP:Machine_check_polls 7045 ± 5% +5.4% 7423 ± 6% interrupts.CPU13.NMI:Non-maskable_interrupts 7045 ± 5% +5.4% 7423 ± 6% interrupts.CPU13.PMI:Performance_monitoring_interrupts 2614 ± 10% +27.8% 3341 ± 7% interrupts.CPU13.RES:Rescheduling_interrupts 178.60 ± 21% +57.6% 281.40 ± 13% interrupts.CPU13.TLB:TLB_shootdowns 1.80 ±200% -100.0% 0.00 interrupts.CPU14.179:PCI-MSI.31983631-edge.i40e-eth1-TxRx-14 0.00 -100.0% 0.00 interrupts.CPU14.216:PCI-MSI.31983668-edge.i40e-eth1-TxRx-51 0.00 -100.0% 0.00 interrupts.CPU14.300:PCI-MSI.71680-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU14.300:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU14.301:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU14.301:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU14.302:PCI-MSI.77824-edge.ioat-msix 4459 ± 3% +18.8% 5300 ± 3% interrupts.CPU14.CAL:Function_call_interrupts 9.00 ± 12% +15.6% 10.40 ± 11% interrupts.CPU14.IWI:IRQ_work_interrupts 68099 +0.3% 68330 interrupts.CPU14.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU14.MCP:Machine_check_polls 7056 ± 5% -15.8% 5940 ± 31% interrupts.CPU14.NMI:Non-maskable_interrupts 7056 ± 5% -15.8% 5940 ± 31% interrupts.CPU14.PMI:Performance_monitoring_interrupts 2605 ± 6% +27.1% 3310 ± 8% interrupts.CPU14.RES:Rescheduling_interrupts 185.80 ± 8% +41.1% 262.20 ± 24% interrupts.CPU14.TLB:TLB_shootdowns 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU15.169:PCI-MSI.31983621-edge.i40e-eth1-TxRx-4 0.20 ±200% -100.0% 0.00 interrupts.CPU15.180:PCI-MSI.31983632-edge.i40e-eth1-TxRx-15 0.00 +2e+101% 0.20 ±200% interrupts.CPU15.217:PCI-MSI.31983669-edge.i40e-eth1-TxRx-52 0.00 -100.0% 0.00 interrupts.CPU15.301:PCI-MSI.73728-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU15.301:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU15.302:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU15.302:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU15.303:PCI-MSI.79872-edge.ioat-msix 10.00 ±200% -100.0% 0.00 interrupts.CPU15.4:IO-APIC.4-edge.ttyS0 4526 ± 4% +19.2% 5394 ± 2% interrupts.CPU15.CAL:Function_call_interrupts 9.20 ± 15% +17.4% 10.80 ± 10% interrupts.CPU15.IWI:IRQ_work_interrupts 68057 +0.4% 68300 interrupts.CPU15.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU15.MCP:Machine_check_polls 7074 ± 5% -6.0% 6650 ± 21% interrupts.CPU15.NMI:Non-maskable_interrupts 7074 ± 5% -6.0% 6650 ± 21% interrupts.CPU15.PMI:Performance_monitoring_interrupts 2639 ± 9% +24.1% 3275 ± 7% interrupts.CPU15.RES:Rescheduling_interrupts 206.20 ± 35% +19.5% 246.40 ± 20% interrupts.CPU15.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU16.170:PCI-MSI.31983622-edge.i40e-eth1-TxRx-5 0.00 +4.6e+102% 4.60 ±127% interrupts.CPU16.181:PCI-MSI.31983633-edge.i40e-eth1-TxRx-16 0.00 -100.0% 0.00 interrupts.CPU16.218:PCI-MSI.31983670-edge.i40e-eth1-TxRx-53 0.00 -100.0% 0.00 interrupts.CPU16.302:PCI-MSI.75776-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU16.302:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU16.303:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU16.303:PCI-MSI.79872-edge.ioat-msix 4458 ± 3% +19.4% 5324 ± 4% interrupts.CPU16.CAL:Function_call_interrupts 8.60 ± 17% +23.3% 10.60 ± 9% interrupts.CPU16.IWI:IRQ_work_interrupts 68043 +0.4% 68310 interrupts.CPU16.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU16.MCP:Machine_check_polls 5745 ± 34% +15.7% 6648 ± 21% interrupts.CPU16.NMI:Non-maskable_interrupts 5745 ± 34% +15.7% 6648 ± 21% interrupts.CPU16.PMI:Performance_monitoring_interrupts 2685 ± 8% +24.8% 3352 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 175.80 ± 9% +61.9% 284.60 ± 16% interrupts.CPU16.TLB:TLB_shootdowns 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU17.171:PCI-MSI.31983623-edge.i40e-eth1-TxRx-6 0.00 -100.0% 0.00 interrupts.CPU17.182:PCI-MSI.31983634-edge.i40e-eth1-TxRx-17 0.20 ±200% -100.0% 0.00 interrupts.CPU17.219:PCI-MSI.31983671-edge.i40e-eth1-TxRx-54 0.00 -100.0% 0.00 interrupts.CPU17.303:PCI-MSI.77824-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU17.303:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU17.304:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU17.315:PCI-MSI.376832-edge.ahci[0000:00:17.0] 4279 ± 8% +22.6% 5246 ± 3% interrupts.CPU17.CAL:Function_call_interrupts 9.00 ± 12% +15.6% 10.40 ± 7% interrupts.CPU17.IWI:IRQ_work_interrupts 67966 +0.6% 68360 interrupts.CPU17.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU17.MCP:Machine_check_polls 6364 ± 22% -7.5% 5885 ± 28% interrupts.CPU17.NMI:Non-maskable_interrupts 6364 ± 22% -7.5% 5885 ± 28% interrupts.CPU17.PMI:Performance_monitoring_interrupts 2603 ± 12% +27.2% 3311 ± 9% interrupts.CPU17.RES:Rescheduling_interrupts 165.80 ± 27% +60.0% 265.20 ± 13% interrupts.CPU17.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU18.172:PCI-MSI.31983624-edge.i40e-eth1-TxRx-7 22.00 ±200% -95.5% 1.00 ±154% interrupts.CPU18.183:PCI-MSI.31983635-edge.i40e-eth1-TxRx-18 0.00 -100.0% 0.00 interrupts.CPU18.220:PCI-MSI.31983672-edge.i40e-eth1-TxRx-55 0.00 -100.0% 0.00 interrupts.CPU18.303:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU18.304:PCI-MSI.79872-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU18.305:PCI-MSI.376832-edge.ahci[0000:00:17.0] 0.00 -100.0% 0.00 interrupts.CPU18.3:IO-APIC.3-edge 4392 ± 5% +23.7% 5432 ± 5% interrupts.CPU18.CAL:Function_call_interrupts 8.80 ± 15% +22.7% 10.80 ± 10% interrupts.CPU18.IWI:IRQ_work_interrupts 68079 +0.3% 68301 interrupts.CPU18.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU18.MCP:Machine_check_polls 6404 ± 24% +16.0% 7428 ± 6% interrupts.CPU18.NMI:Non-maskable_interrupts 6404 ± 24% +16.0% 7428 ± 6% interrupts.CPU18.PMI:Performance_monitoring_interrupts 2523 ± 8% +32.9% 3353 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts 195.00 ± 17% +49.0% 290.60 ± 28% interrupts.CPU18.TLB:TLB_shootdowns 0.00 +2e+101% 0.20 ±200% interrupts.CPU19.173:PCI-MSI.31983625-edge.i40e-eth1-TxRx-8 1.40 ±200% -100.0% 0.00 interrupts.CPU19.184:PCI-MSI.31983636-edge.i40e-eth1-TxRx-19 0.00 -100.0% 0.00 interrupts.CPU19.221:PCI-MSI.31983673-edge.i40e-eth1-TxRx-56 0.00 -100.0% 0.00 interrupts.CPU19.315:PCI-MSI.376832-edge.ahci[0000:00:17.0] 4271 ± 7% +27.1% 5427 interrupts.CPU19.CAL:Function_call_interrupts 9.20 ± 15% +15.2% 10.60 ± 14% interrupts.CPU19.IWI:IRQ_work_interrupts 68063 +0.4% 68357 interrupts.CPU19.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU19.MCP:Machine_check_polls 6361 ± 23% -6.1% 5975 ± 32% interrupts.CPU19.NMI:Non-maskable_interrupts 6361 ± 23% -6.1% 5975 ± 32% interrupts.CPU19.PMI:Performance_monitoring_interrupts 2487 ± 10% +31.5% 3270 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 202.20 ± 15% +7.4% 217.20 ± 16% interrupts.CPU19.TLB:TLB_shootdowns 66.40 ±199% -100.0% 0.00 interrupts.CPU2.167:PCI-MSI.31983619-edge.i40e-eth1-TxRx-2 0.00 -100.0% 0.00 interrupts.CPU2.204:PCI-MSI.31983656-edge.i40e-eth1-TxRx-39 0.00 +2e+101% 0.20 ±200% interrupts.CPU2.252:PCI-MSI.31983704-edge.i40e-eth1-TxRx-87 0.00 -100.0% 0.00 interrupts.CPU2.297:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU2.298:PCI-MSI.69632-edge.ioat-msix 4611 ± 19% +20.0% 5534 ± 4% interrupts.CPU2.CAL:Function_call_interrupts 8.80 ± 15% +20.5% 10.60 ± 9% interrupts.CPU2.IWI:IRQ_work_interrupts 67699 +1.0% 68372 interrupts.CPU2.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU2.MCP:Machine_check_polls 6402 ± 24% +3.7% 6638 ± 21% interrupts.CPU2.NMI:Non-maskable_interrupts 6402 ± 24% +3.7% 6638 ± 21% interrupts.CPU2.PMI:Performance_monitoring_interrupts 2540 ± 10% +28.6% 3266 ± 11% interrupts.CPU2.RES:Rescheduling_interrupts 283.00 ± 65% -7.8% 260.80 ± 8% interrupts.CPU2.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU20.174:PCI-MSI.31983626-edge.i40e-eth1-TxRx-9 0.00 -100.0% 0.00 interrupts.CPU20.185:PCI-MSI.31983637-edge.i40e-eth1-TxRx-20 0.00 +2e+101% 0.20 ±200% interrupts.CPU20.222:PCI-MSI.31983674-edge.i40e-eth1-TxRx-57 0.00 -100.0% 0.00 interrupts.CPU20.315:PCI-MSI.376832-edge.ahci[0000:00:17.0] 4549 ± 4% +19.1% 5416 ± 4% interrupts.CPU20.CAL:Function_call_interrupts 9.40 ± 14% +8.5% 10.20 ± 7% interrupts.CPU20.IWI:IRQ_work_interrupts 68003 +0.5% 68344 interrupts.CPU20.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU20.MCP:Machine_check_polls 7065 ± 5% -27.9% 5097 ± 29% interrupts.CPU20.NMI:Non-maskable_interrupts 7065 ± 5% -27.9% 5097 ± 29% interrupts.CPU20.PMI:Performance_monitoring_interrupts 2660 ± 9% +23.8% 3294 ± 6% interrupts.CPU20.RES:Rescheduling_interrupts 192.80 ± 24% +38.3% 266.60 ± 25% interrupts.CPU20.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU21.175:PCI-MSI.31983627-edge.i40e-eth1-TxRx-10 3.20 ±200% -100.0% 0.00 interrupts.CPU21.186:PCI-MSI.31983638-edge.i40e-eth1-TxRx-21 0.00 +2e+101% 0.20 ±200% interrupts.CPU21.223:PCI-MSI.31983675-edge.i40e-eth1-TxRx-58 10.00 ±200% -100.0% 0.00 interrupts.CPU21.4:IO-APIC.4-edge.ttyS0 4468 ± 5% +20.1% 5366 interrupts.CPU21.CAL:Function_call_interrupts 9.00 ± 17% +17.8% 10.60 ± 9% interrupts.CPU21.IWI:IRQ_work_interrupts 67932 +0.5% 68293 interrupts.CPU21.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU21.MCP:Machine_check_polls 6356 ± 23% +5.0% 6673 ± 21% interrupts.CPU21.NMI:Non-maskable_interrupts 6356 ± 23% +5.0% 6673 ± 21% interrupts.CPU21.PMI:Performance_monitoring_interrupts 2592 ± 9% +29.5% 3357 ± 8% interrupts.CPU21.RES:Rescheduling_interrupts 179.20 ± 10% +43.8% 257.60 ± 20% interrupts.CPU21.TLB:TLB_shootdowns 0.20 ±200% +700.0% 1.60 ±200% interrupts.CPU22.187:PCI-MSI.31983639-edge.i40e-eth1-TxRx-22 0.00 -100.0% 0.00 interrupts.CPU22.224:PCI-MSI.31983676-edge.i40e-eth1-TxRx-59 0.00 -100.0% 0.00 interrupts.CPU22.3:IO-APIC.3-edge 0.00 +8.2e+102% 8.20 ±200% interrupts.CPU22.4:IO-APIC.4-edge.ttyS0 4439 ± 6% +17.0% 5194 ± 10% interrupts.CPU22.CAL:Function_call_interrupts 9.20 ± 15% +17.4% 10.80 ± 10% interrupts.CPU22.IWI:IRQ_work_interrupts 67943 +0.5% 68275 interrupts.CPU22.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU22.MCP:Machine_check_polls 7077 ± 5% +4.9% 7422 ± 6% interrupts.CPU22.NMI:Non-maskable_interrupts 7077 ± 5% +4.9% 7422 ± 6% interrupts.CPU22.PMI:Performance_monitoring_interrupts 2613 ± 14% +22.3% 3196 ± 15% interrupts.CPU22.RES:Rescheduling_interrupts 180.20 ± 33% +69.9% 306.20 ± 36% interrupts.CPU22.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU23.177:PCI-MSI.31983629-edge.i40e-eth1-TxRx-12 0.20 ±200% -100.0% 0.00 interrupts.CPU23.225:PCI-MSI.31983677-edge.i40e-eth1-TxRx-60 0.00 -100.0% 0.00 interrupts.CPU23.295:PCI-MSI.65536-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU23.3:IO-APIC.3-edge 4335 ± 3% +23.6% 5356 ± 3% interrupts.CPU23.CAL:Function_call_interrupts 8.80 ± 15% +22.7% 10.80 ± 15% interrupts.CPU23.IWI:IRQ_work_interrupts 67810 +0.5% 68164 interrupts.CPU23.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU23.MCP:Machine_check_polls 6409 ± 24% +5.6% 6770 ± 25% interrupts.CPU23.NMI:Non-maskable_interrupts 6409 ± 24% +5.6% 6770 ± 25% interrupts.CPU23.PMI:Performance_monitoring_interrupts 2607 ± 8% +25.9% 3282 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 193.80 ± 23% +46.2% 283.40 ± 21% interrupts.CPU23.TLB:TLB_shootdowns 0.40 ±200% +150.0% 1.00 ±200% interrupts.CPU24.189:PCI-MSI.31983641-edge.i40e-eth1-TxRx-24 4385 ± 3% +17.5% 5153 ± 4% interrupts.CPU24.CAL:Function_call_interrupts 9.80 ± 16% +6.1% 10.40 ± 24% interrupts.CPU24.IWI:IRQ_work_interrupts 68076 -0.2% 67931 interrupts.CPU24.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU24.MCP:Machine_check_polls 6095 ± 33% +7.6% 6560 ± 25% interrupts.CPU24.NMI:Non-maskable_interrupts 6095 ± 33% +7.6% 6560 ± 25% interrupts.CPU24.PMI:Performance_monitoring_interrupts 2631 ± 11% +11.2% 2926 ± 8% interrupts.CPU24.RES:Rescheduling_interrupts 1464 ±168% -81.0% 278.80 ± 17% interrupts.CPU24.TLB:TLB_shootdowns 4556 ± 6% +16.8% 5323 ± 8% interrupts.CPU25.CAL:Function_call_interrupts 10.00 ± 12% +0.0% 10.00 ± 12% interrupts.CPU25.IWI:IRQ_work_interrupts 68217 -0.2% 68098 interrupts.CPU25.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU25.MCP:Machine_check_polls 6751 ± 22% +7.2% 7238 ± 6% interrupts.CPU25.NMI:Non-maskable_interrupts 6751 ± 22% +7.2% 7238 ± 6% interrupts.CPU25.PMI:Performance_monitoring_interrupts 2639 ± 7% +15.8% 3057 ± 11% interrupts.CPU25.RES:Rescheduling_interrupts 211.60 ± 20% +27.6% 270.00 ± 21% interrupts.CPU25.TLB:TLB_shootdowns 1.80 ±173% -66.7% 0.60 ±200% interrupts.CPU26.191:PCI-MSI.31983643-edge.i40e-eth1-TxRx-26 0.00 -100.0% 0.00 interrupts.CPU26.305:PCI-MSI.67174400-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU26.306:PCI-MSI.67174400-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU26.307:PCI-MSI.67174400-edge.ioat-msix 4527 ± 2% +17.8% 5334 ± 8% interrupts.CPU26.CAL:Function_call_interrupts 10.00 ± 15% +0.0% 10.00 ± 12% interrupts.CPU26.IWI:IRQ_work_interrupts 67948 +0.2% 68083 interrupts.CPU26.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU26.MCP:Machine_check_polls 6870 ± 26% +5.6% 7257 ± 6% interrupts.CPU26.NMI:Non-maskable_interrupts 6870 ± 26% +5.6% 7257 ± 6% interrupts.CPU26.PMI:Performance_monitoring_interrupts 2706 ± 9% +11.0% 3003 ± 11% interrupts.CPU26.RES:Rescheduling_interrupts 196.20 ± 25% +28.2% 251.60 ± 15% interrupts.CPU26.TLB:TLB_shootdowns 0.60 ±200% +400.0% 3.00 ±168% interrupts.CPU27.192:PCI-MSI.31983644-edge.i40e-eth1-TxRx-27 0.00 -100.0% 0.00 interrupts.CPU27.307:PCI-MSI.67176448-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU27.308:PCI-MSI.67176448-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU27.309:PCI-MSI.67176448-edge.ioat-msix 4694 ± 3% +16.3% 5458 ± 8% interrupts.CPU27.CAL:Function_call_interrupts 10.00 ± 12% -2.0% 9.80 ± 14% interrupts.CPU27.IWI:IRQ_work_interrupts 68152 -0.1% 68096 interrupts.CPU27.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU27.MCP:Machine_check_polls 6747 ± 22% -3.2% 6531 ± 24% interrupts.CPU27.NMI:Non-maskable_interrupts 6747 ± 22% -3.2% 6531 ± 24% interrupts.CPU27.PMI:Performance_monitoring_interrupts 2753 ± 6% +11.5% 3069 ± 12% interrupts.CPU27.RES:Rescheduling_interrupts 173.20 ± 7% +43.1% 247.80 ± 18% interrupts.CPU27.TLB:TLB_shootdowns 66.60 ±200% -99.4% 0.40 ±200% interrupts.CPU28.193:PCI-MSI.31983645-edge.i40e-eth1-TxRx-28 0.00 -100.0% 0.00 interrupts.CPU28.308:PCI-MSI.67178496-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU28.309:PCI-MSI.67178496-edge.ioat-msix 4474 ± 4% +21.4% 5432 ± 7% interrupts.CPU28.CAL:Function_call_interrupts 10.40 ± 16% -3.8% 10.00 ± 12% interrupts.CPU28.IWI:IRQ_work_interrupts 68127 +0.0% 68157 interrupts.CPU28.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU28.MCP:Machine_check_polls 6756 ± 22% +6.6% 7201 ± 6% interrupts.CPU28.NMI:Non-maskable_interrupts 6756 ± 22% +6.6% 7201 ± 6% interrupts.CPU28.PMI:Performance_monitoring_interrupts 2529 ± 13% +20.2% 3041 ± 13% interrupts.CPU28.RES:Rescheduling_interrupts 207.00 ± 15% +40.8% 291.40 ± 25% interrupts.CPU28.TLB:TLB_shootdowns 1.00 ±154% -100.0% 0.00 interrupts.CPU29.194:PCI-MSI.31983646-edge.i40e-eth1-TxRx-29 0.00 -100.0% 0.00 interrupts.CPU29.309:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU29.310:PCI-MSI.67178496-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU29.310:PCI-MSI.67180544-edge.ioat-msix 4637 ± 6% +16.4% 5399 ± 6% interrupts.CPU29.CAL:Function_call_interrupts 10.20 ± 11% -3.9% 9.80 ± 11% interrupts.CPU29.IWI:IRQ_work_interrupts 67986 +0.2% 68127 interrupts.CPU29.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU29.MCP:Machine_check_polls 6744 ± 22% -4.2% 6463 ± 20% interrupts.CPU29.NMI:Non-maskable_interrupts 6744 ± 22% -4.2% 6463 ± 20% interrupts.CPU29.PMI:Performance_monitoring_interrupts 2800 ± 7% +6.5% 2982 ± 10% interrupts.CPU29.RES:Rescheduling_interrupts 190.40 ± 10% +62.5% 309.40 ± 28% interrupts.CPU29.TLB:TLB_shootdowns 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU3.205:PCI-MSI.31983657-edge.i40e-eth1-TxRx-40 0.00 -100.0% 0.00 interrupts.CPU3.253:PCI-MSI.31983705-edge.i40e-eth1-TxRx-88 0.00 -100.0% 0.00 interrupts.CPU3.297:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU3.4:IO-APIC.4-edge.ttyS0 4497 ± 3% +21.6% 5467 ± 2% interrupts.CPU3.CAL:Function_call_interrupts 9.00 ± 17% +15.6% 10.40 ± 14% interrupts.CPU3.IWI:IRQ_work_interrupts 68061 +0.2% 68210 interrupts.CPU3.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU3.MCP:Machine_check_polls 6391 ± 22% -6.3% 5991 ± 32% interrupts.CPU3.NMI:Non-maskable_interrupts 6391 ± 22% -6.3% 5991 ± 32% interrupts.CPU3.PMI:Performance_monitoring_interrupts 2645 ± 9% +28.3% 3394 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts 195.40 ± 13% +32.5% 259.00 ± 18% interrupts.CPU3.TLB:TLB_shootdowns 0.00 +8e+101% 0.80 ±122% interrupts.CPU30.195:PCI-MSI.31983647-edge.i40e-eth1-TxRx-30 0.00 -100.0% 0.00 interrupts.CPU30.309:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU30.310:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU30.310:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU30.311:PCI-MSI.67180544-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU30.311:PCI-MSI.67182592-edge.ioat-msix 4596 ± 3% +17.9% 5420 ± 7% interrupts.CPU30.CAL:Function_call_interrupts 10.20 ± 15% -3.9% 9.80 ± 11% interrupts.CPU30.IWI:IRQ_work_interrupts 68062 +0.1% 68103 interrupts.CPU30.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU30.MCP:Machine_check_polls 6875 ± 26% -6.2% 6449 ± 20% interrupts.CPU30.NMI:Non-maskable_interrupts 6875 ± 26% -6.2% 6449 ± 20% interrupts.CPU30.PMI:Performance_monitoring_interrupts 2697 ± 10% +9.8% 2960 ± 8% interrupts.CPU30.RES:Rescheduling_interrupts 189.60 ± 33% +36.2% 258.20 ± 31% interrupts.CPU30.TLB:TLB_shootdowns 0.00 +1.8e+102% 1.80 ±200% interrupts.CPU31.196:PCI-MSI.31983648-edge.i40e-eth1-TxRx-31 0.00 -100.0% 0.00 interrupts.CPU31.310:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU31.311:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU31.311:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU31.312:PCI-MSI.67182592-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU31.312:PCI-MSI.67184640-edge.ioat-msix 4625 ± 3% +14.1% 5276 ± 12% interrupts.CPU31.CAL:Function_call_interrupts 10.40 ± 11% -5.8% 9.80 ± 11% interrupts.CPU31.IWI:IRQ_work_interrupts 68020 +0.1% 68116 interrupts.CPU31.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU31.MCP:Machine_check_polls 7520 ± 6% -13.9% 6473 ± 20% interrupts.CPU31.NMI:Non-maskable_interrupts 7520 ± 6% -13.9% 6473 ± 20% interrupts.CPU31.PMI:Performance_monitoring_interrupts 2811 ± 9% +5.3% 2961 ± 11% interrupts.CPU31.RES:Rescheduling_interrupts 213.40 ± 5% +23.5% 263.60 ± 25% interrupts.CPU31.TLB:TLB_shootdowns 186.00 ±200% -99.0% 1.80 ±133% interrupts.CPU32.197:PCI-MSI.31983649-edge.i40e-eth1-TxRx-32 0.00 -100.0% 0.00 interrupts.CPU32.311:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU32.312:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU32.312:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU32.313:PCI-MSI.67184640-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU32.313:PCI-MSI.67186688-edge.ioat-msix 4719 ± 3% +13.4% 5354 ± 6% interrupts.CPU32.CAL:Function_call_interrupts 10.20 ± 11% -5.9% 9.60 ± 14% interrupts.CPU32.IWI:IRQ_work_interrupts 68127 +0.1% 68194 interrupts.CPU32.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU32.MCP:Machine_check_polls 7528 ± 6% -23.9% 5731 ± 29% interrupts.CPU32.NMI:Non-maskable_interrupts 7528 ± 6% -23.9% 5731 ± 29% interrupts.CPU32.PMI:Performance_monitoring_interrupts 2762 ± 9% +7.4% 2967 ± 8% interrupts.CPU32.RES:Rescheduling_interrupts 216.60 ± 9% +43.7% 311.20 ± 7% interrupts.CPU32.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU33.312:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU33.313:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU33.313:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU33.314:PCI-MSI.67186688-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU33.314:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU33.315:PCI-MSI.67188736-edge.ioat-msix 4721 ± 4% +12.8% 5326 ± 5% interrupts.CPU33.CAL:Function_call_interrupts 10.20 ± 11% -5.9% 9.60 ± 14% interrupts.CPU33.IWI:IRQ_work_interrupts 68065 +0.2% 68173 interrupts.CPU33.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU33.MCP:Machine_check_polls 7537 ± 6% -23.5% 5765 ± 31% interrupts.CPU33.NMI:Non-maskable_interrupts 7537 ± 6% -23.5% 5765 ± 31% interrupts.CPU33.PMI:Performance_monitoring_interrupts 2826 ± 8% +5.7% 2986 ± 9% interrupts.CPU33.RES:Rescheduling_interrupts 178.80 ± 16% +91.8% 343.00 ± 36% interrupts.CPU33.TLB:TLB_shootdowns 1.60 ±200% -100.0% 0.00 interrupts.CPU34.199:PCI-MSI.31983651-edge.i40e-eth1-TxRx-34 0.00 -100.0% 0.00 interrupts.CPU34.314:PCI-MSI.67188736-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU34.315:PCI-MSI.67188736-edge.ioat-msix 4617 ± 5% +17.1% 5407 ± 6% interrupts.CPU34.CAL:Function_call_interrupts 9.80 ± 16% -2.0% 9.60 ± 14% interrupts.CPU34.IWI:IRQ_work_interrupts 68127 +0.0% 68147 interrupts.CPU34.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU34.MCP:Machine_check_polls 6092 ± 33% -5.8% 5736 ± 29% interrupts.CPU34.NMI:Non-maskable_interrupts 6092 ± 33% -5.8% 5736 ± 29% interrupts.CPU34.PMI:Performance_monitoring_interrupts 2694 ± 8% +11.1% 2992 ± 7% interrupts.CPU34.RES:Rescheduling_interrupts 156.80 ± 18% +90.9% 299.40 ± 38% interrupts.CPU34.TLB:TLB_shootdowns 1.40 ±200% -100.0% 0.00 interrupts.CPU35.200:PCI-MSI.31983652-edge.i40e-eth1-TxRx-35 4526 ± 3% +15.2% 5216 ± 4% interrupts.CPU35.CAL:Function_call_interrupts 10.00 ± 15% -2.0% 9.80 ± 11% interrupts.CPU35.IWI:IRQ_work_interrupts 68149 -0.1% 68108 interrupts.CPU35.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU35.MCP:Machine_check_polls 6887 ± 25% -5.8% 6489 ± 20% interrupts.CPU35.NMI:Non-maskable_interrupts 6887 ± 25% -5.8% 6489 ± 20% interrupts.CPU35.PMI:Performance_monitoring_interrupts 2674 ± 8% +10.4% 2952 ± 8% interrupts.CPU35.RES:Rescheduling_interrupts 244.40 ± 18% +4.6% 255.60 ± 17% interrupts.CPU35.TLB:TLB_shootdowns 0.20 ±200% +7900.0% 16.00 ±165% interrupts.CPU36.201:PCI-MSI.31983653-edge.i40e-eth1-TxRx-36 4527 ± 3% +17.3% 5310 ± 5% interrupts.CPU36.CAL:Function_call_interrupts 10.20 ± 14% -3.9% 9.80 ± 11% interrupts.CPU36.IWI:IRQ_work_interrupts 68114 -0.2% 67970 interrupts.CPU36.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU36.MCP:Machine_check_polls 6745 ± 22% -4.0% 6474 ± 20% interrupts.CPU36.NMI:Non-maskable_interrupts 6745 ± 22% -4.0% 6474 ± 20% interrupts.CPU36.PMI:Performance_monitoring_interrupts 2669 ± 8% +12.4% 3001 ± 8% interrupts.CPU36.RES:Rescheduling_interrupts 245.60 ± 34% +18.3% 290.60 ± 16% interrupts.CPU36.TLB:TLB_shootdowns 0.00 +6e+101% 0.60 ±200% interrupts.CPU37.202:PCI-MSI.31983654-edge.i40e-eth1-TxRx-37 4581 ± 2% +18.4% 5422 ± 6% interrupts.CPU37.CAL:Function_call_interrupts 10.00 ± 12% -4.0% 9.60 ± 14% interrupts.CPU37.IWI:IRQ_work_interrupts 68119 +0.1% 68163 interrupts.CPU37.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU37.MCP:Machine_check_polls 6764 ± 22% -14.6% 5779 ± 31% interrupts.CPU37.NMI:Non-maskable_interrupts 6764 ± 22% -14.6% 5779 ± 31% interrupts.CPU37.PMI:Performance_monitoring_interrupts 2839 ± 9% +7.6% 3056 ± 11% interrupts.CPU37.RES:Rescheduling_interrupts 234.60 ± 25% +49.7% 351.20 ± 21% interrupts.CPU37.TLB:TLB_shootdowns 1.80 ±133% -88.9% 0.20 ±200% interrupts.CPU38.203:PCI-MSI.31983655-edge.i40e-eth1-TxRx-38 4629 ± 2% +14.7% 5307 ± 5% interrupts.CPU38.CAL:Function_call_interrupts 10.20 ± 11% -5.9% 9.60 ± 14% interrupts.CPU38.IWI:IRQ_work_interrupts 67973 +0.2% 68119 interrupts.CPU38.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU38.MCP:Machine_check_polls 7527 ± 6% -23.6% 5751 ± 29% interrupts.CPU38.NMI:Non-maskable_interrupts 7527 ± 6% -23.6% 5751 ± 29% interrupts.CPU38.PMI:Performance_monitoring_interrupts 2732 ± 9% +10.2% 3011 ± 9% interrupts.CPU38.RES:Rescheduling_interrupts 209.00 ± 27% +40.9% 294.40 ± 26% interrupts.CPU38.TLB:TLB_shootdowns 0.80 ±200% -100.0% 0.00 interrupts.CPU39.204:PCI-MSI.31983656-edge.i40e-eth1-TxRx-39 4564 ± 5% +17.5% 5365 ± 4% interrupts.CPU39.CAL:Function_call_interrupts 10.20 ± 11% -3.9% 9.80 ± 14% interrupts.CPU39.IWI:IRQ_work_interrupts 68132 -0.3% 67945 interrupts.CPU39.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU39.MCP:Machine_check_polls 7524 ± 6% -12.6% 6579 ± 24% interrupts.CPU39.NMI:Non-maskable_interrupts 7524 ± 6% -12.6% 6579 ± 24% interrupts.CPU39.PMI:Performance_monitoring_interrupts 2720 ± 6% +12.9% 3070 ± 9% interrupts.CPU39.RES:Rescheduling_interrupts 190.40 ± 11% +60.2% 305.00 ± 41% interrupts.CPU39.TLB:TLB_shootdowns 2.00 ±154% -20.0% 1.60 ±200% interrupts.CPU4.169:PCI-MSI.31983621-edge.i40e-eth1-TxRx-4 0.00 -100.0% 0.00 interrupts.CPU4.206:PCI-MSI.31983658-edge.i40e-eth1-TxRx-41 0.00 -100.0% 0.00 interrupts.CPU4.297:PCI-MSI.67584-edge.ioat-msix 4466 ± 3% +16.6% 5205 ± 8% interrupts.CPU4.CAL:Function_call_interrupts 9.20 ± 15% +17.4% 10.80 ± 10% interrupts.CPU4.IWI:IRQ_work_interrupts 68164 +0.1% 68233 interrupts.CPU4.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU4.MCP:Machine_check_polls 7068 ± 5% +5.1% 7431 ± 5% interrupts.CPU4.NMI:Non-maskable_interrupts 7068 ± 5% +5.1% 7431 ± 5% interrupts.CPU4.PMI:Performance_monitoring_interrupts 2686 ± 7% +17.2% 3149 ± 15% interrupts.CPU4.RES:Rescheduling_interrupts 194.20 ± 14% +39.5% 271.00 ± 21% interrupts.CPU4.TLB:TLB_shootdowns 3.20 ±120% -37.5% 2.00 ±137% interrupts.CPU40.205:PCI-MSI.31983657-edge.i40e-eth1-TxRx-40 4490 ± 3% +14.5% 5142 ± 6% interrupts.CPU40.CAL:Function_call_interrupts 10.20 ± 11% +0.0% 10.20 ± 15% interrupts.CPU40.IWI:IRQ_work_interrupts 68115 +0.1% 68217 interrupts.CPU40.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU40.MCP:Machine_check_polls 7535 ± 6% -13.4% 6523 ± 24% interrupts.CPU40.NMI:Non-maskable_interrupts 7535 ± 6% -13.4% 6523 ± 24% interrupts.CPU40.PMI:Performance_monitoring_interrupts 2665 ± 7% +10.9% 2955 ± 10% interrupts.CPU40.RES:Rescheduling_interrupts 195.00 ± 28% +42.7% 278.20 ± 18% interrupts.CPU40.TLB:TLB_shootdowns 0.00 +2.2e+103% 21.80 ±195% interrupts.CPU41.206:PCI-MSI.31983658-edge.i40e-eth1-TxRx-41 4561 ± 4% +17.5% 5359 ± 7% interrupts.CPU41.CAL:Function_call_interrupts 10.00 ± 12% -2.0% 9.80 ± 14% interrupts.CPU41.IWI:IRQ_work_interrupts 68117 +0.0% 68129 interrupts.CPU41.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU41.MCP:Machine_check_polls 6745 ± 22% -2.9% 6547 ± 25% interrupts.CPU41.NMI:Non-maskable_interrupts 6745 ± 22% -2.9% 6547 ± 25% interrupts.CPU41.PMI:Performance_monitoring_interrupts 2733 ± 8% +10.9% 3031 ± 10% interrupts.CPU41.RES:Rescheduling_interrupts 192.80 ± 24% +69.6% 327.00 ± 20% interrupts.CPU41.TLB:TLB_shootdowns 4437 ± 3% +18.5% 5260 ± 6% interrupts.CPU42.CAL:Function_call_interrupts 10.20 ± 11% -2.0% 10.00 ± 12% interrupts.CPU42.IWI:IRQ_work_interrupts 68130 +0.0% 68163 interrupts.CPU42.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU42.MCP:Machine_check_polls 6749 ± 22% +6.9% 7218 ± 6% interrupts.CPU42.NMI:Non-maskable_interrupts 6749 ± 22% +6.9% 7218 ± 6% interrupts.CPU42.PMI:Performance_monitoring_interrupts 2727 ± 8% +9.0% 2974 ± 8% interrupts.CPU42.RES:Rescheduling_interrupts 171.80 ± 12% +72.6% 296.60 ± 25% interrupts.CPU42.TLB:TLB_shootdowns 0.00 +2.4e+103% 23.60 ±191% interrupts.CPU43.208:PCI-MSI.31983660-edge.i40e-eth1-TxRx-43 4609 ± 3% +14.5% 5277 ± 7% interrupts.CPU43.CAL:Function_call_interrupts 10.00 ± 10% +0.0% 10.00 ± 12% interrupts.CPU43.IWI:IRQ_work_interrupts 68131 -0.0% 68122 interrupts.CPU43.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU43.MCP:Machine_check_polls 6752 ± 22% +7.1% 7229 ± 6% interrupts.CPU43.NMI:Non-maskable_interrupts 6752 ± 22% +7.1% 7229 ± 6% interrupts.CPU43.PMI:Performance_monitoring_interrupts 2782 ± 7% +5.6% 2939 ± 13% interrupts.CPU43.RES:Rescheduling_interrupts 183.20 ± 16% +41.8% 259.80 ± 6% interrupts.CPU43.TLB:TLB_shootdowns 0.00 +4e+101% 0.40 ±200% interrupts.CPU44.209:PCI-MSI.31983661-edge.i40e-eth1-TxRx-44 4671 ± 2% +10.9% 5181 ± 8% interrupts.CPU44.CAL:Function_call_interrupts 10.40 ± 11% -3.8% 10.00 ± 12% interrupts.CPU44.IWI:IRQ_work_interrupts 68163 -0.1% 68126 interrupts.CPU44.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU44.MCP:Machine_check_polls 7518 ± 6% -3.6% 7244 ± 5% interrupts.CPU44.NMI:Non-maskable_interrupts 7518 ± 6% -3.6% 7244 ± 5% interrupts.CPU44.PMI:Performance_monitoring_interrupts 2769 ± 9% +6.1% 2939 ± 14% interrupts.CPU44.RES:Rescheduling_interrupts 187.40 ± 17% +64.8% 308.80 ± 20% interrupts.CPU44.TLB:TLB_shootdowns 4610 ± 4% +14.8% 5290 ± 3% interrupts.CPU45.CAL:Function_call_interrupts 10.20 ± 11% -2.0% 10.00 ± 12% interrupts.CPU45.IWI:IRQ_work_interrupts 68195 -0.2% 68068 interrupts.CPU45.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU45.MCP:Machine_check_polls 6742 ± 22% -4.0% 6475 ± 20% interrupts.CPU45.NMI:Non-maskable_interrupts 6742 ± 22% -4.0% 6475 ± 20% interrupts.CPU45.PMI:Performance_monitoring_interrupts 2785 ± 9% +7.9% 3004 ± 11% interrupts.CPU45.RES:Rescheduling_interrupts 198.60 ± 37% +38.7% 275.40 ± 33% interrupts.CPU45.TLB:TLB_shootdowns 0.00 +1.8e+102% 1.80 ±200% interrupts.CPU46.211:PCI-MSI.31983663-edge.i40e-eth1-TxRx-46 4664 ± 2% +15.7% 5397 ± 6% interrupts.CPU46.CAL:Function_call_interrupts 10.00 ± 10% +0.0% 10.00 ± 16% interrupts.CPU46.IWI:IRQ_work_interrupts 68175 -0.1% 68134 interrupts.CPU46.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU46.MCP:Machine_check_polls 6748 ± 22% -2.5% 6580 ± 24% interrupts.CPU46.NMI:Non-maskable_interrupts 6748 ± 22% -2.5% 6580 ± 24% interrupts.CPU46.PMI:Performance_monitoring_interrupts 2822 ± 8% +8.1% 3052 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts 218.20 ± 24% +26.0% 275.00 ± 28% interrupts.CPU46.TLB:TLB_shootdowns 4606 ± 5% +15.0% 5299 ± 2% interrupts.CPU47.CAL:Function_call_interrupts 10.00 ± 10% +2.0% 10.20 ± 14% interrupts.CPU47.IWI:IRQ_work_interrupts 68142 -0.1% 68060 interrupts.CPU47.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU47.MCP:Machine_check_polls 6761 ± 22% +7.2% 7246 ± 6% interrupts.CPU47.NMI:Non-maskable_interrupts 6761 ± 22% +7.2% 7246 ± 6% interrupts.CPU47.PMI:Performance_monitoring_interrupts 2882 ± 6% +7.6% 3100 ± 10% interrupts.CPU47.RES:Rescheduling_interrupts 193.60 ± 33% +23.2% 238.60 ± 25% interrupts.CPU47.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU48.178:PCI-MSI.31983630-edge.i40e-eth1-TxRx-13 0.00 +2e+101% 0.20 ±200% interrupts.CPU48.226:PCI-MSI.31983678-edge.i40e-eth1-TxRx-61 0.00 -100.0% 0.00 interrupts.CPU48.295:PCI-MSI.65536-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU48.3:IO-APIC.3-edge 0.00 -100.0% 0.00 interrupts.CPU48.4:IO-APIC.4-edge.ttyS0 4403 ± 3% +17.2% 5160 ± 4% interrupts.CPU48.CAL:Function_call_interrupts 9.00 ± 12% +20.0% 10.80 ± 10% interrupts.CPU48.IWI:IRQ_work_interrupts 68276 +0.1% 68362 interrupts.CPU48.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU48.MCP:Machine_check_polls 7061 ± 6% +5.2% 7428 ± 5% interrupts.CPU48.NMI:Non-maskable_interrupts 7061 ± 6% +5.2% 7428 ± 5% interrupts.CPU48.PMI:Performance_monitoring_interrupts 2538 ± 8% +25.5% 3185 ± 8% interrupts.CPU48.RES:Rescheduling_interrupts 165.20 ± 21% +54.6% 255.40 ± 16% interrupts.CPU48.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU49.179:PCI-MSI.31983631-edge.i40e-eth1-TxRx-14 0.40 ±200% +100.0% 0.80 ±200% interrupts.CPU49.214:PCI-MSI.31983666-edge.i40e-eth1-TxRx-49 0.00 -100.0% 0.00 interrupts.CPU49.227:PCI-MSI.31983679-edge.i40e-eth1-TxRx-62 4384 ± 4% +21.0% 5303 ± 4% interrupts.CPU49.CAL:Function_call_interrupts 8.40 ± 9% +26.2% 10.60 ± 14% interrupts.CPU49.IWI:IRQ_work_interrupts 68199 +0.2% 68340 interrupts.CPU49.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU49.MCP:Machine_check_polls 4842 ± 30% +39.6% 6761 ± 25% interrupts.CPU49.NMI:Non-maskable_interrupts 4842 ± 30% +39.6% 6761 ± 25% interrupts.CPU49.PMI:Performance_monitoring_interrupts 2632 ± 7% +27.0% 3341 ± 7% interrupts.CPU49.RES:Rescheduling_interrupts 164.80 ± 25% +76.0% 290.00 ± 27% interrupts.CPU49.TLB:TLB_shootdowns 0.00 +4e+101% 0.40 ±200% interrupts.CPU5.170:PCI-MSI.31983622-edge.i40e-eth1-TxRx-5 0.00 -100.0% 0.00 interrupts.CPU5.255:PCI-MSI.31983707-edge.i40e-eth1-TxRx-90 4531 ± 4% +18.0% 5345 ± 4% interrupts.CPU5.CAL:Function_call_interrupts 9.60 ± 23% +10.4% 10.60 ± 11% interrupts.CPU5.IWI:IRQ_work_interrupts 68080 +0.3% 68315 interrupts.CPU5.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU5.MCP:Machine_check_polls 7090 ± 5% -6.1% 6659 ± 21% interrupts.CPU5.NMI:Non-maskable_interrupts 7090 ± 5% -6.1% 6659 ± 21% interrupts.CPU5.PMI:Performance_monitoring_interrupts 2619 ± 9% +26.2% 3306 ± 12% interrupts.CPU5.RES:Rescheduling_interrupts 271.20 ± 77% +4.4% 283.20 ± 11% interrupts.CPU5.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU50.180:PCI-MSI.31983632-edge.i40e-eth1-TxRx-15 0.00 +2.4e+102% 2.40 ±161% interrupts.CPU50.215:PCI-MSI.31983667-edge.i40e-eth1-TxRx-50 0.00 -100.0% 0.00 interrupts.CPU50.228:PCI-MSI.31983680-edge.i40e-eth1-TxRx-63 4285 ± 3% +22.4% 5243 ± 2% interrupts.CPU50.CAL:Function_call_interrupts 8.40 ± 14% +28.6% 10.80 ± 10% interrupts.CPU50.IWI:IRQ_work_interrupts 68057 +0.5% 68369 interrupts.CPU50.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU50.MCP:Machine_check_polls 4957 ± 35% +49.5% 7410 ± 6% interrupts.CPU50.NMI:Non-maskable_interrupts 4957 ± 35% +49.5% 7410 ± 6% interrupts.CPU50.PMI:Performance_monitoring_interrupts 2599 ± 9% +25.2% 3253 ± 6% interrupts.CPU50.RES:Rescheduling_interrupts 174.20 ± 20% +51.7% 264.20 ± 20% interrupts.CPU50.TLB:TLB_shootdowns 0.00 +2e+101% 0.20 ±200% interrupts.CPU51.181:PCI-MSI.31983633-edge.i40e-eth1-TxRx-16 0.80 ±200% -100.0% 0.00 interrupts.CPU51.216:PCI-MSI.31983668-edge.i40e-eth1-TxRx-51 0.00 -100.0% 0.00 interrupts.CPU51.229:PCI-MSI.31983681-edge.i40e-eth1-TxRx-64 0.00 -100.0% 0.00 interrupts.CPU51.297:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU51.3:IO-APIC.3-edge 4450 ± 4% +19.5% 5318 ± 2% interrupts.CPU51.CAL:Function_call_interrupts 8.80 ± 8% +22.7% 10.80 ± 10% interrupts.CPU51.IWI:IRQ_work_interrupts 68090 +0.6% 68505 interrupts.CPU51.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU51.MCP:Machine_check_polls 6309 ± 19% +17.7% 7429 ± 6% interrupts.CPU51.NMI:Non-maskable_interrupts 6309 ± 19% +17.7% 7429 ± 6% interrupts.CPU51.PMI:Performance_monitoring_interrupts 2667 ± 11% +22.9% 3279 ± 9% interrupts.CPU51.RES:Rescheduling_interrupts 170.80 ± 22% +70.3% 290.80 ± 15% interrupts.CPU51.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU52.182:PCI-MSI.31983634-edge.i40e-eth1-TxRx-17 1.80 ±200% -100.0% 0.00 interrupts.CPU52.217:PCI-MSI.31983669-edge.i40e-eth1-TxRx-52 0.20 ±200% -100.0% 0.00 interrupts.CPU52.230:PCI-MSI.31983682-edge.i40e-eth1-TxRx-65 0.00 -100.0% 0.00 interrupts.CPU52.3:IO-APIC.3-edge 0.00 +1e+103% 10.40 ±200% interrupts.CPU52.4:IO-APIC.4-edge.ttyS0 4436 ± 3% +21.3% 5380 ± 2% interrupts.CPU52.CAL:Function_call_interrupts 8.60 ± 9% +23.3% 10.60 ± 11% interrupts.CPU52.IWI:IRQ_work_interrupts 68090 +0.3% 68294 interrupts.CPU52.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU52.MCP:Machine_check_polls 5594 ± 28% +19.0% 6657 ± 21% interrupts.CPU52.NMI:Non-maskable_interrupts 5594 ± 28% +19.0% 6657 ± 21% interrupts.CPU52.PMI:Performance_monitoring_interrupts 2661 ± 5% +25.4% 3337 ± 8% interrupts.CPU52.RES:Rescheduling_interrupts 188.00 ± 20% +32.8% 249.60 ± 27% interrupts.CPU52.TLB:TLB_shootdowns 0.00 +4e+101% 0.40 ±122% interrupts.CPU53.183:PCI-MSI.31983635-edge.i40e-eth1-TxRx-18 0.00 +1.6e+103% 15.60 ±199% interrupts.CPU53.218:PCI-MSI.31983670-edge.i40e-eth1-TxRx-53 0.00 -100.0% 0.00 interrupts.CPU53.231:PCI-MSI.31983683-edge.i40e-eth1-TxRx-66 0.00 -100.0% 0.00 interrupts.CPU53.297:PCI-MSI.67584-edge.ioat-msix 4439 ± 5% +19.0% 5282 ± 4% interrupts.CPU53.CAL:Function_call_interrupts 9.40 ± 19% +12.8% 10.60 ± 11% interrupts.CPU53.IWI:IRQ_work_interrupts 67978 +0.5% 68339 interrupts.CPU53.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU53.MCP:Machine_check_polls 7087 ± 5% -6.2% 6649 ± 21% interrupts.CPU53.NMI:Non-maskable_interrupts 7087 ± 5% -6.2% 6649 ± 21% interrupts.CPU53.PMI:Performance_monitoring_interrupts 2612 ± 8% +24.4% 3251 ± 8% interrupts.CPU53.RES:Rescheduling_interrupts 173.60 ± 19% +54.1% 267.60 ± 8% interrupts.CPU53.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU54.184:PCI-MSI.31983636-edge.i40e-eth1-TxRx-19 50.60 ±192% -100.0% 0.00 interrupts.CPU54.219:PCI-MSI.31983671-edge.i40e-eth1-TxRx-54 0.00 +2e+101% 0.20 ±200% interrupts.CPU54.232:PCI-MSI.31983684-edge.i40e-eth1-TxRx-67 0.00 -100.0% 0.00 interrupts.CPU54.3:IO-APIC.3-edge 4287 ± 3% +23.3% 5284 ± 7% interrupts.CPU54.CAL:Function_call_interrupts 8.80 ± 15% +22.7% 10.80 ± 10% interrupts.CPU54.IWI:IRQ_work_interrupts 68008 +0.5% 68357 interrupts.CPU54.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU54.MCP:Machine_check_polls 5630 ± 29% +31.9% 7429 ± 5% interrupts.CPU54.NMI:Non-maskable_interrupts 5630 ± 29% +31.9% 7429 ± 5% interrupts.CPU54.PMI:Performance_monitoring_interrupts 2608 ± 9% +28.3% 3346 ± 12% interrupts.CPU54.RES:Rescheduling_interrupts 187.80 ± 22% +22.5% 230.00 ± 17% interrupts.CPU54.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU55.185:PCI-MSI.31983637-edge.i40e-eth1-TxRx-20 1.80 ±200% +66.7% 3.00 ±200% interrupts.CPU55.220:PCI-MSI.31983672-edge.i40e-eth1-TxRx-55 0.20 ±200% -100.0% 0.00 interrupts.CPU55.233:PCI-MSI.31983685-edge.i40e-eth1-TxRx-68 0.00 +9.6e+102% 9.60 ±200% interrupts.CPU55.4:IO-APIC.4-edge.ttyS0 4210 ± 4% +26.9% 5342 ± 5% interrupts.CPU55.CAL:Function_call_interrupts 9.00 ± 18% +22.2% 11.00 ± 9% interrupts.CPU55.IWI:IRQ_work_interrupts 68038 +0.5% 68408 interrupts.CPU55.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU55.MCP:Machine_check_polls 6409 ± 24% +4.0% 6663 ± 21% interrupts.CPU55.NMI:Non-maskable_interrupts 6409 ± 24% +4.0% 6663 ± 21% interrupts.CPU55.PMI:Performance_monitoring_interrupts 2582 ± 6% +29.0% 3332 ± 4% interrupts.CPU55.RES:Rescheduling_interrupts 181.00 ± 19% +60.0% 289.60 ± 24% interrupts.CPU55.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU56.186:PCI-MSI.31983638-edge.i40e-eth1-TxRx-21 1.80 ±123% -22.2% 1.40 ±200% interrupts.CPU56.221:PCI-MSI.31983673-edge.i40e-eth1-TxRx-56 0.20 ±200% -100.0% 0.00 interrupts.CPU56.234:PCI-MSI.31983686-edge.i40e-eth1-TxRx-69 4165 ± 12% +29.8% 5407 ± 3% interrupts.CPU56.CAL:Function_call_interrupts 8.80 ± 18% +20.5% 10.60 ± 11% interrupts.CPU56.IWI:IRQ_work_interrupts 68059 +0.4% 68358 interrupts.CPU56.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU56.MCP:Machine_check_polls 4963 ± 37% +34.1% 6654 ± 21% interrupts.CPU56.NMI:Non-maskable_interrupts 4963 ± 37% +34.1% 6654 ± 21% interrupts.CPU56.PMI:Performance_monitoring_interrupts 2465 ± 14% +33.5% 3291 ± 6% interrupts.CPU56.RES:Rescheduling_interrupts 172.60 ± 28% +70.7% 294.60 ± 24% interrupts.CPU56.TLB:TLB_shootdowns 0.40 ±122% -50.0% 0.20 ±200% interrupts.CPU57.187:PCI-MSI.31983639-edge.i40e-eth1-TxRx-22 0.40 ±200% +46350.0% 185.80 ±200% interrupts.CPU57.222:PCI-MSI.31983674-edge.i40e-eth1-TxRx-57 4201 ± 7% +27.3% 5348 ± 2% interrupts.CPU57.CAL:Function_call_interrupts 9.00 ± 12% +17.8% 10.60 ± 11% interrupts.CPU57.IWI:IRQ_work_interrupts 68050 +0.5% 68384 interrupts.CPU57.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU57.MCP:Machine_check_polls 6387 ± 23% +4.1% 6651 ± 21% interrupts.CPU57.NMI:Non-maskable_interrupts 6387 ± 23% +4.1% 6651 ± 21% interrupts.CPU57.PMI:Performance_monitoring_interrupts 2431 ± 12% +38.1% 3357 ± 5% interrupts.CPU57.RES:Rescheduling_interrupts 188.60 ± 20% +43.6% 270.80 ± 26% interrupts.CPU57.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU58.188:PCI-MSI.31983640-edge.i40e-eth1-TxRx-23 0.00 +2.4e+102% 2.40 ±133% interrupts.CPU58.223:PCI-MSI.31983675-edge.i40e-eth1-TxRx-58 0.00 -100.0% 0.00 interrupts.CPU58.236:PCI-MSI.31983688-edge.i40e-eth1-TxRx-71 4337 ± 5% +19.4% 5179 ± 3% interrupts.CPU58.CAL:Function_call_interrupts 8.80 ± 15% +20.5% 10.60 ± 12% interrupts.CPU58.IWI:IRQ_work_interrupts 68072 +0.4% 68340 interrupts.CPU58.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU58.MCP:Machine_check_polls 6420 ± 23% +4.4% 6700 ± 23% interrupts.CPU58.NMI:Non-maskable_interrupts 6420 ± 23% +4.4% 6700 ± 23% interrupts.CPU58.PMI:Performance_monitoring_interrupts 2626 ± 9% +24.2% 3262 ± 8% interrupts.CPU58.RES:Rescheduling_interrupts 192.20 ± 23% +45.2% 279.00 ± 25% interrupts.CPU58.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU59.189:PCI-MSI.31983641-edge.i40e-eth1-TxRx-24 0.80 ±200% +975.0% 8.60 ±200% interrupts.CPU59.224:PCI-MSI.31983676-edge.i40e-eth1-TxRx-59 0.00 -100.0% 0.00 interrupts.CPU59.237:PCI-MSI.31983689-edge.i40e-eth1-TxRx-72 4369 ± 5% +21.3% 5302 ± 3% interrupts.CPU59.CAL:Function_call_interrupts 8.60 ± 15% +23.3% 10.60 ± 12% interrupts.CPU59.IWI:IRQ_work_interrupts 68058 +0.5% 68397 interrupts.CPU59.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU59.MCP:Machine_check_polls 5705 ± 33% +17.5% 6706 ± 24% interrupts.CPU59.NMI:Non-maskable_interrupts 5705 ± 33% +17.5% 6706 ± 24% interrupts.CPU59.PMI:Performance_monitoring_interrupts 2637 ± 11% +26.3% 3332 ± 7% interrupts.CPU59.RES:Rescheduling_interrupts 176.80 ± 24% +40.8% 249.00 ± 13% interrupts.CPU59.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU6.171:PCI-MSI.31983623-edge.i40e-eth1-TxRx-6 0.00 +4e+101% 0.40 ±122% interrupts.CPU6.208:PCI-MSI.31983660-edge.i40e-eth1-TxRx-43 0.00 -100.0% 0.00 interrupts.CPU6.256:PCI-MSI.31983708-edge.i40e-eth1-TxRx-91 4276 ± 2% +24.5% 5324 ± 2% interrupts.CPU6.CAL:Function_call_interrupts 9.20 ± 15% +15.2% 10.60 ± 11% interrupts.CPU6.IWI:IRQ_work_interrupts 68076 +0.4% 68333 interrupts.CPU6.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU6.MCP:Machine_check_polls 7075 ± 5% -6.0% 6647 ± 21% interrupts.CPU6.NMI:Non-maskable_interrupts 7075 ± 5% -6.0% 6647 ± 21% interrupts.CPU6.PMI:Performance_monitoring_interrupts 2566 ± 8% +28.2% 3290 ± 6% interrupts.CPU6.RES:Rescheduling_interrupts 357.40 ± 95% -28.0% 257.20 ± 15% interrupts.CPU6.TLB:TLB_shootdowns 5.00 ±126% -80.0% 1.00 ±200% interrupts.CPU60.225:PCI-MSI.31983677-edge.i40e-eth1-TxRx-60 0.20 ±200% -100.0% 0.00 interrupts.CPU60.238:PCI-MSI.31983690-edge.i40e-eth1-TxRx-73 0.00 -100.0% 0.00 interrupts.CPU60.295:PCI-MSI.65536-edge.ioat-msix 4285 ± 4% +19.3% 5111 ± 8% interrupts.CPU60.CAL:Function_call_interrupts 9.00 ± 12% +15.6% 10.40 ± 14% interrupts.CPU60.IWI:IRQ_work_interrupts 68109 +0.4% 68368 interrupts.CPU60.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU60.MCP:Machine_check_polls 7062 ± 5% -15.2% 5992 ± 32% interrupts.CPU60.NMI:Non-maskable_interrupts 7062 ± 5% -15.2% 5992 ± 32% interrupts.CPU60.PMI:Performance_monitoring_interrupts 2561 ± 7% +22.9% 3147 ± 11% interrupts.CPU60.RES:Rescheduling_interrupts 169.20 ± 19% +65.7% 280.40 ± 30% interrupts.CPU60.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU61.191:PCI-MSI.31983643-edge.i40e-eth1-TxRx-26 10.80 ±200% -38.9% 6.60 ±192% interrupts.CPU61.226:PCI-MSI.31983678-edge.i40e-eth1-TxRx-61 0.20 ±200% -100.0% 0.00 interrupts.CPU61.239:PCI-MSI.31983691-edge.i40e-eth1-TxRx-74 4313 ± 3% +22.3% 5275 ± 4% interrupts.CPU61.CAL:Function_call_interrupts 9.20 ± 15% +17.4% 10.80 ± 10% interrupts.CPU61.IWI:IRQ_work_interrupts 67764 +0.9% 68407 interrupts.CPU61.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU61.MCP:Machine_check_polls 6363 ± 23% +16.6% 7419 ± 5% interrupts.CPU61.NMI:Non-maskable_interrupts 6363 ± 23% +16.6% 7419 ± 5% interrupts.CPU61.PMI:Performance_monitoring_interrupts 2532 ± 7% +32.3% 3349 ± 8% interrupts.CPU61.RES:Rescheduling_interrupts 190.40 ± 12% +28.4% 244.40 ± 17% interrupts.CPU61.TLB:TLB_shootdowns 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU62.192:PCI-MSI.31983644-edge.i40e-eth1-TxRx-27 9.60 ±200% -85.4% 1.40 ±200% interrupts.CPU62.227:PCI-MSI.31983679-edge.i40e-eth1-TxRx-62 0.00 -100.0% 0.00 interrupts.CPU62.240:PCI-MSI.31983692-edge.i40e-eth1-TxRx-75 0.00 -100.0% 0.00 interrupts.CPU62.314:PCI-MSI.376832-edge.ahci[0000:00:17.0] 4263 ± 5% +22.1% 5207 ± 5% interrupts.CPU62.CAL:Function_call_interrupts 9.00 ± 18% +17.8% 10.60 ± 9% interrupts.CPU62.IWI:IRQ_work_interrupts 68058 +0.5% 68381 interrupts.CPU62.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU62.MCP:Machine_check_polls 6407 ± 24% +3.6% 6640 ± 21% interrupts.CPU62.NMI:Non-maskable_interrupts 6407 ± 24% +3.6% 6640 ± 21% interrupts.CPU62.PMI:Performance_monitoring_interrupts 2534 ± 12% +30.3% 3301 ± 7% interrupts.CPU62.RES:Rescheduling_interrupts 180.60 ± 22% +61.9% 292.40 ± 11% interrupts.CPU62.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU63.193:PCI-MSI.31983645-edge.i40e-eth1-TxRx-28 1.40 ±200% -100.0% 0.00 interrupts.CPU63.228:PCI-MSI.31983680-edge.i40e-eth1-TxRx-63 0.00 -100.0% 0.00 interrupts.CPU63.295:PCI-MSI.65536-edge.ioat-msix 4450 ± 2% +21.3% 5399 ± 4% interrupts.CPU63.CAL:Function_call_interrupts 9.20 ± 15% +15.2% 10.60 ± 11% interrupts.CPU63.IWI:IRQ_work_interrupts 68116 +0.4% 68420 interrupts.CPU63.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU63.MCP:Machine_check_polls 7076 ± 5% -6.1% 6645 ± 21% interrupts.CPU63.NMI:Non-maskable_interrupts 7076 ± 5% -6.1% 6645 ± 21% interrupts.CPU63.PMI:Performance_monitoring_interrupts 2612 ± 6% +28.2% 3349 ± 6% interrupts.CPU63.RES:Rescheduling_interrupts 161.00 ± 21% +49.8% 241.20 ± 22% interrupts.CPU63.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU64.194:PCI-MSI.31983646-edge.i40e-eth1-TxRx-29 4.40 ±123% -100.0% 0.00 interrupts.CPU64.229:PCI-MSI.31983681-edge.i40e-eth1-TxRx-64 0.00 -100.0% 0.00 interrupts.CPU64.242:PCI-MSI.31983694-edge.i40e-eth1-TxRx-77 0.00 -100.0% 0.00 interrupts.CPU64.295:PCI-MSI.65536-edge.ioat-msix 8.60 ±200% -100.0% 0.00 interrupts.CPU64.4:IO-APIC.4-edge.ttyS0 4490 ± 4% +18.0% 5300 ± 5% interrupts.CPU64.CAL:Function_call_interrupts 8.60 ± 15% +23.3% 10.60 ± 11% interrupts.CPU64.IWI:IRQ_work_interrupts 68107 +0.4% 68402 interrupts.CPU64.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU64.MCP:Machine_check_polls 5705 ± 33% +16.5% 6648 ± 21% interrupts.CPU64.NMI:Non-maskable_interrupts 5705 ± 33% +16.5% 6648 ± 21% interrupts.CPU64.PMI:Performance_monitoring_interrupts 2639 ± 8% +23.8% 3266 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts 166.80 ± 18% +69.7% 283.00 ± 7% interrupts.CPU64.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU65.195:PCI-MSI.31983647-edge.i40e-eth1-TxRx-30 39.80 ±200% -100.0% 0.00 interrupts.CPU65.230:PCI-MSI.31983682-edge.i40e-eth1-TxRx-65 0.00 +4e+101% 0.40 ±122% interrupts.CPU65.243:PCI-MSI.31983695-edge.i40e-eth1-TxRx-78 0.00 -100.0% 0.00 interrupts.CPU65.295:PCI-MSI.65536-edge.ioat-msix 4227 ± 4% +27.1% 5373 ± 5% interrupts.CPU65.CAL:Function_call_interrupts 9.00 ± 12% +17.8% 10.60 ± 11% interrupts.CPU65.IWI:IRQ_work_interrupts 68040 +0.5% 68375 interrupts.CPU65.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU65.MCP:Machine_check_polls 7082 ± 5% -6.0% 6659 ± 21% interrupts.CPU65.NMI:Non-maskable_interrupts 7082 ± 5% -6.0% 6659 ± 21% interrupts.CPU65.PMI:Performance_monitoring_interrupts 2568 ± 9% +33.1% 3418 ± 9% interrupts.CPU65.RES:Rescheduling_interrupts 145.00 ± 6% +75.9% 255.00 ± 7% interrupts.CPU65.TLB:TLB_shootdowns 0.00 +2e+101% 0.20 ±200% interrupts.CPU66.196:PCI-MSI.31983648-edge.i40e-eth1-TxRx-31 0.00 -100.0% 0.00 interrupts.CPU66.231:PCI-MSI.31983683-edge.i40e-eth1-TxRx-66 4304 ± 5% +24.8% 5372 ± 5% interrupts.CPU66.CAL:Function_call_interrupts 8.60 ± 15% +25.6% 10.80 ± 10% interrupts.CPU66.IWI:IRQ_work_interrupts 67976 +0.4% 68238 interrupts.CPU66.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU66.MCP:Machine_check_polls 5688 ± 32% +30.6% 7426 ± 6% interrupts.CPU66.NMI:Non-maskable_interrupts 5688 ± 32% +30.6% 7426 ± 6% interrupts.CPU66.PMI:Performance_monitoring_interrupts 2571 ± 8% +25.8% 3235 ± 5% interrupts.CPU66.RES:Rescheduling_interrupts 164.60 ± 17% +99.1% 327.80 ± 24% interrupts.CPU66.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU67.197:PCI-MSI.31983649-edge.i40e-eth1-TxRx-32 0.00 +1.6e+102% 1.60 ±200% interrupts.CPU67.232:PCI-MSI.31983684-edge.i40e-eth1-TxRx-67 0.00 +2e+101% 0.20 ±200% interrupts.CPU67.245:PCI-MSI.31983697-edge.i40e-eth1-TxRx-80 4308 ± 3% +21.2% 5220 ± 4% interrupts.CPU67.CAL:Function_call_interrupts 9.20 ± 15% +17.4% 10.80 ± 10% interrupts.CPU67.IWI:IRQ_work_interrupts 68038 +0.5% 68384 interrupts.CPU67.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU67.MCP:Machine_check_polls 7055 ± 5% +5.3% 7427 ± 5% interrupts.CPU67.NMI:Non-maskable_interrupts 7055 ± 5% +5.3% 7427 ± 5% interrupts.CPU67.PMI:Performance_monitoring_interrupts 2557 ± 9% +28.5% 3286 ± 9% interrupts.CPU67.RES:Rescheduling_interrupts 1288 ±172% -80.8% 247.60 ± 19% interrupts.CPU67.TLB:TLB_shootdowns 9.60 ±170% -100.0% 0.00 interrupts.CPU68.233:PCI-MSI.31983685-edge.i40e-eth1-TxRx-68 0.00 -100.0% 0.00 interrupts.CPU68.246:PCI-MSI.31983698-edge.i40e-eth1-TxRx-81 4356 ± 3% +21.8% 5306 ± 3% interrupts.CPU68.CAL:Function_call_interrupts 9.40 ± 14% +10.6% 10.40 ± 9% interrupts.CPU68.IWI:IRQ_work_interrupts 68128 +0.4% 68372 interrupts.CPU68.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU68.MCP:Machine_check_polls 7052 ± 5% -16.8% 5869 ± 28% interrupts.CPU68.NMI:Non-maskable_interrupts 7052 ± 5% -16.8% 5869 ± 28% interrupts.CPU68.PMI:Performance_monitoring_interrupts 2652 ± 6% +24.4% 3298 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 195.60 ± 19% +56.2% 305.60 ± 10% interrupts.CPU68.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU69.199:PCI-MSI.31983651-edge.i40e-eth1-TxRx-34 1.00 ±154% -100.0% 0.00 interrupts.CPU69.234:PCI-MSI.31983686-edge.i40e-eth1-TxRx-69 0.20 ±200% -100.0% 0.00 interrupts.CPU69.247:PCI-MSI.31983699-edge.i40e-eth1-TxRx-82 4451 ± 5% +20.6% 5368 ± 4% interrupts.CPU69.CAL:Function_call_interrupts 9.20 ± 17% +15.2% 10.60 ± 11% interrupts.CPU69.IWI:IRQ_work_interrupts 68104 +0.4% 68343 interrupts.CPU69.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU69.MCP:Machine_check_polls 6411 ± 24% +4.1% 6674 ± 21% interrupts.CPU69.NMI:Non-maskable_interrupts 6411 ± 24% +4.1% 6674 ± 21% interrupts.CPU69.PMI:Performance_monitoring_interrupts 2662 ± 10% +26.8% 3376 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts 192.40 ± 11% +39.7% 268.80 ± 10% interrupts.CPU69.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU7.172:PCI-MSI.31983624-edge.i40e-eth1-TxRx-7 0.00 -100.0% 0.00 interrupts.CPU7.209:PCI-MSI.31983661-edge.i40e-eth1-TxRx-44 0.20 ±200% -100.0% 0.00 interrupts.CPU7.257:PCI-MSI.31983709-edge.i40e-eth1-TxRx-92 0.00 -100.0% 0.00 interrupts.CPU7.295:PCI-MSI.65536-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU7.299:PCI-MSI.71680-edge.ioat-msix 4414 ± 3% +22.7% 5415 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 9.20 ± 15% +15.2% 10.60 ± 11% interrupts.CPU7.IWI:IRQ_work_interrupts 68114 +0.3% 68313 interrupts.CPU7.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU7.MCP:Machine_check_polls 5660 ± 31% +17.6% 6656 ± 21% interrupts.CPU7.NMI:Non-maskable_interrupts 5660 ± 31% +17.6% 6656 ± 21% interrupts.CPU7.PMI:Performance_monitoring_interrupts 2637 ± 4% +25.7% 3315 ± 9% interrupts.CPU7.RES:Rescheduling_interrupts 172.60 ± 20% +57.1% 271.20 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU70.200:PCI-MSI.31983652-edge.i40e-eth1-TxRx-35 0.00 -100.0% 0.00 interrupts.CPU70.248:PCI-MSI.31983700-edge.i40e-eth1-TxRx-83 4454 ± 5% +21.8% 5424 ± 4% interrupts.CPU70.CAL:Function_call_interrupts 9.00 ± 18% +20.0% 10.80 ± 10% interrupts.CPU70.IWI:IRQ_work_interrupts 67957 +0.5% 68307 interrupts.CPU70.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU70.MCP:Machine_check_polls 6395 ± 24% +16.1% 7425 ± 5% interrupts.CPU70.NMI:Non-maskable_interrupts 6395 ± 24% +16.1% 7425 ± 5% interrupts.CPU70.PMI:Performance_monitoring_interrupts 2655 ± 11% +26.4% 3355 ± 7% interrupts.CPU70.RES:Rescheduling_interrupts 216.40 ± 27% +115.5% 466.40 ± 79% interrupts.CPU70.TLB:TLB_shootdowns 0.20 ±200% -100.0% 0.00 interrupts.CPU71.201:PCI-MSI.31983653-edge.i40e-eth1-TxRx-36 0.00 +1.4e+102% 1.40 ±200% interrupts.CPU71.236:PCI-MSI.31983688-edge.i40e-eth1-TxRx-71 4295 ± 4% +22.0% 5239 ± 3% interrupts.CPU71.CAL:Function_call_interrupts 8.60 ± 17% +23.3% 10.60 ± 12% interrupts.CPU71.IWI:IRQ_work_interrupts 68007 +0.5% 68342 interrupts.CPU71.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU71.MCP:Machine_check_polls 5726 ± 34% +17.2% 6712 ± 23% interrupts.CPU71.NMI:Non-maskable_interrupts 5726 ± 34% +17.2% 6712 ± 23% interrupts.CPU71.PMI:Performance_monitoring_interrupts 2631 ± 9% +24.3% 3271 ± 7% interrupts.CPU71.RES:Rescheduling_interrupts 163.60 ± 12% +83.9% 300.80 ± 23% interrupts.CPU71.TLB:TLB_shootdowns 0.00 +1.2e+102% 1.20 ±200% interrupts.CPU72.237:PCI-MSI.31983689-edge.i40e-eth1-TxRx-72 4509 ± 4% +14.0% 5140 ± 6% interrupts.CPU72.CAL:Function_call_interrupts 9.60 ± 14% +2.1% 9.80 ± 14% interrupts.CPU72.IWI:IRQ_work_interrupts 68129 +0.1% 68170 interrupts.CPU72.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU72.MCP:Machine_check_polls 5312 ± 37% +23.0% 6532 ± 24% interrupts.CPU72.NMI:Non-maskable_interrupts 5312 ± 37% +23.0% 6532 ± 24% interrupts.CPU72.PMI:Performance_monitoring_interrupts 2678 ± 7% +9.4% 2930 ± 9% interrupts.CPU72.RES:Rescheduling_interrupts 192.00 ± 32% +61.8% 310.60 ± 23% interrupts.CPU72.TLB:TLB_shootdowns 19.40 ±159% -100.0% 0.00 interrupts.CPU73.238:PCI-MSI.31983690-edge.i40e-eth1-TxRx-73 4655 ± 3% +18.5% 5518 ± 7% interrupts.CPU73.CAL:Function_call_interrupts 10.00 ± 10% -2.0% 9.80 ± 14% interrupts.CPU73.IWI:IRQ_work_interrupts 68102 +0.1% 68149 interrupts.CPU73.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU73.MCP:Machine_check_polls 6758 ± 22% -3.0% 6558 ± 24% interrupts.CPU73.NMI:Non-maskable_interrupts 6758 ± 22% -3.0% 6558 ± 24% interrupts.CPU73.PMI:Performance_monitoring_interrupts 2728 ± 7% +12.7% 3076 ± 12% interrupts.CPU73.RES:Rescheduling_interrupts 170.20 ± 19% +39.7% 237.80 ± 21% interrupts.CPU73.TLB:TLB_shootdowns 188.20 ±198% -100.0% 0.00 interrupts.CPU74.239:PCI-MSI.31983691-edge.i40e-eth1-TxRx-74 4630 ± 2% +11.3% 5154 ± 4% interrupts.CPU74.CAL:Function_call_interrupts 10.00 ± 10% -4.0% 9.60 ± 16% interrupts.CPU74.IWI:IRQ_work_interrupts 68137 +0.1% 68183 interrupts.CPU74.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU74.MCP:Machine_check_polls 6751 ± 22% -12.9% 5882 ± 33% interrupts.CPU74.NMI:Non-maskable_interrupts 6751 ± 22% -12.9% 5882 ± 33% interrupts.CPU74.PMI:Performance_monitoring_interrupts 2776 ± 10% +8.6% 3015 ± 11% interrupts.CPU74.RES:Rescheduling_interrupts 161.20 ± 13% +62.5% 262.00 ± 23% interrupts.CPU74.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU75.240:PCI-MSI.31983692-edge.i40e-eth1-TxRx-75 4580 ± 3% +15.3% 5281 ± 5% interrupts.CPU75.CAL:Function_call_interrupts 10.00 ± 10% -2.0% 9.80 ± 14% interrupts.CPU75.IWI:IRQ_work_interrupts 68156 +0.2% 68287 interrupts.CPU75.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU75.MCP:Machine_check_polls 6740 ± 22% -3.3% 6516 ± 23% interrupts.CPU75.NMI:Non-maskable_interrupts 6740 ± 22% -3.3% 6516 ± 23% interrupts.CPU75.PMI:Performance_monitoring_interrupts 2672 ± 5% +13.5% 3031 ± 8% interrupts.CPU75.RES:Rescheduling_interrupts 202.40 ± 31% +44.2% 291.80 ± 9% interrupts.CPU75.TLB:TLB_shootdowns 4740 ± 2% +13.0% 5358 ± 5% interrupts.CPU76.CAL:Function_call_interrupts 10.20 ± 11% -5.9% 9.60 ± 16% interrupts.CPU76.IWI:IRQ_work_interrupts 68132 +0.1% 68175 interrupts.CPU76.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU76.MCP:Machine_check_polls 6752 ± 22% -13.4% 5846 ± 34% interrupts.CPU76.NMI:Non-maskable_interrupts 6752 ± 22% -13.4% 5846 ± 34% interrupts.CPU76.PMI:Performance_monitoring_interrupts 2804 ± 7% +6.1% 2974 ± 11% interrupts.CPU76.RES:Rescheduling_interrupts 194.40 ± 18% +55.0% 301.40 ± 11% interrupts.CPU76.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU77.242:PCI-MSI.31983694-edge.i40e-eth1-TxRx-77 4600 ± 3% +12.7% 5185 ± 8% interrupts.CPU77.CAL:Function_call_interrupts 10.20 ± 11% -2.0% 10.00 ± 12% interrupts.CPU77.IWI:IRQ_work_interrupts 68140 +0.0% 68170 interrupts.CPU77.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU77.MCP:Machine_check_polls 7509 ± 6% -3.4% 7255 ± 6% interrupts.CPU77.NMI:Non-maskable_interrupts 7509 ± 6% -3.4% 7255 ± 6% interrupts.CPU77.PMI:Performance_monitoring_interrupts 2760 ± 8% +5.1% 2902 ± 12% interrupts.CPU77.RES:Rescheduling_interrupts 177.60 ± 22% +62.8% 289.20 ± 27% interrupts.CPU77.TLB:TLB_shootdowns 0.00 +3.2e+102% 3.20 ±145% interrupts.CPU78.243:PCI-MSI.31983695-edge.i40e-eth1-TxRx-78 4557 ± 4% +16.4% 5306 ± 5% interrupts.CPU78.CAL:Function_call_interrupts 10.40 ± 11% -7.7% 9.60 ± 16% interrupts.CPU78.IWI:IRQ_work_interrupts 68153 -0.0% 68147 interrupts.CPU78.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU78.MCP:Machine_check_polls 7528 ± 6% -22.2% 5860 ± 34% interrupts.CPU78.NMI:Non-maskable_interrupts 7528 ± 6% -22.2% 5860 ± 34% interrupts.CPU78.PMI:Performance_monitoring_interrupts 2772 ± 10% +7.9% 2991 ± 7% interrupts.CPU78.RES:Rescheduling_interrupts 223.80 ± 24% +17.2% 262.20 ± 22% interrupts.CPU78.TLB:TLB_shootdowns 4575 ± 4% +21.1% 5540 ± 6% interrupts.CPU79.CAL:Function_call_interrupts 10.60 ± 12% -5.7% 10.00 ± 12% interrupts.CPU79.IWI:IRQ_work_interrupts 68158 -0.0% 68135 interrupts.CPU79.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU79.MCP:Machine_check_polls 7540 ± 6% -3.9% 7245 ± 6% interrupts.CPU79.NMI:Non-maskable_interrupts 7540 ± 6% -3.9% 7245 ± 6% interrupts.CPU79.PMI:Performance_monitoring_interrupts 2746 ± 6% +11.3% 3057 ± 7% interrupts.CPU79.RES:Rescheduling_interrupts 232.60 ± 24% +22.2% 284.20 ± 15% interrupts.CPU79.TLB:TLB_shootdowns 0.00 +2e+101% 0.20 ±200% interrupts.CPU8.173:PCI-MSI.31983625-edge.i40e-eth1-TxRx-8 0.20 ±200% -100.0% 0.00 interrupts.CPU8.258:PCI-MSI.31983710-edge.i40e-eth1-TxRx-93 4541 ± 5% +19.3% 5419 ± 3% interrupts.CPU8.CAL:Function_call_interrupts 9.20 ± 17% +19.6% 11.00 ± 11% interrupts.CPU8.IWI:IRQ_work_interrupts 67961 +0.7% 68416 interrupts.CPU8.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU8.MCP:Machine_check_polls 6402 ± 24% +16.1% 7430 ± 6% interrupts.CPU8.NMI:Non-maskable_interrupts 6402 ± 24% +16.1% 7430 ± 6% interrupts.CPU8.PMI:Performance_monitoring_interrupts 2671 ± 11% +22.8% 3280 ± 7% interrupts.CPU8.RES:Rescheduling_interrupts 190.40 ± 15% +69.4% 322.60 ± 9% interrupts.CPU8.TLB:TLB_shootdowns 0.00 +2e+101% 0.20 ±200% interrupts.CPU80.245:PCI-MSI.31983697-edge.i40e-eth1-TxRx-80 4542 ± 2% +13.1% 5137 ± 4% interrupts.CPU80.CAL:Function_call_interrupts 10.20 ± 11% -3.9% 9.80 ± 14% interrupts.CPU80.IWI:IRQ_work_interrupts 68123 +0.1% 68171 interrupts.CPU80.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU80.MCP:Machine_check_polls 6745 ± 22% -2.8% 6555 ± 24% interrupts.CPU80.NMI:Non-maskable_interrupts 6745 ± 22% -2.8% 6555 ± 24% interrupts.CPU80.PMI:Performance_monitoring_interrupts 2774 ± 7% +8.4% 3007 ± 12% interrupts.CPU80.RES:Rescheduling_interrupts 223.40 ± 14% +21.2% 270.80 ± 24% interrupts.CPU80.TLB:TLB_shootdowns 0.00 +6e+101% 0.60 ±200% interrupts.CPU81.246:PCI-MSI.31983698-edge.i40e-eth1-TxRx-81 4622 ± 3% +14.9% 5312 ± 4% interrupts.CPU81.CAL:Function_call_interrupts 10.20 ± 11% -3.9% 9.80 ± 14% interrupts.CPU81.IWI:IRQ_work_interrupts 68141 +0.1% 68208 interrupts.CPU81.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU81.MCP:Machine_check_polls 7540 ± 6% -12.6% 6592 ± 24% interrupts.CPU81.NMI:Non-maskable_interrupts 7540 ± 6% -12.6% 6592 ± 24% interrupts.CPU81.PMI:Performance_monitoring_interrupts 2813 ± 9% +8.6% 3056 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts 197.80 ± 19% +47.3% 291.40 ± 28% interrupts.CPU81.TLB:TLB_shootdowns 26.60 ±196% -100.0% 0.00 interrupts.CPU82.247:PCI-MSI.31983699-edge.i40e-eth1-TxRx-82 4500 ± 2% +12.8% 5074 ± 9% interrupts.CPU82.CAL:Function_call_interrupts 10.20 ± 11% -2.0% 10.00 ± 12% interrupts.CPU82.IWI:IRQ_work_interrupts 68047 -0.0% 68038 interrupts.CPU82.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU82.MCP:Machine_check_polls 7528 ± 6% -3.8% 7246 ± 6% interrupts.CPU82.NMI:Non-maskable_interrupts 7528 ± 6% -3.8% 7246 ± 6% interrupts.CPU82.PMI:Performance_monitoring_interrupts 2743 ± 8% +6.3% 2917 ± 15% interrupts.CPU82.RES:Rescheduling_interrupts 190.60 ± 18% +30.4% 248.60 ± 36% interrupts.CPU82.TLB:TLB_shootdowns 0.00 +6.6e+103% 66.00 ±194% interrupts.CPU83.248:PCI-MSI.31983700-edge.i40e-eth1-TxRx-83 4440 ± 2% +13.3% 5029 ± 9% interrupts.CPU83.CAL:Function_call_interrupts 10.00 ± 10% -2.0% 9.80 ± 14% interrupts.CPU83.IWI:IRQ_work_interrupts 68052 +0.1% 68153 interrupts.CPU83.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU83.MCP:Machine_check_polls 6756 ± 22% -3.2% 6540 ± 24% interrupts.CPU83.NMI:Non-maskable_interrupts 6756 ± 22% -3.2% 6540 ± 24% interrupts.CPU83.PMI:Performance_monitoring_interrupts 2696 ± 9% +5.8% 2854 ± 16% interrupts.CPU83.RES:Rescheduling_interrupts 203.40 ± 10% +58.9% 323.20 ± 21% interrupts.CPU83.TLB:TLB_shootdowns 4564 ± 5% +16.0% 5294 ± 7% interrupts.CPU84.CAL:Function_call_interrupts 10.80 ± 17% -7.4% 10.00 ± 12% interrupts.CPU84.IWI:IRQ_work_interrupts 68114 +0.0% 68142 interrupts.CPU84.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU84.MCP:Machine_check_polls 6787 ± 22% +6.9% 7255 ± 5% interrupts.CPU84.NMI:Non-maskable_interrupts 6787 ± 22% +6.9% 7255 ± 5% interrupts.CPU84.PMI:Performance_monitoring_interrupts 2777 ± 6% +10.7% 3075 ± 10% interrupts.CPU84.RES:Rescheduling_interrupts 195.40 ± 16% +53.3% 299.60 ± 13% interrupts.CPU84.TLB:TLB_shootdowns 4637 ± 2% +13.3% 5255 ± 6% interrupts.CPU85.CAL:Function_call_interrupts 10.20 ± 11% -2.0% 10.00 ± 12% interrupts.CPU85.IWI:IRQ_work_interrupts 68134 +0.1% 68220 interrupts.CPU85.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU85.MCP:Machine_check_polls 6760 ± 22% +7.1% 7241 ± 6% interrupts.CPU85.NMI:Non-maskable_interrupts 6760 ± 22% +7.1% 7241 ± 6% interrupts.CPU85.PMI:Performance_monitoring_interrupts 2791 ± 7% +11.5% 3112 ± 12% interrupts.CPU85.RES:Rescheduling_interrupts 195.60 ± 21% +48.6% 290.60 ± 33% interrupts.CPU85.TLB:TLB_shootdowns 0.00 +1e+103% 10.40 ±200% interrupts.CPU86.251:PCI-MSI.31983703-edge.i40e-eth1-TxRx-86 4524 ± 5% +15.0% 5202 ± 5% interrupts.CPU86.CAL:Function_call_interrupts 9.80 ± 9% +0.0% 9.80 ± 14% interrupts.CPU86.IWI:IRQ_work_interrupts 68124 +0.1% 68162 interrupts.CPU86.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU86.MCP:Machine_check_polls 6004 ± 29% +9.2% 6556 ± 24% interrupts.CPU86.NMI:Non-maskable_interrupts 6004 ± 29% +9.2% 6556 ± 24% interrupts.CPU86.PMI:Performance_monitoring_interrupts 2778 ± 7% +7.6% 2989 ± 9% interrupts.CPU86.RES:Rescheduling_interrupts 220.20 ± 17% +33.1% 293.00 ± 34% interrupts.CPU86.TLB:TLB_shootdowns 0.40 ±200% +15450.0% 62.20 ±190% interrupts.CPU87.252:PCI-MSI.31983704-edge.i40e-eth1-TxRx-87 4528 ± 3% +16.2% 5262 ± 6% interrupts.CPU87.CAL:Function_call_interrupts 9.80 ± 9% +0.0% 9.80 ± 14% interrupts.CPU87.IWI:IRQ_work_interrupts 68126 +0.1% 68193 interrupts.CPU87.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU87.MCP:Machine_check_polls 5971 ± 29% +9.9% 6564 ± 25% interrupts.CPU87.NMI:Non-maskable_interrupts 5971 ± 29% +9.9% 6564 ± 25% interrupts.CPU87.PMI:Performance_monitoring_interrupts 2768 ± 9% +7.9% 2987 ± 9% interrupts.CPU87.RES:Rescheduling_interrupts 218.60 ± 19% +30.5% 285.20 ± 16% interrupts.CPU87.TLB:TLB_shootdowns 22.00 ±200% -100.0% 0.00 interrupts.CPU88.253:PCI-MSI.31983705-edge.i40e-eth1-TxRx-88 4602 ± 3% +16.8% 5375 ± 5% interrupts.CPU88.CAL:Function_call_interrupts 9.80 ± 14% +0.0% 9.80 ± 9% interrupts.CPU88.IWI:IRQ_work_interrupts 68146 +0.0% 68158 interrupts.CPU88.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU88.MCP:Machine_check_polls 6088 ± 34% +19.0% 7246 ± 6% interrupts.CPU88.NMI:Non-maskable_interrupts 6088 ± 34% +19.0% 7246 ± 6% interrupts.CPU88.PMI:Performance_monitoring_interrupts 2766 ± 6% +10.0% 3043 ± 10% interrupts.CPU88.RES:Rescheduling_interrupts 203.00 ± 20% +38.8% 281.80 ± 26% interrupts.CPU88.TLB:TLB_shootdowns 4295 ± 3% +21.6% 5223 ± 5% interrupts.CPU89.CAL:Function_call_interrupts 10.00 ± 10% +0.0% 10.00 ± 12% interrupts.CPU89.IWI:IRQ_work_interrupts 68108 -0.1% 68039 interrupts.CPU89.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU89.MCP:Machine_check_polls 6758 ± 22% +6.9% 7227 ± 6% interrupts.CPU89.NMI:Non-maskable_interrupts 6758 ± 22% +6.9% 7227 ± 6% interrupts.CPU89.PMI:Performance_monitoring_interrupts 2651 ± 14% +10.4% 2925 ± 10% interrupts.CPU89.RES:Rescheduling_interrupts 187.80 ± 17% +28.3% 241.00 ± 40% interrupts.CPU89.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU9.174:PCI-MSI.31983626-edge.i40e-eth1-TxRx-9 0.20 ±200% +0.0% 0.20 ±200% interrupts.CPU9.211:PCI-MSI.31983663-edge.i40e-eth1-TxRx-46 0.00 -100.0% 0.00 interrupts.CPU9.295:PCI-MSI.65536-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU9.297:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU9.298:PCI-MSI.67584-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU9.298:PCI-MSI.69632-edge.ioat-msix 0.00 -100.0% 0.00 interrupts.CPU9.300:PCI-MSI.73728-edge.ioat-msix 4420 +22.2% 5401 ± 3% interrupts.CPU9.CAL:Function_call_interrupts 9.20 ± 15% +15.2% 10.60 ± 14% interrupts.CPU9.IWI:IRQ_work_interrupts 68099 +0.4% 68385 interrupts.CPU9.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU9.MCP:Machine_check_polls 7068 ± 5% -4.2% 6770 ± 25% interrupts.CPU9.NMI:Non-maskable_interrupts 7068 ± 5% -4.2% 6770 ± 25% interrupts.CPU9.PMI:Performance_monitoring_interrupts 2621 ± 7% +26.9% 3326 ± 8% interrupts.CPU9.RES:Rescheduling_interrupts 169.00 ± 14% +61.7% 273.20 ± 14% interrupts.CPU9.TLB:TLB_shootdowns 0.00 -100.0% 0.00 interrupts.CPU90.255:PCI-MSI.31983707-edge.i40e-eth1-TxRx-90 4647 +11.7% 5193 ± 8% interrupts.CPU90.CAL:Function_call_interrupts 10.00 ± 10% -2.0% 9.80 ± 14% interrupts.CPU90.IWI:IRQ_work_interrupts 68152 +0.0% 68162 interrupts.CPU90.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU90.MCP:Machine_check_polls 5983 ± 29% +9.2% 6534 ± 23% interrupts.CPU90.NMI:Non-maskable_interrupts 5983 ± 29% +9.2% 6534 ± 23% interrupts.CPU90.PMI:Performance_monitoring_interrupts 2774 ± 7% +10.1% 3054 ± 10% interrupts.CPU90.RES:Rescheduling_interrupts 165.60 ± 16% +87.9% 311.20 ± 27% interrupts.CPU90.TLB:TLB_shootdowns 2.80 ±200% -42.9% 1.60 ±200% interrupts.CPU91.256:PCI-MSI.31983708-edge.i40e-eth1-TxRx-91 4682 ± 5% +12.7% 5277 ± 7% interrupts.CPU91.CAL:Function_call_interrupts 10.00 ± 10% +0.0% 10.00 ± 12% interrupts.CPU91.IWI:IRQ_work_interrupts 68119 -0.0% 68113 interrupts.CPU91.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU91.MCP:Machine_check_polls 6758 ± 22% +6.9% 7226 ± 6% interrupts.CPU91.NMI:Non-maskable_interrupts 6758 ± 22% +6.9% 7226 ± 6% interrupts.CPU91.PMI:Performance_monitoring_interrupts 2781 ± 6% +8.0% 3004 ± 7% interrupts.CPU91.RES:Rescheduling_interrupts 188.40 ± 15% +49.5% 281.60 ± 22% interrupts.CPU91.TLB:TLB_shootdowns 16.00 ±196% -100.0% 0.00 interrupts.CPU92.257:PCI-MSI.31983709-edge.i40e-eth1-TxRx-92 4580 ± 4% +16.3% 5328 ± 6% interrupts.CPU92.CAL:Function_call_interrupts 9.60 ± 14% +0.0% 9.60 ± 16% interrupts.CPU92.IWI:IRQ_work_interrupts 68155 +0.0% 68164 interrupts.CPU92.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU92.MCP:Machine_check_polls 5319 ± 37% +9.9% 5844 ± 33% interrupts.CPU92.NMI:Non-maskable_interrupts 5319 ± 37% +9.9% 5844 ± 33% interrupts.CPU92.PMI:Performance_monitoring_interrupts 2739 ± 4% +11.3% 3047 ± 9% interrupts.CPU92.RES:Rescheduling_interrupts 231.60 ± 8% +17.3% 271.60 ± 21% interrupts.CPU92.TLB:TLB_shootdowns 1.40 ±139% -100.0% 0.00 interrupts.CPU93.258:PCI-MSI.31983710-edge.i40e-eth1-TxRx-93 4652 ± 4% +15.4% 5366 ± 3% interrupts.CPU93.CAL:Function_call_interrupts 10.00 ± 10% +0.0% 10.00 ± 17% interrupts.CPU93.IWI:IRQ_work_interrupts 68149 -0.1% 68087 interrupts.CPU93.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU93.MCP:Machine_check_polls 5975 ± 29% +10.0% 6571 ± 24% interrupts.CPU93.NMI:Non-maskable_interrupts 5975 ± 29% +10.0% 6571 ± 24% interrupts.CPU93.PMI:Performance_monitoring_interrupts 2729 ± 7% +13.2% 3091 ± 7% interrupts.CPU93.RES:Rescheduling_interrupts 217.60 ± 26% +25.2% 272.40 ± 25% interrupts.CPU93.TLB:TLB_shootdowns 4487 ± 2% +18.0% 5293 ± 5% interrupts.CPU94.CAL:Function_call_interrupts 9.80 ± 9% +0.0% 9.80 ± 14% interrupts.CPU94.IWI:IRQ_work_interrupts 68128 -0.0% 68097 interrupts.CPU94.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU94.MCP:Machine_check_polls 5977 ± 29% +9.5% 6544 ± 24% interrupts.CPU94.NMI:Non-maskable_interrupts 5977 ± 29% +9.5% 6544 ± 24% interrupts.CPU94.PMI:Performance_monitoring_interrupts 2745 ± 8% +10.2% 3024 ± 10% interrupts.CPU94.RES:Rescheduling_interrupts 249.60 ± 28% +26.4% 315.60 ± 24% interrupts.CPU94.TLB:TLB_shootdowns 4019 +13.8% 4575 ± 5% interrupts.CPU95.CAL:Function_call_interrupts 9.40 ± 8% -4.3% 9.00 ± 15% interrupts.CPU95.IWI:IRQ_work_interrupts 68113 +0.1% 68189 interrupts.CPU95.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.CPU95.MCP:Machine_check_polls 5217 ± 32% +26.3% 6590 ± 24% interrupts.CPU95.NMI:Non-maskable_interrupts 5217 ± 32% +26.3% 6590 ± 24% interrupts.CPU95.PMI:Performance_monitoring_interrupts 2259 ± 11% +6.7% 2411 ± 8% interrupts.CPU95.RES:Rescheduling_interrupts 168.00 ± 12% -6.0% 158.00 ± 15% interrupts.CPU95.TLB:TLB_shootdowns 915.80 +7.6% 985.00 ± 2% interrupts.IWI:IRQ_work_interrupts 6535684 +0.2% 6550271 interrupts.LOC:Local_timer_interrupts 0.00 -100.0% 0.00 interrupts.MCP:Machine_check_polls 633150 ± 3% +1.3% 641334 ± 3% interrupts.NMI:Non-maskable_interrupts 633150 ± 3% +1.3% 641334 ± 3% interrupts.PMI:Performance_monitoring_interrupts 255665 +18.0% 301580 interrupts.RES:Rescheduling_interrupts 0.00 -100.0% 0.00 interrupts.RTR:APIC_ICR_read_retries 21070 ± 12% +27.4% 26852 ± 14% interrupts.TLB:TLB_shootdowns stress-ng.spawn.ops_per_sec 18500 +-------------------------------------------------------------------+ | O O O O O O O O O O | 18000 |-+ O O O O O O O O O O O O | 17500 |-+ O O O O O O | | O | 17000 |-+ | | | 16500 |-+ .+.. +| | + + | 16000 |-+ : | 15500 |.+.. .+. .+. .+ : | | +.+ +..+.+.+.+. +. .+. + .+.. .+ : | 15000 |-+ + + + + .+. .+.: | | +.+..+.+ +. + | 14500 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

[xfs] 237d7887ae: xfstests.xfs.513.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 237d7887ae723af7d978e8b9a385fdff416f357b ("xfs: show the proper user quota options") https://git.kernel.org/cgit/fs/xfs/xfs-linux.git xfs-5.11-merge in testcase: xfstests version: xfstests-x86_64-d41dcbd-1_20201215 with following parameters: disk: 4HDD fs: xfs test: xfs-group-51 ucode: 0x21 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2020-12-18 01:24:14 export TEST_DIR=/fs/sdb1 2020-12-18 01:24:14 export TEST_DEV=/dev/sdb1 2020-12-18 01:24:14 export FSTYP=xfs 2020-12-18 01:24:14 export SCRATCH_MNT=/fs/scratch 2020-12-18 01:24:14 mkdir /fs/scratch -p 2020-12-18 01:24:14 export SCRATCH_DEV=/dev/sdb4 2020-12-18 01:24:14 export SCRATCH_LOGDEV=/dev/sdb2 2020-12-18 01:24:14 export SCRATCH_XFS_LIST_METADATA_FIELDS=u3.sfdir3.hdr.parent.i4 2020-12-18 01:24:14 export SCRATCH_XFS_LIST_FUZZ_VERBS=random 2020-12-18 01:24:14 sed "s:^:xfs/:" //lkp/benchmarks/xfstests/tests/xfs-group-51 2020-12-18 01:24:14 ./check xfs/510 xfs/511 xfs/512 xfs/513 xfs/514 xfs/515 xfs/516 xfs/517 xfs/518 xfs/519 FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 lkp-ivb-d05 5.10.0-rc5-00026-g237d7887ae72 #1 SMP Fri Dec 18 08:12:46 CST 2020 MKFS_OPTIONS -- -f -bsize=4096 /dev/sdb4 MOUNT_OPTIONS -- /dev/sdb4 /fs/scratch xfs/510 1s xfs/511 2s xfs/512 2s xfs/513 - output mismatch (see /lkp/benchmarks/xfstests/results//xfs/513.out.bad) --- tests/xfs/513.out 2020-12-15 12:31:26.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//xfs/513.out.bad 2020-12-18 01:24:44.695412845 +0000 @@ -77,7 +77,11 @@ TEST: "-o usrquota" "pass" "usrquota" "true" TEST: "-o quota" "pass" "usrquota" "true" TEST: "-o uqnoenforce" "pass" "usrquota" "true" +[FAILED]: mount /dev/loop0 /fs/sdb1/513.mnt -o uqnoenforce +ERROR: did not expect to find "usrquota" in "rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,uqnoenforce" TEST: "-o qnoenforce" "pass" "usrquota" "true" +[FAILED]: mount /dev/loop0 /fs/sdb1/513.mnt -o qnoenforce ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/xfs/513.out /lkp/benchmarks/xfstests/results//xfs/513.out.bad' to see the entire diff) xfs/514 - output mismatch (see /lkp/benchmarks/xfstests/results//xfs/514.out.bad) --- tests/xfs/514.out 2020-12-15 12:31:26.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//xfs/514.out.bad 2020-12-18 01:24:46.007412903 +0000 @@ -1,2 +1,5 @@ QA output created by 514 Silence is golden +attr_remove not documented in the xfs_db manpage +attr_set not documented in the xfs_db manpage +logformat not documented in the xfs_db manpage ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/xfs/514.out /lkp/benchmarks/xfstests/results//xfs/514.out.bad' to see the entire diff) xfs/515 - output mismatch (see /lkp/benchmarks/xfstests/results//xfs/515.out.bad) --- tests/xfs/515.out 2020-12-15 12:31:26.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//xfs/515.out.bad 2020-12-18 01:24:47.082412951 +0000 @@ -1,2 +1,3 @@ QA output created by 515 Silence is golden +limit not documented in the xfs_quota manpage ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/xfs/515.out /lkp/benchmarks/xfstests/results//xfs/515.out.bad' to see the entire diff) xfs/516 32s xfs/517 [not run] rmapbt not supported by scratch filesystem type: xfs xfs/518 3s xfs/519 [not run] Reflink not supported by scratch filesystem type: xfs Ran: xfs/510 xfs/511 xfs/512 xfs/513 xfs/514 xfs/515 xfs/516 xfs/517 xfs/518 xfs/519 Not run: xfs/517 xfs/519 Failures: xfs/513 xfs/514 xfs/515 Failed 3 of 10 tests To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

9 months, 1 week

2
1
0 / 0

[perf/x86] e506d1dac0: stress-ng.sigsuspend.ops_per_sec 58.5% improvement

by kernel test robot

Greeting, FYI, we noticed a 58.5% improvement of stress-ng.sigsuspend.ops_per_sec due to commit: commit: e506d1dac0edb2df82f2aa0582e814f9cd9aa07d ("perf/x86: Make dummy_iregs static") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: stress-ng on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory with following parameters: nr_threads: 100% disk: 1HDD testtime: 30s class: interrupt cpufreq_governor: performance ucode: 0x5002f01 Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: interrupt/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/stress-ng/30s/0x5002f01 commit: 76a4efa809 ("perf/arch: Remove perf_sample_data::regs_user_copy") e506d1dac0 ("perf/x86: Make dummy_iregs static") 76a4efa80900fc40 e506d1dac0edb2df82f2aa0582e ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :6 17% 1:6 kmsg.BTRFS_error(device_sda1):bdev/dev/sda1_errs:wr#,rd#,flush#,corrupt#,gen %stddev %change %stddev \ | \ 19676113 ± 2% +38.4% 27232673 stress-ng.sigrt.ops 655554 ± 2% +38.2% 906196 stress-ng.sigrt.ops_per_sec 19953963 ± 4% +58.5% 31619237 stress-ng.sigsuspend.ops 665085 ± 4% +58.5% 1053922 stress-ng.sigsuspend.ops_per_sec 1.199e+08 ± 3% +40.0% 1.677e+08 ± 6% stress-ng.time.involuntary_context_switches 3.251e+08 ± 6% +21.7% 3.958e+08 ± 5% stress-ng.time.voluntary_context_switches 92940673 -2.8% 90309178 interrupts.CAL:Function_call_interrupts 672419 ± 4% +23.8% 832364 ± 5% vmstat.system.cs 46309 ± 4% -9.3% 41992 ± 4% slabinfo.Acpi-State.active_objs 912.33 ± 4% -9.4% 826.17 ± 4% slabinfo.Acpi-State.active_slabs 46559 ± 4% -9.5% 42156 ± 4% slabinfo.Acpi-State.num_objs 912.33 ± 4% -9.4% 826.17 ± 4% slabinfo.Acpi-State.num_slabs 476933 ± 4% +8.1% 515358 ± 3% sched_debug.cpu.avg_idle.avg 2404226 ± 5% +17.0% 2813799 ± 6% sched_debug.cpu.nr_switches.avg 2707460 ± 3% +21.0% 3276452 ± 8% sched_debug.cpu.nr_switches.max 2046442 ± 5% +12.4% 2300906 ± 7% sched_debug.cpu.nr_switches.min 150123 ± 14% +47.4% 221244 ± 20% sched_debug.cpu.nr_switches.stddev 2494371 ± 5% +16.3% 2900267 ± 5% sched_debug.cpu.sched_count.avg 3030416 ± 5% +15.6% 3504469 ± 9% sched_debug.cpu.sched_count.max 1526367 ± 8% +17.8% 1797379 ± 7% sched_debug.cpu.ttwu_count.avg 1737242 ± 7% +18.8% 2064291 ± 7% sched_debug.cpu.ttwu_count.max 1321299 ± 8% +15.6% 1527713 ± 10% sched_debug.cpu.ttwu_count.min 1053766 ± 2% +16.9% 1231415 ± 5% sched_debug.cpu.ttwu_local.avg 1254194 +19.1% 1493169 ± 7% sched_debug.cpu.ttwu_local.max 23.91 -2.4 21.54 ± 6% perf-stat.i.cache-miss-rate% 59213457 ± 4% -17.8% 48681553 ± 9% perf-stat.i.cache-misses 604543 ± 4% +27.6% 771225 ± 4% perf-stat.i.context-switches 22934 ± 5% +30.1% 29829 ± 11% perf-stat.i.cycles-between-cache-misses 4814712 ± 12% +21.2% 5834775 ± 12% perf-stat.i.iTLB-loads 14029258 ± 2% -27.2% 10214682 ± 20% perf-stat.i.node-load-misses 3338 ± 5% +16.6% 3892 ± 9% perf-stat.overall.cycles-between-cache-misses 69.59 -5.9 63.73 ± 3% perf-stat.overall.node-load-miss-rate% 66908064 ± 4% -14.0% 57544495 ± 8% perf-stat.ps.cache-misses 673703 ± 4% +23.6% 832871 ± 5% perf-stat.ps.context-switches 61553 ± 13% +12.4% 69199 ± 12% perf-stat.ps.cpu-migrations 5187429 ± 10% +19.1% 6177199 ± 11% perf-stat.ps.iTLB-loads 13948921 ± 2% -25.7% 10365005 ± 18% perf-stat.ps.node-load-misses 28.53 ± 70% -14.1 14.39 ±141% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_pwrite64.do_syscall_64 28.41 ± 70% -14.1 14.34 ±141% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_pwrite64 19.52 ± 70% -9.4 10.10 ±141% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 13.61 ± 70% -6.7 6.89 ±141% perf-profile.calltrace.cycles-pp.__clear_extent_bit.clear_extent_bit.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter 13.61 ± 70% -6.7 6.89 ±141% perf-profile.calltrace.cycles-pp.clear_extent_bit.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 13.52 ± 70% -6.7 6.84 ±141% perf-profile.calltrace.cycles-pp.clear_state_bit.__clear_extent_bit.clear_extent_bit.btrfs_dirty_pages.btrfs_buffered_write 13.51 ± 70% -6.7 6.83 ±141% perf-profile.calltrace.cycles-pp.btrfs_clear_delalloc_extent.clear_state_bit.__clear_extent_bit.clear_extent_bit.btrfs_dirty_pages 7.95 ± 71% -4.2 3.79 ±142% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 7.68 ± 71% -4.0 3.66 ±142% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 7.67 ± 71% -4.0 3.66 ±142% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 7.45 ± 71% -3.9 3.57 ±142% perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_clear_delalloc_extent.clear_state_bit.__clear_extent_bit.clear_extent_bit 7.44 ± 71% -3.9 3.56 ±142% perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_clear_delalloc_extent.clear_state_bit.__clear_extent_bit 7.39 ± 71% -3.9 3.54 ±142% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_clear_delalloc_extent.clear_state_bit 7.34 ± 71% -3.8 3.50 ±143% perf-profile.calltrace.cycles-pp._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 7.27 ± 71% -3.8 3.48 ±142% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_clear_delalloc_extent 7.22 ± 71% -3.8 3.44 ±143% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 5.75 ± 72% -2.6 3.13 ±142% perf-profile.calltrace.cycles-pp.__set_extent_bit.set_extent_bit.btrfs_set_extent_delalloc.btrfs_dirty_pages.btrfs_buffered_write 5.75 ± 72% -2.6 3.13 ±142% perf-profile.calltrace.cycles-pp.set_extent_bit.btrfs_set_extent_delalloc.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter 5.75 ± 72% -2.6 3.13 ±142% perf-profile.calltrace.cycles-pp.btrfs_set_extent_delalloc.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 5.67 ± 72% -2.6 3.09 ±142% perf-profile.calltrace.cycles-pp.set_state_bits.__set_extent_bit.set_extent_bit.btrfs_set_extent_delalloc.btrfs_dirty_pages 5.67 ± 72% -2.6 3.09 ±142% perf-profile.calltrace.cycles-pp.btrfs_set_delalloc_extent.set_state_bits.__set_extent_bit.set_extent_bit.btrfs_set_extent_delalloc 5.47 ± 72% -2.5 2.98 ±142% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_clear_delalloc_extent.clear_state_bit.__clear_extent_bit.clear_extent_bit 5.43 ± 72% -2.5 2.96 ±142% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_set_delalloc_extent.set_state_bits.__set_extent_bit.set_extent_bit 5.32 ± 73% -2.4 2.91 ±142% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_set_delalloc_extent.set_state_bits.__set_extent_bit 5.32 ± 73% -2.4 2.92 ±142% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_clear_delalloc_extent.clear_state_bit.__clear_extent_bit 28.53 ± 70% -14.1 14.39 ±141% perf-profile.children.cycles-pp.btrfs_file_write_iter 28.42 ± 70% -14.1 14.34 ±141% perf-profile.children.cycles-pp.btrfs_buffered_write 19.52 ± 70% -9.4 10.10 ±141% perf-profile.children.cycles-pp.btrfs_dirty_pages 13.72 ± 70% -6.8 6.94 ±141% perf-profile.children.cycles-pp.__clear_extent_bit 13.61 ± 70% -6.7 6.89 ±141% perf-profile.children.cycles-pp.clear_extent_bit 13.57 ± 70% -6.7 6.87 ±141% perf-profile.children.cycles-pp.clear_state_bit 13.52 ± 70% -6.7 6.84 ±141% perf-profile.children.cycles-pp.btrfs_clear_delalloc_extent 7.95 ± 71% -4.2 3.79 ±142% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 7.93 ± 71% -4.2 3.77 ±142% perf-profile.children.cycles-pp.__reserve_bytes 7.68 ± 71% -4.0 3.66 ±142% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 7.48 ± 71% -3.9 3.58 ±142% perf-profile.children.cycles-pp.btrfs_inode_rsv_release 7.46 ± 71% -3.9 3.57 ±142% perf-profile.children.cycles-pp.btrfs_block_rsv_release 5.91 ± 72% -2.7 3.21 ±142% perf-profile.children.cycles-pp.__set_extent_bit 5.81 ± 72% -2.7 3.16 ±142% perf-profile.children.cycles-pp.set_extent_bit 5.75 ± 72% -2.6 3.13 ±142% perf-profile.children.cycles-pp.btrfs_set_extent_delalloc 5.69 ± 72% -2.6 3.10 ±142% perf-profile.children.cycles-pp.set_state_bits 5.68 ± 72% -2.6 3.09 ±142% perf-profile.children.cycles-pp.btrfs_set_delalloc_extent stress-ng.sigsuspend.ops_per_sec 1.1e+06 +----------------------------------------------------------------+ 1.05e+06 |-+ O O O | | O O | 1e+06 |-+ O O O | 950000 |-O O O O | | O | 900000 |-+ | 850000 |-+ O | 800000 |-+ | | | 750000 |-+ + +. | 700000 |-+ +..O + : + +.+ .+. .+. .+.. | |+ .+.+. + +.+ : .+ + .+ + +.+ .+ +. .+.+. +| 650000 |-+ + +.+ +. + .+ + + | 600000 +----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP December 2020