May 2020 - LKP - Ml01.01.Org

[btrfs] ba60dcc6ad: aim7.jobs-per-min 17.2% improvement

by kernel test robot

Greeting, FYI, we noticed a 17.2% improvement of aim7.jobs-per-min due to commit: commit: ba60dcc6ad75409ddb24dbc3524be91ff5673522 ("btrfs: speed up btrfs_get_##bits helpers") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: aim7 on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory with following parameters: disk: 4BRD_12G md: RAID0 fs: btrfs test: sync_disk_rw load: 20 cpufreq_governor: performance ucode: 0x2000065 test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/20/RAID0/debian-x86_64-20191114.cgz/lkp-skl-2sp7/sync_disk_rw/aim7/0x2000065 commit: 77b3e2e295 ("btrfs: add separate bounds checker for set/get helpers") ba60dcc6ad ("btrfs: speed up btrfs_get_##bits helpers") 77b3e2e2959c41ad ba60dcc6ad75409ddb24dbc3524 ---------------- --------------------------- %stddev %change %stddev \ | \ 652.44 +17.2% 764.57 aim7.jobs-per-min 183.94 -14.7% 156.97 aim7.time.elapsed_time 183.94 -14.7% 156.97 aim7.time.elapsed_time.max 1070 -3.0% 1037 aim7.time.system_time 8.73 ± 2% +1.2 9.95 mpstat.cpu.all.sys% 148537 -11.9% 130806 ± 4% meminfo.AnonHugePages 56166 +17.4% 65950 ± 2% meminfo.max_used_kB 511566 -14.2% 439035 proc-vmstat.pgfault 917692 -6.6% 857299 proc-vmstat.pgfree 91.00 -1.6% 89.50 vmstat.cpu.id 187077 +17.4% 219708 vmstat.io.bo 302600 +16.4% 352080 vmstat.system.cs 91.23 -1.3% 90.03 iostat.cpu.idle 8.67 ± 2% +13.7% 9.86 iostat.cpu.system 11387 +16.9% 13309 iostat.md0.w/s 219488 +17.5% 257954 iostat.md0.wkB/s 0.15 ± 17% +23.9% 0.19 ± 4% sched_debug.cfs_rq:/.nr_running.avg 278.81 ± 14% +67.6% 467.42 ± 30% sched_debug.cfs_rq:/.removed.load_avg.max 278.81 ± 14% +67.6% 467.42 ± 30% sched_debug.cfs_rq:/.removed.runnable_avg.max 378.07 ± 11% +25.4% 474.07 ± 19% sched_debug.cfs_rq:/.runnable_avg.avg 1830 ± 6% +28.9% 2358 ± 14% sched_debug.cfs_rq:/.runnable_avg.max 403.14 ± 6% +27.0% 511.80 ± 14% sched_debug.cfs_rq:/.runnable_avg.stddev 180.25 ± 7% +20.0% 216.35 ± 9% sched_debug.cfs_rq:/.util_avg.avg 757.02 ± 8% +14.7% 868.67 ± 8% sched_debug.cfs_rq:/.util_avg.max 190.63 ± 2% +15.7% 220.63 ± 8% sched_debug.cfs_rq:/.util_avg.stddev 29.49 ± 17% +25.4% 36.97 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.avg 341.71 ± 6% +15.8% 395.83 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.max 86.51 ± 8% +20.8% 104.52 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev 109927 ± 11% -27.0% 80219 ± 16% sched_debug.cpu.clock.avg 109929 ± 11% -27.0% 80221 ± 16% sched_debug.cpu.clock.max 109924 ± 11% -27.0% 80215 ± 16% sched_debug.cpu.clock.min 109927 ± 11% -27.0% 80219 ± 16% sched_debug.cpu.clock_task.avg 109929 ± 11% -27.0% 80221 ± 16% sched_debug.cpu.clock_task.max 109924 ± 11% -27.0% 80215 ± 16% sched_debug.cpu.clock_task.min 4233 ± 7% -17.7% 3484 ± 9% sched_debug.cpu.curr->pid.max 0.13 ± 10% +21.6% 0.16 ± 4% sched_debug.cpu.nr_running.avg 137916 ± 18% -37.4% 86338 ± 34% sched_debug.cpu.nr_switches.stddev 137866 ± 18% -37.5% 86122 ± 35% sched_debug.cpu.sched_count.stddev 68524 ± 18% -37.6% 42786 ± 35% sched_debug.cpu.sched_goidle.stddev 63756 ± 19% -36.7% 40386 ± 33% sched_debug.cpu.ttwu_count.stddev 109925 ± 11% -27.0% 80216 ± 16% sched_debug.cpu_clk 109423 ± 11% -27.1% 79715 ± 16% sched_debug.ktime 110306 ± 11% -26.9% 80599 ± 16% sched_debug.sched_clk 9.65 ± 9% +32.1% 12.75 ± 12% perf-stat.i.MPKI 1.801e+09 +1.0% 1.819e+09 perf-stat.i.branch-instructions 1.36 ± 7% +0.2 1.57 ± 8% perf-stat.i.branch-miss-rate% 22954195 ± 2% +9.9% 25234835 perf-stat.i.branch-misses 23058509 ± 2% +18.5% 27323213 ± 4% perf-stat.i.cache-misses 89495729 ± 2% +14.9% 1.028e+08 perf-stat.i.cache-references 305404 +16.6% 356069 perf-stat.i.context-switches 2.47 ± 2% +20.9% 2.99 ± 2% perf-stat.i.cpi 2.33e+10 +13.0% 2.632e+10 perf-stat.i.cpu-cycles 116.12 ± 2% +5.7% 122.75 perf-stat.i.cpu-migrations 0.01 ± 92% +0.0 0.03 ± 74% perf-stat.i.dTLB-load-miss-rate% 2.376e+09 -4.9% 2.26e+09 perf-stat.i.dTLB-loads 0.00 ± 88% +0.0 0.01 ± 69% perf-stat.i.dTLB-store-miss-rate% 1.311e+09 -16.5% 1.095e+09 perf-stat.i.dTLB-stores 6160361 +11.4% 6863984 perf-stat.i.iTLB-loads 9.572e+09 -6.2% 8.978e+09 perf-stat.i.instructions 0.42 -15.9% 0.35 perf-stat.i.ipc 0.32 +12.9% 0.37 perf-stat.i.metric.GHz 0.38 +18.3% 0.45 ± 5% perf-stat.i.metric.K/sec 77.65 -5.3% 73.50 perf-stat.i.metric.M/sec 6645684 ± 3% +18.4% 7870425 ± 4% perf-stat.i.node-load-misses 548364 +16.0% 636213 ± 3% perf-stat.i.node-loads 2119446 ± 3% +19.8% 2538223 ± 4% perf-stat.i.node-store-misses 404397 ± 4% +14.6% 463633 perf-stat.i.node-stores 9.35 ± 2% +22.5% 11.45 perf-stat.overall.MPKI 1.27 ± 2% +0.1 1.39 perf-stat.overall.branch-miss-rate% 2.43 +20.4% 2.93 perf-stat.overall.cpi 0.01 ± 28% +0.0 0.01 ± 26% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 31% +0.0 0.00 ± 22% perf-stat.overall.dTLB-store-miss-rate% 0.41 -17.0% 0.34 perf-stat.overall.ipc 1.791e+09 +0.9% 1.807e+09 perf-stat.ps.branch-instructions 22831499 ± 2% +9.8% 25079408 perf-stat.ps.branch-misses 22928411 ± 2% +18.4% 27142974 ± 4% perf-stat.ps.cache-misses 88995180 ± 2% +14.8% 1.022e+08 perf-stat.ps.cache-references 303678 +16.5% 353708 perf-stat.ps.context-switches 2.317e+10 +12.8% 2.615e+10 perf-stat.ps.cpu-cycles 115.48 ± 2% +5.6% 121.97 perf-stat.ps.cpu-migrations 2.363e+09 -5.0% 2.245e+09 perf-stat.ps.dTLB-loads 1.304e+09 -16.6% 1.088e+09 perf-stat.ps.dTLB-stores 6125836 +11.3% 6818767 perf-stat.ps.iTLB-loads 9.518e+09 -6.3% 8.92e+09 perf-stat.ps.instructions 6608080 ± 3% +18.3% 7818361 ± 4% perf-stat.ps.node-load-misses 545275 +15.9% 632022 ± 3% perf-stat.ps.node-loads 2107458 ± 3% +19.6% 2521437 ± 4% perf-stat.ps.node-store-misses 402129 ± 4% +14.5% 460585 perf-stat.ps.node-stores 1.76e+12 -20.1% 1.405e+12 perf-stat.total.instructions 158.25 ± 5% -16.0% 133.00 ± 4% interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc 370.00 -14.5% 316.50 interrupts.9:IO-APIC.9-fasteoi.acpi 370201 -15.8% 311767 interrupts.CPU0.LOC:Local_timer_interrupts 370.00 -14.5% 316.50 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 370634 -15.9% 311529 interrupts.CPU1.LOC:Local_timer_interrupts 370287 -15.8% 311709 interrupts.CPU10.LOC:Local_timer_interrupts 370363 -15.8% 311706 interrupts.CPU11.LOC:Local_timer_interrupts 370330 -15.8% 311986 interrupts.CPU12.LOC:Local_timer_interrupts 370410 -15.9% 311631 interrupts.CPU13.LOC:Local_timer_interrupts 370298 -15.8% 311808 interrupts.CPU14.LOC:Local_timer_interrupts 370379 -15.8% 311836 interrupts.CPU15.LOC:Local_timer_interrupts 370365 -15.8% 311720 interrupts.CPU16.LOC:Local_timer_interrupts 370381 -15.8% 312007 interrupts.CPU17.LOC:Local_timer_interrupts 370307 -15.5% 312750 interrupts.CPU18.LOC:Local_timer_interrupts 370322 -15.5% 312834 interrupts.CPU19.LOC:Local_timer_interrupts 7226 ± 14% -22.5% 5601 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts 370368 -15.9% 311621 interrupts.CPU2.LOC:Local_timer_interrupts 370250 -15.5% 312869 interrupts.CPU20.LOC:Local_timer_interrupts 6701 ± 16% -24.3% 5075 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts 370366 -15.5% 312834 interrupts.CPU21.LOC:Local_timer_interrupts 370301 -15.4% 313123 interrupts.CPU22.LOC:Local_timer_interrupts 370302 -15.5% 312870 interrupts.CPU23.LOC:Local_timer_interrupts 370346 -15.5% 313001 interrupts.CPU24.LOC:Local_timer_interrupts 370343 -15.5% 312840 interrupts.CPU25.LOC:Local_timer_interrupts 370086 -15.4% 313027 interrupts.CPU26.LOC:Local_timer_interrupts 370347 -15.4% 313152 interrupts.CPU27.LOC:Local_timer_interrupts 370388 -15.5% 313139 interrupts.CPU28.LOC:Local_timer_interrupts 370264 -15.5% 312901 interrupts.CPU29.LOC:Local_timer_interrupts 370360 -15.9% 311495 interrupts.CPU3.LOC:Local_timer_interrupts 370388 -15.4% 313303 interrupts.CPU30.LOC:Local_timer_interrupts 370329 -15.4% 313134 interrupts.CPU31.LOC:Local_timer_interrupts 370327 -15.5% 313029 interrupts.CPU32.LOC:Local_timer_interrupts 370375 -15.5% 313149 interrupts.CPU33.LOC:Local_timer_interrupts 370331 -15.5% 312891 interrupts.CPU34.LOC:Local_timer_interrupts 370337 -15.5% 313117 interrupts.CPU35.LOC:Local_timer_interrupts 370323 -15.7% 312000 interrupts.CPU36.LOC:Local_timer_interrupts 370333 -15.8% 311899 interrupts.CPU37.LOC:Local_timer_interrupts 370344 -15.8% 311845 interrupts.CPU38.LOC:Local_timer_interrupts 370385 -15.8% 311961 interrupts.CPU39.LOC:Local_timer_interrupts 370382 -15.9% 311475 interrupts.CPU4.LOC:Local_timer_interrupts 370361 -15.8% 311990 interrupts.CPU40.LOC:Local_timer_interrupts 370341 -15.8% 311964 interrupts.CPU41.LOC:Local_timer_interrupts 370352 -15.8% 311965 interrupts.CPU42.LOC:Local_timer_interrupts 370269 -15.7% 312031 interrupts.CPU43.LOC:Local_timer_interrupts 370365 -15.7% 312182 interrupts.CPU44.LOC:Local_timer_interrupts 370280 -15.8% 311899 interrupts.CPU45.LOC:Local_timer_interrupts 370280 -15.7% 312058 interrupts.CPU46.LOC:Local_timer_interrupts 370356 -15.7% 312066 interrupts.CPU47.LOC:Local_timer_interrupts 370295 -15.7% 312055 interrupts.CPU48.LOC:Local_timer_interrupts 370371 -15.8% 312037 interrupts.CPU49.LOC:Local_timer_interrupts 370308 -15.9% 311591 interrupts.CPU5.LOC:Local_timer_interrupts 370363 -15.7% 312242 interrupts.CPU50.LOC:Local_timer_interrupts 370380 -15.7% 312307 interrupts.CPU51.LOC:Local_timer_interrupts 370377 -15.7% 312179 interrupts.CPU52.LOC:Local_timer_interrupts 370051 -15.6% 312363 interrupts.CPU53.LOC:Local_timer_interrupts 370295 -15.5% 312971 interrupts.CPU54.LOC:Local_timer_interrupts 370298 -15.5% 312934 interrupts.CPU55.LOC:Local_timer_interrupts 370466 -15.5% 313096 interrupts.CPU56.LOC:Local_timer_interrupts 370342 -15.4% 313214 interrupts.CPU57.LOC:Local_timer_interrupts 370106 -15.4% 313102 interrupts.CPU58.LOC:Local_timer_interrupts 370288 -15.4% 313119 interrupts.CPU59.LOC:Local_timer_interrupts 370365 -15.8% 311706 interrupts.CPU6.LOC:Local_timer_interrupts 370331 -15.5% 313054 interrupts.CPU60.LOC:Local_timer_interrupts 370213 -15.5% 312903 interrupts.CPU61.LOC:Local_timer_interrupts 480.75 ± 57% +336.7% 2099 ± 37% interrupts.CPU61.NMI:Non-maskable_interrupts 480.75 ± 57% +336.7% 2099 ± 37% interrupts.CPU61.PMI:Performance_monitoring_interrupts 370309 -15.6% 312580 interrupts.CPU62.LOC:Local_timer_interrupts 370269 -15.5% 312844 interrupts.CPU63.LOC:Local_timer_interrupts 370334 -15.6% 312722 interrupts.CPU64.LOC:Local_timer_interrupts 370360 -15.5% 312803 interrupts.CPU65.LOC:Local_timer_interrupts 370318 -15.5% 313023 interrupts.CPU66.LOC:Local_timer_interrupts 370357 -15.5% 313002 interrupts.CPU67.LOC:Local_timer_interrupts 370352 -15.5% 313043 interrupts.CPU68.LOC:Local_timer_interrupts 370358 -15.5% 312881 interrupts.CPU69.LOC:Local_timer_interrupts 158.25 ± 5% -16.0% 133.00 ± 4% interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc 370320 -15.7% 312094 interrupts.CPU7.LOC:Local_timer_interrupts 370325 -15.4% 313367 interrupts.CPU70.LOC:Local_timer_interrupts 370488 -15.4% 313267 interrupts.CPU71.LOC:Local_timer_interrupts 370341 -15.8% 311701 interrupts.CPU8.LOC:Local_timer_interrupts 370413 -15.8% 311795 interrupts.CPU9.LOC:Local_timer_interrupts 26663876 -15.6% 22496138 interrupts.LOC:Local_timer_interrupts 2.87 ± 3% -1.1 1.80 ± 3% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.btree_write_cache_pages 2.95 ± 3% -1.1 1.88 ± 3% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.btree_write_cache_pages.do_writepages 3.33 ± 3% -1.0 2.29 ± 3% perf-profile.calltrace.cycles-pp.submit_one_bio.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents 3.33 ± 3% -1.0 2.29 ± 3% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range 3.52 ± 3% -1.0 2.48 ± 3% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_sync_log 3.52 ± 3% -1.0 2.49 ± 4% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter 3.52 ± 3% -1.0 2.49 ± 4% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_sync_log.btrfs_sync_file 3.53 ± 3% -1.0 2.50 ± 3% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter.new_sync_write 44.42 -1.0 43.41 perf-profile.calltrace.cycles-pp.secondary_startup_64 5.51 ± 4% -1.0 4.54 ± 4% perf-profile.calltrace.cycles-pp.btrfs_sync_log.btrfs_sync_file.btrfs_file_write_iter.new_sync_write.vfs_write 0.92 ± 4% -0.5 0.39 ± 57% perf-profile.calltrace.cycles-pp.btrfs_get_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio 1.00 ± 5% -0.4 0.55 ± 4% perf-profile.calltrace.cycles-pp.btrfs_get_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio 0.61 ± 3% -0.4 0.26 ±100% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.62 ± 8% -0.3 0.29 ±101% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.btrfs_work_helper.process_one_work 0.62 ± 8% -0.3 0.29 ±101% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.btrfs_work_helper.process_one_work.worker_thread 2.08 ± 4% -0.3 1.76 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 1.92 ± 4% -0.3 1.62 ± 2% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 1.19 ± 6% -0.2 0.97 ± 2% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.61 ± 3% -0.1 0.53 ± 2% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.62 ± 5% +0.1 0.68 ± 5% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_next_old_leaf.btrfs_csum_file_blocks.btrfs_log_changed_extents.btrfs_log_inode 0.70 ± 5% +0.1 0.84 ± 9% perf-profile.calltrace.cycles-pp.cow_file_range.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages 0.72 ± 4% +0.1 0.87 ± 9% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages 0.82 ± 4% +0.2 0.97 ± 8% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__filemap_fdatawrite_range 0.70 ± 5% +0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages 0.90 ± 3% +0.2 1.06 ± 6% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__filemap_fdatawrite_range.btrfs_fdatawrite_range 1.41 ± 2% +0.2 1.58 ± 4% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_fdatawrite_range.start_ordered_ops.btrfs_sync_file 1.41 ± 2% +0.2 1.59 ± 4% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_fdatawrite_range.start_ordered_ops.btrfs_sync_file.btrfs_file_write_iter 1.41 ± 2% +0.2 1.59 ± 4% perf-profile.calltrace.cycles-pp.btrfs_fdatawrite_range.start_ordered_ops.btrfs_sync_file.btrfs_file_write_iter.new_sync_write 1.42 ± 2% +0.2 1.60 ± 4% perf-profile.calltrace.cycles-pp.start_ordered_ops.btrfs_sync_file.btrfs_file_write_iter.new_sync_write.vfs_write 1.40 ± 2% +0.2 1.58 ± 4% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__filemap_fdatawrite_range.btrfs_fdatawrite_range.start_ordered_ops 1.07 ± 2% +0.2 1.32 ± 9% perf-profile.calltrace.cycles-pp.__btrfs_update_delayed_inode.btrfs_commit_inode_delayed_items.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe 0.90 ± 3% +0.2 1.15 ± 11% perf-profile.calltrace.cycles-pp.btrfs_read_lock_root_node.btrfs_search_slot.btrfs_lookup_inode.__btrfs_update_delayed_inode.btrfs_commit_inode_delayed_items 1.11 ± 2% +0.3 1.37 ± 9% perf-profile.calltrace.cycles-pp.btrfs_commit_inode_delayed_items.btrfs_log_inode.btrfs_log_inode_parent.btrfs_log_dentry_safe.btrfs_sync_file 0.99 ± 3% +0.3 1.25 ± 10% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_inode.__btrfs_update_delayed_inode.btrfs_commit_inode_delayed_items.btrfs_log_inode 0.99 ± 3% +0.3 1.25 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_inode.__btrfs_update_delayed_inode.btrfs_commit_inode_delayed_items.btrfs_log_inode.btrfs_log_inode_parent 4.69 ± 4% +0.5 5.23 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.prepare_to_wait_event.btrfs_tree_lock.btrfs_lock_root_node 11.90 ± 4% +1.2 13.06 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.prepare_to_wait_event.btrfs_tree_read_lock.btrfs_read_lock_root_node 12.08 ± 4% +1.2 13.26 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.btrfs_tree_read_lock.btrfs_read_lock_root_node.btrfs_search_slot 1.24 ± 3% -1.2 0.08 ± 6% perf-profile.children.cycles-pp.map_private_extent_buffer 2.94 ± 3% -1.1 1.86 ± 3% perf-profile.children.cycles-pp.check_leaf 2.96 ± 3% -1.1 1.88 ± 3% perf-profile.children.cycles-pp.btree_csum_one_bio 3.34 ± 3% -1.1 2.29 ± 3% perf-profile.children.cycles-pp.btree_submit_bio_hook 3.53 ± 3% -1.0 2.49 ± 3% perf-profile.children.cycles-pp.btree_write_cache_pages 3.53 ± 3% -1.0 2.50 ± 3% perf-profile.children.cycles-pp.btrfs_write_marked_extents 3.84 ± 3% -1.0 2.81 ± 3% perf-profile.children.cycles-pp.submit_one_bio 44.42 -1.0 43.41 perf-profile.children.cycles-pp.secondary_startup_64 44.42 -1.0 43.41 perf-profile.children.cycles-pp.cpu_startup_entry 44.43 -1.0 43.41 perf-profile.children.cycles-pp.do_idle 5.51 ± 4% -1.0 4.54 ± 4% perf-profile.children.cycles-pp.btrfs_sync_log 40.87 -0.9 40.00 ± 2% perf-profile.children.cycles-pp.cpuidle_enter 40.86 -0.9 40.00 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state 4.93 ± 3% -0.9 4.08 perf-profile.children.cycles-pp.do_writepages 4.94 ± 3% -0.9 4.08 perf-profile.children.cycles-pp.__filemap_fdatawrite_range 1.10 ± 4% -0.5 0.62 ± 4% perf-profile.children.cycles-pp.btrfs_get_32 1.16 ± 5% -0.5 0.69 ± 4% perf-profile.children.cycles-pp.btrfs_get_64 2.37 ± 3% -0.3 2.06 ± 2% perf-profile.children.cycles-pp.apic_timer_interrupt 2.17 ± 3% -0.3 1.89 ± 2% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 0.45 ± 4% -0.2 0.24 ± 6% perf-profile.children.cycles-pp.btrfs_get_8 1.36 ± 4% -0.2 1.17 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt 0.50 ± 10% -0.1 0.37 ± 5% perf-profile.children.cycles-pp.btrfs_commit_inode_delayed_inode 0.62 ± 8% -0.1 0.49 ± 18% perf-profile.children.cycles-pp.add_pending_csums 0.73 ± 2% -0.1 0.62 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.68 ± 2% -0.1 0.60 perf-profile.children.cycles-pp.irq_exit 0.52 ± 3% -0.1 0.44 ± 5% perf-profile.children.cycles-pp.tick_sched_timer 0.49 ± 9% -0.1 0.41 ± 6% perf-profile.children.cycles-pp.clockevents_program_event 0.42 ± 2% -0.1 0.35 ± 4% perf-profile.children.cycles-pp.tick_sched_handle 0.41 ± 2% -0.1 0.35 ± 5% perf-profile.children.cycles-pp.update_process_times 0.48 ± 4% -0.1 0.42 ± 6% perf-profile.children.cycles-pp.__softirqentry_text_start 0.07 ± 17% -0.0 0.03 ±102% perf-profile.children.cycles-pp.btrfs_try_granting_tickets 0.10 ± 8% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.native_irq_return_iret 0.18 ± 5% -0.0 0.16 ± 9% perf-profile.children.cycles-pp.rcu_core 0.10 ± 5% -0.0 0.07 ± 14% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.05 +0.0 0.06 perf-profile.children.cycles-pp.mark_extent_buffer_accessed 0.12 ± 4% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.write_one_eb 0.06 ± 14% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.__btrfs_map_block 0.26 ± 6% +0.0 0.29 ± 2% perf-profile.children.cycles-pp.copy_page 0.26 ± 6% +0.0 0.29 ± 2% perf-profile.children.cycles-pp.copy_extent_buffer_full 0.01 ±173% +0.0 0.05 ± 9% perf-profile.children.cycles-pp.xas_load 0.09 ± 17% +0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_dec_block_group_reservations 0.43 ± 6% +0.1 0.52 ± 11% perf-profile.children.cycles-pp.find_free_extent 0.46 ± 6% +0.1 0.56 ± 11% perf-profile.children.cycles-pp.btrfs_reserve_extent 0.70 ± 4% +0.1 0.84 ± 9% perf-profile.children.cycles-pp.cow_file_range 0.72 ± 4% +0.1 0.87 ± 9% perf-profile.children.cycles-pp.writepage_delalloc 0.82 ± 4% +0.2 0.97 ± 8% perf-profile.children.cycles-pp.__extent_writepage 0.70 ± 5% +0.2 0.85 ± 9% perf-profile.children.cycles-pp.btrfs_run_delalloc_range 0.90 ± 3% +0.2 1.06 ± 6% perf-profile.children.cycles-pp.extent_write_cache_pages 1.41 ± 2% +0.2 1.59 ± 4% perf-profile.children.cycles-pp.btrfs_fdatawrite_range 1.41 ± 2% +0.2 1.59 ± 4% perf-profile.children.cycles-pp.extent_writepages 1.42 ± 2% +0.2 1.60 ± 4% perf-profile.children.cycles-pp.start_ordered_ops 1.11 ± 2% +0.3 1.37 ± 9% perf-profile.children.cycles-pp.btrfs_commit_inode_delayed_items 10.62 ± 2% +0.6 11.21 perf-profile.children.cycles-pp.finish_wait 24.91 +1.4 26.30 perf-profile.children.cycles-pp.btrfs_read_lock_root_node 24.71 +1.4 26.12 perf-profile.children.cycles-pp.btrfs_tree_read_lock 39.35 +1.8 41.17 perf-profile.children.cycles-pp.btrfs_search_slot 35.44 ± 2% +1.9 37.34 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 33.51 ± 3% +1.9 35.44 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 1.18 ± 3% -1.1 0.07 ± 12% perf-profile.self.cycles-pp.map_private_extent_buffer 0.21 ± 6% -0.0 0.18 ± 7% perf-profile.self.cycles-pp.btrfs_get_8 0.21 -0.0 0.18 ± 2% perf-profile.self.cycles-pp.cpuidle_enter_state 0.23 ± 4% -0.0 0.21 ± 2% perf-profile.self.cycles-pp.btrfs_root_node 0.10 ± 8% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.native_irq_return_iret 0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.31 ± 4% +0.0 0.34 ± 3% perf-profile.self.cycles-pp.check_leaf 35.42 ± 2% +1.9 37.30 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 50628 -16.5% 42279 softirqs.CPU0.RCU 65901 ± 5% -12.9% 57372 ± 2% softirqs.CPU0.TIMER 51669 ± 2% -17.9% 42408 ± 2% softirqs.CPU1.RCU 64113 ± 4% -14.9% 54542 ± 2% softirqs.CPU1.TIMER 51947 -16.3% 43473 ± 2% softirqs.CPU10.RCU 27302 ± 2% -22.0% 21295 ± 20% softirqs.CPU10.SCHED 63529 ± 4% -15.3% 53798 softirqs.CPU10.TIMER 52267 ± 2% -15.2% 44340 ± 2% softirqs.CPU11.RCU 27100 ± 2% -16.1% 22733 ± 8% softirqs.CPU11.SCHED 63366 ± 5% -12.4% 55530 ± 2% softirqs.CPU11.TIMER 53004 ± 2% -16.4% 44313 ± 2% softirqs.CPU12.RCU 64026 ± 4% -14.8% 54544 ± 3% softirqs.CPU12.TIMER 52803 ± 2% -16.7% 43996 ± 4% softirqs.CPU13.RCU 26897 ± 2% -18.7% 21858 ± 15% softirqs.CPU13.SCHED 51555 ± 2% -16.5% 43071 softirqs.CPU14.RCU 26699 ± 2% -12.7% 23312 ± 2% softirqs.CPU14.SCHED 61910 ± 3% -13.4% 53634 ± 3% softirqs.CPU14.TIMER 51122 ± 2% -16.5% 42685 softirqs.CPU15.RCU 26662 ± 4% -21.2% 21000 ± 19% softirqs.CPU15.SCHED 62304 ± 3% -13.2% 54076 softirqs.CPU15.TIMER 50555 ± 2% -14.9% 43014 softirqs.CPU16.RCU 26394 -11.1% 23475 ± 3% softirqs.CPU16.SCHED 62620 ± 5% -11.5% 55436 ± 2% softirqs.CPU16.TIMER 51164 ± 2% -15.7% 43121 softirqs.CPU17.RCU 26688 ± 2% -14.8% 22738 ± 3% softirqs.CPU17.SCHED 63194 ± 3% -14.6% 53994 ± 3% softirqs.CPU17.TIMER 53406 -15.8% 44974 softirqs.CPU18.RCU 75466 ± 20% -25.2% 56453 ± 3% softirqs.CPU18.TIMER 54197 -15.9% 45562 softirqs.CPU19.RCU 63505 ± 5% -12.5% 55541 ± 3% softirqs.CPU19.TIMER 51429 -16.5% 42937 ± 2% softirqs.CPU2.RCU 63984 ± 2% -15.3% 54212 ± 2% softirqs.CPU2.TIMER 56406 ± 4% -14.3% 48325 ± 4% softirqs.CPU20.RCU 28071 ± 2% -15.3% 23774 softirqs.CPU20.SCHED 66361 ± 3% -13.3% 57505 ± 2% softirqs.CPU20.TIMER 55456 ± 4% -18.1% 45417 softirqs.CPU21.RCU 64992 ± 5% -14.6% 55524 ± 2% softirqs.CPU21.TIMER 54723 -16.9% 45473 ± 3% softirqs.CPU22.RCU 27692 -12.7% 24170 ± 3% softirqs.CPU22.SCHED 54264 -16.1% 45509 softirqs.CPU23.RCU 71202 ± 22% -25.1% 53343 softirqs.CPU23.TIMER 54510 -14.6% 46567 ± 2% softirqs.CPU24.RCU 63641 ± 5% -12.1% 55950 ± 3% softirqs.CPU24.TIMER 52822 ± 5% -13.4% 45759 ± 2% softirqs.CPU25.RCU 27468 ± 2% -17.9% 22560 ± 20% softirqs.CPU25.SCHED 63382 ± 3% -14.7% 54083 ± 3% softirqs.CPU25.TIMER 54776 -14.3% 46940 softirqs.CPU26.RCU 27611 ± 2% -15.3% 23373 ± 2% softirqs.CPU26.SCHED 64043 ± 4% -13.4% 55443 ± 3% softirqs.CPU26.TIMER 54535 -15.8% 45942 ± 2% softirqs.CPU27.RCU 62297 ± 3% -14.1% 53522 ± 3% softirqs.CPU27.TIMER 55451 -16.4% 46332 softirqs.CPU28.RCU 26766 ± 2% -13.0% 23293 ± 2% softirqs.CPU28.SCHED 63095 ± 4% -14.0% 54261 softirqs.CPU28.TIMER 54550 -18.1% 44652 ± 6% softirqs.CPU29.RCU 26847 ± 2% -13.1% 23319 ± 2% softirqs.CPU29.SCHED 62397 ± 4% -13.7% 53879 softirqs.CPU29.TIMER 52386 ± 2% -18.3% 42820 softirqs.CPU3.RCU 62910 ± 3% -14.3% 53936 softirqs.CPU3.TIMER 53033 -14.7% 45231 ± 2% softirqs.CPU30.RCU 27005 ± 3% -17.3% 22339 ± 8% softirqs.CPU30.SCHED 63234 ± 3% -13.6% 54637 ± 2% softirqs.CPU30.TIMER 53590 -14.8% 45652 ± 2% softirqs.CPU31.RCU 63358 ± 4% -13.4% 54844 ± 3% softirqs.CPU31.TIMER 53078 -15.0% 45106 ± 2% softirqs.CPU32.RCU 61888 ± 4% -12.2% 54334 ± 5% softirqs.CPU32.TIMER 53009 -15.5% 44803 softirqs.CPU33.RCU 62227 ± 4% -13.4% 53900 softirqs.CPU33.TIMER 53026 -14.3% 45451 ± 2% softirqs.CPU34.RCU 26717 ± 3% -14.2% 22922 ± 2% softirqs.CPU34.SCHED 63419 ± 4% -14.2% 54383 ± 3% softirqs.CPU34.TIMER 53486 -15.2% 45351 softirqs.CPU35.RCU 26200 ± 3% -11.8% 23108 ± 2% softirqs.CPU35.SCHED 62867 ± 3% -13.8% 54172 softirqs.CPU35.TIMER 54607 ± 3% -15.7% 46022 softirqs.CPU36.RCU 25625 ± 3% -18.0% 21015 ± 6% softirqs.CPU36.SCHED 60287 -14.0% 51858 softirqs.CPU36.TIMER 55478 ± 2% -17.4% 45816 softirqs.CPU37.RCU 60435 ± 2% -13.3% 52414 softirqs.CPU37.TIMER 55094 -18.0% 45176 softirqs.CPU38.RCU 25963 -11.2% 23043 ± 4% softirqs.CPU38.SCHED 60274 -13.2% 52304 softirqs.CPU38.TIMER 55288 -16.7% 46049 ± 2% softirqs.CPU39.RCU 26067 ± 3% -10.5% 23337 ± 3% softirqs.CPU39.SCHED 60756 -14.4% 52033 softirqs.CPU39.TIMER 51904 ± 2% -13.9% 44710 ± 4% softirqs.CPU4.RCU 27738 ± 4% -11.1% 24654 ± 6% softirqs.CPU4.SCHED 63308 ± 4% -13.4% 54816 softirqs.CPU4.TIMER 55784 -17.1% 46259 ± 2% softirqs.CPU40.RCU 25084 ± 5% -8.1% 23045 ± 3% softirqs.CPU40.SCHED 61541 ± 3% -15.1% 52227 ± 2% softirqs.CPU40.TIMER 53783 ± 2% -15.5% 45430 ± 3% softirqs.CPU41.RCU 60792 ± 2% -14.2% 52138 softirqs.CPU41.TIMER 55629 ± 2% -18.1% 45580 ± 2% softirqs.CPU42.RCU 25823 ± 3% -10.7% 23057 softirqs.CPU43.SCHED 59932 -13.1% 52067 softirqs.CPU43.TIMER 56045 -17.1% 46471 softirqs.CPU44.RCU 25795 ± 3% -10.1% 23190 softirqs.CPU44.SCHED 61027 -14.7% 52070 softirqs.CPU44.TIMER 55240 ± 3% -16.3% 46229 softirqs.CPU45.RCU 60268 -13.9% 51904 softirqs.CPU45.TIMER 56208 -16.1% 47158 softirqs.CPU46.RCU 61110 ± 2% -14.2% 52403 softirqs.CPU46.TIMER 56569 ± 2% -16.1% 47439 softirqs.CPU47.RCU 25324 ± 4% -8.9% 23068 softirqs.CPU47.SCHED 60711 ± 4% -12.2% 53314 softirqs.CPU47.TIMER 56199 ± 2% -15.0% 47768 softirqs.CPU48.RCU 26190 ± 2% -20.0% 20949 ± 14% softirqs.CPU48.SCHED 59970 -13.2% 52077 softirqs.CPU48.TIMER 56726 ± 3% -18.6% 46174 ± 6% softirqs.CPU49.RCU 25692 ± 2% -10.9% 22896 softirqs.CPU49.SCHED 59913 -12.8% 52270 softirqs.CPU49.TIMER 50938 ± 2% -15.6% 42973 softirqs.CPU5.RCU 27553 -15.8% 23192 ± 5% softirqs.CPU5.SCHED 61661 ± 3% -13.4% 53373 softirqs.CPU5.TIMER 56469 ± 3% -17.3% 46676 ± 4% softirqs.CPU50.RCU 26198 ± 3% -24.0% 19915 ± 15% softirqs.CPU50.SCHED 59599 -10.7% 53236 ± 3% softirqs.CPU50.TIMER 55247 ± 4% -15.4% 46751 softirqs.CPU51.RCU 60767 -14.5% 51956 softirqs.CPU51.TIMER 55745 ± 4% -14.9% 47419 softirqs.CPU52.RCU 61472 ± 2% -13.5% 53190 ± 2% softirqs.CPU52.TIMER 56503 ± 3% -15.0% 48035 ± 2% softirqs.CPU53.RCU 27145 ± 8% -18.7% 22060 ± 2% softirqs.CPU53.SCHED 61064 -14.2% 52372 softirqs.CPU53.TIMER 53087 ± 4% -12.7% 46366 softirqs.CPU54.RCU 26527 ± 2% -12.0% 23349 ± 3% softirqs.CPU54.SCHED 61342 ± 3% -12.2% 53887 ± 2% softirqs.CPU54.TIMER 54199 -15.3% 45880 softirqs.CPU55.RCU 26603 ± 5% -12.0% 23419 ± 2% softirqs.CPU55.SCHED 61387 ± 4% -13.1% 53360 ± 2% softirqs.CPU55.TIMER 55244 ± 3% -16.7% 46007 ± 2% softirqs.CPU56.RCU 25474 ± 4% -11.3% 22605 ± 2% softirqs.CPU56.SCHED 59462 ± 6% -13.4% 51512 ± 4% softirqs.CPU56.TIMER 54429 -15.0% 46285 ± 2% softirqs.CPU57.RCU 26382 ± 3% -18.1% 21613 ± 10% softirqs.CPU57.SCHED 61347 ± 2% -14.4% 52522 ± 2% softirqs.CPU57.TIMER 53835 -15.0% 45772 ± 8% softirqs.CPU58.RCU 27577 ± 6% -16.3% 23086 ± 3% softirqs.CPU58.SCHED 59866 ± 3% -11.5% 53004 ± 3% softirqs.CPU58.TIMER 50478 ± 3% -13.8% 43496 ± 2% softirqs.CPU59.RCU 25784 ± 4% -13.4% 22337 ± 4% softirqs.CPU59.SCHED 60158 ± 4% -13.1% 52289 ± 3% softirqs.CPU59.TIMER 52218 ± 2% -19.6% 41975 ± 4% softirqs.CPU6.RCU 26774 ± 4% -19.4% 21584 ± 18% softirqs.CPU6.SCHED 63762 ± 4% -14.7% 54407 ± 3% softirqs.CPU6.TIMER 49746 -15.3% 42137 softirqs.CPU60.RCU 61687 ± 4% -13.6% 53309 softirqs.CPU60.TIMER 49322 ± 2% -15.3% 41791 softirqs.CPU61.RCU 27039 ± 6% -15.0% 22980 softirqs.CPU61.SCHED 69574 ± 23% -22.9% 53612 ± 2% softirqs.CPU61.TIMER 49573 -15.3% 41980 softirqs.CPU62.RCU 61992 ± 3% -14.8% 52809 softirqs.CPU62.TIMER 48362 ± 2% -14.5% 41336 ± 3% softirqs.CPU63.RCU 25400 ± 2% -15.8% 21386 ± 4% softirqs.CPU63.SCHED 60986 ± 3% -13.6% 52667 ± 2% softirqs.CPU63.TIMER 49828 ± 2% -15.8% 41961 softirqs.CPU64.RCU 25943 ± 4% -16.2% 21741 ± 6% softirqs.CPU64.SCHED 60424 ± 2% -11.4% 53536 ± 3% softirqs.CPU64.TIMER 49368 -14.9% 42020 ± 2% softirqs.CPU65.RCU 49457 -16.1% 41504 softirqs.CPU66.RCU 26077 ± 4% -12.5% 22805 ± 4% softirqs.CPU66.SCHED 60464 ± 4% -12.5% 52897 ± 2% softirqs.CPU66.TIMER 49340 -15.5% 41681 ± 2% softirqs.CPU67.RCU 60540 ± 4% -13.4% 52445 ± 3% softirqs.CPU67.TIMER 48991 ± 2% -14.1% 42084 softirqs.CPU68.RCU 60573 ± 4% -12.3% 53132 ± 2% softirqs.CPU68.TIMER 48876 ± 2% -15.0% 41530 softirqs.CPU69.RCU 25521 -13.4% 22097 ± 3% softirqs.CPU69.SCHED 60505 ± 3% -13.3% 52464 ± 2% softirqs.CPU69.TIMER 51990 ± 2% -15.6% 43890 softirqs.CPU7.RCU 72260 ± 23% -25.5% 53822 softirqs.CPU7.TIMER 49780 -15.0% 42312 softirqs.CPU70.RCU 26098 -14.6% 22284 ± 3% softirqs.CPU70.SCHED 60561 ± 2% -12.4% 53026 softirqs.CPU70.TIMER 49983 -15.4% 42287 softirqs.CPU71.RCU 59954 ± 2% -12.7% 52333 ± 2% softirqs.CPU71.TIMER 52360 ± 2% -16.1% 43946 ± 2% softirqs.CPU8.RCU 27306 ± 4% -17.8% 22445 ± 10% softirqs.CPU8.SCHED 63117 ± 4% -14.1% 54208 softirqs.CPU8.TIMER 51060 ± 3% -16.6% 42603 ± 2% softirqs.CPU9.RCU 27044 ± 3% -12.4% 23694 ± 2% softirqs.CPU9.SCHED 61727 ± 4% -13.8% 53224 ± 2% softirqs.CPU9.TIMER 3819240 -15.8% 3214468 softirqs.RCU 1860868 ± 2% -11.9% 1638708 softirqs.SCHED 4500265 ± 2% -13.4% 3895407 softirqs.TIMER aim7.jobs-per-min 800 +---------------------------------------------------------------------+ | OO O OO O OO O OO O OO O O O OO O | 700 |.+ +.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++. | 600 |-: : +.++.| | : : | 500 |-: : | | : : | 400 |-: : | | :: | 300 |-+:: | 200 |-+:: | | :: | 100 |-+: | | : | 0 +---------------------------------------------------------------------+ aim7.time.elapsed_time 200 +---------------------------------------------------------------------+ 180 |-+ .+.++.| |.+ +.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++ | 160 |-:O : OO O OO O OO O OO O OO O OO O OO O OO O OO O | 140 |-: : | | : : | 120 |-: : | 100 |-: : | 80 |-+:: | | :: | 60 |-+:: | 40 |-+:: | | : | 20 |-+: | 0 +---------------------------------------------------------------------+ aim7.time.elapsed_time.max 200 +---------------------------------------------------------------------+ 180 |-+ .+.++.| |.+ +.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++.+.++ | 160 |-:O : OO O OO O OO O OO O OO O OO O OO O OO O OO O | 140 |-: : | | : : | 120 |-: : | 100 |-: : | 80 |-+:: | | :: | 60 |-+:: | 40 |-+:: | | : | 20 |-+: | 0 +---------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

8 months, 1 week

1
0
0 / 0

[treewide] a8c30e1fa8: kernel_BUG_at_mm/slub.c

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: a8c30e1fa8ff56edec6549e61ebe187aefe3d094 ("treewide: Replace one-element array with flexible-array") https://git.kernel.org/cgit/linux/kernel/git/gustavoars/linux.git testing/fam1 in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------+----------+------------+ | | v5.7-rc3 | a8c30e1fa8 | +--------------------------------------------------+----------+------------+ | boot_successes | 451 | 0 | | boot_failures | 12 | 44 | | BUG:unable_to_handle_page_fault_for_address | 1 | 8 | | Oops:#[##] | 1 | 8 | | RIP:memchr | 1 | | | Kernel_panic-not_syncing:Fatal_exception | 1 | 44 | | BUG:kernel_hang_in_boot_stage | 5 | | | BUG:kernel_hang_in_test_stage | 1 | | | INFO:rcu_sched_detected_stalls_on_CPUs/tasks | 1 | | | RIP:async_page_fault | 1 | | | RIP:iov_iter_copy_from_user_atomic | 1 | | | WARNING:at_net/sched/sch_generic.c:#dev_watchdog | 2 | | | RIP:dev_watchdog | 2 | | | RIP:clear_page_rep | 2 | | | BUG:kernel_hang_in_early-boot_stage | 2 | | | RIP:__kmalloc | 0 | 41 | | stack_segment:#[##] | 0 | 33 | | kernel_BUG_at_mm/slub.c | 0 | 3 | | invalid_opcode:#[##] | 0 | 3 | | RIP:kfree | 0 | 3 | +--------------------------------------------------+----------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 4.193210] kernel BUG at mm/slub.c:4004! [ 4.195274] invalid opcode: 0000 [#1] SMP PTI [ 4.197044] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3-00001-ga8c30e1fa8ff5 #1 [ 4.199254] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 4.199254] RIP: 0010:kfree+0xd1/0x1ba [ 4.199254] Code: b6 ff ff 84 c0 75 58 48 89 ef e8 a2 b6 ff ff 48 89 ee bf 10 00 00 00 89 c3 e8 8b b6 ff ff 84 c0 75 0a 48 8b 45 08 a8 01 75 02 <0f> 0b 48 8b 45 00 88 d9 83 ca ff d3 e2 be 06 00 00 00 48 63 d2 48 [ 4.199254] RSP: 0000:ffffc900000139e8 EFLAGS: 00010246 [ 4.199254] RAX: ffffea0008014108 RBX: 0000000000000000 RCX: ffffc900000139e0 [ 4.199254] RDX: 0000000000000000 RSI: ffffea0008014100 RDI: 0000000000000010 [ 4.199254] RBP: ffffea0008014100 R08: ffffc90000013964 R09: 0000000000000002 [ 4.199254] R10: 0000000000000000 R11: ffff88822015db40 R12: ffffffff81617093 [ 4.199254] R13: 0000000000000000 R14: ffff888229750090 R15: ffffc90000013b60 [ 4.199254] FS: 0000000000000000(0000) GS:ffff88823fd00000(0000) knlGS:0000000000000000 [ 4.199254] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4.199254] CR2: 00000000f7ef1000 CR3: 000000000280a000 CR4: 00000000000006e0 [ 4.199254] Call Trace: [ 4.199254] acpi_ns_get_node_unlocked+0xac/0xd8 [ 4.199254] ? trace_hardirqs_on+0x2e/0x3a [ 4.199254] ? down_timeout+0x41/0x48 [ 4.199254] ? acpi_ns_get_node+0x40/0x55 [ 4.199254] acpi_ns_get_node+0x40/0x55 [ 4.199254] acpi_ns_evaluate+0x4c/0x24e [ 4.199254] acpi_ut_evaluate_object+0x6b/0x185 [ 4.199254] acpi_rs_get_prt_method_data+0x33/0x71 [ 4.199254] ? acpi_rs_validate_parameters+0x39/0x4b [ 4.199254] acpi_get_irq_routing_table+0x35/0x50 [ 4.199254] acpi_pci_irq_find_prt_entry+0x7f/0x29d [ 4.199254] ? rdinit_setup+0x2b/0x2b [ 4.199254] acpi_pci_irq_lookup+0x35/0x15d [ 4.199254] acpi_pci_irq_enable+0x71/0x20d [ 4.199254] ? pci_bus_read_config_word+0x49/0x6b [ 4.199254] ? pci_enable_resources+0x2f/0x12f [ 4.199254] do_pci_enable_device+0x53/0xd5 [ 4.199254] pci_enable_device_flags+0xce/0xf5 [ 4.199254] virtio_pci_probe+0x7f/0x137 [ 4.199254] local_pci_probe+0x41/0x7f [ 4.199254] pci_device_probe+0x140/0x198 [ 4.199254] really_probe+0x1a9/0x3bb [ 4.199254] driver_probe_device+0xd4/0x109 [ 4.199254] device_driver_attach+0x3b/0x53 [ 4.199254] __driver_attach+0x112/0x11a [ 4.199254] ? device_driver_attach+0x53/0x53 [ 4.199254] bus_for_each_dev+0x71/0xa3 [ 4.199254] bus_add_driver+0x12c/0x1de [ 4.199254] ? virtio_mmio_init+0x13/0x13 [ 4.199254] driver_register+0x99/0xce [ 4.199254] ? virtio_mmio_init+0x13/0x13 [ 4.199254] do_one_initcall+0x9d/0x1c0 [ 4.199254] kernel_init_freeable+0x1e3/0x246 [ 4.199254] ? rest_init+0xc6/0xc6 [ 4.199254] kernel_init+0xa/0xff [ 4.199254] ret_from_fork+0x3a/0x50 [ 4.199254] Modules linked in: [ 4.299545] ---[ end trace eb46f27d11e5fdb3 ]--- To reproduce: # build kernel cd linux cp config-5.7.0-rc3-00001-ga8c30e1fa8ff5 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

8 months, 1 week

1
0
0 / 0

[tracepoint] 01edfaf177: WARNING:at_kernel/static_call.c:#__static_call_update

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 01edfaf177311fba9941254793431d2044b00fd7 ("[PATCH v4 12/18] tracepoint: Optimize using static_call()") url: https://github.com/0day-ci/linux/commits/Peter-Zijlstra/Add-static_call/2... in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 1.990628] WARNING: CPU: 0 PID: 0 at kernel/static_call.c:113 __static_call_update+0x83/0x29c [ 1.990660] Modules linked in: [ 1.990703] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.7.0-rc3-next-20200501-00012-g01edfaf177311f #1 [ 1.990742] RIP: 0010:__static_call_update+0x83/0x29c [ 1.990781] Code: 4c 89 f2 48 ff 05 d1 9d 83 06 e8 8e 09 d2 ff 48 ff 05 d5 9d 83 06 80 3d 8e a2 83 06 00 0f 85 f3 01 00 00 48 ff 05 c9 9d 83 06 <0f> 0b 48 ff 05 c8 9d 83 06 48 ff 05 c9 9d 83 06 e9 b0 01 00 00 48 [ 1.990814] RSP: 0000:ffffffff83c07cf8 EFLAGS: 00010002 [ 1.990850] RAX: 0000000000000000 RBX: ffffffff86682620 RCX: 0000000000000000 [ 1.990884] RDX: fffffbfff0780f87 RSI: ffffffff83c918c0 RDI: fffffbfff0780f93 [ 1.990916] RBP: ffffffff812943ec R08: fffffbfff07e3551 R09: fffffbfff07e3551 [ 1.990950] R10: ffffffff83f1aa87 R11: 0000000000000001 R12: ffffffff83f6f0c0 [ 1.990982] R13: 00000000ffffffff R14: ffffffff812943ec R15: ffffffff812943ec [ 1.991017] FS: 0000000000000000(0000) GS:ffff8881e8000000(0000) knlGS:0000000000000000 [ 1.991049] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.991081] CR2: ffff88823ffff000 CR3: 0000000003c78000 CR4: 00000000000006b0 [ 1.991119] Call Trace: [ 1.991164] ? __SCT__tp_func_sched_waking+0x8/0x8 [ 1.991202] ? allocate_probes+0x2d/0x4d [ 1.991250] ? probe_sched_switch+0x4f/0x4f [ 1.991286] ? __SCT__tp_func_sched_waking+0x8/0x8 [ 1.991331] ? probe_sched_switch+0x4f/0x4f [ 1.991366] tracepoint_update_call+0x6d/0x79 [ 1.991416] tracepoint_probe_register_prio+0x337/0x47f [ 1.991499] tracepoint_probe_register+0x16/0x1e [ 1.991538] tracing_start_sched_switch+0xc3/0x1ba [ 1.991627] tracing_start_cmdline_record+0x11/0x19 [ 1.991664] trace_printk_start_comm+0x1c/0x24 [ 1.991725] trace_event_init+0x1b8/0x20a [ 1.991775] trace_init+0xc/0x14 [ 1.991809] start_kernel+0x8ed/0xe78 [ 1.991862] ? thread_stack_cache_init+0xd/0xd [ 1.991905] ? __early_make_pgtable+0x1b9/0x204 [ 1.991963] ? early_make_pgtable+0x27/0x2f [ 1.992000] ? early_idt_handler_common+0x35/0x4c [ 1.992131] x86_64_start_reservations+0x42/0x4a [ 1.992170] x86_64_start_kernel+0xe1/0xea [ 1.992216] secondary_startup_64+0xa4/0xb0 [ 1.992336] irq event stamp: 0 [ 1.992371] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 1.992403] hardirqs last disabled at (0): [<0000000000000000>] 0x0 [ 1.992437] softirqs last enabled at (0): [<0000000000000000>] 0x0 [ 1.992469] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 1.992562] ---[ end trace 15ea566c311b9aea ]--- To reproduce: # build kernel cd linux cp config-5.7.0-rc3-next-20200501-00012-g01edfaf177311f .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 1 week

2
1
0 / 0

[selftests/bpf] da43712a72: kernel-selftests.bpf.make_fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 77bb53cb094828a31cd3c5b402899810f63073c1 ("selftests/bpf: Fix perf_buffer test on systems w/ offline CPUs") https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y in testcase: kernel-selftests with following parameters: group: kselftests-bpf test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> KERNEL SELFTESTS: linux_headers_dir is /usr/src/linux-headers-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d 2020-05-08 12:56:15 sed -i s/default_timeout=45/default_timeout=300/ kselftest/runner.sh 2020-05-08 12:56:15 make -C bpf make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf' gcc -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf -Iverifier -c -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o test_stub.c make -C ../../../lib/bpf OUTPUT=/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/ make[1]: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/lib/bpf' Auto-detecting system features: ... libelf: [ [32mon[m ] ... bpf: [ [32mon[m ] HOSTCC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/fixdep.o HOSTLD /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/fixdep-in.o LINK /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/fixdep MKDIR /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/ CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/libbpf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/bpf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/nlattr.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/btf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/libbpf_errno.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/str_error.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/netlink.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/bpf_prog_linfo.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/libbpf_probes.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/xsk.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/hashmap.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/btf_dump.o LD /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/staticobjs/libbpf-in.o LINK /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a MKDIR /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/ CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/libbpf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/bpf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/nlattr.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/btf.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/libbpf_errno.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/str_error.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/netlink.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/bpf_prog_linfo.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/libbpf_probes.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/xsk.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/hashmap.o CC /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/btf_dump.o LD /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/sharedobjs/libbpf-in.o LINK /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.so.0.0.5 GEN /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.pc LINK /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_libbpf make[1]: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/lib/bpf' gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf -Iverifier test_verifier.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_verifier gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program test_tag.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_tag gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf test_maps.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a map_tests/sk_storage_map.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_maps gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program test_lru_map.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_lru_map gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program test_lpm_map.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_lpm_map gcc -g -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf test_progs.c /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_stub.o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c trace_helpers.c prog_tests/attach_probe.c prog_tests/stacktrace_map.c prog_tests/raw_tp_writable_test_run.c prog_tests/xdp_adjust_tail.c prog_tests/signal_pending.c prog_tests/sockopt_inherit.c prog_tests/send_signal.c prog_tests/stacktrace_build_id.c prog_tests/tcp_rtt.c prog_tests/reference_tracking.c prog_tests/bpf_verif_scale.c prog_tests/prog_run_xattr.c prog_tests/task_fd_query_tp.c prog_tests/tp_attach_query.c prog_tests/get_stack_raw_tp.c prog_tests/pkt_md_access.c prog_tests/stacktrace_build_id_nmi.c prog_tests/global_data.c prog_tests/queue_stack_map.c prog_tests/pkt_access.c prog_tests/spinlock.c prog_tests/sockopt.c prog_tests/task_fd_query_rawtp.c prog_tests/xdp.c prog_tests/sockopt_sk.c prog_tests/stacktrace_map_raw_tp.c prog_tests/skb_ctx.c prog_tests/tcp_estats.c prog_tests/perf_buffer.c prog_tests/raw_tp_writable_reject_nbd_invalid.c prog_tests/sockopt_multi.c prog_tests/flow_dissector.c prog_tests/l4lb_all.c prog_tests/flow_dissector_load_bytes.c prog_tests/obj_name.c prog_tests/map_lock.c prog_tests/xdp_noinline.c prog_tests/core_reloc.c prog_tests/bpf_obj_id.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_progs prog_tests/stacktrace_build_id_nmi.c: In function ‘test_stacktrace_build_id_nmi’: prog_tests/stacktrace_build_id_nmi.c:55:3: error: label ‘cleanup’ used but not defined goto cleanup; ^~~~ prog_tests/perf_buffer.c: In function ‘test_perf_buffer’: prog_tests/perf_buffer.c:39:8: warning: implicit declaration of function ‘parse_cpu_mask_file’ [-Wimplicit-function-declaration] err = parse_cpu_mask_file("/sys/devices/system/cpu/online", ^~~~~~~~~~~~~~~~~~~ ../lib.mk:138: recipe for target '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_progs' failed make: *** [/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf/test_progs] Error 1 make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-da43712a7262891317883d4b3a909fb18dac4b1d/tools/testing/selftests/bpf' To reproduce: # build kernel cd linux cp config-5.4.18-00145-g77bb53cb09482 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 1 week

4
6
0 / 0

[selftests] 98bfc3414b: kernel-selftests.netfilter.nft_nat.sh.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 98bfc3414bda335dbd7fec58bde6266f991801d7 ("selftests: netfilter: add simple masq/redirect test cases") https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y in testcase: kernel-selftests with following parameters: group: kselftests-02 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> selftests: netfilter: nft_nat.sh ======================================== /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1 ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2 ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2 ^^^^^^ /dev/stdin:2:8-13: Error: Could not process rule: No such file or directory chain output { ^^^^^^ /dev/stdin:4:3-38: Error: Could not process rule: No such file or directory ip daddr 10.0.1.99 dnat to 10.0.2.99 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip nat output ^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns2in counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns2out counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2out ^^^^^^ /dev/stdin:2:8-13: Error: Could not process rule: No such file or directory chain output { ^^^^^^ /dev/stdin:4:3-41: Error: Could not process rule: No such file or directory ip6 daddr dead:1::99 dnat to dead:2::99 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SKIP: Could not add add ip6 dnat hook Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ /dev/stdin:2:8-18: Error: Could not process rule: No such file or directory chain postrouting { ^^^^^^^^^^^ /dev/stdin:4:3-27: Error: Could not process rule: No such file or directory meta oif veth0 masquerade ^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip nat postrouting ^^^ ERROR: Could not flush nat postrouting Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ /dev/stdin:2:8-18: Error: Could not process rule: No such file or directory chain postrouting { ^^^^^^^^^^^ /dev/stdin:4:3-27: Error: Could not process rule: No such file or directory meta oif veth0 masquerade ^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip6 nat postrouting ^^^ ERROR: Could not flush ip6 nat postrouting Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ /dev/stdin:2:8-17: Error: Could not process rule: No such file or directory chain prerouting { ^^^^^^^^^^ /dev/stdin:4:3-80: Error: Could not process rule: No such file or directory meta iif veth1 ip protocol icmp ip saddr 10.0.2.99 ip daddr 10.0.1.99 redirect ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: Could not process rule: No such file or directory delete table ip nat ^^^ ERROR: Could not delete nat table Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ /dev/stdin:2:8-17: Error: Could not process rule: No such file or directory chain prerouting { ^^^^^^^^^^ /dev/stdin:4:3-87: Error: Could not process rule: No such file or directory meta iif veth1 meta l4proto icmpv6 ip6 saddr dead:2::99 ip6 daddr dead:1::99 redirect ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: Could not process rule: No such file or directory delete table ip6 nat ^^^ ERROR: Could not delete ip6 nat table not ok 1..2 selftests: netfilter: nft_nat.sh [FAIL] To reproduce: # build kernel cd linux cp config-4.19.28-00103-g5c39e08fc417b .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 1 week

1
0
0 / 0

[treewide] 509c0326e0: stack_segment:#[##]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 509c0326e04ff7096a9f3346bc9adcff9859e089 ("treewide: Replace one-element array with flexible-array") https://git.kernel.org/cgit/linux/kernel/git/gustavoars/linux.git testing/fam1 in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 0.395752] ACPI: Interpreter enabled [ 0.396239] ACPI: (supports S0 S3 S4 S5) [ 0.396522] ACPI: Using IOAPIC for interrupt routing [ 0.397140] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.397643] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.398308] stack segment: 0000 [#1] SMP PTI [ 0.398518] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3-00001-g509c0326e04ff #1 [ 0.398518] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 0.398518] RIP: 0010:__kmalloc_track_caller+0x98/0x270 [ 0.398518] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 67 54 d7 7e 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b [ 0.398518] RSP: 0000:ffffc90000013a38 EFLAGS: 00010206 [ 0.398518] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000470 [ 0.398518] RDX: 000000000000046f RSI: 0000000000000cc0 RDI: ffff88822a403a40 [ 0.398518] RBP: 0035304130504e50 R08: 00000000000300c0 R09: 0000000000000000 [ 0.398518] R10: 0000000000000000 R11: ffff88822a059438 R12: 0000000000000cc0 [ 0.398518] R13: 000000000000000c R14: ffff88822a403a40 R15: ffff88822a403a40 [ 0.398518] FS: 0000000000000000(0000) GS:ffff88823fc00000(0000) knlGS:0000000000000000 [ 0.398518] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.398518] CR2: 0000000000000000 CR3: 000000000260a000 CR4: 00000000000406f0 [ 0.398518] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.398518] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 0.398518] Call Trace: [ 0.398518] ? __kernfs_new_node+0x40/0x1e0 [ 0.398518] kstrdup+0x2d/0x60 [ 0.398518] __kernfs_new_node+0x40/0x1e0 [ 0.398518] ? kernfs_create_dir_ns+0x50/0x80 [ 0.398518] ? acpi_ns_lookup+0x81b/0x874 [ 0.398518] ? _cond_resched+0x19/0x30 [ 0.398518] ? kernfs_activate+0x63/0x80 [ 0.398518] kernfs_new_node+0x34/0x60 [ 0.398518] kernfs_create_link+0x35/0xb0 [ 0.398518] sysfs_do_create_link_sd+0x67/0xc0 [ 0.398518] bus_add_device+0x6e/0x150 [ 0.398518] device_add+0x38b/0x840 [ 0.398518] ? kmem_cache_alloc_trace+0x21e/0x230 [ 0.398518] acpi_device_add+0x1dc/0x3f0 [ 0.398518] ? acpi_free_pnp_ids+0x50/0x50 [ 0.398518] acpi_add_single_object+0x25c/0x6d0 [ 0.398518] ? acpi_ut_release_mutex+0x122/0x128 [ 0.398518] ? acpi_get_data_full+0x7e/0x90 [ 0.398518] ? rdinit_setup+0x2b/0x2b [ 0.398518] acpi_bus_check_add+0xd2/0x260 [ 0.398518] ? acpi_ev_gpe_detect+0x140/0x152 [ 0.398518] ? acpi_sleep_proc_init+0x24/0x24 [ 0.398518] acpi_bus_scan+0x31/0x90 [ 0.398518] acpi_scan_init+0xec/0x232 [ 0.398518] ? acpi_sleep_proc_init+0x24/0x24 [ 0.398518] acpi_init+0x2f4/0x354 [ 0.398518] do_one_initcall+0x46/0x220 [ 0.398518] kernel_init_freeable+0x206/0x280 [ 0.398518] ? rest_init+0xd0/0xd0 [ 0.398518] kernel_init+0xa/0x110 [ 0.398518] ret_from_fork+0x35/0x40 [ 0.398518] Modules linked in: [ 0.398532] ---[ end trace 2101f75645609710 ]--- [ 0.399057] RIP: 0010:__kmalloc_track_caller+0x98/0x270 [ 0.399522] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 67 54 d7 7e 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b [ 0.400522] RSP: 0000:ffffc90000013a38 EFLAGS: 00010206 [ 0.401096] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000470 [ 0.401521] RDX: 000000000000046f RSI: 0000000000000cc0 RDI: ffff88822a403a40 [ 0.402254] RBP: 0035304130504e50 R08: 00000000000300c0 R09: 0000000000000000 [ 0.402520] R10: 0000000000000000 R11: ffff88822a059438 R12: 0000000000000cc0 [ 0.403256] R13: 000000000000000c R14: ffff88822a403a40 R15: ffff88822a403a40 [ 0.403522] FS: 0000000000000000(0000) GS:ffff88823fc00000(0000) knlGS:0000000000000000 [ 0.404423] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.404520] CR2: 0000000000000000 CR3: 000000000260a000 CR4: 00000000000406f0 [ 0.405253] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.405520] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 0.406263] Kernel panic - not syncing: Fatal exception Elapsed time: 60 qemu-img create -f qcow2 disk-vm-snb-ssd-55-0 256G qemu-img create -f qcow2 disk-vm-snb-ssd-55-1 256G To reproduce: # build kernel cd linux cp config-5.7.0-rc3-00001-g509c0326e04ff .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

[fs] 05c5a0273b: will-it-scale.per_thread_ops -74.6% regression

by kernel test robot

Greeting, FYI, we noticed a -74.6% regression of will-it-scale.per_thread_ops due to commit: commit: 05c5a0273bc25627b9e9bbebc858b74e88f49c5b ("[PATCH v2] fs: jfs: fix a possible data race in txBegin()") url: https://github.com/0day-ci/linux/commits/Jia-Ju-Bai/fs-jfs-fix-a-possible... base: https://github.com/kleikamp/linux-shaggy jfs-next in testcase: will-it-scale on test machine: 8 threads Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz with 16G memory with following parameters: nr_task: 100% mode: thread test: unlink2 cpufreq_governor: performance ucode: 0x21 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+----------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -84.4% regression | | test machine | 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=getppid1 | | | ucode=0x11 | +------------------+----------------------------------------------------------------------+ | testcase: change | netperf: netperf.Throughput_total_tps -71.8% regression | | test machine | 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory | | test parameters | cluster=cs-localhost | | | cpufreq_governor=performance | | | ip=ipv4 | | | nr_threads=1 | | | runtime=300s | | | test=TCP_CRR | | | ucode=0x2000065 | +------------------+----------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-ivb-d01/unlink2/will-it-scale/0x21 commit: 7aba5dcc23 ("jfs: Replace zero-length array with flexible-array member") 05c5a0273b ("fs: jfs: fix a possible data race in txBegin()") 7aba5dcc234635b4 05c5a0273bc25627b9e9bbebc85 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.RIP:kmem_cache_alloc :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x 3:4 -75% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 3:4 -75% :4 dmesg.WARNING:stack_recursion :4 50% 2:4 kmsg.IP-Config:Failed_to_open_erspan0 :4 50% 2:4 kmsg.IP-Config:Failed_to_open_gretap0 1:4 -25% :4 kmsg.cbb7>]usb_hcd_irq 1:4 -25% :4 kmsg.d54b4>]usb_hcd_irq :4 25% 1:4 kmsg.f3a47>]usb_hcd_irq :4 25% 1:4 kmsg.f681a>]usb_hcd_irq 1:4 -25% :4 kmsg.f8c>]usb_hcd_irq 1:4 -25% :4 kmsg.fffb>]usb_hcd_irq :4 50% 2:4 kmsg.kvm:already_loaded_the_other_module %stddev %change %stddev \ | \ 131035 -74.6% 33245 will-it-scale.per_thread_ops 947843 -3.9% 910679 will-it-scale.time.involuntary_context_switches 2048 +10.2% 2257 will-it-scale.time.system_time 332.26 -68.9% 103.44 will-it-scale.time.user_time 1048286 -74.6% 265968 will-it-scale.workload 18.83 +32.4% 24.93 ± 2% boot-time.boot 7.96 -17.9% 6.54 ± 14% boot-time.dhcp 128.66 +34.1% 172.50 ± 2% boot-time.idle 253528 ±141% +1060.4% 2941982 cpuidle.C3.time 1369 ±127% +489.0% 8068 ± 9% cpuidle.C3.usage 7903103 ± 15% -39.2% 4801161 ± 6% cpuidle.C6.time 2.54 ± 4% +0.8 3.39 mpstat.cpu.all.soft% 82.77 +8.7 91.45 mpstat.cpu.all.sys% 14.40 -9.5 4.90 mpstat.cpu.all.usr% 134740 ± 8% +15.1% 155027 ± 2% softirqs.CPU4.TIMER 134471 ± 5% +15.9% 155790 softirqs.CPU7.TIMER 22928 +11.8% 25638 softirqs.SCHED 84.00 +11.9% 94.00 vmstat.cpu.sy 14.00 -71.4% 4.00 vmstat.cpu.us 6741 -5.2% 6391 vmstat.system.cs 147592 ± 3% -13.9% 127112 meminfo.DirectMap4k 49716 +37.7% 68474 meminfo.KReclaimable 49716 +37.7% 68474 meminfo.SReclaimable 22434 ± 4% +7.5% 24125 ± 2% meminfo.Shmem 115521 ± 3% +20.9% 139668 meminfo.Slab 4073 +1.0% 4116 proc-vmstat.nr_kernel_stack 5603 ± 4% +7.6% 6030 ± 2% proc-vmstat.nr_shmem 12529 +37.8% 17269 proc-vmstat.nr_slab_reclaimable 17051 ± 3% +10.9% 18913 proc-vmstat.nr_slab_unreclaimable 30456663 ± 3% -77.4% 6890093 proc-vmstat.numa_hit 30456663 ± 3% -77.4% 6890093 proc-vmstat.numa_local 71202984 ± 4% -54.3% 32526510 proc-vmstat.pgalloc_normal 384490 -1.2% 380065 proc-vmstat.pgfault 71172080 ± 4% -54.3% 32497466 proc-vmstat.pgfree 2.55 ± 11% +46.0% 3.72 ± 3% sched_debug.cfs_rq:/.nr_spread_over.avg 3.94 ± 13% +23.1% 4.85 ± 8% sched_debug.cfs_rq:/.nr_spread_over.stddev 1.00 +250.0% 3.50 ± 66% sched_debug.cfs_rq:/.util_est_enqueued.min 0.45 ± 9% +17.5% 0.53 sched_debug.cpu.nr_running.stddev 398099 ± 16% -25.4% 297012 ± 5% sched_debug.cpu.nr_switches.max 68866 ± 45% +53.8% 105917 ± 10% sched_debug.cpu.nr_switches.min 108672 ± 28% -36.9% 68536 ± 11% sched_debug.cpu.nr_switches.stddev 394087 ± 16% -26.5% 289489 ± 6% sched_debug.cpu.sched_count.max 9159 ± 9% -11.9% 8071 ± 3% sched_debug.cpu.sched_goidle.max 195763 ± 16% -26.3% 144197 ± 6% sched_debug.cpu.ttwu_count.max 31795 ± 51% +51.6% 48206 ± 13% sched_debug.cpu.ttwu_count.min 194370 ± 16% -26.5% 142882 ± 6% sched_debug.cpu.ttwu_local.max 221.50 ± 31% +45.6% 322.50 ± 23% interrupts.53:PCI-MSI.528389-edge.eth3-TxRx-4 406.50 ± 17% -100.0% 0.00 interrupts.CPU0.49:PCI-MSI.528385-edge.eth3-TxRx-0 410.75 ± 51% -100.0% 0.00 interrupts.CPU1.50:PCI-MSI.528386-edge.eth3-TxRx-1 295.25 ± 41% -100.0% 0.00 interrupts.CPU2.51:PCI-MSI.528387-edge.eth3-TxRx-2 10726 ± 33% -33.8% 7098 interrupts.CPU2.NMI:Non-maskable_interrupts 10726 ± 33% -33.8% 7098 interrupts.CPU2.PMI:Performance_monitoring_interrupts 209.50 ± 18% -100.0% 0.00 interrupts.CPU3.52:PCI-MSI.528388-edge.eth3-TxRx-3 221.50 ± 31% -100.0% 0.00 interrupts.CPU4.53:PCI-MSI.528389-edge.eth3-TxRx-4 379.75 ± 23% -100.0% 0.00 interrupts.CPU5.54:PCI-MSI.528390-edge.eth3-TxRx-5 546708 ± 20% +36.4% 745448 interrupts.CPU5.LOC:Local_timer_interrupts 1345 ± 27% +192.1% 3928 ± 19% interrupts.CPU5.RES:Rescheduling_interrupts 213.25 ± 25% -100.0% 0.00 interrupts.CPU6.55:PCI-MSI.528391-edge.eth3-TxRx-6 9758 ± 30% -27.1% 7112 interrupts.CPU6.NMI:Non-maskable_interrupts 9758 ± 30% -27.1% 7112 interrupts.CPU6.PMI:Performance_monitoring_interrupts 179.25 ± 7% -100.0% 0.00 interrupts.CPU7.56:PCI-MSI.528392-edge.eth3-TxRx-7 25363 ± 8% +14.0% 28913 ± 5% interrupts.RES:Rescheduling_interrupts 5.08 ± 3% -61.9% 1.94 perf-stat.i.MPKI 3.074e+09 +45.1% 4.462e+09 perf-stat.i.branch-instructions 1.08 -0.7 0.35 perf-stat.i.branch-miss-rate% 33284020 -54.6% 15106837 perf-stat.i.branch-misses 12.57 ± 2% +6.8 19.34 perf-stat.i.cache-miss-rate% 9810523 ± 2% -8.5% 8977759 perf-stat.i.cache-misses 78283875 ± 3% -40.2% 46849959 perf-stat.i.cache-references 6783 -4.9% 6448 perf-stat.i.context-switches 1.90 -36.6% 1.21 perf-stat.i.cpi 61.66 ± 7% -45.5% 33.59 perf-stat.i.cpu-migrations 2990 ± 2% +8.7% 3250 perf-stat.i.cycles-between-cache-misses 1.84 ± 15% -0.6 1.20 ± 8% perf-stat.i.dTLB-load-miss-rate% 4.474e+09 +52.4% 6.816e+09 perf-stat.i.dTLB-loads 0.32 ± 7% -0.2 0.13 ± 10% perf-stat.i.dTLB-store-miss-rate% 10493472 ± 7% -47.5% 5505011 ± 10% perf-stat.i.dTLB-store-misses 3.233e+09 +30.1% 4.206e+09 perf-stat.i.dTLB-stores 90.39 ± 2% +7.9 98.33 perf-stat.i.iTLB-load-miss-rate% 2871267 ± 2% -65.4% 993192 perf-stat.i.iTLB-load-misses 314540 ± 26% -94.8% 16476 ± 12% perf-stat.i.iTLB-loads 1.54e+10 +57.0% 2.417e+10 perf-stat.i.instructions 5386 +352.4% 24367 perf-stat.i.instructions-per-iTLB-miss 0.53 +57.6% 0.83 perf-stat.i.ipc 0.42 +21.2% 0.50 ± 6% perf-stat.i.metric.K/sec 1358 +42.9% 1941 perf-stat.i.metric.M/sec 5.08 ± 3% -61.9% 1.94 perf-stat.overall.MPKI 1.08 -0.7 0.34 perf-stat.overall.branch-miss-rate% 12.54 ± 2% +6.6 19.16 perf-stat.overall.cache-miss-rate% 1.90 -36.5% 1.21 perf-stat.overall.cpi 2986 ± 2% +8.7% 3248 perf-stat.overall.cycles-between-cache-misses 1.85 ± 15% -0.6 1.20 ± 8% perf-stat.overall.dTLB-load-miss-rate% 0.32 ± 7% -0.2 0.13 ± 10% perf-stat.overall.dTLB-store-miss-rate% 90.18 ± 2% +8.2 98.37 perf-stat.overall.iTLB-load-miss-rate% 5366 +353.6% 24341 perf-stat.overall.instructions-per-iTLB-miss 0.53 +57.6% 0.83 perf-stat.overall.ipc 4425502 +516.9% 27302878 perf-stat.overall.path-length 3.064e+09 +45.1% 4.447e+09 perf-stat.ps.branch-instructions 33174183 -54.6% 15059388 perf-stat.ps.branch-misses 9777682 ± 2% -8.5% 8947700 perf-stat.ps.cache-misses 78021322 ± 3% -40.2% 46693223 perf-stat.ps.cache-references 6761 -4.9% 6427 perf-stat.ps.context-switches 61.46 ± 7% -45.5% 33.48 perf-stat.ps.cpu-migrations 4.459e+09 +52.4% 6.793e+09 perf-stat.ps.dTLB-loads 10458278 ± 7% -47.5% 5486565 ± 10% perf-stat.ps.dTLB-store-misses 3.222e+09 +30.1% 4.192e+09 perf-stat.ps.dTLB-stores 2861644 ± 2% -65.4% 989874 perf-stat.ps.iTLB-load-misses 313490 ± 26% -94.8% 16429 ± 12% perf-stat.ps.iTLB-loads 1.535e+10 +57.0% 2.409e+10 perf-stat.ps.instructions 4.639e+12 +56.5% 7.262e+12 perf-stat.total.instructions 3864 +39.8% 5401 slabinfo.Acpi-Operand.active_objs 3864 +39.8% 5401 slabinfo.Acpi-Operand.num_objs 529.25 ± 5% -100.0% 0.00 slabinfo.Acpi-Parse.active_objs 529.25 ± 5% -100.0% 0.00 slabinfo.Acpi-Parse.num_objs 497.25 ± 8% -17.9% 408.00 slabinfo.Acpi-State.active_objs 497.25 ± 8% -17.9% 408.00 slabinfo.Acpi-State.num_objs 121.50 ± 12% -100.0% 0.00 slabinfo.UDPv6.active_objs 121.50 ± 12% -100.0% 0.00 slabinfo.UDPv6.num_objs 3399 ± 6% -26.6% 2494 ± 4% slabinfo.anon_vma.active_objs 3399 ± 6% -24.3% 2572 ± 5% slabinfo.anon_vma.num_objs 1018 ± 3% -100.0% 0.00 slabinfo.cred_jar.active_objs 1018 ± 3% -100.0% 0.00 slabinfo.cred_jar.num_objs 83981 ± 2% -22.6% 65026 ± 3% slabinfo.dentry.active_objs 4003 ± 2% -41.9% 2327 ± 3% slabinfo.dentry.active_slabs 84073 ± 2% -22.5% 65193 ± 3% slabinfo.dentry.num_objs 4003 ± 2% -41.9% 2327 ± 3% slabinfo.dentry.num_slabs 176.50 ± 2% -100.0% 0.00 slabinfo.dmaengine-unmap-16.active_objs 176.50 ± 2% -100.0% 0.00 slabinfo.dmaengine-unmap-16.num_objs 272.50 ± 9% -100.0% 0.00 slabinfo.file_lock_cache.active_objs 272.50 ± 9% -100.0% 0.00 slabinfo.file_lock_cache.num_objs 39280 ± 6% -48.5% 20210 ± 10% slabinfo.filp.active_objs 1232 ± 5% -48.2% 638.50 ± 10% slabinfo.filp.active_slabs 39445 ± 5% -48.2% 20452 ± 10% slabinfo.filp.num_objs 1232 ± 5% -48.2% 638.50 ± 10% slabinfo.filp.num_slabs 140.25 -100.0% 0.00 slabinfo.fsnotify_mark_connector.active_objs 800.00 ± 6% -100.0% 0.00 slabinfo.fsnotify_mark_connector.num_objs 1548 -20.2% 1235 slabinfo.inode_cache.active_slabs 1548 -20.2% 1235 slabinfo.inode_cache.num_slabs 984.25 +34.0% 1318 slabinfo.kernfs_node_cache.active_slabs 984.25 +34.0% 1318 slabinfo.kernfs_node_cache.num_slabs 1406 -23.1% 1081 slabinfo.kmalloc-192.active_objs 1406 -23.1% 1081 slabinfo.kmalloc-192.num_objs 1839 -26.1% 1360 slabinfo.kmalloc-1k.active_objs 1839 -26.1% 1360 slabinfo.kmalloc-1k.num_objs 1393 +27.4% 1776 slabinfo.kmalloc-256.active_objs 1393 +27.4% 1776 slabinfo.kmalloc-256.num_objs 773.75 ± 2% +76.7% 1367 slabinfo.kmalloc-2k.active_objs 773.75 ± 2% +76.7% 1367 slabinfo.kmalloc-2k.num_objs 220.75 ± 4% +111.1% 466.00 slabinfo.kmalloc-4k.active_objs 222.50 ± 3% +109.4% 466.00 slabinfo.kmalloc-4k.num_objs 1607 ± 11% +33.3% 2143 slabinfo.kmalloc-512.active_objs 1640 ± 10% +30.7% 2143 slabinfo.kmalloc-512.num_objs 1055 ± 6% +54.6% 1632 slabinfo.kmalloc-rcl-64.active_objs 1055 ± 6% +54.6% 1632 slabinfo.kmalloc-rcl-64.num_objs 808.50 ± 2% -19.5% 651.00 ± 3% slabinfo.kmalloc-rcl-96.active_objs 808.50 ± 2% -19.5% 651.00 ± 3% slabinfo.kmalloc-rcl-96.num_objs 448.00 ± 10% -39.3% 272.00 ± 5% slabinfo.kmem_cache_node.active_objs 448.00 ± 10% -39.3% 272.00 ± 5% slabinfo.kmem_cache_node.num_objs 2505 ± 7% -100.0% 0.00 slabinfo.lsm_file_cache.active_objs 2505 ± 7% -100.0% 0.00 slabinfo.lsm_file_cache.num_objs 243.75 ± 8% -100.0% 0.00 slabinfo.mnt_cache.active_objs 243.75 ± 8% -100.0% 0.00 slabinfo.mnt_cache.num_objs 511.50 ± 3% -54.5% 232.50 ± 6% slabinfo.numa_policy.active_objs 511.50 ± 3% -54.5% 232.50 ± 6% slabinfo.numa_policy.num_objs 648.00 ± 4% +111.3% 1369 ± 2% slabinfo.pid.active_objs 648.00 ± 4% +111.3% 1369 ± 2% slabinfo.pid.num_objs 39796 ± 6% -48.2% 20599 ± 10% slabinfo.shmem_inode_cache.active_objs 1732 ± 6% -54.1% 794.50 ± 10% slabinfo.shmem_inode_cache.active_slabs 39842 ± 6% -48.1% 20665 ± 10% slabinfo.shmem_inode_cache.num_objs 1732 ± 6% -54.1% 794.50 ± 10% slabinfo.shmem_inode_cache.num_slabs 637.25 ± 3% -22.4% 494.50 ± 2% slabinfo.task_delay_info.active_objs 637.25 ± 3% -22.4% 494.50 ± 2% slabinfo.task_delay_info.num_objs 110.50 ± 13% +160.6% 288.00 slabinfo.task_group.active_objs 145.50 ± 11% +97.9% 288.00 slabinfo.task_group.num_objs 10.20 -10.2 0.00 perf-profile.calltrace.cycles-pp.shmem_mknod.path_openat.do_filp_open.do_sys_openat2.do_sys_open 7.41 -7.4 0.00 perf-profile.calltrace.cycles-pp.shmem_get_inode.shmem_mknod.path_openat.do_filp_open.do_sys_openat2 14.30 -4.3 9.99 perf-profile.calltrace.cycles-pp.__GI___libc_close 10.95 -2.3 8.64 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 11.10 -2.1 9.01 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_close 3.55 ± 2% -1.5 2.02 perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 2.93 -1.4 1.50 perf-profile.calltrace.cycles-pp.do_dentry_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open 3.48 ± 2% -1.4 2.12 perf-profile.calltrace.cycles-pp.link_path_walk.path_parentat.filename_parentat.do_unlinkat.do_syscall_64 5.73 -1.1 4.65 perf-profile.calltrace.cycles-pp.filename_parentat.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 1.29 ± 4% -1.0 0.26 ±100% perf-profile.calltrace.cycles-pp.strncpy_from_user.getname_flags.do_sys_openat2.do_sys_open.do_syscall_64 48.31 -0.8 47.48 perf-profile.calltrace.cycles-pp.__GI___libc_open 5.33 ± 2% -0.8 4.52 ± 2% perf-profile.calltrace.cycles-pp.path_parentat.filename_parentat.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.53 ± 4% -0.8 1.75 perf-profile.calltrace.cycles-pp.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 2.37 ± 4% -0.7 1.72 ± 2% perf-profile.calltrace.cycles-pp.getname_flags.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 2.42 ± 2% -0.6 1.77 perf-profile.calltrace.cycles-pp.getname_flags.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.40 ± 5% -0.6 0.83 ± 3% perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc.shmem_alloc_inode.alloc_inode 1.42 ± 5% -0.6 0.85 ± 2% perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc.shmem_alloc_inode.alloc_inode.new_inode_pseudo 1.29 ± 5% -0.5 0.79 ± 3% perf-profile.calltrace.cycles-pp.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc.shmem_alloc_inode 1.98 ± 3% -0.4 1.55 perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2 2.20 ± 7% -0.4 1.82 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 0.89 ± 4% -0.4 0.52 perf-profile.calltrace.cycles-pp.__d_lookup_rcu.lookup_fast.walk_component.link_path_walk.path_openat 0.62 -0.4 0.26 ±100% perf-profile.calltrace.cycles-pp.call_rcu.__dentry_kill.dput.do_unlinkat.do_syscall_64 0.90 ± 7% -0.3 0.55 ± 2% perf-profile.calltrace.cycles-pp.__d_lookup_rcu.lookup_fast.walk_component.link_path_walk.path_parentat 1.96 ± 4% -0.3 1.62 perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_parentat.filename_parentat.do_unlinkat 1.03 -0.3 0.72 perf-profile.calltrace.cycles-pp.inode_init_always.alloc_inode.new_inode_pseudo.new_inode.shmem_get_inode 3.27 ± 2% -0.3 2.96 perf-profile.calltrace.cycles-pp.alloc_inode.new_inode_pseudo.new_inode.shmem_get_inode.shmem_mknod 1.65 ± 4% -0.2 1.46 perf-profile.calltrace.cycles-pp.lookup_fast.walk_component.link_path_walk.path_openat.do_filp_open 0.80 ± 2% -0.2 0.63 perf-profile.calltrace.cycles-pp.apparmor_file_alloc_security.security_file_alloc.__alloc_file.alloc_empty_file.path_openat 1.26 -0.2 1.10 ± 2% perf-profile.calltrace.cycles-pp.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.91 ± 3% -0.1 0.78 perf-profile.calltrace.cycles-pp.shmem_free_inode.shmem_evict_inode.evict.do_unlinkat.do_syscall_64 3.58 -0.1 3.49 perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.10 -0.1 1.02 ± 3% perf-profile.calltrace.cycles-pp.__destroy_inode.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.01 ± 3% +0.1 1.06 perf-profile.calltrace.cycles-pp.ima_file_check.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.89 ± 2% +0.1 0.97 ± 5% perf-profile.calltrace.cycles-pp.complete_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 3.79 +0.1 3.88 perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.81 ± 3% +0.1 0.95 perf-profile.calltrace.cycles-pp.security_task_getsecid.ima_file_check.path_openat.do_filp_open.do_sys_openat2 0.71 ± 4% +0.2 0.86 perf-profile.calltrace.cycles-pp.may_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.54 ± 3% +0.2 0.71 ± 5% perf-profile.calltrace.cycles-pp.legitimize_path.unlazy_walk.complete_walk.path_openat.do_filp_open 3.61 ± 2% +0.2 3.79 perf-profile.calltrace.cycles-pp.__alloc_file.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2 5.79 +0.2 5.97 perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 0.57 ± 3% +0.2 0.76 perf-profile.calltrace.cycles-pp.iput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 1.24 ± 2% +0.2 1.44 perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open 0.74 ± 2% +0.2 0.94 ± 4% perf-profile.calltrace.cycles-pp.unlazy_walk.complete_walk.path_openat.do_filp_open.do_sys_openat2 0.96 ± 3% +0.2 1.16 ± 3% perf-profile.calltrace.cycles-pp.dentry_unlink_inode.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.67 ± 3% +0.2 0.90 ± 2% perf-profile.calltrace.cycles-pp.apparmor_task_getsecid.security_task_getsecid.ima_file_check.path_openat.do_filp_open 1.50 +0.3 1.77 perf-profile.calltrace.cycles-pp.security_file_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open 0.73 ± 2% +0.3 1.00 ± 2% perf-profile.calltrace.cycles-pp.complete_walk.path_parentat.filename_parentat.do_unlinkat.do_syscall_64 1.16 ± 14% +0.3 1.46 ± 5% perf-profile.calltrace.cycles-pp.__close_fd.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 1.61 +0.4 1.97 perf-profile.calltrace.cycles-pp.__dentry_kill.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.60 ± 3% +0.4 0.96 ± 2% perf-profile.calltrace.cycles-pp.unlazy_walk.complete_walk.path_parentat.filename_parentat.do_unlinkat 0.77 +0.4 1.14 perf-profile.calltrace.cycles-pp.mnt_want_write.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.82 ± 3% +0.4 1.19 perf-profile.calltrace.cycles-pp.kmem_cache_alloc.getname_flags.do_sys_openat2.do_sys_open.do_syscall_64 0.76 ± 7% +0.4 1.17 perf-profile.calltrace.cycles-pp.mnt_want_write.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.75 ± 5% +0.5 1.21 ± 2% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.getname_flags.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.39 ± 57% +0.5 0.86 ± 2% perf-profile.calltrace.cycles-pp.__follow_mount_rcu.lookup_fast.walk_component.link_path_walk.path_openat 5.16 +0.5 5.64 perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_close 0.00 +0.5 0.52 perf-profile.calltrace.cycles-pp.lock_acquire.__lock_text_start.new_inode_pseudo.new_inode.shmem_get_inode 0.00 +0.5 0.52 perf-profile.calltrace.cycles-pp.mntput_no_expire.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 0.00 +0.5 0.53 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irq.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.57 ± 4% +0.5 1.10 ± 2% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.security_file_alloc.__alloc_file.alloc_empty_file.path_openat 0.00 +0.5 0.53 ± 3% perf-profile.calltrace.cycles-pp.mntput_no_expire.terminate_walk.path_openat.do_filp_open.do_sys_openat2 0.00 +0.5 0.54 ± 4% perf-profile.calltrace.cycles-pp.__lock_text_start.d_add.simple_lookup.lookup_open.path_openat 0.00 +0.5 0.54 ± 4% perf-profile.calltrace.cycles-pp.mntput_no_expire.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.__lock_text_start.new_inode_pseudo.new_inode.shmem_get_inode.shmem_mknod 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.dput.do_unlinkat.do_syscall_64 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.lock_acquire.down_write.vfs_unlink.do_unlinkat.do_syscall_64 0.00 +0.6 0.58 ± 4% perf-profile.calltrace.cycles-pp.do_raw_spin_lock.__alloc_fd.do_sys_openat2.do_sys_open.do_syscall_64 0.00 +0.6 0.59 perf-profile.calltrace.cycles-pp.xas_start.xas_load.xas_find.find_get_entries.shmem_undo_range 0.00 +0.6 0.59 ± 5% perf-profile.calltrace.cycles-pp.__d_instantiate.d_instantiate.shmem_mknod.lookup_open.path_openat 0.00 +0.6 0.59 ± 11% perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.path_openat.do_filp_open.do_sys_openat2 0.00 +0.6 0.60 perf-profile.calltrace.cycles-pp.lock_acquire.find_get_entries.shmem_undo_range.shmem_truncate_range.shmem_evict_inode 0.00 +0.6 0.61 ± 4% perf-profile.calltrace.cycles-pp.lockref_get_not_dead.legitimize_path.unlazy_walk.complete_walk.path_openat 0.00 +0.6 0.61 perf-profile.calltrace.cycles-pp.lock_release.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.6 0.61 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.__wake_up_common_lock.__d_lookup_done.d_add.simple_lookup 0.00 +0.6 0.61 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire._raw_spin_lock_irq.task_work_run.exit_to_usermode_loop.do_syscall_64 0.00 +0.6 0.61 perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.__lock_text_start.d_delete.vfs_unlink 0.00 +0.6 0.62 ± 3% perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.__fput.task_work_run.exit_to_usermode_loop 0.00 +0.6 0.64 ± 3% perf-profile.calltrace.cycles-pp.lockref_get_not_dead.legitimize_path.unlazy_walk.complete_walk.path_parentat 0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.shmem_unlink.vfs_unlink.do_unlinkat 0.00 +0.6 0.64 ± 9% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 0.00 +0.6 0.64 ± 9% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread 0.00 +0.6 0.64 ± 9% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn 0.00 +0.6 0.64 ± 9% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.lock_acquire.d_alloc_parallel.lookup_open.path_openat.do_filp_open 0.55 ± 4% +0.6 1.19 ± 3% perf-profile.calltrace.cycles-pp.path_init.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +0.6 0.65 perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.terminate_walk.path_openat.do_filp_open 0.00 +0.6 0.65 ± 9% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork 0.00 +0.7 0.66 perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.__lock_text_start.evict.do_unlinkat 0.00 +0.7 0.66 perf-profile.calltrace.cycles-pp.xas_load.xas_find.find_get_entries.shmem_undo_range.shmem_truncate_range 0.60 ± 5% +0.7 1.27 perf-profile.calltrace.cycles-pp.path_init.path_parentat.filename_parentat.do_unlinkat.do_syscall_64 0.00 +0.7 0.68 ± 8% perf-profile.calltrace.cycles-pp.ret_from_fork 0.00 +0.7 0.68 ± 8% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.00 +0.7 0.68 ± 2% perf-profile.calltrace.cycles-pp.fsnotify_grab_connector.fsnotify_destroy_marks.__destroy_inode.destroy_inode.do_unlinkat 0.00 +0.7 0.69 ± 2% perf-profile.calltrace.cycles-pp.lock_is_held_type.___might_sleep.dput.do_unlinkat.do_syscall_64 0.00 +0.7 0.70 perf-profile.calltrace.cycles-pp.xas_find.find_get_entries.shmem_undo_range.shmem_truncate_range.shmem_evict_inode 0.00 +0.7 0.71 ± 3% perf-profile.calltrace.cycles-pp.fsnotify_destroy_marks.__destroy_inode.destroy_inode.do_unlinkat.do_syscall_64 0.00 +0.7 0.72 ± 4% perf-profile.calltrace.cycles-pp.fsnotify_grab_connector.fsnotify_destroy_marks.dentry_unlink_inode.vfs_unlink.do_unlinkat 0.00 +0.7 0.73 ± 4% perf-profile.calltrace.cycles-pp.legitimize_path.unlazy_walk.complete_walk.path_parentat.filename_parentat 0.65 ± 10% +0.7 1.39 perf-profile.calltrace.cycles-pp.__d_lookup.d_lookup.lookup_dcache.__lookup_hash.do_unlinkat 0.00 +0.7 0.74 ± 4% perf-profile.calltrace.cycles-pp.__d_lookup.d_lookup.lookup_open.path_openat.do_filp_open 0.00 +0.7 0.74 ± 6% perf-profile.calltrace.cycles-pp.set_root.nd_jump_root.path_init.path_openat.do_filp_open 0.00 +0.7 0.74 perf-profile.calltrace.cycles-pp.__fd_install.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.8 0.75 perf-profile.calltrace.cycles-pp.inode_permission.may_open.path_openat.do_filp_open.do_sys_openat2 0.00 +0.8 0.75 ± 4% perf-profile.calltrace.cycles-pp.fsnotify_destroy_marks.dentry_unlink_inode.vfs_unlink.do_unlinkat.do_syscall_64 0.00 +0.8 0.77 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.8 0.77 ± 3% perf-profile.calltrace.cycles-pp.set_root.nd_jump_root.path_init.path_parentat.filename_parentat 0.00 +0.8 0.78 ± 3% perf-profile.calltrace.cycles-pp.ktime_get_coarse_real_ts64.current_time.shmem_truncate_range.shmem_evict_inode.evict 0.00 +0.8 0.79 perf-profile.calltrace.cycles-pp.down_write.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.nd_jump_root.path_init.path_openat.do_filp_open.do_sys_openat2 0.00 +0.8 0.80 perf-profile.calltrace.cycles-pp.inode_sb_list_add.new_inode.shmem_get_inode.shmem_mknod.lookup_open 0.00 +0.8 0.80 ± 2% perf-profile.calltrace.cycles-pp.aa_get_task_label.apparmor_task_getsecid.security_task_getsecid.ima_file_check.path_openat 0.00 +0.8 0.82 ± 2% perf-profile.calltrace.cycles-pp.nd_jump_root.path_init.path_parentat.filename_parentat.do_unlinkat 0.00 +0.8 0.83 perf-profile.calltrace.cycles-pp.lock_acquire.__lock_text_start.d_delete.vfs_unlink.do_unlinkat 0.00 +0.8 0.83 perf-profile.calltrace.cycles-pp.down_write_nested.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.00 +0.8 0.83 ± 2% perf-profile.calltrace.cycles-pp.current_time.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat 0.00 +0.8 0.84 ± 2% perf-profile.calltrace.cycles-pp.clear_inode.shmem_evict_inode.evict.do_unlinkat.do_syscall_64 0.00 +0.8 0.84 ± 2% perf-profile.calltrace.cycles-pp.ktime_get_coarse_real_ts64.current_time.shmem_mknod.lookup_open.path_openat 0.00 +0.8 0.84 perf-profile.calltrace.cycles-pp.get_cached_acl.get_acl.posix_acl_create.simple_acl_create.shmem_mknod 44.66 +0.9 45.53 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +0.9 0.88 perf-profile.calltrace.cycles-pp.lock_acquire.__lock_text_start.evict.do_unlinkat.do_syscall_64 0.00 +0.9 0.88 perf-profile.calltrace.cycles-pp.ktime_get_coarse_real_ts64.current_time.shmem_get_inode.shmem_mknod.lookup_open 0.00 +0.9 0.88 perf-profile.calltrace.cycles-pp.__lock_text_start.d_delete.vfs_unlink.do_unlinkat.do_syscall_64 0.95 ± 3% +0.9 1.83 perf-profile.calltrace.cycles-pp.find_get_entries.shmem_undo_range.shmem_truncate_range.shmem_evict_inode.evict 0.00 +0.9 0.89 ± 2% perf-profile.calltrace.cycles-pp.__follow_mount_rcu.lookup_fast.walk_component.link_path_walk.path_parentat 0.00 +0.9 0.90 ± 2% perf-profile.calltrace.cycles-pp.current_time.shmem_mknod.lookup_open.path_openat.do_filp_open 0.00 +0.9 0.90 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.91 ± 3% perf-profile.calltrace.cycles-pp.__lock_acquire.lock_acquire.__lock_text_start.lockref_put_or_lock.dput 0.00 +0.9 0.91 perf-profile.calltrace.cycles-pp.get_acl.posix_acl_create.simple_acl_create.shmem_mknod.lookup_open 0.00 +0.9 0.91 ± 2% perf-profile.calltrace.cycles-pp.shmem_reserve_inode.shmem_get_inode.shmem_mknod.lookup_open.path_openat 0.00 +0.9 0.93 ± 2% perf-profile.calltrace.cycles-pp.__lock_text_start.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.93 perf-profile.calltrace.cycles-pp.current_time.shmem_get_inode.shmem_mknod.lookup_open.path_openat 0.00 +0.9 0.93 ± 2% perf-profile.calltrace.cycles-pp.___might_sleep.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.95 ± 2% perf-profile.calltrace.cycles-pp.ktime_get_coarse_real_ts64.current_time.shmem_unlink.vfs_unlink.do_unlinkat 0.00 +1.0 0.97 perf-profile.calltrace.cycles-pp.d_delete.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.13 ±173% +1.0 1.10 ± 2% perf-profile.calltrace.cycles-pp.__sb_start_write.mnt_want_write.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.0 0.98 ± 2% perf-profile.calltrace.cycles-pp.down_write.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.0 0.99 ± 2% perf-profile.calltrace.cycles-pp.current_time.shmem_unlink.vfs_unlink.do_unlinkat.do_syscall_64 0.00 +1.0 1.02 ± 2% perf-profile.calltrace.cycles-pp.posix_acl_create.simple_acl_create.shmem_mknod.lookup_open.path_openat 0.00 +1.1 1.06 ± 2% perf-profile.calltrace.cycles-pp.__sb_start_write.mnt_want_write.path_openat.do_filp_open.do_sys_openat2 44.83 +1.1 45.94 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.lock_acquire.__lock_text_start.lockref_put_or_lock.dput.do_unlinkat 0.00 +1.2 1.21 ± 2% perf-profile.calltrace.cycles-pp.simple_acl_create.shmem_mknod.lookup_open.path_openat.do_filp_open 0.00 +1.2 1.23 ± 2% perf-profile.calltrace.cycles-pp.__lock_text_start.lockref_put_or_lock.dput.do_unlinkat.do_syscall_64 0.13 ±173% +1.2 1.37 ± 4% perf-profile.calltrace.cycles-pp.dput.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +1.3 1.28 ± 3% perf-profile.calltrace.cycles-pp.dput.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 0.00 +1.3 1.29 perf-profile.calltrace.cycles-pp.d_instantiate.shmem_mknod.lookup_open.path_openat.do_filp_open 1.93 ± 2% +1.3 3.22 perf-profile.calltrace.cycles-pp.shmem_undo_range.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat 0.00 +1.3 1.33 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.__d_lookup_done.d_add.simple_lookup.lookup_open 0.00 +1.4 1.35 perf-profile.calltrace.cycles-pp.dput.terminate_walk.path_openat.do_filp_open.do_sys_openat2 0.65 +1.4 2.04 perf-profile.calltrace.cycles-pp.terminate_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 1.36 +1.4 2.75 perf-profile.calltrace.cycles-pp.shmem_unlink.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.4 1.41 perf-profile.calltrace.cycles-pp.dput.shmem_unlink.vfs_unlink.do_unlinkat.do_syscall_64 0.00 +1.4 1.42 perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__d_alloc.d_alloc.d_alloc_parallel.lookup_open 0.00 +1.4 1.44 perf-profile.calltrace.cycles-pp.__d_lookup_done.d_add.simple_lookup.lookup_open.path_openat 0.88 ± 6% +1.5 2.37 perf-profile.calltrace.cycles-pp.__lookup_hash.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.79 ± 7% +1.6 2.35 perf-profile.calltrace.cycles-pp.lookup_dcache.__lookup_hash.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.74 ± 8% +1.6 2.33 perf-profile.calltrace.cycles-pp.d_lookup.lookup_dcache.__lookup_hash.do_unlinkat.do_syscall_64 0.00 +1.6 1.59 perf-profile.calltrace.cycles-pp.d_lookup.lookup_open.path_openat.do_filp_open.do_sys_openat2 0.00 +1.6 1.65 perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.8 1.75 perf-profile.calltrace.cycles-pp.__d_alloc.d_alloc.d_alloc_parallel.lookup_open.path_openat 2.31 +1.8 4.10 perf-profile.calltrace.cycles-pp.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat.do_syscall_64 6.51 +2.0 8.48 perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 3.96 +2.1 6.03 perf-profile.calltrace.cycles-pp.shmem_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.3 2.29 perf-profile.calltrace.cycles-pp.d_add.simple_lookup.lookup_open.path_openat.do_filp_open 4.78 ± 2% +2.4 7.17 perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.00 +2.4 2.42 perf-profile.calltrace.cycles-pp.simple_lookup.lookup_open.path_openat.do_filp_open.do_sys_openat2 0.00 +2.5 2.48 perf-profile.calltrace.cycles-pp.d_alloc.d_alloc_parallel.lookup_open.path_openat.do_filp_open 41.72 +2.8 44.54 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 41.52 +2.9 44.45 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +3.7 3.73 perf-profile.calltrace.cycles-pp.new_inode_pseudo.new_inode.shmem_get_inode.shmem_mknod.lookup_open 3.44 +3.8 7.21 perf-profile.calltrace.cycles-pp.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.00 +4.2 4.22 perf-profile.calltrace.cycles-pp.d_alloc_parallel.lookup_open.path_openat.do_filp_open.do_sys_openat2 35.02 +4.2 39.27 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 34.56 +4.6 39.12 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 0.00 +4.7 4.73 perf-profile.calltrace.cycles-pp.new_inode.shmem_get_inode.shmem_mknod.lookup_open.path_openat 36.35 +4.8 41.16 perf-profile.calltrace.cycles-pp.unlink 32.71 +6.6 39.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 32.87 +6.8 39.69 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink 0.00 +7.4 7.37 perf-profile.calltrace.cycles-pp.shmem_get_inode.shmem_mknod.lookup_open.path_openat.do_filp_open 27.33 +9.3 36.63 perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 0.00 +11.7 11.72 perf-profile.calltrace.cycles-pp.shmem_mknod.lookup_open.path_openat.do_filp_open.do_sys_openat2 0.00 +21.2 21.16 perf-profile.calltrace.cycles-pp.lookup_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open 8.03 ± 4% -8.0 0.00 perf-profile.children.cycles-pp._raw_spin_lock 14.48 -4.4 10.04 perf-profile.children.cycles-pp.__GI___libc_close 4.45 ± 3% -3.4 1.00 ± 5% perf-profile.children.cycles-pp.syscall_return_via_sysret 4.11 ± 3% -3.0 1.10 perf-profile.children.cycles-pp.entry_SYSCALL_64 7.11 -2.9 4.18 perf-profile.children.cycles-pp.link_path_walk 2.62 ± 3% -1.7 0.96 perf-profile.children.cycles-pp.strncpy_from_user 3.10 ± 8% -1.6 1.54 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 2.96 -1.5 1.50 perf-profile.children.cycles-pp.do_dentry_open 4.80 ± 3% -1.3 3.52 perf-profile.children.cycles-pp.getname_flags 5.33 ± 2% -1.2 4.12 perf-profile.children.cycles-pp.irq_exit 1.88 ± 3% -1.2 0.70 ± 2% perf-profile.children.cycles-pp.__check_object_size 2.29 ± 4% -1.1 1.19 perf-profile.children.cycles-pp.__slab_alloc 1.53 ± 4% -1.1 0.43 perf-profile.children.cycles-pp.fsnotify 5.75 -1.1 4.66 perf-profile.children.cycles-pp.filename_parentat 2.23 ± 5% -1.1 1.15 perf-profile.children.cycles-pp.___slab_alloc 1.38 ± 2% -1.0 0.33 ± 7% perf-profile.children.cycles-pp._cond_resched 1.69 ± 4% -1.0 0.69 perf-profile.children.cycles-pp.rcu_cblist_dequeue 48.50 -1.0 47.52 perf-profile.children.cycles-pp.__GI___libc_open 1.96 ± 4% -0.9 1.06 ± 2% perf-profile.children.cycles-pp.new_slab 1.22 -0.9 0.34 ± 5% perf-profile.children.cycles-pp.security_file_open 5.37 ± 2% -0.8 4.53 ± 2% perf-profile.children.cycles-pp.path_parentat 3.99 ± 3% -0.8 3.20 perf-profile.children.cycles-pp.walk_component 1.20 ± 4% -0.8 0.42 ± 9% perf-profile.children.cycles-pp.__might_sleep 2.54 ± 4% -0.8 1.76 ± 2% perf-profile.children.cycles-pp.__x64_sys_unlink 2.12 ± 4% -0.8 1.36 perf-profile.children.cycles-pp.inode_permission 5.30 ± 2% -0.7 4.56 perf-profile.children.cycles-pp.rcu_do_batch 1.01 -0.7 0.28 ± 5% perf-profile.children.cycles-pp.apparmor_file_open 0.97 ± 6% -0.7 0.25 ± 4% perf-profile.children.cycles-pp.__slab_free 5.32 ± 2% -0.7 4.60 perf-profile.children.cycles-pp.rcu_core 1.12 ± 4% -0.7 0.43 ± 5% perf-profile.children.cycles-pp.generic_permission 0.89 ± 6% -0.7 0.21 ± 2% perf-profile.children.cycles-pp.inode_init_once 0.93 ± 3% -0.7 0.27 perf-profile.children.cycles-pp.security_file_free 5.64 -0.6 5.01 perf-profile.children.cycles-pp.smp_apic_timer_interrupt 5.67 -0.6 5.04 perf-profile.children.cycles-pp.apic_timer_interrupt 5.36 ± 2% -0.6 4.74 perf-profile.children.cycles-pp.__softirqentry_text_start 1.89 ± 5% -0.6 1.27 ± 2% perf-profile.children.cycles-pp.__d_lookup_rcu 1.84 ± 2% -0.6 1.23 perf-profile.children.cycles-pp.call_rcu 0.73 ± 9% -0.6 0.13 ± 7% perf-profile.children.cycles-pp.rcu_all_qs 0.81 ± 2% -0.6 0.23 ± 2% perf-profile.children.cycles-pp.apparmor_file_free_security 1.36 ± 2% -0.6 0.80 perf-profile.children.cycles-pp.inode_sb_list_add 0.72 ± 3% -0.5 0.19 perf-profile.children.cycles-pp.__mnt_want_write 0.70 ± 4% -0.5 0.17 ± 3% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.81 ± 2% -0.5 0.28 ± 3% perf-profile.children.cycles-pp.filp_close 0.72 -0.5 0.21 ± 7% perf-profile.children.cycles-pp.security_path_unlink 0.68 ± 4% -0.5 0.18 ± 5% perf-profile.children.cycles-pp.rcu_segcblist_enqueue 0.77 ± 4% -0.4 0.34 perf-profile.children.cycles-pp.file_free_rcu 0.71 ± 8% -0.4 0.29 perf-profile.children.cycles-pp.__virt_addr_valid 0.67 ± 2% -0.4 0.25 ± 4% perf-profile.children.cycles-pp.security_path_mknod 2.21 ± 7% -0.4 1.83 ± 5% perf-profile.children.cycles-pp.__x64_sys_close 0.50 ± 3% -0.4 0.14 ± 11% perf-profile.children.cycles-pp.__legitimize_mnt 0.44 ± 4% -0.3 0.10 ± 10% perf-profile.children.cycles-pp.__list_del_entry_valid 3.36 ± 4% -0.3 3.02 perf-profile.children.cycles-pp.lookup_fast 0.49 ± 4% -0.3 0.15 ± 6% perf-profile.children.cycles-pp.common_perm_rm 0.47 ± 3% -0.3 0.14 ± 7% perf-profile.children.cycles-pp.fput_many 3.28 ± 2% -0.3 2.97 perf-profile.children.cycles-pp.alloc_inode 1.05 -0.3 0.74 perf-profile.children.cycles-pp.inode_init_always 0.47 ± 4% -0.3 0.17 ± 3% perf-profile.children.cycles-pp.common_perm 0.49 -0.3 0.20 ± 7% perf-profile.children.cycles-pp.may_delete 0.39 ± 11% -0.3 0.10 ± 5% perf-profile.children.cycles-pp.timestamp_truncate 0.44 ± 5% -0.3 0.16 ± 9% perf-profile.children.cycles-pp.common_perm_create 0.41 ± 3% -0.3 0.13 perf-profile.children.cycles-pp.drop_nlink 0.52 ± 7% -0.3 0.24 ± 4% perf-profile.children.cycles-pp.shmem_free_in_core_inode 0.41 ± 12% -0.3 0.14 ± 3% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.43 ± 4% -0.3 0.16 perf-profile.children.cycles-pp.memset_erms 0.38 ± 2% -0.3 0.11 ± 4% perf-profile.children.cycles-pp.__pthread_enable_asynccancel 0.33 ± 3% -0.3 0.08 ± 12% perf-profile.children.cycles-pp.__list_add_valid 0.34 ± 5% -0.2 0.10 ± 15% perf-profile.children.cycles-pp.__fsnotify_parent 0.43 ± 6% -0.2 0.18 ± 8% perf-profile.children.cycles-pp.__srcu_read_lock 0.38 ± 3% -0.2 0.14 ± 7% perf-profile.children.cycles-pp.__check_heap_object 0.34 ± 5% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.__d_drop 0.31 ± 3% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.___d_drop 0.33 ± 11% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.security_inode_permission 0.29 ± 3% -0.2 0.08 ± 6% perf-profile.children.cycles-pp.task_work_add 0.30 ± 8% -0.2 0.09 perf-profile.children.cycles-pp.security_inode_init_security 0.32 ± 3% -0.2 0.11 ± 9% perf-profile.children.cycles-pp.set_cached_acl 0.27 ± 2% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.__d_rehash 4.93 ± 2% -0.2 4.74 perf-profile.children.cycles-pp.new_inode 0.24 ± 7% -0.2 0.06 ± 9% perf-profile.children.cycles-pp.__srcu_read_unlock 0.82 ± 2% -0.2 0.63 perf-profile.children.cycles-pp.apparmor_file_alloc_security 0.28 ± 13% -0.2 0.10 ± 5% perf-profile.children.cycles-pp.map_id_range_down 0.29 ± 2% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.__pthread_disable_asynccancel 0.26 ± 3% -0.2 0.09 perf-profile.children.cycles-pp.d_set_d_op 0.24 ± 12% -0.2 0.07 ± 14% perf-profile.children.cycles-pp.prandom_u32 0.44 ± 5% -0.2 0.28 perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.39 ± 6% -0.2 0.23 ± 4% perf-profile.children.cycles-pp.i_callback 1.27 -0.2 1.10 ± 3% perf-profile.children.cycles-pp.destroy_inode 0.18 ± 6% -0.2 0.03 ±100% perf-profile.children.cycles-pp.always_delete_dentry 0.23 ± 8% -0.2 0.07 ± 14% perf-profile.children.cycles-pp.memcpy_erms 0.23 ± 4% -0.2 0.07 ± 28% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.18 -0.2 0.03 ±100% perf-profile.children.cycles-pp.shmem_create 0.23 -0.1 0.09 ± 17% perf-profile.children.cycles-pp.ihold 0.37 ± 6% -0.1 0.23 ± 6% perf-profile.children.cycles-pp.get_page_from_freelist 0.19 ± 6% -0.1 0.06 perf-profile.children.cycles-pp.map_id_up 0.92 ± 2% -0.1 0.79 perf-profile.children.cycles-pp.shmem_free_inode 0.33 ± 6% -0.1 0.20 ± 10% perf-profile.children.cycles-pp.__lookup_mnt 3.66 -0.1 3.53 perf-profile.children.cycles-pp.__fput 0.15 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.path_get 0.15 ± 11% -0.1 0.03 ±100% perf-profile.children.cycles-pp.current_umask 0.17 ± 8% -0.1 0.05 perf-profile.children.cycles-pp.__x64_sys_open 0.18 ± 12% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.make_kuid 0.17 ± 4% -0.1 0.06 perf-profile.children.cycles-pp.file_ra_state_init 0.27 ± 4% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.put_cpu_partial 0.13 ± 5% -0.1 0.03 ±100% perf-profile.children.cycles-pp.apparmor_path_mknod 0.17 ± 9% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.mnt_drop_write 0.15 ± 11% -0.1 0.05 perf-profile.children.cycles-pp.make_kgid 0.18 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.restore_nameidata 0.12 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.prandom_u32_state 0.25 ± 4% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.unfreeze_partials 0.15 ± 12% -0.1 0.06 ± 16% perf-profile.children.cycles-pp.should_failslab 1.13 -0.1 1.04 ± 3% perf-profile.children.cycles-pp.__destroy_inode 0.17 ± 12% -0.1 0.09 ± 22% perf-profile.children.cycles-pp.putname 0.11 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.process_measurement 0.10 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__wake_up_bit 0.12 ± 10% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.security_d_instantiate 0.15 ± 12% -0.1 0.09 perf-profile.children.cycles-pp._find_next_bit 0.12 ± 13% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.d_flags_for_inode 0.14 ± 7% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.security_inode_free 0.11 ± 11% -0.1 0.06 perf-profile.children.cycles-pp.get_unused_fd_flags 0.11 ± 7% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.legitimize_links 0.12 ± 4% -0.0 0.08 perf-profile.children.cycles-pp.inode_init_owner 0.15 ± 8% -0.0 0.12 perf-profile.children.cycles-pp.__free_pages_ok 0.09 ± 4% -0.0 0.06 perf-profile.children.cycles-pp.rcu_segcblist_pend_cbs 0.11 ± 14% -0.0 0.08 perf-profile.children.cycles-pp.free_one_page 0.16 ± 7% +0.0 0.18 ± 5% perf-profile.children.cycles-pp.expand_files 0.00 +0.1 0.05 perf-profile.children.cycles-pp.__fsnotify_inode_delete 0.03 ±100% +0.1 0.08 ± 5% perf-profile.children.cycles-pp.update_load_avg 0.00 +0.1 0.06 perf-profile.children.cycles-pp.account_system_index_time 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.execve 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.__x64_sys_execve 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.__do_execve_file 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.ksys_write 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.vfs_write 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.new_sync_write 1.02 ± 4% +0.1 1.08 perf-profile.children.cycles-pp.ima_file_check 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.lockdep_sys_exit 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.clockevents_program_event 0.00 +0.1 0.07 ± 28% perf-profile.children.cycles-pp.__handle_mm_fault 0.01 ±173% +0.1 0.10 ± 26% perf-profile.children.cycles-pp.page_fault 0.00 +0.1 0.08 ± 29% perf-profile.children.cycles-pp.handle_mm_fault 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.run_timer_softirq 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.1 0.09 ± 22% perf-profile.children.cycles-pp.do_page_fault 0.50 ± 3% +0.1 0.60 ± 5% perf-profile.children.cycles-pp.__d_instantiate 0.00 +0.1 0.10 perf-profile.children.cycles-pp.ktime_get 0.00 +0.1 0.11 ± 4% perf-profile.children.cycles-pp.fs_reclaim_release 0.00 +0.1 0.11 ± 9% perf-profile.children.cycles-pp.static_obj 0.00 +0.1 0.11 ± 4% perf-profile.children.cycles-pp.debug_mutex_init 0.00 +0.1 0.12 ± 13% perf-profile.children.cycles-pp.update_curr 0.45 ± 10% +0.1 0.56 perf-profile.children.cycles-pp._atomic_dec_and_lock 0.09 ± 20% +0.1 0.21 ± 12% perf-profile.children.cycles-pp.task_tick_fair 0.07 ± 11% +0.1 0.20 perf-profile.children.cycles-pp.__wake_up_common 0.74 ± 4% +0.1 0.87 perf-profile.children.cycles-pp.may_open 0.00 +0.1 0.14 ± 3% perf-profile.children.cycles-pp.__mutex_init 0.82 ± 3% +0.1 0.95 perf-profile.children.cycles-pp.security_task_getsecid 0.11 ± 16% +0.1 0.26 ± 5% perf-profile.children.cycles-pp.scheduler_tick 0.00 +0.2 0.17 ± 3% perf-profile.children.cycles-pp.trace_hardirqs_on_caller 5.83 +0.2 6.00 perf-profile.children.cycles-pp.exit_to_usermode_loop 0.00 +0.2 0.17 ± 5% perf-profile.children.cycles-pp.__put_unused_fd 0.55 ± 4% +0.2 0.73 perf-profile.children.cycles-pp.xas_find 0.98 ± 2% +0.2 1.17 ± 3% perf-profile.children.cycles-pp.dentry_unlink_inode 3.62 +0.2 3.81 perf-profile.children.cycles-pp.__alloc_file 0.00 +0.2 0.18 ± 2% perf-profile.children.cycles-pp.__init_waitqueue_head 0.00 +0.2 0.19 ± 5% perf-profile.children.cycles-pp.__rwlock_init 3.54 ± 2% +0.2 3.75 perf-profile.children.cycles-pp.new_inode_pseudo 0.00 +0.2 0.21 ± 2% perf-profile.children.cycles-pp.trace_hardirqs_off_caller 0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.__need_fs_reclaim 0.67 ± 3% +0.2 0.91 perf-profile.children.cycles-pp.apparmor_task_getsecid 0.15 ± 18% +0.2 0.39 perf-profile.children.cycles-pp.update_process_times 0.15 ± 16% +0.2 0.40 perf-profile.children.cycles-pp.tick_sched_handle 1.51 +0.3 1.77 perf-profile.children.cycles-pp.security_file_alloc 0.94 ± 3% +0.3 1.21 ± 2% perf-profile.children.cycles-pp.simple_acl_create 0.18 ± 16% +0.3 0.45 ± 3% perf-profile.children.cycles-pp.tick_sched_timer 1.17 ± 14% +0.3 1.46 ± 5% perf-profile.children.cycles-pp.__close_fd 0.00 +0.3 0.29 perf-profile.children.cycles-pp.alloc_slab_page 0.36 ± 8% +0.3 0.66 perf-profile.children.cycles-pp.xas_load 0.19 ± 9% +0.3 0.50 perf-profile.children.cycles-pp._raw_spin_trylock 0.10 ± 10% +0.3 0.41 ± 2% perf-profile.children.cycles-pp.__init_rwsem 3.07 ± 4% +0.3 3.40 perf-profile.children.cycles-pp.kmem_cache_free 1.63 ± 2% +0.4 1.98 ± 3% perf-profile.children.cycles-pp.complete_walk 1.64 +0.4 2.00 perf-profile.children.cycles-pp.__dentry_kill 0.40 ± 9% +0.4 0.76 perf-profile.children.cycles-pp.__fd_install 0.24 ± 7% +0.4 0.60 perf-profile.children.cycles-pp.xas_start 2.11 ± 2% +0.4 2.49 perf-profile.children.cycles-pp.d_alloc 0.00 +0.4 0.40 perf-profile.children.cycles-pp.__address_space_init_once 0.00 +0.4 0.41 perf-profile.children.cycles-pp.reacquire_held_locks 0.21 ± 5% +0.4 0.63 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +0.4 0.42 ± 3% perf-profile.children.cycles-pp.trace_hardirqs_on_thunk 0.00 +0.4 0.42 perf-profile.children.cycles-pp.register_lock_class 0.00 +0.4 0.43 perf-profile.children.cycles-pp.tracer_hardirqs_on 0.26 ± 11% +0.4 0.69 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.87 ± 2% +0.4 1.31 perf-profile.children.cycles-pp.d_instantiate 0.22 ± 6% +0.5 0.69 perf-profile.children.cycles-pp.__sb_end_write 0.99 ± 3% +0.5 1.46 ± 5% perf-profile.children.cycles-pp.legitimize_path 0.18 ± 2% +0.5 0.65 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 5.18 +0.5 5.66 perf-profile.children.cycles-pp.task_work_run 0.35 ± 5% +0.5 0.84 ± 2% perf-profile.children.cycles-pp.clear_inode 0.00 +0.5 0.49 ± 8% perf-profile.children.cycles-pp.trace_hardirqs_off_thunk 0.33 ± 8% +0.5 0.82 ± 2% perf-profile.children.cycles-pp.aa_get_task_label 1.92 ± 3% +0.5 2.42 perf-profile.children.cycles-pp.simple_lookup 0.00 +0.5 0.51 ± 6% perf-profile.children.cycles-pp.do_raw_spin_unlock 0.49 ± 8% +0.5 1.03 ± 2% perf-profile.children.cycles-pp.posix_acl_create 1.38 ± 2% +0.6 1.94 ± 3% perf-profile.children.cycles-pp.unlazy_walk 0.31 ± 13% +0.6 0.87 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt 0.00 +0.6 0.58 ± 3% perf-profile.children.cycles-pp.lockdep_init_map 0.38 ± 2% +0.6 0.97 perf-profile.children.cycles-pp.d_delete 0.00 +0.6 0.59 ± 2% perf-profile.children.cycles-pp.tracer_hardirqs_off 0.87 +0.6 1.47 perf-profile.children.cycles-pp.fsnotify_destroy_marks 0.04 ± 58% +0.6 0.65 ± 9% perf-profile.children.cycles-pp.smpboot_thread_fn 0.03 ±100% +0.6 0.64 ± 9% perf-profile.children.cycles-pp.run_ksoftirqd 0.06 ± 13% +0.6 0.68 ± 8% perf-profile.children.cycles-pp.kthread 0.07 ± 17% +0.6 0.69 ± 9% perf-profile.children.cycles-pp.ret_from_fork 0.34 ± 8% +0.6 0.99 ± 6% perf-profile.children.cycles-pp.lockref_get 0.76 ± 2% +0.7 1.42 perf-profile.children.cycles-pp.fsnotify_grab_connector 0.76 ± 5% +0.7 1.44 perf-profile.children.cycles-pp.__d_lookup_done 0.18 ± 14% +0.7 0.92 perf-profile.children.cycles-pp.get_acl 1.58 ± 3% +0.7 2.33 perf-profile.children.cycles-pp.mnt_want_write 0.00 +0.8 0.78 ± 7% perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online 0.08 ± 16% +0.8 0.87 perf-profile.children.cycles-pp.get_cached_acl 0.98 ± 4% +0.8 1.78 ± 2% perf-profile.children.cycles-pp.__follow_mount_rcu 0.82 ± 3% +0.8 1.63 ± 3% perf-profile.children.cycles-pp.nd_jump_root 1.49 ± 3% +0.8 2.31 perf-profile.children.cycles-pp.d_add 0.00 +0.8 0.82 perf-profile.children.cycles-pp.trace_hardirqs_on 0.00 +0.8 0.83 perf-profile.children.cycles-pp.down_write_nested 0.49 +0.9 1.36 perf-profile.children.cycles-pp.__wake_up_common_lock 0.90 ± 2% +0.9 1.77 perf-profile.children.cycles-pp.down_write 0.97 ± 3% +0.9 1.86 perf-profile.children.cycles-pp.find_get_entries 0.00 +0.9 0.92 perf-profile.children.cycles-pp.trace_hardirqs_off 0.29 ± 7% +1.0 1.25 ± 5% perf-profile.children.cycles-pp.lockref_get_not_dead 0.00 +1.0 0.97 ± 2% perf-profile.children.cycles-pp.lockdep_hardirqs_off 0.56 ± 4% +1.0 1.54 ± 4% perf-profile.children.cycles-pp.set_root 0.77 ± 2% +1.0 1.75 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.00 +1.0 1.00 perf-profile.children.cycles-pp.rcu_is_watching 1.02 +1.1 2.12 perf-profile.children.cycles-pp.terminate_walk 0.00 +1.1 1.15 perf-profile.children.cycles-pp.rcu_read_lock_any_held 0.37 ± 7% +1.2 1.60 ± 2% perf-profile.children.cycles-pp.mntput_no_expire 1.98 ± 2% +1.3 3.24 perf-profile.children.cycles-pp.shmem_undo_range 0.92 ± 8% +1.3 2.19 ± 2% perf-profile.children.cycles-pp.__sb_start_write 1.17 ± 3% +1.3 2.47 ± 2% perf-profile.children.cycles-pp.path_init 0.00 +1.3 1.31 perf-profile.children.cycles-pp.match_held_lock 0.00 +1.3 1.31 perf-profile.children.cycles-pp.__raw_spin_lock_init 0.85 ± 8% +1.3 2.17 perf-profile.children.cycles-pp.__d_lookup 0.00 +1.3 1.33 ± 3% perf-profile.children.cycles-pp._raw_spin_unlock_irq 1.38 +1.4 2.75 perf-profile.children.cycles-pp.shmem_unlink 0.00 +1.4 1.40 perf-profile.children.cycles-pp.debug_check_no_locks_freed 0.89 ± 7% +1.5 2.37 perf-profile.children.cycles-pp.__lookup_hash 10.25 +1.5 11.73 perf-profile.children.cycles-pp.shmem_mknod 0.80 ± 7% +1.6 2.35 perf-profile.children.cycles-pp.lookup_dcache 2.70 +1.6 4.26 perf-profile.children.cycles-pp.d_alloc_parallel 0.00 +1.8 1.76 perf-profile.children.cycles-pp.rcu_read_lock_held 0.00 +1.8 1.77 ± 2% perf-profile.children.cycles-pp.rcu_read_lock_held_common 2.31 +1.8 4.11 perf-profile.children.cycles-pp.shmem_truncate_range 6.65 ± 2% +1.9 8.58 perf-profile.children.cycles-pp.kmem_cache_alloc 6.54 +2.0 8.50 perf-profile.children.cycles-pp.evict 0.00 +2.0 2.04 perf-profile.children.cycles-pp.rcu_read_lock_sched_held 3.98 +2.0 6.03 perf-profile.children.cycles-pp.shmem_evict_inode 0.00 +2.2 2.17 perf-profile.children.cycles-pp.find_held_lock 4.83 ± 2% +2.4 7.19 perf-profile.children.cycles-pp.vfs_unlink 1.15 ± 3% +2.5 3.67 perf-profile.children.cycles-pp.current_time 0.00 +2.7 2.70 perf-profile.children.cycles-pp.mark_held_locks 0.00 +2.8 2.75 perf-profile.children.cycles-pp.fs_reclaim_acquire 41.72 +2.8 44.57 perf-profile.children.cycles-pp.do_sys_open 41.57 +2.9 44.50 perf-profile.children.cycles-pp.do_sys_openat2 1.00 ± 6% +2.9 3.94 perf-profile.children.cycles-pp.d_lookup 0.00 +3.2 3.16 ± 3% perf-profile.children.cycles-pp.do_raw_spin_lock 1.01 +3.2 4.18 perf-profile.children.cycles-pp.lockref_put_or_lock 0.27 ± 19% +3.2 3.48 perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.00 +4.0 3.96 perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled 0.00 +4.2 4.18 perf-profile.children.cycles-pp.lockdep_hardirqs_on 35.05 +4.2 39.29 perf-profile.children.cycles-pp.do_filp_open 34.71 +4.5 39.17 perf-profile.children.cycles-pp.path_openat 36.56 +4.7 41.22 perf-profile.children.cycles-pp.unlink 0.00 +4.7 4.69 perf-profile.children.cycles-pp._raw_spin_unlock 1.89 +4.7 6.59 perf-profile.children.cycles-pp.___might_sleep 88.53 +5.3 93.86 perf-profile.children.cycles-pp.do_syscall_64 88.99 +6.0 94.97 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.00 +6.5 6.52 perf-profile.children.cycles-pp.mark_lock 5.30 +7.5 12.79 perf-profile.children.cycles-pp.dput 0.00 +9.0 9.00 perf-profile.children.cycles-pp.lock_is_held_type 27.44 +9.2 36.69 perf-profile.children.cycles-pp.do_unlinkat 0.00 +11.0 11.03 perf-profile.children.cycles-pp.lock_release 0.00 +14.2 14.16 perf-profile.children.cycles-pp.__lock_text_start 0.00 +21.2 21.17 perf-profile.children.cycles-pp.lookup_open 0.00 +23.8 23.81 perf-profile.children.cycles-pp.__lock_acquire 0.00 +32.4 32.39 perf-profile.children.cycles-pp.lock_acquire 7.78 ± 2% -5.8 2.02 ± 6% perf-profile.self.cycles-pp.do_syscall_64 4.44 ± 3% -3.5 0.99 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret 3.65 ± 4% -2.7 0.98 perf-profile.self.cycles-pp.entry_SYSCALL_64 2.55 ± 2% -1.7 0.87 perf-profile.self.cycles-pp.kmem_cache_alloc 2.91 ± 8% -1.6 1.32 ± 7% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 1.67 ± 2% -1.4 0.29 perf-profile.self.cycles-pp.path_openat 1.74 ± 6% -1.2 0.55 perf-profile.self.cycles-pp.__d_lookup_rcu 1.68 ± 2% -1.2 0.49 ± 2% perf-profile.self.cycles-pp.link_path_walk 1.42 ± 4% -1.0 0.41 perf-profile.self.cycles-pp.fsnotify 1.66 ± 4% -1.0 0.69 perf-profile.self.cycles-pp.rcu_cblist_dequeue 1.75 ± 5% -0.9 0.86 perf-profile.self.cycles-pp.kmem_cache_free 1.76 ± 2% -0.8 0.95 perf-profile.self.cycles-pp.___might_sleep 1.02 ± 2% -0.7 0.28 ± 7% perf-profile.self.cycles-pp.generic_permission 0.96 ± 7% -0.7 0.24 ± 4% perf-profile.self.cycles-pp.__slab_free 0.94 -0.7 0.26 perf-profile.self.cycles-pp.apparmor_file_open 0.76 ± 2% -0.7 0.07 perf-profile.self.cycles-pp._raw_spin_lock_irq 0.97 ± 4% -0.7 0.30 ± 3% perf-profile.self.cycles-pp.call_rcu 0.74 ± 2% -0.7 0.09 ± 11% perf-profile.self.cycles-pp.lockref_put_or_lock 0.99 ± 5% -0.7 0.34 ± 8% perf-profile.self.cycles-pp.__might_sleep 0.80 ± 2% -0.6 0.15 ± 6% perf-profile.self.cycles-pp.__alloc_file 0.82 ± 6% -0.6 0.21 ± 2% perf-profile.self.cycles-pp.inode_init_once 0.74 ± 5% -0.6 0.20 ± 2% perf-profile.self.cycles-pp.apparmor_file_free_security 0.81 ± 2% -0.5 0.27 ± 5% perf-profile.self.cycles-pp.shmem_get_inode 0.67 ± 4% -0.5 0.17 ± 2% perf-profile.self.cycles-pp.rcu_segcblist_enqueue 0.67 ± 3% -0.5 0.18 ± 2% perf-profile.self.cycles-pp.__mnt_want_write 0.70 ± 4% -0.5 0.22 ± 4% perf-profile.self.cycles-pp.__check_object_size 0.61 ± 3% -0.5 0.14 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.73 ± 5% -0.5 0.27 ± 3% perf-profile.self.cycles-pp.inode_permission 0.69 ± 3% -0.5 0.23 ± 4% perf-profile.self.cycles-pp.do_dentry_open 0.64 ± 4% -0.5 0.18 ± 8% perf-profile.self.cycles-pp.apparmor_file_alloc_security 0.56 ± 9% -0.5 0.11 ± 4% perf-profile.self.cycles-pp.rcu_all_qs 0.68 ± 7% -0.5 0.23 ± 4% perf-profile.self.cycles-pp.__d_lookup 0.69 ± 4% -0.4 0.24 ± 2% perf-profile.self.cycles-pp.strncpy_from_user 0.61 ± 2% -0.4 0.17 ± 8% perf-profile.self.cycles-pp._cond_resched 0.63 ± 5% -0.4 0.20 ± 4% perf-profile.self.cycles-pp.do_sys_openat2 0.76 ± 4% -0.4 0.34 perf-profile.self.cycles-pp.file_free_rcu 0.67 ± 6% -0.4 0.24 ± 6% perf-profile.self.cycles-pp.__virt_addr_valid 0.58 ± 2% -0.4 0.16 ± 6% perf-profile.self.cycles-pp.walk_component 0.97 ± 3% -0.4 0.55 ± 4% perf-profile.self.cycles-pp.dput 0.53 ± 6% -0.4 0.11 ± 4% perf-profile.self.cycles-pp.getname_flags 0.56 ± 4% -0.4 0.15 ± 3% perf-profile.self.cycles-pp.lookup_fast 0.56 ± 3% -0.4 0.16 perf-profile.self.cycles-pp.inode_init_always 0.60 ± 3% -0.4 0.24 ± 2% perf-profile.self.cycles-pp.__fput 0.52 ± 2% -0.3 0.17 ± 8% perf-profile.self.cycles-pp.__destroy_inode 0.62 ± 7% -0.3 0.28 ± 5% perf-profile.self.cycles-pp.__follow_mount_rcu 0.46 ± 7% -0.3 0.12 ± 8% perf-profile.self.cycles-pp.__legitimize_mnt 0.43 ± 7% -0.3 0.09 ± 11% perf-profile.self.cycles-pp.shmem_reserve_inode 0.43 ± 5% -0.3 0.10 ± 15% perf-profile.self.cycles-pp.task_work_run 0.42 ± 6% -0.3 0.09 perf-profile.self.cycles-pp.current_time 0.42 ± 3% -0.3 0.10 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid 0.41 ± 4% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.new_slab 0.44 ± 4% -0.3 0.12 ± 4% perf-profile.self.cycles-pp.up_write 0.44 ± 4% -0.3 0.12 ± 4% perf-profile.self.cycles-pp.common_perm 0.45 ± 14% -0.3 0.14 perf-profile.self.cycles-pp.__sb_start_write 0.40 ± 5% -0.3 0.09 ± 11% perf-profile.self.cycles-pp.down_write 0.42 -0.3 0.12 ± 12% perf-profile.self.cycles-pp.shmem_undo_range 0.47 ± 4% -0.3 0.18 ± 2% perf-profile.self.cycles-pp.do_unlinkat 0.49 ± 5% -0.3 0.20 ± 15% perf-profile.self.cycles-pp.set_root 0.50 ± 8% -0.3 0.23 ± 2% perf-profile.self.cycles-pp.shmem_free_in_core_inode 0.47 ± 4% -0.3 0.21 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.38 ± 3% -0.3 0.12 perf-profile.self.cycles-pp.drop_nlink 0.39 ± 13% -0.3 0.13 perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.36 ± 9% -0.3 0.10 perf-profile.self.cycles-pp.shmem_free_inode 0.34 ± 12% -0.3 0.09 ± 11% perf-profile.self.cycles-pp.timestamp_truncate 0.35 ± 5% -0.3 0.10 ± 15% perf-profile.self.cycles-pp.__d_alloc 0.39 ± 4% -0.2 0.14 ± 7% perf-profile.self.cycles-pp.shmem_evict_inode 0.33 ± 5% -0.2 0.09 ± 11% perf-profile.self.cycles-pp.__pthread_enable_asynccancel 0.35 ± 2% -0.2 0.11 ± 4% perf-profile.self.cycles-pp.vfs_unlink 0.35 ± 3% -0.2 0.11 ± 4% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.38 ± 2% -0.2 0.14 perf-profile.self.cycles-pp.memset_erms 0.38 ± 12% -0.2 0.15 ± 10% perf-profile.self.cycles-pp.__fd_install 0.30 ± 8% -0.2 0.07 perf-profile.self.cycles-pp.apparmor_task_getsecid 0.31 ± 2% -0.2 0.08 ± 12% perf-profile.self.cycles-pp.__list_add_valid 0.39 ± 6% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.find_get_entries 0.38 ± 5% -0.2 0.15 ± 13% perf-profile.self.cycles-pp.__alloc_fd 0.33 ± 5% -0.2 0.10 perf-profile.self.cycles-pp.__GI___libc_open 0.30 ± 3% -0.2 0.08 ± 6% perf-profile.self.cycles-pp.d_add 0.31 ± 8% -0.2 0.09 ± 17% perf-profile.self.cycles-pp.__fsnotify_parent 0.35 ± 4% -0.2 0.14 ± 3% perf-profile.self.cycles-pp.__check_heap_object 0.39 ± 5% -0.2 0.17 ± 5% perf-profile.self.cycles-pp.__srcu_read_lock 0.29 ± 3% -0.2 0.08 ± 12% perf-profile.self.cycles-pp.__dentry_kill 0.29 -0.2 0.08 ± 6% perf-profile.self.cycles-pp.___d_drop 0.31 ± 11% -0.2 0.10 ± 10% perf-profile.self.cycles-pp.security_inode_permission 0.29 ± 7% -0.2 0.08 perf-profile.self.cycles-pp.security_inode_init_security 0.26 ± 9% -0.2 0.06 ± 9% perf-profile.self.cycles-pp.common_perm_create 0.29 ± 3% -0.2 0.09 ± 11% perf-profile.self.cycles-pp.unlink 0.29 ± 4% -0.2 0.09 ± 17% perf-profile.self.cycles-pp.__GI___libc_close 0.29 ± 11% -0.2 0.09 ± 17% perf-profile.self.cycles-pp._atomic_dec_and_lock 0.26 ± 4% -0.2 0.06 ± 9% perf-profile.self.cycles-pp.terminate_walk 0.23 ± 8% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__srcu_read_unlock 0.47 ± 4% -0.2 0.28 perf-profile.self.cycles-pp.d_alloc_parallel 0.29 ± 5% -0.2 0.10 ± 5% perf-profile.self.cycles-pp.set_cached_acl 0.29 ± 3% -0.2 0.10 ± 5% perf-profile.self.cycles-pp.filename_parentat 0.24 ± 9% -0.2 0.05 perf-profile.self.cycles-pp.simple_lookup 0.24 ± 2% -0.2 0.06 ± 9% perf-profile.self.cycles-pp.__d_rehash 0.31 ± 6% -0.2 0.13 ± 7% perf-profile.self.cycles-pp.aa_get_task_label 0.24 ± 7% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.shmem_unlink 0.20 ± 5% -0.2 0.03 ±100% perf-profile.self.cycles-pp.complete_walk 0.24 ± 3% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.task_work_add 0.24 ± 2% -0.2 0.07 perf-profile.self.cycles-pp.do_filp_open 0.27 ± 5% -0.2 0.10 ± 10% perf-profile.self.cycles-pp.nd_jump_root 0.26 ± 6% -0.2 0.09 ± 11% perf-profile.self.cycles-pp.shmem_mknod 0.19 ± 9% -0.2 0.03 ±100% perf-profile.self.cycles-pp.common_perm_rm 0.34 ± 2% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.path_init 0.18 ± 2% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__close_fd 0.33 ± 7% -0.2 0.17 perf-profile.self.cycles-pp.mntput_no_expire 0.25 ± 11% -0.2 0.10 ± 5% perf-profile.self.cycles-pp.__d_lookup_done 0.27 ± 5% -0.2 0.11 ± 4% perf-profile.self.cycles-pp.unlazy_walk 0.22 ± 12% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.evict 0.18 ± 6% -0.2 0.03 ±100% perf-profile.self.cycles-pp.___slab_alloc 0.38 ± 6% -0.2 0.22 ± 6% perf-profile.self.cycles-pp.i_callback 0.25 ± 3% -0.2 0.10 ± 10% perf-profile.self.cycles-pp.__pthread_disable_asynccancel 0.24 ± 14% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.map_id_range_down 0.23 ± 4% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.d_set_d_op 0.21 ± 5% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.__d_instantiate 0.17 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp._raw_spin_trylock 0.23 ± 7% -0.1 0.09 ± 11% perf-profile.self.cycles-pp.may_open 0.22 ± 8% -0.1 0.08 perf-profile.self.cycles-pp.__sb_end_write 0.23 ± 8% -0.1 0.09 ± 17% perf-profile.self.cycles-pp.may_delete 0.21 ± 5% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.__x64_sys_close 0.20 ± 3% -0.1 0.07 ± 23% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.20 ± 7% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.memcpy_erms 0.20 ± 6% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.ihold 0.20 ± 6% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.xas_find 0.24 -0.1 0.11 ± 9% perf-profile.self.cycles-pp.iput 0.19 ± 8% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.legitimize_path 0.19 ± 7% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.posix_acl_create 0.17 ± 6% -0.1 0.06 ± 9% perf-profile.self.cycles-pp.map_id_up 0.30 ± 7% -0.1 0.18 ± 8% perf-profile.self.cycles-pp.__lookup_mnt 0.17 ± 4% -0.1 0.06 perf-profile.self.cycles-pp.d_alloc 0.16 ± 10% -0.1 0.05 perf-profile.self.cycles-pp.file_ra_state_init 0.13 ± 12% -0.1 0.03 ±100% perf-profile.self.cycles-pp.security_file_open 0.13 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp.dentry_unlink_inode 0.12 ± 13% -0.1 0.03 ±100% perf-profile.self.cycles-pp.security_task_getsecid 0.17 ± 8% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.xas_start 0.15 ± 10% -0.1 0.06 ± 9% perf-profile.self.cycles-pp.mnt_drop_write 0.17 ± 5% -0.1 0.07 perf-profile.self.cycles-pp.fput_many 0.15 ± 4% -0.1 0.06 ± 16% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.16 ± 5% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist 0.11 ± 15% -0.1 0.03 ±100% perf-profile.self.cycles-pp.current_umask 0.14 ± 8% -0.1 0.06 perf-profile.self.cycles-pp.do_sys_open 0.11 ± 4% -0.1 0.03 ±100% perf-profile.self.cycles-pp.d_instantiate 0.14 ± 15% -0.1 0.07 ± 23% perf-profile.self.cycles-pp.putname 0.10 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp.fsnotify_destroy_marks 0.10 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp.process_measurement 0.10 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp.security_file_free 0.15 ± 5% -0.1 0.07 perf-profile.self.cycles-pp.restore_nameidata 0.15 ± 5% -0.1 0.08 ± 12% perf-profile.self.cycles-pp.expand_files 0.14 ± 8% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.simple_acl_create 0.12 ± 13% -0.1 0.06 ± 16% perf-profile.self.cycles-pp.filp_close 0.09 ± 17% -0.1 0.03 ±100% perf-profile.self.cycles-pp.security_path_mknod 0.09 ± 11% -0.1 0.03 ±100% perf-profile.self.cycles-pp.security_inode_free 0.09 ± 16% -0.1 0.03 ±100% perf-profile.self.cycles-pp.get_acl 0.11 ± 6% -0.1 0.05 perf-profile.self.cycles-pp.security_d_instantiate 0.12 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.inode_init_owner 0.14 ± 11% -0.1 0.08 ± 5% perf-profile.self.cycles-pp._find_next_bit 0.11 ± 7% -0.0 0.06 perf-profile.self.cycles-pp.get_unused_fd_flags 0.07 ± 10% -0.0 0.03 ±100% perf-profile.self.cycles-pp.should_failslab 0.10 ± 14% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.d_flags_for_inode 0.09 ± 19% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.ima_file_check 0.09 ± 4% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.rcu_do_batch 0.09 ± 14% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.legitimize_links 0.08 ± 6% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.rcu_segcblist_pend_cbs 0.07 ± 20% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.get_cached_acl 0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_off_thunk 0.00 +0.1 0.07 ± 23% perf-profile.self.cycles-pp.trace_hardirqs_off_caller 0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.trace_hardirqs_on_thunk 0.00 +0.1 0.07 perf-profile.self.cycles-pp._raw_spin_unlock_irq 0.00 +0.1 0.08 ± 6% perf-profile.self.cycles-pp.static_obj 0.00 +0.1 0.08 ± 12% perf-profile.self.cycles-pp.__put_unused_fd 0.00 +0.1 0.09 perf-profile.self.cycles-pp.reacquire_held_locks 0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.trace_hardirqs_on_caller 0.00 +0.1 0.12 ± 13% perf-profile.self.cycles-pp.rcu_read_lock_any_held 0.00 +0.1 0.12 ± 4% perf-profile.self.cycles-pp.fs_reclaim_acquire 0.00 +0.2 0.16 ± 9% perf-profile.self.cycles-pp.rcu_read_lock_held 0.00 +0.2 0.17 perf-profile.self.cycles-pp.__need_fs_reclaim 0.00 +0.2 0.20 perf-profile.self.cycles-pp.rcu_read_lock_sched_held 0.00 +0.3 0.29 perf-profile.self.cycles-pp.__raw_spin_lock_init 0.00 +0.4 0.36 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock 0.00 +0.4 0.36 perf-profile.self.cycles-pp.tracer_hardirqs_on 0.00 +0.4 0.36 ± 4% perf-profile.self.cycles-pp.lookup_open 0.00 +0.4 0.42 perf-profile.self.cycles-pp.register_lock_class 0.00 +0.4 0.43 ± 4% perf-profile.self.cycles-pp.do_raw_spin_unlock 0.00 +0.5 0.46 ± 2% perf-profile.self.cycles-pp.lockdep_init_map 0.00 +0.5 0.46 ± 3% perf-profile.self.cycles-pp.trace_hardirqs_off 0.00 +0.5 0.48 perf-profile.self.cycles-pp.trace_hardirqs_on 0.00 +0.5 0.50 ± 2% perf-profile.self.cycles-pp.rcu_read_lock_held_common 0.00 +0.5 0.53 ± 4% perf-profile.self.cycles-pp.__lock_text_start 0.00 +0.6 0.55 ± 2% perf-profile.self.cycles-pp.tracer_hardirqs_off 0.00 +0.7 0.70 ± 9% perf-profile.self.cycles-pp.rcu_lockdep_current_cpu_online 0.00 +0.9 0.85 ± 2% perf-profile.self.cycles-pp.rcu_is_watching 0.00 +1.0 0.96 ± 2% perf-profile.self.cycles-pp.lockdep_hardirqs_off 0.00 +1.0 1.04 perf-profile.self.cycles-pp.match_held_lock 0.00 +1.2 1.17 perf-profile.self.cycles-pp.mark_held_locks 0.00 +1.3 1.28 perf-profile.self.cycles-pp.debug_check_no_locks_freed 0.00 +1.5 1.54 perf-profile.self.cycles-pp.do_raw_spin_lock 0.00 +1.6 1.58 ± 2% perf-profile.self.cycles-pp.lockdep_hardirqs_on 0.00 +1.6 1.65 perf-profile.self.cycles-pp.find_held_lock 0.00 +3.4 3.38 perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled 0.00 +6.2 6.20 perf-profile.self.cycles-pp.mark_lock 0.00 +7.1 7.12 perf-profile.self.cycles-pp.lock_acquire 0.00 +7.3 7.32 perf-profile.self.cycles-pp.lock_release 0.00 +7.8 7.84 perf-profile.self.cycles-pp.lock_is_held_type 0.00 +18.5 18.52 perf-profile.self.cycles-pp.__lock_acquire will-it-scale.per_thread_ops 140000 +------------------------------------------------------------------+ |..+.+..+.+..+.+..+..+.+..+.+..+..+.+..+.+..+.+..+..+.+..+.+..+.+..| 120000 |-+ | | | 100000 |-+ | | | 80000 |-+ | | | 60000 |-+O O O O | | | 40000 |-+ | | O O O O O O O O O O O O O O O | 20000 |-+ | | | 0 +------------------------------------------------------------------+ will-it-scale.workload 1.2e+06 +-----------------------------------------------------------------+ | | 1e+06 |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+.+..+.+..+.+..+.+..| | | | | 800000 |-+ | | | 600000 |-+ | | O O O O | 400000 |-+ | | | | O O O O O O O O O O O O O O O | 200000 |-+ | | | 0 +-----------------------------------------------------------------+ will-it-scale.time.user_time 350 +---------------------------------------------------------------------+ |..+.+..+..+.+. +..+.+..+..+.+..+..+.+..+..+.+..+..+..+.+..+..+.+..| 300 |-+ | | | 250 |-+ | | | 200 |-+ | | O O O O | 150 |-+ | | | 100 |-+ O O O O O O O O O O O O O O O | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ will-it-scale.time.system_time 2500 +--------------------------------------------------------------------+ | O O O O O O O O O O O O O O O | | O O O O .+.. | 2000 |..+.+..+..+.+..+..+.+..+..+ +..+.+..+.+..+..+.+..+..+.+..+..+.+..| | | | | 1500 |-+ | | | 1000 |-+ | | | | | 500 |-+ | | | | | 0 +--------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-knm01: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-knm01/getppid1/will-it-scale/0x11 commit: 7aba5dcc23 ("jfs: Replace zero-length array with flexible-array member") 05c5a0273b ("fs: jfs: fix a possible data race in txBegin()") 7aba5dcc234635b4 05c5a0273bc25627b9e9bbebc85 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 2:4 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 1:4 -25% :4 dmesg.WARNING:at_ip___perf_sw_event/0x :4 100% 4:4 kmsg.IP-Config:Failed_to_open_erspan0 :4 100% 4:4 kmsg.IP-Config:Failed_to_open_gretap0 :4 100% 4:4 kmsg.kvm:already_loaded_the_other_module %stddev %change %stddev \ | \ 317295 -84.4% 49580 will-it-scale.per_process_ops 312.61 +4.8% 327.65 will-it-scale.time.elapsed_time 312.61 +4.8% 327.65 will-it-scale.time.elapsed_time.max 791.75 +32.1% 1046 ± 9% will-it-scale.time.involuntary_context_switches 3.00 +108.3% 6.25 ± 6% will-it-scale.time.percent_of_cpu_this_job_got 4.74 +238.1% 16.02 ± 8% will-it-scale.time.system_time 91381248 -84.4% 14279312 will-it-scale.workload 1.12e+09 ± 4% +59.6% 1.788e+09 ± 8% cpuidle.C1.time 2320793 ± 4% +41.7% 3289207 ± 17% cpuidle.C1.usage 7868 ± 20% +2593.2% 211899 ± 68% cpuidle.POLL.time 45.68 +37.3% 62.72 boot-time.boot 27.69 ± 4% +39.1% 38.51 boot-time.dhcp 11411 ± 2% +41.0% 16092 boot-time.idle 2.14 +24.3% 2.66 boot-time.smp_boot 1.23 ± 6% +0.9 2.12 ± 6% mpstat.cpu.all.idle% 0.00 ± 17% +0.0 0.00 ± 26% mpstat.cpu.all.soft% 54.74 +28.4 83.17 mpstat.cpu.all.sys% 44.03 -29.3 14.70 ± 3% mpstat.cpu.all.usr% 54.00 +51.9% 82.00 vmstat.cpu.sy 43.25 -67.6% 14.00 ± 5% vmstat.cpu.us 2242 -14.0% 1927 ± 3% vmstat.system.cs 587834 -24.5% 444073 ± 12% vmstat.system.in 5449 +204.3% 16584 ± 39% numa-vmstat.node0.nr_inactive_anon 10990 ± 12% +258.5% 39396 ± 22% numa-vmstat.node0.nr_mapped 31223 ± 2% +15.0% 35891 ± 2% numa-vmstat.node0.nr_shmem 25404 +38.1% 35088 numa-vmstat.node0.nr_slab_reclaimable 75484 +15.5% 87149 numa-vmstat.node0.nr_slab_unreclaimable 5449 +204.3% 16583 ± 39% numa-vmstat.node0.nr_zone_inactive_anon 2084888 -31.5% 1428939 numa-vmstat.node0.numa_hit 2084902 -31.5% 1428952 numa-vmstat.node0.numa_local 4495 ± 4% +46.0% 6560 ± 8% numa-vmstat.node1.nr_slab_reclaimable 2907 ± 18% +53.2% 4453 ± 8% numa-vmstat.node1.nr_slab_unreclaimable 7604318 ± 5% -11.8% 6709760 meminfo.DirectMap2M 1465469 ± 4% -64.2% 525020 meminfo.DirectMap4k 22057 +201.7% 66539 ± 38% meminfo.Inactive 21800 +204.1% 66293 ± 38% meminfo.Inactive(anon) 119635 +39.4% 166765 meminfo.KReclaimable 54582 ± 10% +209.8% 169070 ± 20% meminfo.Mapped 2965860 +11.5% 3307267 ± 2% meminfo.Memused 119635 +39.4% 166765 meminfo.SReclaimable 313588 +16.9% 366537 meminfo.SUnreclaim 124948 ± 2% +15.3% 144011 ± 3% meminfo.Shmem 433223 +23.1% 533303 meminfo.Slab 176743 +16.8% 206477 meminfo.VmallocUsed 22057 +201.2% 66436 ± 38% numa-meminfo.node0.Inactive 21800 +203.6% 66190 ± 38% numa-meminfo.node0.Inactive(anon) 101650 +38.2% 140446 numa-meminfo.node0.KReclaimable 43485 ± 12% +262.4% 157609 ± 21% numa-meminfo.node0.Mapped 2353458 +13.1% 2660838 ± 2% numa-meminfo.node0.MemUsed 101650 +38.2% 140446 numa-meminfo.node0.SReclaimable 301953 +15.5% 348662 numa-meminfo.node0.SUnreclaim 124909 ± 2% +15.1% 143725 ± 3% numa-meminfo.node0.Shmem 403603 +21.2% 489109 numa-meminfo.node0.Slab 17980 ± 4% +46.0% 26243 ± 8% numa-meminfo.node1.KReclaimable 17980 ± 4% +46.0% 26243 ± 8% numa-meminfo.node1.SReclaimable 11630 ± 18% +53.2% 17813 ± 8% numa-meminfo.node1.SUnreclaim 29610 ± 8% +48.8% 44056 ± 7% numa-meminfo.node1.Slab 113909 +3.0% 117371 proc-vmstat.nr_anon_pages 301248 +2.0% 307405 proc-vmstat.nr_file_pages 5449 +203.5% 16542 ± 38% proc-vmstat.nr_inactive_anon 13705 ± 10% +208.9% 42333 ± 19% proc-vmstat.nr_mapped 15428 -1.5% 15193 proc-vmstat.nr_page_table_pages 31216 ± 2% +15.0% 35896 ± 2% proc-vmstat.nr_shmem 29904 +39.3% 41665 proc-vmstat.nr_slab_reclaimable 78394 +16.9% 91612 proc-vmstat.nr_slab_unreclaimable 5449 +203.5% 16542 ± 38% proc-vmstat.nr_zone_inactive_anon 979.25 ±110% +389.8% 4796 ± 8% proc-vmstat.numa_hint_faults 979.25 ±110% +389.8% 4796 ± 8% proc-vmstat.numa_hint_faults_local 941145 -5.4% 890601 proc-vmstat.numa_hit 941145 -5.4% 890601 proc-vmstat.numa_local 1024187 -4.7% 976206 proc-vmstat.pgalloc_normal 873632 -6.6% 815925 proc-vmstat.pgfault 1210449 ± 3% -39.4% 733287 ± 11% proc-vmstat.pgfree 92.98 -68.6 24.34 ±173% perf-profile.calltrace.cycles-pp.getppid 47.25 ± 4% -25.6 21.64 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.getppid 26.10 ± 4% -25.5 0.59 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.getppid 44.62 ± 4% -24.6 19.98 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.getppid 14.16 ± 6% -13.7 0.51 ±173% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.getppid 27.98 ± 6% -10.4 17.55 ±173% perf-profile.calltrace.cycles-pp.__x64_sys_getppid.do_syscall_64.entry_SYSCALL_64_after_hwframe.getppid 14.98 ± 4% -6.4 8.57 ±173% perf-profile.calltrace.cycles-pp.__task_pid_nr_ns.__x64_sys_getppid.do_syscall_64.entry_SYSCALL_64_after_hwframe.getppid 98.22 -73.6 24.65 ±173% perf-profile.children.cycles-pp.getppid 47.88 ± 4% -26.1 21.81 ±173% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 45.32 ± 4% -25.1 20.19 ±173% perf-profile.children.cycles-pp.do_syscall_64 24.32 ± 5% -23.7 0.60 ±173% perf-profile.children.cycles-pp.entry_SYSCALL_64 15.64 ± 5% -15.1 0.57 ±173% perf-profile.children.cycles-pp.syscall_return_via_sysret 28.10 ± 6% -10.5 17.59 ±173% perf-profile.children.cycles-pp.__x64_sys_getppid 15.06 ± 4% -6.4 8.61 ±173% perf-profile.children.cycles-pp.__task_pid_nr_ns 7.78 ± 19% -1.7 6.08 ±173% perf-profile.children.cycles-pp.apic_timer_interrupt 7.33 ± 20% -1.3 6.01 ±173% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 5.64 ± 20% -0.7 4.94 ±173% perf-profile.children.cycles-pp.hrtimer_interrupt 22.26 ± 5% -21.7 0.53 ±173% perf-profile.self.cycles-pp.entry_SYSCALL_64 15.61 ± 5% -15.0 0.57 ±173% perf-profile.self.cycles-pp.syscall_return_via_sysret 14.02 ± 4% -11.3 2.76 ±173% perf-profile.self.cycles-pp.__task_pid_nr_ns 10.85 ± 4% -10.2 0.62 ±173% perf-profile.self.cycles-pp.do_syscall_64 12.08 ± 9% -9.7 2.34 ±173% perf-profile.self.cycles-pp.__x64_sys_getppid 8.58 -8.3 0.32 ±173% perf-profile.self.cycles-pp.getppid 60.73 -92.4% 4.65 ±173% perf-stat.i.MPKI 8.96 -7.8 1.13 ±173% perf-stat.i.branch-miss-rate% 3.664e+08 -78.1% 80177268 ±173% perf-stat.i.branch-misses 1.254e+09 -86.7% 1.668e+08 ±173% perf-stat.i.cache-references 2223 -76.5% 521.96 ±173% perf-stat.i.context-switches 21.47 -87.1% 2.77 ±173% perf-stat.i.cpi 4.431e+11 -77.4% 1.002e+11 ±173% perf-stat.i.cpu-cycles 255.37 -80.9% 48.72 ±173% perf-stat.i.cpu-migrations 8683 -88.0% 1042 ±173% perf-stat.i.cycles-between-cache-misses 1.91 -1.6 0.28 ±173% perf-stat.i.iTLB-load-miss-rate% 2760 -75.1% 686.60 ±173% perf-stat.i.minor-faults 2760 -75.1% 686.60 ±173% perf-stat.i.page-faults 61.07 -92.3% 4.67 ±173% perf-stat.overall.MPKI 8.96 -7.8 1.12 ±173% perf-stat.overall.branch-miss-rate% 21.55 -87.0% 2.81 ±173% perf-stat.overall.cpi 8672 -87.9% 1048 ±173% perf-stat.overall.cycles-between-cache-misses 1.92 -1.6 0.28 ±173% perf-stat.overall.iTLB-load-miss-rate% 3.645e+08 -78.0% 80033922 ±173% perf-stat.ps.branch-misses 1.253e+09 -86.6% 1.685e+08 ±173% perf-stat.ps.cache-references 2159 -77.4% 488.54 ±173% perf-stat.ps.context-switches 286876 -75.0% 71639 ±173% perf-stat.ps.cpu-clock 4.421e+11 -77.1% 1.013e+11 ±173% perf-stat.ps.cpu-cycles 232.60 ± 2% -84.1% 36.89 ±173% perf-stat.ps.cpu-migrations 2674 -77.8% 594.49 ±173% perf-stat.ps.minor-faults 2674 -77.8% 594.49 ±173% perf-stat.ps.page-faults 286876 -75.0% 71639 ±173% perf-stat.ps.task-clock 36717 ± 55% +111.8% 77780 ± 19% sched_debug.cfs_rq:/.MIN_vruntime.avg 10573717 ± 55% +111.9% 22400810 ± 19% sched_debug.cfs_rq:/.MIN_vruntime.max 622006 ± 55% +111.8% 1317686 ± 19% sched_debug.cfs_rq:/.MIN_vruntime.stddev 113082 ± 2% -25.2% 84619 ± 3% sched_debug.cfs_rq:/.exec_clock.min 2174 ± 4% +80.1% 3915 ± 19% sched_debug.cfs_rq:/.exec_clock.stddev 350219 ± 26% +81.1% 634377 ± 16% sched_debug.cfs_rq:/.load.max 4.36 ± 14% +38.2% 6.03 ± 7% sched_debug.cfs_rq:/.load_avg.avg 307.45 ± 22% +131.3% 711.24 ± 21% sched_debug.cfs_rq:/.load_avg.max 20.98 ± 27% +109.4% 43.94 ± 18% sched_debug.cfs_rq:/.load_avg.stddev 36717 ± 55% +111.8% 77780 ± 19% sched_debug.cfs_rq:/.max_vruntime.avg 10573720 ± 55% +111.9% 22400815 ± 19% sched_debug.cfs_rq:/.max_vruntime.max 622006 ± 55% +111.8% 1317687 ± 19% sched_debug.cfs_rq:/.max_vruntime.stddev 37523550 -25.6% 27902434 ± 3% sched_debug.cfs_rq:/.min_vruntime.min 901238 ± 4% +59.0% 1433317 ± 18% sched_debug.cfs_rq:/.min_vruntime.stddev 2.09 ± 14% +57.6% 3.30 ± 17% sched_debug.cfs_rq:/.nr_spread_over.avg 94.65 ± 6% +60.1% 151.53 ± 52% sched_debug.cfs_rq:/.nr_spread_over.max 7.34 ± 6% +47.6% 10.84 ± 41% sched_debug.cfs_rq:/.nr_spread_over.stddev 3.77 ± 6% +29.7% 4.89 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.avg 292.95 ± 20% +111.1% 618.43 ± 16% sched_debug.cfs_rq:/.runnable_load_avg.max 18.38 ± 19% +102.9% 37.28 ± 15% sched_debug.cfs_rq:/.runnable_load_avg.stddev 349946 ± 26% +81.1% 633684 ± 16% sched_debug.cfs_rq:/.runnable_weight.max 250152 ± 51% +233.8% 835124 ± 20% sched_debug.cfs_rq:/.spread0.avg 620948 ± 19% +147.3% 1535605 ± 10% sched_debug.cfs_rq:/.spread0.max -7540824 +115.4% -16246462 sched_debug.cfs_rq:/.spread0.min 550536 ± 7% +125.5% 1241459 ± 21% sched_debug.cfs_rq:/.spread0.stddev 484.00 ± 15% -42.2% 279.99 ± 32% sched_debug.cfs_rq:/.util_avg.min 57.30 ± 9% +25.8% 72.08 sched_debug.cfs_rq:/.util_avg.stddev 370.07 -11.6% 327.26 sched_debug.cfs_rq:/.util_est_enqueued.avg 801.30 ± 7% +11.2% 891.38 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.max 0.90 ± 19% +273.1% 3.36 ± 47% sched_debug.cfs_rq:/.util_est_enqueued.min 120.90 +48.2% 179.17 sched_debug.cfs_rq:/.util_est_enqueued.stddev 2278609 ± 5% +28.2% 2920845 ± 13% sched_debug.cpu.avg_idle.avg 2025 ± 2% -31.4% 1389 ± 5% sched_debug.cpu.clock.stddev 2025 ± 2% -31.4% 1389 ± 5% sched_debug.cpu.clock_task.stddev 342.60 ± 7% +16.8% 400.21 ± 5% sched_debug.cpu.curr->pid.stddev 791314 ± 3% +29.0% 1020928 ± 11% sched_debug.cpu.max_idle_balance_cost.avg 0.12 ± 4% +33.4% 0.16 ± 3% sched_debug.cpu.nr_running.stddev 2011 ± 9% +14.2% 2296 ± 7% sched_debug.cpu.nr_switches.stddev 0.00 ± 49% -83.3% 0.00 ±128% sched_debug.cpu.nr_uninterruptible.avg 233.75 ± 3% +31.8% 308.00 sched_debug.cpu.nr_uninterruptible.max 16.05 ± 5% +26.3% 20.27 sched_debug.cpu.nr_uninterruptible.stddev 368.35 -30.2% 257.27 ± 5% sched_debug.cpu.sched_count.min 204.20 ± 5% -8.8% 186.15 ± 3% sched_debug.cpu.sched_goidle.stddev 135.25 -32.7% 91.07 ± 4% sched_debug.cpu.ttwu_count.min 128.70 -31.9% 87.58 ± 5% sched_debug.cpu.ttwu_local.min 613.86 ± 11% +25.7% 771.81 ± 12% sched_debug.cpu.ttwu_local.stddev 0.00 +183.3% 0.00 ± 17% sched_debug.rt_rq:/.rt_nr_migratory.avg 0.20 +183.3% 0.57 ± 17% sched_debug.rt_rq:/.rt_nr_migratory.max 0.01 +183.3% 0.03 ± 17% sched_debug.rt_rq:/.rt_nr_migratory.stddev 0.00 +183.3% 0.00 ± 17% sched_debug.rt_rq:/.rt_nr_running.avg 0.20 +183.3% 0.57 ± 17% sched_debug.rt_rq:/.rt_nr_running.max 0.01 +183.3% 0.03 ± 17% sched_debug.rt_rq:/.rt_nr_running.stddev 0.33 ± 65% +243.5% 1.13 ± 14% sched_debug.rt_rq:/.rt_time.avg 94.55 ± 65% +237.4% 319.05 ± 15% sched_debug.rt_rq:/.rt_time.max 5.56 ± 65% +237.5% 18.77 ± 15% sched_debug.rt_rq:/.rt_time.stddev 44931 +41.9% 63777 slabinfo.Acpi-Operand.active_objs 802.75 +42.0% 1140 slabinfo.Acpi-Operand.active_slabs 44987 +41.9% 63859 slabinfo.Acpi-Operand.num_objs 802.75 +42.0% 1140 slabinfo.Acpi-Operand.num_slabs 1460 ± 14% -100.0% 0.00 slabinfo.Acpi-Parse.active_objs 1460 ± 14% -100.0% 0.00 slabinfo.Acpi-Parse.num_objs 174.75 ± 6% -100.0% 0.00 slabinfo.UDPv6.active_objs 174.75 ± 6% -100.0% 0.00 slabinfo.UDPv6.num_objs 2366 ± 4% -36.3% 1508 ± 8% slabinfo.UNIX.active_objs 2366 ± 4% -36.3% 1508 ± 8% slabinfo.UNIX.num_objs 136.25 ± 26% -50.8% 67.00 ± 23% slabinfo.btrfs_inode.active_objs 136.25 ± 26% -50.8% 67.00 ± 23% slabinfo.btrfs_inode.num_objs 14784 -100.0% 0.00 slabinfo.cred_jar.active_objs 351.50 -100.0% 0.00 slabinfo.cred_jar.active_slabs 14784 -100.0% 0.00 slabinfo.cred_jar.num_objs 351.50 -100.0% 0.00 slabinfo.cred_jar.num_slabs 3296 -26.5% 2423 slabinfo.dentry.active_slabs 3296 -26.5% 2423 slabinfo.dentry.num_slabs 979.00 ± 2% -100.0% 0.00 slabinfo.dmaengine-unmap-16.active_objs 979.00 ± 2% -100.0% 0.00 slabinfo.dmaengine-unmap-16.num_objs 1631 ± 9% -100.0% 0.00 slabinfo.file_lock_cache.active_objs 1631 ± 9% -100.0% 0.00 slabinfo.file_lock_cache.num_objs 13322 -13.6% 11508 slabinfo.files_cache.active_objs 13322 -13.6% 11508 slabinfo.files_cache.num_objs 1622 ± 9% -100.0% 0.00 slabinfo.fsnotify_mark_connector.active_objs 1622 ± 9% -100.0% 0.00 slabinfo.fsnotify_mark_connector.num_objs 1553 +54.2% 2395 slabinfo.inode_cache.active_slabs 1553 +54.2% 2395 slabinfo.inode_cache.num_slabs 2227 +17.8% 2624 slabinfo.kernfs_node_cache.active_slabs 2227 +17.8% 2624 slabinfo.kernfs_node_cache.num_slabs 9477 ± 2% +50.2% 14236 slabinfo.kmalloc-2k.active_objs 594.50 ± 2% +50.4% 894.00 slabinfo.kmalloc-2k.active_slabs 9516 ± 2% +50.4% 14314 slabinfo.kmalloc-2k.num_objs 594.50 ± 2% +50.4% 894.00 slabinfo.kmalloc-2k.num_slabs 3474 +10.3% 3833 slabinfo.kmalloc-4k.active_objs 657.00 ± 7% +29.1% 848.00 ± 11% slabinfo.kmem_cache.active_objs 657.00 ± 7% +29.1% 848.00 ± 11% slabinfo.kmem_cache.num_objs 712.00 ± 6% +40.6% 1001 ± 9% slabinfo.kmem_cache_node.active_objs 768.00 ± 5% +35.4% 1040 ± 9% slabinfo.kmem_cache_node.num_objs 50671 -100.0% 0.00 slabinfo.lsm_file_cache.active_objs 297.25 -100.0% 0.00 slabinfo.lsm_file_cache.active_slabs 50671 -100.0% 0.00 slabinfo.lsm_file_cache.num_objs 297.25 -100.0% 0.00 slabinfo.lsm_file_cache.num_slabs 9112 -28.3% 6531 slabinfo.mm_struct.active_objs 9112 -28.3% 6531 slabinfo.mm_struct.num_objs 1105 ± 15% -100.0% 0.00 slabinfo.mnt_cache.active_objs 1105 ± 15% -100.0% 0.00 slabinfo.mnt_cache.num_objs 111.75 ± 15% -100.0% 0.00 slabinfo.nfs_read_data.active_objs 111.75 ± 15% -100.0% 0.00 slabinfo.nfs_read_data.num_objs 817.50 ± 21% +102.1% 1652 ± 5% slabinfo.numa_policy.active_objs 817.50 ± 21% +102.1% 1652 ± 5% slabinfo.numa_policy.num_objs 20726 -10.6% 18539 slabinfo.pid.active_objs 323.75 +36.2% 441.00 slabinfo.pid.active_slabs 20726 -10.6% 18539 slabinfo.pid.num_objs 323.75 +36.2% 441.00 slabinfo.pid.num_slabs 2215 +9.3% 2421 ± 4% slabinfo.proc_dir_entry.active_objs 2215 +10.5% 2448 ± 3% slabinfo.proc_dir_entry.num_objs 34174 -15.1% 29003 slabinfo.proc_inode_cache.active_objs 697.00 +30.0% 906.25 slabinfo.proc_inode_cache.active_slabs 34177 -15.1% 29018 slabinfo.proc_inode_cache.num_objs 697.00 +30.0% 906.25 slabinfo.proc_inode_cache.num_slabs 13886 ± 2% -29.4% 9803 ± 2% slabinfo.shmem_inode_cache.active_objs 13886 ± 2% -29.4% 9803 ± 2% slabinfo.shmem_inode_cache.num_objs 10728 -16.0% 9011 ± 2% slabinfo.signal_cache.active_objs 10728 -16.0% 9011 ± 2% slabinfo.signal_cache.num_objs 1091 ± 10% -19.8% 875.00 ± 3% slabinfo.skbuff_fclone_cache.active_objs 1091 ± 10% -19.8% 875.00 ± 3% slabinfo.skbuff_fclone_cache.num_objs 4677 ± 2% -13.7% 4037 ± 4% slabinfo.sock_inode_cache.active_objs 4677 ± 2% -13.7% 4037 ± 4% slabinfo.sock_inode_cache.num_objs 16428 +16.5% 19132 slabinfo.task_delay_info.active_objs 16428 +16.5% 19132 slabinfo.task_delay_info.num_objs 3233 +11.7% 3611 slabinfo.task_struct.active_objs 3235 -44.1% 1807 slabinfo.task_struct.active_slabs 3235 +11.7% 3614 slabinfo.task_struct.num_objs 3235 -44.1% 1807 slabinfo.task_struct.num_slabs 1840 +25.2% 2303 slabinfo.trace_event_file.active_objs 1840 +25.2% 2303 slabinfo.trace_event_file.num_objs 136122 +18.3% 161019 ± 2% softirqs.CPU0.RCU 141474 +11.5% 157772 ± 3% softirqs.CPU1.RCU 136882 +16.7% 159760 ± 2% softirqs.CPU10.RCU 137022 ± 2% +15.3% 157981 ± 2% softirqs.CPU100.RCU 135840 +16.1% 157706 ± 2% softirqs.CPU101.RCU 134628 +14.3% 153914 ± 3% softirqs.CPU102.RCU 134794 +14.8% 154776 ± 2% softirqs.CPU103.RCU 138793 ± 12% +23.3% 171188 softirqs.CPU103.TIMER 136139 +15.7% 157563 ± 2% softirqs.CPU104.RCU 128409 ± 17% +26.4% 162302 ± 7% softirqs.CPU104.TIMER 136866 +16.6% 159616 softirqs.CPU105.RCU 136264 +16.4% 158562 ± 2% softirqs.CPU106.RCU 133141 ± 15% +27.0% 169069 ± 2% softirqs.CPU106.TIMER 137216 +16.4% 159786 softirqs.CPU107.RCU 132918 ± 14% +24.8% 165942 ± 5% softirqs.CPU107.TIMER 134727 +15.8% 156042 ± 2% softirqs.CPU108.RCU 133863 ± 14% +27.2% 170329 softirqs.CPU108.TIMER 135173 +17.6% 159019 ± 2% softirqs.CPU109.RCU 136304 +21.3% 165311 ± 4% softirqs.CPU11.RCU 134604 ± 2% +15.3% 155152 ± 4% softirqs.CPU110.RCU 131367 ± 15% +31.9% 173222 softirqs.CPU110.TIMER 135398 +15.4% 156202 ± 2% softirqs.CPU111.RCU 134911 ± 15% +23.0% 165956 ± 5% softirqs.CPU111.TIMER 133883 +16.7% 156266 ± 3% softirqs.CPU112.RCU 141469 ± 11% +18.2% 167195 ± 3% softirqs.CPU112.TIMER 134135 +14.8% 153961 ± 2% softirqs.CPU113.RCU 136991 ± 13% +25.0% 171183 softirqs.CPU113.TIMER 135577 +15.5% 156650 ± 2% softirqs.CPU114.RCU 136176 +14.7% 156184 ± 2% softirqs.CPU115.RCU 131694 ± 4% +18.9% 156553 softirqs.CPU116.RCU 132117 ± 15% +24.5% 164428 ± 6% softirqs.CPU116.TIMER 136732 +15.1% 157372 softirqs.CPU117.RCU 135603 ± 15% +24.0% 168171 ± 3% softirqs.CPU117.TIMER 137465 +16.5% 160083 ± 2% softirqs.CPU118.RCU 135363 +17.5% 159032 ± 2% softirqs.CPU119.RCU 137113 +17.3% 160838 ± 3% softirqs.CPU12.RCU 139539 ± 4% +12.3% 156719 ± 2% softirqs.CPU120.RCU 134836 +14.5% 154436 softirqs.CPU121.RCU 133522 ± 14% +27.6% 170377 softirqs.CPU121.TIMER 135000 +16.0% 156582 ± 2% softirqs.CPU122.RCU 134900 +14.1% 153984 ± 2% softirqs.CPU123.RCU 136555 +20.7% 164775 ± 7% softirqs.CPU124.RCU 136657 ± 2% +16.3% 158891 ± 2% softirqs.CPU125.RCU 133790 +16.9% 156440 softirqs.CPU126.RCU 131813 ± 14% +24.9% 164630 ± 4% softirqs.CPU126.TIMER 136264 +15.6% 157552 softirqs.CPU127.RCU 136723 ± 14% +23.3% 168646 ± 2% softirqs.CPU127.TIMER 134476 +17.5% 157992 ± 2% softirqs.CPU128.RCU 125895 ± 17% +33.9% 168615 ± 3% softirqs.CPU128.TIMER 135534 +16.9% 158376 softirqs.CPU129.RCU 124429 ± 15% +32.9% 165356 ± 5% softirqs.CPU129.TIMER 136483 +16.3% 158766 softirqs.CPU13.RCU 135648 +16.9% 158546 ± 2% softirqs.CPU130.RCU 129520 ± 16% +28.1% 165925 ± 4% softirqs.CPU130.TIMER 136893 +16.4% 159402 softirqs.CPU131.RCU 130865 ± 16% +26.6% 165714 ± 4% softirqs.CPU131.TIMER 133844 +13.5% 151873 ± 2% softirqs.CPU132.RCU 126615 ± 16% +34.8% 170717 softirqs.CPU132.TIMER 134961 +15.0% 155246 ± 2% softirqs.CPU133.RCU 133249 ± 14% +23.4% 164456 ± 5% softirqs.CPU133.TIMER 134421 ± 2% +16.6% 156773 softirqs.CPU134.RCU 133500 ± 15% +26.0% 168208 ± 2% softirqs.CPU134.TIMER 133934 ± 2% +19.0% 159395 ± 2% softirqs.CPU135.RCU 135741 +15.7% 156989 ± 2% softirqs.CPU136.RCU 128719 ± 13% +31.3% 168997 ± 2% softirqs.CPU136.TIMER 135502 +16.9% 158422 ± 2% softirqs.CPU137.RCU 133778 +17.3% 156923 ± 2% softirqs.CPU138.RCU 135180 ± 12% +22.5% 165561 ± 5% softirqs.CPU138.TIMER 134124 +16.8% 156620 ± 2% softirqs.CPU139.RCU 136993 ± 11% +24.8% 170970 softirqs.CPU139.TIMER 137051 +16.5% 159673 ± 3% softirqs.CPU14.RCU 136169 +16.7% 158881 ± 2% softirqs.CPU140.RCU 135832 +17.0% 158947 ± 2% softirqs.CPU141.RCU 133104 +17.8% 156860 softirqs.CPU142.RCU 131263 ± 13% +25.0% 164128 ± 5% softirqs.CPU142.TIMER 136069 +16.7% 158818 softirqs.CPU143.RCU 139876 ± 10% +20.5% 168600 ± 2% softirqs.CPU143.TIMER 136733 ± 2% +15.7% 158216 softirqs.CPU144.RCU 133096 ± 13% +26.0% 167646 ± 2% softirqs.CPU144.TIMER 136869 +14.9% 157216 ± 2% softirqs.CPU145.RCU 130255 ± 15% +26.3% 164490 ± 4% softirqs.CPU145.TIMER 136154 +16.0% 157934 ± 2% softirqs.CPU146.RCU 127249 ± 16% +27.2% 161846 ± 5% softirqs.CPU146.TIMER 136106 +15.7% 157496 ± 2% softirqs.CPU147.RCU 124747 ± 16% +28.9% 160778 ± 7% softirqs.CPU147.TIMER 135267 +15.0% 155538 ± 2% softirqs.CPU148.RCU 131092 ± 16% +30.0% 170485 softirqs.CPU148.TIMER 135225 +14.9% 155407 ± 3% softirqs.CPU149.RCU 133039 ± 12% +23.6% 164457 ± 5% softirqs.CPU149.TIMER 136113 +16.5% 158520 ± 2% softirqs.CPU15.RCU 135740 +15.6% 156925 ± 2% softirqs.CPU150.RCU 135689 ± 14% +25.6% 170382 softirqs.CPU150.TIMER 134770 +18.4% 159631 ± 2% softirqs.CPU151.RCU 135960 ± 2% +16.6% 158553 ± 2% softirqs.CPU152.RCU 131917 ± 14% +27.1% 167640 ± 3% softirqs.CPU152.TIMER 136105 +16.5% 158566 ± 2% softirqs.CPU153.RCU 129771 ± 16% +26.1% 163591 ± 6% softirqs.CPU153.TIMER 135048 ± 2% +17.6% 158782 softirqs.CPU154.RCU 129429 ± 14% +30.1% 168332 ± 2% softirqs.CPU154.TIMER 135523 +16.4% 157776 ± 2% softirqs.CPU155.RCU 126533 ± 16% +29.7% 164084 ± 4% softirqs.CPU155.TIMER 134974 +15.8% 156348 ± 2% softirqs.CPU156.RCU 132900 ± 13% +27.2% 169004 ± 2% softirqs.CPU156.TIMER 136170 +16.6% 158713 ± 2% softirqs.CPU157.RCU 135501 +14.5% 155128 ± 2% softirqs.CPU158.RCU 128829 ± 15% +32.3% 170469 softirqs.CPU158.TIMER 134457 ± 2% +15.7% 155605 ± 2% softirqs.CPU159.RCU 136530 ± 12% +22.2% 166897 ± 4% softirqs.CPU159.TIMER 139229 +14.9% 159981 ± 2% softirqs.CPU16.RCU 136188 +17.9% 160520 softirqs.CPU160.RCU 140773 ± 10% +20.2% 169151 ± 5% softirqs.CPU160.TIMER 135849 +16.8% 158729 ± 3% softirqs.CPU161.RCU 138370 ± 12% +23.8% 171253 ± 3% softirqs.CPU161.TIMER 136067 +16.5% 158496 softirqs.CPU162.RCU 128389 ± 13% +26.4% 162259 ± 7% softirqs.CPU162.TIMER 135252 +16.2% 157106 softirqs.CPU163.RCU 128540 ± 15% +30.7% 167959 ± 3% softirqs.CPU163.TIMER 132546 ± 2% +18.0% 156410 ± 2% softirqs.CPU164.RCU 128264 ± 15% +27.1% 163082 ± 5% softirqs.CPU164.TIMER 136482 +16.5% 159026 ± 2% softirqs.CPU165.RCU 140243 ± 12% +19.9% 168132 ± 2% softirqs.CPU165.TIMER 136260 +16.3% 158452 ± 2% softirqs.CPU166.RCU 136353 ± 2% +17.8% 160571 ± 3% softirqs.CPU167.RCU 136143 +17.0% 159330 softirqs.CPU168.RCU 135671 ± 2% +17.6% 159555 softirqs.CPU169.RCU 132921 ± 15% +24.5% 165490 ± 5% softirqs.CPU169.TIMER 136549 +16.7% 159402 ± 2% softirqs.CPU17.RCU 135557 +15.3% 156287 ± 2% softirqs.CPU170.RCU 127181 ± 15% +29.3% 164402 ± 4% softirqs.CPU170.TIMER 134990 +17.0% 157991 ± 2% softirqs.CPU171.RCU 132633 ± 15% +26.1% 167274 ± 3% softirqs.CPU171.TIMER 136983 +14.9% 157400 ± 2% softirqs.CPU172.RCU 137464 ± 2% +14.2% 156969 ± 2% softirqs.CPU173.RCU 135433 ± 13% +22.6% 165995 ± 4% softirqs.CPU173.TIMER 135835 +16.3% 157934 ± 2% softirqs.CPU174.RCU 134792 ± 13% +22.0% 164392 ± 5% softirqs.CPU174.TIMER 134619 +16.5% 156887 softirqs.CPU175.RCU 135494 ± 15% +25.3% 169744 ± 2% softirqs.CPU175.TIMER 133640 +16.4% 155593 ± 2% softirqs.CPU176.RCU 138712 ± 12% +22.0% 169161 ± 2% softirqs.CPU176.TIMER 133588 +18.1% 157754 ± 3% softirqs.CPU177.RCU 139623 ± 11% +18.8% 165853 ± 5% softirqs.CPU177.TIMER 135256 +17.3% 158703 ± 2% softirqs.CPU178.RCU 136018 +16.9% 159022 ± 2% softirqs.CPU179.RCU 132648 ± 13% +21.9% 161705 ± 7% softirqs.CPU179.TIMER 133400 ± 2% +20.0% 160027 ± 2% softirqs.CPU18.RCU 134738 ± 2% +15.7% 155843 ± 2% softirqs.CPU180.RCU 134248 ± 13% +25.1% 167915 ± 2% softirqs.CPU180.TIMER 133572 +16.6% 155811 ± 2% softirqs.CPU181.RCU 130620 ± 15% +24.9% 163149 ± 5% softirqs.CPU181.TIMER 136824 +15.4% 157928 ± 2% softirqs.CPU182.RCU 136419 ± 2% +16.1% 158371 ± 2% softirqs.CPU183.RCU 134990 +16.9% 157769 ± 2% softirqs.CPU184.RCU 134085 +17.6% 157730 ± 2% softirqs.CPU185.RCU 134334 +15.6% 155328 ± 2% softirqs.CPU186.RCU 138382 ± 12% +22.1% 168923 ± 2% softirqs.CPU186.TIMER 135548 ± 3% +15.5% 156514 ± 2% softirqs.CPU187.RCU 136563 +15.4% 157588 softirqs.CPU188.RCU 128008 ± 15% +27.8% 163621 ± 5% softirqs.CPU188.TIMER 134339 ± 3% +21.6% 163329 ± 3% softirqs.CPU189.RCU 125695 ± 14% +31.4% 165139 ± 10% softirqs.CPU189.TIMER 137171 ± 2% +17.3% 160919 ± 2% softirqs.CPU19.RCU 135309 ± 2% +16.7% 157923 ± 2% softirqs.CPU190.RCU 140465 ± 11% +19.6% 168002 ± 2% softirqs.CPU190.TIMER 135788 +15.6% 156996 ± 2% softirqs.CPU191.RCU 137224 ± 13% +19.9% 164593 ± 4% softirqs.CPU191.TIMER 134866 +16.3% 156859 ± 2% softirqs.CPU192.RCU 126223 ± 14% +29.1% 162979 ± 5% softirqs.CPU192.TIMER 134101 ± 2% +18.0% 158236 softirqs.CPU193.RCU 133525 ± 13% +25.5% 167618 ± 3% softirqs.CPU193.TIMER 135493 +16.7% 158106 ± 2% softirqs.CPU194.RCU 135519 ± 2% +19.8% 162384 ± 6% softirqs.CPU195.RCU 135219 +14.8% 155269 ± 3% softirqs.CPU196.RCU 129504 ± 16% +28.4% 166258 ± 4% softirqs.CPU196.TIMER 134542 +15.6% 155523 ± 2% softirqs.CPU197.RCU 133568 ± 14% +27.0% 169658 softirqs.CPU197.TIMER 134805 +17.6% 158501 ± 2% softirqs.CPU198.RCU 138035 ± 11% +19.6% 165157 ± 5% softirqs.CPU198.TIMER 134511 +20.9% 162557 ± 3% softirqs.CPU199.RCU 136297 ± 13% +27.6% 173880 ± 5% softirqs.CPU199.TIMER 135267 +20.0% 162370 ± 3% softirqs.CPU2.RCU 134846 ± 2% +19.2% 160770 ± 2% softirqs.CPU20.RCU 134024 +16.8% 156541 ± 2% softirqs.CPU200.RCU 134584 +17.5% 158192 ± 2% softirqs.CPU201.RCU 133058 +15.7% 153923 softirqs.CPU202.RCU 138555 ± 12% +23.5% 171065 softirqs.CPU202.TIMER 133540 +19.6% 159755 ± 6% softirqs.CPU203.RCU 135050 +16.7% 157670 ± 2% softirqs.CPU204.RCU 136644 +16.1% 158708 softirqs.CPU205.RCU 135789 ± 2% +14.1% 154977 ± 2% softirqs.CPU206.RCU 131118 ± 17% +27.8% 167525 ± 3% softirqs.CPU206.TIMER 132798 +18.1% 156889 softirqs.CPU207.RCU 129223 ± 14% +26.4% 163363 ± 5% softirqs.CPU207.TIMER 134869 +16.5% 157143 ± 2% softirqs.CPU208.RCU 125191 ± 15% +31.2% 164211 ± 5% softirqs.CPU208.TIMER 133403 +17.4% 156592 ± 2% softirqs.CPU209.RCU 128000 ± 17% +30.7% 167338 ± 3% softirqs.CPU209.TIMER 136113 +16.4% 158495 ± 2% softirqs.CPU21.RCU 135457 +17.3% 158935 ± 2% softirqs.CPU210.RCU 130469 ± 15% +24.7% 162654 ± 6% softirqs.CPU210.TIMER 134808 +16.8% 157489 ± 2% softirqs.CPU211.RCU 134350 +16.2% 156049 ± 2% softirqs.CPU212.RCU 133561 ± 15% +23.2% 164604 ± 3% softirqs.CPU212.TIMER 134039 +15.8% 155224 ± 3% softirqs.CPU213.RCU 130643 ± 15% +30.4% 170370 softirqs.CPU213.TIMER 134371 +16.1% 155960 ± 2% softirqs.CPU214.RCU 133411 +15.0% 153368 ± 2% softirqs.CPU215.RCU 132121 ± 14% +27.7% 168722 ± 2% softirqs.CPU215.TIMER 136707 +15.9% 158472 softirqs.CPU216.RCU 127939 ± 17% +26.2% 161490 ± 6% softirqs.CPU216.TIMER 135466 +16.1% 157209 ± 2% softirqs.CPU217.RCU 126617 ± 17% +28.9% 163243 ± 6% softirqs.CPU217.TIMER 135361 +16.8% 158112 softirqs.CPU218.RCU 123234 ± 16% +33.8% 164848 ± 5% softirqs.CPU218.TIMER 135629 +16.4% 157826 ± 2% softirqs.CPU219.RCU 128453 ± 17% +32.1% 169666 ± 2% softirqs.CPU219.TIMER 136444 +17.4% 160201 ± 2% softirqs.CPU22.RCU 134386 +17.5% 157935 ± 2% softirqs.CPU220.RCU 134256 +17.4% 157627 ± 2% softirqs.CPU221.RCU 134507 ± 13% +24.9% 168014 ± 2% softirqs.CPU221.TIMER 134258 +14.8% 154092 ± 3% softirqs.CPU222.RCU 129387 ± 15% +26.8% 164053 ± 4% softirqs.CPU222.TIMER 134871 +15.2% 155341 softirqs.CPU223.RCU 132127 ± 14% +28.9% 170302 softirqs.CPU223.TIMER 133327 +15.3% 153671 ± 2% softirqs.CPU224.RCU 138260 ± 10% +22.9% 169936 softirqs.CPU224.TIMER 133735 +17.1% 156650 ± 2% softirqs.CPU225.RCU 138505 ± 13% +19.0% 164777 ± 4% softirqs.CPU225.TIMER 134084 +16.7% 156487 ± 2% softirqs.CPU226.RCU 135468 +17.0% 158545 softirqs.CPU227.RCU 135038 +16.9% 157859 ± 2% softirqs.CPU228.RCU 133805 ± 15% +24.9% 167102 ± 3% softirqs.CPU228.TIMER 136331 +16.5% 158758 ± 2% softirqs.CPU229.RCU 136425 +15.4% 157392 ± 3% softirqs.CPU23.RCU 135465 +16.5% 157822 softirqs.CPU230.RCU 125011 ± 17% +30.4% 162957 ± 6% softirqs.CPU230.TIMER 136872 +16.1% 158870 softirqs.CPU231.RCU 126641 ± 16% +28.6% 162807 ± 6% softirqs.CPU231.TIMER 134721 ± 2% +14.1% 153726 ± 3% softirqs.CPU232.RCU 128484 ± 17% +30.7% 167989 ± 3% softirqs.CPU232.TIMER 134926 ± 2% +13.9% 153702 ± 2% softirqs.CPU233.RCU 131153 ± 15% +29.9% 170303 ± 2% softirqs.CPU233.TIMER 135031 +16.9% 157914 softirqs.CPU234.RCU 127507 ± 15% +27.0% 161940 ± 7% softirqs.CPU234.TIMER 134846 +13.6% 153179 softirqs.CPU235.RCU 130764 ± 13% +28.8% 168412 ± 3% softirqs.CPU235.TIMER 136784 +15.8% 158440 ± 2% softirqs.CPU236.RCU 130322 ± 16% +24.3% 162052 ± 6% softirqs.CPU236.TIMER 136928 +15.5% 158182 ± 2% softirqs.CPU237.RCU 132211 ± 15% +25.8% 166384 ± 3% softirqs.CPU237.TIMER 136159 +16.1% 158147 softirqs.CPU238.RCU 134970 +16.5% 157263 ± 2% softirqs.CPU239.RCU 136224 +17.1% 159551 ± 2% softirqs.CPU24.RCU 136102 +16.5% 158555 ± 3% softirqs.CPU240.RCU 136857 +16.8% 159808 softirqs.CPU241.RCU 134772 +15.8% 156043 ± 3% softirqs.CPU242.RCU 136618 ± 13% +20.2% 164269 ± 4% softirqs.CPU242.TIMER 133417 +17.0% 156099 softirqs.CPU243.RCU 137097 ± 2% +14.2% 156546 ± 2% softirqs.CPU244.RCU 135943 +14.0% 154955 ± 3% softirqs.CPU245.RCU 128818 ± 14% +28.5% 165496 ± 4% softirqs.CPU245.TIMER 133326 +15.1% 153409 ± 2% softirqs.CPU246.RCU 136651 ± 12% +24.1% 169589 ± 2% softirqs.CPU246.TIMER 132974 ± 2% +14.8% 152708 ± 4% softirqs.CPU247.RCU 134423 ± 13% +24.4% 167218 ± 3% softirqs.CPU247.TIMER 135542 +15.6% 156739 ± 2% softirqs.CPU248.RCU 136116 ± 12% +19.4% 162565 ± 5% softirqs.CPU248.TIMER 134289 ± 2% +16.1% 155932 ± 2% softirqs.CPU249.RCU 129577 ± 14% +24.9% 161796 ± 6% softirqs.CPU249.TIMER 136436 +16.4% 158828 ± 2% softirqs.CPU25.RCU 136072 ± 2% +15.6% 157307 ± 2% softirqs.CPU250.RCU 136091 +16.3% 158314 ± 2% softirqs.CPU251.RCU 134192 ± 2% +14.7% 153939 ± 2% softirqs.CPU252.RCU 138790 ± 11% +22.5% 170006 softirqs.CPU252.TIMER 134765 ± 2% +14.6% 154418 ± 3% softirqs.CPU253.RCU 138142 ± 12% +20.6% 166657 ± 3% softirqs.CPU253.TIMER 134656 +15.3% 155269 ± 2% softirqs.CPU254.RCU 130121 ± 14% +29.0% 167793 ± 3% softirqs.CPU254.TIMER 132564 ± 2% +16.7% 154653 ± 2% softirqs.CPU255.RCU 129547 ± 14% +28.0% 165844 ± 3% softirqs.CPU255.TIMER 134116 +16.7% 156468 ± 2% softirqs.CPU256.RCU 133251 +16.2% 154903 softirqs.CPU257.RCU 131146 ± 13% +27.4% 167140 ± 3% softirqs.CPU257.TIMER 134222 +14.7% 153907 ± 2% softirqs.CPU258.RCU 129530 ± 15% +28.9% 166935 ± 3% softirqs.CPU258.TIMER 134196 ± 2% +14.8% 154025 ± 2% softirqs.CPU259.RCU 131710 ± 15% +28.4% 169160 ± 2% softirqs.CPU259.TIMER 136342 +16.3% 158630 ± 2% softirqs.CPU26.RCU 135829 ± 2% +16.3% 157924 softirqs.CPU260.RCU 130603 ± 14% +23.0% 160651 ± 6% softirqs.CPU260.TIMER 135476 +16.6% 157943 softirqs.CPU261.RCU 132915 ± 14% +22.0% 162120 ± 4% softirqs.CPU261.TIMER 135349 +16.3% 157397 softirqs.CPU262.RCU 123981 ± 15% +32.1% 163805 ± 5% softirqs.CPU262.TIMER 135012 +16.0% 156621 ± 2% softirqs.CPU263.RCU 128218 ± 14% +28.0% 164167 ± 4% softirqs.CPU263.TIMER 135655 +15.7% 156928 softirqs.CPU264.RCU 134460 +17.4% 157832 softirqs.CPU265.RCU 132478 +18.3% 156699 softirqs.CPU266.RCU 135330 +17.6% 159124 softirqs.CPU267.RCU 133468 +17.8% 157275 ± 2% softirqs.CPU268.RCU 133616 +17.9% 157492 softirqs.CPU269.RCU 136118 +16.5% 158550 ± 2% softirqs.CPU27.RCU 135526 ± 3% +15.7% 156745 ± 2% softirqs.CPU270.RCU 133527 +17.7% 157157 softirqs.CPU271.RCU 133481 +17.9% 157379 ± 2% softirqs.CPU272.RCU 133798 +16.5% 155820 softirqs.CPU273.RCU 133916 +17.0% 156619 softirqs.CPU274.RCU 117232 ± 12% +19.4% 140000 ± 8% softirqs.CPU274.TIMER 133694 +17.4% 156961 softirqs.CPU275.RCU 134963 +17.3% 158375 softirqs.CPU276.RCU 133845 +16.5% 155972 softirqs.CPU277.RCU 134900 +17.7% 158813 softirqs.CPU278.RCU 132688 ± 2% +18.0% 156596 softirqs.CPU279.RCU 138839 ± 3% +14.5% 158925 ± 2% softirqs.CPU28.RCU 133058 +17.4% 156222 ± 2% softirqs.CPU280.RCU 133371 +17.1% 156160 softirqs.CPU281.RCU 117712 ± 12% +25.2% 147392 ± 7% softirqs.CPU281.TIMER 133917 ± 2% +17.6% 157463 softirqs.CPU282.RCU 132643 +18.5% 157180 softirqs.CPU283.RCU 135339 +16.7% 157966 softirqs.CPU284.RCU 137306 ± 5% +13.8% 156270 ± 2% softirqs.CPU285.RCU 133565 +18.5% 158324 softirqs.CPU286.RCU 118008 ± 3% -37.6% 73633 ± 27% softirqs.CPU287.RCU 135762 ± 2% +16.2% 157819 ± 3% softirqs.CPU29.RCU 121461 ± 11% +18.6% 144001 ± 11% softirqs.CPU29.TIMER 136130 +18.5% 161313 ± 3% softirqs.CPU3.RCU 136517 +15.4% 157570 ± 3% softirqs.CPU30.RCU 135672 ± 2% +16.3% 157790 ± 3% softirqs.CPU31.RCU 136093 ± 2% +16.5% 158493 ± 2% softirqs.CPU32.RCU 135819 +16.3% 157981 ± 2% softirqs.CPU33.RCU 136045 +16.6% 158651 ± 2% softirqs.CPU34.RCU 136036 +16.1% 157991 ± 3% softirqs.CPU35.RCU 136367 +15.9% 158029 ± 2% softirqs.CPU36.RCU 136415 +16.0% 158267 ± 2% softirqs.CPU37.RCU 135495 +16.9% 158383 ± 2% softirqs.CPU38.RCU 133862 ± 2% +18.4% 158462 ± 2% softirqs.CPU39.RCU 139891 ± 2% +14.8% 160621 ± 3% softirqs.CPU4.RCU 137907 ± 3% +14.1% 157330 ± 2% softirqs.CPU40.RCU 126886 ± 14% +32.9% 168585 ± 2% softirqs.CPU40.TIMER 134862 +17.4% 158289 ± 2% softirqs.CPU41.RCU 136127 +15.7% 157538 ± 2% softirqs.CPU42.RCU 137264 +15.1% 158010 ± 2% softirqs.CPU43.RCU 135564 ± 2% +16.1% 157411 ± 2% softirqs.CPU44.RCU 137153 +16.0% 159128 softirqs.CPU45.RCU 135429 +20.0% 162513 ± 3% softirqs.CPU46.RCU 137255 +16.7% 160158 softirqs.CPU47.RCU 141517 ± 6% +14.6% 162184 ± 3% softirqs.CPU48.RCU 135456 +18.3% 160229 ± 4% softirqs.CPU49.RCU 136394 +16.9% 159418 ± 3% softirqs.CPU5.RCU 135717 +16.9% 158683 ± 2% softirqs.CPU50.RCU 136054 +14.9% 156285 ± 2% softirqs.CPU51.RCU 140056 ± 4% +13.5% 159015 softirqs.CPU52.RCU 135150 +16.9% 157937 ± 2% softirqs.CPU53.RCU 121644 ± 14% +27.7% 155339 ± 9% softirqs.CPU53.TIMER 137354 ± 3% +17.2% 160971 softirqs.CPU54.RCU 137624 ± 2% +14.1% 157046 ± 2% softirqs.CPU55.RCU 127234 ± 13% +23.9% 157671 ± 8% softirqs.CPU55.TIMER 137565 ± 2% +16.5% 160319 softirqs.CPU56.RCU 135005 +16.5% 157295 softirqs.CPU57.RCU 137088 +17.0% 160329 softirqs.CPU58.RCU 135693 +15.3% 156411 ± 2% softirqs.CPU59.RCU 137401 ± 2% +15.5% 158681 ± 2% softirqs.CPU6.RCU 137779 ± 4% +15.2% 158669 ± 2% softirqs.CPU60.RCU 139142 ± 4% +13.1% 157315 ± 2% softirqs.CPU61.RCU 123269 ± 11% +26.0% 155280 ± 9% softirqs.CPU61.TIMER 135199 +17.5% 158886 ± 2% softirqs.CPU62.RCU 135140 +16.9% 158003 ± 2% softirqs.CPU63.RCU 134345 +18.1% 158697 ± 2% softirqs.CPU64.RCU 115622 ± 12% +36.6% 157953 ± 7% softirqs.CPU64.TIMER 134620 +17.9% 158739 ± 2% softirqs.CPU65.RCU 134549 +18.2% 158987 ± 2% softirqs.CPU66.RCU 135446 +16.5% 157838 ± 3% softirqs.CPU67.RCU 135023 +17.2% 158266 ± 2% softirqs.CPU68.RCU 136314 +17.5% 160203 ± 2% softirqs.CPU69.RCU 135652 +15.5% 156704 ± 4% softirqs.CPU7.RCU 134925 +15.9% 156424 ± 2% softirqs.CPU70.RCU 117081 ± 11% +30.0% 152210 ± 10% softirqs.CPU70.TIMER 136671 +16.4% 159085 softirqs.CPU71.RCU 137570 ± 4% +14.0% 156848 ± 2% softirqs.CPU72.RCU 136097 ± 3% +13.4% 154393 ± 3% softirqs.CPU73.RCU 127851 ± 14% +29.2% 165181 ± 4% softirqs.CPU73.TIMER 138146 +16.3% 160673 softirqs.CPU74.RCU 138008 +15.0% 158727 ± 2% softirqs.CPU75.RCU 134606 +14.6% 154312 ± 2% softirqs.CPU76.RCU 134161 ± 14% +24.9% 167598 ± 3% softirqs.CPU76.TIMER 134901 +14.9% 155021 ± 2% softirqs.CPU77.RCU 138735 ± 11% +23.1% 170750 ± 2% softirqs.CPU77.TIMER 136452 +16.1% 158483 ± 3% softirqs.CPU78.RCU 135411 +16.4% 157584 ± 2% softirqs.CPU79.RCU 136963 +16.4% 159359 ± 3% softirqs.CPU8.RCU 135725 +16.1% 157543 ± 2% softirqs.CPU80.RCU 128336 ± 14% +31.3% 168470 ± 4% softirqs.CPU80.TIMER 136315 +14.2% 155629 ± 2% softirqs.CPU81.RCU 126034 ± 16% +29.1% 162698 ± 7% softirqs.CPU81.TIMER 135780 +15.9% 157365 softirqs.CPU82.RCU 134289 ± 13% +28.5% 172531 ± 2% softirqs.CPU82.TIMER 136519 +14.6% 156476 ± 2% softirqs.CPU83.RCU 130685 ± 15% +29.2% 168852 ± 3% softirqs.CPU83.TIMER 137126 +16.7% 160052 ± 2% softirqs.CPU84.RCU 128777 ± 14% +24.6% 160416 ± 7% softirqs.CPU84.TIMER 138181 +16.2% 160583 ± 2% softirqs.CPU85.RCU 130075 ± 14% +24.1% 161442 ± 7% softirqs.CPU85.TIMER 136908 +15.3% 157806 ± 2% softirqs.CPU86.RCU 129723 ± 14% +27.4% 165325 ± 5% softirqs.CPU86.TIMER 136955 +16.6% 159720 ± 2% softirqs.CPU87.RCU 125579 ± 17% +32.2% 166036 ± 5% softirqs.CPU87.TIMER 136361 ± 2% +15.7% 157758 ± 2% softirqs.CPU88.RCU 136618 ± 14% +26.4% 172626 ± 2% softirqs.CPU88.TIMER 136285 +14.6% 156212 ± 4% softirqs.CPU89.RCU 124471 ± 16% +36.1% 169366 ± 3% softirqs.CPU89.TIMER 136277 +17.1% 159573 ± 2% softirqs.CPU9.RCU 133641 ± 4% +17.9% 157622 softirqs.CPU90.RCU 133829 ± 14% +27.8% 171070 ± 2% softirqs.CPU90.TIMER 136342 +15.7% 157686 ± 2% softirqs.CPU91.RCU 131999 ± 14% +27.0% 167696 ± 3% softirqs.CPU91.TIMER 135498 +17.2% 158869 ± 2% softirqs.CPU92.RCU 132999 ± 14% +24.9% 166101 ± 4% softirqs.CPU92.TIMER 137579 +15.0% 158270 ± 2% softirqs.CPU93.RCU 121635 ± 16% +35.1% 164365 ± 5% softirqs.CPU93.TIMER 137168 +16.4% 159648 ± 2% softirqs.CPU94.RCU 127455 ± 14% +26.8% 161617 ± 8% softirqs.CPU94.TIMER 137715 +15.9% 159637 ± 2% softirqs.CPU95.RCU 137684 +17.1% 161206 softirqs.CPU96.RCU 127265 ± 15% +28.4% 163372 ± 8% softirqs.CPU96.TIMER 138330 +14.7% 158682 ± 2% softirqs.CPU97.RCU 126906 ± 15% +27.7% 162036 ± 7% softirqs.CPU97.TIMER 137783 +17.8% 162361 ± 4% softirqs.CPU98.RCU 138273 ± 2% +14.7% 158552 ± 2% softirqs.CPU99.RCU 139442 ± 11% +18.6% 165309 ± 4% softirqs.CPU99.TIMER 39026939 +16.2% 45344026 ± 2% softirqs.RCU 585776 ± 8% +37.4% 804678 softirqs.SCHED 309.50 ± 16% +676.1% 2402 ±128% interrupts.31:IR-PCI-MSI.2097154-edge.eth0-TxRx-1 400773 ± 4% -14.8% 341578 interrupts.CAL:Function_call_interrupts 956.25 ± 29% -36.5% 607.25 interrupts.CPU0.CAL:Function_call_interrupts 5651 ± 32% -22.0% 4407 ± 33% interrupts.CPU0.NMI:Non-maskable_interrupts 5651 ± 32% -22.0% 4407 ± 33% interrupts.CPU0.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.4% 1192 interrupts.CPU1.CAL:Function_call_interrupts 1391 ± 4% -14.6% 1189 interrupts.CPU10.CAL:Function_call_interrupts 1250 ± 76% -63.6% 455.50 ± 56% interrupts.CPU10.RES:Rescheduling_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU100.CAL:Function_call_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU101.CAL:Function_call_interrupts 1389 ± 4% -14.6% 1186 interrupts.CPU102.CAL:Function_call_interrupts 625510 -33.7% 414949 ± 28% interrupts.CPU102.LOC:Local_timer_interrupts 4570 ± 34% -28.0% 3290 interrupts.CPU102.NMI:Non-maskable_interrupts 4570 ± 34% -28.0% 3290 interrupts.CPU102.PMI:Performance_monitoring_interrupts 49.00 ± 48% +102.6% 99.25 ± 25% interrupts.CPU102.RES:Rescheduling_interrupts 1388 ± 4% -14.5% 1186 interrupts.CPU103.CAL:Function_call_interrupts 626381 -36.3% 399047 ± 27% interrupts.CPU103.LOC:Local_timer_interrupts 3659 -10.6% 3272 interrupts.CPU103.NMI:Non-maskable_interrupts 3659 -10.6% 3272 interrupts.CPU103.PMI:Performance_monitoring_interrupts 1388 ± 4% -14.5% 1187 interrupts.CPU104.CAL:Function_call_interrupts 626082 -36.2% 399517 ± 16% interrupts.CPU104.LOC:Local_timer_interrupts 5418 ± 33% -23.7% 4136 ± 33% interrupts.CPU104.NMI:Non-maskable_interrupts 5418 ± 33% -23.7% 4136 ± 33% interrupts.CPU104.PMI:Performance_monitoring_interrupts 582.25 ±130% -88.3% 68.25 ± 19% interrupts.CPU104.RES:Rescheduling_interrupts 1388 ± 4% -14.5% 1186 interrupts.CPU105.CAL:Function_call_interrupts 625997 -32.3% 423589 ± 17% interrupts.CPU105.LOC:Local_timer_interrupts 5447 ± 33% -24.0% 4139 ± 33% interrupts.CPU105.NMI:Non-maskable_interrupts 5447 ± 33% -24.0% 4139 ± 33% interrupts.CPU105.PMI:Performance_monitoring_interrupts 1388 ± 4% -24.8% 1043 ± 23% interrupts.CPU106.CAL:Function_call_interrupts 626513 -36.2% 399917 ± 23% interrupts.CPU106.LOC:Local_timer_interrupts 5440 ± 33% -39.0% 3320 interrupts.CPU106.NMI:Non-maskable_interrupts 5440 ± 33% -39.0% 3320 interrupts.CPU106.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU107.CAL:Function_call_interrupts 626383 -32.3% 423818 ± 24% interrupts.CPU107.LOC:Local_timer_interrupts 6362 ± 24% -34.7% 4153 ± 33% interrupts.CPU107.NMI:Non-maskable_interrupts 6362 ± 24% -34.7% 4153 ± 33% interrupts.CPU107.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU108.CAL:Function_call_interrupts 625471 -43.6% 353037 ± 9% interrupts.CPU108.LOC:Local_timer_interrupts 5414 ± 32% -26.6% 3973 ± 32% interrupts.CPU108.NMI:Non-maskable_interrupts 5414 ± 32% -26.6% 3973 ± 32% interrupts.CPU108.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU109.CAL:Function_call_interrupts 626077 -26.1% 462414 ± 21% interrupts.CPU109.LOC:Local_timer_interrupts 309.50 ± 16% +676.1% 2402 ±128% interrupts.CPU11.31:IR-PCI-MSI.2097154-edge.eth0-TxRx-1 1392 ± 4% -14.6% 1189 interrupts.CPU11.CAL:Function_call_interrupts 1388 ± 4% -14.5% 1187 interrupts.CPU110.CAL:Function_call_interrupts 626218 -41.5% 366233 ± 16% interrupts.CPU110.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU111.CAL:Function_call_interrupts 626244 -37.2% 393169 ± 16% interrupts.CPU111.LOC:Local_timer_interrupts 1388 ± 4% -14.5% 1187 interrupts.CPU112.CAL:Function_call_interrupts 626280 -32.8% 421029 ± 26% interrupts.CPU112.LOC:Local_timer_interrupts 6348 ± 24% -35.0% 4123 ± 33% interrupts.CPU112.NMI:Non-maskable_interrupts 6348 ± 24% -35.0% 4123 ± 33% interrupts.CPU112.PMI:Performance_monitoring_interrupts 1390 ± 5% -14.6% 1187 interrupts.CPU113.CAL:Function_call_interrupts 626445 -38.9% 383002 ± 22% interrupts.CPU113.LOC:Local_timer_interrupts 5446 ± 33% -41.4% 3193 ± 2% interrupts.CPU113.NMI:Non-maskable_interrupts 5446 ± 33% -41.4% 3193 ± 2% interrupts.CPU113.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU114.CAL:Function_call_interrupts 5529 ± 33% -23.8% 4212 ± 34% interrupts.CPU114.NMI:Non-maskable_interrupts 5529 ± 33% -23.8% 4212 ± 34% interrupts.CPU114.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU115.CAL:Function_call_interrupts 6491 ± 25% -50.1% 3237 ± 6% interrupts.CPU115.NMI:Non-maskable_interrupts 6491 ± 25% -50.1% 3237 ± 6% interrupts.CPU115.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU116.CAL:Function_call_interrupts 626282 -35.1% 406165 ± 19% interrupts.CPU116.LOC:Local_timer_interrupts 1390 ± 4% -14.6% 1187 interrupts.CPU117.CAL:Function_call_interrupts 626266 -37.6% 390945 ± 20% interrupts.CPU117.LOC:Local_timer_interrupts 1389 ± 4% -14.3% 1190 interrupts.CPU118.CAL:Function_call_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU119.CAL:Function_call_interrupts 1393 ± 4% -14.6% 1189 interrupts.CPU12.CAL:Function_call_interrupts 5665 ± 33% -21.7% 4433 ± 34% interrupts.CPU12.NMI:Non-maskable_interrupts 5665 ± 33% -21.7% 4433 ± 34% interrupts.CPU12.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU120.CAL:Function_call_interrupts 626172 -35.5% 403682 ± 11% interrupts.CPU120.LOC:Local_timer_interrupts 1388 ± 5% -14.5% 1187 interrupts.CPU121.CAL:Function_call_interrupts 625597 -44.3% 348336 ± 8% interrupts.CPU121.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1188 interrupts.CPU122.CAL:Function_call_interrupts 626247 -33.1% 419247 ± 25% interrupts.CPU122.LOC:Local_timer_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU123.CAL:Function_call_interrupts 626315 -41.1% 369075 ± 17% interrupts.CPU123.LOC:Local_timer_interrupts 6344 ± 24% -49.3% 3217 interrupts.CPU123.NMI:Non-maskable_interrupts 6344 ± 24% -49.3% 3217 interrupts.CPU123.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.7% 1187 interrupts.CPU124.CAL:Function_call_interrupts 1389 ± 4% -14.5% 1188 interrupts.CPU125.CAL:Function_call_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU127.CAL:Function_call_interrupts 6359 ± 25% -34.4% 4172 ± 34% interrupts.CPU127.NMI:Non-maskable_interrupts 6359 ± 25% -34.4% 4172 ± 34% interrupts.CPU127.PMI:Performance_monitoring_interrupts 1390 ± 4% -14.5% 1188 interrupts.CPU128.CAL:Function_call_interrupts 626232 -41.4% 366829 ± 13% interrupts.CPU128.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1188 interrupts.CPU129.CAL:Function_call_interrupts 626275 -37.8% 389452 ± 15% interrupts.CPU129.LOC:Local_timer_interrupts 5421 ± 33% -23.4% 4153 ± 35% interrupts.CPU129.NMI:Non-maskable_interrupts 5421 ± 33% -23.4% 4153 ± 35% interrupts.CPU129.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.5% 1190 interrupts.CPU13.CAL:Function_call_interrupts 5644 ± 33% -20.8% 4470 ± 35% interrupts.CPU13.NMI:Non-maskable_interrupts 5644 ± 33% -20.8% 4470 ± 35% interrupts.CPU13.PMI:Performance_monitoring_interrupts 1388 ± 4% -14.4% 1188 interrupts.CPU130.CAL:Function_call_interrupts 626230 -31.7% 427973 ± 24% interrupts.CPU130.LOC:Local_timer_interrupts 34.00 ± 29% +239.7% 115.50 ± 33% interrupts.CPU130.RES:Rescheduling_interrupts 1388 ± 4% -14.4% 1187 interrupts.CPU131.CAL:Function_call_interrupts 626315 -31.8% 427416 ± 25% interrupts.CPU131.LOC:Local_timer_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU132.CAL:Function_call_interrupts 625496 -43.6% 352695 ± 10% interrupts.CPU132.LOC:Local_timer_interrupts 1385 ± 4% -14.3% 1187 interrupts.CPU133.CAL:Function_call_interrupts 626314 -40.3% 373915 ± 9% interrupts.CPU133.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU134.CAL:Function_call_interrupts 626243 -38.3% 386083 ± 17% interrupts.CPU134.LOC:Local_timer_interrupts 5510 ± 33% -26.6% 4045 ± 34% interrupts.CPU134.NMI:Non-maskable_interrupts 5510 ± 33% -26.6% 4045 ± 34% interrupts.CPU134.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU135.CAL:Function_call_interrupts 626330 -24.0% 476296 ± 18% interrupts.CPU135.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1188 interrupts.CPU136.CAL:Function_call_interrupts 625268 -43.6% 352820 ± 7% interrupts.CPU136.LOC:Local_timer_interrupts 5565 ± 33% -30.6% 3862 ± 30% interrupts.CPU136.NMI:Non-maskable_interrupts 5565 ± 33% -30.6% 3862 ± 30% interrupts.CPU136.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU137.CAL:Function_call_interrupts 3689 -9.0% 3357 interrupts.CPU137.NMI:Non-maskable_interrupts 3689 -9.0% 3357 interrupts.CPU137.PMI:Performance_monitoring_interrupts 16.25 ± 42% +269.2% 60.00 ± 79% interrupts.CPU137.RES:Rescheduling_interrupts 1383 ± 4% -14.2% 1187 interrupts.CPU138.CAL:Function_call_interrupts 625452 -32.3% 423598 ± 24% interrupts.CPU138.LOC:Local_timer_interrupts 3675 -10.4% 3293 interrupts.CPU138.NMI:Non-maskable_interrupts 3675 -10.4% 3293 interrupts.CPU138.PMI:Performance_monitoring_interrupts 1382 ± 4% -14.1% 1187 interrupts.CPU139.CAL:Function_call_interrupts 626086 -43.0% 357090 ± 11% interrupts.CPU139.LOC:Local_timer_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU14.CAL:Function_call_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU140.CAL:Function_call_interrupts 1390 ± 4% -14.6% 1187 interrupts.CPU141.CAL:Function_call_interrupts 5347 ± 33% -36.9% 3373 interrupts.CPU141.NMI:Non-maskable_interrupts 5347 ± 33% -36.9% 3373 interrupts.CPU141.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.3% 1188 interrupts.CPU142.CAL:Function_call_interrupts 625813 -34.7% 408791 ± 22% interrupts.CPU142.LOC:Local_timer_interrupts 1386 ± 4% -14.3% 1188 interrupts.CPU143.CAL:Function_call_interrupts 626244 -41.9% 363876 ± 11% interrupts.CPU143.LOC:Local_timer_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU144.CAL:Function_call_interrupts 626512 -41.4% 367098 ± 11% interrupts.CPU144.LOC:Local_timer_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU145.CAL:Function_call_interrupts 626214 -32.4% 423255 ± 24% interrupts.CPU145.LOC:Local_timer_interrupts 1390 ± 4% -14.6% 1188 interrupts.CPU146.CAL:Function_call_interrupts 626107 -37.5% 391390 ± 13% interrupts.CPU146.LOC:Local_timer_interrupts 5388 ± 33% -24.3% 4078 ± 35% interrupts.CPU146.NMI:Non-maskable_interrupts 5388 ± 33% -24.3% 4078 ± 35% interrupts.CPU146.PMI:Performance_monitoring_interrupts 1398 ± 3% -15.1% 1187 interrupts.CPU147.CAL:Function_call_interrupts 626274 -32.5% 422994 ± 19% interrupts.CPU147.LOC:Local_timer_interrupts 5449 ± 33% -39.2% 3313 interrupts.CPU147.NMI:Non-maskable_interrupts 5449 ± 33% -39.2% 3313 interrupts.CPU147.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.6% 1187 interrupts.CPU148.CAL:Function_call_interrupts 626256 -42.1% 362911 ± 14% interrupts.CPU148.LOC:Local_timer_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU149.CAL:Function_call_interrupts 626077 -41.2% 368377 ± 6% interrupts.CPU149.LOC:Local_timer_interrupts 3683 -11.6% 3254 ± 4% interrupts.CPU149.NMI:Non-maskable_interrupts 3683 -11.6% 3254 ± 4% interrupts.CPU149.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.6% 1189 interrupts.CPU15.CAL:Function_call_interrupts 1391 ± 4% -14.6% 1187 interrupts.CPU150.CAL:Function_call_interrupts 626327 -43.3% 355179 ± 11% interrupts.CPU150.LOC:Local_timer_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU151.CAL:Function_call_interrupts 626205 -27.7% 452981 ± 23% interrupts.CPU151.LOC:Local_timer_interrupts 23.25 ± 67% +338.7% 102.00 ± 89% interrupts.CPU151.RES:Rescheduling_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU152.CAL:Function_call_interrupts 626058 -32.7% 421336 ± 28% interrupts.CPU152.LOC:Local_timer_interrupts 18.50 ± 18% +279.7% 70.25 ± 79% interrupts.CPU152.RES:Rescheduling_interrupts 1387 ± 4% -14.4% 1188 interrupts.CPU153.CAL:Function_call_interrupts 626116 -29.3% 442396 ± 23% interrupts.CPU153.LOC:Local_timer_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU154.CAL:Function_call_interrupts 626003 -42.6% 359610 ± 7% interrupts.CPU154.LOC:Local_timer_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU155.CAL:Function_call_interrupts 626214 -36.3% 398719 ± 16% interrupts.CPU155.LOC:Local_timer_interrupts 1390 ± 4% -14.5% 1188 interrupts.CPU156.CAL:Function_call_interrupts 626320 -44.4% 348144 ± 7% interrupts.CPU156.LOC:Local_timer_interrupts 1391 ± 4% -14.6% 1187 interrupts.CPU157.CAL:Function_call_interrupts 626830 -25.2% 469011 ± 21% interrupts.CPU157.LOC:Local_timer_interrupts 1392 ± 4% -14.6% 1188 interrupts.CPU158.CAL:Function_call_interrupts 625891 -43.6% 353089 ± 10% interrupts.CPU158.LOC:Local_timer_interrupts 1392 ± 4% -14.7% 1188 interrupts.CPU159.CAL:Function_call_interrupts 626439 -41.5% 366414 ± 7% interrupts.CPU159.LOC:Local_timer_interrupts 1389 ± 5% -14.5% 1187 interrupts.CPU16.CAL:Function_call_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU160.CAL:Function_call_interrupts 626471 -32.9% 420443 ± 24% interrupts.CPU160.LOC:Local_timer_interrupts 1391 ± 4% -14.7% 1187 interrupts.CPU161.CAL:Function_call_interrupts 626412 -43.4% 354657 ± 10% interrupts.CPU161.LOC:Local_timer_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU162.CAL:Function_call_interrupts 626281 -36.8% 395674 ± 10% interrupts.CPU162.LOC:Local_timer_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU163.CAL:Function_call_interrupts 626172 -43.4% 354152 ± 10% interrupts.CPU163.LOC:Local_timer_interrupts 626617 -30.3% 436801 ± 25% interrupts.CPU164.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU165.CAL:Function_call_interrupts 626511 -34.1% 412650 ± 26% interrupts.CPU165.LOC:Local_timer_interrupts 1390 ± 4% -14.5% 1189 interrupts.CPU166.CAL:Function_call_interrupts 1392 ± 4% -14.8% 1187 interrupts.CPU167.CAL:Function_call_interrupts 5408 ± 33% -22.1% 4211 ± 33% interrupts.CPU167.NMI:Non-maskable_interrupts 5408 ± 33% -22.1% 4211 ± 33% interrupts.CPU167.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.8% 1187 interrupts.CPU168.CAL:Function_call_interrupts 625522 -29.5% 441026 ± 22% interrupts.CPU168.LOC:Local_timer_interrupts 5467 ± 32% -23.5% 4182 ± 35% interrupts.CPU168.NMI:Non-maskable_interrupts 5467 ± 32% -23.5% 4182 ± 35% interrupts.CPU168.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU169.CAL:Function_call_interrupts 626238 -36.6% 396971 ± 16% interrupts.CPU169.LOC:Local_timer_interrupts 1387 ± 5% -14.4% 1188 interrupts.CPU17.CAL:Function_call_interrupts 5668 ± 33% -35.5% 3655 interrupts.CPU17.NMI:Non-maskable_interrupts 5668 ± 33% -35.5% 3655 interrupts.CPU17.PMI:Performance_monitoring_interrupts 1390 ± 4% -14.4% 1189 interrupts.CPU170.CAL:Function_call_interrupts 626211 -33.9% 414185 ± 20% interrupts.CPU170.LOC:Local_timer_interrupts 1387 ± 4% -14.5% 1187 interrupts.CPU171.CAL:Function_call_interrupts 626212 -38.8% 383154 ± 15% interrupts.CPU171.LOC:Local_timer_interrupts 1390 ± 4% -14.6% 1187 interrupts.CPU172.CAL:Function_call_interrupts 626232 -39.3% 379988 ± 8% interrupts.CPU172.LOC:Local_timer_interrupts 1393 ± 4% -14.7% 1187 interrupts.CPU173.CAL:Function_call_interrupts 626682 -42.7% 359247 ± 10% interrupts.CPU173.LOC:Local_timer_interrupts 4609 ± 35% -30.3% 3210 interrupts.CPU173.NMI:Non-maskable_interrupts 4609 ± 35% -30.3% 3210 interrupts.CPU173.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.4% 1192 interrupts.CPU174.CAL:Function_call_interrupts 624983 -38.8% 382288 ± 12% interrupts.CPU174.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU175.CAL:Function_call_interrupts 626347 -43.4% 354602 ± 10% interrupts.CPU175.LOC:Local_timer_interrupts 1390 ± 5% -14.6% 1187 interrupts.CPU176.CAL:Function_call_interrupts 626075 -44.5% 347497 ± 6% interrupts.CPU176.LOC:Local_timer_interrupts 4602 ± 35% -31.9% 3136 ± 4% interrupts.CPU176.NMI:Non-maskable_interrupts 4602 ± 35% -31.9% 3136 ± 4% interrupts.CPU176.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU177.CAL:Function_call_interrupts 625189 -40.2% 373605 ± 9% interrupts.CPU177.LOC:Local_timer_interrupts 5503 ± 33% -26.3% 4054 ± 31% interrupts.CPU177.NMI:Non-maskable_interrupts 5503 ± 33% -26.3% 4054 ± 31% interrupts.CPU177.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU178.CAL:Function_call_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU179.CAL:Function_call_interrupts 626004 -23.9% 476264 ± 21% interrupts.CPU179.LOC:Local_timer_interrupts 3722 -9.5% 3370 interrupts.CPU179.NMI:Non-maskable_interrupts 3722 -9.5% 3370 interrupts.CPU179.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU18.CAL:Function_call_interrupts 496.50 ± 40% -46.6% 265.00 ± 29% interrupts.CPU18.RES:Rescheduling_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU180.CAL:Function_call_interrupts 625723 -36.4% 397718 ± 21% interrupts.CPU180.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU181.CAL:Function_call_interrupts 6367 ± 25% -46.9% 3379 interrupts.CPU181.NMI:Non-maskable_interrupts 6367 ± 25% -46.9% 3379 interrupts.CPU181.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU182.CAL:Function_call_interrupts 5522 ± 32% -23.5% 4224 ± 34% interrupts.CPU182.NMI:Non-maskable_interrupts 5522 ± 32% -23.5% 4224 ± 34% interrupts.CPU182.PMI:Performance_monitoring_interrupts 1394 ± 4% -14.8% 1188 interrupts.CPU183.CAL:Function_call_interrupts 5578 ± 33% -39.5% 3377 interrupts.CPU183.NMI:Non-maskable_interrupts 5578 ± 33% -39.5% 3377 interrupts.CPU183.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU184.CAL:Function_call_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU185.CAL:Function_call_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU186.CAL:Function_call_interrupts 625833 -36.4% 398159 ± 26% interrupts.CPU186.LOC:Local_timer_interrupts 21.50 ± 6% +408.1% 109.25 ± 94% interrupts.CPU186.RES:Rescheduling_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU187.CAL:Function_call_interrupts 626133 -32.4% 423112 ± 22% interrupts.CPU187.LOC:Local_timer_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU188.CAL:Function_call_interrupts 626244 -39.3% 380057 ± 10% interrupts.CPU188.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU189.CAL:Function_call_interrupts 626320 -28.8% 445867 ± 18% interrupts.CPU189.LOC:Local_timer_interrupts 5550 ± 33% -40.8% 3288 interrupts.CPU189.NMI:Non-maskable_interrupts 5550 ± 33% -40.8% 3288 interrupts.CPU189.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.5% 1190 interrupts.CPU19.CAL:Function_call_interrupts 5683 ± 33% -19.6% 4569 ± 35% interrupts.CPU19.NMI:Non-maskable_interrupts 5683 ± 33% -19.6% 4569 ± 35% interrupts.CPU19.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU190.CAL:Function_call_interrupts 625375 -36.0% 400222 ± 24% interrupts.CPU190.LOC:Local_timer_interrupts 6322 ± 25% -35.6% 4069 ± 34% interrupts.CPU190.NMI:Non-maskable_interrupts 6322 ± 25% -35.6% 4069 ± 34% interrupts.CPU190.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.8% 1187 interrupts.CPU191.CAL:Function_call_interrupts 1389 ± 5% -14.5% 1187 interrupts.CPU192.CAL:Function_call_interrupts 626193 -33.9% 414105 ± 22% interrupts.CPU192.LOC:Local_timer_interrupts 69.00 ± 58% +185.1% 196.75 ± 77% interrupts.CPU192.RES:Rescheduling_interrupts 1390 ± 4% -14.6% 1187 interrupts.CPU193.CAL:Function_call_interrupts 626463 -39.6% 378686 ± 14% interrupts.CPU193.LOC:Local_timer_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU194.CAL:Function_call_interrupts 626205 -26.8% 458694 ± 22% interrupts.CPU194.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU195.CAL:Function_call_interrupts 626559 -31.2% 431084 ± 24% interrupts.CPU195.LOC:Local_timer_interrupts 5433 ± 33% -22.5% 4213 ± 34% interrupts.CPU195.NMI:Non-maskable_interrupts 5433 ± 33% -22.5% 4213 ± 34% interrupts.CPU195.PMI:Performance_monitoring_interrupts 1394 ± 4% -14.8% 1187 interrupts.CPU196.CAL:Function_call_interrupts 626261 -43.1% 356508 ± 6% interrupts.CPU196.LOC:Local_timer_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU197.CAL:Function_call_interrupts 626366 -45.4% 342272 ± 5% interrupts.CPU197.LOC:Local_timer_interrupts 1389 ± 4% -14.6% 1187 interrupts.CPU198.CAL:Function_call_interrupts 626231 -31.4% 429407 ± 21% interrupts.CPU198.LOC:Local_timer_interrupts 1395 ± 4% -14.8% 1188 interrupts.CPU199.CAL:Function_call_interrupts 626473 -39.8% 377303 ± 17% interrupts.CPU199.LOC:Local_timer_interrupts 3710 +77.8% 6597 ± 2% interrupts.CPU199.NMI:Non-maskable_interrupts 3710 +77.8% 6597 ± 2% interrupts.CPU199.PMI:Performance_monitoring_interrupts 1394 ± 4% -14.6% 1190 interrupts.CPU2.CAL:Function_call_interrupts 6574 ± 24% -31.9% 4477 ± 33% interrupts.CPU2.NMI:Non-maskable_interrupts 6574 ± 24% -31.9% 4477 ± 33% interrupts.CPU2.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.5% 1189 interrupts.CPU20.CAL:Function_call_interrupts 5690 ± 33% -22.2% 4425 ± 34% interrupts.CPU20.NMI:Non-maskable_interrupts 5690 ± 33% -22.2% 4425 ± 34% interrupts.CPU20.PMI:Performance_monitoring_interrupts 615.50 ± 20% -41.8% 358.50 ± 33% interrupts.CPU20.RES:Rescheduling_interrupts 1392 ± 4% -14.7% 1188 interrupts.CPU200.CAL:Function_call_interrupts 1393 ± 4% -14.7% 1188 interrupts.CPU201.CAL:Function_call_interrupts 4614 ± 34% -27.3% 3352 interrupts.CPU201.NMI:Non-maskable_interrupts 4614 ± 34% -27.3% 3352 interrupts.CPU201.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.7% 1187 interrupts.CPU202.CAL:Function_call_interrupts 626282 -43.7% 352487 ± 10% interrupts.CPU202.LOC:Local_timer_interrupts 3629 +70.3% 6179 interrupts.CPU202.NMI:Non-maskable_interrupts 3629 +70.3% 6179 interrupts.CPU202.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.7% 1188 interrupts.CPU203.CAL:Function_call_interrupts 626378 -37.1% 393719 ± 15% interrupts.CPU203.LOC:Local_timer_interrupts 5446 ± 34% -40.4% 3248 interrupts.CPU203.NMI:Non-maskable_interrupts 5446 ± 34% -40.4% 3248 interrupts.CPU203.PMI:Performance_monitoring_interrupts 1393 ± 4% -14.8% 1187 interrupts.CPU204.CAL:Function_call_interrupts 5567 ± 33% -23.8% 4241 ± 34% interrupts.CPU204.NMI:Non-maskable_interrupts 5567 ± 33% -23.8% 4241 ± 34% interrupts.CPU204.PMI:Performance_monitoring_interrupts 1395 ± 4% -14.9% 1187 interrupts.CPU205.CAL:Function_call_interrupts 5524 ± 33% -24.4% 4178 ± 34% interrupts.CPU205.NMI:Non-maskable_interrupts 5524 ± 33% -24.4% 4178 ± 34% interrupts.CPU205.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.7% 1188 interrupts.CPU206.CAL:Function_call_interrupts 626053 -33.4% 416932 ± 29% interrupts.CPU206.LOC:Local_timer_interrupts 1392 ± 4% -14.5% 1190 interrupts.CPU207.CAL:Function_call_interrupts 626396 -31.4% 429857 ± 24% interrupts.CPU207.LOC:Local_timer_interrupts 1394 ± 4% -14.8% 1187 interrupts.CPU208.CAL:Function_call_interrupts 625691 -31.8% 426743 ± 23% interrupts.CPU208.LOC:Local_timer_interrupts 1394 ± 4% -14.3% 1195 interrupts.CPU209.CAL:Function_call_interrupts 625876 -37.0% 394044 ± 18% interrupts.CPU209.LOC:Local_timer_interrupts 4552 ± 35% -27.3% 3307 interrupts.CPU209.NMI:Non-maskable_interrupts 4552 ± 35% -27.3% 3307 interrupts.CPU209.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.7% 1188 interrupts.CPU21.CAL:Function_call_interrupts 5408 ± 32% -37.6% 3372 interrupts.CPU210.NMI:Non-maskable_interrupts 5408 ± 32% -37.6% 3372 interrupts.CPU210.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.5% 1187 interrupts.CPU211.CAL:Function_call_interrupts 626151 -30.1% 437371 ± 22% interrupts.CPU211.LOC:Local_timer_interrupts 1394 ± 4% -14.8% 1188 interrupts.CPU212.CAL:Function_call_interrupts 625782 -39.5% 378414 ± 11% interrupts.CPU212.LOC:Local_timer_interrupts 5431 ± 33% -26.1% 4014 ± 33% interrupts.CPU212.NMI:Non-maskable_interrupts 5431 ± 33% -26.1% 4014 ± 33% interrupts.CPU212.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.7% 1187 interrupts.CPU213.CAL:Function_call_interrupts 626276 -43.9% 351178 ± 9% interrupts.CPU213.LOC:Local_timer_interrupts 1390 ± 4% -14.5% 1188 interrupts.CPU214.CAL:Function_call_interrupts 625725 -37.9% 388318 ± 11% interrupts.CPU214.LOC:Local_timer_interrupts 1394 ± 4% -14.7% 1189 interrupts.CPU215.CAL:Function_call_interrupts 626305 -44.8% 345769 ± 6% interrupts.CPU215.LOC:Local_timer_interrupts 4373 ± 34% -30.6% 3036 interrupts.CPU215.NMI:Non-maskable_interrupts 4373 ± 34% -30.6% 3036 interrupts.CPU215.PMI:Performance_monitoring_interrupts 1394 ± 4% -14.8% 1187 interrupts.CPU216.CAL:Function_call_interrupts 626187 -28.4% 448139 ± 24% interrupts.CPU216.LOC:Local_timer_interrupts 1394 ± 4% -14.8% 1187 interrupts.CPU217.CAL:Function_call_interrupts 626232 -31.3% 430254 ± 23% interrupts.CPU217.LOC:Local_timer_interrupts 5516 ± 33% -23.9% 4197 ± 34% interrupts.CPU217.NMI:Non-maskable_interrupts 5516 ± 33% -23.9% 4197 ± 34% interrupts.CPU217.PMI:Performance_monitoring_interrupts 1394 ± 4% -14.8% 1188 interrupts.CPU218.CAL:Function_call_interrupts 626092 -35.2% 406015 ± 20% interrupts.CPU218.LOC:Local_timer_interrupts 1393 ± 4% -14.7% 1187 interrupts.CPU219.CAL:Function_call_interrupts 626430 -37.0% 394469 ± 22% interrupts.CPU219.LOC:Local_timer_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU22.CAL:Function_call_interrupts 1394 ± 4% -14.8% 1188 interrupts.CPU220.CAL:Function_call_interrupts 625982 -26.7% 459012 ± 22% interrupts.CPU220.LOC:Local_timer_interrupts 1396 ± 4% -14.9% 1188 interrupts.CPU221.CAL:Function_call_interrupts 626227 -35.9% 401325 ± 24% interrupts.CPU221.LOC:Local_timer_interrupts 1401 ± 5% -15.1% 1189 interrupts.CPU222.CAL:Function_call_interrupts 626245 -39.0% 381742 ± 12% interrupts.CPU222.LOC:Local_timer_interrupts 3602 -10.2% 3236 interrupts.CPU222.NMI:Non-maskable_interrupts 3602 -10.2% 3236 interrupts.CPU222.PMI:Performance_monitoring_interrupts 1405 ± 4% -15.4% 1188 interrupts.CPU223.CAL:Function_call_interrupts 626166 -43.7% 352624 ± 10% interrupts.CPU223.LOC:Local_timer_interrupts 1410 ± 4% -15.7% 1189 interrupts.CPU224.CAL:Function_call_interrupts 626412 -35.5% 404159 ± 29% interrupts.CPU224.LOC:Local_timer_interrupts 1409 ± 4% -15.7% 1188 interrupts.CPU225.CAL:Function_call_interrupts 626501 -32.0% 425753 ± 25% interrupts.CPU225.LOC:Local_timer_interrupts 5434 ± 33% -24.6% 4097 ± 35% interrupts.CPU225.NMI:Non-maskable_interrupts 5434 ± 33% -24.6% 4097 ± 35% interrupts.CPU225.PMI:Performance_monitoring_interrupts 1409 ± 4% -15.6% 1188 interrupts.CPU226.CAL:Function_call_interrupts 42.75 ± 49% +398.2% 213.00 ±103% interrupts.CPU226.RES:Rescheduling_interrupts 1409 ± 4% -15.7% 1188 interrupts.CPU227.CAL:Function_call_interrupts 1409 ± 4% -15.7% 1187 interrupts.CPU228.CAL:Function_call_interrupts 626375 -34.9% 407838 ± 27% interrupts.CPU228.LOC:Local_timer_interrupts 1410 ± 4% -15.7% 1188 interrupts.CPU229.CAL:Function_call_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU23.CAL:Function_call_interrupts 4715 ± 35% -27.4% 3421 interrupts.CPU23.NMI:Non-maskable_interrupts 4715 ± 35% -27.4% 3421 interrupts.CPU23.PMI:Performance_monitoring_interrupts 399.25 ± 48% -63.0% 147.75 ± 65% interrupts.CPU23.RES:Rescheduling_interrupts 1410 ± 4% -15.7% 1188 interrupts.CPU230.CAL:Function_call_interrupts 625778 -33.4% 416531 ± 18% interrupts.CPU230.LOC:Local_timer_interrupts 1409 ± 4% -15.7% 1188 interrupts.CPU231.CAL:Function_call_interrupts 626395 -27.9% 451655 ± 20% interrupts.CPU231.LOC:Local_timer_interrupts 11.25 ± 31% +291.1% 44.00 ± 80% interrupts.CPU231.RES:Rescheduling_interrupts 1408 ± 4% -15.5% 1190 interrupts.CPU232.CAL:Function_call_interrupts 626163 -42.7% 359003 ± 9% interrupts.CPU232.LOC:Local_timer_interrupts 1407 ± 4% -15.5% 1188 interrupts.CPU233.CAL:Function_call_interrupts 626101 -43.1% 356548 ± 11% interrupts.CPU233.LOC:Local_timer_interrupts 6389 ± 25% -49.4% 3234 ± 2% interrupts.CPU233.NMI:Non-maskable_interrupts 6389 ± 25% -49.4% 3234 ± 2% interrupts.CPU233.PMI:Performance_monitoring_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU234.CAL:Function_call_interrupts 626094 -31.6% 428493 ± 21% interrupts.CPU234.LOC:Local_timer_interrupts 1409 ± 4% -15.6% 1189 interrupts.CPU235.CAL:Function_call_interrupts 626309 -45.1% 343921 ± 5% interrupts.CPU235.LOC:Local_timer_interrupts 17.00 ± 20% +260.3% 61.25 ± 72% interrupts.CPU235.RES:Rescheduling_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU236.CAL:Function_call_interrupts 626322 -31.4% 429411 ± 22% interrupts.CPU236.LOC:Local_timer_interrupts 1410 ± 4% -15.8% 1187 interrupts.CPU237.CAL:Function_call_interrupts 626186 -35.4% 404790 ± 21% interrupts.CPU237.LOC:Local_timer_interrupts 5455 ± 33% -39.1% 3323 interrupts.CPU237.NMI:Non-maskable_interrupts 5455 ± 33% -39.1% 3323 interrupts.CPU237.PMI:Performance_monitoring_interrupts 1410 ± 4% -15.7% 1189 interrupts.CPU238.CAL:Function_call_interrupts 626483 -33.9% 414019 ± 16% interrupts.CPU238.LOC:Local_timer_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU239.CAL:Function_call_interrupts 626281 -31.6% 428086 ± 19% interrupts.CPU239.LOC:Local_timer_interrupts 1392 ± 4% -14.6% 1189 interrupts.CPU24.CAL:Function_call_interrupts 720.50 ± 86% -62.4% 271.25 ± 44% interrupts.CPU24.RES:Rescheduling_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU240.CAL:Function_call_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU241.CAL:Function_call_interrupts 3696 -8.7% 3373 interrupts.CPU241.NMI:Non-maskable_interrupts 3696 -8.7% 3373 interrupts.CPU241.PMI:Performance_monitoring_interrupts 1405 ± 4% -15.4% 1188 interrupts.CPU242.CAL:Function_call_interrupts 626326 -36.7% 396466 ± 19% interrupts.CPU242.LOC:Local_timer_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU243.CAL:Function_call_interrupts 626233 -34.9% 407710 ± 17% interrupts.CPU243.LOC:Local_timer_interrupts 3659 -9.7% 3303 interrupts.CPU243.NMI:Non-maskable_interrupts 3659 -9.7% 3303 interrupts.CPU243.PMI:Performance_monitoring_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU244.CAL:Function_call_interrupts 5567 ± 33% -25.6% 4143 ± 35% interrupts.CPU244.NMI:Non-maskable_interrupts 5567 ± 33% -25.6% 4143 ± 35% interrupts.CPU244.PMI:Performance_monitoring_interrupts 1411 ± 4% -15.9% 1187 interrupts.CPU245.CAL:Function_call_interrupts 626606 -38.0% 388502 ± 15% interrupts.CPU245.LOC:Local_timer_interrupts 4639 ± 35% -30.2% 3236 interrupts.CPU245.NMI:Non-maskable_interrupts 4639 ± 35% -30.2% 3236 interrupts.CPU245.PMI:Performance_monitoring_interrupts 1411 ± 4% -15.8% 1188 interrupts.CPU246.CAL:Function_call_interrupts 625290 -41.8% 363817 ± 15% interrupts.CPU246.LOC:Local_timer_interrupts 119.50 ± 53% -63.8% 43.25 ± 57% interrupts.CPU246.RES:Rescheduling_interrupts 1410 ± 4% -15.8% 1187 interrupts.CPU247.CAL:Function_call_interrupts 6427 ± 24% -36.3% 4092 ± 34% interrupts.CPU247.NMI:Non-maskable_interrupts 6427 ± 24% -36.3% 4092 ± 34% interrupts.CPU247.PMI:Performance_monitoring_interrupts 1410 ± 4% -15.8% 1187 interrupts.CPU248.CAL:Function_call_interrupts 625767 -30.7% 433663 ± 25% interrupts.CPU248.LOC:Local_timer_interrupts 5441 ± 33% -38.4% 3351 interrupts.CPU248.NMI:Non-maskable_interrupts 5441 ± 33% -38.4% 3351 interrupts.CPU248.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.8% 1188 interrupts.CPU249.CAL:Function_call_interrupts 625347 -28.9% 444798 ± 23% interrupts.CPU249.LOC:Local_timer_interrupts 5481 ± 33% -23.2% 4211 ± 34% interrupts.CPU249.NMI:Non-maskable_interrupts 5481 ± 33% -23.2% 4211 ± 34% interrupts.CPU249.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU25.CAL:Function_call_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU250.CAL:Function_call_interrupts 5434 ± 32% -22.7% 4202 ± 33% interrupts.CPU250.NMI:Non-maskable_interrupts 5434 ± 32% -22.7% 4202 ± 33% interrupts.CPU250.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU251.CAL:Function_call_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU252.CAL:Function_call_interrupts 625741 -41.8% 364480 ± 15% interrupts.CPU252.LOC:Local_timer_interrupts 5477 ± 33% -41.0% 3231 ± 2% interrupts.CPU252.NMI:Non-maskable_interrupts 5477 ± 33% -41.0% 3231 ± 2% interrupts.CPU252.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU253.CAL:Function_call_interrupts 626149 -34.7% 408653 ± 27% interrupts.CPU253.LOC:Local_timer_interrupts 1412 ± 4% -15.9% 1188 interrupts.CPU254.CAL:Function_call_interrupts 626370 -41.6% 365534 ± 14% interrupts.CPU254.LOC:Local_timer_interrupts 1412 ± 4% -15.8% 1188 interrupts.CPU255.CAL:Function_call_interrupts 626180 -41.7% 365269 ± 9% interrupts.CPU255.LOC:Local_timer_interrupts 1412 ± 4% -15.8% 1188 interrupts.CPU256.CAL:Function_call_interrupts 1412 ± 4% -15.8% 1188 interrupts.CPU257.CAL:Function_call_interrupts 626384 -39.3% 379951 ± 17% interrupts.CPU257.LOC:Local_timer_interrupts 1409 ± 4% -15.6% 1188 interrupts.CPU258.CAL:Function_call_interrupts 626200 -36.8% 395576 ± 21% interrupts.CPU258.LOC:Local_timer_interrupts 1412 ± 4% -15.8% 1188 interrupts.CPU259.CAL:Function_call_interrupts 626122 -35.6% 403275 ± 28% interrupts.CPU259.LOC:Local_timer_interrupts 5503 ± 33% -26.6% 4040 ± 33% interrupts.CPU259.NMI:Non-maskable_interrupts 5503 ± 33% -26.6% 4040 ± 33% interrupts.CPU259.PMI:Performance_monitoring_interrupts 1402 ± 5% -15.3% 1188 interrupts.CPU26.CAL:Function_call_interrupts 1412 ± 4% -15.5% 1193 interrupts.CPU260.CAL:Function_call_interrupts 6462 ± 24% -47.8% 3370 interrupts.CPU260.NMI:Non-maskable_interrupts 6462 ± 24% -47.8% 3370 interrupts.CPU260.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU261.CAL:Function_call_interrupts 1413 ± 4% -15.7% 1191 interrupts.CPU262.CAL:Function_call_interrupts 625496 -30.8% 432776 ± 25% interrupts.CPU262.LOC:Local_timer_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU263.CAL:Function_call_interrupts 626079 -31.7% 427800 ± 26% interrupts.CPU263.LOC:Local_timer_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU264.CAL:Function_call_interrupts 4691 ± 34% -27.0% 3426 ± 2% interrupts.CPU264.NMI:Non-maskable_interrupts 4691 ± 34% -27.0% 3426 ± 2% interrupts.CPU264.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.5% 1192 interrupts.CPU265.CAL:Function_call_interrupts 5626 ± 33% -22.4% 4366 ± 34% interrupts.CPU265.NMI:Non-maskable_interrupts 5626 ± 33% -22.4% 4366 ± 34% interrupts.CPU265.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU266.CAL:Function_call_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU267.CAL:Function_call_interrupts 5632 ± 33% -23.8% 4291 ± 32% interrupts.CPU267.NMI:Non-maskable_interrupts 5632 ± 33% -23.8% 4291 ± 32% interrupts.CPU267.PMI:Performance_monitoring_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU268.CAL:Function_call_interrupts 1413 ± 4% -15.6% 1192 interrupts.CPU269.CAL:Function_call_interrupts 1388 ± 4% -14.4% 1188 interrupts.CPU27.CAL:Function_call_interrupts 1408 ± 4% -15.4% 1192 interrupts.CPU270.CAL:Function_call_interrupts 1408 ± 4% -15.3% 1192 interrupts.CPU271.CAL:Function_call_interrupts 1412 ± 4% -15.6% 1191 interrupts.CPU272.CAL:Function_call_interrupts 1412 ± 4% -15.6% 1192 interrupts.CPU273.CAL:Function_call_interrupts 5592 ± 33% -24.0% 4247 ± 34% interrupts.CPU273.NMI:Non-maskable_interrupts 5592 ± 33% -24.0% 4247 ± 34% interrupts.CPU273.PMI:Performance_monitoring_interrupts 1411 ± 4% -15.5% 1192 interrupts.CPU274.CAL:Function_call_interrupts 1411 ± 4% -15.5% 1192 interrupts.CPU275.CAL:Function_call_interrupts 5608 ± 33% -23.6% 4282 ± 32% interrupts.CPU275.NMI:Non-maskable_interrupts 5608 ± 33% -23.6% 4282 ± 32% interrupts.CPU275.PMI:Performance_monitoring_interrupts 6560 ± 25% -34.9% 4268 ± 33% interrupts.CPU276.NMI:Non-maskable_interrupts 6560 ± 25% -34.9% 4268 ± 33% interrupts.CPU276.PMI:Performance_monitoring_interrupts 1416 ± 4% -15.7% 1193 interrupts.CPU277.CAL:Function_call_interrupts 1418 ± 4% -15.9% 1193 interrupts.CPU278.CAL:Function_call_interrupts 1419 ± 4% -15.9% 1194 interrupts.CPU279.CAL:Function_call_interrupts 1392 ± 4% -14.6% 1190 interrupts.CPU28.CAL:Function_call_interrupts 5670 ± 33% -22.6% 4386 ± 32% interrupts.CPU28.NMI:Non-maskable_interrupts 5670 ± 33% -22.6% 4386 ± 32% interrupts.CPU28.PMI:Performance_monitoring_interrupts 1419 ± 4% -15.8% 1195 interrupts.CPU280.CAL:Function_call_interrupts 6585 ± 25% -35.1% 4270 ± 34% interrupts.CPU280.NMI:Non-maskable_interrupts 6585 ± 25% -35.1% 4270 ± 34% interrupts.CPU280.PMI:Performance_monitoring_interrupts 1419 ± 4% -15.5% 1199 interrupts.CPU281.CAL:Function_call_interrupts 5615 ± 33% -39.2% 3413 interrupts.CPU281.NMI:Non-maskable_interrupts 5615 ± 33% -39.2% 3413 interrupts.CPU281.PMI:Performance_monitoring_interrupts 1420 ± 4% -16.0% 1193 interrupts.CPU282.CAL:Function_call_interrupts 5666 ± 33% -24.7% 4264 ± 33% interrupts.CPU282.NMI:Non-maskable_interrupts 5666 ± 33% -24.7% 4264 ± 33% interrupts.CPU282.PMI:Performance_monitoring_interrupts 1417 ± 4% -15.9% 1192 interrupts.CPU283.CAL:Function_call_interrupts 1415 ± 4% -15.7% 1193 interrupts.CPU284.CAL:Function_call_interrupts 1420 ± 4% -16.1% 1192 interrupts.CPU285.CAL:Function_call_interrupts 1420 ± 4% -15.9% 1194 interrupts.CPU286.CAL:Function_call_interrupts 5503 ± 33% -22.5% 4265 ± 33% interrupts.CPU286.NMI:Non-maskable_interrupts 5503 ± 33% -22.5% 4265 ± 33% interrupts.CPU286.PMI:Performance_monitoring_interrupts 6353 ± 24% -33.9% 4197 ± 33% interrupts.CPU287.NMI:Non-maskable_interrupts 6353 ± 24% -33.9% 4197 ± 33% interrupts.CPU287.PMI:Performance_monitoring_interrupts 15.00 ± 11% +2371.7% 370.75 ± 59% interrupts.CPU287.TLB:TLB_shootdowns 1388 ± 4% -14.2% 1191 interrupts.CPU29.CAL:Function_call_interrupts 5647 ± 33% -23.7% 4311 ± 32% interrupts.CPU29.NMI:Non-maskable_interrupts 5647 ± 33% -23.7% 4311 ± 32% interrupts.CPU29.PMI:Performance_monitoring_interrupts 1392 ± 4% -14.5% 1189 interrupts.CPU3.CAL:Function_call_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU30.CAL:Function_call_interrupts 5666 ± 33% -21.5% 4445 ± 32% interrupts.CPU30.NMI:Non-maskable_interrupts 5666 ± 33% -21.5% 4445 ± 32% interrupts.CPU30.PMI:Performance_monitoring_interrupts 1073 ± 72% -76.8% 249.50 ± 38% interrupts.CPU30.RES:Rescheduling_interrupts 1392 ± 4% -14.5% 1189 interrupts.CPU31.CAL:Function_call_interrupts 6591 ± 24% -45.4% 3596 interrupts.CPU31.NMI:Non-maskable_interrupts 6591 ± 24% -45.4% 3596 interrupts.CPU31.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.6% 1188 interrupts.CPU32.CAL:Function_call_interrupts 873.25 ± 42% -71.4% 249.50 ± 32% interrupts.CPU32.RES:Rescheduling_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU33.CAL:Function_call_interrupts 6630 ± 24% -45.8% 3590 ± 2% interrupts.CPU33.NMI:Non-maskable_interrupts 6630 ± 24% -45.8% 3590 ± 2% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU34.CAL:Function_call_interrupts 1285 ± 72% -83.2% 215.25 ± 25% interrupts.CPU34.RES:Rescheduling_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU35.CAL:Function_call_interrupts 1392 ± 4% -14.5% 1190 interrupts.CPU36.CAL:Function_call_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU37.CAL:Function_call_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU38.CAL:Function_call_interrupts 1936 ±106% -88.4% 224.00 ± 31% interrupts.CPU38.RES:Rescheduling_interrupts 1391 ± 4% -14.5% 1190 interrupts.CPU39.CAL:Function_call_interrupts 1391 ± 4% -14.4% 1191 interrupts.CPU4.CAL:Function_call_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU40.CAL:Function_call_interrupts 1277 ± 44% -81.9% 231.00 ± 49% interrupts.CPU40.RES:Rescheduling_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU41.CAL:Function_call_interrupts 5619 ± 33% -37.9% 3486 interrupts.CPU41.NMI:Non-maskable_interrupts 5619 ± 33% -37.9% 3486 interrupts.CPU41.PMI:Performance_monitoring_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU42.CAL:Function_call_interrupts 5622 ± 33% -23.1% 4321 ± 35% interrupts.CPU42.NMI:Non-maskable_interrupts 5622 ± 33% -23.1% 4321 ± 35% interrupts.CPU42.PMI:Performance_monitoring_interrupts 824.75 ± 47% -60.8% 323.00 ± 71% interrupts.CPU42.RES:Rescheduling_interrupts 1392 ± 4% -14.5% 1190 interrupts.CPU43.CAL:Function_call_interrupts 1391 ± 4% -14.4% 1191 interrupts.CPU44.CAL:Function_call_interrupts 337.00 ± 8% -36.6% 213.50 ± 10% interrupts.CPU44.RES:Rescheduling_interrupts 1390 ± 4% -14.4% 1190 interrupts.CPU45.CAL:Function_call_interrupts 251.50 ± 28% -59.8% 101.00 ± 43% interrupts.CPU45.RES:Rescheduling_interrupts 1392 ± 4% -14.4% 1191 interrupts.CPU46.CAL:Function_call_interrupts 377.25 ± 27% -51.9% 181.50 ± 29% interrupts.CPU46.RES:Rescheduling_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU47.CAL:Function_call_interrupts 1393 ± 5% -14.5% 1191 interrupts.CPU48.CAL:Function_call_interrupts 6604 ± 24% -47.1% 3490 interrupts.CPU48.NMI:Non-maskable_interrupts 6604 ± 24% -47.1% 3490 interrupts.CPU48.PMI:Performance_monitoring_interrupts 1390 ± 5% -14.4% 1190 interrupts.CPU49.CAL:Function_call_interrupts 1390 ± 4% -14.4% 1190 interrupts.CPU5.CAL:Function_call_interrupts 370.75 ± 40% -49.0% 189.00 ± 23% interrupts.CPU5.RES:Rescheduling_interrupts 1386 ± 4% -14.1% 1190 interrupts.CPU50.CAL:Function_call_interrupts 597.25 ± 29% -59.1% 244.00 ± 19% interrupts.CPU50.RES:Rescheduling_interrupts 1390 ± 4% -14.5% 1189 interrupts.CPU51.CAL:Function_call_interrupts 1390 ± 4% -14.3% 1191 interrupts.CPU52.CAL:Function_call_interrupts 1202 ± 96% -75.5% 295.00 ± 22% interrupts.CPU52.RES:Rescheduling_interrupts 1390 ± 4% -14.3% 1191 interrupts.CPU53.CAL:Function_call_interrupts 4.75 ± 17% +1552.6% 78.50 ±157% interrupts.CPU53.TLB:TLB_shootdowns 1393 ± 4% -14.6% 1190 interrupts.CPU54.CAL:Function_call_interrupts 638.50 ± 38% -77.2% 145.75 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts 1384 ± 4% -13.7% 1193 interrupts.CPU55.CAL:Function_call_interrupts 1390 ± 4% -14.4% 1189 interrupts.CPU56.CAL:Function_call_interrupts 1393 ± 4% -14.7% 1189 interrupts.CPU57.CAL:Function_call_interrupts 1389 ± 4% -14.3% 1191 interrupts.CPU58.CAL:Function_call_interrupts 499.75 ± 28% -55.8% 221.00 ± 28% interrupts.CPU58.RES:Rescheduling_interrupts 1387 ± 4% -14.1% 1191 interrupts.CPU59.CAL:Function_call_interrupts 1392 ± 4% -14.5% 1190 interrupts.CPU6.CAL:Function_call_interrupts 5659 ± 33% -21.2% 4457 ± 34% interrupts.CPU6.NMI:Non-maskable_interrupts 5659 ± 33% -21.2% 4457 ± 34% interrupts.CPU6.PMI:Performance_monitoring_interrupts 489.25 ± 21% -28.7% 348.75 ± 8% interrupts.CPU6.RES:Rescheduling_interrupts 1390 ± 4% -14.3% 1191 interrupts.CPU60.CAL:Function_call_interrupts 502.00 ± 13% -53.0% 236.00 ± 24% interrupts.CPU60.RES:Rescheduling_interrupts 1391 ± 4% -14.5% 1189 interrupts.CPU61.CAL:Function_call_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU62.CAL:Function_call_interrupts 5646 ± 33% -24.6% 4256 ± 35% interrupts.CPU62.NMI:Non-maskable_interrupts 5646 ± 33% -24.6% 4256 ± 35% interrupts.CPU62.PMI:Performance_monitoring_interrupts 720.00 ± 17% -62.0% 273.25 ± 24% interrupts.CPU62.RES:Rescheduling_interrupts 1391 ± 4% -14.4% 1191 interrupts.CPU63.CAL:Function_call_interrupts 1392 ± 4% -14.3% 1193 interrupts.CPU64.CAL:Function_call_interrupts 867.50 ± 76% -68.1% 276.50 ± 42% interrupts.CPU64.RES:Rescheduling_interrupts 1391 ± 4% -14.4% 1190 interrupts.CPU65.CAL:Function_call_interrupts 1389 ± 4% -14.2% 1192 interrupts.CPU66.CAL:Function_call_interrupts 323.25 ± 13% -43.3% 183.25 ± 23% interrupts.CPU66.RES:Rescheduling_interrupts 1389 ± 5% -14.2% 1191 interrupts.CPU67.CAL:Function_call_interrupts 1391 ± 4% -14.3% 1192 interrupts.CPU68.CAL:Function_call_interrupts 1391 ± 4% -14.5% 1190 interrupts.CPU69.CAL:Function_call_interrupts 1392 ± 4% -14.6% 1189 interrupts.CPU7.CAL:Function_call_interrupts 5661 ± 33% -38.6% 3474 ± 2% interrupts.CPU7.NMI:Non-maskable_interrupts 5661 ± 33% -38.6% 3474 ± 2% interrupts.CPU7.PMI:Performance_monitoring_interrupts 479.25 ± 45% -56.4% 209.00 ± 23% interrupts.CPU7.RES:Rescheduling_interrupts 1389 ± 4% -14.4% 1189 interrupts.CPU71.CAL:Function_call_interrupts 1388 ± 4% -14.4% 1188 interrupts.CPU72.CAL:Function_call_interrupts 626242 -30.8% 433221 ± 25% interrupts.CPU72.LOC:Local_timer_interrupts 6458 ± 25% -35.4% 4171 ± 33% interrupts.CPU72.NMI:Non-maskable_interrupts 6458 ± 25% -35.4% 4171 ± 33% interrupts.CPU72.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU73.CAL:Function_call_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU74.CAL:Function_call_interrupts 1387 ± 4% -14.6% 1185 interrupts.CPU75.CAL:Function_call_interrupts 6509 ± 25% -35.0% 4228 ± 34% interrupts.CPU75.NMI:Non-maskable_interrupts 6509 ± 25% -35.0% 4228 ± 34% interrupts.CPU75.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.5% 1188 interrupts.CPU76.CAL:Function_call_interrupts 626012 -41.1% 368808 ± 13% interrupts.CPU76.LOC:Local_timer_interrupts 1388 ± 4% -25.0% 1041 ± 24% interrupts.CPU77.CAL:Function_call_interrupts 626170 -43.1% 356486 ± 11% interrupts.CPU77.LOC:Local_timer_interrupts 1387 ± 4% -14.5% 1185 interrupts.CPU78.CAL:Function_call_interrupts 1387 ± 4% -14.5% 1185 interrupts.CPU79.CAL:Function_call_interrupts 5516 ± 33% -22.6% 4267 ± 35% interrupts.CPU79.NMI:Non-maskable_interrupts 5516 ± 33% -22.6% 4267 ± 35% interrupts.CPU79.PMI:Performance_monitoring_interrupts 1390 ± 4% -14.3% 1191 interrupts.CPU8.CAL:Function_call_interrupts 1388 ± 4% -14.6% 1185 interrupts.CPU80.CAL:Function_call_interrupts 626125 -42.2% 361733 ± 11% interrupts.CPU80.LOC:Local_timer_interrupts 1384 ± 4% -14.3% 1186 interrupts.CPU81.CAL:Function_call_interrupts 626113 -41.4% 366749 ± 7% interrupts.CPU81.LOC:Local_timer_interrupts 1387 ± 4% -14.4% 1187 interrupts.CPU82.CAL:Function_call_interrupts 624751 -43.7% 351795 ± 9% interrupts.CPU82.LOC:Local_timer_interrupts 3628 -11.2% 3221 ± 2% interrupts.CPU82.NMI:Non-maskable_interrupts 3628 -11.2% 3221 ± 2% interrupts.CPU82.PMI:Performance_monitoring_interrupts 1388 ± 4% -14.6% 1185 interrupts.CPU83.CAL:Function_call_interrupts 626816 -41.1% 369469 ± 13% interrupts.CPU83.LOC:Local_timer_interrupts 1385 ± 4% -14.5% 1185 interrupts.CPU84.CAL:Function_call_interrupts 5494 ± 33% -37.9% 3412 interrupts.CPU84.NMI:Non-maskable_interrupts 5494 ± 33% -37.9% 3412 interrupts.CPU84.PMI:Performance_monitoring_interrupts 1388 ± 4% -25.0% 1041 ± 23% interrupts.CPU85.CAL:Function_call_interrupts 626454 -27.0% 457589 ± 22% interrupts.CPU85.LOC:Local_timer_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU86.CAL:Function_call_interrupts 626005 -34.3% 411017 ± 22% interrupts.CPU86.LOC:Local_timer_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU87.CAL:Function_call_interrupts 626191 -35.4% 404306 ± 16% interrupts.CPU87.LOC:Local_timer_interrupts 1389 ± 4% -14.6% 1186 interrupts.CPU88.CAL:Function_call_interrupts 625872 -44.5% 347325 ± 7% interrupts.CPU88.LOC:Local_timer_interrupts 3650 -11.1% 3243 ± 2% interrupts.CPU88.NMI:Non-maskable_interrupts 3650 -11.1% 3243 ± 2% interrupts.CPU88.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU89.CAL:Function_call_interrupts 626366 -43.0% 356980 ± 9% interrupts.CPU89.LOC:Local_timer_interrupts 5494 ± 33% -27.0% 4010 ± 32% interrupts.CPU89.NMI:Non-maskable_interrupts 5494 ± 33% -27.0% 4010 ± 32% interrupts.CPU89.PMI:Performance_monitoring_interrupts 1390 ± 4% -14.3% 1191 interrupts.CPU9.CAL:Function_call_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU90.CAL:Function_call_interrupts 626013 -39.3% 379762 ± 19% interrupts.CPU90.LOC:Local_timer_interrupts 5536 ± 33% -24.8% 4164 ± 33% interrupts.CPU90.NMI:Non-maskable_interrupts 5536 ± 33% -24.8% 4164 ± 33% interrupts.CPU90.PMI:Performance_monitoring_interrupts 1387 ± 4% -14.5% 1185 interrupts.CPU91.CAL:Function_call_interrupts 626180 -32.8% 421091 ± 24% interrupts.CPU91.LOC:Local_timer_interrupts 5526 ± 33% -23.4% 4235 ± 34% interrupts.CPU91.NMI:Non-maskable_interrupts 5526 ± 33% -23.4% 4235 ± 34% interrupts.CPU91.PMI:Performance_monitoring_interrupts 1388 ± 4% -24.9% 1042 ± 23% interrupts.CPU92.CAL:Function_call_interrupts 626143 -34.7% 408843 ± 22% interrupts.CPU92.LOC:Local_timer_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU93.CAL:Function_call_interrupts 626257 -34.0% 413157 ± 19% interrupts.CPU93.LOC:Local_timer_interrupts 13.00 ± 12% +588.5% 89.50 ± 95% interrupts.CPU93.RES:Rescheduling_interrupts 1387 ± 4% -14.5% 1186 interrupts.CPU94.CAL:Function_call_interrupts 1388 ± 4% -14.6% 1186 interrupts.CPU95.CAL:Function_call_interrupts 626418 -24.9% 470417 ± 20% interrupts.CPU95.LOC:Local_timer_interrupts 5553 ± 33% -38.3% 3424 interrupts.CPU95.NMI:Non-maskable_interrupts 5553 ± 33% -38.3% 3424 interrupts.CPU95.PMI:Performance_monitoring_interrupts 1388 ± 4% -14.5% 1187 interrupts.CPU96.CAL:Function_call_interrupts 625121 -32.5% 422088 ± 14% interrupts.CPU96.LOC:Local_timer_interrupts 5569 ± 33% -27.4% 4043 ± 34% interrupts.CPU96.NMI:Non-maskable_interrupts 5569 ± 33% -27.4% 4043 ± 34% interrupts.CPU96.PMI:Performance_monitoring_interrupts 1389 ± 4% -14.6% 1186 interrupts.CPU97.CAL:Function_call_interrupts 626209 -34.6% 409693 ± 16% interrupts.CPU97.LOC:Local_timer_interrupts 1381 ± 5% -14.1% 1187 interrupts.CPU98.CAL:Function_call_interrupts 5555 ± 33% -38.4% 3419 interrupts.CPU98.NMI:Non-maskable_interrupts 5555 ± 33% -38.4% 3419 interrupts.CPU98.PMI:Performance_monitoring_interrupts 1383 ± 5% -14.2% 1187 interrupts.CPU99.CAL:Function_call_interrupts 626199 -32.8% 420723 ± 26% interrupts.CPU99.LOC:Local_timer_interrupts 107.25 ± 4% +174.1% 294.00 ± 26% interrupts.IWI:IRQ_work_interrupts 1.803e+08 -21.4% 1.418e+08 ± 12% interrupts.LOC:Local_timer_interrupts 60735 ± 5% -39.5% 36730 ± 4% interrupts.RES:Rescheduling_interrupts *************************************************************************************************** lkp-skl-2sp7: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory ========================================================================================= cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode: cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.6/1/debian-x86_64-20191114.cgz/300s/lkp-skl-2sp7/TCP_CRR/netperf/0x2000065 commit: 7aba5dcc23 ("jfs: Replace zero-length array with flexible-array member") 05c5a0273b ("fs: jfs: fix a possible data race in txBegin()") 7aba5dcc234635b4 05c5a0273bc25627b9e9bbebc85 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 100% 4:4 kmsg.IP-Config:Failed_to_open_erspan0 :4 100% 4:4 kmsg.IP-Config:Failed_to_open_gretap0 :4 100% 4:4 kmsg.kvm:already_loaded_the_other_module %stddev %change %stddev \ | \ 32033 -71.8% 9048 netperf.Throughput_total_tps 32033 -71.8% 9048 netperf.Throughput_tps 34270 ± 19% -32.2% 23234 ± 11% netperf.time.involuntary_context_switches 87.50 -1.7% 86.00 netperf.time.percent_of_cpu_this_job_got 237.76 +7.1% 254.71 netperf.time.system_time 30.66 -66.6% 10.25 ± 3% netperf.time.user_time 17926206 -69.8% 5405643 netperf.time.voluntary_context_switches 9610194 -71.8% 2714625 netperf.workload 0.25 -0.1 0.12 ± 3% mpstat.cpu.all.usr% 181085 -60.2% 72003 vmstat.system.cs 24.19 ± 2% +8.7% 26.30 boot-time.boot 15.27 ± 2% +12.6% 17.18 boot-time.dhcp 1449 ± 2% +10.6% 1602 boot-time.idle 1.589e+08 -16.1% 1.334e+08 cpuidle.C1.time 10004352 -12.6% 8738852 cpuidle.C1.usage 24458189 ± 3% -76.6% 5718567 ± 15% cpuidle.POLL.time 18015660 -95.3% 840897 ± 17% cpuidle.POLL.usage 42120 ± 2% +9.9% 46309 ± 5% softirqs.CPU10.SCHED 41095 ± 4% +10.9% 45593 ± 5% softirqs.CPU27.SCHED 25223 ± 54% +70.0% 42867 softirqs.CPU36.SCHED 166155 ± 8% -10.1% 149301 softirqs.CPU65.RCU 28947742 -71.8% 8152772 softirqs.NET_RX 12415372 -9.2% 11268919 softirqs.RCU 8089600 ± 11% -23.4% 6200320 ± 8% meminfo.DirectMap2M 503148 ± 8% -41.3% 295276 ± 8% meminfo.DirectMap4k 75713 +34.4% 101777 meminfo.KReclaimable 22720 +11.6% 25360 meminfo.Percpu 75713 +34.4% 101777 meminfo.SReclaimable 152955 +16.4% 178056 meminfo.SUnreclaim 228668 +22.4% 279834 meminfo.Slab 137288 +17.8% 161678 meminfo.VmallocUsed 33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.KReclaimable 33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.SReclaimable 74713 ± 16% +34.8% 100736 ± 11% numa-meminfo.node0.SUnreclaim 108029 ± 16% +47.4% 159219 ± 11% numa-meminfo.node0.Slab 101746 ± 44% +68.1% 171066 ± 12% numa-meminfo.node1.Active 101684 ± 44% +68.1% 170963 ± 12% numa-meminfo.node1.Active(anon) 60012 ± 62% +94.5% 116727 ± 18% numa-meminfo.node1.AnonHugePages 98606 ± 46% +69.1% 166702 ± 12% numa-meminfo.node1.AnonPages 5648 +3.4% 5842 proc-vmstat.nr_shmem 18920 +34.5% 25446 proc-vmstat.nr_slab_reclaimable 38238 +16.4% 44513 proc-vmstat.nr_slab_unreclaimable 895359 ± 5% -17.9% 734777 proc-vmstat.numa_hit 872027 ± 5% -18.4% 711460 proc-vmstat.numa_local 1080 ± 9% +27.4% 1375 ± 4% proc-vmstat.pgactivate 1530529 ± 15% -42.6% 878573 ± 2% proc-vmstat.pgalloc_normal 1505342 ± 15% -43.5% 849856 ± 2% proc-vmstat.pgfree 8335 ± 18% +75.4% 14620 ± 12% numa-vmstat.node0.nr_slab_reclaimable 18678 ± 16% +34.8% 25183 ± 11% numa-vmstat.node0.nr_slab_unreclaimable 647500 ± 10% -21.7% 507083 ± 8% numa-vmstat.node0.numa_hit 494986 ± 13% -27.6% 358327 ± 13% numa-vmstat.node0.numa_local 25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_active_anon 24661 ± 46% +69.0% 41679 ± 12% numa-vmstat.node1.nr_anon_pages 25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_zone_active_anon 725209 ± 8% -16.4% 606495 ± 8% numa-vmstat.node1.numa_hit 702021 ± 8% -17.9% 576047 ± 10% numa-vmstat.node1.numa_local 49724 ± 16% -42.5% 28572 ± 16% sched_debug.cfs_rq:/.load.avg 113065 ± 11% -24.3% 85601 ± 10% sched_debug.cfs_rq:/.load.stddev 57662 ± 7% -47.5% 30301 ± 9% sched_debug.cfs_rq:/.min_vruntime.max 6576 ± 8% -34.6% 4300 ± 19% sched_debug.cfs_rq:/.min_vruntime.min 10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.min_vruntime.stddev 0.16 ± 8% -29.7% 0.11 ± 8% sched_debug.cfs_rq:/.nr_running.avg 0.36 ± 2% -13.7% 0.31 ± 4% sched_debug.cfs_rq:/.nr_running.stddev 35.45 ± 20% -52.9% 16.69 sched_debug.cfs_rq:/.runnable_load_avg.avg 95.13 ± 14% -32.7% 63.99 sched_debug.cfs_rq:/.runnable_load_avg.stddev 50290 ± 14% -43.7% 28330 ± 15% sched_debug.cfs_rq:/.runnable_weight.avg 113143 ± 10% -24.9% 84975 ± 10% sched_debug.cfs_rq:/.runnable_weight.stddev 33703 ± 43% -68.0% 10796 ± 42% sched_debug.cfs_rq:/.spread0.max 10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.spread0.stddev 140.66 ± 10% -17.8% 115.68 ± 7% sched_debug.cfs_rq:/.util_avg.avg 247.04 ± 4% -11.6% 218.36 ± 4% sched_debug.cfs_rq:/.util_avg.stddev 51.66 ± 11% -41.4% 30.29 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.avg 179.93 ± 3% -26.3% 132.58 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev 1.26 ± 2% +10.1% 1.39 ± 3% sched_debug.cpu.clock.stddev 1.26 ± 2% +10.2% 1.39 ± 3% sched_debug.cpu.clock_task.stddev 365656 ± 9% -57.9% 154017 sched_debug.cpu.nr_switches.avg 3644073 ± 21% -81.6% 671977 ± 17% sched_debug.cpu.nr_switches.max 1315 ± 5% +863.7% 12673 ± 91% sched_debug.cpu.nr_switches.min 592717 ± 15% -75.6% 144437 ± 18% sched_debug.cpu.nr_switches.stddev 341.50 ± 96% +267.0% 1253 ± 7% interrupts.CPU12.RES:Rescheduling_interrupts 145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.NMI:Non-maskable_interrupts 145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.PMI:Performance_monitoring_interrupts 168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.NMI:Non-maskable_interrupts 168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.PMI:Performance_monitoring_interrupts 316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.NMI:Non-maskable_interrupts 316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3796 ± 83% -82.0% 685.00 ±119% interrupts.CPU18.RES:Rescheduling_interrupts 2353 ± 49% -84.5% 364.00 ±134% interrupts.CPU20.RES:Rescheduling_interrupts 107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.NMI:Non-maskable_interrupts 107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.PMI:Performance_monitoring_interrupts 89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.NMI:Non-maskable_interrupts 89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.PMI:Performance_monitoring_interrupts 93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.NMI:Non-maskable_interrupts 93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.PMI:Performance_monitoring_interrupts 87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.NMI:Non-maskable_interrupts 87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.PMI:Performance_monitoring_interrupts 104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.NMI:Non-maskable_interrupts 104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1951 ± 71% -80.2% 387.00 ±113% interrupts.CPU33.RES:Rescheduling_interrupts 117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.NMI:Non-maskable_interrupts 117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.PMI:Performance_monitoring_interrupts 142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.NMI:Non-maskable_interrupts 142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.PMI:Performance_monitoring_interrupts 320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.NMI:Non-maskable_interrupts 320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.PMI:Performance_monitoring_interrupts 106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.NMI:Non-maskable_interrupts 106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.PMI:Performance_monitoring_interrupts 87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.NMI:Non-maskable_interrupts 87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.PMI:Performance_monitoring_interrupts 23466 +23.8% 29040 ± 5% interrupts.NMI:Non-maskable_interrupts 23466 +23.8% 29040 ± 5% interrupts.PMI:Performance_monitoring_interrupts 13.69 ± 25% -71.4% 3.92 ± 48% perf-stat.i.MPKI 8.78e+08 +90.0% 1.668e+09 perf-stat.i.branch-instructions 2.30 ± 14% -1.5 0.83 ± 33% perf-stat.i.branch-miss-rate% 20600243 ± 12% -31.0% 14224189 ± 29% perf-stat.i.branch-misses 2.48 ± 18% +1.7 4.22 ± 14% perf-stat.i.cache-miss-rate% 57749473 ± 23% -44.2% 32231609 ± 44% perf-stat.i.cache-references 182988 -60.2% 72782 perf-stat.i.context-switches 1.65 ± 5% -39.4% 1.00 ± 5% perf-stat.i.cpi 7.078e+09 ± 3% +18.6% 8.393e+09 ± 3% perf-stat.i.cpu-cycles 85.15 -8.1% 78.28 perf-stat.i.cpu-migrations 0.09 ± 32% -0.1 0.02 ±106% perf-stat.i.dTLB-load-miss-rate% 1.284e+09 +89.1% 2.428e+09 perf-stat.i.dTLB-loads 0.01 ± 51% -0.0 0.00 ±133% perf-stat.i.dTLB-store-miss-rate% 7.676e+08 +81.0% 1.39e+09 perf-stat.i.dTLB-stores 28.71 ± 22% -19.9 8.79 ± 72% perf-stat.i.iTLB-load-miss-rate% 661655 ± 18% -65.2% 230579 ± 60% perf-stat.i.iTLB-load-misses 1672859 ± 15% +53.1% 2560787 ± 13% perf-stat.i.iTLB-loads 4.377e+09 +92.6% 8.432e+09 perf-stat.i.instructions 7111 ± 22% +568.3% 47528 ± 35% perf-stat.i.instructions-per-iTLB-miss 0.61 ± 4% +63.4% 1.00 ± 4% perf-stat.i.ipc 0.10 ± 3% +18.6% 0.12 ± 3% perf-stat.i.metric.GHz 41.51 +84.7% 76.66 perf-stat.i.metric.M/sec 131741 ± 7% +48.3% 195398 ± 7% perf-stat.i.node-load-misses 36778 ± 31% -55.6% 16347 ± 16% perf-stat.i.node-stores 13.22 ± 23% -70.9% 3.85 ± 46% perf-stat.overall.MPKI 2.35 ± 12% -1.5 0.86 ± 30% perf-stat.overall.branch-miss-rate% 2.34 ± 18% +1.9 4.20 ± 14% perf-stat.overall.cache-miss-rate% 1.62 ± 4% -38.4% 1.00 ± 5% perf-stat.overall.cpi 0.08 ± 30% -0.1 0.02 ±103% perf-stat.overall.dTLB-load-miss-rate% 0.01 ± 50% -0.0 0.00 ±132% perf-stat.overall.dTLB-store-miss-rate% 28.68 ± 22% -19.9 8.73 ± 71% perf-stat.overall.iTLB-load-miss-rate% 6894 ± 21% +579.7% 46861 ± 36% perf-stat.overall.instructions-per-iTLB-miss 0.62 ± 4% +62.5% 1.01 ± 4% perf-stat.overall.ipc 138819 +581.5% 946109 perf-stat.overall.path-length 8.751e+08 +90.0% 1.662e+09 perf-stat.ps.branch-instructions 20537517 ± 12% -30.9% 14182248 ± 29% perf-stat.ps.branch-misses 57557495 ± 23% -44.2% 32125500 ± 44% perf-stat.ps.cache-references 182374 -60.2% 72538 perf-stat.ps.context-switches 7.055e+09 ± 3% +18.6% 8.366e+09 ± 3% perf-stat.ps.cpu-cycles 84.88 -8.1% 78.04 perf-stat.ps.cpu-migrations 1.28e+09 +89.1% 2.42e+09 perf-stat.ps.dTLB-loads 7.651e+08 +81.0% 1.385e+09 perf-stat.ps.dTLB-stores 659450 ± 18% -65.2% 229817 ± 60% perf-stat.ps.iTLB-load-misses 1667327 ± 15% +53.1% 2552296 ± 13% perf-stat.ps.iTLB-loads 4.363e+09 +92.6% 8.405e+09 perf-stat.ps.instructions 131326 ± 7% +48.3% 194773 ± 7% perf-stat.ps.node-load-misses 36665 ± 31% -55.5% 16301 ± 16% perf-stat.ps.node-stores 1.334e+12 +92.5% 2.568e+12 perf-stat.total.instructions 1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.active_objs 1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.num_objs 172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.active_objs 172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.num_objs 1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.active_objs 1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.num_objs 497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.active_objs 497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.num_objs 3990 -100.0% 0.00 slabinfo.cred_jar.active_objs 3990 -100.0% 0.00 slabinfo.cred_jar.num_objs 1917 -26.9% 1400 slabinfo.dentry.active_slabs 1917 -26.9% 1400 slabinfo.dentry.num_slabs 2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.active_objs 2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.num_objs 761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.active_objs 761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.num_objs 3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.active_objs 3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.num_objs 19419 ± 6% +25.2% 24310 ± 8% slabinfo.filp.active_objs 613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.active_slabs 19628 ± 6% +24.4% 24408 ± 8% slabinfo.filp.num_objs 613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.num_slabs 2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.active_objs 2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.num_objs 1007 +58.3% 1594 slabinfo.inode_cache.active_slabs 1007 +58.3% 1594 slabinfo.inode_cache.num_slabs 2517 -35.4% 1625 slabinfo.kernfs_node_cache.active_slabs 2517 -35.4% 1625 slabinfo.kernfs_node_cache.num_slabs 4927 ± 2% -13.8% 4248 ± 4% slabinfo.kmalloc-1k.active_objs 5040 ± 2% -15.1% 4280 ± 4% slabinfo.kmalloc-1k.num_objs 4837 ± 4% +28.0% 6193 slabinfo.kmalloc-2k.active_objs 4870 ± 5% +28.1% 6237 slabinfo.kmalloc-2k.num_objs 1191 +19.1% 1419 slabinfo.kmalloc-4k.active_objs 1219 +18.3% 1442 slabinfo.kmalloc-4k.num_objs 3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.active_objs 3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.num_objs 387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.active_objs 387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.num_objs 759.00 ± 10% -36.2% 484.00 ± 5% slabinfo.kmem_cache_node.active_objs 816.00 ± 10% -39.2% 496.00 ± 5% slabinfo.kmem_cache_node.num_objs 12875 -100.0% 0.00 slabinfo.lsm_file_cache.active_objs 12875 -100.0% 0.00 slabinfo.lsm_file_cache.num_objs 2271 -27.9% 1639 slabinfo.mm_struct.active_objs 2271 -27.9% 1639 slabinfo.mm_struct.num_objs 790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.active_objs 790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.num_objs 123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.active_objs 123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.num_objs 294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.active_objs 294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.num_objs 3128 +73.2% 5418 slabinfo.pid.active_objs 3128 +73.2% 5418 slabinfo.pid.num_objs 10638 -13.0% 9252 slabinfo.proc_inode_cache.active_objs 10638 -12.9% 9266 slabinfo.proc_inode_cache.num_objs 5651 -15.6% 4771 slabinfo.shmem_inode_cache.active_objs 5651 -15.6% 4771 slabinfo.shmem_inode_cache.num_objs 2794 -11.6% 2469 slabinfo.signal_cache.active_objs 2794 -11.6% 2469 slabinfo.signal_cache.num_objs 13766 ± 10% -46.0% 7439 ± 16% slabinfo.sock_inode_cache.active_objs 13865 ± 10% -45.9% 7507 ± 16% slabinfo.sock_inode_cache.num_objs 4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.active_objs 4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.num_objs 940.25 +13.2% 1064 slabinfo.task_struct.active_objs 947.50 -43.4% 536.00 slabinfo.task_struct.active_slabs 947.50 +13.2% 1072 slabinfo.task_struct.num_objs 947.50 -43.4% 536.00 slabinfo.task_struct.num_slabs 1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.active_objs 1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.num_objs 1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.active_slabs 1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.num_slabs 38.42 ± 3% -6.9 31.49 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 38.56 ± 3% -6.9 31.66 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 13.48 ± 3% -2.6 10.86 ± 5% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack 13.07 ± 3% -2.5 10.59 ± 5% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start 14.11 ± 3% -2.3 11.79 ± 5% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip 12.53 ± 3% -2.3 10.22 ± 5% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core 12.39 ± 3% -2.3 10.12 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv 13.88 ± 3% -2.2 11.64 ± 5% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq 12.77 ± 3% -2.2 10.53 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action 14.41 ± 3% -2.2 12.18 ± 5% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb 14.41 ± 3% -2.2 12.21 ± 5% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit 14.29 ± 3% -2.1 12.15 ± 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2 12.58 ± 3% -2.1 10.46 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog 14.30 ± 3% -2.1 12.18 ± 5% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output 8.47 ± 3% -1.7 6.73 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.43 ± 3% -1.7 6.72 perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.31 ± 3% -1.6 6.68 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.00 ± 3% -1.6 7.39 ± 3% perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64 9.22 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.20 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.21 ± 3% -1.6 6.63 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64 9.07 ± 3% -1.6 7.51 ± 3% perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.81 -1.3 3.51 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.77 ± 2% -1.3 3.50 ± 3% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.98 ± 3% -1.2 5.73 perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 9.35 ± 3% -1.2 8.11 ± 4% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames 9.27 ± 3% -1.2 8.06 ± 5% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit 4.46 ± 2% -1.1 3.37 ± 3% perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.45 -1.1 3.37 ± 4% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect 4.49 ± 2% -1.1 3.42 ± 4% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect 4.05 -1.1 2.99 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect 4.52 ± 2% -1.0 3.48 ± 4% perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 4.40 ± 2% -1.0 3.36 ± 3% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64 6.03 ± 3% -0.9 5.18 ± 2% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto 6.02 ± 3% -0.8 5.17 ± 2% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 5.18 ± 4% -0.8 4.39 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release 7.17 ± 2% -0.8 6.38 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 1.03 ± 2% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg 4.58 ± 4% -0.7 3.85 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg 0.98 ± 2% -0.7 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg 3.51 ± 3% -0.7 2.80 ± 4% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock 3.77 ± 6% -0.7 3.07 ± 3% perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect 5.87 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release.sock_close 5.86 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release 4.98 ± 4% -0.6 4.33 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close 3.29 ± 3% -0.6 2.64 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process 3.42 ± 3% -0.6 2.78 ± 4% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock 3.35 ± 3% -0.6 2.71 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv 0.93 ± 6% -0.6 0.28 ±100% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable 4.42 ± 5% -0.6 3.81 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked 7.44 ± 4% -0.6 6.85 ± 6% perf-profile.calltrace.cycles-pp.tcp_close.inet_release.__sock_release.sock_close.__fput 9.05 ± 4% -0.6 8.46 ± 5% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.33 ± 5% -0.6 3.74 ± 2% perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 3.25 ± 5% -0.6 2.70 ± 3% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect 7.74 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 7.61 ± 4% -0.5 7.07 ± 6% perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run 7.72 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_usermode_loop 0.93 ± 7% -0.5 0.40 ± 57% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.94 ± 6% -0.5 0.42 ± 58% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process 0.95 ± 7% -0.5 0.43 ± 58% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv 1.55 ± 5% -0.5 1.03 perf-profile.calltrace.cycles-pp.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 4.02 -0.5 3.50 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 2.68 ± 5% -0.5 2.20 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect 1.10 ± 3% -0.5 0.62 ± 5% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto 2.69 ± 5% -0.5 2.21 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect 1.11 ± 7% -0.5 0.64 ± 9% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 1.03 ± 5% -0.5 0.56 ± 3% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 1.15 ± 3% -0.4 0.72 ± 7% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 3.07 ± 4% -0.4 2.64 ± 3% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect 1.03 ± 6% -0.4 0.60 ± 9% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu 0.99 ± 4% -0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.93 ± 6% -0.3 0.61 ± 6% perf-profile.calltrace.cycles-pp.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv 0.88 ± 5% -0.3 0.56 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established 1.40 ± 5% -0.3 1.12 ± 3% perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 1.52 ± 10% -0.2 1.28 ± 6% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.64 ± 2% -0.1 0.53 ± 3% perf-profile.calltrace.cycles-pp.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file.__sys_accept4 0.58 ± 5% -0.1 0.53 ± 2% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file 2.08 ± 2% +0.1 2.21 ± 3% perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 0.64 ± 4% +0.3 0.93 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 1.15 ± 3% +0.3 1.44 ± 7% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.96 ± 4% +0.3 1.26 ± 10% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv 0.98 ± 4% +0.3 1.30 ± 10% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.34 ±101% +0.5 0.81 ± 18% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.35 ±101% +0.5 0.85 ± 18% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.00 +0.6 0.57 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_get_next_event.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select 0.00 +0.6 0.59 ± 12% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv 0.13 ±173% +0.6 0.72 ± 5% perf-profile.calltrace.cycles-pp.run_timer_softirq.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +0.6 0.61 ± 6% perf-profile.calltrace.cycles-pp.hrtimer_next_event_without.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.13 ±173% +0.6 0.75 ± 7% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv 0.00 +0.6 0.62 ± 4% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established 0.00 +0.6 0.62 ± 12% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.16 ±173% +0.7 0.82 ± 36% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt 0.89 ± 11% +0.7 1.58 ± 4% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle 0.13 ±173% +0.7 0.83 ± 20% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.42 ± 18% +0.7 2.15 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt 0.58 ± 61% +0.7 1.33 ± 35% perf-profile.calltrace.cycles-pp.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 1.05 ± 16% +1.0 2.08 ± 25% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 1.12 ± 18% +1.0 2.15 ± 27% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 0.00 +1.1 1.15 ± 13% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 2.12 ± 11% +1.2 3.28 ± 2% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 1.41 ± 13% +1.3 2.71 ± 24% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.39 ±101% +1.4 1.77 ± 38% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +1.5 1.48 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start 0.00 +1.5 1.49 ± 30% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 3.14 ± 13% +1.6 4.75 ± 11% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 1.35 ± 25% +1.8 3.14 ± 15% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +1.9 1.86 ± 6% perf-profile.calltrace.cycles-pp.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit 1.62 ± 20% +2.2 3.83 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 3.29 ± 11% +2.4 5.72 ± 4% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 3.98 ± 7% +2.9 6.83 ± 2% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.40 ± 14% +3.9 6.30 ± 24% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 4.14 ± 4% +6.5 10.63 ± 11% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 48.02 ± 4% +7.9 55.92 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 47.19 ± 4% +8.0 55.23 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 56.36 ± 2% +8.1 64.47 perf-profile.calltrace.cycles-pp.secondary_startup_64 54.68 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 54.66 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 54.62 ± 3% +8.7 63.31 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 10.59 ± 2% +10.3 20.89 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 8.93 ± 2% +10.6 19.50 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 38.73 ± 3% -6.7 32.02 ± 2% perf-profile.children.cycles-pp.do_syscall_64 38.86 ± 3% -6.7 32.19 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 13.52 ± 3% -2.7 10.87 ± 5% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 13.11 ± 3% -2.5 10.60 ± 5% perf-profile.children.cycles-pp.ip_rcv 17.80 ± 3% -2.5 15.33 ± 4% perf-profile.children.cycles-pp.__tcp_transmit_skb 13.39 ± 2% -2.4 10.95 ± 4% perf-profile.children.cycles-pp.tcp_v4_do_rcv 2.72 -2.4 0.35 ± 4% perf-profile.children.cycles-pp.poll_idle 14.15 ± 3% -2.3 11.81 ± 5% perf-profile.children.cycles-pp.net_rx_action 12.56 ± 3% -2.3 10.22 ± 5% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu 12.46 ± 3% -2.3 10.12 ± 5% perf-profile.children.cycles-pp.tcp_v4_rcv 12.80 ± 3% -2.3 10.53 ± 5% perf-profile.children.cycles-pp.ip_local_deliver 13.93 ± 3% -2.3 11.66 ± 5% perf-profile.children.cycles-pp.process_backlog 14.46 ± 3% -2.2 12.24 ± 5% perf-profile.children.cycles-pp.do_softirq 12.61 ± 3% -2.1 10.46 ± 5% perf-profile.children.cycles-pp.ip_local_deliver_finish 14.34 ± 3% -2.1 12.20 ± 5% perf-profile.children.cycles-pp.do_softirq_own_stack 16.07 ± 3% -1.8 14.25 ± 4% perf-profile.children.cycles-pp.ip_finish_output2 16.42 ± 3% -1.8 14.64 ± 4% perf-profile.children.cycles-pp.ip_output 16.86 ± 3% -1.8 15.08 ± 4% perf-profile.children.cycles-pp.__ip_queue_xmit 8.47 ± 3% -1.7 6.73 perf-profile.children.cycles-pp.__x64_sys_sendto 14.60 ± 3% -1.7 12.89 ± 4% perf-profile.children.cycles-pp.__local_bh_enable_ip 8.43 ± 3% -1.7 6.72 perf-profile.children.cycles-pp.__sys_sendto 8.32 ± 3% -1.6 6.68 perf-profile.children.cycles-pp.sock_sendmsg 9.00 ± 3% -1.6 7.39 ± 3% perf-profile.children.cycles-pp.__inet_stream_connect 9.22 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__x64_sys_connect 8.22 ± 3% -1.6 6.63 perf-profile.children.cycles-pp.tcp_sendmsg 9.20 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__sys_connect 9.08 ± 3% -1.6 7.51 ± 3% perf-profile.children.cycles-pp.inet_stream_connect 6.35 ± 2% -1.5 4.81 ± 3% perf-profile.children.cycles-pp.__release_sock 11.90 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.__tcp_push_pending_frames 11.88 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.tcp_write_xmit 4.82 ± 2% -1.3 3.51 ± 3% perf-profile.children.cycles-pp.__x64_sys_recvfrom 4.77 ± 2% -1.3 3.50 ± 3% perf-profile.children.cycles-pp.__sys_recvfrom 8.17 ± 3% -1.3 6.92 ± 5% perf-profile.children.cycles-pp.tcp_rcv_state_process 6.99 ± 3% -1.3 5.73 perf-profile.children.cycles-pp.tcp_sendmsg_locked 6.10 ± 2% -1.2 4.88 ± 3% perf-profile.children.cycles-pp.release_sock 1.47 ± 5% -1.2 0.32 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64 1.45 ± 7% -1.1 0.35 ± 10% perf-profile.children.cycles-pp.syscall_return_via_sysret 5.09 -1.1 3.99 ± 4% perf-profile.children.cycles-pp.tcp_rcv_established 4.46 ± 2% -1.1 3.37 ± 3% perf-profile.children.cycles-pp.inet_recvmsg 4.41 ± 2% -1.0 3.36 ± 3% perf-profile.children.cycles-pp.tcp_recvmsg 1.83 ± 8% -1.0 0.83 ± 14% perf-profile.children.cycles-pp.tcp_ack 0.91 ± 8% -0.7 0.20 ± 18% perf-profile.children.cycles-pp.__inet_lookup_established 3.77 ± 6% -0.7 3.08 ± 3% perf-profile.children.cycles-pp.tcp_connect 2.63 ± 2% -0.6 2.00 ± 6% perf-profile.children.cycles-pp.try_to_wake_up 2.75 ± 2% -0.6 2.13 ± 6% perf-profile.children.cycles-pp.__wake_up_common 1.19 ± 6% -0.6 0.59 ± 11% perf-profile.children.cycles-pp.read_tsc 7.45 ± 4% -0.6 6.86 ± 6% perf-profile.children.cycles-pp.tcp_close 9.06 ± 4% -0.6 8.47 ± 5% perf-profile.children.cycles-pp.exit_to_usermode_loop 4.33 ± 5% -0.6 3.74 ± 2% perf-profile.children.cycles-pp.tcp_v4_connect 3.03 ± 2% -0.6 2.45 ± 6% perf-profile.children.cycles-pp.__wake_up_common_lock 7.75 ± 4% -0.6 7.19 ± 5% perf-profile.children.cycles-pp.sock_close 7.72 ± 4% -0.5 7.19 ± 5% perf-profile.children.cycles-pp.__sock_release 7.61 ± 4% -0.5 7.07 ± 6% perf-profile.children.cycles-pp.inet_release 1.56 ± 6% -0.5 1.04 perf-profile.children.cycles-pp.tcp_child_process 0.98 ± 6% -0.4 0.53 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair 0.95 ± 6% -0.4 0.54 ± 11% perf-profile.children.cycles-pp.autoremove_wake_function 0.68 ± 5% -0.4 0.27 ± 11% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 2.34 ± 3% -0.4 1.94 ± 6% perf-profile.children.cycles-pp.sock_def_readable 0.90 ± 4% -0.4 0.52 ± 7% perf-profile.children.cycles-pp.__kfree_skb 0.98 ± 5% -0.4 0.60 ± 9% perf-profile.children.cycles-pp.activate_task 0.96 ± 8% -0.4 0.58 ± 17% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 0.99 ± 5% -0.4 0.64 ± 9% perf-profile.children.cycles-pp.ttwu_do_activate 0.78 ± 5% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.enqueue_entity 1.06 ± 5% -0.3 0.73 ± 6% perf-profile.children.cycles-pp.sock_def_wakeup 0.39 ± 10% -0.3 0.11 ± 7% perf-profile.children.cycles-pp.skb_release_all 1.40 ± 5% -0.3 1.12 ± 3% perf-profile.children.cycles-pp.tcp_conn_request 0.53 ± 9% -0.3 0.27 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.36 ± 10% -0.3 0.10 ± 10% perf-profile.children.cycles-pp.skb_release_head_state 0.35 ± 12% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.skb_release_data 0.48 ± 13% -0.2 0.24 ± 13% perf-profile.children.cycles-pp.__skb_datagram_iter 0.29 ± 10% -0.2 0.06 ± 63% perf-profile.children.cycles-pp.dst_release 1.59 ± 9% -0.2 1.35 ± 4% perf-profile.children.cycles-pp.schedule_idle 0.34 ± 11% -0.2 0.11 ± 17% perf-profile.children.cycles-pp.__switch_to 0.48 ± 12% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.skb_copy_datagram_iter 0.27 ± 9% -0.2 0.04 ± 58% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.55 ± 9% -0.2 0.32 ± 11% perf-profile.children.cycles-pp.set_next_entity 0.69 ± 4% -0.2 0.47 ± 6% perf-profile.children.cycles-pp.native_write_msr 0.29 ± 20% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.ipv4_mtu 0.27 ± 6% -0.2 0.06 ± 14% perf-profile.children.cycles-pp.call_rcu 0.45 ± 11% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.sched_clock 1.13 ± 4% -0.2 0.93 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_bh 0.51 ± 7% -0.2 0.30 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair 0.71 ± 3% -0.2 0.52 ± 3% perf-profile.children.cycles-pp.sk_reset_timer 0.50 ± 5% -0.2 0.31 ± 6% perf-profile.children.cycles-pp.tcp_event_new_data_sent 0.29 ± 5% -0.2 0.10 ± 23% perf-profile.children.cycles-pp.__slab_free 0.90 ± 4% -0.2 0.71 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair 0.24 ± 11% -0.2 0.06 ± 26% perf-profile.children.cycles-pp.ipv4_dst_check 0.67 ± 13% -0.2 0.50 ± 2% perf-profile.children.cycles-pp.__next_timer_interrupt 0.22 ± 6% -0.2 0.05 ± 58% perf-profile.children.cycles-pp.__ksize 0.40 ± 14% -0.2 0.23 ± 8% perf-profile.children.cycles-pp.native_sched_clock 0.19 ± 15% -0.2 0.03 ±102% perf-profile.children.cycles-pp.nf_hook_slow 0.20 ± 18% -0.2 0.04 ± 60% perf-profile.children.cycles-pp.sock_rfree 0.22 ± 3% -0.2 0.07 ± 19% perf-profile.children.cycles-pp.sockfd_lookup_light 0.23 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.__switch_to_asm 0.52 ± 5% -0.1 0.38 ± 2% perf-profile.children.cycles-pp.lapic_next_deadline 0.21 ± 9% -0.1 0.07 ± 16% perf-profile.children.cycles-pp.__check_object_size 0.47 ± 15% -0.1 0.33 ± 7% perf-profile.children.cycles-pp._find_next_bit 0.17 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp._cond_resched 0.25 ± 20% -0.1 0.12 ± 23% perf-profile.children.cycles-pp.__x64_sys_close 0.27 ± 18% -0.1 0.14 ± 38% perf-profile.children.cycles-pp.update_ts_time_stats 0.50 ± 13% -0.1 0.36 ± 13% perf-profile.children.cycles-pp.tcp_create_openreq_child 0.17 ± 9% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.validate_xmit_skb 0.48 ± 9% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.update_rq_clock 0.18 ± 16% -0.1 0.05 ± 58% perf-profile.children.cycles-pp.__might_sleep 0.21 ± 14% -0.1 0.08 ± 59% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit 0.23 ± 10% -0.1 0.10 ± 19% perf-profile.children.cycles-pp.rcu_eqs_exit 0.25 ± 11% -0.1 0.13 ± 22% perf-profile.children.cycles-pp.nr_iowait_cpu 0.19 ± 11% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.__fget_light 0.21 ± 15% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.run_ksoftirqd 0.20 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.select_idle_sibling 0.20 ± 24% -0.1 0.08 ± 74% perf-profile.children.cycles-pp.wake_up_klogd_work_func 0.15 ± 10% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.update_cfs_group 0.21 ± 23% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.36 ± 6% -0.1 0.25 ± 16% perf-profile.children.cycles-pp.__netif_receive_skb_core 0.23 ± 19% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.smpboot_thread_fn 0.44 ± 7% -0.1 0.34 ± 4% perf-profile.children.cycles-pp.tcp_send_fin 1.51 ± 5% -0.1 1.41 ± 5% perf-profile.children.cycles-pp.wait_woken 0.14 ± 19% -0.1 0.03 ±100% perf-profile.children.cycles-pp.simple_copy_to_iter 0.16 ± 29% -0.1 0.06 ± 66% perf-profile.children.cycles-pp.rb_next 0.36 ± 12% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.inet_csk_destroy_sock 0.27 ± 13% -0.1 0.17 ± 16% perf-profile.children.cycles-pp.__sk_dst_check 0.76 ± 7% -0.1 0.67 ± 6% perf-profile.children.cycles-pp.dequeue_entity 0.15 ± 12% -0.1 0.06 ± 58% perf-profile.children.cycles-pp.sk_forced_mem_schedule 0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.inet_reqsk_alloc 0.27 ± 6% -0.1 0.18 ± 15% perf-profile.children.cycles-pp.memcpy_erms 0.37 ± 11% -0.1 0.28 ± 15% perf-profile.children.cycles-pp.inet_csk_clone_lock 0.14 ± 18% -0.1 0.05 ± 64% perf-profile.children.cycles-pp.inet_sk_rebuild_header 0.42 ± 8% -0.1 0.34 ± 16% perf-profile.children.cycles-pp.native_irq_return_iret 0.18 ± 10% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.rcu_eqs_enter 0.11 ± 24% -0.1 0.04 ±107% perf-profile.children.cycles-pp.native_apic_mem_write 0.15 ± 25% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.tcp_current_mss 0.19 ± 10% -0.1 0.12 ± 17% perf-profile.children.cycles-pp._copy_to_iter 0.11 ± 14% -0.1 0.04 ± 60% perf-profile.children.cycles-pp.reweight_entity 0.16 ± 23% -0.1 0.09 ± 37% perf-profile.children.cycles-pp.menu_reflect 0.12 ± 8% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.sock_recvmsg 0.12 ± 29% -0.1 0.05 ± 67% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust 0.12 ± 19% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.tcp_release_cb 0.10 ± 23% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.run_local_timers 0.25 ± 12% -0.1 0.19 ± 14% perf-profile.children.cycles-pp.__tcp_send_ack 0.09 ± 25% -0.1 0.03 ±100% perf-profile.children.cycles-pp.rb_insert_color 0.26 ± 5% -0.1 0.20 ± 10% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.12 ± 3% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.security_socket_recvmsg 0.12 ± 28% -0.1 0.07 ± 31% perf-profile.children.cycles-pp.arch_cpu_idle_enter 0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.tcp_add_backlog 0.12 ± 13% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.security_sk_alloc 0.11 ± 13% -0.1 0.06 ± 20% perf-profile.children.cycles-pp.apparmor_sk_alloc_security 0.19 ± 13% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.tcp_make_synack 0.07 ± 11% -0.0 0.03 ±102% perf-profile.children.cycles-pp.security_socket_getsockopt 0.20 ± 6% -0.0 0.16 ± 10% perf-profile.children.cycles-pp.sk_prot_alloc 0.10 ± 27% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.tcp_get_metrics 0.11 ± 17% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__update_load_avg_se 0.11 ± 14% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.pick_next_entity 0.11 ± 9% -0.0 0.08 ± 23% perf-profile.children.cycles-pp.inode_init_always 0.15 ± 12% -0.0 0.12 ± 15% perf-profile.children.cycles-pp.remove_wait_queue 0.07 ± 16% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.read 0.06 ± 14% +0.0 0.09 ± 17% perf-profile.children.cycles-pp.forkshell 0.07 ± 13% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.security_sk_free 0.06 +0.0 0.10 ± 15% perf-profile.children.cycles-pp.inet_csk_reqsk_queue_drop 0.14 ± 7% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.evict 0.11 ± 6% +0.0 0.15 ± 12% perf-profile.children.cycles-pp.sock_alloc_inode 0.06 ± 66% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.add_wait_queue 0.03 ±100% +0.1 0.08 ± 16% perf-profile.children.cycles-pp.tick_sched_do_timer 0.08 ± 17% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.try_to_del_timer_sync 0.04 ± 58% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.down_write 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_group_exit 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_exit 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.alloc_set_pte 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.bpf_sk_storage_free 0.09 ± 38% +0.1 0.15 ± 19% perf-profile.children.cycles-pp.execve 0.01 ±173% +0.1 0.07 ± 10% perf-profile.children.cycles-pp.update_rt_rq_load_avg 0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.sock_owned_by_me 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.clear_inode 0.01 ±173% +0.1 0.07 ± 22% perf-profile.children.cycles-pp.__sk_free 0.00 +0.1 0.06 ± 17% perf-profile.children.cycles-pp.__fd_install 0.18 ± 15% +0.1 0.25 ± 12% perf-profile.children.cycles-pp.del_timer 0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.__libc_start_main 0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.main 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.update_dl_rq_load_avg 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.rcu_read_lock_bh_held 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.lockdep_init_map 0.00 +0.1 0.07 ± 25% perf-profile.children.cycles-pp.netlink_has_listeners 0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.ksys_read 0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.vfs_read 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.register_lock_class 0.00 +0.1 0.07 ± 21% perf-profile.children.cycles-pp.inet_twsk_hashdance 0.32 ± 9% +0.1 0.39 ± 5% perf-profile.children.cycles-pp.alloc_empty_file 0.01 ±173% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.cmd_stat 0.00 +0.1 0.07 ± 15% perf-profile.children.cycles-pp.inet_csk_init_xmit_timers 0.32 ± 6% +0.1 0.39 ± 9% perf-profile.children.cycles-pp.inet_create 0.31 ± 11% +0.1 0.38 ± 6% perf-profile.children.cycles-pp.__alloc_file 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.do_filp_open 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.path_openat 0.01 ±173% +0.1 0.09 ± 7% perf-profile.children.cycles-pp.__note_gp_changes 0.00 +0.1 0.08 ± 20% perf-profile.children.cycles-pp.trace_hardirqs_off_caller 0.33 ± 6% +0.1 0.41 ± 9% perf-profile.children.cycles-pp.tcp_done 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.inet_sk_state_store 0.14 ± 7% +0.1 0.22 ± 13% perf-profile.children.cycles-pp.sk_alloc 0.06 ± 13% +0.1 0.14 ± 32% perf-profile.children.cycles-pp.rcu_gp_kthread 0.03 ±100% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.tcp_update_metrics 0.04 ± 59% +0.1 0.12 ± 27% perf-profile.children.cycles-pp._copy_from_user 0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.inet_put_port 0.16 ± 17% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__x64_sys_setsockopt 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.detach_if_pending 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.cgroup_sk_alloc 0.01 ±173% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.enqueue_timer 0.21 ± 8% +0.1 0.30 ± 7% perf-profile.children.cycles-pp.__inet_bind 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_open 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_openat2 0.15 ± 14% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__sys_setsockopt 0.03 ±100% +0.1 0.12 ± 11% perf-profile.children.cycles-pp.filemap_map_pages 0.01 ±173% +0.1 0.10 ± 4% perf-profile.children.cycles-pp.__update_idle_core 0.00 +0.1 0.09 ± 27% perf-profile.children.cycles-pp.ret_from_intr 0.17 ± 10% +0.1 0.26 ± 9% perf-profile.children.cycles-pp.newidle_balance 0.00 +0.1 0.10 ± 11% perf-profile.children.cycles-pp.init_timer_key 0.03 ±100% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.pick_next_task_idle 0.00 +0.1 0.10 ± 21% perf-profile.children.cycles-pp.do_raw_spin_trylock 0.00 +0.1 0.10 ± 18% perf-profile.children.cycles-pp._raw_write_lock_bh 0.01 ±173% +0.1 0.12 ± 3% perf-profile.children.cycles-pp.set_next_task_idle 0.00 +0.1 0.11 ± 20% perf-profile.children.cycles-pp.restore_regs_and_return_to_kernel 0.15 ± 10% +0.1 0.26 ± 16% perf-profile.children.cycles-pp.inet_csk_route_req 0.09 ± 8% +0.1 0.21 ± 5% perf-profile.children.cycles-pp.sock_setsockopt 0.24 ± 10% +0.1 0.36 ± 6% perf-profile.children.cycles-pp.__kmalloc_reserve 0.22 ± 8% +0.1 0.33 ± 9% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.00 +0.1 0.12 ± 32% perf-profile.children.cycles-pp.__local_bh_enable 2.08 ± 2% +0.1 2.21 ± 3% perf-profile.children.cycles-pp.sk_wait_data 0.39 ± 7% +0.1 0.51 ± 9% perf-profile.children.cycles-pp.tcp_mstamp_refresh 0.09 ± 20% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.tcp_set_state 0.09 ± 26% +0.1 0.22 ± 11% perf-profile.children.cycles-pp.del_timer_sync 0.03 ±100% +0.1 0.15 ± 14% perf-profile.children.cycles-pp.cpuacct_charge 0.07 ± 62% +0.1 0.20 ± 6% perf-profile.children.cycles-pp.__handle_mm_fault 0.06 ± 61% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.handle_pte_fault 0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch 0.00 +0.1 0.14 ± 7% perf-profile.children.cycles-pp.trace_softirqs_on 0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.lock_unpin_lock 0.34 ± 19% +0.1 0.48 ± 10% perf-profile.children.cycles-pp.enqueue_to_backlog 0.09 ± 8% +0.1 0.24 ± 11% perf-profile.children.cycles-pp.dput 0.20 ± 13% +0.1 0.35 ± 7% perf-profile.children.cycles-pp.__kmalloc_node_track_caller 0.00 +0.1 0.15 ± 29% perf-profile.children.cycles-pp.trace_softirqs_off 0.00 +0.2 0.15 ± 12% perf-profile.children.cycles-pp.lockdep_rtnl_is_held 0.00 +0.2 0.15 ± 14% perf-profile.children.cycles-pp.trace_hardirqs_on_thunk 0.23 ± 12% +0.2 0.39 ± 11% perf-profile.children.cycles-pp.__sk_destruct 0.07 ± 64% +0.2 0.22 ± 4% perf-profile.children.cycles-pp.handle_mm_fault 0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.__raw_spin_lock_init 0.00 +0.2 0.16 ± 15% perf-profile.children.cycles-pp._raw_spin_unlock_bh 0.00 +0.2 0.16 ± 13% perf-profile.children.cycles-pp.lock_pin_lock 0.06 ± 11% +0.2 0.22 ± 14% perf-profile.children.cycles-pp.trigger_load_balance 0.10 ± 28% +0.2 0.26 ± 4% perf-profile.children.cycles-pp.do_page_fault 0.22 ± 21% +0.2 0.39 ± 15% perf-profile.children.cycles-pp.fib_table_lookup 0.11 ± 31% +0.2 0.28 ± 6% perf-profile.children.cycles-pp.page_fault 0.16 ± 7% +0.2 0.33 ± 9% perf-profile.children.cycles-pp.sk_filter_trim_cap 0.16 ± 49% +0.2 0.34 ± 29% perf-profile.children.cycles-pp.enqueue_hrtimer 0.00 +0.2 0.18 ± 14% perf-profile.children.cycles-pp._raw_spin_lock_nested 0.19 ± 26% +0.2 0.38 ± 25% perf-profile.children.cycles-pp.finish_task_switch 0.09 ± 21% +0.2 0.29 ± 9% perf-profile.children.cycles-pp.___perf_sw_event 0.19 ± 14% +0.2 0.40 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.00 +0.2 0.21 ± 31% perf-profile.children.cycles-pp.trace_hardirqs_off_thunk 0.00 +0.2 0.21 ± 11% perf-profile.children.cycles-pp.tracer_hardirqs_on 0.11 ± 15% +0.2 0.33 ± 12% perf-profile.children.cycles-pp.raise_softirq 0.18 ± 4% +0.2 0.40 ± 8% perf-profile.children.cycles-pp.kfree 0.31 +0.2 0.53 ± 14% perf-profile.children.cycles-pp.ktime_get_with_offset 1.40 ± 5% +0.2 1.63 ± 2% perf-profile.children.cycles-pp.dev_hard_start_xmit 0.39 ± 9% +0.3 0.65 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc 0.22 ± 37% +0.3 0.49 ± 9% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.3 0.27 ± 12% perf-profile.children.cycles-pp.tracer_hardirqs_off 0.18 ± 26% +0.3 0.46 ± 8% perf-profile.children.cycles-pp.update_blocked_averages 0.27 ± 8% +0.3 0.54 ± 3% perf-profile.children.cycles-pp.update_curr 0.00 +0.3 0.28 ± 6% perf-profile.children.cycles-pp.debug_check_no_locks_freed 0.00 +0.3 0.29 ± 14% perf-profile.children.cycles-pp.do_raw_spin_unlock 0.00 +0.3 0.32 ± 10% perf-profile.children.cycles-pp.reacquire_held_locks 0.39 ± 10% +0.4 0.74 ± 13% perf-profile.children.cycles-pp.kmem_cache_free 0.31 ± 7% +0.4 0.67 ± 5% perf-profile.children.cycles-pp.ip_route_output_flow 0.39 ± 12% +0.4 0.75 ± 7% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu 0.48 ± 51% +0.4 0.85 ± 18% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.00 +0.4 0.38 ± 13% perf-profile.children.cycles-pp.__local_bh_disable_ip 0.62 ± 11% +0.4 1.01 ± 4% perf-profile.children.cycles-pp.netif_rx_internal 0.00 +0.4 0.40 ± 10% perf-profile.children.cycles-pp.trace_hardirqs_on 2.03 ± 6% +0.4 2.44 ± 4% perf-profile.children.cycles-pp.__dev_queue_xmit 0.33 ± 28% +0.4 0.75 ± 40% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.4 0.42 ± 7% perf-profile.children.cycles-pp.fs_reclaim_acquire 0.20 ± 6% +0.4 0.63 ± 6% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.00 +0.4 0.43 ± 21% perf-profile.children.cycles-pp._raw_spin_unlock_irq 0.43 ± 11% +0.4 0.87 ± 6% perf-profile.children.cycles-pp.ip_route_output_key_hash 0.47 ± 15% +0.4 0.91 ± 20% perf-profile.children.cycles-pp.scheduler_tick 2.42 +0.4 2.87 ± 7% perf-profile.children.cycles-pp.tcp_data_queue 0.13 ± 20% +0.4 0.58 ± 10% perf-profile.children.cycles-pp.__might_fault 0.13 ± 18% +0.5 0.58 ± 8% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.28 ± 65% +0.5 0.74 ± 5% perf-profile.children.cycles-pp.run_timer_softirq 0.00 +0.6 0.58 ± 5% perf-profile.children.cycles-pp.rcu_is_watching 0.64 ± 10% +0.6 1.23 ± 4% perf-profile.children.cycles-pp.netif_rx 0.70 ± 34% +0.7 1.36 ± 34% perf-profile.children.cycles-pp.irq_enter 0.92 ± 11% +0.7 1.60 ± 4% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.49 ± 10% +0.8 1.25 ± 3% perf-profile.children.cycles-pp.___might_sleep 0.00 +0.8 0.79 ± 5% perf-profile.children.cycles-pp.trace_hardirqs_off 0.00 +0.8 0.82 ± 5% perf-profile.children.cycles-pp.find_held_lock 0.22 ± 8% +0.9 1.08 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq 1.29 ± 16% +0.9 2.23 ± 25% perf-profile.children.cycles-pp.update_process_times 1.35 ± 16% +0.9 2.29 ± 26% perf-profile.children.cycles-pp.tick_sched_handle 0.26 ± 31% +0.9 1.20 ± 13% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.00 +1.0 1.03 ± 2% perf-profile.children.cycles-pp.mark_held_locks 0.00 +1.0 1.04 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock 0.70 ± 14% +1.1 1.76 ± 4% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.49 ± 11% +1.1 1.58 ± 30% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.00 +1.1 1.15 ± 11% perf-profile.children.cycles-pp.lockdep_hardirqs_off 2.52 ± 10% +1.2 3.70 ± 2% perf-profile.children.cycles-pp.rcu_core 0.00 +1.2 1.19 ± 3% perf-profile.children.cycles-pp.match_held_lock 1.65 ± 14% +1.3 2.90 ± 23% perf-profile.children.cycles-pp.tick_sched_timer 0.00 +1.4 1.37 ± 11% perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online 3.23 ± 13% +1.6 4.83 ± 11% perf-profile.children.cycles-pp.menu_select 0.00 +1.7 1.68 ± 3% perf-profile.children.cycles-pp.rcu_read_lock_held 2.81 ± 9% +1.7 4.56 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +1.8 1.76 ± 4% perf-profile.children.cycles-pp.lockdep_hardirqs_on 0.00 +1.8 1.79 ± 12% perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled 1.40 ± 24% +1.8 3.21 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event 0.00 +2.2 2.19 perf-profile.children.cycles-pp.mark_lock 1.68 ± 20% +2.2 3.88 ± 13% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +2.5 2.46 ± 9% perf-profile.children.cycles-pp.__lock_text_start 0.00 +2.7 2.65 ± 4% perf-profile.children.cycles-pp.rcu_read_lock_held_common 4.25 ± 6% +2.9 7.11 ± 2% perf-profile.children.cycles-pp.irq_exit 0.00 +3.0 2.97 ± 4% perf-profile.children.cycles-pp.do_raw_spin_lock 2.18 ± 33% +3.4 5.57 ± 4% perf-profile.children.cycles-pp.ktime_get 2.75 ± 16% +3.9 6.65 ± 25% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.00 +4.7 4.71 ± 3% perf-profile.children.cycles-pp.lock_release 0.00 +4.7 4.74 perf-profile.children.cycles-pp.rcu_read_lock_sched_held 0.00 +6.2 6.18 ± 2% perf-profile.children.cycles-pp.lock_is_held_type 4.58 ± 8% +6.6 11.14 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt 49.50 ± 3% +7.4 56.87 perf-profile.children.cycles-pp.cpuidle_enter 49.48 ± 3% +7.4 56.85 perf-profile.children.cycles-pp.cpuidle_enter_state 56.39 ± 2% +8.1 64.48 perf-profile.children.cycles-pp.do_idle 56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.secondary_startup_64 56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.cpu_startup_entry 54.68 ± 3% +8.7 63.34 perf-profile.children.cycles-pp.start_secondary 10.58 ± 4% +10.5 21.06 ± 9% perf-profile.children.cycles-pp.apic_timer_interrupt 9.66 ± 4% +10.6 20.30 ± 9% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 0.00 +10.8 10.77 ± 3% perf-profile.children.cycles-pp.__lock_acquire 0.00 +14.2 14.15 ± 2% perf-profile.children.cycles-pp.lock_acquire 2.65 ± 2% -2.3 0.32 ± 8% perf-profile.self.cycles-pp.poll_idle 2.35 ± 8% -1.8 0.58 ± 6% perf-profile.self.cycles-pp.do_syscall_64 1.62 ± 6% -1.5 0.12 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.47 ± 6% -1.1 0.32 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64 1.45 ± 7% -1.1 0.35 ± 10% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.81 ± 9% -0.6 0.17 ± 16% perf-profile.self.cycles-pp.__inet_lookup_established 1.14 ± 5% -0.6 0.56 ± 13% perf-profile.self.cycles-pp.read_tsc 1.26 ± 15% -0.6 0.70 ± 21% perf-profile.self.cycles-pp.menu_select 0.81 ± 8% -0.5 0.32 ± 4% perf-profile.self.cycles-pp.__sched_text_start 0.56 ± 10% -0.4 0.17 ± 15% perf-profile.self.cycles-pp.tcp_recvmsg 0.63 ± 4% -0.4 0.25 ± 14% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.50 ± 10% -0.4 0.14 ± 19% perf-profile.self.cycles-pp.__tcp_transmit_skb 0.46 ± 10% -0.4 0.11 ± 7% perf-profile.self.cycles-pp.tcp_ack 0.42 ± 12% -0.3 0.11 ± 12% perf-profile.self.cycles-pp.__dev_queue_xmit 0.47 ± 6% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.tcp_v4_rcv 0.50 ± 14% -0.3 0.22 ± 7% perf-profile.self.cycles-pp.do_idle 0.57 ± 7% -0.3 0.29 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.48 ± 12% -0.3 0.21 ± 4% perf-profile.self.cycles-pp.___might_sleep 0.35 ± 12% -0.3 0.09 ± 18% perf-profile.self.cycles-pp.skb_release_data 0.36 ± 5% -0.3 0.10 ± 11% perf-profile.self.cycles-pp.__netif_receive_skb_core 0.38 ± 11% -0.3 0.12 ± 9% perf-profile.self.cycles-pp.set_next_entity 0.29 ± 17% -0.2 0.04 ± 60% perf-profile.self.cycles-pp.sock_def_readable 0.31 ± 12% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.tcp_clean_rtx_queue 0.26 ± 11% -0.2 0.03 ±100% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.29 ± 12% -0.2 0.06 ± 64% perf-profile.self.cycles-pp.dst_release 0.35 ± 10% -0.2 0.12 ± 5% perf-profile.self.cycles-pp.ip_finish_output2 0.69 ± 4% -0.2 0.47 ± 6% perf-profile.self.cycles-pp.native_write_msr 0.32 ± 9% -0.2 0.10 ± 19% perf-profile.self.cycles-pp.__switch_to 0.35 ± 14% -0.2 0.14 ± 21% perf-profile.self.cycles-pp.update_rq_clock 0.29 ± 8% -0.2 0.09 ± 12% perf-profile.self.cycles-pp.__ip_queue_xmit 0.27 ± 20% -0.2 0.07 ± 10% perf-profile.self.cycles-pp.ipv4_mtu 1.03 ± 7% -0.2 0.83 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state 0.24 ± 8% -0.2 0.05 ± 60% perf-profile.self.cycles-pp.tcp_sendmsg_locked 0.28 ± 5% -0.2 0.10 ± 23% perf-profile.self.cycles-pp.__slab_free 0.27 ± 7% -0.2 0.09 ± 21% perf-profile.self.cycles-pp.select_task_rq_fair 0.22 ± 6% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.__ksize 0.20 ± 20% -0.2 0.03 ±105% perf-profile.self.cycles-pp.net_rx_action 0.39 ± 15% -0.2 0.22 ± 9% perf-profile.self.cycles-pp.native_sched_clock 0.21 ± 8% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.aa_sk_perm 0.24 ± 9% -0.2 0.07 ± 15% perf-profile.self.cycles-pp.loopback_xmit 0.22 ± 12% -0.2 0.06 ± 16% perf-profile.self.cycles-pp.enqueue_entity 0.25 ± 10% -0.2 0.09 ± 17% perf-profile.self.cycles-pp.process_backlog 0.22 ± 21% -0.2 0.06 ± 20% perf-profile.self.cycles-pp.fib_table_lookup 0.19 ± 16% -0.2 0.04 ±107% perf-profile.self.cycles-pp.ipv4_dst_check 0.23 ± 14% -0.2 0.08 ± 21% perf-profile.self.cycles-pp.tcp_rcv_established 0.19 ± 23% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.enqueue_to_backlog 0.23 ± 11% -0.1 0.08 ± 23% perf-profile.self.cycles-pp.__switch_to_asm 0.19 ± 16% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.sock_rfree 0.35 ± 14% -0.1 0.20 ± 17% perf-profile.self.cycles-pp.note_gp_changes 0.46 ± 14% -0.1 0.32 ± 6% perf-profile.self.cycles-pp._find_next_bit 0.21 ± 18% -0.1 0.08 ± 27% perf-profile.self.cycles-pp.kmem_cache_alloc 0.21 ± 23% -0.1 0.07 ± 17% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.21 ± 14% -0.1 0.08 ± 59% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit 0.19 ± 11% -0.1 0.07 ± 13% perf-profile.self.cycles-pp.__fget_light 0.16 ± 16% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.__might_sleep 0.24 ± 12% -0.1 0.13 ± 22% perf-profile.self.cycles-pp.nr_iowait_cpu 0.16 ± 2% -0.1 0.05 ± 60% perf-profile.self.cycles-pp.tcp_rcv_state_process 0.14 ± 24% -0.1 0.03 ±100% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.15 ± 10% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.update_cfs_group 0.17 ± 8% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.tcp_write_xmit 0.14 ± 13% -0.1 0.04 ±100% perf-profile.self.cycles-pp.__kmalloc_node_track_caller 0.19 ± 18% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.17 ± 11% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.mod_timer 0.17 ± 17% -0.1 0.08 ± 20% perf-profile.self.cycles-pp.ip_local_deliver 0.14 ± 15% -0.1 0.05 ± 58% perf-profile.self.cycles-pp.sk_forced_mem_schedule 0.27 ± 6% -0.1 0.18 ± 13% perf-profile.self.cycles-pp.memcpy_erms 0.18 ± 8% -0.1 0.09 ± 17% perf-profile.self.cycles-pp.ip_output 0.14 ± 27% -0.1 0.06 ± 66% perf-profile.self.cycles-pp.rb_next 0.18 ± 10% -0.1 0.10 ± 15% perf-profile.self.cycles-pp.rcu_eqs_enter 0.42 ± 8% -0.1 0.34 ± 16% perf-profile.self.cycles-pp.native_irq_return_iret 0.30 ± 14% -0.1 0.22 ± 11% perf-profile.self.cycles-pp.__softirqentry_text_start 0.14 ± 8% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.kmem_cache_alloc_node 0.11 ± 14% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.reweight_entity 0.18 ± 6% -0.1 0.11 ± 21% perf-profile.self.cycles-pp.kfree 0.12 ± 20% -0.1 0.05 ± 67% perf-profile.self.cycles-pp.menu_reflect 0.12 ± 12% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.dequeue_entity 0.12 ± 13% -0.1 0.07 ± 25% perf-profile.self.cycles-pp.newidle_balance 0.08 ± 23% -0.1 0.03 ±100% perf-profile.self.cycles-pp.run_local_timers 0.08 ± 19% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rb_insert_color 0.24 ± 12% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.__next_timer_interrupt 0.12 ± 18% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.resched_curr 0.11 ± 14% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__update_load_avg_se 0.12 ± 24% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.update_curr 0.10 ± 14% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu 0.09 ± 15% -0.0 0.04 ± 59% perf-profile.self.cycles-pp.rcu_eqs_exit 0.07 ± 22% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.dev_hard_start_xmit 0.11 ± 4% -0.0 0.08 ± 22% perf-profile.self.cycles-pp.finish_task_switch 0.10 ± 15% -0.0 0.07 ± 21% perf-profile.self.cycles-pp.inet_csk_accept 0.08 ± 8% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.release_sock 0.08 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_idle_exit 0.05 ± 62% +0.0 0.08 ± 15% perf-profile.self.cycles-pp.__wake_up_common 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.register_lock_class 0.15 ± 27% +0.1 0.22 ± 15% perf-profile.self.cycles-pp.rcu_core 0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.reacquire_held_locks 0.03 ±100% +0.1 0.12 ± 18% perf-profile.self.cycles-pp.smp_apic_timer_interrupt 0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp._raw_spin_unlock 0.00 +0.1 0.10 ± 25% perf-profile.self.cycles-pp.__lock_text_start 0.00 +0.1 0.10 ± 21% perf-profile.self.cycles-pp.do_raw_spin_trylock 0.00 +0.1 0.11 ± 10% perf-profile.self.cycles-pp.lock_pin_lock 0.00 +0.1 0.12 ± 14% perf-profile.self.cycles-pp.lock_unpin_lock 0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.trace_softirqs_on 0.00 +0.1 0.14 ± 31% perf-profile.self.cycles-pp.trace_softirqs_off 0.00 +0.2 0.18 ± 11% perf-profile.self.cycles-pp.rcu_read_lock_held 0.00 +0.2 0.18 ± 17% perf-profile.self.cycles-pp.tracer_hardirqs_on 0.00 +0.2 0.21 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_on 0.00 +0.2 0.24 ± 10% perf-profile.self.cycles-pp.tracer_hardirqs_off 0.00 +0.3 0.28 ± 13% perf-profile.self.cycles-pp.do_raw_spin_unlock 0.00 +0.3 0.28 ± 6% perf-profile.self.cycles-pp.debug_check_no_locks_freed 0.00 +0.3 0.28 ± 3% perf-profile.self.cycles-pp.__local_bh_disable_ip 0.00 +0.4 0.39 ± 14% perf-profile.self.cycles-pp.rcu_read_lock_sched_held 0.00 +0.4 0.43 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_off 0.00 +0.5 0.48 ± 6% perf-profile.self.cycles-pp.rcu_is_watching 0.00 +0.5 0.51 ± 9% perf-profile.self.cycles-pp.find_held_lock 0.00 +0.5 0.53 ± 3% perf-profile.self.cycles-pp.mark_held_locks 0.00 +0.7 0.68 ± 8% perf-profile.self.cycles-pp.rcu_read_lock_held_common 0.00 +0.8 0.75 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_on 0.00 +0.9 0.88 ± 6% perf-profile.self.cycles-pp.match_held_lock 0.00 +1.1 1.13 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_off 0.00 +1.3 1.30 ± 5% perf-profile.self.cycles-pp.do_raw_spin_lock 0.00 +1.3 1.31 ± 11% perf-profile.self.cycles-pp.rcu_lockdep_current_cpu_online 0.00 +1.5 1.55 ± 12% perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled 0.00 +2.1 2.07 perf-profile.self.cycles-pp.mark_lock 0.00 +3.3 3.29 ± 2% perf-profile.self.cycles-pp.lock_release 0.00 +3.3 3.30 ± 2% perf-profile.self.cycles-pp.lock_acquire 0.00 +5.4 5.41 perf-profile.self.cycles-pp.lock_is_held_type 0.00 +9.1 9.07 ± 4% perf-profile.self.cycles-pp.__lock_acquire Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

Re: [Jfs-discussion] [fs] 05c5a0273b: netperf.Throughput_total_tps -71.8% regression

by Rong Chen

On 5/14/20 12:27 PM, Christian Kujau wrote: > On Tue, 12 May 2020, kernel test robot wrote: >> FYI, we noticed a -71.8% regression of netperf.Throughput_total_tps due to commit: > As noted in this report, netperf is used to "measure various aspect of > networking performance". Are we sure the bisect is correct? JFS is a > filesystem and is not touching net/ in any way. So, having not attempted > to reproduce this, maybe the JFS commit is a red herring? > > C. Hi, The commit also causes -74.6% regression of will-it-scale.per_thread_ops: in testcase: will-it-scale on test machine: 8 threads Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz with 16G memory with following parameters: nr_task: 100% mode: thread test: unlink2 cpufreq_governor: performance ucode: 0x21 I'll send another report for this regression. Best Regards, Rong Chen

8 months, 2 weeks

1
0
0 / 0

Re: fd21dfbb51: BUG:kernel_NULL_pointer_dereference,address

by Rong Chen

On 5/13/20 5:48 PM, Christian Brauner wrote: > On Wed, May 13, 2020 at 04:50:09PM +0800, kernel test robot wrote: >> Greeting, >> >> FYI, we noticed the following commit (built with gcc-7): >> >> commit: fd21dfbb51796e455d734a4b0ba412cfe48b678f ("nsproxy: attach to namespaces via pidfds") >> https://git.kernel.org/cgit/linux/kernel/git/brauner/linux.git setns_pidfd > Well you just saved my ass! :) > > I've pushed a tiny fix for that obvious braino to > > https://git.kernel.org/cgit/linux/kernel/git/brauner/linux.git setns_pidfd > > just now. > Can you rerun this test on there, please? I tested the head commit 2b40c5db73e239531ea54991087f4edc07fbb08e, the problem has been fixed. Best Regards, Rong Chen

8 months, 2 weeks

1
0
0 / 0

Re: [tracing] 06e0a548ba: WARNING:at_kernel/trace/ring_buffer.c:#ring_buffer_iter_peek

by Steven Rostedt

On Wed, 13 May 2020 18:15:57 +0200 Sven Schnelle <svens(a)linux.ibm.com> wrote: > Thanks for looking into this. I've attached my /proc/config.gz to this Mail. > The x86 system is my Laptop which is a Thinkpad X280 with 4 HT CPUs (so 8 cpus > in total). I've tried disabling preemption, but this didn't help. > > It's always this check that causes the loop: > > if (iter->head >= rb_page_size(iter->head_page)) { > rb_inc_iter(iter); > goto again; > } > > On the first loop iter->head is some value > 0 and rb_page_size returns > 0, afterwards it gets twice to this check with both values 0. The third > time the warning is triggered. Maybe that information helps. I figured out what was causing this, and that's just that the writer and the iterator could end up almost "in sync" where the writer writes to each of the pages the iterator is trying to read, and this can trigger the three failures of "zero commits" per page. I had a way to detect this, but then realized that it may be possible for an active writer to possibly trigger the other failures to get an event, that I just decided to force it to try three times, and simply return NULL if an active writer is messing with the iterator. The iterator is a "best effort" to read the buffer if there's an active writer. The consumer read (trace_pipe) is made for that. This patch should solve you issues. (care to give a Tested-by: if it works for you?) -- Steve From 29b36cd00ddcae278430e70e55dd64855e9a54b3 Mon Sep 17 00:00:00 2001 From: "Steven Rostedt (VMware)" <rostedt(a)goodmis.org> Date: Wed, 13 May 2020 15:18:01 -0400 Subject: [PATCH] ring-buffer: Don't deactivate the ring buffer on failed iterator reads If the function tracer is running and the trace file is read (which uses the ring buffer iterator), the iterator can get in sync with the writes, and caues it to fail to find a page with content it can read three times. This causes a warning and deactivation of the ring buffer code. Looking at the other cases of failure to get an event, it appears that there's a chance that the writer could cause them too. Since the iterator is a "best effort" to read the ring buffer if there's an active writer (the consumer reader is made for this case "see trace_pipe"), if it fails to get an event after three tries, simply give up and return NULL. Don't warn, nor disable the ring buffer on this failure. Link: https://lore.kernel.org/r/20200429090508.GG5770@shao2-debian Reported-by: kernel test robot <lkp(a)intel.com> Fixes: ff84c50cfb4b ("ring-buffer: Do not die if rb_iter_peek() fails more than thrice") Signed-off-by: Steven Rostedt (VMware) <rostedt(a)goodmis.org> --- kernel/trace/ring_buffer.c | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 6f0b42ceeb00..448d5f528764 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -4034,7 +4034,6 @@ rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts) struct ring_buffer_per_cpu *cpu_buffer; struct ring_buffer_event *event; int nr_loops = 0; - bool failed = false; if (ts) *ts = 0; @@ -4056,19 +4055,14 @@ rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts) return NULL; /* - * We repeat when a time extend is encountered or we hit - * the end of the page. Since the time extend is always attached - * to a data event, we should never loop more than three times. - * Once for going to next page, once on time extend, and - * finally once to get the event. - * We should never hit the following condition more than thrice, - * unless the buffer is very small, and there's a writer - * that is causing the reader to fail getting an event. + * As the writer can mess with what the iterator is trying + * to read, just give up if we fail to get an event after + * three tries. The iterator is not as reliable when reading + * the ring buffer with an active write as the consumer is. + * Do not warn if the three failures is reached. */ - if (++nr_loops > 3) { - RB_WARN_ON(cpu_buffer, !failed); + if (++nr_loops > 3) return NULL; - } if (rb_per_cpu_empty(cpu_buffer)) return NULL; @@ -4079,10 +4073,8 @@ rb_iter_peek(struct ring_buffer_iter *iter, u64 *ts) } event = rb_iter_head_event(iter); - if (!event) { - failed = true; + if (!event) goto again; - } switch (event->type_len) { case RINGBUF_TYPE_PADDING: -- 2.20.1

8 months, 2 weeks

1
0
0 / 0

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP May 2020