Greeting, FYI, we noticed a -14.4% regression of fsmark.files_per_sec due to commit: commit: 53c5e5a1d87e3341d06a00746c772dc7eff6a65b ("btrfs: simplify the logic in need_preemptive_flushing") https://github.com/kdave/btrfs-devel.git for-next-20210104 in testcase: fsmark on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: iterations: 1x nr_threads: 1t disk: 1BRD_48G fs: btrfs filesize: 4M test_size: 24G sync_method: fsyncBeforeClose cpufreq_governor: performance ucode: 0x5003003 test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload. test-url: https://sourceforge.net/projects/fsmark/ In addition to that, the commit also has significant impact on the following tests: +------------------+------------------------------------------------------------------------+ | testcase: change | fio-basic: fio.write_iops -59.9% regression | | test machine | 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory | | test parameters | bs=4k | | | cpufreq_governor=performance | | | disk=1SSD | | | fs=btrfs | | | ioengine=sync | | | nr_task=8 | | | runtime=300s | | | rw=write | | | test_size=256g | | | ucode=0x4003003 | +------------------+------------------------------------------------------------------------+ | testcase: change | aim7: aim7.jobs-per-min -3.5% regression | | test machine | 144 threads Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | disk=1BRD_48G | | | fs=btrfs | | | load=4 | | | test=creat-clo | | | ucode=0x700001e | +------------------+------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com&gt; Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode: gcc-9/performance/1BRD_48G/4M/btrfs/1x/x86_64-rhel-8.3/1t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2ap2/24G/fsmark/0x5003003 commit: 5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size") 53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing") 5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 0:4 1% 0:4 perf-profile.children.cycles-pp.error_entry 0:4 1% 0:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 239.48 -14.4% 205.05 fsmark.files_per_sec 27.49 +15.7% 31.79 fsmark.time.elapsed_time 27.49 +15.7% 31.79 fsmark.time.elapsed_time.max 26.81 +15.9% 31.08 fsmark.time.system_time 0.21 ± 3% -0.0 0.18 mpstat.cpu.all.usr% 1922 ± 11% +23.6% 2377 ± 7% sched_debug.cpu.avg_idle.min 836321 -12.2% 734615 vmstat.io.bo 1840 ± 9% -25.0% 1379 ± 17% meminfo.Writeback 1813949 ± 2% -12.0% 1595838 meminfo.max_used_kB 2706 ± 48% +221.4% 8700 ± 71% softirqs.NET_RX 1101600 +10.6% 1218682 softirqs.SCHED 6106677 ± 99% -99.8% 14876 ± 35% numa-numastat.node2.local_node 6193240 ± 98% -98.4% 101442 ± 5% numa-numastat.node2.numa_hit 346322 ± 75% -81.2% 64945 ± 57% numa-numastat.node3.other_node 4.651e+09 ± 23% +29.3% 6.013e+09 cpuidle.C1E.time 10275557 ± 15% +24.2% 12766663 cpuidle.C1E.usage 37470 ± 5% +34.0% 50206 ± 24% cpuidle.POLL.time 9995 +36.8% 13675 ± 9% cpuidle.POLL.usage 486.00 ±170% +117.8% 1058 ± 91% numa-meminfo.node0.Dirty 3199694 ±158% +97.8% 6328688 ± 94% numa-meminfo.node0.FilePages 2886568 ±173% +107.5% 5989331 ± 98% numa-meminfo.node0.Inactive(file) 5952654 ± 96% -98.3% 99890 ± 49% numa-meminfo.node2.Inactive 13431706 ± 92% -93.0% 936825 ± 12% numa-meminfo.node2.MemUsed 65018 ± 45% +153.4% 164769 ± 58% numa-meminfo.node3.AnonPages 65783 ± 44% +156.2% 168505 ± 57% numa-meminfo.node3.Inactive(anon) 6776 ± 5% +23.7% 8383 ± 14% numa-meminfo.node3.KernelStack 1129 ± 62% +387.2% 5504 ± 68% numa-meminfo.node3.PageTables 107132 ± 3% -5.8% 100954 ± 2% proc-vmstat.nr_anon_pages 109883 ± 3% -5.7% 103640 ± 2% proc-vmstat.nr_inactive_anon 425.25 ± 16% -21.4% 334.25 ± 13% proc-vmstat.nr_writeback 109883 ± 3% -5.7% 103640 ± 2% proc-vmstat.nr_zone_inactive_anon 593623 ± 3% +16.5% 691600 ± 8% proc-vmstat.numa_foreign 593623 ± 3% +16.5% 691600 ± 8% proc-vmstat.numa_miss 853428 ± 2% +11.5% 951407 ± 5% proc-vmstat.numa_other 150853 ± 5% +9.8% 165681 ± 2% proc-vmstat.pgfault 7385 ± 2% +13.0% 8347 proc-vmstat.pgreuse 796087 ±157% +98.2% 1577690 ± 94% numa-vmstat.node0.nr_file_pages 717788 ±173% +108.0% 1492825 ± 98% numa-vmstat.node0.nr_inactive_file 2259 ± 24% -20.2% 1803 ± 6% numa-vmstat.node0.nr_mapped 717788 ±173% +108.0% 1492819 ± 98% numa-vmstat.node0.nr_zone_inactive_file 15938 ± 26% -21.1% 12572 ± 21% numa-vmstat.node1.nr_slab_unreclaimable 16329 ± 44% +150.8% 40954 ± 58% numa-vmstat.node3.nr_anon_pages 16522 ± 44% +152.7% 41751 ± 58% numa-vmstat.node3.nr_inactive_anon 6772 ± 5% +23.7% 8374 ± 14% numa-vmstat.node3.nr_kernel_stack 280.75 ± 63% +388.2% 1370 ± 68% numa-vmstat.node3.nr_page_table_pages 16521 ± 44% +152.7% 41751 ± 58% numa-vmstat.node3.nr_zone_inactive_anon 499113 ± 5% +689.0% 3938239 ± 84% numa-vmstat.node3.numa_hit 384758 ± 12% +892.7% 3819327 ± 86% numa-vmstat.node3.numa_local 11.83 ± 2% -11.7% 10.44 perf-stat.i.MPKI 1.272e+09 +3.3% 1.314e+09 perf-stat.i.branch-instructions 1.78 -0.2 1.54 perf-stat.i.branch-miss-rate% 27489113 ± 3% -10.0% 24734118 perf-stat.i.branch-misses 69581732 ± 2% -7.2% 64558270 perf-stat.i.cache-references 2.57 ± 2% -9.6% 2.32 perf-stat.i.cpi 1.598e+10 ± 2% -7.1% 1.484e+10 perf-stat.i.cpu-cycles 251.20 -2.8% 244.05 perf-stat.i.cpu-migrations 7.777e+08 -5.2% 7.369e+08 perf-stat.i.dTLB-stores 56.13 ± 2% -1.5 54.62 perf-stat.i.iTLB-load-miss-rate% 5000319 ± 4% -4.6% 4768521 perf-stat.i.iTLB-load-misses 3899306 +1.6% 3960144 perf-stat.i.iTLB-loads 1311 ± 3% +6.6% 1397 perf-stat.i.instructions-per-iTLB-miss 0.41 +9.7% 0.44 perf-stat.i.ipc 7.20 ± 3% -13.5% 6.22 ± 5% perf-stat.i.major-faults 0.08 ± 2% -7.1% 0.08 perf-stat.i.metric.GHz 1297657 ± 3% -5.6% 1224413 ± 2% perf-stat.i.node-load-misses 10.63 -8.9% 9.69 perf-stat.overall.MPKI 2.16 ± 2% -0.3 1.88 perf-stat.overall.branch-miss-rate% 2.44 -8.8% 2.22 perf-stat.overall.cpi 56.15 ± 2% -1.5 54.63 perf-stat.overall.iTLB-load-miss-rate% 1313 ± 3% +6.4% 1397 perf-stat.overall.instructions-per-iTLB-miss 0.41 +9.6% 0.45 perf-stat.overall.ipc 1.226e+09 +3.5% 1.269e+09 perf-stat.ps.branch-instructions 26520603 ± 2% -10.1% 23847387 perf-stat.ps.branch-misses 67153636 ± 2% -6.9% 62502238 perf-stat.ps.cache-references 1.541e+10 ± 2% -6.8% 1.435e+10 perf-stat.ps.cpu-cycles 242.48 -2.7% 235.91 perf-stat.ps.cpu-migrations 7.492e+08 -4.9% 7.123e+08 perf-stat.ps.dTLB-stores 4819104 ± 4% -4.2% 4617865 perf-stat.ps.iTLB-load-misses 3758462 +2.0% 3835078 perf-stat.ps.iTLB-loads 6.317e+09 +2.1% 6.452e+09 perf-stat.ps.instructions 7.02 ± 3% -14.5% 6.00 perf-stat.ps.major-faults 1.807e+11 +16.6% 2.108e+11 perf-stat.total.instructions 0.86 ± 6% -0.1 0.75 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.82 ± 6% -0.1 0.71 ± 13% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.79 ± 6% +0.1 0.86 ± 2% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.00 +0.9 0.85 ± 15% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 5.52 ± 10% +1.9 7.38 ± 13% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 5.50 ± 10% +1.9 7.36 ± 12% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 5.60 ± 10% +1.9 7.47 ± 13% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.58 ± 10% +1.9 7.45 ± 13% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.41 ± 8% +2.0 9.40 ± 8% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.1 2.12 ± 12% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 0.00 +2.2 2.17 ± 12% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 0.00 +2.2 2.17 ± 12% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +2.2 2.23 ± 12% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.88 ± 7% -0.1 0.77 ± 14% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.52 ± 14% -0.1 0.41 ± 15% perf-profile.children.cycles-pp.end_page_writeback 0.46 ± 13% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.test_clear_page_writeback 0.24 ± 8% -0.0 0.19 ± 9% perf-profile.children.cycles-pp.__clear_extent_bit 0.24 ± 8% -0.0 0.19 ± 9% perf-profile.children.cycles-pp.clear_extent_bit 0.16 ± 5% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.alloc_extent_state 0.14 ± 6% +0.0 0.18 ± 21% perf-profile.children.cycles-pp.__schedule 0.31 ± 6% +0.0 0.35 ± 7% perf-profile.children.cycles-pp.update_rq_clock 0.04 ± 59% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush 0.04 ± 60% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.btrfs_alloc_tree_block 0.21 ± 4% +0.1 0.28 perf-profile.children.cycles-pp.arch_scale_freq_tick 0.18 ± 10% +0.1 0.26 ± 15% perf-profile.children.cycles-pp.btrfs_search_slot 0.12 ± 19% +0.1 0.20 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block 0.12 ± 19% +0.1 0.20 ± 15% perf-profile.children.cycles-pp.btrfs_cow_block 0.83 ± 3% +0.1 0.93 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.1 0.14 ± 17% perf-profile.children.cycles-pp.find_next_bit 0.12 ± 16% +0.6 0.67 ± 15% perf-profile.children.cycles-pp._find_next_bit 0.00 +1.0 0.99 ± 15% perf-profile.children.cycles-pp.cpumask_next 5.52 ± 10% +1.9 7.38 ± 13% perf-profile.children.cycles-pp.btrfs_file_write_iter 5.50 ± 10% +1.9 7.36 ± 13% perf-profile.children.cycles-pp.btrfs_buffered_write 7.49 ± 8% +2.0 9.48 ± 8% perf-profile.children.cycles-pp.ksys_write 7.42 ± 8% +2.0 9.41 ± 8% perf-profile.children.cycles-pp.new_sync_write 7.46 ± 8% +2.0 9.46 ± 8% perf-profile.children.cycles-pp.vfs_write 0.11 ± 12% +2.1 2.23 ± 12% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 0.06 ± 11% +2.2 2.25 ± 12% perf-profile.children.cycles-pp.__reserve_bytes 0.00 +2.2 2.19 ± 12% perf-profile.children.cycles-pp.__percpu_counter_sum 0.03 ±100% +2.2 2.24 ± 12% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 0.21 ± 4% +0.1 0.28 perf-profile.self.cycles-pp.arch_scale_freq_tick 0.83 ± 3% +0.1 0.92 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.1 0.14 ± 17% perf-profile.self.cycles-pp.find_next_bit 0.00 +0.3 0.30 ± 15% perf-profile.self.cycles-pp.cpumask_next 0.11 ± 14% +0.5 0.66 ± 16% perf-profile.self.cycles-pp._find_next_bit 0.00 +1.2 1.17 ± 10% perf-profile.self.cycles-pp.__percpu_counter_sum 2506 ± 18% -29.0% 1779 ± 19% interrupts.CPU0.CAL:Function_call_interrupts 57908 +15.2% 66697 interrupts.CPU0.LOC:Local_timer_interrupts 58115 +14.8% 66696 interrupts.CPU1.LOC:Local_timer_interrupts 58148 +14.7% 66677 interrupts.CPU10.LOC:Local_timer_interrupts 58110 +14.7% 66638 interrupts.CPU100.LOC:Local_timer_interrupts 58085 +14.7% 66623 interrupts.CPU101.LOC:Local_timer_interrupts 58066 +14.8% 66634 interrupts.CPU102.LOC:Local_timer_interrupts 57776 +15.4% 66656 interrupts.CPU103.LOC:Local_timer_interrupts 58053 +14.8% 66625 interrupts.CPU104.LOC:Local_timer_interrupts 58042 +14.5% 66472 interrupts.CPU105.LOC:Local_timer_interrupts 58036 +14.8% 66631 interrupts.CPU106.LOC:Local_timer_interrupts 58034 +14.8% 66597 interrupts.CPU107.LOC:Local_timer_interrupts 57578 ± 2% +15.7% 66607 interrupts.CPU108.LOC:Local_timer_interrupts 58018 +14.8% 66613 interrupts.CPU109.LOC:Local_timer_interrupts 58074 +14.7% 66640 interrupts.CPU11.LOC:Local_timer_interrupts 58051 +14.7% 66592 interrupts.CPU110.LOC:Local_timer_interrupts 58069 +14.8% 66653 interrupts.CPU111.LOC:Local_timer_interrupts 58050 +14.8% 66617 interrupts.CPU112.LOC:Local_timer_interrupts 58054 +14.8% 66638 interrupts.CPU113.LOC:Local_timer_interrupts 58097 +14.7% 66635 interrupts.CPU114.LOC:Local_timer_interrupts 58048 +14.7% 66602 interrupts.CPU115.LOC:Local_timer_interrupts 232.50 ± 20% -31.2% 160.00 ± 34% interrupts.CPU115.NMI:Non-maskable_interrupts 232.50 ± 20% -31.2% 160.00 ± 34% interrupts.CPU115.PMI:Performance_monitoring_interrupts 58068 +14.7% 66620 interrupts.CPU116.LOC:Local_timer_interrupts 58073 +14.7% 66606 interrupts.CPU117.LOC:Local_timer_interrupts 58073 +14.7% 66601 interrupts.CPU118.LOC:Local_timer_interrupts 57930 +15.0% 66637 interrupts.CPU119.LOC:Local_timer_interrupts 226.25 ± 21% -42.4% 130.25 ± 14% interrupts.CPU119.NMI:Non-maskable_interrupts 226.25 ± 21% -42.4% 130.25 ± 14% interrupts.CPU119.PMI:Performance_monitoring_interrupts 58108 +14.8% 66699 interrupts.CPU12.LOC:Local_timer_interrupts 58024 +14.8% 66604 interrupts.CPU120.LOC:Local_timer_interrupts 58007 +14.5% 66430 interrupts.CPU121.LOC:Local_timer_interrupts 58028 +14.8% 66625 interrupts.CPU122.LOC:Local_timer_interrupts 58032 +14.7% 66585 interrupts.CPU123.LOC:Local_timer_interrupts 58036 +14.8% 66606 interrupts.CPU124.LOC:Local_timer_interrupts 58033 +14.8% 66615 interrupts.CPU125.LOC:Local_timer_interrupts 58024 +14.8% 66627 interrupts.CPU126.LOC:Local_timer_interrupts 58015 +14.8% 66593 interrupts.CPU127.LOC:Local_timer_interrupts 58011 +14.8% 66598 interrupts.CPU128.LOC:Local_timer_interrupts 58016 +14.8% 66594 interrupts.CPU129.LOC:Local_timer_interrupts 58089 +14.9% 66753 interrupts.CPU13.LOC:Local_timer_interrupts 58005 +14.8% 66591 interrupts.CPU130.LOC:Local_timer_interrupts 58016 +14.8% 66623 interrupts.CPU131.LOC:Local_timer_interrupts 57791 +15.3% 66640 interrupts.CPU132.LOC:Local_timer_interrupts 58027 +14.7% 66578 interrupts.CPU133.LOC:Local_timer_interrupts 58043 +14.7% 66594 interrupts.CPU134.LOC:Local_timer_interrupts 58005 +14.8% 66600 interrupts.CPU135.LOC:Local_timer_interrupts 58003 +14.8% 66594 interrupts.CPU136.LOC:Local_timer_interrupts 58019 +14.8% 66590 interrupts.CPU137.LOC:Local_timer_interrupts 58044 +14.7% 66588 interrupts.CPU138.LOC:Local_timer_interrupts 58036 +14.7% 66575 interrupts.CPU139.LOC:Local_timer_interrupts 58111 +14.7% 66629 interrupts.CPU14.LOC:Local_timer_interrupts 58032 +14.7% 66581 interrupts.CPU140.LOC:Local_timer_interrupts 58026 +14.8% 66593 interrupts.CPU141.LOC:Local_timer_interrupts 58003 +14.8% 66605 interrupts.CPU142.LOC:Local_timer_interrupts 57998 +14.9% 66617 interrupts.CPU143.LOC:Local_timer_interrupts 57886 +15.0% 66570 interrupts.CPU144.LOC:Local_timer_interrupts 918.25 ± 88% -85.3% 135.25 ± 10% interrupts.CPU144.NMI:Non-maskable_interrupts 918.25 ± 88% -85.3% 135.25 ± 10% interrupts.CPU144.PMI:Performance_monitoring_interrupts 58017 +14.8% 66607 interrupts.CPU145.LOC:Local_timer_interrupts 58017 +14.8% 66597 interrupts.CPU146.LOC:Local_timer_interrupts 58016 +14.8% 66583 interrupts.CPU147.LOC:Local_timer_interrupts 57938 +14.9% 66573 interrupts.CPU148.LOC:Local_timer_interrupts 58031 +14.8% 66600 interrupts.CPU149.LOC:Local_timer_interrupts 58103 +14.7% 66639 interrupts.CPU15.LOC:Local_timer_interrupts 58004 +14.8% 66581 interrupts.CPU150.LOC:Local_timer_interrupts 58000 +14.7% 66527 interrupts.CPU151.LOC:Local_timer_interrupts 57945 +14.9% 66586 interrupts.CPU152.LOC:Local_timer_interrupts 58027 +14.7% 66574 interrupts.CPU153.LOC:Local_timer_interrupts 58036 +14.7% 66567 interrupts.CPU154.LOC:Local_timer_interrupts 58031 +14.8% 66597 interrupts.CPU155.LOC:Local_timer_interrupts 58012 +14.8% 66582 interrupts.CPU156.LOC:Local_timer_interrupts 58009 +14.8% 66599 interrupts.CPU157.LOC:Local_timer_interrupts 58069 +14.7% 66584 interrupts.CPU158.LOC:Local_timer_interrupts 57997 +14.8% 66596 interrupts.CPU159.LOC:Local_timer_interrupts 58081 +14.7% 66633 interrupts.CPU16.LOC:Local_timer_interrupts 58014 +14.8% 66583 interrupts.CPU160.LOC:Local_timer_interrupts 58016 +14.9% 66633 interrupts.CPU161.LOC:Local_timer_interrupts 57998 +14.8% 66579 interrupts.CPU162.LOC:Local_timer_interrupts 58019 +14.7% 66563 interrupts.CPU163.LOC:Local_timer_interrupts 57996 +14.8% 66558 interrupts.CPU164.LOC:Local_timer_interrupts 58019 +14.7% 66570 interrupts.CPU165.LOC:Local_timer_interrupts 58041 +14.7% 66581 interrupts.CPU166.LOC:Local_timer_interrupts 57854 +15.1% 66600 interrupts.CPU167.LOC:Local_timer_interrupts 57991 +14.8% 66594 interrupts.CPU168.LOC:Local_timer_interrupts 58009 +14.8% 66610 interrupts.CPU169.LOC:Local_timer_interrupts 58089 +14.7% 66643 interrupts.CPU17.LOC:Local_timer_interrupts 58035 +14.8% 66595 interrupts.CPU170.LOC:Local_timer_interrupts 57732 +15.4% 66622 interrupts.CPU171.LOC:Local_timer_interrupts 58030 +14.8% 66599 interrupts.CPU172.LOC:Local_timer_interrupts 58019 +13.7% 65945 interrupts.CPU173.LOC:Local_timer_interrupts 180.00 ± 26% +225.0% 585.00 ± 56% interrupts.CPU173.NMI:Non-maskable_interrupts 180.00 ± 26% +225.0% 585.00 ± 56% interrupts.CPU173.PMI:Performance_monitoring_interrupts 58013 +14.5% 66447 interrupts.CPU174.LOC:Local_timer_interrupts 58015 +14.8% 66594 interrupts.CPU175.LOC:Local_timer_interrupts 58014 +14.8% 66592 interrupts.CPU176.LOC:Local_timer_interrupts 58033 +14.7% 66588 interrupts.CPU177.LOC:Local_timer_interrupts 57906 +15.0% 66602 interrupts.CPU178.LOC:Local_timer_interrupts 58020 +14.8% 66636 interrupts.CPU179.LOC:Local_timer_interrupts 58075 +14.8% 66648 interrupts.CPU18.LOC:Local_timer_interrupts 58017 +14.8% 66587 interrupts.CPU180.LOC:Local_timer_interrupts 58019 +14.8% 66622 interrupts.CPU181.LOC:Local_timer_interrupts 58042 +14.7% 66586 interrupts.CPU182.LOC:Local_timer_interrupts 58006 +14.8% 66582 interrupts.CPU183.LOC:Local_timer_interrupts 57993 +14.8% 66583 interrupts.CPU184.LOC:Local_timer_interrupts 58008 +14.8% 66583 interrupts.CPU185.LOC:Local_timer_interrupts 58018 +14.8% 66592 interrupts.CPU186.LOC:Local_timer_interrupts 57998 +14.7% 66524 interrupts.CPU187.LOC:Local_timer_interrupts 58005 +14.8% 66581 interrupts.CPU188.LOC:Local_timer_interrupts 58010 +14.6% 66507 interrupts.CPU189.LOC:Local_timer_interrupts 58109 +14.7% 66660 interrupts.CPU19.LOC:Local_timer_interrupts 58006 +14.8% 66580 interrupts.CPU190.LOC:Local_timer_interrupts 58048 +14.7% 66595 interrupts.CPU191.LOC:Local_timer_interrupts 57029 ± 3% +16.1% 66183 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts 58085 +14.7% 66645 interrupts.CPU20.LOC:Local_timer_interrupts 58075 +14.8% 66680 interrupts.CPU21.LOC:Local_timer_interrupts 58095 +14.7% 66656 interrupts.CPU22.LOC:Local_timer_interrupts 58050 +14.8% 66636 interrupts.CPU23.LOC:Local_timer_interrupts 58056 +14.8% 66641 interrupts.CPU24.LOC:Local_timer_interrupts 58014 +14.9% 66641 interrupts.CPU25.LOC:Local_timer_interrupts 58088 +14.7% 66646 interrupts.CPU26.LOC:Local_timer_interrupts 1797 ± 8% +33.5% 2400 ± 35% interrupts.CPU27.CAL:Function_call_interrupts 58030 +13.1% 65636 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 58061 +14.7% 66620 interrupts.CPU28.LOC:Local_timer_interrupts 58048 +14.8% 66621 interrupts.CPU29.LOC:Local_timer_interrupts 58023 +14.8% 66631 interrupts.CPU3.LOC:Local_timer_interrupts 58074 +14.7% 66614 interrupts.CPU30.LOC:Local_timer_interrupts 58100 +14.7% 66620 interrupts.CPU31.LOC:Local_timer_interrupts 58061 +14.7% 66615 interrupts.CPU32.LOC:Local_timer_interrupts 58096 +14.2% 66368 interrupts.CPU33.LOC:Local_timer_interrupts 58066 +14.7% 66616 interrupts.CPU34.LOC:Local_timer_interrupts 58066 +14.7% 66626 interrupts.CPU35.LOC:Local_timer_interrupts 58025 +14.7% 66580 interrupts.CPU36.LOC:Local_timer_interrupts 58036 +14.8% 66616 interrupts.CPU37.LOC:Local_timer_interrupts 58072 +14.7% 66612 interrupts.CPU38.LOC:Local_timer_interrupts 58052 +14.7% 66610 interrupts.CPU39.LOC:Local_timer_interrupts 58094 +14.8% 66666 interrupts.CPU4.LOC:Local_timer_interrupts 58063 +14.7% 66614 interrupts.CPU40.LOC:Local_timer_interrupts 58052 +14.8% 66627 interrupts.CPU41.LOC:Local_timer_interrupts 58035 +14.8% 66616 interrupts.CPU42.LOC:Local_timer_interrupts 58048 +14.8% 66636 interrupts.CPU43.LOC:Local_timer_interrupts 58080 +14.7% 66631 interrupts.CPU44.LOC:Local_timer_interrupts 58050 +14.8% 66625 interrupts.CPU45.LOC:Local_timer_interrupts 58060 +14.7% 66621 interrupts.CPU46.LOC:Local_timer_interrupts 58072 +14.7% 66631 interrupts.CPU47.LOC:Local_timer_interrupts 58005 +14.6% 66448 interrupts.CPU48.LOC:Local_timer_interrupts 58069 +14.7% 66614 interrupts.CPU49.LOC:Local_timer_interrupts 58103 +14.7% 66625 interrupts.CPU5.LOC:Local_timer_interrupts 58049 +14.8% 66613 interrupts.CPU50.LOC:Local_timer_interrupts 58056 +14.7% 66615 interrupts.CPU51.LOC:Local_timer_interrupts 57464 ± 2% +15.9% 66581 interrupts.CPU52.LOC:Local_timer_interrupts 58050 +14.7% 66598 interrupts.CPU53.LOC:Local_timer_interrupts 58046 +14.7% 66584 interrupts.CPU54.LOC:Local_timer_interrupts 58080 +14.0% 66219 interrupts.CPU55.LOC:Local_timer_interrupts 57786 +15.2% 66582 interrupts.CPU56.LOC:Local_timer_interrupts 58058 +14.7% 66596 interrupts.CPU57.LOC:Local_timer_interrupts 58066 +14.5% 66482 interrupts.CPU58.LOC:Local_timer_interrupts 58044 +14.7% 66581 interrupts.CPU59.LOC:Local_timer_interrupts 58093 +14.7% 66625 interrupts.CPU6.LOC:Local_timer_interrupts 58038 +14.8% 66609 interrupts.CPU60.LOC:Local_timer_interrupts 58033 +14.7% 66590 interrupts.CPU61.LOC:Local_timer_interrupts 58015 +14.8% 66605 interrupts.CPU62.LOC:Local_timer_interrupts 58081 +14.7% 66604 interrupts.CPU63.LOC:Local_timer_interrupts 58048 +14.8% 66634 interrupts.CPU64.LOC:Local_timer_interrupts 58025 +14.4% 66400 interrupts.CPU65.LOC:Local_timer_interrupts 58043 +14.7% 66597 interrupts.CPU66.LOC:Local_timer_interrupts 58050 +14.7% 66585 interrupts.CPU67.LOC:Local_timer_interrupts 58083 +14.6% 66573 interrupts.CPU68.LOC:Local_timer_interrupts 58043 +14.7% 66601 interrupts.CPU69.LOC:Local_timer_interrupts 58012 +14.9% 66639 interrupts.CPU7.LOC:Local_timer_interrupts 58046 +14.7% 66580 interrupts.CPU70.LOC:Local_timer_interrupts 58040 +14.8% 66619 interrupts.CPU71.LOC:Local_timer_interrupts 57810 +15.1% 66538 interrupts.CPU72.LOC:Local_timer_interrupts 58076 +14.7% 66616 interrupts.CPU73.LOC:Local_timer_interrupts 58033 +14.8% 66636 interrupts.CPU74.LOC:Local_timer_interrupts 57667 +15.5% 66598 interrupts.CPU75.LOC:Local_timer_interrupts 58034 +14.8% 66622 interrupts.CPU76.LOC:Local_timer_interrupts 58045 +14.7% 66568 interrupts.CPU77.LOC:Local_timer_interrupts 176.50 ± 25% +328.0% 755.50 ± 91% interrupts.CPU77.NMI:Non-maskable_interrupts 176.50 ± 25% +328.0% 755.50 ± 91% interrupts.CPU77.PMI:Performance_monitoring_interrupts 58031 +13.8% 66059 interrupts.CPU78.LOC:Local_timer_interrupts 58026 +14.8% 66603 interrupts.CPU79.LOC:Local_timer_interrupts 58056 +14.7% 66597 interrupts.CPU8.LOC:Local_timer_interrupts 58027 +14.3% 66306 interrupts.CPU80.LOC:Local_timer_interrupts 58030 +14.7% 66538 interrupts.CPU81.LOC:Local_timer_interrupts 57995 +14.9% 66616 interrupts.CPU82.LOC:Local_timer_interrupts 58047 +14.7% 66558 interrupts.CPU83.LOC:Local_timer_interrupts 58036 +14.8% 66608 interrupts.CPU84.LOC:Local_timer_interrupts 58038 +14.8% 66607 interrupts.CPU85.LOC:Local_timer_interrupts 58032 +14.8% 66609 interrupts.CPU86.LOC:Local_timer_interrupts 57907 +15.1% 66632 interrupts.CPU87.LOC:Local_timer_interrupts 58018 +14.8% 66621 interrupts.CPU88.LOC:Local_timer_interrupts 58029 +14.5% 66471 interrupts.CPU89.LOC:Local_timer_interrupts 58101 +14.6% 66596 interrupts.CPU9.LOC:Local_timer_interrupts 58065 +14.7% 66609 interrupts.CPU90.LOC:Local_timer_interrupts 58050 +14.5% 66495 interrupts.CPU91.LOC:Local_timer_interrupts 58057 +14.7% 66612 interrupts.CPU92.LOC:Local_timer_interrupts 58026 +14.8% 66612 interrupts.CPU93.LOC:Local_timer_interrupts 58044 +14.8% 66610 interrupts.CPU94.LOC:Local_timer_interrupts 58046 +14.8% 66615 interrupts.CPU95.LOC:Local_timer_interrupts 58050 +14.8% 66616 interrupts.CPU96.LOC:Local_timer_interrupts 58071 +14.6% 66547 interrupts.CPU97.LOC:Local_timer_interrupts 57871 +15.1% 66606 interrupts.CPU98.LOC:Local_timer_interrupts 58066 +14.8% 66642 interrupts.CPU99.LOC:Local_timer_interrupts 11139415 +14.8% 12784101 interrupts.LOC:Local_timer_interrupts fsmark.files_per_sec 250 +---------------------------------------------------------------------+ 245 |-+ + + | | + + +++. .+ .++ + ++ + +. .+ :+ .++ +. | 240 |++.+ :+ + + ++++ ++++ +++.++ ++.++ ++++ ++++ + + ++ ++| 235 |-+ + + | | | 230 |-+ | 225 |-+ | 220 |-+ | | | 215 |-+ | 210 |-+ OO O O | |OO O OO OOOO O O O OO OOOO OOOOO O O | 205 |-+ O OO O O OO | 200 +---------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap1: 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory ========================================================================================= bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/ucode: 4k/gcc-9/performance/1SSD/btrfs/sync/x86_64-rhel-8.3/8/debian-10.4-x86_64-20200603.cgz/300s/write/lkp-csl-2ap1/256g/fio-basic/0x4003003 commit: 5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size") 53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing") 5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 100% 4:4 stderr.Events_enabled :4 100% 4:4 stderr.[perf_record:Captured_and_wrote#MB/tmp/lkp/perf-sched.data(#samples)] :4 100% 4:4 stderr.[perf_record:Woken_up#times_to_write_data] %stddev %change %stddev \ | \ 0.01 +1.1 1.12 ± 8% fio.latency_100us% 62.86 ± 3% -62.7 0.12 ±132% fio.latency_10us% 19.90 ± 15% -19.8 0.12 ±149% fio.latency_20us% 0.01 -0.0 0.00 fio.latency_2us% 16.87 ± 14% -16.9 0.01 fio.latency_4us% 0.03 ± 2% -0.0 0.00 fio.latency_50ms% 0.33 ± 13% +98.3 98.64 fio.latency_50us% 127.41 +137.0% 301.96 fio.time.elapsed_time 127.41 +137.0% 301.96 fio.time.elapsed_time.max 5.369e+08 -4.1% 5.151e+08 fio.time.file_system_outputs 1594 ± 6% +293.4% 6274 ± 3% fio.time.involuntary_context_switches 420.75 ± 3% +89.9% 799.00 fio.time.percent_of_cpu_this_job_got 480.22 ± 4% +390.9% 2357 fio.time.system_time 36993 -13.3% 32063 fio.time.voluntary_context_switches 67108864 -4.1% 64389238 fio.workload 2090 -59.9% 838.40 fio.write_bw_MBps 11616 ± 3% +263.6% 42240 fio.write_clat_90%_us 13248 ± 3% +239.1% 44928 fio.write_clat_95%_us 17408 ± 2% +189.7% 50432 fio.write_clat_99%_us 14517 +154.7% 36975 fio.write_clat_mean_us 388357 -98.6% 5502 ± 9% fio.write_clat_stddev 535275 -59.9% 214629 fio.write_iops 39055446 ± 26% +79.5% 70092416 ± 32% cpuidle.C1E.usage 248292 ± 3% +40.8% 349605 ± 19% cpuidle.POLL.usage 2.26 ± 2% -92.8% 0.16 ± 3% iostat.cpu.iowait 3.70 ± 2% +39.6% 5.16 iostat.cpu.system 176.92 +99.6% 353.08 uptime.boot 31486 +102.3% 63696 uptime.idle 2.29 ± 2% -2.1 0.16 ± 3% mpstat.cpu.all.iowait% 1.12 ± 2% -0.3 0.84 ± 8% mpstat.cpu.all.irq% 0.09 ± 3% -0.0 0.07 mpstat.cpu.all.soft% 2.53 ± 3% +1.7 4.27 mpstat.cpu.all.sys% 0.28 -0.2 0.13 ± 2% mpstat.cpu.all.usr% 93.00 +1.1% 94.00 vmstat.cpu.id 1837418 -57.0% 789462 ± 2% vmstat.io.bo 1.293e+08 -10.7% 1.155e+08 ± 3% vmstat.memory.cache 65719106 +22.0% 80155172 ± 5% vmstat.memory.free 4.25 ± 10% -100.0% 0.00 vmstat.procs.b 5.50 ± 9% +50.0% 8.25 ± 5% vmstat.procs.r 50381 ± 2% -73.0% 13588 vmstat.system.cs 8058918 ± 89% +181.2% 22659297 ± 5% numa-numastat.node0.local_node 8122514 ± 88% +179.8% 22724178 ± 5% numa-numastat.node0.numa_hit 8081647 ± 67% -99.7% 23653 ±173% numa-numastat.node0.numa_miss 8145258 ± 67% -98.9% 88543 ± 72% numa-numastat.node0.other_node 6489935 ± 95% -96.4% 232918 ± 76% numa-numastat.node1.numa_foreign 12996328 ± 70% -100.0% 0.00 numa-numastat.node2.numa_foreign 6677566 ± 15% +71.1% 11424129 ± 28% numa-numastat.node3.local_node 3818447 ± 87% -97.1% 108841 ±173% numa-numastat.node3.numa_foreign 6737072 ± 15% +70.6% 11495370 ± 28% numa-numastat.node3.numa_hit 10218977 ± 29% -100.0% 0.00 numa-numastat.node3.numa_miss 10278486 ± 29% -99.3% 71246 ± 21% numa-numastat.node3.other_node 117151 +62.7% 190593 ± 4% meminfo.Active 5415 +1142.0% 67255 ± 8% meminfo.Active(anon) 133307 ± 2% +32.2% 176190 meminfo.AnonHugePages 1.284e+08 -10.5% 1.149e+08 ± 3% meminfo.Cached 133470 ± 17% +31.5% 175470 ± 10% meminfo.CmaFree 1.276e+08 -10.6% 1.14e+08 ± 3% meminfo.Inactive 1.261e+08 -10.7% 1.126e+08 ± 3% meminfo.Inactive(file) 66215274 +21.3% 80314069 ± 5% meminfo.MemFree 1.315e+08 -10.7% 1.174e+08 ± 3% meminfo.Memused 309160 -11.6% 273349 meminfo.SUnreclaim 722723 -8.7% 659694 meminfo.Slab 19767477 ± 2% -86.3% 2702733 ± 4% meminfo.Writeback 1522111 -59.9% 610959 ± 3% meminfo.max_used_kB 6716 ± 51% +593.0% 46543 ± 17% numa-meminfo.node0.Active 4579 ± 93% +880.8% 44916 ± 18% numa-meminfo.node0.Active(file) 2632191 ± 17% +39.0% 3658947 ± 9% numa-meminfo.node0.Dirty 29868469 ± 23% +30.9% 39092332 ± 4% numa-meminfo.node0.FilePages 29706221 ± 23% +30.8% 38862897 ± 4% numa-meminfo.node0.Inactive 29324351 ± 25% +31.3% 38502867 ± 3% numa-meminfo.node0.Inactive(file) 18493244 ± 37% -49.2% 9395979 ± 19% numa-meminfo.node0.MemFree 30661618 ± 22% +29.7% 39758883 ± 4% numa-meminfo.node0.MemUsed 4820384 ± 13% -80.9% 922511 ± 11% numa-meminfo.node0.Writeback 7716 ± 7% -11.8% 6808 ± 8% numa-meminfo.node1.KernelStack 4733390 ± 14% -84.5% 733265 ± 12% numa-meminfo.node1.Writeback 3085152 ± 8% -28.5% 2205686 ± 23% numa-meminfo.node2.Dirty 36156759 ± 9% -28.6% 25803094 ± 19% numa-meminfo.node2.FilePages 35933307 ± 9% -28.7% 25615675 ± 19% numa-meminfo.node2.Inactive 35585652 ± 9% -29.9% 24928486 ± 21% numa-meminfo.node2.Inactive(file) 12572134 ± 28% +83.4% 23052389 ± 21% numa-meminfo.node2.MemFree 36962324 ± 9% -28.4% 26482069 ± 19% numa-meminfo.node2.MemUsed 5324212 ± 4% -89.0% 584306 ± 26% numa-meminfo.node2.Writeback 945.50 ± 92% +6446.5% 61897 ± 8% numa-meminfo.node3.Active(anon) 2861051 ± 10% -37.8% 1778186 ± 30% numa-meminfo.node3.Dirty 32510001 ± 15% -34.6% 21269226 ± 26% numa-meminfo.node3.FilePages 32349393 ± 16% -35.0% 21023730 ± 27% numa-meminfo.node3.Inactive 31940087 ± 15% -35.3% 20657807 ± 28% numa-meminfo.node3.Inactive(file) 16148531 ± 32% +71.1% 27626525 ± 20% numa-meminfo.node3.MemFree 33343003 ± 15% -34.4% 21865009 ± 26% numa-meminfo.node3.MemUsed 4867117 ± 12% -90.6% 457350 ± 27% numa-meminfo.node3.Writeback 1272 ± 2% -52.9% 598.75 ± 4% slabinfo.biovec-max.active_objs 1361 -49.1% 693.50 ± 3% slabinfo.biovec-max.num_objs 8179 +9.3% 8943 ± 3% slabinfo.btrfs_extent_buffer.num_objs 1605 ± 4% +6.6% 1712 ± 5% slabinfo.btrfs_ordered_extent.active_objs 1605 ± 4% +6.6% 1712 ± 5% slabinfo.btrfs_ordered_extent.num_objs 2117 ± 5% +23.1% 2607 ± 3% slabinfo.buffer_head.active_objs 2157 ± 5% +20.9% 2608 ± 3% slabinfo.buffer_head.num_objs 5316 ± 11% +103.5% 10821 ± 7% slabinfo.dmaengine-unmap-16.active_objs 127.50 ± 10% +103.3% 259.25 ± 7% slabinfo.dmaengine-unmap-16.active_slabs 5385 ± 10% +102.5% 10906 ± 7% slabinfo.dmaengine-unmap-16.num_objs 127.50 ± 10% +103.3% 259.25 ± 7% slabinfo.dmaengine-unmap-16.num_slabs 8840 ± 11% +22.0% 10780 ± 8% slabinfo.fsnotify_mark_connector.active_objs 8929 ± 11% +21.7% 10865 ± 7% slabinfo.fsnotify_mark_connector.num_objs 5642 ± 3% -7.6% 5214 ± 2% slabinfo.kmalloc-rcl-64.active_objs 5642 ± 3% -7.6% 5214 ± 2% slabinfo.kmalloc-rcl-64.num_objs 2207 ± 3% -21.4% 1735 ± 3% slabinfo.mnt_cache.active_objs 2291 ± 2% -20.2% 1828 ± 3% slabinfo.mnt_cache.num_objs 983.00 ± 11% +14.4% 1124 ± 5% slabinfo.proc_inode_cache.active_slabs 11805 ± 11% +14.4% 13504 ± 5% slabinfo.proc_inode_cache.num_objs 983.00 ± 11% +14.4% 1124 ± 5% slabinfo.proc_inode_cache.num_slabs 588271 -9.0% 535256 ± 2% slabinfo.radix_tree_node.active_objs 42019 -9.0% 38232 ± 2% slabinfo.radix_tree_node.active_slabs 588272 -9.0% 535256 ± 2% slabinfo.radix_tree_node.num_objs 42019 -9.0% 38232 ± 2% slabinfo.radix_tree_node.num_slabs 3437 ± 3% +16.0% 3986 slabinfo.trace_event_file.active_objs 3444 ± 3% +16.1% 3999 slabinfo.trace_event_file.num_objs 21837 +25.3% 27365 slabinfo.vmap_area.active_objs 21863 +25.4% 27407 slabinfo.vmap_area.num_objs 676874 ± 43% +122.4% 1505328 ± 39% proc-vmstat.compact_free_scanned 661.00 ± 10% +83.2% 1211 ± 26% proc-vmstat.kswapd_high_wmark_hit_quickly 216.50 ± 55% +880.4% 2122 ± 65% proc-vmstat.kswapd_low_wmark_hit_quickly 1352 +1138.6% 16751 ± 8% proc-vmstat.nr_active_anon 27860 +10.5% 30795 ± 2% proc-vmstat.nr_active_file 67188751 -4.1% 64460748 proc-vmstat.nr_dirtied 2830874 ± 2% -8.6% 2588405 proc-vmstat.nr_dirty 32053250 -10.4% 28710117 ± 3% proc-vmstat.nr_file_pages 33479 ± 17% +31.1% 43876 ± 10% proc-vmstat.nr_free_cma 16593949 +21.1% 20098175 ± 5% proc-vmstat.nr_free_pages 31485171 -10.7% 28118905 ± 3% proc-vmstat.nr_inactive_file 283883 +1.6% 288335 proc-vmstat.nr_mapped 279467 +7.2% 299680 proc-vmstat.nr_shmem 103212 -6.5% 96498 proc-vmstat.nr_slab_reclaimable 77257 -11.5% 68335 proc-vmstat.nr_slab_unreclaimable 4936950 ± 2% -86.4% 673846 ± 4% proc-vmstat.nr_writeback 1352 +1138.6% 16751 ± 8% proc-vmstat.nr_zone_active_anon 27873 +10.5% 30800 ± 2% proc-vmstat.nr_zone_active_file 31485279 -10.7% 28118952 ± 3% proc-vmstat.nr_zone_inactive_file 7768354 -58.0% 3262485 proc-vmstat.nr_zone_write_pending 29916365 ± 9% -70.7% 8750752 ± 50% proc-vmstat.numa_foreign 38401814 ± 7% +49.2% 57291640 ± 6% proc-vmstat.numa_hit 38140485 ± 7% +49.5% 57031158 ± 6% proc-vmstat.numa_local 29916365 ± 9% -70.7% 8750752 ± 50% proc-vmstat.numa_miss 30177694 ± 9% -70.1% 9011234 ± 48% proc-vmstat.numa_other 955.00 ± 16% +269.9% 3532 ± 33% proc-vmstat.pageoutrun 13359 +322.7% 56468 ± 8% proc-vmstat.pgactivate 68321287 -3.5% 65940158 proc-vmstat.pgalloc_normal 1085032 +61.4% 1750813 proc-vmstat.pgfault 28022 +131.2% 64789 proc-vmstat.pgreuse 27637 ± 82% +244.0% 95087 ± 21% sched_debug.cfs_rq:/.load.avg 628606 ± 13% +36.0% 854981 sched_debug.cfs_rq:/.load.max 107249 ± 45% +124.2% 240458 ± 10% sched_debug.cfs_rq:/.load.stddev 86480 ± 7% +19.2% 103104 ± 5% sched_debug.cfs_rq:/.min_vruntime.avg 113366 ± 2% +78.5% 202336 ± 3% sched_debug.cfs_rq:/.min_vruntime.max 7058 ± 7% +232.3% 23457 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev 0.05 ± 42% +109.5% 0.11 ± 19% sched_debug.cfs_rq:/.nr_running.avg 0.21 ± 20% +38.2% 0.28 ± 7% sched_debug.cfs_rq:/.nr_running.stddev 79.54 ± 7% +41.7% 112.71 ± 16% sched_debug.cfs_rq:/.runnable_avg.avg 834.79 ± 3% +22.7% 1024 sched_debug.cfs_rq:/.runnable_avg.max 160.25 ± 4% +69.1% 270.95 ± 8% sched_debug.cfs_rq:/.runnable_avg.stddev 36862 ± 34% +197.7% 109730 ± 9% sched_debug.cfs_rq:/.spread0.max 7074 ± 7% +231.6% 23457 ± 11% sched_debug.cfs_rq:/.spread0.stddev 79.49 ± 7% +41.7% 112.67 ± 16% sched_debug.cfs_rq:/.util_avg.avg 834.58 ± 4% +22.7% 1024 sched_debug.cfs_rq:/.util_avg.max 160.21 ± 4% +69.1% 270.90 ± 8% sched_debug.cfs_rq:/.util_avg.stddev 13.27 ± 53% +520.7% 82.36 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.avg 682.00 ± 11% +50.3% 1025 sched_debug.cfs_rq:/.util_est_enqueued.max 74.42 ± 17% +225.1% 241.91 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.stddev 2970391 ± 34% -52.6% 1408456 ± 6% sched_debug.cpu.avg_idle.max 35686 ± 14% +188.3% 102883 ± 26% sched_debug.cpu.avg_idle.min 282434 ± 23% -50.1% 140876 ± 6% sched_debug.cpu.avg_idle.stddev 93684 ± 14% +90.0% 178021 ± 6% sched_debug.cpu.clock.avg 93701 ± 14% +90.0% 178034 ± 6% sched_debug.cpu.clock.max 93667 ± 14% +90.0% 178007 ± 6% sched_debug.cpu.clock.min 9.76 ± 6% -23.1% 7.50 ± 5% sched_debug.cpu.clock.stddev 93008 ± 14% +90.1% 176795 ± 6% sched_debug.cpu.clock_task.avg 93308 ± 14% +89.8% 177069 ± 6% sched_debug.cpu.clock_task.max 84353 ± 16% +100.2% 168853 ± 7% sched_debug.cpu.clock_task.min 121.07 ± 24% +79.9% 217.78 ± 2% sched_debug.cpu.curr->pid.avg 5219 ± 9% +49.6% 7806 ± 5% sched_debug.cpu.curr->pid.max 674.76 ± 12% +45.6% 982.71 ± 2% sched_debug.cpu.curr->pid.stddev 1485195 ± 34% -52.2% 710548 ± 8% sched_debug.cpu.max_idle_balance_cost.max 89431 ± 54% -70.3% 26578 ± 21% sched_debug.cpu.max_idle_balance_cost.stddev 0.03 ± 25% +53.8% 0.05 ± 3% sched_debug.cpu.nr_running.avg 0.18 ± 13% +26.8% 0.22 sched_debug.cpu.nr_running.stddev 824.12 ± 13% +36.8% 1126 ± 12% sched_debug.cpu.nr_switches.min 0.02 ± 22% -88.0% 0.00 ± 25% sched_debug.cpu.nr_uninterruptible.avg 93667 ± 14% +90.0% 178008 ± 6% sched_debug.cpu_clk 92796 ± 14% +90.9% 177135 ± 6% sched_debug.ktime 94042 ± 14% +89.7% 178380 ± 6% sched_debug.sched_clk 1145 ± 92% +880.4% 11230 ± 18% numa-vmstat.node0.nr_active_file 8172691 ± 31% +55.7% 12720947 ± 9% numa-vmstat.node0.nr_dirtied 655987 ± 17% +39.5% 914941 ± 9% numa-vmstat.node0.nr_dirty 7465216 ± 23% +30.9% 9772897 ± 4% numa-vmstat.node0.nr_file_pages 4625156 ± 37% -49.2% 2349212 ± 19% numa-vmstat.node0.nr_free_pages 7328653 ± 25% +31.3% 9625646 ± 3% numa-vmstat.node0.nr_inactive_file 1206608 ± 13% -80.9% 230245 ± 10% numa-vmstat.node0.nr_writeback 6309984 ± 36% +83.5% 11575697 ± 9% numa-vmstat.node0.nr_written 1148 ± 92% +878.0% 11232 ± 18% numa-vmstat.node0.nr_zone_active_file 7328667 ± 25% +31.3% 9625676 ± 3% numa-vmstat.node0.nr_zone_inactive_file 1862745 ± 14% -38.5% 1145274 ± 9% numa-vmstat.node0.nr_zone_write_pending 4611857 ± 95% +192.1% 13471453 ± 8% numa-vmstat.node0.numa_hit 4514907 ± 97% +196.2% 13371363 ± 8% numa-vmstat.node0.numa_local 4280076 ± 69% -100.0% 1088 ±173% numa-vmstat.node0.numa_miss 4377034 ± 68% -97.7% 101182 ± 60% numa-vmstat.node0.numa_other 7710 ± 7% -11.8% 6799 ± 9% numa-vmstat.node1.nr_kernel_stack 1184420 ± 14% -84.5% 183552 ± 12% numa-vmstat.node1.nr_writeback 1870949 ± 14% -54.0% 861047 ± 8% numa-vmstat.node1.nr_zone_write_pending 3181676 ±125% -97.3% 84818 ±125% numa-vmstat.node1.numa_foreign 10400753 ± 13% -37.3% 6523007 ± 23% numa-vmstat.node2.nr_dirtied 769495 ± 8% -28.3% 551406 ± 23% numa-vmstat.node2.nr_dirty 9042967 ± 9% -28.7% 6451632 ± 19% numa-vmstat.node2.nr_file_pages 3139286 ± 27% +83.6% 5762258 ± 22% numa-vmstat.node2.nr_free_pages 8899645 ± 9% -30.0% 6232951 ± 21% numa-vmstat.node2.nr_inactive_file 1332398 ± 4% -89.0% 146368 ± 26% numa-vmstat.node2.nr_writeback 8899674 ± 9% -30.0% 6232954 ± 21% numa-vmstat.node2.nr_zone_inactive_file 2102021 ± 5% -66.8% 697822 ± 24% numa-vmstat.node2.nr_zone_write_pending 7084511 ± 73% -100.0% 0.00 numa-vmstat.node2.numa_foreign 236.25 ± 92% +6461.2% 15500 ± 8% numa-vmstat.node3.nr_active_anon 8919994 ± 20% -40.8% 5283319 ± 31% numa-vmstat.node3.nr_dirtied 715123 ± 10% -37.8% 444600 ± 30% numa-vmstat.node3.nr_dirty 8132346 ± 15% -34.6% 5317735 ± 26% numa-vmstat.node3.nr_file_pages 33475 ± 17% +31.0% 43866 ± 10% numa-vmstat.node3.nr_free_cma 4032260 ± 32% +71.3% 6906250 ± 20% numa-vmstat.node3.nr_free_pages 7990046 ± 15% -35.4% 5165114 ± 28% numa-vmstat.node3.nr_inactive_file 1215444 ± 12% -90.6% 114335 ± 27% numa-vmstat.node3.nr_writeback 236.25 ± 92% +6461.2% 15500 ± 8% numa-vmstat.node3.nr_zone_active_anon 7990067 ± 15% -35.4% 5165114 ± 28% numa-vmstat.node3.nr_zone_inactive_file 1930722 ± 10% -71.0% 558966 ± 29% numa-vmstat.node3.nr_zone_write_pending 1901104 ± 90% -99.7% 5165 ±173% numa-vmstat.node3.numa_foreign 4150742 ± 14% +45.5% 6039465 ± 26% numa-vmstat.node3.numa_hit 4016578 ± 15% +46.4% 5878550 ± 27% numa-vmstat.node3.numa_local 5594513 ± 41% -100.0% 0.00 numa-vmstat.node3.numa_miss 5728682 ± 40% -97.2% 160916 ± 8% numa-vmstat.node3.numa_other 11.78 ± 3% -40.4% 7.02 ± 6% perf-stat.i.MPKI 2.286e+09 -4.8% 2.177e+09 perf-stat.i.branch-instructions 0.65 -0.2 0.49 perf-stat.i.branch-miss-rate% 15863136 ± 2% -30.8% 10985192 ± 2% perf-stat.i.branch-misses 87495643 ± 4% -47.7% 45751362 ± 7% perf-stat.i.cache-misses 1.392e+08 ± 3% -45.8% 75512046 ± 7% perf-stat.i.cache-references 51205 ± 2% -73.5% 13590 perf-stat.i.context-switches 2.65 +64.0% 4.34 perf-stat.i.cpi 3.067e+10 ± 2% +43.6% 4.404e+10 perf-stat.i.cpu-cycles 455.07 ± 5% +169.3% 1225 ± 9% perf-stat.i.cycles-between-cache-misses 273457 ± 65% -67.1% 89970 ± 17% perf-stat.i.dTLB-load-misses 3.329e+09 -22.9% 2.566e+09 perf-stat.i.dTLB-loads 64168 ± 37% -52.7% 30346 ± 7% perf-stat.i.dTLB-store-misses 1.672e+09 -41.6% 9.756e+08 perf-stat.i.dTLB-stores 61.66 ± 2% -8.9 52.72 perf-stat.i.iTLB-load-miss-rate% 7608055 ± 6% -29.7% 5350529 perf-stat.i.iTLB-load-misses 4712088 +1.8% 4794810 perf-stat.i.iTLB-loads 1.164e+10 -10.9% 1.036e+10 perf-stat.i.instructions 1532 ± 6% +26.0% 1931 perf-stat.i.instructions-per-iTLB-miss 0.39 -39.4% 0.24 perf-stat.i.ipc 2212 -57.9% 930.49 perf-stat.i.major-faults 0.16 ± 2% +44.1% 0.23 perf-stat.i.metric.GHz 0.83 ± 18% -38.0% 0.52 ± 16% perf-stat.i.metric.K/sec 38.83 -22.1% 30.26 perf-stat.i.metric.M/sec 3836 -0.9% 3801 perf-stat.i.minor-faults 13804926 ± 4% -40.0% 8289551 ± 3% perf-stat.i.node-load-misses 1907846 ± 9% -44.5% 1058077 ± 7% perf-stat.i.node-loads 4552731 ± 6% -34.0% 3003065 ± 4% perf-stat.i.node-store-misses 287448 ± 48% -42.0% 166584 ± 14% perf-stat.i.node-stores 6049 -21.8% 4731 perf-stat.i.page-faults 11.96 ± 3% -39.1% 7.28 ± 6% perf-stat.overall.MPKI 0.70 -0.2 0.51 perf-stat.overall.branch-miss-rate% 2.63 +61.4% 4.25 perf-stat.overall.cpi 351.94 ± 5% +174.9% 967.33 ± 7% perf-stat.overall.cycles-between-cache-misses 61.70 ± 2% -9.0 52.74 perf-stat.overall.iTLB-load-miss-rate% 1537 ± 6% +26.1% 1937 perf-stat.overall.instructions-per-iTLB-miss 0.38 -38.0% 0.24 perf-stat.overall.ipc 22102 +119.7% 48551 perf-stat.overall.path-length 2.268e+09 -4.4% 2.17e+09 perf-stat.ps.branch-instructions 15776375 ± 2% -30.4% 10975709 ± 2% perf-stat.ps.branch-misses 86567030 ± 4% -47.3% 45589562 ± 7% perf-stat.ps.cache-misses 1.381e+08 ± 3% -45.5% 75270910 ± 7% perf-stat.ps.cache-references 50561 ± 2% -73.2% 13544 perf-stat.ps.context-switches 3.04e+10 ± 2% +44.3% 4.387e+10 perf-stat.ps.cpu-cycles 3.302e+09 -22.6% 2.557e+09 perf-stat.ps.dTLB-loads 63958 ± 37% -52.3% 30512 ± 7% perf-stat.ps.dTLB-store-misses 1.658e+09 -41.4% 9.723e+08 perf-stat.ps.dTLB-stores 7542975 ± 6% -29.3% 5331814 perf-stat.ps.iTLB-load-misses 4669996 +2.3% 4777284 perf-stat.ps.iTLB-loads 1.155e+10 -10.6% 1.033e+10 perf-stat.ps.instructions 2191 -57.5% 931.32 perf-stat.ps.major-faults 13652567 ± 4% -39.5% 8255968 ± 3% perf-stat.ps.node-load-misses 1887494 ± 9% -44.1% 1054935 ± 7% perf-stat.ps.node-loads 4506248 ± 6% -33.7% 2989856 ± 4% perf-stat.ps.node-store-misses 286096 ± 48% -41.9% 166311 ± 14% perf-stat.ps.node-stores 6008 -21.4% 4723 perf-stat.ps.page-faults 1.483e+12 +110.8% 3.126e+12 perf-stat.total.instructions 0.01 ± 53% -87.5% 0.00 ±173% perf-sched.sch_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 0.00 ± 23% -66.7% 0.00 perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 5.36 ± 62% -99.9% 0.01 ± 10% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.07 ± 39% -91.8% 0.01 ± 26% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 0.11 ± 33% -85.0% 0.02 ± 22% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 0.01 ± 17% +162.1% 0.02 ± 12% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.05 ± 54% -87.5% 0.01 ± 28% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 1.42 ± 60% +230.1% 4.68 ± 19% perf-sched.sch_delay.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 0.06 ± 17% -73.2% 0.02 ± 11% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 ±173% +2066.7% 0.02 ± 15% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 22.77 ± 59% -99.9% 0.01 ± 15% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.00 ±173% +1150.0% 0.01 ± 14% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 20.53 ± 34% -85.8% 2.90 ± 83% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 2.30 +197.5% 6.84 ± 10% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork 0.02 ± 27% -81.2% 0.00 perf-sched.total_sch_delay.average.ms 27.75 ± 20% -74.1% 7.19 ± 16% perf-sched.total_sch_delay.max.ms 0.32 ± 18% +7823.9% 25.73 ± 13% perf-sched.total_wait_and_delay.average.ms 3635 ± 4% +2913.9% 109554 ± 17% perf-sched.total_wait_and_delay.count.ms 158.86 ± 5% +5057.9% 8194 ± 6% perf-sched.total_wait_and_delay.max.ms 0.31 ± 19% +8233.6% 25.73 ± 13% perf-sched.total_wait_time.average.ms 158.85 ± 5% +5058.2% 8194 ± 6% perf-sched.total_wait_time.max.ms 0.51 ±173% +44191.4% 226.99 ± 2% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 158.86 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 0.07 ± 2% +45.3% 0.10 ± 19% perf-sched.wait_and_delay.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 0.04 ± 6% +1.2e+05% 46.19 perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.02 ± 14% +21001.2% 4.38 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 6.88 ± 59% +8627.8% 600.56 ± 56% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 3.86 ± 5% +13.4% 4.38 perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 1.25 ± 26% +60259.2% 756.75 perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 7.66 ± 11% +2755.3% 218.84 ± 10% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 1.50 ±173% +20216.7% 304.75 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 1.00 -100.0% 0.00 perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 2255 ± 2% +4082.4% 94312 ± 19% perf-sched.wait_and_delay.count.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 660.75 ± 17% +428.1% 3489 perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read 312.50 ± 17% +575.9% 2112 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 3.25 ± 59% +376.9% 15.50 ± 65% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 41.00 ± 3% +5462.8% 2280 perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 314.75 ± 17% +694.1% 2499 perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork 29.50 ± 56% +7337.3% 2194 ± 9% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 2.83 ±173% +1.3e+05% 3654 ± 31% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 158.86 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 3.27 ± 46% +27193.5% 892.91 ± 76% perf-sched.wait_and_delay.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 0.11 ± 29% +9.2e+05% 1018 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 5.50 ± 3% +18112.0% 1000 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 22.75 ± 59% +28414.4% 6485 ± 53% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 51.79 ±114% +3718.9% 1977 ± 32% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 75.64 ± 19% +8319.0% 6368 ± 16% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork 0.53 ±166% +42910.9% 226.99 ± 2% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 158.85 ± 5% -99.9% 0.13 ±173% perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 0.07 ± 2% +48.9% 0.10 ± 20% perf-sched.wait_time.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 0.04 ± 8% +1.3e+05% 46.19 perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.02 ± 14% +21001.2% 4.38 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 1.57 ± 88% +38243.4% 600.55 ± 56% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 3.85 ± 5% +13.5% 4.37 perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 1.18 ± 28% +63976.9% 756.75 perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 7.55 ± 12% +2796.8% 218.82 ± 10% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 2.90 ±167% +1.3e+05% 3654 ± 31% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 158.85 ± 5% -99.7% 0.46 ±173% perf-sched.wait_time.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex 2.79 ± 57% +31839.1% 892.54 ± 76% perf-sched.wait_time.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle 0.11 ± 32% +9.6e+05% 1018 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 5.50 ± 3% +18111.8% 1000 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr 6.38 ± 88% +1e+05% 6485 ± 53% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 42.79 ±150% +4522.0% 1977 ± 32% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork 75.61 ± 19% +8322.7% 6368 ± 16% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork 23273 ± 5% +79.4% 41758 ± 8% softirqs.CPU0.SCHED 19224 ± 3% +102.6% 38950 softirqs.CPU1.SCHED 17936 ± 3% +117.1% 38939 ± 3% softirqs.CPU10.SCHED 17462 ± 4% +113.1% 37218 ± 6% softirqs.CPU100.SCHED 9336 ± 24% -56.9% 4027 ± 3% softirqs.CPU101.RCU 16134 ± 17% +114.7% 34644 ± 19% softirqs.CPU102.SCHED 18025 ± 2% +89.8% 34220 ± 19% softirqs.CPU103.SCHED 17976 ± 3% +114.2% 38508 softirqs.CPU104.SCHED 17940 ± 3% +110.4% 37752 ± 3% softirqs.CPU105.SCHED 17976 ± 3% +112.8% 38259 softirqs.CPU106.SCHED 17892 ± 5% +105.6% 36785 ± 5% softirqs.CPU107.SCHED 17998 ± 3% +93.0% 34732 ± 18% softirqs.CPU108.SCHED 17812 ± 4% +104.9% 36492 ± 5% softirqs.CPU109.SCHED 509.25 ±129% +2290.8% 12175 ± 77% softirqs.CPU11.NET_RX 17542 ± 3% +97.5% 34636 ± 19% softirqs.CPU11.SCHED 17930 ± 3% +107.5% 37206 ± 4% softirqs.CPU110.SCHED 16041 ± 15% +131.7% 37174 ± 7% softirqs.CPU111.SCHED 17512 ± 7% +85.2% 32425 ± 19% softirqs.CPU112.SCHED 17732 ± 4% +101.9% 35800 ± 14% softirqs.CPU113.SCHED 17731 ± 5% +99.1% 35301 ± 17% softirqs.CPU114.SCHED 18034 ± 2% +73.3% 31260 ± 22% softirqs.CPU115.SCHED 18000 ± 2% +71.9% 30947 ± 23% softirqs.CPU116.SCHED 18037 ± 2% +71.8% 30995 ± 25% softirqs.CPU117.SCHED 17819 ± 4% +115.9% 38469 softirqs.CPU118.SCHED 18069 ± 2% +106.0% 37229 ± 5% softirqs.CPU119.SCHED 16670 ± 4% +63.4% 27233 ± 16% softirqs.CPU120.SCHED 17658 ± 3% +101.2% 35533 ± 16% softirqs.CPU122.SCHED 16398 ± 16% +136.4% 38771 softirqs.CPU123.SCHED 17833 ± 3% +117.8% 38834 softirqs.CPU124.SCHED 17981 ± 2% +116.6% 38948 softirqs.CPU125.SCHED 17859 ± 2% +134.7% 41917 ± 12% softirqs.CPU126.SCHED 17735 ± 3% +115.3% 38177 softirqs.CPU127.SCHED 17671 ± 3% +117.7% 38463 softirqs.CPU128.SCHED 17888 ± 2% +89.5% 33904 ± 19% softirqs.CPU129.SCHED 17864 ± 2% +115.3% 38462 softirqs.CPU130.SCHED 17937 ± 2% +116.3% 38796 softirqs.CPU131.SCHED 17823 ± 2% +116.6% 38613 softirqs.CPU132.SCHED 17817 ± 2% +115.6% 38423 softirqs.CPU133.SCHED 17467 ± 6% +120.5% 38521 softirqs.CPU134.SCHED 17443 ± 5% +90.5% 33227 ± 20% softirqs.CPU135.SCHED 17409 ± 5% +119.3% 38180 softirqs.CPU136.SCHED 17427 ± 6% +122.5% 38779 softirqs.CPU137.SCHED 17994 +115.9% 38853 softirqs.CPU138.SCHED 18000 ± 2% +112.9% 38319 softirqs.CPU139.SCHED 17685 ± 3% +96.7% 34793 ± 20% softirqs.CPU140.SCHED 17559 ± 4% +120.8% 38767 softirqs.CPU141.SCHED 17880 ± 2% +117.1% 38810 softirqs.CPU142.SCHED 18018 ± 2% +114.8% 38700 softirqs.CPU143.SCHED 15816 ± 13% +52.0% 24042 ± 16% softirqs.CPU144.SCHED 17373 ± 4% +120.4% 38293 softirqs.CPU145.SCHED 17900 +117.5% 38927 softirqs.CPU146.SCHED 15928 ± 18% +145.0% 39017 softirqs.CPU147.SCHED 15242 ± 18% +154.5% 38791 softirqs.CPU148.SCHED 17218 ± 4% +125.7% 38866 softirqs.CPU149.SCHED 17388 ± 6% +106.6% 35926 ± 9% softirqs.CPU15.SCHED 17972 ± 2% +115.0% 38643 softirqs.CPU150.SCHED 16587 ± 8% +132.9% 38637 softirqs.CPU151.SCHED 17468 ± 2% +121.6% 38703 softirqs.CPU152.SCHED 17060 ± 7% +127.6% 38826 softirqs.CPU153.SCHED 17641 ± 2% +119.1% 38646 softirqs.CPU154.SCHED 17892 +78.1% 31866 ± 23% softirqs.CPU155.SCHED 17924 +115.9% 38701 softirqs.CPU156.SCHED 17856 +115.9% 38551 softirqs.CPU157.SCHED 17950 +114.0% 38405 softirqs.CPU158.SCHED 17923 +113.7% 38302 softirqs.CPU159.SCHED 17541 ± 2% +98.6% 34837 ± 20% softirqs.CPU160.SCHED 17833 +116.4% 38583 softirqs.CPU161.SCHED 14192 ± 29% +173.0% 38749 softirqs.CPU162.SCHED 17193 ± 3% +122.9% 38327 softirqs.CPU163.SCHED 16821 ± 9% +131.5% 38948 softirqs.CPU164.SCHED 16698 ± 8% +133.0% 38903 softirqs.CPU165.SCHED 16020 ± 12% +138.9% 38264 ± 2% softirqs.CPU166.SCHED 16919 ± 8% +130.1% 38934 softirqs.CPU167.SCHED 16632 ± 2% +32.4% 22019 ± 17% softirqs.CPU168.SCHED 17823 ± 2% +116.6% 38604 ± 2% softirqs.CPU169.SCHED 17983 +119.0% 39386 softirqs.CPU170.SCHED 17669 +122.6% 39324 softirqs.CPU171.SCHED 17927 +116.3% 38781 softirqs.CPU172.SCHED 18141 +116.2% 39216 softirqs.CPU173.SCHED 17888 +118.6% 39099 softirqs.CPU174.SCHED 18076 +116.9% 39214 softirqs.CPU175.SCHED 17994 +115.3% 38746 softirqs.CPU176.SCHED 17615 ± 3% +121.8% 39065 softirqs.CPU177.SCHED 17871 ± 2% +119.4% 39203 softirqs.CPU178.SCHED 16330 ± 19% +138.6% 38966 softirqs.CPU179.SCHED 16932 ± 10% +73.9% 29442 ± 31% softirqs.CPU18.SCHED 17972 +109.4% 37633 ± 6% softirqs.CPU180.SCHED 17896 +114.7% 38419 softirqs.CPU181.SCHED 18036 +116.6% 39073 softirqs.CPU182.SCHED 17883 ± 2% +118.8% 39123 softirqs.CPU183.SCHED 6988 ± 28% -49.0% 3563 ± 6% softirqs.CPU184.RCU 18068 +118.3% 39452 softirqs.CPU184.SCHED 18037 +116.9% 39131 softirqs.CPU185.SCHED 18445 ± 2% +111.6% 39021 softirqs.CPU186.SCHED 17861 ± 2% +119.1% 39126 softirqs.CPU187.SCHED 18094 +116.5% 39179 softirqs.CPU188.SCHED 18041 +116.1% 38988 softirqs.CPU189.SCHED 17981 +118.7% 39332 softirqs.CPU190.SCHED 18050 +115.7% 38940 softirqs.CPU191.SCHED 17923 ± 3% +113.4% 38253 softirqs.CPU2.SCHED 17588 ± 3% +89.1% 33267 ± 24% softirqs.CPU20.SCHED 17894 ± 2% +88.0% 33642 ± 21% softirqs.CPU21.SCHED 17032 ± 10% +122.3% 37859 softirqs.CPU22.SCHED 15841 ± 19% +138.4% 37766 softirqs.CPU23.SCHED 15987 ± 4% +38.6% 22162 ± 22% softirqs.CPU24.SCHED 17548 ± 4% +93.9% 34034 ± 19% softirqs.CPU25.SCHED 17581 ± 3% +111.1% 37114 ± 4% softirqs.CPU26.SCHED 17571 ± 3% +116.2% 37995 softirqs.CPU27.SCHED 17797 ± 2% +113.3% 37968 softirqs.CPU28.SCHED 17877 +113.8% 38215 softirqs.CPU29.SCHED 17757 ± 2% +95.3% 34689 ± 17% softirqs.CPU3.SCHED 17713 ± 3% +92.0% 34004 ± 20% softirqs.CPU30.SCHED 16785 ± 6% +126.0% 37939 softirqs.CPU31.SCHED 17665 ± 3% +113.8% 37762 softirqs.CPU32.SCHED 17874 ± 2% +112.8% 38032 softirqs.CPU33.SCHED 17826 ± 2% +111.6% 37717 softirqs.CPU34.SCHED 17848 +112.7% 37964 softirqs.CPU35.SCHED 17898 +111.8% 37914 softirqs.CPU36.SCHED 18035 ± 2% +109.2% 37733 softirqs.CPU37.SCHED 17727 ± 2% +113.0% 37751 softirqs.CPU38.SCHED 17779 ± 2% +106.8% 36770 ± 5% softirqs.CPU39.SCHED 16919 ± 6% +123.6% 37824 softirqs.CPU4.SCHED 17784 ± 4% +110.5% 37430 softirqs.CPU40.SCHED 17540 ± 4% +116.5% 37977 softirqs.CPU41.SCHED 17772 ± 2% +113.9% 38021 softirqs.CPU42.SCHED 17473 ± 4% +114.7% 37515 softirqs.CPU43.SCHED 17717 ± 2% +115.0% 38090 softirqs.CPU44.SCHED 17551 ± 5% +116.2% 37950 softirqs.CPU45.SCHED 17784 +113.6% 37995 softirqs.CPU46.SCHED 17887 ± 2% +112.5% 38019 softirqs.CPU47.SCHED 17224 ± 2% +97.3% 33983 ± 16% softirqs.CPU49.SCHED 17391 +94.1% 33758 ± 20% softirqs.CPU5.SCHED 17685 +120.2% 38937 ± 3% softirqs.CPU50.SCHED 8016 ± 27% -46.0% 4329 ± 2% softirqs.CPU51.RCU 17841 +113.4% 38072 softirqs.CPU51.SCHED 17600 ± 3% +116.9% 38171 softirqs.CPU52.SCHED 17239 ± 2% +121.2% 38137 softirqs.CPU53.SCHED 17822 +113.4% 38030 softirqs.CPU54.SCHED 17055 ± 5% +121.8% 37830 softirqs.CPU55.SCHED 17653 ± 2% +115.4% 38022 softirqs.CPU56.SCHED 17894 +112.4% 38004 softirqs.CPU57.SCHED 17582 ± 3% +115.4% 37869 softirqs.CPU58.SCHED 17678 +84.8% 32662 ± 17% softirqs.CPU59.SCHED 17560 ± 4% +90.6% 33468 ± 20% softirqs.CPU6.SCHED 17824 +112.4% 37864 softirqs.CPU60.SCHED 17914 +115.9% 38674 ± 3% softirqs.CPU61.SCHED 17675 +113.5% 37735 softirqs.CPU62.SCHED 17695 +113.5% 37774 softirqs.CPU63.SCHED 17783 +114.3% 38110 softirqs.CPU64.SCHED 17911 +112.9% 38129 softirqs.CPU65.SCHED 17941 +112.6% 38136 softirqs.CPU66.SCHED 17838 +110.9% 37629 softirqs.CPU67.SCHED 17747 +115.4% 38234 softirqs.CPU68.SCHED 17318 ± 3% +120.1% 38109 softirqs.CPU69.SCHED 17585 +104.6% 35981 ± 10% softirqs.CPU70.SCHED 17432 ± 2% +117.9% 37991 softirqs.CPU71.SCHED 14703 ± 5% +50.7% 22151 ± 12% softirqs.CPU72.SCHED 17016 ± 4% +97.0% 33521 ± 24% softirqs.CPU73.SCHED 17008 ± 6% +125.7% 38384 softirqs.CPU74.SCHED 17574 ± 2% +119.6% 38600 softirqs.CPU75.SCHED 17676 ± 3% +115.6% 38107 softirqs.CPU76.SCHED 17819 +115.6% 38416 softirqs.CPU77.SCHED 17807 +114.1% 38120 softirqs.CPU78.SCHED 17659 +116.9% 38296 softirqs.CPU79.SCHED 17796 ± 2% +113.1% 37930 softirqs.CPU8.SCHED 15987 ± 18% +137.9% 38037 softirqs.CPU80.SCHED 17684 ± 2% +116.2% 38237 softirqs.CPU81.SCHED 17850 +115.5% 38466 softirqs.CPU82.SCHED 17789 +115.0% 38252 softirqs.CPU83.SCHED 17743 +113.3% 37843 softirqs.CPU85.SCHED 17804 +115.6% 38388 softirqs.CPU86.SCHED 17931 +114.2% 38416 softirqs.CPU87.SCHED 17271 ± 4% +121.8% 38309 softirqs.CPU88.SCHED 17798 +115.9% 38423 softirqs.CPU89.SCHED 17862 +113.5% 38140 softirqs.CPU90.SCHED 17849 +119.2% 39127 ± 4% softirqs.CPU91.SCHED 17846 +115.3% 38424 softirqs.CPU92.SCHED 17699 +116.5% 38316 softirqs.CPU93.SCHED 17868 +115.6% 38524 softirqs.CPU94.SCHED 18023 ± 2% +112.8% 38355 softirqs.CPU97.SCHED 16630 ± 15% +134.1% 38934 softirqs.CPU98.SCHED 18017 ± 3% +115.8% 38889 softirqs.CPU99.SCHED 6591 ± 56% +299.9% 26359 ± 47% softirqs.NET_RX 1256534 ± 8% -37.6% 783562 softirqs.RCU 3375418 +108.0% 7022041 softirqs.SCHED 44803 ± 3% +32.1% 59178 softirqs.TIMER 775.00 ±142% +2464.5% 19875 ± 92% interrupts.32:PCI-MSI.524290-edge.eth0-TxRx-1 258.50 +134.8% 607.00 interrupts.9:IO-APIC.9-fasteoi.acpi 257240 +135.8% 606454 interrupts.CPU0.LOC:Local_timer_interrupts 0.00 +2e+105% 1951 ±146% interrupts.CPU0.NMI:Non-maskable_interrupts 0.00 +2e+105% 1951 ±146% interrupts.CPU0.PMI:Performance_monitoring_interrupts 258.50 +134.8% 607.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 257926 +136.8% 610646 interrupts.CPU1.LOC:Local_timer_interrupts 0.50 ±100% +64450.0% 322.75 ± 32% interrupts.CPU1.NMI:Non-maskable_interrupts 0.50 ±100% +64450.0% 322.75 ± 32% interrupts.CPU1.PMI:Performance_monitoring_interrupts 258039 +135.1% 606641 interrupts.CPU10.LOC:Local_timer_interrupts 258014 +136.7% 610642 interrupts.CPU100.LOC:Local_timer_interrupts 257987 +135.0% 606348 interrupts.CPU101.LOC:Local_timer_interrupts 257966 +135.0% 606229 interrupts.CPU102.LOC:Local_timer_interrupts 257652 +135.0% 605468 interrupts.CPU103.LOC:Local_timer_interrupts 257994 +134.7% 605530 interrupts.CPU104.LOC:Local_timer_interrupts 257551 +135.4% 606204 interrupts.CPU105.LOC:Local_timer_interrupts 257967 +135.2% 606681 interrupts.CPU106.LOC:Local_timer_interrupts 257986 +135.1% 606441 interrupts.CPU107.LOC:Local_timer_interrupts 257993 +134.7% 605584 interrupts.CPU108.LOC:Local_timer_interrupts 257977 +135.1% 606448 interrupts.CPU109.LOC:Local_timer_interrupts 775.00 ±142% +2464.5% 19875 ± 92% interrupts.CPU11.32:PCI-MSI.524290-edge.eth0-TxRx-1 258008 +135.1% 606672 interrupts.CPU11.LOC:Local_timer_interrupts 257964 +135.1% 606566 interrupts.CPU110.LOC:Local_timer_interrupts 257917 +135.0% 606175 interrupts.CPU111.LOC:Local_timer_interrupts 257983 +135.2% 606679 interrupts.CPU112.LOC:Local_timer_interrupts 9.00 ±134% +597.2% 62.75 ± 88% interrupts.CPU112.RES:Rescheduling_interrupts 257969 +134.9% 606062 interrupts.CPU113.LOC:Local_timer_interrupts 7.00 ±111% +553.6% 45.75 ± 99% interrupts.CPU113.RES:Rescheduling_interrupts 258017 +135.0% 606228 interrupts.CPU114.LOC:Local_timer_interrupts 7.50 ±150% +753.3% 64.00 ± 76% interrupts.CPU114.RES:Rescheduling_interrupts 258012 +135.1% 606643 interrupts.CPU115.LOC:Local_timer_interrupts 1.25 ± 87% +6020.0% 76.50 ± 82% interrupts.CPU115.RES:Rescheduling_interrupts 258004 +134.9% 606081 interrupts.CPU116.LOC:Local_timer_interrupts 1.00 ±100% +8200.0% 83.00 ± 70% interrupts.CPU116.RES:Rescheduling_interrupts 258008 +135.1% 606547 interrupts.CPU117.LOC:Local_timer_interrupts 1.50 ±173% +1.6e+05% 2394 ±146% interrupts.CPU117.NMI:Non-maskable_interrupts 1.50 ±173% +1.6e+05% 2394 ±146% interrupts.CPU117.PMI:Performance_monitoring_interrupts 1.25 ± 87% +4300.0% 55.00 ±116% interrupts.CPU117.RES:Rescheduling_interrupts 257990 +135.7% 608177 interrupts.CPU118.LOC:Local_timer_interrupts 258010 +135.1% 606654 interrupts.CPU119.LOC:Local_timer_interrupts 258067 +135.0% 606502 interrupts.CPU12.LOC:Local_timer_interrupts 257434 +135.3% 605760 interrupts.CPU120.LOC:Local_timer_interrupts 258027 +134.6% 605384 interrupts.CPU121.LOC:Local_timer_interrupts 255276 +137.2% 605433 interrupts.CPU122.LOC:Local_timer_interrupts 256928 +135.3% 604514 interrupts.CPU123.LOC:Local_timer_interrupts 257505 +135.2% 605681 interrupts.CPU124.LOC:Local_timer_interrupts 257792 +134.9% 605545 interrupts.CPU125.LOC:Local_timer_interrupts 256996 +135.6% 605474 interrupts.CPU126.LOC:Local_timer_interrupts 257022 +135.6% 605654 interrupts.CPU127.LOC:Local_timer_interrupts 258006 +134.8% 605780 interrupts.CPU128.LOC:Local_timer_interrupts 257568 +135.2% 605783 interrupts.CPU129.LOC:Local_timer_interrupts 258047 +135.1% 606566 interrupts.CPU13.LOC:Local_timer_interrupts 31.75 ±165% +392.9% 156.50 ± 65% interrupts.CPU13.RES:Rescheduling_interrupts 257934 +134.9% 605825 interrupts.CPU130.LOC:Local_timer_interrupts 257688 +135.1% 605785 interrupts.CPU131.LOC:Local_timer_interrupts 257934 +134.9% 605802 interrupts.CPU132.LOC:Local_timer_interrupts 258031 +134.8% 605766 interrupts.CPU133.LOC:Local_timer_interrupts 257182 +135.6% 605817 interrupts.CPU134.LOC:Local_timer_interrupts 0.50 ±173% +55750.0% 279.25 ± 8% interrupts.CPU134.NMI:Non-maskable_interrupts 0.50 ±173% +55750.0% 279.25 ± 8% interrupts.CPU134.PMI:Performance_monitoring_interrupts 256554 +136.0% 605423 interrupts.CPU135.LOC:Local_timer_interrupts 256889 +135.6% 605342 interrupts.CPU136.LOC:Local_timer_interrupts 257503 +135.2% 605773 interrupts.CPU137.LOC:Local_timer_interrupts 257142 +135.6% 605805 interrupts.CPU138.LOC:Local_timer_interrupts 0.25 ±173% +1e+05% 261.00 ± 25% interrupts.CPU138.NMI:Non-maskable_interrupts 0.25 ±173% +1e+05% 261.00 ± 25% interrupts.CPU138.PMI:Performance_monitoring_interrupts 1951 ± 13% +44.1% 2812 ± 28% interrupts.CPU139.CAL:Function_call_interrupts 257098 +135.6% 605636 interrupts.CPU139.LOC:Local_timer_interrupts 257999 +135.1% 606552 interrupts.CPU14.LOC:Local_timer_interrupts 257879 +135.0% 605887 interrupts.CPU140.LOC:Local_timer_interrupts 0.25 ±173% +1e+05% 262.50 ± 25% interrupts.CPU140.NMI:Non-maskable_interrupts 0.25 ±173% +1e+05% 262.50 ± 25% interrupts.CPU140.PMI:Performance_monitoring_interrupts 257988 +134.8% 605783 interrupts.CPU141.LOC:Local_timer_interrupts 0.25 ±173% +1.8e+05% 457.75 ± 59% interrupts.CPU141.NMI:Non-maskable_interrupts 0.25 ±173% +1.8e+05% 457.75 ± 59% interrupts.CPU141.PMI:Performance_monitoring_interrupts 256928 +135.8% 605788 interrupts.CPU142.LOC:Local_timer_interrupts 257551 +135.0% 605360 interrupts.CPU143.LOC:Local_timer_interrupts 258028 +134.8% 605867 interrupts.CPU144.LOC:Local_timer_interrupts 258006 +134.7% 605541 interrupts.CPU145.LOC:Local_timer_interrupts 258024 +134.8% 605783 interrupts.CPU146.LOC:Local_timer_interrupts 257614 +135.0% 605470 interrupts.CPU147.LOC:Local_timer_interrupts 257820 +134.9% 605659 interrupts.CPU148.LOC:Local_timer_interrupts 257485 +135.3% 605741 interrupts.CPU149.LOC:Local_timer_interrupts 258573 +134.6% 606547 interrupts.CPU15.LOC:Local_timer_interrupts 257711 +134.8% 605001 interrupts.CPU150.LOC:Local_timer_interrupts 257968 +134.4% 604801 interrupts.CPU151.LOC:Local_timer_interrupts 258008 +134.6% 605227 interrupts.CPU152.LOC:Local_timer_interrupts 340.25 ± 55% -95.3% 16.00 ± 74% interrupts.CPU152.RES:Rescheduling_interrupts 258013 +135.3% 607224 interrupts.CPU153.LOC:Local_timer_interrupts 257749 +135.6% 607168 interrupts.CPU154.LOC:Local_timer_interrupts 258017 +134.8% 605925 interrupts.CPU155.LOC:Local_timer_interrupts 3.25 ±102% +2130.8% 72.50 ± 93% interrupts.CPU155.RES:Rescheduling_interrupts 257962 +136.2% 609340 interrupts.CPU156.LOC:Local_timer_interrupts 257730 +135.1% 605840 interrupts.CPU157.LOC:Local_timer_interrupts 257296 +135.3% 605306 interrupts.CPU158.LOC:Local_timer_interrupts 257269 +135.4% 605609 interrupts.CPU159.LOC:Local_timer_interrupts 258025 +135.0% 606370 interrupts.CPU16.LOC:Local_timer_interrupts 1842 ± 6% +17.6% 2166 ± 6% interrupts.CPU160.CAL:Function_call_interrupts 257442 +135.3% 605884 interrupts.CPU160.LOC:Local_timer_interrupts 257136 +135.6% 605841 interrupts.CPU161.LOC:Local_timer_interrupts 257820 +135.0% 605922 interrupts.CPU162.LOC:Local_timer_interrupts 95.25 ± 80% -89.2% 10.25 ± 68% interrupts.CPU162.RES:Rescheduling_interrupts 257981 +134.9% 605945 interrupts.CPU163.LOC:Local_timer_interrupts 0.50 ±173% +51000.0% 255.50 ± 24% interrupts.CPU163.NMI:Non-maskable_interrupts 0.50 ±173% +51000.0% 255.50 ± 24% interrupts.CPU163.PMI:Performance_monitoring_interrupts 1934 ± 13% +23.0% 2378 ± 8% interrupts.CPU164.CAL:Function_call_interrupts 257995 +134.9% 605908 interrupts.CPU164.LOC:Local_timer_interrupts 257989 +134.9% 605899 interrupts.CPU165.LOC:Local_timer_interrupts 257990 +134.7% 605509 interrupts.CPU166.LOC:Local_timer_interrupts 257814 +134.8% 605385 interrupts.CPU167.LOC:Local_timer_interrupts 257441 +135.4% 605963 interrupts.CPU168.LOC:Local_timer_interrupts 20.00 ± 30% +732.5% 166.50 ± 29% interrupts.CPU168.RES:Rescheduling_interrupts 257218 +135.7% 606154 interrupts.CPU169.LOC:Local_timer_interrupts 0.50 ±173% +1.2e+05% 597.50 ± 80% interrupts.CPU169.NMI:Non-maskable_interrupts 0.50 ±173% +1.2e+05% 597.50 ± 80% interrupts.CPU169.PMI:Performance_monitoring_interrupts 258040 +135.0% 606392 interrupts.CPU17.LOC:Local_timer_interrupts 257869 +135.1% 606194 interrupts.CPU170.LOC:Local_timer_interrupts 257719 +135.2% 606164 interrupts.CPU171.LOC:Local_timer_interrupts 1.50 ±173% +18966.7% 286.00 ± 25% interrupts.CPU171.NMI:Non-maskable_interrupts 1.50 ±173% +18966.7% 286.00 ± 25% interrupts.CPU171.PMI:Performance_monitoring_interrupts 257274 +135.6% 606183 interrupts.CPU172.LOC:Local_timer_interrupts 0.00 +3.1e+104% 306.50 ± 30% interrupts.CPU172.NMI:Non-maskable_interrupts 0.00 +3.1e+104% 306.50 ± 30% interrupts.CPU172.PMI:Performance_monitoring_interrupts 256796 +136.0% 606108 interrupts.CPU173.LOC:Local_timer_interrupts 0.50 ±100% +57050.0% 285.75 ± 25% interrupts.CPU173.NMI:Non-maskable_interrupts 0.50 ±100% +57050.0% 285.75 ± 25% interrupts.CPU173.PMI:Performance_monitoring_interrupts 255382 +137.3% 606147 interrupts.CPU174.LOC:Local_timer_interrupts 256716 +135.9% 605483 interrupts.CPU175.LOC:Local_timer_interrupts 0.50 ±100% +48500.0% 243.00 ± 34% interrupts.CPU175.NMI:Non-maskable_interrupts 0.50 ±100% +48500.0% 243.00 ± 34% interrupts.CPU175.PMI:Performance_monitoring_interrupts 257752 +135.1% 605873 interrupts.CPU176.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 304.25 ± 30% interrupts.CPU176.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 304.25 ± 30% interrupts.CPU176.PMI:Performance_monitoring_interrupts 257997 +135.0% 606185 interrupts.CPU177.LOC:Local_timer_interrupts 257592 +135.2% 605961 interrupts.CPU178.LOC:Local_timer_interrupts 257711 +135.2% 606176 interrupts.CPU179.LOC:Local_timer_interrupts 1.50 ±173% +18883.3% 284.75 ± 24% interrupts.CPU179.NMI:Non-maskable_interrupts 1.50 ±173% +18883.3% 284.75 ± 24% interrupts.CPU179.PMI:Performance_monitoring_interrupts 260122 ± 2% +135.2% 611875 interrupts.CPU18.LOC:Local_timer_interrupts 257975 +135.0% 606231 interrupts.CPU180.LOC:Local_timer_interrupts 257985 +135.0% 606187 interrupts.CPU181.LOC:Local_timer_interrupts 0.25 ±173% +1e+05% 261.00 ± 42% interrupts.CPU181.NMI:Non-maskable_interrupts 0.25 ±173% +1e+05% 261.00 ± 42% interrupts.CPU181.PMI:Performance_monitoring_interrupts 257981 +134.9% 605957 interrupts.CPU182.LOC:Local_timer_interrupts 258014 +134.9% 606058 interrupts.CPU183.LOC:Local_timer_interrupts 258003 +134.9% 606132 interrupts.CPU184.LOC:Local_timer_interrupts 257628 +135.3% 606131 interrupts.CPU185.LOC:Local_timer_interrupts 256870 +135.9% 605987 interrupts.CPU186.LOC:Local_timer_interrupts 257549 +135.2% 605680 interrupts.CPU187.LOC:Local_timer_interrupts 257981 +135.0% 606175 interrupts.CPU188.LOC:Local_timer_interrupts 1.25 ±173% +22280.0% 279.75 ± 25% interrupts.CPU188.NMI:Non-maskable_interrupts 1.25 ±173% +22280.0% 279.75 ± 25% interrupts.CPU188.PMI:Performance_monitoring_interrupts 257956 +135.0% 606126 interrupts.CPU189.LOC:Local_timer_interrupts 0.75 ±173% +41700.0% 313.50 ± 34% interrupts.CPU189.NMI:Non-maskable_interrupts 0.75 ±173% +41700.0% 313.50 ± 34% interrupts.CPU189.PMI:Performance_monitoring_interrupts 258147 +135.2% 607115 interrupts.CPU19.LOC:Local_timer_interrupts 1.00 ±122% +10250.0% 103.50 ± 86% interrupts.CPU19.RES:Rescheduling_interrupts 87.75 ± 47% -63.5% 32.00 ± 88% interrupts.CPU19.TLB:TLB_shootdowns 256298 +136.5% 606167 interrupts.CPU190.LOC:Local_timer_interrupts 257838 +135.1% 606204 interrupts.CPU191.LOC:Local_timer_interrupts 2.25 ± 72% +13733.3% 311.25 ± 26% interrupts.CPU191.NMI:Non-maskable_interrupts 2.25 ± 72% +13733.3% 311.25 ± 26% interrupts.CPU191.PMI:Performance_monitoring_interrupts 257840 +135.3% 606664 interrupts.CPU2.LOC:Local_timer_interrupts 258010 +135.1% 606461 interrupts.CPU20.LOC:Local_timer_interrupts 97.25 ± 31% -70.2% 29.00 ±109% interrupts.CPU20.TLB:TLB_shootdowns 258033 +134.7% 605617 interrupts.CPU21.LOC:Local_timer_interrupts 258053 +135.2% 607010 interrupts.CPU22.LOC:Local_timer_interrupts 258009 +135.1% 606487 interrupts.CPU23.LOC:Local_timer_interrupts 259088 +133.8% 605797 interrupts.CPU24.LOC:Local_timer_interrupts 29.00 ± 42% +427.6% 153.00 ± 32% interrupts.CPU24.RES:Rescheduling_interrupts 258265 +134.6% 605784 interrupts.CPU25.LOC:Local_timer_interrupts 258019 +134.8% 605798 interrupts.CPU26.LOC:Local_timer_interrupts 258509 +134.3% 605788 interrupts.CPU27.LOC:Local_timer_interrupts 0.25 ±173% +1.7e+05% 427.00 ± 73% interrupts.CPU27.NMI:Non-maskable_interrupts 0.25 ±173% +1.7e+05% 427.00 ± 73% interrupts.CPU27.PMI:Performance_monitoring_interrupts 258270 +134.6% 605794 interrupts.CPU28.LOC:Local_timer_interrupts 258015 +134.8% 605812 interrupts.CPU29.LOC:Local_timer_interrupts 0.25 ±173% +1e+05% 259.25 ± 23% interrupts.CPU29.NMI:Non-maskable_interrupts 0.25 ±173% +1e+05% 259.25 ± 23% interrupts.CPU29.PMI:Performance_monitoring_interrupts 258002 +135.0% 606425 interrupts.CPU3.LOC:Local_timer_interrupts 258053 +134.8% 605796 interrupts.CPU30.LOC:Local_timer_interrupts 0.25 ±173% +73500.0% 184.00 ± 32% interrupts.CPU30.NMI:Non-maskable_interrupts 0.25 ±173% +73500.0% 184.00 ± 32% interrupts.CPU30.PMI:Performance_monitoring_interrupts 258163 +134.7% 605804 interrupts.CPU31.LOC:Local_timer_interrupts 258086 +134.7% 605795 interrupts.CPU32.LOC:Local_timer_interrupts 258056 +134.7% 605784 interrupts.CPU33.LOC:Local_timer_interrupts 258071 +134.7% 605817 interrupts.CPU34.LOC:Local_timer_interrupts 258062 +134.8% 605813 interrupts.CPU35.LOC:Local_timer_interrupts 258033 +134.8% 605820 interrupts.CPU36.LOC:Local_timer_interrupts 258047 +134.8% 605794 interrupts.CPU37.LOC:Local_timer_interrupts 258035 +134.8% 605828 interrupts.CPU38.LOC:Local_timer_interrupts 0.25 ±173% +1.8e+05% 443.25 ± 57% interrupts.CPU38.NMI:Non-maskable_interrupts 0.25 ±173% +1.8e+05% 443.25 ± 57% interrupts.CPU38.PMI:Performance_monitoring_interrupts 258037 +134.8% 605808 interrupts.CPU39.LOC:Local_timer_interrupts 0.25 ±173% +1.8e+05% 448.50 ± 77% interrupts.CPU39.NMI:Non-maskable_interrupts 0.25 ±173% +1.8e+05% 448.50 ± 77% interrupts.CPU39.PMI:Performance_monitoring_interrupts 257903 +135.1% 606264 interrupts.CPU4.LOC:Local_timer_interrupts 0.25 ±173% +1e+05% 257.75 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts 0.25 ±173% +1e+05% 257.75 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts 257882 +134.9% 605785 interrupts.CPU40.LOC:Local_timer_interrupts 0.25 ±173% +2.3e+05% 574.25 ± 99% interrupts.CPU40.NMI:Non-maskable_interrupts 0.25 ±173% +2.3e+05% 574.25 ± 99% interrupts.CPU40.PMI:Performance_monitoring_interrupts 257998 +134.6% 605337 interrupts.CPU41.LOC:Local_timer_interrupts 48.75 ± 83% -86.2% 6.75 ±109% interrupts.CPU41.RES:Rescheduling_interrupts 258079 +134.7% 605611 interrupts.CPU42.LOC:Local_timer_interrupts 0.50 ±173% +52250.0% 261.75 ± 25% interrupts.CPU42.NMI:Non-maskable_interrupts 0.50 ±173% +52250.0% 261.75 ± 25% interrupts.CPU42.PMI:Performance_monitoring_interrupts 257555 +135.2% 605682 interrupts.CPU43.LOC:Local_timer_interrupts 258011 +134.8% 605819 interrupts.CPU44.LOC:Local_timer_interrupts 110.00 ± 58% -96.1% 4.25 ± 56% interrupts.CPU44.RES:Rescheduling_interrupts 258046 +134.7% 605702 interrupts.CPU45.LOC:Local_timer_interrupts 101.00 ± 89% -97.3% 2.75 ±110% interrupts.CPU45.RES:Rescheduling_interrupts 258036 +134.8% 605870 interrupts.CPU46.LOC:Local_timer_interrupts 258037 +134.7% 605737 interrupts.CPU47.LOC:Local_timer_interrupts 257967 +134.9% 605917 interrupts.CPU48.LOC:Local_timer_interrupts 257674 +135.2% 605935 interrupts.CPU49.LOC:Local_timer_interrupts 258062 +135.1% 606595 interrupts.CPU5.LOC:Local_timer_interrupts 258045 +134.9% 606025 interrupts.CPU50.LOC:Local_timer_interrupts 258020 +134.8% 605923 interrupts.CPU51.LOC:Local_timer_interrupts 257886 +135.0% 605918 interrupts.CPU52.LOC:Local_timer_interrupts 258018 +134.8% 605948 interrupts.CPU53.LOC:Local_timer_interrupts 258032 +134.8% 605884 interrupts.CPU54.LOC:Local_timer_interrupts 55.25 ± 76% -95.5% 2.50 ± 60% interrupts.CPU54.RES:Rescheduling_interrupts 257864 +135.0% 605919 interrupts.CPU55.LOC:Local_timer_interrupts 258008 +134.8% 605925 interrupts.CPU56.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 299.25 ± 4% interrupts.CPU56.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 299.25 ± 4% interrupts.CPU56.PMI:Performance_monitoring_interrupts 258009 +134.9% 605943 interrupts.CPU57.LOC:Local_timer_interrupts 257781 +135.1% 606003 interrupts.CPU58.LOC:Local_timer_interrupts 258019 +134.8% 605916 interrupts.CPU59.LOC:Local_timer_interrupts 257973 +135.1% 606373 interrupts.CPU6.LOC:Local_timer_interrupts 258077 +134.8% 605916 interrupts.CPU60.LOC:Local_timer_interrupts 257983 +134.8% 605737 interrupts.CPU61.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 293.00 ± 3% interrupts.CPU61.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 293.00 ± 3% interrupts.CPU61.PMI:Performance_monitoring_interrupts 260661 ± 2% +132.5% 605907 interrupts.CPU62.LOC:Local_timer_interrupts 258162 +134.7% 605888 interrupts.CPU63.LOC:Local_timer_interrupts 258056 +134.8% 605938 interrupts.CPU64.LOC:Local_timer_interrupts 257625 +135.2% 605921 interrupts.CPU65.LOC:Local_timer_interrupts 2005 ± 9% +51.1% 3030 ± 44% interrupts.CPU66.CAL:Function_call_interrupts 257972 +134.9% 605902 interrupts.CPU66.LOC:Local_timer_interrupts 257840 +135.0% 605920 interrupts.CPU67.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 291.25 ± 2% interrupts.CPU67.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 291.25 ± 2% interrupts.CPU67.PMI:Performance_monitoring_interrupts 255873 +136.8% 605914 interrupts.CPU68.LOC:Local_timer_interrupts 257918 +134.9% 605936 interrupts.CPU69.LOC:Local_timer_interrupts 257750 +135.5% 607125 interrupts.CPU7.LOC:Local_timer_interrupts 258004 +134.8% 605804 interrupts.CPU70.LOC:Local_timer_interrupts 1865 ± 11% +16.2% 2167 ± 7% interrupts.CPU71.CAL:Function_call_interrupts 260658 ± 2% +132.5% 605957 interrupts.CPU71.LOC:Local_timer_interrupts 257805 +135.0% 605950 interrupts.CPU72.LOC:Local_timer_interrupts 258043 +134.9% 606168 interrupts.CPU73.LOC:Local_timer_interrupts 0.25 ±173% +9.2e+05% 2310 ±153% interrupts.CPU73.NMI:Non-maskable_interrupts 0.25 ±173% +9.2e+05% 2310 ±153% interrupts.CPU73.PMI:Performance_monitoring_interrupts 258016 +134.9% 606153 interrupts.CPU74.LOC:Local_timer_interrupts 258012 +135.0% 606224 interrupts.CPU75.LOC:Local_timer_interrupts 258034 +134.9% 606203 interrupts.CPU76.LOC:Local_timer_interrupts 257998 +135.0% 606202 interrupts.CPU77.LOC:Local_timer_interrupts 0.25 ±173% +1.1e+05% 286.75 ± 26% interrupts.CPU77.NMI:Non-maskable_interrupts 0.25 ±173% +1.1e+05% 286.75 ± 26% interrupts.CPU77.PMI:Performance_monitoring_interrupts 257982 +135.0% 606209 interrupts.CPU78.LOC:Local_timer_interrupts 258005 +135.0% 606194 interrupts.CPU79.LOC:Local_timer_interrupts 258005 +135.1% 606517 interrupts.CPU8.LOC:Local_timer_interrupts 257963 +135.0% 606187 interrupts.CPU80.LOC:Local_timer_interrupts 258018 +134.9% 606189 interrupts.CPU81.LOC:Local_timer_interrupts 258015 +135.0% 606248 interrupts.CPU82.LOC:Local_timer_interrupts 258001 +135.0% 606206 interrupts.CPU83.LOC:Local_timer_interrupts 258067 +134.9% 606176 interrupts.CPU84.LOC:Local_timer_interrupts 258026 +134.9% 606002 interrupts.CPU85.LOC:Local_timer_interrupts 1941 ± 12% +51.5% 2940 ± 24% interrupts.CPU86.CAL:Function_call_interrupts 258049 +134.9% 606232 interrupts.CPU86.LOC:Local_timer_interrupts 258021 +134.9% 606195 interrupts.CPU87.LOC:Local_timer_interrupts 258025 +134.9% 606192 interrupts.CPU88.LOC:Local_timer_interrupts 258019 +135.0% 606238 interrupts.CPU89.LOC:Local_timer_interrupts 257891 +135.2% 606490 interrupts.CPU9.LOC:Local_timer_interrupts 258039 +134.9% 606176 interrupts.CPU90.LOC:Local_timer_interrupts 257929 +135.0% 606125 interrupts.CPU91.LOC:Local_timer_interrupts 258045 +134.9% 606114 interrupts.CPU92.LOC:Local_timer_interrupts 256784 +136.0% 606132 interrupts.CPU93.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 307.50 ± 32% interrupts.CPU93.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 307.50 ± 32% interrupts.CPU93.PMI:Performance_monitoring_interrupts 255472 +137.3% 606111 interrupts.CPU94.LOC:Local_timer_interrupts 257726 +135.1% 606036 interrupts.CPU95.LOC:Local_timer_interrupts 0.25 ±173% +1.2e+05% 305.25 ± 28% interrupts.CPU95.NMI:Non-maskable_interrupts 0.25 ±173% +1.2e+05% 305.25 ± 28% interrupts.CPU95.PMI:Performance_monitoring_interrupts 257710 +135.1% 605816 interrupts.CPU96.LOC:Local_timer_interrupts 18.25 ± 62% +352.1% 82.50 ±104% interrupts.CPU96.RES:Rescheduling_interrupts 257967 +135.2% 606640 interrupts.CPU97.LOC:Local_timer_interrupts 0.50 ±100% +60400.0% 302.50 ± 19% interrupts.CPU97.NMI:Non-maskable_interrupts 0.50 ±100% +60400.0% 302.50 ± 19% interrupts.CPU97.PMI:Performance_monitoring_interrupts 257967 +135.1% 606427 interrupts.CPU98.LOC:Local_timer_interrupts 258011 +135.0% 606237 interrupts.CPU99.LOC:Local_timer_interrupts 49504415 +135.1% 1.164e+08 interrupts.LOC:Local_timer_interrupts 0.00 +1.9e+104% 192.00 interrupts.MCP:Machine_check_polls 17.50 ± 24% +8.5e+05% 148487 ± 7% interrupts.NMI:Non-maskable_interrupts 17.50 ± 24% +8.5e+05% 148487 ± 7% interrupts.PMI:Performance_monitoring_interrupts 10597 ± 10% -46.8% 5641 ± 2% interrupts.RES:Rescheduling_interrupts 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode 15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare 13.10 ±116% -13.1 0.00 perf-profile.calltrace.cycles-pp.task_work_run.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart 10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.seq_read_iter.proc_reg_read_iter.new_sync_read.vfs_read.ksys_read 10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.proc_reg_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 10.09 ±143% -10.1 0.00 perf-profile.calltrace.cycles-pp.__fput.task_work_run.do_exit.do_group_exit.get_signal 9.02 ±118% -9.0 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.02 ±118% -9.0 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.00 ±139% -9.0 0.00 perf-profile.calltrace.cycles-pp.perf_release.__fput.task_work_run.do_exit.do_group_exit 9.00 ±139% -9.0 0.00 perf-profile.calltrace.cycles-pp.perf_event_release_kernel.perf_release.__fput.task_work_run.do_exit 7.15 ± 68% -7.1 0.00 perf-profile.calltrace.cycles-pp.show_interrupts.seq_read_iter.proc_reg_read_iter.new_sync_read.vfs_read 6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.smp_call_function_single.event_function_call.perf_remove_from_context.perf_event_release_kernel.perf_release 6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.event_function_call.perf_remove_from_context.perf_event_release_kernel.perf_release.__fput 6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.perf_remove_from_context.perf_event_release_kernel.perf_release.__fput.task_work_run 6.39 ±127% -6.4 0.00 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 6.39 ±127% -6.4 0.00 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 6.34 ±109% -6.3 0.00 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.34 ±109% -6.3 0.00 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.__libc_start_main 4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.main.__libc_start_main 4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.run_builtin.main.__libc_start_main 4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.__libc_start_main 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry 3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.read 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.lookup_fast.walk_component.path_lookupat.filename_lookup.vfs_statx 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.walk_component.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat 0.00 +0.6 0.55 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu 0.00 +0.6 0.62 ± 6% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 0.00 +0.6 0.63 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp._find_next_bit.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes 0.00 +0.7 0.73 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +0.7 0.74 ± 5% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +0.8 0.79 ± 6% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.8 0.81 ± 11% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.00 +0.8 0.85 ± 12% perf-profile.calltrace.cycles-pp.pagecache_get_page.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +0.9 0.87 ± 12% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.00 +0.9 0.87 ± 12% perf-profile.calltrace.cycles-pp.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +0.9 0.88 ± 8% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.00 +0.9 0.93 ± 18% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write 0.00 +0.9 0.94 ± 18% perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter 0.00 +0.9 0.95 ± 18% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +1.0 0.98 ± 17% perf-profile.calltrace.cycles-pp.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +1.0 1.05 ± 10% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 0.00 +1.1 1.07 ± 32% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages 0.00 +1.2 1.15 ± 11% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 0.00 +1.2 1.22 ± 6% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 0.00 +1.3 1.29 ± 13% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +1.6 1.61 ± 32% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode 0.00 +2.1 2.05 ± 32% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work.worker_thread 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn 0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb 0.00 +2.2 2.25 ± 28% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +2.2 2.25 ± 28% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 0.00 +2.3 2.34 ± 28% perf-profile.calltrace.cycles-pp.ret_from_fork 0.00 +2.3 2.34 ± 28% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.00 +2.8 2.81 ± 7% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +2.8 2.83 ± 7% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +2.8 2.84 ± 7% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +4.0 4.00 ± 13% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 1.09 ±173% +4.1 5.15 ± 5% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 1.09 ±173% +8.1 9.19 ± 4% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.00 +11.6 11.61 ± 5% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +14.8 14.81 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 0.00 +15.1 15.06 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 0.00 +17.0 17.02 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 0.00 +17.2 17.20 ± 12% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 0.00 +17.3 17.31 ± 12% perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +17.3 17.34 ± 12% perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +19.6 19.56 ± 11% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 0.00 +19.6 19.56 ± 11% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +19.9 19.86 ± 11% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 15.69 ± 61% +26.3 42.00 ± 12% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +41.7 41.71 ± 12% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 0.00 +41.8 41.82 ± 12% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.00 +41.8 41.84 ± 12% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +41.9 41.93 ± 12% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +42.0 41.96 ± 12% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 19.48 ± 62% -19.5 0.00 perf-profile.children.cycles-pp.do_group_exit 19.48 ± 62% -19.5 0.00 perf-profile.children.cycles-pp.do_exit 16.98 ± 75% -17.0 0.00 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 16.98 ± 75% -17.0 0.00 perf-profile.children.cycles-pp.exit_to_user_mode_prepare 15.60 ± 87% -15.6 0.00 perf-profile.children.cycles-pp.arch_do_signal_or_restart 15.60 ± 87% -15.6 0.00 perf-profile.children.cycles-pp.get_signal 14.48 ±102% -14.5 0.00 perf-profile.children.cycles-pp.task_work_run 12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.ksys_read 12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.vfs_read 12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.seq_read_iter 11.47 ±123% -11.5 0.00 perf-profile.children.cycles-pp.__fput 10.41 ± 93% -10.4 0.00 perf-profile.children.cycles-pp.new_sync_read 10.41 ± 93% -10.4 0.00 perf-profile.children.cycles-pp.proc_reg_read_iter 9.00 ±139% -9.0 0.00 perf-profile.children.cycles-pp.perf_release 9.00 ±139% -9.0 0.00 perf-profile.children.cycles-pp.perf_event_release_kernel 7.15 ± 68% -7.1 0.00 perf-profile.children.cycles-pp.show_interrupts 6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.smp_call_function_single 6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.event_function_call 6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.perf_remove_from_context 6.39 ±127% -6.4 0.00 perf-profile.children.cycles-pp.schedule_idle 6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_sys_open 6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_sys_openat2 6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_filp_open 6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.path_openat 4.70 ±103% -4.7 0.00 perf-profile.children.cycles-pp.lookup_fast 4.70 ±103% -4.7 0.00 perf-profile.children.cycles-pp.walk_component 4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.__libc_start_main 4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.main 4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.run_builtin 4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.cmd_record 3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.__x64_sys_exit_group 3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.mmput 3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.exit_mmap 3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.pick_next_task_fair 3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.read 3.56 ±104% -3.6 0.00 perf-profile.children.cycles-pp.seq_printf 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__do_sys_newlstat 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__d_lookup 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.filename_lookup 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.path_lookupat 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__lxstat64 3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.vfs_statx 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.dec_zone_page_state 0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.arch_scale_freq_tick 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.btrfs_write_check 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.rcu_idle_exit 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.free_extent_state 0.00 +0.1 0.07 ± 21% perf-profile.children.cycles-pp.merge_state 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.__mod_lruvec_page_state 0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.__mod_node_page_state 0.00 +0.1 0.09 ± 26% perf-profile.children.cycles-pp.crc_24 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.update_rq_clock 0.00 +0.1 0.09 ± 23% perf-profile.children.cycles-pp.find_get_entry 0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.btrfs_transaction_in_commit 0.00 +0.1 0.09 ± 20% perf-profile.children.cycles-pp.calc_global_load_tick 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.__mod_memcg_state 0.00 +0.1 0.10 ± 23% perf-profile.children.cycles-pp.crc_32 0.00 +0.1 0.10 ± 23% perf-profile.children.cycles-pp.rmqueue 0.00 +0.1 0.10 ± 24% perf-profile.children.cycles-pp.kernel_fpu_begin 0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.__mod_lruvec_state 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.blk_mq_submit_bio 0.00 +0.1 0.10 ± 15% perf-profile.children.cycles-pp.account_process_tick 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.run_one_async_done 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.btrfs_map_bio 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.submit_bio 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.submit_bio_noacct 0.00 +0.1 0.11 ± 25% perf-profile.children.cycles-pp.node_dirty_ok 0.00 +0.1 0.11 ± 33% perf-profile.children.cycles-pp.crc_41 0.00 +0.1 0.11 ± 23% perf-profile.children.cycles-pp.__crc32c_pcl_intel_finup 0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.clear_state_bit 0.00 +0.1 0.12 ± 13% perf-profile.children.cycles-pp.native_sched_clock 0.00 +0.1 0.12 ± 11% perf-profile.children.cycles-pp.update_sd_lb_stats 0.00 +0.1 0.12 ± 14% perf-profile.children.cycles-pp.__pagevec_lru_add 0.00 +0.1 0.12 ± 15% perf-profile.children.cycles-pp.sched_clock 0.00 +0.1 0.12 ± 34% perf-profile.children.cycles-pp.btrfs_find_delalloc_range 0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.account_page_dirtied 0.00 +0.1 0.12 ± 12% perf-profile.children.cycles-pp.find_busiest_group 0.00 +0.1 0.13 ± 29% perf-profile.children.cycles-pp._raw_read_lock 0.00 +0.1 0.13 ± 35% perf-profile.children.cycles-pp.crc_96 0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.btrfs_run_delalloc_range 0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.run_delalloc_nocow 0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.find_get_pages_contig 0.00 +0.1 0.14 ± 6% perf-profile.children.cycles-pp.lock_extent_bits 0.00 +0.1 0.14 ± 11% perf-profile.children.cycles-pp.lru_cache_add 0.00 +0.1 0.14 ± 10% perf-profile.children.cycles-pp.sched_clock_cpu 0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.__lookup_extent_mapping 0.00 +0.2 0.15 ± 35% perf-profile.children.cycles-pp.lock_delalloc_pages 0.00 +0.2 0.15 ± 11% perf-profile.children.cycles-pp.load_balance 0.00 +0.2 0.15 ± 32% perf-profile.children.cycles-pp.btrfs_get_chunk_map 0.00 +0.2 0.15 ± 9% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.00 +0.2 0.16 ± 17% perf-profile.children.cycles-pp.btrfs_drop_pages 0.00 +0.2 0.16 ± 14% perf-profile.children.cycles-pp.mem_cgroup_charge 0.00 +0.2 0.16 ± 23% perf-profile.children.cycles-pp.crc_72 0.00 +0.2 0.16 ± 6% perf-profile.children.cycles-pp.read_tsc 0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.find_next_bit 0.00 +0.2 0.17 ± 8% perf-profile.children.cycles-pp.lapic_next_deadline 0.00 +0.2 0.17 ± 13% perf-profile.children.cycles-pp.btrfs_work_helper 0.00 +0.2 0.18 ± 10% perf-profile.children.cycles-pp.xas_load 0.00 +0.2 0.19 ± 18% perf-profile.children.cycles-pp.btrfs_get_alloc_profile 0.00 +0.2 0.21 ± 29% perf-profile.children.cycles-pp.btrfs_get_io_geometry 0.00 +0.2 0.21 ± 17% perf-profile.children.cycles-pp.calc_available_free_space 0.00 +0.2 0.22 ± 29% perf-profile.children.cycles-pp.btrfs_bio_fits_in_stripe 0.00 +0.2 0.23 ± 20% perf-profile.children.cycles-pp.btrfs_get_extent 0.00 +0.2 0.23 ± 6% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.2 0.23 ± 11% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need 0.00 +0.2 0.25 ± 29% perf-profile.children.cycles-pp.crc_42 0.00 +0.3 0.26 ± 13% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.00 +0.3 0.27 ± 25% perf-profile.children.cycles-pp.get_page_from_freelist 0.00 +0.3 0.27 ± 29% perf-profile.children.cycles-pp.submit_extent_page 0.00 +0.3 0.28 ± 33% perf-profile.children.cycles-pp.__process_pages_contig 0.00 +0.3 0.29 ± 32% perf-profile.children.cycles-pp.find_lock_delalloc_range 0.00 +0.3 0.29 ± 22% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.00 +0.3 0.30 ± 35% perf-profile.children.cycles-pp.clear_page_dirty_for_io 0.00 +0.3 0.30 ± 37% perf-profile.children.cycles-pp.test_clear_page_writeback 0.00 +0.3 0.31 ± 24% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.00 +0.3 0.32 ± 14% perf-profile.children.cycles-pp.kmem_cache_alloc 0.00 +0.3 0.33 ± 8% perf-profile.children.cycles-pp.scheduler_tick 0.00 +0.3 0.33 ± 38% perf-profile.children.cycles-pp.end_page_writeback 0.00 +0.3 0.34 ± 4% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.4 0.36 ± 13% perf-profile.children.cycles-pp.btrfs_set_delalloc_extent 0.00 +0.4 0.37 ± 18% perf-profile.children.cycles-pp.alloc_extent_state 0.00 +0.4 0.37 ± 13% perf-profile.children.cycles-pp.set_state_bits 0.00 +0.4 0.37 ± 11% perf-profile.children.cycles-pp.btrfs_reserve_data_bytes 0.00 +0.4 0.38 ± 11% perf-profile.children.cycles-pp.__add_to_page_cache_locked 0.00 +0.4 0.38 ± 10% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.00 +0.4 0.41 ± 8% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.00 +0.4 0.42 ± 35% perf-profile.children.cycles-pp.__test_set_page_writeback 0.00 +0.4 0.42 ± 26% perf-profile.children.cycles-pp.crc_128 0.00 +0.4 0.42 ± 10% perf-profile.children.cycles-pp.btrfs_check_data_free_space 0.00 +0.4 0.43 ± 32% perf-profile.children.cycles-pp.writepage_delalloc 0.00 +0.4 0.43 ± 16% perf-profile.children.cycles-pp.clear_extent_bit 0.00 +0.4 0.43 ± 11% perf-profile.children.cycles-pp.irqtime_account_irq 0.00 +0.5 0.50 ± 12% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +0.5 0.51 ± 14% perf-profile.children.cycles-pp.btrfs_set_extent_delalloc 0.00 +0.5 0.52 ± 13% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.00 +0.5 0.53 ± 10% perf-profile.children.cycles-pp.add_to_page_cache_lru 0.00 +0.5 0.53 ± 35% perf-profile.children.cycles-pp.btrfs_set_range_writeback 0.00 +0.6 0.55 ± 11% perf-profile.children.cycles-pp.set_extent_bit 0.00 +0.6 0.55 ± 4% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.6 0.62 ± 12% perf-profile.children.cycles-pp.__clear_extent_bit 0.00 +0.7 0.66 ±117% perf-profile.children.cycles-pp.start_kernel 0.00 +0.7 0.69 ± 6% perf-profile.children.cycles-pp.update_process_times 0.00 +0.7 0.70 ± 7% perf-profile.children.cycles-pp.tick_sched_handle 0.00 +0.7 0.73 ± 12% perf-profile.children.cycles-pp.tick_nohz_next_event 0.00 +0.8 0.79 ± 5% perf-profile.children.cycles-pp.tick_irq_enter 0.00 +0.8 0.80 ± 5% perf-profile.children.cycles-pp.irq_enter_rcu 0.00 +0.8 0.81 ± 11% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +0.9 0.87 ± 12% perf-profile.children.cycles-pp.prepare_pages 0.00 +0.9 0.93 ± 11% perf-profile.children.cycles-pp.pagecache_get_page 0.00 +0.9 0.94 ± 18% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.00 +0.9 0.95 ± 18% perf-profile.children.cycles-pp.copyin 0.00 +1.0 0.96 ± 18% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 0.00 +1.0 0.96 ± 8% perf-profile.children.cycles-pp.tick_sched_timer 0.00 +1.0 0.98 ± 17% perf-profile.children.cycles-pp.btrfs_copy_from_user 0.00 +1.1 1.08 ± 32% perf-profile.children.cycles-pp.__extent_writepage_io 0.00 +1.1 1.09 ± 10% perf-profile.children.cycles-pp.clockevents_program_event 0.00 +1.3 1.29 ± 13% perf-profile.children.cycles-pp.btrfs_dirty_pages 0.00 +1.3 1.33 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.00 +1.6 1.62 ± 32% perf-profile.children.cycles-pp.__extent_writepage 0.00 +2.1 2.05 ± 32% perf-profile.children.cycles-pp.extent_write_cache_pages 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.wb_workfn 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.wb_writeback 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.__writeback_inodes_wb 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.writeback_sb_inodes 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.__writeback_single_inode 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.extent_writepages 0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.do_writepages 0.00 +2.2 2.25 ± 28% perf-profile.children.cycles-pp.process_one_work 0.00 +2.2 2.25 ± 28% perf-profile.children.cycles-pp.worker_thread 0.00 +2.3 2.29 ± 10% perf-profile.children.cycles-pp.ktime_get 0.00 +2.3 2.34 ± 28% perf-profile.children.cycles-pp.kthread 0.00 +2.3 2.34 ± 28% perf-profile.children.cycles-pp.ret_from_fork 0.00 +3.0 2.97 ± 7% perf-profile.children.cycles-pp.hrtimer_interrupt 0.00 +3.0 3.01 ± 7% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.00 +4.0 4.02 ± 13% perf-profile.children.cycles-pp.__percpu_counter_sum 1.09 ±173% +4.3 5.36 ± 5% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.00 +11.7 11.67 ± 5% perf-profile.children.cycles-pp.menu_select 0.00 +17.3 17.33 ± 12% perf-profile.children.cycles-pp.btrfs_block_rsv_release 0.00 +17.3 17.34 ± 12% perf-profile.children.cycles-pp.btrfs_inode_rsv_release 0.00 +19.6 19.56 ± 11% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 0.00 +19.9 19.85 ± 11% perf-profile.children.cycles-pp.__reserve_bytes 0.00 +19.9 19.86 ± 11% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 0.00 +32.0 31.95 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.00 +33.4 33.40 ± 11% perf-profile.children.cycles-pp._raw_spin_lock 0.00 +41.7 41.73 ± 12% perf-profile.children.cycles-pp.btrfs_buffered_write 0.00 +41.8 41.82 ± 12% perf-profile.children.cycles-pp.btrfs_file_write_iter 0.00 +41.9 41.85 ± 12% perf-profile.children.cycles-pp.new_sync_write 0.00 +41.9 41.95 ± 12% perf-profile.children.cycles-pp.vfs_write 0.00 +42.0 41.98 ± 12% perf-profile.children.cycles-pp.ksys_write 6.82 ±130% -6.8 0.00 perf-profile.self.cycles-pp.smp_call_function_single 4.67 ±100% -4.7 0.00 perf-profile.self.cycles-pp.show_interrupts 3.31 ±102% -3.3 0.00 perf-profile.self.cycles-pp.__d_lookup 0.00 +0.1 0.05 perf-profile.self.cycles-pp.dec_zone_page_state 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rebalance_domains 0.00 +0.1 0.06 ± 15% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.arch_scale_freq_tick 0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.free_extent_state 0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.1 0.07 ± 10% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.00 +0.1 0.07 ± 19% perf-profile.self.cycles-pp.btrfs_get_alloc_profile 0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.btrfs_dirty_pages 0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.08 ± 23% perf-profile.self.cycles-pp.test_clear_page_writeback 0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.btrfs_get_extent 0.00 +0.1 0.08 ± 15% perf-profile.self.cycles-pp.do_idle 0.00 +0.1 0.08 ± 15% perf-profile.self.cycles-pp.btrfs_reserve_data_bytes 0.00 +0.1 0.08 ± 21% perf-profile.self.cycles-pp.__mod_lruvec_page_state 0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.update_sd_lb_stats 0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.__mod_node_page_state 0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.crc_24 0.00 +0.1 0.09 ± 17% perf-profile.self.cycles-pp.btrfs_file_write_iter 0.00 +0.1 0.09 ± 14% perf-profile.self.cycles-pp.__mod_memcg_state 0.00 +0.1 0.09 ± 20% perf-profile.self.cycles-pp.calc_global_load_tick 0.00 +0.1 0.10 ± 28% perf-profile.self.cycles-pp.crc_32 0.00 +0.1 0.10 ± 23% perf-profile.self.cycles-pp.kernel_fpu_begin 0.00 +0.1 0.10 ± 28% perf-profile.self.cycles-pp.clear_page_dirty_for_io 0.00 +0.1 0.10 ± 17% perf-profile.self.cycles-pp.btrfs_set_delalloc_extent 0.00 +0.1 0.10 ± 15% perf-profile.self.cycles-pp.account_process_tick 0.00 +0.1 0.11 ± 32% perf-profile.self.cycles-pp.__test_set_page_writeback 0.00 +0.1 0.11 ± 11% perf-profile.self.cycles-pp.native_sched_clock 0.00 +0.1 0.11 ± 32% perf-profile.self.cycles-pp.__process_pages_contig 0.00 +0.1 0.12 ± 19% perf-profile.self.cycles-pp.__clear_extent_bit 0.00 +0.1 0.12 ± 21% perf-profile.self.cycles-pp.tick_nohz_next_event 0.00 +0.1 0.13 ± 29% perf-profile.self.cycles-pp._raw_read_lock 0.00 +0.1 0.13 ± 35% perf-profile.self.cycles-pp.crc_96 0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.xas_load 0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.__lookup_extent_mapping 0.00 +0.1 0.14 ± 18% perf-profile.self.cycles-pp.update_process_times 0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.btrfs_drop_pages 0.00 +0.2 0.15 ± 8% perf-profile.self.cycles-pp.read_tsc 0.00 +0.2 0.16 ± 23% perf-profile.self.cycles-pp.crc_72 0.00 +0.2 0.16 ± 11% perf-profile.self.cycles-pp.find_next_bit 0.00 +0.2 0.17 ± 8% perf-profile.self.cycles-pp.lapic_next_deadline 0.00 +0.2 0.25 ± 28% perf-profile.self.cycles-pp.crc_42 0.00 +0.3 0.26 ± 17% perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +0.3 0.27 ± 26% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.00 +0.3 0.30 ± 8% perf-profile.self.cycles-pp.__reserve_bytes 0.00 +0.3 0.34 ± 4% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.4 0.38 ± 13% perf-profile.self.cycles-pp.irqtime_account_irq 0.00 +0.4 0.38 ± 10% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.00 +0.4 0.41 ± 12% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.4 0.42 ± 27% perf-profile.self.cycles-pp.crc_128 0.00 +0.5 0.47 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.5 0.52 ± 13% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.00 +0.9 0.94 ± 18% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.00 +1.5 1.47 ± 10% perf-profile.self.cycles-pp._raw_spin_lock 0.00 +2.2 2.17 ± 10% perf-profile.self.cycles-pp.ktime_get 0.00 +2.5 2.47 ± 15% perf-profile.self.cycles-pp.__percpu_counter_sum 0.00 +7.5 7.47 ± 9% perf-profile.self.cycles-pp.cpuidle_enter_state 0.00 +10.8 10.81 ± 5% perf-profile.self.cycles-pp.menu_select 0.00 +31.8 31.83 ± 11% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath *************************************************************************************************** lkp-cpl-4sp1: 144 threads Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/1BRD_48G/btrfs/x86_64-rhel-8.3/4/debian-10.4-x86_64-20200603.cgz/lkp-cpl-4sp1/creat-clo/aim7/0x700001e commit: 5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size") 53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing") 5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7 ---------------- --------------------------- %stddev %change %stddev \ | \ 2188 -3.5% 2112 aim7.jobs-per-min 10.99 +3.7% 11.40 aim7.time.elapsed_time 10.99 +3.7% 11.40 aim7.time.elapsed_time.max 2735 ± 3% +3.9% 2841 aim7.time.maximum_resident_set_size 240926 ± 5% -99.1% 2250 ± 3% aim7.time.voluntary_context_switches 1.99 +10.9% 2.20 ± 12% iostat.cpu.system 0.61 ± 2% +0.0 0.66 ± 4% mpstat.cpu.all.sys% 220726 ± 23% +69.2% 373544 ± 16% softirqs.RCU 37875 ± 4% -89.6% 3955 ± 2% vmstat.system.cs 249812 ± 8% -88.5% 28754 ± 11% cpuidle.POLL.time 196378 ± 6% -96.4% 6990 ± 3% cpuidle.POLL.usage 5390 ± 9% +21.0% 6521 ± 9% numa-meminfo.node1.KernelStack 70376 ± 17% -20.5% 55945 ± 12% numa-meminfo.node3.SUnreclaim 5412 ± 8% +20.4% 6514 ± 9% numa-vmstat.node1.nr_kernel_stack 17593 ± 17% -20.5% 13987 ± 12% numa-vmstat.node3.nr_slab_unreclaimable 13523 ± 6% -77.7% 3022 ± 3% perf-stat.i.context-switches 12463 ± 7% -77.7% 2783 ± 3% perf-stat.ps.context-switches 716.75 +5.0% 752.25 ± 3% proc-vmstat.nr_active_file 716.75 +5.0% 752.25 ± 3% proc-vmstat.nr_zone_active_file 21939 ± 44% -75.9% 5285 ± 29% sched_debug.cfs_rq:/.load_avg.max 2732 ± 25% -51.6% 1323 ± 12% sched_debug.cfs_rq:/.load_avg.stddev 6380 ± 74% +209.7% 19761 ± 28% sched_debug.cfs_rq:/.spread0.max 160664 ± 4% -14.1% 137974 ± 7% interrupts.CAL:Function_call_interrupts 1054 ± 24% +42.9% 1506 ± 25% interrupts.CPU1.CAL:Function_call_interrupts 911.50 +12.3% 1023 ± 6% interrupts.CPU104.CAL:Function_call_interrupts 1795 ± 29% -43.4% 1016 ± 10% interrupts.CPU2.CAL:Function_call_interrupts 776.50 ± 10% -58.9% 319.50 ± 34% interrupts.RES:Rescheduling_interrupts 0.01 ± 89% +365.1% 0.05 ± 41% perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 0.02 ±109% +354.5% 0.08 ± 10% perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 0.01 ± 23% +11226.9% 0.74 ±171% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.03 ± 65% +3965.6% 1.27 ±163% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.81 ± 25% -28.8% 0.58 ± 15% perf-sched.total_wait_time.average.ms 1681 ± 3% -8.0% 1547 ± 8% slabinfo.files_cache.active_objs 1681 ± 3% -8.0% 1547 ± 8% slabinfo.files_cache.num_objs 3846 ± 9% -16.7% 3203 ± 5% slabinfo.fsnotify_mark_connector.active_objs 3846 ± 9% -16.7% 3203 ± 5% slabinfo.fsnotify_mark_connector.num_objs 1903 ± 4% -19.5% 1532 ± 13% slabinfo.khugepaged_mm_slot.active_objs 1903 ± 4% -19.5% 1532 ± 13% slabinfo.khugepaged_mm_slot.num_objs 10.00 ±122% -8.6 1.35 ±173% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.read 5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.new_sync_read.vfs_read 5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read 3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel 3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.__open64_nocancel 11.25 ±110% -9.2 2.03 ±173% perf-profile.children.cycles-pp.ksys_read 11.25 ±110% -9.2 2.03 ±173% perf-profile.children.cycles-pp.vfs_read 10.00 ±122% -8.6 1.35 ±173% perf-profile.children.cycles-pp.new_sync_read 6.25 ±103% -5.6 0.68 ±173% perf-profile.children.cycles-pp.read 5.00 ±100% -5.0 0.00 perf-profile.children.cycles-pp.pipe_read 5.00 ±100% -4.3 0.68 ±173% perf-profile.children.cycles-pp.schedule 5.00 ±100% -3.6 1.35 ±173% perf-profile.children.cycles-pp.__schedule 3.59 ±106% -1.1 2.50 ±173% perf-profile.children.cycles-pp.__open64_nocancel Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang