February 2019 - LKP - Ml01.01.Org

[mm/gup] cdaa813278: stress-ng.numa.ops_per_sec 4671.0% improvement

by kernel test robot

Greeting, FYI, we noticed a 4671.0% improvement of stress-ng.numa.ops_per_sec due to commit: commit: cdaa813278ddc616ee201eacda77f63996b5dd2d ("[PATCH 4/6] mm/gup: track gup-pinned pages") url: https://github.com/0day-ci/linux/commits/john-hubbard-gmail-com/RFC-v2-mm... in testcase: stress-ng on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory with following parameters: nr_threads: 50% disk: 1HDD testtime: 5s class: memory cpufreq_governor: performance ucode: 0xb00002e In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: ltp.cve-2017-18075.pass -100.0% undefined | | test machine | qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=futex1 | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: | | test machine | 192 threads Skylake-4S with 704G memory | | test parameters | class=cpu | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=1s | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.numa.ops_per_sec 401.7% improvement | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | class=pipe | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=1s | +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: stress-ng.vm-splice.ops_per_sec -18.5% regression | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=futex2 | | | ucode=0xb00002e | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory | | test parameters | class=os | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=1s | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.futex.ops_per_sec -58.0% regression | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | class=vm | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=1s | | | ucode=0xb00002e | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.vm-splice.ops -58.3% undefined | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory | | test parameters | class=pipe | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=60s | | | ucode=0xb00002e | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: kernel_selftests.memfd.run_fuse_test.sh.pass -100.0% undefined | | test machine | qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G | | test parameters | class=pipe | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=60s | +------------------+---------------------------------------------------------------------------+ | testcase: change | kvm-unit-tests: stress-ng.vm-splice.ops -99.3% undefined | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory | +------------------+---------------------------------------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: memory/gcc-7/performance/1HDD/x86_64-rhel-7.2/50%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep3/stress-ng/5s/0xb00002e commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- %stddev %change %stddev \ | \ 17845 ± 2% +25.1% 22326 ± 22% stress-ng.memcpy.ops 3568 ± 2% +25.1% 4464 ± 22% stress-ng.memcpy.ops_per_sec 55.50 ± 2% +3969.8% 2258 ± 3% stress-ng.numa.ops 9.41 ± 4% +4671.0% 449.07 ± 3% stress-ng.numa.ops_per_sec 277857 ± 5% +50.9% 419386 ± 4% stress-ng.time.involuntary_context_switches 326.50 ± 10% -16.2% 273.50 ± 3% stress-ng.vm-addr.ops 65.30 ± 10% -16.2% 54.71 ± 3% stress-ng.vm-addr.ops_per_sec 0.01 ±113% +0.0 0.02 ± 36% mpstat.cpu.iowait% 181260 +15.7% 209701 vmstat.system.in 64963 ± 32% +315.5% 269897 ± 38% numa-numastat.node0.other_node 66083 ± 33% +904.9% 664072 ± 18% numa-numastat.node1.other_node 1670 ± 2% +36.4% 2279 ± 3% slabinfo.numa_policy.active_objs 1670 ± 2% +36.4% 2279 ± 3% slabinfo.numa_policy.num_objs 38695830 +29.5% 50110710 turbostat.IRQ 10.08 -2.9% 9.79 turbostat.RAMWatt 917912 ± 10% +77.1% 1625317 ± 27% meminfo.Active 917736 ± 10% +77.1% 1624942 ± 27% meminfo.Active(anon) 425637 ± 10% +43.8% 611934 ± 18% meminfo.Inactive 425296 ± 10% +43.8% 611561 ± 18% meminfo.Inactive(anon) 3195353 ± 3% +26.7% 4047153 ± 14% meminfo.Memused 49.89 ± 5% -26.7% 36.55 ± 6% perf-stat.i.MPKI 4.372e+08 ± 2% -5.1% 4.151e+08 ± 3% perf-stat.i.cache-references 9.75 ± 5% -8.8% 8.90 ± 3% perf-stat.i.cpi 4.84 ± 3% -8.2% 4.44 ± 4% perf-stat.overall.MPKI 4.361e+08 ± 2% -5.2% 4.137e+08 ± 3% perf-stat.ps.cache-references 27667 ± 19% +32.4% 36627 ± 17% softirqs.CPU0.RCU 24079 ± 6% -12.6% 21054 ± 8% softirqs.CPU24.SCHED 29249 ± 4% +14.0% 33351 ± 4% softirqs.CPU25.RCU 29618 ± 4% +16.8% 34600 ± 8% softirqs.CPU27.RCU 37158 ± 3% -12.6% 32476 ± 12% softirqs.CPU69.RCU 446518 ± 23% +87.1% 835393 ± 18% numa-meminfo.node0.Active 446388 ± 23% +87.1% 835224 ± 18% numa-meminfo.node0.Active(anon) 279797 ± 60% +81.1% 506718 ± 7% numa-meminfo.node0.Inactive 279545 ± 60% +81.2% 506611 ± 7% numa-meminfo.node0.Inactive(anon) 57425 ± 7% +16.8% 67077 ± 3% numa-meminfo.node0.KReclaimable 1666995 ± 17% +37.9% 2299473 ± 4% numa-meminfo.node0.MemUsed 57425 ± 7% +16.8% 67077 ± 3% numa-meminfo.node0.SReclaimable 59307 ± 7% -18.7% 48201 ± 4% numa-meminfo.node1.KReclaimable 59307 ± 7% -18.7% 48201 ± 4% numa-meminfo.node1.SReclaimable 114842 ± 19% +86.4% 214054 ± 15% numa-vmstat.node0.nr_active_anon 71405 ± 61% +78.6% 127550 ± 8% numa-vmstat.node0.nr_inactive_anon 257.00 ± 12% -43.0% 146.50 ± 30% numa-vmstat.node0.nr_isolated_anon 14332 ± 7% +17.2% 16792 ± 3% numa-vmstat.node0.nr_slab_reclaimable 114840 ± 19% +86.4% 214050 ± 15% numa-vmstat.node0.nr_zone_active_anon 71404 ± 61% +78.6% 127549 ± 8% numa-vmstat.node0.nr_zone_inactive_anon 39507 ± 32% +250.0% 138280 ± 37% numa-vmstat.node0.numa_other 313.75 ± 5% -57.1% 134.50 ± 25% numa-vmstat.node1.nr_isolated_anon 14816 ± 7% -18.5% 12069 ± 4% numa-vmstat.node1.nr_slab_reclaimable 172051 ± 7% +177.8% 477913 ± 13% numa-vmstat.node1.numa_other 231722 ± 8% +72.6% 399865 ± 27% proc-vmstat.nr_active_anon 291.00 ± 16% +18.6% 345.00 ± 19% proc-vmstat.nr_anon_transparent_hugepages 107048 ± 8% +43.8% 153932 ± 19% proc-vmstat.nr_inactive_anon 643.25 ± 6% -55.9% 283.75 ± 25% proc-vmstat.nr_isolated_anon 231722 ± 8% +72.6% 399865 ± 27% proc-vmstat.nr_zone_active_anon 107048 ± 8% +43.8% 153932 ± 19% proc-vmstat.nr_zone_inactive_anon 131052 ± 3% +612.7% 933974 ± 3% proc-vmstat.numa_other 6.265e+08 ± 3% -13.6% 5.411e+08 ± 5% proc-vmstat.pgalloc_normal 6.264e+08 ± 3% -13.7% 5.403e+08 ± 5% proc-vmstat.pgfree 7486 ±122% +2087.9% 163793 ± 6% proc-vmstat.pgmigrate_fail 1047865 ± 18% +134.0% 2452064 ± 15% proc-vmstat.pgmigrate_success 234795 ± 26% -60.5% 92683 ± 78% proc-vmstat.thp_deferred_split_page 417.00 ±145% +3214.7% 13822 ±144% proc-vmstat.unevictable_pgs_cleared 417.50 ±144% +3211.2% 13824 ±144% proc-vmstat.unevictable_pgs_stranded 46.55 -46.6 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_move_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 46.55 -46.6 0.00 perf-profile.calltrace.cycles-pp.kernel_move_pages.__x64_sys_move_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 46.47 -46.5 0.00 perf-profile.calltrace.cycles-pp.do_move_pages_to_node.kernel_move_pages.__x64_sys_move_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 46.47 -46.5 0.00 perf-profile.calltrace.cycles-pp.migrate_pages.do_move_pages_to_node.kernel_move_pages.__x64_sys_move_pages.do_syscall_64 45.32 -45.3 0.00 perf-profile.calltrace.cycles-pp.move_to_new_page.migrate_pages.do_move_pages_to_node.kernel_move_pages.__x64_sys_move_pages 45.32 -45.3 0.00 perf-profile.calltrace.cycles-pp.migrate_page.move_to_new_page.migrate_pages.do_move_pages_to_node.kernel_move_pages 45.22 -45.2 0.00 perf-profile.calltrace.cycles-pp.migrate_page_copy.migrate_page.move_to_new_page.migrate_pages.do_move_pages_to_node 43.53 -43.0 0.55 ± 62% perf-profile.calltrace.cycles-pp.copy_page.migrate_page_copy.migrate_page.move_to_new_page.migrate_pages 78.97 ± 2% -6.3 72.65 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 79.20 ± 2% -6.2 72.97 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 5.01 ± 3% -0.9 4.11 ± 3% perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.01 ± 3% -0.9 4.12 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.94 ± 3% -0.9 4.05 ± 3% perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 4.99 ± 3% -0.9 4.10 ± 3% perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.48 ± 3% -0.5 2.02 ± 3% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 2.47 ± 3% -0.5 2.02 ± 3% perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 2.37 ± 3% -0.4 1.93 ± 3% perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap 2.35 ± 3% -0.4 1.92 ± 3% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region 2.38 ± 2% -0.4 1.96 ± 3% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap 2.38 ± 3% -0.4 1.96 ± 3% perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap.__vm_munmap 2.38 ± 3% -0.4 1.96 ± 3% perf-profile.calltrace.cycles-pp.lru_add_drain.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 2.21 ± 3% -0.4 1.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu 2.19 ± 3% -0.4 1.78 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu 2.23 ± 2% -0.4 1.83 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region 2.20 ± 2% -0.4 1.81 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain 0.58 ± 5% +0.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.__vfs_read.vfs_read.ksys_read 0.58 ± 7% +0.2 0.78 ± 4% perf-profile.calltrace.cycles-pp.anon_pipe_buf_release.pipe_read.__vfs_read.vfs_read.ksys_read 0.70 ± 16% +0.2 0.92 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_read 0.65 ± 5% +0.2 0.88 perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 0.66 ± 5% +0.2 0.89 ± 2% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.70 ± 16% +0.2 0.94 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_read.__vfs_read 0.81 ± 5% +0.3 1.10 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_read.__vfs_read.vfs_read 0.83 ± 14% +0.3 1.12 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_read.__vfs_read.vfs_read 0.40 ± 57% +0.3 0.71 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.__vfs_write.vfs_write.ksys_write 0.58 ± 5% +0.3 0.92 ± 28% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.__vfs_write 0.28 ±100% +0.4 0.64 ± 8% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_write.__vfs_write.vfs_write.ksys_write 0.65 ± 5% +0.4 1.02 ± 25% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.__vfs_write.vfs_write 0.26 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.atime_needs_update.touch_atime.pipe_read.__vfs_read.vfs_read 1.04 ± 5% +0.4 1.44 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.__vfs_write.vfs_write.ksys_write 1.06 ± 5% +0.4 1.46 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.__vfs_read.vfs_read.ksys_read 1.21 ± 5% +0.4 1.64 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.14 ± 8% +0.5 1.61 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.__vfs_read.vfs_read.ksys_read 1.30 ± 5% +0.5 1.78 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.__vfs_read 1.16 ± 9% +0.5 1.64 perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.pipe_write.__vfs_write.vfs_write 0.13 ±173% +0.5 0.63 ± 7% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 1.37 ± 6% +0.5 1.86 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.__vfs_read.vfs_read 1.34 ± 6% +0.5 1.85 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.13 ±173% +0.5 0.64 perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 0.14 ±173% +0.5 0.67 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +0.5 0.54 ± 5% perf-profile.calltrace.cycles-pp.rmap_walk_file.remove_migration_ptes.migrate_pages.migrate_to_node.do_migrate_pages 1.25 ± 4% +0.6 1.82 ± 14% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.__vfs_write.vfs_write.ksys_write 0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.6 0.59 ± 6% perf-profile.calltrace.cycles-pp.rmap_walk_anon.try_to_unmap.migrate_pages.migrate_to_node.do_migrate_pages 0.00 +0.6 0.60 ± 8% perf-profile.calltrace.cycles-pp.find_next_bit.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64 1.65 ± 7% +0.6 2.27 ± 4% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.__vfs_write.vfs_write.ksys_write 0.00 +0.6 0.63 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.queue_pages_pte_range.__walk_page_range.walk_page_range.queue_pages_range 0.00 +0.7 0.65 ± 5% perf-profile.calltrace.cycles-pp.remove_migration_ptes.migrate_pages.migrate_to_node.do_migrate_pages.kernel_migrate_pages 1.89 ± 7% +0.7 2.56 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_read.__vfs_read.vfs_read.ksys_read 0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.queue_pages_test_walk.walk_page_range.queue_pages_range.migrate_to_node.do_migrate_pages 2.05 ± 5% +0.7 2.80 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.__vfs_read.vfs_read.ksys_read 0.00 +0.8 0.82 ± 23% perf-profile.calltrace.cycles-pp.migrate_page.move_to_new_page.migrate_pages.migrate_to_node.do_migrate_pages 0.00 +0.8 0.82 ± 23% perf-profile.calltrace.cycles-pp.move_to_new_page.migrate_pages.migrate_to_node.do_migrate_pages.kernel_migrate_pages 0.00 +0.8 0.85 ± 5% perf-profile.calltrace.cycles-pp._vm_normal_page.queue_pages_pte_range.__walk_page_range.walk_page_range.queue_pages_range 2.79 ± 4% +0.9 3.68 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 0.00 +0.9 0.89 ± 3% perf-profile.calltrace.cycles-pp.bitmap_ord_to_pos.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64 2.80 ± 6% +0.9 3.73 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 0.00 +1.2 1.21 ± 7% perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_mask.on_each_cpu_cond_mask.flush_tlb_mm_range.ptep_clear_flush 0.00 +1.3 1.25 ± 7% perf-profile.calltrace.cycles-pp.on_each_cpu_mask.on_each_cpu_cond_mask.flush_tlb_mm_range.ptep_clear_flush.try_to_unmap_one 0.00 +1.4 1.38 ± 7% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.flush_tlb_mm_range.ptep_clear_flush.try_to_unmap_one.rmap_walk_file 3.73 ± 7% +1.4 5.11 ± 2% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.__vfs_write.vfs_write.ksys_write 0.00 +1.6 1.56 ± 7% perf-profile.calltrace.cycles-pp.flush_tlb_mm_range.ptep_clear_flush.try_to_unmap_one.rmap_walk_file.try_to_unmap 0.00 +1.6 1.63 ± 7% perf-profile.calltrace.cycles-pp.ptep_clear_flush.try_to_unmap_one.rmap_walk_file.try_to_unmap.migrate_pages 0.00 +2.2 2.20 ± 6% perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_file.try_to_unmap.migrate_pages.migrate_to_node 0.00 +2.3 2.30 ± 6% perf-profile.calltrace.cycles-pp.rmap_walk_file.try_to_unmap.migrate_pages.migrate_to_node.do_migrate_pages 0.00 +2.9 2.90 ± 6% perf-profile.calltrace.cycles-pp.try_to_unmap.migrate_pages.migrate_to_node.do_migrate_pages.kernel_migrate_pages 8.94 ± 5% +3.4 12.33 perf-profile.calltrace.cycles-pp.pipe_read.__vfs_read.vfs_read.ksys_read.do_syscall_64 9.24 ± 5% +3.5 12.73 perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.65 ± 18% +3.7 17.34 ± 6% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 13.65 ± 18% +3.7 17.35 ± 6% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 13.65 ± 18% +3.7 17.35 ± 6% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 13.89 ± 17% +3.9 17.84 ± 6% perf-profile.calltrace.cycles-pp.secondary_startup_64 10.58 ± 5% +4.0 14.54 perf-profile.calltrace.cycles-pp.pipe_write.__vfs_write.vfs_write.ksys_write.do_syscall_64 10.90 ± 5% +4.1 14.95 perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.36 ± 5% +4.3 15.62 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.83 ± 5% +4.4 16.24 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +4.5 4.54 ± 5% perf-profile.calltrace.cycles-pp.migrate_pages.migrate_to_node.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages 12.69 ± 5% +4.7 17.35 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.15 ± 5% +4.8 17.99 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +5.1 5.10 ± 3% perf-profile.calltrace.cycles-pp.__bitmap_weight.bitmap_bitremap.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages 0.00 +6.7 6.65 ± 4% perf-profile.calltrace.cycles-pp.__bitmap_weight.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64 0.00 +7.9 7.85 ± 4% perf-profile.calltrace.cycles-pp.bitmap_bitremap.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64 0.00 +8.3 8.31 ± 6% perf-profile.calltrace.cycles-pp.queue_pages_pte_range.__walk_page_range.walk_page_range.queue_pages_range.migrate_to_node 0.00 +9.4 9.39 ± 6% perf-profile.calltrace.cycles-pp.__walk_page_range.walk_page_range.queue_pages_range.migrate_to_node.do_migrate_pages 0.00 +10.3 10.28 ± 6% perf-profile.calltrace.cycles-pp.walk_page_range.queue_pages_range.migrate_to_node.do_migrate_pages.kernel_migrate_pages 0.00 +10.3 10.31 ± 6% perf-profile.calltrace.cycles-pp.queue_pages_range.migrate_to_node.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages 0.63 ± 6% +14.3 14.88 ± 4% perf-profile.calltrace.cycles-pp.migrate_to_node.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64 1.25 ± 3% +31.2 32.42 ± 4% perf-profile.calltrace.cycles-pp.do_migrate_pages.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.26 ± 3% +31.4 32.62 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_migrate_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.26 ± 3% +31.4 32.62 ± 4% perf-profile.calltrace.cycles-pp.kernel_migrate_pages.__x64_sys_migrate_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe 360536 ± 3% +1526.6% 5864634 ± 6% interrupts.CAL:Function_call_interrupts 4122 ± 31% +65.0% 6803 interrupts.CPU0.NMI:Non-maskable_interrupts 4122 ± 31% +65.0% 6803 interrupts.CPU0.PMI:Performance_monitoring_interrupts 6247 ± 9% +23.4% 7708 ± 17% interrupts.CPU0.RES:Rescheduling_interrupts 3735 ± 15% +2257.0% 88044 ± 33% interrupts.CPU1.CAL:Function_call_interrupts 6593 ± 16% +23.6% 8150 ± 11% interrupts.CPU1.RES:Rescheduling_interrupts 1605 ± 48% +5360.4% 87680 ± 33% interrupts.CPU1.TLB:TLB_shootdowns 3668 ± 27% +2395.9% 91551 ± 17% interrupts.CPU10.CAL:Function_call_interrupts 3705 ± 45% +87.8% 6957 ± 14% interrupts.CPU10.NMI:Non-maskable_interrupts 3705 ± 45% +87.8% 6957 ± 14% interrupts.CPU10.PMI:Performance_monitoring_interrupts 3692 ± 5% +50.9% 5571 ± 2% interrupts.CPU10.RES:Rescheduling_interrupts 1194 ± 74% +7530.5% 91146 ± 18% interrupts.CPU10.TLB:TLB_shootdowns 3833 ± 29% +2767.2% 109921 ± 10% interrupts.CPU11.CAL:Function_call_interrupts 3631 ± 2% +86.9% 6787 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts 1569 ± 76% +6909.2% 109992 ± 11% interrupts.CPU11.TLB:TLB_shootdowns 3557 ± 35% +2148.0% 79971 ± 54% interrupts.CPU12.CAL:Function_call_interrupts 3760 ± 11% +58.0% 5942 ± 22% interrupts.CPU12.RES:Rescheduling_interrupts 1104 ±119% +7079.3% 79277 ± 56% interrupts.CPU12.TLB:TLB_shootdowns 3639 ± 32% +2257.8% 85817 ± 28% interrupts.CPU13.CAL:Function_call_interrupts 3497 ± 65% +76.1% 6159 ± 26% interrupts.CPU13.NMI:Non-maskable_interrupts 3497 ± 65% +76.1% 6159 ± 26% interrupts.CPU13.PMI:Performance_monitoring_interrupts 3799 ± 3% +46.3% 5558 ± 11% interrupts.CPU13.RES:Rescheduling_interrupts 1329 ± 89% +6373.1% 86075 ± 29% interrupts.CPU13.TLB:TLB_shootdowns 3942 ± 31% +2512.7% 103013 ± 12% interrupts.CPU14.CAL:Function_call_interrupts 3736 ± 4% +76.7% 6601 ± 14% interrupts.CPU14.RES:Rescheduling_interrupts 1491 ± 75% +6811.2% 103097 ± 12% interrupts.CPU14.TLB:TLB_shootdowns 3605 ± 33% +2212.0% 83354 ± 58% interrupts.CPU15.CAL:Function_call_interrupts 3624 ± 62% +85.4% 6720 ± 22% interrupts.CPU15.NMI:Non-maskable_interrupts 3624 ± 62% +85.4% 6720 ± 22% interrupts.CPU15.PMI:Performance_monitoring_interrupts 3751 ± 3% +41.9% 5325 ± 19% interrupts.CPU15.RES:Rescheduling_interrupts 1208 ± 98% +6788.3% 83227 ± 59% interrupts.CPU15.TLB:TLB_shootdowns 3738 ± 28% +2823.1% 109274 ± 29% interrupts.CPU16.CAL:Function_call_interrupts 3584 ± 3% +73.9% 6232 ± 17% interrupts.CPU16.RES:Rescheduling_interrupts 1749 ± 76% +6151.7% 109343 ± 29% interrupts.CPU16.TLB:TLB_shootdowns 3590 ± 36% +2164.9% 81327 ± 44% interrupts.CPU17.CAL:Function_call_interrupts 3515 ± 66% +67.5% 5887 ± 31% interrupts.CPU17.NMI:Non-maskable_interrupts 3515 ± 66% +67.5% 5887 ± 31% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3602 ± 2% +59.5% 5746 ± 14% interrupts.CPU17.RES:Rescheduling_interrupts 1191 ±109% +6686.6% 80845 ± 46% interrupts.CPU17.TLB:TLB_shootdowns 4548 ± 23% +2020.8% 96469 ± 48% interrupts.CPU18.CAL:Function_call_interrupts 3822 ± 8% +52.3% 5822 ± 21% interrupts.CPU18.RES:Rescheduling_interrupts 2195 ± 50% +4291.9% 96414 ± 48% interrupts.CPU18.TLB:TLB_shootdowns 3596 ± 15% +1942.4% 73450 ± 82% interrupts.CPU2.CAL:Function_call_interrupts 5367 ± 6% +33.2% 7150 ± 6% interrupts.CPU2.NMI:Non-maskable_interrupts 5367 ± 6% +33.2% 7150 ± 6% interrupts.CPU2.PMI:Performance_monitoring_interrupts 3608 ± 33% +1494.0% 57512 ± 48% interrupts.CPU20.CAL:Function_call_interrupts 3723 ± 3% +36.6% 5085 ± 11% interrupts.CPU20.RES:Rescheduling_interrupts 1159 ± 97% +4754.8% 56304 ± 50% interrupts.CPU20.TLB:TLB_shootdowns 3686 ± 30% +2621.0% 100315 ± 33% interrupts.CPU21.CAL:Function_call_interrupts 3574 ± 67% +77.6% 6349 ± 25% interrupts.CPU21.NMI:Non-maskable_interrupts 3574 ± 67% +77.6% 6349 ± 25% interrupts.CPU21.PMI:Performance_monitoring_interrupts 3835 ± 13% +65.2% 6337 ± 15% interrupts.CPU21.RES:Rescheduling_interrupts 1162 ± 98% +8509.6% 100065 ± 34% interrupts.CPU21.TLB:TLB_shootdowns 3687 ± 27% +2336.0% 89813 ± 50% interrupts.CPU22.CAL:Function_call_interrupts 3810 +83.5% 6993 ± 27% interrupts.CPU22.RES:Rescheduling_interrupts 1484 ± 66% +5953.7% 89867 ± 51% interrupts.CPU22.TLB:TLB_shootdowns 3974 ± 17% +2750.3% 113270 ± 15% interrupts.CPU23.CAL:Function_call_interrupts 3926 ± 11% +80.7% 7096 ± 9% interrupts.CPU23.RES:Rescheduling_interrupts 1464 ± 53% +7676.5% 113848 ± 15% interrupts.CPU23.TLB:TLB_shootdowns 4345 ± 14% +2287.9% 103770 ± 13% interrupts.CPU24.CAL:Function_call_interrupts 4072 ± 9% +84.4% 7507 ± 13% interrupts.CPU24.RES:Rescheduling_interrupts 1858 ± 38% +5495.7% 103967 ± 14% interrupts.CPU24.TLB:TLB_shootdowns 3807 ± 26% +2463.3% 97585 ± 30% interrupts.CPU25.CAL:Function_call_interrupts 3852 ± 4% +78.3% 6870 ± 14% interrupts.CPU25.RES:Rescheduling_interrupts 1368 ± 72% +7011.0% 97296 ± 31% interrupts.CPU25.TLB:TLB_shootdowns 3574 ± 21% +1729.3% 65390 ± 48% interrupts.CPU26.CAL:Function_call_interrupts 6245 ± 7% -30.6% 4334 ± 26% interrupts.CPU26.NMI:Non-maskable_interrupts 6245 ± 7% -30.6% 4334 ± 26% interrupts.CPU26.PMI:Performance_monitoring_interrupts 4092 ± 18% +37.4% 5623 ± 14% interrupts.CPU26.RES:Rescheduling_interrupts 1168 ± 64% +5429.2% 64594 ± 50% interrupts.CPU26.TLB:TLB_shootdowns 6377 ± 10% -34.5% 4179 ± 23% interrupts.CPU27.NMI:Non-maskable_interrupts 6377 ± 10% -34.5% 4179 ± 23% interrupts.CPU27.PMI:Performance_monitoring_interrupts 6425 ± 14% -38.7% 3937 ± 23% interrupts.CPU28.NMI:Non-maskable_interrupts 6425 ± 14% -38.7% 3937 ± 23% interrupts.CPU28.PMI:Performance_monitoring_interrupts 4880 ± 4% +894.4% 48528 ± 19% interrupts.CPU29.CAL:Function_call_interrupts 3951 ± 6% +25.6% 4962 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts 2365 ± 13% +1898.5% 47274 ± 20% interrupts.CPU29.TLB:TLB_shootdowns 3890 ± 13% +3046.4% 122404 ± 11% interrupts.CPU3.CAL:Function_call_interrupts 4834 ± 22% +53.2% 7407 ± 4% interrupts.CPU3.NMI:Non-maskable_interrupts 4834 ± 22% +53.2% 7407 ± 4% interrupts.CPU3.PMI:Performance_monitoring_interrupts 4600 ± 3% +89.1% 8699 ± 25% interrupts.CPU3.RES:Rescheduling_interrupts 1585 ± 50% +7626.2% 122460 ± 12% interrupts.CPU3.TLB:TLB_shootdowns 4598 ± 15% +1179.7% 58846 ± 45% interrupts.CPU31.CAL:Function_call_interrupts 3672 ± 3% +61.8% 5941 ± 14% interrupts.CPU31.RES:Rescheduling_interrupts 2274 ± 27% +2438.3% 57732 ± 47% interrupts.CPU31.TLB:TLB_shootdowns 4554 ± 21% +1114.9% 55329 ± 56% interrupts.CPU32.CAL:Function_call_interrupts 3741 +44.6% 5408 ± 21% interrupts.CPU32.RES:Rescheduling_interrupts 2105 ± 41% +2460.0% 53894 ± 59% interrupts.CPU32.TLB:TLB_shootdowns 3907 ± 30% +860.8% 37539 ± 65% interrupts.CPU35.CAL:Function_call_interrupts 3871 ± 9% +22.8% 4752 ± 18% interrupts.CPU35.RES:Rescheduling_interrupts 1638 ± 73% +2093.4% 35938 ± 68% interrupts.CPU35.TLB:TLB_shootdowns 4647 ± 23% +1627.4% 80281 ± 12% interrupts.CPU36.CAL:Function_call_interrupts 3821 ± 5% +57.4% 6015 ± 9% interrupts.CPU36.RES:Rescheduling_interrupts 2181 ± 45% +3557.4% 79776 ± 12% interrupts.CPU36.TLB:TLB_shootdowns 7070 ± 14% -30.3% 4924 ± 24% interrupts.CPU38.NMI:Non-maskable_interrupts 7070 ± 14% -30.3% 4924 ± 24% interrupts.CPU38.PMI:Performance_monitoring_interrupts 3711 ± 12% +1962.2% 76543 ± 53% interrupts.CPU4.CAL:Function_call_interrupts 4987 ± 21% +43.4% 7152 ± 3% interrupts.CPU4.NMI:Non-maskable_interrupts 4987 ± 21% +43.4% 7152 ± 3% interrupts.CPU4.PMI:Performance_monitoring_interrupts 3815 ± 2% +60.4% 6121 ± 20% interrupts.CPU4.RES:Rescheduling_interrupts 1324 ± 33% +5629.5% 75872 ± 55% interrupts.CPU4.TLB:TLB_shootdowns 4648 ± 22% +814.8% 42517 ± 39% interrupts.CPU43.CAL:Function_call_interrupts 2282 ± 35% +1704.8% 41186 ± 41% interrupts.CPU43.TLB:TLB_shootdowns 3602 ± 29% +2533.9% 94873 ± 42% interrupts.CPU45.CAL:Function_call_interrupts 4435 ± 44% +62.7% 7217 ± 10% interrupts.CPU45.NMI:Non-maskable_interrupts 4435 ± 44% +62.7% 7217 ± 10% interrupts.CPU45.PMI:Performance_monitoring_interrupts 3712 ± 4% +57.8% 5856 ± 21% interrupts.CPU45.RES:Rescheduling_interrupts 1180 ± 98% +7913.7% 94601 ± 44% interrupts.CPU45.TLB:TLB_shootdowns 4762 ± 26% +49.6% 7124 ± 12% interrupts.CPU46.NMI:Non-maskable_interrupts 4762 ± 26% +49.6% 7124 ± 12% interrupts.CPU46.PMI:Performance_monitoring_interrupts 1165 ±115% +5808.7% 68836 ± 84% interrupts.CPU47.TLB:TLB_shootdowns 3486 ± 30% +2453.1% 89006 ± 60% interrupts.CPU48.CAL:Function_call_interrupts 4565 ± 46% +55.6% 7104 ± 14% interrupts.CPU48.NMI:Non-maskable_interrupts 4565 ± 46% +55.6% 7104 ± 14% interrupts.CPU48.PMI:Performance_monitoring_interrupts 3695 ± 2% +70.3% 6295 ± 20% interrupts.CPU48.RES:Rescheduling_interrupts 1073 ±111% +8148.9% 88511 ± 62% interrupts.CPU48.TLB:TLB_shootdowns 4106 ± 47% +69.1% 6944 ± 14% interrupts.CPU49.NMI:Non-maskable_interrupts 4106 ± 47% +69.1% 6944 ± 14% interrupts.CPU49.PMI:Performance_monitoring_interrupts 1185 ± 95% +5391.3% 65112 ± 98% interrupts.CPU49.TLB:TLB_shootdowns 3601 ± 8% +2568.7% 96098 ± 29% interrupts.CPU5.CAL:Function_call_interrupts 3785 ± 26% +91.1% 7233 ± 3% interrupts.CPU5.NMI:Non-maskable_interrupts 3785 ± 26% +91.1% 7233 ± 3% interrupts.CPU5.PMI:Performance_monitoring_interrupts 3668 ± 2% +74.1% 6386 ± 16% interrupts.CPU5.RES:Rescheduling_interrupts 1137 ± 29% +8322.2% 95802 ± 30% interrupts.CPU5.TLB:TLB_shootdowns 3745 ± 41% +84.0% 6890 ± 18% interrupts.CPU50.NMI:Non-maskable_interrupts 3745 ± 41% +84.0% 6890 ± 18% interrupts.CPU50.PMI:Performance_monitoring_interrupts 654.50 ± 68% +12823.4% 84583 ± 66% interrupts.CPU50.TLB:TLB_shootdowns 3754 ± 59% +84.3% 6921 ± 18% interrupts.CPU51.NMI:Non-maskable_interrupts 3754 ± 59% +84.3% 6921 ± 18% interrupts.CPU51.PMI:Performance_monitoring_interrupts 3616 ± 4% +63.4% 5907 ± 23% interrupts.CPU51.RES:Rescheduling_interrupts 1051 ±115% +7965.0% 84783 ± 67% interrupts.CPU51.TLB:TLB_shootdowns 3537 ± 35% +1280.8% 48845 ± 62% interrupts.CPU52.CAL:Function_call_interrupts 1067 ±112% +4353.9% 47545 ± 65% interrupts.CPU52.TLB:TLB_shootdowns 1112 ±109% +4960.5% 56285 ± 97% interrupts.CPU53.TLB:TLB_shootdowns 3841 ± 56% +80.5% 6932 ± 20% interrupts.CPU54.NMI:Non-maskable_interrupts 3841 ± 56% +80.5% 6932 ± 20% interrupts.CPU54.PMI:Performance_monitoring_interrupts 3561 +69.5% 6036 ± 23% interrupts.CPU54.RES:Rescheduling_interrupts 3543 ± 32% +1357.6% 51648 ± 60% interrupts.CPU55.CAL:Function_call_interrupts 3537 ± 2% +39.5% 4933 ± 13% interrupts.CPU55.RES:Rescheduling_interrupts 1144 ±103% +4304.4% 50397 ± 63% interrupts.CPU55.TLB:TLB_shootdowns 3550 ± 35% +2444.4% 90343 ± 56% interrupts.CPU56.CAL:Function_call_interrupts 3650 ± 65% +94.0% 7082 ± 11% interrupts.CPU56.NMI:Non-maskable_interrupts 3650 ± 65% +94.0% 7082 ± 11% interrupts.CPU56.PMI:Performance_monitoring_interrupts 3651 +65.0% 6023 ± 23% interrupts.CPU56.RES:Rescheduling_interrupts 1136 ±112% +7824.5% 90062 ± 58% interrupts.CPU56.TLB:TLB_shootdowns 2575 ± 33% +170.5% 6966 ± 17% interrupts.CPU57.NMI:Non-maskable_interrupts 2575 ± 33% +170.5% 6966 ± 17% interrupts.CPU57.PMI:Performance_monitoring_interrupts 3533 ± 3% +63.2% 5768 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts 1119 ±104% +7722.0% 87547 ± 60% interrupts.CPU57.TLB:TLB_shootdowns 3626 +52.3% 5525 ± 18% interrupts.CPU58.RES:Rescheduling_interrupts 3579 ± 35% +2484.6% 92516 ± 52% interrupts.CPU59.CAL:Function_call_interrupts 2676 ± 32% +152.7% 6762 ± 23% interrupts.CPU59.NMI:Non-maskable_interrupts 2676 ± 32% +152.7% 6762 ± 23% interrupts.CPU59.PMI:Performance_monitoring_interrupts 3615 +63.4% 5907 ± 22% interrupts.CPU59.RES:Rescheduling_interrupts 1144 ±104% +7945.0% 92075 ± 53% interrupts.CPU59.TLB:TLB_shootdowns 3821 ± 15% +1528.2% 62212 ± 38% interrupts.CPU6.CAL:Function_call_interrupts 3942 ± 39% +71.2% 6748 ± 9% interrupts.CPU6.NMI:Non-maskable_interrupts 3942 ± 39% +71.2% 6748 ± 9% interrupts.CPU6.PMI:Performance_monitoring_interrupts 3903 ± 4% +47.5% 5759 ± 22% interrupts.CPU6.RES:Rescheduling_interrupts 1496 ± 36% +3987.9% 61185 ± 39% interrupts.CPU6.TLB:TLB_shootdowns 2798 ± 22% +139.4% 6699 ± 21% interrupts.CPU60.NMI:Non-maskable_interrupts 2798 ± 22% +139.4% 6699 ± 21% interrupts.CPU60.PMI:Performance_monitoring_interrupts 2957 ± 17% +123.3% 6604 ± 22% interrupts.CPU61.NMI:Non-maskable_interrupts 2957 ± 17% +123.3% 6604 ± 22% interrupts.CPU61.PMI:Performance_monitoring_interrupts 3680 ± 37% +2380.7% 91308 ± 40% interrupts.CPU62.CAL:Function_call_interrupts 4022 ± 14% +37.9% 5547 ± 11% interrupts.CPU62.RES:Rescheduling_interrupts 1136 ±124% +7892.6% 90855 ± 42% interrupts.CPU62.TLB:TLB_shootdowns 3634 ± 36% +2919.3% 109735 ± 24% interrupts.CPU63.CAL:Function_call_interrupts 3521 ± 26% +114.0% 7538 ± 2% interrupts.CPU63.NMI:Non-maskable_interrupts 3521 ± 26% +114.0% 7538 ± 2% interrupts.CPU63.PMI:Performance_monitoring_interrupts 3948 ± 10% +65.9% 6551 ± 18% interrupts.CPU63.RES:Rescheduling_interrupts 1152 ±110% +9401.5% 109528 ± 24% interrupts.CPU63.TLB:TLB_shootdowns 3592 ± 34% +3034.8% 112602 ± 26% interrupts.CPU64.CAL:Function_call_interrupts 2961 ± 17% +148.8% 7369 ± 9% interrupts.CPU64.NMI:Non-maskable_interrupts 2961 ± 17% +148.8% 7369 ± 9% interrupts.CPU64.PMI:Performance_monitoring_interrupts 4193 ± 19% +66.8% 6996 ± 10% interrupts.CPU64.RES:Rescheduling_interrupts 1104 ±106% +10101.5% 112650 ± 27% interrupts.CPU64.TLB:TLB_shootdowns 3509 ± 33% +2502.5% 91320 ± 30% interrupts.CPU65.CAL:Function_call_interrupts 2965 ± 17% +145.6% 7283 ± 6% interrupts.CPU65.NMI:Non-maskable_interrupts 2965 ± 17% +145.6% 7283 ± 6% interrupts.CPU65.PMI:Performance_monitoring_interrupts 3859 ± 8% +52.9% 5899 ± 13% interrupts.CPU65.RES:Rescheduling_interrupts 1037 ±112% +8669.9% 91009 ± 31% interrupts.CPU65.TLB:TLB_shootdowns 3370 ± 10% +2036.3% 71994 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 3469 ± 44% +72.9% 5999 ± 24% interrupts.CPU7.NMI:Non-maskable_interrupts 3469 ± 44% +72.9% 5999 ± 24% interrupts.CPU7.PMI:Performance_monitoring_interrupts 3809 ± 4% +47.7% 5625 ± 11% interrupts.CPU7.RES:Rescheduling_interrupts 988.50 ± 61% +7115.2% 71322 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 3654 ± 4% +80.9% 6612 ± 24% interrupts.CPU77.RES:Rescheduling_interrupts 3569 ± 25% +2737.1% 101271 ± 22% interrupts.CPU8.CAL:Function_call_interrupts 3412 ± 47% +81.7% 6202 ± 26% interrupts.CPU8.NMI:Non-maskable_interrupts 3412 ± 47% +81.7% 6202 ± 26% interrupts.CPU8.PMI:Performance_monitoring_interrupts 3822 ± 6% +72.8% 6605 ± 13% interrupts.CPU8.RES:Rescheduling_interrupts 2371 ± 57% +4169.2% 101244 ± 23% interrupts.CPU8.TLB:TLB_shootdowns 3280 ± 25% +2403.5% 82113 ± 56% interrupts.CPU9.CAL:Function_call_interrupts 3406 ± 50% +101.8% 6873 ± 11% interrupts.CPU9.NMI:Non-maskable_interrupts 3406 ± 50% +101.8% 6873 ± 11% interrupts.CPU9.PMI:Performance_monitoring_interrupts 3636 ± 6% +66.1% 6041 ± 24% interrupts.CPU9.RES:Rescheduling_interrupts 898.75 ±100% +8966.6% 81486 ± 58% interrupts.CPU9.TLB:TLB_shootdowns 441700 ± 6% +18.5% 523360 ± 5% interrupts.NMI:Non-maskable_interrupts 441700 ± 6% +18.5% 523360 ± 5% interrupts.PMI:Performance_monitoring_interrupts 344450 ± 2% +42.8% 491951 ± 2% interrupts.RES:Rescheduling_interrupts 153414 ± 10% +3677.4% 5795118 ± 6% interrupts.TLB:TLB_shootdowns stress-ng.numa.ops 3500 +-+------------------------------------------------------------------+ | O O | 3000 +-+ O O O OO O O | O O O O O O O O OO | 2500 +-+ O O | | O O O O OO O OO | 2000 +-+ O O | | | 1500 +-+ | | | 1000 +-+ | | | 500 +-+ | | | 0 +-+------------------------------------------------------------------+ stress-ng.numa.ops_per_sec 700 +-+-------------------------------------------------------------------+ | O | 600 +-+ O O O OO O O O | O O O O O O O O O O | 500 +-+ O | | O OO OO O O OO O | 400 +-+ O O | | | 300 +-+ | | | 200 +-+ | | | 100 +-+ | | | 0 +-+-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** vm-snb-8G: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G ========================================================================================= compiler/kconfig/rootfs/tbox_group/test/testcase: gcc-7/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/vm-snb-8G/cve/ltp commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :12 8% 1:12 dmesg.BUG:soft_lockup-CPU##stuck_for#s :12 83% 10:12 dmesg.Kernel_panic-not_syncing:System_is_deadlocked_on_memory :12 8% 1:12 dmesg.Kernel_panic-not_syncing:softlockup:hung_tasks :12 83% 10:12 dmesg.Out_of_memory:Kill_process :12 83% 10:12 dmesg.Out_of_memory_and_no_killable_processes :12 8% 1:12 dmesg.RIP:free_reserved_area %stddev %change %stddev \ | \ 1.00 -100.0% 0.00 ltp.cve-2017-18075.pass 783.53 ± 40% +165.9% 2083 ltp.time.elapsed_time 783.53 ± 40% +165.9% 2083 ltp.time.elapsed_time.max 1171525 ± 21% -69.1% 362268 ltp.time.involuntary_context_switches 11374889 ± 27% -89.7% 1171686 ltp.time.minor_page_faults 103.08 ± 9% -46.6% 55.00 ltp.time.percent_of_cpu_this_job_got 199802 ± 12% -36.0% 127912 ltp.time.voluntary_context_switches 557835 ± 3% +469.2% 3175044 meminfo.Active 557602 ± 3% +469.4% 3174826 meminfo.Active(anon) 204797 -51.1% 100217 meminfo.CmaFree 709192 ± 3% -11.8% 625168 meminfo.Inactive 22617 ± 16% -50.8% 11135 meminfo.Inactive(anon) 4772734 -52.9% 2247863 meminfo.MemAvailable 4273603 -57.4% 1820264 meminfo.MemFree 3888307 +63.1% 6341643 meminfo.Memused *************************************************************************************************** lkp-skl-4sp1: 192 threads Skylake-4S with 704G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-skl-4sp1/futex1/will-it-scale commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.42 ± 41% -37.6% 0.27 ± 38% turbostat.CPU%c1 436529 +18.1% 515400 ± 2% meminfo.Active 432417 +18.2% 511272 ± 2% meminfo.Active(anon) 1.01 ±100% +1140.2% 12.50 ±141% irq_exception_noise.__do_page_fault.50th 1.10 ±100% +1059.7% 12.70 ±137% irq_exception_noise.__do_page_fault.60th 1.18 ±100% +1002.0% 13.00 ±133% irq_exception_noise.__do_page_fault.70th 1.32 ±100% +934.8% 13.71 ±123% irq_exception_noise.__do_page_fault.80th 545804 ± 35% -56.4% 238144 ± 29% numa-numastat.node0.local_node 568271 ± 32% -54.7% 257664 ± 24% numa-numastat.node0.numa_hit 232178 ± 44% +135.2% 546083 ± 30% numa-numastat.node2.local_node 268049 ± 38% +114.9% 576076 ± 27% numa-numastat.node2.numa_hit 330370 ± 19% -39.1% 201131 ± 50% numa-numastat.node3.local_node 342159 ± 15% -31.7% 233865 ± 42% numa-numastat.node3.numa_hit 108167 +18.2% 127807 ± 2% proc-vmstat.nr_active_anon 6541 -1.8% 6425 proc-vmstat.nr_inactive_anon 8474 -2.4% 8271 ± 2% proc-vmstat.nr_mapped 108167 +18.2% 127807 ± 2% proc-vmstat.nr_zone_active_anon 6541 -1.8% 6425 proc-vmstat.nr_zone_inactive_anon 29259 +3.5% 30283 proc-vmstat.pgactivate 1477807 -2.1% 1446297 proc-vmstat.pgfree 13.29 ± 5% -2.6 10.72 ± 4% perf-stat.i.cache-miss-rate% 1107530 ± 4% -16.6% 923800 ± 3% perf-stat.i.cache-misses 57.31 -2.0% 56.14 perf-stat.i.cpu-migrations 429491 ± 3% +14.9% 493355 ± 4% perf-stat.i.cycles-between-cache-misses 51.31 +10.9 62.22 ± 6% perf-stat.i.iTLB-load-miss-rate% 3.369e+08 +2.2% 3.443e+08 perf-stat.i.iTLB-load-misses 3.2e+08 -32.8% 2.149e+08 ± 16% perf-stat.i.iTLB-loads 863.04 -2.4% 842.12 perf-stat.i.instructions-per-iTLB-miss 355.57 ± 6% -13.9% 306.21 ± 5% sched_debug.cfs_rq:/.exec_clock.stddev 0.46 +163.6% 1.21 ± 10% sched_debug.cfs_rq:/.nr_spread_over.avg 619.50 ± 6% -13.9% 533.25 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.max 128891 +42.6% 183838 ± 30% sched_debug.cpu.avg_idle.stddev 11367 ± 2% +14.6% 13028 sched_debug.cpu.curr->pid.max 2547 ± 15% +30.3% 3321 sched_debug.cpu.curr->pid.min -11.50 -23.9% -8.75 sched_debug.cpu.nr_uninterruptible.min 10647 ± 20% -40.5% 6331 ± 17% sched_debug.cpu.sched_goidle.max 800.32 ± 19% -31.8% 545.75 ± 5% sched_debug.cpu.sched_goidle.stddev 20627 +16.3% 23980 ± 6% softirqs.CPU0.RCU 112466 ± 4% -8.4% 102965 ± 4% softirqs.CPU0.TIMER 20826 +10.9% 23088 ± 6% softirqs.CPU110.RCU 20775 ± 8% +22.1% 25361 ± 5% softirqs.CPU112.RCU 20835 ± 10% +22.8% 25592 ± 5% softirqs.CPU113.RCU 21208 ± 6% +21.7% 25811 ± 4% softirqs.CPU114.RCU 20691 ± 7% +24.5% 25764 ± 6% softirqs.CPU115.RCU 20809 ± 8% +22.8% 25550 ± 5% softirqs.CPU116.RCU 20870 ± 9% +23.4% 25754 ± 4% softirqs.CPU117.RCU 20714 ± 8% +23.4% 25568 ± 5% softirqs.CPU118.RCU 20652 ± 8% +22.6% 25310 ± 5% softirqs.CPU119.RCU 20909 +10.8% 23171 ± 7% softirqs.CPU13.RCU 20488 +10.1% 22548 ± 5% softirqs.CPU14.RCU 20955 ± 6% +21.3% 25421 ± 6% softirqs.CPU16.RCU 19946 +15.2% 22982 ± 11% softirqs.CPU173.RCU 133691 ± 19% -22.1% 104181 ± 4% softirqs.CPU176.TIMER 21426 ± 3% +17.6% 25194 ± 4% softirqs.CPU18.RCU 20370 ± 6% +24.1% 25275 ± 3% softirqs.CPU19.RCU 21029 ± 4% +19.6% 25146 ± 5% softirqs.CPU20.RCU 20971 ± 8% +19.7% 25095 ± 4% softirqs.CPU21.RCU 20759 ± 6% +20.2% 24958 ± 5% softirqs.CPU22.RCU 20627 ± 6% +21.6% 25077 ± 5% softirqs.CPU23.RCU 20693 +10.5% 22867 ± 6% softirqs.CPU5.RCU 18951 ± 4% +21.1% 22953 ± 12% softirqs.CPU75.RCU 110575 ± 4% -7.5% 102329 ± 2% softirqs.CPU98.TIMER 57149 +37.8% 78776 ± 10% numa-vmstat.node0.nr_file_pages 1470 +84.0% 2706 ± 37% numa-vmstat.node0.nr_mapped 1020 ± 74% +1687.7% 18243 ± 45% numa-vmstat.node0.nr_shmem 16.50 ± 33% +8503.0% 1419 ±122% numa-vmstat.node1.nr_inactive_anon 6205 +19.9% 7437 ± 10% numa-vmstat.node1.nr_kernel_stack 594.50 ± 3% +141.9% 1438 ± 52% numa-vmstat.node1.nr_page_table_pages 56.00 ± 33% +2705.4% 1571 ±112% numa-vmstat.node1.nr_shmem 3072 ± 5% +119.8% 6751 ± 17% numa-vmstat.node1.nr_slab_reclaimable 11562 ± 2% +31.6% 15218 ± 9% numa-vmstat.node1.nr_slab_unreclaimable 16.50 ± 33% +8503.0% 1419 ±122% numa-vmstat.node1.nr_zone_inactive_anon 266341 ± 12% +131.7% 617001 ± 9% numa-vmstat.node1.numa_hit 140430 ± 23% +258.4% 503291 ± 12% numa-vmstat.node1.numa_local 1028 -100.0% 0.00 numa-vmstat.node2.nr_active_file 71145 -19.0% 57616 ± 2% numa-vmstat.node2.nr_file_pages 4668 ± 12% -67.9% 1500 ±104% numa-vmstat.node2.nr_inactive_anon 544.50 -100.0% 0.00 numa-vmstat.node2.nr_inactive_file 9199 ± 4% -28.3% 6593 ± 5% numa-vmstat.node2.nr_kernel_stack 3552 -50.5% 1758 ± 33% numa-vmstat.node2.nr_mapped 2955 ± 6% -65.5% 1018 ± 38% numa-vmstat.node2.nr_page_table_pages 4996 ± 11% -65.4% 1727 ± 95% numa-vmstat.node2.nr_shmem 64607 -13.5% 55888 numa-vmstat.node2.nr_unevictable 1028 -100.0% 0.00 numa-vmstat.node2.nr_zone_active_file 4668 ± 12% -67.9% 1500 ±104% numa-vmstat.node2.nr_zone_inactive_anon 544.50 -100.0% 0.00 numa-vmstat.node2.nr_zone_inactive_file 64607 -13.5% 55888 numa-vmstat.node2.nr_zone_unevictable 31367 ± 22% -70.8% 9145 ±104% numa-vmstat.node3.nr_active_anon 11913 ± 59% -87.6% 1476 ± 52% numa-vmstat.node3.nr_anon_pages 1000 ± 17% -72.0% 279.75 ±150% numa-vmstat.node3.nr_inactive_anon 698.00 ± 11% -15.7% 588.25 numa-vmstat.node3.nr_page_table_pages 6430 ± 14% -39.1% 3918 ± 20% numa-vmstat.node3.nr_slab_reclaimable 31366 ± 22% -70.8% 9145 ±104% numa-vmstat.node3.nr_zone_active_anon 1000 ± 17% -72.0% 279.75 ±150% numa-vmstat.node3.nr_zone_inactive_anon 345814 ± 42% -45.4% 188830 ± 51% numa-vmstat.node3.numa_local 228598 +37.8% 315090 ± 10% numa-meminfo.node0.FilePages 3152 ± 89% +354.3% 14320 ± 50% numa-meminfo.node0.Inactive 5883 +80.6% 10625 ± 34% numa-meminfo.node0.Mapped 4082 ± 74% +1687.1% 72956 ± 45% numa-meminfo.node0.Shmem 68.50 ± 31% +8993.8% 6229 ±107% numa-meminfo.node1.Inactive 68.50 ± 31% +8191.2% 5679 ±122% numa-meminfo.node1.Inactive(anon) 12286 ± 5% +119.8% 27005 ± 17% numa-meminfo.node1.KReclaimable 6207 +19.9% 7441 ± 10% numa-meminfo.node1.KernelStack 2372 ± 3% +142.7% 5757 ± 52% numa-meminfo.node1.PageTables 12286 ± 5% +119.8% 27005 ± 17% numa-meminfo.node1.SReclaimable 46242 ± 2% +31.6% 60873 ± 9% numa-meminfo.node1.SUnreclaim 223.50 ± 33% +2711.9% 6284 ±112% numa-meminfo.node1.Shmem 58528 +50.1% 87879 ± 11% numa-meminfo.node1.Slab 4112 -100.0% 0.00 numa-meminfo.node2.Active(file) 284580 -19.0% 230466 ± 2% numa-meminfo.node2.FilePages 20854 ± 11% -71.2% 6004 ±104% numa-meminfo.node2.Inactive 18673 ± 12% -67.8% 6004 ±104% numa-meminfo.node2.Inactive(anon) 2180 -100.0% 0.00 numa-meminfo.node2.Inactive(file) 9203 ± 4% -28.3% 6594 ± 5% numa-meminfo.node2.KernelStack 13712 -48.6% 7054 ± 33% numa-meminfo.node2.Mapped 883222 ± 6% -16.2% 740514 ± 13% numa-meminfo.node2.MemUsed 11829 ± 6% -65.8% 4040 ± 38% numa-meminfo.node2.PageTables 19985 ± 11% -65.4% 6911 ± 95% numa-meminfo.node2.Shmem 258431 -13.5% 223554 numa-meminfo.node2.Unevictable 125256 ± 22% -70.8% 36547 ±104% numa-meminfo.node3.Active 125256 ± 22% -70.8% 36547 ±104% numa-meminfo.node3.Active(anon) 24571 ± 83% -96.0% 974.75 ±173% numa-meminfo.node3.AnonHugePages 47677 ± 59% -87.7% 5876 ± 52% numa-meminfo.node3.AnonPages 3993 ± 18% -72.1% 1115 ±149% numa-meminfo.node3.Inactive 3991 ± 18% -72.1% 1115 ±149% numa-meminfo.node3.Inactive(anon) 25714 ± 14% -39.1% 15671 ± 20% numa-meminfo.node3.KReclaimable 696078 ± 2% -14.9% 592511 ± 8% numa-meminfo.node3.MemUsed 2772 ± 11% -15.9% 2332 numa-meminfo.node3.PageTables 25714 ± 14% -39.1% 15671 ± 20% numa-meminfo.node3.SReclaimable 78860 ± 8% -16.7% 65670 ± 6% numa-meminfo.node3.Slab 1702 ± 25% -44.9% 937.50 ± 17% interrupts.CPU0.RES:Rescheduling_interrupts 6102 ± 5% -3.6% 5884 ± 5% interrupts.CPU100.CAL:Function_call_interrupts 6091 ± 5% -4.1% 5842 ± 5% interrupts.CPU101.CAL:Function_call_interrupts 6100 ± 5% -4.3% 5841 ± 6% interrupts.CPU102.CAL:Function_call_interrupts 6099 ± 5% -3.6% 5882 ± 5% interrupts.CPU103.CAL:Function_call_interrupts 6099 ± 5% -3.5% 5883 ± 5% interrupts.CPU104.CAL:Function_call_interrupts 6099 ± 5% -3.6% 5882 ± 5% interrupts.CPU106.CAL:Function_call_interrupts 6119 ± 5% -3.8% 5886 ± 5% interrupts.CPU107.CAL:Function_call_interrupts 6115 ± 5% -8.8% 5575 ± 4% interrupts.CPU108.CAL:Function_call_interrupts 6115 ± 5% -3.6% 5894 ± 4% interrupts.CPU109.CAL:Function_call_interrupts 6118 ± 5% -3.9% 5879 ± 5% interrupts.CPU110.CAL:Function_call_interrupts 6117 ± 5% -3.8% 5884 ± 4% interrupts.CPU111.CAL:Function_call_interrupts 291.50 ± 86% -90.1% 29.00 ± 33% interrupts.CPU111.RES:Rescheduling_interrupts 6117 ± 5% -3.6% 5897 ± 5% interrupts.CPU112.CAL:Function_call_interrupts 6117 ± 5% -3.5% 5902 ± 5% interrupts.CPU113.CAL:Function_call_interrupts 6116 ± 5% -3.5% 5904 ± 5% interrupts.CPU114.CAL:Function_call_interrupts 6116 ± 5% -3.6% 5897 ± 5% interrupts.CPU115.CAL:Function_call_interrupts 6117 ± 5% -3.6% 5897 ± 5% interrupts.CPU116.CAL:Function_call_interrupts 6118 ± 5% -3.6% 5896 ± 5% interrupts.CPU117.CAL:Function_call_interrupts 6088 ± 5% -5.7% 5744 interrupts.CPU12.CAL:Function_call_interrupts 1079 ± 78% -77.3% 245.50 ±134% interrupts.CPU120.RES:Rescheduling_interrupts 949.50 ± 91% -93.4% 63.00 ± 35% interrupts.CPU122.RES:Rescheduling_interrupts 6099 ± 5% -3.6% 5879 ± 4% interrupts.CPU13.CAL:Function_call_interrupts 2647 ± 18% -82.0% 477.00 ± 33% interrupts.CPU13.RES:Rescheduling_interrupts 208.50 ± 51% -51.3% 101.50 ± 36% interrupts.CPU14.RES:Rescheduling_interrupts 53.00 ± 50% +502.4% 319.25 ± 68% interrupts.CPU140.RES:Rescheduling_interrupts 20.00 ± 55% +392.5% 98.50 ± 35% interrupts.CPU141.RES:Rescheduling_interrupts 6093 ± 5% -4.2% 5838 ± 5% interrupts.CPU144.CAL:Function_call_interrupts 206.00 ± 59% -63.7% 74.75 ± 75% interrupts.CPU145.RES:Rescheduling_interrupts 6100 ± 5% -3.9% 5865 ± 5% interrupts.CPU15.CAL:Function_call_interrupts 8.00 +575.0% 54.00 ± 60% interrupts.CPU152.RES:Rescheduling_interrupts 969.00 ± 67% -91.8% 79.25 ± 56% interrupts.CPU153.RES:Rescheduling_interrupts 922.00 ± 91% -74.7% 233.50 ±134% interrupts.CPU159.RES:Rescheduling_interrupts 6098 ± 5% -3.9% 5863 ± 5% interrupts.CPU16.CAL:Function_call_interrupts 6097 ± 5% -4.2% 5841 ± 4% interrupts.CPU17.CAL:Function_call_interrupts 1077 ± 89% -94.5% 58.75 ± 78% interrupts.CPU176.RES:Rescheduling_interrupts 710144 ± 12% -13.5% 614236 interrupts.CPU182.LOC:Local_timer_interrupts 303.50 ± 78% -92.6% 22.50 ± 50% interrupts.CPU186.RES:Rescheduling_interrupts 6096 ± 5% -6.4% 5704 interrupts.CPU19.CAL:Function_call_interrupts 271.00 ± 71% -86.6% 36.25 ± 81% interrupts.CPU190.RES:Rescheduling_interrupts 6109 ± 5% -3.8% 5875 ± 5% interrupts.CPU26.CAL:Function_call_interrupts 6114 ± 5% -4.1% 5864 ± 5% interrupts.CPU27.CAL:Function_call_interrupts 6112 ± 5% -4.1% 5864 ± 5% interrupts.CPU28.CAL:Function_call_interrupts 6085 ± 6% -3.7% 5859 ± 5% interrupts.CPU29.CAL:Function_call_interrupts 780665 ± 20% -21.3% 614413 interrupts.CPU3.LOC:Local_timer_interrupts 1969 ± 90% -88.1% 235.00 ±106% interrupts.CPU3.RES:Rescheduling_interrupts 6085 ± 6% -3.8% 5855 ± 5% interrupts.CPU30.CAL:Function_call_interrupts 6079 ± 6% -3.7% 5853 ± 5% interrupts.CPU31.CAL:Function_call_interrupts 6080 ± 6% -3.8% 5847 ± 5% interrupts.CPU32.CAL:Function_call_interrupts 6077 ± 6% -3.7% 5850 ± 5% interrupts.CPU33.CAL:Function_call_interrupts 806.00 ± 11% -73.2% 216.00 ± 74% interrupts.CPU35.RES:Rescheduling_interrupts 1731 ± 3% -83.6% 283.75 ± 58% interrupts.CPU37.RES:Rescheduling_interrupts 45.50 ± 45% +1009.3% 504.75 ± 86% interrupts.CPU40.RES:Rescheduling_interrupts 41.50 ± 20% +222.3% 133.75 ± 47% interrupts.CPU44.RES:Rescheduling_interrupts 86.50 ± 26% +414.7% 445.25 ± 45% interrupts.CPU49.RES:Rescheduling_interrupts 129.50 ± 30% +1750.4% 2396 ± 94% interrupts.CPU53.RES:Rescheduling_interrupts 239.50 ± 51% +291.6% 938.00 ± 34% interrupts.CPU56.RES:Rescheduling_interrupts 6153 ± 5% -4.4% 5881 ± 5% interrupts.CPU58.CAL:Function_call_interrupts 87.00 ± 8% +1240.2% 1166 ± 58% interrupts.CPU63.RES:Rescheduling_interrupts 244.50 ± 29% -51.7% 118.00 ± 30% interrupts.CPU69.RES:Rescheduling_interrupts 1242 ± 64% -82.3% 220.00 ± 33% interrupts.CPU75.RES:Rescheduling_interrupts 173.50 ± 18% -43.4% 98.25 ± 10% interrupts.CPU76.RES:Rescheduling_interrupts 2162 ± 60% -76.4% 510.50 ±123% interrupts.CPU77.RES:Rescheduling_interrupts 74.00 ± 5% +285.5% 285.25 ± 45% interrupts.CPU8.RES:Rescheduling_interrupts 633.00 ± 34% -56.6% 274.50 ± 97% interrupts.CPU82.RES:Rescheduling_interrupts 6229 ± 7% -5.3% 5898 ± 5% interrupts.CPU84.CAL:Function_call_interrupts 6123 ± 5% -3.6% 5900 ± 5% interrupts.CPU86.CAL:Function_call_interrupts 943284 ± 34% -34.8% 614662 interrupts.CPU86.LOC:Local_timer_interrupts 6122 ± 5% -3.6% 5899 ± 5% interrupts.CPU87.CAL:Function_call_interrupts 6120 ± 5% -3.6% 5898 ± 5% interrupts.CPU88.CAL:Function_call_interrupts 6118 ± 5% -3.5% 5902 ± 5% interrupts.CPU89.CAL:Function_call_interrupts 1286 ± 83% -66.7% 428.50 ±118% interrupts.CPU9.RES:Rescheduling_interrupts 6114 ± 5% -3.5% 5900 ± 5% interrupts.CPU91.CAL:Function_call_interrupts 6118 ± 5% -3.4% 5908 ± 5% interrupts.CPU92.CAL:Function_call_interrupts 6117 ± 5% -3.4% 5907 ± 5% interrupts.CPU93.CAL:Function_call_interrupts 6118 ± 5% -3.4% 5907 ± 5% interrupts.CPU94.CAL:Function_call_interrupts 6074 ± 5% -3.8% 5841 ± 5% interrupts.CPU95.CAL:Function_call_interrupts 8980 ± 11% -9.6% 8117 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 6116 ± 5% -3.8% 5882 ± 5% interrupts.CPU96.CAL:Function_call_interrupts 6116 ± 5% -5.4% 5786 ± 2% interrupts.CPU97.CAL:Function_call_interrupts 6115 ± 5% -3.7% 5890 ± 5% interrupts.CPU99.CAL:Function_call_interrupts 846020 ± 26% -27.6% 612307 interrupts.CPU99.LOC:Local_timer_interrupts 311.00 -34.5% 203.75 ± 44% interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime: cpu/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/stress-ng/1s commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- %stddev %change %stddev \ | \ 66.75 ± 8% +260.3% 240.50 ± 3% stress-ng.numa.ops 46.23 ± 5% +401.7% 231.92 ± 3% stress-ng.numa.ops_per_sec 44912 ± 3% +6.8% 47988 stress-ng.time.voluntary_context_switches 34176 ± 8% -38.7% 20951 ± 11% numa-numastat.node1.numa_hit 34176 ± 8% -38.7% 20951 ± 11% numa-numastat.node1.other_node 162.25 ± 13% -97.5% 4.00 ±106% numa-vmstat.node0.nr_isolated_anon 180.50 ± 13% -100.0% 0.00 numa-vmstat.node1.nr_isolated_anon 22.10 ± 5% -21.7% 17.31 ± 4% perf-stat.i.MPKI 14.96 -4.5% 14.28 perf-stat.overall.MPKI 7.58 +0.1 7.65 perf-stat.overall.branch-miss-rate% 18.33 +0.4 18.73 perf-stat.overall.cache-miss-rate% 1831 +2.2% 1871 perf-stat.overall.cycles-between-cache-misses 1.46 ± 33% -1.1 0.32 ±103% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 1.44 ± 35% -0.8 0.63 ± 78% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 1.70 ± 15% -0.6 1.11 ± 62% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.55 ± 66% +2.6 3.12 ± 75% perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__ioctl.perf_evlist__enable 0.55 ± 66% +2.6 3.13 ± 74% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__ioctl.perf_evlist__enable.cmd_record.run_builtin 0.55 ± 66% +2.6 3.13 ± 74% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__ioctl.perf_evlist__enable.cmd_record 2626 ± 10% +17.0% 3072 ± 6% slabinfo.avtab_node.active_objs 2626 ± 10% +17.0% 3072 ± 6% slabinfo.avtab_node.num_objs 44473 +7.0% 47586 ± 4% slabinfo.filp.active_objs 44478 +7.1% 47622 ± 4% slabinfo.filp.num_objs 828.75 ± 9% -14.2% 711.00 ± 4% slabinfo.skbuff_fclone_cache.active_objs 828.75 ± 9% -14.2% 711.00 ± 4% slabinfo.skbuff_fclone_cache.num_objs 431.00 ± 30% -99.0% 4.50 ±101% proc-vmstat.nr_isolated_anon 34176 ± 8% -38.7% 20951 ± 11% proc-vmstat.numa_other 23394485 ± 9% -74.5% 5964109 ± 40% proc-vmstat.pgalloc_normal 23265642 ± 9% -75.0% 5825344 ± 41% proc-vmstat.pgfree 592.75 ± 15% +472.5% 3393 ± 8% proc-vmstat.pgmigrate_fail 51229 ± 8% -86.2% 7061 ± 15% proc-vmstat.pgmigrate_success 40681 ± 10% -83.7% 6633 ± 71% proc-vmstat.thp_deferred_split_page 497.44 ± 3% -16.4% 415.64 ± 4% sched_debug.cfs_rq:/.exec_clock.stddev 4.59 ± 20% -24.7% 3.45 ± 10% sched_debug.cfs_rq:/.load_avg.avg 321.12 ± 46% -63.6% 116.75 ± 20% sched_debug.cfs_rq:/.load_avg.max 23.22 ± 35% -51.5% 11.26 ± 12% sched_debug.cfs_rq:/.load_avg.stddev 14.50 ± 22% +62.9% 23.62 ± 23% sched_debug.cfs_rq:/.nr_spread_over.max 1.68 ± 12% +26.8% 2.13 ± 18% sched_debug.cfs_rq:/.nr_spread_over.stddev 1078 ± 5% +17.5% 1267 ± 9% sched_debug.cfs_rq:/.util_avg.max 9496 ± 18% -15.6% 8019 ± 4% softirqs.CPU0.RCU 40692 ± 5% -9.7% 36753 ± 2% softirqs.CPU0.TIMER 40960 ± 5% -9.7% 36968 ± 5% softirqs.CPU10.TIMER 42943 ± 5% -7.9% 39563 ± 5% softirqs.CPU114.TIMER 7967 ± 15% -14.7% 6795 ± 5% softirqs.CPU128.RCU 44489 ± 5% -13.2% 38635 ± 5% softirqs.CPU128.TIMER 43555 ± 7% -13.6% 37648 ± 5% softirqs.CPU144.TIMER 7955 ± 16% -15.9% 6693 ± 4% softirqs.CPU145.RCU 43111 ± 7% -11.9% 37982 ± 4% softirqs.CPU160.TIMER 7755 ± 18% -14.0% 6667 ± 6% softirqs.CPU186.RCU 43744 ± 6% -7.9% 40296 ± 5% softirqs.CPU187.TIMER 41152 ± 7% -9.2% 37362 ± 4% softirqs.CPU190.TIMER 43610 ± 3% -11.9% 38408 ± 5% softirqs.CPU198.TIMER 42378 ± 7% -11.4% 37538 ± 4% softirqs.CPU240.TIMER 8735 ± 18% -21.6% 6845 ± 8% softirqs.CPU262.RCU 37455 ± 4% -7.7% 34575 ± 3% softirqs.CPU270.TIMER 8857 ± 20% -25.2% 6622 ± 5% softirqs.CPU271.RCU 40008 ± 5% -9.7% 36122 ± 2% softirqs.CPU32.TIMER 45805 ± 22% -17.4% 37825 ± 4% softirqs.CPU37.TIMER 40227 ± 4% -10.2% 36125 ± 4% softirqs.CPU55.TIMER 44908 ± 6% -12.3% 39385 ± 5% softirqs.CPU64.TIMER 12303 ± 8% -24.0% 9348 ± 18% softirqs.CPU69.RCU 51650 ± 4% -17.8% 42473 ± 2% softirqs.CPU69.TIMER 44028 ± 6% -9.6% 39794 ± 3% softirqs.CPU86.TIMER 2009 ± 12% -24.9% 1509 ± 17% interrupts.CPU0.RES:Rescheduling_interrupts 956.00 ±136% -91.7% 79.00 ± 24% interrupts.CPU10.RES:Rescheduling_interrupts 58.50 ± 38% +127.4% 133.00 ± 42% interrupts.CPU101.RES:Rescheduling_interrupts 33.25 ± 29% +226.3% 108.50 ± 94% interrupts.CPU104.RES:Rescheduling_interrupts 7122 ± 3% +10.9% 7897 ± 2% interrupts.CPU115.CAL:Function_call_interrupts 56.25 ± 27% +98.2% 111.50 ± 62% interrupts.CPU118.RES:Rescheduling_interrupts 446.25 ± 38% -48.8% 228.50 ± 18% interrupts.CPU118.TLB:TLB_shootdowns 7016 ± 5% +11.8% 7841 interrupts.CPU124.CAL:Function_call_interrupts 241.00 ±120% -77.3% 54.75 ± 23% interrupts.CPU124.RES:Rescheduling_interrupts 103.00 ± 41% -54.6% 46.75 ± 28% interrupts.CPU125.RES:Rescheduling_interrupts 37.00 ± 8% +303.4% 149.25 ± 76% interrupts.CPU156.RES:Rescheduling_interrupts 66.25 ± 81% +108.3% 138.00 ± 64% interrupts.CPU162.RES:Rescheduling_interrupts 34.75 ± 25% +656.1% 262.75 ±116% interrupts.CPU168.RES:Rescheduling_interrupts 117.25 ± 20% -42.9% 67.00 ± 29% interrupts.CPU193.RES:Rescheduling_interrupts 8393 ± 3% +11.5% 9359 ± 4% interrupts.CPU205.CAL:Function_call_interrupts 87.75 ± 39% -44.7% 48.50 ± 28% interrupts.CPU205.RES:Rescheduling_interrupts 6863 ± 5% +12.5% 7720 ± 2% interrupts.CPU21.CAL:Function_call_interrupts 47.25 ± 9% +66.7% 78.75 ± 27% interrupts.CPU224.RES:Rescheduling_interrupts 47.50 ± 33% +81.1% 86.00 ± 29% interrupts.CPU232.RES:Rescheduling_interrupts 32.25 ± 14% +106.2% 66.50 ± 50% interrupts.CPU241.RES:Rescheduling_interrupts 328.25 ± 25% +167.6% 878.50 ± 60% interrupts.CPU248.TLB:TLB_shootdowns 36.75 ± 43% +105.4% 75.50 ± 40% interrupts.CPU256.RES:Rescheduling_interrupts 377.75 ± 20% +29.8% 490.25 ± 15% interrupts.CPU264.TLB:TLB_shootdowns 73.50 ± 45% +204.8% 224.00 ± 61% interrupts.CPU28.RES:Rescheduling_interrupts 79.25 ± 37% +76.3% 139.75 ± 35% interrupts.CPU32.RES:Rescheduling_interrupts 6892 ± 6% +14.9% 7919 ± 3% interrupts.CPU37.CAL:Function_call_interrupts 6892 ± 4% +16.9% 8055 ± 8% interrupts.CPU39.CAL:Function_call_interrupts 51.75 ± 15% +68.1% 87.00 ± 34% interrupts.CPU39.RES:Rescheduling_interrupts 476.00 ± 17% +102.9% 966.00 ± 31% interrupts.CPU40.TLB:TLB_shootdowns 6847 ± 5% +12.6% 7711 ± 2% interrupts.CPU41.CAL:Function_call_interrupts 6853 ± 4% +10.9% 7603 ± 3% interrupts.CPU48.CAL:Function_call_interrupts 522.50 ± 27% -48.1% 271.00 ± 74% interrupts.CPU54.TLB:TLB_shootdowns 454.25 ± 29% +85.5% 842.75 ± 49% interrupts.CPU60.TLB:TLB_shootdowns 300.00 ± 32% +148.1% 744.25 ± 55% interrupts.CPU61.TLB:TLB_shootdowns 52.75 ± 23% +130.3% 121.50 ± 50% interrupts.CPU64.RES:Rescheduling_interrupts 52.00 ± 22% +122.6% 115.75 ± 62% interrupts.CPU70.RES:Rescheduling_interrupts 6972 ± 5% +8.9% 7595 ± 5% interrupts.CPU77.CAL:Function_call_interrupts 6923 ± 4% +10.5% 7650 ± 2% interrupts.CPU8.CAL:Function_call_interrupts 7233 ± 5% +11.3% 8049 ± 2% interrupts.CPU81.CAL:Function_call_interrupts 585.25 ± 10% -37.6% 365.00 ± 27% interrupts.CPU88.TLB:TLB_shootdowns *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime: pipe/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/stress-ng/1s commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 kmsg.DHCP/BOOTP:Reply_not_for_us_on_eth#,op[#]xid[#] :4 25% 1:4 dmesg.RIP:get_gup_pin_page 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 25% 1:4 dmesg.WARNING:at_mm/gup.c:#get_gup_pin_page 1:4 -25% :4 dmesg.WARNING:stack_recursion %stddev %change %stddev \ | \ 18012 -10.5% 16117 ± 8% stress-ng.time.percent_of_cpu_this_job_got 1592 -4.9% 1513 stress-ng.time.system_time 3995345 -47.5% 2099227 stress-ng.vm-splice.ops 4001986 -18.5% 3259638 ± 7% stress-ng.vm-splice.ops_per_sec 0.24 ±173% +1.3 1.53 ± 44% perf-profile.calltrace.cycles-pp.cpu_load_update.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer 185055 ± 59% +38.4% 256144 ± 49% meminfo.Active 184967 ± 59% +38.4% 256057 ± 49% meminfo.Active(anon) 185946 ± 58% +40.8% 261876 ± 50% numa-meminfo.node0.Active 185858 ± 58% +40.9% 261789 ± 50% numa-meminfo.node0.Active(anon) 17098 ± 6% -11.5% 15128 ± 3% numa-meminfo.node1.SUnreclaim 47648 ± 62% +42.2% 67747 ± 51% numa-vmstat.node0.nr_active_anon 47622 ± 62% +42.2% 67725 ± 51% numa-vmstat.node0.nr_zone_active_anon 4274 ± 6% -11.5% 3782 ± 3% numa-vmstat.node1.nr_slab_unreclaimable 24880 +0.9% 25102 proc-vmstat.nr_slab_reclaimable 18690590 +1.3% 18925901 proc-vmstat.numa_hit 18690590 +1.3% 18925901 proc-vmstat.numa_local 10842 ± 3% +7.3% 11637 ± 5% softirqs.CPU13.TIMER 14451 ± 26% -26.5% 10615 ± 5% softirqs.CPU18.TIMER 10649 ± 5% +15.8% 12333 ± 10% softirqs.CPU56.TIMER 3.37 ± 7% -44.3% 1.88 ± 33% sched_debug.cfs_rq:/.exec_clock.avg 22.72 ± 6% -29.9% 15.92 ± 26% sched_debug.cfs_rq:/.exec_clock.stddev 5.80 ± 34% -40.9% 3.43 ± 11% sched_debug.cfs_rq:/.load_avg.avg 1020 -51.2% 498.00 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.max 78.11 ± 12% -33.3% 52.06 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.stddev 7243 ± 12% +79.0% 12964 ± 17% sched_debug.cpu.avg_idle.min 2420 ± 18% -19.7% 1943 perf-stat.i.cycles-between-cache-misses 137.18 ± 5% +16.2% 159.47 ± 6% perf-stat.i.instructions-per-iTLB-miss 1969 ± 4% -7.2% 1827 ± 2% perf-stat.overall.cycles-between-cache-misses 0.90 ± 3% -0.0 0.85 perf-stat.overall.iTLB-load-miss-rate% 111.89 ± 2% +4.8% 117.28 perf-stat.overall.instructions-per-iTLB-miss 1.822e+11 -12.8% 1.589e+11 ± 11% perf-stat.ps.cpu-cycles 10509 ± 3% -12.6% 9183 ± 10% perf-stat.ps.minor-faults 457963 ± 2% -12.0% 402874 ± 11% perf-stat.ps.msec 10647 ± 4% -13.0% 9267 ± 10% perf-stat.ps.page-faults 1840 ± 34% -35.6% 1184 ± 19% interrupts.CPU12.RES:Rescheduling_interrupts 792.50 ± 7% +33.7% 1059 ± 17% interrupts.CPU126.RES:Rescheduling_interrupts 743.00 ± 50% +119.2% 1628 ± 20% interrupts.CPU154.RES:Rescheduling_interrupts 1490 ± 9% -37.5% 931.50 ± 21% interrupts.CPU16.RES:Rescheduling_interrupts 880.50 ± 30% +70.6% 1502 ± 10% interrupts.CPU194.RES:Rescheduling_interrupts 931.00 ± 23% +49.5% 1391 ± 18% interrupts.CPU216.RES:Rescheduling_interrupts 1158 ± 11% -31.0% 798.75 ± 30% interrupts.CPU236.RES:Rescheduling_interrupts 1253 ± 21% -32.7% 843.00 ± 31% interrupts.CPU241.RES:Rescheduling_interrupts 1700 ± 37% -58.2% 710.00 ± 48% interrupts.CPU243.RES:Rescheduling_interrupts 1399 ± 20% -40.4% 834.25 ± 35% interrupts.CPU255.RES:Rescheduling_interrupts 1064 ± 19% +32.0% 1405 ± 7% interrupts.CPU256.RES:Rescheduling_interrupts 996.75 ± 24% +32.0% 1316 ± 11% interrupts.CPU262.RES:Rescheduling_interrupts 1129 ± 22% +46.2% 1652 ± 16% interrupts.CPU267.RES:Rescheduling_interrupts 1553 ± 19% -34.4% 1019 ± 25% interrupts.CPU39.RES:Rescheduling_interrupts 1110 ± 5% +25.3% 1390 ± 14% interrupts.CPU40.RES:Rescheduling_interrupts 1073 ± 18% +45.5% 1562 ± 4% interrupts.CPU41.RES:Rescheduling_interrupts 1478 ± 20% -35.3% 956.50 ± 22% interrupts.CPU47.RES:Rescheduling_interrupts 1554 ± 7% -29.6% 1094 ± 24% interrupts.CPU54.RES:Rescheduling_interrupts 1613 ± 18% -37.6% 1007 ± 38% interrupts.CPU57.RES:Rescheduling_interrupts 946.25 ± 15% +45.7% 1379 ± 10% interrupts.CPU64.RES:Rescheduling_interrupts 780.00 ± 31% +81.5% 1415 ± 12% interrupts.CPU84.RES:Rescheduling_interrupts *************************************************************************************************** lkp-bdw-ep3b: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep3b/futex2/will-it-scale/0xb00002e commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- %stddev %change %stddev \ | \ 26674 ± 17% -22.3% 20716 ± 3% softirqs.CPU30.RCU 26755 ± 20% -21.3% 21064 ± 3% softirqs.CPU59.RCU 295164 +11.4% 328898 meminfo.Active 294996 +11.4% 328730 meminfo.Active(anon) 295368 -25.0% 221640 ± 14% meminfo.DirectMap4k 24.48 +0.2 24.66 perf-profile.calltrace.cycles-pp.get_futex_key.futex_wait_setup.futex_wait.do_futex.__x64_sys_futex 14.88 +0.2 15.09 perf-profile.calltrace.cycles-pp.gup_pgd_range.get_user_pages_fast.get_futex_key.futex_wait_setup.futex_wait 50.45 +0.2 50.67 perf-profile.calltrace.cycles-pp.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe 73752 +11.4% 82177 proc-vmstat.nr_active_anon 73752 +11.4% 82177 proc-vmstat.nr_zone_active_anon 710645 -2.3% 694065 proc-vmstat.pgfree 2.108e+08 ± 15% -53.8% 97334654 ± 18% cpuidle.C6.time 262362 ± 3% -54.2% 120108 ± 14% cpuidle.C6.usage 27692 ± 77% -79.8% 5606 ± 27% cpuidle.POLL.time 4498 ± 40% -44.2% 2509 ± 8% cpuidle.POLL.usage 362291 -41.5% 211769 ± 18% numa-numastat.node0.local_node 379384 -42.7% 217528 ± 19% numa-numastat.node0.numa_hit 17093 -66.3% 5760 ±120% numa-numastat.node0.other_node 289553 +50.8% 436751 ± 9% numa-numastat.node1.local_node 289591 +54.7% 448130 ± 9% numa-numastat.node1.numa_hit 250685 ± 3% -56.5% 108986 ± 17% turbostat.C6 0.77 ± 16% -0.4 0.34 ± 20% turbostat.C6% 0.55 ± 24% -58.3% 0.23 ± 32% turbostat.CPU%c6 0.35 ± 23% -54.2% 0.16 ± 55% turbostat.Pkg%pc2 0.03 ± 66% -91.7% 0.00 ±173% turbostat.Pkg%pc6 34.58 ± 11% +30.4% 45.08 ± 4% sched_debug.cfs_rq:/.nr_spread_over.max 3.94 ± 7% +25.1% 4.92 ± 5% sched_debug.cfs_rq:/.nr_spread_over.stddev 2.39 ± 2% +183.0% 6.78 ±106% sched_debug.cpu.cpu_load[0].stddev 24.58 +107.3% 50.96 ± 62% sched_debug.cpu.cpu_load[1].max 2.26 ± 6% +118.1% 4.94 ± 64% sched_debug.cpu.cpu_load[1].stddev 25212 ± 22% -25.8% 18696 ± 27% sched_debug.cpu.sched_count.max 125453 -33.3% 83670 ± 19% numa-meminfo.node0.AnonHugePages 168582 -21.2% 132815 ± 14% numa-meminfo.node0.AnonPages 124539 +18.2% 147258 ± 4% numa-meminfo.node1.Active 124539 +18.2% 147216 ± 4% numa-meminfo.node1.Active(anon) 43851 ± 4% +99.6% 87509 ± 18% numa-meminfo.node1.AnonHugePages 75101 ± 2% +48.9% 111789 ± 16% numa-meminfo.node1.AnonPages 2816 ± 2% +22.4% 3447 ± 17% numa-meminfo.node1.PageTables 54950 +11.2% 61121 ± 7% numa-meminfo.node1.SUnreclaim 42139 -21.2% 33199 ± 14% numa-vmstat.node0.nr_anon_pages 711043 -19.2% 574713 ± 14% numa-vmstat.node0.numa_hit 693926 -18.0% 568865 ± 13% numa-vmstat.node0.numa_local 17117 -65.8% 5847 ±117% numa-vmstat.node0.numa_other 31111 +18.3% 36809 ± 4% numa-vmstat.node1.nr_active_anon 18777 ± 2% +48.8% 27945 ± 16% numa-vmstat.node1.nr_anon_pages 699.50 ± 3% +22.0% 853.50 ± 16% numa-vmstat.node1.nr_page_table_pages 13737 +11.2% 15279 ± 7% numa-vmstat.node1.nr_slab_unreclaimable 31111 +18.3% 36809 ± 4% numa-vmstat.node1.nr_zone_active_anon 426549 +32.8% 566367 ± 15% numa-vmstat.node1.numa_hit 288820 +44.7% 417867 ± 20% numa-vmstat.node1.numa_local 0.00 -0.0 0.00 ± 2% perf-stat.i.dTLB-load-miss-rate% 195436 -4.1% 187470 ± 3% perf-stat.i.dTLB-load-misses 67.36 -0.7 66.69 perf-stat.i.iTLB-load-miss-rate% 1.66e+08 +2.8% 1.707e+08 perf-stat.i.iTLB-loads 51056 -8.1% 46912 ± 2% perf-stat.i.node-load-misses 7153 -30.0% 5006 ± 21% perf-stat.i.node-loads 0.00 -0.0 0.00 ± 3% perf-stat.overall.dTLB-load-miss-rate% 67.37 -0.7 66.69 perf-stat.overall.iTLB-load-miss-rate% 194906 -4.1% 186955 ± 3% perf-stat.ps.dTLB-load-misses 1.655e+08 +2.8% 1.701e+08 perf-stat.ps.iTLB-loads 50904 -8.1% 46767 ± 2% perf-stat.ps.node-load-misses 7136 -30.0% 4996 ± 21% perf-stat.ps.node-loads 352.50 ± 10% +32.5% 467.00 ± 14% interrupts.32:PCI-MSI.3145729-edge.eth0-TxRx-0 222.00 ± 16% +84.0% 408.50 ± 27% interrupts.34:PCI-MSI.3145731-edge.eth0-TxRx-2 275.00 ± 12% -25.7% 204.25 ± 17% interrupts.37:PCI-MSI.3145734-edge.eth0-TxRx-5 352.50 ± 10% +32.5% 467.00 ± 14% interrupts.CPU11.32:PCI-MSI.3145729-edge.eth0-TxRx-0 651.50 ± 59% -76.4% 154.00 ± 59% interrupts.CPU11.RES:Rescheduling_interrupts 222.00 ± 16% +84.0% 408.50 ± 27% interrupts.CPU13.34:PCI-MSI.3145731-edge.eth0-TxRx-2 1011 ± 2% -82.7% 175.25 ± 50% interrupts.CPU13.RES:Rescheduling_interrupts 1680 ± 26% -64.1% 602.75 ± 85% interrupts.CPU14.RES:Rescheduling_interrupts 472.50 ± 53% -54.1% 217.00 ± 95% interrupts.CPU15.RES:Rescheduling_interrupts 275.00 ± 12% -25.7% 204.25 ± 17% interrupts.CPU16.37:PCI-MSI.3145734-edge.eth0-TxRx-5 635.00 ± 62% -54.5% 289.00 ±113% interrupts.CPU17.RES:Rescheduling_interrupts 903.00 ± 12% -60.4% 357.50 ± 82% interrupts.CPU18.RES:Rescheduling_interrupts 109.00 ± 30% +627.1% 792.50 ± 99% interrupts.CPU24.RES:Rescheduling_interrupts 146.00 ± 8% +365.9% 680.25 ± 62% interrupts.CPU28.RES:Rescheduling_interrupts 1609 ± 53% -79.9% 323.50 ± 83% interrupts.CPU3.RES:Rescheduling_interrupts 125.50 ± 25% +1227.3% 1665 ± 59% interrupts.CPU30.RES:Rescheduling_interrupts 381.00 ± 48% +240.0% 1295 ± 33% interrupts.CPU31.RES:Rescheduling_interrupts 127.50 ± 29% +1944.3% 2606 ± 69% interrupts.CPU36.RES:Rescheduling_interrupts 132.00 ± 22% +141.3% 318.50 ± 23% interrupts.CPU40.RES:Rescheduling_interrupts 255.50 ± 69% +377.2% 1219 ± 71% interrupts.CPU42.RES:Rescheduling_interrupts 965.00 ± 90% -93.8% 60.00 ± 93% interrupts.CPU45.RES:Rescheduling_interrupts 945.50 ± 96% -97.6% 22.75 ± 54% interrupts.CPU49.RES:Rescheduling_interrupts 231.50 ± 85% -88.1% 27.50 ± 89% interrupts.CPU50.RES:Rescheduling_interrupts 345.50 ± 91% -95.0% 17.25 ± 70% interrupts.CPU57.RES:Rescheduling_interrupts 71.50 ± 44% -47.6% 37.50 ± 88% interrupts.CPU59.RES:Rescheduling_interrupts 7877 -37.5% 4926 ± 34% interrupts.CPU6.NMI:Non-maskable_interrupts 7877 -37.5% 4926 ± 34% interrupts.CPU6.PMI:Performance_monitoring_interrupts 1167 ± 44% -88.0% 139.75 ± 60% interrupts.CPU6.RES:Rescheduling_interrupts 331.50 ± 91% -94.7% 17.50 ± 62% interrupts.CPU61.RES:Rescheduling_interrupts 136.50 ± 4% -81.7% 25.00 ± 57% interrupts.CPU64.RES:Rescheduling_interrupts 7897 -37.8% 4913 ± 34% interrupts.CPU7.NMI:Non-maskable_interrupts 7897 -37.8% 4913 ± 34% interrupts.CPU7.PMI:Performance_monitoring_interrupts 1356 ± 18% -65.8% 463.75 ± 6% interrupts.CPU7.RES:Rescheduling_interrupts 27.50 ± 5% +335.5% 119.75 ±101% interrupts.CPU73.RES:Rescheduling_interrupts 203.00 ± 13% -80.3% 40.00 ± 31% interrupts.CPU75.RES:Rescheduling_interrupts 7885 -37.4% 4934 ± 34% interrupts.CPU8.NMI:Non-maskable_interrupts 7885 -37.4% 4934 ± 34% interrupts.CPU8.PMI:Performance_monitoring_interrupts 1232 ± 51% -80.8% 236.50 ± 85% interrupts.CPU8.RES:Rescheduling_interrupts 295.00 ± 61% -58.7% 121.75 ± 65% interrupts.CPU87.RES:Rescheduling_interrupts *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime: os/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/stress-ng/1s commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 7:4 58% 10:4 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.key_lookup.lookup_user_key.keyctl_set_timeout :4 50% 2:4 dmesg.RIP:get_gup_pin_page :4 50% 2:4 dmesg.WARNING:at_mm/gup.c:#get_gup_pin_page :4 100% 4:4 kmsg.Memory_failure:#:dirty_LRU_page_still_referenced_by#users :4 100% 4:4 kmsg.Memory_failure:#:recovery_action_for_dirty_LRU_page:Failed 4:4 -100% :4 kmsg.Memory_failure:#:recovery_action_for_dirty_LRU_page:Recovered %stddev %change %stddev \ | \ 291197 -55.0% 131136 ± 11% stress-ng.futex.ops 290079 -58.0% 121851 ± 12% stress-ng.futex.ops_per_sec 2015 ± 80% +203.6% 6119 ± 61% stress-ng.io.ops_per_sec 122468 ± 2% +22.4% 149941 ± 2% stress-ng.mlock.ops_per_sec 65.00 ± 6% +251.2% 228.25 ± 5% stress-ng.numa.ops 45.61 ± 5% +383.3% 220.41 ± 5% stress-ng.numa.ops_per_sec 625254 ± 6% -23.1% 480881 ± 10% stress-ng.sem.ops 624395 ± 6% -23.1% 480000 ± 10% stress-ng.sem.ops_per_sec 14903791 -2.6% 14517111 stress-ng.time.voluntary_context_switches 306260 ± 4% -8.3% 280934 ± 5% stress-ng.userfaultfd.ops 306179 ± 4% -8.2% 280922 ± 5% stress-ng.userfaultfd.ops_per_sec 4157856 -49.5% 2098955 stress-ng.vm-splice.ops 4161453 -13.4% 3603810 stress-ng.vm-splice.ops_per_sec 1.53e+08 ± 2% +3.1% 1.578e+08 perf-stat.ps.iTLB-load-misses 4293169 ± 66% -44.9% 2363760 ± 2% cpuidle.POLL.time 81481 ± 48% -34.3% 53539 ± 2% cpuidle.POLL.usage 33283 ± 6% -59.2% 13575 ± 11% numa-numastat.node1.numa_hit 33283 ± 6% -59.2% 13575 ± 11% numa-numastat.node1.other_node 564127 ± 4% +185.9% 1612672 ± 19% meminfo.Active 563846 ± 4% +185.9% 1612232 ± 19% meminfo.Active(anon) 485.00 ± 22% +118.9% 1061 ± 7% meminfo.Inactive(file) 4144644 +26.3% 5236228 ± 6% meminfo.Memused 510156 ± 3% +205.4% 1558069 ± 20% numa-meminfo.node0.Active 509874 ± 3% +205.5% 1557628 ± 20% numa-meminfo.node0.Active(anon) 474.50 ± 21% +123.8% 1061 ± 9% numa-meminfo.node0.Inactive(file) 3534355 +30.6% 4617529 ± 7% numa-meminfo.node0.MemUsed 123321 ± 4% +218.3% 392482 ± 20% numa-vmstat.node0.nr_active_anon 70.00 ± 19% +58.9% 111.25 ± 3% numa-vmstat.node0.nr_active_file 119.00 ± 21% +124.2% 266.75 ± 8% numa-vmstat.node0.nr_inactive_file 123242 ± 4% +218.4% 392451 ± 20% numa-vmstat.node0.nr_zone_active_anon 70.00 ± 19% +58.9% 111.25 ± 3% numa-vmstat.node0.nr_zone_active_file 119.00 ± 21% +124.2% 266.75 ± 8% numa-vmstat.node0.nr_zone_inactive_file 3053 ± 6% -14.4% 2615 ± 12% sched_debug.cfs_rq:/.exec_clock.stddev 1.46 ± 6% -15.1% 1.24 ± 5% sched_debug.cfs_rq:/.nr_running.max 585.54 ± 15% -35.4% 378.27 ± 19% sched_debug.cfs_rq:/.runnable_load_avg.max 567.38 ± 15% -37.7% 353.29 ± 11% sched_debug.cpu.cpu_load[0].max 93234 ± 21% -24.0% 70869 ± 10% sched_debug.cpu.curr->pid.avg 185032 ± 4% -13.7% 159654 ± 10% sched_debug.cpu.curr->pid.max 23033 ± 9% -9.2% 20906 ± 6% sched_debug.cpu.sched_goidle.max 2420 ± 6% -13.9% 2085 ± 4% sched_debug.cpu.sched_goidle.stddev 38.68 ± 40% -60.9% 15.14 ± 30% sched_debug.rt_rq:/.rt_time.max 7.03 ± 67% -61.5% 2.71 ± 40% sched_debug.rt_rq:/.rt_time.stddev 134783 ± 5% +190.2% 391176 ± 17% proc-vmstat.nr_active_anon 70.25 ± 17% +58.4% 111.25 ± 4% proc-vmstat.nr_active_file 121.75 ± 20% +117.9% 265.25 ± 7% proc-vmstat.nr_inactive_file 131.00 ± 23% -99.8% 0.25 ±173% proc-vmstat.nr_isolated_anon 105352 -2.8% 102442 proc-vmstat.nr_slab_reclaimable 134783 ± 5% +190.2% 391176 ± 17% proc-vmstat.nr_zone_active_anon 70.25 ± 17% +58.4% 111.25 ± 4% proc-vmstat.nr_zone_active_file 121.75 ± 20% +117.9% 265.25 ± 7% proc-vmstat.nr_zone_inactive_file 864.50 ± 25% +638.5% 6384 ± 83% proc-vmstat.numa_huge_pte_updates 33283 ± 6% -59.2% 13575 ± 11% proc-vmstat.numa_other 445254 ± 25% +634.9% 3272027 ± 83% proc-vmstat.numa_pte_updates 466.00 ± 12% +382.9% 2250 ± 9% proc-vmstat.pgmigrate_fail 49887 ± 6% -92.6% 3698 ± 33% proc-vmstat.pgmigrate_success 7822768 -1.3% 7721955 proc-vmstat.unevictable_pgs_culled 352423 ± 2% +4.2% 367091 proc-vmstat.unevictable_pgs_mlocked 351801 ± 2% +4.1% 366356 proc-vmstat.unevictable_pgs_munlocked 351393 ± 2% +4.1% 365940 proc-vmstat.unevictable_pgs_rescued 11075 ± 8% -16.5% 9245 ± 9% softirqs.CPU100.NET_RX 8323 ± 9% +47.3% 12259 ± 23% softirqs.CPU111.NET_RX 10330 ± 11% -17.8% 8494 ± 5% softirqs.CPU120.NET_RX 28219 ± 4% -13.9% 24292 ± 3% softirqs.CPU123.RCU 19316 ± 6% -8.9% 17600 ± 4% softirqs.CPU127.SCHED 11518 ± 2% -26.4% 8477 ± 9% softirqs.CPU128.NET_RX 6724 ± 22% +67.4% 11258 ± 24% softirqs.CPU131.NET_RX 8571 ± 33% +62.2% 13903 ± 24% softirqs.CPU138.NET_RX 7441 ± 4% +33.9% 9965 ± 14% softirqs.CPU141.NET_RX 8270 ± 16% +44.7% 11966 ± 15% softirqs.CPU146.NET_RX 11298 ± 20% -36.7% 7155 ± 18% softirqs.CPU148.NET_RX 8935 ± 23% +33.2% 11900 ± 29% softirqs.CPU153.NET_RX 19787 ± 5% -9.6% 17897 ± 3% softirqs.CPU163.SCHED 26941 ± 5% -9.5% 24385 ± 3% softirqs.CPU169.RCU 12370 ± 25% -34.7% 8074 ± 13% softirqs.CPU177.NET_RX 20748 ± 6% -10.9% 18487 ± 4% softirqs.CPU181.SCHED 20707 ± 9% -16.1% 17369 ± 2% softirqs.CPU186.SCHED 19622 ± 6% -10.7% 17522 ± 3% softirqs.CPU187.SCHED 19406 ± 6% -8.9% 17672 ± 2% softirqs.CPU194.SCHED 32272 ± 8% -14.2% 27679 ± 7% softirqs.CPU20.RCU 12603 ± 21% -29.4% 8893 ± 16% softirqs.CPU209.NET_RX 11432 ± 9% -14.7% 9749 ± 14% softirqs.CPU21.NET_RX 9075 ± 9% -19.4% 7310 ± 7% softirqs.CPU214.NET_RX 8118 ± 9% +55.0% 12584 ± 24% softirqs.CPU240.NET_RX 9232 ± 16% -27.3% 6708 ± 9% softirqs.CPU244.NET_RX 8203 ± 15% +32.8% 10895 ± 7% softirqs.CPU265.NET_RX 14174 ± 8% -31.4% 9727 ± 23% softirqs.CPU267.NET_RX 17401 ± 11% -55.9% 7675 ± 21% softirqs.CPU28.NET_RX 30370 ± 4% -12.7% 26507 ± 3% softirqs.CPU3.RCU 11791 ± 4% -34.4% 7732 ± 13% softirqs.CPU46.NET_RX 19641 ± 7% -10.5% 17574 ± 4% softirqs.CPU51.SCHED 9879 ± 18% +21.5% 12003 ± 13% softirqs.CPU53.NET_RX 9572 ± 7% -8.3% 8781 ± 6% softirqs.CPU58.NET_RX 11786 ± 12% -33.7% 7818 ± 4% softirqs.CPU6.NET_RX 19919 ± 8% -12.9% 17341 ± 4% softirqs.CPU69.SCHED 8365 ± 15% +64.2% 13734 ± 35% softirqs.CPU76.NET_RX 7648 ± 11% +38.0% 10557 ± 17% softirqs.CPU77.NET_RX 9286 ± 15% +44.2% 13394 ± 16% softirqs.CPU88.NET_RX 27697 ± 4% -11.6% 24482 ± 3% softirqs.CPU90.RCU 8763 ± 16% +34.9% 11818 ± 22% softirqs.CPU94.NET_RX 2680 ± 15% -51.8% 1293 ± 28% interrupts.CPU100.NMI:Non-maskable_interrupts 2680 ± 15% -51.8% 1293 ± 28% interrupts.CPU100.PMI:Performance_monitoring_interrupts 2219 ± 20% -30.4% 1545 ± 22% interrupts.CPU104.NMI:Non-maskable_interrupts 2219 ± 20% -30.4% 1545 ± 22% interrupts.CPU104.PMI:Performance_monitoring_interrupts 2314 ± 19% -31.7% 1580 ± 10% interrupts.CPU106.TLB:TLB_shootdowns 2699 ± 18% -30.8% 1868 ± 10% interrupts.CPU108.TLB:TLB_shootdowns 17763 ± 25% +63.0% 28961 ± 13% interrupts.CPU113.RES:Rescheduling_interrupts 16871 ± 22% +88.1% 31734 ± 27% interrupts.CPU125.RES:Rescheduling_interrupts 24894 ± 18% +27.6% 31769 ± 8% interrupts.CPU128.RES:Rescheduling_interrupts 1755 ± 23% +77.4% 3112 ± 57% interrupts.CPU130.TLB:TLB_shootdowns 2644 ± 31% -35.0% 1719 ± 20% interrupts.CPU142.TLB:TLB_shootdowns 1164 ± 14% +64.3% 1912 ± 46% interrupts.CPU145.NMI:Non-maskable_interrupts 1164 ± 14% +64.3% 1912 ± 46% interrupts.CPU145.PMI:Performance_monitoring_interrupts 23656 ± 38% +53.7% 36365 ± 30% interrupts.CPU150.RES:Rescheduling_interrupts 27724 ± 6% -30.2% 19354 ± 16% interrupts.CPU151.RES:Rescheduling_interrupts 2491 ± 23% -29.6% 1754 ± 18% interrupts.CPU155.TLB:TLB_shootdowns 2153 ± 17% +16.1% 2499 ± 16% interrupts.CPU167.TLB:TLB_shootdowns 2170 ± 18% -20.0% 1735 ± 11% interrupts.CPU169.TLB:TLB_shootdowns 2004 ± 30% -33.3% 1337 ± 17% interrupts.CPU170.NMI:Non-maskable_interrupts 2004 ± 30% -33.3% 1337 ± 17% interrupts.CPU170.PMI:Performance_monitoring_interrupts 2079 ± 20% -25.1% 1557 ± 8% interrupts.CPU171.TLB:TLB_shootdowns 1754 ± 10% +44.1% 2527 ± 17% interrupts.CPU187.TLB:TLB_shootdowns 1632 ± 12% +45.6% 2376 ± 21% interrupts.CPU191.TLB:TLB_shootdowns 3271 ± 49% -50.5% 1618 ± 49% interrupts.CPU199.NMI:Non-maskable_interrupts 3271 ± 49% -50.5% 1618 ± 49% interrupts.CPU199.PMI:Performance_monitoring_interrupts 3481 ± 41% -52.1% 1668 ± 33% interrupts.CPU200.NMI:Non-maskable_interrupts 3481 ± 41% -52.1% 1668 ± 33% interrupts.CPU200.PMI:Performance_monitoring_interrupts 20086 ± 22% +84.1% 36982 ± 5% interrupts.CPU212.RES:Rescheduling_interrupts 1738 ± 12% +17.8% 2048 ± 5% interrupts.CPU212.TLB:TLB_shootdowns 18973 ± 23% +51.8% 28802 ± 19% interrupts.CPU216.RES:Rescheduling_interrupts 2301 ± 28% -51.6% 1113 ± 41% interrupts.CPU218.NMI:Non-maskable_interrupts 2301 ± 28% -51.6% 1113 ± 41% interrupts.CPU218.PMI:Performance_monitoring_interrupts 2267 ± 10% -16.8% 1887 ± 15% interrupts.CPU220.TLB:TLB_shootdowns 15399 ± 14% +61.3% 24836 ± 7% interrupts.CPU222.RES:Rescheduling_interrupts 36076 ± 16% -43.8% 20263 ± 36% interrupts.CPU226.RES:Rescheduling_interrupts 19194 ± 10% +84.8% 35478 ± 13% interrupts.CPU227.RES:Rescheduling_interrupts 2425 ± 8% -19.9% 1942 ± 6% interrupts.CPU227.TLB:TLB_shootdowns 2303 ± 22% -23.9% 1753 ± 8% interrupts.CPU234.TLB:TLB_shootdowns 2837 ± 8% -49.9% 1421 ± 11% interrupts.CPU239.NMI:Non-maskable_interrupts 2837 ± 8% -49.9% 1421 ± 11% interrupts.CPU239.PMI:Performance_monitoring_interrupts 17386 ± 17% +84.6% 32093 ± 30% interrupts.CPU240.RES:Rescheduling_interrupts 18801 ± 55% +92.8% 36248 ± 23% interrupts.CPU246.RES:Rescheduling_interrupts 2469 ± 31% -29.8% 1732 ± 17% interrupts.CPU247.TLB:TLB_shootdowns 31757 ± 27% -35.3% 20541 ± 31% interrupts.CPU250.RES:Rescheduling_interrupts 36905 ± 40% -46.7% 19670 ± 8% interrupts.CPU252.RES:Rescheduling_interrupts 20156 ± 12% +44.7% 29156 ± 7% interrupts.CPU253.RES:Rescheduling_interrupts 4880 ± 30% -48.0% 2539 ± 10% interrupts.CPU26.TLB:TLB_shootdowns 30073 ± 26% -37.2% 18889 ± 28% interrupts.CPU260.RES:Rescheduling_interrupts 24487 ± 4% -23.7% 18693 ± 23% interrupts.CPU261.RES:Rescheduling_interrupts 2201 ± 26% -42.9% 1257 ± 29% interrupts.CPU262.NMI:Non-maskable_interrupts 2201 ± 26% -42.9% 1257 ± 29% interrupts.CPU262.PMI:Performance_monitoring_interrupts 16758 ± 30% +76.3% 29544 ± 16% interrupts.CPU263.RES:Rescheduling_interrupts 2958 ± 56% -54.4% 1350 ± 25% interrupts.CPU266.NMI:Non-maskable_interrupts 2958 ± 56% -54.4% 1350 ± 25% interrupts.CPU266.PMI:Performance_monitoring_interrupts 29594 ± 23% -35.9% 18979 ± 33% interrupts.CPU268.RES:Rescheduling_interrupts 2391 ± 20% -42.8% 1367 ± 25% interrupts.CPU27.NMI:Non-maskable_interrupts 2391 ± 20% -42.8% 1367 ± 25% interrupts.CPU27.PMI:Performance_monitoring_interrupts 17135 ± 26% +74.3% 29873 ± 20% interrupts.CPU32.RES:Rescheduling_interrupts 32963 ± 29% -31.4% 22602 ± 27% interrupts.CPU36.RES:Rescheduling_interrupts 2328 ± 28% -34.4% 1527 ± 50% interrupts.CPU39.NMI:Non-maskable_interrupts 2328 ± 28% -34.4% 1527 ± 50% interrupts.CPU39.PMI:Performance_monitoring_interrupts 16869 ± 24% +69.0% 28507 ± 22% interrupts.CPU39.RES:Rescheduling_interrupts 2584 ± 21% -26.7% 1895 ± 16% interrupts.CPU51.TLB:TLB_shootdowns 3155 ± 16% -36.1% 2015 ± 20% interrupts.CPU52.TLB:TLB_shootdowns 4951 ± 28% -60.4% 1961 ± 25% interrupts.CPU55.TLB:TLB_shootdowns 4382 ± 24% -50.2% 2183 ± 14% interrupts.CPU56.TLB:TLB_shootdowns 15835 ± 21% +108.6% 33033 ± 30% interrupts.CPU6.RES:Rescheduling_interrupts 3549 ± 29% -34.9% 2310 ± 16% interrupts.CPU61.TLB:TLB_shootdowns 3466 ± 41% -51.2% 1691 ± 58% interrupts.CPU67.NMI:Non-maskable_interrupts 3466 ± 41% -51.2% 1691 ± 58% interrupts.CPU67.PMI:Performance_monitoring_interrupts 35989 ± 19% -36.0% 23024 ± 27% interrupts.CPU75.RES:Rescheduling_interrupts 2210 ± 6% -30.4% 1538 ± 14% interrupts.CPU76.TLB:TLB_shootdowns 3307 ± 45% -62.2% 1250 ± 14% interrupts.CPU78.NMI:Non-maskable_interrupts 3307 ± 45% -62.2% 1250 ± 14% interrupts.CPU78.PMI:Performance_monitoring_interrupts 1604 +54.6% 2480 ± 12% interrupts.CPU78.TLB:TLB_shootdowns 2863 ± 57% -42.8% 1636 ± 11% interrupts.CPU90.NMI:Non-maskable_interrupts 2863 ± 57% -42.8% 1636 ± 11% interrupts.CPU90.PMI:Performance_monitoring_interrupts 2308 ± 11% -21.7% 1808 ± 8% interrupts.CPU91.TLB:TLB_shootdowns 18040 ± 36% +89.6% 34197 ± 9% interrupts.CPU94.RES:Rescheduling_interrupts 21085 ± 18% +51.0% 31835 ± 16% interrupts.CPU97.RES:Rescheduling_interrupts *************************************************************************************************** lkp-bdw-ep3: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: vm/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep3/stress-ng/1s/0xb00002e commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 100% 4:4 kmsg.Memory_failure:#:dirty_LRU_page_still_referenced_by#users :4 100% 4:4 kmsg.Memory_failure:#:recovery_action_for_dirty_LRU_page:Failed 4:4 -100% :4 kmsg.Memory_failure:#:recovery_action_for_dirty_LRU_page:Recovered :4 100% 4:4 dmesg.RIP:get_gup_pin_page 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 100% 4:4 dmesg.WARNING:at_mm/gup.c:#get_gup_pin_page 1:4 -25% :4 dmesg.WARNING:stack_recursion %stddev %change %stddev \ | \ 7765 -3.4% 7499 stress-ng.time.percent_of_cpu_this_job_got 1880 -5.1% 1784 stress-ng.time.system_time 5023675 ± 2% -58.3% 2095111 stress-ng.vm-splice.ops 17.91 ± 8% +12.9% 20.22 ± 6% iostat.cpu.idle 2406 -3.4% 2325 turbostat.Avg_MHz 59555797 -11.7% 52587698 vmstat.memory.free 15.47 ± 9% +2.3 17.80 ± 7% mpstat.cpu.idle% 0.10 ± 8% -0.0 0.08 ± 8% mpstat.cpu.soft% 34300 ± 2% -5.0% 32577 perf-stat.ps.major-faults 170288 -0.9% 168728 perf-stat.ps.msec 1131 ± 7% -17.7% 930.75 ± 14% slabinfo.nsproxy.active_objs 1131 ± 7% -17.7% 930.75 ± 14% slabinfo.nsproxy.num_objs 4662 ± 9% +69.2% 7887 ± 49% softirqs.CPU22.RCU 3963 ± 8% +151.7% 9974 ± 50% softirqs.CPU78.RCU 2905249 ± 10% +192.0% 8483775 ± 3% meminfo.Active 2905081 ± 10% +192.0% 8483607 ± 3% meminfo.Active(anon) 23150145 ± 4% -9.9% 20851826 ± 3% meminfo.Committed_AS 1365340 ± 5% +84.5% 2518784 ± 4% meminfo.Inactive 1364952 ± 5% +84.1% 2512920 ± 4% meminfo.Inactive(anon) 387.25 ± 6% +1414.2% 5863 ± 4% meminfo.Inactive(file) 59176014 -11.4% 52417164 meminfo.MemAvailable 59432778 -11.4% 52671931 meminfo.MemFree 6419352 ± 3% +105.3% 13180195 ± 2% meminfo.Memused 1.88 ± 6% +110.3% 3.96 ± 39% sched_debug.cpu.cpu_load[1].avg 19.25 ± 2% +629.9% 140.50 ± 62% sched_debug.cpu.cpu_load[1].max 4.41 ± 4% +258.0% 15.78 ± 53% sched_debug.cpu.cpu_load[1].stddev 2.31 ± 14% +94.2% 4.48 ± 31% sched_debug.cpu.cpu_load[2].avg 26.25 ± 47% +471.4% 150.00 ± 41% sched_debug.cpu.cpu_load[2].max 4.71 ± 21% +248.5% 16.42 ± 38% sched_debug.cpu.cpu_load[2].stddev 37.25 ± 37% +208.1% 114.75 ± 37% sched_debug.cpu.cpu_load[3].max 5.87 ± 19% +125.8% 13.25 ± 34% sched_debug.cpu.cpu_load[3].stddev 33.00 ± 19% +122.0% 73.25 ± 46% sched_debug.cpu.cpu_load[4].max 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.static_key_slow_dec.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.__static_key_slow_dec_cpuslocked.static_key_slow_dec.sw_perf_event_destroy._free_event.perf_event_release_kernel 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.__jump_label_update.__static_key_slow_dec_cpuslocked.static_key_slow_dec.sw_perf_event_destroy._free_event 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.arch_jump_label_transform.__jump_label_update.__static_key_slow_dec_cpuslocked.static_key_slow_dec.sw_perf_event_destroy 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.__jump_label_transform.arch_jump_label_transform.__jump_label_update.__static_key_slow_dec_cpuslocked.static_key_slow_dec 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.smp_call_function_many.on_each_cpu.text_poke_bp.__jump_label_transform.arch_jump_label_transform 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.on_each_cpu.text_poke_bp.__jump_label_transform.arch_jump_label_transform.__jump_label_update 11.46 ± 47% -7.5 3.97 ± 98% perf-profile.calltrace.cycles-pp.text_poke_bp.__jump_label_transform.arch_jump_label_transform.__jump_label_update.__static_key_slow_dec_cpuslocked 11.46 ± 47% -6.7 4.78 ± 69% perf-profile.calltrace.cycles-pp._free_event.perf_event_release_kernel.perf_release.__fput.task_work_run 11.46 ± 47% -6.7 4.78 ± 69% perf-profile.calltrace.cycles-pp.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release.__fput 695203 ± 5% +213.3% 2177910 ± 3% proc-vmstat.nr_active_anon 1473798 -11.8% 1299346 proc-vmstat.nr_dirty_background_threshold 2952003 -11.8% 2603072 proc-vmstat.nr_dirty_threshold 14852961 -11.8% 13107633 proc-vmstat.nr_free_pages 377237 ± 7% +69.4% 639157 ± 3% proc-vmstat.nr_inactive_anon 97.00 ± 3% +1395.6% 1450 ± 6% proc-vmstat.nr_inactive_file 695202 ± 5% +213.3% 2177910 ± 3% proc-vmstat.nr_zone_active_anon 377237 ± 7% +69.4% 639157 ± 3% proc-vmstat.nr_zone_inactive_anon 97.00 ± 3% +1395.6% 1450 ± 6% proc-vmstat.nr_zone_inactive_file 12792774 ± 2% -10.7% 11419593 ± 7% proc-vmstat.pgactivate 2113 -3.3% 2043 ± 2% proc-vmstat.thp_split_page 1404817 ± 4% +213.6% 4405232 ± 9% numa-meminfo.node0.Active 1404733 ± 4% +213.6% 4405148 ± 9% numa-meminfo.node0.Active(anon) 828406 ± 12% +51.0% 1250665 ± 10% numa-meminfo.node0.Inactive 828190 ± 12% +50.7% 1247741 ± 10% numa-meminfo.node0.Inactive(anon) 214.75 ± 59% +1261.6% 2924 ± 10% numa-meminfo.node0.Inactive(file) 29574615 -11.6% 26131731 ± 2% numa-meminfo.node0.MemFree 3284371 ± 6% +104.8% 6727255 ± 8% numa-meminfo.node0.MemUsed 1476404 ± 11% +204.2% 4491068 ± 2% numa-meminfo.node1.Active 1476320 ± 11% +204.2% 4490984 ± 2% numa-meminfo.node1.Active(anon) 684877 ± 19% +87.8% 1286317 ± 9% numa-meminfo.node1.Inactive 684690 ± 19% +87.5% 1283519 ± 9% numa-meminfo.node1.Inactive(anon) 186.50 ± 65% +1400.3% 2798 ± 6% numa-meminfo.node1.Inactive(file) 29743730 -12.1% 26154353 numa-meminfo.node1.MemFree 3249412 ± 9% +110.5% 6838786 ± 3% numa-meminfo.node1.MemUsed 377498 ± 12% +193.5% 1108039 ± 9% numa-vmstat.node0.nr_active_anon 7374428 -11.6% 6515488 ± 2% numa-vmstat.node0.nr_free_pages 203915 ± 20% +57.8% 321729 ± 8% numa-vmstat.node0.nr_inactive_anon 55.25 ± 59% +1211.3% 724.50 ± 11% numa-vmstat.node0.nr_inactive_file 377455 ± 12% +193.5% 1108012 ± 9% numa-vmstat.node0.nr_zone_active_anon 203887 ± 20% +57.8% 321697 ± 8% numa-vmstat.node0.nr_zone_inactive_anon 55.25 ± 59% +1211.3% 724.50 ± 11% numa-vmstat.node0.nr_zone_inactive_file 392594 ± 12% +181.8% 1106200 ± 3% numa-vmstat.node1.nr_active_anon 7400171 -11.6% 6543657 numa-vmstat.node1.nr_free_pages 185991 ± 20% +78.6% 332154 ± 8% numa-vmstat.node1.nr_inactive_anon 44.75 ± 65% +1458.7% 697.50 ± 5% numa-vmstat.node1.nr_inactive_file 392558 ± 12% +181.8% 1106188 ± 3% numa-vmstat.node1.nr_zone_active_anon 185980 ± 20% +78.6% 332130 ± 8% numa-vmstat.node1.nr_zone_inactive_anon 44.75 ± 65% +1458.7% 697.50 ± 5% numa-vmstat.node1.nr_zone_inactive_file 92917 -2.7% 90421 interrupts.CAL:Function_call_interrupts 537.00 ± 10% +28.6% 690.50 ± 12% interrupts.CPU13.RES:Rescheduling_interrupts 574.25 ± 6% +44.4% 829.50 ± 29% interrupts.CPU17.RES:Rescheduling_interrupts 4162 ± 12% -22.4% 3231 ± 5% interrupts.CPU22.TLB:TLB_shootdowns 4051 ± 11% -20.9% 3203 ± 3% interrupts.CPU23.TLB:TLB_shootdowns 4017 ± 5% -17.4% 3318 ± 3% interrupts.CPU24.TLB:TLB_shootdowns 4249 ± 10% -22.7% 3283 ± 8% interrupts.CPU25.TLB:TLB_shootdowns 4003 ± 7% -17.8% 3291 ± 2% interrupts.CPU26.TLB:TLB_shootdowns 4161 ± 8% -19.8% 3336 ± 7% interrupts.CPU27.TLB:TLB_shootdowns 4063 ± 5% -17.9% 3337 ± 5% interrupts.CPU28.TLB:TLB_shootdowns 3970 ± 6% -17.6% 3271 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 4093 ± 7% -20.1% 3270 ± 3% interrupts.CPU30.TLB:TLB_shootdowns 4001 ± 7% -15.2% 3392 ± 9% interrupts.CPU31.TLB:TLB_shootdowns 4246 ± 7% -22.6% 3284 ± 3% interrupts.CPU32.TLB:TLB_shootdowns 3949 ± 6% -14.9% 3361 ± 6% interrupts.CPU33.TLB:TLB_shootdowns 3949 ± 5% -15.6% 3332 ± 8% interrupts.CPU34.TLB:TLB_shootdowns 4057 ± 9% -20.4% 3228 ± 3% interrupts.CPU35.TLB:TLB_shootdowns 4105 ± 7% -22.5% 3180 ± 8% interrupts.CPU36.TLB:TLB_shootdowns 3957 ± 7% -18.7% 3217 ± 7% interrupts.CPU37.TLB:TLB_shootdowns 4071 ± 8% -20.7% 3229 ± 4% interrupts.CPU38.TLB:TLB_shootdowns 4256 ± 18% -22.8% 3286 ± 4% interrupts.CPU39.TLB:TLB_shootdowns 4334 ± 2% -22.0% 3381 ± 6% interrupts.CPU40.TLB:TLB_shootdowns 3858 ± 5% -17.2% 3195 ± 7% interrupts.CPU42.TLB:TLB_shootdowns 4212 ± 9% -20.3% 3356 ± 3% interrupts.CPU43.TLB:TLB_shootdowns 3922 ± 9% -17.7% 3227 ± 4% interrupts.CPU66.TLB:TLB_shootdowns 4021 ± 3% -16.1% 3372 ± 5% interrupts.CPU67.TLB:TLB_shootdowns 4176 ± 4% -19.0% 3381 ± 4% interrupts.CPU68.TLB:TLB_shootdowns 4149 ± 5% -39.5% 2508 ± 57% interrupts.CPU69.TLB:TLB_shootdowns 3993 ± 8% -18.1% 3270 ± 3% interrupts.CPU70.TLB:TLB_shootdowns 4059 ± 4% -17.4% 3352 ± 4% interrupts.CPU71.TLB:TLB_shootdowns 514.50 ± 5% +49.7% 770.00 ± 24% interrupts.CPU72.RES:Rescheduling_interrupts 3994 ± 5% -20.9% 3158 ± 2% interrupts.CPU72.TLB:TLB_shootdowns 4034 ± 7% -17.1% 3342 ± 3% interrupts.CPU73.TLB:TLB_shootdowns 4018 ± 7% -15.1% 3412 ± 4% interrupts.CPU74.TLB:TLB_shootdowns 4052 ± 7% -14.6% 3459 ± 5% interrupts.CPU75.TLB:TLB_shootdowns 4060 ± 6% -19.6% 3263 ± 5% interrupts.CPU76.TLB:TLB_shootdowns 4025 ± 5% -20.1% 3217 ± 2% interrupts.CPU77.TLB:TLB_shootdowns 4071 ± 6% -21.5% 3197 ± 2% interrupts.CPU78.TLB:TLB_shootdowns 4045 ± 5% -17.7% 3330 ± 7% interrupts.CPU79.TLB:TLB_shootdowns 454.00 ± 36% +51.5% 688.00 ± 13% interrupts.CPU80.RES:Rescheduling_interrupts 3963 ± 5% -18.5% 3229 ± 5% interrupts.CPU81.TLB:TLB_shootdowns 4202 ± 5% -25.3% 3139 ± 5% interrupts.CPU82.TLB:TLB_shootdowns 3843 ± 3% -15.6% 3242 interrupts.CPU83.TLB:TLB_shootdowns 4169 ± 5% -40.3% 2489 ± 57% interrupts.CPU86.TLB:TLB_shootdowns 4062 ± 6% -17.6% 3345 ± 2% interrupts.CPU87.TLB:TLB_shootdowns *************************************************************************************************** vm-snb-4G: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G ========================================================================================= compiler/group/kconfig/rootfs/tbox_group/testcase: gcc-7/kselftests-02/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/vm-snb-4G/kernel_selftests commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 4:12 -25% 1:12 kmsg.unregister_netdevice:waiting_for_veth_A-R1_to_become_free.Usage_count= 1:12 -8% :12 kmsg.veth0:Failed_to_cycle_device_veth0;route_tables_might_be_wrong :12 100% 12:12 kernel_selftests.memfd.run_fuse_test.sh.fail :12 8% 1:12 kernel_selftests.net.ip_defrag.sh.fail %stddev %change %stddev \ | \ 1.00 -100.0% 0.00 kernel_selftests.memfd.run_fuse_test.sh.pass 361623 +223.2% 1168751 meminfo.Active 361623 +223.2% 1168751 meminfo.Active(anon) 127769 ± 21% -94.8% 6662 ± 26% meminfo.CmaFree 1941240 -41.3% 1139756 meminfo.MemAvailable 1895394 -42.4% 1091194 ± 2% meminfo.MemFree 2137756 +37.6% 2941952 meminfo.Memused 881.70 ± 5% +52.8% 1347 ± 4% meminfo.max_used_kB *************************************************************************************************** lkp-bdw-ep3: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: pipe/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep3/stress-ng/60s/0xb00002e commit: 9627026352 ("mm: page_cache_add_speculative(): refactoring") cdaa813278 ("mm/gup: track gup-pinned pages") 96270263521248d5 cdaa813278ddc616ee201eacda7 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 100% 4:4 dmesg.RIP:get_gup_pin_page :4 100% 4:4 dmesg.WARNING:at_mm/gup.c:#get_gup_pin_page %stddev %change %stddev \ | \ 360.22 -16.3% 301.59 stress-ng.time.elapsed_time 360.22 -16.3% 301.59 stress-ng.time.elapsed_time.max 8342 -1.3% 8234 stress-ng.time.percent_of_cpu_this_job_got 26890 -19.3% 21707 stress-ng.time.system_time 2.989e+08 -99.3% 2099213 stress-ng.vm-splice.ops 12010366 +10.0% 13211802 meminfo.Committed_AS 5812 ± 4% +18.0% 6856 ± 3% meminfo.max_used_kB 5.64 ± 2% +1.3 6.94 ± 2% mpstat.cpu.idle% 9.90 +1.8 11.68 ± 2% mpstat.cpu.usr% 107366 ± 93% -78.7% 22825 ± 44% numa-meminfo.node0.AnonPages 74249 ± 6% -13.8% 63969 ± 9% numa-meminfo.node0.SUnreclaim 5.90 ± 2% +22.8% 7.24 iostat.cpu.idle 84.23 -3.7% 81.11 iostat.cpu.system 9.87 +18.0% 11.64 ± 2% iostat.cpu.user 16463 +2.3% 16841 proc-vmstat.nr_kernel_stack 275921 ± 93% -50.8% 135706 ±150% proc-vmstat.numa_pte_updates 1000818 -14.3% 857522 proc-vmstat.pgfault 83.75 -3.6% 80.75 vmstat.cpu.sy 9.25 ± 4% +21.6% 11.25 ± 3% vmstat.cpu.us 1766858 +20.5% 2129192 vmstat.system.cs 185535 +1.5% 188310 vmstat.system.in 26840 ± 93% -78.8% 5697 ± 44% numa-vmstat.node0.nr_anon_pages 18563 ± 6% -13.8% 15992 ± 9% numa-vmstat.node0.nr_slab_unreclaimable 1.066e+09 ± 2% -28.2% 7.648e+08 ± 7% numa-vmstat.node0.numa_hit 1.066e+09 ± 2% -28.2% 7.648e+08 ± 7% numa-vmstat.node0.numa_local 9.301e+08 ± 2% -17.2% 7.701e+08 ± 6% numa-vmstat.node1.numa_hit 9.3e+08 ± 2% -17.2% 7.699e+08 ± 6% numa-vmstat.node1.numa_local 2.15 ± 2% -0.1 2.03 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.87 -0.1 0.81 perf-profile.calltrace.cycles-pp.avc_has_perm.file_has_perm.security_file_permission.vfs_read.ksys_read 1.35 -0.0 1.30 perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.83 -0.0 0.80 perf-profile.calltrace.cycles-pp.avc_has_perm.file_has_perm.security_file_permission.vfs_write.ksys_write 1.31 -0.0 1.27 perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 0.55 +0.0 0.57 perf-profile.calltrace.cycles-pp.__inode_security_revalidate.selinux_file_permission.security_file_permission.vfs_write.ksys_write 2.09 +0.1 2.15 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pipe_write.__vfs_write.vfs_write.ksys_write 2.62 ± 2% +0.1 2.68 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_write.__vfs_write.vfs_write.ksys_write 2650 -1.3% 2616 turbostat.Avg_MHz 3.85 ± 2% +0.7 4.55 turbostat.C1% 0.54 ± 7% +0.2 0.70 ± 12% turbostat.C1E% 4.71 ± 2% +21.7% 5.73 turbostat.CPU%c1 77.50 ± 2% -14.2% 66.50 ± 2% turbostat.CoreTmp 67302470 -14.9% 57247592 turbostat.IRQ 81.75 ± 2% -13.1% 71.00 turbostat.PkgTmp 261.28 +3.9% 271.39 turbostat.PkgWatt 1285315 ± 8% +15.7% 1487427 ± 6% sched_debug.cfs_rq:/.MIN_vruntime.stddev 173175 -16.1% 145379 sched_debug.cfs_rq:/.exec_clock.avg 174188 -16.1% 146168 sched_debug.cfs_rq:/.exec_clock.max 171345 -16.0% 143970 sched_debug.cfs_rq:/.exec_clock.min 5689 ± 7% -27.5% 4124 ± 10% sched_debug.cfs_rq:/.load.min 168.54 ± 6% +12.3% 189.25 ± 6% sched_debug.cfs_rq:/.load_avg.max 1285315 ± 8% +15.7% 1487427 ± 6% sched_debug.cfs_rq:/.max_vruntime.stddev 16247293 -15.8% 13678528 sched_debug.cfs_rq:/.min_vruntime.avg 17621646 ± 2% -13.2% 15289779 ± 3% sched_debug.cfs_rq:/.min_vruntime.max 14916399 -18.2% 12204755 ± 3% sched_debug.cfs_rq:/.min_vruntime.min 0.13 ± 6% +21.7% 0.16 ± 8% sched_debug.cfs_rq:/.nr_running.stddev 4.96 ± 4% -20.3% 3.96 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.min 5426 ± 8% -19.0% 4397 ± 11% sched_debug.cfs_rq:/.runnable_weight.min 10113 ± 21% -48.9% 5166 ± 13% sched_debug.cpu.avg_idle.min 222767 -13.4% 192825 sched_debug.cpu.clock.avg 222785 -13.4% 192841 sched_debug.cpu.clock.max 222745 -13.4% 192807 sched_debug.cpu.clock.min 11.90 ± 12% -17.7% 9.80 ± 12% sched_debug.cpu.clock.stddev 222767 -13.4% 192825 sched_debug.cpu.clock_task.avg 222785 -13.4% 192841 sched_debug.cpu.clock_task.max 222745 -13.4% 192807 sched_debug.cpu.clock_task.min 11.90 ± 12% -17.7% 9.80 ± 12% sched_debug.cpu.clock_task.stddev 5.21 ± 3% -12.9% 4.54 ± 3% sched_debug.cpu.cpu_load[0].min 4955 -19.3% 4001 sched_debug.cpu.curr->pid.avg 6909 -11.6% 6108 sched_debug.cpu.curr->pid.max 11634 ± 7% -10.9% 10363 sched_debug.cpu.load.avg 193768 -15.5% 163653 sched_debug.cpu.nr_load_updates.avg 197917 -15.6% 167117 sched_debug.cpu.nr_load_updates.max 191959 -15.6% 162097 sched_debug.cpu.nr_load_updates.min 0.50 ± 6% +10.8% 0.56 ± 6% sched_debug.cpu.nr_running.stddev 4644436 ± 2% -7.8% 4280609 ± 2% sched_debug.cpu.nr_switches.avg 4644663 ± 2% -7.8% 4280771 ± 2% sched_debug.cpu.sched_count.avg 409748 ± 4% -22.5% 317516 ± 3% sched_debug.cpu.sched_goidle.avg 572470 ± 11% -22.4% 444043 ± 13% sched_debug.cpu.sched_goidle.max 286861 ± 7% -22.8% 221401 ± 8% sched_debug.cpu.sched_goidle.min 222744 -13.4% 192807 sched_debug.cpu_clk 219014 -13.7% 189076 sched_debug.ktime 223427 -13.4% 193488 sched_debug.sched_clk 14.20 +15.8% 16.44 perf-stat.i.MPKI 3.166e+10 +17.1% 3.708e+10 perf-stat.i.branch-instructions 1.57 +0.0 1.59 perf-stat.i.branch-miss-rate% 4.847e+08 +16.6% 5.653e+08 ± 2% perf-stat.i.branch-misses 12.07 ± 23% -7.1 4.97 ± 55% perf-stat.i.cache-miss-rate% 9.997e+08 +18.7% 1.187e+09 perf-stat.i.cache-references 1782714 +20.5% 2148963 perf-stat.i.context-switches 4.01 -60.1% 1.60 perf-stat.i.cpi 269394 ± 5% +18.6% 319628 ± 2% perf-stat.i.cpu-migrations 0.19 ± 3% +0.0 0.22 perf-stat.i.dTLB-load-miss-rate% 47856194 ± 3% +19.6% 57235440 perf-stat.i.dTLB-load-misses 3.932e+10 +16.8% 4.592e+10 perf-stat.i.dTLB-loads 2.459e+10 +16.7% 2.871e+10 perf-stat.i.dTLB-stores 72.03 +1.3 73.35 perf-stat.i.iTLB-load-miss-rate% 92906511 ± 2% +16.5% 1.083e+08 ± 3% perf-stat.i.iTLB-load-misses 35482612 +12.6% 39969391 ± 2% perf-stat.i.iTLB-loads 1.521e+11 +17.0% 1.779e+11 perf-stat.i.instructions 28571 ± 3% +14.4% 32688 ± 6% perf-stat.i.instructions-per-iTLB-miss 0.64 +16.9% 0.75 perf-stat.i.ipc 2696 +1.9% 2746 perf-stat.i.minor-faults 115872 ± 4% +33.3% 154453 ± 11% perf-stat.i.node-loads 69.92 ± 5% -6.0 63.95 ± 2% perf-stat.i.node-store-miss-rate% 2696 +1.9% 2746 perf-stat.i.page-faults 1.53 -15.3% 1.30 perf-stat.overall.cpi 0.65 +18.1% 0.77 perf-stat.overall.ipc 3.153e+10 +16.9% 3.684e+10 perf-stat.ps.branch-instructions 4.827e+08 +16.3% 5.616e+08 ± 2% perf-stat.ps.branch-misses 9.972e+08 +18.4% 1.181e+09 perf-stat.ps.cache-references 1778082 +20.3% 2138597 perf-stat.ps.context-switches 2.319e+11 -1.2% 2.292e+11 perf-stat.ps.cpu-cycles 268859 ± 5% +18.4% 318278 ± 2% perf-stat.ps.cpu-migrations 47762824 ± 3% +19.4% 57005870 perf-stat.ps.dTLB-load-misses 3.916e+10 +16.5% 4.563e+10 perf-stat.ps.dTLB-loads 2.449e+10 +16.5% 2.852e+10 perf-stat.ps.dTLB-stores 92569828 ± 2% +16.2% 1.076e+08 ± 3% perf-stat.ps.iTLB-load-misses 35350893 +12.4% 39724157 ± 2% perf-stat.ps.iTLB-loads 1.515e+11 +16.7% 1.767e+11 perf-stat.ps.instructions 2690 +1.6% 2733 perf-stat.ps.minor-faults 115582 ± 4% +33.0% 153669 ± 11% perf-stat.ps.node-loads 2690 +1.6% 2733 perf-stat.ps.page-faults 5.463e+13 -2.2% 5.344e+13 perf-stat.total.instructions 423.00 -16.0% 355.50 interrupts.9:IO-APIC.9-fasteoi.acpi 334285 -13.8% 288189 interrupts.CAL:Function_call_interrupts 3718 ± 5% -11.2% 3301 interrupts.CPU0.CAL:Function_call_interrupts 721763 -16.2% 605183 interrupts.CPU0.LOC:Local_timer_interrupts 25452 ± 12% +29.7% 33009 ± 11% interrupts.CPU0.RES:Rescheduling_interrupts 423.00 -16.0% 355.50 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 3841 -13.6% 3317 interrupts.CPU1.CAL:Function_call_interrupts 721450 -16.2% 604748 interrupts.CPU1.LOC:Local_timer_interrupts 3776 ± 2% -12.2% 3317 interrupts.CPU10.CAL:Function_call_interrupts 722124 -16.2% 604931 interrupts.CPU10.LOC:Local_timer_interrupts 22581 ± 13% +44.5% 32630 ± 25% interrupts.CPU10.RES:Rescheduling_interrupts 3835 -13.4% 3321 interrupts.CPU11.CAL:Function_call_interrupts 721997 -16.2% 605063 interrupts.CPU11.LOC:Local_timer_interrupts 21969 ± 12% +39.6% 30662 ± 10% interrupts.CPU11.RES:Rescheduling_interrupts 3828 -12.7% 3340 interrupts.CPU12.CAL:Function_call_interrupts 721760 -16.2% 604828 interrupts.CPU12.LOC:Local_timer_interrupts 3790 -12.8% 3303 interrupts.CPU13.CAL:Function_call_interrupts 721659 -16.2% 604978 interrupts.CPU13.LOC:Local_timer_interrupts 3796 -12.6% 3317 interrupts.CPU14.CAL:Function_call_interrupts 721830 -16.2% 605051 interrupts.CPU14.LOC:Local_timer_interrupts 3745 ± 2% -11.6% 3313 interrupts.CPU15.CAL:Function_call_interrupts 721696 -16.1% 605247 interrupts.CPU15.LOC:Local_timer_interrupts 3796 ± 2% -13.1% 3298 interrupts.CPU16.CAL:Function_call_interrupts 721615 -16.1% 605304 interrupts.CPU16.LOC:Local_timer_interrupts 721686 -16.2% 604977 interrupts.CPU17.LOC:Local_timer_interrupts 23557 ± 11% +31.8% 31037 ± 17% interrupts.CPU17.RES:Rescheduling_interrupts 3841 -13.9% 3308 interrupts.CPU18.CAL:Function_call_interrupts 721620 -16.2% 604958 interrupts.CPU18.LOC:Local_timer_interrupts 22764 ± 17% +33.8% 30451 ± 8% interrupts.CPU18.RES:Rescheduling_interrupts 3790 ± 2% -12.8% 3304 interrupts.CPU19.CAL:Function_call_interrupts 721479 -16.1% 604999 interrupts.CPU19.LOC:Local_timer_interrupts 3845 -14.0% 3307 interrupts.CPU2.CAL:Function_call_interrupts 721284 -16.1% 605094 interrupts.CPU2.LOC:Local_timer_interrupts 22880 ± 19% +42.0% 32484 ± 12% interrupts.CPU2.RES:Rescheduling_interrupts 721360 -16.1% 604946 interrupts.CPU20.LOC:Local_timer_interrupts 21116 ± 18% +39.1% 29366 ± 17% interrupts.CPU20.RES:Rescheduling_interrupts 3811 -13.5% 3297 interrupts.CPU21.CAL:Function_call_interrupts 721473 -16.2% 604796 interrupts.CPU21.LOC:Local_timer_interrupts 3827 -15.3% 3241 ± 3% interrupts.CPU22.CAL:Function_call_interrupts 721210 -16.2% 604237 interrupts.CPU22.LOC:Local_timer_interrupts 721987 -16.2% 604725 interrupts.CPU23.LOC:Local_timer_interrupts 3821 -19.4% 3081 ± 6% interrupts.CPU24.CAL:Function_call_interrupts 721544 -16.2% 604685 interrupts.CPU24.LOC:Local_timer_interrupts 3786 -15.6% 3194 ± 3% interrupts.CPU25.CAL:Function_call_interrupts 721135 -16.1% 604853 interrupts.CPU25.LOC:Local_timer_interrupts 3827 -15.2% 3245 ± 2% interrupts.CPU26.CAL:Function_call_interrupts 721813 -16.2% 605191 interrupts.CPU26.LOC:Local_timer_interrupts 3821 -25.4% 2851 ± 26% interrupts.CPU27.CAL:Function_call_interrupts 721056 -16.1% 605078 interrupts.CPU27.LOC:Local_timer_interrupts 3818 -15.1% 3241 ± 3% interrupts.CPU28.CAL:Function_call_interrupts 721295 -16.1% 604867 interrupts.CPU28.LOC:Local_timer_interrupts 3839 -14.2% 3293 interrupts.CPU29.CAL:Function_call_interrupts 721791 -16.2% 604635 interrupts.CPU29.LOC:Local_timer_interrupts 3818 -13.1% 3318 interrupts.CPU3.CAL:Function_call_interrupts 721555 -16.2% 604385 interrupts.CPU3.LOC:Local_timer_interrupts 22807 ± 12% +32.9% 30316 ± 13% interrupts.CPU3.RES:Rescheduling_interrupts 3829 -14.3% 3280 interrupts.CPU30.CAL:Function_call_interrupts 721877 -16.2% 604644 interrupts.CPU30.LOC:Local_timer_interrupts 29149 ± 5% -15.8% 24552 ± 10% interrupts.CPU30.RES:Rescheduling_interrupts 3838 -13.9% 3304 interrupts.CPU31.CAL:Function_call_interrupts 721927 -16.2% 604849 interrupts.CPU31.LOC:Local_timer_interrupts 3836 -14.2% 3290 interrupts.CPU32.CAL:Function_call_interrupts 720992 -16.1% 604786 interrupts.CPU32.LOC:Local_timer_interrupts 3826 -14.0% 3292 interrupts.CPU33.CAL:Function_call_interrupts 721853 -16.2% 604700 interrupts.CPU33.LOC:Local_timer_interrupts 3809 -13.8% 3283 interrupts.CPU34.CAL:Function_call_interrupts 721742 -16.2% 605038 interrupts.CPU34.LOC:Local_timer_interrupts 3833 -14.0% 3296 interrupts.CPU35.CAL:Function_call_interrupts 721732 -16.2% 604841 interrupts.CPU35.LOC:Local_timer_interrupts 3700 ± 6% -12.7% 3229 ± 3% interrupts.CPU36.CAL:Function_call_interrupts 721598 -16.2% 604849 interrupts.CPU36.LOC:Local_timer_interrupts 3835 -14.1% 3293 interrupts.CPU37.CAL:Function_call_interrupts 721245 -16.2% 604498 interrupts.CPU37.LOC:Local_timer_interrupts 3831 -13.8% 3301 interrupts.CPU38.CAL:Function_call_interrupts 721366 -16.1% 605049 interrupts.CPU38.LOC:Local_timer_interrupts 3843 -15.3% 3253 ± 2% interrupts.CPU39.CAL:Function_call_interrupts 721951 -16.2% 604693 interrupts.CPU39.LOC:Local_timer_interrupts 3823 -13.2% 3316 interrupts.CPU4.CAL:Function_call_interrupts 721523 -16.2% 604953 interrupts.CPU4.LOC:Local_timer_interrupts 22707 ± 16% +33.2% 30241 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 3840 -14.2% 3294 interrupts.CPU40.CAL:Function_call_interrupts 721636 -16.2% 604810 interrupts.CPU40.LOC:Local_timer_interrupts 3847 -18.8% 3122 ± 9% interrupts.CPU41.CAL:Function_call_interrupts 721928 -16.2% 604884 interrupts.CPU41.LOC:Local_timer_interrupts 3839 -14.3% 3290 interrupts.CPU42.CAL:Function_call_interrupts 721753 -16.2% 604811 interrupts.CPU42.LOC:Local_timer_interrupts 3778 -13.8% 3255 interrupts.CPU43.CAL:Function_call_interrupts 721315 -16.2% 604691 interrupts.CPU43.LOC:Local_timer_interrupts 3841 -13.7% 3314 interrupts.CPU44.CAL:Function_call_interrupts 721306 -16.2% 604640 interrupts.CPU44.LOC:Local_timer_interrupts 22357 ± 16% +47.9% 33078 ± 15% interrupts.CPU44.RES:Rescheduling_interrupts 3832 -15.0% 3256 ± 3% interrupts.CPU45.CAL:Function_call_interrupts 721470 -16.2% 604760 interrupts.CPU45.LOC:Local_timer_interrupts 3846 -18.1% 3149 ± 9% interrupts.CPU46.CAL:Function_call_interrupts 721294 -16.2% 604606 interrupts.CPU46.LOC:Local_timer_interrupts 22430 ± 19% +39.5% 31283 ± 10% interrupts.CPU46.RES:Rescheduling_interrupts 3795 -12.5% 3320 interrupts.CPU47.CAL:Function_call_interrupts 721274 -16.2% 604302 interrupts.CPU47.LOC:Local_timer_interrupts 22255 ± 13% +29.9% 28907 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts 3841 -14.2% 3295 interrupts.CPU48.CAL:Function_call_interrupts 721357 -16.2% 604782 interrupts.CPU48.LOC:Local_timer_interrupts 20959 ± 17% +36.5% 28600 ± 7% interrupts.CPU48.RES:Rescheduling_interrupts 3835 -13.3% 3324 interrupts.CPU49.CAL:Function_call_interrupts 721311 -16.1% 605127 interrupts.CPU49.LOC:Local_timer_interrupts 22089 ± 15% +33.1% 29406 ± 11% interrupts.CPU49.RES:Rescheduling_interrupts 3842 -14.0% 3305 interrupts.CPU5.CAL:Function_call_interrupts 721561 -16.1% 605092 interrupts.CPU5.LOC:Local_timer_interrupts 24272 ± 14% +25.3% 30413 ± 11% interrupts.CPU5.RES:Rescheduling_interrupts 3848 -13.7% 3319 interrupts.CPU50.CAL:Function_call_interrupts 721251 -16.1% 604921 interrupts.CPU50.LOC:Local_timer_interrupts 3827 -13.3% 3318 interrupts.CPU51.CAL:Function_call_interrupts 721474 -16.2% 604842 interrupts.CPU51.LOC:Local_timer_interrupts 3812 -12.5% 3337 interrupts.CPU52.CAL:Function_call_interrupts 721284 -16.1% 604942 interrupts.CPU52.LOC:Local_timer_interrupts 22236 ± 10% +33.3% 29642 ± 8% interrupts.CPU52.RES:Rescheduling_interrupts 721452 -16.1% 605377 interrupts.CPU53.LOC:Local_timer_interrupts 21926 ± 15% +38.2% 30305 ± 12% interrupts.CPU53.RES:Rescheduling_interrupts 721468 -16.1% 605249 interrupts.CPU54.LOC:Local_timer_interrupts 22522 ± 13% +36.4% 30717 ± 17% interrupts.CPU54.RES:Rescheduling_interrupts 3842 -13.9% 3307 interrupts.CPU55.CAL:Function_call_interrupts 721620 -16.2% 604986 interrupts.CPU55.LOC:Local_timer_interrupts 22201 ± 16% +31.7% 29239 ± 11% interrupts.CPU55.RES:Rescheduling_interrupts 3811 -13.3% 3303 interrupts.CPU56.CAL:Function_call_interrupts 721916 -16.2% 605011 interrupts.CPU56.LOC:Local_timer_interrupts 721636 -16.2% 604961 interrupts.CPU57.LOC:Local_timer_interrupts 3841 -13.9% 3309 interrupts.CPU58.CAL:Function_call_interrupts 721637 -16.1% 605154 interrupts.CPU58.LOC:Local_timer_interrupts 3795 -12.7% 3314 interrupts.CPU59.CAL:Function_call_interrupts 721926 -16.1% 605366 interrupts.CPU59.LOC:Local_timer_interrupts 3841 -17.0% 3189 ± 7% interrupts.CPU6.CAL:Function_call_interrupts 721537 -16.1% 605025 interrupts.CPU6.LOC:Local_timer_interrupts 21627 ± 16% +36.9% 29599 ± 15% interrupts.CPU6.RES:Rescheduling_interrupts 3839 -13.5% 3319 interrupts.CPU60.CAL:Function_call_interrupts 721689 -16.2% 604938 interrupts.CPU60.LOC:Local_timer_interrupts 22724 ± 12% +31.4% 29852 ± 13% interrupts.CPU60.RES:Rescheduling_interrupts 721538 -16.1% 605018 interrupts.CPU61.LOC:Local_timer_interrupts 22521 ± 15% +33.2% 29995 ± 19% interrupts.CPU61.RES:Rescheduling_interrupts 3795 ± 2% -12.7% 3313 interrupts.CPU62.CAL:Function_call_interrupts 721565 -16.2% 604946 interrupts.CPU62.LOC:Local_timer_interrupts 22054 ± 17% +38.7% 30598 ± 11% interrupts.CPU62.RES:Rescheduling_interrupts 721884 -16.2% 605030 interrupts.CPU63.LOC:Local_timer_interrupts 3772 ± 3% -12.3% 3309 interrupts.CPU64.CAL:Function_call_interrupts 721981 -16.2% 605040 interrupts.CPU64.LOC:Local_timer_interrupts 21437 ± 20% +33.6% 28639 ± 19% interrupts.CPU64.RES:Rescheduling_interrupts 3779 -19.3% 3051 ± 14% interrupts.CPU65.CAL:Function_call_interrupts 721390 -16.1% 605132 interrupts.CPU65.LOC:Local_timer_interrupts 3825 -13.5% 3309 interrupts.CPU66.CAL:Function_call_interrupts 722064 -16.3% 604606 interrupts.CPU66.LOC:Local_timer_interrupts 3819 -26.5% 2808 ± 24% interrupts.CPU67.CAL:Function_call_interrupts 721439 -16.2% 604744 interrupts.CPU67.LOC:Local_timer_interrupts 3821 -13.5% 3306 interrupts.CPU68.CAL:Function_call_interrupts 722060 -16.3% 604673 interrupts.CPU68.LOC:Local_timer_interrupts 3821 -13.7% 3297 interrupts.CPU69.CAL:Function_call_interrupts 722106 -16.3% 604542 interrupts.CPU69.LOC:Local_timer_interrupts 3842 -14.2% 3297 interrupts.CPU7.CAL:Function_call_interrupts 721363 -16.1% 605295 interrupts.CPU7.LOC:Local_timer_interrupts 22221 ± 16% +40.9% 31317 ± 19% interrupts.CPU7.RES:Rescheduling_interrupts 3826 -13.6% 3305 interrupts.CPU70.CAL:Function_call_interrupts 721764 -16.2% 604627 interrupts.CPU70.LOC:Local_timer_interrupts 3809 -13.5% 3295 interrupts.CPU71.CAL:Function_call_interrupts 722080 -16.2% 604781 interrupts.CPU71.LOC:Local_timer_interrupts 3818 -14.2% 3274 ± 2% interrupts.CPU72.CAL:Function_call_interrupts 721916 -16.2% 604918 interrupts.CPU72.LOC:Local_timer_interrupts 27835 ± 14% -17.6% 22945 ± 10% interrupts.CPU72.RES:Rescheduling_interrupts 3813 -13.2% 3308 interrupts.CPU73.CAL:Function_call_interrupts 721747 -16.2% 604707 interrupts.CPU73.LOC:Local_timer_interrupts 721746 -16.2% 604838 interrupts.CPU74.LOC:Local_timer_interrupts 3832 -14.7% 3269 ± 2% interrupts.CPU75.CAL:Function_call_interrupts 721781 -16.2% 604619 interrupts.CPU75.LOC:Local_timer_interrupts 3779 -12.4% 3310 interrupts.CPU76.CAL:Function_call_interrupts 720970 -16.1% 604618 interrupts.CPU76.LOC:Local_timer_interrupts 3809 ± 2% -15.7% 3211 ± 4% interrupts.CPU77.CAL:Function_call_interrupts 722062 -16.3% 604188 interrupts.CPU77.LOC:Local_timer_interrupts 3843 -14.6% 3284 ± 2% interrupts.CPU78.CAL:Function_call_interrupts 721715 -16.2% 604636 interrupts.CPU78.LOC:Local_timer_interrupts 3840 -13.9% 3306 interrupts.CPU79.CAL:Function_call_interrupts 721613 -16.2% 604526 interrupts.CPU79.LOC:Local_timer_interrupts 3763 ± 4% -11.5% 3331 interrupts.CPU8.CAL:Function_call_interrupts 721367 -16.1% 605107 interrupts.CPU8.LOC:Local_timer_interrupts 22471 ± 17% +38.2% 31048 ± 9% interrupts.CPU8.RES:Rescheduling_interrupts 3830 -13.9% 3298 interrupts.CPU80.CAL:Function_call_interrupts 721678 -16.2% 604714 interrupts.CPU80.LOC:Local_timer_interrupts 3776 ± 3% -12.5% 3306 interrupts.CPU81.CAL:Function_call_interrupts 721263 -16.1% 604803 interrupts.CPU81.LOC:Local_timer_interrupts 3837 -13.9% 3305 interrupts.CPU82.CAL:Function_call_interrupts 721961 -16.2% 604781 interrupts.CPU82.LOC:Local_timer_interrupts 3849 -14.8% 3280 interrupts.CPU83.CAL:Function_call_interrupts 721888 -16.2% 604619 interrupts.CPU83.LOC:Local_timer_interrupts 3802 -13.2% 3300 interrupts.CPU84.CAL:Function_call_interrupts 721937 -16.2% 604653 interrupts.CPU84.LOC:Local_timer_interrupts 3843 -13.8% 3313 interrupts.CPU85.CAL:Function_call_interrupts 721834 -16.2% 604711 interrupts.CPU85.LOC:Local_timer_interrupts 3839 -14.3% 3291 interrupts.CPU86.CAL:Function_call_interrupts 721871 -16.2% 604809 interrupts.CPU86.LOC:Local_timer_interrupts 3847 -14.6% 3287 interrupts.CPU87.CAL:Function_call_interrupts 722149 -16.3% 604789 interrupts.CPU87.LOC:Local_timer_interrupts 3838 -14.1% 3298 interrupts.CPU9.CAL:Function_call_interrupts 721253 -16.2% 604595 interrupts.CPU9.LOC:Local_timer_interrupts 22535 ± 19% +36.1% 30680 ± 11% interrupts.CPU9.RES:Rescheduling_interrupts 63503101 -16.2% 53227032 interrupts.LOC:Local_timer_interrupts 310.00 ± 3% +11.0% 344.00 ± 7% interrupts.TLB:TLB_shootdowns 20431 ± 15% -19.7% 16415 ± 4% softirqs.CPU0.RCU 34582 ± 2% +17.5% 40639 ± 8% softirqs.CPU0.SCHED 133704 -12.9% 116517 ± 3% softirqs.CPU0.TIMER 20017 ± 16% -18.6% 16300 softirqs.CPU1.RCU 32010 ± 3% +17.3% 37561 ± 8% softirqs.CPU1.SCHED 132385 ± 2% -14.8% 112812 softirqs.CPU1.TIMER 19743 ± 16% -20.2% 15764 ± 3% softirqs.CPU10.RCU 30881 ± 3% +20.6% 37253 ± 10% softirqs.CPU10.SCHED 130801 ± 2% -8.5% 119711 ± 6% softirqs.CPU10.TIMER 30607 ± 2% +20.4% 36850 ± 8% softirqs.CPU11.SCHED 133304 ± 2% -11.9% 117463 ± 2% softirqs.CPU11.TIMER 30848 +20.1% 37052 ± 9% softirqs.CPU12.SCHED 131208 -11.5% 116174 ± 3% softirqs.CPU12.TIMER 22042 ± 17% -28.7% 15719 ± 5% softirqs.CPU13.RCU 30520 ± 5% +21.7% 37133 ± 11% softirqs.CPU13.SCHED 135598 ± 4% -15.4% 114708 ± 2% softirqs.CPU13.TIMER 31132 ± 4% +15.4% 35911 ± 10% softirqs.CPU14.SCHED 148384 ± 17% -22.8% 114512 softirqs.CPU14.TIMER 30303 +20.6% 36560 ± 10% softirqs.CPU15.SCHED 133852 -14.1% 115008 softirqs.CPU15.TIMER 31199 ± 2% +18.5% 36961 ± 8% softirqs.CPU16.SCHED 133767 ± 2% -13.1% 116191 softirqs.CPU16.TIMER 19872 ± 17% -17.2% 16453 ± 2% softirqs.CPU17.RCU 30661 ± 2% +19.2% 36554 ± 9% softirqs.CPU17.SCHED 133274 ± 2% -15.1% 113122 ± 2% softirqs.CPU17.TIMER 20088 ± 15% -17.5% 16582 ± 2% softirqs.CPU18.RCU 31180 ± 3% +15.6% 36055 ± 9% softirqs.CPU18.SCHED 134796 ± 4% -15.7% 113667 softirqs.CPU18.TIMER 22481 ± 20% -26.8% 16451 ± 2% softirqs.CPU19.RCU 30809 ± 2% +19.5% 36811 ± 7% softirqs.CPU19.SCHED 132271 ± 2% -12.4% 115843 ± 3% softirqs.CPU19.TIMER 19797 ± 16% -17.5% 16326 ± 2% softirqs.CPU2.RCU 30922 ± 3% +21.6% 37612 ± 8% softirqs.CPU2.SCHED 25248 ± 29% -24.3% 19119 ± 24% softirqs.CPU20.RCU 29985 ± 2% +22.1% 36608 ± 8% softirqs.CPU20.SCHED 128772 ± 2% -10.9% 114692 softirqs.CPU20.TIMER 19935 ± 17% -19.2% 16110 ± 3% softirqs.CPU21.RCU 31495 ± 3% +16.4% 36651 ± 7% softirqs.CPU21.SCHED 130837 ± 3% -13.2% 113573 ± 2% softirqs.CPU21.TIMER 40257 ± 5% -13.7% 34739 ± 7% softirqs.CPU22.SCHED 133177 ± 2% -16.6% 111009 ± 3% softirqs.CPU22.TIMER 40524 ± 5% -17.6% 33383 ± 9% softirqs.CPU23.SCHED 133547 ± 4% -18.1% 109369 softirqs.CPU23.TIMER 39849 ± 3% -15.3% 33755 ± 9% softirqs.CPU24.SCHED 131526 ± 2% -17.7% 108249 ± 2% softirqs.CPU24.TIMER 39996 ± 4% -13.6% 34576 ± 6% softirqs.CPU25.SCHED 131627 ± 3% -16.1% 110442 ± 3% softirqs.CPU25.TIMER 39898 ± 5% -14.9% 33947 ± 7% softirqs.CPU26.SCHED 132681 -16.5% 110841 ± 4% softirqs.CPU26.TIMER 40079 ± 7% -15.6% 33826 ± 6% softirqs.CPU27.SCHED 133577 -18.1% 109462 ± 2% softirqs.CPU27.TIMER 40228 ± 5% -15.4% 34036 ± 6% softirqs.CPU28.SCHED 131075 ± 3% -17.9% 107666 ± 2% softirqs.CPU28.TIMER 39312 ± 3% -14.8% 33499 ± 8% softirqs.CPU29.SCHED 134423 ± 3% -18.4% 109702 ± 3% softirqs.CPU29.TIMER 19384 ± 17% -17.8% 15929 softirqs.CPU3.RCU 30436 ± 3% +21.2% 36897 ± 11% softirqs.CPU3.SCHED 132319 ± 3% -13.4% 114525 ± 2% softirqs.CPU3.TIMER 22908 ± 16% -22.4% 17781 ± 3% softirqs.CPU30.RCU 39762 ± 3% -15.0% 33803 ± 8% softirqs.CPU30.SCHED 134269 ± 2% -17.7% 110531 ± 2% softirqs.CPU30.TIMER 41024 ± 5% -17.4% 33894 ± 7% softirqs.CPU31.SCHED 149024 ± 17% -24.6% 112375 ± 3% softirqs.CPU31.TIMER 39978 ± 4% -15.5% 33774 ± 7% softirqs.CPU32.SCHED 131148 -17.2% 108564 ± 3% softirqs.CPU32.TIMER 39979 ± 4% -13.8% 34476 ± 9% softirqs.CPU33.SCHED 134094 ± 4% -18.1% 109836 ± 2% softirqs.CPU33.TIMER 22786 ± 16% -22.9% 17569 ± 4% softirqs.CPU34.RCU 40552 ± 4% -17.3% 33554 ± 7% softirqs.CPU34.SCHED 131088 ± 2% -17.0% 108798 ± 3% softirqs.CPU34.TIMER 39532 ± 3% -14.1% 33963 ± 7% softirqs.CPU35.SCHED 132211 ± 2% -15.9% 111137 ± 3% softirqs.CPU35.TIMER 38671 ± 4% -13.2% 33557 ± 9% softirqs.CPU36.SCHED 147868 ± 17% -27.0% 107982 softirqs.CPU36.TIMER 23249 ± 15% -22.1% 18100 ± 5% softirqs.CPU37.RCU 40190 ± 5% -15.3% 34023 ± 7% softirqs.CPU37.SCHED 132884 ± 3% -15.9% 111703 ± 2% softirqs.CPU37.TIMER 22745 ± 16% -23.2% 17469 ± 4% softirqs.CPU38.RCU 40569 ± 5% -16.3% 33950 ± 9% softirqs.CPU38.SCHED 135133 -18.3% 110441 ± 2% softirqs.CPU38.TIMER 39935 ± 5% -15.3% 33825 ± 8% softirqs.CPU39.SCHED 133753 ± 2% -17.5% 110412 softirqs.CPU39.TIMER 30946 ± 4% +17.7% 36409 ± 9% softirqs.CPU4.SCHED 130762 ± 2% -13.6% 113024 ± 2% softirqs.CPU4.TIMER 40335 ± 3% -16.4% 33703 ± 9% softirqs.CPU40.SCHED 130983 ± 2% -15.6% 110597 ± 2% softirqs.CPU40.TIMER 40423 ± 5% -17.1% 33525 ± 8% softirqs.CPU41.SCHED 132224 ± 4% -18.0% 108363 softirqs.CPU41.TIMER 40723 ± 4% -17.3% 33682 ± 7% softirqs.CPU42.SCHED 135049 -17.8% 110952 ± 3% softirqs.CPU42.TIMER 39349 ± 3% -14.8% 33536 ± 8% softirqs.CPU43.SCHED 134203 ± 2% -17.6% 110610 ± 2% softirqs.CPU43.TIMER 17084 ± 18% -18.0% 14000 ± 3% softirqs.CPU44.RCU 30946 +22.1% 37787 ± 9% softirqs.CPU44.SCHED 130568 -13.3% 113197 ± 3% softirqs.CPU44.TIMER 20252 ± 15% -19.8% 16238 ± 2% softirqs.CPU45.RCU 31638 ± 4% +16.3% 36780 ± 9% softirqs.CPU45.SCHED 132331 ± 2% -14.9% 112550 softirqs.CPU45.TIMER 24973 ± 13% -16.0% 20983 ± 2% softirqs.CPU46.RCU 30862 +22.1% 37672 ± 6% softirqs.CPU46.SCHED 137074 ± 3% -11.2% 121691 ± 4% softirqs.CPU46.TIMER 19750 ± 17% -18.6% 16079 ± 2% softirqs.CPU47.RCU 30289 ± 3% +21.4% 36762 ± 9% softirqs.CPU47.SCHED 130548 ± 3% -12.7% 113914 softirqs.CPU47.TIMER 19693 ± 16% -17.0% 16336 ± 4% softirqs.CPU48.RCU 30593 ± 5% +19.9% 36684 ± 9% softirqs.CPU48.SCHED 130499 ± 3% -13.6% 112753 ± 2% softirqs.CPU48.TIMER 30877 ± 2% +19.9% 37018 ± 9% softirqs.CPU49.SCHED 130875 ± 4% -13.0% 113833 softirqs.CPU49.TIMER 22365 ± 36% -27.9% 16131 softirqs.CPU5.RCU 30822 +20.2% 37048 ± 9% softirqs.CPU5.SCHED 130397 ± 3% -11.8% 115024 softirqs.CPU5.TIMER 22536 ± 37% -26.9% 16472 ± 4% softirqs.CPU50.RCU 30543 +20.3% 36752 ± 9% softirqs.CPU50.SCHED 133918 ± 3% -13.5% 115860 ± 4% softirqs.CPU50.TIMER 131702 ± 2% -9.6% 118995 ± 6% softirqs.CPU51.TIMER 20176 ± 17% -21.1% 15915 ± 3% softirqs.CPU52.RCU 30631 ± 3% +22.2% 37419 ± 7% softirqs.CPU52.SCHED 131629 -13.5% 113828 softirqs.CPU52.TIMER 31078 +21.4% 37736 ± 9% softirqs.CPU53.SCHED 133106 ± 3% -14.2% 114257 ± 3% softirqs.CPU53.TIMER 30982 ± 4% +19.3% 36962 ± 9% softirqs.CPU54.SCHED 25187 ± 31% -24.8% 18932 ± 27% softirqs.CPU55.RCU 30622 ± 3% +19.6% 36613 ± 9% softirqs.CPU55.SCHED 19796 ± 14% -18.3% 16167 ± 3% softirqs.CPU56.RCU 30828 +18.8% 36619 ± 9% softirqs.CPU56.SCHED 130194 ± 2% -12.1% 114503 ± 2% softirqs.CPU56.TIMER 30577 ± 5% +21.1% 37016 ± 10% softirqs.CPU57.SCHED 147818 ± 17% -24.6% 111515 ± 2% softirqs.CPU57.TIMER 25343 ± 33% -36.6% 16078 ± 4% softirqs.CPU58.RCU 31181 ± 2% +16.6% 36349 ± 10% softirqs.CPU58.SCHED 139070 ± 7% -16.4% 116283 ± 2% softirqs.CPU58.TIMER 19741 ± 16% -15.3% 16722 ± 5% softirqs.CPU59.RCU 30636 ± 2% +20.1% 36788 ± 9% softirqs.CPU59.SCHED 133696 -14.0% 114933 softirqs.CPU59.TIMER 30639 ± 2% +19.3% 36549 ± 9% softirqs.CPU6.SCHED 31080 +19.0% 36975 ± 7% softirqs.CPU60.SCHED 133580 ± 3% -13.5% 115606 softirqs.CPU60.TIMER 19131 ± 16% -16.9% 15898 ± 2% softirqs.CPU61.RCU 30495 ± 3% +20.2% 36664 ± 10% softirqs.CPU61.SCHED 133103 -15.3% 112675 softirqs.CPU61.TIMER 31019 ± 2% +18.8% 36849 ± 8% softirqs.CPU62.SCHED 134432 ± 4% -15.8% 113144 softirqs.CPU62.TIMER 30882 ± 2% +16.9% 36105 ± 8% softirqs.CPU63.SCHED 131772 ± 2% -12.7% 115005 ± 3% softirqs.CPU63.TIMER 19316 ± 16% -18.8% 15694 ± 4% softirqs.CPU64.RCU 30286 ± 2% +20.5% 36497 ± 8% softirqs.CPU64.SCHED 128582 ± 2% -10.1% 115656 ± 4% softirqs.CPU64.TIMER 19223 ± 17% -18.1% 15749 ± 3% softirqs.CPU65.RCU 30985 ± 2% +18.6% 36751 ± 8% softirqs.CPU65.SCHED 130355 ± 3% -13.0% 113407 ± 2% softirqs.CPU65.TIMER 40027 ± 4% -14.7% 34153 ± 7% softirqs.CPU66.SCHED 131990 ± 3% -16.9% 109630 ± 3% softirqs.CPU66.TIMER 40103 ± 5% -16.4% 33528 ± 7% softirqs.CPU67.SCHED 133065 ± 4% -18.5% 108445 softirqs.CPU67.TIMER 39282 -14.5% 33576 ± 9% softirqs.CPU68.SCHED 131993 ± 2% -18.9% 107006 ± 2% softirqs.CPU68.TIMER 131342 ± 3% -16.3% 109974 ± 3% softirqs.CPU69.TIMER 19684 ± 18% -16.4% 16446 ± 4% softirqs.CPU7.RCU 30804 ± 2% +19.7% 36877 ± 11% softirqs.CPU7.SCHED 39499 ± 4% -14.1% 33939 ± 7% softirqs.CPU70.SCHED 132374 -17.3% 109508 ± 4% softirqs.CPU70.TIMER 39816 ± 4% -13.6% 34387 ± 6% softirqs.CPU71.SCHED 133540 -18.0% 109519 ± 2% softirqs.CPU71.TIMER 40073 ± 4% -15.8% 33726 ± 6% softirqs.CPU72.SCHED 132172 ± 4% -18.5% 107744 softirqs.CPU72.TIMER 39089 ± 3% -14.1% 33596 ± 7% softirqs.CPU73.SCHED 131584 -16.4% 110040 ± 3% softirqs.CPU73.TIMER 40315 ± 4% -16.5% 33655 ± 7% softirqs.CPU74.SCHED 133455 -17.6% 109941 softirqs.CPU74.TIMER 40427 ± 5% -16.7% 33659 ± 8% softirqs.CPU75.SCHED 137065 ± 5% -18.6% 111601 ± 3% softirqs.CPU75.TIMER 39577 ± 3% -14.0% 34031 ± 7% softirqs.CPU76.SCHED 130476 -16.6% 108755 ± 4% softirqs.CPU76.TIMER 40386 ± 5% -14.9% 34356 ± 7% softirqs.CPU77.SCHED 134414 ± 4% -18.8% 109199 ± 3% softirqs.CPU77.TIMER 40179 ± 5% -16.2% 33656 ± 7% softirqs.CPU78.SCHED 131087 ± 2% -17.4% 108250 ± 3% softirqs.CPU78.TIMER 39263 ± 3% -14.2% 33686 ± 7% softirqs.CPU79.SCHED 130805 ± 2% -16.3% 109502 ± 2% softirqs.CPU79.TIMER 19896 ± 16% -19.7% 15987 ± 2% softirqs.CPU8.RCU 30563 ± 3% +24.0% 37906 ± 7% softirqs.CPU8.SCHED 131375 ± 2% -12.8% 114531 softirqs.CPU8.TIMER 39276 ± 2% -14.9% 33416 ± 9% softirqs.CPU80.SCHED 138042 ± 7% -21.9% 107876 ± 2% softirqs.CPU80.TIMER 21626 ± 17% -24.1% 16414 ± 4% softirqs.CPU81.RCU 40018 ± 4% -15.6% 33781 ± 7% softirqs.CPU81.SCHED 134117 ± 4% -16.8% 111583 ± 3% softirqs.CPU81.TIMER 134747 -18.0% 110540 ± 3% softirqs.CPU82.TIMER 133707 ± 2% -17.6% 110191 softirqs.CPU83.TIMER 40079 ± 4% -16.3% 33556 ± 10% softirqs.CPU84.SCHED 130536 ± 2% -15.8% 109959 softirqs.CPU84.TIMER 40427 ± 5% -16.9% 33585 ± 7% softirqs.CPU85.SCHED 131805 ± 4% -18.4% 107549 softirqs.CPU85.TIMER 40785 ± 5% -17.5% 33639 ± 7% softirqs.CPU86.SCHED 134917 ± 2% -18.1% 110473 ± 3% softirqs.CPU86.TIMER 39496 ± 3% -15.7% 33280 ± 8% softirqs.CPU87.SCHED 135851 ± 5% -19.3% 109698 ± 2% softirqs.CPU87.TIMER 30947 ± 2% +20.1% 37173 ± 9% softirqs.CPU9.SCHED 133360 ± 3% -14.3% 114339 ± 3% softirqs.CPU9.TIMER 4299 ±103% -66.6% 1438 ± 9% softirqs.NET_RX 1788927 ± 16% -15.2% 1516780 ± 4% softirqs.RCU 11741892 -15.5% 9927286 softirqs.TIMER Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 8 months

1
0
0 / 0

[signal] b3eedf8d5b: BUG:unable_to_handle_kernel

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: b3eedf8d5b385cacfc80f04bc6750df4f971b807 ("[PATCH] signal: always allocate siginfo for SI_TKILL") url: https://github.com/0day-ci/linux/commits/Jack-Andersen/signal-always-allo... in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------------------------+----------+------------+ | | v5.0-rc4 | b3eedf8d5b | +-------------------------------------------------------------------------+----------+------------+ | boot_successes | 151 | 6 | | boot_failures | 337 | 5 | | invoked_oom-killer:gfp_mask=0x | 212 | 5 | | Mem-Info | 212 | 5 | | Out_of_memory_and_no_killable_processes | 210 | 2 | | Kernel_panic-not_syncing:System_is_deadlocked_on_memory | 210 | 2 | | BUG:kernel_in_stage | 6 | | | End_of_test:RCU_HOTPLUG | 86 | | | WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_stats_print[rcutorture] | 33 | | | RIP:rcu_torture_stats_print[rcutorture] | 33 | | | End_of_test:FAILURE | 33 | | | BUG:unable_to_handle_kernel | 0 | 3 | | Oops:#[##] | 0 | 3 | | RIP:__send_signal | 0 | 3 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 3 | +-------------------------------------------------------------------------+----------+------------+ [ 89.016930] BUG: unable to handle kernel NULL pointer dereference at 0000000000000009 [ 89.019907] #PF error: [normal kernel read fault] [ 89.022004] PGD 0 P4D 0 [ 89.023198] Oops: 0000 [#1] SMP PTI [ 89.024568] CPU: 1 PID: 1162 Comm: modprobe Not tainted 5.0.0-rc4-00001-gb3eedf8 #7 [ 89.026701] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 [ 89.028993] RIP: 0010:__send_signal+0x9d/0x5b0 [ 89.030589] Code: 44 8d 53 ff 4d 8b 70 10 44 89 d1 45 89 d1 49 d3 ee 41 83 e6 01 0f 85 9f 01 00 00 b8 00 01 04 00 4c 0f a3 d0 0f 83 e8 01 00 00 <41> 83 7c 24 08 fa 0f 85 dd 00 00 00 f6 45 26 20 0f 85 d3 00 00 00 [ 89.035619] RSP: 0018:ffffaad380ac7a10 EFLAGS: 00010047 [ 89.037427] RAX: 0000000000040100 RBX: 0000000000000009 RCX: 0000000000000008 [ 89.039592] RDX: ffff8eadbebd2201 RSI: ffff8eadbec81e01 RDI: 0000000000000000 [ 89.041758] RBP: ffff8eadbec81e00 R08: ffff8eadbcbd2240 R09: 0000000000000008 [ 89.043930] R10: 0000000000000008 R11: 00000000ffffffff R12: 0000000000000001 [ 89.046115] R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000 [ 89.048296] FS: 00007fa3ad97d700(0000) GS:ffff8eadab500000(0000) knlGS:0000000000000000 [ 89.050663] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 89.052609] CR2: 0000000000000009 CR3: 00000000005bc000 CR4: 00000000000406e0 [ 89.054820] DR0: 00007f22859a8000 DR1: 0000000000000000 DR2: 0000000000000000 [ 89.057032] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 [ 89.059246] Call Trace: [ 89.060631] ? vprintk_emit+0x1ae/0x2a0 [ 89.062253] do_send_sig_info+0x4a/0x90 [ 89.063895] __oom_kill_process+0xb6/0x260 [ 89.065578] oom_kill_process+0x1de/0x2a0 [ 89.067246] out_of_memory+0x2e8/0x510 [ 89.068876] __alloc_pages_slowpath+0xac4/0xdd0 [ 89.070651] __alloc_pages_nodemask+0x2e5/0x300 [ 89.072424] __vmalloc_node_range+0x205/0x2b0 [ 89.074163] ? load_module+0x1864/0x2990 [ 89.075830] module_alloc+0x71/0xc0 [ 89.077425] ? load_module+0x1864/0x2990 [ 89.079091] load_module+0x1864/0x2990 [ 89.080734] ? vfs_read+0x125/0x150 [ 89.082314] ? kernel_read_file+0x1a8/0x1d0 [ 89.084039] ? __do_sys_finit_module+0xe9/0x110 [ 89.085834] __do_sys_finit_module+0xe9/0x110 [ 89.087597] do_syscall_64+0x5b/0x180 [ 89.089212] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 89.091105] RIP: 0033:0x7fa3ad4a8229 [ 89.092742] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3f 4c 2b 00 f7 d8 64 89 01 48 [ 89.098162] RSP: 002b:00007ffc69600e18 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 89.100601] RAX: ffffffffffffffda RBX: 000055e2740303b0 RCX: 00007fa3ad4a8229 [ 89.102960] RDX: 0000000000000000 RSI: 000055e273502638 RDI: 0000000000000001 [ 89.105292] RBP: 000055e273502638 R08: 0000000000000000 R09: 0000000000000001 [ 89.107610] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000 [ 89.109895] R13: 000055e27402f590 R14: 0000000000040000 R15: 0000000000000000 [ 89.112155] Modules linked in: ip_tunnel ieee802154_socket ieee802154 vmw_vsock_vmci_transport vsock vmw_vmci af_key rose kcm nfc hidp cmtp kernelcapi bnep rfcomm bluetooth ecdh_generic rfkill can_bcm can_raw can pptp gre l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppoe pppox ppp_generic slhc crypto_user ib_core nfnetlink scsi_transport_iscsi dccp_ipv6 atm netrom appletalk psnap llc ax25 sctp libcrc32c dccp_ipv4 dccp virtio_blk ppdev bochs_drm crct10dif_pclmul crc32_pclmul crc32c_intel ttm ghash_clmulni_intel drm_kms_helper snd_pcm snd_timer drm joydev snd serio_raw soundcore pcspkr i6300esb floppy parport_pc parport qemu_fw_cfg virtio_pci virtio_ring ata_generic i2c_piix4 virtio pata_acpi [ 89.128902] CR2: 0000000000000009 [ 89.130034] ---[ end trace f01d4d182d151a78 ]--- To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 8 months

1
0
0 / 0

[mm/page_alloc.c] 148c6673c9: vm-scalability.free_time 102.6% regression

by kernel test robot

Greeting, FYI, we noticed a 102.6% regression of vm-scalability.free_time due to commit: commit: 148c6673c935129351f353601720dc6a160d64bb ("mm/page_alloc.c: Set ppc->high fraction default to 512") https://git.kernel.org/cgit/linux/kernel/git/saeed/linux.git net-next-mlx4 in testcase: vm-scalability on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory with following parameters: runtime: 300s test: lru-file-mmap-read cpufreq_governor: performance test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ In addition to that, the commit also has significant impact on the following tests: +------------------+------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops 14.3% improvement | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=page_fault2 | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.median -19.1% regression | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-readonce | +------------------+------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops 8.8% improvement | | test machine | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory | | test parameters | cpufreq_governor=performance | | | test=malloc1 | | | ucode=0x20 | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.free_time 64.3% regression | | test machine | 104 threads Skylake with 192G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-mmap-read | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.free_time 8.5% regression | | test machine | 40 threads Skylake with 64G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=anon-r-rand | +------------------+------------------------------------------------------------------------+ | testcase: change | reaim: reaim.jobs_per_min 1.5% improvement | | test machine | 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory | | test parameters | cpufreq_governor=performance | | | nr_task=100t | | | runtime=300s | | | test=sieve | | | ucode=0x42d | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.free_time -76.2% improvement | | test machine | 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=anon-r-seq | | | ucode=0x12 | +------------------+------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops 17.2% improvement | | test machine | 104 threads Skylake with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=page_fault2 | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-mmap-read-rand | +------------------+------------------------------------------------------------------------+ | testcase: change | fileio: fileio.request_latency_95%_ms 11356.3% regression | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory | | test parameters | cpufreq_governor=performance | | | disk=1HDD | | | filenum=1024f | | | fs=f2fs | | | iomode=sync | | | nr_threads=100% | | | period=600s | | | rwmode=rndrw | | | size=64G | | | ucode=0xb00002e | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.median -29.6% regression | | test machine | 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-readonce | | | ucode=0x12 | +------------------+------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops 25.8% improvement | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=100% | | | test=page_fault2 | +------------------+------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -51.9% regression | | test machine | 192 threads Skylake-4S with 704G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=page_fault2 | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.median -18.3% regression | | test machine | 104 threads Skylake with 192G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-readonce | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.median -22.0% regression | | test machine | 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-readonce | | | ucode=0xb00002e | +------------------+------------------------------------------------------------------------+ | testcase: change | pigz: | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | blocksize=512K | | | cpufreq_governor=performance | | | nr_threads=25% | +------------------+------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.free_time -14.6% improvement | | test machine | 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | test=lru-file-mmap-read | | | ucode=0xb00002e | +------------------+------------------------------------------------------------------------+ | testcase: change | pigz: pigz.throughput 2.7% improvement | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | blocksize=128K | | | cpufreq_governor=performance | | | nr_threads=100% | +------------------+------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.tee.ops_per_sec 17.3% improvement | | test machine | 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory | | test parameters | class=pipe | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=1s | +------------------+------------------------------------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep2/lru-file-mmap-read/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.08 ± 2% +102.6% 0.17 ± 6% vm-scalability.free_time 190785 -16.3% 159667 vm-scalability.median 0.27 ± 16% +192.0% 0.79 ± 13% vm-scalability.stddev 16797990 -16.4% 14039573 vm-scalability.throughput 318.07 +3.6% 329.36 vm-scalability.time.elapsed_time 318.07 +3.6% 329.36 vm-scalability.time.elapsed_time.max 10869 ± 6% -65.2% 3782 ± 35% vm-scalability.time.major_page_faults 2439444 ± 11% -22.4% 1892959 ± 3% vm-scalability.time.maximum_resident_set_size 98279905 -14.0% 84542283 vm-scalability.time.minor_page_faults 7669 -5.8% 7227 vm-scalability.time.percent_of_cpu_this_job_got 2783 -12.4% 2437 vm-scalability.time.user_time 15936 ± 3% -43.8% 8960 ± 6% vm-scalability.time.voluntary_context_switches 4.832e+09 -12.7% 4.217e+09 vm-scalability.workload 10.18 -1.0 9.16 mpstat.cpu.usr% 22523 ± 23% +3117.7% 724745 ±158% cpuidle.POLL.time 8479 ± 12% +486.8% 49760 ±106% cpuidle.POLL.usage 454.74 -1.5% 447.92 pmeter.Average_Active_Power 10662906 -11.4% 9445999 pmeter.performance_per_watt 392.75 ± 15% +35.5% 532.00 ± 12% slabinfo.skbuff_ext_cache.active_objs 392.75 ± 15% +36.0% 534.25 ± 11% slabinfo.skbuff_ext_cache.num_objs 2.267e+08 ± 2% -13.2% 1.968e+08 ± 2% turbostat.IRQ 237.05 -1.7% 232.94 turbostat.PkgWatt 30368 ± 73% +207.8% 93477 ± 17% numa-meminfo.node0.AnonHugePages 5455398 ± 7% -14.3% 4673744 ± 7% numa-meminfo.node0.MemFree 1857052 -14.4% 1589936 ± 4% numa-meminfo.node1.PageTables 79.25 +1.9% 80.75 vmstat.cpu.sy 5137 -4.4% 4910 vmstat.system.cs 395997 ± 4% -13.5% 342451 ± 3% vmstat.system.in 71112 ± 28% +56.8% 111534 ± 7% meminfo.AnonHugePages 294449 ± 8% -33.0% 197358 ± 9% meminfo.DirectMap4k 10228573 ± 2% -13.8% 8812062 ± 3% meminfo.MemFree 3919091 -10.8% 3495108 meminfo.PageTables 527772 -13.3% 457627 meminfo.max_used_kB 5.053e+08 -22.1% 3.934e+08 ± 2% numa-numastat.node0.local_node 36050190 ± 9% +126.9% 81793402 ± 5% numa-numastat.node0.numa_foreign 5.053e+08 -22.1% 3.934e+08 ± 2% numa-numastat.node0.numa_hit 28825403 ± 9% +121.9% 63956966 ± 6% numa-numastat.node0.numa_miss 28827048 ± 9% +121.9% 63964490 ± 6% numa-numastat.node0.other_node 5.065e+08 -24.3% 3.835e+08 ± 2% numa-numastat.node1.local_node 28825403 ± 9% +121.9% 63956966 ± 6% numa-numastat.node1.numa_foreign 5.065e+08 -24.3% 3.836e+08 ± 2% numa-numastat.node1.numa_hit 36050190 ± 9% +126.9% 81793402 ± 5% numa-numastat.node1.numa_miss 36065990 ± 9% +126.8% 81803578 ± 5% numa-numastat.node1.other_node 1410341 ± 5% -16.7% 1174677 ± 7% numa-vmstat.node0.nr_free_pages 393.25 ± 3% -9.7% 355.00 numa-vmstat.node0.nr_isolated_file 18615874 ± 9% +137.1% 44139342 ± 5% numa-vmstat.node0.numa_foreign 3.027e+08 -22.5% 2.347e+08 ± 2% numa-vmstat.node0.numa_hit 3.027e+08 -22.5% 2.347e+08 ± 2% numa-vmstat.node0.numa_local 15119878 ± 13% +133.6% 35321988 ± 6% numa-vmstat.node0.numa_miss 15121763 ± 13% +133.6% 35329659 ± 6% numa-vmstat.node0.numa_other 3592757 ± 2% -12.2% 3155379 ± 3% numa-vmstat.node0.workingset_nodereclaim 1280797 ± 8% -15.5% 1082202 ± 5% numa-vmstat.node1.nr_free_pages 463711 -14.6% 395956 ± 4% numa-vmstat.node1.nr_page_table_pages 15125778 ± 13% +133.6% 35329763 ± 6% numa-vmstat.node1.numa_foreign 3.035e+08 -24.4% 2.296e+08 ± 3% numa-vmstat.node1.numa_hit 3.034e+08 -24.4% 2.295e+08 ± 3% numa-vmstat.node1.numa_local 18621273 ± 9% +137.1% 44149116 ± 5% numa-vmstat.node1.numa_miss 18772519 ± 9% +136.0% 44295194 ± 5% numa-vmstat.node1.numa_other 2552477 -36.8% 1612321 ± 8% numa-vmstat.node1.workingset_nodereclaim 185978 -11.0% 165530 ± 4% proc-vmstat.allocstall_movable 39040 ± 7% -17.4% 32232 ± 10% proc-vmstat.allocstall_normal 6.178e+08 ± 11% -61.0% 2.406e+08 ± 18% proc-vmstat.compact_daemon_free_scanned 11494995 ± 13% -80.7% 2213142 ± 27% proc-vmstat.compact_daemon_migrate_scanned 3099 -55.6% 1375 ± 8% proc-vmstat.compact_daemon_wake 368.50 ± 27% -57.5% 156.50 ± 15% proc-vmstat.compact_fail 6.191e+08 ± 10% -60.9% 2.421e+08 ± 18% proc-vmstat.compact_free_scanned 2216588 ± 3% -60.1% 884408 ± 18% proc-vmstat.compact_isolated 11497797 ± 13% -80.6% 2227952 ± 26% proc-vmstat.compact_migrate_scanned 373.50 ± 26% -57.3% 159.50 ± 15% proc-vmstat.compact_stall 4106 -55.2% 1838 ± 11% proc-vmstat.kswapd_low_wmark_hit_quickly 3075564 -1.2% 3038183 proc-vmstat.nr_dirty_background_threshold 6158650 -1.2% 6083796 proc-vmstat.nr_dirty_threshold 2596006 ± 2% -13.9% 2235375 ± 3% proc-vmstat.nr_free_pages 792.50 ± 4% -6.8% 738.50 proc-vmstat.nr_isolated_file 26325915 +2.1% 26890597 proc-vmstat.nr_mapped 980457 -11.0% 872161 proc-vmstat.nr_page_table_pages 537810 -2.0% 527139 proc-vmstat.nr_slab_reclaimable 34140 +1.1% 34523 proc-vmstat.nr_slab_unreclaimable 64875594 ± 9% +124.7% 1.458e+08 ± 5% proc-vmstat.numa_foreign 1.012e+09 -23.2% 7.77e+08 ± 2% proc-vmstat.numa_hit 1.012e+09 -23.2% 7.77e+08 ± 2% proc-vmstat.numa_local 64875594 ± 9% +124.7% 1.458e+08 ± 5% proc-vmstat.numa_miss 64893044 ± 9% +124.6% 1.458e+08 ± 5% proc-vmstat.numa_other 4209 -53.0% 1980 ± 11% proc-vmstat.pageoutrun 13155143 -13.1% 11433399 proc-vmstat.pgalloc_dma32 1.065e+09 -14.3% 9.127e+08 proc-vmstat.pgalloc_normal 98956663 -13.8% 85312423 proc-vmstat.pgfault 1.079e+09 -14.4% 9.243e+08 proc-vmstat.pgfree 10869 ± 6% -65.2% 3782 ± 35% proc-vmstat.pgmajfault 1104664 ± 3% -59.8% 443543 ± 18% proc-vmstat.pgmigrate_success 1.977e+09 -16.0% 1.661e+09 proc-vmstat.pgscan_direct 1.117e+08 ± 3% +20.6% 1.346e+08 ± 2% proc-vmstat.pgscan_kswapd 1e+09 -16.2% 8.384e+08 proc-vmstat.pgsteal_direct 42425835 ± 2% +23.0% 52190643 proc-vmstat.pgsteal_kswapd 14681817 -18.8% 11925440 ± 2% proc-vmstat.slabs_scanned 6122476 -22.4% 4752262 ± 3% proc-vmstat.workingset_nodereclaim 3098450 -2.5% 3020467 proc-vmstat.workingset_nodes 449.00 ± 7% +158.4% 1160 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev 7568 +21.2% 9175 ± 9% sched_debug.cfs_rq:/.load.min 34.84 ± 4% +266.9% 127.84 ± 21% sched_debug.cfs_rq:/.load_avg.avg 861.29 ± 6% +31.7% 1134 ± 8% sched_debug.cfs_rq:/.load_avg.max 6.83 ± 11% +104.9% 14.00 ± 15% sched_debug.cfs_rq:/.load_avg.min 129.23 ± 2% +31.5% 169.99 ± 6% sched_debug.cfs_rq:/.load_avg.stddev 647896 ± 12% -31.5% 444068 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 0.69 +24.4% 0.86 sched_debug.cfs_rq:/.nr_running.avg 13.87 ± 4% +916.6% 141.04 ± 6% sched_debug.cfs_rq:/.nr_spread_over.avg 151.08 ± 15% +76.4% 266.58 ± 3% sched_debug.cfs_rq:/.nr_spread_over.max 27.20 ± 7% +112.5% 57.81 ± 9% sched_debug.cfs_rq:/.nr_spread_over.stddev 7568 +21.2% 9175 ± 9% sched_debug.cfs_rq:/.runnable_weight.min -2892722 -50.0% -1445196 sched_debug.cfs_rq:/.spread0.min 647562 ± 12% -31.4% 444517 ± 8% sched_debug.cfs_rq:/.spread0.stddev 747.15 +22.2% 913.36 sched_debug.cfs_rq:/.util_avg.avg 452.75 ± 9% +17.1% 530.12 ± 6% sched_debug.cfs_rq:/.util_avg.min 174.83 ± 7% +27.1% 222.13 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.stddev 651271 ± 2% +32.2% 861145 sched_debug.cpu.avg_idle.avg 45944 ± 25% +496.0% 273815 ± 21% sched_debug.cpu.avg_idle.min 275984 -39.8% 166017 ± 2% sched_debug.cpu.avg_idle.stddev 21.06 ± 11% +17.5% 24.73 ± 6% sched_debug.cpu.cpu_load[2].avg 6.67 ± 5% +21.9% 8.12 ± 5% sched_debug.cpu.cpu_load[2].min 21.31 ± 10% +22.6% 26.13 ± 7% sched_debug.cpu.cpu_load[3].avg 6.75 ± 5% +21.0% 8.17 ± 6% sched_debug.cpu.cpu_load[3].min 21.61 ± 9% +33.7% 28.89 ± 11% sched_debug.cpu.cpu_load[4].avg 6.79 ± 4% +21.5% 8.25 ± 7% sched_debug.cpu.cpu_load[4].min 1580 +23.1% 1944 sched_debug.cpu.curr->pid.avg 367.25 ± 5% +68.5% 618.88 ± 29% sched_debug.cpu.curr->pid.min 7568 +21.2% 9174 ± 9% sched_debug.cpu.load.min 155998 -8.9% 142149 sched_debug.cpu.nr_load_updates.min 1000 ± 3% +128.2% 2282 ± 12% sched_debug.cpu.nr_load_updates.stddev 0.72 +24.0% 0.90 sched_debug.cpu.nr_running.avg 187.07 ± 4% +37.7% 257.53 ± 7% sched_debug.cpu.sched_goidle.avg 51.71 ± 13% -58.4% 21.50 ± 56% sched_debug.cpu.sched_goidle.min 253.11 ± 11% +40.8% 356.31 ± 12% sched_debug.cpu.sched_goidle.stddev 874.88 ± 7% -12.5% 765.71 ± 3% sched_debug.cpu.ttwu_count.stddev 5408 ± 9% -12.9% 4712 ± 3% sched_debug.cpu.ttwu_local.max 2170 -13.4% 1879 ± 3% sched_debug.cpu.ttwu_local.min 664.25 ± 8% -22.1% 517.45 ± 7% sched_debug.cpu.ttwu_local.stddev 54.66 ± 4% -8.3 46.36 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node 53.48 ± 4% -8.2 45.24 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 72.95 ± 2% -8.0 64.94 perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 71.61 ± 2% -7.8 63.81 perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 71.54 ± 2% -7.8 63.75 perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages 95.73 -4.1 91.61 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 95.71 -4.1 91.59 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 95.74 -4.1 91.62 perf-profile.calltrace.cycles-pp.page_fault 95.58 -4.1 91.48 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 95.54 -4.1 91.44 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 2.63 ± 29% -1.4 1.19 ± 54% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault 2.63 ± 29% -1.4 1.19 ± 54% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault 2.63 ± 29% -1.4 1.19 ± 53% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault.__do_fault 2.63 ± 29% -1.4 1.21 ± 52% perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 1.73 ± 8% -1.4 0.33 ±101% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 3.02 ± 6% -1.4 1.62 ± 7% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 1.79 ± 8% -1.3 0.48 ± 60% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 1.74 ± 12% -0.3 1.46 ± 6% perf-profile.calltrace.cycles-pp.page_referenced.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 1.40 ± 4% -0.3 1.14 ± 3% perf-profile.calltrace.cycles-pp.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 3.96 ± 5% -0.2 3.73 ± 3% perf-profile.calltrace.cycles-pp.memset_erms.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages 1.27 ± 4% -0.2 1.04 ± 3% perf-profile.calltrace.cycles-pp.rmap_walk_file.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.90 ± 4% -0.2 0.73 ± 3% perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_file.try_to_unmap.shrink_page_list.shrink_inactive_list 0.72 ± 5% -0.1 0.60 ± 4% perf-profile.calltrace.cycles-pp.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 1.34 ± 4% -0.1 1.22 ± 6% perf-profile.calltrace.cycles-pp.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.81 ± 5% -0.1 0.74 ± 6% perf-profile.calltrace.cycles-pp.page_referenced_one.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list 0.00 +1.1 1.07 ± 15% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 0.00 +1.3 1.28 ± 20% perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 6.57 ± 5% +2.3 8.82 ± 5% perf-profile.calltrace.cycles-pp.smp_call_function_many.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 19.76 ± 4% +2.5 22.28 ± 7% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.filemap_fault.__xfs_filemap_fault 19.74 ± 4% +2.5 22.27 ± 7% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.filemap_fault 19.73 ± 4% +2.5 22.26 ± 7% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 19.67 ± 4% +2.5 22.21 ± 7% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 14.04 ± 4% +2.8 16.81 ± 8% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 11.19 ± 7% +3.7 14.89 ± 11% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 11.13 ± 7% +3.7 14.84 ± 11% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 10.39 ± 7% +3.7 14.13 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 10.42 ± 7% +3.7 14.16 ± 12% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 6.85 ± 5% +3.8 10.66 ± 9% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list 6.85 ± 5% +3.8 10.67 ± 9% perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg 6.80 ± 5% +3.8 10.62 ± 9% perf-profile.calltrace.cycles-pp.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list 6.86 ± 5% +3.8 10.68 ± 9% perf-profile.calltrace.cycles-pp.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.00 +4.2 4.18 ± 6% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages 0.00 +4.2 4.18 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq 2.48 ± 3% +4.4 6.90 ± 2% perf-profile.calltrace.cycles-pp.ret_from_fork 2.48 ± 3% +4.4 6.90 ± 2% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.00 +4.5 4.55 ± 6% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work 0.00 +4.6 4.56 ± 6% perf-profile.calltrace.cycles-pp.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work.worker_thread 0.00 +4.6 4.56 ± 6% perf-profile.calltrace.cycles-pp.drain_pages.drain_local_pages_wq.process_one_work.worker_thread.kthread 0.00 +4.6 4.58 ± 7% perf-profile.calltrace.cycles-pp.drain_local_pages_wq.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +4.6 4.60 ± 7% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +4.6 4.61 ± 7% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 22418 ± 16% -18.9% 18186 ± 7% softirqs.CPU12.RCU 20494 ± 15% -18.0% 16807 ± 2% softirqs.CPU20.RCU 21340 ± 6% -17.4% 17626 ± 6% softirqs.CPU23.RCU 20156 ± 3% -20.8% 15960 ± 2% softirqs.CPU25.RCU 20503 ± 6% -14.9% 17445 ± 4% softirqs.CPU27.RCU 22932 ± 14% -25.9% 16997 ± 4% softirqs.CPU28.RCU 6094 ± 4% +29.3% 7880 ± 19% softirqs.CPU28.SCHED 22011 ± 3% -22.2% 17118 ± 7% softirqs.CPU29.RCU 24355 ± 21% -22.6% 18852 ± 3% softirqs.CPU3.RCU 24459 ± 15% -22.2% 19030 ± 11% softirqs.CPU30.RCU 22782 ± 9% -23.9% 17341 ± 12% softirqs.CPU31.RCU 22722 ± 6% -27.0% 16588 ± 2% softirqs.CPU32.RCU 21640 ± 3% -20.8% 17142 ± 12% softirqs.CPU33.RCU 21469 ± 7% -17.1% 17797 ± 5% softirqs.CPU34.RCU 21676 ± 6% -14.2% 18597 ± 10% softirqs.CPU35.RCU 21508 ± 7% -16.1% 18049 ± 6% softirqs.CPU36.RCU 21167 ± 7% -19.3% 17078 ± 4% softirqs.CPU37.RCU 22657 ± 11% -24.8% 17040 ± 4% softirqs.CPU40.RCU 22714 ± 2% -25.1% 17015 ± 6% softirqs.CPU41.RCU 22121 -19.6% 17783 ± 6% softirqs.CPU42.RCU 22511 ± 6% -22.0% 17568 ± 7% softirqs.CPU43.RCU 18617 ± 18% -22.6% 14415 ± 5% softirqs.CPU44.RCU 25191 ± 5% -15.1% 21391 ± 11% softirqs.CPU46.RCU 22206 ± 7% -19.7% 17831 ± 12% softirqs.CPU48.RCU 20847 ± 7% -11.1% 18537 ± 6% softirqs.CPU5.RCU 28974 ± 43% -36.4% 18419 ± 11% softirqs.CPU50.RCU 20225 ± 6% -16.1% 16978 ± 6% softirqs.CPU51.RCU 21802 ± 13% -23.5% 16688 ± 8% softirqs.CPU52.RCU 20567 ± 8% -15.4% 17391 ± 5% softirqs.CPU56.RCU 23826 ± 21% -27.3% 17323 ± 6% softirqs.CPU57.RCU 20493 ± 2% -15.1% 17407 ± 5% softirqs.CPU59.RCU 21884 ± 20% -24.8% 16457 ± 5% softirqs.CPU66.RCU 20744 ± 7% -19.5% 16692 ± 8% softirqs.CPU67.RCU 21513 ± 2% -26.1% 15898 ± 5% softirqs.CPU68.RCU 20934 ± 9% -23.9% 15922 ± 3% softirqs.CPU70.RCU 20941 ± 8% -23.4% 16050 ± 3% softirqs.CPU71.RCU 20428 ± 6% -22.3% 15870 ± 6% softirqs.CPU72.RCU 21634 ± 5% -24.4% 16361 ± 11% softirqs.CPU73.RCU 23287 ± 20% -30.1% 16276 ± 9% softirqs.CPU74.RCU 6019 ± 5% +32.2% 7958 ± 20% softirqs.CPU74.SCHED 20339 ± 3% -26.1% 15031 ± 3% softirqs.CPU75.RCU 30194 ± 54% -47.4% 15870 ± 5% softirqs.CPU76.RCU 20128 ± 4% -20.5% 16003 ± 4% softirqs.CPU77.RCU 22257 ± 10% -24.5% 16806 softirqs.CPU78.RCU 20882 ± 7% -19.0% 16910 ± 2% softirqs.CPU79.RCU 6259 ± 5% +42.5% 8922 ± 23% softirqs.CPU79.SCHED 20309 ± 8% -14.2% 17416 ± 7% softirqs.CPU80.RCU 20606 ± 6% -20.2% 16446 ± 8% softirqs.CPU81.RCU 6126 ± 4% +39.5% 8549 ± 15% softirqs.CPU81.SCHED 20111 ± 8% -22.9% 15504 ± 5% softirqs.CPU82.RCU 20114 ± 5% -21.6% 15776 softirqs.CPU83.RCU 6082 ± 7% +36.5% 8302 ± 20% softirqs.CPU83.SCHED 21626 ± 9% -23.7% 16509 ± 3% softirqs.CPU84.RCU 20304 ± 4% -20.7% 16101 ± 9% softirqs.CPU85.RCU 6174 ± 6% +45.1% 8960 ± 15% softirqs.CPU85.SCHED 20100 ± 5% -16.6% 16755 ± 8% softirqs.CPU86.RCU 153748 ± 10% -22.7% 118918 softirqs.CPU87.TIMER 1872444 ± 6% -15.3% 1585607 ± 4% softirqs.RCU 2.973e+10 -11.0% 2.645e+10 perf-stat.i.branch-instructions 47997774 -16.2% 40243170 perf-stat.i.branch-misses 13.04 -2.5 10.58 ± 5% perf-stat.i.cache-miss-rate% 50166411 -7.8% 46252162 perf-stat.i.cache-misses 6.39e+08 -17.4% 5.28e+08 perf-stat.i.cache-references 5092 -3.9% 4896 perf-stat.i.context-switches 2.24 +2.8% 2.30 perf-stat.i.cpi 106.53 +31.9% 140.55 ± 3% perf-stat.i.cpu-migrations 5048 -3.2% 4887 perf-stat.i.cycles-between-cache-misses 20431376 ± 5% -15.1% 17342975 perf-stat.i.dTLB-load-misses 2.798e+10 -10.5% 2.504e+10 perf-stat.i.dTLB-loads 876356 ± 4% -20.7% 694887 ± 3% perf-stat.i.dTLB-store-misses 6.172e+09 -17.9% 5.067e+09 perf-stat.i.dTLB-stores 79.50 -1.5 78.02 perf-stat.i.iTLB-load-miss-rate% 2124927 ± 4% -17.6% 1751040 perf-stat.i.iTLB-load-misses 522102 ± 3% -8.7% 476896 ± 3% perf-stat.i.iTLB-loads 1.087e+11 -10.3% 9.749e+10 perf-stat.i.instructions 250153 ± 2% -47.3% 131755 ± 24% perf-stat.i.instructions-per-iTLB-miss 0.51 -10.2% 0.46 perf-stat.i.ipc 32.84 ± 6% -66.1% 11.15 ± 35% perf-stat.i.major-faults 317637 -17.9% 260759 perf-stat.i.minor-faults 65.97 +1.5 67.43 perf-stat.i.node-load-miss-rate% 5647884 +11.6% 6300871 perf-stat.i.node-load-misses 3545704 ± 3% -10.0% 3191252 ± 3% perf-stat.i.node-loads 41.63 ± 2% +3.9 45.56 perf-stat.i.node-store-miss-rate% 5554991 +7.5% 5970401 perf-stat.i.node-store-misses 9591725 -19.2% 7747705 perf-stat.i.node-stores 317672 -17.9% 260814 perf-stat.i.page-faults 5.81 -7.3% 5.39 perf-stat.overall.MPKI 0.16 -0.0 0.15 perf-stat.overall.branch-miss-rate% 7.91 +0.9 8.79 perf-stat.overall.cache-miss-rate% 2.05 +11.2% 2.28 perf-stat.overall.cpi 4464 +8.0% 4820 perf-stat.overall.cycles-between-cache-misses 80.28 -1.7 78.63 perf-stat.overall.iTLB-load-miss-rate% 52007 ± 3% +7.9% 56117 perf-stat.overall.instructions-per-iTLB-miss 0.49 -10.0% 0.44 perf-stat.overall.ipc 61.85 +4.7 66.55 perf-stat.overall.node-load-miss-rate% 37.02 +6.6 43.64 perf-stat.overall.node-store-miss-rate% 7100 +7.1% 7603 perf-stat.overall.path-length 2.939e+10 -10.3% 2.635e+10 perf-stat.ps.branch-instructions 47282796 -15.4% 39985252 perf-stat.ps.branch-misses 49492445 -7.0% 46012507 perf-stat.ps.cache-misses 6.254e+08 -16.3% 5.235e+08 perf-stat.ps.cache-references 5114 -4.5% 4884 perf-stat.ps.context-switches 107.96 +30.2% 140.59 ± 3% perf-stat.ps.cpu-migrations 19951520 ± 5% -13.9% 17173615 perf-stat.ps.dTLB-load-misses 2.77e+10 -9.9% 2.496e+10 perf-stat.ps.dTLB-loads 853028 ± 4% -19.6% 685991 ± 3% perf-stat.ps.dTLB-store-misses 6.072e+09 -17.1% 5.036e+09 perf-stat.ps.dTLB-stores 2073137 ± 4% -16.5% 1731915 perf-stat.ps.iTLB-load-misses 509056 ± 3% -7.5% 470743 ± 2% perf-stat.ps.iTLB-loads 1.076e+11 -9.7% 9.718e+10 perf-stat.ps.instructions 34.11 ± 6% -66.4% 11.47 ± 35% perf-stat.ps.major-faults 310380 -16.7% 258500 perf-stat.ps.minor-faults 5596343 +12.1% 6271980 perf-stat.ps.node-load-misses 3453756 ± 3% -8.7% 3153456 ± 3% perf-stat.ps.node-loads 5538564 +7.6% 5957644 perf-stat.ps.node-store-misses 9422862 -18.3% 7693907 perf-stat.ps.node-stores 310415 -16.7% 258512 perf-stat.ps.page-faults 3.431e+13 -6.5% 3.206e+13 perf-stat.total.instructions 171.75 ± 2% +78.2% 306.00 ± 43% interrupts.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 73072523 -17.6% 60237102 interrupts.CAL:Function_call_interrupts 823484 ± 4% -17.4% 679787 ± 4% interrupts.CPU1.CAL:Function_call_interrupts 1125411 ± 4% -16.6% 938606 ± 4% interrupts.CPU1.TLB:TLB_shootdowns 817712 ± 5% -15.5% 690588 ± 3% interrupts.CPU10.CAL:Function_call_interrupts 1116716 ± 5% -14.3% 956585 ± 3% interrupts.CPU10.TLB:TLB_shootdowns 815653 ± 2% -13.6% 704355 ± 2% interrupts.CPU11.CAL:Function_call_interrupts 1115923 ± 2% -12.6% 975636 ± 2% interrupts.CPU11.TLB:TLB_shootdowns 845717 -17.8% 695270 ± 3% interrupts.CPU12.CAL:Function_call_interrupts 1158033 -16.7% 964888 ± 2% interrupts.CPU12.TLB:TLB_shootdowns 832773 ± 2% -17.1% 689958 ± 5% interrupts.CPU13.CAL:Function_call_interrupts 1138418 ± 2% -16.0% 956672 ± 5% interrupts.CPU13.TLB:TLB_shootdowns 843190 -17.5% 695464 ± 3% interrupts.CPU14.CAL:Function_call_interrupts 1152283 -16.4% 962851 ± 3% interrupts.CPU14.TLB:TLB_shootdowns 843623 -16.7% 702769 ± 3% interrupts.CPU15.CAL:Function_call_interrupts 1152491 -15.5% 974101 ± 3% interrupts.CPU15.TLB:TLB_shootdowns 848015 -15.8% 714049 interrupts.CPU17.CAL:Function_call_interrupts 1160850 -14.5% 992284 interrupts.CPU17.TLB:TLB_shootdowns 845102 -17.1% 700226 interrupts.CPU18.CAL:Function_call_interrupts 1156012 -15.9% 972016 interrupts.CPU18.TLB:TLB_shootdowns 842955 -16.2% 706354 interrupts.CPU19.CAL:Function_call_interrupts 1154616 -15.1% 979973 interrupts.CPU19.TLB:TLB_shootdowns 843344 -15.3% 714128 interrupts.CPU2.CAL:Function_call_interrupts 1151011 -14.2% 988047 interrupts.CPU2.TLB:TLB_shootdowns 835150 -15.8% 703176 interrupts.CPU20.CAL:Function_call_interrupts 1140833 -14.6% 974379 interrupts.CPU20.TLB:TLB_shootdowns 770036 ± 8% -14.8% 656399 ± 2% interrupts.CPU22.CAL:Function_call_interrupts 832678 -17.3% 688902 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 1139814 -16.5% 952031 ± 2% interrupts.CPU23.TLB:TLB_shootdowns 839767 -18.6% 683252 ± 3% interrupts.CPU24.CAL:Function_call_interrupts 1150521 -17.8% 946175 ± 3% interrupts.CPU24.TLB:TLB_shootdowns 801685 ± 4% -14.1% 688570 ± 2% interrupts.CPU25.CAL:Function_call_interrupts 1095515 ± 4% -13.1% 952313 ± 2% interrupts.CPU25.TLB:TLB_shootdowns 842643 -17.0% 699743 ± 2% interrupts.CPU26.CAL:Function_call_interrupts 1155201 -16.2% 968436 ± 2% interrupts.CPU26.TLB:TLB_shootdowns 835410 -17.2% 691606 interrupts.CPU27.CAL:Function_call_interrupts 2548 ± 3% +26.7% 3229 ± 30% interrupts.CPU27.RES:Rescheduling_interrupts 1143798 -16.5% 955531 interrupts.CPU27.TLB:TLB_shootdowns 832850 ± 2% -18.2% 681141 ± 3% interrupts.CPU28.CAL:Function_call_interrupts 1139657 ± 2% -17.3% 943032 ± 3% interrupts.CPU28.TLB:TLB_shootdowns 837232 -17.4% 691608 ± 2% interrupts.CPU29.CAL:Function_call_interrupts 1144711 -16.5% 955689 ± 2% interrupts.CPU29.TLB:TLB_shootdowns 841559 -15.3% 712472 ± 3% interrupts.CPU3.CAL:Function_call_interrupts 2697 ± 5% +43.4% 3867 ± 48% interrupts.CPU3.RES:Rescheduling_interrupts 1152499 -14.2% 988636 ± 2% interrupts.CPU3.TLB:TLB_shootdowns 839971 -19.7% 674179 ± 3% interrupts.CPU30.CAL:Function_call_interrupts 1149644 -18.7% 934298 ± 3% interrupts.CPU30.TLB:TLB_shootdowns 838832 -17.7% 690706 interrupts.CPU31.CAL:Function_call_interrupts 1148283 -16.9% 954218 interrupts.CPU31.TLB:TLB_shootdowns 838361 -17.5% 691288 ± 2% interrupts.CPU32.CAL:Function_call_interrupts 2594 ± 4% +23.1% 3193 ± 16% interrupts.CPU32.RES:Rescheduling_interrupts 1148271 -16.6% 957528 ± 2% interrupts.CPU32.TLB:TLB_shootdowns 834838 -17.0% 693259 ± 2% interrupts.CPU33.CAL:Function_call_interrupts 1141970 -16.0% 958901 ± 2% interrupts.CPU33.TLB:TLB_shootdowns 822999 -16.1% 690228 interrupts.CPU34.CAL:Function_call_interrupts 1126669 -15.2% 955427 interrupts.CPU34.TLB:TLB_shootdowns 830818 ± 3% -17.4% 686140 interrupts.CPU35.CAL:Function_call_interrupts 1138366 ± 3% -16.6% 949078 interrupts.CPU35.TLB:TLB_shootdowns 815498 ± 3% -14.4% 698265 interrupts.CPU36.CAL:Function_call_interrupts 1117460 ± 3% -13.4% 967531 interrupts.CPU36.TLB:TLB_shootdowns 835314 -17.4% 690008 interrupts.CPU37.CAL:Function_call_interrupts 1142213 -16.5% 954135 interrupts.CPU37.TLB:TLB_shootdowns 798294 ± 6% -15.3% 676288 ± 2% interrupts.CPU38.CAL:Function_call_interrupts 2737 ± 11% +35.6% 3710 ± 35% interrupts.CPU38.RES:Rescheduling_interrupts 1090495 ± 7% -14.2% 936038 ± 2% interrupts.CPU38.TLB:TLB_shootdowns 843017 -19.2% 681012 ± 2% interrupts.CPU39.CAL:Function_call_interrupts 1153855 -18.2% 943538 ± 2% interrupts.CPU39.TLB:TLB_shootdowns 809588 -15.1% 687391 ± 2% interrupts.CPU4.CAL:Function_call_interrupts 1105103 -13.8% 952835 ± 2% interrupts.CPU4.TLB:TLB_shootdowns 831172 -17.7% 683839 ± 3% interrupts.CPU40.CAL:Function_call_interrupts 1138032 -16.6% 948679 ± 3% interrupts.CPU40.TLB:TLB_shootdowns 841568 -21.3% 662392 ± 3% interrupts.CPU41.CAL:Function_call_interrupts 1151802 -20.4% 916516 ± 4% interrupts.CPU41.TLB:TLB_shootdowns 813358 ± 2% -16.0% 682962 ± 2% interrupts.CPU42.CAL:Function_call_interrupts 1113750 ± 2% -15.1% 945477 ± 2% interrupts.CPU42.TLB:TLB_shootdowns 824928 -18.9% 668607 ± 4% interrupts.CPU43.CAL:Function_call_interrupts 1130124 -18.0% 927125 ± 4% interrupts.CPU43.TLB:TLB_shootdowns 837296 ± 2% -17.1% 693900 ± 2% interrupts.CPU44.CAL:Function_call_interrupts 1143688 ± 2% -15.7% 964183 ± 2% interrupts.CPU44.TLB:TLB_shootdowns 822858 ± 4% -17.1% 682366 ± 2% interrupts.CPU45.CAL:Function_call_interrupts 1125019 ± 4% -15.8% 947500 ± 2% interrupts.CPU45.TLB:TLB_shootdowns 843488 -17.7% 694343 ± 2% interrupts.CPU46.CAL:Function_call_interrupts 1153293 -16.4% 964554 ± 2% interrupts.CPU46.TLB:TLB_shootdowns 840383 -17.0% 697728 ± 3% interrupts.CPU47.CAL:Function_call_interrupts 1149710 -15.6% 969805 ± 3% interrupts.CPU47.TLB:TLB_shootdowns 807474 ± 2% -16.2% 676777 interrupts.CPU48.CAL:Function_call_interrupts 1101860 ± 2% -14.4% 942739 interrupts.CPU48.TLB:TLB_shootdowns 829467 -16.7% 690936 ± 2% interrupts.CPU49.CAL:Function_call_interrupts 1134255 -15.4% 960037 ± 2% interrupts.CPU49.TLB:TLB_shootdowns 824827 ± 2% -14.4% 705827 ± 2% interrupts.CPU5.CAL:Function_call_interrupts 1127087 ± 2% -13.4% 976370 ± 2% interrupts.CPU5.TLB:TLB_shootdowns 847075 -16.6% 706155 interrupts.CPU50.CAL:Function_call_interrupts 1157251 -15.3% 980134 interrupts.CPU50.TLB:TLB_shootdowns 832652 ± 3% -19.1% 674014 ± 5% interrupts.CPU51.CAL:Function_call_interrupts 1135068 ± 3% -17.6% 935107 ± 5% interrupts.CPU51.TLB:TLB_shootdowns 826673 ± 4% -15.1% 701806 ± 2% interrupts.CPU52.CAL:Function_call_interrupts 1128246 ± 4% -13.6% 974871 interrupts.CPU52.TLB:TLB_shootdowns 847987 -18.0% 695649 interrupts.CPU53.CAL:Function_call_interrupts 1159474 -16.6% 967249 interrupts.CPU53.TLB:TLB_shootdowns 814323 ± 6% -16.9% 676447 ± 2% interrupts.CPU54.CAL:Function_call_interrupts 1113039 ± 6% -15.8% 937486 ± 3% interrupts.CPU54.TLB:TLB_shootdowns 844023 -17.3% 697597 ± 2% interrupts.CPU55.CAL:Function_call_interrupts 1153947 -16.2% 966801 ± 2% interrupts.CPU55.TLB:TLB_shootdowns 849155 -18.8% 689781 ± 2% interrupts.CPU56.CAL:Function_call_interrupts 1162440 -17.5% 958985 ± 2% interrupts.CPU56.TLB:TLB_shootdowns 843499 -18.6% 686893 ± 2% interrupts.CPU57.CAL:Function_call_interrupts 1154373 -17.3% 954114 ± 2% interrupts.CPU57.TLB:TLB_shootdowns 846389 -19.8% 679064 ± 3% interrupts.CPU58.CAL:Function_call_interrupts 1157578 -18.5% 943518 ± 3% interrupts.CPU58.TLB:TLB_shootdowns 842982 -17.3% 696997 interrupts.CPU59.CAL:Function_call_interrupts 2483 ± 5% +20.4% 2989 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts 1151154 -16.0% 966879 interrupts.CPU59.TLB:TLB_shootdowns 847219 -16.3% 708938 ± 2% interrupts.CPU6.CAL:Function_call_interrupts 1158071 -15.2% 981983 ± 2% interrupts.CPU6.TLB:TLB_shootdowns 841608 -18.3% 687866 ± 3% interrupts.CPU61.CAL:Function_call_interrupts 1151090 -17.0% 955215 ± 3% interrupts.CPU61.TLB:TLB_shootdowns 848260 -19.3% 684765 ± 2% interrupts.CPU62.CAL:Function_call_interrupts 1159986 -18.0% 951359 ± 2% interrupts.CPU62.TLB:TLB_shootdowns 844262 -18.0% 692137 ± 3% interrupts.CPU63.CAL:Function_call_interrupts 1156646 -16.6% 964142 ± 3% interrupts.CPU63.TLB:TLB_shootdowns 826508 ± 2% -16.1% 693809 interrupts.CPU64.CAL:Function_call_interrupts 1129045 ± 2% -14.7% 962518 interrupts.CPU64.TLB:TLB_shootdowns 823030 ± 4% -16.0% 691454 ± 2% interrupts.CPU65.CAL:Function_call_interrupts 7841 -40.1% 4695 ± 36% interrupts.CPU65.NMI:Non-maskable_interrupts 7841 -40.1% 4695 ± 36% interrupts.CPU65.PMI:Performance_monitoring_interrupts 1122427 ± 4% -14.5% 959387 ± 2% interrupts.CPU65.TLB:TLB_shootdowns 812587 ± 2% -17.7% 668425 ± 2% interrupts.CPU66.CAL:Function_call_interrupts 1107817 ± 3% -16.2% 928466 ± 2% interrupts.CPU66.TLB:TLB_shootdowns 826927 ± 2% -17.7% 680373 ± 2% interrupts.CPU67.CAL:Function_call_interrupts 1129658 ± 2% -16.1% 947336 ± 2% interrupts.CPU67.TLB:TLB_shootdowns 842393 -23.2% 646633 ± 5% interrupts.CPU68.CAL:Function_call_interrupts 2463 ± 3% +100.2% 4933 ± 43% interrupts.CPU68.RES:Rescheduling_interrupts 1151837 -22.0% 898821 ± 5% interrupts.CPU68.TLB:TLB_shootdowns 838182 -19.6% 674051 ± 2% interrupts.CPU69.CAL:Function_call_interrupts 1145883 -18.4% 935191 ± 2% interrupts.CPU69.TLB:TLB_shootdowns 171.75 ± 2% +78.2% 306.00 ± 43% interrupts.CPU7.41:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 821848 ± 4% -16.1% 689564 ± 4% interrupts.CPU7.CAL:Function_call_interrupts 1121161 ± 5% -14.8% 955201 ± 4% interrupts.CPU7.TLB:TLB_shootdowns 838801 -19.2% 677472 ± 2% interrupts.CPU70.CAL:Function_call_interrupts 1147148 -17.9% 941986 ± 2% interrupts.CPU70.TLB:TLB_shootdowns 839068 -19.0% 679477 interrupts.CPU71.CAL:Function_call_interrupts 1146655 -17.8% 942128 interrupts.CPU71.TLB:TLB_shootdowns 836871 -20.3% 666891 interrupts.CPU72.CAL:Function_call_interrupts 1144974 -19.1% 926362 interrupts.CPU72.TLB:TLB_shootdowns 841709 -20.5% 669094 ± 2% interrupts.CPU73.CAL:Function_call_interrupts 1149707 -19.3% 927817 ± 2% interrupts.CPU73.TLB:TLB_shootdowns 845118 -21.2% 665912 ± 2% interrupts.CPU74.CAL:Function_call_interrupts 2444 ± 2% +87.4% 4580 ± 64% interrupts.CPU74.RES:Rescheduling_interrupts 1154212 -19.9% 924238 interrupts.CPU74.TLB:TLB_shootdowns 828480 -20.3% 660684 interrupts.CPU75.CAL:Function_call_interrupts 2528 ± 5% +114.8% 5431 ± 28% interrupts.CPU75.RES:Rescheduling_interrupts 1133574 -19.1% 916547 interrupts.CPU75.TLB:TLB_shootdowns 833669 ± 2% -20.8% 660103 ± 3% interrupts.CPU76.CAL:Function_call_interrupts 2494 +31.3% 3274 ± 30% interrupts.CPU76.RES:Rescheduling_interrupts 1139405 ± 2% -19.6% 915598 ± 3% interrupts.CPU76.TLB:TLB_shootdowns 810302 ± 2% -16.7% 674633 ± 2% interrupts.CPU77.CAL:Function_call_interrupts 1107301 ± 2% -15.4% 936520 ± 2% interrupts.CPU77.TLB:TLB_shootdowns 828049 ± 3% -19.3% 668023 interrupts.CPU78.CAL:Function_call_interrupts 1132076 ± 3% -18.1% 927579 ± 2% interrupts.CPU78.TLB:TLB_shootdowns 836651 -19.5% 673368 ± 2% interrupts.CPU79.CAL:Function_call_interrupts 2523 ± 3% +161.1% 6589 ± 61% interrupts.CPU79.RES:Rescheduling_interrupts 1143834 -18.2% 935910 ± 2% interrupts.CPU79.TLB:TLB_shootdowns 814224 ± 5% -13.9% 701163 ± 3% interrupts.CPU8.CAL:Function_call_interrupts 1111093 ± 5% -12.5% 972307 ± 2% interrupts.CPU8.TLB:TLB_shootdowns 806179 ± 5% -17.3% 666429 ± 3% interrupts.CPU80.CAL:Function_call_interrupts 2578 ± 9% +91.6% 4939 ± 37% interrupts.CPU80.RES:Rescheduling_interrupts 1103613 ± 5% -16.0% 926732 ± 3% interrupts.CPU80.TLB:TLB_shootdowns 841112 -21.1% 663913 interrupts.CPU81.CAL:Function_call_interrupts 2523 ± 3% +124.3% 5658 ± 42% interrupts.CPU81.RES:Rescheduling_interrupts 1148252 -19.7% 921952 interrupts.CPU81.TLB:TLB_shootdowns 816858 ± 3% -22.8% 630890 ± 2% interrupts.CPU82.CAL:Function_call_interrupts 2606 ± 7% +52.1% 3964 ± 35% interrupts.CPU82.RES:Rescheduling_interrupts 1114566 ± 3% -21.4% 876067 ± 2% interrupts.CPU82.TLB:TLB_shootdowns 831394 ± 2% -22.7% 642859 ± 2% interrupts.CPU83.CAL:Function_call_interrupts 2521 ± 2% +102.1% 5095 ± 58% interrupts.CPU83.RES:Rescheduling_interrupts 1137198 ± 2% -21.5% 893118 ± 2% interrupts.CPU83.TLB:TLB_shootdowns 843687 -21.8% 660122 interrupts.CPU84.CAL:Function_call_interrupts 2417 ± 2% +40.6% 3399 ± 29% interrupts.CPU84.RES:Rescheduling_interrupts 1154852 -20.6% 917423 interrupts.CPU84.TLB:TLB_shootdowns 820356 ± 2% -21.9% 641096 interrupts.CPU85.CAL:Function_call_interrupts 2576 ± 3% +114.7% 5530 ± 40% interrupts.CPU85.RES:Rescheduling_interrupts 1121766 ± 2% -20.7% 889141 interrupts.CPU85.TLB:TLB_shootdowns 829792 ± 2% -20.6% 658801 ± 2% interrupts.CPU86.CAL:Function_call_interrupts 1134392 ± 2% -19.4% 914649 ± 2% interrupts.CPU86.TLB:TLB_shootdowns 817593 -21.8% 638962 ± 4% interrupts.CPU87.CAL:Function_call_interrupts 2559 ± 5% +85.2% 4738 ± 35% interrupts.CPU87.RES:Rescheduling_interrupts 1119006 ± 2% -20.6% 888156 ± 4% interrupts.CPU87.TLB:TLB_shootdowns 842752 -15.2% 714778 interrupts.CPU9.CAL:Function_call_interrupts 1152075 -14.0% 990235 interrupts.CPU9.TLB:TLB_shootdowns 238.75 ± 2% -11.0% 212.50 ± 8% interrupts.IWI:IRQ_work_interrupts 239180 +23.7% 295891 ± 2% interrupts.RES:Rescheduling_interrupts 99906029 -16.4% 83542441 interrupts.TLB:TLB_shootdowns vm-scalability.throughput 1.75e+07 +-+--------------------------------------------------------------+ | ++ ++ + | 1.7e+07 +-++ + + + + :: : +++ : + + + + + +| 1.65e+07 +-+ + +++++++ + + + ++ : : +++++ +++++++ + +++++++ :++ ::+ | | : : :: + : + | 1.6e+07 +-+ +: +: + | | + + | 1.55e+07 +-+ | | | 1.5e+07 +-+ | 1.45e+07 +-+ O | | OO | 1.4e+07 +-OOOO O OOOO O O O OOO OOO | OO O O O O O O | 1.35e+07 +-+--------------------------------------------------------------+ vm-scalability.median 195000 +-+----------------------------------------------------------------+ |+ + + + + +++ : : + + + + + ++ + + ++ ++++ + +| 190000 +-+ + +++++++ + + + : : ++ + + ++ + + :+++++++ : ::+ | 185000 +-+ : : :: + + + | | +: +: | 180000 +-+ + + | 175000 +-+ | | | 170000 +-+ | 165000 +-+ O | | OO O OO OO | 160000 +-OOOO O OOOO O O O O | 155000 +O+ O O O O O O | O | 150000 +-+----------------------------------------------------------------+ vm-scalability.workload 4.9e+09 +-+---------------------------------------------------------------+ |++++ +++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++| 4.8e+09 +-+ + : + | 4.7e+09 +-+ + | | | 4.6e+09 +-+ | | | 4.5e+09 +-+ | | | 4.4e+09 +-+ O | 4.3e+09 +-+ O | | O O O OOO O OO OO | 4.2e+09 +-+OO O O O O O O | OO O O O O | 4.1e+09 +-+------------------O---O----------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/page_fault2/will-it-scale commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 2915 +14.3% 3332 will-it-scale.per_process_ops 793150 +14.3% 906519 will-it-scale.workload 9.653e+08 ± 18% -22.2% 7.514e+08 ± 23% cpuidle.C6.time 10.03 +3.1% 10.35 turbostat.RAMWatt 3.46 ± 14% -1.1 2.39 ± 6% mpstat.cpu.idle% 0.35 -0.1 0.30 mpstat.cpu.soft% 45979687 -43.9% 25810843 numa-meminfo.node0.MemFree 52890063 +38.1% 73058908 numa-meminfo.node0.MemUsed 2.484e+08 +13.5% 2.819e+08 numa-numastat.node0.local_node 2.484e+08 +13.5% 2.819e+08 numa-numastat.node0.numa_hit 1080 ± 10% -13.9% 931.00 slabinfo.skbuff_ext_cache.active_objs 1080 ± 10% -13.9% 931.00 slabinfo.skbuff_ext_cache.num_objs 11538681 -43.4% 6527183 numa-vmstat.node0.nr_free_pages 1.339e+08 +9.3% 1.463e+08 numa-vmstat.node0.numa_hit 1.339e+08 +9.3% 1.463e+08 numa-vmstat.node0.numa_local 93.75 +1.3% 95.00 vmstat.cpu.sy 61780470 -33.4% 41164435 vmstat.memory.free 132475 ± 7% +11.4% 147583 vmstat.system.in 5862400 +16.6% 6835712 ± 6% meminfo.DirectMap2M 323804 ± 12% -57.7% 136924 ± 3% meminfo.DirectMap4k 61267703 -33.0% 41060806 meminfo.MemAvailable 61882211 -32.7% 41675564 meminfo.MemFree 53469352 +37.8% 73675999 meminfo.Memused 1529004 -32.9% 1025827 proc-vmstat.nr_dirty_background_threshold 3062178 -32.9% 2054197 proc-vmstat.nr_dirty_threshold 15492186 -32.5% 10451100 proc-vmstat.nr_free_pages 2.484e+08 +13.5% 2.819e+08 proc-vmstat.numa_hit 2.484e+08 +13.5% 2.819e+08 proc-vmstat.numa_local 2.485e+08 +13.5% 2.82e+08 proc-vmstat.pgalloc_normal 2.388e+08 +14.0% 2.723e+08 proc-vmstat.pgfault 2.484e+08 +13.4% 2.817e+08 proc-vmstat.pgfree 120737 ± 45% -50.1% 60215 ± 77% sched_debug.cfs_rq:/.load.avg 0.95 ± 5% -5.9% 0.89 ± 4% sched_debug.cfs_rq:/.nr_running.avg 44.29 ± 13% +26.5% 56.04 ± 9% sched_debug.cfs_rq:/.nr_spread_over.min 104.50 ± 47% -50.6% 51.58 ± 78% sched_debug.cfs_rq:/.runnable_load_avg.avg 267.00 ± 21% -31.6% 182.61 ± 31% sched_debug.cfs_rq:/.runnable_load_avg.stddev 120728 ± 45% -50.1% 60205 ± 77% sched_debug.cfs_rq:/.runnable_weight.avg 6576590 ± 3% -12.1% 5782356 ± 5% sched_debug.cpu.avg_idle.avg 104.61 ± 47% -50.6% 51.67 ± 78% sched_debug.cpu.cpu_load[0].avg 267.19 ± 22% -31.6% 182.83 ± 32% sched_debug.cpu.cpu_load[0].stddev 103.84 ± 47% -50.4% 51.52 ± 78% sched_debug.cpu.cpu_load[1].avg 266.26 ± 22% -31.6% 182.21 ± 32% sched_debug.cpu.cpu_load[1].stddev 102.73 ± 47% -49.9% 51.45 ± 77% sched_debug.cpu.cpu_load[2].avg 264.20 ± 22% -31.5% 181.04 ± 32% sched_debug.cpu.cpu_load[2].stddev 101.37 ± 46% -49.2% 51.46 ± 76% sched_debug.cpu.cpu_load[3].avg 260.24 ± 21% -31.2% 179.12 ± 31% sched_debug.cpu.cpu_load[3].stddev 99.34 ± 45% -48.5% 51.20 ± 73% sched_debug.cpu.cpu_load[4].avg 252.89 ± 20% -30.7% 175.16 ± 29% sched_debug.cpu.cpu_load[4].stddev 121036 ± 46% -50.4% 60049 ± 76% sched_debug.cpu.load.avg 20902 ± 13% +46.6% 30647 ± 22% sched_debug.cpu.sched_count.max 2057 ± 8% +20.7% 2484 ± 11% sched_debug.cpu.sched_count.stddev 50.32 ± 8% -13.5% 43.51 ± 3% sched_debug.cpu.sched_goidle.avg 10.26 ± 2% +5.1% 10.79 perf-stat.i.MPKI 27.13 -3.3 23.80 perf-stat.i.cache-miss-rate% 88501110 ± 6% -9.5% 80099279 perf-stat.i.cache-misses 3.239e+08 ± 2% +5.1% 3.406e+08 perf-stat.i.cache-references 85.67 ± 3% -7.3% 79.44 perf-stat.i.cpu-migrations 3873 +7.2% 4153 perf-stat.i.cycles-between-cache-misses 62594916 +12.3% 70303900 perf-stat.i.iTLB-load-misses 564.36 -11.8% 497.82 ± 4% perf-stat.i.instructions-per-iTLB-miss 764984 +13.0% 864782 perf-stat.i.minor-faults 759722 +12.6% 855671 perf-stat.i.page-faults 9.84 +7.5% 10.59 perf-stat.overall.MPKI 0.72 +0.1 0.77 perf-stat.overall.branch-miss-rate% 26.64 -3.7 22.97 perf-stat.overall.cache-miss-rate% 3851 +7.7% 4149 perf-stat.overall.cycles-between-cache-misses 0.19 +0.0 0.22 perf-stat.overall.iTLB-load-miss-rate% 523.76 -13.5% 453.18 perf-stat.overall.instructions-per-iTLB-miss 12965864 -12.7% 11322179 perf-stat.overall.path-length 53760641 +8.4% 58298536 perf-stat.ps.branch-misses 81017925 -6.3% 75927473 perf-stat.ps.cache-misses 3.042e+08 +8.7% 3.306e+08 perf-stat.ps.cache-references 83.64 -6.2% 78.48 perf-stat.ps.cpu-migrations 58992694 +16.8% 68905415 perf-stat.ps.iTLB-load-misses 717099 +15.1% 825580 perf-stat.ps.minor-faults 716179 +15.4% 826644 perf-stat.ps.page-faults 36.13 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.munmap 36.12 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 36.12 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.12 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 36.12 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 36.13 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 36.13 ± 5% -5.4 30.75 ± 11% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 32.45 ± 5% -5.1 27.34 ± 11% perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 32.44 ± 5% -5.1 27.34 ± 11% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 32.32 ± 5% -5.1 27.23 ± 11% perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region.__do_munmap 32.28 ± 5% -5.1 27.20 ± 11% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region 31.73 ± 4% -4.9 26.86 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 31.73 ± 4% -4.9 26.86 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range 0.60 ± 4% +0.2 0.81 ± 11% perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.66 ± 4% +0.2 0.89 ± 11% perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 0.67 ± 3% +0.3 0.94 ± 11% perf-profile.calltrace.cycles-pp.copy_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 0.40 ± 57% +0.3 0.71 ± 11% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault 55.72 ± 2% +3.7 59.41 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 55.71 ± 2% +3.7 59.40 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte 56.61 ± 2% +3.7 60.31 ± 4% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault 56.64 ± 2% +3.7 60.36 ± 4% perf-profile.calltrace.cycles-pp.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 57.18 ± 2% +3.9 61.09 ± 4% perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 57.23 ± 2% +3.9 61.15 ± 4% perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 60.26 ± 2% +4.7 64.99 ± 4% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 60.47 ± 2% +4.8 65.28 ± 4% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault.testcase 61.22 ± 2% +5.0 66.22 ± 4% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault.testcase 61.46 ± 2% +5.1 66.53 ± 4% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.testcase 61.78 ± 2% +5.2 66.97 ± 4% perf-profile.calltrace.cycles-pp.page_fault.testcase 62.35 ± 2% +5.4 67.76 ± 4% perf-profile.calltrace.cycles-pp.testcase 803355 +11.2% 893581 ± 15% interrupts.CAL:Function_call_interrupts 128554 ± 9% +16.2% 149339 ± 4% interrupts.CPU1.LOC:Local_timer_interrupts 4761 ± 36% -35.5% 3071 ± 24% interrupts.CPU1.NMI:Non-maskable_interrupts 4761 ± 36% -35.5% 3071 ± 24% interrupts.CPU1.PMI:Performance_monitoring_interrupts 128961 ± 7% +17.4% 151405 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts 5570 ± 27% -43.0% 3173 ± 23% interrupts.CPU10.NMI:Non-maskable_interrupts 5570 ± 27% -43.0% 3173 ± 23% interrupts.CPU10.PMI:Performance_monitoring_interrupts 128081 ± 7% +16.5% 149184 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts 20.50 ± 68% +16892.7% 3483 ±168% interrupts.CPU101.RES:Rescheduling_interrupts 713.75 ±168% -98.9% 7.75 ± 99% interrupts.CPU105.RES:Rescheduling_interrupts 126758 ± 8% +16.5% 147656 ± 2% interrupts.CPU106.LOC:Local_timer_interrupts 127972 ± 8% +13.6% 145433 interrupts.CPU107.LOC:Local_timer_interrupts 127431 ± 8% +15.8% 147616 interrupts.CPU11.LOC:Local_timer_interrupts 10.75 ± 16% +416.3% 55.50 ± 57% interrupts.CPU11.RES:Rescheduling_interrupts 662.50 ±155% -97.6% 15.75 ±137% interrupts.CPU110.RES:Rescheduling_interrupts 128295 ± 8% +13.9% 146152 interrupts.CPU111.LOC:Local_timer_interrupts 4650 ± 26% -32.3% 3147 ± 24% interrupts.CPU111.NMI:Non-maskable_interrupts 4650 ± 26% -32.3% 3147 ± 24% interrupts.CPU111.PMI:Performance_monitoring_interrupts 127789 ± 7% +13.8% 145367 interrupts.CPU113.LOC:Local_timer_interrupts 128012 ± 8% +14.3% 146378 interrupts.CPU115.LOC:Local_timer_interrupts 4735 ± 27% -33.6% 3144 ± 24% interrupts.CPU115.NMI:Non-maskable_interrupts 4735 ± 27% -33.6% 3144 ± 24% interrupts.CPU115.PMI:Performance_monitoring_interrupts 129902 ± 7% +15.2% 149697 ± 3% interrupts.CPU116.LOC:Local_timer_interrupts 126438 ± 7% +14.8% 145140 interrupts.CPU117.LOC:Local_timer_interrupts 127391 ± 8% +17.5% 149689 interrupts.CPU120.LOC:Local_timer_interrupts 128904 ± 9% +17.1% 150954 ± 3% interrupts.CPU121.LOC:Local_timer_interrupts 2956 +14.1% 3372 ± 13% interrupts.CPU122.CAL:Function_call_interrupts 129866 ± 9% +15.6% 150186 interrupts.CPU122.LOC:Local_timer_interrupts 5710 ± 31% -44.0% 3195 ± 24% interrupts.CPU124.NMI:Non-maskable_interrupts 5710 ± 31% -44.0% 3195 ± 24% interrupts.CPU124.PMI:Performance_monitoring_interrupts 128571 ± 8% +21.0% 155526 ± 9% interrupts.CPU125.LOC:Local_timer_interrupts 2962 +14.0% 3375 ± 14% interrupts.CPU126.CAL:Function_call_interrupts 128798 ± 7% +13.0% 145548 interrupts.CPU126.LOC:Local_timer_interrupts 5649 ± 33% -44.1% 3155 ± 26% interrupts.CPU127.NMI:Non-maskable_interrupts 5649 ± 33% -44.1% 3155 ± 26% interrupts.CPU127.PMI:Performance_monitoring_interrupts 128109 ± 6% +13.5% 145341 interrupts.CPU13.LOC:Local_timer_interrupts 131863 ± 9% +14.2% 150633 ± 3% interrupts.CPU130.LOC:Local_timer_interrupts 5570 ± 30% -43.8% 3129 ± 25% interrupts.CPU131.NMI:Non-maskable_interrupts 5570 ± 30% -43.8% 3129 ± 25% interrupts.CPU131.PMI:Performance_monitoring_interrupts 71.50 ± 51% -81.5% 13.25 ± 75% interrupts.CPU131.RES:Rescheduling_interrupts 128700 ± 8% +18.3% 152226 ± 5% interrupts.CPU132.LOC:Local_timer_interrupts 128990 ± 8% +35.6% 174872 ± 27% interrupts.CPU133.LOC:Local_timer_interrupts 127866 ± 8% +15.0% 147010 interrupts.CPU134.LOC:Local_timer_interrupts 127582 ± 8% +18.3% 150886 ± 6% interrupts.CPU137.LOC:Local_timer_interrupts 6576 ± 19% -52.5% 3121 ± 23% interrupts.CPU137.NMI:Non-maskable_interrupts 6576 ± 19% -52.5% 3121 ± 23% interrupts.CPU137.PMI:Performance_monitoring_interrupts 127377 ± 8% +15.2% 146767 interrupts.CPU139.LOC:Local_timer_interrupts 127170 ± 8% +16.3% 147916 interrupts.CPU140.LOC:Local_timer_interrupts 128637 ± 8% +14.2% 146946 interrupts.CPU141.LOC:Local_timer_interrupts 127547 ± 8% +18.5% 151152 ± 8% interrupts.CPU142.LOC:Local_timer_interrupts 128792 ± 8% +13.0% 145510 interrupts.CPU143.LOC:Local_timer_interrupts 128235 ± 7% +17.0% 149998 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts 129104 ± 9% +14.0% 147130 interrupts.CPU147.LOC:Local_timer_interrupts 128110 ± 7% +13.0% 144716 interrupts.CPU149.LOC:Local_timer_interrupts 5675 ± 32% -44.9% 3126 ± 25% interrupts.CPU150.NMI:Non-maskable_interrupts 5675 ± 32% -44.9% 3126 ± 25% interrupts.CPU150.PMI:Performance_monitoring_interrupts 2538 ± 53% -96.7% 82.75 ± 49% interrupts.CPU156.RES:Rescheduling_interrupts 130214 ± 6% +15.1% 149920 ± 3% interrupts.CPU158.LOC:Local_timer_interrupts 130122 ± 5% +12.5% 146337 interrupts.CPU159.LOC:Local_timer_interrupts 128290 ± 8% +13.2% 145228 interrupts.CPU16.LOC:Local_timer_interrupts 2824 ±171% -99.4% 15.75 ±103% interrupts.CPU160.RES:Rescheduling_interrupts 129740 ± 9% +13.6% 147325 ± 2% interrupts.CPU161.LOC:Local_timer_interrupts 6568 ± 19% -52.0% 3149 ± 25% interrupts.CPU161.NMI:Non-maskable_interrupts 6568 ± 19% -52.0% 3149 ± 25% interrupts.CPU161.PMI:Performance_monitoring_interrupts 130062 ± 9% +12.0% 145652 interrupts.CPU162.LOC:Local_timer_interrupts 127653 ± 8% +14.2% 145832 interrupts.CPU163.LOC:Local_timer_interrupts 131743 ± 8% +13.5% 149536 ± 4% interrupts.CPU166.LOC:Local_timer_interrupts 128326 ± 6% +15.7% 148474 ± 4% interrupts.CPU167.LOC:Local_timer_interrupts 7647 ± 8% -58.9% 3146 ± 24% interrupts.CPU168.NMI:Non-maskable_interrupts 7647 ± 8% -58.9% 3146 ± 24% interrupts.CPU168.PMI:Performance_monitoring_interrupts 128803 ± 8% +19.9% 154482 ± 3% interrupts.CPU169.LOC:Local_timer_interrupts 3.25 ± 54% +3453.8% 115.50 ± 80% interrupts.CPU169.RES:Rescheduling_interrupts 5827 ± 38% -45.7% 3167 ± 25% interrupts.CPU170.NMI:Non-maskable_interrupts 5827 ± 38% -45.7% 3167 ± 25% interrupts.CPU170.PMI:Performance_monitoring_interrupts 128888 ± 6% +11.6% 143901 interrupts.CPU173.LOC:Local_timer_interrupts 45.50 ± 87% -70.9% 13.25 ± 64% interrupts.CPU173.RES:Rescheduling_interrupts 126325 ± 7% +16.2% 146781 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts 4.50 ± 45% +5661.1% 259.25 ±127% interrupts.CPU174.RES:Rescheduling_interrupts 128222 ± 8% +14.8% 147140 interrupts.CPU177.LOC:Local_timer_interrupts 4665 ± 27% +37.4% 6411 ± 25% interrupts.CPU178.NMI:Non-maskable_interrupts 4665 ± 27% +37.4% 6411 ± 25% interrupts.CPU178.PMI:Performance_monitoring_interrupts 128503 ± 8% +13.5% 145811 interrupts.CPU179.LOC:Local_timer_interrupts 5625 ± 31% -43.8% 3163 ± 24% interrupts.CPU180.NMI:Non-maskable_interrupts 5625 ± 31% -43.8% 3163 ± 24% interrupts.CPU180.PMI:Performance_monitoring_interrupts 129081 ± 7% +14.0% 147215 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts 129717 ± 7% +12.3% 145686 interrupts.CPU182.LOC:Local_timer_interrupts 128310 ± 8% +14.4% 146837 interrupts.CPU183.LOC:Local_timer_interrupts 129388 ± 8% +15.1% 148976 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts 127563 ± 8% +34.2% 171190 ± 26% interrupts.CPU187.LOC:Local_timer_interrupts 127835 ± 7% +16.5% 148916 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 127978 ± 8% +17.2% 150014 ± 3% interrupts.CPU189.LOC:Local_timer_interrupts 130936 ± 6% +11.5% 146033 interrupts.CPU19.LOC:Local_timer_interrupts 131210 ± 8% +15.1% 151087 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts 135742 ± 12% +13.7% 154404 ± 3% interrupts.CPU192.LOC:Local_timer_interrupts 128820 ± 9% +21.4% 156382 ± 10% interrupts.CPU193.LOC:Local_timer_interrupts 129645 ± 8% +16.3% 150819 ± 3% interrupts.CPU200.LOC:Local_timer_interrupts 127699 ± 8% +13.4% 144810 interrupts.CPU202.LOC:Local_timer_interrupts 130293 ± 10% +14.5% 149212 ± 3% interrupts.CPU205.LOC:Local_timer_interrupts 129036 ± 7% +15.3% 148733 ± 4% interrupts.CPU206.LOC:Local_timer_interrupts 126846 ± 8% +15.5% 146475 interrupts.CPU207.LOC:Local_timer_interrupts 128853 ± 8% +14.6% 147663 interrupts.CPU208.LOC:Local_timer_interrupts 13.75 ± 27% +30838.2% 4254 ± 91% interrupts.CPU208.RES:Rescheduling_interrupts 128632 ± 9% +16.5% 149849 ± 4% interrupts.CPU209.LOC:Local_timer_interrupts 9.75 ± 92% +8276.9% 816.75 ±164% interrupts.CPU210.RES:Rescheduling_interrupts 127942 ± 7% +16.0% 148469 ± 2% interrupts.CPU214.LOC:Local_timer_interrupts 126652 ± 7% +15.7% 146589 interrupts.CPU215.LOC:Local_timer_interrupts 128490 ± 7% +13.2% 145406 interrupts.CPU217.LOC:Local_timer_interrupts 6552 ± 19% -38.0% 4061 ± 44% interrupts.CPU219.NMI:Non-maskable_interrupts 6552 ± 19% -38.0% 4061 ± 44% interrupts.CPU219.PMI:Performance_monitoring_interrupts 130827 ± 7% +12.3% 146900 interrupts.CPU22.LOC:Local_timer_interrupts 5634 ± 26% -44.5% 3127 ± 25% interrupts.CPU221.NMI:Non-maskable_interrupts 5634 ± 26% -44.5% 3127 ± 25% interrupts.CPU221.PMI:Performance_monitoring_interrupts 129557 ± 7% +13.6% 147201 interrupts.CPU227.LOC:Local_timer_interrupts 482.25 ±166% -98.8% 6.00 ± 56% interrupts.CPU227.RES:Rescheduling_interrupts 5899 ± 39% -47.6% 3093 ± 25% interrupts.CPU228.NMI:Non-maskable_interrupts 5899 ± 39% -47.6% 3093 ± 25% interrupts.CPU228.PMI:Performance_monitoring_interrupts 130626 ± 9% +13.3% 147950 ± 2% interrupts.CPU229.LOC:Local_timer_interrupts 128668 ± 6% +14.1% 146842 interrupts.CPU23.LOC:Local_timer_interrupts 129548 ± 9% +12.3% 145442 interrupts.CPU230.LOC:Local_timer_interrupts 127610 ± 8% +14.2% 145697 ± 2% interrupts.CPU231.LOC:Local_timer_interrupts 129211 ± 4% +14.4% 147853 ± 2% interrupts.CPU235.LOC:Local_timer_interrupts 128867 ± 8% +13.7% 146534 ± 2% interrupts.CPU236.LOC:Local_timer_interrupts 127461 ± 8% +19.6% 152411 ± 4% interrupts.CPU237.LOC:Local_timer_interrupts 20.25 ±112% +29906.2% 6076 ±104% interrupts.CPU237.RES:Rescheduling_interrupts 5573 ± 26% -43.0% 3174 ± 26% interrupts.CPU239.NMI:Non-maskable_interrupts 5573 ± 26% -43.0% 3174 ± 26% interrupts.CPU239.PMI:Performance_monitoring_interrupts 129013 ± 5% +12.7% 145372 interrupts.CPU24.LOC:Local_timer_interrupts 126001 ± 7% +18.4% 149203 ± 2% interrupts.CPU242.LOC:Local_timer_interrupts 129851 ± 7% +11.5% 144833 interrupts.CPU243.LOC:Local_timer_interrupts 128229 ± 7% +14.6% 147008 ± 2% interrupts.CPU249.LOC:Local_timer_interrupts 129716 ± 9% +14.0% 147867 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 4754 ± 34% -36.4% 3024 ± 26% interrupts.CPU25.NMI:Non-maskable_interrupts 4754 ± 34% -36.4% 3024 ± 26% interrupts.CPU25.PMI:Performance_monitoring_interrupts 130445 ± 7% +16.4% 151819 ± 2% interrupts.CPU252.LOC:Local_timer_interrupts 128360 ± 6% +13.8% 146119 interrupts.CPU253.LOC:Local_timer_interrupts 127846 ± 7% +15.2% 147293 interrupts.CPU254.LOC:Local_timer_interrupts 126614 ± 7% +16.7% 147763 ± 2% interrupts.CPU255.LOC:Local_timer_interrupts 128654 ± 8% +19.3% 153440 ± 4% interrupts.CPU257.LOC:Local_timer_interrupts 128616 ± 9% +21.5% 156232 ± 10% interrupts.CPU261.LOC:Local_timer_interrupts 6470 ± 20% -51.3% 3154 ± 24% interrupts.CPU267.NMI:Non-maskable_interrupts 6470 ± 20% -51.3% 3154 ± 24% interrupts.CPU267.PMI:Performance_monitoring_interrupts 131577 ± 9% +16.0% 152612 ± 4% interrupts.CPU268.LOC:Local_timer_interrupts 126663 ± 8% +14.6% 145204 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 127569 ± 7% +13.8% 145234 interrupts.CPU270.LOC:Local_timer_interrupts 129927 ± 8% +12.6% 146311 interrupts.CPU271.LOC:Local_timer_interrupts 127514 ± 8% +14.2% 145611 interrupts.CPU3.LOC:Local_timer_interrupts 132021 ± 7% +13.8% 150233 ± 4% interrupts.CPU30.LOC:Local_timer_interrupts 129141 ± 7% +16.6% 150542 ± 3% interrupts.CPU31.LOC:Local_timer_interrupts 127472 ± 8% +18.5% 151101 ± 3% interrupts.CPU33.LOC:Local_timer_interrupts 12.25 ±103% +693.9% 97.25 ±106% interrupts.CPU33.RES:Rescheduling_interrupts 126536 ± 8% +16.2% 147005 interrupts.CPU38.LOC:Local_timer_interrupts 16.50 ± 99% +3939.4% 666.50 ±161% interrupts.CPU38.RES:Rescheduling_interrupts 127088 ± 8% +15.7% 147050 interrupts.CPU4.LOC:Local_timer_interrupts 128312 ± 7% +13.6% 145710 interrupts.CPU43.LOC:Local_timer_interrupts 128263 ± 7% +14.3% 146558 interrupts.CPU44.LOC:Local_timer_interrupts 128207 ± 8% +14.1% 146233 interrupts.CPU45.LOC:Local_timer_interrupts 5717 ± 37% -45.6% 3112 ± 24% interrupts.CPU45.NMI:Non-maskable_interrupts 5717 ± 37% -45.6% 3112 ± 24% interrupts.CPU45.PMI:Performance_monitoring_interrupts 128850 ± 8% +14.1% 147007 interrupts.CPU47.LOC:Local_timer_interrupts 129259 ± 7% +16.3% 150341 ± 3% interrupts.CPU48.LOC:Local_timer_interrupts 127796 ± 8% +14.2% 145996 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts 128889 ± 9% +19.6% 154173 ± 5% interrupts.CPU52.LOC:Local_timer_interrupts 128943 ± 7% +19.6% 154187 ± 6% interrupts.CPU53.LOC:Local_timer_interrupts 5588 ± 28% -44.0% 3131 ± 24% interrupts.CPU53.NMI:Non-maskable_interrupts 5588 ± 28% -44.0% 3131 ± 24% interrupts.CPU53.PMI:Performance_monitoring_interrupts 129251 ± 9% +16.6% 150732 ± 6% interrupts.CPU57.LOC:Local_timer_interrupts 128776 ± 8% +13.1% 145652 interrupts.CPU58.LOC:Local_timer_interrupts 83.50 ± 26% -62.3% 31.50 ±102% interrupts.CPU60.RES:Rescheduling_interrupts 4663 ± 28% -33.1% 3119 ± 24% interrupts.CPU61.NMI:Non-maskable_interrupts 4663 ± 28% -33.1% 3119 ± 24% interrupts.CPU61.PMI:Performance_monitoring_interrupts 4867 ±156% -99.5% 25.25 ±103% interrupts.CPU63.RES:Rescheduling_interrupts 129466 ± 8% +15.0% 148886 interrupts.CPU64.LOC:Local_timer_interrupts 127414 ± 8% +14.3% 145648 interrupts.CPU66.LOC:Local_timer_interrupts 130554 ± 8% +14.1% 148979 ± 3% interrupts.CPU68.LOC:Local_timer_interrupts 129542 ± 9% +16.7% 151239 ± 6% interrupts.CPU69.LOC:Local_timer_interrupts 127572 ± 8% +16.9% 149140 ± 5% interrupts.CPU7.LOC:Local_timer_interrupts 127408 ± 7% +14.9% 146415 interrupts.CPU71.LOC:Local_timer_interrupts 127501 ± 8% +20.7% 153954 ± 5% interrupts.CPU72.LOC:Local_timer_interrupts 6724 ± 28% -52.7% 3181 ± 25% interrupts.CPU72.NMI:Non-maskable_interrupts 6724 ± 28% -52.7% 3181 ± 25% interrupts.CPU72.PMI:Performance_monitoring_interrupts 129276 ± 9% +13.1% 146167 interrupts.CPU73.LOC:Local_timer_interrupts 126917 ± 8% +15.4% 146489 ± 3% interrupts.CPU75.LOC:Local_timer_interrupts 11.25 ± 39% +8428.9% 959.50 ±166% interrupts.CPU78.RES:Rescheduling_interrupts 126883 ± 7% +15.4% 146455 interrupts.CPU79.LOC:Local_timer_interrupts 6.00 ± 88% +21337.5% 1286 ±171% interrupts.CPU79.RES:Rescheduling_interrupts 5641 ± 34% -44.2% 3148 ± 24% interrupts.CPU8.NMI:Non-maskable_interrupts 5641 ± 34% -44.2% 3148 ± 24% interrupts.CPU8.PMI:Performance_monitoring_interrupts 128132 ± 6% +13.5% 145464 interrupts.CPU80.LOC:Local_timer_interrupts 749.50 ±163% -98.5% 11.00 ± 79% interrupts.CPU80.RES:Rescheduling_interrupts 127226 ± 8% +14.4% 145581 interrupts.CPU81.LOC:Local_timer_interrupts 130504 ± 7% +11.9% 145970 interrupts.CPU87.LOC:Local_timer_interrupts 129211 ± 7% +14.6% 148092 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts 128400 ± 7% +14.7% 147294 interrupts.CPU91.LOC:Local_timer_interrupts 128902 ± 8% +14.2% 147189 interrupts.CPU93.LOC:Local_timer_interrupts 127135 ± 8% +14.3% 145313 ± 2% interrupts.CPU95.LOC:Local_timer_interrupts 6.50 ± 23% +1.2e+05% 7818 ±172% interrupts.CPU95.RES:Rescheduling_interrupts 130341 ± 4% +14.1% 148675 ± 3% interrupts.CPU99.LOC:Local_timer_interrupts 49496 +13.1% 55992 softirqs.CPU0.TIMER 50203 ± 2% +12.0% 56224 ± 2% softirqs.CPU1.TIMER 50297 +13.1% 56907 softirqs.CPU10.TIMER 50693 +11.3% 56417 softirqs.CPU101.TIMER 50621 +10.9% 56163 softirqs.CPU102.TIMER 50449 +12.5% 56766 ± 2% softirqs.CPU103.TIMER 47997 ± 9% +18.0% 56661 softirqs.CPU104.TIMER 51068 +18.4% 60460 ± 12% softirqs.CPU105.TIMER 50247 ± 2% +12.6% 56554 softirqs.CPU107.TIMER 49883 +13.2% 56457 softirqs.CPU108.TIMER 50698 +12.4% 56966 softirqs.CPU110.TIMER 50262 +12.9% 56763 softirqs.CPU111.TIMER 50644 +12.8% 57138 softirqs.CPU112.TIMER 50408 +12.6% 56782 softirqs.CPU113.TIMER 50192 ± 2% +19.1% 59754 ± 10% softirqs.CPU114.TIMER 49852 +13.8% 56747 ± 2% softirqs.CPU115.TIMER 50216 +12.0% 56220 softirqs.CPU116.TIMER 50039 ± 2% +14.9% 57481 ± 2% softirqs.CPU117.TIMER 50217 +13.0% 56736 softirqs.CPU118.TIMER 50306 +11.3% 56002 softirqs.CPU119.TIMER 50401 +11.4% 56167 softirqs.CPU120.TIMER 49867 +13.0% 56338 softirqs.CPU121.TIMER 50066 +13.2% 56669 softirqs.CPU122.TIMER 50019 +18.4% 59200 ± 7% softirqs.CPU123.TIMER 49970 +13.1% 56504 softirqs.CPU124.TIMER 49978 +12.9% 56418 softirqs.CPU125.TIMER 50794 ± 2% +11.6% 56701 softirqs.CPU126.TIMER 50111 ± 2% +12.8% 56534 softirqs.CPU127.TIMER 50373 ± 2% +11.9% 56371 softirqs.CPU129.TIMER 5366 ± 99% -64.1% 1926 ± 3% softirqs.CPU13.RCU 50497 +11.7% 56411 softirqs.CPU131.TIMER 50737 +11.5% 56581 softirqs.CPU132.TIMER 50183 +12.7% 56574 softirqs.CPU134.TIMER 49719 ± 2% +14.4% 56899 softirqs.CPU135.TIMER 48773 ± 2% +14.6% 55894 ± 2% softirqs.CPU136.TIMER 48773 ± 2% +15.1% 56147 softirqs.CPU137.TIMER 50191 ± 2% +13.3% 56855 softirqs.CPU138.TIMER 50102 ± 2% +12.8% 56536 softirqs.CPU139.TIMER 49571 +13.6% 56313 softirqs.CPU140.TIMER 49804 +12.8% 56187 softirqs.CPU141.TIMER 50039 +13.0% 56540 softirqs.CPU142.TIMER 50265 ± 2% +12.6% 56607 softirqs.CPU143.TIMER 50086 +12.2% 56188 softirqs.CPU144.TIMER 50139 +12.1% 56219 softirqs.CPU145.TIMER 50414 ± 2% +11.7% 56290 softirqs.CPU146.TIMER 50427 +11.1% 56018 softirqs.CPU147.TIMER 50446 +12.0% 56482 softirqs.CPU148.TIMER 50044 +13.0% 56557 softirqs.CPU149.TIMER 51281 +10.9% 56865 softirqs.CPU15.TIMER 50204 +11.5% 55980 softirqs.CPU150.TIMER 50028 +11.8% 55934 ± 2% softirqs.CPU151.TIMER 50095 +12.1% 56156 softirqs.CPU152.TIMER 50003 +12.2% 56082 softirqs.CPU153.TIMER 50505 +12.5% 56805 softirqs.CPU154.TIMER 50183 +12.4% 56394 softirqs.CPU155.TIMER 48906 +12.1% 54837 softirqs.CPU156.TIMER 48174 +13.9% 54885 ± 2% softirqs.CPU157.TIMER 49882 +12.7% 56219 softirqs.CPU158.TIMER 50167 ± 2% +12.0% 56169 softirqs.CPU159.TIMER 50816 +12.7% 57282 softirqs.CPU16.TIMER 49819 +13.1% 56365 softirqs.CPU160.TIMER 50038 +12.4% 56240 softirqs.CPU161.TIMER 49924 +11.8% 55830 softirqs.CPU162.TIMER 49983 +11.8% 55883 softirqs.CPU163.TIMER 49400 +13.5% 56049 softirqs.CPU164.TIMER 49342 +14.1% 56287 softirqs.CPU165.TIMER 49521 +12.7% 55827 softirqs.CPU166.TIMER 49884 +12.1% 55940 softirqs.CPU167.TIMER 50051 +11.8% 55952 softirqs.CPU169.TIMER 51371 +12.1% 57592 softirqs.CPU17.TIMER 49744 +18.1% 58735 ± 5% softirqs.CPU170.TIMER 49787 +12.5% 55986 softirqs.CPU171.TIMER 50061 +11.9% 56038 softirqs.CPU173.TIMER 49801 ± 2% +12.1% 55831 ± 2% softirqs.CPU174.TIMER 49420 +13.5% 56112 softirqs.CPU175.TIMER 49342 ± 2% +13.1% 55810 softirqs.CPU176.TIMER 49625 +12.6% 55872 softirqs.CPU177.TIMER 50144 ± 2% +11.4% 55862 softirqs.CPU178.TIMER 50219 +11.5% 55988 softirqs.CPU179.TIMER 51029 +12.8% 57572 softirqs.CPU18.TIMER 49532 +19.5% 59187 ± 8% softirqs.CPU180.TIMER 49870 +11.4% 55565 ± 2% softirqs.CPU181.TIMER 49587 +13.0% 56053 softirqs.CPU182.TIMER 49772 ± 2% +13.7% 56604 softirqs.CPU183.TIMER 49529 ± 2% +12.8% 55878 softirqs.CPU184.TIMER 49655 +12.1% 55653 softirqs.CPU185.TIMER 50195 ± 2% +12.6% 56538 softirqs.CPU186.TIMER 49682 +12.4% 55850 ± 2% softirqs.CPU187.TIMER 50004 +11.4% 55710 softirqs.CPU188.TIMER 49724 +12.9% 56116 softirqs.CPU189.TIMER 50861 +13.2% 57554 softirqs.CPU19.TIMER 49716 +13.0% 56157 softirqs.CPU190.TIMER 49478 +13.4% 56113 softirqs.CPU191.TIMER 49650 +12.4% 55787 softirqs.CPU192.TIMER 49762 +20.0% 59719 ± 10% softirqs.CPU193.TIMER 48048 ± 3% +16.4% 55942 softirqs.CPU195.TIMER 49058 ± 3% +13.8% 55850 softirqs.CPU196.TIMER 49382 ± 3% +12.9% 55759 softirqs.CPU197.TIMER 49805 +12.4% 55963 softirqs.CPU198.TIMER 49914 +11.4% 55599 softirqs.CPU199.TIMER 51598 +15.7% 59693 ± 7% softirqs.CPU2.TIMER 49383 +11.6% 55092 softirqs.CPU20.TIMER 49843 +11.2% 55423 softirqs.CPU201.TIMER 49076 ± 2% +14.0% 55956 softirqs.CPU202.TIMER 49591 +12.3% 55672 softirqs.CPU203.TIMER 48302 +13.2% 54700 softirqs.CPU204.TIMER 48478 +13.1% 54810 ± 2% softirqs.CPU205.TIMER 49371 +12.6% 55579 softirqs.CPU206.TIMER 49458 +12.8% 55801 softirqs.CPU207.TIMER 49573 ± 2% +11.7% 55362 ± 2% softirqs.CPU208.TIMER 48986 +13.3% 55520 softirqs.CPU209.TIMER 49015 +14.3% 56048 softirqs.CPU21.TIMER 50680 ± 4% +10.0% 55756 softirqs.CPU210.TIMER 49834 ± 2% +13.6% 56612 ± 2% softirqs.CPU211.TIMER 49370 +13.1% 55815 softirqs.CPU212.TIMER 49365 +13.2% 55894 softirqs.CPU213.TIMER 50003 +12.0% 55999 softirqs.CPU214.TIMER 49605 +12.2% 55640 softirqs.CPU215.TIMER 49881 +12.3% 56030 softirqs.CPU216.TIMER 49699 +12.6% 55983 softirqs.CPU217.TIMER 49504 ± 2% +12.4% 55642 softirqs.CPU218.TIMER 49288 ± 2% +12.5% 55431 ± 2% softirqs.CPU219.TIMER 51162 ± 2% +11.8% 57194 softirqs.CPU22.TIMER 49659 +17.5% 58356 ± 6% softirqs.CPU220.TIMER 50409 ± 3% +10.5% 55720 softirqs.CPU221.TIMER 49491 +12.5% 55669 softirqs.CPU222.TIMER 49204 ± 2% +12.8% 55494 softirqs.CPU223.TIMER 48153 ± 2% +12.3% 54055 ± 2% softirqs.CPU224.TIMER 49270 ± 2% +13.2% 55763 softirqs.CPU226.TIMER 49358 +12.7% 55635 softirqs.CPU227.TIMER 49444 +12.7% 55713 ± 2% softirqs.CPU228.TIMER 48998 +13.3% 55507 softirqs.CPU229.TIMER 51061 ± 2% +11.9% 57145 softirqs.CPU23.TIMER 49316 +12.8% 55640 softirqs.CPU230.TIMER 48910 ± 2% +12.7% 55119 softirqs.CPU231.TIMER 48907 ± 2% +13.5% 55508 softirqs.CPU232.TIMER 49109 +13.6% 55785 softirqs.CPU234.TIMER 49155 +13.2% 55655 softirqs.CPU235.TIMER 49343 +11.2% 54855 ± 2% softirqs.CPU236.TIMER 49362 +13.2% 55856 softirqs.CPU237.TIMER 49838 +21.8% 60682 ± 12% softirqs.CPU238.TIMER 49352 +11.5% 55017 ± 2% softirqs.CPU239.TIMER 50680 +15.0% 58259 ± 2% softirqs.CPU24.TIMER 48924 ± 2% +12.8% 55189 softirqs.CPU240.TIMER 48948 ± 2% +12.9% 55249 softirqs.CPU241.TIMER 49116 +12.5% 55243 softirqs.CPU242.TIMER 49129 +12.6% 55302 softirqs.CPU243.TIMER 49044 ± 2% +13.2% 55529 softirqs.CPU244.TIMER 49397 ± 2% +12.2% 55416 softirqs.CPU245.TIMER 49260 +12.7% 55531 ± 2% softirqs.CPU246.TIMER 49609 ± 3% +12.2% 55683 softirqs.CPU247.TIMER 49301 +11.3% 54860 ± 2% softirqs.CPU248.TIMER 49298 +11.6% 55019 ± 2% softirqs.CPU249.TIMER 51270 +12.5% 57662 softirqs.CPU25.TIMER 48845 +13.1% 55253 softirqs.CPU250.TIMER 49107 +13.3% 55645 softirqs.CPU251.TIMER 49154 +12.2% 55149 softirqs.CPU252.TIMER 48924 +13.0% 55301 softirqs.CPU253.TIMER 48894 +13.7% 55596 softirqs.CPU254.TIMER 48916 +12.6% 55091 ± 2% softirqs.CPU255.TIMER 49172 ± 2% +12.7% 55426 ± 2% softirqs.CPU256.TIMER 48976 +13.5% 55580 softirqs.CPU257.TIMER 49102 +12.5% 55228 softirqs.CPU258.TIMER 49074 +12.7% 55295 softirqs.CPU259.TIMER 50804 +12.8% 57286 softirqs.CPU26.TIMER 49010 +12.5% 55117 softirqs.CPU260.TIMER 49962 ± 2% +18.9% 59384 ± 10% softirqs.CPU261.TIMER 48889 ± 2% +13.2% 55334 softirqs.CPU262.TIMER 48462 ± 2% +14.1% 55298 softirqs.CPU263.TIMER 47236 ± 7% +18.0% 55757 ± 3% softirqs.CPU264.TIMER 48878 ± 2% +13.9% 55683 softirqs.CPU265.TIMER 49213 +13.1% 55646 softirqs.CPU266.TIMER 48790 ± 2% +12.6% 54931 softirqs.CPU267.TIMER 48757 +11.4% 54322 softirqs.CPU268.TIMER 48439 +13.0% 54722 softirqs.CPU269.TIMER 50756 +12.9% 57295 softirqs.CPU27.TIMER 49335 +11.6% 55072 softirqs.CPU270.TIMER 48693 +13.6% 55303 softirqs.CPU271.TIMER 50783 +11.9% 56830 softirqs.CPU28.TIMER 50486 +13.3% 57198 softirqs.CPU29.TIMER 51064 ± 2% +13.2% 57782 softirqs.CPU3.TIMER 50542 ± 2% +13.3% 57288 softirqs.CPU31.TIMER 50817 +12.7% 57294 ± 2% softirqs.CPU32.TIMER 50899 ± 2% +11.6% 56821 ± 2% softirqs.CPU33.TIMER 51145 +10.7% 56597 ± 3% softirqs.CPU34.TIMER 50551 +12.5% 56860 ± 2% softirqs.CPU35.TIMER 50406 ± 2% +12.4% 56670 ± 2% softirqs.CPU36.TIMER 50864 ± 2% +12.8% 57355 softirqs.CPU37.TIMER 50905 +12.1% 57074 softirqs.CPU38.TIMER 50677 +12.8% 57162 softirqs.CPU39.TIMER 52148 ± 2% +10.1% 57402 softirqs.CPU4.TIMER 48420 ± 9% +18.0% 57129 softirqs.CPU40.TIMER 50678 +12.8% 57150 softirqs.CPU42.TIMER 51050 +11.9% 57137 softirqs.CPU43.TIMER 50759 +12.1% 56892 ± 2% softirqs.CPU44.TIMER 50892 +11.8% 56917 softirqs.CPU45.TIMER 50854 +11.9% 56886 softirqs.CPU46.TIMER 50144 +13.0% 56677 softirqs.CPU48.TIMER 5830 ±110% -70.2% 1737 ± 6% softirqs.CPU49.SCHED 50856 +11.8% 56841 softirqs.CPU49.TIMER 50758 +13.5% 57597 softirqs.CPU5.TIMER 51028 +12.6% 57433 softirqs.CPU50.TIMER 50837 +12.1% 56978 ± 2% softirqs.CPU51.TIMER 50892 +11.9% 56945 softirqs.CPU52.TIMER 51146 +11.7% 57136 softirqs.CPU53.TIMER 50100 +13.4% 56798 softirqs.CPU54.TIMER 50752 +12.4% 57055 softirqs.CPU55.TIMER 51051 +11.8% 57057 softirqs.CPU56.TIMER 50903 +10.8% 56419 ± 2% softirqs.CPU57.TIMER 50192 ± 2% +13.7% 57074 softirqs.CPU58.TIMER 50788 +11.9% 56840 softirqs.CPU59.TIMER 51289 +16.6% 59822 ± 7% softirqs.CPU6.TIMER 50076 ± 2% +14.0% 57074 softirqs.CPU60.TIMER 50288 ± 2% +22.5% 61594 ± 12% softirqs.CPU61.TIMER 50800 ± 2% +12.7% 57243 softirqs.CPU62.TIMER 4297 ± 87% -60.0% 1719 ± 10% softirqs.CPU63.SCHED 50803 ± 2% +11.5% 56648 softirqs.CPU63.TIMER 50556 +11.5% 56384 softirqs.CPU64.TIMER 50710 +12.6% 57082 softirqs.CPU65.TIMER 50573 ± 2% +12.5% 56870 softirqs.CPU66.TIMER 50402 +12.5% 56727 softirqs.CPU67.TIMER 51225 ± 5% +18.7% 60793 ± 4% softirqs.CPU68.TIMER 52055 ± 3% +9.9% 57191 softirqs.CPU7.TIMER 51046 +11.6% 56956 softirqs.CPU70.TIMER 50666 ± 2% +12.5% 56996 ± 2% softirqs.CPU72.TIMER 50842 +11.3% 56604 softirqs.CPU73.TIMER 51021 ± 2% +12.7% 57491 ± 2% softirqs.CPU74.TIMER 50768 +11.9% 56804 softirqs.CPU75.TIMER 50862 +12.5% 57205 softirqs.CPU77.TIMER 51158 +10.6% 56605 softirqs.CPU78.TIMER 51134 ± 2% +11.8% 57154 softirqs.CPU79.TIMER 50632 ± 2% +13.4% 57422 softirqs.CPU8.TIMER 50553 ± 2% +12.8% 57037 softirqs.CPU80.TIMER 50601 +12.3% 56818 softirqs.CPU81.TIMER 50786 +10.9% 56322 softirqs.CPU82.TIMER 50537 +11.9% 56569 ± 2% softirqs.CPU83.TIMER 50931 +13.0% 57571 softirqs.CPU84.TIMER 50862 +11.6% 56773 softirqs.CPU85.TIMER 51099 +11.5% 56976 softirqs.CPU86.TIMER 50509 +13.2% 57159 softirqs.CPU87.TIMER 49642 +12.1% 55671 softirqs.CPU88.TIMER 50083 ± 2% +10.6% 55414 ± 3% softirqs.CPU89.TIMER 50970 +13.1% 57652 softirqs.CPU9.TIMER 50395 +15.3% 58095 ± 4% softirqs.CPU90.TIMER 50780 +12.2% 56951 softirqs.CPU91.TIMER 50350 ± 2% +12.8% 56797 softirqs.CPU92.TIMER 50333 +12.2% 56489 softirqs.CPU93.TIMER 50545 +11.8% 56506 softirqs.CPU94.TIMER 50919 ± 2% +11.4% 56708 softirqs.CPU95.TIMER 50768 +11.3% 56505 softirqs.CPU97.TIMER 50389 +12.1% 56504 softirqs.CPU98.TIMER 50478 +12.6% 56839 ± 2% softirqs.CPU99.TIMER 19659 ± 57% -71.8% 5552 ± 57% softirqs.NET_RX 13676325 +12.3% 15355868 softirqs.TIMER *************************************************************************************************** lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep2/lru-file-readonce/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 310312 -19.1% 251078 vm-scalability.median 27328354 -19.2% 22090587 vm-scalability.throughput 190.42 +20.2% 228.95 vm-scalability.time.elapsed_time 190.42 +20.2% 228.95 vm-scalability.time.elapsed_time.max 219233 +13.2% 248278 vm-scalability.time.involuntary_context_switches 7168 -1.8% 7042 vm-scalability.time.percent_of_cpu_this_job_got 13329 +18.4% 15787 vm-scalability.time.system_time 321.79 ± 2% +4.8% 337.12 vm-scalability.time.user_time 1347 +51.1% 2035 ± 13% vm-scalability.time.voluntary_context_switches 14919 ± 7% +113.7% 31879 ± 68% cpuidle.C1E.usage 16455 ± 9% +40.8% 23171 ± 28% cpuidle.POLL.time 0.07 ± 2% +0.0 0.09 ± 5% mpstat.cpu.soft% 1.96 ± 2% -0.3 1.70 ± 8% mpstat.cpu.usr% 3090 -10.8% 2756 ± 5% vmstat.system.cs 181210 -11.7% 160017 ± 12% vmstat.system.in 306210 ± 13% -49.1% 155863 ± 7% meminfo.DirectMap4k 105582 +21.3% 128073 ± 2% meminfo.Shmem 720696 -18.3% 589164 ± 8% meminfo.max_used_kB 149122 ± 12% +28.6% 191831 ± 10% numa-meminfo.node0.Active 148798 ± 12% +28.6% 191353 ± 10% numa-meminfo.node0.Active(anon) 1069800 -16.0% 898396 ± 8% numa-meminfo.node1.KReclaimable 1069800 -16.0% 898396 ± 8% numa-meminfo.node1.SReclaimable 1134459 -15.2% 961818 ± 8% numa-meminfo.node1.Slab 17963880 ± 2% +221.8% 57810217 ± 5% numa-numastat.node0.numa_foreign 17509671 ± 3% +179.4% 48927485 numa-numastat.node0.numa_miss 17515451 ± 3% +179.3% 48929064 numa-numastat.node0.other_node 17509671 ± 3% +179.4% 48927485 numa-numastat.node1.numa_foreign 17963880 ± 2% +221.8% 57810217 ± 5% numa-numastat.node1.numa_miss 17975396 ± 2% +221.7% 57825977 ± 5% numa-numastat.node1.other_node 37255 ± 12% +28.4% 47853 ± 10% numa-vmstat.node0.nr_active_anon 37256 ± 12% +28.4% 47853 ± 10% numa-vmstat.node0.nr_zone_active_anon 9781648 ± 3% +243.1% 33561783 numa-vmstat.node0.numa_foreign 9881578 ± 3% +189.8% 28634049 ± 5% numa-vmstat.node0.numa_miss 9887890 ± 3% +189.6% 28636504 ± 5% numa-vmstat.node0.numa_other 3393930 ± 5% +35.9% 4613590 ± 7% numa-vmstat.node0.workingset_nodereclaim 266909 -16.1% 223966 ± 8% numa-vmstat.node1.nr_slab_reclaimable 9884279 ± 3% +189.8% 28640136 ± 5% numa-vmstat.node1.numa_foreign 9783507 ± 3% +243.2% 33572233 numa-vmstat.node1.numa_miss 9930431 ± 3% +239.6% 33723308 numa-vmstat.node1.numa_other 2771000 ± 6% -28.1% 1992491 ± 11% numa-vmstat.node1.workingset_nodereclaim 328.75 ± 7% +218.2% 1046 ± 16% proc-vmstat.allocstall_normal 698887 ± 6% +89.8% 1326640 ± 9% proc-vmstat.compact_daemon_migrate_scanned 1565 ± 6% -28.0% 1127 ± 7% proc-vmstat.compact_daemon_wake 26.50 ± 37% +412.3% 135.75 ± 27% proc-vmstat.compact_fail 1028872 ± 12% +56.6% 1610829 ± 6% proc-vmstat.compact_isolated 704055 ± 6% +89.7% 1335347 ± 9% proc-vmstat.compact_migrate_scanned 36.00 ± 50% +293.8% 141.75 ± 25% proc-vmstat.compact_stall 3731 -50.1% 1862 ± 3% proc-vmstat.kswapd_low_wmark_hit_quickly 68606 +10.1% 75502 ± 2% proc-vmstat.nr_active_anon 3172709 -6.0% 2983910 proc-vmstat.nr_dirty_background_threshold 6353176 -6.0% 5975118 proc-vmstat.nr_dirty_threshold 28636480 -9.7% 25849697 ± 8% proc-vmstat.nr_file_pages 28380705 -9.8% 25586992 ± 8% proc-vmstat.nr_inactive_file 611.50 ± 2% -7.9% 563.00 ± 7% proc-vmstat.nr_isolated_file 26439 +21.1% 32030 ± 2% proc-vmstat.nr_shmem 544007 -11.0% 484417 ± 8% proc-vmstat.nr_slab_reclaimable 68607 +10.1% 75503 ± 2% proc-vmstat.nr_zone_active_anon 28380710 -9.8% 25586992 ± 8% proc-vmstat.nr_zone_inactive_file 35473552 +200.9% 1.067e+08 ± 3% proc-vmstat.numa_foreign 1.039e+09 -6.9% 9.678e+08 proc-vmstat.numa_hit 1.039e+09 -6.9% 9.678e+08 proc-vmstat.numa_local 35473552 +200.9% 1.067e+08 ± 3% proc-vmstat.numa_miss 35490852 +200.8% 1.068e+08 ± 3% proc-vmstat.numa_other 7283 ± 43% -69.1% 2251 ± 61% proc-vmstat.numa_pages_migrated 3784 -46.7% 2017 ± 2% proc-vmstat.pageoutrun 26732 ± 4% +15.2% 30808 ± 3% proc-vmstat.pgactivate 13091102 +4.2% 13645052 proc-vmstat.pgalloc_dma32 576055 +18.3% 681283 ± 7% proc-vmstat.pgfault 518718 ± 12% +55.1% 804726 ± 6% proc-vmstat.pgmigrate_success 9.97e+08 -5.8% 9.392e+08 proc-vmstat.pgscan_direct 45159281 ± 2% +130.7% 1.042e+08 ± 2% proc-vmstat.pgscan_kswapd 9.97e+08 -5.8% 9.392e+08 proc-vmstat.pgsteal_direct 45159207 ± 2% +130.7% 1.042e+08 ± 2% proc-vmstat.pgsteal_kswapd 14362283 +5.0% 15084966 proc-vmstat.slabs_scanned 6148774 +7.3% 6596785 ± 8% proc-vmstat.workingset_nodereclaim 3092463 -11.8% 2727986 ± 8% proc-vmstat.workingset_nodes 1.497e+10 -9.2% 1.359e+10 ± 8% perf-stat.i.branch-instructions 72873551 -21.2% 57407599 ± 5% perf-stat.i.branch-misses 1.019e+09 -20.8% 8.071e+08 ± 7% perf-stat.i.cache-references 3058 -11.2% 2716 ± 5% perf-stat.i.context-switches 2.78 +26.0% 3.51 ± 22% perf-stat.i.cpi 1.87e+10 -11.7% 1.651e+10 ± 8% perf-stat.i.dTLB-loads 7.816e+09 -21.8% 6.109e+09 ± 7% perf-stat.i.dTLB-stores 23622156 -18.5% 19253831 ± 9% perf-stat.i.iTLB-load-misses 11165228 -19.7% 8962467 ± 9% perf-stat.i.iTLB-loads 6.8e+10 -10.9% 6.061e+10 ± 8% perf-stat.i.instructions 0.38 -11.7% 0.34 ± 8% perf-stat.i.ipc 2894 -6.3% 2712 perf-stat.i.minor-faults 42.60 +10.6 53.18 ± 6% perf-stat.i.node-load-miss-rate% 4202934 +59.3% 6696601 ± 8% perf-stat.i.node-load-misses 24.53 ± 3% +8.8 33.37 ± 9% perf-stat.i.node-store-miss-rate% 3927228 +19.4% 4687672 ± 8% perf-stat.i.node-store-misses 14315546 -21.8% 11191205 ± 8% perf-stat.i.node-stores 2895 -6.3% 2713 perf-stat.i.page-faults 14.98 -11.2% 13.30 perf-stat.overall.MPKI 0.49 -0.1 0.42 ± 2% perf-stat.overall.branch-miss-rate% 6.29 +1.4 7.69 perf-stat.overall.cache-miss-rate% 2.98 +10.1% 3.28 perf-stat.overall.cpi 0.05 ± 18% +0.0 0.07 ± 3% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 10% +0.0 0.01 ± 31% perf-stat.overall.dTLB-store-miss-rate% 2879 +9.5% 3154 perf-stat.overall.instructions-per-iTLB-miss 0.34 -9.2% 0.31 perf-stat.overall.ipc 41.37 +10.8 52.17 perf-stat.overall.node-load-miss-rate% 21.54 +8.0 29.52 perf-stat.overall.node-store-miss-rate% 3018 +12.6% 3398 perf-stat.overall.path-length 1.494e+10 -8.9% 1.361e+10 ± 8% perf-stat.ps.branch-instructions 72655317 -21.2% 57222006 ± 5% perf-stat.ps.branch-misses 1.017e+09 -20.7% 8.065e+08 ± 7% perf-stat.ps.cache-references 3048 -11.1% 2711 ± 5% perf-stat.ps.context-switches 1.866e+10 -11.4% 1.653e+10 ± 8% perf-stat.ps.dTLB-loads 7.799e+09 -21.7% 6.103e+09 ± 7% perf-stat.ps.dTLB-stores 23577878 -18.4% 19250150 ± 8% perf-stat.ps.iTLB-load-misses 11143782 -19.6% 8961566 ± 8% perf-stat.ps.iTLB-loads 6.788e+10 -10.6% 6.067e+10 ± 8% perf-stat.ps.instructions 2874 -6.3% 2693 perf-stat.ps.minor-faults 4191852 +60.6% 6730806 ± 8% perf-stat.ps.node-load-misses 3922388 +19.6% 4692221 ± 8% perf-stat.ps.node-store-misses 14290722 -21.7% 11194431 ± 8% perf-stat.ps.node-stores 2874 -6.3% 2693 perf-stat.ps.page-faults 1.296e+13 +12.6% 1.46e+13 perf-stat.total.instructions 5.72 ± 12% -5.7 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64 5.68 ± 12% -5.7 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 5.68 ± 12% -5.7 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 5.67 ± 12% -5.7 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 5.54 ± 12% -5.5 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 5.29 ± 12% -5.3 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 6.29 ± 2% -1.3 4.98 ± 7% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 4.16 ± 3% -0.7 3.41 ± 6% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.86 -0.6 0.29 ±100% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.shrink_page_list.shrink_inactive_list.shrink_node_memcg 2.10 ± 7% -0.5 1.65 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 1.95 ± 7% -0.4 1.52 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 1.09 -0.4 0.67 ± 18% perf-profile.calltrace.cycles-pp.free_unref_page_list.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 2.33 -0.3 2.07 perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 1.43 ± 2% -0.2 1.20 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 2.04 -0.2 1.82 perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 2.00 -0.2 1.78 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read 1.39 -0.2 1.21 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 1.17 ± 2% -0.2 1.01 ± 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.26 -0.2 1.10 ± 5% perf-profile.calltrace.cycles-pp.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.90 ± 3% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.78 ± 2% -0.1 0.69 ± 6% perf-profile.calltrace.cycles-pp.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list 0.74 ± 2% -0.1 0.66 perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 0.63 ± 2% -0.1 0.58 ± 2% perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 0.60 ± 2% -0.0 0.55 ± 2% perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 0.71 ± 12% +0.4 1.06 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.12 ± 12% +0.5 1.58 ± 13% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 1.12 ± 12% +0.5 1.58 ± 13% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 1.12 ± 12% +0.5 1.58 ± 13% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 1.10 ± 12% +0.5 1.57 ± 13% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 1.10 ± 12% +0.5 1.57 ± 13% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 0.00 +0.7 0.70 perf-profile.calltrace.cycles-pp.__list_del_entry_valid.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 1.32 ± 7% +1.0 2.33 ± 15% perf-profile.calltrace.cycles-pp.ret_from_fork 1.32 ± 7% +1.0 2.33 ± 15% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 17.23 ± 3% +4.8 22.02 perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 17.13 ± 3% +4.8 21.94 perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 16.00 ± 3% +4.9 20.86 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 16.04 ± 3% +4.9 20.91 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 89.66 +5.0 94.62 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 89.52 +5.0 94.50 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 87.81 +5.2 93.02 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.98 ± 3% +5.2 24.19 ± 4% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 87.56 +5.3 92.82 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 28.03 ± 2% +5.3 33.29 ± 3% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 28.05 ± 2% +5.3 33.31 ± 3% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 28.02 ± 2% +5.3 33.28 ± 3% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 27.92 ± 2% +5.3 33.19 ± 3% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 86.71 +5.4 92.08 perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 86.56 +5.4 91.94 perf-profile.calltrace.cycles-pp.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read.do_syscall_64 86.23 +5.4 91.65 perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read 85.88 +5.5 91.33 perf-profile.calltrace.cycles-pp.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read 82.07 +5.9 87.93 perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 82.09 +5.9 87.95 perf-profile.calltrace.cycles-pp.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 14090 ± 28% +53.7% 21660 ± 16% sched_debug.cfs_rq:/.load.avg 738.12 ± 7% +81.5% 1339 ± 47% sched_debug.cfs_rq:/.load_avg.max 5.50 +42.5% 7.84 ± 9% sched_debug.cfs_rq:/.load_avg.min 116.23 ± 3% +49.7% 173.98 ± 36% sched_debug.cfs_rq:/.load_avg.stddev 0.54 +36.6% 0.74 ± 8% sched_debug.cfs_rq:/.nr_running.avg 0.50 +42.5% 0.71 ± 9% sched_debug.cfs_rq:/.nr_running.min 0.15 ± 10% -25.1% 0.11 ± 17% sched_debug.cfs_rq:/.nr_running.stddev 14.95 ± 3% +274.2% 55.95 ± 10% sched_debug.cfs_rq:/.nr_spread_over.avg 342.75 ± 11% -56.9% 147.80 ± 8% sched_debug.cfs_rq:/.nr_spread_over.max 53.90 ± 9% -64.7% 19.01 ± 3% sched_debug.cfs_rq:/.nr_spread_over.stddev 11.37 ± 25% +71.9% 19.55 ± 12% sched_debug.cfs_rq:/.runnable_load_avg.avg 303.06 ± 60% +109.0% 633.27 ± 18% sched_debug.cfs_rq:/.runnable_load_avg.max 5.44 +42.1% 7.72 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.min 39.06 ± 55% +108.1% 81.28 ± 18% sched_debug.cfs_rq:/.runnable_load_avg.stddev 14021 ± 28% +53.9% 21579 ± 16% sched_debug.cfs_rq:/.runnable_weight.avg 5777 +42.3% 8218 ± 8% sched_debug.cfs_rq:/.runnable_weight.min 85493 ± 21% +229.7% 281878 ± 13% sched_debug.cfs_rq:/.spread0.max 609.89 +32.8% 809.66 ± 8% sched_debug.cfs_rq:/.util_avg.avg 1130 ± 5% +7.8% 1217 ± 6% sched_debug.cfs_rq:/.util_avg.max 17.47 ± 14% +1992.2% 365.42 ± 24% sched_debug.cfs_rq:/.util_est_enqueued.avg 0.50 +42.5% 0.71 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.min 89.61 ± 18% +235.8% 300.86 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev 365315 ± 18% +39.4% 509103 ± 3% sched_debug.cpu.avg_idle.min 114827 ± 6% -18.0% 94212 ± 3% sched_debug.cpu.avg_idle.stddev 11.34 ± 25% +78.8% 20.28 ± 13% sched_debug.cpu.cpu_load[0].avg 302.94 ± 60% +129.7% 695.95 ± 15% sched_debug.cpu.cpu_load[0].max 5.44 +43.0% 7.78 ± 8% sched_debug.cpu.cpu_load[0].min 39.06 ± 55% +124.7% 87.77 ± 16% sched_debug.cpu.cpu_load[0].stddev 11.86 ± 21% +68.8% 20.01 ± 12% sched_debug.cpu.cpu_load[1].avg 339.75 ± 48% +96.7% 668.19 ± 13% sched_debug.cpu.cpu_load[1].max 5.44 +43.0% 7.78 ± 8% sched_debug.cpu.cpu_load[1].min 41.82 ± 45% +102.3% 84.60 ± 15% sched_debug.cpu.cpu_load[1].stddev 12.20 ± 19% +64.8% 20.11 ± 11% sched_debug.cpu.cpu_load[2].avg 381.31 ± 43% +74.8% 666.54 ± 12% sched_debug.cpu.cpu_load[2].max 5.44 +43.0% 7.78 ± 8% sched_debug.cpu.cpu_load[2].min 44.48 ± 42% +89.6% 84.34 ± 14% sched_debug.cpu.cpu_load[2].stddev 12.48 ± 17% +63.4% 20.40 ± 10% sched_debug.cpu.cpu_load[3].avg 431.56 ± 34% +56.1% 673.61 ± 9% sched_debug.cpu.cpu_load[3].max 5.44 +43.0% 7.78 ± 8% sched_debug.cpu.cpu_load[3].min 47.96 ± 36% +78.0% 85.35 ± 10% sched_debug.cpu.cpu_load[3].stddev 13.12 ± 14% +58.5% 20.81 ± 10% sched_debug.cpu.cpu_load[4].avg 504.69 ± 23% +38.6% 699.60 ± 6% sched_debug.cpu.cpu_load[4].max 5.44 +43.0% 7.78 ± 8% sched_debug.cpu.cpu_load[4].min 54.72 ± 25% +61.3% 88.25 ± 7% sched_debug.cpu.cpu_load[4].stddev 1223 +33.7% 1635 ± 8% sched_debug.cpu.curr->pid.avg 548.19 ± 4% -19.5% 441.08 ± 8% sched_debug.cpu.curr->pid.stddev 14844 ± 22% +45.8% 21650 ± 16% sched_debug.cpu.load.avg 5777 +42.3% 8218 ± 8% sched_debug.cpu.load.min 0.56 +36.8% 0.77 ± 9% sched_debug.cpu.nr_running.avg 1.50 +26.7% 1.90 ± 13% sched_debug.cpu.nr_running.max 0.50 +42.5% 0.71 ± 9% sched_debug.cpu.nr_running.min 20159 ± 16% -31.2% 13873 ± 9% sched_debug.cpu.nr_switches.max 3327 ± 9% -31.5% 2277 ± 6% sched_debug.cpu.nr_switches.stddev 8293 ± 21% -39.4% 5022 ± 17% sched_debug.cpu.ttwu_count.max 1322 ± 12% -39.1% 805.17 ± 10% sched_debug.cpu.ttwu_count.stddev 7814 ± 26% -40.0% 4687 ± 13% sched_debug.cpu.ttwu_local.max 1215 ± 16% -42.0% 705.60 ± 9% sched_debug.cpu.ttwu_local.stddev 9342 ± 7% +31.3% 12270 ± 16% softirqs.CPU0.SCHED 78377 +21.8% 95488 ± 12% softirqs.CPU10.TIMER 5815 +44.0% 8375 ± 43% softirqs.CPU12.SCHED 5774 ± 2% +28.5% 7423 ± 26% softirqs.CPU13.SCHED 78138 +13.6% 88771 ± 6% softirqs.CPU14.TIMER 5542 ± 4% +43.6% 7961 ± 35% softirqs.CPU15.SCHED 78173 +12.9% 88227 ± 4% softirqs.CPU15.TIMER 5785 ± 2% +35.7% 7850 ± 34% softirqs.CPU16.SCHED 78377 +13.1% 88652 ± 6% softirqs.CPU16.TIMER 17178 ± 13% -23.8% 13082 ± 7% softirqs.CPU18.RCU 5843 ± 3% +33.8% 7816 ± 32% softirqs.CPU18.SCHED 5653 ± 2% +35.5% 7662 ± 32% softirqs.CPU21.SCHED 78002 +13.5% 88570 ± 6% softirqs.CPU21.TIMER 13430 ± 16% +77.1% 23782 ± 31% softirqs.CPU22.RCU 78179 ± 2% +20.7% 94386 ± 6% softirqs.CPU22.TIMER 13076 ± 7% +107.2% 27097 ± 62% softirqs.CPU23.RCU 75366 ± 3% +20.2% 90618 ± 8% softirqs.CPU23.TIMER 13645 ± 11% +27.8% 17438 ± 9% softirqs.CPU24.RCU 75596 ± 3% +19.7% 90467 ± 9% softirqs.CPU24.TIMER 12964 ± 7% +34.3% 17410 ± 8% softirqs.CPU25.RCU 74809 ± 3% +20.4% 90098 ± 8% softirqs.CPU25.TIMER 13795 ± 7% +40.2% 19336 ± 21% softirqs.CPU26.RCU 75617 ± 3% +19.0% 89966 ± 9% softirqs.CPU26.TIMER 12790 ± 9% +40.5% 17969 ± 15% softirqs.CPU27.RCU 5676 ± 3% +33.4% 7574 ± 32% softirqs.CPU27.SCHED 74882 ± 3% +20.0% 89883 ± 8% softirqs.CPU27.TIMER 13134 ± 11% +53.2% 20120 ± 25% softirqs.CPU28.RCU 75122 ± 3% +20.0% 90156 ± 8% softirqs.CPU28.TIMER 14272 ± 12% +22.8% 17526 ± 13% softirqs.CPU29.RCU 74606 ± 3% +21.2% 90428 ± 9% softirqs.CPU29.TIMER 5656 ± 4% +35.3% 7653 ± 31% softirqs.CPU30.SCHED 74764 ± 3% +20.5% 90122 ± 8% softirqs.CPU30.TIMER 74720 ± 3% +20.7% 90168 ± 8% softirqs.CPU31.TIMER 14260 ± 10% +50.5% 21463 ± 24% softirqs.CPU32.RCU 74688 ± 3% +20.9% 90333 ± 8% softirqs.CPU32.TIMER 74534 ± 3% +19.9% 89400 ± 8% softirqs.CPU33.TIMER 74410 ± 3% +20.6% 89723 ± 9% softirqs.CPU34.TIMER 74705 ± 3% +19.7% 89451 ± 9% softirqs.CPU35.TIMER 14085 ± 6% +72.5% 24291 ± 42% softirqs.CPU36.RCU 74686 ± 3% +20.5% 89988 ± 8% softirqs.CPU36.TIMER 13945 ± 6% +31.5% 18332 ± 13% softirqs.CPU37.RCU 74585 ± 3% +19.9% 89425 ± 8% softirqs.CPU37.TIMER 14501 ± 4% +29.8% 18827 ± 12% softirqs.CPU38.RCU 74256 ± 3% +21.1% 89947 ± 8% softirqs.CPU38.TIMER 5602 ± 4% +36.9% 7671 ± 34% softirqs.CPU39.SCHED 74304 ± 3% +21.0% 89941 ± 9% softirqs.CPU39.TIMER 13598 ± 12% +35.7% 18458 ± 11% softirqs.CPU4.RCU 14948 ± 13% +24.3% 18587 ± 13% softirqs.CPU40.RCU 74266 ± 3% +20.7% 89621 ± 8% softirqs.CPU40.TIMER 13590 ± 9% +57.5% 21403 ± 29% softirqs.CPU41.RCU 74621 ± 3% +20.3% 89779 ± 8% softirqs.CPU41.TIMER 14468 ± 11% +33.3% 19289 ± 14% softirqs.CPU42.RCU 13508 ± 18% +49.5% 20192 ± 22% softirqs.CPU43.RCU 75452 ± 2% +19.0% 89811 ± 9% softirqs.CPU43.TIMER 5440 ± 4% +40.3% 7635 ± 33% softirqs.CPU45.SCHED 5784 ± 2% +33.2% 7703 ± 32% softirqs.CPU47.SCHED 5689 ± 6% +39.1% 7915 ± 30% softirqs.CPU49.SCHED 5782 +35.0% 7804 ± 28% softirqs.CPU51.SCHED 77375 ± 2% +16.0% 89719 ± 6% softirqs.CPU51.TIMER 5780 +36.2% 7872 ± 33% softirqs.CPU53.SCHED 5731 ± 4% +34.1% 7685 ± 33% softirqs.CPU57.SCHED 76969 ± 2% +15.6% 88999 ± 6% softirqs.CPU57.TIMER 78041 ± 2% +13.0% 88174 ± 6% softirqs.CPU58.TIMER 14718 ± 13% -18.9% 11941 ± 11% softirqs.CPU59.RCU 5984 ± 6% +33.0% 7959 ± 33% softirqs.CPU59.SCHED 78045 ± 2% +28.7% 100463 ± 25% softirqs.CPU59.TIMER 5883 ± 2% +34.1% 7891 ± 30% softirqs.CPU61.SCHED 5774 ± 2% +35.5% 7824 ± 34% softirqs.CPU63.SCHED 77030 ± 2% +13.3% 87263 ± 6% softirqs.CPU63.TIMER 5786 ± 3% +37.0% 7925 ± 31% softirqs.CPU64.SCHED 77646 +14.0% 88539 ± 6% softirqs.CPU64.TIMER 5464 ± 4% +38.4% 7564 ± 32% softirqs.CPU65.SCHED 77192 +13.5% 87634 ± 6% softirqs.CPU65.TIMER 12858 ± 13% +41.3% 18172 ± 12% softirqs.CPU66.RCU 74150 ± 3% +20.5% 89372 ± 8% softirqs.CPU66.TIMER 13318 ± 6% +28.2% 17078 ± 13% softirqs.CPU67.RCU 74176 ± 3% +20.5% 89399 ± 8% softirqs.CPU67.TIMER 74596 ± 3% +19.9% 89476 ± 9% softirqs.CPU68.TIMER 74722 ± 3% +20.2% 89819 ± 9% softirqs.CPU69.TIMER 77715 ± 2% +22.1% 94866 ± 12% softirqs.CPU7.TIMER 13557 ± 12% +30.3% 17668 ± 11% softirqs.CPU70.RCU 74774 ± 3% +19.9% 89683 ± 8% softirqs.CPU70.TIMER 12591 ± 8% +63.8% 20629 ± 40% softirqs.CPU71.RCU 74521 ± 3% +19.8% 89266 ± 9% softirqs.CPU71.TIMER 13196 ± 10% +44.5% 19067 ± 22% softirqs.CPU72.RCU 74352 ± 3% +20.6% 89666 ± 8% softirqs.CPU72.TIMER 13213 ± 12% +41.2% 18658 ± 17% softirqs.CPU73.RCU 74367 ± 3% +20.5% 89610 ± 8% softirqs.CPU73.TIMER 13854 ± 17% +35.2% 18725 ± 11% softirqs.CPU74.RCU 74344 ± 3% +20.5% 89603 ± 8% softirqs.CPU74.TIMER 14076 ± 6% +45.1% 20424 ± 36% softirqs.CPU75.RCU 73863 ± 3% +21.5% 89713 ± 8% softirqs.CPU75.TIMER 13241 ± 15% +35.9% 17990 ± 10% softirqs.CPU76.RCU 73935 ± 3% +21.2% 89640 ± 8% softirqs.CPU76.TIMER 13772 ± 12% +25.7% 17315 ± 11% softirqs.CPU77.RCU 74036 ± 3% +20.4% 89140 ± 8% softirqs.CPU77.TIMER 13574 ± 8% +30.4% 17707 ± 18% softirqs.CPU78.RCU 73636 ± 3% +21.0% 89120 ± 9% softirqs.CPU78.TIMER 13948 ± 8% +37.4% 19171 ± 16% softirqs.CPU79.RCU 73881 ± 3% +20.6% 89088 ± 8% softirqs.CPU79.TIMER 5400 ± 14% +55.7% 8409 ± 28% softirqs.CPU8.SCHED 78520 +14.8% 90173 ± 6% softirqs.CPU8.TIMER 13338 ± 8% +42.0% 18943 ± 20% softirqs.CPU80.RCU 74425 ± 3% +20.4% 89621 ± 8% softirqs.CPU80.TIMER 14123 ± 6% +28.6% 18163 ± 13% softirqs.CPU81.RCU 74287 ± 3% +19.8% 89033 ± 8% softirqs.CPU81.TIMER 5605 ± 3% +38.4% 7758 ± 32% softirqs.CPU82.SCHED 73973 ± 3% +21.6% 89921 ± 8% softirqs.CPU82.TIMER 5643 ± 6% +34.1% 7565 ± 32% softirqs.CPU83.SCHED 74173 ± 3% +20.2% 89172 ± 9% softirqs.CPU83.TIMER 14178 ± 10% +25.5% 17789 ± 9% softirqs.CPU84.RCU 5828 ± 2% +33.0% 7753 ± 29% softirqs.CPU84.SCHED 74122 ± 3% +21.1% 89769 ± 8% softirqs.CPU84.TIMER 14892 ± 17% +27.4% 18977 ± 17% softirqs.CPU85.RCU 74117 ± 3% +20.6% 89380 ± 8% softirqs.CPU85.TIMER 13365 ± 7% +34.5% 17980 ± 14% softirqs.CPU86.RCU 74396 ± 3% +19.8% 89149 ± 8% softirqs.CPU86.TIMER 12458 ± 6% +46.1% 18204 ± 20% softirqs.CPU87.RCU 6479 +114.3% 13888 ± 49% softirqs.NET_RX 1218947 ± 9% +21.4% 1480339 ± 13% softirqs.RCU 6738143 +16.6% 7857544 ± 7% softirqs.TIMER 357.75 ± 30% +56.8% 561.00 ± 12% interrupts.34:IR-PCI-MSI.1572864-edge.eth0-TxRx-0 118.50 ± 17% +2966.7% 3634 ±164% interrupts.42:IR-PCI-MSI.1572872-edge.eth0-TxRx-8 113.00 ± 11% +77.9% 201.00 ± 38% interrupts.43:IR-PCI-MSI.1572873-edge.eth0-TxRx-9 111.75 ± 9% +84.1% 205.75 ± 44% interrupts.49:IR-PCI-MSI.1572879-edge.eth0-TxRx-15 93.25 +27.1% 118.50 ± 9% interrupts.51:IR-PCI-MSI.1572881-edge.eth0-TxRx-17 93.25 +35.1% 126.00 ± 10% interrupts.52:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 93.50 +26.7% 118.50 ± 9% interrupts.53:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 93.50 +27.3% 119.00 ± 8% interrupts.54:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 95.50 ± 3% +24.1% 118.50 ± 9% interrupts.55:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 93.25 +27.1% 118.50 ± 9% interrupts.56:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 93.75 +26.4% 118.50 ± 9% interrupts.57:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 93.25 +27.3% 118.75 ± 9% interrupts.58:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 93.25 +27.1% 118.50 ± 9% interrupts.59:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 93.25 +27.6% 119.00 ± 8% interrupts.60:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 93.25 +27.3% 118.75 ± 9% interrupts.61:IR-PCI-MSI.1572891-edge.eth0-TxRx-27 95.00 ± 3% +25.5% 119.25 ± 10% interrupts.62:IR-PCI-MSI.1572892-edge.eth0-TxRx-28 94.25 +25.7% 118.50 ± 9% interrupts.63:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 93.75 +26.7% 118.75 ± 9% interrupts.64:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 95.75 ± 4% +23.8% 118.50 ± 9% interrupts.65:IR-PCI-MSI.1572895-edge.eth0-TxRx-31 93.25 +27.3% 118.75 ± 9% interrupts.66:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 93.25 +27.3% 118.75 ± 9% interrupts.67:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 93.25 +27.3% 118.75 ± 9% interrupts.68:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 93.25 +27.1% 118.50 ± 9% interrupts.69:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 93.25 +27.1% 118.50 ± 9% interrupts.70:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 93.25 +27.1% 118.50 ± 9% interrupts.71:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 93.25 +27.1% 118.50 ± 9% interrupts.74:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 95.00 ± 3% +26.3% 120.00 ± 10% interrupts.75:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 93.25 +29.5% 120.75 ± 12% interrupts.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 93.25 +27.1% 118.50 ± 9% interrupts.77:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 93.25 +27.1% 118.50 ± 9% interrupts.78:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 95.75 ± 4% +24.3% 119.00 ± 9% interrupts.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 96.75 ± 6% +22.5% 118.50 ± 9% interrupts.80:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 93.25 +27.1% 118.50 ± 9% interrupts.81:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 93.75 +26.4% 118.50 ± 9% interrupts.82:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 93.25 +28.2% 119.50 ± 8% interrupts.83:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 93.25 +27.1% 118.50 ± 9% interrupts.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 93.25 +28.2% 119.50 ± 9% interrupts.85:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 93.25 +27.1% 118.50 ± 9% interrupts.86:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 93.25 +27.1% 118.50 ± 9% interrupts.87:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 93.25 +27.6% 119.00 ± 9% interrupts.88:IR-PCI-MSI.1572916-edge.eth0-TxRx-52 93.25 +27.1% 118.50 ± 9% interrupts.89:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 93.25 +27.1% 118.50 ± 9% interrupts.90:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 93.25 +27.6% 119.00 ± 9% interrupts.91:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 93.25 +31.1% 122.25 ± 8% interrupts.92:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 95.75 ± 4% +23.8% 118.50 ± 9% interrupts.93:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 95.75 ± 4% +26.4% 121.00 ± 8% interrupts.94:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 95.00 ± 3% +24.7% 118.50 ± 9% interrupts.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 95.75 ± 4% +26.4% 121.00 ± 8% interrupts.96:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 95.75 ± 4% +26.1% 120.75 ± 8% interrupts.97:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 96.75 ± 6% +22.7% 118.75 ± 9% interrupts.98:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 308.00 +26.5% 389.75 ± 7% interrupts.9:IR-IO-APIC.9-fasteoi.acpi 206197 +16.9% 241055 ± 7% interrupts.CAL:Function_call_interrupts 357.75 ± 30% +56.8% 561.00 ± 12% interrupts.CPU0.34:IR-PCI-MSI.1572864-edge.eth0-TxRx-0 2363 +14.3% 2702 ± 8% interrupts.CPU0.CAL:Function_call_interrupts 308.00 +26.5% 389.75 ± 7% interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi 2366 +16.7% 2761 ± 6% interrupts.CPU1.CAL:Function_call_interrupts 5877 ± 24% +26.5% 7435 ± 2% interrupts.CPU1.NMI:Non-maskable_interrupts 5877 ± 24% +26.5% 7435 ± 2% interrupts.CPU1.PMI:Performance_monitoring_interrupts 2361 +16.8% 2759 ± 10% interrupts.CPU10.CAL:Function_call_interrupts 5879 ± 23% +26.4% 7429 ± 2% interrupts.CPU10.NMI:Non-maskable_interrupts 5879 ± 23% +26.4% 7429 ± 2% interrupts.CPU10.PMI:Performance_monitoring_interrupts 2330 +18.0% 2750 ± 7% interrupts.CPU11.CAL:Function_call_interrupts 2370 +16.6% 2762 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 5894 ± 24% +25.3% 7386 ± 2% interrupts.CPU12.NMI:Non-maskable_interrupts 5894 ± 24% +25.3% 7386 ± 2% interrupts.CPU12.PMI:Performance_monitoring_interrupts 131.00 ± 72% +1065.6% 1527 ±144% interrupts.CPU12.RES:Rescheduling_interrupts 2320 ± 6% +19.4% 2771 ± 6% interrupts.CPU13.CAL:Function_call_interrupts 114.25 ± 76% +164.1% 301.75 ± 33% interrupts.CPU13.RES:Rescheduling_interrupts 2202 ± 12% +24.7% 2747 ± 5% interrupts.CPU14.CAL:Function_call_interrupts 111.75 ± 9% +84.1% 205.75 ± 44% interrupts.CPU15.49:IR-PCI-MSI.1572879-edge.eth0-TxRx-15 2237 ± 10% +24.2% 2780 ± 7% interrupts.CPU15.CAL:Function_call_interrupts 5974 ± 23% +24.5% 7440 ± 2% interrupts.CPU15.NMI:Non-maskable_interrupts 5974 ± 23% +24.5% 7440 ± 2% interrupts.CPU15.PMI:Performance_monitoring_interrupts 2371 +16.9% 2773 ± 7% interrupts.CPU16.CAL:Function_call_interrupts 69.25 ± 21% +383.8% 335.00 ± 28% interrupts.CPU16.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU17.51:IR-PCI-MSI.1572881-edge.eth0-TxRx-17 2349 ± 4% +18.2% 2777 ± 7% interrupts.CPU17.CAL:Function_call_interrupts 5622 ± 23% +31.9% 7418 ± 2% interrupts.CPU17.NMI:Non-maskable_interrupts 5622 ± 23% +31.9% 7418 ± 2% interrupts.CPU17.PMI:Performance_monitoring_interrupts 93.25 +35.1% 126.00 ± 10% interrupts.CPU18.52:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 2364 +15.7% 2736 ± 3% interrupts.CPU18.CAL:Function_call_interrupts 5815 ± 25% +27.4% 7410 ± 3% interrupts.CPU18.NMI:Non-maskable_interrupts 5815 ± 25% +27.4% 7410 ± 3% interrupts.CPU18.PMI:Performance_monitoring_interrupts 99.50 ± 19% +169.6% 268.25 ± 26% interrupts.CPU18.RES:Rescheduling_interrupts 93.50 +26.7% 118.50 ± 9% interrupts.CPU19.53:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 2375 +16.9% 2776 ± 6% interrupts.CPU19.CAL:Function_call_interrupts 6463 ± 2% +14.8% 7421 ± 3% interrupts.CPU19.NMI:Non-maskable_interrupts 6463 ± 2% +14.8% 7421 ± 3% interrupts.CPU19.PMI:Performance_monitoring_interrupts 92.75 ± 47% +249.1% 323.75 ± 18% interrupts.CPU19.RES:Rescheduling_interrupts 2361 +16.5% 2751 ± 6% interrupts.CPU2.CAL:Function_call_interrupts 73.00 ± 25% +302.4% 293.75 ± 34% interrupts.CPU2.RES:Rescheduling_interrupts 93.50 +27.3% 119.00 ± 8% interrupts.CPU20.54:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 67.50 ± 32% +310.0% 276.75 ± 37% interrupts.CPU20.RES:Rescheduling_interrupts 95.50 ± 3% +24.1% 118.50 ± 9% interrupts.CPU21.55:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 2334 ± 2% +17.3% 2738 ± 8% interrupts.CPU21.CAL:Function_call_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU22.56:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 5701 ± 26% +29.8% 7402 ± 2% interrupts.CPU22.NMI:Non-maskable_interrupts 5701 ± 26% +29.8% 7402 ± 2% interrupts.CPU22.PMI:Performance_monitoring_interrupts 1220 ± 73% -78.7% 259.75 ± 7% interrupts.CPU22.RES:Rescheduling_interrupts 93.75 +26.4% 118.50 ± 9% interrupts.CPU23.57:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 2398 +15.2% 2762 ± 7% interrupts.CPU23.CAL:Function_call_interrupts 6404 ± 3% +15.6% 7400 ± 2% interrupts.CPU23.NMI:Non-maskable_interrupts 6404 ± 3% +15.6% 7400 ± 2% interrupts.CPU23.PMI:Performance_monitoring_interrupts 93.25 +27.3% 118.75 ± 9% interrupts.CPU24.58:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 2362 +16.6% 2754 ± 6% interrupts.CPU24.CAL:Function_call_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU25.59:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 2364 +17.9% 2788 ± 8% interrupts.CPU25.CAL:Function_call_interrupts 6484 ± 3% +14.3% 7414 ± 2% interrupts.CPU25.NMI:Non-maskable_interrupts 6484 ± 3% +14.3% 7414 ± 2% interrupts.CPU25.PMI:Performance_monitoring_interrupts 93.25 +27.6% 119.00 ± 8% interrupts.CPU26.60:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 2354 +15.9% 2727 ± 4% interrupts.CPU26.CAL:Function_call_interrupts 6271 ± 4% +17.9% 7391 ± 2% interrupts.CPU26.NMI:Non-maskable_interrupts 6271 ± 4% +17.9% 7391 ± 2% interrupts.CPU26.PMI:Performance_monitoring_interrupts 93.25 +27.3% 118.75 ± 9% interrupts.CPU27.61:IR-PCI-MSI.1572891-edge.eth0-TxRx-27 2354 +18.4% 2786 ± 7% interrupts.CPU27.CAL:Function_call_interrupts 6553 ± 3% +13.0% 7407 ± 2% interrupts.CPU27.NMI:Non-maskable_interrupts 6553 ± 3% +13.0% 7407 ± 2% interrupts.CPU27.PMI:Performance_monitoring_interrupts 95.00 ± 3% +25.5% 119.25 ± 10% interrupts.CPU28.62:IR-PCI-MSI.1572892-edge.eth0-TxRx-28 2357 +15.7% 2728 ± 5% interrupts.CPU28.CAL:Function_call_interrupts 94.25 +25.7% 118.50 ± 9% interrupts.CPU29.63:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 78.50 ± 14% +448.7% 430.75 ± 93% interrupts.CPU29.RES:Rescheduling_interrupts 2359 +17.5% 2773 ± 6% interrupts.CPU3.CAL:Function_call_interrupts 113.25 ± 58% +127.6% 257.75 ± 32% interrupts.CPU3.RES:Rescheduling_interrupts 93.75 +26.7% 118.75 ± 9% interrupts.CPU30.64:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 2353 +17.3% 2761 ± 6% interrupts.CPU30.CAL:Function_call_interrupts 6493 ± 3% +13.7% 7381 ± 2% interrupts.CPU30.NMI:Non-maskable_interrupts 6493 ± 3% +13.7% 7381 ± 2% interrupts.CPU30.PMI:Performance_monitoring_interrupts 95.75 ± 4% +23.8% 118.50 ± 9% interrupts.CPU31.65:IR-PCI-MSI.1572895-edge.eth0-TxRx-31 2318 ± 3% +16.8% 2708 ± 5% interrupts.CPU31.CAL:Function_call_interrupts 64.75 ± 26% +366.0% 301.75 ± 52% interrupts.CPU31.RES:Rescheduling_interrupts 93.25 +27.3% 118.75 ± 9% interrupts.CPU32.66:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 2342 +18.0% 2764 ± 7% interrupts.CPU32.CAL:Function_call_interrupts 114.50 ± 53% +141.5% 276.50 ± 46% interrupts.CPU32.RES:Rescheduling_interrupts 93.25 +27.3% 118.75 ± 9% interrupts.CPU33.67:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 2369 +16.2% 2754 ± 7% interrupts.CPU33.CAL:Function_call_interrupts 6449 ± 6% +14.5% 7386 ± 2% interrupts.CPU33.NMI:Non-maskable_interrupts 6449 ± 6% +14.5% 7386 ± 2% interrupts.CPU33.PMI:Performance_monitoring_interrupts 93.25 +27.3% 118.75 ± 9% interrupts.CPU34.68:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 93.25 +27.1% 118.50 ± 9% interrupts.CPU35.69:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 2312 ± 4% +14.8% 2654 ± 6% interrupts.CPU35.CAL:Function_call_interrupts 5513 ± 25% +33.8% 7379 ± 2% interrupts.CPU35.NMI:Non-maskable_interrupts 5513 ± 25% +33.8% 7379 ± 2% interrupts.CPU35.PMI:Performance_monitoring_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU36.70:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 2224 ± 10% +24.3% 2766 ± 7% interrupts.CPU36.CAL:Function_call_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU37.71:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 2371 +15.3% 2734 ± 8% interrupts.CPU37.CAL:Function_call_interrupts 4764 ± 32% +55.1% 7389 ± 2% interrupts.CPU37.NMI:Non-maskable_interrupts 4764 ± 32% +55.1% 7389 ± 2% interrupts.CPU37.PMI:Performance_monitoring_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU38.74:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 2349 +16.2% 2730 ± 7% interrupts.CPU38.CAL:Function_call_interrupts 6391 ± 4% +15.7% 7397 ± 2% interrupts.CPU38.NMI:Non-maskable_interrupts 6391 ± 4% +15.7% 7397 ± 2% interrupts.CPU38.PMI:Performance_monitoring_interrupts 95.50 ± 56% +251.8% 336.00 ± 43% interrupts.CPU38.RES:Rescheduling_interrupts 95.00 ± 3% +26.3% 120.00 ± 10% interrupts.CPU39.75:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 2205 ± 7% +23.8% 2731 ± 8% interrupts.CPU39.CAL:Function_call_interrupts 102.75 ± 65% +240.6% 350.00 ± 47% interrupts.CPU39.RES:Rescheduling_interrupts 2367 +15.7% 2739 ± 7% interrupts.CPU4.CAL:Function_call_interrupts 93.25 +29.5% 120.75 ± 12% interrupts.CPU40.76:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 2382 +14.9% 2737 ± 8% interrupts.CPU40.CAL:Function_call_interrupts 65.25 ± 46% +230.3% 215.50 ± 37% interrupts.CPU40.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU41.77:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 2371 +16.6% 2764 ± 6% interrupts.CPU41.CAL:Function_call_interrupts 5537 ± 25% +33.9% 7414 ± 2% interrupts.CPU41.NMI:Non-maskable_interrupts 5537 ± 25% +33.9% 7414 ± 2% interrupts.CPU41.PMI:Performance_monitoring_interrupts 72.50 ± 25% +238.3% 245.25 ± 24% interrupts.CPU41.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU42.78:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 2360 +14.0% 2691 ± 5% interrupts.CPU42.CAL:Function_call_interrupts 4974 ± 36% +48.2% 7371 ± 2% interrupts.CPU42.NMI:Non-maskable_interrupts 4974 ± 36% +48.2% 7371 ± 2% interrupts.CPU42.PMI:Performance_monitoring_interrupts 95.75 ± 4% +24.3% 119.00 ± 9% interrupts.CPU43.79:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 2345 +16.2% 2726 ± 5% interrupts.CPU43.CAL:Function_call_interrupts 5811 ± 27% +26.8% 7370 ± 2% interrupts.CPU43.NMI:Non-maskable_interrupts 5811 ± 27% +26.8% 7370 ± 2% interrupts.CPU43.PMI:Performance_monitoring_interrupts 96.75 ± 6% +22.5% 118.50 ± 9% interrupts.CPU44.80:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 2320 ± 3% +18.2% 2742 ± 7% interrupts.CPU44.CAL:Function_call_interrupts 4972 ± 31% +49.2% 7417 ± 2% interrupts.CPU44.NMI:Non-maskable_interrupts 4972 ± 31% +49.2% 7417 ± 2% interrupts.CPU44.PMI:Performance_monitoring_interrupts 97.75 ± 46% +299.5% 390.50 ± 35% interrupts.CPU44.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU45.81:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 2351 +18.4% 2782 ± 6% interrupts.CPU45.CAL:Function_call_interrupts 107.00 ± 41% +146.7% 264.00 ± 36% interrupts.CPU45.RES:Rescheduling_interrupts 93.75 +26.4% 118.50 ± 9% interrupts.CPU46.82:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 2365 +16.6% 2758 ± 6% interrupts.CPU46.CAL:Function_call_interrupts 6782 ± 2% +9.6% 7435 ± 2% interrupts.CPU46.NMI:Non-maskable_interrupts 6782 ± 2% +9.6% 7435 ± 2% interrupts.CPU46.PMI:Performance_monitoring_interrupts 62.50 ± 22% +500.0% 375.00 ± 34% interrupts.CPU46.RES:Rescheduling_interrupts 93.25 +28.2% 119.50 ± 8% interrupts.CPU47.83:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 2361 +15.3% 2722 ± 3% interrupts.CPU47.CAL:Function_call_interrupts 46.50 ± 24% +776.9% 407.75 ± 65% interrupts.CPU47.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU48.84:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 2373 +14.1% 2708 ± 7% interrupts.CPU48.CAL:Function_call_interrupts 93.25 +28.2% 119.50 ± 9% interrupts.CPU49.85:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 2365 +17.0% 2768 ± 6% interrupts.CPU49.CAL:Function_call_interrupts 88.50 ± 56% +243.8% 304.25 ± 33% interrupts.CPU49.RES:Rescheduling_interrupts 2263 ± 6% +20.2% 2720 ± 4% interrupts.CPU5.CAL:Function_call_interrupts 131.00 ± 56% +248.1% 456.00 ± 35% interrupts.CPU5.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU50.86:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 2342 +18.3% 2772 ± 7% interrupts.CPU50.CAL:Function_call_interrupts 6780 ± 2% +9.4% 7416 ± 2% interrupts.CPU50.NMI:Non-maskable_interrupts 6780 ± 2% +9.4% 7416 ± 2% interrupts.CPU50.PMI:Performance_monitoring_interrupts 45.00 ± 18% +775.6% 394.00 ± 56% interrupts.CPU50.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU51.87:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 2353 +15.5% 2718 ± 7% interrupts.CPU51.CAL:Function_call_interrupts 119.75 ± 67% +105.4% 246.00 ± 16% interrupts.CPU51.RES:Rescheduling_interrupts 93.25 +27.6% 119.00 ± 9% interrupts.CPU52.88:IR-PCI-MSI.1572916-edge.eth0-TxRx-52 2301 ± 4% +20.9% 2783 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 6528 +13.5% 7409 ± 2% interrupts.CPU52.NMI:Non-maskable_interrupts 6528 +13.5% 7409 ± 2% interrupts.CPU52.PMI:Performance_monitoring_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU53.89:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 2319 ± 2% +20.5% 2795 ± 6% interrupts.CPU53.CAL:Function_call_interrupts 46.50 ± 12% +643.5% 345.75 ± 39% interrupts.CPU53.RES:Rescheduling_interrupts 93.25 +27.1% 118.50 ± 9% interrupts.CPU54.90:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 2340 +18.7% 2778 ± 7% interrupts.CPU54.CAL:Function_call_interrupts 42.50 ± 5% +394.7% 210.25 ± 26% interrupts.CPU54.RES:Rescheduling_interrupts 93.25 +27.6% 119.00 ± 9% interrupts.CPU55.91:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 2231 ± 11% +24.3% 2774 ± 7% interrupts.CPU55.CAL:Function_call_interrupts 93.25 +31.1% 122.25 ± 8% interrupts.CPU56.92:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 2383 +14.7% 2735 ± 8% interrupts.CPU56.CAL:Function_call_interrupts 49.25 ± 21% +307.1% 200.50 ± 60% interrupts.CPU56.RES:Rescheduling_interrupts 95.75 ± 4% +23.8% 118.50 ± 9% interrupts.CPU57.93:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 2379 +16.9% 2782 ± 7% interrupts.CPU57.CAL:Function_call_interrupts 53.25 ± 21% +728.6% 441.25 ± 41% interrupts.CPU57.RES:Rescheduling_interrupts 95.75 ± 4% +26.4% 121.00 ± 8% interrupts.CPU58.94:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 2220 ± 10% +23.8% 2749 ± 4% interrupts.CPU58.CAL:Function_call_interrupts 95.00 ± 3% +24.7% 118.50 ± 9% interrupts.CPU59.95:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 2187 ± 14% +26.5% 2767 ± 7% interrupts.CPU59.CAL:Function_call_interrupts 2349 +19.0% 2795 ± 6% interrupts.CPU6.CAL:Function_call_interrupts 6642 ± 3% +11.6% 7414 ± 2% interrupts.CPU6.NMI:Non-maskable_interrupts 6642 ± 3% +11.6% 7414 ± 2% interrupts.CPU6.PMI:Performance_monitoring_interrupts 75.50 ± 45% +565.9% 502.75 ± 47% interrupts.CPU6.RES:Rescheduling_interrupts 95.75 ± 4% +26.4% 121.00 ± 8% interrupts.CPU60.96:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 2219 ± 12% +24.6% 2765 ± 7% interrupts.CPU60.CAL:Function_call_interrupts 95.75 ± 4% +26.1% 120.75 ± 8% interrupts.CPU61.97:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 2312 ± 4% +20.8% 2792 ± 7% interrupts.CPU61.CAL:Function_call_interrupts 96.75 ± 6% +22.7% 118.75 ± 9% interrupts.CPU62.98:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 2321 ± 3% +19.7% 2778 ± 6% interrupts.CPU62.CAL:Function_call_interrupts 6629 ± 2% +12.2% 7441 ± 2% interrupts.CPU62.NMI:Non-maskable_interrupts 6629 ± 2% +12.2% 7441 ± 2% interrupts.CPU62.PMI:Performance_monitoring_interrupts 49.25 ± 19% +243.1% 169.00 ± 26% interrupts.CPU62.RES:Rescheduling_interrupts 2363 +17.3% 2773 ± 7% interrupts.CPU63.CAL:Function_call_interrupts 2377 +16.6% 2773 ± 6% interrupts.CPU64.CAL:Function_call_interrupts 55.50 ± 28% +298.6% 221.25 ± 40% interrupts.CPU64.RES:Rescheduling_interrupts 2377 +17.1% 2783 ± 7% interrupts.CPU65.CAL:Function_call_interrupts 49.75 ± 16% +344.2% 221.00 ± 31% interrupts.CPU65.RES:Rescheduling_interrupts 2384 +15.8% 2761 ± 7% interrupts.CPU66.CAL:Function_call_interrupts 5606 ± 24% +31.8% 7386 ± 2% interrupts.CPU66.NMI:Non-maskable_interrupts 5606 ± 24% +31.8% 7386 ± 2% interrupts.CPU66.PMI:Performance_monitoring_interrupts 883.50 ± 54% -74.8% 222.75 ± 23% interrupts.CPU66.RES:Rescheduling_interrupts 2360 +16.7% 2756 ± 6% interrupts.CPU67.CAL:Function_call_interrupts 6493 ± 4% +13.6% 7377 ± 2% interrupts.CPU67.NMI:Non-maskable_interrupts 6493 ± 4% +13.6% 7377 ± 2% interrupts.CPU67.PMI:Performance_monitoring_interrupts 58.00 ± 19% +174.6% 159.25 ± 15% interrupts.CPU67.RES:Rescheduling_interrupts 2367 +17.3% 2775 ± 7% interrupts.CPU68.CAL:Function_call_interrupts 4844 ± 32% +52.3% 7380 ± 2% interrupts.CPU68.NMI:Non-maskable_interrupts 4844 ± 32% +52.3% 7380 ± 2% interrupts.CPU68.PMI:Performance_monitoring_interrupts 2337 ± 3% +17.8% 2753 ± 6% interrupts.CPU69.CAL:Function_call_interrupts 5607 ± 23% +31.5% 7375 ± 2% interrupts.CPU69.NMI:Non-maskable_interrupts 5607 ± 23% +31.5% 7375 ± 2% interrupts.CPU69.PMI:Performance_monitoring_interrupts 2399 +13.1% 2713 ± 9% interrupts.CPU7.CAL:Function_call_interrupts 2380 +14.2% 2719 ± 6% interrupts.CPU70.CAL:Function_call_interrupts 50.00 ± 11% +400.0% 250.00 ± 72% interrupts.CPU70.RES:Rescheduling_interrupts 2379 +17.2% 2788 ± 7% interrupts.CPU71.CAL:Function_call_interrupts 5632 ± 25% +31.3% 7396 ± 2% interrupts.CPU71.NMI:Non-maskable_interrupts 5632 ± 25% +31.3% 7396 ± 2% interrupts.CPU71.PMI:Performance_monitoring_interrupts 54.25 ± 28% +633.2% 397.75 ± 88% interrupts.CPU71.RES:Rescheduling_interrupts 6405 ± 4% +15.3% 7382 ± 2% interrupts.CPU73.NMI:Non-maskable_interrupts 6405 ± 4% +15.3% 7382 ± 2% interrupts.CPU73.PMI:Performance_monitoring_interrupts 55.00 ± 39% +338.6% 241.25 ± 46% interrupts.CPU73.RES:Rescheduling_interrupts 2372 +16.1% 2755 ± 6% interrupts.CPU74.CAL:Function_call_interrupts 5617 ± 22% +31.3% 7373 ± 2% interrupts.CPU74.NMI:Non-maskable_interrupts 5617 ± 22% +31.3% 7373 ± 2% interrupts.CPU74.PMI:Performance_monitoring_interrupts 54.50 ± 20% +181.2% 153.25 ± 31% interrupts.CPU74.RES:Rescheduling_interrupts 2379 +16.1% 2763 ± 6% interrupts.CPU75.CAL:Function_call_interrupts 5853 ± 27% +26.3% 7394 ± 2% interrupts.CPU75.NMI:Non-maskable_interrupts 5853 ± 27% +26.3% 7394 ± 2% interrupts.CPU75.PMI:Performance_monitoring_interrupts 50.00 ± 21% +598.0% 349.00 ± 50% interrupts.CPU75.RES:Rescheduling_interrupts 2374 +17.2% 2782 ± 7% interrupts.CPU76.CAL:Function_call_interrupts 6667 ± 3% +10.1% 7341 ± 2% interrupts.CPU76.NMI:Non-maskable_interrupts 6667 ± 3% +10.1% 7341 ± 2% interrupts.CPU76.PMI:Performance_monitoring_interrupts 58.75 ± 34% +333.6% 254.75 ± 50% interrupts.CPU76.RES:Rescheduling_interrupts 2367 +16.4% 2756 ± 7% interrupts.CPU77.CAL:Function_call_interrupts 2360 +14.2% 2694 ± 9% interrupts.CPU78.CAL:Function_call_interrupts 2395 ± 2% +10.2% 2641 ± 9% interrupts.CPU79.CAL:Function_call_interrupts 5337 ± 22% +38.0% 7364 ± 2% interrupts.CPU79.NMI:Non-maskable_interrupts 5337 ± 22% +38.0% 7364 ± 2% interrupts.CPU79.PMI:Performance_monitoring_interrupts 60.50 ± 20% +188.0% 174.25 ± 34% interrupts.CPU79.RES:Rescheduling_interrupts 118.50 ± 17% +2966.7% 3634 ±164% interrupts.CPU8.42:IR-PCI-MSI.1572872-edge.eth0-TxRx-8 2241 ± 2% +23.8% 2774 ± 7% interrupts.CPU8.CAL:Function_call_interrupts 6570 ± 3% +13.2% 7436 ± 2% interrupts.CPU8.NMI:Non-maskable_interrupts 6570 ± 3% +13.2% 7436 ± 2% interrupts.CPU8.PMI:Performance_monitoring_interrupts 2184 ± 15% +24.5% 2720 ± 8% interrupts.CPU80.CAL:Function_call_interrupts 6402 ± 3% +15.3% 7384 ± 2% interrupts.CPU80.NMI:Non-maskable_interrupts 6402 ± 3% +15.3% 7384 ± 2% interrupts.CPU80.PMI:Performance_monitoring_interrupts 2365 +15.7% 2736 ± 7% interrupts.CPU81.CAL:Function_call_interrupts 5745 ± 24% +29.0% 7412 ± 2% interrupts.CPU82.NMI:Non-maskable_interrupts 5745 ± 24% +29.0% 7412 ± 2% interrupts.CPU82.PMI:Performance_monitoring_interrupts 132.75 ± 77% +269.3% 490.25 ± 82% interrupts.CPU82.RES:Rescheduling_interrupts 2387 +13.8% 2717 ± 8% interrupts.CPU83.CAL:Function_call_interrupts 6546 ± 4% +13.4% 7425 interrupts.CPU83.NMI:Non-maskable_interrupts 6546 ± 4% +13.4% 7425 interrupts.CPU83.PMI:Performance_monitoring_interrupts 50.50 ± 19% +548.5% 327.50 ± 66% interrupts.CPU83.RES:Rescheduling_interrupts 2380 +13.4% 2698 ± 9% interrupts.CPU84.CAL:Function_call_interrupts 4640 ± 30% +59.2% 7389 ± 2% interrupts.CPU84.NMI:Non-maskable_interrupts 4640 ± 30% +59.2% 7389 ± 2% interrupts.CPU84.PMI:Performance_monitoring_interrupts 58.75 ± 21% +434.9% 314.25 ± 22% interrupts.CPU84.RES:Rescheduling_interrupts 2372 +16.7% 2767 ± 6% interrupts.CPU85.CAL:Function_call_interrupts 45.25 ± 29% +359.7% 208.00 ± 16% interrupts.CPU85.RES:Rescheduling_interrupts 2369 +17.1% 2773 ± 7% interrupts.CPU86.CAL:Function_call_interrupts 5656 ± 25% +30.3% 7371 ± 2% interrupts.CPU86.NMI:Non-maskable_interrupts 5656 ± 25% +30.3% 7371 ± 2% interrupts.CPU86.PMI:Performance_monitoring_interrupts 60.75 ± 34% +148.6% 151.00 ± 14% interrupts.CPU86.RES:Rescheduling_interrupts 2371 +15.8% 2746 ± 6% interrupts.CPU87.CAL:Function_call_interrupts 6491 ± 2% +13.7% 7382 ± 2% interrupts.CPU87.NMI:Non-maskable_interrupts 6491 ± 2% +13.7% 7382 ± 2% interrupts.CPU87.PMI:Performance_monitoring_interrupts 113.00 ± 11% +77.9% 201.00 ± 38% interrupts.CPU9.43:IR-PCI-MSI.1572873-edge.eth0-TxRx-9 2369 +15.5% 2735 ± 7% interrupts.CPU9.CAL:Function_call_interrupts 123.00 ± 45% +239.6% 417.75 ± 34% interrupts.CPU9.RES:Rescheduling_interrupts 534961 ± 2% +14.5% 612693 ± 3% interrupts.NMI:Non-maskable_interrupts 534961 ± 2% +14.5% 612693 ± 3% interrupts.PMI:Performance_monitoring_interrupts 499.75 ± 19% +82.0% 909.75 ± 28% interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-ivb-d04: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/lkp-ivb-d04/malloc1/will-it-scale/0x20 commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 3:4 -24% 2:4 perf-profile.calltrace.cycles-pp.error_entry %stddev %change %stddev \ | \ 141485 +8.8% 153976 will-it-scale.per_process_ops 54307 +4.6% 56792 will-it-scale.per_thread_ops 0.60 +4.2% 0.63 will-it-scale.scalability 995.33 ± 2% +12.2% 1117 ± 4% will-it-scale.time.involuntary_context_switches 7750 +1.6% 7875 will-it-scale.time.maximum_resident_set_size 10976474 +4.1% 11429608 will-it-scale.time.minor_page_faults 11.54 ± 2% +5.3% 12.14 ± 3% will-it-scale.time.user_time 1598907 +8.7% 1737937 will-it-scale.workload 1349 ± 5% +11.2% 1500 ± 3% slabinfo.cred_jar.num_objs 119830 +18.8% 142354 ± 9% softirqs.CPU1.TIMER 506182 +9.8% 555969 ± 3% softirqs.TIMER 12314 ± 3% -9.6% 11136 ± 6% proc-vmstat.nr_free_cma 1019 -1.1% 1008 proc-vmstat.nr_page_table_pages 1.198e+08 +8.9% 1.305e+08 proc-vmstat.numa_hit 1.198e+08 +8.9% 1.305e+08 proc-vmstat.numa_local 60966777 +8.6% 66234879 proc-vmstat.pgfault 79676 ± 4% +48.3% 118129 ± 14% interrupts.CPU0.RES:Rescheduling_interrupts 64633 ± 14% +32.6% 85673 ± 9% interrupts.CPU1.RES:Rescheduling_interrupts 5802 ± 71% -71.6% 1645 ±172% interrupts.CPU2.NMI:Non-maskable_interrupts 5802 ± 71% -71.6% 1645 ±172% interrupts.CPU2.PMI:Performance_monitoring_interrupts 81976 ± 8% +33.1% 109144 ± 9% interrupts.CPU2.RES:Rescheduling_interrupts 56266 ± 28% +37.7% 77456 ± 12% interrupts.CPU3.RES:Rescheduling_interrupts 282553 ± 7% +38.2% 390403 ± 9% interrupts.RES:Rescheduling_interrupts 88857868 -40.7% 52689101 ± 58% perf-stat.i.cache-references 199937 +8.8% 217455 perf-stat.i.minor-faults 199937 +8.8% 217455 perf-stat.i.page-faults 17.86 ± 10% -31.7% 12.19 ± 44% perf-stat.overall.MPKI 1.32 +0.2 1.51 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.16 +0.0 0.18 ± 11% perf-stat.overall.dTLB-store-miss-rate% 855.49 -7.8% 789.16 ± 5% perf-stat.overall.instructions-per-iTLB-miss 88640533 -40.7% 52570485 ± 58% perf-stat.ps.cache-references 199443 +8.7% 216870 perf-stat.ps.minor-faults 199443 +8.7% 216870 perf-stat.ps.page-faults 517343 ± 14% -17.0% 429469 ± 13% sched_debug.cfs_rq:/.load.max 155355 ± 4% -19.7% 124678 ± 13% sched_debug.cfs_rq:/.load.stddev 353.54 ± 4% -9.9% 318.46 ± 4% sched_debug.cfs_rq:/.load_avg.avg 162.83 ± 4% -14.5% 139.21 ± 6% sched_debug.cfs_rq:/.load_avg.stddev 95.89 ± 5% -17.6% 79.00 ± 7% sched_debug.cpu.cpu_load[3].min 1409393 ± 11% +20.6% 1699641 ± 4% sched_debug.cpu.nr_switches.min 505088 ± 38% -36.5% 320523 ± 9% sched_debug.cpu.nr_switches.stddev 1400699 ± 11% +20.7% 1690383 ± 4% sched_debug.cpu.sched_count.min 502630 ± 38% -36.5% 318920 ± 9% sched_debug.cpu.sched_count.stddev 697023 ± 11% +20.8% 842269 ± 4% sched_debug.cpu.sched_goidle.min 251481 ± 38% -36.6% 159432 ± 9% sched_debug.cpu.sched_goidle.stddev 719504 ± 13% +20.3% 865494 ± 5% sched_debug.cpu.ttwu_count.min 257260 ± 36% -35.5% 165962 ± 11% sched_debug.cpu.ttwu_count.stddev 2991 ± 38% +1210.9% 39210 ± 63% sched_debug.cpu.ttwu_local.min 18.88 ± 5% -12.4 6.45 ±103% perf-profile.calltrace.cycles-pp.page_fault 18.84 ± 5% -12.4 6.45 ±103% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 18.53 ± 5% -12.2 6.30 ±103% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 17.12 ± 5% -11.7 5.46 ±103% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 16.66 ± 5% -11.5 5.13 ±104% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 9.12 -5.3 3.78 ±103% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 8.82 -5.1 3.67 ±103% perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 6.71 ± 10% -5.0 1.67 ±105% perf-profile.calltrace.cycles-pp.__pte_alloc.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 5.24 ± 4% -4.2 1.06 ±106% perf-profile.calltrace.cycles-pp.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 5.65 ± 8% -4.2 1.50 ±105% perf-profile.calltrace.cycles-pp.pte_alloc_one.__pte_alloc.__handle_mm_fault.handle_mm_fault.__do_page_fault 4.92 ± 5% -4.0 0.94 ±106% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault 4.68 ± 5% -3.9 0.77 ±103% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault 5.81 -3.8 1.98 ±104% perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap 4.99 ± 10% -3.8 1.21 ±105% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc.__handle_mm_fault.handle_mm_fault 4.46 ± 10% -3.6 0.89 ±102% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc.__handle_mm_fault 4.88 ± 2% -3.3 1.61 ±105% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region 2.98 ± 2% -2.5 0.52 ±105% perf-profile.calltrace.cycles-pp.clear_page_erms.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault 2.61 ± 4% -2.2 0.36 ±102% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu 2.61 ± 13% -2.1 0.54 ±102% perf-profile.calltrace.cycles-pp.clear_page_erms.get_page_from_freelist.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc 0.86 ± 7% -0.5 0.34 ±101% perf-profile.calltrace.cycles-pp.mem_cgroup_try_charge_delay.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 0.83 ± 11% -0.5 0.33 ±100% perf-profile.calltrace.cycles-pp.free_pages_and_swap_cache.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region *************************************************************************************************** lkp-skl-fpga01: 104 threads Skylake with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-skl-fpga01/lru-file-mmap-read/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.10 +64.3% 0.16 ± 17% vm-scalability.free_time 170190 -17.0% 141215 vm-scalability.median 17692974 -16.9% 14710217 vm-scalability.throughput 321.55 +7.6% 345.86 vm-scalability.time.elapsed_time 321.55 +7.6% 345.86 vm-scalability.time.elapsed_time.max 515745 ± 3% +9.0% 562300 ± 2% vm-scalability.time.involuntary_context_switches 7438 ± 9% -61.7% 2845 ± 22% vm-scalability.time.major_page_faults 98032831 -9.7% 88555246 vm-scalability.time.minor_page_faults 8539 -4.8% 8129 vm-scalability.time.percent_of_cpu_this_job_got 25061 +4.0% 26073 vm-scalability.time.system_time 2399 ± 7% -14.8% 2043 vm-scalability.time.user_time 10461 ± 8% -28.8% 7452 ± 5% vm-scalability.time.voluntary_context_switches 4.832e+09 -8.6% 4.417e+09 vm-scalability.workload 41763 ± 10% +181.4% 117503 ± 21% cpuidle.POLL.time 13432 ± 5% +43.8% 19321 ± 14% cpuidle.POLL.usage 22714134 ± 3% -21.2% 17896955 meminfo.MemFree 789380 ± 2% -16.7% 657771 ± 2% meminfo.max_used_kB 16.62 ± 2% -2.3 14.36 ± 3% mpstat.cpu.idle% 7.68 ± 9% -1.4 6.30 mpstat.cpu.usr% 2362 +1.9% 2408 turbostat.Avg_MHz 31.02 -15.7 15.32 ± 8% turbostat.PKG_% 160.63 -2.9% 156.03 turbostat.RAMWatt 15.25 ± 2% -11.5% 13.50 ± 3% vmstat.cpu.id 76.50 +3.6% 79.25 vmstat.cpu.sy 18135925 ± 3% -12.6% 15848673 ± 2% vmstat.memory.free 3876 ± 2% +3.0% 3993 vmstat.system.cs 422549 -12.1% 371273 vmstat.system.in 1525 ± 6% -8.5% 1395 ± 3% slabinfo.kmalloc-rcl-96.active_objs 1525 ± 6% -8.5% 1395 ± 3% slabinfo.kmalloc-rcl-96.num_objs 166.00 ± 8% +47.9% 245.50 ± 16% slabinfo.nfs_commit_data.active_objs 166.00 ± 8% +47.9% 245.50 ± 16% slabinfo.nfs_commit_data.num_objs 375.50 ± 10% +23.7% 464.50 ± 11% slabinfo.skbuff_ext_cache.active_objs 375.50 ± 10% +23.8% 465.00 ± 11% slabinfo.skbuff_ext_cache.num_objs 249339 ± 16% -39.9% 149877 ± 25% numa-meminfo.node0.Active 248585 ± 16% -39.9% 149430 ± 25% numa-meminfo.node0.Active(anon) 170143 ± 21% -33.5% 113101 ± 20% numa-meminfo.node0.AnonPages 114110 -66.1% 38688 ±111% numa-meminfo.node0.Inactive(anon) 11496532 ± 5% -23.6% 8780581 ± 4% numa-meminfo.node0.MemFree 192761 -61.0% 75127 ± 91% numa-meminfo.node0.Shmem 96251 ± 40% +102.7% 195142 ± 22% numa-meminfo.node1.Active 95293 ± 41% +104.1% 194480 ± 22% numa-meminfo.node1.Active(anon) 81207 ± 44% +72.9% 140426 ± 18% numa-meminfo.node1.AnonPages 11713395 ± 10% -19.7% 9408736 ± 6% numa-meminfo.node1.MemFree 5.086e+08 -18.9% 4.124e+08 ± 3% numa-numastat.node0.local_node 38065392 ± 4% +142.3% 92248073 ± 8% numa-numastat.node0.numa_foreign 5.086e+08 -18.9% 4.124e+08 ± 3% numa-numastat.node0.numa_hit 23606634 ± 8% +196.0% 69868426 ± 12% numa-numastat.node0.numa_miss 23606668 ± 8% +196.0% 69883947 ± 12% numa-numastat.node0.other_node 5.065e+08 -21.3% 3.985e+08 ± 3% numa-numastat.node1.local_node 23606634 ± 8% +196.0% 69868426 ± 12% numa-numastat.node1.numa_foreign 5.066e+08 -21.3% 3.985e+08 ± 3% numa-numastat.node1.numa_hit 38065392 ± 4% +142.3% 92248073 ± 8% numa-numastat.node1.numa_miss 38085698 ± 4% +142.2% 92253227 ± 8% numa-numastat.node1.other_node 62174 ± 15% -39.8% 37417 ± 25% numa-vmstat.node0.nr_active_anon 42531 ± 21% -33.5% 28274 ± 20% numa-vmstat.node0.nr_anon_pages 2923411 ± 6% -24.2% 2215818 ± 5% numa-vmstat.node0.nr_free_pages 28479 -66.1% 9662 ±111% numa-vmstat.node0.nr_inactive_anon 408.75 ± 3% -11.9% 360.00 ± 5% numa-vmstat.node0.nr_isolated_file 48176 -60.9% 18833 ± 90% numa-vmstat.node0.nr_shmem 62176 ± 15% -39.8% 37419 ± 25% numa-vmstat.node0.nr_zone_active_anon 28480 -66.1% 9662 ±111% numa-vmstat.node0.nr_zone_inactive_anon 20914603 ± 5% +145.7% 51396316 ± 8% numa-vmstat.node0.numa_foreign 3.207e+08 -19.3% 2.587e+08 numa-vmstat.node0.numa_hit 3.207e+08 -19.4% 2.587e+08 numa-vmstat.node0.numa_local 12003566 ± 8% +229.7% 39581464 ± 12% numa-vmstat.node0.numa_miss 12003623 ± 8% +229.9% 39597035 ± 12% numa-vmstat.node0.numa_other 2765471 ± 2% -10.5% 2475247 ± 2% numa-vmstat.node0.workingset_nodereclaim 23837 ± 41% +104.5% 48738 ± 22% numa-vmstat.node1.nr_active_anon 20298 ± 44% +72.9% 35092 ± 18% numa-vmstat.node1.nr_anon_pages 2980021 ± 9% -20.6% 2365536 ± 6% numa-vmstat.node1.nr_free_pages 23837 ± 41% +104.5% 48738 ± 22% numa-vmstat.node1.nr_zone_active_anon 12007946 ± 8% +229.7% 39590391 ± 12% numa-vmstat.node1.numa_foreign 3.187e+08 -21.2% 2.511e+08 ± 2% numa-vmstat.node1.numa_hit 3.185e+08 -21.2% 2.509e+08 ± 2% numa-vmstat.node1.numa_local 20919047 ± 5% +145.7% 51407069 ± 8% numa-vmstat.node1.numa_miss 21088733 ± 5% +144.5% 51561954 ± 8% numa-vmstat.node1.numa_other 2209483 -26.4% 1626337 numa-vmstat.node1.workingset_nodereclaim 17387 ± 14% -22.3% 13511 ± 11% softirqs.CPU100.RCU 16818 ± 6% -21.8% 13149 ± 7% softirqs.CPU102.RCU 18206 ± 20% -28.9% 12940 ± 9% softirqs.CPU103.RCU 17826 ± 8% -17.3% 14734 ± 16% softirqs.CPU30.RCU 163.75 ± 14% +3623.4% 6097 ±165% softirqs.CPU31.NET_RX 15945 ± 9% -14.1% 13691 ± 3% softirqs.CPU34.RCU 17538 ± 6% -17.5% 14461 ± 8% softirqs.CPU38.RCU 18608 ± 16% -26.0% 13776 ± 6% softirqs.CPU39.RCU 17923 ± 11% -21.3% 14108 ± 9% softirqs.CPU42.RCU 16826 ± 3% -17.9% 13822 ± 8% softirqs.CPU43.RCU 17380 ± 21% -30.7% 12037 ± 8% softirqs.CPU45.RCU 16554 ± 21% -25.1% 12391 ± 8% softirqs.CPU50.RCU 13796 ± 7% -14.3% 11827 ± 4% softirqs.CPU55.RCU 14945 ± 5% +11.3% 16632 ± 6% softirqs.CPU6.RCU 17982 ± 12% -19.7% 14434 ± 13% softirqs.CPU80.RCU 17422 ± 10% -25.3% 13010 ± 11% softirqs.CPU81.RCU 16163 ± 2% -17.2% 13376 ± 5% softirqs.CPU82.RCU 17737 ± 22% -28.0% 12779 ± 9% softirqs.CPU83.RCU 17126 ± 11% -25.8% 12714 ± 9% softirqs.CPU84.RCU 15861 ± 13% -26.5% 11665 ± 12% softirqs.CPU86.RCU 15700 ± 3% -15.7% 13232 ± 2% softirqs.CPU87.RCU 16274 ± 3% -20.3% 12975 ± 4% softirqs.CPU89.RCU 17108 ± 13% -20.4% 13622 ± 16% softirqs.CPU91.RCU 17793 ± 7% -25.3% 13294 ± 4% softirqs.CPU92.RCU 16304 ± 2% -18.6% 13274 ± 5% softirqs.CPU93.RCU 115980 -5.8% 109244 ± 3% proc-vmstat.allocstall_movable 5.737e+08 ± 18% -71.7% 1.624e+08 ± 21% proc-vmstat.compact_daemon_free_scanned 4415631 ± 20% -84.2% 698061 ± 13% proc-vmstat.compact_daemon_migrate_scanned 1814 ± 3% -61.5% 698.50 ± 19% proc-vmstat.compact_daemon_wake 5.789e+08 ± 17% -71.6% 1.641e+08 ± 21% proc-vmstat.compact_free_scanned 1613867 ± 10% -75.1% 402482 ± 21% proc-vmstat.compact_isolated 4432755 ± 20% -84.0% 709540 ± 13% proc-vmstat.compact_migrate_scanned 39.75 ± 12% +150.9% 99.75 ± 23% proc-vmstat.kswapd_high_wmark_hit_quickly 1651 ± 4% -50.6% 815.00 ± 14% proc-vmstat.kswapd_low_wmark_hit_quickly 4674238 -1.9% 4584907 proc-vmstat.nr_dirty_background_threshold 9359907 -1.9% 9181024 proc-vmstat.nr_dirty_threshold 5775346 ± 2% -21.9% 4512026 proc-vmstat.nr_free_pages 832.25 -5.6% 786.00 ± 3% proc-vmstat.nr_isolated_file 36381035 +4.5% 38024735 proc-vmstat.nr_mapped 887599 -2.5% 865498 proc-vmstat.nr_page_table_pages 729154 -2.0% 714825 proc-vmstat.nr_slab_reclaimable 61672026 ± 5% +162.9% 1.621e+08 ± 10% proc-vmstat.numa_foreign 1.015e+09 -20.1% 8.11e+08 ± 3% proc-vmstat.numa_hit 1.015e+09 -20.1% 8.109e+08 ± 3% proc-vmstat.numa_local 61672026 ± 5% +162.9% 1.621e+08 ± 10% proc-vmstat.numa_miss 61692372 ± 5% +162.8% 1.621e+08 ± 10% proc-vmstat.numa_other 54821 ± 19% -32.2% 37153 ± 32% proc-vmstat.numa_pte_updates 2335 ± 3% -55.5% 1038 ± 12% proc-vmstat.pageoutrun 36447 ± 4% -12.5% 31902 ± 2% proc-vmstat.pgactivate 7041719 -7.2% 6538030 proc-vmstat.pgalloc_dma32 1.072e+09 -9.7% 9.682e+08 proc-vmstat.pgalloc_normal 98714586 -9.5% 89353097 proc-vmstat.pgfault 1.079e+09 -9.7% 9.746e+08 proc-vmstat.pgfree 7438 ± 9% -61.7% 2845 ± 22% proc-vmstat.pgmajfault 803408 ± 10% -74.7% 203107 ± 21% proc-vmstat.pgmigrate_success 1.935e+09 -10.7% 1.727e+09 proc-vmstat.pgscan_direct 1.189e+08 ± 4% +17.2% 1.394e+08 ± 5% proc-vmstat.pgscan_kswapd 9.886e+08 -11.1% 8.791e+08 proc-vmstat.pgsteal_direct 37983596 ± 3% +21.4% 46099245 ± 5% proc-vmstat.pgsteal_kswapd 12573369 -12.5% 10995461 ± 6% proc-vmstat.slabs_scanned 4946192 -17.9% 4060924 ± 3% proc-vmstat.workingset_nodereclaim 4219577 -2.8% 4100775 proc-vmstat.workingset_nodes 58.54 -12.7 45.81 ± 15% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node 57.49 -12.7 44.80 ± 15% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 74.56 -11.5 63.05 ± 7% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 73.50 -10.8 62.69 ± 7% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages 73.55 -10.8 62.75 ± 7% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 5.70 ± 40% -5.5 0.18 ±173% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 5.70 ± 40% -5.5 0.18 ±173% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault.__do_fault 5.70 ± 40% -5.5 0.18 ±173% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault 5.70 ± 40% -5.5 0.18 ±173% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault 95.70 -5.1 90.59 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 95.71 -5.1 90.59 perf-profile.calltrace.cycles-pp.page_fault 95.63 -5.1 90.52 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 95.69 -5.1 90.58 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 95.60 -5.1 90.50 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 5.13 ± 25% -4.0 1.14 ±106% perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 5.07 ± 27% -3.9 1.13 ±106% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault 5.07 ± 27% -3.9 1.13 ±106% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault 5.07 ± 27% -3.9 1.13 ±106% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__do_page_cache_readahead.filemap_fault.__xfs_filemap_fault.__do_fault 3.14 ± 12% -1.4 1.70 ± 12% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 2.12 ± 11% -1.3 0.84 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 2.08 ± 11% -1.3 0.80 ± 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 0.29 ±100% +0.5 0.76 ± 9% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.21 ± 10% +0.8 1.99 ± 20% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.filemap_fault 0.67 ± 17% +0.8 1.50 ± 24% perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 0.67 ± 17% +0.8 1.49 ± 25% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead 0.00 +1.1 1.10 ± 16% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 0.00 +1.4 1.40 ± 18% perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 6.54 ± 4% +3.0 9.53 ± 18% perf-profile.calltrace.cycles-pp.smp_call_function_many.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 19.80 ± 7% +3.8 23.57 ± 12% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.filemap_fault.__xfs_filemap_fault 19.80 ± 7% +3.8 23.57 ± 12% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.filemap_fault 19.76 ± 7% +3.8 23.53 ± 12% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 19.79 ± 7% +3.8 23.56 ± 12% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 14.67 ± 8% +4.0 18.64 ± 11% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 11.48 ± 5% +4.6 16.12 ± 13% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 11.50 ± 5% +4.7 16.16 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 12.05 ± 4% +4.7 16.73 ± 13% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 12.09 ± 4% +4.7 16.77 ± 13% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 6.73 ± 4% +4.7 11.47 ± 19% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list 6.70 ± 4% +4.7 11.44 ± 19% perf-profile.calltrace.cycles-pp.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list 6.74 ± 4% +4.7 11.48 ± 19% perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg 6.74 ± 4% +4.7 11.48 ± 19% perf-profile.calltrace.cycles-pp.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.00 +5.4 5.37 ± 18% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages 0.00 +5.4 5.37 ± 18% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq 0.00 +5.8 5.80 ± 18% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work 0.00 +5.8 5.81 ± 17% perf-profile.calltrace.cycles-pp.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work.worker_thread 0.00 +5.8 5.81 ± 17% perf-profile.calltrace.cycles-pp.drain_pages.drain_local_pages_wq.process_one_work.worker_thread.kthread 0.00 +5.8 5.81 ± 18% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +5.8 5.81 ± 18% perf-profile.calltrace.cycles-pp.drain_local_pages_wq.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +5.8 5.81 ± 18% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 1.93 ± 7% +5.9 7.79 ± 12% perf-profile.calltrace.cycles-pp.ret_from_fork 1.93 ± 7% +5.9 7.79 ± 12% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 70.81 ± 2% +6.9 77.69 ± 4% perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.ondemand_readahead.filemap_fault.__xfs_filemap_fault.__do_fault 70.81 ± 2% +6.9 77.69 ± 4% perf-profile.calltrace.cycles-pp.ondemand_readahead.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 2.23e+10 -16.1% 1.871e+10 perf-stat.i.branch-instructions 43386664 -18.3% 35449723 perf-stat.i.branch-misses 1.301e+08 -16.0% 1.093e+08 perf-stat.i.cache-misses 3.788e+08 -17.8% 3.112e+08 perf-stat.i.cache-references 3799 ± 2% +3.8% 3943 perf-stat.i.context-switches 4.45 -7.2% 4.13 ± 2% perf-stat.i.cpi 2.427e+11 +2.2% 2.48e+11 perf-stat.i.cpu-cycles 99.08 +19.6% 118.52 ± 4% perf-stat.i.cpu-migrations 3228 -13.1% 2806 ± 2% perf-stat.i.cycles-between-cache-misses 18291020 ± 2% -11.0% 16281109 ± 6% perf-stat.i.dTLB-load-misses 2.054e+10 -14.9% 1.747e+10 perf-stat.i.dTLB-loads 1302202 -21.3% 1025333 ± 3% perf-stat.i.dTLB-store-misses 5.582e+09 -17.6% 4.599e+09 perf-stat.i.dTLB-stores 1925537 -17.6% 1587180 perf-stat.i.iTLB-load-misses 801786 ± 4% -11.4% 710122 ± 3% perf-stat.i.iTLB-loads 7.904e+10 -15.8% 6.653e+10 perf-stat.i.instructions 0.43 ± 4% -15.4% 0.37 ± 5% perf-stat.i.ipc 21.84 ± 10% -63.9% 7.87 ± 22% perf-stat.i.major-faults 314027 -17.5% 258924 perf-stat.i.minor-faults 64.16 +2.0 66.19 perf-stat.i.node-load-miss-rate% 5497582 +21.8% 6698043 perf-stat.i.node-load-misses 41.67 +4.8 46.45 ± 2% perf-stat.i.node-store-miss-rate% 4613231 +30.1% 6003777 ± 2% perf-stat.i.node-store-misses 12475024 -27.9% 8988618 ± 3% perf-stat.i.node-stores 314064 -17.5% 259026 perf-stat.i.page-faults 4.78 -2.2% 4.67 perf-stat.overall.MPKI 0.20 -0.0 0.19 perf-stat.overall.branch-miss-rate% 34.39 +0.7 35.12 perf-stat.overall.cache-miss-rate% 3.15 +19.5% 3.76 perf-stat.overall.cpi 1916 +19.6% 2292 perf-stat.overall.cycles-between-cache-misses 0.32 -16.3% 0.27 perf-stat.overall.ipc 61.31 +4.0 65.32 perf-stat.overall.node-load-miss-rate% 27.51 +12.7 40.26 ± 3% perf-stat.overall.node-store-miss-rate% 2.187e+10 -14.8% 1.864e+10 perf-stat.ps.branch-instructions 42684414 -17.4% 35269171 perf-stat.ps.branch-misses 1.276e+08 -14.7% 1.088e+08 perf-stat.ps.cache-misses 3.71e+08 -16.5% 3.098e+08 perf-stat.ps.cache-references 3845 ± 2% +2.9% 3955 perf-stat.ps.context-switches 2.445e+11 +2.0% 2.494e+11 perf-stat.ps.cpu-cycles 101.99 +17.0% 119.28 ± 4% perf-stat.ps.cpu-migrations 17864531 ± 2% -9.4% 16192020 ± 6% perf-stat.ps.dTLB-load-misses 2.019e+10 -13.7% 1.742e+10 perf-stat.ps.dTLB-loads 1269086 ± 2% -19.7% 1019015 ± 2% perf-stat.ps.dTLB-store-misses 5.498e+09 -16.6% 4.588e+09 perf-stat.ps.dTLB-stores 1877330 -16.0% 1577549 perf-stat.ps.iTLB-load-misses 773199 ± 3% -10.4% 693063 ± 3% perf-stat.ps.iTLB-loads 7.767e+10 -14.6% 6.632e+10 perf-stat.ps.instructions 23.10 ± 9% -64.5% 8.19 ± 22% perf-stat.ps.major-faults 306264 -15.9% 257476 perf-stat.ps.minor-faults 5471854 +22.4% 6697652 perf-stat.ps.node-load-misses 4623952 +30.1% 6015704 ± 2% perf-stat.ps.node-store-misses 12183897 -26.7% 8933911 ± 3% perf-stat.ps.node-stores 306287 -15.9% 257484 perf-stat.ps.page-faults 2.502e+13 -8.1% 2.3e+13 perf-stat.total.instructions 17813 ±118% +169.5% 48006 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.avg 1619671 ±136% +208.3% 4992688 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.max 166067 ±128% +193.4% 487213 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.stddev 570.84 ± 9% +32.1% 754.24 ± 17% sched_debug.cfs_rq:/.exec_clock.stddev 20788 ± 11% +20.0% 24947 ± 10% sched_debug.cfs_rq:/.load.avg 97457 ± 13% +21.1% 118045 ± 11% sched_debug.cfs_rq:/.load.stddev 30.28 ± 8% +154.9% 77.20 ± 24% sched_debug.cfs_rq:/.load_avg.avg 832.50 ± 11% +33.5% 1111 ± 10% sched_debug.cfs_rq:/.load_avg.max 5.92 +40.1% 8.29 ± 15% sched_debug.cfs_rq:/.load_avg.min 17813 ±118% +169.5% 48006 ± 37% sched_debug.cfs_rq:/.max_vruntime.avg 1619671 ±136% +208.3% 4992688 ± 37% sched_debug.cfs_rq:/.max_vruntime.max 166067 ±128% +193.4% 487213 ± 37% sched_debug.cfs_rq:/.max_vruntime.stddev 769063 ± 12% -26.9% 562036 ± 3% sched_debug.cfs_rq:/.min_vruntime.stddev 0.69 +18.0% 0.81 ± 8% sched_debug.cfs_rq:/.nr_running.avg 14.48 ± 4% +1538.3% 237.23 ± 2% sched_debug.cfs_rq:/.nr_spread_over.avg 205.08 ± 21% +76.4% 361.83 ± 4% sched_debug.cfs_rq:/.nr_spread_over.max 33.52 ± 12% +125.2% 75.49 ± 6% sched_debug.cfs_rq:/.nr_spread_over.stddev 6.07 ± 40% -80.1% 1.21 ±111% sched_debug.cfs_rq:/.removed.load_avg.avg 33.05 ± 12% -70.3% 9.81 ±103% sched_debug.cfs_rq:/.removed.load_avg.stddev 279.91 ± 40% -79.9% 56.19 ±111% sched_debug.cfs_rq:/.removed.runnable_sum.avg 1523 ± 12% -70.0% 456.27 ±103% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 2.38 ± 50% -77.9% 0.53 ±105% sched_debug.cfs_rq:/.removed.util_avg.avg 12.76 ± 23% -65.1% 4.46 ±100% sched_debug.cfs_rq:/.removed.util_avg.stddev 17.15 ± 12% +29.5% 22.21 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.avg 649.79 ± 14% +29.3% 839.92 ± 8% sched_debug.cfs_rq:/.runnable_load_avg.max 80.30 ± 16% +32.5% 106.39 ± 10% sched_debug.cfs_rq:/.runnable_load_avg.stddev 20738 ± 11% +20.0% 24878 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg 97433 ± 13% +21.1% 118031 ± 11% sched_debug.cfs_rq:/.runnable_weight.stddev -4500054 -47.2% -2374086 sched_debug.cfs_rq:/.spread0.min 768866 ± 12% -26.7% 563539 ± 3% sched_debug.cfs_rq:/.spread0.stddev 170.15 ± 15% +32.2% 225.03 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev 683144 ± 2% +26.5% 864003 sched_debug.cpu.avg_idle.avg 45572 ± 10% +401.0% 228332 ± 9% sched_debug.cpu.avg_idle.min 273649 -37.5% 170912 ± 2% sched_debug.cpu.avg_idle.stddev 17.19 ± 12% +29.5% 22.26 ± 11% sched_debug.cpu.cpu_load[0].avg 649.79 ± 14% +29.3% 839.92 ± 8% sched_debug.cpu.cpu_load[0].max 80.32 ± 16% +32.4% 106.38 ± 10% sched_debug.cpu.cpu_load[0].stddev 17.21 ± 12% +30.1% 22.39 ± 11% sched_debug.cpu.cpu_load[1].avg 649.21 ± 14% +30.5% 847.33 ± 9% sched_debug.cpu.cpu_load[1].max 80.21 ± 16% +33.2% 106.86 ± 10% sched_debug.cpu.cpu_load[1].stddev 17.29 ± 12% +31.1% 22.66 ± 10% sched_debug.cpu.cpu_load[2].avg 651.12 ± 14% +32.1% 860.08 ± 9% sched_debug.cpu.cpu_load[2].max 80.12 ± 16% +34.8% 108.03 ± 9% sched_debug.cpu.cpu_load[2].stddev 17.39 ± 13% +31.7% 22.91 ± 8% sched_debug.cpu.cpu_load[3].avg 661.58 ± 15% +31.5% 869.75 ± 9% sched_debug.cpu.cpu_load[3].max 80.07 ± 18% +36.0% 108.88 ± 8% sched_debug.cpu.cpu_load[3].stddev 17.57 ± 13% +35.2% 23.75 ± 8% sched_debug.cpu.cpu_load[4].avg 696.08 ± 15% +28.8% 896.29 ± 9% sched_debug.cpu.cpu_load[4].max 82.14 ± 18% +35.3% 111.13 ± 5% sched_debug.cpu.cpu_load[4].stddev 1714 +16.3% 1994 ± 8% sched_debug.cpu.curr->pid.avg 19939 ± 9% +27.2% 25367 ± 11% sched_debug.cpu.load.avg 94802 ± 11% +25.9% 119368 ± 11% sched_debug.cpu.load.stddev 1001 ± 11% +119.9% 2203 ± 12% sched_debug.cpu.nr_load_updates.stddev 0.71 +18.1% 0.84 ± 8% sched_debug.cpu.nr_running.avg 4376 ± 5% +11.7% 4889 ± 2% sched_debug.cpu.nr_switches.min 2324 ± 3% -9.0% 2114 ± 3% sched_debug.cpu.nr_switches.stddev 24.75 ± 32% -48.8% 12.67 ± 37% sched_debug.cpu.nr_uninterruptible.max 4116 ± 4% +12.4% 4628 sched_debug.cpu.sched_count.min 1899 -18.8% 1541 ± 6% sched_debug.cpu.sched_count.stddev 160.61 +22.6% 196.88 ± 14% sched_debug.cpu.sched_goidle.avg 2564 ± 6% -21.7% 2007 ± 10% sched_debug.cpu.sched_goidle.max 1715 ± 5% -8.1% 1576 sched_debug.cpu.ttwu_count.min 1231 ± 2% -9.1% 1118 sched_debug.cpu.ttwu_local.min 709.36 ± 3% -12.4% 621.62 ± 3% sched_debug.cpu.ttwu_local.stddev 646.50 -100.0% 0.00 interrupts.9:IO-APIC.9-fasteoi.acpi 71115558 -14.2% 61026077 interrupts.CAL:Function_call_interrupts 693319 -13.9% 596850 ± 3% interrupts.CPU0.CAL:Function_call_interrupts 954317 -11.5% 844600 ± 4% interrupts.CPU0.TLB:TLB_shootdowns 646.50 -100.0% 0.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 692744 ± 2% -12.8% 604221 ± 3% interrupts.CPU1.CAL:Function_call_interrupts 1566 ± 17% +27.7% 2000 ± 6% interrupts.CPU1.RES:Rescheduling_interrupts 955501 ± 2% -10.2% 857976 ± 4% interrupts.CPU1.TLB:TLB_shootdowns 691106 -12.1% 607396 ± 2% interrupts.CPU10.CAL:Function_call_interrupts 1423 ± 8% +23.3% 1754 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts 663717 ± 4% -16.6% 553384 ± 3% interrupts.CPU100.CAL:Function_call_interrupts 910804 ± 4% -14.1% 782723 ± 3% interrupts.CPU100.TLB:TLB_shootdowns 682212 -18.6% 555086 ± 2% interrupts.CPU101.CAL:Function_call_interrupts 1328 ± 10% +157.5% 3421 ± 83% interrupts.CPU101.RES:Rescheduling_interrupts 938521 -16.2% 786338 ± 2% interrupts.CPU101.TLB:TLB_shootdowns 675339 -16.4% 564826 ± 3% interrupts.CPU102.CAL:Function_call_interrupts 1319 ± 12% +143.4% 3211 ± 76% interrupts.CPU102.RES:Rescheduling_interrupts 927260 -13.9% 798252 ± 3% interrupts.CPU102.TLB:TLB_shootdowns 655174 ± 5% -13.2% 568782 ± 3% interrupts.CPU103.CAL:Function_call_interrupts 698434 -13.0% 607474 ± 2% interrupts.CPU13.CAL:Function_call_interrupts 1413 ± 11% +21.3% 1713 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 960038 -10.0% 864425 ± 2% interrupts.CPU13.TLB:TLB_shootdowns 703577 -12.8% 613459 interrupts.CPU14.CAL:Function_call_interrupts 1327 ± 9% +31.1% 1740 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts 970730 -10.2% 871724 interrupts.CPU14.TLB:TLB_shootdowns 703061 -14.4% 601807 ± 2% interrupts.CPU15.CAL:Function_call_interrupts 1359 ± 11% +32.9% 1806 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 970814 -11.9% 854968 ± 2% interrupts.CPU15.TLB:TLB_shootdowns 702259 -14.0% 603789 ± 2% interrupts.CPU16.CAL:Function_call_interrupts 1322 ± 12% +62.1% 2143 ± 36% interrupts.CPU16.RES:Rescheduling_interrupts 965542 -11.4% 855449 ± 3% interrupts.CPU16.TLB:TLB_shootdowns 692349 -10.9% 617151 interrupts.CPU17.CAL:Function_call_interrupts 1473 ± 5% +18.3% 1743 ± 2% interrupts.CPU17.RES:Rescheduling_interrupts 694637 ± 2% -14.5% 593740 ± 3% interrupts.CPU18.CAL:Function_call_interrupts 957723 ± 2% -12.1% 842104 ± 2% interrupts.CPU18.TLB:TLB_shootdowns 703427 -13.8% 606353 ± 3% interrupts.CPU19.CAL:Function_call_interrupts 1369 ± 12% +27.0% 1739 ± 2% interrupts.CPU19.RES:Rescheduling_interrupts 693288 -12.7% 605279 interrupts.CPU2.CAL:Function_call_interrupts 954902 -9.9% 860603 interrupts.CPU2.TLB:TLB_shootdowns 701553 -13.9% 604282 interrupts.CPU20.CAL:Function_call_interrupts 1410 ± 13% +29.8% 1831 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts 967623 -10.8% 863572 interrupts.CPU20.TLB:TLB_shootdowns 700649 -12.6% 612200 interrupts.CPU21.CAL:Function_call_interrupts 1333 ± 8% +29.0% 1719 interrupts.CPU21.RES:Rescheduling_interrupts 694783 ± 3% -11.9% 611893 ± 3% interrupts.CPU22.CAL:Function_call_interrupts 1370 ± 12% +28.6% 1761 ± 6% interrupts.CPU22.RES:Rescheduling_interrupts 698607 -13.7% 603168 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 1388 ± 15% +33.5% 1854 ± 13% interrupts.CPU23.RES:Rescheduling_interrupts 964584 -10.9% 859037 interrupts.CPU23.TLB:TLB_shootdowns 703147 -13.6% 607667 interrupts.CPU24.CAL:Function_call_interrupts 1375 ± 14% +25.0% 1719 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 967828 -10.5% 866348 ± 2% interrupts.CPU24.TLB:TLB_shootdowns 1420 ± 17% +22.8% 1744 ± 4% interrupts.CPU25.RES:Rescheduling_interrupts 649764 ± 3% -13.1% 564901 interrupts.CPU26.CAL:Function_call_interrupts 893070 ± 3% -10.8% 796963 interrupts.CPU26.TLB:TLB_shootdowns 679824 -15.0% 577572 ± 4% interrupts.CPU27.CAL:Function_call_interrupts 1422 ± 11% +25.4% 1783 ± 9% interrupts.CPU27.RES:Rescheduling_interrupts 932994 -12.6% 815406 ± 5% interrupts.CPU27.TLB:TLB_shootdowns 679241 -13.7% 586369 ± 3% interrupts.CPU28.CAL:Function_call_interrupts 6836 ± 24% -31.2% 4700 ± 37% interrupts.CPU28.NMI:Non-maskable_interrupts 6836 ± 24% -31.2% 4700 ± 37% interrupts.CPU28.PMI:Performance_monitoring_interrupts 1407 ± 11% +24.9% 1757 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 673812 ± 2% -12.6% 589005 ± 2% interrupts.CPU29.CAL:Function_call_interrupts 1526 ± 18% +73.5% 2648 ± 45% interrupts.CPU29.RES:Rescheduling_interrupts 924100 ± 3% -10.4% 828231 ± 2% interrupts.CPU29.TLB:TLB_shootdowns 704289 -13.4% 610109 ± 2% interrupts.CPU3.CAL:Function_call_interrupts 1348 ± 10% +37.3% 1850 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts 682507 -11.8% 601733 interrupts.CPU30.CAL:Function_call_interrupts 1392 ± 8% +23.5% 1718 ± 5% interrupts.CPU30.RES:Rescheduling_interrupts 678895 -13.2% 589283 ± 2% interrupts.CPU31.CAL:Function_call_interrupts 1498 ± 5% +23.1% 1845 ± 14% interrupts.CPU31.RES:Rescheduling_interrupts 931631 -10.7% 831559 ± 2% interrupts.CPU31.TLB:TLB_shootdowns 687333 -16.9% 570978 ± 3% interrupts.CPU32.CAL:Function_call_interrupts 1336 ± 13% +104.8% 2737 ± 57% interrupts.CPU32.RES:Rescheduling_interrupts 943010 -14.5% 806401 ± 3% interrupts.CPU32.TLB:TLB_shootdowns 678667 -13.3% 588663 interrupts.CPU33.CAL:Function_call_interrupts 1472 ± 13% +49.8% 2206 ± 37% interrupts.CPU33.RES:Rescheduling_interrupts 932921 -10.8% 832334 interrupts.CPU33.TLB:TLB_shootdowns 680120 ± 2% -12.9% 592713 interrupts.CPU34.CAL:Function_call_interrupts 1430 ± 9% +54.0% 2201 ± 25% interrupts.CPU34.RES:Rescheduling_interrupts 673525 -13.1% 585594 ± 2% interrupts.CPU35.CAL:Function_call_interrupts 1372 ± 11% +163.2% 3611 ± 88% interrupts.CPU35.RES:Rescheduling_interrupts 922652 -10.3% 827452 ± 3% interrupts.CPU35.TLB:TLB_shootdowns 682340 -14.0% 586601 interrupts.CPU36.CAL:Function_call_interrupts 1444 ± 13% +245.9% 4998 ±112% interrupts.CPU36.RES:Rescheduling_interrupts 937932 -11.7% 828119 interrupts.CPU36.TLB:TLB_shootdowns 687065 -16.9% 570647 ± 5% interrupts.CPU37.CAL:Function_call_interrupts 1349 ± 9% +76.9% 2387 ± 22% interrupts.CPU37.RES:Rescheduling_interrupts 941647 -14.4% 806476 ± 5% interrupts.CPU37.TLB:TLB_shootdowns 676722 -13.8% 583149 interrupts.CPU38.CAL:Function_call_interrupts 1425 ± 11% +22.7% 1748 ± 4% interrupts.CPU38.RES:Rescheduling_interrupts 927646 -10.9% 826215 interrupts.CPU38.TLB:TLB_shootdowns 672859 ± 3% -12.6% 588015 interrupts.CPU39.CAL:Function_call_interrupts 1420 ± 15% +104.3% 2902 ± 72% interrupts.CPU39.RES:Rescheduling_interrupts 923066 ± 2% -10.3% 828366 interrupts.CPU39.TLB:TLB_shootdowns 687341 ± 2% -13.1% 597105 ± 4% interrupts.CPU4.CAL:Function_call_interrupts 1357 ± 12% +33.1% 1806 ± 5% interrupts.CPU4.RES:Rescheduling_interrupts 665714 ± 4% -12.8% 580505 interrupts.CPU40.CAL:Function_call_interrupts 1437 ± 9% +24.3% 1786 ± 4% interrupts.CPU40.RES:Rescheduling_interrupts 912124 ± 4% -10.4% 817592 interrupts.CPU40.TLB:TLB_shootdowns 674212 -11.7% 595274 interrupts.CPU41.CAL:Function_call_interrupts 1370 ± 11% +30.7% 1791 ± 8% interrupts.CPU41.RES:Rescheduling_interrupts 924386 -8.8% 842676 ± 2% interrupts.CPU41.TLB:TLB_shootdowns 685445 -15.0% 582349 interrupts.CPU42.CAL:Function_call_interrupts 1357 ± 6% +80.8% 2453 ± 52% interrupts.CPU42.RES:Rescheduling_interrupts 940835 -12.4% 824601 ± 2% interrupts.CPU42.TLB:TLB_shootdowns 681048 -14.8% 580473 interrupts.CPU43.CAL:Function_call_interrupts 1370 ± 8% +67.7% 2297 ± 44% interrupts.CPU43.RES:Rescheduling_interrupts 935698 -12.6% 817615 ± 2% interrupts.CPU43.TLB:TLB_shootdowns 665992 ± 3% -14.5% 569213 ± 5% interrupts.CPU44.CAL:Function_call_interrupts 1428 ± 5% +26.6% 1809 ± 9% interrupts.CPU44.RES:Rescheduling_interrupts 915754 ± 3% -12.4% 802432 ± 5% interrupts.CPU44.TLB:TLB_shootdowns 672307 -17.1% 557560 ± 4% interrupts.CPU45.CAL:Function_call_interrupts 1448 ± 10% +61.3% 2336 ± 33% interrupts.CPU45.RES:Rescheduling_interrupts 926458 -15.2% 785408 ± 4% interrupts.CPU45.TLB:TLB_shootdowns 680228 -14.2% 583681 interrupts.CPU46.CAL:Function_call_interrupts 1373 ± 9% +27.3% 1749 ± 2% interrupts.CPU46.RES:Rescheduling_interrupts 934987 -11.9% 823546 ± 2% interrupts.CPU46.TLB:TLB_shootdowns 677361 ± 2% -13.6% 585181 ± 3% interrupts.CPU47.CAL:Function_call_interrupts 1360 ± 12% +31.5% 1789 ± 7% interrupts.CPU47.RES:Rescheduling_interrupts 929037 ± 2% -10.7% 829484 ± 3% interrupts.CPU47.TLB:TLB_shootdowns 678116 -18.4% 553511 ± 5% interrupts.CPU48.CAL:Function_call_interrupts 1437 ± 9% +60.4% 2306 ± 38% interrupts.CPU48.RES:Rescheduling_interrupts 931304 -16.3% 779815 ± 5% interrupts.CPU48.TLB:TLB_shootdowns 678284 -14.6% 579115 interrupts.CPU49.CAL:Function_call_interrupts 1398 ± 11% +29.3% 1808 ± 7% interrupts.CPU49.RES:Rescheduling_interrupts 929841 -12.0% 817820 interrupts.CPU49.TLB:TLB_shootdowns 675967 ± 4% -11.1% 600750 interrupts.CPU5.CAL:Function_call_interrupts 1501 ± 9% +50.9% 2265 ± 31% interrupts.CPU5.RES:Rescheduling_interrupts 930668 ± 4% -8.4% 852764 interrupts.CPU5.TLB:TLB_shootdowns 682482 -15.2% 578902 ± 2% interrupts.CPU50.CAL:Function_call_interrupts 1375 ± 13% +165.1% 3645 ± 55% interrupts.CPU50.RES:Rescheduling_interrupts 934420 -12.3% 819456 ± 2% interrupts.CPU50.TLB:TLB_shootdowns 642789 ± 5% -8.7% 586834 interrupts.CPU51.CAL:Function_call_interrupts 5226 ± 5% +10.3% 5764 ± 3% interrupts.CPU51.RES:Rescheduling_interrupts 696538 -13.9% 599479 interrupts.CPU52.CAL:Function_call_interrupts 1430 ± 6% +20.8% 1728 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts 960482 -11.3% 851845 interrupts.CPU52.TLB:TLB_shootdowns 692852 ± 3% -14.8% 590006 ± 2% interrupts.CPU53.CAL:Function_call_interrupts 954152 ± 3% -12.0% 840011 ± 2% interrupts.CPU53.TLB:TLB_shootdowns 703326 -16.8% 585211 ± 2% interrupts.CPU54.CAL:Function_call_interrupts 1340 ± 13% +38.3% 1854 ± 10% interrupts.CPU54.RES:Rescheduling_interrupts 968303 -13.9% 833620 ± 2% interrupts.CPU54.TLB:TLB_shootdowns 700662 -14.9% 596044 ± 3% interrupts.CPU55.CAL:Function_call_interrupts 1447 ± 10% +19.3% 1727 ± 2% interrupts.CPU55.RES:Rescheduling_interrupts 965633 -11.9% 850274 ± 3% interrupts.CPU55.TLB:TLB_shootdowns 698513 -14.4% 598186 ± 2% interrupts.CPU56.CAL:Function_call_interrupts 1343 ± 7% +48.2% 1989 ± 16% interrupts.CPU56.RES:Rescheduling_interrupts 963686 -11.4% 853692 ± 2% interrupts.CPU56.TLB:TLB_shootdowns 691567 ± 2% -14.1% 593767 ± 2% interrupts.CPU58.CAL:Function_call_interrupts 1368 ± 16% +27.3% 1742 ± 4% interrupts.CPU58.RES:Rescheduling_interrupts 952265 ± 2% -11.1% 846893 ± 2% interrupts.CPU58.TLB:TLB_shootdowns 694992 ± 2% -14.4% 595234 interrupts.CPU59.CAL:Function_call_interrupts 1421 ± 6% +21.3% 1724 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts 957262 ± 2% -11.5% 846704 interrupts.CPU59.TLB:TLB_shootdowns 695538 -12.1% 611509 interrupts.CPU6.CAL:Function_call_interrupts 956749 -9.0% 870507 interrupts.CPU6.TLB:TLB_shootdowns 678598 ± 4% -15.2% 575719 ± 3% interrupts.CPU60.CAL:Function_call_interrupts 1450 ± 14% +20.3% 1744 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts 932446 ± 4% -12.4% 816408 ± 3% interrupts.CPU60.TLB:TLB_shootdowns 691871 ± 3% -14.0% 595252 ± 2% interrupts.CPU61.CAL:Function_call_interrupts 1448 ± 2% +20.6% 1747 ± 5% interrupts.CPU61.RES:Rescheduling_interrupts 951054 ± 3% -10.9% 847621 ± 3% interrupts.CPU61.TLB:TLB_shootdowns 692000 ± 2% -14.4% 592092 ± 2% interrupts.CPU62.CAL:Function_call_interrupts 1441 ± 4% +22.7% 1768 ± 2% interrupts.CPU62.RES:Rescheduling_interrupts 952040 ± 2% -11.4% 843206 ± 2% interrupts.CPU62.TLB:TLB_shootdowns 703843 -14.4% 602227 interrupts.CPU65.CAL:Function_call_interrupts 1326 ± 16% +28.1% 1699 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 969377 -11.5% 857637 interrupts.CPU65.TLB:TLB_shootdowns 704517 -14.0% 605843 ± 2% interrupts.CPU66.CAL:Function_call_interrupts 1265 ± 8% +33.4% 1688 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts 973100 -11.3% 863481 interrupts.CPU66.TLB:TLB_shootdowns 703645 -15.6% 594213 ± 2% interrupts.CPU67.CAL:Function_call_interrupts 1361 ± 11% +30.4% 1775 ± 8% interrupts.CPU67.RES:Rescheduling_interrupts 970947 -12.9% 845733 ± 2% interrupts.CPU67.TLB:TLB_shootdowns 701434 -15.1% 595559 interrupts.CPU68.CAL:Function_call_interrupts 1331 ± 12% +27.7% 1699 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 967998 -12.4% 847856 interrupts.CPU68.TLB:TLB_shootdowns 693868 ± 2% -12.2% 609177 ± 2% interrupts.CPU69.CAL:Function_call_interrupts 1405 ± 7% +22.4% 1719 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts 699225 -13.0% 608212 ± 2% interrupts.CPU7.CAL:Function_call_interrupts 1436 ± 12% +22.2% 1755 ± 2% interrupts.CPU7.RES:Rescheduling_interrupts 960300 ± 2% -10.0% 864051 ± 2% interrupts.CPU7.TLB:TLB_shootdowns 694558 ± 2% -12.8% 605318 interrupts.CPU70.CAL:Function_call_interrupts 1401 ± 8% +38.1% 1935 ± 27% interrupts.CPU70.RES:Rescheduling_interrupts 956070 ± 2% -9.7% 862886 interrupts.CPU70.TLB:TLB_shootdowns 704274 -16.3% 589660 ± 2% interrupts.CPU71.CAL:Function_call_interrupts 1325 ± 8% +29.2% 1712 interrupts.CPU71.RES:Rescheduling_interrupts 968723 -13.3% 839460 ± 2% interrupts.CPU71.TLB:TLB_shootdowns 695605 -17.6% 573157 ± 3% interrupts.CPU72.CAL:Function_call_interrupts 1378 ± 8% +70.1% 2345 ± 44% interrupts.CPU72.RES:Rescheduling_interrupts 956330 -14.7% 816018 ± 3% interrupts.CPU72.TLB:TLB_shootdowns 699242 -15.1% 593486 interrupts.CPU73.CAL:Function_call_interrupts 1337 ± 11% +30.8% 1749 ± 6% interrupts.CPU73.RES:Rescheduling_interrupts 965399 -12.3% 846408 interrupts.CPU73.TLB:TLB_shootdowns 689511 ± 2% -13.5% 596151 interrupts.CPU74.CAL:Function_call_interrupts 1432 ± 8% +25.0% 1790 ± 9% interrupts.CPU74.RES:Rescheduling_interrupts 950872 ± 2% -10.6% 850539 ± 2% interrupts.CPU74.TLB:TLB_shootdowns 707174 -17.3% 585008 ± 3% interrupts.CPU75.CAL:Function_call_interrupts 1305 ± 9% +33.8% 1746 ± 3% interrupts.CPU75.RES:Rescheduling_interrupts 973369 -14.2% 835155 ± 3% interrupts.CPU75.TLB:TLB_shootdowns 701796 -14.8% 597873 ± 2% interrupts.CPU76.CAL:Function_call_interrupts 1295 ± 11% +50.1% 1943 ± 11% interrupts.CPU76.RES:Rescheduling_interrupts 968552 -12.0% 851993 ± 2% interrupts.CPU76.TLB:TLB_shootdowns 651441 ± 4% -12.8% 568200 ± 2% interrupts.CPU78.CAL:Function_call_interrupts 1606 ± 17% +52.7% 2453 ± 34% interrupts.CPU78.RES:Rescheduling_interrupts 895554 ± 4% -10.1% 805382 ± 2% interrupts.CPU78.TLB:TLB_shootdowns 683204 -15.7% 575816 ± 2% interrupts.CPU79.CAL:Function_call_interrupts 1399 ± 9% +97.3% 2761 ± 66% interrupts.CPU79.RES:Rescheduling_interrupts 936716 -13.1% 814280 ± 3% interrupts.CPU79.TLB:TLB_shootdowns 693626 ± 2% -15.1% 588961 ± 3% interrupts.CPU8.CAL:Function_call_interrupts 1442 ± 14% +28.9% 1859 ± 6% interrupts.CPU8.RES:Rescheduling_interrupts 955941 ± 2% -12.5% 836869 ± 3% interrupts.CPU8.TLB:TLB_shootdowns 682793 -16.6% 569790 ± 4% interrupts.CPU80.CAL:Function_call_interrupts 1325 ± 10% +66.2% 2204 ± 35% interrupts.CPU80.RES:Rescheduling_interrupts 937969 -13.9% 807134 ± 4% interrupts.CPU80.TLB:TLB_shootdowns 679962 -15.6% 573676 ± 3% interrupts.CPU81.CAL:Function_call_interrupts 1392 ± 10% +23.7% 1723 ± 6% interrupts.CPU81.RES:Rescheduling_interrupts 932820 -13.3% 808975 ± 3% interrupts.CPU81.TLB:TLB_shootdowns 681268 -15.0% 578786 ± 2% interrupts.CPU82.CAL:Function_call_interrupts 1337 ± 9% +84.0% 2461 ± 34% interrupts.CPU82.RES:Rescheduling_interrupts 936994 -12.8% 817128 ± 3% interrupts.CPU82.TLB:TLB_shootdowns 684548 -16.1% 574357 interrupts.CPU83.CAL:Function_call_interrupts 1255 ± 10% +35.2% 1697 ± 6% interrupts.CPU83.RES:Rescheduling_interrupts 939985 -13.6% 812315 interrupts.CPU83.TLB:TLB_shootdowns 683340 -15.4% 578407 ± 2% interrupts.CPU84.CAL:Function_call_interrupts 1338 ± 12% +138.6% 3193 ± 46% interrupts.CPU84.RES:Rescheduling_interrupts 938735 -12.4% 821866 ± 2% interrupts.CPU84.TLB:TLB_shootdowns 675547 -15.9% 568057 ± 2% interrupts.CPU85.CAL:Function_call_interrupts 1327 ± 11% +48.7% 1975 ± 14% interrupts.CPU85.RES:Rescheduling_interrupts 927466 -13.1% 805811 ± 2% interrupts.CPU85.TLB:TLB_shootdowns 663382 ± 6% -13.7% 572633 interrupts.CPU86.CAL:Function_call_interrupts 1375 ± 10% +32.2% 1818 ± 4% interrupts.CPU86.RES:Rescheduling_interrupts 910282 ± 5% -11.2% 807992 ± 2% interrupts.CPU86.TLB:TLB_shootdowns 684887 -16.2% 574035 ± 3% interrupts.CPU87.CAL:Function_call_interrupts 1332 ± 8% +53.5% 2045 ± 30% interrupts.CPU87.RES:Rescheduling_interrupts 940740 -13.7% 811836 ± 3% interrupts.CPU87.TLB:TLB_shootdowns 681473 -17.3% 563799 ± 2% interrupts.CPU88.CAL:Function_call_interrupts 1292 ± 11% +365.4% 6014 ±117% interrupts.CPU88.RES:Rescheduling_interrupts 936113 -14.9% 796687 ± 3% interrupts.CPU88.TLB:TLB_shootdowns 687252 -16.2% 575932 ± 2% interrupts.CPU89.CAL:Function_call_interrupts 1336 ± 6% +206.2% 4091 ± 67% interrupts.CPU89.RES:Rescheduling_interrupts 943782 -13.3% 818480 ± 2% interrupts.CPU89.TLB:TLB_shootdowns 679476 ± 4% -12.0% 598126 ± 2% interrupts.CPU9.CAL:Function_call_interrupts 1464 ± 9% +21.2% 1774 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts 937097 ± 4% -9.6% 846848 ± 3% interrupts.CPU9.TLB:TLB_shootdowns 658796 ± 2% -14.9% 560306 interrupts.CPU90.CAL:Function_call_interrupts 1392 ± 11% +39.3% 1940 ± 13% interrupts.CPU90.RES:Rescheduling_interrupts 903083 ± 2% -12.2% 793002 interrupts.CPU90.TLB:TLB_shootdowns 683163 -16.8% 568491 ± 3% interrupts.CPU91.CAL:Function_call_interrupts 1353 ± 11% +52.2% 2060 ± 26% interrupts.CPU91.RES:Rescheduling_interrupts 938623 -14.3% 804154 ± 3% interrupts.CPU91.TLB:TLB_shootdowns 681375 -16.4% 569429 interrupts.CPU92.CAL:Function_call_interrupts 1387 ± 13% +148.6% 3449 ± 65% interrupts.CPU92.RES:Rescheduling_interrupts 935999 -14.2% 803391 interrupts.CPU92.TLB:TLB_shootdowns 670818 ± 2% -14.9% 570630 ± 2% interrupts.CPU93.CAL:Function_call_interrupts 1376 ± 6% +85.2% 2548 ± 26% interrupts.CPU93.RES:Rescheduling_interrupts 921394 ± 3% -12.3% 808132 ± 2% interrupts.CPU93.TLB:TLB_shootdowns 676410 -16.3% 566320 interrupts.CPU94.CAL:Function_call_interrupts 1302 ± 10% +61.4% 2102 ± 28% interrupts.CPU94.RES:Rescheduling_interrupts 926760 -13.7% 799590 interrupts.CPU94.TLB:TLB_shootdowns 679116 ± 2% -16.3% 568659 ± 2% interrupts.CPU95.CAL:Function_call_interrupts 7833 -30.0% 5482 ± 30% interrupts.CPU95.NMI:Non-maskable_interrupts 7833 -30.0% 5482 ± 30% interrupts.CPU95.PMI:Performance_monitoring_interrupts 1323 ± 7% +28.9% 1706 ± 6% interrupts.CPU95.RES:Rescheduling_interrupts 931571 ± 2% -13.6% 805221 ± 2% interrupts.CPU95.TLB:TLB_shootdowns 648229 ± 7% -11.5% 573882 interrupts.CPU96.CAL:Function_call_interrupts 1490 ± 5% +77.7% 2649 ± 47% interrupts.CPU96.RES:Rescheduling_interrupts 680514 -20.1% 543698 ± 7% interrupts.CPU97.CAL:Function_call_interrupts 1366 ± 10% +49.8% 2047 ± 12% interrupts.CPU97.RES:Rescheduling_interrupts 933092 -17.3% 771491 ± 7% interrupts.CPU97.TLB:TLB_shootdowns 681553 -16.3% 570304 interrupts.CPU98.CAL:Function_call_interrupts 1313 ± 12% +42.0% 1865 ± 9% interrupts.CPU98.RES:Rescheduling_interrupts 937173 -13.7% 808446 interrupts.CPU98.TLB:TLB_shootdowns 682309 -18.9% 553182 ± 4% interrupts.CPU99.CAL:Function_call_interrupts 1330 ± 10% +41.9% 1888 ± 9% interrupts.CPU99.RES:Rescheduling_interrupts 936364 -16.5% 781811 ± 4% interrupts.CPU99.TLB:TLB_shootdowns 151511 ± 9% +49.0% 225761 ± 4% interrupts.RES:Rescheduling_interrupts 97797792 -11.5% 86572252 interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-skl-1sp1: 40 threads Skylake with 64G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-skl-1sp1/anon-r-rand/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.59 +8.5% 0.64 vm-scalability.free_time 13.25 -3.8 9.50 ± 57% perf-profile.calltrace.cycles-pp.do_rw_once 2.31 ± 3% -0.8 1.56 ± 58% perf-profile.calltrace.cycles-pp.lrand48_r@plt 562.25 ±100% +270.2% 2081 ± 74% turbostat.C6 0.16 ±102% +276.9% 0.61 ± 74% turbostat.CPU%c1 562.25 ±100% +270.2% 2081 ± 74% cpuidle.C6.usage 12025 ± 79% +203.9% 36547 ± 68% cpuidle.POLL.time 831.00 ±102% +274.8% 3114 ± 74% cpuidle.POLL.usage 867672 ± 4% +20.3% 1043531 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg 867672 ± 4% +20.3% 1043531 ± 10% sched_debug.cfs_rq:/.runnable_weight.max 867672 ± 4% +20.3% 1043531 ± 10% sched_debug.cfs_rq:/.runnable_weight.min *************************************************************************************************** ivb44: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/100t/debian-x86_64-2018-04-03.cgz/300s/ivb44/sieve/reaim/0x42d commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.23 -7.2% 0.21 reaim.child_systime 521.34 -1.6% 513.05 reaim.child_utime 52646 +1.5% 53455 reaim.jobs_per_min 526.46 +1.5% 534.56 reaim.jobs_per_min_child 83.05 +3.0% 85.55 reaim.jti 53524 +1.6% 54373 reaim.max_jobs_per_min 11.40 -1.5% 11.23 reaim.parent_time 16.40 -14.9% 13.95 ± 6% reaim.std_dev_percent 1.55 -14.9% 1.32 ± 5% reaim.std_dev_time 308.95 -1.3% 305.00 reaim.time.elapsed_time 308.95 -1.3% 305.00 reaim.time.elapsed_time.max 730136 +4.0% 759300 reaim.time.involuntary_context_switches 1527809 -2.1% 1495983 reaim.time.minor_page_faults 11991 -1.6% 11800 reaim.time.user_time 1034700 ± 98% -65.4% 358148 ±152% cpuidle.C1.usage 12.68 -4.9% 12.06 turbostat.RAMWatt 3292 +3.0% 3392 vmstat.system.cs 37711 ± 8% +16.5% 43917 ± 7% numa-vmstat.node0.nr_active_anon 36539 ± 11% +18.5% 43287 ± 8% numa-vmstat.node0.nr_anon_pages 37711 ± 8% +16.5% 43917 ± 7% numa-vmstat.node0.nr_zone_active_anon 233268 ± 7% -44.8% 128820 meminfo.DirectMap4k 1713247 +59.9% 2739456 ± 2% meminfo.Memused 26817 ± 3% -8.3% 24581 ± 4% meminfo.Shmem 5849 +85.1% 10829 ± 3% meminfo.max_used_kB 38478 ± 5% -9.4% 34878 ± 6% softirqs.CPU15.RCU 49996 ± 6% -10.5% 44746 ± 4% softirqs.CPU16.RCU 51308 ± 3% -11.0% 45660 ± 3% softirqs.CPU18.RCU 45074 ± 3% -7.4% 41742 ± 6% softirqs.CPU20.RCU 152664 ± 9% +16.6% 177953 ± 7% numa-meminfo.node0.Active 150768 ± 8% +16.5% 175626 ± 7% numa-meminfo.node0.Active(anon) 146071 ± 11% +18.5% 173124 ± 8% numa-meminfo.node0.AnonPages 875373 ± 5% +62.0% 1417768 ± 14% numa-meminfo.node0.MemUsed 106396 ± 19% -30.0% 74474 ± 30% numa-meminfo.node1.AnonHugePages 837790 ± 5% +57.7% 1320900 ± 14% numa-meminfo.node1.MemUsed 1592399 -1.6% 1566815 proc-vmstat.nr_dirty_background_threshold 3188693 -1.6% 3137462 proc-vmstat.nr_dirty_threshold 16034684 -1.6% 15778488 proc-vmstat.nr_free_pages 6705 ± 3% -8.4% 6144 ± 4% proc-vmstat.nr_shmem 239651 ± 3% -14.0% 206060 ± 5% proc-vmstat.numa_hint_faults 2176510 -1.9% 2135475 proc-vmstat.numa_hit 2166960 -1.9% 2125941 proc-vmstat.numa_local 229894 -14.2% 197136 ± 4% proc-vmstat.numa_pages_migrated 4498 ± 14% -23.5% 3442 ± 16% proc-vmstat.pgactivate 2240585 -1.7% 2201618 proc-vmstat.pgalloc_normal 2308066 -1.9% 2263904 proc-vmstat.pgfault 2218407 -1.7% 2180545 proc-vmstat.pgfree 229894 -14.2% 197136 ± 4% proc-vmstat.pgmigrate_success 161870 -1.4% 159562 interrupts.CAL:Function_call_interrupts 431.50 ± 86% +146.3% 1062 ± 36% interrupts.CPU11.RES:Rescheduling_interrupts 5167 ± 35% -66.9% 1709 ± 99% interrupts.CPU25.NMI:Non-maskable_interrupts 5167 ± 35% -66.9% 1709 ± 99% interrupts.CPU25.PMI:Performance_monitoring_interrupts 8239 -57.9% 3469 ± 70% interrupts.CPU26.NMI:Non-maskable_interrupts 8239 -57.9% 3469 ± 70% interrupts.CPU26.PMI:Performance_monitoring_interrupts 1047 ±173% +414.9% 5392 ± 31% interrupts.CPU33.NMI:Non-maskable_interrupts 1047 ±173% +414.9% 5392 ± 31% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1101 ± 17% -38.0% 682.75 ± 40% interrupts.CPU39.RES:Rescheduling_interrupts 6123 ± 32% -98.8% 75.00 ± 69% interrupts.CPU43.NMI:Non-maskable_interrupts 6123 ± 32% -98.8% 75.00 ± 69% interrupts.CPU43.PMI:Performance_monitoring_interrupts 7222 ± 24% -63.4% 2640 ±110% interrupts.CPU6.NMI:Non-maskable_interrupts 7222 ± 24% -63.4% 2640 ±110% interrupts.CPU6.PMI:Performance_monitoring_interrupts 7200 ± 24% -87.8% 879.00 ±172% interrupts.CPU7.NMI:Non-maskable_interrupts 7200 ± 24% -87.8% 879.00 ±172% interrupts.CPU7.PMI:Performance_monitoring_interrupts 6194 ± 33% -68.8% 1931 ±100% interrupts.CPU9.NMI:Non-maskable_interrupts 6194 ± 33% -68.8% 1931 ±100% interrupts.CPU9.PMI:Performance_monitoring_interrupts 275.25 ± 62% +359.3% 1264 ± 92% interrupts.CPU9.RES:Rescheduling_interrupts 210923 ± 12% -27.2% 153491 ± 19% interrupts.NMI:Non-maskable_interrupts 210923 ± 12% -27.2% 153491 ± 19% interrupts.PMI:Performance_monitoring_interrupts 49953 ± 49% +262.4% 181009 ± 59% sched_debug.cfs_rq:/.load.max 31.24 ± 8% +23.2% 38.49 ± 7% sched_debug.cfs_rq:/.load_avg.avg 249.96 ± 25% +120.3% 550.62 ± 5% sched_debug.cfs_rq:/.load_avg.max 49.37 ± 14% +90.1% 93.88 ± 5% sched_debug.cfs_rq:/.load_avg.stddev 1598577 -18.0% 1311245 ± 9% sched_debug.cfs_rq:/.min_vruntime.stddev 0.84 ± 8% -28.0% 0.61 ± 11% sched_debug.cfs_rq:/.nr_running.avg 1.21 ± 23% -23.8% 0.93 ± 13% sched_debug.cfs_rq:/.nr_spread_over.avg 18.36 ± 2% -13.7% 15.84 ± 8% sched_debug.cfs_rq:/.runnable_load_avg.avg 1598559 -18.0% 1311295 ± 9% sched_debug.cfs_rq:/.spread0.stddev 887.34 ± 7% -24.6% 669.03 ± 14% sched_debug.cfs_rq:/.util_avg.avg 388.51 ± 5% -17.0% 322.56 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.avg 70194 ± 8% +24.2% 87204 ± 10% sched_debug.cpu.avg_idle.stddev 18.51 ± 2% -14.0% 15.92 ± 8% sched_debug.cpu.cpu_load[0].avg 49.46 ± 48% +253.3% 174.75 ± 60% sched_debug.cpu.cpu_load[1].max 18.70 ± 2% -13.6% 16.16 ± 7% sched_debug.cpu.cpu_load[2].avg 50.62 ± 52% +209.5% 156.71 ± 58% sched_debug.cpu.cpu_load[2].max 9.82 ± 47% +146.1% 24.17 ± 53% sched_debug.cpu.cpu_load[2].stddev 18.87 ± 2% -17.0% 15.67 ± 4% sched_debug.cpu.cpu_load[3].avg 55.96 ± 52% +115.1% 120.38 ± 57% sched_debug.cpu.cpu_load[3].max 9.96 ± 45% +91.8% 19.10 ± 49% sched_debug.cpu.cpu_load[3].stddev 19.20 ± 2% -21.2% 15.14 ± 5% sched_debug.cpu.cpu_load[4].avg 5883 ± 6% -29.4% 4152 ± 16% sched_debug.cpu.curr->pid.avg 0.00 ± 10% -13.7% 0.00 ± 7% sched_debug.cpu.next_balance.stddev 1.46 ± 7% -18.7% 1.19 ± 9% sched_debug.cpu.nr_running.avg 6288 ± 2% +13.1% 7111 ± 6% sched_debug.cpu.nr_switches.min 8.17 ± 11% -31.1% 5.62 ± 23% sched_debug.cpu.nr_uninterruptible.max 3.02 -13.8% 2.60 ± 10% sched_debug.cpu.nr_uninterruptible.stddev 55208 ± 5% +21.4% 67038 ± 12% sched_debug.cpu.sched_count.max 1.895e+10 +1.2% 1.917e+10 perf-stat.i.branch-instructions 3.382e+08 +0.9% 3.414e+08 perf-stat.i.branch-misses 56664828 ± 3% -11.3% 50263937 perf-stat.i.cache-misses 4.205e+09 +1.8% 4.281e+09 perf-stat.i.cache-references 3285 +3.2% 3389 perf-stat.i.context-switches 2326 ± 3% +14.2% 2657 ± 2% perf-stat.i.cycles-between-cache-misses 2235056 +5.1% 2348695 ± 3% perf-stat.i.dTLB-load-misses 4.554e+09 +1.1% 4.604e+09 perf-stat.i.dTLB-loads 0.08 -0.0 0.08 perf-stat.i.dTLB-store-miss-rate% 8187526 -6.3% 7673337 perf-stat.i.dTLB-store-misses 1.063e+10 +1.2% 1.076e+10 perf-stat.i.dTLB-stores 89.76 +0.9 90.71 perf-stat.i.iTLB-load-miss-rate% 8.794e+10 +1.2% 8.896e+10 perf-stat.i.instructions 469494 ± 2% -10.4% 420724 ± 2% perf-stat.i.node-load-misses 5998757 ± 2% -5.0% 5701475 perf-stat.i.node-loads 5.57 ± 5% -0.8 4.80 ± 6% perf-stat.i.node-store-miss-rate% 1731276 ± 2% -34.9% 1126964 ± 7% perf-stat.i.node-store-misses 54616643 ± 3% -12.3% 47917886 perf-stat.i.node-stores 1.35 ± 3% -0.2 1.17 perf-stat.overall.cache-miss-rate% 1.32 -1.3% 1.30 perf-stat.overall.cpi 2046 ± 3% +12.4% 2299 perf-stat.overall.cycles-between-cache-misses 0.05 +0.0 0.05 ± 3% perf-stat.overall.dTLB-load-miss-rate% 0.08 -0.0 0.07 perf-stat.overall.dTLB-store-miss-rate% 0.76 +1.4% 0.77 perf-stat.overall.ipc 3.08 ± 5% -0.8 2.30 ± 6% perf-stat.overall.node-store-miss-rate% 1.889e+10 +1.2% 1.911e+10 perf-stat.ps.branch-instructions 3.371e+08 +0.9% 3.403e+08 perf-stat.ps.branch-misses 56481680 ± 3% -11.3% 50101279 perf-stat.ps.cache-misses 4.191e+09 +1.8% 4.267e+09 perf-stat.ps.cache-references 3274 +3.2% 3378 perf-stat.ps.context-switches 2227912 +5.1% 2341150 ± 3% perf-stat.ps.dTLB-load-misses 4.539e+09 +1.1% 4.589e+09 perf-stat.ps.dTLB-loads 8160800 -6.3% 7648007 perf-stat.ps.dTLB-store-misses 1.06e+10 +1.2% 1.072e+10 perf-stat.ps.dTLB-stores 8.765e+10 +1.2% 8.866e+10 perf-stat.ps.instructions 467965 ± 2% -10.4% 419341 ± 2% perf-stat.ps.node-load-misses 5979334 ± 2% -5.0% 5682912 perf-stat.ps.node-loads 1725519 ± 2% -34.9% 1123053 ± 7% perf-stat.ps.node-store-misses 54440120 ± 3% -12.3% 47762454 perf-stat.ps.node-stores 53.48 ± 27% -28.7 24.79 ± 30% perf-profile.calltrace.cycles-pp.apic_timer_interrupt 51.74 ± 27% -27.8 23.92 ± 30% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt 41.38 ± 27% -22.0 19.39 ± 31% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 32.74 ± 27% -17.4 15.31 ± 32% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 22.84 ± 27% -11.0 11.80 ± 31% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 22.44 ± 27% -11.0 11.44 ± 31% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 14.74 ± 26% -7.5 7.24 ± 31% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 11.85 ± 27% -6.4 5.48 ± 31% perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer 5.29 ± 33% -3.0 2.33 ± 24% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 4.16 ± 33% -2.3 1.85 ± 24% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 3.68 ± 33% -2.1 1.58 ± 35% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 2.26 ± 27% -1.9 0.31 ±101% perf-profile.calltrace.cycles-pp.page_fault 2.24 ± 26% -1.9 0.31 ±101% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 2.60 ± 36% -1.9 0.70 ± 84% perf-profile.calltrace.cycles-pp.delay_tsc.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write 2.19 ± 27% -1.9 0.30 ±101% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 3.20 ± 24% -1.8 1.39 ± 30% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 2.94 ± 32% -1.6 1.30 ± 20% perf-profile.calltrace.cycles-pp.interrupt_entry 1.92 ± 30% -1.3 0.60 ± 61% perf-profile.calltrace.cycles-pp.account_user_time.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.67 ± 26% -1.2 0.43 ±100% perf-profile.calltrace.cycles-pp.hrtimer_active.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle 1.88 ± 22% -1.2 0.69 ± 78% perf-profile.calltrace.cycles-pp.__remove_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 1.87 ± 30% -1.2 0.70 ± 24% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 1.86 ± 29% -1.2 0.70 ± 24% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.31 ± 32% -1.0 0.31 ±103% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve 1.31 ± 32% -1.0 0.31 ±103% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.31 ± 32% -1.0 0.31 ±103% perf-profile.calltrace.cycles-pp.execve 1.31 ± 32% -1.0 0.31 ±103% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.30 ± 33% -1.0 0.31 ±103% perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.75 ± 19% -0.9 0.82 ± 19% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 1.95 ± 31% -0.9 1.02 ± 31% perf-profile.calltrace.cycles-pp.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt 1.63 ± 39% -0.9 0.73 ± 73% perf-profile.calltrace.cycles-pp.rcu_check_callbacks.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.58 ± 25% -0.8 0.76 ± 74% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 1.58 ± 32% -0.8 0.77 ± 36% perf-profile.calltrace.cycles-pp.interrupt_entry.apic_timer_interrupt 1.20 ± 19% -0.8 0.41 ±101% perf-profile.calltrace.cycles-pp.__update_load_avg_se.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle 1.16 ± 18% -0.8 0.38 ±103% perf-profile.calltrace.cycles-pp.reweight_entity.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle 1.24 ± 17% -0.7 0.52 ± 61% perf-profile.calltrace.cycles-pp.native_write_msr.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt 0.13 ±173% +0.5 0.64 ± 15% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt 0.20 ±173% +0.7 0.90 ± 8% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +0.8 0.82 ± 25% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle 0.00 +1.2 1.19 ± 29% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +1.6 1.65 ± 27% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.20 ±173% +2.1 2.31 ± 15% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.14 ±173% +2.2 2.34 ± 10% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.21 ±173% +2.5 2.66 ± 13% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.22 ±173% +2.7 2.94 ± 26% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.24 ±173% +3.2 3.43 ± 15% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.50 ±173% +6.4 6.86 ± 11% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 0.52 ±173% +6.7 7.23 ± 11% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 17.09 ±118% +31.4 48.46 ± 19% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 17.75 ±119% +38.9 56.64 ± 18% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 18.06 ±119% +42.8 60.85 ± 17% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 18.07 ±119% +42.8 60.90 ± 17% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 18.07 ±119% +42.8 60.90 ± 17% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 18.07 ±119% +43.4 61.49 ± 17% perf-profile.calltrace.cycles-pp.secondary_startup_64 *************************************************************************************************** lkp-hsx04: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-hsx04/anon-r-seq/vm-scalability/0x12 commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 14% 2:4 perf-profile.calltrace.cycles-pp.error_entry %stddev %change %stddev \ | \ 0.49 ± 2% -76.2% 0.12 ± 17% vm-scalability.free_time 161.58 -34.4% 105.92 vm-scalability.time.system_time 0.43 -0.1 0.30 mpstat.cpu.sys% 15814 ± 50% +69.9% 26862 numa-numastat.node3.other_node 116340 ± 3% -9.6% 105201 ± 4% softirqs.CPU121.TIMER 6900550 +21.4% 8379710 ± 5% meminfo.DirectMap2M 318964 ± 17% -54.5% 145144 ± 8% meminfo.DirectMap4k 32316805 ± 3% +5.3e+06 37654607 ± 10% syscalls.sys_close.noise.2% 31898706 ± 3% +5.4e+06 37263448 ± 11% syscalls.sys_close.noise.5% 22032386 ± 5% +4.8e+06 26783409 ± 15% syscalls.sys_close.noise.50% 223235 +63.0% 363897 ± 34% numa-meminfo.node1.FilePages 4801 ± 87% +2729.3% 135855 ± 93% numa-meminfo.node1.Shmem 374532 ± 35% -40.9% 221434 numa-meminfo.node2.FilePages 129667 ± 97% -98.8% 1502 ± 35% numa-meminfo.node2.Inactive 129533 ± 97% -99.2% 1005 ± 18% numa-meminfo.node2.Inactive(anon) 135959 ± 93% -98.7% 1743 ± 29% numa-meminfo.node2.Shmem 218359 +6.0% 231431 ± 5% numa-meminfo.node3.Unevictable 523460 ± 3% +9.9% 575078 ± 5% perf-stat.i.dTLB-load-misses 22330 ± 9% -14.5% 19086 ± 11% perf-stat.i.dTLB-store-misses 30413 ± 3% +11.9% 34043 ± 5% perf-stat.i.iTLB-loads 0.00 ± 3% +0.0 0.00 ± 5% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 9% -0.0 0.00 ± 11% perf-stat.overall.dTLB-store-miss-rate% 522122 ± 3% +9.9% 573731 ± 5% perf-stat.ps.dTLB-load-misses 22400 ± 9% -14.5% 19158 ± 11% perf-stat.ps.dTLB-store-misses 30450 ± 3% +12.0% 34103 ± 6% perf-stat.ps.iTLB-loads 55809 +63.1% 90998 ± 34% numa-vmstat.node1.nr_file_pages 1200 ± 87% +2731.2% 33988 ± 93% numa-vmstat.node1.nr_shmem 93686 ± 35% -40.9% 55358 numa-vmstat.node2.nr_file_pages 32436 ± 97% -99.2% 250.75 ± 18% numa-vmstat.node2.nr_inactive_anon 1263 ± 5% -6.2% 1184 numa-vmstat.node2.nr_mapped 34042 ± 93% -98.7% 435.25 ± 30% numa-vmstat.node2.nr_shmem 32436 ± 97% -99.2% 250.75 ± 18% numa-vmstat.node2.nr_zone_inactive_anon 54589 +6.0% 57857 ± 5% numa-vmstat.node3.nr_unevictable 54589 +6.0% 57857 ± 5% numa-vmstat.node3.nr_zone_unevictable 90741 ± 8% +12.5% 102063 numa-vmstat.node3.numa_other 18317 ± 3% +468.2% 104076 ± 81% sched_debug.cfs_rq:/.load.max 17.79 ± 7% -9.8% 16.04 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.max 1172 ± 8% -9.7% 1058 sched_debug.cfs_rq:/.util_avg.max 64.61 ± 7% -17.3% 53.42 ± 12% sched_debug.cfs_rq:/.util_avg.stddev 66.25 ±102% -75.2% 16.46 ± 7% sched_debug.cpu.cpu_load[0].max 5.56 ± 97% -70.6% 1.64 ± 10% sched_debug.cpu.cpu_load[0].stddev 41.21 ± 81% -60.2% 16.42 ± 5% sched_debug.cpu.cpu_load[1].max 3.49 ± 75% -54.1% 1.60 ± 9% sched_debug.cpu.cpu_load[1].stddev 28.33 ± 57% -41.3% 16.62 ± 5% sched_debug.cpu.cpu_load[2].max 2.46 ± 49% -34.4% 1.61 ± 6% sched_debug.cpu.cpu_load[2].stddev 180.75 ± 9% -16.3% 151.25 ± 2% interrupts.48:PCI-MSI.1572867-edge.eth0-TxRx-3 217.75 ± 23% -23.8% 166.00 ± 10% interrupts.58:PCI-MSI.1572877-edge.eth0-TxRx-13 74.50 ± 89% +1247.3% 1003 ±134% interrupts.CPU10.RES:Rescheduling_interrupts 608.25 ±115% +381.2% 2927 ± 25% interrupts.CPU11.NMI:Non-maskable_interrupts 608.25 ±115% +381.2% 2927 ± 25% interrupts.CPU11.PMI:Performance_monitoring_interrupts 217.75 ± 23% -23.8% 166.00 ± 10% interrupts.CPU13.58:PCI-MSI.1572877-edge.eth0-TxRx-13 70.25 ± 48% +1085.1% 832.50 ± 77% interrupts.CPU16.RES:Rescheduling_interrupts 457.75 ± 37% +98.3% 907.75 ± 50% interrupts.CPU25.RES:Rescheduling_interrupts 441.50 ± 25% +239.3% 1498 ± 33% interrupts.CPU29.RES:Rescheduling_interrupts 180.75 ± 9% -16.3% 151.25 ± 2% interrupts.CPU3.48:PCI-MSI.1572867-edge.eth0-TxRx-3 2258 ± 31% -62.1% 855.25 ±100% interrupts.CPU31.NMI:Non-maskable_interrupts 2258 ± 31% -62.1% 855.25 ±100% interrupts.CPU31.PMI:Performance_monitoring_interrupts 947.00 ± 41% -53.6% 439.25 ± 18% interrupts.CPU37.RES:Rescheduling_interrupts 873.50 ±100% +239.3% 2963 ± 23% interrupts.CPU39.NMI:Non-maskable_interrupts 873.50 ±100% +239.3% 2963 ± 23% interrupts.CPU39.PMI:Performance_monitoring_interrupts 2942 ± 24% -61.4% 1136 ±106% interrupts.CPU53.NMI:Non-maskable_interrupts 2942 ± 24% -61.4% 1136 ±106% interrupts.CPU53.PMI:Performance_monitoring_interrupts 2037 ± 62% -78.7% 433.25 ±173% interrupts.CPU54.NMI:Non-maskable_interrupts 2037 ± 62% -78.7% 433.25 ±173% interrupts.CPU54.PMI:Performance_monitoring_interrupts 1263 ± 60% -64.6% 446.75 ± 91% interrupts.CPU55.RES:Rescheduling_interrupts 88.00 ± 75% +217.0% 279.00 ± 83% interrupts.CPU7.RES:Rescheduling_interrupts 2102 ± 34% +36.9% 2878 ± 23% interrupts.CPU74.NMI:Non-maskable_interrupts 2102 ± 34% +36.9% 2878 ± 23% interrupts.CPU74.PMI:Performance_monitoring_interrupts 3360 -56.5% 1460 ± 19% interrupts.CPU80.NMI:Non-maskable_interrupts 3360 -56.5% 1460 ± 19% interrupts.CPU80.PMI:Performance_monitoring_interrupts 284.50 ± 83% +263.0% 1032 ± 84% interrupts.CPU84.RES:Rescheduling_interrupts 86.25 ± 77% +178.8% 240.50 ± 33% interrupts.CPU87.RES:Rescheduling_interrupts 2537 ± 53% -98.2% 46.25 ±145% interrupts.CPU88.NMI:Non-maskable_interrupts 2537 ± 53% -98.2% 46.25 ±145% interrupts.CPU88.PMI:Performance_monitoring_interrupts 2542 ± 32% -60.1% 1014 ±135% interrupts.CPU90.NMI:Non-maskable_interrupts 2542 ± 32% -60.1% 1014 ±135% interrupts.CPU90.PMI:Performance_monitoring_interrupts 3266 ± 6% -64.2% 1169 ±117% interrupts.CPU91.NMI:Non-maskable_interrupts 3266 ± 6% -64.2% 1169 ±117% interrupts.CPU91.PMI:Performance_monitoring_interrupts 2575 ± 32% -77.8% 572.25 ±117% interrupts.CPU93.NMI:Non-maskable_interrupts 2575 ± 32% -77.8% 572.25 ±117% interrupts.CPU93.PMI:Performance_monitoring_interrupts 1473 ± 66% -75.5% 360.50 ±134% interrupts.CPU94.RES:Rescheduling_interrupts *************************************************************************************************** lkp-skl-fpga01: 104 threads Skylake with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-skl-fpga01/page_fault2/will-it-scale commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 50% 2:4 dmesg.WARNING:at_ip___perf_sw_event/0x 3:4 25% 4:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry 6:4 49% 8:4 perf-profile.calltrace.cycles-pp.error_entry %stddev %change %stddev \ | \ 89400 +17.2% 104814 will-it-scale.per_process_ops 9297688 +17.2% 10900679 will-it-scale.workload 2325 ± 3% -9.8% 2096 ± 6% boot-time.idle 555.75 ± 5% -14.0% 477.75 ± 3% slabinfo.Acpi-ParseExt.active_objs 332113 +21.5% 403621 ± 2% softirqs.RCU 252784 ± 8% -38.5% 155504 ± 9% meminfo.DirectMap4k 22277235 +35.3% 30136935 meminfo.Memused 0.00 ± 28% -0.0 0.00 ± 31% mpstat.cpu.soft% 2.31 +0.5 2.83 ± 2% mpstat.cpu.usr% 11163504 +35.9% 15171129 numa-meminfo.node0.MemUsed 11052278 +35.4% 14959540 numa-meminfo.node1.MemUsed 96.75 -1.6% 95.25 vmstat.cpu.sy 1137 ± 2% +7.3% 1219 vmstat.system.cs 0.43 ± 16% +1.3 1.78 ± 63% turbostat.PKG_% 349.60 +3.0% 360.04 turbostat.PkgWatt 309.04 +7.2% 331.44 turbostat.RAMWatt 1.409e+09 +17.5% 1.655e+09 numa-numastat.node0.local_node 1.409e+09 +17.5% 1.655e+09 numa-numastat.node0.numa_hit 1.404e+09 +16.5% 1.636e+09 numa-numastat.node1.local_node 1.404e+09 +16.5% 1.636e+09 numa-numastat.node1.numa_hit 7.077e+08 +17.4% 8.309e+08 numa-vmstat.node0.numa_hit 7.076e+08 +17.4% 8.309e+08 numa-vmstat.node0.numa_local 7.049e+08 +16.9% 8.24e+08 numa-vmstat.node1.numa_hit 7.047e+08 +16.9% 8.238e+08 numa-vmstat.node1.numa_local 1752433 +1.2% 1773176 proc-vmstat.nr_active_anon 1560708 +5.0% 1638427 proc-vmstat.nr_anon_pages 4333598 -4.5% 4137510 proc-vmstat.nr_dirty_background_threshold 8677828 -4.5% 8285137 proc-vmstat.nr_dirty_threshold 43618122 -4.5% 41654193 proc-vmstat.nr_free_pages 6625 -5.1% 6290 proc-vmstat.nr_page_table_pages 1752432 +1.2% 1773175 proc-vmstat.nr_zone_active_anon 2.812e+09 +17.0% 3.291e+09 proc-vmstat.numa_hit 2.812e+09 +17.0% 3.291e+09 proc-vmstat.numa_local 2.814e+09 +17.0% 3.292e+09 proc-vmstat.pgalloc_normal 2.804e+09 +17.0% 3.28e+09 proc-vmstat.pgfault 2.812e+09 +17.0% 3.29e+09 proc-vmstat.pgfree 154553 ± 25% -65.1% 53958 ± 70% sched_debug.cfs_rq:/.MIN_vruntime.avg 1109781 ± 17% -57.5% 471293 ± 61% sched_debug.cfs_rq:/.MIN_vruntime.stddev 23.30 ± 4% -20.6% 18.51 ± 10% sched_debug.cfs_rq:/.load_avg.avg 64.18 ± 12% -21.9% 50.14 ± 8% sched_debug.cfs_rq:/.load_avg.stddev 154553 ± 25% -65.1% 53958 ± 70% sched_debug.cfs_rq:/.max_vruntime.avg 1109781 ± 17% -57.5% 471293 ± 61% sched_debug.cfs_rq:/.max_vruntime.stddev 244525 ± 5% -40.3% 145987 ± 5% sched_debug.cfs_rq:/.min_vruntime.stddev 149.80 -24.9% 112.45 sched_debug.cfs_rq:/.nr_spread_over.avg 860.54 ± 9% +83.3% 1577 ± 23% sched_debug.cfs_rq:/.nr_spread_over.max 95.25 -41.9% 55.33 ± 2% sched_debug.cfs_rq:/.nr_spread_over.min 119.73 ± 4% +69.1% 202.47 ± 15% sched_debug.cfs_rq:/.nr_spread_over.stddev 10.87 ± 5% -18.1% 8.90 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.avg -1324923 -42.0% -768952 sched_debug.cfs_rq:/.spread0.min 244399 ± 5% -40.4% 145666 ± 5% sched_debug.cfs_rq:/.spread0.stddev 11.05 ± 4% -19.1% 8.94 ± 9% sched_debug.cpu.cpu_load[0].avg 25.16 ± 33% -57.5% 10.69 ± 68% sched_debug.cpu.cpu_load[0].stddev 11.18 ± 3% -19.4% 9.01 ± 7% sched_debug.cpu.cpu_load[1].avg 25.07 ± 32% -54.9% 11.29 ± 56% sched_debug.cpu.cpu_load[1].stddev 11.38 ± 3% -20.8% 9.02 ± 6% sched_debug.cpu.cpu_load[2].avg 24.55 ± 33% -55.6% 10.89 ± 47% sched_debug.cpu.cpu_load[2].stddev 11.49 ± 4% -20.9% 9.09 ± 4% sched_debug.cpu.cpu_load[3].avg 23.25 ± 33% -52.7% 11.00 ± 36% sched_debug.cpu.cpu_load[3].stddev 11.45 ± 5% -19.4% 9.24 ± 3% sched_debug.cpu.cpu_load[4].avg 23.45 ± 31% -44.9% 12.92 ± 23% sched_debug.cpu.cpu_load[4].stddev 16690 ± 9% +65.9% 27695 ± 17% sched_debug.cpu.nr_switches.max 2915 ± 5% +33.1% 3880 ± 10% sched_debug.cpu.nr_switches.stddev 529.13 -17.1% 438.75 sched_debug.cpu.sched_count.min 2797 ± 17% +25.3% 3506 ± 12% sched_debug.cpu.sched_count.stddev 6712 ± 16% +82.3% 12239 ± 19% sched_debug.cpu.ttwu_count.max 1138 ± 6% +44.6% 1646 ± 12% sched_debug.cpu.ttwu_count.stddev 5979 ± 13% +100.8% 12005 ± 19% sched_debug.cpu.ttwu_local.max 1015 ± 7% +56.8% 1592 ± 12% sched_debug.cpu.ttwu_local.stddev 290462 +12.7% 327354 interrupts.CAL:Function_call_interrupts 2557 ± 12% +15.6% 2955 interrupts.CPU1.CAL:Function_call_interrupts 2609 ± 5% +27.6% 3329 ± 14% interrupts.CPU100.CAL:Function_call_interrupts 2708 ± 2% +22.5% 3318 ± 14% interrupts.CPU101.CAL:Function_call_interrupts 2703 ± 2% +16.5% 3149 ± 8% interrupts.CPU102.CAL:Function_call_interrupts 2727 +12.0% 3055 ± 5% interrupts.CPU23.CAL:Function_call_interrupts 66.50 ± 86% +595.9% 462.75 ± 24% interrupts.CPU30.RES:Rescheduling_interrupts 2529 ± 12% +17.0% 2959 ± 2% interrupts.CPU32.CAL:Function_call_interrupts 2676 ± 4% +11.8% 2992 ± 4% interrupts.CPU38.CAL:Function_call_interrupts 2478 ± 17% +20.6% 2989 ± 4% interrupts.CPU39.CAL:Function_call_interrupts 2708 +9.4% 2962 ± 3% interrupts.CPU42.CAL:Function_call_interrupts 2694 +19.2% 3212 ± 10% interrupts.CPU43.CAL:Function_call_interrupts 112.75 ± 58% +474.7% 648.00 ±105% interrupts.CPU44.RES:Rescheduling_interrupts 2691 +26.5% 3404 ± 10% interrupts.CPU45.CAL:Function_call_interrupts 2765 ± 2% +22.1% 3376 ± 11% interrupts.CPU46.CAL:Function_call_interrupts 41.00 ±107% +334.1% 178.00 ± 45% interrupts.CPU46.RES:Rescheduling_interrupts 2760 ± 3% +22.9% 3391 ± 11% interrupts.CPU47.CAL:Function_call_interrupts 2630 ± 4% +28.5% 3379 ± 11% interrupts.CPU48.CAL:Function_call_interrupts 2741 ± 3% +31.1% 3595 ± 10% interrupts.CPU49.CAL:Function_call_interrupts 2766 ± 2% +29.6% 3586 ± 10% interrupts.CPU50.CAL:Function_call_interrupts 2718 ± 2% +31.1% 3564 ± 10% interrupts.CPU52.CAL:Function_call_interrupts 2577 ± 10% +37.9% 3554 ± 10% interrupts.CPU53.CAL:Function_call_interrupts 2701 ± 2% +30.9% 3536 ± 10% interrupts.CPU54.CAL:Function_call_interrupts 2693 +30.9% 3525 ± 10% interrupts.CPU55.CAL:Function_call_interrupts 2688 +30.6% 3511 ± 10% interrupts.CPU56.CAL:Function_call_interrupts 2686 +30.0% 3492 ± 10% interrupts.CPU57.CAL:Function_call_interrupts 2681 +29.5% 3472 ± 10% interrupts.CPU58.CAL:Function_call_interrupts 2635 ± 5% +33.6% 3519 ± 8% interrupts.CPU59.CAL:Function_call_interrupts 2671 +28.9% 3442 ± 10% interrupts.CPU60.CAL:Function_call_interrupts 84.75 ± 45% -92.0% 6.75 ±114% interrupts.CPU60.RES:Rescheduling_interrupts 109.75 ± 35% +328.5% 470.25 ± 68% interrupts.CPU62.RES:Rescheduling_interrupts 2802 ± 5% +19.2% 3341 ± 12% interrupts.CPU78.CAL:Function_call_interrupts 2778 ± 4% +19.7% 3326 ± 12% interrupts.CPU79.CAL:Function_call_interrupts 2729 ± 4% +19.4% 3258 ± 13% interrupts.CPU80.CAL:Function_call_interrupts 2842 ± 6% +14.6% 3257 ± 12% interrupts.CPU83.CAL:Function_call_interrupts 156.75 ± 71% -79.6% 32.00 ± 28% interrupts.CPU89.RES:Rescheduling_interrupts 2842 ± 7% +19.8% 3406 ± 15% interrupts.CPU95.CAL:Function_call_interrupts 2752 ± 3% +22.1% 3362 ± 14% interrupts.CPU98.CAL:Function_call_interrupts 2731 ± 3% +22.4% 3344 ± 14% interrupts.CPU99.CAL:Function_call_interrupts 20946 +10.2% 23084 ± 2% interrupts.RES:Rescheduling_interrupts 36.50 -36.5 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault 38.11 -36.5 1.61 perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault 36.49 -36.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma 38.35 -36.5 1.89 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault 38.63 -36.4 2.27 perf-profile.calltrace.cycles-pp.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 35.55 -35.6 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu_free 35.54 -35.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages 33.07 -33.1 0.00 perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 32.63 -32.6 0.00 perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu_free.unmap_page_range 40.80 -14.8 25.95 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 40.80 -14.8 25.95 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 40.80 -14.8 25.95 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 40.79 -14.8 25.95 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 40.80 -14.8 25.97 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 40.80 -14.8 25.97 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.25 -13.1 23.17 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 36.26 -13.1 23.18 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 34.59 -13.0 21.55 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region 34.80 -13.0 21.81 perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region.__do_munmap 4.53 -1.8 2.74 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 4.53 -1.8 2.74 perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 4.49 -1.8 2.71 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region 4.51 -1.8 2.74 perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap 0.67 ± 2% +0.2 0.84 perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 0.87 +0.2 1.11 perf-profile.calltrace.cycles-pp.__list_del_entry_valid.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault 0.81 +0.3 1.07 perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.71 ± 3% +0.3 1.01 ± 2% perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault 1.01 ± 2% +0.4 1.39 ± 2% perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault 1.42 +0.4 1.82 perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 1.10 ± 2% +0.4 1.51 ± 2% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault 1.29 +0.5 1.74 perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 1.53 +1.1 2.64 perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 0.00 +2.6 2.56 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu 0.00 +2.6 2.56 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu 7.88 +4.5 12.36 perf-profile.calltrace.cycles-pp.copy_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 53.81 +13.3 67.07 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 54.17 +13.4 67.55 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 55.05 +13.6 68.67 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 55.29 +13.7 68.98 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 55.37 +13.7 69.08 perf-profile.calltrace.cycles-pp.page_fault 0.93 +19.4 20.35 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range 0.94 +19.4 20.37 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 1.64 +42.7 44.30 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte 1.69 +42.7 44.36 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 2.62 +43.1 45.68 perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault 2.72 +43.1 45.80 perf-profile.calltrace.cycles-pp.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 4.90 +44.4 49.32 perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 4.94 +44.4 49.38 perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 21.78 +6.1% 23.11 perf-stat.i.MPKI 8.253e+09 +4.8% 8.646e+09 perf-stat.i.branch-instructions 0.50 +0.0 0.52 perf-stat.i.branch-miss-rate% 40993691 +8.7% 44561957 perf-stat.i.branch-misses 84.65 +4.1 88.71 perf-stat.i.cache-miss-rate% 7.813e+08 +17.1% 9.148e+08 perf-stat.i.cache-misses 9.209e+08 +11.7% 1.029e+09 perf-stat.i.cache-references 1090 ± 2% +8.2% 1179 perf-stat.i.context-switches 6.81 -5.2% 6.46 perf-stat.i.cpi 33.80 -4.9% 32.13 perf-stat.i.cpu-migrations 379.66 -14.5% 324.63 perf-stat.i.cycles-between-cache-misses 0.05 ± 3% -0.0 0.02 ± 15% perf-stat.i.dTLB-load-miss-rate% 5304834 ± 3% -52.2% 2533610 ± 16% perf-stat.i.dTLB-load-misses 1.122e+10 +6.1% 1.19e+10 perf-stat.i.dTLB-loads 1.30 +0.1 1.43 perf-stat.i.dTLB-store-miss-rate% 76693587 +20.0% 92034882 perf-stat.i.dTLB-store-misses 5.821e+09 +8.9% 6.342e+09 perf-stat.i.dTLB-stores 18285337 +17.8% 21541708 perf-stat.i.iTLB-load-misses 4.221e+10 +5.3% 4.446e+10 perf-stat.i.instructions 2312 -10.5% 2068 perf-stat.i.instructions-per-iTLB-miss 0.15 +5.4% 0.15 perf-stat.i.ipc 9215819 +17.7% 10846431 perf-stat.i.minor-faults 0.26 ± 2% -0.0 0.25 ± 2% perf-stat.i.node-load-miss-rate% 224799 ± 2% +21.3% 272750 ± 4% perf-stat.i.node-load-misses 1.731e+08 +17.1% 2.027e+08 perf-stat.i.node-loads 1.36 -0.2 1.16 perf-stat.i.node-store-miss-rate% 58853544 ± 2% +16.6% 68638996 perf-stat.i.node-stores 9216162 +17.7% 10845914 perf-stat.i.page-faults 21.82 +6.1% 23.15 perf-stat.overall.MPKI 0.50 +0.0 0.52 perf-stat.overall.branch-miss-rate% 84.85 +4.0 88.89 perf-stat.overall.cache-miss-rate% 6.82 -5.2% 6.46 perf-stat.overall.cpi 368.18 -14.7% 314.00 perf-stat.overall.cycles-between-cache-misses 0.05 ± 3% -0.0 0.02 ± 15% perf-stat.overall.dTLB-load-miss-rate% 1.30 +0.1 1.43 perf-stat.overall.dTLB-store-miss-rate% 2307 -10.6% 2063 perf-stat.overall.instructions-per-iTLB-miss 0.15 +5.5% 0.15 perf-stat.overall.ipc 1.35 -0.2 1.15 perf-stat.overall.node-store-miss-rate% 1376816 -10.7% 1229718 perf-stat.overall.path-length 8.22e+09 +4.8% 8.616e+09 perf-stat.ps.branch-instructions 40754233 +9.0% 44406840 perf-stat.ps.branch-misses 7.782e+08 +17.1% 9.115e+08 perf-stat.ps.cache-misses 9.172e+08 +11.8% 1.025e+09 perf-stat.ps.cache-references 1086 ± 2% +8.2% 1175 perf-stat.ps.context-switches 33.60 -4.7% 32.02 perf-stat.ps.cpu-migrations 5283674 ± 3% -52.2% 2524786 ± 16% perf-stat.ps.dTLB-load-misses 1.117e+10 +6.1% 1.186e+10 perf-stat.ps.dTLB-loads 76366911 +20.1% 91709432 perf-stat.ps.dTLB-store-misses 5.797e+09 +9.0% 6.32e+09 perf-stat.ps.dTLB-stores 18211554 +17.9% 21465765 perf-stat.ps.iTLB-load-misses 4.203e+10 +5.4% 4.43e+10 perf-stat.ps.instructions 9181549 +17.7% 10808307 perf-stat.ps.minor-faults 223260 ± 2% +21.7% 271793 ± 4% perf-stat.ps.node-load-misses 1.724e+08 +17.1% 2.02e+08 perf-stat.ps.node-loads 58587000 ± 2% +16.7% 68394802 perf-stat.ps.node-stores 9179702 +17.7% 10807933 perf-stat.ps.page-faults 1.28e+13 +4.7% 1.34e+13 perf-stat.total.instructions *************************************************************************************************** lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ep2/lru-file-mmap-read-rand/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 599672 +26.4% 757815 vm-scalability.time.involuntary_context_switches 9.493e+08 -13.1% 8.253e+08 vm-scalability.time.major_page_faults 1941387 ± 3% -13.8% 1674108 ± 2% vm-scalability.time.maximum_resident_set_size 63137 +1.9% 64315 vm-scalability.time.minor_page_faults 6830 -7.5% 6317 vm-scalability.time.percent_of_cpu_this_job_got 25703 -7.6% 23758 vm-scalability.time.system_time 294.63 -11.6% 260.60 vm-scalability.time.user_time 5872 ± 3% +82.4% 10713 ± 5% vm-scalability.time.voluntary_context_switches 297652 ± 15% -37.8% 185078 ± 8% meminfo.DirectMap4k 0.02 ± 50% +0.1 0.09 ± 59% mpstat.cpu.soft% 2.383e+08 -13.5% 2.062e+08 turbostat.IRQ 32242 ± 4% +87.6% 60474 ± 22% cpuidle.POLL.time 15664 ± 3% +43.0% 22399 ± 13% cpuidle.POLL.usage 3899 +23.6% 4818 vmstat.system.cs 368843 -11.1% 327749 vmstat.system.in 4139555 ± 3% +11.1% 4600505 ± 2% numa-meminfo.node1.KReclaimable 4139555 ± 3% +11.1% 4600505 ± 2% numa-meminfo.node1.SReclaimable 4200095 ± 4% +11.1% 4666188 ± 2% numa-meminfo.node1.Slab 11501 ± 11% -17.0% 9549 ± 3% slabinfo.proc_inode_cache.active_objs 11689 ± 11% -16.7% 9736 ± 3% slabinfo.proc_inode_cache.num_objs 440.25 ± 7% +42.6% 628.00 ± 8% slabinfo.skbuff_ext_cache.active_objs 441.50 ± 7% +42.2% 628.00 ± 8% slabinfo.skbuff_ext_cache.num_objs 4.361e+08 -20.8% 3.453e+08 numa-numastat.node0.local_node 40925794 +97.2% 80707378 ± 4% numa-numastat.node0.numa_foreign 4.361e+08 -20.8% 3.453e+08 numa-numastat.node0.numa_hit 36068660 +83.3% 66101433 ± 5% numa-numastat.node0.numa_miss 36078718 +83.2% 66107412 ± 5% numa-numastat.node0.other_node 4.398e+08 -23.5% 3.366e+08 numa-numastat.node1.local_node 36068660 +83.3% 66101433 ± 5% numa-numastat.node1.numa_foreign 4.398e+08 -23.4% 3.367e+08 numa-numastat.node1.numa_hit 40925794 +97.2% 80707378 ± 4% numa-numastat.node1.numa_miss 40933151 +97.2% 80718641 ± 4% numa-numastat.node1.other_node 386.25 ± 2% -14.4% 330.50 ± 2% numa-vmstat.node0.nr_isolated_file 24417147 +104.1% 49841869 ± 5% numa-vmstat.node0.numa_foreign 2.846e+08 -20.4% 2.265e+08 numa-vmstat.node0.numa_hit 2.846e+08 -20.4% 2.265e+08 numa-vmstat.node0.numa_local 21765484 ± 2% +85.1% 40294529 ± 5% numa-vmstat.node0.numa_miss 21775855 ± 2% +85.1% 40301018 ± 5% numa-vmstat.node0.numa_other 125946 ± 3% -14.4% 107839 numa-vmstat.node0.workingset_activate 26951 ± 67% +5728.8% 1570940 ± 27% numa-vmstat.node0.workingset_nodereclaim 80648495 -27.0% 58886088 ± 2% numa-vmstat.node0.workingset_refault 1559 ± 3% -17.9% 1280 ± 5% numa-vmstat.node0.workingset_restore 1033201 ± 4% +11.2% 1149435 numa-vmstat.node1.nr_slab_reclaimable 21769821 ± 2% +85.1% 40302724 ± 5% numa-vmstat.node1.numa_foreign 2.872e+08 -23.2% 2.205e+08 ± 2% numa-vmstat.node1.numa_hit 2.871e+08 -23.3% 2.203e+08 ± 2% numa-vmstat.node1.numa_local 24422460 +104.1% 49851832 ± 5% numa-vmstat.node1.numa_miss 24565137 +103.5% 49998659 ± 5% numa-vmstat.node1.numa_other 127650 -14.3% 109442 ± 4% numa-vmstat.node1.workingset_activate 1153 ±173% +85832.3% 991443 ± 54% numa-vmstat.node1.workingset_nodereclaim 81857071 -27.6% 59287759 ± 2% numa-vmstat.node1.workingset_refault 1617 ± 4% -14.0% 1390 ± 10% numa-vmstat.node1.workingset_restore 259678 -17.0% 215573 proc-vmstat.allocstall_movable 1703 ± 16% +23.6% 2105 ± 10% proc-vmstat.allocstall_normal 1.395e+08 ± 11% -48.7% 71549757 ± 43% proc-vmstat.compact_daemon_free_scanned 1003933 ± 63% -63.2% 369627 ±132% proc-vmstat.compact_daemon_migrate_scanned 926.00 ± 6% -42.3% 534.75 ± 24% proc-vmstat.compact_daemon_wake 1.419e+08 ± 12% -48.5% 73046792 ± 41% proc-vmstat.compact_free_scanned 489837 ± 4% -72.0% 137242 ± 22% proc-vmstat.compact_isolated 1008467 ± 62% -62.8% 375359 ±130% proc-vmstat.compact_migrate_scanned 265.50 -62.1% 100.75 ± 5% proc-vmstat.kswapd_high_wmark_hit_quickly 3651 -51.5% 1771 proc-vmstat.kswapd_low_wmark_hit_quickly 397146 +6.0% 420930 proc-vmstat.nr_active_file 2852082 -1.6% 2805195 proc-vmstat.nr_dirty_background_threshold 5711140 -1.6% 5617251 proc-vmstat.nr_dirty_threshold 24648732 -1.2% 24347024 proc-vmstat.nr_file_pages 23980560 -1.4% 23654294 proc-vmstat.nr_inactive_file 771.75 -10.6% 690.25 proc-vmstat.nr_isolated_file 40463 -2.6% 39425 proc-vmstat.nr_shmem 2155487 +7.6% 2319389 proc-vmstat.nr_slab_reclaimable 397137 +6.0% 420896 proc-vmstat.nr_zone_active_file 23980512 -1.4% 23654248 proc-vmstat.nr_zone_inactive_file 76994455 +90.7% 1.468e+08 ± 4% proc-vmstat.numa_foreign 8.759e+08 -22.1% 6.82e+08 proc-vmstat.numa_hit 8.759e+08 -22.1% 6.82e+08 proc-vmstat.numa_local 76994455 +90.7% 1.468e+08 ± 4% proc-vmstat.numa_miss 77011876 +90.7% 1.468e+08 ± 4% proc-vmstat.numa_other 3937 -50.8% 1938 proc-vmstat.pageoutrun 11995297 -14.2% 10293574 proc-vmstat.pgactivate 12961188 -11.6% 11454246 proc-vmstat.pgalloc_dma32 9.421e+08 -13.0% 8.196e+08 proc-vmstat.pgalloc_normal 11871698 -15.3% 10059124 proc-vmstat.pgdeactivate 9.503e+08 -13.1% 8.263e+08 proc-vmstat.pgfault 9.552e+08 -13.0% 8.309e+08 proc-vmstat.pgfree 9.493e+08 -13.1% 8.253e+08 proc-vmstat.pgmajfault 247541 ± 5% -70.5% 73057 ± 18% proc-vmstat.pgmigrate_success 11871698 -15.3% 10059124 proc-vmstat.pgrefill 1.799e+09 -15.1% 1.527e+09 proc-vmstat.pgscan_direct 62913759 +48.0% 93129545 proc-vmstat.pgscan_kswapd 9.006e+08 -15.3% 7.626e+08 proc-vmstat.pgsteal_direct 20365558 +72.4% 35105261 proc-vmstat.pgsteal_kswapd 408633 ± 47% +2489.4% 10581295 ± 20% proc-vmstat.slabs_scanned 253093 -14.2% 217227 ± 2% proc-vmstat.workingset_activate 28085 ± 59% +9037.1% 2566212 ± 31% proc-vmstat.workingset_nodereclaim 1.617e+08 -27.1% 1.179e+08 ± 2% proc-vmstat.workingset_refault 3171 -15.8% 2670 ± 4% proc-vmstat.workingset_restore 1.326e+10 -3.2% 1.284e+10 perf-stat.i.branch-instructions 79973042 -13.4% 69292984 ± 2% perf-stat.i.branch-misses 25.79 +1.0 26.75 perf-stat.i.cache-miss-rate% 1.035e+08 -7.8% 95404454 perf-stat.i.cache-misses 3.693e+08 -11.2% 3.277e+08 perf-stat.i.cache-references 3847 +24.0% 4773 perf-stat.i.context-switches 3.02 +5.3% 3.19 perf-stat.i.cpi 65.77 +70.1% 111.90 perf-stat.i.cpu-migrations 43393953 -11.8% 38256273 perf-stat.i.dTLB-load-misses 1.494e+10 -4.2% 1.431e+10 perf-stat.i.dTLB-loads 1610742 ± 2% -6.6% 1505184 ± 3% perf-stat.i.dTLB-store-misses 4.72e+09 -12.4% 4.136e+09 perf-stat.i.dTLB-stores 6848588 -11.2% 6080133 perf-stat.i.iTLB-load-misses 5.853e+10 -4.1% 5.613e+10 perf-stat.i.instructions 19159 ± 6% +15.5% 22138 ± 4% perf-stat.i.instructions-per-iTLB-miss 0.36 -7.4% 0.33 ± 2% perf-stat.i.ipc 2503190 -13.2% 2172350 perf-stat.i.major-faults 2524 -1.6% 2484 perf-stat.i.minor-faults 24607743 ± 3% -13.2% 21353885 ± 3% perf-stat.i.node-loads 8995692 +4.0% 9352683 perf-stat.i.node-store-misses 11103789 -16.4% 9282543 perf-stat.i.node-stores 2505862 -13.2% 2175090 perf-stat.i.page-faults 6.27 -7.4% 5.81 perf-stat.overall.MPKI 0.60 -0.1 0.54 perf-stat.overall.branch-miss-rate% 28.09 +1.1 29.17 perf-stat.overall.cache-miss-rate% 3.32 +4.1% 3.46 perf-stat.overall.cpi 1886 +8.2% 2041 perf-stat.overall.cycles-between-cache-misses 0.29 -0.0 0.27 perf-stat.overall.dTLB-load-miss-rate% 0.03 ± 2% +0.0 0.04 ± 2% perf-stat.overall.dTLB-store-miss-rate% 8615 +7.9% 9296 perf-stat.overall.instructions-per-iTLB-miss 0.30 -3.9% 0.29 perf-stat.overall.ipc 44.84 +5.4 50.24 perf-stat.overall.node-store-miss-rate% 41062 -3.9% 39454 perf-stat.overall.path-length 1.33e+10 -3.2% 1.287e+10 perf-stat.ps.branch-instructions 79535002 -13.3% 68984260 ± 2% perf-stat.ps.branch-misses 1.033e+08 -7.8% 95308434 perf-stat.ps.cache-misses 3.679e+08 -11.2% 3.268e+08 perf-stat.ps.cache-references 3877 +23.7% 4795 perf-stat.ps.context-switches 66.49 +69.1% 112.41 perf-stat.ps.cpu-migrations 43270179 -11.7% 38189049 perf-stat.ps.dTLB-load-misses 1.498e+10 -4.2% 1.435e+10 perf-stat.ps.dTLB-loads 1599540 ± 2% -6.5% 1495678 ± 2% perf-stat.ps.dTLB-store-misses 4.706e+09 -12.3% 4.127e+09 perf-stat.ps.dTLB-stores 6811861 -11.1% 6055300 perf-stat.ps.iTLB-load-misses 5.868e+10 -4.1% 5.627e+10 perf-stat.ps.instructions 2491568 -13.1% 2165778 perf-stat.ps.major-faults 2509 -1.8% 2465 perf-stat.ps.minor-faults 24532615 ± 3% -13.1% 21312065 ± 3% perf-stat.ps.node-loads 9017673 +3.9% 9370738 perf-stat.ps.node-store-misses 11093045 -16.4% 9279099 perf-stat.ps.node-stores 2494078 -13.1% 2168244 perf-stat.ps.page-faults 2.236e+13 -4.1% 2.144e+13 perf-stat.total.instructions 929.79 ± 20% +47.6% 1372 ± 5% sched_debug.cfs_rq:/.exec_clock.stddev 18377 ± 24% +39.8% 25688 ± 10% sched_debug.cfs_rq:/.load.avg 446807 ± 35% +67.0% 746350 ± 17% sched_debug.cfs_rq:/.load.max 62843 ± 41% +73.1% 108767 ± 15% sched_debug.cfs_rq:/.load.stddev 28.08 ± 21% +163.7% 74.05 ± 4% sched_debug.cfs_rq:/.load_avg.avg 606.86 ± 22% +55.1% 941.07 ± 7% sched_debug.cfs_rq:/.load_avg.max 7.68 ± 5% +45.1% 11.14 ± 13% sched_debug.cfs_rq:/.load_avg.min 90.76 ± 25% +49.1% 135.32 ± 3% sched_debug.cfs_rq:/.load_avg.stddev 11623369 +10.9% 12887579 ± 2% sched_debug.cfs_rq:/.min_vruntime.min 699993 ± 5% -35.4% 452471 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 16.40 ± 5% +903.8% 164.60 ± 5% sched_debug.cfs_rq:/.nr_spread_over.avg 270.96 ± 9% +13.7% 307.96 ± 4% sched_debug.cfs_rq:/.nr_spread_over.max 43.70 ± 5% +80.6% 78.93 ± 5% sched_debug.cfs_rq:/.nr_spread_over.stddev 14.91 ± 22% +52.3% 22.71 ± 10% sched_debug.cfs_rq:/.runnable_load_avg.avg 321.79 ± 45% +121.7% 713.50 ± 16% sched_debug.cfs_rq:/.runnable_load_avg.max 7.29 ± 9% -27.5% 5.29 ± 14% sched_debug.cfs_rq:/.runnable_load_avg.min 44.73 ± 47% +118.8% 97.86 ± 16% sched_debug.cfs_rq:/.runnable_load_avg.stddev 443828 ± 35% +67.8% 744624 ± 17% sched_debug.cfs_rq:/.runnable_weight.max 62573 ± 41% +71.1% 107041 ± 17% sched_debug.cfs_rq:/.runnable_weight.stddev -3062796 -59.5% -1239265 sched_debug.cfs_rq:/.spread0.min 699681 ± 5% -35.2% 453182 ± 8% sched_debug.cfs_rq:/.spread0.stddev 512.75 ± 9% -25.0% 384.64 ± 16% sched_debug.cfs_rq:/.util_avg.min 95.31 ± 18% +37.6% 131.15 ± 15% sched_debug.cfs_rq:/.util_avg.stddev 667.14 -11.2% 592.33 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.avg 255.50 ± 44% -70.4% 75.61 ± 73% sched_debug.cfs_rq:/.util_est_enqueued.min 149.63 ± 18% +38.3% 206.87 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.stddev 151510 ± 27% +62.3% 245870 ± 14% sched_debug.cpu.avg_idle.min 212572 ± 4% -21.2% 167485 ± 2% sched_debug.cpu.avg_idle.stddev 14.95 ± 22% +52.4% 22.79 ± 11% sched_debug.cpu.cpu_load[0].avg 322.14 ± 45% +121.5% 713.46 ± 16% sched_debug.cpu.cpu_load[0].max 7.29 ± 9% -33.3% 4.86 ± 18% sched_debug.cpu.cpu_load[0].min 44.82 ± 47% +118.4% 97.88 ± 16% sched_debug.cpu.cpu_load[0].stddev 14.82 ± 22% +54.2% 22.85 ± 11% sched_debug.cpu.cpu_load[1].avg 316.04 ± 47% +127.1% 717.64 ± 16% sched_debug.cpu.cpu_load[1].max 7.29 ± 9% -28.4% 5.21 ± 19% sched_debug.cpu.cpu_load[1].min 43.58 ± 48% +124.6% 97.87 ± 16% sched_debug.cpu.cpu_load[1].stddev 14.80 ± 22% +56.0% 23.09 ± 11% sched_debug.cpu.cpu_load[2].avg 317.89 ± 48% +131.0% 734.46 ± 15% sched_debug.cpu.cpu_load[2].max 7.32 ± 9% -28.8% 5.21 ± 19% sched_debug.cpu.cpu_load[2].min 43.04 ± 49% +129.0% 98.58 ± 15% sched_debug.cpu.cpu_load[2].stddev 15.00 ± 21% +54.6% 23.19 ± 11% sched_debug.cpu.cpu_load[3].avg 340.79 ± 45% +121.1% 753.64 ± 14% sched_debug.cpu.cpu_load[3].max 7.32 ± 9% -27.3% 5.32 ± 15% sched_debug.cpu.cpu_load[3].min 44.72 ± 47% +121.5% 99.06 ± 16% sched_debug.cpu.cpu_load[3].stddev 15.47 ± 20% +50.5% 23.29 ± 10% sched_debug.cpu.cpu_load[4].avg 394.32 ± 38% +101.0% 792.75 ± 13% sched_debug.cpu.cpu_load[4].max 7.32 ± 9% -25.9% 5.43 ± 13% sched_debug.cpu.cpu_load[4].min 49.87 ± 42% +103.7% 101.59 ± 15% sched_debug.cpu.cpu_load[4].stddev 18811 ± 26% +36.6% 25696 ± 10% sched_debug.cpu.load.avg 446845 ± 35% +67.0% 746337 ± 17% sched_debug.cpu.load.max 64036 ± 43% +69.2% 108378 ± 16% sched_debug.cpu.load.stddev 1212 ± 7% +101.9% 2447 ± 3% sched_debug.cpu.nr_load_updates.stddev 0.22 ± 6% +13.8% 0.25 ± 3% sched_debug.cpu.nr_running.stddev 10418 +17.7% 12262 sched_debug.cpu.nr_switches.avg 7182 ± 2% +33.8% 9612 ± 3% sched_debug.cpu.nr_switches.min 2956 ± 5% -20.7% 2343 ± 2% sched_debug.cpu.nr_switches.stddev 7.29 ± 9% +75.5% 12.79 ± 17% sched_debug.cpu.nr_uninterruptible.max -8.04 +48.0% -11.89 sched_debug.cpu.nr_uninterruptible.min 2.86 ± 4% +51.9% 4.34 ± 8% sched_debug.cpu.nr_uninterruptible.stddev 10942 ± 3% +20.8% 13213 ± 4% sched_debug.cpu.sched_count.avg 55375 ± 23% +62.7% 90098 ± 42% sched_debug.cpu.sched_count.max 7032 +34.0% 9422 ± 2% sched_debug.cpu.sched_count.min 183.57 ± 3% +51.7% 278.51 ± 4% sched_debug.cpu.sched_goidle.avg 2867 ± 11% +22.2% 3503 ± 7% sched_debug.cpu.sched_goidle.max 378.52 ± 8% +22.1% 462.04 ± 5% sched_debug.cpu.sched_goidle.stddev 4635 +17.4% 5442 sched_debug.cpu.ttwu_count.avg 3006 ± 4% +23.1% 3701 ± 6% sched_debug.cpu.ttwu_count.min 1185 ± 6% -12.0% 1043 ± 4% sched_debug.cpu.ttwu_count.stddev 984.72 ± 7% -30.6% 682.92 ± 4% sched_debug.cpu.ttwu_local.stddev 65.59 -7.1 58.53 perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 65.11 -7.0 58.14 perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages 65.61 -6.5 59.07 perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault 65.62 -6.5 59.07 perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault 65.61 -6.4 59.23 ± 2% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 66.82 -6.3 60.52 perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault.__do_fault 43.86 -5.8 38.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 68.00 -5.8 62.24 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 44.61 -5.7 38.93 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node 96.79 -5.1 91.71 perf-profile.calltrace.cycles-pp.page_fault 96.76 -5.1 91.68 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 96.59 -5.1 91.53 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 96.04 -5.0 91.03 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 95.87 -5.0 90.87 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 90.77 -4.4 86.34 perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 90.67 -4.4 86.25 perf-profile.calltrace.cycles-pp.__xfs_filemap_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 90.46 -4.4 86.06 perf-profile.calltrace.cycles-pp.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault.handle_mm_fault 6.84 ± 5% -2.2 4.63 ± 5% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 3.47 ± 9% -2.0 1.48 ± 19% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 3.52 ± 9% -2.0 1.54 ± 18% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 19.64 ± 3% -1.2 18.48 perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 4.03 ± 2% -0.5 3.55 ± 2% perf-profile.calltrace.cycles-pp.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 2.75 ± 2% -0.3 2.46 ± 2% perf-profile.calltrace.cycles-pp.xas_find.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.55 ± 2% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.pagecache_get_page.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 2.46 ± 2% -0.3 2.18 ± 2% perf-profile.calltrace.cycles-pp.xas_load.xas_find.filemap_map_pages.__handle_mm_fault.handle_mm_fault 3.03 ± 2% -0.2 2.80 perf-profile.calltrace.cycles-pp.page_referenced.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 2.75 ± 2% -0.2 2.53 perf-profile.calltrace.cycles-pp.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.62 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.list_lru_del.workingset_update_node.xas_store.__add_to_page_cache_locked.add_to_page_cache_lru 2.84 ± 2% -0.2 2.64 perf-profile.calltrace.cycles-pp.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 4.91 -0.2 4.73 perf-profile.calltrace.cycles-pp.iomap_readpage.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 0.56 ± 2% -0.2 0.39 ± 57% perf-profile.calltrace.cycles-pp.xfs_file_iomap_begin.iomap_apply.iomap_readpage.filemap_fault.__xfs_filemap_fault 2.18 ± 2% -0.2 2.01 perf-profile.calltrace.cycles-pp.page_referenced_one.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list 4.84 -0.2 4.67 perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpage.filemap_fault.__xfs_filemap_fault.__do_fault 2.30 ± 2% -0.2 2.14 perf-profile.calltrace.cycles-pp.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list 1.87 ± 2% -0.2 1.71 ± 2% perf-profile.calltrace.cycles-pp.page_vma_mapped_walk.page_referenced_one.rmap_walk_file.page_referenced.shrink_page_list 1.91 ± 2% -0.2 1.76 ± 3% perf-profile.calltrace.cycles-pp.xas_create.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list 3.02 -0.1 2.90 ± 2% perf-profile.calltrace.cycles-pp.memset_erms.iomap_readpage_actor.iomap_apply.iomap_readpage.filemap_fault 1.40 ± 2% -0.1 1.29 ± 2% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.filemap_fault.__xfs_filemap_fault.__do_fault 0.70 ± 2% -0.1 0.59 ± 4% perf-profile.calltrace.cycles-pp.workingset_update_node.xas_store.__add_to_page_cache_locked.add_to_page_cache_lru.filemap_fault 0.92 -0.1 0.84 ± 2% perf-profile.calltrace.cycles-pp.isolate_lru_pages.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 1.08 -0.1 1.00 ± 3% perf-profile.calltrace.cycles-pp.rmap_walk_file.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg 1.18 ± 2% -0.1 1.10 ± 3% perf-profile.calltrace.cycles-pp.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.81 ± 2% -0.1 0.74 ± 5% perf-profile.calltrace.cycles-pp.xas_store.__add_to_page_cache_locked.add_to_page_cache_lru.filemap_fault.__xfs_filemap_fault 0.71 -0.1 0.64 perf-profile.calltrace.cycles-pp.__isolate_lru_page.isolate_lru_pages.shrink_inactive_list.shrink_node_memcg.shrink_node 0.69 -0.1 0.63 ± 2% perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 0.66 -0.1 0.59 ± 2% perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.76 -0.1 0.71 ± 3% perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_file.try_to_unmap.shrink_page_list.shrink_inactive_list 0.77 ± 9% +0.1 0.92 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.31 ± 8% +0.5 1.83 ± 3% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 1.30 ± 9% +0.5 1.82 ± 3% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 1.31 ± 9% +0.5 1.83 ± 3% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 1.31 ± 9% +0.5 1.83 ± 3% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 1.31 ± 9% +0.5 1.83 ± 3% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 1.05 ± 3% +0.5 1.59 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault.__do_fault 0.26 ±100% +0.6 0.83 ± 3% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 0.14 ±173% +0.8 0.93 ± 9% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault 6.58 ± 5% +1.0 7.54 ± 3% perf-profile.calltrace.cycles-pp.smp_call_function_many.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 0.00 +1.0 0.97 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.filemap_fault 0.00 +1.0 0.97 ± 12% perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.filemap_fault.__xfs_filemap_fault 0.00 +1.4 1.41 ± 5% perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush 16.51 +1.7 18.16 ± 3% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.filemap_fault.__xfs_filemap_fault.__do_fault.__handle_mm_fault 14.81 +1.8 16.63 ± 4% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.filemap_fault.__xfs_filemap_fault.__do_fault 14.76 +1.8 16.59 ± 4% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.filemap_fault.__xfs_filemap_fault 14.08 +1.9 15.95 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.filemap_fault 14.06 +1.9 15.93 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 6.92 ± 5% +2.1 9.02 ± 3% perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg 6.90 ± 5% +2.1 9.00 ± 3% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list.shrink_inactive_list 6.92 ± 5% +2.1 9.02 ± 3% perf-profile.calltrace.cycles-pp.try_to_unmap_flush.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 6.85 ± 5% +2.1 8.96 ± 3% perf-profile.calltrace.cycles-pp.on_each_cpu_mask.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush.shrink_page_list 0.00 +3.7 3.68 ± 13% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages 0.00 +3.7 3.68 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq 0.00 +4.1 4.05 ± 12% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work 0.00 +4.1 4.06 ± 12% perf-profile.calltrace.cycles-pp.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work.worker_thread 0.00 +4.1 4.06 ± 12% perf-profile.calltrace.cycles-pp.drain_pages.drain_local_pages_wq.process_one_work.worker_thread.kthread 0.00 +4.1 4.07 ± 13% perf-profile.calltrace.cycles-pp.drain_local_pages_wq.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +4.1 4.09 ± 12% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +4.1 4.09 ± 12% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 1.51 ± 9% +4.5 5.96 ± 8% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 1.51 ± 9% +4.5 5.97 ± 9% perf-profile.calltrace.cycles-pp.ret_from_fork 155671 ± 5% -11.7% 137495 ± 3% softirqs.CPU0.TIMER 154077 ± 4% -11.5% 136361 ± 4% softirqs.CPU1.TIMER 152648 ± 4% -10.5% 136619 ± 3% softirqs.CPU10.TIMER 151902 ± 4% -10.8% 135510 ± 4% softirqs.CPU11.TIMER 11486 ± 7% +29.5% 14872 ± 6% softirqs.CPU12.RCU 152401 ± 4% -11.2% 135344 ± 4% softirqs.CPU12.TIMER 152403 ± 4% -11.2% 135336 ± 3% softirqs.CPU13.TIMER 11806 ± 5% +19.8% 14144 ± 6% softirqs.CPU14.RCU 152583 ± 5% -10.2% 136962 ± 3% softirqs.CPU14.TIMER 152875 ± 4% -9.6% 138132 ± 2% softirqs.CPU15.TIMER 152915 ± 4% -10.8% 136372 ± 4% softirqs.CPU16.TIMER 6783 ± 7% +48.6% 10081 ± 26% softirqs.CPU17.RCU 153166 ± 3% -12.0% 134842 ± 4% softirqs.CPU17.TIMER 151910 ± 4% -10.6% 135808 ± 3% softirqs.CPU18.TIMER 151219 ± 5% -10.2% 135744 ± 3% softirqs.CPU19.TIMER 11904 ± 12% +25.6% 14957 ± 7% softirqs.CPU2.RCU 152357 ± 4% -11.4% 134977 ± 4% softirqs.CPU2.TIMER 151803 ± 4% -9.1% 138011 ± 3% softirqs.CPU20.TIMER 10529 ± 12% +53.2% 16130 ± 15% softirqs.CPU22.RCU 9554 ± 16% +49.1% 14242 ± 6% softirqs.CPU23.RCU 8991 ± 11% +43.7% 12921 ± 10% softirqs.CPU24.RCU 8581 ± 18% +46.7% 12589 ± 10% softirqs.CPU25.RCU 10096 ± 18% +41.7% 14304 ± 5% softirqs.CPU26.RCU 8705 ± 13% +60.8% 14000 ± 10% softirqs.CPU27.RCU 9425 ± 19% +44.2% 13592 ± 10% softirqs.CPU28.RCU 10570 ± 21% +41.1% 14913 ± 8% softirqs.CPU29.RCU 10921 ± 8% +50.1% 16396 ± 6% softirqs.CPU30.RCU 9282 ± 15% +41.7% 13152 ± 15% softirqs.CPU32.RCU 10728 ± 17% +25.6% 13480 ± 3% softirqs.CPU33.RCU 11399 ± 12% +34.9% 15373 ± 7% softirqs.CPU34.RCU 11611 ± 6% +23.1% 14294 ± 2% softirqs.CPU35.RCU 13037 ± 9% +27.9% 16671 ± 6% softirqs.CPU36.RCU 11950 ± 8% +27.2% 15205 ± 5% softirqs.CPU37.RCU 11609 ± 10% +39.0% 16134 ± 8% softirqs.CPU38.RCU 10335 ± 3% +25.9% 13017 ± 7% softirqs.CPU39.RCU 152751 ± 4% -10.9% 136147 ± 3% softirqs.CPU4.TIMER 11506 ± 7% +32.9% 15294 ± 8% softirqs.CPU41.RCU 11626 ± 6% +22.8% 14281 ± 7% softirqs.CPU43.RCU 6881 ± 12% +44.9% 9972 ± 13% softirqs.CPU44.RCU 152522 ± 3% -10.6% 136320 ± 3% softirqs.CPU44.TIMER 12534 ± 6% +14.9% 14402 ± 6% softirqs.CPU45.RCU 153201 ± 3% -11.8% 135087 ± 4% softirqs.CPU45.TIMER 14354 ± 16% +35.0% 19374 ± 7% softirqs.CPU46.RCU 154910 ± 4% -10.4% 138813 ± 4% softirqs.CPU46.TIMER 11165 ± 7% +24.4% 13891 ± 11% softirqs.CPU47.RCU 152356 ± 4% -10.6% 136280 ± 4% softirqs.CPU47.TIMER 153077 ± 4% -11.8% 134992 ± 3% softirqs.CPU48.TIMER 10702 ± 28% +35.8% 14534 ± 5% softirqs.CPU49.RCU 151230 ± 4% -10.8% 134830 ± 4% softirqs.CPU49.TIMER 12009 ± 7% +27.6% 15325 ± 10% softirqs.CPU5.RCU 153765 ± 3% -11.4% 136214 ± 4% softirqs.CPU5.TIMER 152447 ± 4% -11.6% 134691 ± 4% softirqs.CPU50.TIMER 11093 ± 6% +16.6% 12938 ± 6% softirqs.CPU51.RCU 9334 ± 6% +51.0% 14095 ± 36% softirqs.CPU52.RCU 152581 ± 4% -12.3% 133771 ± 4% softirqs.CPU52.TIMER 152311 ± 4% -11.9% 134237 ± 4% softirqs.CPU53.TIMER 10680 ± 13% +30.4% 13931 ± 11% softirqs.CPU54.RCU 152169 ± 4% -11.3% 134951 ± 4% softirqs.CPU54.TIMER 152817 ± 4% -12.0% 134437 ± 4% softirqs.CPU55.TIMER 152234 ± 4% -11.6% 134623 ± 5% softirqs.CPU56.TIMER 153166 ± 6% -12.5% 134085 ± 4% softirqs.CPU57.TIMER 152402 ± 4% -10.8% 135949 ± 3% softirqs.CPU58.TIMER 152546 ± 4% -10.4% 136653 ± 2% softirqs.CPU59.TIMER 11480 ± 12% +34.1% 15394 ± 9% softirqs.CPU6.RCU 152298 ± 4% -10.7% 135977 ± 4% softirqs.CPU6.TIMER 152458 ± 4% -11.2% 135415 ± 4% softirqs.CPU60.TIMER 7258 ± 9% +28.5% 9323 ± 17% softirqs.CPU61.RCU 167708 ± 12% -20.4% 133522 ± 4% softirqs.CPU61.TIMER 151841 ± 4% -11.7% 134027 ± 4% softirqs.CPU62.TIMER 152312 ± 3% -11.8% 134343 ± 4% softirqs.CPU63.TIMER 151842 ± 4% -10.1% 136553 ± 3% softirqs.CPU64.TIMER 10599 ± 9% +45.5% 15425 ± 11% softirqs.CPU66.RCU 9173 ± 8% +56.6% 14365 ± 12% softirqs.CPU67.RCU 9368 ± 11% +39.4% 13059 ± 10% softirqs.CPU68.RCU 152658 ± 4% -9.4% 138300 ± 4% softirqs.CPU7.TIMER 10194 ± 9% +37.9% 14057 ± 13% softirqs.CPU70.RCU 9442 ± 9% +52.0% 14348 ± 10% softirqs.CPU71.RCU 10062 ± 20% +39.7% 14055 ± 8% softirqs.CPU72.RCU 11783 ± 19% +33.3% 15711 ± 10% softirqs.CPU73.RCU 9484 ± 20% +65.6% 15706 ± 12% softirqs.CPU74.RCU 8613 ± 20% +43.4% 12351 ± 18% softirqs.CPU76.RCU 10272 ± 15% +32.2% 13585 ± 11% softirqs.CPU77.RCU 10110 ± 9% +42.3% 14384 ± 8% softirqs.CPU79.RCU 11621 ± 11% +34.1% 15580 ± 9% softirqs.CPU81.RCU 10415 ± 15% +46.7% 15283 ± 6% softirqs.CPU82.RCU 9177 ± 6% +33.0% 12201 ± 8% softirqs.CPU83.RCU 10952 ± 8% +37.9% 15107 ± 6% softirqs.CPU84.RCU 11378 ± 12% +37.5% 15643 ± 5% softirqs.CPU85.RCU 12808 +14.9% 14720 ± 14% softirqs.CPU86.SCHED 11598 ± 8% +27.1% 14742 ± 6% softirqs.CPU87.RCU 12843 +7.4% 13799 ± 4% softirqs.CPU87.SCHED 173430 ± 12% -12.4% 151901 ± 10% softirqs.CPU87.TIMER 151928 ± 5% -10.8% 135532 ± 4% softirqs.CPU9.TIMER 938265 ± 5% +29.5% 1214844 ± 4% softirqs.RCU 73887112 -16.6% 61613151 interrupts.CAL:Function_call_interrupts 814769 ± 3% -12.3% 714647 ± 5% interrupts.CPU0.CAL:Function_call_interrupts 2697 ± 18% +99.6% 5384 ± 11% interrupts.CPU0.RES:Rescheduling_interrupts 1070879 ± 4% -12.8% 933917 ± 5% interrupts.CPU0.TLB:TLB_shootdowns 847771 -14.5% 724918 interrupts.CPU1.CAL:Function_call_interrupts 1822 ± 31% +92.3% 3503 ± 3% interrupts.CPU1.RES:Rescheduling_interrupts 1118594 -15.2% 948568 interrupts.CPU1.TLB:TLB_shootdowns 833687 ± 2% -15.7% 702869 ± 2% interrupts.CPU10.CAL:Function_call_interrupts 1611 ± 7% +129.9% 3706 ± 9% interrupts.CPU10.RES:Rescheduling_interrupts 1097722 ± 3% -16.3% 918470 ± 2% interrupts.CPU10.TLB:TLB_shootdowns 852751 -15.5% 720949 ± 2% interrupts.CPU11.CAL:Function_call_interrupts 1446 ± 8% +147.1% 3574 ± 9% interrupts.CPU11.RES:Rescheduling_interrupts 1124535 -15.8% 946324 ± 2% interrupts.CPU11.TLB:TLB_shootdowns 843017 -15.2% 715291 ± 2% interrupts.CPU12.CAL:Function_call_interrupts 1488 ± 18% +127.0% 3378 ± 3% interrupts.CPU12.RES:Rescheduling_interrupts 1110870 -15.7% 936512 ± 2% interrupts.CPU12.TLB:TLB_shootdowns 856521 -15.1% 727148 interrupts.CPU13.CAL:Function_call_interrupts 1352 +151.6% 3402 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 1128380 -15.5% 953155 interrupts.CPU13.TLB:TLB_shootdowns 1561 ± 13% +117.3% 3392 ± 6% interrupts.CPU14.RES:Rescheduling_interrupts 841159 ± 2% -11.8% 742218 interrupts.CPU15.CAL:Function_call_interrupts 1351 ± 6% +155.2% 3449 ± 3% interrupts.CPU15.RES:Rescheduling_interrupts 1105766 ± 2% -12.0% 973294 interrupts.CPU15.TLB:TLB_shootdowns 1823 ± 26% +86.7% 3404 ± 7% interrupts.CPU16.RES:Rescheduling_interrupts 849360 -15.7% 716062 interrupts.CPU17.CAL:Function_call_interrupts 1361 ± 8% +145.4% 3342 ± 6% interrupts.CPU17.RES:Rescheduling_interrupts 1119722 -15.8% 942463 interrupts.CPU17.TLB:TLB_shootdowns 850664 -14.1% 730553 interrupts.CPU18.CAL:Function_call_interrupts 1501 ± 20% +121.7% 3328 ± 5% interrupts.CPU18.RES:Rescheduling_interrupts 1122380 -14.6% 958694 interrupts.CPU18.TLB:TLB_shootdowns 846626 -17.6% 697647 ± 6% interrupts.CPU19.CAL:Function_call_interrupts 1309 ± 7% +163.6% 3452 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 1116712 -18.1% 914537 ± 5% interrupts.CPU19.TLB:TLB_shootdowns 846242 -13.5% 731891 interrupts.CPU2.CAL:Function_call_interrupts 1498 ± 11% +128.1% 3419 ± 5% interrupts.CPU2.RES:Rescheduling_interrupts 1114823 -14.0% 958269 interrupts.CPU2.TLB:TLB_shootdowns 852703 -14.9% 725407 ± 2% interrupts.CPU20.CAL:Function_call_interrupts 1389 ± 4% +137.7% 3302 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts 1122749 -15.0% 953892 ± 2% interrupts.CPU20.TLB:TLB_shootdowns 843090 -12.2% 740592 interrupts.CPU21.CAL:Function_call_interrupts 1349 ± 5% +140.3% 3242 ± 5% interrupts.CPU21.RES:Rescheduling_interrupts 1111508 -12.8% 969069 interrupts.CPU21.TLB:TLB_shootdowns 806231 ± 3% -15.0% 685510 ± 2% interrupts.CPU22.CAL:Function_call_interrupts 1677 ± 16% +113.7% 3583 ± 7% interrupts.CPU22.RES:Rescheduling_interrupts 1064241 ± 4% -15.6% 897928 interrupts.CPU22.TLB:TLB_shootdowns 840447 -15.9% 706685 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 1302 ± 7% +157.5% 3352 ± 3% interrupts.CPU23.RES:Rescheduling_interrupts 1111586 -16.9% 924142 ± 2% interrupts.CPU23.TLB:TLB_shootdowns 843809 -17.9% 692783 interrupts.CPU24.CAL:Function_call_interrupts 1395 ± 2% +940.7% 14526 ±132% interrupts.CPU24.RES:Rescheduling_interrupts 1114169 -18.5% 907919 interrupts.CPU24.TLB:TLB_shootdowns 804801 ± 5% -12.7% 702621 interrupts.CPU25.CAL:Function_call_interrupts 1574 ± 16% +155.0% 4013 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts 1059130 ± 6% -13.1% 920817 interrupts.CPU25.TLB:TLB_shootdowns 835997 ± 2% -18.6% 680154 ± 3% interrupts.CPU26.CAL:Function_call_interrupts 1405 ± 7% +148.4% 3491 ± 3% interrupts.CPU26.RES:Rescheduling_interrupts 1101446 ± 2% -19.3% 888887 ± 3% interrupts.CPU26.TLB:TLB_shootdowns 858650 -17.5% 708338 interrupts.CPU27.CAL:Function_call_interrupts 1310 ± 5% +150.3% 3280 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 1134471 -18.2% 927489 interrupts.CPU27.TLB:TLB_shootdowns 827312 ± 2% -13.8% 713291 interrupts.CPU28.CAL:Function_call_interrupts 1421 ± 4% +136.0% 3355 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 1088848 ± 2% -14.1% 934828 interrupts.CPU28.TLB:TLB_shootdowns 849813 -17.9% 697621 ± 2% interrupts.CPU29.CAL:Function_call_interrupts 1408 ± 11% +140.7% 3390 ± 4% interrupts.CPU29.RES:Rescheduling_interrupts 1121472 -18.5% 914044 ± 2% interrupts.CPU29.TLB:TLB_shootdowns 848617 -13.2% 736476 ± 2% interrupts.CPU3.CAL:Function_call_interrupts 1391 ± 7% +153.7% 3529 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts 1119764 -13.9% 964214 interrupts.CPU3.TLB:TLB_shootdowns 848520 -15.4% 717746 interrupts.CPU30.CAL:Function_call_interrupts 1351 ± 3% +171.7% 3670 ± 14% interrupts.CPU30.RES:Rescheduling_interrupts 1120814 -16.0% 941613 interrupts.CPU30.TLB:TLB_shootdowns 837559 ± 3% -15.6% 707224 ± 2% interrupts.CPU31.CAL:Function_call_interrupts 1393 ± 9% +142.0% 3371 ± 6% interrupts.CPU31.RES:Rescheduling_interrupts 1102755 ± 2% -16.1% 925679 ± 2% interrupts.CPU31.TLB:TLB_shootdowns 854041 -18.7% 694158 ± 2% interrupts.CPU32.CAL:Function_call_interrupts 1269 +160.9% 3312 ± 6% interrupts.CPU32.RES:Rescheduling_interrupts 1126296 -19.3% 908920 ± 2% interrupts.CPU32.TLB:TLB_shootdowns 856696 -18.5% 697790 interrupts.CPU33.CAL:Function_call_interrupts 1238 ± 2% +162.3% 3248 ± 2% interrupts.CPU33.RES:Rescheduling_interrupts 1131396 -19.0% 916255 interrupts.CPU33.TLB:TLB_shootdowns 846688 -16.4% 708091 interrupts.CPU34.CAL:Function_call_interrupts 1261 +168.3% 3384 ± 9% interrupts.CPU34.RES:Rescheduling_interrupts 1118891 -16.9% 930255 interrupts.CPU34.TLB:TLB_shootdowns 852958 -17.7% 701911 interrupts.CPU35.CAL:Function_call_interrupts 1237 ± 3% +174.2% 3393 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 1123604 -18.1% 920310 interrupts.CPU35.TLB:TLB_shootdowns 848747 -18.7% 689969 interrupts.CPU36.CAL:Function_call_interrupts 1264 ± 2% +190.3% 3670 ± 9% interrupts.CPU36.RES:Rescheduling_interrupts 1118178 -19.0% 905808 interrupts.CPU36.TLB:TLB_shootdowns 856079 -20.6% 679678 ± 4% interrupts.CPU37.CAL:Function_call_interrupts 1296 ± 2% +204.2% 3944 ± 23% interrupts.CPU37.RES:Rescheduling_interrupts 1130431 -21.3% 889126 ± 4% interrupts.CPU37.TLB:TLB_shootdowns 845950 ± 2% -19.8% 678621 ± 3% interrupts.CPU38.CAL:Function_call_interrupts 1621 ± 14% +152.2% 4088 ± 15% interrupts.CPU38.RES:Rescheduling_interrupts 1115980 ± 2% -20.3% 889290 ± 3% interrupts.CPU38.TLB:TLB_shootdowns 853206 -20.6% 677284 ± 2% interrupts.CPU39.CAL:Function_call_interrupts 1283 ± 2% +162.1% 3364 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts 1124010 -21.0% 888036 ± 2% interrupts.CPU39.TLB:TLB_shootdowns 1898 ± 22% +82.8% 3470 ± 6% interrupts.CPU4.RES:Rescheduling_interrupts 834314 ± 2% -17.6% 687884 interrupts.CPU40.CAL:Function_call_interrupts 1397 ± 9% +138.4% 3331 ± 6% interrupts.CPU40.RES:Rescheduling_interrupts 1100215 ± 2% -17.9% 903489 interrupts.CPU40.TLB:TLB_shootdowns 849999 -19.4% 684913 ± 2% interrupts.CPU41.CAL:Function_call_interrupts 1264 ± 2% +226.9% 4132 ± 34% interrupts.CPU41.RES:Rescheduling_interrupts 1121696 -19.9% 898453 ± 2% interrupts.CPU41.TLB:TLB_shootdowns 834049 ± 2% -18.1% 683236 ± 2% interrupts.CPU43.CAL:Function_call_interrupts 4865 ± 7% +55.4% 7559 ± 11% interrupts.CPU43.RES:Rescheduling_interrupts 1099407 ± 2% -18.4% 896611 ± 2% interrupts.CPU43.TLB:TLB_shootdowns 833956 ± 3% -14.8% 710586 ± 3% interrupts.CPU44.CAL:Function_call_interrupts 1509 ± 13% +125.4% 3401 ± 3% interrupts.CPU44.RES:Rescheduling_interrupts 1096297 ± 3% -14.9% 932685 ± 3% interrupts.CPU44.TLB:TLB_shootdowns 849547 -16.1% 712834 interrupts.CPU45.CAL:Function_call_interrupts 1293 ± 3% +172.9% 3528 ± 13% interrupts.CPU45.RES:Rescheduling_interrupts 1114803 -15.9% 938061 interrupts.CPU45.TLB:TLB_shootdowns 854964 -15.8% 719973 interrupts.CPU46.CAL:Function_call_interrupts 1305 ± 6% +169.0% 3511 ± 2% interrupts.CPU46.RES:Rescheduling_interrupts 1123583 -15.9% 945268 interrupts.CPU46.TLB:TLB_shootdowns 849620 -15.0% 722177 interrupts.CPU47.CAL:Function_call_interrupts 1297 ± 4% +172.8% 3538 ± 7% interrupts.CPU47.RES:Rescheduling_interrupts 1115065 -15.0% 947776 interrupts.CPU47.TLB:TLB_shootdowns 1803 ± 20% +102.4% 3651 ± 2% interrupts.CPU48.RES:Rescheduling_interrupts 842642 -16.1% 707304 interrupts.CPU49.CAL:Function_call_interrupts 1361 ± 14% +149.1% 3392 ± 5% interrupts.CPU49.RES:Rescheduling_interrupts 1105634 -15.8% 930642 interrupts.CPU49.TLB:TLB_shootdowns 844930 -15.2% 716662 ± 2% interrupts.CPU5.CAL:Function_call_interrupts 1390 ± 7% +326.2% 5924 ± 72% interrupts.CPU5.RES:Rescheduling_interrupts 1113774 -15.8% 937888 ± 2% interrupts.CPU5.TLB:TLB_shootdowns 854876 -15.9% 718573 interrupts.CPU50.CAL:Function_call_interrupts 1310 ± 5% +163.8% 3456 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts 1122934 -16.2% 941211 interrupts.CPU50.TLB:TLB_shootdowns 842546 ± 2% -16.3% 705092 ± 2% interrupts.CPU51.CAL:Function_call_interrupts 1776 ± 37% +93.0% 3428 ± 8% interrupts.CPU51.RES:Rescheduling_interrupts 1105813 ± 2% -16.3% 925662 ± 2% interrupts.CPU51.TLB:TLB_shootdowns 836362 ± 4% -15.7% 705404 interrupts.CPU52.CAL:Function_call_interrupts 1458 ± 26% +137.0% 3455 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts 1097031 ± 4% -15.9% 923015 interrupts.CPU52.TLB:TLB_shootdowns 852018 -15.8% 717622 interrupts.CPU53.CAL:Function_call_interrupts 1332 ± 4% +164.0% 3516 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts 1118972 -15.6% 943908 interrupts.CPU53.TLB:TLB_shootdowns 854462 -17.9% 701390 interrupts.CPU54.CAL:Function_call_interrupts 1366 ± 7% +164.6% 3616 ± 9% interrupts.CPU54.RES:Rescheduling_interrupts 1123651 -18.2% 919457 interrupts.CPU54.TLB:TLB_shootdowns 845581 -16.4% 706511 ± 3% interrupts.CPU55.CAL:Function_call_interrupts 1240 ± 4% +193.3% 3637 ± 3% interrupts.CPU55.RES:Rescheduling_interrupts 1110236 -16.7% 925369 ± 2% interrupts.CPU55.TLB:TLB_shootdowns 845406 -15.7% 712954 interrupts.CPU56.CAL:Function_call_interrupts 1338 ± 13% +149.2% 3335 ± 4% interrupts.CPU56.RES:Rescheduling_interrupts 1112073 -15.9% 935051 interrupts.CPU56.TLB:TLB_shootdowns 856970 -17.3% 708772 interrupts.CPU57.CAL:Function_call_interrupts 1353 ± 6% +147.4% 3346 ± 3% interrupts.CPU57.RES:Rescheduling_interrupts 1125910 -17.3% 930639 interrupts.CPU57.TLB:TLB_shootdowns 846527 -14.1% 727075 interrupts.CPU58.CAL:Function_call_interrupts 1399 ± 12% +139.2% 3346 ± 5% interrupts.CPU58.RES:Rescheduling_interrupts 1109644 -13.9% 955255 interrupts.CPU58.TLB:TLB_shootdowns 844275 -14.7% 719842 interrupts.CPU59.CAL:Function_call_interrupts 1280 ± 6% +164.2% 3382 ± 2% interrupts.CPU59.RES:Rescheduling_interrupts 1107026 -14.8% 942928 interrupts.CPU59.TLB:TLB_shootdowns 847285 -13.5% 732526 ± 2% interrupts.CPU6.CAL:Function_call_interrupts 1451 ± 4% +439.5% 7831 ± 88% interrupts.CPU6.RES:Rescheduling_interrupts 1117272 -13.9% 962184 interrupts.CPU6.TLB:TLB_shootdowns 1755 ± 27% +91.3% 3356 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 856357 -17.6% 705761 ± 2% interrupts.CPU61.CAL:Function_call_interrupts 1287 ± 3% +166.4% 3430 interrupts.CPU61.RES:Rescheduling_interrupts 1123124 -17.5% 926165 interrupts.CPU61.TLB:TLB_shootdowns 861067 -17.4% 711615 interrupts.CPU62.CAL:Function_call_interrupts 1379 ± 12% +137.6% 3276 ± 5% interrupts.CPU62.RES:Rescheduling_interrupts 1130950 -17.4% 934189 interrupts.CPU62.TLB:TLB_shootdowns 856281 -18.7% 696216 ± 3% interrupts.CPU63.CAL:Function_call_interrupts 1245 ± 4% +214.1% 3911 ± 20% interrupts.CPU63.RES:Rescheduling_interrupts 1124489 -18.9% 912042 ± 3% interrupts.CPU63.TLB:TLB_shootdowns 851819 -18.0% 698430 ± 2% interrupts.CPU64.CAL:Function_call_interrupts 1305 ± 7% +167.0% 3485 ± 8% interrupts.CPU64.RES:Rescheduling_interrupts 1116110 -17.6% 919282 interrupts.CPU64.TLB:TLB_shootdowns 845244 -16.0% 709808 interrupts.CPU65.CAL:Function_call_interrupts 1296 ± 5% +158.8% 3356 ± 4% interrupts.CPU65.RES:Rescheduling_interrupts 1106696 -15.6% 934351 interrupts.CPU65.TLB:TLB_shootdowns 808718 ± 3% -17.1% 670786 ± 3% interrupts.CPU66.CAL:Function_call_interrupts 1525 ± 13% +171.2% 4138 ± 33% interrupts.CPU66.RES:Rescheduling_interrupts 1062684 ± 4% -17.0% 881987 ± 3% interrupts.CPU66.TLB:TLB_shootdowns 838931 ± 2% -18.5% 683635 interrupts.CPU67.CAL:Function_call_interrupts 1350 ± 4% +197.6% 4020 ± 20% interrupts.CPU67.RES:Rescheduling_interrupts 1102936 ± 2% -18.5% 898555 interrupts.CPU67.TLB:TLB_shootdowns 830872 ± 3% -18.4% 677904 ± 2% interrupts.CPU68.CAL:Function_call_interrupts 1377 ± 10% +143.5% 3353 ± 5% interrupts.CPU68.RES:Rescheduling_interrupts 1089452 ± 3% -18.2% 890636 ± 2% interrupts.CPU68.TLB:TLB_shootdowns 815272 ± 7% -16.3% 682235 ± 2% interrupts.CPU69.CAL:Function_call_interrupts 1446 ± 21% +148.5% 3593 ± 9% interrupts.CPU69.RES:Rescheduling_interrupts 1068586 ± 8% -16.0% 897562 ± 2% interrupts.CPU69.TLB:TLB_shootdowns 839315 ± 2% -13.5% 726052 interrupts.CPU7.CAL:Function_call_interrupts 1528 ± 5% +134.3% 3580 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 1106012 -13.9% 951903 interrupts.CPU7.TLB:TLB_shootdowns 838599 ± 5% -19.8% 672407 ± 2% interrupts.CPU70.CAL:Function_call_interrupts 1370 ± 15% +141.5% 3310 ± 3% interrupts.CPU70.RES:Rescheduling_interrupts 1103645 ± 6% -20.0% 882524 ± 2% interrupts.CPU70.TLB:TLB_shootdowns 860160 -19.2% 694653 interrupts.CPU71.CAL:Function_call_interrupts 1227 ± 3% +169.8% 3312 ± 4% interrupts.CPU71.RES:Rescheduling_interrupts 1130571 -19.2% 913600 interrupts.CPU71.TLB:TLB_shootdowns 827876 ± 3% -16.6% 690247 interrupts.CPU72.CAL:Function_call_interrupts 1355 ± 10% +230.5% 4478 ± 43% interrupts.CPU72.RES:Rescheduling_interrupts 1085983 ± 3% -16.3% 909038 interrupts.CPU72.TLB:TLB_shootdowns 848852 -20.9% 671383 interrupts.CPU73.CAL:Function_call_interrupts 1250 ± 5% +202.8% 3786 ± 20% interrupts.CPU73.RES:Rescheduling_interrupts 1113651 -20.8% 882181 interrupts.CPU73.TLB:TLB_shootdowns 852958 -19.5% 686257 interrupts.CPU74.CAL:Function_call_interrupts 1259 ± 7% +329.4% 5409 ± 43% interrupts.CPU74.RES:Rescheduling_interrupts 1120136 -19.5% 901527 interrupts.CPU74.TLB:TLB_shootdowns 857393 -21.0% 677103 ± 2% interrupts.CPU75.CAL:Function_call_interrupts 1287 ± 5% +235.8% 4322 ± 23% interrupts.CPU75.RES:Rescheduling_interrupts 1126167 -21.2% 887159 ± 2% interrupts.CPU75.TLB:TLB_shootdowns 853415 -23.3% 654861 interrupts.CPU76.CAL:Function_call_interrupts 1212 ± 3% +235.3% 4064 ± 23% interrupts.CPU76.RES:Rescheduling_interrupts 1121486 -23.3% 860207 interrupts.CPU76.TLB:TLB_shootdowns 859113 -21.6% 673412 ± 3% interrupts.CPU77.CAL:Function_call_interrupts 1217 ± 5% +262.3% 4410 ± 24% interrupts.CPU77.RES:Rescheduling_interrupts 1132002 -21.7% 886380 ± 3% interrupts.CPU77.TLB:TLB_shootdowns 854518 -21.5% 671017 ± 2% interrupts.CPU78.CAL:Function_call_interrupts 1244 ± 3% +207.4% 3826 ± 20% interrupts.CPU78.RES:Rescheduling_interrupts 1122885 -21.3% 883247 ± 2% interrupts.CPU78.TLB:TLB_shootdowns 850847 -20.4% 677155 ± 2% interrupts.CPU79.CAL:Function_call_interrupts 1225 ± 4% +179.6% 3426 ± 3% interrupts.CPU79.RES:Rescheduling_interrupts 1116800 -20.4% 888995 ± 2% interrupts.CPU79.TLB:TLB_shootdowns 1808 ± 34% +98.0% 3580 ± 13% interrupts.CPU8.RES:Rescheduling_interrupts 853050 -22.5% 660941 ± 2% interrupts.CPU80.CAL:Function_call_interrupts 1316 ± 10% +211.6% 4102 ± 35% interrupts.CPU80.RES:Rescheduling_interrupts 1120487 -22.4% 869874 interrupts.CPU80.TLB:TLB_shootdowns 849727 -23.0% 654228 ± 2% interrupts.CPU81.CAL:Function_call_interrupts 1359 ± 9% +246.3% 4708 ± 43% interrupts.CPU81.RES:Rescheduling_interrupts 1114279 -22.9% 858982 ± 2% interrupts.CPU81.TLB:TLB_shootdowns 828152 ± 4% -20.6% 657723 ± 3% interrupts.CPU82.CAL:Function_call_interrupts 1394 ± 19% +273.8% 5211 ± 57% interrupts.CPU82.RES:Rescheduling_interrupts 1084984 ± 4% -20.5% 862349 ± 2% interrupts.CPU82.TLB:TLB_shootdowns 851441 -22.3% 661400 interrupts.CPU83.CAL:Function_call_interrupts 1209 ± 2% +229.9% 3989 ± 19% interrupts.CPU83.RES:Rescheduling_interrupts 1118858 -22.2% 870532 interrupts.CPU83.TLB:TLB_shootdowns 840296 ± 2% -20.1% 671202 interrupts.CPU84.CAL:Function_call_interrupts 1301 ± 7% +227.2% 4256 ± 39% interrupts.CPU84.RES:Rescheduling_interrupts 1105893 ± 2% -20.1% 883700 interrupts.CPU84.TLB:TLB_shootdowns 847835 -23.2% 651208 ± 3% interrupts.CPU85.CAL:Function_call_interrupts 1234 ± 4% +250.6% 4326 ± 33% interrupts.CPU85.RES:Rescheduling_interrupts 1112616 -23.2% 854803 ± 3% interrupts.CPU85.TLB:TLB_shootdowns 828071 ± 6% -20.5% 658549 interrupts.CPU86.CAL:Function_call_interrupts 1447 ± 24% +308.6% 5912 ± 71% interrupts.CPU86.RES:Rescheduling_interrupts 1083338 ± 6% -20.2% 864980 interrupts.CPU86.TLB:TLB_shootdowns 809024 ± 4% -19.0% 655464 interrupts.CPU87.CAL:Function_call_interrupts 1385 ± 15% +219.9% 4430 ± 29% interrupts.CPU87.RES:Rescheduling_interrupts 1062720 ± 4% -18.8% 863388 interrupts.CPU87.TLB:TLB_shootdowns 844286 -12.0% 743024 interrupts.CPU9.CAL:Function_call_interrupts 1416 ± 6% +138.4% 3377 ± 6% interrupts.CPU9.RES:Rescheduling_interrupts 1111572 -12.5% 972796 interrupts.CPU9.TLB:TLB_shootdowns 230.25 -27.1% 167.75 interrupts.IWI:IRQ_work_interrupts 130091 ± 2% +166.1% 346125 ± 4% interrupts.RES:Rescheduling_interrupts 97172549 -16.8% 80835240 interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-bdw-ep3c: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory ========================================================================================= compiler/cpufreq_governor/disk/filenum/fs/iomode/kconfig/nr_threads/period/rootfs/rwmode/size/tbox_group/testcase/ucode: gcc-7/performance/1HDD/1024f/f2fs/sync/x86_64-rhel-7.2/100%/600s/debian-x86_64-2018-04-03.cgz/rndrw/64G/lkp-bdw-ep3c/fileio/0xb00002e commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 0.09 ± 11% +11356.3% 9.97 ± 12% fileio.request_latency_95%_ms 1.16 ± 4% +47.8% 1.72 ± 4% fileio.request_latency_avg_ms 308.49 -2.7% 300.07 fileio.requests_per_sec 101.31 ± 18% +36.9% 138.69 ± 4% iostat.sda.rkB/s 285537 ± 18% -54.5% 129889 ± 9% meminfo.DirectMap4k 99.80 ± 18% +37.7% 137.40 ± 4% vmstat.io.bi 1574407 -2.2% 1540252 proc-vmstat.nr_dirty_background_threshold 3152666 -2.2% 3084272 proc-vmstat.nr_dirty_threshold 3546424 ± 8% +12.8% 4000300 ± 2% proc-vmstat.pgfree 16488 ± 47% -86.1% 2294 ±101% proc-vmstat.pgmigrate_success 1186695 ± 2% +33.5% 1584179 ± 2% proc-vmstat.pgscan_kswapd 1185752 ± 2% +33.5% 1582466 ± 2% proc-vmstat.pgsteal_kswapd 157595 ± 2% +26.9% 200033 ± 2% proc-vmstat.slabs_scanned *************************************************************************************************** lkp-hsx04: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-hsx04/lru-file-readonce/vm-scalability/0x12 commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 159951 -29.6% 112660 ± 2% vm-scalability.median 23108650 -29.3% 16329964 ± 2% vm-scalability.throughput 471.69 +15.2% 543.49 vm-scalability.time.elapsed_time 471.69 +15.2% 543.49 vm-scalability.time.elapsed_time.max 228936 +13.9% 260704 vm-scalability.time.involuntary_context_switches 107979 -1.1% 106817 vm-scalability.time.minor_page_faults 5591 +12.3% 6280 ± 2% vm-scalability.time.percent_of_cpu_this_job_got 25568 +30.1% 33268 ± 2% vm-scalability.time.system_time 809.74 +6.0% 858.09 vm-scalability.time.user_time 2098 +8.3% 2272 ± 5% vm-scalability.time.voluntary_context_switches 7174857 ± 6% +20.4% 8637309 ± 10% meminfo.DirectMap2M 306801 ± 11% -50.5% 151829 ± 8% meminfo.DirectMap4k 60.23 -5.9 54.29 ± 2% mpstat.cpu.idle% 0.01 ± 8% +0.0 0.05 ±126% mpstat.cpu.soft% 38.48 +5.9 44.37 ± 3% mpstat.cpu.sys% 512.25 +20.0% 614.75 turbostat.Avg_MHz 42.79 +8.6 51.36 turbostat.Busy% 72.32 +3.0% 74.49 turbostat.RAMWatt 60.00 -11.7% 53.00 ± 2% vmstat.cpu.id 1.671e+08 -12.6% 1.46e+08 ± 4% vmstat.memory.free 58.00 +24.1% 72.00 ± 2% vmstat.procs.r 2311 -6.6% 2158 vmstat.system.cs 43878170 ± 6% -17.1% 36355184 ± 6% numa-meminfo.node2.MemFree 42955 ± 5% +16.2% 49901 ± 4% numa-meminfo.node2.SUnreclaim 2003 ± 19% +257.4% 7161 ± 79% numa-meminfo.node2.Shmem 1303428 ± 8% -25.5% 971322 ± 2% numa-meminfo.node3.KReclaimable 1303428 ± 8% -25.5% 971322 ± 2% numa-meminfo.node3.SReclaimable 1350910 ± 7% -24.3% 1022705 ± 2% numa-meminfo.node3.Slab 36913 ± 3% +8.4% 39997 ± 3% slabinfo.anon_vma_chain.active_objs 36913 ± 3% +8.4% 39997 ± 3% slabinfo.anon_vma_chain.num_objs 12723 +12.5% 14317 ± 3% slabinfo.cred_jar.active_objs 12723 +12.5% 14317 ± 3% slabinfo.cred_jar.num_objs 527.50 ± 10% -21.2% 415.50 ± 7% slabinfo.kmalloc-rcl-128.active_objs 527.50 ± 10% -21.2% 415.50 ± 7% slabinfo.kmalloc-rcl-128.num_objs 1160 ± 11% +46.0% 1694 ± 11% slabinfo.mnt_cache.active_objs 1160 ± 11% +46.0% 1694 ± 11% slabinfo.mnt_cache.num_objs 11120 ± 3% -7.1% 10335 ± 4% slabinfo.skbuff_head_cache.active_objs 2.578e+08 -19.6% 2.072e+08 ± 3% numa-numastat.node0.local_node 9765240 ± 6% +527.5% 61273194 ± 7% numa-numastat.node0.numa_foreign 2.578e+08 -19.6% 2.072e+08 ± 3% numa-numastat.node0.numa_hit 9535411 ± 6% +528.0% 59878960 ± 12% numa-numastat.node0.numa_miss 9555465 ± 6% +526.8% 59895789 ± 12% numa-numastat.node0.other_node 2.584e+08 -19.7% 2.074e+08 ± 3% numa-numastat.node1.local_node 10715796 ± 7% +447.8% 58705063 ± 8% numa-numastat.node1.numa_foreign 2.584e+08 -19.7% 2.074e+08 ± 3% numa-numastat.node1.numa_hit 9308839 ± 8% +549.4% 60450129 ± 8% numa-numastat.node1.numa_miss 9327859 ± 8% +548.3% 60469137 ± 8% numa-numastat.node1.other_node 2.593e+08 -17.3% 2.145e+08 numa-numastat.node2.local_node 10000057 ± 5% +454.7% 55473129 ± 6% numa-numastat.node2.numa_foreign 2.594e+08 -17.3% 2.145e+08 numa-numastat.node2.numa_hit 10449413 ± 10% +425.1% 54872623 ± 2% numa-numastat.node2.numa_miss 10473080 ± 10% +424.2% 54896266 ± 2% numa-numastat.node2.other_node 2.601e+08 -18.2% 2.127e+08 ± 3% numa-numastat.node3.local_node 9331773 ± 4% +524.5% 58276242 ± 8% numa-numastat.node3.numa_foreign 2.601e+08 -18.2% 2.127e+08 ± 3% numa-numastat.node3.numa_hit 10519203 ± 4% +456.4% 58525917 ± 12% numa-numastat.node3.numa_miss 10537154 ± 4% +455.6% 58545882 ± 12% numa-numastat.node3.other_node 7698928 ± 8% +519.3% 47677129 ± 7% numa-vmstat.node0.numa_foreign 2.146e+08 -21.0% 1.696e+08 ± 3% numa-vmstat.node0.numa_hit 2.146e+08 -21.0% 1.696e+08 ± 3% numa-vmstat.node0.numa_local 7651378 ± 6% +519.2% 47377072 ± 12% numa-vmstat.node0.numa_miss 7672255 ± 6% +517.8% 47395926 ± 12% numa-vmstat.node0.numa_other 8564612 ± 8% +431.3% 45502806 ± 8% numa-vmstat.node1.numa_foreign 2.151e+08 -21.1% 1.697e+08 ± 3% numa-vmstat.node1.numa_hit 2.15e+08 -21.1% 1.696e+08 ± 3% numa-vmstat.node1.numa_local 7365803 ± 9% +545.1% 47515619 ± 8% numa-vmstat.node1.numa_miss 7459854 ± 9% +538.2% 47609971 ± 8% numa-vmstat.node1.numa_other 11162894 ± 6% -16.5% 9318413 ± 6% numa-vmstat.node2.nr_free_pages 502.00 ± 19% +257.6% 1795 ± 78% numa-vmstat.node2.nr_shmem 10743 ± 5% +16.2% 12481 ± 4% numa-vmstat.node2.nr_slab_unreclaimable 7950407 ± 6% +448.2% 43587014 ± 6% numa-vmstat.node2.numa_foreign 2.158e+08 -18.5% 1.758e+08 numa-vmstat.node2.numa_hit 2.157e+08 -18.5% 1.757e+08 numa-vmstat.node2.numa_local 8296123 ± 11% +412.9% 42550931 ± 3% numa-vmstat.node2.numa_miss 8394960 ± 11% +408.0% 42649961 ± 3% numa-vmstat.node2.numa_other 325930 ± 8% -26.0% 241040 ± 3% numa-vmstat.node3.nr_slab_reclaimable 7467226 ± 3% +517.8% 46130731 ± 7% numa-vmstat.node3.numa_foreign 2.165e+08 -19.4% 1.745e+08 ± 3% numa-vmstat.node3.numa_hit 2.164e+08 -19.4% 1.744e+08 ± 3% numa-vmstat.node3.numa_local 8367675 ± 5% +443.2% 45454056 ± 11% numa-vmstat.node3.numa_miss 8460511 ± 5% +438.4% 45549245 ± 11% numa-vmstat.node3.numa_other 1869353 ± 10% -26.3% 1377246 ± 5% numa-vmstat.node3.workingset_nodes 40310 ± 22% -36.8% 25468 ± 33% softirqs.CPU1.RCU 38118 ± 3% -27.1% 27807 ± 40% softirqs.CPU101.SCHED 37789 ± 4% -16.7% 31488 ± 20% softirqs.CPU102.SCHED 170153 ± 7% +16.6% 198323 ± 7% softirqs.CPU119.TIMER 174910 ± 5% +12.6% 197017 ± 7% softirqs.CPU123.TIMER 162829 ± 3% +25.1% 203647 ± 13% softirqs.CPU127.TIMER 37212 -9.8% 33580 ± 5% softirqs.CPU128.SCHED 159245 ± 5% +14.9% 183018 ± 6% softirqs.CPU134.TIMER 29813 ± 9% +44.4% 43039 ± 17% softirqs.CPU137.RCU 164212 ± 4% +11.3% 182789 ± 5% softirqs.CPU142.TIMER 29684 ± 8% +33.0% 39467 ± 12% softirqs.CPU143.RCU 21860 ± 19% -40.5% 13008 ± 15% softirqs.CPU16.RCU 17621 ± 15% -30.3% 12276 ± 16% softirqs.CPU17.RCU 39462 ± 28% -44.3% 21987 ± 31% softirqs.CPU3.RCU 37836 ± 4% -26.7% 27737 ± 44% softirqs.CPU34.SCHED 37771 -41.2% 22225 ± 59% softirqs.CPU46.SCHED 169535 ± 7% +16.6% 197650 ± 7% softirqs.CPU48.TIMER 169359 ± 7% +16.9% 197948 ± 7% softirqs.CPU51.TIMER 169823 ± 7% +17.1% 198897 ± 7% softirqs.CPU52.TIMER 166706 ± 7% +13.9% 189872 ± 7% softirqs.CPU54.TIMER 37126 ± 2% -18.8% 30144 ± 19% softirqs.CPU62.SCHED 160554 ± 5% +14.5% 183758 ± 6% softirqs.CPU62.TIMER 43168 ± 39% -42.1% 24976 ± 24% softirqs.CPU7.RCU 162032 ± 6% +13.2% 183443 ± 5% softirqs.CPU70.TIMER 33401 ± 19% -31.3% 22962 ± 28% softirqs.CPU8.RCU 24737 ± 19% -31.1% 17033 ± 32% softirqs.CPU80.RCU 36450 ± 25% -45.6% 19839 ± 23% softirqs.CPU9.RCU 37912 ± 3% -25.6% 28192 ± 40% softirqs.CPU98.SCHED 1301 +51.2% 1968 ± 5% syscalls.sys_close.med 24032353 ± 13% +4.1e+07 65097121 ± 39% syscalls.sys_close.noise.100% 43539949 ± 8% +5e+07 93552431 ± 26% syscalls.sys_close.noise.2% 41134763 ± 8% +4.9e+07 90047291 ± 27% syscalls.sys_close.noise.25% 43449324 ± 8% +5e+07 93407103 ± 26% syscalls.sys_close.noise.5% 36122948 ± 9% +4.5e+07 81450918 ± 30% syscalls.sys_close.noise.50% 29628273 ± 12% +4.2e+07 72003748 ± 34% syscalls.sys_close.noise.75% 79500 ± 3% +1272.1% 1090795 ± 52% syscalls.sys_mmap.max 7675 +104.0% 15656 ± 4% syscalls.sys_mmap.med 5171 +26.4% 6535 ± 10% syscalls.sys_mmap.min 1.348e+08 +4.2e+09 4.316e+09 ± 58% syscalls.sys_mmap.noise.100% 2.285e+08 +4.2e+09 4.462e+09 ± 55% syscalls.sys_mmap.noise.2% 2.135e+08 +4.2e+09 4.43e+09 ± 56% syscalls.sys_mmap.noise.25% 2.276e+08 +4.2e+09 4.46e+09 ± 55% syscalls.sys_mmap.noise.5% 1.801e+08 +4.2e+09 4.384e+09 ± 57% syscalls.sys_mmap.noise.50% 1.549e+08 +4.2e+09 4.341e+09 ± 57% syscalls.sys_mmap.noise.75% 2.494e+10 -85.9% 3.525e+09 ± 40% syscalls.sys_open.max 9579 +95.3% 18704 ± 4% syscalls.sys_open.med 6910 +41.5% 9777 ± 13% syscalls.sys_open.min 2.555e+09 +578.5% 1.733e+10 ± 20% syscalls.sys_read.max 13177 ± 6% -54.1% 6045 ± 4% syscalls.sys_read.med 1760 ± 2% +51.4% 2666 ± 20% syscalls.sys_read.min 7.811e+11 ± 10% -7.6e+11 2.583e+10 ± 78% syscalls.sys_read.noise.100% 7.812e+11 ± 10% -7.6e+11 2.589e+10 ± 78% syscalls.sys_read.noise.2% 7.812e+11 ± 10% -7.6e+11 2.588e+10 ± 78% syscalls.sys_read.noise.25% 7.812e+11 ± 10% -7.6e+11 2.589e+10 ± 78% syscalls.sys_read.noise.5% 7.812e+11 ± 10% -7.6e+11 2.586e+10 ± 78% syscalls.sys_read.noise.50% 7.811e+11 ± 10% -7.6e+11 2.585e+10 ± 78% syscalls.sys_read.noise.75% 90074 ± 62% +1.4e+05% 1.295e+08 ±135% syscalls.sys_write.max 25996 +3.0% 26773 proc-vmstat.allocstall_movable 1.363e+09 ± 4% -73.6% 3.592e+08 ± 10% proc-vmstat.compact_daemon_free_scanned 2935 ± 2% -71.9% 824.75 ± 10% proc-vmstat.compact_daemon_wake 1.366e+09 ± 4% -73.6% 3.609e+08 ± 9% proc-vmstat.compact_free_scanned 2906 -70.2% 866.50 ± 10% proc-vmstat.kswapd_low_wmark_hit_quickly 74663 +2.5% 76558 proc-vmstat.nr_active_anon 66034 +4.8% 69192 proc-vmstat.nr_anon_pages 12949213 -6.9% 12054819 proc-vmstat.nr_dirty_background_threshold 25930088 -6.9% 24139115 proc-vmstat.nr_dirty_threshold 87777480 -5.4% 83004975 proc-vmstat.nr_file_pages 42630023 -9.8% 38448281 ± 4% proc-vmstat.nr_free_pages 87470688 -5.5% 82697166 proc-vmstat.nr_inactive_file 6339 -1.0% 6273 proc-vmstat.nr_mapped 2022 +5.7% 2138 proc-vmstat.nr_page_table_pages 77587 -1.4% 76478 proc-vmstat.nr_shmem 1268379 -7.9% 1167848 ± 2% proc-vmstat.nr_slab_reclaimable 50893 +2.5% 52156 proc-vmstat.nr_slab_unreclaimable 74663 +2.5% 76558 proc-vmstat.nr_zone_active_anon 87470705 -5.5% 82697183 proc-vmstat.nr_zone_inactive_file 39812867 ± 2% +487.1% 2.337e+08 ± 7% proc-vmstat.numa_foreign 1.036e+09 -18.7% 8.419e+08 ± 2% proc-vmstat.numa_hit 1.036e+09 -18.7% 8.418e+08 ± 2% proc-vmstat.numa_local 39812867 ± 2% +487.1% 2.337e+08 ± 7% proc-vmstat.numa_miss 39893681 ± 2% +486.1% 2.338e+08 ± 7% proc-vmstat.numa_other 3517 -66.0% 1196 ± 8% proc-vmstat.pageoutrun 10996 ± 6% -22.1% 8569 ± 10% proc-vmstat.pgactivate 1649248 +4.8% 1729126 proc-vmstat.pgfault 8.165e+08 -10.0% 7.346e+08 proc-vmstat.pgscan_direct 1.29e+08 +68.6% 2.175e+08 ± 5% proc-vmstat.pgscan_kswapd 8.165e+08 -10.0% 7.346e+08 proc-vmstat.pgsteal_direct 1.29e+08 +68.6% 2.175e+08 ± 5% proc-vmstat.pgsteal_kswapd 76624 ± 2% +3780.7% 2973567 ± 32% proc-vmstat.slabs_scanned 7167858 -9.2% 6506875 ± 2% proc-vmstat.workingset_nodes 50.66 -14.7 35.99 ± 10% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 52.75 -13.6 39.15 ± 10% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 43.47 -12.8 30.63 ± 11% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 43.47 -12.8 30.63 ± 11% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead 43.45 -12.8 30.62 ± 11% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 43.47 -12.8 30.64 ± 11% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 43.41 -12.8 30.60 ± 11% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages 33.36 -10.4 23.00 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 34.43 -10.2 24.20 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node 93.36 -4.5 88.90 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 93.18 -4.5 88.73 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.29 ± 3% -1.5 3.80 ± 16% perf-profile.calltrace.cycles-pp.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 5.24 ± 3% -1.5 3.76 ± 16% perf-profile.calltrace.cycles-pp.wakeup_kswapd.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead 1.87 ± 2% -0.5 1.36 ± 18% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 3.04 ± 3% -0.4 2.63 ± 13% perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 2.65 ± 3% -0.4 2.27 ± 13% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read 2.68 ± 3% -0.4 2.31 ± 13% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 0.93 -0.3 0.64 ± 12% perf-profile.calltrace.cycles-pp.xas_store.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list 0.83 ± 4% -0.2 0.59 ± 7% perf-profile.calltrace.cycles-pp.isolate_lru_pages.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 1.08 -0.1 0.96 ± 13% perf-profile.calltrace.cycles-pp.syscall_trace_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.98 ± 4% -0.1 0.86 ± 10% perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 0.93 ± 3% -0.1 0.82 ± 11% perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 0.89 ± 3% -0.1 0.78 ± 11% perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 0.89 ± 28% +0.9 1.77 ± 18% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.73 ± 2% +1.0 2.73 ± 25% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 2.15 ± 27% +1.1 3.24 ± 20% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 2.15 ± 27% +1.1 3.24 ± 20% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 2.15 ± 27% +1.1 3.24 ± 20% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 2.15 ± 27% +1.1 3.24 ± 20% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 2.15 ± 27% +1.1 3.24 ± 20% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 0.00 +1.4 1.43 ± 40% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead 0.00 +1.4 1.43 ± 40% perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 28.06 +13.4 41.43 ± 14% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 28.02 +13.4 41.40 ± 14% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 28.00 +13.4 41.38 ± 14% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 27.87 +13.4 41.26 ± 14% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 19.06 +13.4 32.48 ± 18% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 15.26 +13.5 28.75 ± 21% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 15.31 +13.5 28.81 ± 21% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 16.86 +13.5 30.40 ± 20% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 16.74 +13.6 30.29 ± 20% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 138066 +37.7% 190125 ± 2% sched_debug.cfs_rq:/.exec_clock.avg 143931 +37.0% 197152 ± 3% sched_debug.cfs_rq:/.exec_clock.max 134749 +37.2% 184814 ± 2% sched_debug.cfs_rq:/.exec_clock.min 2008 ± 19% +36.4% 2738 ± 13% sched_debug.cfs_rq:/.load.min 17.49 ± 9% +81.5% 31.74 ± 36% sched_debug.cfs_rq:/.load_avg.avg 89.32 ± 6% +153.2% 226.15 ± 58% sched_debug.cfs_rq:/.load_avg.stddev 20193153 +27.0% 25649338 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg 20970094 +31.5% 27580569 ± 2% sched_debug.cfs_rq:/.min_vruntime.max 14294526 ± 9% +29.4% 18503222 ± 5% sched_debug.cfs_rq:/.min_vruntime.min 1168974 ± 9% +23.7% 1445926 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 34.81 ± 4% +102.4% 70.45 ± 7% sched_debug.cfs_rq:/.nr_spread_over.avg 3.12 ± 30% +1058.3% 36.20 ± 8% sched_debug.cfs_rq:/.nr_spread_over.min 51.25 ± 8% -15.7% 43.18 ± 5% sched_debug.cfs_rq:/.nr_spread_over.stddev 1.72 ± 19% +45.0% 2.49 ± 4% sched_debug.cfs_rq:/.runnable_load_avg.min 1168713 ± 9% +23.7% 1445766 ± 8% sched_debug.cfs_rq:/.spread0.stddev 37.96 ± 56% +234.6% 127.01 ± 35% sched_debug.cfs_rq:/.util_est_enqueued.avg 1030 ± 2% +13.1% 1164 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.max 136.83 ± 18% +64.3% 224.84 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.stddev 295747 ± 14% +27.0% 375634 ± 7% sched_debug.cpu.avg_idle.min 265923 +19.9% 318808 ± 4% sched_debug.cpu.clock.avg 265971 +19.9% 318854 ± 4% sched_debug.cpu.clock.max 265865 +19.9% 318744 ± 4% sched_debug.cpu.clock.min 265923 +19.9% 318808 ± 4% sched_debug.cpu.clock_task.avg 265971 +19.9% 318854 ± 4% sched_debug.cpu.clock_task.max 265865 +19.9% 318744 ± 4% sched_debug.cpu.clock_task.min 1.72 ± 19% +45.0% 2.49 ± 4% sched_debug.cpu.cpu_load[0].min 13.65 ± 11% +34.8% 18.40 ± 15% sched_debug.cpu.cpu_load[1].avg 592.22 +74.3% 1032 ± 60% sched_debug.cpu.cpu_load[1].max 75.75 ± 8% +56.1% 118.28 ± 38% sched_debug.cpu.cpu_load[1].stddev 13.73 ± 11% +33.0% 18.27 ± 14% sched_debug.cpu.cpu_load[2].avg 597.88 +69.8% 1015 ± 57% sched_debug.cpu.cpu_load[2].max 76.09 ± 7% +53.5% 116.77 ± 36% sched_debug.cpu.cpu_load[2].stddev 13.72 ± 11% +29.1% 17.71 ± 9% sched_debug.cpu.cpu_load[3].avg 607.56 ± 2% +54.4% 938.32 ± 45% sched_debug.cpu.cpu_load[3].max 76.23 ± 8% +44.3% 110.02 ± 27% sched_debug.cpu.cpu_load[3].stddev 13.52 ± 11% +27.2% 17.20 ± 4% sched_debug.cpu.cpu_load[4].avg 632.84 ± 4% +38.4% 876.09 ± 31% sched_debug.cpu.cpu_load[4].max 75.89 ± 8% +36.8% 103.82 ± 16% sched_debug.cpu.cpu_load[4].stddev 1647 ± 5% +59.1% 2621 ± 9% sched_debug.cpu.nr_load_updates.stddev 1.41 ± 3% +9.2% 1.54 ± 4% sched_debug.cpu.nr_running.max 0.17 ± 6% +12.7% 0.19 ± 5% sched_debug.cpu.nr_running.stddev 4706 +11.8% 5260 ± 6% sched_debug.cpu.sched_count.avg 1341 ± 3% +18.0% 1582 ± 3% sched_debug.cpu.sched_count.min 490.47 +8.8% 533.85 sched_debug.cpu.ttwu_local.min 265864 +19.9% 318744 ± 4% sched_debug.cpu_clk 261740 +20.2% 314619 ± 4% sched_debug.ktime 271297 +19.5% 324176 ± 3% sched_debug.sched_clk 5.414e+09 +6.5% 5.764e+09 ± 2% perf-stat.i.branch-instructions 5.69 ± 5% +2.0 7.73 ± 4% perf-stat.i.cache-miss-rate% 16895299 +7.3% 18131175 perf-stat.i.cache-misses 1.988e+08 ± 5% -6.0% 1.868e+08 ± 2% perf-stat.i.cache-references 2259 -8.0% 2078 perf-stat.i.context-switches 7.115e+10 +13.9% 8.104e+10 ± 2% perf-stat.i.cpu-cycles 46.80 -7.2% 43.45 perf-stat.i.cpu-migrations 39122191 ± 2% +20.8% 47277484 ± 10% perf-stat.i.dTLB-load-misses 3.761e+09 ± 2% -10.3% 3.372e+09 ± 3% perf-stat.i.dTLB-stores 10186716 ± 3% -12.9% 8874664 ± 3% perf-stat.i.iTLB-load-misses 4774632 ± 5% -13.9% 4110833 ± 5% perf-stat.i.iTLB-loads 3406 -3.6% 3284 perf-stat.i.minor-faults 288264 +0.8% 290711 perf-stat.i.msec 5939570 +20.4% 7148872 ± 2% perf-stat.i.node-load-misses 1987616 +31.5% 2612806 ± 4% perf-stat.i.node-store-misses 5033784 -29.0% 3573151 ± 3% perf-stat.i.node-stores 3408 -3.3% 3294 perf-stat.i.page-faults 7.80 ± 5% -15.9% 6.56 ± 3% perf-stat.overall.MPKI 1.00 ± 9% -0.3 0.72 ± 9% perf-stat.overall.branch-miss-rate% 8.65 ± 5% +1.8 10.49 ± 2% perf-stat.overall.cache-miss-rate% 2.83 +11.8% 3.17 perf-stat.overall.cpi 4212 +9.5% 4611 perf-stat.overall.cycles-between-cache-misses 0.55 ± 2% +0.1 0.62 ± 10% perf-stat.overall.dTLB-load-miss-rate% 0.08 ± 4% +0.0 0.10 ± 5% perf-stat.overall.dTLB-store-miss-rate% 2483 ± 2% +24.6% 3093 ± 3% perf-stat.overall.instructions-per-iTLB-miss 0.35 -10.5% 0.32 perf-stat.overall.ipc 60.68 +3.8 64.43 ± 2% perf-stat.overall.node-load-miss-rate% 28.47 +14.2 42.62 ± 3% perf-stat.overall.node-store-miss-rate% 2846 +24.1% 3533 perf-stat.overall.path-length 5.582e+09 +11.1% 6.2e+09 perf-stat.ps.branch-instructions 55857513 ± 9% -20.7% 44319218 ± 8% perf-stat.ps.branch-misses 17435920 +10.0% 19183129 perf-stat.ps.cache-misses 2.022e+08 ± 5% -9.4% 1.831e+08 ± 2% perf-stat.ps.cache-references 2274 -10.8% 2029 perf-stat.ps.context-switches 7.344e+10 +20.5% 8.846e+10 perf-stat.ps.cpu-cycles 48.62 -4.8% 46.29 perf-stat.ps.cpu-migrations 40262404 ± 2% +19.2% 48006485 ± 11% perf-stat.ps.dTLB-load-misses 7.25e+09 +5.7% 7.667e+09 perf-stat.ps.dTLB-loads 3.85e+09 ± 2% -11.9% 3.393e+09 ± 2% perf-stat.ps.dTLB-stores 10440246 ± 3% -13.5% 9035938 ± 2% perf-stat.ps.iTLB-load-misses 4885236 ± 5% -14.5% 4179292 ± 5% perf-stat.ps.iTLB-loads 2.591e+10 +7.8% 2.793e+10 perf-stat.ps.instructions 3395 -8.9% 3093 perf-stat.ps.minor-faults 6110979 +33.2% 8138580 perf-stat.ps.node-load-misses 3962211 ± 3% +13.5% 4498047 ± 5% perf-stat.ps.node-loads 2065557 +31.8% 2723276 ± 5% perf-stat.ps.node-store-misses 5189749 -29.4% 3664120 ± 3% perf-stat.ps.node-stores 3395 -8.9% 3093 perf-stat.ps.page-faults 1.223e+13 +24.1% 1.518e+13 perf-stat.total.instructions 231.25 +11.7% 258.25 interrupts.100:PCI-MSI.1572919-edge.eth0-TxRx-55 231.25 +11.7% 258.25 interrupts.101:PCI-MSI.1572920-edge.eth0-TxRx-56 231.50 +12.1% 259.50 interrupts.102:PCI-MSI.1572921-edge.eth0-TxRx-57 231.25 +13.0% 261.25 ± 2% interrupts.103:PCI-MSI.1572922-edge.eth0-TxRx-58 231.25 +11.7% 258.25 interrupts.104:PCI-MSI.1572923-edge.eth0-TxRx-59 231.50 +11.8% 258.75 interrupts.105:PCI-MSI.1572924-edge.eth0-TxRx-60 231.25 +11.9% 258.75 interrupts.106:PCI-MSI.1572925-edge.eth0-TxRx-61 231.25 +11.7% 258.25 interrupts.107:PCI-MSI.1572926-edge.eth0-TxRx-62 271.50 ± 14% +53.0% 415.50 ± 47% interrupts.49:PCI-MSI.1572868-edge.eth0-TxRx-4 532.25 ± 33% -48.8% 272.75 ± 2% interrupts.54:PCI-MSI.1572873-edge.eth0-TxRx-9 233.25 +11.8% 260.75 interrupts.64:PCI-MSI.1572883-edge.eth0-TxRx-19 232.00 +11.7% 259.25 interrupts.65:PCI-MSI.1572884-edge.eth0-TxRx-20 231.25 +11.8% 258.50 ± 2% interrupts.66:PCI-MSI.1572885-edge.eth0-TxRx-21 233.25 +10.8% 258.50 interrupts.67:PCI-MSI.1572886-edge.eth0-TxRx-22 231.75 +11.4% 258.25 interrupts.68:PCI-MSI.1572887-edge.eth0-TxRx-23 231.25 +14.5% 264.75 ± 4% interrupts.69:PCI-MSI.1572888-edge.eth0-TxRx-24 231.25 +12.2% 259.50 interrupts.70:PCI-MSI.1572889-edge.eth0-TxRx-25 231.25 +17.3% 271.25 ± 6% interrupts.71:PCI-MSI.1572890-edge.eth0-TxRx-26 231.25 +30.5% 301.75 ± 23% interrupts.72:PCI-MSI.1572891-edge.eth0-TxRx-27 231.25 +14.6% 265.00 ± 3% interrupts.73:PCI-MSI.1572892-edge.eth0-TxRx-28 233.75 +10.9% 259.25 interrupts.74:PCI-MSI.1572893-edge.eth0-TxRx-29 236.75 ± 3% +9.7% 259.75 interrupts.75:PCI-MSI.1572894-edge.eth0-TxRx-30 234.75 ± 2% +14.1% 267.75 ± 5% interrupts.76:PCI-MSI.1572895-edge.eth0-TxRx-31 232.00 +11.9% 259.50 interrupts.77:PCI-MSI.1572896-edge.eth0-TxRx-32 231.25 +13.1% 261.50 ± 2% interrupts.78:PCI-MSI.1572897-edge.eth0-TxRx-33 233.75 +10.9% 259.25 interrupts.80:PCI-MSI.1572899-edge.eth0-TxRx-35 231.25 +30.3% 301.25 ± 21% interrupts.81:PCI-MSI.1572900-edge.eth0-TxRx-36 234.00 +11.5% 261.00 interrupts.82:PCI-MSI.1572901-edge.eth0-TxRx-37 231.75 +12.0% 259.50 interrupts.83:PCI-MSI.1572902-edge.eth0-TxRx-38 231.25 +12.1% 259.25 interrupts.84:PCI-MSI.1572903-edge.eth0-TxRx-39 231.75 +11.4% 258.25 interrupts.85:PCI-MSI.1572904-edge.eth0-TxRx-40 231.25 +11.7% 258.25 interrupts.86:PCI-MSI.1572905-edge.eth0-TxRx-41 231.25 +11.7% 258.25 interrupts.87:PCI-MSI.1572906-edge.eth0-TxRx-42 231.25 +11.8% 258.50 interrupts.88:PCI-MSI.1572907-edge.eth0-TxRx-43 231.75 +11.4% 258.25 interrupts.89:PCI-MSI.1572908-edge.eth0-TxRx-44 231.25 +11.9% 258.75 interrupts.90:PCI-MSI.1572909-edge.eth0-TxRx-45 231.25 +11.8% 258.50 interrupts.91:PCI-MSI.1572910-edge.eth0-TxRx-46 231.50 +11.7% 258.50 ± 2% interrupts.92:PCI-MSI.1572911-edge.eth0-TxRx-47 231.50 +11.6% 258.25 interrupts.93:PCI-MSI.1572912-edge.eth0-TxRx-48 231.75 +11.5% 258.50 interrupts.94:PCI-MSI.1572913-edge.eth0-TxRx-49 231.25 +12.6% 260.50 interrupts.95:PCI-MSI.1572914-edge.eth0-TxRx-50 231.25 +11.7% 258.25 interrupts.96:PCI-MSI.1572915-edge.eth0-TxRx-51 231.50 +12.3% 260.00 interrupts.97:PCI-MSI.1572916-edge.eth0-TxRx-52 234.25 +10.4% 258.50 interrupts.98:PCI-MSI.1572917-edge.eth0-TxRx-53 231.75 +11.7% 258.75 interrupts.99:PCI-MSI.1572918-edge.eth0-TxRx-54 626729 +2.8% 644483 interrupts.CAL:Function_call_interrupts 1767 ± 62% -82.2% 314.75 ±172% interrupts.CPU104.NMI:Non-maskable_interrupts 1767 ± 62% -82.2% 314.75 ±172% interrupts.CPU104.PMI:Performance_monitoring_interrupts 420.25 ± 33% -54.5% 191.25 ± 32% interrupts.CPU108.RES:Rescheduling_interrupts 193.00 ± 17% -52.7% 91.25 ± 15% interrupts.CPU112.RES:Rescheduling_interrupts 2102 ± 66% -85.3% 308.75 ±172% interrupts.CPU115.NMI:Non-maskable_interrupts 2102 ± 66% -85.3% 308.75 ±172% interrupts.CPU115.PMI:Performance_monitoring_interrupts 169.50 ± 34% -55.2% 76.00 ± 16% interrupts.CPU115.RES:Rescheduling_interrupts 2547 ± 33% -73.2% 682.00 ±144% interrupts.CPU120.NMI:Non-maskable_interrupts 2547 ± 33% -73.2% 682.00 ±144% interrupts.CPU120.PMI:Performance_monitoring_interrupts 2087 ± 65% -69.6% 633.50 ±173% interrupts.CPU129.NMI:Non-maskable_interrupts 2087 ± 65% -69.6% 633.50 ±173% interrupts.CPU129.PMI:Performance_monitoring_interrupts 4266 ± 5% +6.1% 4528 ± 5% interrupts.CPU133.CAL:Function_call_interrupts 4244 ± 4% +9.2% 4633 ± 5% interrupts.CPU134.CAL:Function_call_interrupts 2103 ± 34% -56.1% 924.50 ± 59% interrupts.CPU138.NMI:Non-maskable_interrupts 2103 ± 34% -56.1% 924.50 ± 59% interrupts.CPU138.PMI:Performance_monitoring_interrupts 125.50 ± 41% -54.4% 57.25 ± 15% interrupts.CPU140.RES:Rescheduling_interrupts 2085 ± 34% -100.0% 1.00 ±122% interrupts.CPU143.NMI:Non-maskable_interrupts 2085 ± 34% -100.0% 1.00 ±122% interrupts.CPU143.PMI:Performance_monitoring_interrupts 91.25 ± 36% -81.6% 16.75 ± 14% interrupts.CPU18.TLB:TLB_shootdowns 233.25 +11.8% 260.75 interrupts.CPU19.64:PCI-MSI.1572883-edge.eth0-TxRx-19 232.00 +11.7% 259.25 interrupts.CPU20.65:PCI-MSI.1572884-edge.eth0-TxRx-20 231.25 +11.8% 258.50 ± 2% interrupts.CPU21.66:PCI-MSI.1572885-edge.eth0-TxRx-21 233.25 +10.8% 258.50 interrupts.CPU22.67:PCI-MSI.1572886-edge.eth0-TxRx-22 231.75 +11.4% 258.25 interrupts.CPU23.68:PCI-MSI.1572887-edge.eth0-TxRx-23 231.25 +14.5% 264.75 ± 4% interrupts.CPU24.69:PCI-MSI.1572888-edge.eth0-TxRx-24 231.25 +12.2% 259.50 interrupts.CPU25.70:PCI-MSI.1572889-edge.eth0-TxRx-25 231.25 +17.3% 271.25 ± 6% interrupts.CPU26.71:PCI-MSI.1572890-edge.eth0-TxRx-26 4226 ± 2% +9.4% 4625 ± 5% interrupts.CPU26.CAL:Function_call_interrupts 231.25 +30.5% 301.75 ± 23% interrupts.CPU27.72:PCI-MSI.1572891-edge.eth0-TxRx-27 231.25 +14.6% 265.00 ± 3% interrupts.CPU28.73:PCI-MSI.1572892-edge.eth0-TxRx-28 233.75 +10.9% 259.25 interrupts.CPU29.74:PCI-MSI.1572893-edge.eth0-TxRx-29 236.75 ± 3% +9.7% 259.75 interrupts.CPU30.75:PCI-MSI.1572894-edge.eth0-TxRx-30 234.75 ± 2% +14.1% 267.75 ± 5% interrupts.CPU31.76:PCI-MSI.1572895-edge.eth0-TxRx-31 232.00 +11.9% 259.50 interrupts.CPU32.77:PCI-MSI.1572896-edge.eth0-TxRx-32 231.25 +13.1% 261.50 ± 2% interrupts.CPU33.78:PCI-MSI.1572897-edge.eth0-TxRx-33 233.75 +10.9% 259.25 interrupts.CPU35.80:PCI-MSI.1572899-edge.eth0-TxRx-35 231.25 +30.3% 301.25 ± 21% interrupts.CPU36.81:PCI-MSI.1572900-edge.eth0-TxRx-36 234.00 +11.5% 261.00 interrupts.CPU37.82:PCI-MSI.1572901-edge.eth0-TxRx-37 231.75 +12.0% 259.50 interrupts.CPU38.83:PCI-MSI.1572902-edge.eth0-TxRx-38 231.25 +12.1% 259.25 interrupts.CPU39.84:PCI-MSI.1572903-edge.eth0-TxRx-39 271.50 ± 14% +53.0% 415.50 ± 47% interrupts.CPU4.49:PCI-MSI.1572868-edge.eth0-TxRx-4 4289 ± 2% +10.0% 4719 ± 4% interrupts.CPU4.CAL:Function_call_interrupts 724.50 ± 72% -77.0% 166.50 ± 38% interrupts.CPU4.RES:Rescheduling_interrupts 231.75 +11.4% 258.25 interrupts.CPU40.85:PCI-MSI.1572904-edge.eth0-TxRx-40 231.25 +11.7% 258.25 interrupts.CPU41.86:PCI-MSI.1572905-edge.eth0-TxRx-41 231.25 +11.7% 258.25 interrupts.CPU42.87:PCI-MSI.1572906-edge.eth0-TxRx-42 231.25 +11.8% 258.50 interrupts.CPU43.88:PCI-MSI.1572907-edge.eth0-TxRx-43 231.75 +11.4% 258.25 interrupts.CPU44.89:PCI-MSI.1572908-edge.eth0-TxRx-44 4352 ± 4% +7.5% 4680 ± 3% interrupts.CPU44.CAL:Function_call_interrupts 231.25 +11.9% 258.75 interrupts.CPU45.90:PCI-MSI.1572909-edge.eth0-TxRx-45 231.25 +11.8% 258.50 interrupts.CPU46.91:PCI-MSI.1572910-edge.eth0-TxRx-46 231.50 +11.7% 258.50 ± 2% interrupts.CPU47.92:PCI-MSI.1572911-edge.eth0-TxRx-47 231.50 +11.6% 258.25 interrupts.CPU48.93:PCI-MSI.1572912-edge.eth0-TxRx-48 0.75 ±110% +2e+05% 1517 ± 61% interrupts.CPU48.NMI:Non-maskable_interrupts 0.75 ±110% +2e+05% 1517 ± 61% interrupts.CPU48.PMI:Performance_monitoring_interrupts 231.75 +11.5% 258.50 interrupts.CPU49.94:PCI-MSI.1572913-edge.eth0-TxRx-49 231.25 +12.6% 260.50 interrupts.CPU50.95:PCI-MSI.1572914-edge.eth0-TxRx-50 231.25 +11.7% 258.25 interrupts.CPU51.96:PCI-MSI.1572915-edge.eth0-TxRx-51 231.50 +12.3% 260.00 interrupts.CPU52.97:PCI-MSI.1572916-edge.eth0-TxRx-52 234.25 +10.4% 258.50 interrupts.CPU53.98:PCI-MSI.1572917-edge.eth0-TxRx-53 231.75 +11.7% 258.75 interrupts.CPU54.99:PCI-MSI.1572918-edge.eth0-TxRx-54 231.25 +11.7% 258.25 interrupts.CPU55.100:PCI-MSI.1572919-edge.eth0-TxRx-55 231.25 +11.7% 258.25 interrupts.CPU56.101:PCI-MSI.1572920-edge.eth0-TxRx-56 231.50 +12.1% 259.50 interrupts.CPU57.102:PCI-MSI.1572921-edge.eth0-TxRx-57 4289 ± 4% +7.9% 4627 ± 5% interrupts.CPU57.CAL:Function_call_interrupts 231.25 +13.0% 261.25 ± 2% interrupts.CPU58.103:PCI-MSI.1572922-edge.eth0-TxRx-58 231.25 +11.7% 258.25 interrupts.CPU59.104:PCI-MSI.1572923-edge.eth0-TxRx-59 231.50 +11.8% 258.75 interrupts.CPU60.105:PCI-MSI.1572924-edge.eth0-TxRx-60 231.25 +11.9% 258.75 interrupts.CPU61.106:PCI-MSI.1572925-edge.eth0-TxRx-61 231.25 +11.7% 258.25 interrupts.CPU62.107:PCI-MSI.1572926-edge.eth0-TxRx-62 232.25 ± 65% -59.5% 94.00 ± 59% interrupts.CPU67.RES:Rescheduling_interrupts 203.75 ± 45% -62.2% 77.00 ± 46% interrupts.CPU69.RES:Rescheduling_interrupts 2097 ± 65% -84.5% 324.25 ±165% interrupts.CPU70.NMI:Non-maskable_interrupts 2097 ± 65% -84.5% 324.25 ±165% interrupts.CPU70.PMI:Performance_monitoring_interrupts 3915 ± 9% +17.6% 4605 ± 4% interrupts.CPU71.CAL:Function_call_interrupts 341.50 ±172% +653.2% 2572 ± 11% interrupts.CPU71.NMI:Non-maskable_interrupts 341.50 ±172% +653.2% 2572 ± 11% interrupts.CPU71.PMI:Performance_monitoring_interrupts 3723 +34.7% 5016 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts 4252 ± 5% +6.9% 4546 ± 4% interrupts.CPU75.CAL:Function_call_interrupts 4218 ± 4% +7.7% 4543 ± 5% interrupts.CPU8.CAL:Function_call_interrupts 2136 ± 33% -72.8% 580.75 ±100% interrupts.CPU80.NMI:Non-maskable_interrupts 2136 ± 33% -72.8% 580.75 ±100% interrupts.CPU80.PMI:Performance_monitoring_interrupts 2542 ± 32% -85.7% 363.75 ±172% interrupts.CPU81.NMI:Non-maskable_interrupts 2542 ± 32% -85.7% 363.75 ±172% interrupts.CPU81.PMI:Performance_monitoring_interrupts 2096 ± 61% -98.1% 39.25 ±168% interrupts.CPU87.NMI:Non-maskable_interrupts 2096 ± 61% -98.1% 39.25 ±168% interrupts.CPU87.PMI:Performance_monitoring_interrupts 4263 ± 5% +7.2% 4571 ± 5% interrupts.CPU89.CAL:Function_call_interrupts 532.25 ± 33% -48.8% 272.75 ± 2% interrupts.CPU9.54:PCI-MSI.1572873-edge.eth0-TxRx-9 0.75 ±173% +2.1e+05% 1577 ± 67% interrupts.CPU9.NMI:Non-maskable_interrupts 0.75 ±173% +2.1e+05% 1577 ± 67% interrupts.CPU9.PMI:Performance_monitoring_interrupts 574.50 ± 25% -49.2% 292.00 ± 35% interrupts.CPU90.RES:Rescheduling_interrupts 2105 ± 66% -70.5% 621.25 ±173% interrupts.CPU93.NMI:Non-maskable_interrupts 2105 ± 66% -70.5% 621.25 ±173% interrupts.CPU93.PMI:Performance_monitoring_interrupts 3165 ± 9% -61.4% 1223 ± 99% interrupts.CPU98.NMI:Non-maskable_interrupts 3165 ± 9% -61.4% 1223 ± 99% interrupts.CPU98.PMI:Performance_monitoring_interrupts 109.00 ± 12% +165.6% 289.50 ± 34% interrupts.CPU98.RES:Rescheduling_interrupts 201714 ± 7% -26.5% 148264 ± 11% interrupts.NMI:Non-maskable_interrupts 201714 ± 7% -26.5% 148264 ± 11% interrupts.PMI:Performance_monitoring_interrupts 1450 ± 2% -51.5% 704.00 ± 28% interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/thread/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/page_fault2/will-it-scale commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x :4 25% 1:4 dmesg.WARNING:at_ip___perf_sw_event/0x %stddev %change %stddev \ | \ 2137 +25.8% 2688 will-it-scale.per_thread_ops 325.49 -1.4% 320.81 will-it-scale.time.elapsed_time 325.49 -1.4% 320.81 will-it-scale.time.elapsed_time.max 83287 +16.2% 96803 will-it-scale.time.involuntary_context_switches 1.76e+08 +25.2% 2.203e+08 will-it-scale.time.minor_page_faults 12024 +19.1% 14325 will-it-scale.time.percent_of_cpu_this_job_got 38699 +17.5% 45456 will-it-scale.time.system_time 1489661 +26.0% 1876866 will-it-scale.time.voluntary_context_switches 581516 +25.8% 731535 will-it-scale.workload 56.31 -1.8% 55.30 boot-time.boot 1.861e+08 +23.9% 2.305e+08 numa-numastat.node0.local_node 1.861e+08 +23.9% 2.305e+08 numa-numastat.node0.numa_hit 66.25 -8.1 58.19 mpstat.cpu.idle% 0.09 ± 4% +0.0 0.10 ± 2% mpstat.cpu.soft% 33.11 +8.0 41.09 mpstat.cpu.sys% 21449 ± 3% +15.0% 24661 ± 2% numa-meminfo.node0.Mapped 46766835 -48.8% 23956230 ± 3% numa-meminfo.node0.MemFree 52102915 +43.8% 74913520 numa-meminfo.node0.MemUsed 603.00 ± 2% +13.7% 685.50 turbostat.Avg_MHz 50.37 ± 2% +6.9 57.24 turbostat.Busy% 97.45 +2.1% 99.48 turbostat.PkgWatt 11710862 -49.1% 5966411 ± 3% numa-vmstat.node0.nr_free_pages 5453 ± 4% +15.7% 6311 ± 2% numa-vmstat.node0.nr_mapped 1.079e+08 +18.8% 1.282e+08 numa-vmstat.node0.numa_hit 1.079e+08 +18.8% 1.282e+08 numa-vmstat.node0.numa_local 65.75 -11.8% 58.00 vmstat.cpu.id 63019915 -36.1% 40243275 ± 2% vmstat.memory.free 102.75 ± 6% +14.1% 117.25 ± 5% vmstat.procs.r 11235 +22.8% 13797 vmstat.system.cs 5872128 +16.3% 6829056 ± 6% meminfo.DirectMap2M 314076 ± 13% -54.3% 143580 ± 4% meminfo.DirectMap4k 62158781 -36.8% 39276584 ± 2% meminfo.MemAvailable 62774635 -36.5% 39892103 ± 2% meminfo.MemFree 52576927 +43.5% 75459459 meminfo.Memused 211219 +25.3% 264640 meminfo.max_used_kB 1548026 -36.8% 978496 ± 2% proc-vmstat.nr_dirty_background_threshold 3099679 -36.8% 1959394 ± 2% proc-vmstat.nr_dirty_threshold 15679791 -36.4% 9976975 ± 2% proc-vmstat.nr_free_pages 7872 ± 2% +10.3% 8685 proc-vmstat.nr_mapped 6858 +15.5% 7919 ± 3% proc-vmstat.numa_hint_faults 6858 +15.5% 7919 ± 3% proc-vmstat.numa_hint_faults_local 1.862e+08 +23.8% 2.306e+08 proc-vmstat.numa_hit 1.862e+08 +23.8% 2.306e+08 proc-vmstat.numa_local 19257 ± 8% +38.0% 26573 ± 6% proc-vmstat.pgactivate 1.863e+08 +23.8% 2.307e+08 proc-vmstat.pgalloc_normal 1.768e+08 +25.0% 2.211e+08 proc-vmstat.pgfault 1.861e+08 +23.9% 2.306e+08 proc-vmstat.pgfree 19.60 ± 7% -10.2% 17.60 ± 2% perf-stat.i.MPKI 3.541e+09 +17.9% 4.175e+09 perf-stat.i.branch-instructions 2.98 ± 15% -0.7 2.31 ± 7% perf-stat.i.branch-miss-rate% 68202600 ± 3% +7.2% 73081899 ± 3% perf-stat.i.branch-misses 23.25 ± 15% -3.1 20.16 perf-stat.i.cache-miss-rate% 2.468e+08 +14.8% 2.832e+08 perf-stat.i.cache-references 11651 +24.3% 14484 perf-stat.i.context-switches 1.511e+11 +17.3% 1.773e+11 perf-stat.i.cpu-cycles 67.91 ± 3% +16.9% 79.40 ± 2% perf-stat.i.cpu-migrations 2496 ± 8% +16.8% 2915 ± 2% perf-stat.i.cycles-between-cache-misses 0.65 ± 4% -0.1 0.54 ± 5% perf-stat.i.iTLB-load-miss-rate% 68977781 ± 5% +10.5% 76186115 perf-stat.i.iTLB-load-misses 1.499e+10 +17.5% 1.762e+10 perf-stat.i.iTLB-loads 1.513e+10 +17.6% 1.779e+10 perf-stat.i.instructions 205.96 ± 2% +8.0% 222.51 ± 3% perf-stat.i.instructions-per-iTLB-miss 513823 +28.1% 658446 ± 2% perf-stat.i.minor-faults 516702 +28.3% 663095 ± 2% perf-stat.i.page-faults 15.99 ± 2% -2.0% 15.66 perf-stat.overall.MPKI 1.85 ± 3% -0.2 1.69 ± 2% perf-stat.overall.branch-miss-rate% 22.59 ± 11% -2.4 20.14 perf-stat.overall.cache-miss-rate% 2788 ± 8% +13.1% 3154 perf-stat.overall.cycles-between-cache-misses 226.15 ± 3% +5.4% 238.34 perf-stat.overall.instructions-per-iTLB-miss 8787084 -9.4% 7962978 perf-stat.overall.path-length 3.694e+09 +16.1% 4.288e+09 perf-stat.ps.branch-instructions 68237892 ± 2% +6.4% 72573267 ± 3% perf-stat.ps.branch-misses 2.503e+08 +13.5% 2.841e+08 perf-stat.ps.cache-references 11129 +22.9% 13681 perf-stat.ps.context-switches 1.561e+11 +15.6% 1.805e+11 perf-stat.ps.cpu-cycles 67.39 ± 3% +15.6% 77.93 ± 2% perf-stat.ps.cpu-migrations 69392712 ± 5% +9.7% 76142593 perf-stat.ps.iTLB-load-misses 1.566e+10 +15.8% 1.813e+10 perf-stat.ps.iTLB-loads 1.566e+10 +15.8% 1.815e+10 perf-stat.ps.instructions 540047 +27.2% 687035 perf-stat.ps.minor-faults 540857 +27.2% 687815 perf-stat.ps.page-faults 5.11e+12 +14.0% 5.825e+12 perf-stat.total.instructions 73358 +16.5% 85459 sched_debug.cfs_rq:/.exec_clock.avg 79095 +13.0% 89393 sched_debug.cfs_rq:/.exec_clock.max 69934 +16.6% 81559 sched_debug.cfs_rq:/.exec_clock.min 745.22 ± 4% -17.4% 615.84 ± 9% sched_debug.cfs_rq:/.exec_clock.stddev 1.84 ± 15% +55.5% 2.85 ± 8% sched_debug.cfs_rq:/.load_avg.avg 17191895 ± 2% +17.4% 20191179 ± 3% sched_debug.cfs_rq:/.min_vruntime.avg 17496368 ± 2% +17.2% 20511606 ± 3% sched_debug.cfs_rq:/.min_vruntime.max 15904939 +17.2% 18638923 ± 2% sched_debug.cfs_rq:/.min_vruntime.min 0.15 ± 11% +114.0% 0.33 ± 22% sched_debug.cfs_rq:/.nr_running.avg 0.30 ± 3% +35.4% 0.41 sched_debug.cfs_rq:/.nr_running.stddev 2.64 ± 14% +31.0% 3.46 sched_debug.cfs_rq:/.nr_spread_over.avg 1.04 ± 11% +68.1% 1.74 ± 17% sched_debug.cfs_rq:/.runnable_load_avg.avg 225.11 ± 9% +98.7% 447.34 ± 16% sched_debug.cfs_rq:/.util_avg.avg 67.58 ± 21% +71.6% 116.00 ± 26% sched_debug.cfs_rq:/.util_avg.min 125.85 +23.2% 154.98 ± 2% sched_debug.cfs_rq:/.util_avg.stddev 87.11 ± 24% +151.3% 218.90 ± 37% sched_debug.cfs_rq:/.util_est_enqueued.avg 693.08 ± 11% +38.5% 960.17 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max 137.85 ± 11% +84.2% 253.86 ± 13% sched_debug.cfs_rq:/.util_est_enqueued.stddev 145.35 ± 3% +17.6% 170.87 ± 8% sched_debug.cpu.clock.stddev 145.35 ± 3% +17.6% 170.87 ± 8% sched_debug.cpu.clock_task.stddev 1.06 ± 11% +67.0% 1.77 ± 16% sched_debug.cpu.cpu_load[0].avg 1.12 ± 17% +62.0% 1.81 ± 18% sched_debug.cpu.cpu_load[1].avg 1.11 ± 16% +64.2% 1.82 ± 18% sched_debug.cpu.cpu_load[2].avg 1.09 ± 14% +66.4% 1.81 ± 19% sched_debug.cpu.cpu_load[3].avg 1.07 ± 12% +68.7% 1.80 ± 19% sched_debug.cpu.cpu_load[4].avg 565.62 ± 10% +109.2% 1183 ± 22% sched_debug.cpu.curr->pid.avg 1114 ± 4% +30.7% 1456 sched_debug.cpu.curr->pid.stddev 0.00 ± 2% +23.9% 0.00 ± 6% sched_debug.cpu.next_balance.stddev 0.15 ± 10% +113.7% 0.33 ± 22% sched_debug.cpu.nr_running.avg 0.31 ± 3% +34.6% 0.41 sched_debug.cpu.nr_running.stddev 7057 +18.2% 8340 sched_debug.cpu.nr_switches.avg 5908 +21.8% 7194 ± 2% sched_debug.cpu.nr_switches.min 0.69 ± 2% -25.8% 0.51 ± 14% sched_debug.cpu.nr_uninterruptible.avg 6797 +19.0% 8090 sched_debug.cpu.sched_count.avg 5815 +21.6% 7073 sched_debug.cpu.sched_count.min 3075 +19.5% 3673 sched_debug.cpu.sched_goidle.avg 2694 +21.8% 3281 ± 2% sched_debug.cpu.sched_goidle.min 3263 +19.6% 3902 sched_debug.cpu.ttwu_count.avg 2133 +28.0% 2729 ± 2% sched_debug.cpu.ttwu_count.min 20.46 ± 20% -10.7 9.79 ± 12% perf-profile.calltrace.cycles-pp.secondary_startup_64 20.39 ± 20% -10.6 9.76 ± 12% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 20.39 ± 20% -10.6 9.76 ± 12% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 20.35 ± 20% -10.6 9.74 ± 12% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 15.97 ± 19% -8.8 7.17 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 9.98 ± 16% -5.1 4.89 ± 13% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 9.52 ± 15% -4.9 4.67 ± 13% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 5.84 ± 13% -2.9 2.97 ± 13% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 4.24 ± 41% -2.5 1.74 ± 10% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 4.34 ± 14% -2.2 2.13 ± 13% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 3.10 ± 15% -1.5 1.60 ± 14% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 2.40 ± 21% -1.3 1.14 ± 13% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 2.39 ± 19% -1.0 1.38 ± 17% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 1.76 ± 24% -1.0 0.79 ± 11% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 1.91 ± 24% -0.9 0.97 ± 20% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 1.47 ± 40% -0.8 0.63 ± 13% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 1.10 ± 17% -0.8 0.29 ±100% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.63 ± 13% -0.8 0.86 ± 11% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 1.21 ± 14% -0.6 0.63 ± 11% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.64 ± 13% -0.3 0.38 ± 57% perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 0.66 ± 17% -0.3 0.41 ± 59% perf-profile.calltrace.cycles-pp.call_rwsem_down_read_failed.down_read.__do_page_fault.do_page_fault.page_fault 0.64 ± 13% -0.3 0.38 ± 57% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 0.92 ± 11% -0.2 0.74 ± 7% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.munmap 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 0.74 ± 13% -0.2 0.58 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 0.70 ± 13% -0.1 0.55 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 0.82 ± 8% +0.1 0.92 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 0.78 ± 8% +0.1 0.90 ± 6% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.__lru_cache_add 0.80 ± 9% +0.1 0.92 ± 6% perf-profile.calltrace.cycles-pp.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte 0.43 ± 58% +0.2 0.64 ± 8% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore 0.49 ± 58% +0.2 0.73 ± 7% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn 76.37 ± 5% +10.9 87.24 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault.testcase 76.69 ± 5% +10.9 87.57 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.testcase 77.09 ± 5% +10.9 87.99 perf-profile.calltrace.cycles-pp.page_fault.testcase 77.75 ± 5% +10.9 88.69 perf-profile.calltrace.cycles-pp.testcase 62.60 ± 7% +11.1 73.70 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte 62.61 ± 7% +11.1 73.72 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 63.95 ± 7% +11.3 75.22 ± 2% perf-profile.calltrace.cycles-pp.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 69.81 ± 6% +11.3 81.08 ± 2% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 63.80 ± 7% +11.3 75.07 ± 2% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault 70.13 ± 6% +11.3 81.42 ± 2% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault.testcase 64.89 ± 7% +11.3 76.18 ± 2% perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 64.95 ± 7% +11.3 76.25 ± 2% perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 163.75 ± 18% +159.4% 424.75 ± 56% interrupts.CPU100.RES:Rescheduling_interrupts 1366 ± 9% +66.8% 2278 ± 36% interrupts.CPU101.NMI:Non-maskable_interrupts 1366 ± 9% +66.8% 2278 ± 36% interrupts.CPU101.PMI:Performance_monitoring_interrupts 1711 ± 40% +64.9% 2822 ± 39% interrupts.CPU102.NMI:Non-maskable_interrupts 1711 ± 40% +64.9% 2822 ± 39% interrupts.CPU102.PMI:Performance_monitoring_interrupts 1320 ± 8% +73.2% 2286 ± 30% interrupts.CPU103.NMI:Non-maskable_interrupts 1320 ± 8% +73.2% 2286 ± 30% interrupts.CPU103.PMI:Performance_monitoring_interrupts 1651 ± 29% +100.5% 3310 ± 28% interrupts.CPU104.NMI:Non-maskable_interrupts 1651 ± 29% +100.5% 3310 ± 28% interrupts.CPU104.PMI:Performance_monitoring_interrupts 1682 ± 31% +93.4% 3253 ± 28% interrupts.CPU105.NMI:Non-maskable_interrupts 1682 ± 31% +93.4% 3253 ± 28% interrupts.CPU105.PMI:Performance_monitoring_interrupts 1339 ± 11% +71.6% 2298 ± 35% interrupts.CPU107.NMI:Non-maskable_interrupts 1339 ± 11% +71.6% 2298 ± 35% interrupts.CPU107.PMI:Performance_monitoring_interrupts 180.50 ± 37% +94.3% 350.75 ± 26% interrupts.CPU108.RES:Rescheduling_interrupts 1366 ± 11% +137.8% 3250 ± 26% interrupts.CPU11.NMI:Non-maskable_interrupts 1366 ± 11% +137.8% 3250 ± 26% interrupts.CPU11.PMI:Performance_monitoring_interrupts 178.25 ± 40% +180.5% 500.00 ± 90% interrupts.CPU117.RES:Rescheduling_interrupts 1370 ± 9% +69.9% 2328 ± 34% interrupts.CPU118.NMI:Non-maskable_interrupts 1370 ± 9% +69.9% 2328 ± 34% interrupts.CPU118.PMI:Performance_monitoring_interrupts 237.00 ± 17% +58.1% 374.75 ± 45% interrupts.CPU118.RES:Rescheduling_interrupts 1364 ± 10% +67.1% 2280 ± 29% interrupts.CPU12.NMI:Non-maskable_interrupts 1364 ± 10% +67.1% 2280 ± 29% interrupts.CPU12.PMI:Performance_monitoring_interrupts 1632 ± 30% +72.8% 2821 ± 35% interrupts.CPU120.NMI:Non-maskable_interrupts 1632 ± 30% +72.8% 2821 ± 35% interrupts.CPU120.PMI:Performance_monitoring_interrupts 1328 ± 8% +112.5% 2822 ± 40% interrupts.CPU121.NMI:Non-maskable_interrupts 1328 ± 8% +112.5% 2822 ± 40% interrupts.CPU121.PMI:Performance_monitoring_interrupts 1591 ± 26% +73.0% 2753 ± 31% interrupts.CPU125.NMI:Non-maskable_interrupts 1591 ± 26% +73.0% 2753 ± 31% interrupts.CPU125.PMI:Performance_monitoring_interrupts 1664 ± 37% +70.3% 2834 ± 38% interrupts.CPU126.NMI:Non-maskable_interrupts 1664 ± 37% +70.3% 2834 ± 38% interrupts.CPU126.PMI:Performance_monitoring_interrupts 187.50 ± 26% +33.9% 251.00 ± 20% interrupts.CPU129.RES:Rescheduling_interrupts 180.25 ± 18% +40.4% 253.00 ± 28% interrupts.CPU13.RES:Rescheduling_interrupts 1323 ± 9% +42.4% 1884 ± 4% interrupts.CPU131.NMI:Non-maskable_interrupts 1323 ± 9% +42.4% 1884 ± 4% interrupts.CPU131.PMI:Performance_monitoring_interrupts 1345 ± 6% +80.3% 2426 ± 43% interrupts.CPU134.NMI:Non-maskable_interrupts 1345 ± 6% +80.3% 2426 ± 43% interrupts.CPU134.PMI:Performance_monitoring_interrupts 1350 ± 7% +105.1% 2770 ± 29% interrupts.CPU140.NMI:Non-maskable_interrupts 1350 ± 7% +105.1% 2770 ± 29% interrupts.CPU140.PMI:Performance_monitoring_interrupts 1639 ± 28% +73.9% 2850 ± 36% interrupts.CPU148.NMI:Non-maskable_interrupts 1639 ± 28% +73.9% 2850 ± 36% interrupts.CPU148.PMI:Performance_monitoring_interrupts 1663 ± 26% +98.1% 3295 ± 28% interrupts.CPU149.NMI:Non-maskable_interrupts 1663 ± 26% +98.1% 3295 ± 28% interrupts.CPU149.PMI:Performance_monitoring_interrupts 1346 ± 13% +104.6% 2754 ± 29% interrupts.CPU15.NMI:Non-maskable_interrupts 1346 ± 13% +104.6% 2754 ± 29% interrupts.CPU15.PMI:Performance_monitoring_interrupts 1344 ± 7% +37.0% 1842 ± 7% interrupts.CPU151.NMI:Non-maskable_interrupts 1344 ± 7% +37.0% 1842 ± 7% interrupts.CPU151.PMI:Performance_monitoring_interrupts 1349 ± 12% +37.7% 1858 ± 9% interrupts.CPU152.NMI:Non-maskable_interrupts 1349 ± 12% +37.7% 1858 ± 9% interrupts.CPU152.PMI:Performance_monitoring_interrupts 1342 ± 9% +143.9% 3273 ± 26% interrupts.CPU153.NMI:Non-maskable_interrupts 1342 ± 9% +143.9% 3273 ± 26% interrupts.CPU153.PMI:Performance_monitoring_interrupts 1359 ± 9% +75.5% 2385 ± 44% interrupts.CPU154.NMI:Non-maskable_interrupts 1359 ± 9% +75.5% 2385 ± 44% interrupts.CPU154.PMI:Performance_monitoring_interrupts 1735 ± 42% +63.5% 2838 ± 38% interrupts.CPU155.NMI:Non-maskable_interrupts 1735 ± 42% +63.5% 2838 ± 38% interrupts.CPU155.PMI:Performance_monitoring_interrupts 1351 ± 9% +106.5% 2791 ± 29% interrupts.CPU156.NMI:Non-maskable_interrupts 1351 ± 9% +106.5% 2791 ± 29% interrupts.CPU156.PMI:Performance_monitoring_interrupts 1321 ± 10% +73.0% 2285 ± 34% interrupts.CPU158.NMI:Non-maskable_interrupts 1321 ± 10% +73.0% 2285 ± 34% interrupts.CPU158.PMI:Performance_monitoring_interrupts 1298 ± 11% +77.8% 2309 ± 35% interrupts.CPU159.NMI:Non-maskable_interrupts 1298 ± 11% +77.8% 2309 ± 35% interrupts.CPU159.PMI:Performance_monitoring_interrupts 1356 ± 10% +142.8% 3294 ± 26% interrupts.CPU160.NMI:Non-maskable_interrupts 1356 ± 10% +142.8% 3294 ± 26% interrupts.CPU160.PMI:Performance_monitoring_interrupts 167.75 ± 35% +53.5% 257.50 ± 20% interrupts.CPU162.RES:Rescheduling_interrupts 1645 ± 25% +67.1% 2750 ± 30% interrupts.CPU164.NMI:Non-maskable_interrupts 1645 ± 25% +67.1% 2750 ± 30% interrupts.CPU164.PMI:Performance_monitoring_interrupts 1378 ± 7% +70.2% 2345 ± 33% interrupts.CPU168.NMI:Non-maskable_interrupts 1378 ± 7% +70.2% 2345 ± 33% interrupts.CPU168.PMI:Performance_monitoring_interrupts 1661 ± 27% +65.8% 2755 ± 29% interrupts.CPU169.NMI:Non-maskable_interrupts 1661 ± 27% +65.8% 2755 ± 29% interrupts.CPU169.PMI:Performance_monitoring_interrupts 1332 ± 10% +80.4% 2403 ± 44% interrupts.CPU173.NMI:Non-maskable_interrupts 1332 ± 10% +80.4% 2403 ± 44% interrupts.CPU173.PMI:Performance_monitoring_interrupts 1650 ± 30% +67.1% 2757 ± 32% interrupts.CPU175.NMI:Non-maskable_interrupts 1650 ± 30% +67.1% 2757 ± 32% interrupts.CPU175.PMI:Performance_monitoring_interrupts 1615 ± 31% +73.8% 2807 ± 37% interrupts.CPU179.NMI:Non-maskable_interrupts 1615 ± 31% +73.8% 2807 ± 37% interrupts.CPU179.PMI:Performance_monitoring_interrupts 1333 ± 9% +81.3% 2418 ± 43% interrupts.CPU18.NMI:Non-maskable_interrupts 1333 ± 9% +81.3% 2418 ± 43% interrupts.CPU18.PMI:Performance_monitoring_interrupts 1308 ± 7% +116.0% 2825 ± 35% interrupts.CPU180.NMI:Non-maskable_interrupts 1308 ± 7% +116.0% 2825 ± 35% interrupts.CPU180.PMI:Performance_monitoring_interrupts 565.50 ± 41% -50.8% 278.25 ± 32% interrupts.CPU180.RES:Rescheduling_interrupts 1629 ± 30% +68.0% 2737 ± 30% interrupts.CPU182.NMI:Non-maskable_interrupts 1629 ± 30% +68.0% 2737 ± 30% interrupts.CPU182.PMI:Performance_monitoring_interrupts 138.00 ± 18% +71.0% 236.00 ± 8% interrupts.CPU183.RES:Rescheduling_interrupts 174.50 ± 32% +223.9% 565.25 ± 86% interrupts.CPU184.RES:Rescheduling_interrupts 1656 ± 36% +70.0% 2816 ± 38% interrupts.CPU185.NMI:Non-maskable_interrupts 1656 ± 36% +70.0% 2816 ± 38% interrupts.CPU185.PMI:Performance_monitoring_interrupts 216.75 ± 45% +152.9% 548.25 ± 70% interrupts.CPU185.RES:Rescheduling_interrupts 1328 ± 8% +72.8% 2296 ± 35% interrupts.CPU186.NMI:Non-maskable_interrupts 1328 ± 8% +72.8% 2296 ± 35% interrupts.CPU186.PMI:Performance_monitoring_interrupts 1592 ± 27% +78.8% 2848 ± 37% interrupts.CPU189.NMI:Non-maskable_interrupts 1592 ± 27% +78.8% 2848 ± 37% interrupts.CPU189.PMI:Performance_monitoring_interrupts 173.75 ± 22% +75.5% 305.00 ± 42% interrupts.CPU190.RES:Rescheduling_interrupts 1355 ± 8% +36.8% 1854 ± 7% interrupts.CPU193.NMI:Non-maskable_interrupts 1355 ± 8% +36.8% 1854 ± 7% interrupts.CPU193.PMI:Performance_monitoring_interrupts 1334 ± 9% +142.8% 3239 ± 20% interrupts.CPU195.NMI:Non-maskable_interrupts 1334 ± 9% +142.8% 3239 ± 20% interrupts.CPU195.PMI:Performance_monitoring_interrupts 167.25 ± 24% +65.0% 276.00 ± 23% interrupts.CPU197.RES:Rescheduling_interrupts 1329 ± 6% +110.1% 2793 ± 36% interrupts.CPU199.NMI:Non-maskable_interrupts 1329 ± 6% +110.1% 2793 ± 36% interrupts.CPU199.PMI:Performance_monitoring_interrupts 1643 ± 28% +71.7% 2820 ± 38% interrupts.CPU2.NMI:Non-maskable_interrupts 1643 ± 28% +71.7% 2820 ± 38% interrupts.CPU2.PMI:Performance_monitoring_interrupts 1713 ± 45% +64.1% 2811 ± 36% interrupts.CPU209.NMI:Non-maskable_interrupts 1713 ± 45% +64.1% 2811 ± 36% interrupts.CPU209.PMI:Performance_monitoring_interrupts 177.75 ± 28% +96.1% 348.50 ± 43% interrupts.CPU21.RES:Rescheduling_interrupts 1338 ± 6% +72.6% 2310 ± 34% interrupts.CPU211.NMI:Non-maskable_interrupts 1338 ± 6% +72.6% 2310 ± 34% interrupts.CPU211.PMI:Performance_monitoring_interrupts 1617 ± 26% +69.3% 2738 ± 29% interrupts.CPU215.NMI:Non-maskable_interrupts 1617 ± 26% +69.3% 2738 ± 29% interrupts.CPU215.PMI:Performance_monitoring_interrupts 188.75 ± 21% +63.4% 308.50 ± 28% interrupts.CPU216.RES:Rescheduling_interrupts 2050 ± 38% +83.5% 3762 ± 7% interrupts.CPU217.NMI:Non-maskable_interrupts 2050 ± 38% +83.5% 3762 ± 7% interrupts.CPU217.PMI:Performance_monitoring_interrupts 167.00 ± 19% +76.0% 294.00 ± 35% interrupts.CPU218.RES:Rescheduling_interrupts 1673 ± 31% +66.9% 2793 ± 36% interrupts.CPU219.NMI:Non-maskable_interrupts 1673 ± 31% +66.9% 2793 ± 36% interrupts.CPU219.PMI:Performance_monitoring_interrupts 169.75 ± 29% +86.5% 316.50 ± 22% interrupts.CPU219.RES:Rescheduling_interrupts 1371 ± 10% +100.3% 2747 ± 28% interrupts.CPU22.NMI:Non-maskable_interrupts 1371 ± 10% +100.3% 2747 ± 28% interrupts.CPU22.PMI:Performance_monitoring_interrupts 1738 ± 42% +61.7% 2811 ± 38% interrupts.CPU225.NMI:Non-maskable_interrupts 1738 ± 42% +61.7% 2811 ± 38% interrupts.CPU225.PMI:Performance_monitoring_interrupts 178.50 ± 24% +53.8% 274.50 ± 25% interrupts.CPU225.RES:Rescheduling_interrupts 1314 ± 7% +74.0% 2287 ± 31% interrupts.CPU227.NMI:Non-maskable_interrupts 1314 ± 7% +74.0% 2287 ± 31% interrupts.CPU227.PMI:Performance_monitoring_interrupts 1337 ± 9% +114.0% 2862 ± 37% interrupts.CPU231.NMI:Non-maskable_interrupts 1337 ± 9% +114.0% 2862 ± 37% interrupts.CPU231.PMI:Performance_monitoring_interrupts 161.25 ± 45% +76.1% 284.00 ± 29% interrupts.CPU231.RES:Rescheduling_interrupts 1331 ± 9% +73.5% 2309 ± 32% interrupts.CPU234.NMI:Non-maskable_interrupts 1331 ± 9% +73.5% 2309 ± 32% interrupts.CPU234.PMI:Performance_monitoring_interrupts 1326 ± 8% +109.7% 2781 ± 32% interrupts.CPU238.NMI:Non-maskable_interrupts 1326 ± 8% +109.7% 2781 ± 32% interrupts.CPU238.PMI:Performance_monitoring_interrupts 1666 ± 26% +91.9% 3198 ± 20% interrupts.CPU24.NMI:Non-maskable_interrupts 1666 ± 26% +91.9% 3198 ± 20% interrupts.CPU24.PMI:Performance_monitoring_interrupts 164.50 ± 23% +56.8% 258.00 ± 11% interrupts.CPU24.RES:Rescheduling_interrupts 1628 ± 31% +72.4% 2806 ± 35% interrupts.CPU240.NMI:Non-maskable_interrupts 1628 ± 31% +72.4% 2806 ± 35% interrupts.CPU240.PMI:Performance_monitoring_interrupts 1347 ± 8% +179.8% 3768 ± 7% interrupts.CPU241.NMI:Non-maskable_interrupts 1347 ± 8% +179.8% 3768 ± 7% interrupts.CPU241.PMI:Performance_monitoring_interrupts 1318 ± 11% +39.3% 1835 ± 6% interrupts.CPU242.NMI:Non-maskable_interrupts 1318 ± 11% +39.3% 1835 ± 6% interrupts.CPU242.PMI:Performance_monitoring_interrupts 175.50 ± 44% +65.5% 290.50 ± 17% interrupts.CPU244.RES:Rescheduling_interrupts 1335 ± 9% +111.4% 2822 ± 36% interrupts.CPU247.NMI:Non-maskable_interrupts 1335 ± 9% +111.4% 2822 ± 36% interrupts.CPU247.PMI:Performance_monitoring_interrupts 1754 ± 45% +59.5% 2798 ± 32% interrupts.CPU25.NMI:Non-maskable_interrupts 1754 ± 45% +59.5% 2798 ± 32% interrupts.CPU25.PMI:Performance_monitoring_interrupts 147.50 ± 15% +41.7% 209.00 ± 4% interrupts.CPU25.RES:Rescheduling_interrupts 1314 ± 9% +82.7% 2401 ± 42% interrupts.CPU250.NMI:Non-maskable_interrupts 1314 ± 9% +82.7% 2401 ± 42% interrupts.CPU250.PMI:Performance_monitoring_interrupts 1309 ± 9% +78.2% 2332 ± 34% interrupts.CPU251.NMI:Non-maskable_interrupts 1309 ± 9% +78.2% 2332 ± 34% interrupts.CPU251.PMI:Performance_monitoring_interrupts 1379 ± 12% +100.5% 2765 ± 33% interrupts.CPU253.NMI:Non-maskable_interrupts 1379 ± 12% +100.5% 2765 ± 33% interrupts.CPU253.PMI:Performance_monitoring_interrupts 1339 ± 9% +105.0% 2747 ± 33% interrupts.CPU254.NMI:Non-maskable_interrupts 1339 ± 9% +105.0% 2747 ± 33% interrupts.CPU254.PMI:Performance_monitoring_interrupts 1670 ± 38% +70.2% 2842 ± 39% interrupts.CPU257.NMI:Non-maskable_interrupts 1670 ± 38% +70.2% 2842 ± 39% interrupts.CPU257.PMI:Performance_monitoring_interrupts 1319 ± 9% +115.8% 2848 ± 38% interrupts.CPU259.NMI:Non-maskable_interrupts 1319 ± 9% +115.8% 2848 ± 38% interrupts.CPU259.PMI:Performance_monitoring_interrupts 1337 ± 11% +36.9% 1830 ± 7% interrupts.CPU26.NMI:Non-maskable_interrupts 1337 ± 11% +36.9% 1830 ± 7% interrupts.CPU26.PMI:Performance_monitoring_interrupts 1644 ± 24% +73.2% 2847 ± 38% interrupts.CPU264.NMI:Non-maskable_interrupts 1644 ± 24% +73.2% 2847 ± 38% interrupts.CPU264.PMI:Performance_monitoring_interrupts 1612 ± 28% +74.3% 2810 ± 38% interrupts.CPU265.NMI:Non-maskable_interrupts 1612 ± 28% +74.3% 2810 ± 38% interrupts.CPU265.PMI:Performance_monitoring_interrupts 1687 ± 46% +65.6% 2795 ± 35% interrupts.CPU269.NMI:Non-maskable_interrupts 1687 ± 46% +65.6% 2795 ± 35% interrupts.CPU269.PMI:Performance_monitoring_interrupts 1333 ± 11% +41.8% 1890 ± 5% interrupts.CPU29.NMI:Non-maskable_interrupts 1333 ± 11% +41.8% 1890 ± 5% interrupts.CPU29.PMI:Performance_monitoring_interrupts 185.50 ± 34% +72.2% 319.50 ± 24% interrupts.CPU30.RES:Rescheduling_interrupts 1630 ± 25% +74.6% 2847 ± 39% interrupts.CPU33.NMI:Non-maskable_interrupts 1630 ± 25% +74.6% 2847 ± 39% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1346 ± 10% +149.7% 3361 ± 28% interrupts.CPU35.NMI:Non-maskable_interrupts 1346 ± 10% +149.7% 3361 ± 28% interrupts.CPU35.PMI:Performance_monitoring_interrupts 1371 ± 9% +66.9% 2289 ± 36% interrupts.CPU38.NMI:Non-maskable_interrupts 1371 ± 9% +66.9% 2289 ± 36% interrupts.CPU38.PMI:Performance_monitoring_interrupts 1319 ± 13% +144.1% 3219 ± 20% interrupts.CPU39.NMI:Non-maskable_interrupts 1319 ± 13% +144.1% 3219 ± 20% interrupts.CPU39.PMI:Performance_monitoring_interrupts 1351 ± 10% +35.3% 1828 ± 7% interrupts.CPU40.NMI:Non-maskable_interrupts 1351 ± 10% +35.3% 1828 ± 7% interrupts.CPU40.PMI:Performance_monitoring_interrupts 1635 ± 26% +71.2% 2799 ± 32% interrupts.CPU43.NMI:Non-maskable_interrupts 1635 ± 26% +71.2% 2799 ± 32% interrupts.CPU43.PMI:Performance_monitoring_interrupts 1654 ± 30% +94.8% 3221 ± 19% interrupts.CPU45.NMI:Non-maskable_interrupts 1654 ± 30% +94.8% 3221 ± 19% interrupts.CPU45.PMI:Performance_monitoring_interrupts 1355 ± 8% +108.1% 2820 ± 36% interrupts.CPU49.NMI:Non-maskable_interrupts 1355 ± 8% +108.1% 2820 ± 36% interrupts.CPU49.PMI:Performance_monitoring_interrupts 1755 ± 43% +60.0% 2807 ± 40% interrupts.CPU5.NMI:Non-maskable_interrupts 1755 ± 43% +60.0% 2807 ± 40% interrupts.CPU5.PMI:Performance_monitoring_interrupts 237.25 ± 71% +76.6% 419.00 ± 56% interrupts.CPU53.RES:Rescheduling_interrupts 1635 ± 31% +69.3% 2769 ± 30% interrupts.CPU59.NMI:Non-maskable_interrupts 1635 ± 31% +69.3% 2769 ± 30% interrupts.CPU59.PMI:Performance_monitoring_interrupts 199.00 ± 38% +83.8% 365.75 ± 25% interrupts.CPU6.RES:Rescheduling_interrupts 155.25 ± 28% +54.8% 240.25 ± 21% interrupts.CPU60.RES:Rescheduling_interrupts 1694 ± 40% +67.0% 2830 ± 39% interrupts.CPU66.NMI:Non-maskable_interrupts 1694 ± 40% +67.0% 2830 ± 39% interrupts.CPU66.PMI:Performance_monitoring_interrupts 1365 ± 10% +135.8% 3220 ± 21% interrupts.CPU67.NMI:Non-maskable_interrupts 1365 ± 10% +135.8% 3220 ± 21% interrupts.CPU67.PMI:Performance_monitoring_interrupts 1728 ± 43% +60.1% 2767 ± 38% interrupts.CPU69.NMI:Non-maskable_interrupts 1728 ± 43% +60.1% 2767 ± 38% interrupts.CPU69.PMI:Performance_monitoring_interrupts 214.25 ± 20% +133.0% 499.25 ± 41% interrupts.CPU74.RES:Rescheduling_interrupts 1332 ± 10% +35.7% 1807 ± 7% interrupts.CPU75.NMI:Non-maskable_interrupts 1332 ± 10% +35.7% 1807 ± 7% interrupts.CPU75.PMI:Performance_monitoring_interrupts 226.25 ± 22% +89.9% 429.75 ± 28% interrupts.CPU75.RES:Rescheduling_interrupts 1671 ± 31% +70.4% 2847 ± 38% interrupts.CPU76.NMI:Non-maskable_interrupts 1671 ± 31% +70.4% 2847 ± 38% interrupts.CPU76.PMI:Performance_monitoring_interrupts 1624 ± 27% +70.7% 2773 ± 28% interrupts.CPU8.NMI:Non-maskable_interrupts 1624 ± 27% +70.7% 2773 ± 28% interrupts.CPU8.PMI:Performance_monitoring_interrupts 161.00 ± 17% +65.1% 265.75 ± 13% interrupts.CPU83.RES:Rescheduling_interrupts 1689 ± 40% +93.8% 3273 ± 26% interrupts.CPU87.NMI:Non-maskable_interrupts 1689 ± 40% +93.8% 3273 ± 26% interrupts.CPU87.PMI:Performance_monitoring_interrupts 2409 ± 29% +54.1% 3713 ± 7% interrupts.CPU9.NMI:Non-maskable_interrupts 2409 ± 29% +54.1% 3713 ± 7% interrupts.CPU9.PMI:Performance_monitoring_interrupts 1346 ± 10% +37.0% 1843 ± 7% interrupts.CPU90.NMI:Non-maskable_interrupts 1346 ± 10% +37.0% 1843 ± 7% interrupts.CPU90.PMI:Performance_monitoring_interrupts 1323 ± 8% +71.7% 2272 ± 34% interrupts.CPU91.NMI:Non-maskable_interrupts 1323 ± 8% +71.7% 2272 ± 34% interrupts.CPU91.PMI:Performance_monitoring_interrupts 1309 ± 9% +111.1% 2763 ± 28% interrupts.CPU93.NMI:Non-maskable_interrupts 1309 ± 9% +111.1% 2763 ± 28% interrupts.CPU93.PMI:Performance_monitoring_interrupts 1360 ± 7% +140.5% 3272 ± 27% interrupts.CPU95.NMI:Non-maskable_interrupts 1360 ± 7% +140.5% 3272 ± 27% interrupts.CPU95.PMI:Performance_monitoring_interrupts 11.25 ± 36% +795.6% 100.75 ± 2% interrupts.IWI:IRQ_work_interrupts 487657 ± 8% +39.0% 677714 ± 7% interrupts.NMI:Non-maskable_interrupts 487657 ± 8% +39.0% 677714 ± 7% interrupts.PMI:Performance_monitoring_interrupts 18473 ± 4% -9.1% 16783 ± 3% softirqs.CPU1.SCHED 12341 ± 4% +27.8% 15777 ± 8% softirqs.CPU10.RCU 11067 ± 4% +12.5% 12447 ± 3% softirqs.CPU100.RCU 100554 ± 4% -10.5% 90007 ± 5% softirqs.CPU100.TIMER 10816 ± 6% +11.5% 12057 ± 3% softirqs.CPU101.RCU 11218 ± 5% +11.2% 12474 ± 3% softirqs.CPU102.RCU 10750 ± 4% +13.8% 12237 ± 3% softirqs.CPU103.RCU 17412 -8.2% 15988 ± 2% softirqs.CPU103.SCHED 98778 ± 5% -7.8% 91100 ± 5% softirqs.CPU103.TIMER 11004 ± 3% +12.4% 12364 softirqs.CPU104.RCU 100357 ± 4% -8.6% 91723 ± 3% softirqs.CPU104.TIMER 10713 ± 4% +14.3% 12249 ± 2% softirqs.CPU105.RCU 100357 ± 2% -8.3% 91987 ± 5% softirqs.CPU105.TIMER 10980 ± 3% +12.1% 12309 ± 3% softirqs.CPU106.RCU 99117 ± 4% -10.5% 88688 ± 5% softirqs.CPU106.TIMER 17299 -8.3% 15862 ± 2% softirqs.CPU107.SCHED 10848 ± 3% +13.5% 12309 softirqs.CPU108.RCU 99577 ± 4% -10.2% 89445 ± 2% softirqs.CPU108.TIMER 10803 ± 3% +11.0% 11990 ± 2% softirqs.CPU109.RCU 102128 ± 4% -10.3% 91597 ± 5% softirqs.CPU109.TIMER 104340 ± 7% -10.5% 93389 ± 4% softirqs.CPU110.TIMER 10871 ± 3% +13.5% 12341 softirqs.CPU112.RCU 98806 ± 5% -7.9% 91017 ± 5% softirqs.CPU113.TIMER 10973 +11.8% 12272 softirqs.CPU114.RCU 99081 ± 5% -8.4% 90768 ± 2% softirqs.CPU114.TIMER 100067 ± 2% -10.1% 89968 ± 4% softirqs.CPU116.TIMER 11169 +8.5% 12122 ± 3% softirqs.CPU117.RCU 99337 ± 3% -9.6% 89790 ± 4% softirqs.CPU117.TIMER 10921 ± 2% +20.1% 13117 ± 14% softirqs.CPU118.RCU 101560 ± 5% -10.9% 90457 softirqs.CPU119.TIMER 12478 +11.0% 13846 ± 3% softirqs.CPU12.RCU 101862 ± 2% -7.4% 94302 ± 5% softirqs.CPU120.TIMER 10891 ± 7% +8.8% 11851 ± 4% softirqs.CPU121.RCU 101820 ± 4% -13.8% 87815 ± 3% softirqs.CPU122.TIMER 98300 ± 2% -9.6% 88827 ± 4% softirqs.CPU123.TIMER 10686 ± 3% +16.3% 12423 ± 2% softirqs.CPU124.RCU 100237 ± 4% -12.4% 87782 ± 2% softirqs.CPU124.TIMER 10841 ± 2% +11.9% 12133 softirqs.CPU125.RCU 96727 ± 3% -8.1% 88932 ± 4% softirqs.CPU125.TIMER 10769 +12.0% 12065 ± 3% softirqs.CPU126.RCU 101522 -9.2% 92209 softirqs.CPU126.TIMER 10698 ± 5% +9.3% 11696 ± 3% softirqs.CPU127.RCU 17155 -7.0% 15959 ± 2% softirqs.CPU128.SCHED 98178 ± 5% -7.6% 90680 ± 4% softirqs.CPU128.TIMER 11033 +11.3% 12279 ± 2% softirqs.CPU129.RCU 10948 ± 5% +11.0% 12154 ± 3% softirqs.CPU130.RCU 102218 ± 4% -12.0% 89962 ± 3% softirqs.CPU130.TIMER 10758 ± 4% +19.4% 12841 ± 12% softirqs.CPU131.RCU 17352 -8.1% 15940 ± 2% softirqs.CPU131.SCHED 10875 ± 6% +12.4% 12219 ± 8% softirqs.CPU132.RCU 101841 -12.3% 89273 ± 4% softirqs.CPU132.TIMER 105128 ± 2% -13.7% 90677 ± 5% softirqs.CPU133.TIMER 10628 ± 3% +11.5% 11854 ± 3% softirqs.CPU137.RCU 10555 +13.8% 12007 ± 5% softirqs.CPU138.RCU 100887 ± 6% -10.5% 90303 ± 4% softirqs.CPU138.TIMER 99377 ± 3% -10.4% 89081 softirqs.CPU139.TIMER 91747 ± 5% -8.7% 83803 ± 3% softirqs.CPU14.TIMER 98185 ± 4% -10.7% 87710 ± 3% softirqs.CPU140.TIMER 10852 ± 6% +9.3% 11857 ± 4% softirqs.CPU141.RCU 98914 -12.2% 86818 ± 2% softirqs.CPU141.TIMER 10849 ± 3% +10.4% 11977 ± 4% softirqs.CPU146.RCU 17269 -7.7% 15934 ± 2% softirqs.CPU146.SCHED 10669 ± 4% +10.9% 11833 ± 3% softirqs.CPU149.RCU 96678 ± 3% -9.1% 87844 ± 3% softirqs.CPU150.TIMER 10419 ± 5% +15.1% 11996 ± 2% softirqs.CPU151.RCU 106517 ± 12% -18.0% 87311 ± 4% softirqs.CPU151.TIMER 102691 ± 3% -16.1% 86112 ± 2% softirqs.CPU152.TIMER 99114 ± 2% -12.9% 86365 ± 2% softirqs.CPU153.TIMER 10623 ± 4% +10.9% 11783 ± 3% softirqs.CPU154.RCU 98682 -11.8% 87079 ± 4% softirqs.CPU154.TIMER 10610 ± 3% +12.4% 11923 ± 5% softirqs.CPU155.RCU 100882 ± 3% -11.0% 89748 ± 3% softirqs.CPU155.TIMER 10748 ± 3% +10.4% 11868 ± 2% softirqs.CPU156.RCU 98452 ± 3% -6.8% 91739 softirqs.CPU156.TIMER 10823 ± 3% +9.7% 11871 ± 2% softirqs.CPU157.RCU 99080 ± 5% -10.9% 88286 ± 4% softirqs.CPU157.TIMER 17351 ± 2% -7.8% 16004 ± 2% softirqs.CPU158.SCHED 97048 ± 4% -8.9% 88371 ± 2% softirqs.CPU158.TIMER 10869 ± 3% +18.4% 12874 ± 13% softirqs.CPU159.RCU 94609 ± 4% -12.2% 83090 ± 3% softirqs.CPU16.TIMER 10723 ± 4% +18.4% 12700 ± 11% softirqs.CPU161.RCU 99838 ± 4% -10.6% 89279 ± 3% softirqs.CPU162.TIMER 10520 ± 2% +14.2% 12012 ± 3% softirqs.CPU163.RCU 101533 ± 4% -9.6% 91781 ± 5% softirqs.CPU163.TIMER 10504 ± 3% +9.4% 11490 softirqs.CPU164.RCU 99576 ± 5% -10.7% 88957 ± 3% softirqs.CPU165.TIMER 10479 ± 5% +9.3% 11449 ± 4% softirqs.CPU166.RCU 10456 ± 4% +19.1% 12451 ± 9% softirqs.CPU167.RCU 10450 ± 3% +11.9% 11691 ± 5% softirqs.CPU168.RCU 97838 ± 4% -10.4% 87620 ± 3% softirqs.CPU168.TIMER 10694 ± 2% +11.5% 11929 ± 3% softirqs.CPU169.RCU 101328 ± 3% -11.2% 90019 ± 4% softirqs.CPU169.TIMER 10580 +11.9% 11843 ± 4% softirqs.CPU170.RCU 99084 ± 3% -10.2% 88993 ± 3% softirqs.CPU170.TIMER 10763 ± 3% +9.6% 11797 ± 4% softirqs.CPU171.RCU 17236 -7.1% 16009 ± 3% softirqs.CPU172.SCHED 102042 -10.1% 91686 ± 3% softirqs.CPU172.TIMER 17299 -7.7% 15961 ± 3% softirqs.CPU173.SCHED 97802 ± 2% -8.3% 89716 ± 3% softirqs.CPU173.TIMER 10536 ± 3% +10.5% 11642 ± 5% softirqs.CPU174.RCU 10545 ± 3% +10.2% 11625 ± 3% softirqs.CPU175.RCU 94357 ± 5% -8.6% 86239 ± 4% softirqs.CPU175.TIMER 10694 +10.2% 11783 ± 3% softirqs.CPU176.RCU 10363 ± 5% +13.6% 11769 ± 3% softirqs.CPU178.RCU 103549 ± 8% -13.3% 89754 ± 3% softirqs.CPU179.TIMER 93069 ± 4% -9.7% 84018 ± 3% softirqs.CPU18.TIMER 100587 ± 4% -8.8% 91714 ± 6% softirqs.CPU180.TIMER 98392 ± 4% -9.2% 89369 ± 3% softirqs.CPU181.TIMER 10442 +11.5% 11644 ± 3% softirqs.CPU182.RCU 10613 ± 3% +10.6% 11733 ± 2% softirqs.CPU183.RCU 100747 ± 3% -10.2% 90442 ± 2% softirqs.CPU183.TIMER 10617 ± 2% +11.9% 11879 softirqs.CPU184.RCU 98900 ± 3% -8.3% 90653 ± 2% softirqs.CPU185.TIMER 99019 ± 4% -9.5% 89629 ± 3% softirqs.CPU186.TIMER 101675 ± 2% -11.4% 90086 softirqs.CPU187.TIMER 10741 ± 4% +15.9% 12446 ± 7% softirqs.CPU188.RCU 17256 -8.6% 15774 ± 3% softirqs.CPU188.SCHED 10568 ± 4% +9.1% 11535 ± 3% softirqs.CPU189.RCU 17282 -8.4% 15832 ± 2% softirqs.CPU189.SCHED 99399 ± 5% -8.8% 90684 ± 2% softirqs.CPU189.TIMER 12793 ± 18% +20.3% 15392 ± 12% softirqs.CPU19.RCU 17380 -8.7% 15873 softirqs.CPU19.SCHED 93553 ± 4% -9.6% 84576 ± 3% softirqs.CPU19.TIMER 99431 -12.4% 87111 ± 3% softirqs.CPU190.TIMER 97115 ± 4% -10.8% 86645 ± 2% softirqs.CPU191.TIMER 101263 ± 2% -13.7% 87411 ± 2% softirqs.CPU192.TIMER 98911 ± 2% -9.7% 89270 ± 3% softirqs.CPU195.TIMER 97190 ± 5% -8.0% 89382 ± 5% softirqs.CPU196.TIMER 10576 ± 3% +15.4% 12204 ± 4% softirqs.CPU197.RCU 10664 ± 4% +10.3% 11766 softirqs.CPU198.RCU 17275 -7.0% 16069 ± 2% softirqs.CPU198.SCHED 102671 ± 4% -11.8% 90567 ± 2% softirqs.CPU198.TIMER 17149 ± 2% -6.9% 15964 ± 2% softirqs.CPU199.SCHED 17817 ± 2% -9.5% 16127 ± 2% softirqs.CPU2.SCHED 17228 ± 2% -7.4% 15950 ± 2% softirqs.CPU20.SCHED 10415 ± 4% +18.4% 12331 ± 11% softirqs.CPU200.RCU 100544 ± 3% -13.0% 87434 ± 3% softirqs.CPU201.TIMER 10629 ± 5% +12.7% 11983 softirqs.CPU202.RCU 10338 ± 2% +12.9% 11675 softirqs.CPU203.RCU 97420 ± 5% -8.0% 89649 softirqs.CPU203.TIMER 10502 ± 2% +16.6% 12246 ± 4% softirqs.CPU206.RCU 100171 ± 6% -11.0% 89102 ± 2% softirqs.CPU206.TIMER 10398 +23.6% 12847 ± 17% softirqs.CPU207.RCU 98195 ± 3% -8.4% 89966 ± 3% softirqs.CPU207.TIMER 10402 ± 6% +11.9% 11639 softirqs.CPU208.RCU 100257 ± 3% -10.9% 89343 ± 2% softirqs.CPU208.TIMER 10345 ± 6% +14.8% 11874 ± 2% softirqs.CPU209.RCU 100699 ± 4% -11.9% 88676 ± 2% softirqs.CPU209.TIMER 12393 +18.7% 14710 ± 9% softirqs.CPU21.RCU 10473 ± 2% +9.6% 11480 ± 4% softirqs.CPU210.RCU 10446 ± 6% +20.8% 12624 ± 14% softirqs.CPU211.RCU 10389 ± 4% +13.5% 11789 ± 6% softirqs.CPU212.RCU 95528 ± 4% -7.6% 88305 ± 3% softirqs.CPU213.TIMER 10661 ± 2% +11.0% 11837 ± 4% softirqs.CPU214.RCU 97571 ± 4% -8.2% 89610 ± 3% softirqs.CPU214.TIMER 10556 ± 3% +21.8% 12861 ± 13% softirqs.CPU216.RCU 10299 ± 4% +29.0% 13287 ± 23% softirqs.CPU217.RCU 97938 ± 2% -9.7% 88474 ± 5% softirqs.CPU217.TIMER 96514 ± 2% -9.9% 86964 ± 4% softirqs.CPU218.TIMER 102617 ± 6% -15.9% 86253 ± 2% softirqs.CPU219.TIMER 17430 ± 2% -8.4% 15958 softirqs.CPU22.SCHED 93730 ± 5% -9.6% 84700 ± 3% softirqs.CPU22.TIMER 99964 ± 2% -11.1% 88893 ± 6% softirqs.CPU220.TIMER 98620 ± 2% -9.6% 89154 ± 5% softirqs.CPU221.TIMER 10342 ± 2% +14.6% 11852 ± 8% softirqs.CPU222.RCU 105885 ± 7% -15.7% 89275 ± 4% softirqs.CPU223.TIMER 17573 ± 3% -8.9% 16016 softirqs.CPU224.SCHED 97387 ± 4% -9.1% 88529 ± 3% softirqs.CPU224.TIMER 97157 ± 4% -9.5% 87900 ± 2% softirqs.CPU225.TIMER 9986 ± 6% +15.7% 11557 ± 2% softirqs.CPU226.RCU 17215 ± 2% -7.1% 15985 ± 2% softirqs.CPU226.SCHED 98264 ± 2% -10.7% 87721 softirqs.CPU226.TIMER 17221 -7.7% 15902 ± 2% softirqs.CPU227.SCHED 96092 ± 3% -7.3% 89092 ± 2% softirqs.CPU227.TIMER 10251 ± 5% +21.9% 12493 ± 16% softirqs.CPU228.RCU 100504 ± 6% -9.3% 91135 ± 3% softirqs.CPU228.TIMER 100469 ± 2% -9.4% 91059 ± 3% softirqs.CPU229.TIMER 12791 ± 4% +20.8% 15448 ± 11% softirqs.CPU23.RCU 17558 ± 2% -9.4% 15900 ± 2% softirqs.CPU23.SCHED 93218 ± 6% -10.4% 83499 softirqs.CPU23.TIMER 98247 ± 4% -7.1% 91276 ± 4% softirqs.CPU230.TIMER 100951 ± 5% -10.5% 90402 ± 3% softirqs.CPU231.TIMER 105368 ± 4% -11.6% 93185 ± 2% softirqs.CPU232.TIMER 10369 ± 5% +11.5% 11561 ± 2% softirqs.CPU233.RCU 10431 ± 5% +10.4% 11518 ± 4% softirqs.CPU234.RCU 10412 ± 2% +13.4% 11808 ± 5% softirqs.CPU235.RCU 97291 ± 5% -12.0% 85630 ± 3% softirqs.CPU236.TIMER 97723 ± 3% -10.1% 87826 ± 2% softirqs.CPU237.TIMER 96603 ± 3% -9.9% 87038 ± 3% softirqs.CPU238.TIMER 10537 ± 3% +11.6% 11765 ± 2% softirqs.CPU239.RCU 94484 ± 5% -9.5% 85466 softirqs.CPU24.TIMER 10167 ± 3% +13.8% 11570 ± 5% softirqs.CPU240.RCU 97538 ± 4% -9.5% 88305 ± 3% softirqs.CPU240.TIMER 10183 ± 3% +14.7% 11680 ± 3% softirqs.CPU241.RCU 10608 ± 3% +11.9% 11867 ± 3% softirqs.CPU242.RCU 96373 ± 4% -9.3% 87408 ± 3% softirqs.CPU242.TIMER 10316 ± 3% +10.9% 11440 ± 3% softirqs.CPU243.RCU 99319 ± 2% -9.4% 90023 ± 3% softirqs.CPU244.TIMER 17282 -7.6% 15973 softirqs.CPU245.SCHED 99826 ± 4% -9.4% 90484 ± 4% softirqs.CPU245.TIMER 17092 ± 3% -6.2% 16029 ± 2% softirqs.CPU246.SCHED 10225 +15.1% 11771 ± 2% softirqs.CPU247.RCU 99806 ± 10% -11.8% 87994 ± 3% softirqs.CPU247.TIMER 10182 ± 2% +12.3% 11433 ± 2% softirqs.CPU248.RCU 17205 ± 2% -7.6% 15893 ± 2% softirqs.CPU249.SCHED 17205 -7.6% 15894 ± 2% softirqs.CPU25.SCHED 10473 ± 2% +13.7% 11908 ± 2% softirqs.CPU250.RCU 96253 ± 3% -7.2% 89343 ± 2% softirqs.CPU252.TIMER 100816 ± 4% -11.2% 89491 softirqs.CPU254.TIMER 10138 +11.8% 11335 softirqs.CPU255.RCU 10912 ± 3% +11.5% 12169 ± 6% softirqs.CPU257.RCU 11075 ± 2% +8.8% 12053 softirqs.CPU258.RCU 92922 ± 4% -11.1% 82595 softirqs.CPU259.TIMER 17374 -9.6% 15710 ± 5% softirqs.CPU260.SCHED 92766 ± 5% -12.0% 81644 softirqs.CPU260.TIMER 10948 ± 4% +20.1% 13148 ± 12% softirqs.CPU261.RCU 92608 ± 5% -10.3% 83083 ± 3% softirqs.CPU263.TIMER 11249 ± 3% +8.9% 12249 ± 3% softirqs.CPU264.RCU 92996 ± 5% -9.0% 84582 ± 2% softirqs.CPU265.TIMER 105673 ± 24% -22.0% 82442 ± 3% softirqs.CPU266.TIMER 10781 ± 4% +23.9% 13356 ± 12% softirqs.CPU267.RCU 94124 ± 4% -12.1% 82722 ± 3% softirqs.CPU268.TIMER 10629 ± 2% +23.2% 13098 ± 11% softirqs.CPU269.RCU 93065 ± 4% -10.7% 83153 ± 4% softirqs.CPU269.TIMER 12153 +19.6% 14529 ± 10% softirqs.CPU27.RCU 17327 -8.3% 15897 ± 2% softirqs.CPU27.SCHED 11736 ± 4% +22.9% 14423 ± 9% softirqs.CPU28.RCU 12155 ± 4% +22.7% 14913 ± 9% softirqs.CPU29.RCU 17339 -8.3% 15907 ± 2% softirqs.CPU29.SCHED 17457 -9.2% 15859 ± 2% softirqs.CPU3.SCHED 12285 ± 2% +12.2% 13782 ± 2% softirqs.CPU30.RCU 92849 ± 6% -7.6% 85779 ± 3% softirqs.CPU30.TIMER 17343 -8.4% 15880 ± 2% softirqs.CPU31.SCHED 11863 ± 3% +21.9% 14462 ± 13% softirqs.CPU32.RCU 12882 ± 13% +18.6% 15282 ± 13% softirqs.CPU34.RCU 17418 -8.0% 16022 ± 2% softirqs.CPU34.SCHED 93617 ± 5% -10.7% 83595 ± 2% softirqs.CPU34.TIMER 17544 -8.9% 15979 ± 2% softirqs.CPU35.SCHED 97042 ± 4% -10.0% 87319 ± 2% softirqs.CPU36.TIMER 95017 ± 4% -9.2% 86260 ± 2% softirqs.CPU37.TIMER 12131 ± 3% +17.2% 14215 ± 11% softirqs.CPU39.RCU 17422 -8.8% 15881 ± 2% softirqs.CPU39.SCHED 93400 ± 5% -8.1% 85878 ± 3% softirqs.CPU39.TIMER 13358 ± 12% +19.6% 15970 ± 16% softirqs.CPU4.RCU 94103 ± 5% -7.5% 87045 ± 2% softirqs.CPU40.TIMER 17296 -8.0% 15915 ± 2% softirqs.CPU41.SCHED 97200 ± 11% -12.6% 84905 ± 2% softirqs.CPU42.TIMER 11828 +22.9% 14539 ± 12% softirqs.CPU44.RCU 93513 ± 5% -8.5% 85542 ± 3% softirqs.CPU45.TIMER 95798 ± 4% -9.4% 86827 ± 2% softirqs.CPU49.TIMER 12414 ± 3% +21.7% 15109 ± 12% softirqs.CPU5.RCU 94612 ± 4% -10.7% 84449 ± 3% softirqs.CPU5.TIMER 96294 ± 7% -11.2% 85463 softirqs.CPU50.TIMER 12988 ± 15% +17.2% 15216 ± 12% softirqs.CPU51.RCU 17393 -8.0% 15999 ± 2% softirqs.CPU51.SCHED 96788 ± 4% -9.3% 87807 ± 2% softirqs.CPU51.TIMER 94045 ± 7% -12.0% 82722 softirqs.CPU52.TIMER 17466 -10.6% 15611 softirqs.CPU53.SCHED 12090 ± 2% +16.9% 14132 ± 12% softirqs.CPU55.RCU 12064 ± 3% +18.2% 14262 ± 11% softirqs.CPU56.RCU 93720 ± 5% -10.6% 83755 softirqs.CPU56.TIMER 17256 -8.0% 15879 ± 2% softirqs.CPU58.SCHED 94304 ± 5% -10.4% 84519 ± 3% softirqs.CPU59.TIMER 11958 +11.4% 13323 ± 4% softirqs.CPU60.RCU 11877 ± 2% +14.1% 13551 softirqs.CPU62.RCU 17256 ± 2% -7.6% 15953 softirqs.CPU62.SCHED 17213 ± 2% -7.1% 15984 ± 2% softirqs.CPU63.SCHED 17292 -7.9% 15925 softirqs.CPU64.SCHED 101158 ± 5% -13.5% 87518 ± 5% softirqs.CPU64.TIMER 11243 ± 3% +10.9% 12468 ± 5% softirqs.CPU65.RCU 100712 ± 3% -13.0% 87636 ± 3% softirqs.CPU65.TIMER 11245 ± 3% +11.5% 12541 ± 3% softirqs.CPU66.RCU 99280 ± 4% -11.6% 87750 ± 2% softirqs.CPU68.TIMER 11162 ± 2% +23.0% 13732 ± 19% softirqs.CPU69.RCU 17629 -8.5% 16127 ± 4% softirqs.CPU70.SCHED 101359 ± 5% -9.4% 91824 ± 5% softirqs.CPU70.TIMER 11320 +11.9% 12666 ± 4% softirqs.CPU71.RCU 100833 ± 6% -9.9% 90861 ± 3% softirqs.CPU71.TIMER 11074 +9.8% 12162 ± 2% softirqs.CPU72.RCU 11156 ± 3% +11.1% 12392 ± 3% softirqs.CPU73.RCU 100871 ± 3% -11.4% 89375 ± 5% softirqs.CPU73.TIMER 11326 ± 4% +9.3% 12375 ± 2% softirqs.CPU74.RCU 102866 ± 3% -13.0% 89443 ± 3% softirqs.CPU74.TIMER 10983 ± 5% +17.7% 12927 ± 2% softirqs.CPU75.RCU 11060 ± 3% +13.7% 12571 ± 2% softirqs.CPU77.RCU 99193 ± 4% -10.7% 88602 ± 2% softirqs.CPU78.TIMER 11133 ± 6% +11.1% 12370 ± 3% softirqs.CPU79.RCU 101991 -12.0% 89748 ± 2% softirqs.CPU79.TIMER 10824 ± 5% +11.9% 12109 ± 2% softirqs.CPU80.RCU 17260 -8.0% 15881 ± 2% softirqs.CPU80.SCHED 100029 ± 3% -8.9% 91159 ± 5% softirqs.CPU80.TIMER 11022 +11.6% 12296 ± 2% softirqs.CPU81.RCU 97866 ± 2% -9.5% 88538 ± 3% softirqs.CPU82.TIMER 10667 ± 4% +23.6% 13188 ± 13% softirqs.CPU83.RCU 96970 ± 2% -10.0% 87321 ± 3% softirqs.CPU83.TIMER 99683 ± 3% -11.8% 87933 ± 6% softirqs.CPU84.TIMER 101844 ± 2% -12.5% 89143 ± 5% softirqs.CPU85.TIMER 10809 +19.9% 12959 ± 12% softirqs.CPU86.RCU 99247 ± 3% -7.5% 91761 ± 2% softirqs.CPU87.TIMER 11015 ± 4% +15.4% 12712 ± 6% softirqs.CPU88.RCU 10994 +9.1% 11992 ± 3% softirqs.CPU89.RCU 12299 ± 4% +25.2% 15402 ± 12% softirqs.CPU9.RCU 17485 -8.8% 15943 ± 2% softirqs.CPU9.SCHED 10979 ± 5% +13.0% 12411 ± 7% softirqs.CPU90.RCU 100345 ± 3% -8.4% 91879 ± 2% softirqs.CPU90.TIMER 10973 ± 3% +14.8% 12598 ± 7% softirqs.CPU91.RCU 98869 ± 3% -8.6% 90336 ± 2% softirqs.CPU91.TIMER 17454 -9.3% 15830 softirqs.CPU92.SCHED 101000 ± 4% -11.4% 89514 ± 2% softirqs.CPU92.TIMER 11043 ± 3% +18.2% 13054 ± 9% softirqs.CPU94.RCU 10978 ± 6% +10.8% 12161 ± 5% softirqs.CPU98.RCU 10816 ± 5% +12.7% 12188 ± 4% softirqs.CPU99.RCU 3090798 ± 3% +10.6% 3418176 ± 2% softirqs.RCU 26564794 ± 3% -9.1% 24149339 ± 2% softirqs.TIMER *************************************************************************************************** lkp-skl-4sp1: 192 threads Skylake-4S with 704G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-skl-4sp1/page_fault2/will-it-scale commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -17% 1:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry 3:4 -16% 2:4 perf-profile.calltrace.cycles-pp.error_entry :4 27% 1:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry.testcase 0:4 53% 2:4 perf-profile.calltrace.cycles-pp.error_entry.testcase %stddev %change %stddev \ | \ 39778 -51.9% 19128 will-it-scale.per_process_ops 7637514 -51.9% 3672843 will-it-scale.workload 3349 ± 4% +121.1% 7406 ± 9% kthread_noise.total_time 14704 ± 4% -13.5% 12719 ± 8% slabinfo.skbuff_head_cache.active_objs 475824 ± 8% +41.7% 674399 ± 4% softirqs.RCU 0.01 ± 9% -0.0 0.01 ± 14% mpstat.cpu.soft% 2.12 ± 2% +2.7 4.78 ± 5% mpstat.cpu.usr% 95.75 -2.1% 93.75 vmstat.cpu.sy 3900 +26.8% 4947 ± 11% vmstat.system.cs 326838 ± 11% +19.1% 389273 ± 2% vmstat.system.in 327905 ± 22% -50.4% 162574 ± 4% meminfo.DirectMap4k 40230471 +47.8% 59449219 meminfo.Memused 59325 -8.6% 54225 ± 2% meminfo.PageTables 181968 ± 2% +41.3% 257034 meminfo.max_used_kB 0.69 ± 91% -76.8% 0.16 ± 27% turbostat.CPU%c1 98288699 ± 10% +16.4% 1.144e+08 ± 3% turbostat.IRQ 435.85 -2.7% 424.24 turbostat.PkgWatt 250.39 -16.6% 208.71 turbostat.RAMWatt 6.616e+08 -87.0% 85682633 ± 3% numa-numastat.node2.local_node 6.616e+08 -87.0% 85718836 ± 3% numa-numastat.node2.numa_hit 25355 ± 25% +41.3% 35836 numa-numastat.node2.other_node 6.735e+08 -87.3% 85754685 ± 2% numa-numastat.node3.local_node 6.736e+08 -87.3% 85781904 ± 2% numa-numastat.node3.numa_hit 10822 ± 3% -9.1% 9835 ± 6% numa-vmstat.node0.nr_kernel_stack 5995 ± 6% -29.6% 4220 ± 26% numa-vmstat.node0.nr_page_table_pages 20792 ± 9% -11.9% 18324 ± 8% numa-vmstat.node0.nr_slab_unreclaimable 724842 +11.6% 808719 ± 2% numa-vmstat.node2.nr_anon_pages 2.734e+08 ± 2% -83.5% 45181715 ± 3% numa-vmstat.node2.numa_hit 2.733e+08 ± 2% -83.5% 45057604 ± 3% numa-vmstat.node2.numa_local 2858 ± 7% -17.0% 2372 ± 3% numa-vmstat.node3.nr_page_table_pages 2.777e+08 ± 3% -83.8% 44940277 ± 2% numa-vmstat.node3.numa_hit 2.776e+08 ± 3% -83.9% 44824813 ± 2% numa-vmstat.node3.numa_local 81377 ± 25% -71.0% 23630 ± 58% numa-meminfo.node0.AnonHugePages 10875 ± 2% -9.6% 9833 ± 6% numa-meminfo.node0.KernelStack 10291170 +40.4% 14451454 ± 2% numa-meminfo.node0.MemUsed 24263 ± 6% -30.4% 16895 ± 25% numa-meminfo.node0.PageTables 83135 ± 9% -11.8% 73303 ± 8% numa-meminfo.node0.SUnreclaim 9927685 +42.8% 14179566 ± 4% numa-meminfo.node1.MemUsed 2895675 +11.7% 3234188 ± 2% numa-meminfo.node2.AnonPages 10008832 +51.9% 15203532 ± 6% numa-meminfo.node2.MemUsed 15173 ±173% +266.8% 55658 ± 63% numa-meminfo.node3.AnonHugePages 9948513 +56.1% 15525891 ± 6% numa-meminfo.node3.MemUsed 11422 ± 7% -16.6% 9522 ± 2% numa-meminfo.node3.PageTables 2928712 +6.5% 3118835 proc-vmstat.nr_anon_pages 17068511 -2.8% 16589653 proc-vmstat.nr_dirty_background_threshold 34178822 -2.8% 33219927 proc-vmstat.nr_dirty_threshold 1.715e+08 -2.8% 1.667e+08 proc-vmstat.nr_free_pages 14817 -8.5% 13561 ± 2% proc-vmstat.nr_page_table_pages 250.25 ± 53% +1360.6% 3655 ± 73% proc-vmstat.numa_hint_faults 78.75 ± 54% +3793.7% 3066 ± 91% proc-vmstat.numa_hint_faults_local 2.286e+09 -51.4% 1.112e+09 proc-vmstat.numa_hit 2.285e+09 -51.4% 1.111e+09 proc-vmstat.numa_local 39159 -53.7% 18136 ± 66% proc-vmstat.pgactivate 2.287e+09 -51.3% 1.114e+09 proc-vmstat.pgalloc_normal 2.274e+09 -51.4% 1.104e+09 proc-vmstat.pgfault 2.284e+09 -51.5% 1.107e+09 proc-vmstat.pgfree 34.57 -34.6 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault 34.56 -34.6 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma 36.41 -32.7 3.67 ± 2% perf-profile.calltrace.cycles-pp.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 36.20 -32.7 3.50 ± 2% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault.__do_page_fault 36.03 -32.6 3.40 ± 2% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault.handle_mm_fault 31.50 ± 3% -31.5 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu_free 31.50 ± 3% -31.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages 29.57 ± 3% -29.6 0.00 perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 29.17 ± 3% -29.2 0.00 perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu_free.unmap_page_range 37.36 ± 2% -22.1 15.30 ± 2% perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 37.35 ± 2% -22.1 15.30 ± 2% perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 30.77 ± 2% -19.9 10.90 ± 2% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region 31.14 ± 2% -19.6 11.59 ± 2% perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region.__do_munmap 33.17 ± 2% -19.4 13.80 ± 2% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 33.18 ± 2% -19.4 13.82 ± 2% perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 4.10 ± 2% -2.7 1.38 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region 4.17 ± 2% -2.7 1.45 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 4.17 ± 2% -2.7 1.45 perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 4.14 ± 2% -2.7 1.44 perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.unmap_region.__do_munmap 0.00 +0.5 0.54 ± 3% perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 0.00 +0.6 0.55 ± 2% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 0.00 +0.7 0.66 ± 2% perf-profile.calltrace.cycles-pp.xas_load.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault 0.00 +0.7 0.67 ± 2% perf-profile.calltrace.cycles-pp.free_pages_and_swap_cache.tlb_flush_mmu_free.unmap_page_range.unmap_vmas.unmap_region 2.10 ± 5% +1.1 3.20 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 0.00 +1.2 1.22 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu 0.00 +1.2 1.22 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu 0.81 ± 9% +2.1 2.93 ± 3% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.__handle_mm_fault 1.42 ± 11% +2.2 3.64 perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 1.18 ± 11% +2.2 3.41 perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault 1.12 ± 11% +2.2 3.36 perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault 0.93 ± 12% +2.3 3.18 perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault 1.29 ± 11% +2.3 3.55 perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.49 ± 58% +9.2 9.66 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range 0.49 ± 58% +9.2 9.67 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu_free.unmap_page_range.unmap_vmas 0.00 +9.3 9.31 ± 13% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.copy_page.__handle_mm_fault.handle_mm_fault.__do_page_fault 1.07 ± 15% +18.4 19.45 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte 1.09 ± 15% +18.4 19.48 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault 1.84 ± 3% +18.7 20.57 ± 7% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault 1.92 ± 2% +18.8 20.68 ± 7% perf-profile.calltrace.cycles-pp.__lru_cache_add.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault 58.16 +20.2 78.37 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 4.65 ± 3% +20.7 25.32 ± 5% perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault 4.68 ± 3% +20.8 25.52 ± 5% perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 14.70 ± 7% +22.3 36.97 ± 2% perf-profile.calltrace.cycles-pp.copy_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 24.18 +10.5% 26.72 perf-stat.i.MPKI 7.795e+09 -55.3% 3.486e+09 perf-stat.i.branch-instructions 0.56 ± 2% +0.1 0.67 ± 2% perf-stat.i.branch-miss-rate% 41969684 -42.7% 24028297 ± 2% perf-stat.i.branch-misses 6.355e+08 -51.9% 3.059e+08 perf-stat.i.cache-misses 9.595e+08 -51.4% 4.666e+08 perf-stat.i.cache-references 3566 ± 4% +31.5% 4689 ± 12% perf-stat.i.context-switches 12.19 ± 2% +94.6% 23.74 perf-stat.i.cpi 61.94 +8.5% 67.18 ± 2% perf-stat.i.cpu-migrations 822.12 ± 4% +66.5% 1369 perf-stat.i.cycles-between-cache-misses 3902442 ± 15% -59.2% 1590249 ± 60% perf-stat.i.dTLB-load-misses 1.053e+10 -53.9% 4.855e+09 ± 2% perf-stat.i.dTLB-loads 1.20 +0.2 1.36 perf-stat.i.dTLB-store-miss-rate% 62889409 -47.1% 33243935 ± 2% perf-stat.i.dTLB-store-misses 5.225e+09 -53.0% 2.457e+09 perf-stat.i.dTLB-stores 15595687 -53.1% 7309864 ± 3% perf-stat.i.iTLB-load-misses 1.001e+08 ± 11% -45.3% 54768985 ± 16% perf-stat.i.iTLB-loads 3.918e+10 -55.0% 1.763e+10 perf-stat.i.instructions 2856 ± 17% -15.6% 2410 ± 3% perf-stat.i.instructions-per-iTLB-miss 0.10 ± 3% -55.3% 0.04 perf-stat.i.ipc 7620320 -52.0% 3659097 perf-stat.i.minor-faults 97.59 -4.6 92.96 ± 2% perf-stat.i.node-load-miss-rate% 184977 +17.0% 216342 ± 5% perf-stat.i.node-load-misses 4347 ± 12% +265.0% 15871 ± 37% perf-stat.i.node-loads 97.50 -6.6 90.94 perf-stat.i.node-store-miss-rate% 913177 ± 2% -64.8% 321857 ± 2% perf-stat.i.node-store-misses 17816 ± 6% +79.2% 31934 ± 10% perf-stat.i.node-stores 7632782 -52.0% 3661552 perf-stat.i.page-faults 24.48 +8.1% 26.47 perf-stat.overall.MPKI 0.54 +0.2 0.69 ± 2% perf-stat.overall.branch-miss-rate% 66.24 -0.7 65.56 perf-stat.overall.cache-miss-rate% 10.84 +116.7% 23.48 perf-stat.overall.cpi 668.30 +102.5% 1353 perf-stat.overall.cycles-between-cache-misses 1.19 +0.1 1.34 perf-stat.overall.dTLB-store-miss-rate% 0.09 -53.8% 0.04 perf-stat.overall.ipc 97.64 -4.5 93.17 ± 2% perf-stat.overall.node-load-miss-rate% 98.03 -7.1 90.97 perf-stat.overall.node-store-miss-rate% 1521066 -5.5% 1437723 perf-stat.overall.path-length 7.557e+09 -54.3% 3.453e+09 perf-stat.ps.branch-instructions 40556622 -41.4% 23747265 ± 2% perf-stat.ps.branch-misses 6.155e+08 -50.8% 3.031e+08 perf-stat.ps.cache-misses 9.293e+08 -50.3% 4.623e+08 perf-stat.ps.cache-references 3682 +26.3% 4649 ± 12% perf-stat.ps.context-switches 62.18 +7.5% 66.85 ± 2% perf-stat.ps.cpu-migrations 3783897 ± 14% -58.3% 1576456 ± 60% perf-stat.ps.dTLB-load-misses 1.02e+10 -52.8% 4.81e+09 ± 3% perf-stat.ps.dTLB-loads 60936451 -45.9% 32963378 ± 2% perf-stat.ps.dTLB-store-misses 5.049e+09 -51.8% 2.435e+09 perf-stat.ps.dTLB-stores 15088884 -51.9% 7251206 ± 3% perf-stat.ps.iTLB-load-misses 92844515 ± 12% -41.1% 54719539 ± 15% perf-stat.ps.iTLB-loads 3.796e+10 -54.0% 1.747e+10 perf-stat.ps.instructions 7411382 -50.9% 3639119 perf-stat.ps.minor-faults 181409 +18.3% 214687 ± 5% perf-stat.ps.node-load-misses 4374 ± 9% +260.3% 15763 ± 37% perf-stat.ps.node-loads 882207 -63.8% 319321 ± 2% perf-stat.ps.node-store-misses 17754 ± 5% +78.5% 31698 ± 10% perf-stat.ps.node-stores 7408109 -50.9% 3637653 perf-stat.ps.page-faults 1.162e+13 -54.5% 5.281e+12 perf-stat.total.instructions 409568 ± 22% -56.9% 176342 ± 44% sched_debug.cfs_rq:/.MIN_vruntime.avg 2867615 ± 23% -36.1% 1832651 ± 31% sched_debug.cfs_rq:/.MIN_vruntime.stddev 151792 -17.1% 125862 sched_debug.cfs_rq:/.exec_clock.avg 152869 -16.5% 127587 sched_debug.cfs_rq:/.exec_clock.max 134970 ± 2% -24.0% 102544 ± 9% sched_debug.cfs_rq:/.exec_clock.min 19810 ± 9% -49.1% 10083 ± 22% sched_debug.cfs_rq:/.load.avg 107777 ± 10% -42.8% 61624 ± 34% sched_debug.cfs_rq:/.load.stddev 22.62 ± 9% -42.2% 13.07 ± 14% sched_debug.cfs_rq:/.load_avg.avg 1001 ± 35% -41.3% 588.05 ± 13% sched_debug.cfs_rq:/.load_avg.max 100.11 ± 28% -43.8% 56.23 ± 15% sched_debug.cfs_rq:/.load_avg.stddev 409568 ± 22% -56.9% 176342 ± 44% sched_debug.cfs_rq:/.max_vruntime.avg 2867615 ± 23% -36.1% 1832651 ± 31% sched_debug.cfs_rq:/.max_vruntime.stddev 30743045 -17.3% 25416045 sched_debug.cfs_rq:/.min_vruntime.avg 31155455 -16.5% 26018339 sched_debug.cfs_rq:/.min_vruntime.max 25797828 ± 2% -24.1% 19576607 ± 9% sched_debug.cfs_rq:/.min_vruntime.min 0.16 ± 17% -36.6% 0.10 ± 13% sched_debug.cfs_rq:/.nr_running.stddev 107.36 -60.3% 42.57 sched_debug.cfs_rq:/.nr_spread_over.avg 572.29 ± 7% -48.9% 292.55 ± 14% sched_debug.cfs_rq:/.nr_spread_over.max 37.88 ± 13% -80.5% 7.40 ± 14% sched_debug.cfs_rq:/.nr_spread_over.min 52.36 ± 11% -31.5% 35.88 ± 7% sched_debug.cfs_rq:/.nr_spread_over.stddev 12.34 ± 7% -49.7% 6.21 ± 15% sched_debug.cfs_rq:/.runnable_load_avg.avg 644.75 ± 24% -53.5% 299.60 ± 35% sched_debug.cfs_rq:/.runnable_load_avg.max 62.25 ± 20% -62.9% 23.12 ± 37% sched_debug.cfs_rq:/.runnable_load_avg.stddev 19778 ± 9% -49.1% 10063 ± 22% sched_debug.cfs_rq:/.runnable_weight.avg 107739 ± 10% -42.8% 61590 ± 34% sched_debug.cfs_rq:/.runnable_weight.stddev 206772 -11.3% 183434 ± 2% sched_debug.cpu.clock.avg 207080 -10.8% 184765 ± 2% sched_debug.cpu.clock.max 206458 -11.9% 181908 ± 2% sched_debug.cpu.clock.min 174.19 ± 8% +373.0% 823.94 ± 3% sched_debug.cpu.clock.stddev 206772 -11.3% 183434 ± 2% sched_debug.cpu.clock_task.avg 207080 -10.8% 184765 ± 2% sched_debug.cpu.clock_task.max 206458 -11.9% 181908 ± 2% sched_debug.cpu.clock_task.min 174.19 ± 8% +373.0% 823.94 ± 3% sched_debug.cpu.clock_task.stddev 12.44 ± 6% -47.3% 6.55 ± 15% sched_debug.cpu.cpu_load[0].avg 62.44 ± 19% -55.5% 27.76 ± 45% sched_debug.cpu.cpu_load[0].stddev 12.53 ± 6% -49.6% 6.32 ± 12% sched_debug.cpu.cpu_load[1].avg 62.05 ± 19% -61.2% 24.08 ± 38% sched_debug.cpu.cpu_load[1].stddev 12.54 ± 6% -50.4% 6.22 ± 12% sched_debug.cpu.cpu_load[2].avg 642.71 ± 23% -54.3% 294.00 ± 35% sched_debug.cpu.cpu_load[2].max 61.30 ± 19% -63.5% 22.40 ± 33% sched_debug.cpu.cpu_load[2].stddev 12.41 ± 6% -50.6% 6.13 ± 11% sched_debug.cpu.cpu_load[3].avg 633.54 ± 23% -56.0% 278.85 ± 33% sched_debug.cpu.cpu_load[3].max 59.25 ± 19% -64.5% 21.03 ± 32% sched_debug.cpu.cpu_load[3].stddev 12.14 ± 9% -49.8% 6.10 ± 10% sched_debug.cpu.cpu_load[4].avg 625.96 ± 24% -57.0% 269.25 ± 33% sched_debug.cpu.cpu_load[4].max 56.61 ± 21% -64.1% 20.35 ± 32% sched_debug.cpu.cpu_load[4].stddev 10828 ± 6% -12.9% 9432 sched_debug.cpu.curr->pid.max 20492 ± 10% -52.2% 9803 ± 28% sched_debug.cpu.load.avg 112135 ± 8% -48.6% 57672 ± 52% sched_debug.cpu.load.stddev 1239001 ± 21% +44.8% 1794100 ± 13% sched_debug.cpu.max_idle_balance_cost.max 73492 ± 34% +81.9% 133652 ± 26% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 6% +183.2% 0.00 ± 5% sched_debug.cpu.next_balance.stddev 160939 -14.4% 137736 sched_debug.cpu.nr_load_updates.avg 172859 -14.3% 148136 sched_debug.cpu.nr_load_updates.max 157547 -15.8% 132645 sched_debug.cpu.nr_load_updates.min 2088 ± 26% -40.2% 1248 ± 10% sched_debug.cpu.nr_load_updates.stddev 142183 ± 24% +117.1% 308717 ± 34% sched_debug.cpu.nr_switches.max 541.42 -10.9% 482.60 sched_debug.cpu.nr_switches.min 15637 ± 12% +74.9% 27354 ± 31% sched_debug.cpu.nr_switches.stddev 141073 ± 25% +117.7% 307114 ± 34% sched_debug.cpu.sched_count.max 484.42 ± 6% -22.9% 373.60 ± 3% sched_debug.cpu.sched_count.min 15484 ± 12% +74.9% 27078 ± 32% sched_debug.cpu.sched_count.stddev 116.37 ± 6% -8.4% 106.60 ± 4% sched_debug.cpu.sched_goidle.avg 69996 ± 24% +119.4% 153559 ± 35% sched_debug.cpu.ttwu_count.max 216.92 -16.3% 181.55 sched_debug.cpu.ttwu_count.min 7680 ± 12% +76.2% 13536 ± 32% sched_debug.cpu.ttwu_count.stddev 69558 ± 24% +120.2% 153170 ± 35% sched_debug.cpu.ttwu_local.max 190.08 -16.5% 158.80 sched_debug.cpu.ttwu_local.min 7624 ± 12% +77.2% 13508 ± 32% sched_debug.cpu.ttwu_local.stddev 206449 -11.9% 181871 ± 2% sched_debug.cpu_clk 202520 -12.1% 177942 ± 2% sched_debug.ktime 2.52 -10.0% 2.27 sched_debug.rt_rq:/.rt_runtime.stddev 213727 -12.8% 186368 sched_debug.sched_clk 443866 ± 8% +27.0% 563493 ± 5% interrupts.CPU0.LOC:Local_timer_interrupts 441945 ± 7% +27.0% 561138 ± 6% interrupts.CPU1.LOC:Local_timer_interrupts 442168 ± 7% +26.8% 560709 ± 6% interrupts.CPU10.LOC:Local_timer_interrupts 441832 ± 7% +26.9% 560592 ± 6% interrupts.CPU100.LOC:Local_timer_interrupts 442346 ± 7% +26.7% 560650 ± 6% interrupts.CPU101.LOC:Local_timer_interrupts 441614 ± 7% +27.0% 560850 ± 6% interrupts.CPU102.LOC:Local_timer_interrupts 443184 ± 7% +26.5% 560499 ± 6% interrupts.CPU103.LOC:Local_timer_interrupts 441615 ± 7% +26.9% 560510 ± 6% interrupts.CPU104.LOC:Local_timer_interrupts 441529 ± 7% +26.9% 560472 ± 6% interrupts.CPU105.LOC:Local_timer_interrupts 441611 ± 7% +26.9% 560326 ± 6% interrupts.CPU106.LOC:Local_timer_interrupts 441886 ± 7% +26.9% 560746 ± 6% interrupts.CPU107.LOC:Local_timer_interrupts 444837 ± 8% +26.1% 560988 ± 6% interrupts.CPU108.LOC:Local_timer_interrupts 441552 ± 7% +27.0% 560746 ± 6% interrupts.CPU109.LOC:Local_timer_interrupts 442269 ± 7% +26.8% 560919 ± 6% interrupts.CPU11.LOC:Local_timer_interrupts 441617 ± 7% +26.9% 560549 ± 6% interrupts.CPU110.LOC:Local_timer_interrupts 441570 ± 7% +27.0% 560892 ± 6% interrupts.CPU111.LOC:Local_timer_interrupts 443022 ± 7% +26.6% 560741 ± 6% interrupts.CPU112.LOC:Local_timer_interrupts 441964 ± 7% +27.1% 561946 ± 6% interrupts.CPU113.LOC:Local_timer_interrupts 441789 ± 7% +26.9% 560493 ± 6% interrupts.CPU114.LOC:Local_timer_interrupts 441952 ± 7% +26.9% 560882 ± 6% interrupts.CPU115.LOC:Local_timer_interrupts 442589 ± 7% +26.7% 560921 ± 6% interrupts.CPU116.LOC:Local_timer_interrupts 443197 ± 7% +26.5% 560753 ± 6% interrupts.CPU117.LOC:Local_timer_interrupts 441734 ± 7% +26.9% 560762 ± 6% interrupts.CPU118.LOC:Local_timer_interrupts 440690 ± 7% +27.1% 559993 ± 6% interrupts.CPU119.LOC:Local_timer_interrupts 444765 ± 7% +26.2% 561154 ± 6% interrupts.CPU12.LOC:Local_timer_interrupts 437923 ± 9% +28.2% 561312 ± 6% interrupts.CPU120.LOC:Local_timer_interrupts 460450 +22.2% 562447 ± 6% interrupts.CPU121.LOC:Local_timer_interrupts 440049 ± 9% +34.2% 590338 ± 10% interrupts.CPU122.LOC:Local_timer_interrupts 439176 ± 9% +28.1% 562613 ± 6% interrupts.CPU123.LOC:Local_timer_interrupts 438950 ± 9% +28.3% 563078 ± 6% interrupts.CPU124.LOC:Local_timer_interrupts 438223 ± 9% +35.9% 595403 ± 13% interrupts.CPU125.LOC:Local_timer_interrupts 439205 ± 9% +28.4% 564022 ± 6% interrupts.CPU126.LOC:Local_timer_interrupts 438515 ± 9% +28.4% 562902 ± 6% interrupts.CPU127.LOC:Local_timer_interrupts 438003 ± 9% +28.2% 561680 ± 6% interrupts.CPU128.LOC:Local_timer_interrupts 438381 ± 9% +28.1% 561766 ± 6% interrupts.CPU129.LOC:Local_timer_interrupts 4628 +8.6% 5026 ± 4% interrupts.CPU13.CAL:Function_call_interrupts 441915 ± 7% +27.0% 561029 ± 6% interrupts.CPU13.LOC:Local_timer_interrupts 456533 ± 12% +23.2% 562614 ± 6% interrupts.CPU130.LOC:Local_timer_interrupts 446775 ± 10% +25.9% 562316 ± 6% interrupts.CPU131.LOC:Local_timer_interrupts 439796 ± 9% +27.9% 562386 ± 6% interrupts.CPU132.LOC:Local_timer_interrupts 443246 ± 10% +26.9% 562318 ± 6% interrupts.CPU133.LOC:Local_timer_interrupts 438946 ± 9% +28.1% 562147 ± 6% interrupts.CPU134.LOC:Local_timer_interrupts 438482 ± 9% +28.2% 562159 ± 6% interrupts.CPU135.LOC:Local_timer_interrupts 438851 ± 9% +28.2% 562409 ± 6% interrupts.CPU136.LOC:Local_timer_interrupts 439005 ± 9% +28.0% 562114 ± 6% interrupts.CPU137.LOC:Local_timer_interrupts 438350 ± 9% +28.2% 562056 ± 6% interrupts.CPU138.LOC:Local_timer_interrupts 437968 ± 9% +28.4% 562403 ± 6% interrupts.CPU139.LOC:Local_timer_interrupts 441269 ± 7% +27.1% 561051 ± 6% interrupts.CPU14.LOC:Local_timer_interrupts 438473 ± 9% +28.2% 562323 ± 6% interrupts.CPU140.LOC:Local_timer_interrupts 441766 ± 8% +27.1% 561681 ± 6% interrupts.CPU141.LOC:Local_timer_interrupts 438209 ± 9% +28.2% 561663 ± 6% interrupts.CPU142.LOC:Local_timer_interrupts 438170 ± 9% +28.1% 561438 ± 6% interrupts.CPU143.LOC:Local_timer_interrupts 6105 -10.2% 5480 ± 4% interrupts.CPU145.NMI:Non-maskable_interrupts 6105 -10.2% 5480 ± 4% interrupts.CPU145.PMI:Performance_monitoring_interrupts 441758 ± 7% +27.0% 560883 ± 6% interrupts.CPU15.LOC:Local_timer_interrupts 6092 -10.0% 5484 ± 4% interrupts.CPU150.NMI:Non-maskable_interrupts 6092 -10.0% 5484 ± 4% interrupts.CPU150.PMI:Performance_monitoring_interrupts 6107 -20.2% 4870 ± 25% interrupts.CPU151.NMI:Non-maskable_interrupts 6107 -20.2% 4870 ± 25% interrupts.CPU151.PMI:Performance_monitoring_interrupts 6113 -20.1% 4884 ± 25% interrupts.CPU152.NMI:Non-maskable_interrupts 6113 -20.1% 4884 ± 25% interrupts.CPU152.PMI:Performance_monitoring_interrupts 6114 -20.2% 4877 ± 25% interrupts.CPU153.NMI:Non-maskable_interrupts 6114 -20.2% 4877 ± 25% interrupts.CPU153.PMI:Performance_monitoring_interrupts 6121 -32.4% 4139 ± 32% interrupts.CPU154.NMI:Non-maskable_interrupts 6121 -32.4% 4139 ± 32% interrupts.CPU154.PMI:Performance_monitoring_interrupts 6.00 ± 48% +1133.3% 74.00 ± 83% interrupts.CPU158.RES:Rescheduling_interrupts 442392 ± 7% +26.8% 560825 ± 6% interrupts.CPU16.LOC:Local_timer_interrupts 6110 -10.2% 5485 ± 4% interrupts.CPU160.NMI:Non-maskable_interrupts 6110 -10.2% 5485 ± 4% interrupts.CPU160.PMI:Performance_monitoring_interrupts 6086 -10.0% 5477 ± 3% interrupts.CPU161.NMI:Non-maskable_interrupts 6086 -10.0% 5477 ± 3% interrupts.CPU161.PMI:Performance_monitoring_interrupts 6104 -32.7% 4107 ± 34% interrupts.CPU165.NMI:Non-maskable_interrupts 6104 -32.7% 4107 ± 34% interrupts.CPU165.PMI:Performance_monitoring_interrupts 6123 -32.9% 4108 ± 34% interrupts.CPU166.NMI:Non-maskable_interrupts 6123 -32.9% 4108 ± 34% interrupts.CPU166.PMI:Performance_monitoring_interrupts 6090 -10.3% 5465 ± 4% interrupts.CPU169.NMI:Non-maskable_interrupts 6090 -10.3% 5465 ± 4% interrupts.CPU169.PMI:Performance_monitoring_interrupts 154.00 ± 54% +116.2% 333.00 ± 26% interrupts.CPU169.RES:Rescheduling_interrupts 441690 ± 7% +27.2% 561996 ± 6% interrupts.CPU17.LOC:Local_timer_interrupts 109.00 ± 76% +138.5% 260.00 ± 17% interrupts.CPU170.RES:Rescheduling_interrupts 6110 -10.5% 5469 ± 4% interrupts.CPU172.NMI:Non-maskable_interrupts 6110 -10.5% 5469 ± 4% interrupts.CPU172.PMI:Performance_monitoring_interrupts 153.50 ± 69% +174.3% 421.00 ± 47% interrupts.CPU172.RES:Rescheduling_interrupts 6094 -33.1% 4076 ± 31% interrupts.CPU177.NMI:Non-maskable_interrupts 6094 -33.1% 4076 ± 31% interrupts.CPU177.PMI:Performance_monitoring_interrupts 6085 -21.5% 4778 ± 22% interrupts.CPU178.NMI:Non-maskable_interrupts 6085 -21.5% 4778 ± 22% interrupts.CPU178.PMI:Performance_monitoring_interrupts 441721 ± 7% +26.8% 559955 ± 6% interrupts.CPU18.LOC:Local_timer_interrupts 6099 -21.1% 4811 ± 25% interrupts.CPU180.NMI:Non-maskable_interrupts 6099 -21.1% 4811 ± 25% interrupts.CPU180.PMI:Performance_monitoring_interrupts 6108 -9.9% 5506 ± 4% interrupts.CPU182.NMI:Non-maskable_interrupts 6108 -9.9% 5506 ± 4% interrupts.CPU182.PMI:Performance_monitoring_interrupts 6091 -9.5% 5514 ± 4% interrupts.CPU183.NMI:Non-maskable_interrupts 6091 -9.5% 5514 ± 4% interrupts.CPU183.PMI:Performance_monitoring_interrupts 6116 -9.8% 5519 ± 4% interrupts.CPU184.NMI:Non-maskable_interrupts 6116 -9.8% 5519 ± 4% interrupts.CPU184.PMI:Performance_monitoring_interrupts 6110 -20.3% 4871 ± 25% interrupts.CPU185.NMI:Non-maskable_interrupts 6110 -20.3% 4871 ± 25% interrupts.CPU185.PMI:Performance_monitoring_interrupts 6116 -20.1% 4888 ± 25% interrupts.CPU186.NMI:Non-maskable_interrupts 6116 -20.1% 4888 ± 25% interrupts.CPU186.PMI:Performance_monitoring_interrupts 6107 -9.9% 5501 ± 4% interrupts.CPU187.NMI:Non-maskable_interrupts 6107 -9.9% 5501 ± 4% interrupts.CPU187.PMI:Performance_monitoring_interrupts 6103 -20.0% 4885 ± 25% interrupts.CPU188.NMI:Non-maskable_interrupts 6103 -20.0% 4885 ± 25% interrupts.CPU188.PMI:Performance_monitoring_interrupts 441941 ± 7% +26.9% 560739 ± 6% interrupts.CPU19.LOC:Local_timer_interrupts 442228 ± 7% +26.9% 561169 ± 6% interrupts.CPU2.LOC:Local_timer_interrupts 440938 ± 8% +27.2% 561005 ± 6% interrupts.CPU20.LOC:Local_timer_interrupts 442892 ± 7% +26.6% 560878 ± 6% interrupts.CPU21.LOC:Local_timer_interrupts 441484 ± 7% +27.1% 560924 ± 6% interrupts.CPU22.LOC:Local_timer_interrupts 440543 ± 7% +27.1% 559865 ± 6% interrupts.CPU23.LOC:Local_timer_interrupts 438386 ± 9% +28.0% 561312 ± 6% interrupts.CPU24.LOC:Local_timer_interrupts 462406 +21.7% 562847 ± 6% interrupts.CPU25.LOC:Local_timer_interrupts 5395 ± 25% -33.2% 3603 ± 32% interrupts.CPU25.NMI:Non-maskable_interrupts 5395 ± 25% -33.2% 3603 ± 32% interrupts.CPU25.PMI:Performance_monitoring_interrupts 438496 ± 9% +39.6% 612302 ± 15% interrupts.CPU26.LOC:Local_timer_interrupts 438103 ± 9% +28.5% 562794 ± 6% interrupts.CPU27.LOC:Local_timer_interrupts 439193 ± 9% +28.1% 562819 ± 6% interrupts.CPU28.LOC:Local_timer_interrupts 438266 ± 9% +40.7% 616674 ± 18% interrupts.CPU29.LOC:Local_timer_interrupts 442329 ± 7% +27.6% 564291 ± 7% interrupts.CPU3.LOC:Local_timer_interrupts 439031 ± 9% +28.2% 562871 ± 6% interrupts.CPU30.LOC:Local_timer_interrupts 438943 ± 9% +28.3% 563258 ± 6% interrupts.CPU31.LOC:Local_timer_interrupts 438882 ± 9% +28.1% 562021 ± 6% interrupts.CPU32.LOC:Local_timer_interrupts 438928 ± 9% +27.0% 557626 ± 5% interrupts.CPU33.LOC:Local_timer_interrupts 454289 ± 12% +23.8% 562341 ± 6% interrupts.CPU34.LOC:Local_timer_interrupts 445842 ± 10% +26.1% 562155 ± 6% interrupts.CPU35.LOC:Local_timer_interrupts 443773 ± 7% +26.6% 561772 ± 6% interrupts.CPU36.LOC:Local_timer_interrupts 445000 ± 10% +26.2% 561695 ± 6% interrupts.CPU37.LOC:Local_timer_interrupts 439502 ± 9% +27.9% 562101 ± 6% interrupts.CPU38.LOC:Local_timer_interrupts 438776 ± 9% +27.8% 560948 ± 6% interrupts.CPU39.LOC:Local_timer_interrupts 442720 ± 7% +26.7% 560836 ± 6% interrupts.CPU4.LOC:Local_timer_interrupts 439316 ± 9% +27.7% 561122 ± 6% interrupts.CPU40.LOC:Local_timer_interrupts 439119 ± 9% +28.0% 562073 ± 6% interrupts.CPU41.LOC:Local_timer_interrupts 438408 ± 9% +28.0% 560983 ± 6% interrupts.CPU42.LOC:Local_timer_interrupts 438940 ± 9% +28.1% 562209 ± 6% interrupts.CPU43.LOC:Local_timer_interrupts 6158 -29.4% 4350 ± 31% interrupts.CPU43.NMI:Non-maskable_interrupts 6158 -29.4% 4350 ± 31% interrupts.CPU43.PMI:Performance_monitoring_interrupts 438457 ± 9% +28.2% 562141 ± 6% interrupts.CPU44.LOC:Local_timer_interrupts 6141 -29.0% 4363 ± 31% interrupts.CPU44.NMI:Non-maskable_interrupts 6141 -29.0% 4363 ± 31% interrupts.CPU44.PMI:Performance_monitoring_interrupts 442624 ± 7% +26.9% 561592 ± 6% interrupts.CPU45.LOC:Local_timer_interrupts 437782 ± 9% +28.3% 561834 ± 6% interrupts.CPU46.LOC:Local_timer_interrupts 438078 ± 9% +28.1% 561199 ± 6% interrupts.CPU47.LOC:Local_timer_interrupts 6100 -11.2% 5416 ± 4% interrupts.CPU48.NMI:Non-maskable_interrupts 6100 -11.2% 5416 ± 4% interrupts.CPU48.PMI:Performance_monitoring_interrupts 4630 +8.2% 5012 ± 5% interrupts.CPU5.CAL:Function_call_interrupts 442026 ± 7% +26.9% 560985 ± 6% interrupts.CPU5.LOC:Local_timer_interrupts 6119 -20.5% 4866 ± 26% interrupts.CPU51.NMI:Non-maskable_interrupts 6119 -20.5% 4866 ± 26% interrupts.CPU51.PMI:Performance_monitoring_interrupts 30.00 ± 97% +403.3% 151.00 ± 85% interrupts.CPU51.RES:Rescheduling_interrupts 6122 -10.9% 5452 ± 4% interrupts.CPU54.NMI:Non-maskable_interrupts 6122 -10.9% 5452 ± 4% interrupts.CPU54.PMI:Performance_monitoring_interrupts 6150 -22.5% 4765 ± 25% interrupts.CPU56.NMI:Non-maskable_interrupts 6150 -22.5% 4765 ± 25% interrupts.CPU56.PMI:Performance_monitoring_interrupts 6136 -11.2% 5450 ± 4% interrupts.CPU57.NMI:Non-maskable_interrupts 6136 -11.2% 5450 ± 4% interrupts.CPU57.PMI:Performance_monitoring_interrupts 441694 ± 7% +27.0% 561033 ± 6% interrupts.CPU6.LOC:Local_timer_interrupts 19.75 ± 89% +524.1% 123.25 ± 33% interrupts.CPU62.RES:Rescheduling_interrupts 4633 +7.3% 4969 ± 4% interrupts.CPU65.CAL:Function_call_interrupts 6100 -33.3% 4068 ± 31% interrupts.CPU65.NMI:Non-maskable_interrupts 6100 -33.3% 4068 ± 31% interrupts.CPU65.PMI:Performance_monitoring_interrupts 6118 -22.1% 4768 ± 22% interrupts.CPU66.NMI:Non-maskable_interrupts 6118 -22.1% 4768 ± 22% interrupts.CPU66.PMI:Performance_monitoring_interrupts 4629 ± 2% +8.4% 5017 ± 4% interrupts.CPU7.CAL:Function_call_interrupts 443667 ± 7% +26.4% 560890 ± 6% interrupts.CPU7.LOC:Local_timer_interrupts 6096 -10.4% 5462 ± 4% interrupts.CPU71.NMI:Non-maskable_interrupts 6096 -10.4% 5462 ± 4% interrupts.CPU71.PMI:Performance_monitoring_interrupts 6103 -22.1% 4753 ± 22% interrupts.CPU73.NMI:Non-maskable_interrupts 6103 -22.1% 4753 ± 22% interrupts.CPU73.PMI:Performance_monitoring_interrupts 227.50 ± 57% +113.4% 485.50 ± 28% interrupts.CPU73.RES:Rescheduling_interrupts 6091 -10.4% 5460 ± 4% interrupts.CPU74.NMI:Non-maskable_interrupts 6091 -10.4% 5460 ± 4% interrupts.CPU74.PMI:Performance_monitoring_interrupts 156.75 ± 88% +189.3% 453.50 ± 18% interrupts.CPU74.RES:Rescheduling_interrupts 78.75 ± 50% +308.3% 321.50 ± 20% interrupts.CPU75.RES:Rescheduling_interrupts 6085 -10.3% 5461 ± 4% interrupts.CPU76.NMI:Non-maskable_interrupts 6085 -10.3% 5461 ± 4% interrupts.CPU76.PMI:Performance_monitoring_interrupts 152.25 ± 93% +187.0% 437.00 ± 20% interrupts.CPU77.RES:Rescheduling_interrupts 5316 ± 25% -35.3% 3442 ± 33% interrupts.CPU79.NMI:Non-maskable_interrupts 5316 ± 25% -35.3% 3442 ± 33% interrupts.CPU79.PMI:Performance_monitoring_interrupts 442068 ± 7% +26.9% 560877 ± 6% interrupts.CPU8.LOC:Local_timer_interrupts 6084 -42.6% 3490 ± 38% interrupts.CPU80.NMI:Non-maskable_interrupts 6084 -42.6% 3490 ± 38% interrupts.CPU80.PMI:Performance_monitoring_interrupts 111.00 ± 79% +138.3% 264.50 ± 63% interrupts.CPU81.RES:Rescheduling_interrupts 6113 -33.7% 4052 ± 30% interrupts.CPU82.NMI:Non-maskable_interrupts 6113 -33.7% 4052 ± 30% interrupts.CPU82.PMI:Performance_monitoring_interrupts 6106 -21.9% 4772 ± 25% interrupts.CPU83.NMI:Non-maskable_interrupts 6106 -21.9% 4772 ± 25% interrupts.CPU83.PMI:Performance_monitoring_interrupts 6105 -20.0% 4885 ± 25% interrupts.CPU86.NMI:Non-maskable_interrupts 6105 -20.0% 4885 ± 25% interrupts.CPU86.PMI:Performance_monitoring_interrupts 6084 -19.7% 4886 ± 25% interrupts.CPU87.NMI:Non-maskable_interrupts 6084 -19.7% 4886 ± 25% interrupts.CPU87.PMI:Performance_monitoring_interrupts 6105 -10.2% 5484 ± 4% interrupts.CPU88.NMI:Non-maskable_interrupts 6105 -10.2% 5484 ± 4% interrupts.CPU88.PMI:Performance_monitoring_interrupts 441843 ± 7% +27.0% 561181 ± 6% interrupts.CPU9.LOC:Local_timer_interrupts 6107 -21.3% 4804 ± 25% interrupts.CPU90.NMI:Non-maskable_interrupts 6107 -21.3% 4804 ± 25% interrupts.CPU90.PMI:Performance_monitoring_interrupts 6082 -32.5% 4106 ± 34% interrupts.CPU93.NMI:Non-maskable_interrupts 6082 -32.5% 4106 ± 34% interrupts.CPU93.PMI:Performance_monitoring_interrupts 6085 -9.9% 5484 ± 4% interrupts.CPU94.NMI:Non-maskable_interrupts 6085 -9.9% 5484 ± 4% interrupts.CPU94.PMI:Performance_monitoring_interrupts 441067 ± 7% +28.3% 565771 ± 5% interrupts.CPU96.LOC:Local_timer_interrupts 441841 ± 7% +27.0% 560947 ± 6% interrupts.CPU97.LOC:Local_timer_interrupts 441029 ± 7% +27.2% 561113 ± 6% interrupts.CPU98.LOC:Local_timer_interrupts 442341 ± 7% +26.8% 560839 ± 6% interrupts.CPU99.LOC:Local_timer_interrupts 95239511 ± 11% +17.0% 1.115e+08 ± 3% interrupts.LOC:Local_timer_interrupts *************************************************************************************************** lkp-skl-fpga01: 104 threads Skylake with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-skl-fpga01/lru-file-readonce/vm-scalability commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 267382 -18.3% 218553 vm-scalability.median 27868489 -18.4% 22751891 vm-scalability.throughput 202.53 +17.9% 238.78 vm-scalability.time.elapsed_time 202.53 +17.9% 238.78 vm-scalability.time.elapsed_time.max 178282 +13.0% 201467 vm-scalability.time.involuntary_context_switches 7847 -1.0% 7767 vm-scalability.time.percent_of_cpu_this_job_got 15577 +17.0% 18218 vm-scalability.time.system_time 1592 ± 2% +78.1% 2836 ± 24% vm-scalability.time.voluntary_context_switches 2.674e+09 ± 54% -59.3% 1.09e+09 ±130% cpuidle.C6.time 24.54 -2.9 21.68 mpstat.cpu.idle% 3883 ± 4% -14.5% 3321 ± 3% slabinfo.kmalloc-rcl-64.active_objs 3883 ± 4% -14.5% 3321 ± 3% slabinfo.kmalloc-rcl-64.num_objs 14059840 ± 2% -10.5% 12586661 ± 4% numa-meminfo.node0.MemFree 1441736 -9.5% 1305441 numa-meminfo.node1.KReclaimable 14332989 ± 4% -18.7% 11653080 ± 5% numa-meminfo.node1.MemFree 1441736 -9.5% 1305441 numa-meminfo.node1.SReclaimable 2123 +4.0% 2208 turbostat.Avg_MHz 12.56 ± 54% -8.3 4.29 ±130% turbostat.C6% 43114799 +10.5% 47658339 turbostat.IRQ 24.11 -11.8 12.35 ± 4% turbostat.PKG_% 184.74 -2.9% 179.40 turbostat.RAMWatt 24.50 ± 2% -11.2% 21.75 vmstat.cpu.id 73.00 +4.1% 76.00 vmstat.cpu.sy 27802573 ± 2% -16.7% 23150343 vmstat.memory.free 2580 -7.0% 2401 vmstat.system.cs 210070 -5.0% 199664 vmstat.system.in 19316136 ± 7% +221.0% 62011283 ± 5% numa-numastat.node0.numa_foreign 12247697 ± 6% +386.8% 59622208 ± 2% numa-numastat.node0.numa_miss 12261264 ± 6% +386.3% 59630731 ± 2% numa-numastat.node0.other_node 12247697 ± 6% +386.8% 59622208 ± 2% numa-numastat.node1.numa_foreign 19316136 ± 7% +221.0% 62011283 ± 5% numa-numastat.node1.numa_miss 19322987 ± 7% +221.0% 62022985 ± 5% numa-numastat.node1.other_node 281733 +10.8% 312061 meminfo.Active 281509 +10.8% 311848 meminfo.Active(anon) 38909 ± 2% -15.4% 32922 ± 8% meminfo.CmaFree 5693218 +11.3% 6334910 ± 8% meminfo.DirectMap2M 293863 ± 8% -40.7% 174364 ± 12% meminfo.DirectMap4k 28234850 -15.2% 23944316 meminfo.MemFree 137242 +24.3% 170648 meminfo.Shmem 1000970 -11.9% 882068 meminfo.max_used_kB 11770 +52.3% 17929 ± 25% softirqs.CPU101.RCU 10077 ± 9% +15.5% 11636 ± 10% softirqs.CPU13.RCU 10509 ± 9% +35.2% 14212 ± 20% softirqs.CPU14.RCU 9486 ± 4% +64.8% 15629 ± 31% softirqs.CPU26.RCU 9380 +47.4% 13826 ± 28% softirqs.CPU29.RCU 12736 ± 10% +56.1% 19880 ± 26% softirqs.CPU32.RCU 11713 ± 5% +40.2% 16423 ± 16% softirqs.CPU42.RCU 10910 ± 14% +30.8% 14271 ± 19% softirqs.CPU46.RCU 10362 ± 3% +87.7% 19448 ± 19% softirqs.CPU49.RCU 9997 ± 5% +13.2% 11312 ± 7% softirqs.CPU65.RCU 9738 ± 5% +44.7% 14090 ± 26% softirqs.CPU66.RCU 11519 ± 2% +14.0% 13137 ± 12% softirqs.CPU83.RCU 11596 ± 9% +33.6% 15494 ± 23% softirqs.CPU86.RCU 11473 ± 15% +29.3% 14835 ± 23% softirqs.CPU88.RCU 1245125 ± 4% +14.1% 1421069 softirqs.RCU 3541718 ± 2% -10.6% 3167441 ± 5% numa-vmstat.node0.nr_free_pages 11429039 ± 9% +222.5% 36862004 ± 6% numa-vmstat.node0.numa_foreign 3.336e+08 -10.2% 2.994e+08 numa-vmstat.node0.numa_hit 3.336e+08 -10.2% 2.994e+08 numa-vmstat.node0.numa_local 6568178 ± 5% +438.9% 35396855 ± 2% numa-vmstat.node0.numa_miss 6581780 ± 5% +437.9% 35405485 ± 2% numa-vmstat.node0.numa_other 2688017 +22.4% 3290377 ± 4% numa-vmstat.node0.workingset_nodereclaim 2061926 -8.9% 1878294 numa-vmstat.node0.workingset_nodes 9846 ± 3% -14.4% 8424 ± 9% numa-vmstat.node1.nr_free_cma 3616031 ± 3% -18.5% 2946334 ± 6% numa-vmstat.node1.nr_free_pages 360110 -9.4% 326092 numa-vmstat.node1.nr_slab_reclaimable 6569127 ± 5% +439.0% 35404580 ± 2% numa-vmstat.node1.numa_foreign 3.35e+08 -10.6% 2.996e+08 numa-vmstat.node1.numa_hit 3.348e+08 -10.6% 2.994e+08 numa-vmstat.node1.numa_local 11431797 ± 9% +222.5% 36870344 ± 6% numa-vmstat.node1.numa_miss 11588020 ± 9% +219.6% 37032021 ± 6% numa-vmstat.node1.numa_other 85092 +1.1% 86037 proc-vmstat.allocstall_movable 6.64e+08 ± 7% -57.2% 2.843e+08 ± 22% proc-vmstat.compact_daemon_free_scanned 1959 -56.4% 854.50 ± 5% proc-vmstat.compact_daemon_wake 6.642e+08 ± 7% -57.2% 2.845e+08 ± 22% proc-vmstat.compact_free_scanned 1997 -50.1% 997.50 ± 5% proc-vmstat.kswapd_low_wmark_hit_quickly 70387 +10.8% 77956 proc-vmstat.nr_active_anon 4763370 -7.6% 4399812 proc-vmstat.nr_dirty_background_threshold 9538388 -7.6% 8810383 proc-vmstat.nr_dirty_threshold 41128365 -6.3% 38536932 proc-vmstat.nr_file_pages 9849 ± 3% -15.7% 8307 ± 8% proc-vmstat.nr_free_cma 7054662 -14.7% 6014873 proc-vmstat.nr_free_pages 28642 +1.3% 29022 proc-vmstat.nr_inactive_anon 40864685 -6.4% 38263422 proc-vmstat.nr_inactive_file 670.50 -10.2% 602.00 ± 4% proc-vmstat.nr_isolated_file 6582 +5.4% 6939 ± 2% proc-vmstat.nr_mapped 2397 +1.8% 2441 proc-vmstat.nr_page_table_pages 34380 +23.9% 42584 proc-vmstat.nr_shmem 726733 -7.4% 672630 proc-vmstat.nr_slab_reclaimable 70388 +10.8% 77957 proc-vmstat.nr_zone_active_anon 28642 +1.3% 29022 proc-vmstat.nr_zone_inactive_anon 40864645 -6.4% 38263432 proc-vmstat.nr_zone_inactive_file 31563834 ± 6% +285.4% 1.216e+08 ± 4% proc-vmstat.numa_foreign 1.043e+09 -8.6% 9.53e+08 proc-vmstat.numa_hit 1.043e+09 -8.6% 9.53e+08 proc-vmstat.numa_local 31563834 ± 6% +285.4% 1.216e+08 ± 4% proc-vmstat.numa_miss 31584258 ± 6% +285.2% 1.217e+08 ± 4% proc-vmstat.numa_other 2896 -59.9% 1162 ± 3% proc-vmstat.pageoutrun 19386 ± 8% +82.1% 35309 ± 7% proc-vmstat.pgactivate 6969446 +4.0% 7250881 proc-vmstat.pgalloc_dma32 612923 +11.9% 685914 proc-vmstat.pgfault 1941 ± 42% -65.8% 664.25 ± 55% proc-vmstat.pgmigrate_fail 9.901e+08 -7.1% 9.199e+08 proc-vmstat.pgscan_direct 24.00 ±119% +5025.0% 1230 ± 55% proc-vmstat.pgscan_direct_throttle 36197782 ± 2% +200.7% 1.089e+08 ± 4% proc-vmstat.pgscan_kswapd 9.901e+08 -7.1% 9.199e+08 proc-vmstat.pgsteal_direct 36197730 ± 2% +200.7% 1.089e+08 ± 4% proc-vmstat.pgsteal_kswapd 12326029 ± 2% +10.1% 13569879 proc-vmstat.slabs_scanned 5045797 +6.8% 5388502 proc-vmstat.workingset_nodereclaim 4160739 -8.8% 3795344 proc-vmstat.workingset_nodes 8.25e+09 -13.7% 7.12e+09 perf-stat.i.branch-instructions 0.92 ± 7% -0.1 0.85 ± 4% perf-stat.i.branch-miss-rate% 66123531 -17.9% 54296195 perf-stat.i.branch-misses 1.828e+08 -13.4% 1.583e+08 perf-stat.i.cache-misses 5.226e+08 -17.7% 4.303e+08 perf-stat.i.cache-references 2538 -7.7% 2342 perf-stat.i.context-switches 4.50 +25.4% 5.65 perf-stat.i.cpi 2.186e+11 +3.8% 2.268e+11 perf-stat.i.cpu-cycles 47.13 +17.3% 55.27 ± 2% perf-stat.i.cpu-migrations 1127 +18.7% 1338 perf-stat.i.cycles-between-cache-misses 0.08 ± 16% +0.1 0.13 ± 3% perf-stat.i.dTLB-load-miss-rate% 9783258 ± 10% +49.5% 14624305 ± 10% perf-stat.i.dTLB-load-misses 1.104e+10 -14.4% 9.45e+09 perf-stat.i.dTLB-loads 5399804 ± 7% -19.0% 4372243 ± 5% perf-stat.i.dTLB-store-misses 5.816e+09 -17.0% 4.829e+09 perf-stat.i.dTLB-stores 10501739 -15.1% 8920664 perf-stat.i.iTLB-load-misses 4.06e+10 -14.4% 3.475e+10 perf-stat.i.instructions 0.37 ± 5% -14.3% 0.32 ± 5% perf-stat.i.ipc 2930 -4.6% 2796 perf-stat.i.minor-faults 18.09 +9.4 27.49 ± 2% perf-stat.i.node-load-miss-rate% 5243773 ± 3% +133.0% 12216989 perf-stat.i.node-load-misses 61747639 -18.9% 50075226 ± 2% perf-stat.i.node-loads 21.73 ± 2% +11.3 33.03 ± 2% perf-stat.i.node-store-miss-rate% 2896223 ± 2% +86.2% 5391881 perf-stat.i.node-store-misses 19754443 -26.9% 14433885 perf-stat.i.node-stores 2931 -4.5% 2800 perf-stat.i.page-faults 12.88 -3.8% 12.39 perf-stat.overall.MPKI 0.80 -0.0 0.76 perf-stat.overall.branch-miss-rate% 35.00 +1.8 36.81 perf-stat.overall.cache-miss-rate% 5.39 +22.0% 6.58 perf-stat.overall.cpi 1196 +20.6% 1442 perf-stat.overall.cycles-between-cache-misses 0.09 ± 10% +0.1 0.15 ± 9% perf-stat.overall.dTLB-load-miss-rate% 91.86 -1.4 90.44 perf-stat.overall.iTLB-load-miss-rate% 0.19 -18.1% 0.15 perf-stat.overall.ipc 7.82 ± 2% +12.0 19.79 ± 2% perf-stat.overall.node-load-miss-rate% 12.79 ± 2% +14.4 27.21 perf-stat.overall.node-store-miss-rate% 8.267e+09 -14.0% 7.112e+09 perf-stat.ps.branch-instructions 66151618 -18.4% 54000381 perf-stat.ps.branch-misses 1.835e+08 -13.7% 1.584e+08 perf-stat.ps.cache-misses 5.242e+08 -17.9% 4.302e+08 perf-stat.ps.cache-references 2535 -8.1% 2331 perf-stat.ps.context-switches 2.194e+11 +4.1% 2.284e+11 perf-stat.ps.cpu-cycles 47.28 +17.7% 55.63 ± 2% perf-stat.ps.cpu-migrations 9829320 ± 10% +49.0% 14641191 ± 10% perf-stat.ps.dTLB-load-misses 1.107e+10 -14.6% 9.45e+09 perf-stat.ps.dTLB-loads 5403667 ± 7% -20.0% 4325550 ± 6% perf-stat.ps.dTLB-store-misses 5.831e+09 -17.2% 4.827e+09 perf-stat.ps.dTLB-stores 10536239 -15.2% 8937656 perf-stat.ps.iTLB-load-misses 4.069e+10 -14.7% 3.471e+10 perf-stat.ps.instructions 2906 -5.1% 2759 perf-stat.ps.minor-faults 5255162 ± 3% +134.9% 12345602 perf-stat.ps.node-load-misses 61965059 -19.2% 50077399 ± 2% perf-stat.ps.node-loads 2906646 ± 2% +85.8% 5401881 perf-stat.ps.node-store-misses 19823244 -27.1% 14453478 perf-stat.ps.node-stores 2906 -5.1% 2759 perf-stat.ps.page-faults 8.06 ± 9% -8.1 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64 8.02 ± 9% -8.0 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 8.02 ± 9% -8.0 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 8.02 ± 9% -8.0 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 7.67 ± 9% -7.7 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 6.98 ± 10% -7.0 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 5.89 -2.1 3.80 ± 9% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 4.29 -1.7 2.60 ± 9% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 2.71 -1.4 1.31 ± 15% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 2.57 -1.4 1.19 ± 16% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 5.35 -1.2 4.12 perf-profile.calltrace.cycles-pp.iomap_set_range_uptodate.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages 10.26 -1.1 9.12 perf-profile.calltrace.cycles-pp.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 0.73 ± 3% -0.4 0.31 ±100% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.88 ± 3% -0.4 0.52 ± 60% perf-profile.calltrace.cycles-pp.free_unref_page_list.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 4.10 -0.3 3.84 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 1.35 -0.3 1.09 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 3.91 -0.2 3.68 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 3.87 -0.2 3.65 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read 1.29 -0.2 1.10 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 0.90 ± 2% -0.1 0.76 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.87 ± 2% -0.1 0.73 ± 5% perf-profile.calltrace.cycles-pp.__delete_from_page_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.68 ± 2% -0.1 0.58 ± 2% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.65 ± 2% -0.1 0.55 perf-profile.calltrace.cycles-pp.__pagevec_lru_add_fn.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 0.81 -0.1 0.72 ± 4% perf-profile.calltrace.cycles-pp.isolate_lru_pages.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 4.48 +0.2 4.64 perf-profile.calltrace.cycles-pp.memset_erms.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages 0.57 ± 8% +0.5 1.07 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 0.98 ± 5% +0.7 1.69 ± 13% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 0.98 ± 6% +0.7 1.69 ± 13% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 0.98 ± 6% +0.7 1.69 ± 13% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 0.97 ± 6% +0.7 1.68 ± 13% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 0.97 ± 6% +0.7 1.68 ± 13% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 0.00 +0.7 0.73 ± 5% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 1.21 ± 2% +1.0 2.23 ± 25% perf-profile.calltrace.cycles-pp.ret_from_fork 1.21 ± 2% +1.0 2.23 ± 25% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 2.09 ± 5% +2.7 4.77 ± 39% perf-profile.calltrace.cycles-pp._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 2.08 ± 5% +2.7 4.76 ± 39% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead 3.09 ± 4% +2.8 5.87 ± 31% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 24.64 +8.2 32.84 ± 2% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 24.63 +8.2 32.83 ± 2% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 24.62 +8.2 32.82 ± 2% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 24.56 +8.2 32.76 ± 2% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 86.83 +8.2 95.03 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 86.72 +8.2 94.95 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 85.48 +8.4 93.93 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 85.26 +8.5 93.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 84.67 +8.6 93.25 perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 84.56 +8.6 93.16 perf-profile.calltrace.cycles-pp.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read.do_syscall_64 84.27 +8.7 92.92 perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read 83.99 +8.7 92.70 perf-profile.calltrace.cycles-pp.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read 12.59 ± 2% +9.0 21.61 ± 5% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 12.51 ± 2% +9.0 21.54 ± 5% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 11.54 ± 2% +9.1 20.66 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 11.59 ± 2% +9.1 20.70 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 78.70 +9.2 87.87 perf-profile.calltrace.cycles-pp.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 78.69 +9.2 87.86 perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 14.06 ± 2% +9.4 23.45 ± 3% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 7081 ±173% +1129.2% 87047 ± 86% sched_debug.cfs_rq:/.MIN_vruntime.avg 736468 ±173% +384.4% 3567746 ± 18% sched_debug.cfs_rq:/.MIN_vruntime.max 71868 ±173% +587.8% 494327 ± 37% sched_debug.cfs_rq:/.MIN_vruntime.stddev 83062 +19.7% 99400 ± 10% sched_debug.cfs_rq:/.exec_clock.avg 87627 +15.4% 101139 ± 10% sched_debug.cfs_rq:/.exec_clock.max 82237 +19.3% 98133 ± 9% sched_debug.cfs_rq:/.exec_clock.min 13487 ± 22% +78.4% 24057 ± 35% sched_debug.cfs_rq:/.load.avg 458516 ± 21% +56.8% 719174 ± 16% sched_debug.cfs_rq:/.load.max 4923 +33.3% 6563 ± 11% sched_debug.cfs_rq:/.load.min 58854 ± 30% +73.6% 102173 ± 27% sched_debug.cfs_rq:/.load.stddev 751.19 ± 8% +19.9% 900.35 ± 8% sched_debug.cfs_rq:/.load_avg.max 4.50 +33.9% 6.03 ± 12% sched_debug.cfs_rq:/.load_avg.min 7081 ±173% +1129.2% 87047 ± 86% sched_debug.cfs_rq:/.max_vruntime.avg 736468 ±173% +384.4% 3567746 ± 18% sched_debug.cfs_rq:/.max_vruntime.max 71868 ±173% +587.8% 494327 ± 37% sched_debug.cfs_rq:/.max_vruntime.stddev 9278141 +13.5% 10533815 ± 9% sched_debug.cfs_rq:/.min_vruntime.avg 9451389 +16.7% 11030401 ± 9% sched_debug.cfs_rq:/.min_vruntime.max 389746 ± 6% +44.8% 564234 ± 9% sched_debug.cfs_rq:/.min_vruntime.stddev 0.54 +32.2% 0.71 ± 10% sched_debug.cfs_rq:/.nr_running.avg 0.50 +35.0% 0.68 ± 11% sched_debug.cfs_rq:/.nr_running.min 15.27 ± 2% +197.4% 45.40 ± 16% sched_debug.cfs_rq:/.nr_spread_over.avg 313.06 ± 15% -55.7% 138.84 ± 15% sched_debug.cfs_rq:/.nr_spread_over.max 48.96 ± 9% -64.9% 17.19 ± 12% sched_debug.cfs_rq:/.nr_spread_over.stddev 11.37 ± 20% +54.9% 17.61 ± 18% sched_debug.cfs_rq:/.runnable_load_avg.avg 421.56 ± 24% +55.2% 654.21 ± 14% sched_debug.cfs_rq:/.runnable_load_avg.max 4.44 ± 2% +34.4% 5.96 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.min 49.82 ± 28% +63.0% 81.22 ± 22% sched_debug.cfs_rq:/.runnable_load_avg.stddev 13455 ± 22% +78.1% 23968 ± 35% sched_debug.cfs_rq:/.runnable_weight.avg 457920 ± 22% +57.0% 718706 ± 16% sched_debug.cfs_rq:/.runnable_weight.max 4923 +33.3% 6563 ± 11% sched_debug.cfs_rq:/.runnable_weight.min 58788 ± 30% +73.8% 102155 ± 27% sched_debug.cfs_rq:/.runnable_weight.stddev 146373 ± 27% +175.3% 402939 ± 29% sched_debug.cfs_rq:/.spread0.max -2122820 +48.1% -3143140 sched_debug.cfs_rq:/.spread0.min 389642 ± 5% +44.8% 564306 ± 9% sched_debug.cfs_rq:/.spread0.stddev 591.25 +28.8% 761.66 ± 10% sched_debug.cfs_rq:/.util_avg.avg 77.27 ± 75% +338.0% 338.48 ± 29% sched_debug.cfs_rq:/.util_est_enqueued.avg 916.06 ± 17% +33.8% 1225 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max 0.50 +35.0% 0.68 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.min 133357 ± 17% -31.2% 91775 ± 6% sched_debug.cpu.avg_idle.stddev 122520 +12.6% 137996 ± 10% sched_debug.cpu.clock.max 122520 +12.6% 137996 ± 10% sched_debug.cpu.clock_task.max 11.39 ± 19% +54.7% 17.61 ± 18% sched_debug.cpu.cpu_load[0].avg 422.12 ± 24% +55.0% 654.21 ± 14% sched_debug.cpu.cpu_load[0].max 4.44 ± 2% +34.4% 5.96 ± 11% sched_debug.cpu.cpu_load[0].min 49.91 ± 28% +62.7% 81.22 ± 22% sched_debug.cpu.cpu_load[0].stddev 11.61 ± 18% +52.4% 17.69 ± 18% sched_debug.cpu.cpu_load[1].avg 437.50 ± 21% +50.9% 660.10 ± 14% sched_debug.cpu.cpu_load[1].max 4.44 ± 2% +34.4% 5.96 ± 11% sched_debug.cpu.cpu_load[1].min 50.56 ± 27% +60.9% 81.35 ± 22% sched_debug.cpu.cpu_load[1].stddev 11.72 ± 18% +53.0% 17.93 ± 18% sched_debug.cpu.cpu_load[2].avg 456.19 ± 18% +47.8% 674.36 ± 15% sched_debug.cpu.cpu_load[2].max 4.50 +32.5% 5.96 ± 11% sched_debug.cpu.cpu_load[2].min 51.23 ± 26% +60.4% 82.15 ± 22% sched_debug.cpu.cpu_load[2].stddev 11.75 ± 18% +55.0% 18.21 ± 18% sched_debug.cpu.cpu_load[3].avg 480.75 ± 16% +44.1% 692.58 ± 16% sched_debug.cpu.cpu_load[3].max 4.50 +32.5% 5.96 ± 11% sched_debug.cpu.cpu_load[3].min 52.12 ± 24% +60.6% 83.71 ± 23% sched_debug.cpu.cpu_load[3].stddev 12.00 ± 15% +55.0% 18.60 ± 19% sched_debug.cpu.cpu_load[4].avg 4.50 +32.5% 5.96 ± 11% sched_debug.cpu.cpu_load[4].min 1269 ± 2% +29.3% 1640 ± 10% sched_debug.cpu.curr->pid.avg 13488 ± 22% +85.9% 25077 ± 33% sched_debug.cpu.load.avg 458287 ± 21% +57.1% 720140 ± 16% sched_debug.cpu.load.max 4923 +33.3% 6563 ± 11% sched_debug.cpu.load.min 58840 ± 30% +78.7% 105123 ± 27% sched_debug.cpu.load.stddev 818.92 ± 8% +18.4% 969.53 ± 10% sched_debug.cpu.nr_load_updates.stddev 0.55 +33.8% 0.74 ± 10% sched_debug.cpu.nr_running.avg 1.56 ± 6% +14.4% 1.79 ± 8% sched_debug.cpu.nr_running.max 0.50 +35.0% 0.68 ± 11% sched_debug.cpu.nr_running.min 0.20 ± 6% +16.9% 0.23 ± 5% sched_debug.cpu.nr_running.stddev 17168 ± 11% -25.2% 12847 ± 12% sched_debug.cpu.nr_switches.max 2839 ± 3% -22.8% 2193 ± 6% sched_debug.cpu.nr_switches.stddev 14906 ± 8% -35.7% 9589 ± 7% sched_debug.cpu.sched_count.max 2462 -31.6% 1685 ± 6% sched_debug.cpu.sched_count.stddev 6915 ± 13% -30.5% 4803 ± 14% sched_debug.cpu.ttwu_count.max 1122 ± 4% -31.4% 769.95 ± 10% sched_debug.cpu.ttwu_count.stddev 6611 ± 17% -39.2% 4017 ± 12% sched_debug.cpu.ttwu_local.max 1011 ± 4% -36.1% 646.38 ± 5% sched_debug.cpu.ttwu_local.stddev 409.00 +17.5% 480.50 interrupts.9:IO-APIC.9-fasteoi.acpi 253060 +8.7% 275187 interrupts.CAL:Function_call_interrupts 409.00 +17.5% 480.50 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 3846 ± 21% +90.6% 7329 ± 6% interrupts.CPU100.NMI:Non-maskable_interrupts 3846 ± 21% +90.6% 7329 ± 6% interrupts.CPU100.PMI:Performance_monitoring_interrupts 2333 ± 5% +14.5% 2670 interrupts.CPU101.CAL:Function_call_interrupts 4026 ± 11% +82.5% 7349 ± 6% interrupts.CPU101.NMI:Non-maskable_interrupts 4026 ± 11% +82.5% 7349 ± 6% interrupts.CPU101.PMI:Performance_monitoring_interrupts 2487 ± 43% +195.0% 7339 ± 6% interrupts.CPU102.NMI:Non-maskable_interrupts 2487 ± 43% +195.0% 7339 ± 6% interrupts.CPU102.PMI:Performance_monitoring_interrupts 3667 ± 23% +102.5% 7427 ± 5% interrupts.CPU13.NMI:Non-maskable_interrupts 3667 ± 23% +102.5% 7427 ± 5% interrupts.CPU13.PMI:Performance_monitoring_interrupts 2318 ± 3% +14.5% 2653 interrupts.CPU16.CAL:Function_call_interrupts 2978 ± 32% +116.5% 6449 ± 23% interrupts.CPU16.NMI:Non-maskable_interrupts 2978 ± 32% +116.5% 6449 ± 23% interrupts.CPU16.PMI:Performance_monitoring_interrupts 3568 ± 23% +107.8% 7416 ± 6% interrupts.CPU17.NMI:Non-maskable_interrupts 3568 ± 23% +107.8% 7416 ± 6% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3777 ± 36% +97.5% 7458 ± 5% interrupts.CPU18.NMI:Non-maskable_interrupts 3777 ± 36% +97.5% 7458 ± 5% interrupts.CPU18.PMI:Performance_monitoring_interrupts 62.00 ± 31% +243.5% 213.00 ± 35% interrupts.CPU18.RES:Rescheduling_interrupts 75.75 ± 43% +395.0% 375.00 ± 25% interrupts.CPU2.RES:Rescheduling_interrupts 3470 ± 23% +112.6% 7378 ± 6% interrupts.CPU20.NMI:Non-maskable_interrupts 3470 ± 23% +112.6% 7378 ± 6% interrupts.CPU20.PMI:Performance_monitoring_interrupts 4069 ± 9% +82.6% 7430 ± 5% interrupts.CPU22.NMI:Non-maskable_interrupts 4069 ± 9% +82.6% 7430 ± 5% interrupts.CPU22.PMI:Performance_monitoring_interrupts 2404 ± 2% +11.0% 2668 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 3378 ± 33% +119.1% 7401 ± 6% interrupts.CPU23.NMI:Non-maskable_interrupts 3378 ± 33% +119.1% 7401 ± 6% interrupts.CPU23.PMI:Performance_monitoring_interrupts 2435 +10.1% 2682 interrupts.CPU24.CAL:Function_call_interrupts 55.75 ± 43% +313.9% 230.75 ± 48% interrupts.CPU24.RES:Rescheduling_interrupts 4300 ± 9% +71.9% 7392 ± 6% interrupts.CPU25.NMI:Non-maskable_interrupts 4300 ± 9% +71.9% 7392 ± 6% interrupts.CPU25.PMI:Performance_monitoring_interrupts 751.25 ± 17% -50.3% 373.25 ± 23% interrupts.CPU26.RES:Rescheduling_interrupts 3772 ± 30% +95.1% 7358 ± 6% interrupts.CPU27.NMI:Non-maskable_interrupts 3772 ± 30% +95.1% 7358 ± 6% interrupts.CPU27.PMI:Performance_monitoring_interrupts 3795 ± 4% +93.6% 7347 ± 6% interrupts.CPU28.NMI:Non-maskable_interrupts 3795 ± 4% +93.6% 7347 ± 6% interrupts.CPU28.PMI:Performance_monitoring_interrupts 2436 +10.4% 2688 interrupts.CPU29.CAL:Function_call_interrupts 78.75 ± 23% +301.9% 316.50 ± 63% interrupts.CPU29.RES:Rescheduling_interrupts 2440 +8.3% 2641 ± 2% interrupts.CPU30.CAL:Function_call_interrupts 3732 ± 18% +96.8% 7345 ± 6% interrupts.CPU31.NMI:Non-maskable_interrupts 3732 ± 18% +96.8% 7345 ± 6% interrupts.CPU31.PMI:Performance_monitoring_interrupts 3069 ± 39% +139.9% 7363 ± 6% interrupts.CPU32.NMI:Non-maskable_interrupts 3069 ± 39% +139.9% 7363 ± 6% interrupts.CPU32.PMI:Performance_monitoring_interrupts 2246 ± 9% +18.3% 2657 interrupts.CPU33.CAL:Function_call_interrupts 3845 ± 26% +91.4% 7361 ± 6% interrupts.CPU35.NMI:Non-maskable_interrupts 3845 ± 26% +91.4% 7361 ± 6% interrupts.CPU35.PMI:Performance_monitoring_interrupts 75.50 ± 30% +51.7% 114.50 ± 14% interrupts.CPU36.RES:Rescheduling_interrupts 2399 ± 3% +12.7% 2703 ± 3% interrupts.CPU37.CAL:Function_call_interrupts 4156 ± 16% +77.6% 7379 ± 6% interrupts.CPU37.NMI:Non-maskable_interrupts 4156 ± 16% +77.6% 7379 ± 6% interrupts.CPU37.PMI:Performance_monitoring_interrupts 4168 ± 9% +76.1% 7341 ± 6% interrupts.CPU38.NMI:Non-maskable_interrupts 4168 ± 9% +76.1% 7341 ± 6% interrupts.CPU38.PMI:Performance_monitoring_interrupts 4207 ± 27% +53.6% 6460 ± 23% interrupts.CPU4.NMI:Non-maskable_interrupts 4207 ± 27% +53.6% 6460 ± 23% interrupts.CPU4.PMI:Performance_monitoring_interrupts 2276 ± 12% +17.7% 2679 interrupts.CPU40.CAL:Function_call_interrupts 4314 ± 6% +70.3% 7347 ± 6% interrupts.CPU40.NMI:Non-maskable_interrupts 4314 ± 6% +70.3% 7347 ± 6% interrupts.CPU40.PMI:Performance_monitoring_interrupts 4192 ± 6% +75.3% 7350 ± 6% interrupts.CPU43.NMI:Non-maskable_interrupts 4192 ± 6% +75.3% 7350 ± 6% interrupts.CPU43.PMI:Performance_monitoring_interrupts 2439 +7.6% 2626 ± 3% interrupts.CPU44.CAL:Function_call_interrupts 3870 ± 23% +90.1% 7359 ± 6% interrupts.CPU46.NMI:Non-maskable_interrupts 3870 ± 23% +90.1% 7359 ± 6% interrupts.CPU46.PMI:Performance_monitoring_interrupts 2370 ± 19% +171.8% 6443 ± 23% interrupts.CPU5.NMI:Non-maskable_interrupts 2370 ± 19% +171.8% 6443 ± 23% interrupts.CPU5.PMI:Performance_monitoring_interrupts 120.75 ± 44% +154.0% 306.75 ± 62% interrupts.CPU5.RES:Rescheduling_interrupts 4625 ± 5% +58.3% 7324 ± 6% interrupts.CPU51.NMI:Non-maskable_interrupts 4625 ± 5% +58.3% 7324 ± 6% interrupts.CPU51.PMI:Performance_monitoring_interrupts 2140 +22.5% 2623 ± 3% interrupts.CPU51.RES:Rescheduling_interrupts 116.25 ± 50% +75.5% 204.00 ± 22% interrupts.CPU52.RES:Rescheduling_interrupts 4156 ± 11% +79.1% 7445 ± 5% interrupts.CPU53.NMI:Non-maskable_interrupts 4156 ± 11% +79.1% 7445 ± 5% interrupts.CPU53.PMI:Performance_monitoring_interrupts 71.50 ± 31% +216.1% 226.00 ± 28% interrupts.CPU54.RES:Rescheduling_interrupts 3095 ± 30% +107.9% 6437 ± 23% interrupts.CPU55.NMI:Non-maskable_interrupts 3095 ± 30% +107.9% 6437 ± 23% interrupts.CPU55.PMI:Performance_monitoring_interrupts 4426 ± 20% +67.8% 7425 ± 6% interrupts.CPU58.NMI:Non-maskable_interrupts 4426 ± 20% +67.8% 7425 ± 6% interrupts.CPU58.PMI:Performance_monitoring_interrupts 73.00 ± 58% +177.7% 202.75 ± 42% interrupts.CPU58.RES:Rescheduling_interrupts 3590 ± 10% +106.1% 7400 ± 6% interrupts.CPU6.NMI:Non-maskable_interrupts 3590 ± 10% +106.1% 7400 ± 6% interrupts.CPU6.PMI:Performance_monitoring_interrupts 120.50 ± 57% +122.4% 268.00 ± 20% interrupts.CPU6.RES:Rescheduling_interrupts 4060 ± 9% +83.6% 7453 ± 5% interrupts.CPU60.NMI:Non-maskable_interrupts 4060 ± 9% +83.6% 7453 ± 5% interrupts.CPU60.PMI:Performance_monitoring_interrupts 3626 ± 26% +103.7% 7386 ± 6% interrupts.CPU62.NMI:Non-maskable_interrupts 3626 ± 26% +103.7% 7386 ± 6% interrupts.CPU62.PMI:Performance_monitoring_interrupts 50.25 ± 10% +404.0% 253.25 ± 59% interrupts.CPU62.RES:Rescheduling_interrupts 3778 ± 29% +96.0% 7405 ± 5% interrupts.CPU63.NMI:Non-maskable_interrupts 3778 ± 29% +96.0% 7405 ± 5% interrupts.CPU63.PMI:Performance_monitoring_interrupts 3826 ± 22% +93.1% 7389 ± 6% interrupts.CPU64.NMI:Non-maskable_interrupts 3826 ± 22% +93.1% 7389 ± 6% interrupts.CPU64.PMI:Performance_monitoring_interrupts 3266 ± 37% +126.1% 7387 ± 6% interrupts.CPU66.NMI:Non-maskable_interrupts 3266 ± 37% +126.1% 7387 ± 6% interrupts.CPU66.PMI:Performance_monitoring_interrupts 2821 ± 30% +127.5% 6419 ± 23% interrupts.CPU67.NMI:Non-maskable_interrupts 2821 ± 30% +127.5% 6419 ± 23% interrupts.CPU67.PMI:Performance_monitoring_interrupts 3319 ± 29% +122.4% 7381 ± 6% interrupts.CPU68.NMI:Non-maskable_interrupts 3319 ± 29% +122.4% 7381 ± 6% interrupts.CPU68.PMI:Performance_monitoring_interrupts 3055 ± 24% +114.9% 6566 ± 28% interrupts.CPU69.NMI:Non-maskable_interrupts 3055 ± 24% +114.9% 6566 ± 28% interrupts.CPU69.PMI:Performance_monitoring_interrupts 48.75 +296.9% 193.50 ± 83% interrupts.CPU70.RES:Rescheduling_interrupts 3719 ± 28% +98.7% 7390 ± 6% interrupts.CPU71.NMI:Non-maskable_interrupts 3719 ± 28% +98.7% 7390 ± 6% interrupts.CPU71.PMI:Performance_monitoring_interrupts 3756 ± 19% +98.0% 7439 ± 6% interrupts.CPU72.NMI:Non-maskable_interrupts 3756 ± 19% +98.0% 7439 ± 6% interrupts.CPU72.PMI:Performance_monitoring_interrupts 49.00 ± 14% +243.9% 168.50 ± 50% interrupts.CPU72.RES:Rescheduling_interrupts 3488 ± 21% +113.3% 7440 ± 6% interrupts.CPU76.NMI:Non-maskable_interrupts 3488 ± 21% +113.3% 7440 ± 6% interrupts.CPU76.PMI:Performance_monitoring_interrupts 42.50 ± 7% +294.1% 167.50 ± 38% interrupts.CPU76.RES:Rescheduling_interrupts 3074 ± 31% +84.3% 5666 ± 38% interrupts.CPU77.NMI:Non-maskable_interrupts 3074 ± 31% +84.3% 5666 ± 38% interrupts.CPU77.PMI:Performance_monitoring_interrupts 2363 ± 4% +10.9% 2619 ± 3% interrupts.CPU78.CAL:Function_call_interrupts 3008 ± 34% +145.7% 7392 ± 6% interrupts.CPU78.NMI:Non-maskable_interrupts 3008 ± 34% +145.7% 7392 ± 6% interrupts.CPU78.PMI:Performance_monitoring_interrupts 107.00 ± 79% -75.7% 26.00 ± 66% interrupts.CPU78.TLB:TLB_shootdowns 3956 ± 12% +88.5% 7457 ± 5% interrupts.CPU8.NMI:Non-maskable_interrupts 3956 ± 12% +88.5% 7457 ± 5% interrupts.CPU8.PMI:Performance_monitoring_interrupts 4247 ± 3% +73.5% 7370 ± 6% interrupts.CPU80.NMI:Non-maskable_interrupts 4247 ± 3% +73.5% 7370 ± 6% interrupts.CPU80.PMI:Performance_monitoring_interrupts 4303 ± 12% +70.8% 7347 ± 6% interrupts.CPU81.NMI:Non-maskable_interrupts 4303 ± 12% +70.8% 7347 ± 6% interrupts.CPU81.PMI:Performance_monitoring_interrupts 2379 ± 2% +12.4% 2674 interrupts.CPU84.CAL:Function_call_interrupts 2254 ± 12% +17.6% 2652 interrupts.CPU85.CAL:Function_call_interrupts 4027 ± 7% +81.4% 7305 ± 6% interrupts.CPU85.NMI:Non-maskable_interrupts 4027 ± 7% +81.4% 7305 ± 6% interrupts.CPU85.PMI:Performance_monitoring_interrupts 3105 ± 39% +104.7% 6357 ± 23% interrupts.CPU87.NMI:Non-maskable_interrupts 3105 ± 39% +104.7% 6357 ± 23% interrupts.CPU87.PMI:Performance_monitoring_interrupts 4019 ± 5% +82.8% 7348 ± 6% interrupts.CPU88.NMI:Non-maskable_interrupts 4019 ± 5% +82.8% 7348 ± 6% interrupts.CPU88.PMI:Performance_monitoring_interrupts 2391 ± 3% +12.2% 2682 interrupts.CPU89.CAL:Function_call_interrupts 3851 ± 7% +90.6% 7341 ± 7% interrupts.CPU89.NMI:Non-maskable_interrupts 3851 ± 7% +90.6% 7341 ± 7% interrupts.CPU89.PMI:Performance_monitoring_interrupts 3222 ± 27% +128.6% 7367 ± 6% interrupts.CPU91.NMI:Non-maskable_interrupts 3222 ± 27% +128.6% 7367 ± 6% interrupts.CPU91.PMI:Performance_monitoring_interrupts 3514 ± 28% +110.0% 7380 ± 5% interrupts.CPU92.NMI:Non-maskable_interrupts 3514 ± 28% +110.0% 7380 ± 5% interrupts.CPU92.PMI:Performance_monitoring_interrupts 45.25 ± 11% +97.2% 89.25 ± 11% interrupts.CPU93.RES:Rescheduling_interrupts 2405 ± 2% +11.0% 2668 ± 2% interrupts.CPU94.CAL:Function_call_interrupts 44.00 ± 2% +95.5% 86.00 ± 25% interrupts.CPU94.RES:Rescheduling_interrupts 4226 ± 2% +74.0% 7353 ± 6% interrupts.CPU95.NMI:Non-maskable_interrupts 4226 ± 2% +74.0% 7353 ± 6% interrupts.CPU95.PMI:Performance_monitoring_interrupts 43.75 ± 7% +385.1% 212.25 ±102% interrupts.CPU96.RES:Rescheduling_interrupts 72.00 ± 32% +142.0% 174.25 ± 28% interrupts.CPU97.RES:Rescheduling_interrupts 3933 ± 7% +86.7% 7341 ± 6% interrupts.CPU98.NMI:Non-maskable_interrupts 3933 ± 7% +86.7% 7341 ± 6% interrupts.CPU98.PMI:Performance_monitoring_interrupts 3318 ± 20% +122.8% 7391 ± 6% interrupts.CPU99.NMI:Non-maskable_interrupts 3318 ± 20% +122.8% 7391 ± 6% interrupts.CPU99.PMI:Performance_monitoring_interrupts 96.00 ± 7% +143.8% 234.00 ± 17% interrupts.IWI:IRQ_work_interrupts 392529 ± 7% +76.8% 694028 ± 6% interrupts.NMI:Non-maskable_interrupts 392529 ± 7% +76.8% 694028 ± 6% interrupts.PMI:Performance_monitoring_interrupts *************************************************************************************************** lkp-bdw-ex2: 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ex2/lru-file-readonce/vm-scalability/0xb00002e commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at_ip__slab_free/0x %stddev %change %stddev \ | \ 149757 -22.0% 116870 ± 2% vm-scalability.median 29024147 -22.1% 22614887 ± 2% vm-scalability.throughput 283.88 +13.8% 323.02 vm-scalability.time.elapsed_time 283.88 +13.8% 323.02 vm-scalability.time.elapsed_time.max 176801 +19.9% 211950 vm-scalability.time.involuntary_context_switches 9980 +4.5% 10428 ± 2% vm-scalability.time.percent_of_cpu_this_job_got 27969 +18.9% 33254 vm-scalability.time.system_time 361.20 ± 2% +17.9% 425.95 ± 2% vm-scalability.time.user_time 0.68 ± 3% +0.1 0.82 ± 3% mpstat.cpu.usr% 5323161 ±104% -98.1% 103428 ± 15% cpuidle.POLL.time 334639 ±110% -87.4% 42155 ± 7% cpuidle.POLL.usage 4125 ± 4% +9.6% 4520 ± 4% slabinfo.kmalloc-rcl-64.active_objs 4125 ± 4% +9.6% 4520 ± 4% slabinfo.kmalloc-rcl-64.num_objs 1392 +11.8% 1557 ± 2% turbostat.Avg_MHz 54.35 +6.3 60.63 ± 2% turbostat.Busy% 440.27 +3.3% 454.93 turbostat.PkgWatt 47.25 ± 2% -10.1% 42.50 ± 4% vmstat.cpu.id 51.00 ± 2% +9.3% 55.75 ± 3% vmstat.cpu.sy 3.752e+08 ± 2% -10.8% 3.347e+08 ± 2% vmstat.memory.cache 100.75 ± 2% +15.9% 116.75 ± 3% vmstat.procs.r 329415 +20.4% 396485 ± 3% meminfo.Active 328936 +20.4% 395953 ± 3% meminfo.Active(anon) 3.688e+08 ± 2% -11.7% 3.257e+08 ± 2% meminfo.Cached 354589 ± 8% -57.9% 149388 ± 8% meminfo.DirectMap4k 3.678e+08 ± 2% -11.7% 3.247e+08 ± 2% meminfo.Inactive 3.676e+08 ± 2% -11.7% 3.244e+08 ± 2% meminfo.Inactive(file) 32189 ± 2% +19.1% 38323 ± 9% meminfo.Mapped 428.00 ±173% +125.3% 964.25 ± 83% meminfo.Mlocked 331152 +20.9% 400361 ± 4% meminfo.Shmem 98666548 ± 3% -12.3% 86550447 ± 3% numa-meminfo.node0.FilePages 98416859 ± 3% -12.3% 86311727 ± 3% numa-meminfo.node0.Inactive 98338794 ± 3% -12.3% 86291896 ± 3% numa-meminfo.node0.Inactive(file) 74554608 ± 3% -12.7% 65056366 ± 3% numa-meminfo.node1.FilePages 74295173 ± 3% -12.8% 64762087 ± 3% numa-meminfo.node1.Inactive 74227718 ± 3% -12.8% 64697317 ± 3% numa-meminfo.node1.Inactive(file) 24516 ±108% +244.5% 84471 ± 52% numa-meminfo.node2.Active 24476 ±108% +244.7% 84380 ± 52% numa-meminfo.node2.Active(anon) 23235 ±112% +171.1% 62998 ± 55% numa-meminfo.node2.AnonPages 1.441e+08 ± 2% -11.5% 1.276e+08 ± 4% numa-meminfo.node2.FilePages 1.439e+08 ± 2% -11.5% 1.273e+08 ± 4% numa-meminfo.node2.Inactive 827.25 ± 53% +7333.5% 61493 ±166% numa-meminfo.node2.Inactive(anon) 1.439e+08 ± 2% -11.6% 1.272e+08 ± 4% numa-meminfo.node2.Inactive(file) 5787 ± 3% +16.5% 6742 ± 10% numa-meminfo.node2.Mapped 2111 ± 12% +3829.5% 82952 ±124% numa-meminfo.node2.Shmem 48843863 ± 6% -11.5% 43227567 ± 2% numa-meminfo.node3.FilePages 48591680 ± 6% -11.6% 42943754 ± 2% numa-meminfo.node3.Inactive 48468125 ± 7% -11.7% 42816103 ± 2% numa-meminfo.node3.Inactive(file) 2.433e+08 -22.8% 1.878e+08 numa-numastat.node0.local_node 53300661 ± 3% +76.3% 93991570 numa-numastat.node0.numa_foreign 2.433e+08 -22.8% 1.878e+08 numa-numastat.node0.numa_hit 47555438 ± 2% +123.3% 1.062e+08 ± 3% numa-numastat.node0.numa_miss 47569462 ± 2% +123.3% 1.062e+08 ± 3% numa-numastat.node0.other_node 2.137e+08 -22.9% 1.648e+08 ± 2% numa-numastat.node1.local_node 76801030 +34.5% 1.033e+08 numa-numastat.node1.numa_foreign 2.137e+08 -22.9% 1.648e+08 ± 2% numa-numastat.node1.numa_hit 25699468 ± 7% +182.9% 72697000 ± 3% numa-numastat.node1.numa_miss 25708830 ± 7% +182.8% 72706419 ± 3% numa-numastat.node1.other_node 2.201e+08 -14.3% 1.886e+08 numa-numastat.node2.local_node 9556272 ± 16% +735.3% 79819118 ± 4% numa-numastat.node2.numa_foreign 2.201e+08 -14.3% 1.886e+08 numa-numastat.node2.numa_hit 1.493e+08 +13.0% 1.688e+08 ± 2% numa-numastat.node2.numa_miss 1.493e+08 +13.0% 1.688e+08 ± 2% numa-numastat.node2.other_node 1.713e+08 ± 2% -24.6% 1.293e+08 numa-numastat.node3.local_node 86932029 +46.7% 1.275e+08 ± 3% numa-numastat.node3.numa_foreign 1.713e+08 ± 2% -24.6% 1.293e+08 numa-numastat.node3.numa_hit 4021174 ± 34% +1316.9% 56976999 ± 2% numa-numastat.node3.numa_miss 4035220 ± 34% +1312.3% 56989607 ± 2% numa-numastat.node3.other_node 16810 ± 28% +61.5% 27143 ± 13% softirqs.CPU0.RCU 21255 ± 16% +54.9% 32929 ± 21% softirqs.CPU1.RCU 19349 ± 26% +51.0% 29218 ± 20% softirqs.CPU10.RCU 18178 ± 19% +55.4% 28256 ± 4% softirqs.CPU100.RCU 17805 ± 23% +54.3% 27468 ± 4% softirqs.CPU101.RCU 19843 ± 15% +44.9% 28746 ± 7% softirqs.CPU103.RCU 16931 ± 8% +75.5% 29707 ± 34% softirqs.CPU105.RCU 16620 ± 17% +65.3% 27477 ± 25% softirqs.CPU107.RCU 20242 ± 36% +49.8% 30318 ± 14% softirqs.CPU11.RCU 15980 ± 31% +77.6% 28386 ± 6% softirqs.CPU112.RCU 19796 ± 5% -8.6% 18089 ± 4% softirqs.CPU112.SCHED 11978 ± 23% +56.2% 18715 ± 8% softirqs.CPU114.RCU 18643 ± 19% +82.8% 34089 ± 18% softirqs.CPU12.RCU 13720 ± 39% +33.1% 18256 ± 2% softirqs.CPU122.SCHED 14471 ± 18% +30.7% 18907 ± 11% softirqs.CPU129.RCU 13947 ± 22% +36.5% 19038 ± 12% softirqs.CPU131.RCU 15119 ± 29% +128.3% 34509 ± 22% softirqs.CPU16.RCU 13855 ± 11% +59.4% 22080 ± 28% softirqs.CPU177.RCU 15513 ± 14% +54.5% 23964 ± 31% softirqs.CPU178.RCU 14342 ± 35% +104.2% 29284 ± 41% softirqs.CPU18.RCU 15605 ± 17% +86.4% 29092 ± 43% softirqs.CPU181.RCU 15357 ± 33% +68.1% 25807 ± 12% softirqs.CPU20.RCU 17225 ± 3% +46.8% 25292 ± 50% softirqs.CPU29.SCHED 17120 ± 17% +103.8% 34898 ± 38% softirqs.CPU3.RCU 20996 ± 44% +109.0% 43888 ± 48% softirqs.CPU4.RCU 17242 ± 24% +77.7% 30646 ± 19% softirqs.CPU5.RCU 14106 ± 24% +50.2% 21188 ± 22% softirqs.CPU58.RCU 16569 ± 5% +82.3% 30204 ± 65% softirqs.CPU58.SCHED 14260 ± 23% +27.7% 18217 ± 5% softirqs.CPU61.SCHED 16236 ± 25% +47.5% 23948 ± 24% softirqs.CPU63.RCU 15675 ± 11% +14.8% 18001 ± 4% softirqs.CPU63.SCHED 11557 ± 17% +68.0% 19412 ± 26% softirqs.CPU66.RCU 16534 ± 3% +27.4% 21069 ± 20% softirqs.CPU67.SCHED 16592 ± 3% +12.1% 18601 ± 5% softirqs.CPU68.SCHED 13740 ± 10% +75.2% 24069 ± 40% softirqs.CPU81.RCU 15212 ± 26% +49.2% 22701 ± 26% softirqs.CPU82.RCU 15060 ± 22% +64.4% 24753 ± 30% softirqs.CPU89.RCU 16376 ± 18% +103.4% 33305 ± 36% softirqs.CPU9.RCU 19775 ± 27% +43.3% 28337 ± 6% softirqs.CPU99.RCU 9064 ± 2% +148.0% 22483 ± 56% softirqs.NET_RX 24487361 ± 3% -12.1% 21530346 ± 3% numa-vmstat.node0.nr_file_pages 24405353 ± 3% -12.0% 21465639 ± 3% numa-vmstat.node0.nr_inactive_file 22.50 ±173% +175.6% 62.00 ± 85% numa-vmstat.node0.nr_mlock 24405355 ± 3% -12.0% 21465570 ± 3% numa-vmstat.node0.nr_zone_inactive_file 40876228 ± 4% +68.7% 68971758 ± 2% numa-vmstat.node0.numa_foreign 1.858e+08 -23.3% 1.425e+08 numa-vmstat.node0.numa_hit 1.858e+08 -23.3% 1.425e+08 numa-vmstat.node0.numa_local 33943980 ± 3% +129.7% 77965910 ± 3% numa-vmstat.node0.numa_miss 33959249 ± 3% +129.6% 77984122 ± 3% numa-vmstat.node0.numa_other 18504940 ± 3% -12.5% 16196899 ± 3% numa-vmstat.node1.nr_file_pages 18423198 ± 3% -12.6% 16106972 ± 4% numa-vmstat.node1.nr_inactive_file 18423160 ± 3% -12.6% 16106931 ± 4% numa-vmstat.node1.nr_zone_inactive_file 58534808 +29.3% 75671126 numa-vmstat.node1.numa_foreign 1.614e+08 -22.7% 1.247e+08 ± 2% numa-vmstat.node1.numa_hit 1.613e+08 -22.7% 1.246e+08 ± 2% numa-vmstat.node1.numa_local 17656843 ± 11% +194.0% 51908928 ± 5% numa-vmstat.node1.numa_miss 17742462 ± 11% +193.1% 51994956 ± 5% numa-vmstat.node1.numa_other 108492 ± 26% +955.2% 1144846 ± 23% numa-vmstat.node1.workingset_nodereclaim 6097 ±108% +246.7% 21142 ± 52% numa-vmstat.node2.nr_active_anon 5784 ±112% +171.9% 15730 ± 55% numa-vmstat.node2.nr_anon_pages 35712617 ± 2% -11.1% 31742917 ± 4% numa-vmstat.node2.nr_file_pages 207.25 ± 53% +7335.2% 15409 ±166% numa-vmstat.node2.nr_inactive_anon 35655102 ± 2% -11.2% 31665009 ± 4% numa-vmstat.node2.nr_inactive_file 1448 ± 2% +14.4% 1656 ± 9% numa-vmstat.node2.nr_mapped 22.75 ±173% +135.2% 53.50 ± 84% numa-vmstat.node2.nr_mlock 531.75 ± 12% +3819.4% 20841 ±124% numa-vmstat.node2.nr_shmem 6097 ±108% +246.7% 21142 ± 52% numa-vmstat.node2.nr_zone_active_anon 207.25 ± 53% +7335.2% 15409 ±166% numa-vmstat.node2.nr_zone_inactive_anon 35655092 ± 2% -11.2% 31664952 ± 4% numa-vmstat.node2.nr_zone_inactive_file 4825057 ± 15% +1104.6% 58121318 ± 4% numa-vmstat.node2.numa_foreign 1.653e+08 -12.9% 1.439e+08 numa-vmstat.node2.numa_hit 1.652e+08 -12.9% 1.438e+08 numa-vmstat.node2.numa_local 12116726 ± 6% -11.2% 10763478 ± 2% numa-vmstat.node3.nr_file_pages 12022889 ± 7% -11.3% 10660471 ± 2% numa-vmstat.node3.nr_inactive_file 12022575 ± 7% -11.3% 10660512 ± 2% numa-vmstat.node3.nr_zone_inactive_file 64859908 ± 2% +44.0% 93368498 ± 2% numa-vmstat.node3.numa_foreign 1.289e+08 ± 2% -24.9% 96858785 numa-vmstat.node3.numa_hit 1.288e+08 ± 2% -24.9% 96770034 numa-vmstat.node3.numa_local 2170453 ± 31% +1797.2% 41177633 ± 3% numa-vmstat.node3.numa_miss 2260238 ± 30% +1725.8% 41266426 ± 3% numa-vmstat.node3.numa_other 32345 ± 99% +214.5% 101717 ± 33% numa-vmstat.node3.workingset_nodereclaim 24695 +10.8% 27358 proc-vmstat.allocstall_movable 6.346e+08 ± 3% -29.9% 4.446e+08 ± 23% proc-vmstat.compact_daemon_free_scanned 2001184 ± 11% -26.0% 1480770 ± 19% proc-vmstat.compact_daemon_migrate_scanned 1893 ± 4% -46.6% 1010 ± 11% proc-vmstat.compact_daemon_wake 6.401e+08 ± 2% -30.2% 4.465e+08 ± 23% proc-vmstat.compact_free_scanned 3695636 ± 12% -44.5% 2050018 ± 19% proc-vmstat.compact_isolated 2016804 ± 11% -26.3% 1485887 ± 19% proc-vmstat.compact_migrate_scanned 1620 ± 4% -38.6% 995.25 ± 6% proc-vmstat.kswapd_low_wmark_hit_quickly 82304 +20.3% 98994 ± 3% proc-vmstat.nr_active_anon 12129113 -8.1% 11149117 proc-vmstat.nr_dirty_background_threshold 24287881 -8.1% 22325496 proc-vmstat.nr_dirty_threshold 91989167 ± 2% -11.5% 81401595 ± 2% proc-vmstat.nr_file_pages 67413 +1.7% 68586 proc-vmstat.nr_inactive_anon 91674177 ± 2% -11.6% 81066982 ± 2% proc-vmstat.nr_inactive_file 28849 +0.9% 29118 proc-vmstat.nr_kernel_stack 8211 ± 2% +18.4% 9726 ± 8% proc-vmstat.nr_mapped 108.00 ±173% +122.2% 240.00 ± 83% proc-vmstat.nr_mlock 2744 +5.1% 2886 ± 2% proc-vmstat.nr_page_table_pages 82952 +20.8% 100187 ± 4% proc-vmstat.nr_shmem 1213328 ± 2% -4.9% 1153566 proc-vmstat.nr_slab_reclaimable 82305 +20.3% 98994 ± 3% proc-vmstat.nr_zone_active_anon 67413 +1.7% 68586 proc-vmstat.nr_zone_inactive_anon 91673929 ± 2% -11.6% 81067074 ± 2% proc-vmstat.nr_zone_inactive_file 2.266e+08 +78.6% 4.046e+08 proc-vmstat.numa_foreign 3863 ± 10% +38.0% 5330 ± 17% proc-vmstat.numa_hint_faults 1260 ± 35% +95.9% 2469 ± 27% proc-vmstat.numa_hint_faults_local 8.485e+08 -21.0% 6.705e+08 proc-vmstat.numa_hit 8.484e+08 -21.0% 6.705e+08 proc-vmstat.numa_local 2.266e+08 +78.6% 4.046e+08 proc-vmstat.numa_miss 2.266e+08 +78.6% 4.047e+08 proc-vmstat.numa_other 2345 ± 2% -40.8% 1388 ± 6% proc-vmstat.pageoutrun 26369 ± 6% +92.2% 50677 ± 15% proc-vmstat.pgactivate 2704986 ± 2% +9.2% 2955110 ± 2% proc-vmstat.pgalloc_dma32 1091694 ± 2% +7.0% 1168439 ± 2% proc-vmstat.pgfault 79.25 ± 11% +4201.3% 3408 ± 75% proc-vmstat.pgmigrate_fail 1847086 ± 12% -44.4% 1026471 ± 19% proc-vmstat.pgmigrate_success 7.274e+08 -3.2% 7.043e+08 proc-vmstat.pgscan_direct 2.26e+08 ± 4% +14.8% 2.595e+08 proc-vmstat.pgscan_kswapd 7.274e+08 -3.2% 7.043e+08 proc-vmstat.pgsteal_direct 2.26e+08 ± 4% +14.8% 2.595e+08 proc-vmstat.pgsteal_kswapd 539278 ± 18% +838.1% 5059105 ± 8% proc-vmstat.slabs_scanned 159579 ± 25% +936.9% 1654700 ± 7% proc-vmstat.workingset_nodereclaim 6704363 ± 3% -5.6% 6330017 proc-vmstat.workingset_nodes 67262838 ± 3% -12.7% 58735393 ± 6% perf-stat.i.branch-misses 7.163e+08 -9.2% 6.503e+08 perf-stat.i.cache-references 3.25 ± 5% +11.1% 3.61 ± 8% perf-stat.i.cpi 2.474e+11 +9.8% 2.718e+11 ± 4% perf-stat.i.cpu-cycles 42.40 ± 2% +15.1% 48.79 ± 3% perf-stat.i.cpu-migrations 7743016 ± 9% +51.1% 11699161 ± 10% perf-stat.i.dTLB-load-misses 0.05 ± 27% +0.0 0.06 ± 20% perf-stat.i.dTLB-store-miss-rate% 822758 ± 35% +43.2% 1178183 ± 23% perf-stat.i.dTLB-store-misses 5.521e+09 -10.1% 4.961e+09 ± 2% perf-stat.i.dTLB-stores 64.57 ± 10% +10.3 74.87 ± 7% perf-stat.i.iTLB-load-miss-rate% 16065655 -10.0% 14462032 ± 2% perf-stat.i.iTLB-load-misses 8683333 ± 5% -20.1% 6940831 ± 4% perf-stat.i.iTLB-loads 385229 +2.5% 394939 perf-stat.i.msec 49.90 ± 5% +5.4 55.27 ± 2% perf-stat.i.node-store-miss-rate% 6135382 ± 3% +9.5% 6718006 ± 2% perf-stat.i.node-store-misses 7985591 ± 2% -27.4% 5797653 ± 3% perf-stat.i.node-stores 10.55 -16.6% 8.80 ± 3% perf-stat.overall.MPKI 0.42 -0.1 0.32 ± 6% perf-stat.overall.branch-miss-rate% 7.28 ± 2% +0.5 7.78 ± 2% perf-stat.overall.cache-miss-rate% 3.74 +6.4% 3.98 perf-stat.overall.cpi 4877 +19.5% 5828 perf-stat.overall.cycles-between-cache-misses 0.04 ± 10% +0.0 0.06 ± 10% perf-stat.overall.dTLB-load-miss-rate% 0.01 ± 35% +0.0 0.02 ± 22% perf-stat.overall.dTLB-store-miss-rate% 65.30 +2.3 67.59 perf-stat.overall.iTLB-load-miss-rate% 4206 +19.4% 5024 ± 3% perf-stat.overall.instructions-per-iTLB-miss 0.27 -6.0% 0.25 perf-stat.overall.ipc 43.63 +10.2 53.79 perf-stat.overall.node-store-miss-rate% 4722 +19.8% 5658 ± 2% perf-stat.overall.path-length 1.635e+10 +7.1% 1.75e+10 ± 3% perf-stat.ps.branch-instructions 67885613 ± 2% -17.3% 56158420 ± 4% perf-stat.ps.branch-misses 54478471 ± 3% -6.2% 51103235 ± 2% perf-stat.ps.cache-misses 7.483e+08 -12.2% 6.571e+08 perf-stat.ps.cache-references 2.656e+11 +12.1% 2.978e+11 ± 2% perf-stat.ps.cpu-cycles 45.86 +18.0% 54.10 ± 3% perf-stat.ps.cpu-migrations 7776798 ± 8% +45.9% 11346164 ± 9% perf-stat.ps.dTLB-load-misses 5.754e+09 -12.8% 5.02e+09 ± 2% perf-stat.ps.dTLB-stores 16872171 -11.7% 14890921 ± 2% perf-stat.ps.iTLB-load-misses 8972982 ± 4% -20.4% 7143013 ± 4% perf-stat.ps.iTLB-loads 3692 -5.8% 3479 perf-stat.ps.minor-faults 7881782 ± 4% +6.8% 8417494 perf-stat.ps.node-load-misses 2817377 ± 3% +14.3% 3221371 ± 5% perf-stat.ps.node-loads 6539081 ± 3% +8.5% 7091829 ± 2% perf-stat.ps.node-store-misses 8444206 -27.8% 6093062 ± 3% perf-stat.ps.node-stores 3693 -5.8% 3479 perf-stat.ps.page-faults 2.028e+13 +19.8% 2.43e+13 ± 2% perf-stat.total.instructions 30.74 ± 20% -30.7 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64 30.63 ± 19% -30.6 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 30.63 ± 19% -30.6 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 30.62 ± 19% -30.6 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 30.20 ± 20% -30.2 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 29.35 ± 20% -29.4 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 21.40 ± 16% -18.4 3.03 ± 21% perf-profile.calltrace.cycles-pp.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 21.31 ± 16% -18.3 3.01 ± 21% perf-profile.calltrace.cycles-pp.wakeup_kswapd.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead 7.36 ± 16% -6.3 1.08 ± 20% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 10.33 ± 8% -3.4 6.94 ± 6% perf-profile.calltrace.cycles-pp.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 4.10 ± 7% -2.1 1.98 ± 6% perf-profile.calltrace.cycles-pp.iomap_set_range_uptodate.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages 3.28 ± 11% -1.7 1.56 ± 17% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 3.28 ± 11% -1.7 1.56 ± 17% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 3.28 ± 11% -1.7 1.56 ± 17% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 3.46 ± 14% -1.6 1.82 ± 18% perf-profile.calltrace.cycles-pp.ret_from_fork 3.46 ± 14% -1.6 1.82 ± 18% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 2.98 ± 14% -1.4 1.55 ± 17% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 2.98 ± 14% -1.4 1.55 ± 16% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 2.74 ± 13% -1.4 1.34 ± 4% perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 2.58 ± 13% -1.4 1.21 ± 4% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 2.06 ± 15% -1.4 0.69 ± 20% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 2.56 ± 13% -1.4 1.20 ± 4% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.generic_file_read_iter.xfs_file_buffered_aio_read 6.03 ± 9% -1.3 4.71 ± 6% perf-profile.calltrace.cycles-pp.memset_erms.iomap_readpage_actor.iomap_readpages_actor.iomap_apply.iomap_readpages 1.04 ± 9% -0.4 0.60 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 0.93 ± 10% -0.3 0.65 ± 9% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 0.00 +0.7 0.71 ± 19% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.32 ± 32% +2.3 3.65 ± 25% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.13 ±173% +2.6 2.71 ± 36% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.00 +2.6 2.63 ± 37% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 0.72 ± 61% +3.5 4.25 ± 21% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 16.91 ± 6% +16.1 33.01 ± 14% perf-profile.calltrace.cycles-pp.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 16.88 ± 6% +16.1 32.99 ± 14% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead 16.88 ± 6% +16.1 33.00 ± 14% perf-profile.calltrace.cycles-pp.iomap_readpages.read_pages.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter 16.79 ± 6% +16.2 32.94 ± 14% perf-profile.calltrace.cycles-pp.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages.__do_page_cache_readahead 40.48 ± 10% +19.0 59.49 ± 7% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read 5.24 ± 7% +19.4 24.69 ± 22% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 5.18 ± 7% +19.5 24.64 ± 22% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 6.26 ± 6% +19.6 25.88 ± 20% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 4.33 ± 8% +19.7 23.98 ± 23% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 4.22 ± 9% +19.7 23.94 ± 23% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 5.51 ± 26% +29.6 35.16 ± 15% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages 5.51 ± 26% +30.3 35.85 ± 15% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node 63.49 ± 8% +32.9 96.36 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 63.40 ± 8% +32.9 96.31 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 62.53 ± 8% +33.1 95.60 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 62.36 ± 8% +33.1 95.51 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 61.95 ± 8% +33.2 95.15 perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 61.88 ± 8% +33.2 95.09 perf-profile.calltrace.cycles-pp.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read.do_syscall_64 61.66 ± 8% +33.3 94.94 perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read.ksys_read 61.45 ± 8% +33.3 94.78 perf-profile.calltrace.cycles-pp.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read.vfs_read 6.53 ± 25% +33.6 40.08 ± 12% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages 6.53 ± 25% +33.6 40.10 ± 12% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 6.61 ± 26% +34.0 40.61 ± 13% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 6.61 ± 26% +34.0 40.61 ± 13% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 6.61 ± 26% +34.0 40.61 ± 13% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead 57.78 ± 8% +35.0 92.79 perf-profile.calltrace.cycles-pp.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter.__vfs_read 57.77 ± 8% +35.0 92.79 perf-profile.calltrace.cycles-pp.__do_page_cache_readahead.ondemand_readahead.generic_file_read_iter.xfs_file_buffered_aio_read.xfs_file_read_iter 95264 +29.7% 123547 sched_debug.cfs_rq:/.exec_clock.avg 105896 ± 6% +21.2% 128329 sched_debug.cfs_rq:/.exec_clock.max 87871 +34.2% 117885 sched_debug.cfs_rq:/.exec_clock.min 3113 ± 5% -46.7% 1658 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev 11232 ± 3% +39.2% 15633 ± 17% sched_debug.cfs_rq:/.load.avg 611246 ± 2% +11.7% 682682 ± 3% sched_debug.cfs_rq:/.load.max 70385 ± 3% +26.8% 89232 ± 10% sched_debug.cfs_rq:/.load.stddev 13.69 ± 19% +190.3% 39.74 ± 80% sched_debug.cfs_rq:/.load_avg.avg 80.92 ± 8% +278.8% 306.49 ±119% sched_debug.cfs_rq:/.load_avg.stddev 18658844 +22.2% 22794459 sched_debug.cfs_rq:/.min_vruntime.avg 19438672 +24.4% 24186120 sched_debug.cfs_rq:/.min_vruntime.max 11351689 ± 7% +42.7% 16198558 ± 3% sched_debug.cfs_rq:/.min_vruntime.min 912409 ± 4% +19.0% 1086091 ± 3% sched_debug.cfs_rq:/.min_vruntime.stddev 0.42 +20.0% 0.50 ± 3% sched_debug.cfs_rq:/.nr_running.avg 28.47 ± 9% +133.1% 66.36 ± 4% sched_debug.cfs_rq:/.nr_spread_over.avg 3.95 ± 17% +767.1% 34.25 ± 12% sched_debug.cfs_rq:/.nr_spread_over.min 8.92 ± 4% +34.1% 11.96 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.avg 589.65 ± 3% +11.6% 658.04 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max 59.65 ± 5% +25.2% 74.67 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev 10938 ± 5% +42.7% 15609 ± 17% sched_debug.cfs_rq:/.runnable_weight.avg 610397 ± 2% +11.7% 681818 ± 3% sched_debug.cfs_rq:/.runnable_weight.max 69142 ± 3% +29.0% 89191 ± 10% sched_debug.cfs_rq:/.runnable_weight.stddev 912286 ± 4% +18.8% 1084064 ± 3% sched_debug.cfs_rq:/.spread0.stddev 441.12 +18.5% 522.75 ± 2% sched_debug.cfs_rq:/.util_avg.avg 136.64 ± 40% +90.0% 259.67 ± 26% sched_debug.cfs_rq:/.util_est_enqueued.avg 184631 ± 4% +13.8% 210194 sched_debug.cpu.clock.avg 184662 ± 4% +13.9% 210354 sched_debug.cpu.clock.max 184592 ± 4% +13.8% 210099 sched_debug.cpu.clock.min 184631 ± 4% +13.8% 210194 sched_debug.cpu.clock_task.avg 184662 ± 4% +13.9% 210354 sched_debug.cpu.clock_task.max 184592 ± 4% +13.8% 210099 sched_debug.cpu.clock_task.min 9.43 ± 6% +26.1% 11.90 ± 7% sched_debug.cpu.cpu_load[0].avg 9.33 ± 5% +28.2% 11.96 ± 8% sched_debug.cpu.cpu_load[1].avg 62.06 ± 9% +20.6% 74.82 ± 6% sched_debug.cpu.cpu_load[1].stddev 9.28 ± 3% +30.2% 12.08 ± 6% sched_debug.cpu.cpu_load[2].avg 631.25 ± 4% +8.9% 687.17 ± 6% sched_debug.cpu.cpu_load[2].max 61.37 ± 6% +23.7% 75.89 ± 4% sched_debug.cpu.cpu_load[2].stddev 9.23 ± 2% +37.8% 12.72 ± 3% sched_debug.cpu.cpu_load[3].avg 61.96 ± 4% +24.8% 77.30 sched_debug.cpu.cpu_load[3].stddev 9.17 ± 2% +52.5% 13.99 ± 5% sched_debug.cpu.cpu_load[4].avg 63.50 ± 5% +25.4% 79.66 ± 2% sched_debug.cpu.cpu_load[4].stddev 1359 +19.4% 1622 ± 2% sched_debug.cpu.curr->pid.avg 10944 ± 5% +44.9% 15860 ± 17% sched_debug.cpu.load.avg 610821 ± 2% +15.6% 705869 ± 7% sched_debug.cpu.load.max 69155 ± 3% +31.3% 90813 ± 10% sched_debug.cpu.load.stddev 121577 +8.9% 132381 ± 3% sched_debug.cpu.nr_load_updates.avg 129292 +15.2% 149001 ± 7% sched_debug.cpu.nr_load_updates.max 119185 +8.1% 128893 ± 3% sched_debug.cpu.nr_load_updates.min 1277 ± 4% +78.1% 2274 ± 7% sched_debug.cpu.nr_load_updates.stddev 0.43 +20.9% 0.52 ± 3% sched_debug.cpu.nr_running.avg 1.40 +22.0% 1.71 ± 12% sched_debug.cpu.nr_running.max 2364 +13.7% 2687 sched_debug.cpu.nr_switches.avg 2459 +14.5% 2815 sched_debug.cpu.sched_count.avg 806.55 ± 2% +13.6% 916.38 ± 3% sched_debug.cpu.sched_count.min 855.88 +12.3% 960.87 sched_debug.cpu.ttwu_count.avg 9197 ± 2% +24.4% 11445 ± 10% sched_debug.cpu.ttwu_count.max 275.65 ± 2% +10.3% 304.08 sched_debug.cpu.ttwu_count.min 628.78 ± 2% +15.6% 726.59 ± 2% sched_debug.cpu.ttwu_local.avg 239.65 ± 2% +10.2% 264.17 sched_debug.cpu.ttwu_local.min 184592 ± 4% +13.8% 210099 sched_debug.cpu_clk 179423 ± 4% +14.2% 204930 sched_debug.ktime 1.51 +25.0% 1.89 sched_debug.rt_rq:/.rt_runtime.stddev 189822 ± 4% +12.9% 214356 sched_debug.sched_clk 150.25 ± 5% +54.6% 232.25 ± 34% interrupts.118:IR-PCI-MSI.1574920-edge.eth1-TxRx-8 157.00 ± 4% +49.5% 234.75 ± 45% interrupts.126:IR-PCI-MSI.1574924-edge.eth1-TxRx-12 171.25 ± 25% +49.2% 255.50 ± 43% interrupts.128:IR-PCI-MSI.1574926-edge.eth1-TxRx-14 143.50 ± 4% +9.6% 157.25 ± 3% interrupts.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17 141.25 ± 2% +25.7% 177.50 ± 18% interrupts.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18 142.50 ± 3% +10.2% 157.00 ± 2% interrupts.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19 140.75 ± 2% +12.6% 158.50 ± 4% interrupts.136:IR-PCI-MSI.1574934-edge.eth1-TxRx-22 141.00 +10.8% 156.25 ± 2% interrupts.137:IR-PCI-MSI.1574935-edge.eth1-TxRx-23 140.50 +11.0% 156.00 ± 2% interrupts.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24 141.00 +12.8% 159.00 ± 4% interrupts.139:IR-PCI-MSI.1574937-edge.eth1-TxRx-25 140.50 +12.1% 157.50 ± 3% interrupts.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26 140.50 +11.4% 156.50 interrupts.141:IR-PCI-MSI.1574939-edge.eth1-TxRx-27 140.50 +11.2% 156.25 ± 2% interrupts.142:IR-PCI-MSI.1574940-edge.eth1-TxRx-28 140.50 +13.0% 158.75 ± 3% interrupts.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29 141.00 +12.9% 159.25 interrupts.144:IR-PCI-MSI.1574942-edge.eth1-TxRx-30 140.50 +11.4% 156.50 ± 2% interrupts.145:IR-PCI-MSI.1574943-edge.eth1-TxRx-31 140.75 +10.8% 156.00 ± 2% interrupts.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33 140.50 +11.2% 156.25 ± 2% interrupts.149:IR-PCI-MSI.1574947-edge.eth1-TxRx-35 140.75 +11.2% 156.50 ± 2% interrupts.150:IR-PCI-MSI.1574948-edge.eth1-TxRx-36 140.75 +11.4% 156.75 ± 2% interrupts.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37 140.50 +11.4% 156.50 ± 2% interrupts.152:IR-PCI-MSI.1574950-edge.eth1-TxRx-38 140.50 +12.3% 157.75 ± 3% interrupts.154:IR-PCI-MSI.1574952-edge.eth1-TxRx-40 140.50 +11.0% 156.00 ± 2% interrupts.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41 140.75 +10.8% 156.00 ± 2% interrupts.156:IR-PCI-MSI.1574954-edge.eth1-TxRx-42 140.50 +11.0% 156.00 ± 2% interrupts.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43 140.75 +11.2% 156.50 ± 2% interrupts.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44 140.75 +11.2% 156.50 ± 2% interrupts.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45 140.50 +11.2% 156.25 ± 2% interrupts.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46 140.50 +11.2% 156.25 ± 2% interrupts.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47 141.00 +10.6% 156.00 ± 2% interrupts.162:IR-PCI-MSI.1574960-edge.eth1-TxRx-48 140.50 +12.3% 157.75 ± 2% interrupts.163:IR-PCI-MSI.1574961-edge.eth1-TxRx-49 140.50 +11.0% 156.00 ± 2% interrupts.164:IR-PCI-MSI.1574962-edge.eth1-TxRx-50 140.50 +11.4% 156.50 ± 2% interrupts.165:IR-PCI-MSI.1574963-edge.eth1-TxRx-51 140.50 +60.5% 225.50 ± 52% interrupts.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53 140.50 +11.2% 156.25 ± 2% interrupts.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55 140.75 +11.2% 156.50 ± 2% interrupts.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56 140.50 +11.2% 156.25 ± 2% interrupts.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58 141.00 +12.4% 158.50 ± 3% interrupts.173:IR-PCI-MSI.1574971-edge.eth1-TxRx-59 141.00 +10.6% 156.00 ± 2% interrupts.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60 140.50 +11.6% 156.75 ± 2% interrupts.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61 140.50 +11.6% 156.75 ± 2% interrupts.176:IR-PCI-MSI.1574974-edge.eth1-TxRx-62 622415 +18.0% 734601 ± 9% interrupts.CAL:Function_call_interrupts 3155 ± 3% +21.6% 3838 ± 9% interrupts.CPU0.CAL:Function_call_interrupts 18048 ±124% -93.3% 1215 ± 20% interrupts.CPU0.RES:Rescheduling_interrupts 3221 +19.8% 3861 ± 9% interrupts.CPU1.CAL:Function_call_interrupts 2284 ± 54% +174.1% 6260 ± 16% interrupts.CPU1.NMI:Non-maskable_interrupts 2284 ± 54% +174.1% 6260 ± 16% interrupts.CPU1.PMI:Performance_monitoring_interrupts 2211 ± 84% +149.9% 5525 ± 35% interrupts.CPU100.NMI:Non-maskable_interrupts 2211 ± 84% +149.9% 5525 ± 35% interrupts.CPU100.PMI:Performance_monitoring_interrupts 401.75 ± 75% -73.2% 107.50 ± 56% interrupts.CPU100.RES:Rescheduling_interrupts 2637 ± 24% +131.6% 6107 ± 14% interrupts.CPU101.NMI:Non-maskable_interrupts 2637 ± 24% +131.6% 6107 ± 14% interrupts.CPU101.PMI:Performance_monitoring_interrupts 3242 +16.7% 3782 ± 8% interrupts.CPU102.CAL:Function_call_interrupts 1597 ± 52% +246.0% 5527 ± 34% interrupts.CPU102.NMI:Non-maskable_interrupts 1597 ± 52% +246.0% 5527 ± 34% interrupts.CPU102.PMI:Performance_monitoring_interrupts 2702 ± 24% +132.9% 6292 ± 10% interrupts.CPU103.NMI:Non-maskable_interrupts 2702 ± 24% +132.9% 6292 ± 10% interrupts.CPU103.PMI:Performance_monitoring_interrupts 3162 ± 5% +20.4% 3806 ± 8% interrupts.CPU104.CAL:Function_call_interrupts 1300 ± 92% +306.2% 5284 ± 28% interrupts.CPU104.NMI:Non-maskable_interrupts 1300 ± 92% +306.2% 5284 ± 28% interrupts.CPU104.PMI:Performance_monitoring_interrupts 1211 ± 67% +405.4% 6120 ± 16% interrupts.CPU105.NMI:Non-maskable_interrupts 1211 ± 67% +405.4% 6120 ± 16% interrupts.CPU105.PMI:Performance_monitoring_interrupts 2114 ± 76% +187.6% 6080 ± 14% interrupts.CPU106.NMI:Non-maskable_interrupts 2114 ± 76% +187.6% 6080 ± 14% interrupts.CPU106.PMI:Performance_monitoring_interrupts 1888 ± 55% +224.2% 6120 ± 16% interrupts.CPU107.NMI:Non-maskable_interrupts 1888 ± 55% +224.2% 6120 ± 16% interrupts.CPU107.PMI:Performance_monitoring_interrupts 1496 ± 56% +260.4% 5391 ± 20% interrupts.CPU108.NMI:Non-maskable_interrupts 1496 ± 56% +260.4% 5391 ± 20% interrupts.CPU108.PMI:Performance_monitoring_interrupts 3232 +19.8% 3873 ± 9% interrupts.CPU109.CAL:Function_call_interrupts 2670 ± 44% +127.8% 6084 ± 15% interrupts.CPU109.NMI:Non-maskable_interrupts 2670 ± 44% +127.8% 6084 ± 15% interrupts.CPU109.PMI:Performance_monitoring_interrupts 3237 +18.8% 3845 ± 9% interrupts.CPU11.CAL:Function_call_interrupts 3202 ± 2% +19.8% 3835 ± 8% interrupts.CPU110.CAL:Function_call_interrupts 2307 ±103% +165.1% 6117 ± 14% interrupts.CPU110.NMI:Non-maskable_interrupts 2307 ±103% +165.1% 6117 ± 14% interrupts.CPU110.PMI:Performance_monitoring_interrupts 1054 ± 63% +329.2% 4527 ± 37% interrupts.CPU111.NMI:Non-maskable_interrupts 1054 ± 63% +329.2% 4527 ± 37% interrupts.CPU111.PMI:Performance_monitoring_interrupts 151.75 ± 48% -46.8% 80.75 ± 54% interrupts.CPU111.RES:Rescheduling_interrupts 3234 +18.4% 3828 ± 9% interrupts.CPU112.CAL:Function_call_interrupts 950.25 ± 68% +479.8% 5509 ± 25% interrupts.CPU112.NMI:Non-maskable_interrupts 950.25 ± 68% +479.8% 5509 ± 25% interrupts.CPU112.PMI:Performance_monitoring_interrupts 1296 ± 39% +326.9% 5533 ± 32% interrupts.CPU113.NMI:Non-maskable_interrupts 1296 ± 39% +326.9% 5533 ± 32% interrupts.CPU113.PMI:Performance_monitoring_interrupts 1686 ± 62% +219.7% 5390 ± 20% interrupts.CPU114.NMI:Non-maskable_interrupts 1686 ± 62% +219.7% 5390 ± 20% interrupts.CPU114.PMI:Performance_monitoring_interrupts 2020 ± 69% +117.3% 4390 ± 23% interrupts.CPU116.NMI:Non-maskable_interrupts 2020 ± 69% +117.3% 4390 ± 23% interrupts.CPU116.PMI:Performance_monitoring_interrupts 2090 ± 51% +113.5% 4462 ± 32% interrupts.CPU117.NMI:Non-maskable_interrupts 2090 ± 51% +113.5% 4462 ± 32% interrupts.CPU117.PMI:Performance_monitoring_interrupts 2267 ± 35% +129.0% 5193 ± 22% interrupts.CPU118.NMI:Non-maskable_interrupts 2267 ± 35% +129.0% 5193 ± 22% interrupts.CPU118.PMI:Performance_monitoring_interrupts 3190 ± 3% +19.8% 3823 ± 8% interrupts.CPU119.CAL:Function_call_interrupts 157.00 ± 4% +49.5% 234.75 ± 45% interrupts.CPU12.126:IR-PCI-MSI.1574924-edge.eth1-TxRx-12 3252 +18.1% 3840 ± 9% interrupts.CPU12.CAL:Function_call_interrupts 2365 ± 90% +126.2% 5351 ± 26% interrupts.CPU121.NMI:Non-maskable_interrupts 2365 ± 90% +126.2% 5351 ± 26% interrupts.CPU121.PMI:Performance_monitoring_interrupts 139.75 ± 70% -59.6% 56.50 ± 24% interrupts.CPU124.RES:Rescheduling_interrupts 2947 ± 18% +30.1% 3835 ± 8% interrupts.CPU125.CAL:Function_call_interrupts 1644 ± 55% +286.1% 6348 ± 9% interrupts.CPU125.NMI:Non-maskable_interrupts 1644 ± 55% +286.1% 6348 ± 9% interrupts.CPU125.PMI:Performance_monitoring_interrupts 3092 ± 53% +105.9% 6366 ± 10% interrupts.CPU126.NMI:Non-maskable_interrupts 3092 ± 53% +105.9% 6366 ± 10% interrupts.CPU126.PMI:Performance_monitoring_interrupts 1205 ± 57% +360.5% 5552 ± 32% interrupts.CPU127.NMI:Non-maskable_interrupts 1205 ± 57% +360.5% 5552 ± 32% interrupts.CPU127.PMI:Performance_monitoring_interrupts 2959 ± 61% +110.2% 6219 ± 12% interrupts.CPU128.NMI:Non-maskable_interrupts 2959 ± 61% +110.2% 6219 ± 12% interrupts.CPU128.PMI:Performance_monitoring_interrupts 88.00 ± 41% -41.8% 51.25 ± 41% interrupts.CPU128.RES:Rescheduling_interrupts 3215 ± 3% +18.5% 3811 ± 8% interrupts.CPU129.CAL:Function_call_interrupts 1823 ± 94% +243.0% 6254 ± 11% interrupts.CPU129.NMI:Non-maskable_interrupts 1823 ± 94% +243.0% 6254 ± 11% interrupts.CPU129.PMI:Performance_monitoring_interrupts 3290 ± 46% +89.6% 6239 ± 11% interrupts.CPU130.NMI:Non-maskable_interrupts 3290 ± 46% +89.6% 6239 ± 11% interrupts.CPU130.PMI:Performance_monitoring_interrupts 3266 +17.8% 3847 ± 9% interrupts.CPU131.CAL:Function_call_interrupts 1213 ± 85% +407.0% 6151 ± 13% interrupts.CPU131.NMI:Non-maskable_interrupts 1213 ± 85% +407.0% 6151 ± 13% interrupts.CPU131.PMI:Performance_monitoring_interrupts 1788 ± 61% +246.5% 6197 ± 12% interrupts.CPU132.NMI:Non-maskable_interrupts 1788 ± 61% +246.5% 6197 ± 12% interrupts.CPU132.PMI:Performance_monitoring_interrupts 1504 ± 48% +149.2% 3748 ± 21% interrupts.CPU133.NMI:Non-maskable_interrupts 1504 ± 48% +149.2% 3748 ± 21% interrupts.CPU133.PMI:Performance_monitoring_interrupts 90.50 ± 47% -52.8% 42.75 ± 22% interrupts.CPU133.RES:Rescheduling_interrupts 3260 +15.2% 3755 ± 7% interrupts.CPU134.CAL:Function_call_interrupts 1984 ± 44% +180.8% 5571 ± 30% interrupts.CPU134.NMI:Non-maskable_interrupts 1984 ± 44% +180.8% 5571 ± 30% interrupts.CPU134.PMI:Performance_monitoring_interrupts 3268 +17.0% 3826 ± 8% interrupts.CPU135.CAL:Function_call_interrupts 3246 +18.7% 3852 ± 9% interrupts.CPU136.CAL:Function_call_interrupts 1254 ± 58% +332.9% 5430 ± 26% interrupts.CPU136.NMI:Non-maskable_interrupts 1254 ± 58% +332.9% 5430 ± 26% interrupts.CPU136.PMI:Performance_monitoring_interrupts 3678 ± 30% +68.8% 6208 ± 12% interrupts.CPU137.NMI:Non-maskable_interrupts 3678 ± 30% +68.8% 6208 ± 12% interrupts.CPU137.PMI:Performance_monitoring_interrupts 1196 ±155% -95.9% 49.25 ± 13% interrupts.CPU137.RES:Rescheduling_interrupts 3245 +18.7% 3850 ± 8% interrupts.CPU138.CAL:Function_call_interrupts 1727 ± 48% +268.4% 6364 ± 10% interrupts.CPU138.NMI:Non-maskable_interrupts 1727 ± 48% +268.4% 6364 ± 10% interrupts.CPU138.PMI:Performance_monitoring_interrupts 2699 ± 35% +132.7% 6280 ± 10% interrupts.CPU139.NMI:Non-maskable_interrupts 2699 ± 35% +132.7% 6280 ± 10% interrupts.CPU139.PMI:Performance_monitoring_interrupts 171.25 ± 25% +49.2% 255.50 ± 43% interrupts.CPU14.128:IR-PCI-MSI.1574926-edge.eth1-TxRx-14 3238 +17.4% 3800 ± 9% interrupts.CPU14.CAL:Function_call_interrupts 1477 ± 76% +267.9% 5436 ± 26% interrupts.CPU141.NMI:Non-maskable_interrupts 1477 ± 76% +267.9% 5436 ± 26% interrupts.CPU141.PMI:Performance_monitoring_interrupts 753.00 ±104% -82.3% 133.00 ± 70% interrupts.CPU141.RES:Rescheduling_interrupts 3313 ± 48% +61.2% 5342 ± 20% interrupts.CPU142.NMI:Non-maskable_interrupts 3313 ± 48% +61.2% 5342 ± 20% interrupts.CPU142.PMI:Performance_monitoring_interrupts 2077 ± 54% +160.5% 5412 ± 26% interrupts.CPU143.NMI:Non-maskable_interrupts 2077 ± 54% +160.5% 5412 ± 26% interrupts.CPU143.PMI:Performance_monitoring_interrupts 1301 ±142% -89.8% 133.00 ± 25% interrupts.CPU144.RES:Rescheduling_interrupts 3254 +18.9% 3869 ± 9% interrupts.CPU145.CAL:Function_call_interrupts 2946 ± 16% +30.9% 3856 ± 9% interrupts.CPU149.CAL:Function_call_interrupts 3231 +17.0% 3781 ± 8% interrupts.CPU15.CAL:Function_call_interrupts 1426 ±107% +240.0% 4849 ± 43% interrupts.CPU15.NMI:Non-maskable_interrupts 1426 ±107% +240.0% 4849 ± 43% interrupts.CPU15.PMI:Performance_monitoring_interrupts 3821 ± 43% +59.7% 6102 ± 16% interrupts.CPU150.NMI:Non-maskable_interrupts 3821 ± 43% +59.7% 6102 ± 16% interrupts.CPU150.PMI:Performance_monitoring_interrupts 3253 +18.9% 3869 ± 9% interrupts.CPU151.CAL:Function_call_interrupts 3212 ± 2% +20.0% 3856 ± 8% interrupts.CPU153.CAL:Function_call_interrupts 3171 +22.3% 3878 ± 9% interrupts.CPU154.CAL:Function_call_interrupts 3269 +18.6% 3879 ± 8% interrupts.CPU155.CAL:Function_call_interrupts 121.25 ± 26% -47.4% 63.75 ± 70% interrupts.CPU155.RES:Rescheduling_interrupts 3258 +17.5% 3830 ± 8% interrupts.CPU156.CAL:Function_call_interrupts 84.50 ± 63% +75.7% 148.50 ± 34% interrupts.CPU157.RES:Rescheduling_interrupts 3998 ± 32% +54.2% 6167 ± 13% interrupts.CPU158.NMI:Non-maskable_interrupts 3998 ± 32% +54.2% 6167 ± 13% interrupts.CPU158.PMI:Performance_monitoring_interrupts 1649 ± 89% +233.8% 5505 ± 28% interrupts.CPU16.NMI:Non-maskable_interrupts 1649 ± 89% +233.8% 5505 ± 28% interrupts.CPU16.PMI:Performance_monitoring_interrupts 3769 ± 30% +60.7% 6058 ± 16% interrupts.CPU160.NMI:Non-maskable_interrupts 3769 ± 30% +60.7% 6058 ± 16% interrupts.CPU160.PMI:Performance_monitoring_interrupts 3062 ± 9% +100.0% 6125 ± 15% interrupts.CPU161.NMI:Non-maskable_interrupts 3062 ± 9% +100.0% 6125 ± 15% interrupts.CPU161.PMI:Performance_monitoring_interrupts 3259 +18.7% 3869 ± 9% interrupts.CPU162.CAL:Function_call_interrupts 3259 +17.6% 3834 ± 9% interrupts.CPU163.CAL:Function_call_interrupts 2847 ± 34% +87.8% 5347 ± 31% interrupts.CPU164.NMI:Non-maskable_interrupts 2847 ± 34% +87.8% 5347 ± 31% interrupts.CPU164.PMI:Performance_monitoring_interrupts 3720 ± 22% +63.7% 6091 ± 15% interrupts.CPU165.NMI:Non-maskable_interrupts 3720 ± 22% +63.7% 6091 ± 15% interrupts.CPU165.PMI:Performance_monitoring_interrupts 3265 +18.2% 3861 ± 8% interrupts.CPU169.CAL:Function_call_interrupts 143.50 ± 4% +9.6% 157.25 ± 3% interrupts.CPU17.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17 3243 +19.2% 3867 ± 9% interrupts.CPU17.CAL:Function_call_interrupts 2607 ± 58% +143.6% 6351 ± 13% interrupts.CPU17.NMI:Non-maskable_interrupts 2607 ± 58% +143.6% 6351 ± 13% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3216 ± 3% +16.4% 3743 ± 9% interrupts.CPU174.CAL:Function_call_interrupts 2672 ± 28% +131.4% 6183 ± 13% interrupts.CPU174.NMI:Non-maskable_interrupts 2672 ± 28% +131.4% 6183 ± 13% interrupts.CPU174.PMI:Performance_monitoring_interrupts 3761 ± 15% +67.4% 6295 ± 11% interrupts.CPU178.NMI:Non-maskable_interrupts 3761 ± 15% +67.4% 6295 ± 11% interrupts.CPU178.PMI:Performance_monitoring_interrupts 141.25 ± 2% +25.7% 177.50 ± 18% interrupts.CPU18.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18 3261 +19.7% 3905 ± 7% interrupts.CPU18.CAL:Function_call_interrupts 3260 +17.8% 3842 ± 8% interrupts.CPU181.CAL:Function_call_interrupts 3274 +17.8% 3856 ± 8% interrupts.CPU183.CAL:Function_call_interrupts 3265 +18.1% 3854 ± 9% interrupts.CPU186.CAL:Function_call_interrupts 3211 +20.4% 3868 ± 9% interrupts.CPU187.CAL:Function_call_interrupts 3254 +18.1% 3844 ± 9% interrupts.CPU188.CAL:Function_call_interrupts 142.50 ± 3% +10.2% 157.00 ± 2% interrupts.CPU19.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19 3248 +18.6% 3852 ± 9% interrupts.CPU19.CAL:Function_call_interrupts 6351 ± 14% -27.8% 4584 ± 27% interrupts.CPU19.NMI:Non-maskable_interrupts 6351 ± 14% -27.8% 4584 ± 27% interrupts.CPU19.PMI:Performance_monitoring_interrupts 80.00 ± 29% +6769.4% 5495 ±169% interrupts.CPU19.RES:Rescheduling_interrupts 3266 +16.1% 3793 ± 8% interrupts.CPU190.CAL:Function_call_interrupts 3231 +17.2% 3785 ± 8% interrupts.CPU191.CAL:Function_call_interrupts 3243 +19.6% 3880 ± 9% interrupts.CPU2.CAL:Function_call_interrupts 140.75 ± 2% +12.6% 158.50 ± 4% interrupts.CPU22.136:IR-PCI-MSI.1574934-edge.eth1-TxRx-22 141.00 +10.8% 156.25 ± 2% interrupts.CPU23.137:IR-PCI-MSI.1574935-edge.eth1-TxRx-23 3184 ± 2% +20.2% 3828 ± 9% interrupts.CPU23.CAL:Function_call_interrupts 140.50 +11.0% 156.00 ± 2% interrupts.CPU24.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24 141.00 +12.8% 159.00 ± 4% interrupts.CPU25.139:IR-PCI-MSI.1574937-edge.eth1-TxRx-25 140.50 +12.1% 157.50 ± 3% interrupts.CPU26.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26 140.50 +11.4% 156.50 interrupts.CPU27.141:IR-PCI-MSI.1574939-edge.eth1-TxRx-27 140.50 +11.2% 156.25 ± 2% interrupts.CPU28.142:IR-PCI-MSI.1574940-edge.eth1-TxRx-28 140.50 +13.0% 158.75 ± 3% interrupts.CPU29.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29 3257 +18.3% 3853 ± 9% interrupts.CPU29.CAL:Function_call_interrupts 3118 ± 4% +23.0% 3835 ± 9% interrupts.CPU3.CAL:Function_call_interrupts 141.00 +12.9% 159.25 interrupts.CPU30.144:IR-PCI-MSI.1574942-edge.eth1-TxRx-30 3263 +17.4% 3832 ± 8% interrupts.CPU30.CAL:Function_call_interrupts 140.50 +11.4% 156.50 ± 2% interrupts.CPU31.145:IR-PCI-MSI.1574943-edge.eth1-TxRx-31 1845 ± 50% +241.0% 6294 ± 11% interrupts.CPU31.NMI:Non-maskable_interrupts 1845 ± 50% +241.0% 6294 ± 11% interrupts.CPU31.PMI:Performance_monitoring_interrupts 78.50 ± 35% +124.5% 176.25 ± 52% interrupts.CPU31.RES:Rescheduling_interrupts 3265 +17.1% 3823 ± 8% interrupts.CPU32.CAL:Function_call_interrupts 140.75 +10.8% 156.00 ± 2% interrupts.CPU33.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33 2649 ± 88% +107.6% 5499 ± 28% interrupts.CPU33.NMI:Non-maskable_interrupts 2649 ± 88% +107.6% 5499 ± 28% interrupts.CPU33.PMI:Performance_monitoring_interrupts 140.50 +11.2% 156.25 ± 2% interrupts.CPU35.149:IR-PCI-MSI.1574947-edge.eth1-TxRx-35 140.75 +11.2% 156.50 ± 2% interrupts.CPU36.150:IR-PCI-MSI.1574948-edge.eth1-TxRx-36 140.75 +11.4% 156.75 ± 2% interrupts.CPU37.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37 3251 +17.8% 3831 ± 8% interrupts.CPU37.CAL:Function_call_interrupts 140.50 +11.4% 156.50 ± 2% interrupts.CPU38.152:IR-PCI-MSI.1574950-edge.eth1-TxRx-38 129.50 ± 39% -56.8% 56.00 ± 28% interrupts.CPU38.RES:Rescheduling_interrupts 140.50 +12.3% 157.75 ± 3% interrupts.CPU40.154:IR-PCI-MSI.1574952-edge.eth1-TxRx-40 140.50 +11.0% 156.00 ± 2% interrupts.CPU41.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41 3214 ± 2% +20.3% 3868 ± 9% interrupts.CPU41.CAL:Function_call_interrupts 140.75 +10.8% 156.00 ± 2% interrupts.CPU42.156:IR-PCI-MSI.1574954-edge.eth1-TxRx-42 140.50 +11.0% 156.00 ± 2% interrupts.CPU43.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43 238.50 ± 66% -73.6% 63.00 ± 42% interrupts.CPU43.RES:Rescheduling_interrupts 140.75 +11.2% 156.50 ± 2% interrupts.CPU44.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44 3252 +18.8% 3863 ± 9% interrupts.CPU44.CAL:Function_call_interrupts 3503 ± 42% +78.3% 6247 ± 12% interrupts.CPU44.NMI:Non-maskable_interrupts 3503 ± 42% +78.3% 6247 ± 12% interrupts.CPU44.PMI:Performance_monitoring_interrupts 246.00 ± 67% -81.6% 45.25 ± 8% interrupts.CPU44.RES:Rescheduling_interrupts 140.75 +11.2% 156.50 ± 2% interrupts.CPU45.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45 2032 ± 97% +164.0% 5366 ± 27% interrupts.CPU45.NMI:Non-maskable_interrupts 2032 ± 97% +164.0% 5366 ± 27% interrupts.CPU45.PMI:Performance_monitoring_interrupts 140.50 +11.2% 156.25 ± 2% interrupts.CPU46.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46 140.50 +11.2% 156.25 ± 2% interrupts.CPU47.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47 3254 +20.0% 3906 ± 9% interrupts.CPU47.CAL:Function_call_interrupts 2604 ± 57% +139.9% 6247 ± 11% interrupts.CPU47.NMI:Non-maskable_interrupts 2604 ± 57% +139.9% 6247 ± 11% interrupts.CPU47.PMI:Performance_monitoring_interrupts 141.00 +10.6% 156.00 ± 2% interrupts.CPU48.162:IR-PCI-MSI.1574960-edge.eth1-TxRx-48 3064 ± 6% +25.2% 3835 ± 9% interrupts.CPU48.CAL:Function_call_interrupts 140.50 +12.3% 157.75 ± 2% interrupts.CPU49.163:IR-PCI-MSI.1574961-edge.eth1-TxRx-49 3259 +17.8% 3838 ± 9% interrupts.CPU5.CAL:Function_call_interrupts 140.50 +11.0% 156.00 ± 2% interrupts.CPU50.164:IR-PCI-MSI.1574962-edge.eth1-TxRx-50 140.50 +11.4% 156.50 ± 2% interrupts.CPU51.165:IR-PCI-MSI.1574963-edge.eth1-TxRx-51 140.50 +60.5% 225.50 ± 52% interrupts.CPU53.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53 140.50 +11.2% 156.25 ± 2% interrupts.CPU55.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55 3198 ± 2% +14.5% 3662 ± 9% interrupts.CPU55.CAL:Function_call_interrupts 140.75 +11.2% 156.50 ± 2% interrupts.CPU56.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56 140.50 +11.2% 156.25 ± 2% interrupts.CPU58.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58 141.00 +12.4% 158.50 ± 3% interrupts.CPU59.173:IR-PCI-MSI.1574971-edge.eth1-TxRx-59 369.25 ± 84% -83.3% 61.75 ± 49% interrupts.CPU59.RES:Rescheduling_interrupts 1767 ± 72% +253.4% 6245 ± 14% interrupts.CPU6.NMI:Non-maskable_interrupts 1767 ± 72% +253.4% 6245 ± 14% interrupts.CPU6.PMI:Performance_monitoring_interrupts 141.00 +10.6% 156.00 ± 2% interrupts.CPU60.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60 140.50 +11.6% 156.75 ± 2% interrupts.CPU61.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61 140.50 +11.6% 156.75 ± 2% interrupts.CPU62.176:IR-PCI-MSI.1574974-edge.eth1-TxRx-62 7211 -16.1% 6052 ± 15% interrupts.CPU62.NMI:Non-maskable_interrupts 7211 -16.1% 6052 ± 15% interrupts.CPU62.PMI:Performance_monitoring_interrupts 3231 +19.4% 3857 ± 9% interrupts.CPU64.CAL:Function_call_interrupts 838.75 ±130% -91.7% 69.25 ± 36% interrupts.CPU64.RES:Rescheduling_interrupts 69.00 ± 31% +66.7% 115.00 ± 50% interrupts.CPU68.RES:Rescheduling_interrupts 321.75 ± 12% -23.2% 247.25 ± 17% interrupts.CPU72.RES:Rescheduling_interrupts 60.50 ± 5% +170.2% 163.50 ± 35% interrupts.CPU74.RES:Rescheduling_interrupts 3256 +18.5% 3857 ± 9% interrupts.CPU75.CAL:Function_call_interrupts 79.25 ± 21% +472.2% 453.50 ± 9% interrupts.CPU75.RES:Rescheduling_interrupts 3245 +18.8% 3854 ± 9% interrupts.CPU77.CAL:Function_call_interrupts 4569 ± 25% +34.8% 6159 ± 14% interrupts.CPU77.NMI:Non-maskable_interrupts 4569 ± 25% +34.8% 6159 ± 14% interrupts.CPU77.PMI:Performance_monitoring_interrupts 150.25 ± 5% +54.6% 232.25 ± 34% interrupts.CPU8.118:IR-PCI-MSI.1574920-edge.eth1-TxRx-8 3212 ± 2% +19.9% 3850 ± 9% interrupts.CPU80.CAL:Function_call_interrupts 192.75 ± 61% -54.2% 88.25 ± 81% interrupts.CPU80.RES:Rescheduling_interrupts 4394 ± 27% +40.2% 6162 ± 13% interrupts.CPU81.NMI:Non-maskable_interrupts 4394 ± 27% +40.2% 6162 ± 13% interrupts.CPU81.PMI:Performance_monitoring_interrupts 4742 ± 19% +32.9% 6300 ± 11% interrupts.CPU82.NMI:Non-maskable_interrupts 4742 ± 19% +32.9% 6300 ± 11% interrupts.CPU82.PMI:Performance_monitoring_interrupts 3237 +18.3% 3830 ± 8% interrupts.CPU84.CAL:Function_call_interrupts 3254 +18.3% 3849 ± 9% interrupts.CPU89.CAL:Function_call_interrupts 1761 ±125% +212.2% 5498 ± 27% interrupts.CPU9.NMI:Non-maskable_interrupts 1761 ±125% +212.2% 5498 ± 27% interrupts.CPU9.PMI:Performance_monitoring_interrupts 3249 +18.9% 3864 ± 7% interrupts.CPU90.CAL:Function_call_interrupts 3238 +18.5% 3837 ± 8% interrupts.CPU91.CAL:Function_call_interrupts 170.50 ± 63% -76.0% 41.00 ± 11% interrupts.CPU92.RES:Rescheduling_interrupts 3501 ± 7% +33.2% 4663 interrupts.CPU95.RES:Rescheduling_interrupts 3236 +19.2% 3858 ± 9% interrupts.CPU96.CAL:Function_call_interrupts 2546 ± 57% +139.5% 6097 ± 15% interrupts.CPU96.NMI:Non-maskable_interrupts 2546 ± 57% +139.5% 6097 ± 15% interrupts.CPU96.PMI:Performance_monitoring_interrupts 1551 ± 47% +296.9% 6159 ± 16% interrupts.CPU97.NMI:Non-maskable_interrupts 1551 ± 47% +296.9% 6159 ± 16% interrupts.CPU97.PMI:Performance_monitoring_interrupts 1417 ± 70% +332.2% 6125 ± 14% interrupts.CPU98.NMI:Non-maskable_interrupts 1417 ± 70% +332.2% 6125 ± 14% interrupts.CPU98.PMI:Performance_monitoring_interrupts 3183 ± 3% +20.0% 3820 ± 8% interrupts.CPU99.CAL:Function_call_interrupts 1987 ± 93% +225.8% 6474 ± 7% interrupts.CPU99.NMI:Non-maskable_interrupts 1987 ± 93% +225.8% 6474 ± 7% interrupts.CPU99.PMI:Performance_monitoring_interrupts 160.50 ± 12% +130.1% 369.25 ± 24% interrupts.IWI:IRQ_work_interrupts 764314 ± 9% +42.4% 1088399 ± 13% interrupts.NMI:Non-maskable_interrupts 764314 ± 9% +42.4% 1088399 ± 13% interrupts.PMI:Performance_monitoring_interrupts *************************************************************************************************** lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= blocksize/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/testcase: 512K/gcc-7/performance/x86_64-rhel-7.2/25%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep2/pigz commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 83.20 +24.9% 103.96 ± 2% pigz.time.system_time 0.34 +0.1 0.42 ± 4% mpstat.cpu.sys% 13.78 +1.6% 14.00 turbostat.RAMWatt 15649 ± 15% -72.6% 4283 ±171% numa-numastat.node0.other_node 1521 ±160% +746.1% 12875 ± 56% numa-numastat.node1.other_node 273712 ± 15% -41.3% 160560 ± 13% meminfo.DirectMap4k 1936313 +109.4% 4054029 ± 6% meminfo.Memused 6655 +144.5% 16270 ± 6% meminfo.max_used_kB 3228648 -1.6% 3175785 proc-vmstat.nr_dirty_background_threshold 6465192 -1.6% 6359335 proc-vmstat.nr_dirty_threshold 32488498 -1.6% 31959080 proc-vmstat.nr_free_pages 810.75 ± 13% -39.2% 493.25 ± 44% numa-vmstat.node0.nr_page_table_pages 8162 ± 13% +28.1% 10457 ± 6% numa-vmstat.node0.nr_slab_reclaimable 16240 ± 16% -71.3% 4668 ±156% numa-vmstat.node0.numa_other 9595 ± 12% -23.0% 7391 ± 7% numa-vmstat.node1.nr_slab_reclaimable 137985 +8.7% 150014 ± 4% numa-vmstat.node1.numa_other 32650 ± 13% +28.1% 41830 ± 6% numa-meminfo.node0.KReclaimable 1019000 ± 6% +102.0% 2058285 ± 10% numa-meminfo.node0.MemUsed 3249 ± 13% -39.3% 1971 ± 44% numa-meminfo.node0.PageTables 32650 ± 13% +28.1% 41830 ± 6% numa-meminfo.node0.SReclaimable 38382 ± 12% -23.0% 29567 ± 7% numa-meminfo.node1.KReclaimable 917645 ± 7% +117.5% 1995816 ± 21% numa-meminfo.node1.MemUsed 38382 ± 12% -23.0% 29567 ± 7% numa-meminfo.node1.SReclaimable 96.86 -1.3 95.58 perf-profile.calltrace.cycles-pp.secondary_startup_64 0.32 ±100% +0.7 1.01 ± 17% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.__vfs_write.vfs_write 0.32 ±100% +0.7 1.01 ± 17% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.__vfs_write 0.79 ± 61% +0.7 1.50 ± 17% perf-profile.calltrace.cycles-pp.pipe_write.__vfs_write.vfs_write.ksys_write.do_syscall_64 0.79 ± 61% +0.7 1.50 ± 17% perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.34 ±100% +0.7 1.04 ± 17% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.__vfs_write.vfs_write.ksys_write 0.79 ± 61% +0.7 1.50 ± 17% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.79 ± 61% +0.7 1.51 ± 17% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 20.87 ± 7% -17.0% 17.32 ± 10% sched_debug.cfs_rq:/.load_avg.avg 52.13 ± 6% -15.1% 44.28 ± 12% sched_debug.cfs_rq:/.load_avg.stddev 6.09 ± 25% -53.2% 2.85 ± 57% sched_debug.cfs_rq:/.removed.load_avg.avg 30.88 ± 12% -39.5% 18.68 ± 57% sched_debug.cfs_rq:/.removed.load_avg.stddev 280.39 ± 25% -53.1% 131.58 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.avg 1422 ± 12% -39.3% 862.99 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 2.63 ± 20% -52.9% 1.24 ± 59% sched_debug.cfs_rq:/.removed.util_avg.avg 13.60 ± 8% -39.1% 8.28 ± 59% sched_debug.cfs_rq:/.removed.util_avg.stddev -56401 +1362.8% -825066 sched_debug.cfs_rq:/.spread0.avg -941225 +81.9% -1712152 sched_debug.cfs_rq:/.spread0.min 31954 ± 37% -33.5% 21246 ± 57% softirqs.CPU0.SCHED 7178 ±167% -97.9% 147.25 ± 28% softirqs.CPU11.NET_RX 122191 ± 23% -24.9% 91737 ± 12% softirqs.CPU12.TIMER 31327 ± 17% +26.8% 39720 ± 2% softirqs.CPU18.SCHED 34939 ± 10% -19.0% 28293 ± 11% softirqs.CPU36.RCU 38735 -18.4% 31618 ± 16% softirqs.CPU38.SCHED 25281 ± 55% -49.5% 12757 ±116% softirqs.CPU4.SCHED 22964 ± 22% +38.9% 31905 ± 18% softirqs.CPU46.RCU 105138 ± 11% +39.9% 147123 ± 5% softirqs.CPU5.TIMER 128051 ± 18% -29.5% 90300 ± 8% softirqs.CPU57.TIMER 34163 ± 14% -34.0% 22543 ± 30% softirqs.CPU66.RCU 37676 ± 8% -21.5% 29587 ± 15% softirqs.CPU68.SCHED 31983 ± 36% -57.7% 13535 ±109% softirqs.CPU7.SCHED 31040 ± 22% -25.4% 23160 ± 11% softirqs.CPU81.RCU 31684 ± 20% -30.4% 22062 ± 14% softirqs.CPU85.RCU 450.25 ± 48% -64.6% 159.25 ± 5% interrupts.38:IR-PCI-MSI.1572868-edge.eth0-TxRx-4 10852 ±167% -98.1% 203.00 ± 40% interrupts.45:IR-PCI-MSI.1572875-edge.eth0-TxRx-11 374.75 ± 34% +70.3% 638.25 ± 19% interrupts.CPU0.RES:Rescheduling_interrupts 10852 ±167% -98.1% 203.00 ± 40% interrupts.CPU11.45:IR-PCI-MSI.1572875-edge.eth0-TxRx-11 16.25 ± 72% +88935.4% 14468 ±171% interrupts.CPU11.RES:Rescheduling_interrupts 7000 ± 40% -51.3% 3406 interrupts.CPU16.CAL:Function_call_interrupts 10328 ± 51% -50.8% 5080 ± 57% interrupts.CPU20.CAL:Function_call_interrupts 333.25 ± 92% -91.1% 29.50 ±115% interrupts.CPU35.RES:Rescheduling_interrupts 450.25 ± 48% -64.6% 159.25 ± 5% interrupts.CPU4.38:IR-PCI-MSI.1572868-edge.eth0-TxRx-4 6474 ± 23% -65.5% 2232 ± 62% interrupts.CPU51.NMI:Non-maskable_interrupts 6474 ± 23% -65.5% 2232 ± 62% interrupts.CPU51.PMI:Performance_monitoring_interrupts 11.00 ± 57% +806.8% 99.75 ± 59% interrupts.CPU62.RES:Rescheduling_interrupts 6.75 ±141% +1374.1% 99.50 ± 93% interrupts.CPU63.RES:Rescheduling_interrupts 2595 ± 44% +161.0% 6773 ± 29% interrupts.CPU75.NMI:Non-maskable_interrupts 2595 ± 44% +161.0% 6773 ± 29% interrupts.CPU75.PMI:Performance_monitoring_interrupts 519.00 ± 41% +724.7% 4280 ± 84% interrupts.CPU77.NMI:Non-maskable_interrupts 519.00 ± 41% +724.7% 4280 ± 84% interrupts.CPU77.PMI:Performance_monitoring_interrupts 170301 ± 21% +34.1% 228420 ± 10% interrupts.RES:Rescheduling_interrupts 3.31 ± 4% -14.5% 2.83 ± 2% perf-stat.i.MPKI 1.58 ± 5% +1.7 3.24 ± 2% perf-stat.i.cache-miss-rate% 5259609 +77.6% 9341172 perf-stat.i.cache-misses 3.372e+08 ± 4% -14.1% 2.897e+08 ± 2% perf-stat.i.cache-references 12677 -44.5% 7030 perf-stat.i.cycles-between-cache-misses 0.01 ± 11% -0.0 0.00 ± 3% perf-stat.i.dTLB-load-miss-rate% 1935554 ± 10% -15.9% 1627264 ± 3% perf-stat.i.dTLB-load-misses 93.96 -34.1 59.83 ± 5% perf-stat.i.node-load-miss-rate% 128527 ± 11% +1114.4% 1560801 ± 6% perf-stat.i.node-loads 93.08 -38.4 54.64 ± 2% perf-stat.i.node-store-miss-rate% 74546 ± 5% +1231.2% 992355 ± 10% perf-stat.i.node-stores 3.30 ± 4% -14.5% 2.82 ± 2% perf-stat.overall.MPKI 1.56 ± 5% +1.7 3.23 ± 2% perf-stat.overall.cache-miss-rate% 12419 -43.7% 6997 perf-stat.overall.cycles-between-cache-misses 0.01 ± 11% -0.0 0.00 ± 3% perf-stat.overall.dTLB-load-miss-rate% 94.67 -34.8 59.86 ± 5% perf-stat.overall.node-load-miss-rate% 93.79 -39.4 54.43 perf-stat.overall.node-store-miss-rate% 5242026 +77.6% 9309914 perf-stat.ps.cache-misses 3.36e+08 ± 4% -14.1% 2.887e+08 ± 2% perf-stat.ps.cache-references 1929104 ± 10% -15.9% 1621870 ± 3% perf-stat.ps.dTLB-load-misses 128140 ± 11% +1114.0% 1555599 ± 6% perf-stat.ps.node-loads 74314 ± 5% +1230.9% 989040 ± 10% perf-stat.ps.node-stores *************************************************************************************************** lkp-bdw-ex2: 160 threads Intel(R) Xeon(R) CPU E7-8890 v4 @ 2.20GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.2/debian-x86_64-2018-04-03.cgz/300s/lkp-bdw-ex2/lru-file-mmap-read/vm-scalability/0xb00002e commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 0.23 ± 4% -14.6% 0.19 ± 5% vm-scalability.free_time 99543 -18.7% 80901 ± 2% vm-scalability.median 19214447 -20.0% 15365055 vm-scalability.throughput 380.77 +9.1% 415.40 vm-scalability.time.elapsed_time 380.77 +9.1% 415.40 vm-scalability.time.elapsed_time.max 363458 ± 3% +21.0% 439886 ± 2% vm-scalability.time.involuntary_context_switches 8296910 ± 2% -49.6% 4182269 ± 6% vm-scalability.time.maximum_resident_set_size 98596935 -5.7% 92981737 vm-scalability.time.minor_page_faults 44148 +12.2% 49531 vm-scalability.time.system_time 4.832e+09 -5.1% 4.585e+09 vm-scalability.workload 330604 ± 9% -51.9% 159124 meminfo.DirectMap4k 4241 ± 7% -18.0% 3478 ± 10% slabinfo.eventpoll_pwq.active_objs 4241 ± 7% -18.0% 3478 ± 10% slabinfo.eventpoll_pwq.num_objs 1730 +7.1% 1853 turbostat.Avg_MHz 2.431e+08 ± 8% +12.0% 2.721e+08 ± 7% turbostat.IRQ 479.26 +1.4% 486.16 turbostat.PkgWatt 33.75 -13.3% 29.25 ± 4% vmstat.cpu.id 61.00 +7.0% 65.25 vmstat.cpu.sy 128.25 +12.9% 144.75 vmstat.procs.r 3054 +3.3% 3156 vmstat.system.cs 1.808e+08 ± 2% -15.5% 1.528e+08 ± 7% numa-numastat.node0.local_node 1.808e+08 ± 2% -15.5% 1.528e+08 ± 7% numa-numastat.node0.numa_hit 1.024e+08 ± 5% +14.7% 1.174e+08 ± 5% numa-numastat.node0.numa_miss 1.024e+08 ± 5% +14.7% 1.174e+08 ± 5% numa-numastat.node0.other_node 1.82e+08 -25.1% 1.363e+08 ± 9% numa-numastat.node1.local_node 1.007e+08 +21.7% 1.225e+08 ± 4% numa-numastat.node1.numa_foreign 1.82e+08 -25.1% 1.363e+08 ± 9% numa-numastat.node1.numa_hit 59479126 ± 7% +41.3% 84038800 ± 14% numa-numastat.node1.numa_miss 59497820 ± 7% +41.3% 84057470 ± 14% numa-numastat.node1.other_node 1.978e+08 ± 2% -12.4% 1.733e+08 ± 5% numa-numastat.node2.local_node 72949158 ± 5% +33.3% 97259359 ± 12% numa-numastat.node2.numa_foreign 1.978e+08 ± 2% -12.4% 1.733e+08 ± 5% numa-numastat.node2.numa_hit 1.281e+08 -24.7% 96451089 ± 6% numa-numastat.node3.local_node 1.281e+08 -24.7% 96466674 ± 6% numa-numastat.node3.numa_hit 45052118 ± 9% +47.0% 66235889 ± 7% numa-numastat.node3.numa_miss 45061583 ± 9% +47.0% 66251475 ± 7% numa-numastat.node3.other_node 26069 ±134% +377.1% 124367 ± 21% numa-meminfo.node0.AnonHugePages 51016 ± 84% +232.9% 169847 ± 9% numa-meminfo.node0.AnonPages 1.038e+08 ± 2% -9.2% 94251029 ± 2% numa-meminfo.node0.FilePages 1.035e+08 ± 2% -9.2% 93946853 ± 2% numa-meminfo.node0.Inactive 1.034e+08 ± 2% -9.3% 93861431 ± 2% numa-meminfo.node0.Inactive(file) 78403380 ± 3% -9.3% 71077879 ± 5% numa-meminfo.node1.FilePages 78177263 ± 3% -9.4% 70825327 ± 5% numa-meminfo.node1.Inactive 78114744 ± 3% -9.4% 70763883 ± 5% numa-meminfo.node1.Inactive(file) 561416 +13.1% 634774 ± 3% numa-meminfo.node1.PageTables 1.566e+08 -10.3% 1.405e+08 ± 2% numa-meminfo.node2.FilePages 1.564e+08 -10.3% 1.403e+08 ± 2% numa-meminfo.node2.Inactive 1.564e+08 -10.3% 1.402e+08 ± 2% numa-meminfo.node2.Inactive(file) 8801 ± 87% +861.7% 84639 ±120% numa-meminfo.node2.Shmem 725010 ± 3% -19.0% 587245 ± 2% numa-meminfo.node3.KReclaimable 431257 ± 3% -14.1% 370588 ± 3% numa-meminfo.node3.PageTables 725010 ± 3% -19.0% 587245 ± 2% numa-meminfo.node3.SReclaimable 784111 ± 3% -18.0% 643301 numa-meminfo.node3.Slab 71.42 ± 2% -7.9 63.52 ± 8% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead.filemap_fault.__xfs_filemap_fault 4.65 ± 12% -2.4 2.23 ± 50% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 3.77 ± 14% -2.3 1.51 ± 71% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_node_memcg 3.74 ± 14% -2.3 1.48 ± 72% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list 97.58 -0.7 96.90 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault 97.59 -0.7 96.91 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 97.60 -0.7 96.92 perf-profile.calltrace.cycles-pp.page_fault 97.50 -0.7 96.83 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 97.48 -0.7 96.81 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault 0.88 ± 4% -0.1 0.73 ± 7% perf-profile.calltrace.cycles-pp.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node 0.79 ± 3% -0.1 0.66 ± 8% perf-profile.calltrace.cycles-pp.rmap_walk_file.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.80 -0.1 0.69 ± 14% perf-profile.calltrace.cycles-pp.rmap_walk_file.page_referenced.shrink_page_list.shrink_inactive_list.shrink_node_memcg 0.42 ± 58% +0.2 0.67 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 0.99 ± 11% +0.4 1.41 ± 21% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat 1.65 ± 12% +0.6 2.20 ± 12% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_node_memcg.shrink_node.balance_pgdat.kswapd 1.65 ± 12% +0.6 2.20 ± 12% perf-profile.calltrace.cycles-pp.shrink_node.balance_pgdat.kswapd.kthread.ret_from_fork 1.65 ± 12% +0.6 2.20 ± 12% perf-profile.calltrace.cycles-pp.shrink_node_memcg.shrink_node.balance_pgdat.kswapd.kthread 1.65 ± 12% +0.6 2.20 ± 12% perf-profile.calltrace.cycles-pp.kswapd.kthread.ret_from_fork 1.65 ± 12% +0.6 2.20 ± 12% perf-profile.calltrace.cycles-pp.balance_pgdat.kswapd.kthread.ret_from_fork 1.81 ± 12% +0.6 2.38 ± 12% perf-profile.calltrace.cycles-pp.ret_from_fork 1.81 ± 12% +0.6 2.38 ± 12% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.00 +1.5 1.54 ± 15% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 0.86 ± 8% +3.2 4.02 ± 14% perf-profile.calltrace.cycles-pp.wakeup_kswapd.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead 0.87 ± 8% +3.2 4.05 ± 15% perf-profile.calltrace.cycles-pp.wake_all_kswapds.__alloc_pages_slowpath.__alloc_pages_nodemask.__do_page_cache_readahead.ondemand_readahead 17.74 ± 10% +6.9 24.64 ± 21% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages.read_pages 16.83 ± 11% +7.0 23.79 ± 21% perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply.iomap_readpages 16.79 ± 11% +7.0 23.77 ± 21% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor.iomap_apply 16.29 ± 11% +7.0 23.27 ± 22% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.iomap_readpages_actor 16.26 ± 11% +7.0 23.25 ± 22% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru 2383 ± 4% +89.9% 4525 ± 15% proc-vmstat.allocstall_normal 3.517e+08 ± 8% -59.9% 1.409e+08 ± 11% proc-vmstat.compact_daemon_free_scanned 1125324 ± 26% -25.7% 835780 ± 11% proc-vmstat.compact_daemon_migrate_scanned 1166 ± 3% -76.3% 276.00 ± 26% proc-vmstat.compact_daemon_wake 3.524e+08 ± 8% -59.7% 1.421e+08 ± 10% proc-vmstat.compact_free_scanned 1130064 ± 26% -25.7% 839367 ± 10% proc-vmstat.compact_migrate_scanned 841.75 ± 8% -64.5% 298.75 ± 8% proc-vmstat.kswapd_low_wmark_hit_quickly 65873 +1.2% 66658 proc-vmstat.nr_anon_pages 12053701 -8.6% 11021817 proc-vmstat.nr_dirty_background_threshold 24136875 -8.6% 22070585 proc-vmstat.nr_dirty_threshold 98251122 -8.7% 89738956 ± 2% proc-vmstat.nr_file_pages 97908208 -8.7% 89394380 ± 2% proc-vmstat.nr_inactive_file 681102 +7.1% 729507 proc-vmstat.nr_page_table_pages 1237802 ± 2% -7.0% 1151495 ± 2% proc-vmstat.nr_slab_reclaimable 97907950 -8.7% 89394476 ± 2% proc-vmstat.nr_zone_inactive_file 3.887e+08 ± 2% +17.5% 4.569e+08 ± 5% proc-vmstat.numa_foreign 5637 ± 4% -23.7% 4298 ± 5% proc-vmstat.numa_hint_faults 2696 ± 7% -20.2% 2152 ± 6% proc-vmstat.numa_hint_faults_local 6.887e+08 -18.9% 5.589e+08 ± 6% proc-vmstat.numa_hit 6.887e+08 -18.9% 5.588e+08 ± 6% proc-vmstat.numa_local 3.887e+08 ± 2% +17.5% 4.569e+08 ± 5% proc-vmstat.numa_miss 3.887e+08 ± 2% +17.5% 4.569e+08 ± 5% proc-vmstat.numa_other 2495 ± 68% -67.3% 816.50 ± 13% proc-vmstat.numa_pages_migrated 1488 ± 2% -68.0% 476.75 ± 16% proc-vmstat.pageoutrun 1.077e+09 -5.7% 1.015e+09 proc-vmstat.pgalloc_normal 99830266 -5.6% 94256905 proc-vmstat.pgfault 1.08e+09 -5.7% 1.018e+09 proc-vmstat.pgfree 3.788e+08 ± 2% -6.3% 3.55e+08 ± 4% proc-vmstat.pgscan_kswapd 8.787e+08 -4.2% 8.417e+08 proc-vmstat.pgsteal_direct 491026 ± 50% +551.5% 3199083 ± 36% proc-vmstat.slabs_scanned 90046 ± 62% +933.3% 930413 ± 31% proc-vmstat.workingset_nodereclaim 12723 ± 85% +233.5% 42429 ± 9% numa-vmstat.node0.nr_anon_pages 25790306 ± 2% -9.4% 23372081 ± 2% numa-vmstat.node0.nr_file_pages 25686340 ± 2% -9.4% 23274336 ± 2% numa-vmstat.node0.nr_inactive_file 25686340 ± 2% -9.4% 23274316 ± 2% numa-vmstat.node0.nr_zone_inactive_file 62032774 ± 7% +19.6% 74196846 ± 7% numa-vmstat.node0.numa_foreign 1.3e+08 ± 3% -15.6% 1.097e+08 ± 6% numa-vmstat.node0.numa_hit 1.3e+08 ± 3% -15.6% 1.097e+08 ± 6% numa-vmstat.node0.numa_local 68045758 ± 5% +18.1% 80353794 ± 5% numa-vmstat.node0.numa_miss 68056461 ± 5% +18.1% 80358092 ± 5% numa-vmstat.node0.numa_other 19460734 ± 3% -9.3% 17652514 ± 5% numa-vmstat.node1.nr_file_pages 19388592 ± 3% -9.4% 17573830 ± 5% numa-vmstat.node1.nr_inactive_file 139055 ± 2% +13.5% 157771 ± 3% numa-vmstat.node1.nr_page_table_pages 19388576 ± 3% -9.4% 17573800 ± 5% numa-vmstat.node1.nr_zone_inactive_file 71481033 ± 2% +17.4% 83911277 ± 3% numa-vmstat.node1.numa_foreign 1.312e+08 ± 2% -25.2% 98147821 ± 9% numa-vmstat.node1.numa_hit 1.311e+08 ± 2% -25.2% 98052793 ± 9% numa-vmstat.node1.numa_local 34872180 ± 11% +61.1% 56192708 ± 15% numa-vmstat.node1.numa_miss 34967065 ± 11% +61.0% 56287779 ± 15% numa-vmstat.node1.numa_other 45322 ± 44% +1136.4% 560367 ± 19% numa-vmstat.node1.workingset_nodereclaim 38875861 -10.4% 34846949 ± 3% numa-vmstat.node2.nr_file_pages 38815791 -10.4% 34768369 ± 3% numa-vmstat.node2.nr_inactive_file 2226 ± 86% +855.2% 21266 ±120% numa-vmstat.node2.nr_shmem 38815763 -10.4% 34768319 ± 3% numa-vmstat.node2.nr_zone_inactive_file 50674246 ± 5% +31.0% 66385790 ± 10% numa-vmstat.node2.numa_foreign 1.416e+08 ± 2% -11.9% 1.247e+08 ± 6% numa-vmstat.node2.numa_hit 1.416e+08 ± 2% -11.9% 1.246e+08 ± 6% numa-vmstat.node2.numa_local 107079 ± 4% -14.1% 92004 ± 3% numa-vmstat.node3.nr_page_table_pages 182145 ± 3% -19.2% 147205 ± 2% numa-vmstat.node3.nr_slab_reclaimable 75943837 ± 4% +15.1% 87431284 ± 4% numa-vmstat.node3.numa_foreign 88612247 -22.4% 68762157 ± 6% numa-vmstat.node3.numa_hit 88527041 -22.4% 68670483 ± 6% numa-vmstat.node3.numa_local 31842656 ± 8% +42.5% 45364810 ± 5% numa-vmstat.node3.numa_miss 31927892 ± 8% +42.4% 45456524 ± 5% numa-vmstat.node3.numa_other 1036634 ± 4% -19.1% 838338 ± 4% numa-vmstat.node3.workingset_nodes 3.051e+10 -4.5% 2.915e+10 perf-stat.i.branch-instructions 7.75 ± 3% +1.3 9.10 ± 4% perf-stat.i.cache-miss-rate% 5.653e+08 -11.4% 5.007e+08 perf-stat.i.cache-references 2960 +2.7% 3039 perf-stat.i.context-switches 2.70 ± 5% +17.1% 3.16 ± 9% perf-stat.i.cpi 3.123e+11 +7.0% 3.342e+11 perf-stat.i.cpu-cycles 72.12 ± 3% +34.6% 97.05 ± 4% perf-stat.i.cpu-migrations 15137945 +31.8% 19955324 ± 5% perf-stat.i.dTLB-load-misses 2.914e+10 -3.8% 2.802e+10 perf-stat.i.dTLB-loads 5.163e+09 -12.0% 4.545e+09 perf-stat.i.dTLB-stores 1.139e+11 -3.6% 1.098e+11 perf-stat.i.instructions 0.40 ± 4% -15.0% 0.34 ± 6% perf-stat.i.ipc 253323 -16.3% 212011 ± 2% perf-stat.i.minor-faults 8077542 +12.6% 9097817 ± 3% perf-stat.i.node-load-misses 8156922 ± 2% +8.0% 8805563 ± 3% perf-stat.i.node-store-misses 5699831 -19.6% 4583716 ± 3% perf-stat.i.node-stores 253679 -15.6% 214201 perf-stat.i.page-faults 4.83 -9.5% 4.37 perf-stat.overall.MPKI 8.50 +1.2 9.69 ± 2% perf-stat.overall.cache-miss-rate% 2.77 +10.8% 3.07 perf-stat.overall.cpi 6748 +7.4% 7249 perf-stat.overall.cycles-between-cache-misses 0.05 ± 2% +0.0 0.07 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.36 -9.7% 0.33 perf-stat.overall.ipc 59.41 +6.3 65.72 perf-stat.overall.node-store-miss-rate% 9415 +12.6% 10599 perf-stat.overall.path-length 3.195e+10 -4.0% 3.066e+10 perf-stat.ps.branch-instructions 5.766e+08 -12.4% 5.048e+08 perf-stat.ps.cache-references 3.306e+11 +7.2% 3.545e+11 perf-stat.ps.cpu-cycles 77.02 ± 3% +36.1% 104.84 ± 3% perf-stat.ps.cpu-migrations 15419654 ± 2% +29.7% 19995292 ± 5% perf-stat.ps.dTLB-load-misses 3.054e+10 -3.3% 2.953e+10 perf-stat.ps.dTLB-loads 5.29e+09 -12.8% 4.615e+09 perf-stat.ps.dTLB-stores 1.194e+11 -3.2% 1.156e+11 perf-stat.ps.instructions 261922 -14.4% 224111 perf-stat.ps.minor-faults 8359208 +14.4% 9564791 ± 2% perf-stat.ps.node-load-misses 8585846 +6.5% 9148169 ± 2% perf-stat.ps.node-store-misses 5866543 -18.7% 4770903 ± 3% perf-stat.ps.node-stores 261923 -14.4% 224112 perf-stat.ps.page-faults 4.549e+13 +6.8% 4.86e+13 perf-stat.total.instructions 146322 +12.5% 164677 ± 4% sched_debug.cfs_rq:/.exec_clock.min 3282 ± 9% -39.8% 1974 ± 34% sched_debug.cfs_rq:/.exec_clock.stddev 12942 ± 7% +145.1% 31723 ± 64% sched_debug.cfs_rq:/.load.avg 715794 +19.1% 852777 ± 7% sched_debug.cfs_rq:/.load.max 80424 ± 4% +54.9% 124602 ± 15% sched_debug.cfs_rq:/.load.stddev 15.70 ± 3% +199.3% 46.99 ± 48% sched_debug.cfs_rq:/.load_avg.avg 812.93 ± 2% +235.8% 2730 ±107% sched_debug.cfs_rq:/.load_avg.max 92.38 ± 2% +182.3% 260.75 ± 77% sched_debug.cfs_rq:/.load_avg.stddev 0.57 +12.3% 0.64 ± 7% sched_debug.cfs_rq:/.nr_running.avg 45.06 ± 6% +166.1% 119.89 ± 14% sched_debug.cfs_rq:/.nr_spread_over.avg 9.79 ± 8% +582.1% 66.75 ± 18% sched_debug.cfs_rq:/.nr_spread_over.min 11.88 ± 8% +153.4% 30.10 ± 67% sched_debug.cfs_rq:/.runnable_load_avg.avg 697.64 +19.3% 832.26 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.max 75.35 ± 6% +57.0% 118.29 ± 16% sched_debug.cfs_rq:/.runnable_load_avg.stddev 12924 ± 7% +145.3% 31708 ± 64% sched_debug.cfs_rq:/.runnable_weight.avg 715033 +19.2% 852595 ± 7% sched_debug.cfs_rq:/.runnable_weight.max 80371 ± 4% +55.0% 124592 ± 15% sched_debug.cfs_rq:/.runnable_weight.stddev 2622597 ±128% -139.8% -1044828 sched_debug.cfs_rq:/.spread0.avg 3897757 ± 85% -71.2% 1123051 ± 50% sched_debug.cfs_rq:/.spread0.max 594.56 +10.4% 656.49 ± 5% sched_debug.cfs_rq:/.util_avg.avg 152112 ± 11% -17.0% 126240 ± 3% sched_debug.cpu.avg_idle.stddev 11.80 ± 9% +155.5% 30.14 ± 67% sched_debug.cpu.cpu_load[0].avg 697.54 +19.6% 834.44 ± 7% sched_debug.cpu.cpu_load[0].max 75.04 ± 6% +57.7% 118.35 ± 16% sched_debug.cpu.cpu_load[0].stddev 11.88 ± 8% +146.3% 29.25 ± 64% sched_debug.cpu.cpu_load[1].avg 703.07 ± 2% +20.4% 846.45 ± 7% sched_debug.cpu.cpu_load[1].max 75.31 ± 6% +54.9% 116.68 ± 15% sched_debug.cpu.cpu_load[1].stddev 12.06 ± 7% +118.4% 26.34 ± 52% sched_debug.cpu.cpu_load[2].avg 709.89 ± 2% +18.8% 843.29 ± 7% sched_debug.cpu.cpu_load[2].max 75.65 ± 5% +49.4% 113.05 ± 10% sched_debug.cpu.cpu_load[2].stddev 12.29 ± 7% +91.8% 23.56 ± 37% sched_debug.cpu.cpu_load[3].avg 724.54 ± 3% +21.6% 880.81 ± 7% sched_debug.cpu.cpu_load[3].max 76.60 ± 5% +45.8% 111.65 ± 7% sched_debug.cpu.cpu_load[3].stddev 12.52 ± 7% +72.8% 21.64 ± 24% sched_debug.cpu.cpu_load[4].avg 78.36 ± 5% +42.4% 111.55 ± 7% sched_debug.cpu.cpu_load[4].stddev 1916 +9.7% 2101 ± 5% sched_debug.cpu.curr->pid.avg 13727 ± 7% +131.1% 31720 ± 64% sched_debug.cpu.load.avg 85274 ± 7% +46.1% 124599 ± 15% sched_debug.cpu.load.stddev 1462 ± 9% +73.7% 2539 ± 3% sched_debug.cpu.nr_load_updates.stddev 0.58 +13.8% 0.66 ± 8% sched_debug.cpu.nr_running.avg -7.75 +41.9% -11.00 sched_debug.cpu.nr_uninterruptible.min 1508 ± 2% +12.8% 1702 ± 8% sched_debug.cpu.sched_count.min 1327 ± 2% -12.9% 1155 ± 7% sched_debug.cpu.ttwu_count.stddev 9211 ± 9% -31.3% 6325 ± 3% sched_debug.cpu.ttwu_local.max 1051 ± 3% -14.6% 898.23 ± 5% sched_debug.cpu.ttwu_local.stddev 19496 ± 2% -13.6% 16843 ± 4% softirqs.CPU124.SCHED 19122 ± 3% -13.8% 16480 ± 3% softirqs.CPU126.SCHED 26530 ± 50% -37.0% 16709 ± 3% softirqs.CPU127.SCHED 26796 ± 53% -38.3% 16523 ± 3% softirqs.CPU132.SCHED 18659 ± 5% -11.2% 16563 ± 4% softirqs.CPU134.SCHED 19039 ± 3% -13.4% 16491 ± 3% softirqs.CPU136.SCHED 19163 ± 5% -13.0% 16674 ± 5% softirqs.CPU137.SCHED 18948 ± 4% -9.8% 17089 ± 4% softirqs.CPU138.SCHED 18688 ± 3% -10.7% 16697 ± 3% softirqs.CPU139.SCHED 18491 ± 3% -11.2% 16414 ± 3% softirqs.CPU140.SCHED 18667 ± 4% -9.1% 16960 ± 2% softirqs.CPU141.SCHED 140329 ± 3% +10.9% 155641 ± 4% softirqs.CPU169.TIMER 140988 ± 3% +10.0% 155137 ± 4% softirqs.CPU170.TIMER 142711 ± 2% +8.8% 155340 ± 3% softirqs.CPU171.TIMER 139544 ± 3% +11.4% 155411 ± 3% softirqs.CPU173.TIMER 140477 ± 2% +10.7% 155463 ± 3% softirqs.CPU174.TIMER 138970 ± 3% +11.2% 154492 ± 3% softirqs.CPU175.TIMER 139728 ± 4% +11.1% 155294 ± 3% softirqs.CPU176.TIMER 138840 ± 3% +11.7% 155036 ± 3% softirqs.CPU177.TIMER 140018 ± 2% +10.6% 154801 ± 3% softirqs.CPU178.TIMER 139763 ± 3% +11.0% 155202 ± 3% softirqs.CPU179.TIMER 23963 ± 34% -31.2% 16476 ± 5% softirqs.CPU18.SCHED 140399 ± 2% +10.3% 154855 ± 3% softirqs.CPU180.TIMER 140094 ± 3% +10.6% 154971 ± 4% softirqs.CPU181.TIMER 22766 ± 30% +40.7% 32033 ± 13% softirqs.CPU182.RCU 142632 ± 2% +8.8% 155124 ± 4% softirqs.CPU182.TIMER 140684 ± 2% +10.1% 154922 ± 3% softirqs.CPU185.TIMER 23093 ± 21% +55.8% 35988 ± 27% softirqs.CPU186.RCU 140208 ± 3% +11.2% 155873 ± 3% softirqs.CPU186.TIMER 141405 ± 3% +9.6% 154967 ± 3% softirqs.CPU187.TIMER 23015 ± 24% +36.7% 31453 ± 23% softirqs.CPU189.RCU 142420 ± 3% +9.1% 155374 ± 4% softirqs.CPU189.TIMER 140654 ± 2% +10.0% 154764 ± 3% softirqs.CPU190.TIMER 141215 ± 2% +9.6% 154724 ± 3% softirqs.CPU191.TIMER 18432 ± 3% -10.8% 16445 ± 3% softirqs.CPU27.SCHED 18155 ± 4% -7.9% 16730 ± 4% softirqs.CPU29.SCHED 25114 ± 44% -34.3% 16491 ± 3% softirqs.CPU31.SCHED 19271 ± 5% -14.8% 16427 ± 4% softirqs.CPU34.SCHED 18734 ± 4% -12.0% 16493 ± 3% softirqs.CPU38.SCHED 18417 ± 3% -9.6% 16658 softirqs.CPU39.SCHED 18704 ± 5% -10.2% 16792 ± 5% softirqs.CPU44.SCHED 18523 ± 4% -10.2% 16626 ± 3% softirqs.CPU45.SCHED 18972 ± 3% -12.1% 16675 ± 3% softirqs.CPU46.SCHED 18568 ± 5% -11.6% 16409 ± 3% softirqs.CPU47.SCHED 143008 ± 2% +10.3% 157719 ± 4% softirqs.CPU72.TIMER 141321 ± 2% +11.2% 157080 ± 4% softirqs.CPU73.TIMER 141658 ± 2% +10.9% 157144 ± 4% softirqs.CPU74.TIMER 143549 ± 2% +9.5% 157155 ± 4% softirqs.CPU75.TIMER 140485 ± 2% +11.7% 156970 ± 4% softirqs.CPU76.TIMER 140406 ± 3% +11.6% 156742 ± 3% softirqs.CPU77.TIMER 141343 ± 3% +11.1% 157023 ± 3% softirqs.CPU78.TIMER 140677 ± 3% +11.8% 157279 ± 3% softirqs.CPU79.TIMER 140022 ± 3% +12.0% 156792 ± 3% softirqs.CPU81.TIMER 141168 ± 2% +11.1% 156775 ± 3% softirqs.CPU82.TIMER 140581 ± 3% +11.5% 156700 ± 3% softirqs.CPU83.TIMER 141291 ± 2% +11.0% 156803 ± 3% softirqs.CPU84.TIMER 22257 ± 16% +105.5% 45738 ± 72% softirqs.CPU85.RCU 141917 ± 2% +10.5% 156757 ± 4% softirqs.CPU85.TIMER 143293 ± 3% +9.6% 156988 ± 4% softirqs.CPU86.TIMER 142660 ± 2% +9.9% 156730 ± 4% softirqs.CPU89.TIMER 23086 ± 18% +74.0% 40174 ± 33% softirqs.CPU90.RCU 141582 ± 2% +10.5% 156488 ± 3% softirqs.CPU90.TIMER 23967 ± 22% +77.1% 42441 ± 34% softirqs.CPU93.RCU 143091 ± 3% +9.8% 157096 ± 3% softirqs.CPU93.TIMER 142056 ± 3% +10.1% 156402 ± 3% softirqs.CPU94.TIMER 142401 ± 2% +9.9% 156443 ± 4% softirqs.CPU95.TIMER 197.75 ± 6% +29.5% 256.00 ± 26% interrupts.115:IR-PCI-MSI.1574917-edge.eth1-TxRx-5 218.25 ± 12% +90.6% 416.00 ± 52% interrupts.117:IR-PCI-MSI.1574919-edge.eth1-TxRx-7 195.00 ± 3% +52.8% 298.00 ± 40% interrupts.124:IR-PCI-MSI.1574922-edge.eth1-TxRx-10 201.75 ± 4% +217.1% 639.75 ±112% interrupts.126:IR-PCI-MSI.1574924-edge.eth1-TxRx-12 186.50 +10.1% 205.25 interrupts.130:IR-PCI-MSI.1574928-edge.eth1-TxRx-16 186.75 +10.0% 205.50 interrupts.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17 187.00 +9.5% 204.75 interrupts.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18 192.00 ± 3% +10.9% 213.00 ± 5% interrupts.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19 187.25 +12.0% 209.75 ± 4% interrupts.135:IR-PCI-MSI.1574933-edge.eth1-TxRx-21 187.75 +12.9% 212.00 ± 5% interrupts.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24 187.50 +33.5% 250.25 ± 30% interrupts.139:IR-PCI-MSI.1574937-edge.eth1-TxRx-25 186.75 +10.2% 205.75 interrupts.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26 186.50 +10.2% 205.50 interrupts.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29 187.75 +9.7% 206.00 interrupts.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33 189.50 ± 2% +66.1% 314.75 ± 59% interrupts.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37 191.00 ± 3% +8.1% 206.50 ± 2% interrupts.153:IR-PCI-MSI.1574951-edge.eth1-TxRx-39 187.75 ± 2% +10.4% 207.25 interrupts.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41 186.75 +9.6% 204.75 interrupts.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43 186.75 +9.9% 205.25 interrupts.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44 186.50 +11.4% 207.75 ± 2% interrupts.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45 187.00 +10.2% 206.00 ± 2% interrupts.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46 187.00 +9.9% 205.50 interrupts.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47 186.50 +10.1% 205.25 interrupts.166:IR-PCI-MSI.1574964-edge.eth1-TxRx-52 186.50 +10.6% 206.25 ± 2% interrupts.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53 187.00 +10.8% 207.25 ± 2% interrupts.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55 186.50 +62.2% 302.50 ± 54% interrupts.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56 186.75 +13.4% 211.75 ± 2% interrupts.171:IR-PCI-MSI.1574969-edge.eth1-TxRx-57 186.50 +13.1% 211.00 ± 4% interrupts.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58 186.50 +10.9% 206.75 interrupts.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60 186.50 +9.9% 205.00 interrupts.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61 53698095 +7.5% 57743002 interrupts.CAL:Function_call_interrupts 254909 ± 10% +17.3% 298898 ± 2% interrupts.CPU0.CAL:Function_call_interrupts 405763 ± 10% +14.4% 464369 ± 4% interrupts.CPU0.TLB:TLB_shootdowns 270140 ± 2% +11.8% 301941 ± 3% interrupts.CPU1.CAL:Function_call_interrupts 538752 ± 20% +21.0% 651896 ± 18% interrupts.CPU1.LOC:Local_timer_interrupts 434684 +7.9% 469029 ± 4% interrupts.CPU1.TLB:TLB_shootdowns 195.00 ± 3% +52.8% 298.00 ± 40% interrupts.CPU10.124:IR-PCI-MSI.1574922-edge.eth1-TxRx-10 252424 ± 13% +20.7% 304589 ± 4% interrupts.CPU10.CAL:Function_call_interrupts 394441 ± 14% +19.8% 472609 ± 4% interrupts.CPU10.TLB:TLB_shootdowns 256138 ± 8% +17.8% 301674 ± 5% interrupts.CPU100.CAL:Function_call_interrupts 193.50 ± 77% +9145.9% 17890 ±170% interrupts.CPU100.RES:Rescheduling_interrupts 156.50 ± 30% +94.4% 304.25 ± 19% interrupts.CPU101.RES:Rescheduling_interrupts 539515 ± 19% +20.7% 651403 ± 18% interrupts.CPU102.LOC:Local_timer_interrupts 126.00 ± 13% +179.2% 351.75 ± 48% interrupts.CPU102.RES:Rescheduling_interrupts 541336 ± 19% +20.1% 650053 ± 18% interrupts.CPU103.LOC:Local_timer_interrupts 7239 -18.2% 5924 ± 23% interrupts.CPU103.NMI:Non-maskable_interrupts 7239 -18.2% 5924 ± 23% interrupts.CPU103.PMI:Performance_monitoring_interrupts 108.00 ± 23% +12469.2% 13574 ±168% interrupts.CPU103.RES:Rescheduling_interrupts 538254 ± 19% +20.6% 649347 ± 18% interrupts.CPU104.LOC:Local_timer_interrupts 144.50 ± 49% +135.5% 340.25 ± 46% interrupts.CPU104.RES:Rescheduling_interrupts 254075 ± 9% +20.7% 306757 interrupts.CPU106.CAL:Function_call_interrupts 399189 ± 8% +19.8% 478199 interrupts.CPU106.TLB:TLB_shootdowns 265502 ± 3% +12.5% 298648 interrupts.CPU107.CAL:Function_call_interrupts 540413 ± 19% +20.1% 649197 ± 18% interrupts.CPU107.LOC:Local_timer_interrupts 96.00 ± 22% +395.3% 475.50 ± 62% interrupts.CPU107.RES:Rescheduling_interrupts 538494 ± 19% +22.1% 657367 ± 17% interrupts.CPU108.LOC:Local_timer_interrupts 138.25 ± 35% +219.2% 441.25 ± 56% interrupts.CPU108.RES:Rescheduling_interrupts 266470 +11.3% 296574 interrupts.CPU109.CAL:Function_call_interrupts 537756 ± 20% +20.9% 650060 ± 18% interrupts.CPU109.LOC:Local_timer_interrupts 538917 ± 19% +21.4% 654061 ± 18% interrupts.CPU11.LOC:Local_timer_interrupts 124.75 ± 38% +122.8% 278.00 ± 40% interrupts.CPU11.RES:Rescheduling_interrupts 274033 ± 2% +10.5% 302693 ± 2% interrupts.CPU111.CAL:Function_call_interrupts 538480 ± 19% +20.2% 647416 ± 18% interrupts.CPU111.LOC:Local_timer_interrupts 73.00 ± 38% +204.1% 222.00 ± 49% interrupts.CPU111.RES:Rescheduling_interrupts 540260 ± 19% +19.5% 645594 ± 18% interrupts.CPU112.LOC:Local_timer_interrupts 540547 ± 19% +20.5% 651244 ± 18% interrupts.CPU113.LOC:Local_timer_interrupts 7299 -18.9% 5917 ± 24% interrupts.CPU113.NMI:Non-maskable_interrupts 7299 -18.9% 5917 ± 24% interrupts.CPU113.PMI:Performance_monitoring_interrupts 128.25 ± 77% +88.3% 241.50 ± 39% interrupts.CPU113.RES:Rescheduling_interrupts 538109 ± 19% +20.7% 649662 ± 18% interrupts.CPU114.LOC:Local_timer_interrupts 257287 ± 8% +12.4% 289070 ± 3% interrupts.CPU115.CAL:Function_call_interrupts 546154 ± 18% +19.0% 649740 ± 18% interrupts.CPU115.LOC:Local_timer_interrupts 264738 ± 2% +10.7% 292963 interrupts.CPU116.CAL:Function_call_interrupts 541256 ± 19% +19.8% 648170 ± 18% interrupts.CPU116.LOC:Local_timer_interrupts 7275 -30.0% 5095 ± 33% interrupts.CPU117.NMI:Non-maskable_interrupts 7275 -30.0% 5095 ± 33% interrupts.CPU117.PMI:Performance_monitoring_interrupts 540426 ± 19% +20.0% 648576 ± 18% interrupts.CPU118.LOC:Local_timer_interrupts 422235 +10.3% 465569 ± 4% interrupts.CPU118.TLB:TLB_shootdowns 275052 ± 4% +9.3% 300614 ± 6% interrupts.CPU119.CAL:Function_call_interrupts 539764 ± 19% +20.0% 647739 ± 18% interrupts.CPU119.LOC:Local_timer_interrupts 86.50 ± 43% +155.5% 221.00 ± 34% interrupts.CPU119.RES:Rescheduling_interrupts 201.75 ± 4% +217.1% 639.75 ±112% interrupts.CPU12.126:IR-PCI-MSI.1574924-edge.eth1-TxRx-12 270935 ± 5% +10.6% 299611 ± 4% interrupts.CPU12.CAL:Function_call_interrupts 540696 ± 19% +24.0% 670292 ± 16% interrupts.CPU12.LOC:Local_timer_interrupts 545199 ± 18% +17.9% 642725 ± 18% interrupts.CPU122.LOC:Local_timer_interrupts 539304 ± 19% +19.0% 641812 ± 18% interrupts.CPU123.LOC:Local_timer_interrupts 546689 ± 18% +17.8% 644213 ± 18% interrupts.CPU124.LOC:Local_timer_interrupts 543898 ± 19% +20.6% 655876 ± 15% interrupts.CPU125.LOC:Local_timer_interrupts 546382 ± 19% +17.4% 641472 ± 17% interrupts.CPU126.LOC:Local_timer_interrupts 308438 ± 2% -7.8% 284322 ± 4% interrupts.CPU127.CAL:Function_call_interrupts 542124 ± 19% +19.4% 647461 ± 16% interrupts.CPU127.LOC:Local_timer_interrupts 7286 -30.8% 5044 ± 32% interrupts.CPU127.NMI:Non-maskable_interrupts 7286 -30.8% 5044 ± 32% interrupts.CPU127.PMI:Performance_monitoring_interrupts 478527 ± 2% -11.7% 422345 ± 6% interrupts.CPU127.TLB:TLB_shootdowns 542707 ± 19% +18.4% 642307 ± 17% interrupts.CPU128.LOC:Local_timer_interrupts 269272 ± 5% +6.7% 287335 ± 4% interrupts.CPU13.CAL:Function_call_interrupts 538587 ± 19% +21.9% 656282 ± 18% interrupts.CPU13.LOC:Local_timer_interrupts 540534 ± 19% +19.2% 644104 ± 17% interrupts.CPU130.LOC:Local_timer_interrupts 7277 -19.2% 5879 ± 22% interrupts.CPU130.NMI:Non-maskable_interrupts 7277 -19.2% 5879 ± 22% interrupts.CPU130.PMI:Performance_monitoring_interrupts 538467 ± 20% +18.9% 640023 ± 18% interrupts.CPU132.LOC:Local_timer_interrupts 540300 ± 19% +19.8% 647453 ± 16% interrupts.CPU134.LOC:Local_timer_interrupts 7259 -18.9% 5885 ± 22% interrupts.CPU134.NMI:Non-maskable_interrupts 7259 -18.9% 5885 ± 22% interrupts.CPU134.PMI:Performance_monitoring_interrupts 543505 ± 19% +17.7% 639974 ± 18% interrupts.CPU135.LOC:Local_timer_interrupts 95.00 ± 7% +92.9% 183.25 ± 38% interrupts.CPU136.RES:Rescheduling_interrupts 85.50 ± 35% +147.1% 211.25 ± 64% interrupts.CPU137.RES:Rescheduling_interrupts 542153 ± 19% +17.9% 639317 ± 18% interrupts.CPU139.LOC:Local_timer_interrupts 71.50 ± 39% +677.3% 555.75 ±106% interrupts.CPU139.RES:Rescheduling_interrupts 539602 ± 19% +18.6% 640147 ± 18% interrupts.CPU140.LOC:Local_timer_interrupts 538627 ± 19% +21.0% 651502 ± 17% interrupts.CPU144.LOC:Local_timer_interrupts 544865 ± 18% +19.4% 650795 ± 18% interrupts.CPU145.LOC:Local_timer_interrupts 303613 +11.0% 336937 ± 2% interrupts.CPU146.CAL:Function_call_interrupts 540377 ± 19% +21.2% 654817 ± 17% interrupts.CPU146.LOC:Local_timer_interrupts 540064 ± 19% +20.7% 651799 ± 18% interrupts.CPU147.LOC:Local_timer_interrupts 305856 ± 3% +11.0% 339566 ± 2% interrupts.CPU148.CAL:Function_call_interrupts 542683 ± 19% +19.4% 647852 ± 18% interrupts.CPU148.LOC:Local_timer_interrupts 541661 ± 19% +20.0% 650025 ± 18% interrupts.CPU149.LOC:Local_timer_interrupts 539881 ± 19% +21.6% 656225 ± 18% interrupts.CPU15.LOC:Local_timer_interrupts 89.25 ± 19% +167.8% 239.00 ± 13% interrupts.CPU15.RES:Rescheduling_interrupts 541985 ± 19% +20.0% 650169 ± 18% interrupts.CPU150.LOC:Local_timer_interrupts 540791 ± 19% +19.8% 647950 ± 18% interrupts.CPU151.LOC:Local_timer_interrupts 101.00 ± 16% +231.2% 334.50 ± 91% interrupts.CPU151.RES:Rescheduling_interrupts 538408 ± 19% +21.2% 652752 ± 17% interrupts.CPU152.LOC:Local_timer_interrupts 82.25 ± 25% +192.4% 240.50 ± 24% interrupts.CPU152.RES:Rescheduling_interrupts 305245 ± 4% +13.0% 344930 ± 2% interrupts.CPU153.CAL:Function_call_interrupts 541279 ± 19% +20.7% 653300 ± 17% interrupts.CPU153.LOC:Local_timer_interrupts 7275 -20.2% 5802 ± 22% interrupts.CPU153.NMI:Non-maskable_interrupts 7275 -20.2% 5802 ± 22% interrupts.CPU153.PMI:Performance_monitoring_interrupts 448157 ± 3% +10.1% 493421 ± 3% interrupts.CPU153.TLB:TLB_shootdowns 540904 ± 19% +20.2% 649897 ± 18% interrupts.CPU154.LOC:Local_timer_interrupts 113.00 ± 45% +5616.4% 6459 ±168% interrupts.CPU154.RES:Rescheduling_interrupts 541323 ± 19% +20.5% 652368 ± 18% interrupts.CPU155.LOC:Local_timer_interrupts 539888 ± 19% +20.8% 652010 ± 17% interrupts.CPU156.LOC:Local_timer_interrupts 105.00 ± 26% +47.6% 155.00 ± 26% interrupts.CPU156.RES:Rescheduling_interrupts 538866 ± 19% +20.6% 649649 ± 18% interrupts.CPU157.LOC:Local_timer_interrupts 539999 ± 19% +20.5% 650482 ± 17% interrupts.CPU158.LOC:Local_timer_interrupts 7312 -20.3% 5825 ± 22% interrupts.CPU158.NMI:Non-maskable_interrupts 7312 -20.3% 5825 ± 22% interrupts.CPU158.PMI:Performance_monitoring_interrupts 539832 ± 19% +20.3% 649376 ± 18% interrupts.CPU159.LOC:Local_timer_interrupts 186.50 +10.1% 205.25 interrupts.CPU16.130:IR-PCI-MSI.1574928-edge.eth1-TxRx-16 540127 ± 19% +20.6% 651201 ± 18% interrupts.CPU16.LOC:Local_timer_interrupts 538514 ± 19% +20.9% 651157 ± 18% interrupts.CPU160.LOC:Local_timer_interrupts 92.50 ± 36% +83.8% 170.00 ± 26% interrupts.CPU160.RES:Rescheduling_interrupts 538197 ± 20% +20.7% 649566 ± 18% interrupts.CPU161.LOC:Local_timer_interrupts 285829 ± 7% +16.1% 331859 ± 4% interrupts.CPU162.CAL:Function_call_interrupts 539794 ± 20% +20.3% 649196 ± 18% interrupts.CPU162.LOC:Local_timer_interrupts 542663 ± 19% +20.1% 651922 ± 17% interrupts.CPU163.LOC:Local_timer_interrupts 543820 ± 19% +19.3% 648877 ± 18% interrupts.CPU164.LOC:Local_timer_interrupts 7290 -31.1% 5024 ± 32% interrupts.CPU164.NMI:Non-maskable_interrupts 7290 -31.1% 5024 ± 32% interrupts.CPU164.PMI:Performance_monitoring_interrupts 542758 ± 19% +19.5% 648373 ± 18% interrupts.CPU165.LOC:Local_timer_interrupts 540588 ± 19% +20.5% 651149 ± 17% interrupts.CPU167.LOC:Local_timer_interrupts 7127 -52.2% 3404 ± 5% interrupts.CPU168.NMI:Non-maskable_interrupts 7127 -52.2% 3404 ± 5% interrupts.CPU168.PMI:Performance_monitoring_interrupts 244836 +9.7% 268535 ± 4% interrupts.CPU169.CAL:Function_call_interrupts 541617 ± 19% +22.4% 662679 ± 16% interrupts.CPU169.LOC:Local_timer_interrupts 186.75 +10.0% 205.50 interrupts.CPU17.131:IR-PCI-MSI.1574929-edge.eth1-TxRx-17 540858 ± 19% +20.9% 653952 ± 19% interrupts.CPU17.LOC:Local_timer_interrupts 241660 ± 6% +13.8% 274997 ± 4% interrupts.CPU170.CAL:Function_call_interrupts 542488 ± 19% +20.6% 654496 ± 16% interrupts.CPU170.LOC:Local_timer_interrupts 371384 ± 7% +11.8% 415074 ± 4% interrupts.CPU170.TLB:TLB_shootdowns 242121 ± 5% +16.0% 280945 ± 2% interrupts.CPU171.CAL:Function_call_interrupts 376541 ± 7% +14.4% 430803 ± 3% interrupts.CPU171.TLB:TLB_shootdowns 542151 ± 19% +21.3% 657582 ± 16% interrupts.CPU172.LOC:Local_timer_interrupts 7085 -29.1% 5022 ± 29% interrupts.CPU172.NMI:Non-maskable_interrupts 7085 -29.1% 5022 ± 29% interrupts.CPU172.PMI:Performance_monitoring_interrupts 240899 ± 3% +13.5% 273395 ± 5% interrupts.CPU173.CAL:Function_call_interrupts 541208 ± 19% +21.2% 656008 ± 16% interrupts.CPU173.LOC:Local_timer_interrupts 369290 ± 4% +12.5% 415566 ± 5% interrupts.CPU173.TLB:TLB_shootdowns 227863 ± 3% +16.0% 264249 ± 5% interrupts.CPU174.CAL:Function_call_interrupts 350987 ± 5% +14.1% 400420 ± 5% interrupts.CPU174.TLB:TLB_shootdowns 238834 ± 4% +16.6% 278474 ± 3% interrupts.CPU175.CAL:Function_call_interrupts 373355 ± 6% +13.0% 421812 ± 4% interrupts.CPU175.TLB:TLB_shootdowns 235752 ± 5% +17.1% 276163 ± 4% interrupts.CPU176.CAL:Function_call_interrupts 362982 ± 6% +15.5% 419078 ± 5% interrupts.CPU176.TLB:TLB_shootdowns 541447 ± 19% +21.5% 658031 ± 15% interrupts.CPU177.LOC:Local_timer_interrupts 147.75 ± 48% +2620.3% 4019 ±163% interrupts.CPU177.RES:Rescheduling_interrupts 247600 ± 4% +14.2% 282868 ± 4% interrupts.CPU178.CAL:Function_call_interrupts 87.25 ± 12% +214.0% 274.00 ± 66% interrupts.CPU178.RES:Rescheduling_interrupts 382356 ± 4% +14.0% 436065 ± 5% interrupts.CPU178.TLB:TLB_shootdowns 241472 ± 6% +16.1% 280320 interrupts.CPU179.CAL:Function_call_interrupts 371820 ± 6% +14.8% 426942 interrupts.CPU179.TLB:TLB_shootdowns 187.00 +9.5% 204.75 interrupts.CPU18.132:IR-PCI-MSI.1574930-edge.eth1-TxRx-18 538689 ± 19% +21.6% 654863 ± 18% interrupts.CPU18.LOC:Local_timer_interrupts 235652 ± 3% +15.8% 272978 ± 3% interrupts.CPU181.CAL:Function_call_interrupts 542706 ± 19% +21.2% 657607 ± 15% interrupts.CPU181.LOC:Local_timer_interrupts 80.25 ± 33% +91.9% 154.00 ± 24% interrupts.CPU181.RES:Rescheduling_interrupts 363735 ± 2% +12.2% 408143 ± 5% interrupts.CPU181.TLB:TLB_shootdowns 232778 ± 6% +21.1% 282009 ± 4% interrupts.CPU182.CAL:Function_call_interrupts 356162 ± 9% +21.4% 432541 ± 4% interrupts.CPU182.TLB:TLB_shootdowns 235311 +18.5% 278801 ± 4% interrupts.CPU183.CAL:Function_call_interrupts 543730 ± 19% +20.8% 656877 ± 16% interrupts.CPU183.LOC:Local_timer_interrupts 370116 +15.1% 425984 ± 6% interrupts.CPU183.TLB:TLB_shootdowns 236140 ± 7% +15.7% 273246 interrupts.CPU184.CAL:Function_call_interrupts 361940 ± 6% +14.2% 413381 ± 2% interrupts.CPU184.TLB:TLB_shootdowns 240094 ± 2% +17.6% 282320 ± 3% interrupts.CPU185.CAL:Function_call_interrupts 373915 ± 2% +14.6% 428620 ± 5% interrupts.CPU185.TLB:TLB_shootdowns 540897 ± 19% +22.8% 664231 ± 17% interrupts.CPU186.LOC:Local_timer_interrupts 238146 ± 3% +14.9% 273691 ± 7% interrupts.CPU187.CAL:Function_call_interrupts 542778 ± 19% +21.5% 659277 ± 15% interrupts.CPU187.LOC:Local_timer_interrupts 130.00 ± 67% +15364.6% 20104 ±171% interrupts.CPU187.RES:Rescheduling_interrupts 362251 ± 3% +14.7% 415565 ± 9% interrupts.CPU187.TLB:TLB_shootdowns 233794 ± 5% +16.6% 272593 ± 4% interrupts.CPU188.CAL:Function_call_interrupts 7101 ± 2% -28.8% 5058 ± 32% interrupts.CPU188.NMI:Non-maskable_interrupts 7101 ± 2% -28.8% 5058 ± 32% interrupts.CPU188.PMI:Performance_monitoring_interrupts 354236 ± 4% +16.7% 413453 ± 4% interrupts.CPU188.TLB:TLB_shootdowns 236717 ± 2% +15.8% 274099 ± 3% interrupts.CPU189.CAL:Function_call_interrupts 541842 ± 19% +21.0% 655869 ± 16% interrupts.CPU189.LOC:Local_timer_interrupts 360252 ± 3% +15.5% 416268 ± 4% interrupts.CPU189.TLB:TLB_shootdowns 192.00 ± 3% +10.9% 213.00 ± 5% interrupts.CPU19.133:IR-PCI-MSI.1574931-edge.eth1-TxRx-19 540779 ± 19% +21.9% 659130 ± 16% interrupts.CPU190.LOC:Local_timer_interrupts 102.00 ± 37% +165.4% 270.75 ± 53% interrupts.CPU190.RES:Rescheduling_interrupts 237645 ± 6% +17.4% 278882 ± 4% interrupts.CPU191.CAL:Function_call_interrupts 7096 -40.8% 4197 ± 30% interrupts.CPU191.NMI:Non-maskable_interrupts 7096 -40.8% 4197 ± 30% interrupts.CPU191.PMI:Performance_monitoring_interrupts 115.75 ± 20% +56.6% 181.25 ± 14% interrupts.CPU191.RES:Rescheduling_interrupts 538458 ± 19% +21.7% 655406 ± 17% interrupts.CPU2.LOC:Local_timer_interrupts 539855 ± 19% +22.1% 659223 ± 16% interrupts.CPU20.LOC:Local_timer_interrupts 178.50 ± 56% +125.6% 402.75 ± 43% interrupts.CPU20.RES:Rescheduling_interrupts 187.25 +12.0% 209.75 ± 4% interrupts.CPU21.135:IR-PCI-MSI.1574933-edge.eth1-TxRx-21 542028 ± 19% +20.3% 652018 ± 18% interrupts.CPU21.LOC:Local_timer_interrupts 540724 ± 19% +20.5% 651349 ± 18% interrupts.CPU22.LOC:Local_timer_interrupts 7256 -19.1% 5869 ± 21% interrupts.CPU22.NMI:Non-maskable_interrupts 7256 -19.1% 5869 ± 21% interrupts.CPU22.PMI:Performance_monitoring_interrupts 270299 +11.7% 301958 ± 4% interrupts.CPU23.CAL:Function_call_interrupts 100.25 ± 16% +210.0% 310.75 ± 57% interrupts.CPU23.RES:Rescheduling_interrupts 438048 ± 3% +6.6% 467164 ± 5% interrupts.CPU23.TLB:TLB_shootdowns 187.75 +12.9% 212.00 ± 5% interrupts.CPU24.138:IR-PCI-MSI.1574936-edge.eth1-TxRx-24 187.50 +33.5% 250.25 ± 30% interrupts.CPU25.139:IR-PCI-MSI.1574937-edge.eth1-TxRx-25 186.75 +10.2% 205.75 interrupts.CPU26.140:IR-PCI-MSI.1574938-edge.eth1-TxRx-26 543634 ± 19% +19.8% 651384 ± 18% interrupts.CPU26.LOC:Local_timer_interrupts 540673 ± 19% +20.1% 649485 ± 18% interrupts.CPU27.LOC:Local_timer_interrupts 540215 ± 19% +20.1% 648845 ± 18% interrupts.CPU28.LOC:Local_timer_interrupts 7309 -19.7% 5867 ± 22% interrupts.CPU28.NMI:Non-maskable_interrupts 7309 -19.7% 5867 ± 22% interrupts.CPU28.PMI:Performance_monitoring_interrupts 186.50 +10.2% 205.50 interrupts.CPU29.143:IR-PCI-MSI.1574941-edge.eth1-TxRx-29 541202 ± 19% +19.7% 647935 ± 18% interrupts.CPU29.LOC:Local_timer_interrupts 542415 ± 19% +20.8% 655455 ± 18% interrupts.CPU31.LOC:Local_timer_interrupts 540412 ± 19% +19.5% 645872 ± 18% interrupts.CPU32.LOC:Local_timer_interrupts 187.75 +9.7% 206.00 interrupts.CPU33.147:IR-PCI-MSI.1574945-edge.eth1-TxRx-33 168.75 ± 83% +743.3% 1423 ±107% interrupts.CPU33.RES:Rescheduling_interrupts 540120 ± 19% +20.0% 647960 ± 17% interrupts.CPU34.LOC:Local_timer_interrupts 539074 ± 19% +20.1% 647521 ± 18% interrupts.CPU36.LOC:Local_timer_interrupts 189.50 ± 2% +66.1% 314.75 ± 59% interrupts.CPU37.151:IR-PCI-MSI.1574949-edge.eth1-TxRx-37 544930 ± 18% +19.5% 651311 ± 17% interrupts.CPU37.LOC:Local_timer_interrupts 7243 -18.0% 5937 ± 22% interrupts.CPU37.NMI:Non-maskable_interrupts 7243 -18.0% 5937 ± 22% interrupts.CPU37.PMI:Performance_monitoring_interrupts 540036 ± 19% +21.0% 653540 ± 17% interrupts.CPU38.LOC:Local_timer_interrupts 7259 -18.6% 5911 ± 24% interrupts.CPU38.NMI:Non-maskable_interrupts 7259 -18.6% 5911 ± 24% interrupts.CPU38.PMI:Performance_monitoring_interrupts 191.00 ± 3% +8.1% 206.50 ± 2% interrupts.CPU39.153:IR-PCI-MSI.1574951-edge.eth1-TxRx-39 540434 ± 19% +19.6% 646433 ± 18% interrupts.CPU39.LOC:Local_timer_interrupts 7243 -17.8% 5954 ± 23% interrupts.CPU39.NMI:Non-maskable_interrupts 7243 -17.8% 5954 ± 23% interrupts.CPU39.PMI:Performance_monitoring_interrupts 540872 ± 19% +19.3% 645453 ± 18% interrupts.CPU40.LOC:Local_timer_interrupts 187.75 ± 2% +10.4% 207.25 interrupts.CPU41.155:IR-PCI-MSI.1574953-edge.eth1-TxRx-41 540484 ± 19% +19.7% 647111 ± 18% interrupts.CPU41.LOC:Local_timer_interrupts 540667 ± 19% +20.0% 648663 ± 17% interrupts.CPU42.LOC:Local_timer_interrupts 186.75 +9.6% 204.75 interrupts.CPU43.157:IR-PCI-MSI.1574955-edge.eth1-TxRx-43 539381 ± 19% +19.8% 646270 ± 18% interrupts.CPU43.LOC:Local_timer_interrupts 186.75 +9.9% 205.25 interrupts.CPU44.158:IR-PCI-MSI.1574956-edge.eth1-TxRx-44 540624 ± 19% +20.2% 649611 ± 18% interrupts.CPU44.LOC:Local_timer_interrupts 186.50 +11.4% 207.75 ± 2% interrupts.CPU45.159:IR-PCI-MSI.1574957-edge.eth1-TxRx-45 7247 -18.6% 5897 ± 22% interrupts.CPU45.NMI:Non-maskable_interrupts 7247 -18.6% 5897 ± 22% interrupts.CPU45.PMI:Performance_monitoring_interrupts 115.50 ± 28% +109.3% 241.75 ± 20% interrupts.CPU45.RES:Rescheduling_interrupts 187.00 +10.2% 206.00 ± 2% interrupts.CPU46.160:IR-PCI-MSI.1574958-edge.eth1-TxRx-46 546326 ± 18% +19.0% 650389 ± 18% interrupts.CPU46.LOC:Local_timer_interrupts 187.00 +9.9% 205.50 interrupts.CPU47.161:IR-PCI-MSI.1574959-edge.eth1-TxRx-47 539681 ± 19% +21.4% 655000 ± 17% interrupts.CPU48.LOC:Local_timer_interrupts 297666 ± 4% +13.3% 337322 ± 3% interrupts.CPU49.CAL:Function_call_interrupts 542397 ± 19% +20.6% 653981 ± 18% interrupts.CPU49.LOC:Local_timer_interrupts 197.75 ± 6% +29.5% 256.00 ± 26% interrupts.CPU5.115:IR-PCI-MSI.1574917-edge.eth1-TxRx-5 543219 ± 19% +20.9% 656491 ± 17% interrupts.CPU5.LOC:Local_timer_interrupts 129.75 ± 12% +163.4% 341.75 ± 41% interrupts.CPU5.RES:Rescheduling_interrupts 538363 ± 19% +24.2% 668775 ± 16% interrupts.CPU50.LOC:Local_timer_interrupts 112.25 ± 23% +773.3% 980.25 ±126% interrupts.CPU50.RES:Rescheduling_interrupts 541054 ± 19% +21.0% 654804 ± 18% interrupts.CPU51.LOC:Local_timer_interrupts 143.50 ± 28% +118.3% 313.25 ± 18% interrupts.CPU51.RES:Rescheduling_interrupts 186.50 +10.1% 205.25 interrupts.CPU52.166:IR-PCI-MSI.1574964-edge.eth1-TxRx-52 315772 ± 2% +9.2% 344834 ± 4% interrupts.CPU52.CAL:Function_call_interrupts 541595 ± 19% +20.6% 653335 ± 17% interrupts.CPU52.LOC:Local_timer_interrupts 186.50 +10.6% 206.25 ± 2% interrupts.CPU53.167:IR-PCI-MSI.1574965-edge.eth1-TxRx-53 543567 ± 19% +20.9% 657395 ± 18% interrupts.CPU53.LOC:Local_timer_interrupts 541122 ± 19% +21.8% 659015 ± 17% interrupts.CPU54.LOC:Local_timer_interrupts 187.00 +10.8% 207.25 ± 2% interrupts.CPU55.169:IR-PCI-MSI.1574967-edge.eth1-TxRx-55 539942 ± 19% +21.2% 654145 ± 18% interrupts.CPU55.LOC:Local_timer_interrupts 7294 -33.0% 4885 ± 27% interrupts.CPU55.NMI:Non-maskable_interrupts 7294 -33.0% 4885 ± 27% interrupts.CPU55.PMI:Performance_monitoring_interrupts 186.50 +62.2% 302.50 ± 54% interrupts.CPU56.170:IR-PCI-MSI.1574968-edge.eth1-TxRx-56 539011 ± 19% +21.8% 656480 ± 17% interrupts.CPU56.LOC:Local_timer_interrupts 99.75 ± 31% +4463.4% 4552 ±160% interrupts.CPU56.RES:Rescheduling_interrupts 186.75 +13.4% 211.75 ± 2% interrupts.CPU57.171:IR-PCI-MSI.1574969-edge.eth1-TxRx-57 306775 ± 4% +11.0% 340460 ± 3% interrupts.CPU57.CAL:Function_call_interrupts 539066 ± 19% +21.5% 655208 ± 17% interrupts.CPU57.LOC:Local_timer_interrupts 186.50 +13.1% 211.00 ± 4% interrupts.CPU58.172:IR-PCI-MSI.1574970-edge.eth1-TxRx-58 541659 ± 19% +21.0% 655197 ± 18% interrupts.CPU58.LOC:Local_timer_interrupts 219.25 ± 76% +162.6% 575.75 ± 65% interrupts.CPU58.RES:Rescheduling_interrupts 538311 ± 19% +21.8% 655845 ± 17% interrupts.CPU59.LOC:Local_timer_interrupts 7290 -19.7% 5853 ± 23% interrupts.CPU59.NMI:Non-maskable_interrupts 7290 -19.7% 5853 ± 23% interrupts.CPU59.PMI:Performance_monitoring_interrupts 540426 ± 19% +21.2% 655199 ± 18% interrupts.CPU6.LOC:Local_timer_interrupts 186.50 +10.9% 206.75 interrupts.CPU60.174:IR-PCI-MSI.1574972-edge.eth1-TxRx-60 543220 ± 19% +20.2% 652812 ± 18% interrupts.CPU60.LOC:Local_timer_interrupts 186.50 +9.9% 205.00 interrupts.CPU61.175:IR-PCI-MSI.1574973-edge.eth1-TxRx-61 541985 ± 19% +21.4% 657809 ± 17% interrupts.CPU61.LOC:Local_timer_interrupts 540718 ± 20% +21.7% 657795 ± 17% interrupts.CPU62.LOC:Local_timer_interrupts 538260 ± 19% +21.6% 654528 ± 17% interrupts.CPU63.LOC:Local_timer_interrupts 104.25 ± 51% +77.9% 185.50 ± 33% interrupts.CPU63.RES:Rescheduling_interrupts 537795 ± 20% +21.4% 653100 ± 18% interrupts.CPU64.LOC:Local_timer_interrupts 7238 -32.3% 4897 ± 27% interrupts.CPU64.NMI:Non-maskable_interrupts 7238 -32.3% 4897 ± 27% interrupts.CPU64.PMI:Performance_monitoring_interrupts 538375 ± 20% +21.8% 655532 ± 18% interrupts.CPU65.LOC:Local_timer_interrupts 96.25 ± 34% +2305.7% 2315 ±157% interrupts.CPU65.RES:Rescheduling_interrupts 541216 ± 19% +20.8% 653940 ± 18% interrupts.CPU66.LOC:Local_timer_interrupts 539188 ± 19% +21.9% 657526 ± 17% interrupts.CPU67.LOC:Local_timer_interrupts 537928 ± 20% +21.6% 653906 ± 17% interrupts.CPU68.LOC:Local_timer_interrupts 121.75 ± 42% +68.0% 204.50 ± 26% interrupts.CPU68.RES:Rescheduling_interrupts 539066 ± 19% +21.5% 654731 ± 17% interrupts.CPU69.LOC:Local_timer_interrupts 218.25 ± 12% +90.6% 416.00 ± 52% interrupts.CPU7.117:IR-PCI-MSI.1574919-edge.eth1-TxRx-7 275986 ± 4% +12.9% 311689 ± 7% interrupts.CPU7.CAL:Function_call_interrupts 541779 ± 19% +20.2% 651472 ± 18% interrupts.CPU7.LOC:Local_timer_interrupts 437364 ± 6% +11.4% 487197 ± 7% interrupts.CPU7.TLB:TLB_shootdowns 537604 ± 20% +21.8% 654589 ± 17% interrupts.CPU70.LOC:Local_timer_interrupts 7284 -32.4% 4927 ± 26% interrupts.CPU70.NMI:Non-maskable_interrupts 7284 -32.4% 4927 ± 26% interrupts.CPU70.PMI:Performance_monitoring_interrupts 541179 ± 19% +21.7% 658843 ± 17% interrupts.CPU71.LOC:Local_timer_interrupts 7270 -19.6% 5847 ± 23% interrupts.CPU71.NMI:Non-maskable_interrupts 7270 -19.6% 5847 ± 23% interrupts.CPU71.PMI:Performance_monitoring_interrupts 557930 ± 18% +22.3% 682389 ± 14% interrupts.CPU72.LOC:Local_timer_interrupts 262229 ± 2% +9.5% 287014 ± 3% interrupts.CPU73.CAL:Function_call_interrupts 543128 ± 19% +22.9% 667640 ± 16% interrupts.CPU73.LOC:Local_timer_interrupts 123.00 ± 50% +100.6% 246.75 ± 6% interrupts.CPU73.RES:Rescheduling_interrupts 542726 ± 19% +22.5% 664815 ± 16% interrupts.CPU74.LOC:Local_timer_interrupts 126.25 ± 29% +95.8% 247.25 ± 27% interrupts.CPU74.RES:Rescheduling_interrupts 233414 ± 7% +18.8% 277393 ± 3% interrupts.CPU75.CAL:Function_call_interrupts 362258 ± 9% +16.3% 421464 ± 4% interrupts.CPU75.TLB:TLB_shootdowns 248530 ± 5% +13.8% 282876 interrupts.CPU76.CAL:Function_call_interrupts 541415 ± 19% +22.7% 664387 ± 16% interrupts.CPU76.LOC:Local_timer_interrupts 386104 ± 5% +11.7% 431283 ± 2% interrupts.CPU76.TLB:TLB_shootdowns 246491 ± 4% +15.9% 285607 ± 3% interrupts.CPU77.CAL:Function_call_interrupts 257272 ± 4% +9.4% 281541 ± 2% interrupts.CPU79.CAL:Function_call_interrupts 267352 ± 5% +15.1% 307824 ± 6% interrupts.CPU8.CAL:Function_call_interrupts 541165 ± 19% +20.5% 652070 ± 17% interrupts.CPU8.LOC:Local_timer_interrupts 429547 ± 5% +12.9% 484875 ± 6% interrupts.CPU8.TLB:TLB_shootdowns 253102 ± 4% +11.5% 282213 ± 2% interrupts.CPU80.CAL:Function_call_interrupts 246088 ± 5% +14.0% 280557 ± 4% interrupts.CPU81.CAL:Function_call_interrupts 541391 ± 19% +23.6% 669374 ± 15% interrupts.CPU81.LOC:Local_timer_interrupts 105.25 ± 28% +121.6% 233.25 ± 20% interrupts.CPU81.RES:Rescheduling_interrupts 241043 ± 2% +16.2% 280149 ± 6% interrupts.CPU82.CAL:Function_call_interrupts 543329 ± 19% +22.2% 664212 ± 16% interrupts.CPU82.LOC:Local_timer_interrupts 376457 ± 2% +13.1% 425712 ± 8% interrupts.CPU82.TLB:TLB_shootdowns 543327 ± 19% +21.9% 662439 ± 16% interrupts.CPU83.LOC:Local_timer_interrupts 7152 -18.5% 5831 ± 21% interrupts.CPU83.NMI:Non-maskable_interrupts 7152 -18.5% 5831 ± 21% interrupts.CPU83.PMI:Performance_monitoring_interrupts 541628 ± 19% +22.8% 665108 ± 16% interrupts.CPU84.LOC:Local_timer_interrupts 176.00 ± 59% +75.3% 308.50 ± 24% interrupts.CPU84.RES:Rescheduling_interrupts 254687 ± 3% +7.9% 274808 ± 5% interrupts.CPU85.CAL:Function_call_interrupts 543512 ± 19% +22.8% 667177 ± 15% interrupts.CPU85.LOC:Local_timer_interrupts 7104 -28.7% 5068 ± 32% interrupts.CPU85.NMI:Non-maskable_interrupts 7104 -28.7% 5068 ± 32% interrupts.CPU85.PMI:Performance_monitoring_interrupts 100.25 ± 39% +1384.0% 1487 ±155% interrupts.CPU85.RES:Rescheduling_interrupts 548792 ± 18% +23.8% 679539 ± 14% interrupts.CPU86.LOC:Local_timer_interrupts 175.00 ± 42% +74.7% 305.75 ± 18% interrupts.CPU86.RES:Rescheduling_interrupts 226308 ± 4% +20.5% 272796 ± 4% interrupts.CPU87.CAL:Function_call_interrupts 547617 ± 18% +21.8% 666821 ± 15% interrupts.CPU87.LOC:Local_timer_interrupts 350978 ± 5% +17.9% 413700 ± 5% interrupts.CPU87.TLB:TLB_shootdowns 226245 ± 7% +24.0% 280568 ± 3% interrupts.CPU88.CAL:Function_call_interrupts 347711 ± 8% +23.7% 430000 ± 4% interrupts.CPU88.TLB:TLB_shootdowns 245912 ± 3% +15.1% 283130 ± 4% interrupts.CPU89.CAL:Function_call_interrupts 542643 ± 19% +22.7% 665681 ± 15% interrupts.CPU89.LOC:Local_timer_interrupts 383169 ± 5% +12.1% 429524 ± 5% interrupts.CPU89.TLB:TLB_shootdowns 267597 +12.6% 301417 ± 2% interrupts.CPU9.CAL:Function_call_interrupts 543850 ± 19% +23.5% 671584 ± 15% interrupts.CPU90.LOC:Local_timer_interrupts 541042 ± 19% +23.1% 665788 ± 17% interrupts.CPU91.LOC:Local_timer_interrupts 234911 ± 3% +17.9% 276858 ± 2% interrupts.CPU92.CAL:Function_call_interrupts 358828 ± 5% +17.4% 421226 ± 2% interrupts.CPU92.TLB:TLB_shootdowns 239072 ± 4% +17.7% 281340 ± 5% interrupts.CPU93.CAL:Function_call_interrupts 543322 ± 19% +21.9% 662329 ± 16% interrupts.CPU93.LOC:Local_timer_interrupts 367623 ± 7% +16.8% 429249 ± 5% interrupts.CPU93.TLB:TLB_shootdowns 247885 ± 3% +13.5% 281430 ± 5% interrupts.CPU94.CAL:Function_call_interrupts 541125 ± 19% +22.7% 663728 ± 16% interrupts.CPU94.LOC:Local_timer_interrupts 246971 ± 2% +11.4% 275061 ± 5% interrupts.CPU95.CAL:Function_call_interrupts 545672 ± 19% +21.9% 665018 ± 15% interrupts.CPU95.LOC:Local_timer_interrupts 6254 +18.0% 7381 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 545654 ± 21% +18.5% 646628 ± 18% interrupts.CPU96.LOC:Local_timer_interrupts 7257 -29.6% 5106 ± 33% interrupts.CPU96.NMI:Non-maskable_interrupts 7257 -29.6% 5106 ± 33% interrupts.CPU96.PMI:Performance_monitoring_interrupts 263983 ± 2% +10.3% 291249 ± 4% interrupts.CPU97.CAL:Function_call_interrupts 539109 ± 20% +20.9% 651896 ± 17% interrupts.CPU97.LOC:Local_timer_interrupts 538437 ± 19% +21.1% 652180 ± 18% interrupts.CPU98.LOC:Local_timer_interrupts 83.75 ± 41% +341.2% 369.50 ± 47% interrupts.CPU98.RES:Rescheduling_interrupts 1.042e+08 ± 19% +20.4% 1.254e+08 ± 17% interrupts.LOC:Local_timer_interrupts *************************************************************************************************** lkp-bdw-ep2: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= blocksize/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/testcase: 128K/gcc-7/performance/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-bdw-ep2/pigz commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- %stddev %change %stddev \ | \ 8.338e+08 +2.7% 8.565e+08 pigz.throughput 511250 -29.6% 359913 pigz.time.involuntary_context_switches 8206 -2.4% 8008 pigz.time.percent_of_cpu_this_job_got 297.62 +22.7% 365.04 pigz.time.system_time 24338 -2.7% 23675 pigz.time.user_time 13485631 +15.2% 15539672 pigz.time.voluntary_context_switches 7.14 +2.4 9.52 ± 2% mpstat.cpu.idle% 1.13 +0.3 1.38 mpstat.cpu.sys% 271152 ± 6% -40.6% 161072 ± 14% meminfo.DirectMap4k 2033284 +301.6% 8165301 meminfo.Memused 7106 +351.4% 32079 meminfo.max_used_kB 7.00 +32.1% 9.25 ± 4% vmstat.cpu.id 91.00 -3.0% 88.25 vmstat.cpu.us 88844 +15.5% 102616 vmstat.system.cs 207592 -12.7% 181230 ± 21% vmstat.system.in 2938584 -13.3% 2547452 ± 8% cpuidle.C1.usage 6.07e+08 ± 2% +18.3% 7.18e+08 ± 5% cpuidle.C1E.time 9.065e+08 ± 4% +32.3% 1.199e+09 ± 11% cpuidle.C3.time 5286688 ± 3% +35.4% 7155915 ± 6% cpuidle.C3.usage 767840 +49.5% 1147947 ± 32% cpuidle.POLL.time 140873 +51.7% 213646 ± 17% cpuidle.POLL.usage 2448 +1.9% 2496 turbostat.Avg_MHz 2935997 -13.4% 2543835 ± 8% turbostat.C1 2.29 ± 2% +0.4 2.70 ± 5% turbostat.C1E% 5286391 ± 3% +35.4% 7155579 ± 6% turbostat.C3 3.42 ± 4% +1.1 4.51 ± 11% turbostat.C3% 6.57 +29.6% 8.52 turbostat.CPU%c1 278.91 +2.4% 285.70 turbostat.PkgWatt 14.00 +13.9% 15.96 turbostat.RAMWatt 33446 ± 9% +49.8% 50095 ± 10% numa-vmstat.node0.nr_active_anon 33051 ± 9% +34.4% 44426 ± 15% numa-vmstat.node0.nr_anon_pages 2738 ± 78% +260.7% 9878 ± 31% numa-vmstat.node0.nr_shmem 8095 ± 17% +28.0% 10362 ± 5% numa-vmstat.node0.nr_slab_reclaimable 33446 ± 9% +49.8% 50095 ± 10% numa-vmstat.node0.nr_zone_active_anon 17840 ± 2% -73.9% 4651 ±157% numa-vmstat.node0.numa_other 61920 ± 4% -23.7% 47223 ± 11% numa-vmstat.node1.nr_active_anon 9462 ± 18% -71.4% 2708 ±124% numa-vmstat.node1.nr_shmem 9691 ± 14% -22.0% 7555 ± 6% numa-vmstat.node1.nr_slab_reclaimable 61920 ± 4% -23.7% 47223 ± 11% numa-vmstat.node1.nr_zone_active_anon 95384 +2.0% 97274 proc-vmstat.nr_active_anon 87818 +1.7% 89341 proc-vmstat.nr_anon_pages 97.25 +4.6% 101.75 proc-vmstat.nr_anon_transparent_hugepages 3226236 -4.7% 3073086 proc-vmstat.nr_dirty_background_threshold 6460361 -4.7% 6153687 proc-vmstat.nr_dirty_threshold 32464338 -4.7% 30930590 proc-vmstat.nr_free_pages 1265 +1.1% 1280 proc-vmstat.nr_page_table_pages 95384 +2.0% 97274 proc-vmstat.nr_zone_active_anon 81185303 +2.9% 83579505 proc-vmstat.numa_hit 81168110 +2.9% 83562307 proc-vmstat.numa_local 81422001 +2.9% 83803349 proc-vmstat.pgalloc_normal 81372148 +2.9% 83764722 proc-vmstat.pgfree 133878 ± 9% +49.7% 200392 ± 10% numa-meminfo.node0.Active 133752 ± 9% +49.8% 200350 ± 10% numa-meminfo.node0.Active(anon) 132167 ± 9% +34.4% 177638 ± 15% numa-meminfo.node0.AnonPages 32384 ± 17% +28.0% 41451 ± 5% numa-meminfo.node0.KReclaimable 979174 ± 8% +325.1% 4162644 ± 2% numa-meminfo.node0.MemUsed 32384 ± 17% +28.0% 41451 ± 5% numa-meminfo.node0.SReclaimable 10956 ± 78% +260.7% 39521 ± 31% numa-meminfo.node0.Shmem 247763 ± 4% -23.8% 188902 ± 11% numa-meminfo.node1.Active 247721 ± 4% -23.8% 188776 ± 11% numa-meminfo.node1.Active(anon) 38767 ± 14% -22.0% 30225 ± 6% numa-meminfo.node1.KReclaimable 1053870 ± 7% +280.2% 4006503 numa-meminfo.node1.MemUsed 38767 ± 14% -22.0% 30225 ± 6% numa-meminfo.node1.SReclaimable 37816 ± 18% -71.3% 10852 ±124% numa-meminfo.node1.Shmem 13.33 ± 22% -39.7% 8.04 ± 38% sched_debug.cfs_rq:/.nr_spread_over.max 1.66 ± 22% -29.8% 1.16 ± 24% sched_debug.cfs_rq:/.nr_spread_over.stddev 5.61 ± 16% +41.1% 7.92 ± 40% sched_debug.cpu.cpu_load[1].stddev 157462 +13.2% 178259 sched_debug.cpu.nr_switches.avg 175376 +12.5% 197230 sched_debug.cpu.nr_switches.max 142466 +15.2% 164056 sched_debug.cpu.nr_switches.min 0.00 +175.0% 0.01 ± 30% sched_debug.cpu.nr_uninterruptible.avg 157579 +13.2% 178394 sched_debug.cpu.sched_count.avg 202031 ± 5% +13.6% 229597 ± 2% sched_debug.cpu.sched_count.max 142206 +15.1% 163725 sched_debug.cpu.sched_count.min 74241 +15.4% 85703 sched_debug.cpu.sched_goidle.avg 81416 +14.8% 93450 sched_debug.cpu.sched_goidle.max 67754 +17.4% 79515 sched_debug.cpu.sched_goidle.min 79375 +12.7% 89493 sched_debug.cpu.ttwu_count.avg 69638 +15.5% 80454 sched_debug.cpu.ttwu_count.min 3921 -28.8% 2792 sched_debug.cpu.ttwu_local.avg 6038 ± 4% -27.7% 4364 ± 6% sched_debug.cpu.ttwu_local.max 2171 ± 5% -26.1% 1604 ± 7% sched_debug.cpu.ttwu_local.min 830.88 ± 12% -30.6% 576.92 ± 10% sched_debug.cpu.ttwu_local.stddev 5.43 ± 26% -5.4 0.00 perf-profile.calltrace.cycles-pp.thermal_interrupt 5.38 ± 26% -5.4 0.00 perf-profile.calltrace.cycles-pp.smp_thermal_interrupt.thermal_interrupt 5.35 ± 26% -5.4 0.00 perf-profile.calltrace.cycles-pp.intel_thermal_interrupt.smp_thermal_interrupt.thermal_interrupt 5.02 ± 26% -5.0 0.00 perf-profile.calltrace.cycles-pp.pkg_thermal_notify.intel_thermal_interrupt.smp_thermal_interrupt.thermal_interrupt 4.83 ± 26% -4.8 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pkg_thermal_notify.intel_thermal_interrupt.smp_thermal_interrupt.thermal_interrupt 4.67 ± 26% -4.7 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pkg_thermal_notify.intel_thermal_interrupt.smp_thermal_interrupt 8.13 ± 3% -0.9 7.18 ± 7% perf-profile.calltrace.cycles-pp.apic_timer_interrupt 7.81 ± 3% -0.9 6.95 ± 7% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt 6.71 ± 3% -0.6 6.10 ± 6% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.99 ± 6% -0.1 0.88 ± 2% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pipe_write.__vfs_write.vfs_write.ksys_write 0.62 ± 8% -0.1 0.52 perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.pipe_write.__vfs_write.vfs_write 1.16 ± 3% +0.1 1.24 ± 2% perf-profile.calltrace.cycles-pp.wake_up_q.futex_wake.do_futex.__x64_sys_futex.do_syscall_64 0.61 ± 4% +0.1 0.69 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.wake_up_q.futex_wake.do_futex 0.60 ± 4% +0.1 0.68 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.wake_up_q.futex_wake 1.14 ± 2% +0.1 1.23 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_up_q.futex_wake.do_futex.__x64_sys_futex 0.56 ± 5% +0.1 0.65 ± 5% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.wake_up_q 1.49 ± 3% +0.1 1.60 ± 2% perf-profile.calltrace.cycles-pp.futex_wake.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.71 ± 5% +0.2 0.94 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle 2.81 ± 2% +0.9 3.74 ± 4% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.__vfs_write.vfs_write.ksys_write 4.81 ± 2% +0.9 5.76 ± 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.80 ± 2% +0.9 5.75 ± 3% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.76 ± 2% +1.0 5.72 ± 3% perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.75 ± 2% +1.0 5.71 ± 3% perf-profile.calltrace.cycles-pp.pipe_write.__vfs_write.vfs_write.ksys_write.do_syscall_64 2.54 +1.0 3.50 ± 5% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.__vfs_write.vfs_write 2.50 +1.0 3.48 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.__vfs_write 9.04 ± 2% +1.3 10.36 ± 2% perf-profile.calltrace.cycles-pp.pipe_read.__vfs_read.vfs_read.ksys_read.do_syscall_64 9.06 +1.3 10.39 ± 2% perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.13 +1.3 10.46 ± 2% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.12 +1.3 10.46 ± 2% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.48 ± 6% +1.7 4.20 ± 4% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.pipe_read.__vfs_read.vfs_read 2.50 ± 6% +1.7 4.22 ± 3% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_read.__vfs_read.vfs_read.ksys_read 17.46 ± 2% +2.3 19.77 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 17.43 ± 2% +2.3 19.74 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 62.28 ± 2% +4.1 66.37 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 65.89 ± 2% +4.1 70.03 perf-profile.calltrace.cycles-pp.secondary_startup_64 65.16 ± 2% +4.2 69.33 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 65.16 ± 2% +4.2 69.33 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 65.14 ± 2% +4.2 69.31 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 61.28 ± 3% +4.2 65.48 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 6.83 -3.2% 6.62 perf-stat.i.MPKI 2.954e+10 +2.9% 3.039e+10 perf-stat.i.branch-instructions 1.829e+09 +2.6% 1.877e+09 perf-stat.i.branch-misses 1.22 +0.4 1.62 perf-stat.i.cache-miss-rate% 18462323 +32.8% 24524865 perf-stat.i.cache-misses 89511 +15.7% 103532 perf-stat.i.context-switches 2.146e+11 +2.2% 2.192e+11 perf-stat.i.cpu-cycles 1365 -2.6% 1330 perf-stat.i.cpu-migrations 11662 -23.2% 8955 perf-stat.i.cycles-between-cache-misses 0.00 ± 3% -0.0 0.00 ± 3% perf-stat.i.dTLB-load-miss-rate% 7.286e+10 +2.8% 7.487e+10 perf-stat.i.dTLB-loads 3.278e+10 +2.7% 3.367e+10 perf-stat.i.dTLB-stores 39.80 ± 6% -5.0 34.77 ± 3% perf-stat.i.iTLB-load-miss-rate% 1299874 +17.2% 1523701 ± 3% perf-stat.i.iTLB-loads 2.227e+11 +2.8% 2.288e+11 perf-stat.i.instructions 95.28 -6.1 89.16 perf-stat.i.node-load-miss-rate% 6148481 +4.1% 6401043 perf-stat.i.node-load-misses 305439 ± 2% +156.4% 783244 ± 2% perf-stat.i.node-loads 87.33 -24.3 63.07 perf-stat.i.node-store-miss-rate% 5795094 +8.0% 6260805 perf-stat.i.node-store-misses 837584 ± 2% +337.5% 3664117 perf-stat.i.node-stores 6.83 -3.1% 6.62 perf-stat.overall.MPKI 1.21 +0.4 1.62 perf-stat.overall.cache-miss-rate% 11621 -23.1% 8936 perf-stat.overall.cycles-between-cache-misses 0.00 ± 3% -0.0 0.00 ± 3% perf-stat.overall.dTLB-load-miss-rate% 40.54 ± 7% -5.2 35.33 ± 4% perf-stat.overall.iTLB-load-miss-rate% 95.27 -6.2 89.10 perf-stat.overall.node-load-miss-rate% 87.37 -24.3 63.08 perf-stat.overall.node-store-miss-rate% 2.942e+10 +2.9% 3.027e+10 perf-stat.ps.branch-instructions 1.821e+09 +2.7% 1.87e+09 perf-stat.ps.branch-misses 18386302 +32.9% 24430572 perf-stat.ps.cache-misses 89154 +15.7% 103135 perf-stat.ps.context-switches 2.137e+11 +2.2% 2.183e+11 perf-stat.ps.cpu-cycles 1360 -2.6% 1325 perf-stat.ps.cpu-migrations 7.256e+10 +2.8% 7.458e+10 perf-stat.ps.dTLB-loads 3.265e+10 +2.7% 3.354e+10 perf-stat.ps.dTLB-stores 1294832 +17.2% 1518095 ± 3% perf-stat.ps.iTLB-loads 2.217e+11 +2.8% 2.28e+11 perf-stat.ps.instructions 6123665 +4.1% 6376385 perf-stat.ps.node-load-misses 304224 ± 2% +156.5% 780242 ± 2% perf-stat.ps.node-loads 5771771 +8.1% 6236602 perf-stat.ps.node-store-misses 834241 ± 2% +337.5% 3650057 perf-stat.ps.node-stores 6.669e+13 +2.9% 6.865e+13 perf-stat.total.instructions 249.75 ± 47% -35.5% 161.00 ± 5% interrupts.38:IR-PCI-MSI.1572868-edge.eth0-TxRx-4 431.50 +18.4% 510.75 interrupts.9:IR-IO-APIC.9-fasteoi.acpi 61799 ± 3% -59.0% 25343 ± 44% interrupts.CPU0.TRM:Thermal_event_interrupts 431.50 +18.4% 510.75 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU1.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU10.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU11.TRM:Thermal_event_interrupts 4004 ± 5% -13.8% 3451 ± 9% interrupts.CPU12.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU12.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU13.TRM:Thermal_event_interrupts 4255 ± 8% -13.1% 3698 ± 7% interrupts.CPU14.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU14.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU15.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU16.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU17.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU18.TRM:Thermal_event_interrupts 3830 ± 7% -11.8% 3377 ± 9% interrupts.CPU19.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU19.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU2.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU20.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU21.TRM:Thermal_event_interrupts 3710 ± 7% +18.1% 4384 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 4254 ± 31% +75.7% 7475 interrupts.CPU3.NMI:Non-maskable_interrupts 4254 ± 31% +75.7% 7475 interrupts.CPU3.PMI:Performance_monitoring_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU3.TRM:Thermal_event_interrupts 3166 ± 6% +17.1% 3709 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts 3300 ± 4% +16.8% 3854 ± 6% interrupts.CPU39.RES:Rescheduling_interrupts 249.75 ± 47% -35.5% 161.00 ± 5% interrupts.CPU4.38:IR-PCI-MSI.1572868-edge.eth0-TxRx-4 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU4.TRM:Thermal_event_interrupts 3130 ± 6% +18.0% 3692 ± 12% interrupts.CPU41.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU44.TRM:Thermal_event_interrupts 3680 ± 5% -14.1% 3160 ± 9% interrupts.CPU45.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU45.TRM:Thermal_event_interrupts 3698 ± 3% -10.1% 3324 ± 5% interrupts.CPU46.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU46.TRM:Thermal_event_interrupts 5954 ± 22% +26.0% 7500 interrupts.CPU47.NMI:Non-maskable_interrupts 5954 ± 22% +26.0% 7500 interrupts.CPU47.PMI:Performance_monitoring_interrupts 61805 ± 3% -59.0% 25343 ± 44% interrupts.CPU47.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU48.TRM:Thermal_event_interrupts 61805 ± 3% -59.0% 25343 ± 44% interrupts.CPU49.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU5.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU50.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU51.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU52.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU53.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU54.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU55.TRM:Thermal_event_interrupts 3767 ± 6% -14.1% 3236 ± 8% interrupts.CPU56.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU56.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU57.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU58.TRM:Thermal_event_interrupts 3798 ± 8% -15.0% 3229 ± 3% interrupts.CPU59.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU59.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU6.TRM:Thermal_event_interrupts 61803 ± 3% -59.0% 25343 ± 44% interrupts.CPU60.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU61.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU62.TRM:Thermal_event_interrupts 3338 ± 5% -12.1% 2934 ± 6% interrupts.CPU63.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU63.TRM:Thermal_event_interrupts 61805 ± 3% -59.0% 25343 ± 44% interrupts.CPU64.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU65.TRM:Thermal_event_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU7.TRM:Thermal_event_interrupts 3179 ± 9% +13.2% 3600 ± 6% interrupts.CPU77.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU8.TRM:Thermal_event_interrupts 3081 ± 7% +12.4% 3465 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts 61806 ± 3% -59.0% 25343 ± 44% interrupts.CPU9.TRM:Thermal_event_interrupts 2719472 ± 3% -59.0% 1115092 ± 44% interrupts.TRM:Thermal_event_interrupts 25605 ± 4% -15.6% 21604 ± 4% softirqs.CPU0.RCU 27101 ± 10% -17.2% 22448 ± 3% softirqs.CPU1.RCU 11215 ± 2% +20.7% 13532 ± 6% softirqs.CPU1.SCHED 24097 ± 3% -12.0% 21206 ± 3% softirqs.CPU10.RCU 11027 +11.9% 12337 softirqs.CPU10.SCHED 24603 ± 3% -14.0% 21156 softirqs.CPU11.RCU 10798 +14.3% 12343 ± 2% softirqs.CPU11.SCHED 24676 ± 7% -15.5% 20842 ± 3% softirqs.CPU12.RCU 10896 +13.1% 12324 softirqs.CPU12.SCHED 24701 ± 5% -14.0% 21250 ± 3% softirqs.CPU13.RCU 11001 +12.2% 12342 softirqs.CPU13.SCHED 26286 ± 3% -13.6% 22719 ± 7% softirqs.CPU14.RCU 10876 +13.0% 12285 softirqs.CPU14.SCHED 27217 ± 4% -18.3% 22246 ± 6% softirqs.CPU15.RCU 26699 ± 4% -17.0% 22160 ± 6% softirqs.CPU16.RCU 10812 +12.8% 12197 softirqs.CPU16.SCHED 24655 ± 6% -13.2% 21400 ± 6% softirqs.CPU17.RCU 11058 ± 2% +12.2% 12404 ± 2% softirqs.CPU17.SCHED 26454 ± 3% -17.8% 21733 ± 4% softirqs.CPU18.RCU 10909 ± 2% +16.5% 12708 ± 4% softirqs.CPU18.SCHED 10855 +16.1% 12599 ± 5% softirqs.CPU19.SCHED 25053 ± 3% -14.4% 21437 ± 4% softirqs.CPU2.RCU 10658 ± 4% +12.1% 11951 ± 2% softirqs.CPU2.SCHED 27016 ± 5% -15.5% 22818 ± 4% softirqs.CPU20.RCU 11107 ± 6% +10.2% 12235 softirqs.CPU20.SCHED 27073 ± 5% -16.7% 22552 ± 3% softirqs.CPU21.RCU 10792 +15.3% 12439 ± 2% softirqs.CPU21.SCHED 26657 ± 4% -14.8% 22702 ± 5% softirqs.CPU22.RCU 10935 +12.5% 12298 softirqs.CPU22.SCHED 27378 ± 2% -17.6% 22547 ± 3% softirqs.CPU23.RCU 10818 +13.6% 12292 softirqs.CPU23.SCHED 26527 ± 3% -15.0% 22560 ± 3% softirqs.CPU24.RCU 10809 +13.4% 12255 ± 2% softirqs.CPU24.SCHED 25743 ± 3% -15.2% 21828 ± 4% softirqs.CPU25.RCU 10818 +14.5% 12387 softirqs.CPU25.SCHED 25864 ± 5% -14.5% 22105 ± 6% softirqs.CPU26.RCU 10861 +13.6% 12333 softirqs.CPU26.SCHED 25869 ± 4% -15.4% 21896 ± 4% softirqs.CPU27.RCU 10841 +14.8% 12447 softirqs.CPU27.SCHED 26220 ± 4% -17.0% 21754 ± 9% softirqs.CPU28.RCU 10764 +13.9% 12264 ± 2% softirqs.CPU28.SCHED 27116 ± 5% -14.6% 23157 ± 11% softirqs.CPU29.RCU 10613 +17.3% 12451 ± 3% softirqs.CPU29.SCHED 25778 ± 4% -15.4% 21818 ± 7% softirqs.CPU3.RCU 10791 +14.9% 12398 ± 2% softirqs.CPU3.SCHED 26863 ± 5% -12.0% 23649 ± 4% softirqs.CPU30.RCU 10707 +14.8% 12290 softirqs.CPU30.SCHED 27211 ± 4% -16.8% 22644 ± 4% softirqs.CPU31.RCU 10698 +16.2% 12437 softirqs.CPU31.SCHED 26191 ± 4% -13.4% 22693 ± 7% softirqs.CPU32.RCU 10718 +15.0% 12325 ± 3% softirqs.CPU32.SCHED 26093 ± 6% -13.1% 22663 ± 4% softirqs.CPU33.RCU 10744 +16.9% 12558 ± 2% softirqs.CPU33.SCHED 26391 ± 7% -16.7% 21987 ± 3% softirqs.CPU34.RCU 10861 +14.6% 12443 softirqs.CPU34.SCHED 26840 ± 8% -17.7% 22099 ± 4% softirqs.CPU35.RCU 10890 ± 2% +12.9% 12294 softirqs.CPU35.SCHED 26889 ± 8% -16.9% 22348 ± 4% softirqs.CPU36.RCU 10870 +13.5% 12337 ± 2% softirqs.CPU36.SCHED 26678 ± 3% -13.5% 23075 ± 6% softirqs.CPU37.RCU 10682 +14.7% 12252 softirqs.CPU37.SCHED 27102 ± 6% -17.7% 22313 ± 4% softirqs.CPU38.RCU 10761 +14.2% 12294 softirqs.CPU38.SCHED 10756 +15.2% 12387 softirqs.CPU39.SCHED 24415 ± 3% -13.4% 21140 ± 5% softirqs.CPU4.RCU 10964 +13.2% 12406 softirqs.CPU4.SCHED 26427 ± 4% -19.4% 21297 ± 9% softirqs.CPU40.RCU 10722 +15.6% 12394 softirqs.CPU40.SCHED 26377 ± 5% -17.6% 21741 ± 9% softirqs.CPU41.RCU 10695 ± 2% +16.5% 12461 softirqs.CPU41.SCHED 27384 ± 3% -19.3% 22095 ± 8% softirqs.CPU42.RCU 10792 +15.2% 12436 softirqs.CPU42.SCHED 27114 ± 3% -17.3% 22424 ± 3% softirqs.CPU43.RCU 10750 +14.3% 12292 ± 2% softirqs.CPU43.SCHED 21179 ± 7% -14.8% 18053 ± 5% softirqs.CPU44.RCU 10701 +14.1% 12209 softirqs.CPU44.SCHED 25747 ± 4% -16.1% 21612 ± 4% softirqs.CPU45.RCU 10797 +13.5% 12255 softirqs.CPU45.SCHED 10811 +15.2% 12449 softirqs.CPU46.SCHED 26297 ± 4% -20.1% 21013 ± 4% softirqs.CPU47.RCU 10645 ± 2% +15.7% 12314 softirqs.CPU47.SCHED 24961 ± 5% -12.9% 21741 ± 8% softirqs.CPU48.RCU 10720 ± 2% +14.4% 12269 softirqs.CPU48.SCHED 24458 ± 4% -14.6% 20887 ± 4% softirqs.CPU49.RCU 10897 ± 2% +12.0% 12202 softirqs.CPU49.SCHED 24220 ± 3% -13.1% 21044 ± 3% softirqs.CPU5.RCU 11006 +13.9% 12534 softirqs.CPU5.SCHED 25205 ± 4% -14.7% 21503 ± 4% softirqs.CPU50.RCU 10789 +12.9% 12178 softirqs.CPU50.SCHED 25244 ± 4% -14.4% 21614 ± 3% softirqs.CPU51.RCU 10814 ± 2% +12.0% 12108 ± 2% softirqs.CPU51.SCHED 24276 ± 4% -13.8% 20930 ± 4% softirqs.CPU52.RCU 10744 +14.7% 12319 softirqs.CPU52.SCHED 25735 ± 4% -17.0% 21352 ± 5% softirqs.CPU53.RCU 10855 ± 3% +12.6% 12218 ± 2% softirqs.CPU53.SCHED 10752 +14.8% 12342 ± 2% softirqs.CPU54.SCHED 24636 ± 5% -14.6% 21039 ± 3% softirqs.CPU55.RCU 11088 ± 6% +11.3% 12346 softirqs.CPU55.SCHED 24093 ± 4% -13.0% 20967 ± 2% softirqs.CPU56.RCU 10756 +14.5% 12314 softirqs.CPU56.SCHED 24444 ± 2% -11.1% 21721 ± 4% softirqs.CPU57.RCU 10753 +15.1% 12373 softirqs.CPU57.SCHED 27007 ± 3% -17.4% 22316 softirqs.CPU58.RCU 10644 +16.3% 12379 ± 2% softirqs.CPU58.SCHED 27563 ± 3% -16.6% 22983 ± 5% softirqs.CPU59.RCU 10795 +14.5% 12355 softirqs.CPU59.SCHED 24976 ± 3% -12.8% 21774 ± 4% softirqs.CPU6.RCU 10900 ± 2% +13.9% 12412 softirqs.CPU6.SCHED 26143 ± 4% -15.9% 21987 ± 7% softirqs.CPU60.RCU 10834 +13.1% 12258 ± 2% softirqs.CPU60.SCHED 10775 ± 2% +15.9% 12484 ± 2% softirqs.CPU61.SCHED 10797 ± 2% +14.9% 12410 softirqs.CPU62.SCHED 24050 ± 6% -11.9% 21187 ± 9% softirqs.CPU63.RCU 10707 +14.4% 12245 softirqs.CPU63.SCHED 26283 ± 4% -15.4% 22233 ± 4% softirqs.CPU64.RCU 10741 +14.0% 12245 softirqs.CPU64.SCHED 26944 ± 6% -17.8% 22139 ± 4% softirqs.CPU65.RCU 10623 +14.7% 12187 ± 3% softirqs.CPU65.SCHED 88513 ± 3% +38.1% 122221 ± 27% softirqs.CPU65.TIMER 26090 ± 4% -14.8% 22227 ± 6% softirqs.CPU66.RCU 10750 +15.3% 12391 softirqs.CPU66.SCHED 27171 ± 6% -19.0% 22012 ± 5% softirqs.CPU67.RCU 10724 +14.2% 12251 softirqs.CPU67.SCHED 26829 ± 4% -16.7% 22351 ± 6% softirqs.CPU68.RCU 10802 +13.9% 12306 ± 3% softirqs.CPU68.SCHED 26521 -16.5% 22150 ± 5% softirqs.CPU69.RCU 10682 +15.8% 12374 softirqs.CPU69.SCHED 25097 ± 4% -11.0% 22341 ± 5% softirqs.CPU7.RCU 10868 +12.7% 12247 softirqs.CPU7.SCHED 25889 ± 6% -16.0% 21744 ± 7% softirqs.CPU70.RCU 10952 ± 2% +13.2% 12401 softirqs.CPU70.SCHED 25508 ± 4% -16.4% 21327 ± 5% softirqs.CPU71.RCU 10676 +15.5% 12334 softirqs.CPU71.SCHED 26629 ± 5% -17.7% 21924 ± 5% softirqs.CPU72.RCU 10806 ± 2% +12.6% 12164 softirqs.CPU72.SCHED 26358 ± 4% -16.2% 22086 ± 7% softirqs.CPU73.RCU 10582 +16.0% 12278 softirqs.CPU73.SCHED 26155 ± 3% -16.4% 21859 ± 7% softirqs.CPU74.RCU 10640 +15.8% 12323 softirqs.CPU74.SCHED 10651 +14.6% 12205 softirqs.CPU75.SCHED 22487 ± 5% -11.9% 19802 ± 4% softirqs.CPU76.RCU 10687 +15.2% 12316 softirqs.CPU76.SCHED 10720 +15.6% 12389 softirqs.CPU77.SCHED 10659 +14.9% 12244 softirqs.CPU78.SCHED 22183 ± 5% -10.9% 19762 ± 3% softirqs.CPU79.RCU 10615 +15.7% 12286 softirqs.CPU79.SCHED 23979 ± 2% -13.3% 20797 ± 3% softirqs.CPU8.RCU 11009 +12.9% 12425 softirqs.CPU8.SCHED 10789 ± 2% +12.5% 12136 softirqs.CPU80.SCHED 23137 ± 6% -11.8% 20411 ± 3% softirqs.CPU81.RCU 10676 +14.3% 12203 softirqs.CPU81.SCHED 11040 ± 4% +11.0% 12252 ± 2% softirqs.CPU82.SCHED 21799 ± 6% -10.7% 19467 ± 3% softirqs.CPU83.RCU 10825 +13.0% 12231 softirqs.CPU83.SCHED 22300 ± 6% -13.3% 19328 ± 5% softirqs.CPU84.RCU 10564 +17.3% 12395 softirqs.CPU84.SCHED 23316 ± 6% -14.2% 20015 ± 7% softirqs.CPU85.RCU 10699 +18.2% 12642 ± 4% softirqs.CPU85.SCHED 23049 ± 6% -10.9% 20537 ± 3% softirqs.CPU86.RCU 10667 +15.0% 12263 softirqs.CPU86.SCHED 10410 +17.5% 12233 softirqs.CPU87.SCHED 124354 ± 26% -28.5% 88963 softirqs.CPU87.TIMER 25208 ± 3% -15.2% 21371 ± 2% softirqs.CPU9.RCU 10819 ± 3% +15.5% 12498 ± 2% softirqs.CPU9.SCHED 2232449 ± 4% -14.5% 1909367 ± 4% softirqs.RCU 954191 +14.2% 1089550 softirqs.SCHED *************************************************************************************************** lkp-knm02: 272 threads Intel(R) Xeon Phi(TM) CPU 7255 @ 1.10GHz with 112G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime: pipe/gcc-7/performance/1HDD/x86_64-rhel-7.2/100%/debian-x86_64-2018-04-03.cgz/lkp-knm02/stress-ng/1s commit: 82342cc3fb ("net/mlx4: Change number of max MSIXs from 64 to 1024") 148c6673c9 ("mm/page_alloc.c: Set ppc->high fraction default to 512") 82342cc3fb1ccc8a 148c6673c935129351f35360172 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 208793 +17.2% 244666 stress-ng.tee.ops 207629 +17.3% 243548 stress-ng.tee.ops_per_sec 438523 +39.2% 610238 ± 4% stress-ng.time.involuntary_context_switches 114781 +15.3% 132388 stress-ng.time.minor_page_faults 18156 +9.4% 19871 stress-ng.time.percent_of_cpu_this_job_got 1585 +2.1% 1619 stress-ng.time.system_time 170.33 +5.1% 179.01 stress-ng.time.user_time 2309083 ± 3% +4.9% 2423124 ± 2% stress-ng.time.voluntary_context_switches 2623984 ± 19% -28.8% 1867986 ± 13% cpuidle.C1.usage 4435 ± 8% -21.1% 3499 ± 3% cpuidle.POLL.usage 49.84 ± 5% -12.1% 43.82 iostat.cpu.idle 45.20 ± 5% +12.0% 50.63 iostat.cpu.system 4.96 ± 5% +12.0% 5.55 iostat.cpu.user 719.25 ± 3% +7.6% 774.00 ± 2% turbostat.Avg_MHz 6015541 ± 6% -7.4% 5570264 ± 2% turbostat.IRQ 8.78 ± 5% +22.4% 10.75 turbostat.RAMWatt 45.75 ± 6% -6.9 38.84 mpstat.cpu.idle% 0.02 ± 48% -0.0 0.01 ± 27% mpstat.cpu.soft% 48.88 ± 5% +6.2 55.12 mpstat.cpu.sys% 5.35 ± 5% +0.7 6.03 mpstat.cpu.usr% 4634520 ± 20% -37.6% 2893787 ± 26% sched_debug.cpu.avg_idle.max 559563 ± 20% -37.3% 350974 ± 29% sched_debug.cpu.avg_idle.stddev 30792 ± 64% -53.6% 14290 ± 23% sched_debug.cpu.load.stddev 299577 ± 28% -45.3% 163789 ± 39% sched_debug.cpu.max_idle_balance_cost.stddev 50.25 ± 4% -10.9% 44.75 vmstat.cpu.id 44.00 ± 5% +11.9% 49.25 vmstat.cpu.sy 183.25 ± 6% +12.1% 205.50 ± 2% vmstat.procs.r 303530 ± 5% +11.3% 337723 ± 2% vmstat.system.cs 30652 +200.5% 92113 ± 38% numa-vmstat.node0.nr_active_anon 30884 +198.5% 92184 ± 37% numa-vmstat.node0.nr_anon_pages 16.75 ± 19% +661.2% 127.50 ± 48% numa-vmstat.node0.nr_anon_transparent_hugepages 5050 ± 7% +34.6% 6798 ± 10% numa-vmstat.node0.nr_page_table_pages 30659 +200.2% 92045 ± 38% numa-vmstat.node0.nr_zone_active_anon 122478 +195.4% 361856 ± 38% numa-meminfo.node0.Active 122394 +195.6% 361768 ± 38% numa-meminfo.node0.Active(anon) 34978 ± 20% +656.9% 264750 ± 48% numa-meminfo.node0.AnonHugePages 123291 +193.5% 361876 ± 37% numa-meminfo.node0.AnonPages 1728367 +152.9% 4370923 ± 3% numa-meminfo.node0.MemUsed 20306 ± 4% +31.6% 26726 ± 11% numa-meminfo.node0.PageTables 11510 ± 5% -9.7% 10391 ± 3% softirqs.CPU106.TIMER 11007 ± 4% -7.9% 10141 ± 2% softirqs.CPU126.TIMER 9591 ± 4% -9.3% 8700 softirqs.CPU259.TIMER 11169 ± 4% -6.9% 10395 ± 2% softirqs.CPU39.TIMER 11552 ± 3% -11.6% 10209 ± 5% softirqs.CPU6.TIMER 12087 ± 13% -12.7% 10551 ± 2% softirqs.CPU64.TIMER 11327 ± 8% -9.5% 10249 ± 2% softirqs.CPU93.TIMER 122711 +194.7% 361595 ± 38% meminfo.Active 122627 +194.8% 361507 ± 38% meminfo.Active(anon) 34606 ± 19% +668.1% 265821 ± 48% meminfo.AnonHugePages 123553 +192.9% 361825 ± 37% meminfo.AnonPages 1954800 ± 3% +16.7% 2282021 ± 3% meminfo.Committed_AS 6110720 ± 6% +16.1% 7092736 meminfo.DirectMap2M 337628 ± 8% -57.9% 142044 ± 6% meminfo.DirectMap4k 2266327 +116.9% 4915123 ± 3% meminfo.Memused 20845 ± 5% +28.3% 26741 ± 9% meminfo.PageTables 276349 ± 4% +164.8% 731749 ± 6% meminfo.max_used_kB 30599 +200.4% 91932 ± 38% proc-vmstat.nr_active_anon 30827 +198.4% 91982 ± 38% proc-vmstat.nr_anon_pages 16.50 ± 21% +687.9% 130.00 ± 49% proc-vmstat.nr_anon_transparent_hugepages 2805434 -2.3% 2740715 proc-vmstat.nr_dirty_background_threshold 5617729 -2.3% 5488162 proc-vmstat.nr_dirty_threshold 28273112 -2.3% 27625125 proc-vmstat.nr_free_pages 38822 +3.2% 40069 proc-vmstat.nr_kernel_stack 5130 ± 5% +34.7% 6908 ± 11% proc-vmstat.nr_page_table_pages 24901 -1.0% 24641 proc-vmstat.nr_slab_reclaimable 30599 +200.4% 91932 ± 38% proc-vmstat.nr_zone_active_anon 18755810 +2.4% 19199074 proc-vmstat.numa_hit 18755810 +2.4% 19199074 proc-vmstat.numa_local 9488 ± 20% +2028.9% 202001 ± 55% proc-vmstat.numa_pte_updates 18803764 +5.7% 19874483 ± 2% proc-vmstat.pgalloc_normal 156554 ± 2% +9.3% 171049 proc-vmstat.pgfault 18697117 +5.5% 19727099 ± 2% proc-vmstat.pgfree 4.91 ± 2% +0.4 5.29 perf-stat.i.branch-miss-rate% 4.654e+08 ± 5% +10.4% 5.136e+08 ± 5% perf-stat.i.branch-misses 1.05e+09 ± 4% +11.2% 1.168e+09 ± 2% perf-stat.i.cache-references 2.182e+11 +7.1% 2.336e+11 perf-stat.i.cpu-cycles 8240 ± 24% +206.8% 25278 ± 10% perf-stat.i.cpu-migrations 4.178e+08 ± 4% +19.4% 4.99e+08 ± 7% perf-stat.i.iTLB-load-misses 134.77 ± 7% -14.6% 115.13 ± 6% perf-stat.i.instructions-per-iTLB-miss 12150 ± 9% +28.1% 15566 ± 2% perf-stat.i.minor-faults 545666 ± 2% -3.8% 524874 perf-stat.i.msec 12266 ± 9% +28.2% 15721 ± 3% perf-stat.i.page-faults 4.77 +0.4 5.13 perf-stat.overall.branch-miss-rate% 0.94 ± 2% +0.1 1.04 perf-stat.overall.iTLB-load-miss-rate% 106.72 ± 2% -10.1% 95.93 perf-stat.overall.instructions-per-iTLB-miss 3.772e+08 ± 3% +7.8% 4.066e+08 ± 2% perf-stat.ps.branch-misses 8.776e+08 ± 2% +9.1% 9.577e+08 ± 5% perf-stat.ps.cache-references 1.8e+11 ± 2% +5.0% 1.891e+11 ± 4% perf-stat.ps.cpu-cycles 8273 ± 18% +174.4% 22700 ± 11% perf-stat.ps.cpu-migrations 3.479e+08 +14.2% 3.973e+08 ± 2% perf-stat.ps.iTLB-load-misses 10453 ± 7% +24.9% 13059 ± 4% perf-stat.ps.minor-faults 458464 -6.8% 427496 ± 4% perf-stat.ps.msec 10474 ± 7% +25.9% 13185 ± 4% perf-stat.ps.page-faults 2.816e+11 ± 3% -13.3% 2.441e+11 ± 6% perf-stat.total.instructions 17913 ± 8% -9.2% 16263 ± 3% interrupts.CPU100.LOC:Local_timer_interrupts 17891 ± 7% -9.6% 16175 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts 17865 ± 7% -10.1% 16062 ± 3% interrupts.CPU103.LOC:Local_timer_interrupts 17839 ± 8% -8.6% 16310 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts 17790 ± 8% -8.4% 16301 ± 3% interrupts.CPU109.LOC:Local_timer_interrupts 17863 ± 7% -9.7% 16134 ± 3% interrupts.CPU11.LOC:Local_timer_interrupts 17991 ± 7% -9.1% 16351 ± 3% interrupts.CPU112.LOC:Local_timer_interrupts 17936 ± 9% -9.2% 16290 ± 2% interrupts.CPU113.LOC:Local_timer_interrupts 17721 ± 7% -9.1% 16110 ± 3% interrupts.CPU114.LOC:Local_timer_interrupts 17871 ± 8% -9.4% 16184 ± 3% interrupts.CPU115.LOC:Local_timer_interrupts 17907 ± 5% -9.2% 16256 ± 3% interrupts.CPU118.LOC:Local_timer_interrupts 17554 ± 6% -7.3% 16274 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts 17859 ± 6% -9.1% 16236 ± 3% interrupts.CPU120.LOC:Local_timer_interrupts 711.50 ± 24% +30.1% 926.00 ± 13% interrupts.CPU120.RES:Rescheduling_interrupts 882.75 ± 37% +88.8% 1666 ± 31% interrupts.CPU124.RES:Rescheduling_interrupts 18033 ± 5% -9.8% 16260 ± 3% interrupts.CPU126.LOC:Local_timer_interrupts 693.50 ± 24% +43.2% 993.25 ± 16% interrupts.CPU126.RES:Rescheduling_interrupts 1267 ± 16% -48.9% 647.75 ± 15% interrupts.CPU128.RES:Rescheduling_interrupts 17574 ± 7% -7.5% 16253 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts 17571 ± 6% -8.5% 16082 ± 3% interrupts.CPU133.LOC:Local_timer_interrupts 17819 ± 7% -9.2% 16171 ± 3% interrupts.CPU134.LOC:Local_timer_interrupts 17777 ± 7% -9.4% 16098 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts 805.50 ± 22% +44.1% 1160 ± 20% interrupts.CPU135.RES:Rescheduling_interrupts 17562 ± 9% -9.0% 15980 ± 2% interrupts.CPU136.LOC:Local_timer_interrupts 17625 ± 6% -8.4% 16145 ± 3% interrupts.CPU137.LOC:Local_timer_interrupts 17790 ± 7% -9.4% 16119 ± 3% interrupts.CPU138.LOC:Local_timer_interrupts 17906 ± 7% -9.3% 16239 ± 4% interrupts.CPU14.LOC:Local_timer_interrupts 794.00 ± 16% +28.0% 1016 ± 17% interrupts.CPU141.RES:Rescheduling_interrupts 17709 ± 7% -9.7% 15991 ± 3% interrupts.CPU143.LOC:Local_timer_interrupts 17679 ± 7% -8.1% 16244 ± 3% interrupts.CPU146.LOC:Local_timer_interrupts 17609 ± 8% -9.3% 15970 ± 2% interrupts.CPU149.LOC:Local_timer_interrupts 17944 ± 6% -10.6% 16036 ± 3% interrupts.CPU150.LOC:Local_timer_interrupts 1272 ± 16% +26.3% 1606 ± 8% interrupts.CPU150.RES:Rescheduling_interrupts 17916 ± 7% -10.5% 16028 ± 4% interrupts.CPU151.LOC:Local_timer_interrupts 17767 ± 6% -9.2% 16131 ± 2% interrupts.CPU152.LOC:Local_timer_interrupts 17615 ± 8% -8.5% 16115 ± 2% interrupts.CPU153.LOC:Local_timer_interrupts 17903 ± 7% -9.5% 16208 ± 4% interrupts.CPU154.LOC:Local_timer_interrupts 17866 ± 7% -11.1% 15890 ± 2% interrupts.CPU157.LOC:Local_timer_interrupts 17907 ± 7% -9.4% 16231 ± 2% interrupts.CPU159.LOC:Local_timer_interrupts 17937 ± 7% -9.4% 16247 ± 2% interrupts.CPU160.LOC:Local_timer_interrupts 17683 ± 7% -8.7% 16137 ± 2% interrupts.CPU162.LOC:Local_timer_interrupts 17721 ± 6% -9.6% 16015 ± 2% interrupts.CPU163.LOC:Local_timer_interrupts 991.50 ± 25% +42.8% 1415 ± 18% interrupts.CPU163.RES:Rescheduling_interrupts 17763 ± 8% -8.3% 16284 ± 5% interrupts.CPU165.LOC:Local_timer_interrupts 18097 ± 7% -10.4% 16212 ± 3% interrupts.CPU166.LOC:Local_timer_interrupts 17885 ± 7% -9.8% 16129 ± 3% interrupts.CPU167.LOC:Local_timer_interrupts 17995 ± 6% -9.9% 16211 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts 17801 ± 7% -9.3% 16138 ± 3% interrupts.CPU171.LOC:Local_timer_interrupts 1557 ± 19% -33.9% 1030 ± 30% interrupts.CPU174.RES:Rescheduling_interrupts 17700 ± 7% -9.5% 16024 ± 2% interrupts.CPU175.LOC:Local_timer_interrupts 17975 ± 8% -9.4% 16277 interrupts.CPU176.LOC:Local_timer_interrupts 17773 ± 7% -8.8% 16217 ± 3% interrupts.CPU177.LOC:Local_timer_interrupts 17999 ± 8% -10.3% 16141 ± 4% interrupts.CPU183.LOC:Local_timer_interrupts 17831 ± 7% -8.2% 16377 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts 17920 ± 7% -9.1% 16297 ± 2% interrupts.CPU189.LOC:Local_timer_interrupts 17759 ± 6% -8.4% 16270 interrupts.CPU190.LOC:Local_timer_interrupts 17767 ± 7% -7.8% 16383 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts 17783 ± 6% -8.8% 16217 ± 3% interrupts.CPU192.LOC:Local_timer_interrupts 802.75 ± 16% +91.2% 1535 ± 23% interrupts.CPU192.RES:Rescheduling_interrupts 17719 ± 6% -8.6% 16201 ± 3% interrupts.CPU193.LOC:Local_timer_interrupts 17711 ± 6% -8.0% 16286 ± 2% interrupts.CPU195.LOC:Local_timer_interrupts 17872 ± 8% -10.7% 15968 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts 17761 ± 7% -9.3% 16106 ± 2% interrupts.CPU200.LOC:Local_timer_interrupts 17780 ± 7% -9.5% 16085 ± 3% interrupts.CPU201.LOC:Local_timer_interrupts 17724 ± 7% -9.6% 16019 ± 2% interrupts.CPU204.LOC:Local_timer_interrupts 17901 ± 7% -9.7% 16167 ± 3% interrupts.CPU205.LOC:Local_timer_interrupts 1390 ± 17% -36.6% 880.75 ± 38% interrupts.CPU208.RES:Rescheduling_interrupts 17750 ± 7% -9.9% 15999 ± 4% interrupts.CPU210.LOC:Local_timer_interrupts 963.75 ± 17% +68.5% 1624 ± 9% interrupts.CPU210.RES:Rescheduling_interrupts 17736 ± 6% -8.9% 16152 ± 3% interrupts.CPU211.LOC:Local_timer_interrupts 682.25 ± 18% +113.2% 1454 ± 3% interrupts.CPU212.RES:Rescheduling_interrupts 17707 ± 6% -8.3% 16232 ± 2% interrupts.CPU213.LOC:Local_timer_interrupts 17633 ± 7% -8.4% 16155 ± 3% interrupts.CPU214.LOC:Local_timer_interrupts 17652 ± 7% -9.1% 16046 ± 4% interrupts.CPU216.LOC:Local_timer_interrupts 17714 ± 7% -9.5% 16036 ± 3% interrupts.CPU217.LOC:Local_timer_interrupts 971.00 ± 9% +15.9% 1125 ± 7% interrupts.CPU217.RES:Rescheduling_interrupts 17764 ± 6% -9.5% 16078 ± 4% interrupts.CPU218.LOC:Local_timer_interrupts 17723 ± 6% -9.0% 16133 ± 4% interrupts.CPU219.LOC:Local_timer_interrupts 1069 ± 23% +26.5% 1353 ± 8% interrupts.CPU219.RES:Rescheduling_interrupts 17940 ± 7% -9.6% 16216 ± 3% interrupts.CPU22.LOC:Local_timer_interrupts 17588 ± 8% -7.2% 16329 ± 2% interrupts.CPU220.LOC:Local_timer_interrupts 17609 ± 7% -8.7% 16079 ± 3% interrupts.CPU223.LOC:Local_timer_interrupts 17692 ± 6% -9.2% 16062 ± 3% interrupts.CPU224.LOC:Local_timer_interrupts 17761 ± 7% -9.9% 16000 ± 3% interrupts.CPU225.LOC:Local_timer_interrupts 17880 ± 7% -9.5% 16174 ± 3% interrupts.CPU226.LOC:Local_timer_interrupts 17973 ± 7% -10.4% 16102 ± 3% interrupts.CPU227.LOC:Local_timer_interrupts 17869 ± 7% -9.4% 16194 ± 3% interrupts.CPU228.LOC:Local_timer_interrupts 17778 ± 7% -8.5% 16265 ± 3% interrupts.CPU229.LOC:Local_timer_interrupts 17945 ± 7% -9.0% 16334 ± 3% interrupts.CPU23.LOC:Local_timer_interrupts 820.75 ± 8% +43.8% 1180 ± 23% interrupts.CPU23.RES:Rescheduling_interrupts 17883 ± 7% -9.8% 16137 ± 3% interrupts.CPU230.LOC:Local_timer_interrupts 17720 ± 7% -10.0% 15955 ± 2% interrupts.CPU231.LOC:Local_timer_interrupts 606.25 ± 51% +85.5% 1124 ± 24% interrupts.CPU232.RES:Rescheduling_interrupts 17799 ± 8% -9.5% 16110 ± 3% interrupts.CPU233.LOC:Local_timer_interrupts 17747 ± 7% -9.1% 16130 ± 2% interrupts.CPU234.LOC:Local_timer_interrupts 17905 ± 7% -10.1% 16102 ± 2% interrupts.CPU235.LOC:Local_timer_interrupts 17829 ± 6% -10.0% 16053 ± 2% interrupts.CPU237.LOC:Local_timer_interrupts 761.00 ± 9% +69.2% 1287 ± 21% interrupts.CPU240.RES:Rescheduling_interrupts 17745 ± 7% -8.3% 16272 ± 3% interrupts.CPU241.LOC:Local_timer_interrupts 17856 ± 7% -10.2% 16035 ± 4% interrupts.CPU244.LOC:Local_timer_interrupts 928.00 ± 34% +58.3% 1469 ± 11% interrupts.CPU244.RES:Rescheduling_interrupts 17857 ± 8% -9.1% 16228 ± 2% interrupts.CPU245.LOC:Local_timer_interrupts 17944 ± 8% -9.4% 16265 ± 3% interrupts.CPU246.LOC:Local_timer_interrupts 18107 ± 7% -10.6% 16189 ± 3% interrupts.CPU25.LOC:Local_timer_interrupts 17778 ± 8% -10.1% 15979 ± 4% interrupts.CPU250.LOC:Local_timer_interrupts 17881 ± 7% -9.8% 16125 ± 3% interrupts.CPU251.LOC:Local_timer_interrupts 607.25 ± 33% +82.8% 1110 ± 14% interrupts.CPU253.RES:Rescheduling_interrupts 17801 ± 7% -8.3% 16328 ± 3% interrupts.CPU254.LOC:Local_timer_interrupts 17810 ± 6% -9.1% 16195 ± 3% interrupts.CPU255.LOC:Local_timer_interrupts 17760 ± 6% -8.6% 16233 ± 3% interrupts.CPU258.LOC:Local_timer_interrupts 17927 ± 7% -9.5% 16220 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 17668 ± 6% -7.7% 16300 ± 2% interrupts.CPU260.LOC:Local_timer_interrupts 1020 ± 29% +36.8% 1395 ± 17% interrupts.CPU261.RES:Rescheduling_interrupts 17882 ± 6% -9.9% 16109 ± 2% interrupts.CPU263.LOC:Local_timer_interrupts 17815 ± 6% -8.9% 16221 ± 2% interrupts.CPU265.LOC:Local_timer_interrupts 17728 ± 5% -9.1% 16114 ± 2% interrupts.CPU267.LOC:Local_timer_interrupts 17867 ± 6% -10.3% 16028 ± 3% interrupts.CPU268.LOC:Local_timer_interrupts 966.50 ± 9% +38.2% 1335 ± 12% interrupts.CPU268.RES:Rescheduling_interrupts 727.50 ± 23% +95.1% 1419 ± 36% interrupts.CPU269.RES:Rescheduling_interrupts 17783 ± 7% -9.8% 16036 ± 3% interrupts.CPU270.LOC:Local_timer_interrupts 17862 ± 7% -10.0% 16083 ± 3% interrupts.CPU271.LOC:Local_timer_interrupts 17977 ± 6% -9.0% 16354 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts 18010 ± 7% -10.2% 16180 ± 4% interrupts.CPU3.LOC:Local_timer_interrupts 17565 ± 5% -7.0% 16344 interrupts.CPU30.LOC:Local_timer_interrupts 17925 ± 6% -10.7% 16010 ± 3% interrupts.CPU31.LOC:Local_timer_interrupts 17867 ± 7% -9.3% 16207 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 17784 ± 7% -9.5% 16093 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 17923 ± 7% -10.3% 16076 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 17687 ± 6% -8.6% 16169 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts 17759 ± 6% -10.0% 15980 ± 3% interrupts.CPU4.LOC:Local_timer_interrupts 17889 ± 8% -9.6% 16170 ± 3% interrupts.CPU40.LOC:Local_timer_interrupts 17941 ± 7% -9.5% 16235 ± 3% interrupts.CPU41.LOC:Local_timer_interrupts 17992 ± 7% -9.4% 16304 ± 3% interrupts.CPU43.LOC:Local_timer_interrupts 1182 ± 21% +38.4% 1637 ± 10% interrupts.CPU44.RES:Rescheduling_interrupts 17664 ± 9% -8.6% 16138 ± 4% interrupts.CPU45.LOC:Local_timer_interrupts 17900 ± 8% -8.9% 16299 ± 3% interrupts.CPU46.LOC:Local_timer_interrupts 17572 ± 7% -9.4% 15922 ± 3% interrupts.CPU5.LOC:Local_timer_interrupts 17957 ± 5% -9.2% 16306 ± 3% interrupts.CPU55.LOC:Local_timer_interrupts 17885 ± 7% -9.2% 16242 ± 3% interrupts.CPU56.LOC:Local_timer_interrupts 693.50 ± 50% +89.3% 1312 ± 19% interrupts.CPU56.RES:Rescheduling_interrupts 18009 ± 6% -9.4% 16323 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts 17697 ± 6% -7.7% 16335 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts 17883 ± 7% -9.1% 16257 ± 3% interrupts.CPU61.LOC:Local_timer_interrupts 17709 ± 6% -9.7% 15985 ± 3% interrupts.CPU64.LOC:Local_timer_interrupts 17915 ± 6% -9.6% 16201 ± 3% interrupts.CPU66.LOC:Local_timer_interrupts 18209 ± 8% -10.7% 16252 ± 3% interrupts.CPU67.LOC:Local_timer_interrupts 17660 ± 6% -7.7% 16302 ± 3% interrupts.CPU69.LOC:Local_timer_interrupts 1274 ± 20% -33.2% 851.00 ± 24% interrupts.CPU7.RES:Rescheduling_interrupts 17862 ± 7% -9.5% 16168 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts 17538 ± 8% -9.3% 15908 ± 3% interrupts.CPU73.LOC:Local_timer_interrupts 764.75 ± 54% +69.2% 1294 ± 15% interrupts.CPU73.RES:Rescheduling_interrupts 17802 ± 7% -10.1% 16012 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 17865 ± 7% -9.8% 16107 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 17959 ± 9% -10.5% 16078 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 17878 ± 7% -9.2% 16230 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts 17665 ± 7% -9.2% 16034 ± 3% interrupts.CPU81.LOC:Local_timer_interrupts 17879 ± 7% -10.1% 16078 ± 3% interrupts.CPU82.LOC:Local_timer_interrupts 17823 ± 6% -9.3% 16174 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts 17865 ± 7% -9.7% 16131 ± 3% interrupts.CPU86.LOC:Local_timer_interrupts 17916 ± 8% -9.7% 16171 ± 3% interrupts.CPU89.LOC:Local_timer_interrupts 628.00 ± 17% +81.4% 1139 ± 16% interrupts.CPU89.RES:Rescheduling_interrupts 842.00 ± 14% +39.3% 1173 ± 20% interrupts.CPU9.RES:Rescheduling_interrupts 18024 ± 6% -9.8% 16260 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts 723.50 ± 18% +75.7% 1271 ± 10% interrupts.CPU90.RES:Rescheduling_interrupts 17993 ± 7% -10.3% 16146 ± 3% interrupts.CPU93.LOC:Local_timer_interrupts 17681 ± 7% -9.1% 16071 ± 3% interrupts.CPU94.LOC:Local_timer_interrupts 17716 ± 8% -9.3% 16071 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts 17925 ± 6% -10.4% 16066 ± 2% interrupts.CPU97.LOC:Local_timer_interrupts 17997 ± 6% -10.3% 16150 ± 3% interrupts.CPU98.LOC:Local_timer_interrupts 17662 ± 7% -8.8% 16108 interrupts.CPU99.LOC:Local_timer_interrupts 4837465 ± 7% -8.9% 4406579 ± 2% interrupts.LOC:Local_timer_interrupts Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 8 months

1
0
0 / 0

[rcu] 7e28c5af4e: will-it-scale.per_process_ops 84.7% improvement

by kernel test robot

Greeting, FYI, we noticed a 84.7% improvement of will-it-scale.per_process_ops due to commit: commit: 7e28c5af4ef6b539334aa5de40feca0c041c94df ("rcu: Eliminate ->rcu_qs_ctr from the rcu_dynticks structure") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: nr_task: 100% mode: process test: brk1 cpufreq_governor: performance test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase: gcc-7/performance/x86_64-rhel-7.2/process/100%/debian-x86_64-2018-04-03.cgz/lkp-knm01/brk1/will-it-scale commit: c5bacd9417 ("rcu: Motivate Tiny RCU forward progress") 7e28c5af4e ("rcu: Eliminate ->rcu_qs_ctr from the rcu_dynticks structure") c5bacd94173ec49d 7e28c5af4ef6b539334aa5de40 ---------------- -------------------------- %stddev %change %stddev \ | \ 48605 +84.7% 89756 ± 4% will-it-scale.per_process_ops 13998509 +84.7% 25850040 ± 4% will-it-scale.workload 0.01 ± 63% -0.0 0.00 ±131% mpstat.cpu.soft% 11.86 ± 3% -1.2 10.65 ± 4% mpstat.cpu.usr% 662910 +15.8% 767829 ± 3% numa-numastat.node0.local_node 662889 +15.8% 767801 ± 3% numa-numastat.node0.numa_hit 2149 ± 4% -16.9% 1785 ± 2% vmstat.system.cs 727237 ± 3% -14.9% 618668 vmstat.system.in 284156 ± 4% +32.5% 376625 numa-meminfo.node0.Active 283908 ± 4% +32.6% 376374 numa-meminfo.node0.Active(anon) 110629 ± 15% -77.7% 24625 ± 18% numa-meminfo.node0.Inactive 110393 ± 15% -77.9% 24400 ± 19% numa-meminfo.node0.Inactive(anon) 284255 ± 4% +32.5% 376666 meminfo.Active 284007 ± 4% +32.5% 376415 meminfo.Active(anon) 110855 ± 15% -77.8% 24620 ± 18% meminfo.Inactive 110619 ± 15% -77.9% 24395 ± 19% meminfo.Inactive(anon) 8.00 ±173% +11303.1% 912.25 ± 87% meminfo.Mlocked 11072 -15.7% 9338 ± 3% meminfo.max_used_kB 71004 ± 4% +32.6% 94124 numa-vmstat.node0.nr_active_anon 27812 ± 15% -78.1% 6100 ± 19% numa-vmstat.node0.nr_inactive_anon 1.00 ±173% +13100.0% 132.00 ± 87% numa-vmstat.node0.nr_mlock 71002 ± 4% +32.6% 94122 numa-vmstat.node0.nr_zone_active_anon 27811 ± 15% -78.1% 6100 ± 19% numa-vmstat.node0.nr_zone_inactive_anon 1.00 ±173% +9450.0% 95.50 ± 87% numa-vmstat.node1.nr_mlock 70939 ± 4% +32.8% 94186 proc-vmstat.nr_active_anon 27523 ± 15% -77.8% 6101 ± 19% proc-vmstat.nr_inactive_anon 2.25 ±173% +10044.4% 228.25 ± 86% proc-vmstat.nr_mlock 70939 ± 4% +32.8% 94186 proc-vmstat.nr_zone_active_anon 27523 ± 15% -77.8% 6101 ± 19% proc-vmstat.nr_zone_inactive_anon 680060 +15.8% 787707 ± 2% proc-vmstat.numa_hit 680060 +15.8% 787707 ± 2% proc-vmstat.numa_local 140509 ± 10% -45.3% 76820 ± 24% proc-vmstat.numa_pte_updates 895.50 ± 30% +480.0% 5193 ± 36% proc-vmstat.pgactivate 774283 +13.9% 882060 ± 3% proc-vmstat.pgalloc_normal 653457 +21.9% 796258 ± 3% proc-vmstat.pgfault 610954 ± 2% +32.4% 808662 ± 3% proc-vmstat.pgfree 5299 ± 6% +106.4% 10939 ± 36% softirqs.CPU10.RCU 106293 ± 5% +28.2% 136274 ± 15% softirqs.CPU126.TIMER 108234 ± 6% +26.3% 136696 ± 15% softirqs.CPU127.TIMER 109747 ± 7% +26.4% 138696 ± 11% softirqs.CPU135.TIMER 111732 ± 8% +24.4% 138967 ± 11% softirqs.CPU138.TIMER 107031 ± 8% +26.7% 135558 ± 14% softirqs.CPU148.TIMER 100841 +23.1% 124155 ± 15% softirqs.CPU15.TIMER 104640 ± 4% +27.8% 133719 ± 16% softirqs.CPU172.TIMER 103142 ± 3% +30.9% 135023 ± 13% softirqs.CPU190.TIMER 106698 ± 10% +27.5% 136023 ± 13% softirqs.CPU197.TIMER 110268 ± 2% +22.5% 135052 ± 12% softirqs.CPU205.TIMER 106111 ± 8% +23.9% 131436 ± 14% softirqs.CPU206.TIMER 106894 ± 6% +28.7% 137574 ± 14% softirqs.CPU213.TIMER 108418 ± 8% +28.3% 139100 ± 12% softirqs.CPU214.TIMER 100170 ± 10% +32.8% 132978 ± 15% softirqs.CPU215.TIMER 100836 ± 2% +30.3% 131406 ± 17% softirqs.CPU217.TIMER 112497 ± 8% +22.9% 138291 ± 13% softirqs.CPU219.TIMER 108669 ± 10% +22.1% 132720 ± 15% softirqs.CPU220.TIMER 108584 ± 8% +25.5% 136230 ± 13% softirqs.CPU227.TIMER 106251 ± 6% +28.6% 136686 ± 11% softirqs.CPU236.TIMER 105296 ± 7% +31.3% 138251 ± 12% softirqs.CPU238.TIMER 3915 ± 8% +77.4% 6945 ± 30% softirqs.CPU249.RCU 101465 ± 2% +26.5% 128388 ± 19% softirqs.CPU271.TIMER 104242 ± 8% +23.7% 128907 ± 17% softirqs.CPU70.TIMER 107020 ± 7% +26.9% 135845 ± 13% softirqs.CPU92.TIMER 9.659e+09 ± 10% +16.3% 1.124e+10 ± 3% perf-stat.i.branch-instructions 6.69 +1.0 7.69 ± 2% perf-stat.i.branch-miss-rate% 6.443e+08 ± 10% +33.3% 8.59e+08 ± 2% perf-stat.i.branch-misses 4.10 ± 6% +0.9 5.04 ± 10% perf-stat.i.cache-miss-rate% 67464898 ± 10% +25.2% 84491315 ± 4% perf-stat.i.cache-misses 2820 ± 13% -32.4% 1906 ± 3% perf-stat.i.context-switches 15.34 -37.0% 9.67 ± 3% perf-stat.i.cpi 7.067e+11 ± 11% -30.2% 4.932e+11 ± 4% perf-stat.i.cpu-cycles 71.93 ± 10% -41.9% 41.79 ± 9% perf-stat.i.cpu-migrations 2.36 -0.0 2.33 perf-stat.i.iTLB-load-miss-rate% 1.106e+09 ± 10% +15.1% 1.273e+09 ± 2% perf-stat.i.iTLB-load-misses 4.569e+10 ± 10% +17.0% 5.345e+10 ± 3% perf-stat.i.iTLB-loads 4.621e+10 ± 10% +16.1% 5.364e+10 ± 3% perf-stat.i.instructions 0.07 +64.1% 0.11 ± 3% perf-stat.i.ipc 3524 ± 11% -18.3% 2878 perf-stat.i.minor-faults 993359 ± 11% -32.0% 675215 ± 2% perf-stat.i.msec 3529 ± 11% -18.5% 2878 perf-stat.i.page-faults 6.67 +1.0 7.65 perf-stat.overall.branch-miss-rate% 3.91 +0.6 4.54 ± 3% perf-stat.overall.cache-miss-rate% 15.28 -39.8% 9.20 ± 3% perf-stat.overall.cpi 2.36 -0.0 2.33 perf-stat.overall.iTLB-load-miss-rate% 41.78 +0.9% 42.13 perf-stat.overall.instructions-per-iTLB-miss 0.07 +66.4% 0.11 ± 3% perf-stat.overall.ipc 580565 -5.5% 548645 perf-stat.overall.path-length 1.699e+12 +74.9% 2.971e+12 ± 4% perf-stat.total.branch-instructions 1.133e+11 +100.6% 2.272e+11 ± 5% perf-stat.total.branch-misses 1.186e+10 +88.0% 2.23e+10 perf-stat.total.cache-misses 3.032e+11 +62.2% 4.918e+11 ± 2% perf-stat.total.cache-references 1.242e+14 +4.8% 1.302e+14 perf-stat.total.cpu-cycles 12654 -13.0% 11014 ± 5% perf-stat.total.cpu-migrations 1.945e+11 +73.0% 3.365e+11 ± 3% perf-stat.total.iTLB-load-misses 8.038e+12 +75.8% 1.413e+13 ± 4% perf-stat.total.iTLB-loads 8.127e+12 +74.5% 1.418e+13 ± 4% perf-stat.total.instructions 619183 +22.9% 760791 ± 3% perf-stat.total.minor-faults 620031 +22.7% 760750 ± 3% perf-stat.total.page-faults 23472574 ± 28% -50.0% 11736977 ± 60% sched_debug.cfs_rq:/.MIN_vruntime.max 1380731 ± 28% -46.9% 733243 ± 66% sched_debug.cfs_rq:/.MIN_vruntime.stddev 109109 +17.1% 127734 sched_debug.cfs_rq:/.exec_clock.avg 115740 +12.4% 130080 sched_debug.cfs_rq:/.exec_clock.max 4252 ± 4% -50.4% 2110 ± 36% sched_debug.cfs_rq:/.exec_clock.stddev 6243 ± 12% -14.0% 5369 ± 15% sched_debug.cfs_rq:/.load.avg 887713 ± 24% -30.6% 616115 ± 43% sched_debug.cfs_rq:/.load.max 53788 ± 23% -30.4% 37427 ± 40% sched_debug.cfs_rq:/.load.stddev 23472574 ± 28% -50.0% 11736977 ± 60% sched_debug.cfs_rq:/.max_vruntime.max 1380731 ± 28% -46.9% 733243 ± 66% sched_debug.cfs_rq:/.max_vruntime.stddev 36862110 +17.8% 43411425 sched_debug.cfs_rq:/.min_vruntime.avg 39448691 +12.5% 44387703 sched_debug.cfs_rq:/.min_vruntime.max 1668979 ± 3% -45.0% 917652 ± 30% sched_debug.cfs_rq:/.min_vruntime.stddev 1.38 ± 9% -9.1% 1.25 ± 6% sched_debug.cfs_rq:/.nr_running.max 0.56 ± 19% +42.2% 0.80 sched_debug.cfs_rq:/.nr_running.min 0.07 ± 13% -32.7% 0.05 ± 15% sched_debug.cfs_rq:/.nr_running.stddev 2.25 ± 3% -51.5% 1.09 ± 2% sched_debug.cfs_rq:/.nr_spread_over.avg 6235 ± 12% -14.0% 5361 ± 16% sched_debug.cfs_rq:/.runnable_weight.avg 887713 ± 24% -30.6% 616088 ± 43% sched_debug.cfs_rq:/.runnable_weight.max 53791 ± 23% -30.4% 37423 ± 40% sched_debug.cfs_rq:/.runnable_weight.stddev 513480 ± 46% -68.1% 163689 ± 16% sched_debug.cfs_rq:/.spread0.avg 992070 ± 25% -55.6% 440313 ± 8% sched_debug.cfs_rq:/.spread0.max 271.31 ± 25% +54.1% 418.15 ± 23% sched_debug.cfs_rq:/.util_avg.min 71.69 ± 6% -14.9% 61.03 ± 7% sched_debug.cfs_rq:/.util_avg.stddev 638.86 +18.5% 756.85 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.avg 131.07 ± 2% -17.0% 108.78 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.stddev 1617727 ± 4% -24.5% 1222141 ± 11% sched_debug.cpu.avg_idle.avg 167800 ± 2% +13.3% 190124 ± 4% sched_debug.cpu.clock.avg 174380 +10.3% 192313 ± 4% sched_debug.cpu.clock.max 161230 ± 2% +16.5% 187888 ± 4% sched_debug.cpu.clock.min 3797 ± 4% -66.4% 1276 ± 8% sched_debug.cpu.clock.stddev 167800 ± 2% +13.3% 190124 ± 4% sched_debug.cpu.clock_task.avg 174380 +10.3% 192313 ± 4% sched_debug.cpu.clock_task.max 161230 ± 2% +16.5% 187888 ± 4% sched_debug.cpu.clock_task.min 3797 ± 4% -66.4% 1276 ± 8% sched_debug.cpu.clock_task.stddev 5416 ± 2% +15.6% 6262 ± 5% sched_debug.cpu.curr->pid.max 2045 ± 18% +20.6% 2466 ± 3% sched_debug.cpu.curr->pid.min 892728 ± 7% -37.5% 557989 ± 5% sched_debug.cpu.max_idle_balance_cost.avg 4113479 ± 18% -51.6% 1989769 ± 36% sched_debug.cpu.max_idle_balance_cost.max 461057 ± 17% -67.6% 149491 ± 36% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 4% -65.8% 0.00 ± 23% sched_debug.cpu.next_balance.stddev 117031 +16.4% 136279 sched_debug.cpu.nr_load_updates.avg 139472 ± 3% +16.0% 161763 ± 4% sched_debug.cpu.nr_load_updates.max 109218 +22.1% 133390 sched_debug.cpu.nr_load_updates.min 3910 ± 3% -46.0% 2111 ± 19% sched_debug.cpu.nr_load_updates.stddev 0.16 ± 4% -19.4% 0.13 ± 3% sched_debug.cpu.nr_running.stddev 1108 +12.5% 1247 ± 2% sched_debug.cpu.nr_switches.avg 26767 ± 16% +60.1% 42860 ± 15% sched_debug.cpu.nr_switches.max 2342 ± 4% +34.2% 3144 ± 7% sched_debug.cpu.nr_switches.stddev 29066 ± 11% +83.5% 53341 ± 36% sched_debug.cpu.sched_count.max 2473 ± 2% +61.3% 3988 ± 33% sched_debug.cpu.sched_count.stddev 275.25 ± 3% +33.5% 367.54 sched_debug.cpu.ttwu_count.avg 10910 ± 17% +67.3% 18254 ± 13% sched_debug.cpu.ttwu_count.max 50.31 ± 2% +19.9% 60.30 ± 2% sched_debug.cpu.ttwu_count.min 832.06 ± 10% +46.1% 1215 ± 7% sched_debug.cpu.ttwu_count.stddev 176.33 +42.8% 251.81 ± 2% sched_debug.cpu.ttwu_local.avg 10535 ± 18% +69.7% 17877 ± 14% sched_debug.cpu.ttwu_local.max 31.56 +15.2% 36.35 sched_debug.cpu.ttwu_local.min 729.04 ± 13% +57.2% 1146 ± 9% sched_debug.cpu.ttwu_local.stddev 161164 ± 2% +16.6% 187864 ± 4% sched_debug.cpu_clk 160513 ± 2% +16.6% 187213 ± 4% sched_debug.ktime 1.31 ± 24% +39.1% 1.82 sched_debug.rt_rq:/.rt_runtime.stddev 161891 ± 2% +16.3% 188213 ± 4% sched_debug.sched_clk 745.75 ± 61% -87.9% 90.25 ±173% interrupts.36:IR-PCI-MSI.2621442-edge.eth1-TxRx-1 961.75 ± 48% -75.5% 235.25 ±173% interrupts.37:IR-PCI-MSI.2621443-edge.eth1-TxRx-2 4649 ± 16% -25.2% 3475 ± 3% interrupts.CPU100.NMI:Non-maskable_interrupts 4649 ± 16% -25.2% 3475 ± 3% interrupts.CPU100.PMI:Performance_monitoring_interrupts 30.50 ± 17% +277.9% 115.25 ± 49% interrupts.CPU109.RES:Rescheduling_interrupts 552027 ± 5% +12.2% 619466 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts 274.50 ± 59% -44.9% 151.25 ± 92% interrupts.CPU116.RES:Rescheduling_interrupts 37.50 ± 47% +323.3% 158.75 ± 43% interrupts.CPU121.RES:Rescheduling_interrupts 7275 ± 12% -45.9% 3937 ± 25% interrupts.CPU125.NMI:Non-maskable_interrupts 7275 ± 12% -45.9% 3937 ± 25% interrupts.CPU125.PMI:Performance_monitoring_interrupts 5151 ± 34% -32.6% 3470 ± 6% interrupts.CPU140.NMI:Non-maskable_interrupts 5151 ± 34% -32.6% 3470 ± 6% interrupts.CPU140.PMI:Performance_monitoring_interrupts 5589 ± 28% -26.0% 4134 ± 32% interrupts.CPU153.NMI:Non-maskable_interrupts 5589 ± 28% -26.0% 4134 ± 32% interrupts.CPU153.PMI:Performance_monitoring_interrupts 745.75 ± 61% -87.9% 90.25 ±173% interrupts.CPU16.36:IR-PCI-MSI.2621442-edge.eth1-TxRx-1 961.75 ± 48% -75.5% 235.25 ±173% interrupts.CPU17.37:IR-PCI-MSI.2621443-edge.eth1-TxRx-2 6606 ± 25% -38.6% 4057 ± 27% interrupts.CPU171.NMI:Non-maskable_interrupts 6606 ± 25% -38.6% 4057 ± 27% interrupts.CPU171.PMI:Performance_monitoring_interrupts 5916 ± 21% -28.9% 4205 ± 28% interrupts.CPU172.NMI:Non-maskable_interrupts 5916 ± 21% -28.9% 4205 ± 28% interrupts.CPU172.PMI:Performance_monitoring_interrupts 85.50 ± 36% -79.2% 17.75 ± 72% interrupts.CPU176.RES:Rescheduling_interrupts 74.00 ± 79% -79.1% 15.50 ± 27% interrupts.CPU177.RES:Rescheduling_interrupts 6513 ± 21% -36.4% 4139 ± 24% interrupts.CPU180.NMI:Non-maskable_interrupts 6513 ± 21% -36.4% 4139 ± 24% interrupts.CPU180.PMI:Performance_monitoring_interrupts 17.50 ± 32% +371.4% 82.50 ±101% interrupts.CPU181.RES:Rescheduling_interrupts 41.00 ±101% +232.9% 136.50 ± 79% interrupts.CPU183.RES:Rescheduling_interrupts 5998 ± 31% -29.3% 4238 ± 18% interrupts.CPU192.NMI:Non-maskable_interrupts 5998 ± 31% -29.3% 4238 ± 18% interrupts.CPU192.PMI:Performance_monitoring_interrupts 5812 ± 21% -40.6% 3450 ± 4% interrupts.CPU198.NMI:Non-maskable_interrupts 5812 ± 21% -40.6% 3450 ± 4% interrupts.CPU198.PMI:Performance_monitoring_interrupts 5605 ± 22% -27.0% 4093 ± 26% interrupts.CPU200.NMI:Non-maskable_interrupts 5605 ± 22% -27.0% 4093 ± 26% interrupts.CPU200.PMI:Performance_monitoring_interrupts 5528 ± 27% -38.1% 3423 ± 2% interrupts.CPU212.NMI:Non-maskable_interrupts 5528 ± 27% -38.1% 3423 ± 2% interrupts.CPU212.PMI:Performance_monitoring_interrupts 6447 ± 27% -31.2% 4433 ± 39% interrupts.CPU216.NMI:Non-maskable_interrupts 6447 ± 27% -31.2% 4433 ± 39% interrupts.CPU216.PMI:Performance_monitoring_interrupts 561218 ± 5% +10.4% 619618 ± 2% interrupts.CPU220.LOC:Local_timer_interrupts 201.00 ±134% -85.0% 30.25 ± 53% interrupts.CPU228.RES:Rescheduling_interrupts 5757 ± 22% -35.4% 3719 ± 4% interrupts.CPU229.NMI:Non-maskable_interrupts 5757 ± 22% -35.4% 3719 ± 4% interrupts.CPU229.PMI:Performance_monitoring_interrupts 165.75 ± 46% -70.6% 48.75 ±103% interrupts.CPU230.RES:Rescheduling_interrupts 23.75 ± 38% +346.3% 106.00 ± 87% interrupts.CPU236.RES:Rescheduling_interrupts 4576 ± 15% -26.0% 3385 ± 3% interrupts.CPU248.NMI:Non-maskable_interrupts 4576 ± 15% -26.0% 3385 ± 3% interrupts.CPU248.PMI:Performance_monitoring_interrupts 606.75 ±140% -92.7% 44.00 ± 69% interrupts.CPU252.RES:Rescheduling_interrupts 80.00 ± 62% -67.5% 26.00 ± 30% interrupts.CPU256.RES:Rescheduling_interrupts 5696 ± 30% -26.9% 4164 ± 26% interrupts.CPU259.NMI:Non-maskable_interrupts 5696 ± 30% -26.9% 4164 ± 26% interrupts.CPU259.PMI:Performance_monitoring_interrupts 1796 ± 99% -76.2% 428.00 ± 45% interrupts.CPU26.RES:Rescheduling_interrupts 109.50 ± 70% -66.0% 37.25 ± 55% interrupts.CPU266.RES:Rescheduling_interrupts 143.00 ± 38% -69.1% 44.25 ± 90% interrupts.CPU271.RES:Rescheduling_interrupts 5526 ± 29% -34.5% 3619 interrupts.CPU278.NMI:Non-maskable_interrupts 5526 ± 29% -34.5% 3619 interrupts.CPU278.PMI:Performance_monitoring_interrupts 94.75 ± 42% -55.1% 42.50 ± 64% interrupts.CPU278.RES:Rescheduling_interrupts 6671 ± 19% -31.4% 4579 ± 31% interrupts.CPU281.NMI:Non-maskable_interrupts 6671 ± 19% -31.4% 4579 ± 31% interrupts.CPU281.PMI:Performance_monitoring_interrupts 7140 ± 25% -45.1% 3917 ± 2% interrupts.CPU37.NMI:Non-maskable_interrupts 7140 ± 25% -45.1% 3917 ± 2% interrupts.CPU37.PMI:Performance_monitoring_interrupts 6176 ± 29% -37.9% 3834 ± 3% interrupts.CPU38.NMI:Non-maskable_interrupts 6176 ± 29% -37.9% 3834 ± 3% interrupts.CPU38.PMI:Performance_monitoring_interrupts 968.50 ±109% -75.0% 241.75 ± 42% interrupts.CPU38.RES:Rescheduling_interrupts 6057 ± 24% -28.2% 4352 ± 23% interrupts.CPU41.NMI:Non-maskable_interrupts 6057 ± 24% -28.2% 4352 ± 23% interrupts.CPU41.PMI:Performance_monitoring_interrupts 120.00 ± 99% +376.2% 571.50 ± 85% interrupts.CPU47.RES:Rescheduling_interrupts 177.75 ± 31% +247.4% 617.50 ± 88% interrupts.CPU48.RES:Rescheduling_interrupts 110.00 ± 83% +116.6% 238.25 ± 26% interrupts.CPU49.RES:Rescheduling_interrupts 6925 ± 22% -45.4% 3781 ± 3% interrupts.CPU50.NMI:Non-maskable_interrupts 6925 ± 22% -45.4% 3781 ± 3% interrupts.CPU50.PMI:Performance_monitoring_interrupts 180.25 ± 97% +685.2% 1415 ±115% interrupts.CPU51.RES:Rescheduling_interrupts 182.50 ± 53% +1719.2% 3320 ± 62% interrupts.CPU52.RES:Rescheduling_interrupts 5043 ± 18% -26.0% 3730 ± 5% interrupts.CPU57.NMI:Non-maskable_interrupts 5043 ± 18% -26.0% 3730 ± 5% interrupts.CPU57.PMI:Performance_monitoring_interrupts 3655 ± 9% +33.0% 4863 ± 21% interrupts.CPU60.NMI:Non-maskable_interrupts 3655 ± 9% +33.0% 4863 ± 21% interrupts.CPU60.PMI:Performance_monitoring_interrupts 6130 ± 24% -28.1% 4410 ± 21% interrupts.CPU64.NMI:Non-maskable_interrupts 6130 ± 24% -28.1% 4410 ± 21% interrupts.CPU64.PMI:Performance_monitoring_interrupts 5811 ± 25% -36.1% 3713 interrupts.CPU76.NMI:Non-maskable_interrupts 5811 ± 25% -36.1% 3713 interrupts.CPU76.PMI:Performance_monitoring_interrupts 6036 ± 28% -39.4% 3660 ± 5% interrupts.CPU80.NMI:Non-maskable_interrupts 6036 ± 28% -39.4% 3660 ± 5% interrupts.CPU80.PMI:Performance_monitoring_interrupts 6494 ± 25% -43.9% 3640 ± 5% interrupts.CPU82.NMI:Non-maskable_interrupts 6494 ± 25% -43.9% 3640 ± 5% interrupts.CPU82.PMI:Performance_monitoring_interrupts 6881 ± 13% -48.8% 3522 ± 3% interrupts.CPU84.NMI:Non-maskable_interrupts 6881 ± 13% -48.8% 3522 ± 3% interrupts.CPU84.PMI:Performance_monitoring_interrupts 110.75 ± 19% +119.4% 243.00 ± 19% interrupts.CPU9.RES:Rescheduling_interrupts will-it-scale.per_process_ops 95000 +-+-----------------------------------O-----------------------------+ 90000 +-+ O O O O O | | O O OO O O | 85000 +-+ OOO O O O | 80000 +-+ OO O O O | | O O O | 75000 +-+O O | 70000 OO+ O O O | 65000 +-+ | | | 60000 +-+ | 55000 +-+ + + .+ .+ ++. | | .+++.+ ++.+ :.+ + ++ + +++.+++.++ + .+++.+ +. + | 50000 +-+ + + :+ +++ + .++ + ++.+++.+| 45000 +-+-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 8 months

1
0
0 / 0

[xfs] ea3e90ae01: aim7.jobs-per-min -23.5% regression

by kernel test robot

Greeting, FYI, we noticed a -23.5% regression of aim7.jobs-per-min due to commit: commit: ea3e90ae01e5d776b06334d89282c65b55cc4732 ("xfs: deferred inode inactivation") https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git djwong-wtf in testcase: aim7 on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory with following parameters: disk: 1BRD_48G fs: xfs test: disk_wrt load: 3000 cpufreq_governor: performance test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | aim7: aim7.jobs-per-min -34.4% regression | | test machine | 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory | | test parameters | cpufreq_governor=performance | | | disk=4BRD_12G | | | fs=xfs | | | load=3000 | | | md=RAID0 | | | test=disk_cp | +------------------+-----------------------------------------------------------------------+ | testcase: change | aim7: aim7.jobs-per-min -36.1% regression | | test machine | 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory | | test parameters | cpufreq_governor=performance | | | disk=1BRD_48G | | | fs=xfs | | | load=3000 | | | test=disk_cp | +------------------+-----------------------------------------------------------------------+ | testcase: change | aim7: aim7.jobs-per-min -22.8% regression | | test machine | 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory | | test parameters | cpufreq_governor=performance | | | disk=4BRD_12G | | | fs=xfs | | | load=3000 | | | md=RAID0 | | | test=disk_wrt | +------------------+-----------------------------------------------------------------------+ | testcase: change | aim7: aim7.jobs-per-min -3.6% regression | | test machine | 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory | | test parameters | cpufreq_governor=performance | | | disk=1BRD_48G | | | fs=xfs | | | load=3000 | | | test=disk_rw | +------------------+-----------------------------------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase: gcc-7/performance/1BRD_48G/xfs/x86_64-rhel-7.2/3000/debian-x86_64-2018-04-03.cgz/lkp-ivb-ep01/disk_wrt/aim7 commit: 1410e363f1 ("xfs: refactor eofblocks inode match code") ea3e90ae01 ("xfs: deferred inode inactivation") 1410e363f1c8ee34 ea3e90ae01e5d776b06334d892 ---------------- -------------------------- fail:runs %reproduction fail:runs | | | 2:4 -50% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x :4 75% 3:4 stderr.AIM_Multiuser_Benchmark-Suite_VII_Testing_over :4 75% 3:4 stderr.Child##::No_space_left_on_device :4 75% 3:4 stderr.Child##::No_such_file_or_directory :4 75% 3:4 stderr.Failed_to_execute :4 75% 3:4 stderr.disk1.c:No_space_left_on_device :4 25% 1:4 stderr.disk1.c:No_such_file_or_directory :4 75% 3:4 stderr.disk_wrt/fs/ram0 :4 75% 3:4 stderr.disk_wrt:cannot_create/fs/ram0/tmpb :4 75% 3:4 stderr.disk_wrt:cannot_write/fs/ram0/tmpb %stddev %change %stddev \ | \ 183256 -23.5% 140165 aim7.jobs-per-min 98.64 +30.7% 128.93 aim7.time.elapsed_time 98.64 +30.7% 128.93 aim7.time.elapsed_time.max 2942 ± 5% +72.5% 5076 aim7.time.involuntary_context_switches 150347 -4.5% 143525 aim7.time.minor_page_faults 1145 +2.3% 1171 aim7.time.system_time 105.14 -11.3% 93.28 aim7.time.user_time 1088728 -18.4% 888576 aim7.time.voluntary_context_switches 28.90 -5.3 23.57 mpstat.cpu.sys% 2.82 ± 2% -0.8 1.97 mpstat.cpu.usr% 748771 ± 6% +13.6% 850841 softirqs.RCU 1698774 ± 7% +33.9% 2273908 softirqs.TIMER 447.25 ± 42% +416.0% 2308 vmstat.io.bo 12.00 -16.7% 10.00 vmstat.procs.r 22971 -15.5% 19419 vmstat.system.cs 1.511e+08 ± 97% -66.7% 50283389 cpuidle.C1.time 4.829e+08 ± 54% +157.0% 1.241e+09 cpuidle.C3.time 1619863 ± 55% +175.7% 4465852 cpuidle.C3.usage 1.963e+09 ± 6% +22.2% 2.399e+09 cpuidle.C6.time 3129 ± 3% -21.4% 2461 numa-meminfo.node0.Dirty 41377 ± 58% +94.2% 80360 numa-meminfo.node1.AnonHugePages 3058 ± 4% -42.8% 1748 numa-meminfo.node1.Dirty 3257 ± 18% -43.4% 1845 numa-meminfo.node1.Inactive(file) 39372316 ± 2% +13.8% 44791654 numa-numastat.node0.local_node 39374967 ± 2% +13.8% 44791761 numa-numastat.node0.numa_hit 37866081 ± 2% -13.5% 32760615 numa-numastat.node1.local_node 37871325 ± 2% -13.5% 32768405 numa-numastat.node1.numa_hit 91221 +24.1% 113175 meminfo.AnonHugePages 4460544 ± 11% -11.9% 3928064 meminfo.DirectMap2M 6162 ± 7% -29.7% 4331 meminfo.Dirty 7228 ± 6% -24.1% 5488 meminfo.Inactive(file) 82887 +22.1% 101191 meminfo.KReclaimable 53296 -18.8% 43277 meminfo.KernelStack 104268 ± 7% -21.7% 81666 meminfo.PageTables 82887 +22.1% 101191 meminfo.SReclaimable 245621 +12.3% 275896 meminfo.Slab 22543 -21.4% 17709 meminfo.max_used_kB 419.00 -11.7% 370.00 turbostat.Avg_MHz 34.57 -5.7 28.83 turbostat.Busy% 3.78 ± 97% -2.8 0.96 turbostat.C1% 1619503 ± 55% +175.7% 4465464 turbostat.C3 12.05 ± 54% +11.7 23.76 turbostat.C3% 39.05 -1.6% 38.43 turbostat.CorWatt 8132763 +29.9% 10567462 turbostat.IRQ 3.90 ± 19% -55.2% 1.75 turbostat.Pkg%pc2 65.06 -1.1% 64.32 turbostat.PkgWatt 30.48 -1.5% 30.01 turbostat.RAMWatt 8010 +30.3% 10440 turbostat.SMI 17506003 ± 3% +26.4% 22129161 numa-vmstat.node0.nr_dirtied 801.25 ± 2% -19.6% 644.00 numa-vmstat.node0.nr_dirty 30.00 ±104% +306.7% 122.00 numa-vmstat.node0.nr_mlock 2281 ± 67% +121.3% 5047 numa-vmstat.node0.nr_written 736.00 ± 3% -16.0% 618.00 numa-vmstat.node0.nr_zone_write_pending 18000788 ± 4% +25.4% 22573406 numa-vmstat.node0.numa_hit 17938349 ± 4% +25.2% 22453943 numa-vmstat.node0.numa_local 16401874 ± 5% -13.4% 14202478 numa-vmstat.node1.nr_dirtied 759.75 ± 2% -40.0% 456.00 numa-vmstat.node1.nr_dirty 816.50 ± 16% -41.9% 474.00 numa-vmstat.node1.nr_inactive_file 810.00 ± 17% -40.7% 480.00 numa-vmstat.node1.nr_zone_inactive_file 682.50 ± 4% -38.9% 417.00 numa-vmstat.node1.nr_zone_write_pending 16831813 ± 5% -12.5% 14735042 numa-vmstat.node1.numa_hit 16737137 ± 5% -12.2% 14697376 numa-vmstat.node1.numa_local 89520 -8.2% 82199 proc-vmstat.nr_active_anon 88370 -8.3% 81005 proc-vmstat.nr_anon_pages 1563 ± 3% -32.3% 1058 proc-vmstat.nr_dirty 1837 -26.5% 1351 proc-vmstat.nr_inactive_file 53302 -19.1% 43097 proc-vmstat.nr_kernel_stack 59.00 ±100% +255.9% 210.00 proc-vmstat.nr_mlock 26088 ± 7% -22.2% 20308 proc-vmstat.nr_page_table_pages 20725 +22.0% 25283 proc-vmstat.nr_slab_reclaimable 40701 +7.1% 43591 proc-vmstat.nr_slab_unreclaimable 7660 ± 62% +118.4% 16727 proc-vmstat.nr_written 89520 -8.2% 82199 proc-vmstat.nr_zone_active_anon 1837 -26.5% 1351 proc-vmstat.nr_zone_inactive_file 1425 ± 2% -32.2% 967.00 proc-vmstat.nr_zone_write_pending 1126 ±159% +1323.2% 16032 proc-vmstat.numa_pages_migrated 422562 +16.7% 493239 proc-vmstat.pgfault 1126 ±159% +1323.2% 16032 proc-vmstat.pgmigrate_success 45196 ± 42% +574.2% 304712 proc-vmstat.pgpgout 10.94 ± 11% -14.6% 9.35 perf-stat.i.MPKI 2.809e+09 -11.5% 2.487e+09 perf-stat.i.branch-instructions 51267609 ± 3% -14.8% 43705622 perf-stat.i.branch-misses 10302747 ± 5% -19.4% 8305142 perf-stat.i.cache-misses 1.103e+08 ± 2% -17.4% 91100594 perf-stat.i.cache-references 23263 -15.5% 19657 perf-stat.i.context-switches 1.679e+10 -10.4% 1.504e+10 perf-stat.i.cpu-cycles 570.71 ± 2% -67.5% 185.40 perf-stat.i.cpu-migrations 1609 ± 3% +21.8% 1960 perf-stat.i.cycles-between-cache-misses 66929844 ± 14% -24.0% 50837864 perf-stat.i.dTLB-load-misses 4.228e+09 -10.9% 3.769e+09 perf-stat.i.dTLB-loads 2.775e+09 ± 2% -16.2% 2.325e+09 perf-stat.i.dTLB-stores 81.99 ± 2% -3.6 78.36 perf-stat.i.iTLB-load-miss-rate% 5890227 ± 20% -34.9% 3833787 perf-stat.i.iTLB-load-misses 1331587 ± 2% -17.3% 1101350 perf-stat.i.iTLB-loads 1.406e+10 -11.1% 1.251e+10 perf-stat.i.instructions 2622 ± 14% +25.7% 3296 perf-stat.i.instructions-per-iTLB-miss 4041 -9.1% 3675 perf-stat.i.minor-faults 41.78 -1.0 40.82 perf-stat.i.node-load-miss-rate% 1454704 ± 4% -18.8% 1180600 perf-stat.i.node-load-misses 1975488 -16.4% 1652485 perf-stat.i.node-loads 31.25 ± 3% -1.3 29.95 perf-stat.i.node-store-miss-rate% 2327502 ± 2% -39.2% 1415221 perf-stat.i.node-store-misses 4843416 -36.0% 3100429 perf-stat.i.node-stores 4041 -9.1% 3675 perf-stat.i.page-faults 7.84 ± 2% -7.1% 7.29 perf-stat.overall.MPKI 1632 ± 3% +10.9% 1810 perf-stat.overall.cycles-between-cache-misses 81.13 ± 3% -3.4 77.68 perf-stat.overall.iTLB-load-miss-rate% 2475 ± 17% +31.8% 3261 perf-stat.overall.instructions-per-iTLB-miss 42.39 -0.7 41.67 perf-stat.overall.node-load-miss-rate% 32.46 -1.1 31.34 perf-stat.overall.node-store-miss-rate% 2.781e+09 -11.2% 2.468e+09 perf-stat.ps.branch-instructions 50756832 ± 3% -14.6% 43368170 perf-stat.ps.branch-misses 10198206 ± 5% -19.2% 8241618 perf-stat.ps.cache-misses 1.091e+08 ± 2% -17.2% 90400364 perf-stat.ps.cache-references 23024 -15.3% 19507 perf-stat.ps.context-switches 1.662e+10 -10.2% 1.492e+10 perf-stat.ps.cpu-cycles 564.90 ± 2% -67.4% 183.98 perf-stat.ps.cpu-migrations 66243199 ± 14% -23.8% 50446460 perf-stat.ps.dTLB-load-misses 4.184e+09 -10.6% 3.74e+09 perf-stat.ps.dTLB-loads 2.747e+09 ± 2% -16.0% 2.307e+09 perf-stat.ps.dTLB-stores 5829978 ± 21% -34.7% 3804317 perf-stat.ps.iTLB-load-misses 1317945 ± 2% -17.1% 1092904 perf-stat.ps.iTLB-loads 1.392e+10 -10.8% 1.241e+10 perf-stat.ps.instructions 4002 -8.9% 3647 perf-stat.ps.minor-faults 1439804 ± 4% -18.6% 1171552 perf-stat.ps.node-load-misses 1955278 -16.1% 1639830 perf-stat.ps.node-loads 2303657 ± 2% -39.0% 1404416 perf-stat.ps.node-store-misses 4793893 -35.8% 3076781 perf-stat.ps.node-stores 4002 -8.9% 3647 perf-stat.ps.page-faults 1.383e+12 +17.0% 1.618e+12 perf-stat.total.instructions 117845 -12.0% 103706 slabinfo.anon_vma_chain.active_objs 2778 ± 2% +446.2% 15176 slabinfo.dmaengine-unmap-16.active_objs 66.50 ± 3% +666.9% 510.00 slabinfo.dmaengine-unmap-16.active_slabs 2818 ± 3% +661.2% 21457 slabinfo.dmaengine-unmap-16.num_objs 66.50 ± 3% +666.9% 510.00 slabinfo.dmaengine-unmap-16.num_slabs 1665 ± 6% +27.8% 2128 slabinfo.eventpoll_pwq.active_objs 1665 ± 6% +27.8% 2128 slabinfo.eventpoll_pwq.num_objs 2121 +61.2% 3420 slabinfo.kmalloc-128.active_objs 2127 +60.8% 3420 slabinfo.kmalloc-128.num_objs 18752 +132.4% 43581 slabinfo.kmalloc-16.active_objs 73.25 +148.5% 182.00 slabinfo.kmalloc-16.active_slabs 18752 +148.8% 46649 slabinfo.kmalloc-16.num_objs 73.25 +148.5% 182.00 slabinfo.kmalloc-16.num_slabs 746.00 +15.3% 860.00 slabinfo.kmalloc-4k.active_objs 758.75 +15.6% 877.00 slabinfo.kmalloc-4k.num_objs 6690 +247.7% 23258 slabinfo.kmalloc-512.active_objs 219.00 +291.3% 857.00 slabinfo.kmalloc-512.active_slabs 7032 +290.3% 27449 slabinfo.kmalloc-512.num_objs 219.00 +291.3% 857.00 slabinfo.kmalloc-512.num_slabs 412.00 +114.8% 885.00 slabinfo.kmalloc-8k.active_objs 105.25 +124.2% 236.00 slabinfo.kmalloc-8k.active_slabs 422.25 +123.6% 944.00 slabinfo.kmalloc-8k.num_objs 105.25 +124.2% 236.00 slabinfo.kmalloc-8k.num_slabs 1008 ± 18% -42.9% 576.00 slabinfo.kmalloc-rcl-128.active_objs 1008 ± 18% -42.9% 576.00 slabinfo.kmalloc-rcl-128.num_objs 2414 ± 10% +16.6% 2815 slabinfo.kmalloc-rcl-64.active_objs 2414 ± 10% +16.6% 2815 slabinfo.kmalloc-rcl-64.num_objs 1543 ± 6% -29.3% 1092 slabinfo.kmalloc-rcl-96.active_objs 1543 ± 6% -29.3% 1092 slabinfo.kmalloc-rcl-96.num_objs 1397 ± 3% +24.6% 1741 slabinfo.mnt_cache.active_objs 1397 ± 3% +24.6% 1741 slabinfo.mnt_cache.num_objs 2695 -16.6% 2249 slabinfo.names_cache.active_objs 2726 -16.3% 2282 slabinfo.names_cache.num_objs 1117 ± 2% -9.8% 1007 slabinfo.nsproxy.active_objs 1117 ± 2% -9.8% 1007 slabinfo.nsproxy.num_objs 22454 -8.6% 20527 slabinfo.proc_inode_cache.active_objs 20542 +12.4% 23100 slabinfo.radix_tree_node.active_objs 20749 +12.1% 23250 slabinfo.radix_tree_node.num_objs 3939 -13.6% 3402 slabinfo.sighand_cache.active_objs 4584 -12.8% 4000 slabinfo.signal_cache.active_objs 1947 ± 4% +15.2% 2244 slabinfo.sock_inode_cache.active_objs 1947 ± 4% +15.2% 2244 slabinfo.sock_inode_cache.num_objs 3564 -17.6% 2936 slabinfo.task_struct.active_objs 1223 -9.7% 1105 slabinfo.task_struct.active_slabs 3672 -9.7% 3317 slabinfo.task_struct.num_objs 1223 -9.7% 1105 slabinfo.task_struct.num_slabs 95963 -15.5% 81120 slabinfo.vm_area_struct.active_objs 2430 -12.1% 2135 slabinfo.vm_area_struct.active_slabs 97232 -12.2% 85406 slabinfo.vm_area_struct.num_objs 2430 -12.1% 2135 slabinfo.vm_area_struct.num_slabs 1080 +60.1% 1729 slabinfo.xfs_buf_item.active_objs 1080 +60.1% 1729 slabinfo.xfs_buf_item.num_objs 1106 ± 6% +100.5% 2218 slabinfo.xfs_efd_item.active_objs 1106 ± 6% +100.5% 2218 slabinfo.xfs_efd_item.num_objs 2376 +486.4% 13934 slabinfo.xfs_inode.active_objs 71.25 +757.5% 611.00 slabinfo.xfs_inode.active_slabs 2434 +754.6% 20804 slabinfo.xfs_inode.num_objs 71.25 +757.5% 611.00 slabinfo.xfs_inode.num_slabs 8677 +70.1% 14762 sched_debug.cfs_rq:/.exec_clock.avg 10740 ± 6% +76.3% 18939 sched_debug.cfs_rq:/.exec_clock.max 7873 ± 3% +30.8% 10295 sched_debug.cfs_rq:/.exec_clock.min 591.52 ± 38% +353.3% 2681 sched_debug.cfs_rq:/.exec_clock.stddev 16972 ± 11% +114.8% 36462 sched_debug.cfs_rq:/.load.avg 54921 ± 4% +1193.8% 710558 sched_debug.cfs_rq:/.load.max 21620 ± 2% +431.1% 114823 sched_debug.cfs_rq:/.load.stddev 141516 ± 2% +43.8% 203440 sched_debug.cfs_rq:/.min_vruntime.avg 232226 ± 5% +53.1% 355483 sched_debug.cfs_rq:/.min_vruntime.max 18767 ± 13% +171.1% 50884 sched_debug.cfs_rq:/.min_vruntime.stddev 0.37 ± 15% -21.6% 0.29 sched_debug.cfs_rq:/.nr_running.avg 1.00 +33.3% 1.33 sched_debug.cfs_rq:/.nr_spread_over.max 8.26 ± 9% +97.5% 16.31 sched_debug.cfs_rq:/.runnable_load_avg.avg 43.50 ± 8% +748.3% 369.00 sched_debug.cfs_rq:/.runnable_load_avg.max 11.52 ± 3% +416.4% 59.48 sched_debug.cfs_rq:/.runnable_load_avg.stddev 15680 ± 11% +126.8% 35568 sched_debug.cfs_rq:/.runnable_weight.avg 46021 ± 7% +1442.0% 709671 sched_debug.cfs_rq:/.runnable_weight.max 19481 +482.9% 113551 sched_debug.cfs_rq:/.runnable_weight.stddev -559.46 +4100.4% -23499 sched_debug.cfs_rq:/.spread0.avg 90079 ± 17% +42.7% 128560 sched_debug.cfs_rq:/.spread0.max -17348 +473.3% -99457 sched_debug.cfs_rq:/.spread0.min 18774 ± 13% +171.0% 50884 sched_debug.cfs_rq:/.spread0.stddev 530.73 ± 6% -34.4% 348.26 sched_debug.cfs_rq:/.util_avg.avg 1375 ± 9% -20.9% 1088 sched_debug.cfs_rq:/.util_avg.max 179.50 ± 18% -71.4% 51.33 sched_debug.cfs_rq:/.util_avg.min 254.48 ± 8% -25.4% 189.84 sched_debug.cfs_rq:/.util_avg.stddev 63.01 ± 13% -27.3% 45.82 sched_debug.cfs_rq:/.util_est_enqueued.avg 527.50 ± 18% +33.0% 701.67 sched_debug.cfs_rq:/.util_est_enqueued.max 514399 ± 2% +22.6% 630419 sched_debug.cpu.avg_idle.avg 259027 ± 5% -27.7% 187213 sched_debug.cpu.avg_idle.stddev 65370 +44.3% 94338 sched_debug.cpu.clock.avg 65396 +44.3% 94359 sched_debug.cpu.clock.max 65342 +44.3% 94319 sched_debug.cpu.clock.min 15.67 ± 9% -23.1% 12.04 sched_debug.cpu.clock.stddev 65370 +44.3% 94338 sched_debug.cpu.clock_task.avg 65396 +44.3% 94359 sched_debug.cpu.clock_task.max 65342 +44.3% 94319 sched_debug.cpu.clock_task.min 15.67 ± 9% -23.1% 12.04 sched_debug.cpu.clock_task.stddev 8.62 ± 13% +42.7% 12.30 sched_debug.cpu.cpu_load[0].avg 45.62 ± 7% +405.6% 230.67 sched_debug.cpu.cpu_load[0].max 11.64 ± 3% +236.3% 39.14 sched_debug.cpu.cpu_load[0].stddev 9.86 ± 11% +60.3% 15.80 sched_debug.cpu.cpu_load[1].avg 58.25 ± 42% +467.7% 330.67 sched_debug.cpu.cpu_load[1].max 12.57 ± 22% +322.3% 53.10 sched_debug.cpu.cpu_load[1].stddev 10.95 ± 12% +36.6% 14.97 sched_debug.cpu.cpu_load[2].avg 70.00 ± 36% +333.3% 303.33 sched_debug.cpu.cpu_load[2].max 13.50 ± 24% +255.8% 48.03 sched_debug.cpu.cpu_load[2].stddev 11.84 ± 10% +18.0% 13.97 sched_debug.cpu.cpu_load[3].avg 73.75 ± 11% +251.6% 259.33 sched_debug.cpu.cpu_load[3].max 14.27 ± 8% +189.2% 41.28 sched_debug.cpu.cpu_load[3].stddev 157.12 ± 8% +78.8% 281.00 sched_debug.cpu.cpu_load[4].max 25.76 ± 6% +72.1% 44.35 sched_debug.cpu.cpu_load[4].stddev 1007 ± 16% -31.4% 690.52 sched_debug.cpu.curr->pid.avg 3872 +32.4% 5129 sched_debug.cpu.curr->pid.max 0.00 ± 8% -16.4% 0.00 sched_debug.cpu.next_balance.stddev 37560 +79.5% 67410 sched_debug.cpu.nr_load_updates.avg 45621 ± 4% +71.2% 78109 sched_debug.cpu.nr_load_updates.max 34675 +84.5% 63961 sched_debug.cpu.nr_load_updates.min 0.39 ± 11% -24.7% 0.29 sched_debug.cpu.nr_running.avg 17652 +71.9% 30342 sched_debug.cpu.nr_switches.avg 25548 ± 4% +158.3% 65991 sched_debug.cpu.nr_switches.max 14707 ± 3% +20.4% 17704 sched_debug.cpu.nr_switches.min 2410 ± 6% +369.0% 11308 sched_debug.cpu.nr_switches.stddev 37.31 -23.9% 28.38 sched_debug.cpu.nr_uninterruptible.avg 70.12 ± 9% -13.5% 60.67 sched_debug.cpu.nr_uninterruptible.max 8.25 ± 80% -156.6% -4.67 ± 21% sched_debug.cpu.nr_uninterruptible.min 16014 +79.4% 28727 sched_debug.cpu.sched_count.avg 18645 ± 3% +229.6% 61448 sched_debug.cpu.sched_count.max 14268 ± 3% +16.7% 16651 sched_debug.cpu.sched_count.min 991.56 ± 17% +973.3% 10641 sched_debug.cpu.sched_count.stddev 7933 +78.6% 14173 sched_debug.cpu.sched_goidle.avg 9228 ± 4% +229.8% 30438 sched_debug.cpu.sched_goidle.max 7085 ± 3% +16.3% 8239 sched_debug.cpu.sched_goidle.min 482.50 ± 15% +991.2% 5264 sched_debug.cpu.sched_goidle.stddev 7980 +79.3% 14310 sched_debug.cpu.ttwu_count.avg 9942 +217.2% 31537 sched_debug.cpu.ttwu_count.max 7128 ± 4% +26.0% 8984 sched_debug.cpu.ttwu_count.min 597.48 ± 18% +776.2% 5235 sched_debug.cpu.ttwu_count.stddev 182.07 ± 2% +196.1% 539.14 sched_debug.cpu.ttwu_local.avg 364.38 ± 8% +179.3% 1017 sched_debug.cpu.ttwu_local.max 127.50 ± 2% +145.0% 312.33 sched_debug.cpu.ttwu_local.min 48.30 ± 8% +231.2% 159.98 sched_debug.cpu.ttwu_local.stddev 65342 +44.3% 94317 sched_debug.cpu_clk 62900 +46.1% 91878 sched_debug.ktime 65817 +44.0% 94792 sched_debug.sched_clk 36.96 -5.6 31.31 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 29.84 -5.5 24.38 perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_write.__vfs_write.vfs_write.ksys_write.do_syscall_64 31.53 -5.4 26.16 perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 35.67 -5.2 30.42 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 25.09 -4.2 20.88 perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.__vfs_write.vfs_write.ksys_write 24.88 -4.2 20.69 perf-profile.calltrace.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.__vfs_write.vfs_write 48.72 -3.8 44.88 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 48.40 -3.8 44.61 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.73 -3.2 15.56 perf-profile.calltrace.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.__vfs_write 9.78 -1.7 8.10 perf-profile.calltrace.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 4.32 ± 2% -1.6 2.69 perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.31 ± 2% -1.6 2.69 perf-profile.calltrace.cycles-pp.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.30 ± 2% -1.6 2.69 perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.28 ± 2% -1.6 2.67 perf-profile.calltrace.cycles-pp.dput.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 4.27 ± 2% -1.6 2.66 perf-profile.calltrace.cycles-pp.dentry_kill.dput.__fput.task_work_run.exit_to_usermode_loop 4.26 ± 2% -1.6 2.66 perf-profile.calltrace.cycles-pp.__dentry_kill.dentry_kill.dput.__fput.task_work_run 7.05 ± 2% -1.3 5.73 perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write 6.82 ± 2% -1.2 5.58 perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply 4.58 ± 3% -0.8 3.80 perf-profile.calltrace.cycles-pp.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.__vfs_write 2.64 ± 4% -0.8 1.88 perf-profile.calltrace.cycles-pp.xfs_file_aio_write_checks.xfs_file_buffered_aio_write.__vfs_write.vfs_write.ksys_write 3.35 ± 2% -0.6 2.72 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 4.34 -0.6 3.73 perf-profile.calltrace.cycles-pp.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 3.63 -0.6 3.03 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 3.05 ± 3% -0.5 2.55 perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dentry_kill.dput.__fput 2.74 ± 3% -0.5 2.25 perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor 3.01 ± 3% -0.5 2.53 perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.evict.__dentry_kill.dentry_kill.dput 1.36 ± 5% -0.5 0.90 perf-profile.calltrace.cycles-pp.file_update_time.xfs_file_aio_write_checks.xfs_file_buffered_aio_write.__vfs_write.vfs_write 2.14 ± 2% -0.4 1.76 perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 1.71 ± 3% -0.4 1.33 perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor 1.12 ± 12% -0.4 0.74 perf-profile.calltrace.cycles-pp.xfs_ilock.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write 1.66 ± 4% -0.3 1.31 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply 1.72 ± 4% -0.3 1.38 perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write 2.09 ± 4% -0.3 1.77 perf-profile.calltrace.cycles-pp.iomap_read_page_sync.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write 0.92 ± 15% -0.3 0.60 perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.92 ± 4% -0.3 0.63 perf-profile.calltrace.cycles-pp.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat 0.78 ± 19% -0.3 0.50 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.45 ± 5% -0.3 1.17 perf-profile.calltrace.cycles-pp.memset_erms.iomap_read_page_sync.iomap_write_begin.iomap_write_actor.iomap_apply 0.79 ± 8% -0.3 0.52 perf-profile.calltrace.cycles-pp.xfs_ilock.xfs_file_buffered_aio_write.__vfs_write.vfs_write.ksys_write 1.38 ± 3% -0.3 1.12 perf-profile.calltrace.cycles-pp.__generic_write_end.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write 0.91 ± 4% -0.3 0.65 perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.evict.__dentry_kill.dentry_kill 0.87 ± 4% -0.2 0.62 perf-profile.calltrace.cycles-pp.down_write.xfs_ilock.xfs_file_iomap_begin.iomap_apply.iomap_file_buffered_write 1.58 -0.2 1.33 perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin 1.03 -0.2 0.79 perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin 0.87 ± 4% -0.2 0.63 perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.evict.__dentry_kill 1.04 ± 5% -0.2 0.82 perf-profile.calltrace.cycles-pp.__lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin 0.95 ± 6% -0.2 0.74 perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.__lru_cache_add.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin 1.41 ± 3% -0.2 1.21 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor 1.85 -0.2 1.67 perf-profile.calltrace.cycles-pp.iomap_set_page_dirty.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write 0.90 ± 2% -0.1 0.77 perf-profile.calltrace.cycles-pp.__set_page_dirty.iomap_set_page_dirty.iomap_write_end.iomap_write_actor.iomap_apply 0.90 ± 5% -0.1 0.78 perf-profile.calltrace.cycles-pp.iomap_set_range_uptodate.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write 0.89 ± 2% -0.1 0.77 perf-profile.calltrace.cycles-pp.delete_from_page_cache_batch.truncate_inode_pages_range.evict.__dentry_kill.dentry_kill 0.71 ± 7% -0.1 0.61 perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.do_unlinkat 0.63 ± 6% -0.1 0.54 perf-profile.calltrace.cycles-pp.__cancel_dirty_page.truncate_cleanup_page.truncate_inode_pages_range.evict.__dentry_kill 1.42 ± 2% -0.1 1.33 perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.37 ± 3% -0.1 1.30 perf-profile.calltrace.cycles-pp.xfs_remove.xfs_vn_unlink.vfs_unlink.do_unlinkat.do_syscall_64 1.37 ± 3% -0.1 1.31 perf-profile.calltrace.cycles-pp.xfs_vn_unlink.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.68 ± 5% +0.1 0.76 perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat 0.75 ± 6% +0.3 1.10 perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.69 ± 11% +0.4 1.14 perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.14 ±173% +0.5 0.61 perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.46 ± 59% +0.5 0.96 perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 1.15 ± 5% +0.5 1.67 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.55 ± 4% +0.5 1.10 perf-profile.calltrace.cycles-pp.xfs_ialloc.xfs_dir_ialloc.xfs_create.xfs_generic_create.path_openat 0.56 ± 4% +0.6 1.11 perf-profile.calltrace.cycles-pp.xfs_dir_ialloc.xfs_create.xfs_generic_create.path_openat.do_filp_open 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 1.39 ± 2% +0.6 2.03 perf-profile.calltrace.cycles-pp.xfs_generic_create.path_openat.do_filp_open.do_sys_open.do_syscall_64 1.34 ± 2% +0.7 2.00 perf-profile.calltrace.cycles-pp.xfs_create.xfs_generic_create.path_openat.do_filp_open.do_sys_open 0.00 +0.7 0.66 perf-profile.calltrace.cycles-pp.__inode_security_revalidate.selinux_file_permission.security_file_permission.vfs_write.ksys_write 0.00 +0.7 0.67 perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree 0.00 +0.8 0.75 perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc 0.65 ± 7% +0.8 1.40 perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +0.8 0.77 perf-profile.calltrace.cycles-pp.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive 0.00 +0.8 0.84 perf-profile.calltrace.cycles-pp.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode 0.13 ±173% +0.9 1.00 perf-profile.calltrace.cycles-pp.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc.xfs_create.xfs_generic_create 0.00 +0.9 0.87 perf-profile.calltrace.cycles-pp.xfs_dialloc_ag.xfs_dialloc.xfs_ialloc.xfs_dir_ialloc.xfs_create 0.00 +1.0 1.02 perf-profile.calltrace.cycles-pp.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode.xfs_reclaim_inodes_pag 2.08 ± 8% +1.1 3.21 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 2.21 ± 7% +1.2 3.41 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 42.53 ± 2% +1.2 43.75 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 42.55 ± 2% +1.2 43.78 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 42.55 ± 2% +1.2 43.78 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 0.00 +1.3 1.31 perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_inactive_inode.xfs_reclaim_inodes_pag.xfs_inactive_inodes 0.00 +1.5 1.46 perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_inactive_inode.xfs_reclaim_inodes_pag.xfs_inactive_inodes.xfs_inactive_worker 0.00 +1.5 1.48 perf-profile.calltrace.cycles-pp.xfs_inactive_worker.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +1.5 1.48 perf-profile.calltrace.cycles-pp.xfs_inactive_inodes.xfs_inactive_worker.process_one_work.worker_thread.kthread 0.00 +1.5 1.48 perf-profile.calltrace.cycles-pp.xfs_reclaim_inodes_pag.xfs_inactive_inodes.xfs_inactive_worker.process_one_work.worker_thread 0.00 +1.5 1.48 perf-profile.calltrace.cycles-pp.xfs_inactive_inode.xfs_reclaim_inodes_pag.xfs_inactive_inodes.xfs_inactive_worker.process_one_work 2.40 ± 3% +1.5 3.90 perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.81 ± 6% +1.6 2.40 perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.81 ± 6% +1.6 2.40 perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.do_unlinkat.do_syscall_64 0.82 ± 6% +1.6 2.41 perf-profile.calltrace.cycles-pp.down_write.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.7 1.71 perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat 0.00 +1.7 1.71 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +1.7 1.72 perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.do_unlinkat 0.00 +1.7 1.72 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 0.00 +1.8 1.75 perf-profile.calltrace.cycles-pp.ret_from_fork 0.00 +1.8 1.75 perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.76 ± 5% +1.8 2.54 perf-profile.calltrace.cycles-pp.down_write.path_openat.do_filp_open.do_sys_open.do_syscall_64 0.76 ± 5% +1.8 2.54 perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open.do_sys_open 0.75 ± 5% +1.8 2.54 perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open 0.52 ±173% +2.0 2.49 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_kernel 0.55 ±173% +2.2 2.70 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 0.57 ±173% +2.2 2.82 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 0.57 ±173% +2.2 2.82 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64 0.57 ±173% +2.2 2.82 perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64 2.72 ± 2% +2.4 5.16 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.72 +2.4 5.16 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.75 ± 2% +2.5 5.20 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 43.11 +3.5 46.60 perf-profile.calltrace.cycles-pp.secondary_startup_64 aim7.jobs-per-min 200000 +-+----------------------------------------------------------------+ 180000 +-+.+..+.+.+.+..+.+.+.+..+ +.+..+.+.+.+.+..+.+.+.+..+.+.+.+..+.+.| | : : | 160000 +-+ : : | 140000 O-O O O O O O O O O O O O O O O O O O O | | : : | 120000 +-+ : : | 100000 +-+ : : | 80000 +-+ : : | | : : | 60000 +-+ : : | 40000 +-+ : | | : | 20000 +-+ : | 0 +-+---------------------------------------O------------------------+ aim7.time.user_time 140 +-+-------------------------------------------------------------------+ | | 120 +-+ + | | .+..+.+.+..+ +..+. .+..+. .+.. + + .+.. | 100 +-+..+.+.+..+.+ : : + + + +..+.+.+..+.+ +.| O O O O O O O O O O O O O O O O O O O | 80 +-+ O : : | | : : | 60 +-+ : : | | : : | 40 +-+ : : | | : : | 20 +-+ : O | | : O O | 0 +-+-------------------------------------------------------------------+ aim7.time.elapsed_time 140 +-+-------------------------------------------------------------------+ | O O O O O O O O O O O O O O | 120 O-+ O O O O O | | | 100 +-+..+.+.+..+.+.+..+.+.+..+ +..+.+.+..+.+.+..+.+.+..+.+.+..+.+.+..+.| | : : | 80 +-+ : : | | : : | 60 +-+ : : | | : : | 40 +-+ : : | | : : O | 20 +-+ : O | | : O | 0 +-+-------------------------------------------------------------------+ aim7.time.elapsed_time.max 140 +-+-------------------------------------------------------------------+ | O O O O O O O O O O O O O O | 120 O-+ O O O O O | | | 100 +-+..+.+.+..+.+.+..+.+.+..+ +..+.+.+..+.+.+..+.+.+..+.+.+..+.+.+..+.| | : : | 80 +-+ : : | | : : | 60 +-+ : : | | : : | 40 +-+ : : | | : : O | 20 +-+ : O | | : O | 0 +-+-------------------------------------------------------------------+ aim7.time.voluntary_context_switches 1.2e+06 +-+---------------------------------------------------------------+ |.+.+..+.+.+.+.+..+.+.+.+ +.+.+.+.+..+.+.+.+.+..+.+.+.+.+..+.+.| 1e+06 +-+ : : | | : : | O O O O O O O O O O O O O :O O O O O O O | 800000 +-+ : : | | : : | 600000 +-+ : : | | : : | 400000 +-+ : : | | : : | | :: | 200000 +-+ : O | | : O O | 0 +-+---------------------------------------------------------------+ aim7.time.involuntary_context_switches 16000 +-+-----------------------------------------------------------------+ | O | 14000 +-+ | 12000 +-+ | | | 10000 +-+ | | | 8000 +-+ O | O O O | 6000 +-O O O O O O O O O | 4000 +-+ O O O O O O O | | .+. .+..+.+ +..+.+.+..+.+.+.+..+.+. .+..+. .+.+.. | 2000 +-+..+.+.+.+..+ + : : + + +.| | : : O O | 0 +-+-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-ivb-ep01: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase: gcc-7/performance/4BRD_12G/xfs/x86_64-rhel-7.2/3000/RAID0/debian-x86_64-2018-04-03.cgz/lkp-ivb-ep01/disk_cp/aim7 commit: 1410e363f1 ("xfs: refactor eofblocks inode match code") ea3e90ae01 ("xfs: deferred inode inactivation") 1410e363f1c8ee34 ea3e90ae01e5d776b06334d892 ---------------- -------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 246235 -34.4% 161597 aim7.jobs-per-min 74.13 ± 2% +51.0% 111.92 aim7.time.elapsed_time 74.13 ± 2% +51.0% 111.92 aim7.time.elapsed_time.max 72548 +50.4% 109137 ± 3% aim7.time.involuntary_context_switches 2120 +10.6% 2345 aim7.time.system_time 105.68 +31.5% 138.99 aim7.time.user_time 769040 -16.7% 640759 aim7.time.voluntary_context_switches 42554 ± 2% +27.9% 54446 interrupts.CAL:Function_call_interrupts 12.15 ± 21% +729.6% 100.77 iostat.md0.w/s 825.28 ± 42% +230.6% 2728 ± 3% iostat.md0.wkB/s 5.08e+08 ± 17% +193.6% 1.492e+09 ± 9% cpuidle.C6.time 786744 ± 8% +225.7% 2562270 ± 22% cpuidle.C6.usage 14557 ± 9% +77.1% 25776 ± 8% cpuidle.POLL.time 26.52 ± 4% +17.6 44.15 mpstat.cpu.idle% 69.74 -17.2 52.57 mpstat.cpu.sys% 3.68 ± 3% -0.4 3.24 mpstat.cpu.usr% 425707 ± 2% +45.1% 617877 ± 5% softirqs.RCU 334218 ± 2% +36.8% 457343 softirqs.SCHED 1239373 +54.3% 1912002 ± 4% softirqs.TIMER 803.25 ± 43% +225.2% 2612 ± 3% vmstat.io.bo 28.75 ± 2% -12.2% 25.25 ± 3% vmstat.procs.r 20992 -15.2% 17803 vmstat.system.cs 8353 ± 6% -28.0% 6012 ± 3% numa-meminfo.node0.Dirty 8486 ± 7% -28.8% 6044 ± 5% numa-meminfo.node0.Inactive(file) 67740 ± 44% +37.6% 93210 ± 39% numa-meminfo.node0.SUnreclaim 8225 ± 4% -24.9% 6178 ± 7% numa-meminfo.node1.Dirty 8240 ± 3% -23.9% 6274 ± 5% numa-meminfo.node1.Inactive(file) 68342 ± 3% +44.8% 98943 ± 2% meminfo.AnonHugePages 16729 ± 5% -27.3% 12156 ± 3% meminfo.Dirty 36652 ± 2% -13.5% 31688 meminfo.Inactive 16891 ± 5% -27.1% 12310 ± 3% meminfo.Inactive(file) 81220 +30.7% 106154 meminfo.KReclaimable 81220 +30.7% 106154 meminfo.SReclaimable 161182 +13.5% 182969 meminfo.SUnreclaim 242403 +19.3% 289124 meminfo.Slab 30636 ± 2% -30.4% 21337 meminfo.max_used_kB 2161 ± 2% -43.3% 1226 turbostat.Avg_MHz 75.00 -17.3 57.74 turbostat.Busy% 2881 -26.3% 2124 turbostat.Bzy_MHz 783741 ± 8% +226.5% 2558841 ± 22% turbostat.C6 16.72 ± 16% +16.2 32.89 ± 9% turbostat.C6% 14.65 ± 10% +102.5% 29.67 ± 9% turbostat.CPU%c1 118.64 ± 2% -33.2% 79.29 ± 2% turbostat.CorWatt 6276593 ± 3% +48.1% 9292777 turbostat.IRQ 6.12 ± 10% -57.8% 2.58 ± 16% turbostat.Pkg%pc2 146.28 -27.4% 106.21 turbostat.PkgWatt 38.88 -9.2% 35.30 turbostat.RAMWatt 6050 ± 2% +42.1% 8600 turbostat.SMI 15441029 ± 5% +17.3% 18110417 ± 3% numa-vmstat.node0.nr_dirtied 2004 ± 7% -26.4% 1475 ± 7% numa-vmstat.node0.nr_dirty 2056 ± 7% -27.5% 1491 ± 9% numa-vmstat.node0.nr_inactive_file 16930 ± 44% +37.7% 23309 ± 39% numa-vmstat.node0.nr_slab_unreclaimable 2056 ± 7% -27.6% 1488 ± 9% numa-vmstat.node0.nr_zone_inactive_file 1705 ± 6% -18.0% 1398 ± 9% numa-vmstat.node0.nr_zone_write_pending 15942385 ± 5% +16.8% 18618196 ± 3% numa-vmstat.node0.numa_hit 15909481 ± 5% +16.8% 18585265 ± 3% numa-vmstat.node0.numa_local 16135495 ± 2% +11.3% 17961471 ± 4% numa-vmstat.node1.nr_dirtied 2026 ± 3% -27.0% 1478 numa-vmstat.node1.nr_dirty 2016 ± 3% -25.5% 1502 ± 3% numa-vmstat.node1.nr_inactive_file 2013 ± 3% -25.5% 1500 ± 3% numa-vmstat.node1.nr_zone_inactive_file 1766 -21.0% 1395 ± 3% numa-vmstat.node1.nr_zone_write_pending 16554298 ± 2% +11.4% 18436270 ± 5% numa-vmstat.node1.numa_hit 16429705 ± 2% +11.5% 18311887 ± 5% numa-vmstat.node1.numa_local 278.25 ± 2% +4.9% 291.75 proc-vmstat.nr_active_file 4119 ± 4% -27.8% 2975 ± 3% proc-vmstat.nr_dirty 4940 -2.0% 4841 proc-vmstat.nr_inactive_anon 4155 ± 4% -27.5% 3014 ± 2% proc-vmstat.nr_inactive_file 52063 -6.7% 48571 proc-vmstat.nr_kernel_stack 7623 ± 6% -7.9% 7017 ± 3% proc-vmstat.nr_shmem 20299 +30.7% 26534 proc-vmstat.nr_slab_reclaimable 40282 +13.5% 45711 proc-vmstat.nr_slab_unreclaimable 278.25 ± 2% +4.9% 291.75 proc-vmstat.nr_zone_active_file 4940 -2.0% 4841 proc-vmstat.nr_zone_inactive_anon 4155 ± 4% -27.5% 3013 ± 2% proc-vmstat.nr_zone_inactive_file 3539 ± 2% -20.4% 2817 ± 3% proc-vmstat.nr_zone_write_pending 3784 ± 17% -32.5% 2555 ± 10% proc-vmstat.pgactivate 364648 +25.4% 457274 proc-vmstat.pgfault 61924 ± 44% +384.4% 299936 ± 3% proc-vmstat.pgpgout 1318 ± 3% +12.5% 1484 slabinfo.Acpi-ParseExt.active_objs 1318 ± 3% +12.5% 1484 slabinfo.Acpi-ParseExt.num_objs 2487 ± 2% +571.7% 16709 slabinfo.dmaengine-unmap-16.active_objs 61.50 ± 5% +928.5% 632.50 ± 2% slabinfo.dmaengine-unmap-16.active_slabs 2603 ± 5% +921.0% 26576 ± 2% slabinfo.dmaengine-unmap-16.num_objs 61.50 ± 5% +928.5% 632.50 ± 2% slabinfo.dmaengine-unmap-16.num_slabs 1838 ± 3% +23.1% 2263 slabinfo.ebitmap_node.active_objs 1838 ± 3% +23.1% 2263 slabinfo.ebitmap_node.num_objs 2214 ± 3% +67.9% 3718 ± 2% slabinfo.kmalloc-128.active_objs 2219 ± 2% +67.6% 3718 ± 2% slabinfo.kmalloc-128.num_objs 19264 +104.4% 39366 slabinfo.kmalloc-16.active_objs 19264 +123.6% 43076 slabinfo.kmalloc-16.num_objs 752.50 +19.2% 897.25 ± 2% slabinfo.kmalloc-4k.active_objs 761.25 +21.1% 922.25 ± 3% slabinfo.kmalloc-4k.num_objs 6140 +305.5% 24898 slabinfo.kmalloc-512.active_objs 201.25 +385.0% 976.00 slabinfo.kmalloc-512.active_slabs 6456 +384.0% 31246 slabinfo.kmalloc-512.num_objs 201.25 +385.0% 976.00 slabinfo.kmalloc-512.num_slabs 453.25 +100.8% 910.00 slabinfo.kmalloc-8k.active_objs 116.50 +110.9% 245.75 slabinfo.kmalloc-8k.active_slabs 467.50 +110.5% 984.00 slabinfo.kmalloc-8k.num_objs 116.50 +110.9% 245.75 slabinfo.kmalloc-8k.num_slabs 1495 ± 5% +24.3% 1858 slabinfo.mnt_cache.active_objs 1495 ± 5% +24.3% 1858 slabinfo.mnt_cache.num_objs 17764 +14.3% 20299 slabinfo.radix_tree_node.active_objs 17866 +14.8% 20510 slabinfo.radix_tree_node.num_objs 1121 +91.8% 2150 ± 6% slabinfo.xfs_buf_item.active_objs 1121 +91.8% 2150 ± 6% slabinfo.xfs_buf_item.num_objs 1041 ± 3% +164.8% 2758 ± 2% slabinfo.xfs_efd_item.active_objs 1041 ± 3% +164.8% 2758 ± 2% slabinfo.xfs_efd_item.num_objs 2166 ± 2% +651.2% 16272 slabinfo.xfs_inode.active_objs 66.50 +1061.7% 772.50 ± 2% slabinfo.xfs_inode.active_slabs 2283 +1051.1% 26281 ± 2% slabinfo.xfs_inode.num_objs 66.50 +1061.7% 772.50 ± 2% slabinfo.xfs_inode.num_slabs 15.17 ± 11% -22.6% 11.73 ± 7% perf-stat.i.MPKI 5.136e+09 -26.7% 3.767e+09 perf-stat.i.branch-instructions 2.62 ± 6% -0.4 2.19 ± 4% perf-stat.i.branch-miss-rate% 72620415 ± 3% -22.2% 56513455 perf-stat.i.branch-misses 41990561 -30.7% 29088990 perf-stat.i.cache-misses 2.722e+08 -34.5% 1.781e+08 perf-stat.i.cache-references 21460 -15.8% 18064 perf-stat.i.context-switches 3.59 ± 4% -29.6% 2.53 ± 3% perf-stat.i.cpi 8.628e+10 ± 2% -42.9% 4.928e+10 perf-stat.i.cpu-cycles 2351 ± 2% -64.5% 835.66 ± 2% perf-stat.i.cpu-migrations 1866 -21.5% 1464 perf-stat.i.cycles-between-cache-misses 1.557e+08 ± 19% -28.8% 1.109e+08 ± 13% perf-stat.i.dTLB-load-misses 7.951e+09 ± 2% -26.7% 5.828e+09 perf-stat.i.dTLB-loads 5.176e+09 ± 2% -29.4% 3.654e+09 perf-stat.i.dTLB-stores 87.76 -6.8 81.01 ± 5% perf-stat.i.iTLB-load-miss-rate% 2.571e+10 -26.5% 1.891e+10 perf-stat.i.instructions 0.31 ± 5% +52.5% 0.47 perf-stat.i.ipc 4602 ± 2% -15.4% 3892 perf-stat.i.minor-faults 15075092 ± 2% -40.3% 8992397 perf-stat.i.node-load-misses 17489617 ± 2% -39.6% 10556012 perf-stat.i.node-loads 16554564 ± 2% -31.5% 11347620 perf-stat.i.node-store-misses 22193571 ± 2% -29.6% 15619967 perf-stat.i.node-stores 4602 ± 2% -15.4% 3892 perf-stat.i.page-faults 10.58 -11.0% 9.42 perf-stat.overall.MPKI 1.41 ± 4% +0.1 1.50 perf-stat.overall.branch-miss-rate% 15.43 +0.9 16.33 perf-stat.overall.cache-miss-rate% 3.36 -22.3% 2.61 perf-stat.overall.cpi 2054 -17.6% 1693 perf-stat.overall.cycles-between-cache-misses 87.31 -6.4 80.89 ± 6% perf-stat.overall.iTLB-load-miss-rate% 0.30 +28.8% 0.38 perf-stat.overall.ipc 42.72 -0.6 42.08 perf-stat.overall.node-store-miss-rate% 5.065e+09 -26.3% 3.732e+09 perf-stat.ps.branch-instructions 71633997 ± 3% -21.8% 56003741 perf-stat.ps.branch-misses 41409137 -30.4% 28818359 perf-stat.ps.cache-misses 2.684e+08 -34.2% 1.765e+08 perf-stat.ps.cache-references 21162 -15.4% 17902 perf-stat.ps.context-switches 8.508e+10 ± 2% -42.6% 4.881e+10 perf-stat.ps.cpu-cycles 2318 ± 2% -64.3% 827.96 ± 2% perf-stat.ps.cpu-migrations 1.535e+08 ± 19% -28.4% 1.099e+08 ± 13% perf-stat.ps.dTLB-load-misses 7.841e+09 ± 2% -26.4% 5.774e+09 perf-stat.ps.dTLB-loads 5.104e+09 ± 2% -29.1% 3.62e+09 perf-stat.ps.dTLB-stores 2.536e+10 -26.1% 1.873e+10 perf-stat.ps.instructions 4542 ± 2% -15.1% 3857 perf-stat.ps.minor-faults 14865391 ± 2% -40.1% 8908375 perf-stat.ps.node-load-misses 17246354 ± 2% -39.4% 10457333 perf-stat.ps.node-loads 16324282 ± 2% -31.1% 11241253 perf-stat.ps.node-store-misses 21885039 ± 2% -29.3% 15473569 perf-stat.ps.node-stores 4542 ± 2% -15.1% 3857 perf-stat.ps.page-faults 1.9e+12 +10.8% 2.104e+12 perf-stat.total.instructions 21284 ± 2% -26.0% 15759 sched_debug.cfs_rq:/.exec_clock.avg 23850 ± 2% -23.3% 18285 sched_debug.cfs_rq:/.exec_clock.max 21005 ± 2% -27.2% 15294 sched_debug.cfs_rq:/.exec_clock.min 50861 ± 5% +1011.8% 565458 ± 90% sched_debug.cfs_rq:/.load.max 63.27 ± 14% +212.1% 197.43 ± 52% sched_debug.cfs_rq:/.load_avg.avg 638.88 ± 24% +631.7% 4674 ± 83% sched_debug.cfs_rq:/.load_avg.max 138.17 ± 16% +479.8% 801.09 ± 76% sched_debug.cfs_rq:/.load_avg.stddev 760165 ± 3% -34.8% 495838 sched_debug.cfs_rq:/.min_vruntime.avg 846487 ± 2% -28.3% 606585 ± 2% sched_debug.cfs_rq:/.min_vruntime.max 741279 ± 3% -37.1% 466061 sched_debug.cfs_rq:/.min_vruntime.min 16779 ± 11% +33.7% 22431 ± 10% sched_debug.cfs_rq:/.min_vruntime.stddev 35429 ± 19% +1449.4% 548957 ± 92% sched_debug.cfs_rq:/.runnable_weight.max 86926 ± 25% +33.6% 116140 ± 8% sched_debug.cfs_rq:/.spread0.max 16774 ± 11% +33.4% 22374 ± 10% sched_debug.cfs_rq:/.spread0.stddev 152.58 ± 10% -57.7% 64.56 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg 170.31 ± 11% -24.0% 129.42 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.stddev 414415 ± 2% +44.4% 598518 ± 4% sched_debug.cpu.avg_idle.avg 9.49 ± 6% +91.1% 18.14 ± 23% sched_debug.cpu.clock.stddev 9.49 ± 6% +91.1% 18.14 ± 23% sched_debug.cpu.clock_task.stddev 12.45 ± 5% +46.5% 18.23 ± 15% sched_debug.cpu.cpu_load[0].avg 37.75 ± 12%