November 2020 - LKP - Ml01.01.Org

[btrfs] 96bed17ad9: fio.write_iops -59.7% regression

by kernel test robot

Greeting, FYI, we noticed a -59.7% regression of fio.write_iops due to commit: commit: 96bed17ad9d425ff6958a2e6f87179453a3d76f2 ("btrfs: simplify the logic in need_preemptive_flushing") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: fio-basic on test machine: 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory with following parameters: disk: 1SSD fs: btrfs runtime: 300s nr_task: 8 rw: write bs: 4k ioengine: sync test_size: 256g cpufreq_governor: performance ucode: 0x4002f01 test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user. test-url: https://github.com/axboe/fio If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/ucode: 4k/gcc-9/performance/1SSD/btrfs/sync/x86_64-rhel-8.3/8/debian-10.4-x86_64-20200603.cgz/300s/write/lkp-csl-2ap1/256g/fio-basic/0x4002f01 commit: 77adc5cb16 ("btrfs: rework btrfs_calc_reclaim_metadata_size") 96bed17ad9 ("btrfs: simplify the logic in need_preemptive_flushing") 77adc5cb164dcf5d 96bed17ad9d425ff6958a2e6f87 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 kmsg.ACPI_BIOS_Error(bug) 1:4 -25% :4 kmsg.ACPI_Error %stddev %change %stddev \ | \ 0.01 +1.4 1.39 ± 48% fio.latency_100us% 49.66 ± 7% -48.9 0.75 ±168% fio.latency_10us% 36.19 ± 10% -35.2 1.04 ±169% fio.latency_20us% 0.01 -0.0 0.00 fio.latency_2us% 13.28 ± 15% -13.3 0.01 ± 57% fio.latency_4us% 0.03 ± 9% -0.0 0.00 ±173% fio.latency_50ms% 0.83 ± 15% +96.0 96.82 ± 2% fio.latency_50us% 129.75 ± 2% +132.3% 301.45 fio.time.elapsed_time 129.75 ± 2% +132.3% 301.45 fio.time.elapsed_time.max 5.369e+08 -5.4% 5.078e+08 fio.time.file_system_outputs 1925 ± 7% +255.6% 6846 ± 8% fio.time.involuntary_context_switches 482.00 ± 5% +66.0% 800.25 fio.time.percent_of_cpu_this_job_got 564.84 ± 4% +317.3% 2356 fio.time.system_time 67108864 -5.4% 63471135 fio.workload 2048 ± 2% -59.7% 826.44 fio.write_bw_MBps 13376 ± 2% +221.5% 43008 fio.write_clat_90%_us 15072 ± 2% +201.5% 45440 fio.write_clat_95%_us 19520 ± 2% +160.3% 50816 ± 3% fio.write_clat_99%_us 14866 ± 2% +152.4% 37514 fio.write_clat_mean_us 360348 ± 5% -98.3% 5952 ± 21% fio.write_clat_stddev 524538 ± 2% -59.7% 211569 fio.write_iops 34.69 -0.9% 34.38 boot-time.boot 1.97 ± 6% -92.3% 0.15 ± 6% iostat.cpu.iowait 3.90 ± 4% +29.1% 5.03 iostat.cpu.system 168.55 ± 2% +101.7% 339.96 uptime.boot 29866 ± 2% +105.1% 61259 uptime.idle 1.99 ± 6% -1.8 0.15 ± 6% mpstat.cpu.all.iowait% 1.03 ± 4% -0.3 0.72 ± 10% mpstat.cpu.all.irq% 2.82 ± 6% +1.4 4.26 mpstat.cpu.all.sys% 0.29 ± 6% -0.2 0.12 mpstat.cpu.all.usr% 93.00 +1.1% 94.00 vmstat.cpu.id 1805132 ± 2% -56.5% 785909 ± 4% vmstat.io.bo 65013138 +11.3% 72328489 ± 4% vmstat.memory.free 3.75 ± 11% -100.0% 0.00 vmstat.procs.b 6.50 ± 7% +23.1% 8.00 vmstat.procs.r 49100 -72.8% 13352 ± 3% vmstat.system.cs 394325 -9.7% 356099 ± 16% vmstat.system.in 2064649 ±100% -100.0% 0.00 numa-numastat.node0.numa_foreign 1469474 ± 45% -98.9% 16866 ±102% numa-numastat.node0.numa_miss 1485006 ± 44% -98.3% 24665 ±103% numa-numastat.node0.other_node 2998955 ± 87% -92.5% 224907 ±132% numa-numastat.node1.numa_foreign 2047311 ±100% -100.0% 0.00 numa-numastat.node1.numa_miss 2078424 ± 98% -98.9% 23355 ± 57% numa-numastat.node1.other_node 2691737 ± 77% -91.6% 224907 ±132% numa-numastat.node2.numa_miss 2715144 ± 76% -90.6% 256010 ±116% numa-numastat.node2.other_node 1449274 ± 52% -98.8% 16866 ±102% numa-numastat.node3.numa_foreign 115940 +20.5% 139716 ± 3% meminfo.Active 3895 +284.7% 14986 ± 4% meminfo.Active(anon) 112045 +11.3% 124729 ± 3% meminfo.Active(file) 111136 ± 12% +48.7% 165206 meminfo.AnonHugePages 358017 ± 4% -15.3% 303181 ± 4% meminfo.AnonPages 65473338 +10.9% 72609414 ± 4% meminfo.MemFree 12802 ± 9% -20.4% 10188 ± 6% meminfo.PageTables 344826 -10.5% 308454 meminfo.SUnreclaim 19443863 -86.7% 2586903 ± 4% meminfo.Writeback 1499568 ± 2% -56.6% 650338 meminfo.max_used_kB 2900 ± 60% +985.9% 31493 ± 35% numa-meminfo.node0.Active 1167 ± 91% +2406.3% 29267 ± 38% numa-meminfo.node0.Active(file) 17540 ± 54% +363.7% 81329 ± 47% numa-meminfo.node0.AnonHugePages 89292 ± 33% +78.4% 159337 ± 36% numa-meminfo.node0.AnonPages 4775891 ± 6% -86.8% 629756 ± 5% numa-meminfo.node0.Writeback 5027255 ± 8% -86.9% 658431 ± 6% numa-meminfo.node1.Writeback 2389 ±105% +1897.0% 47724 ± 60% numa-meminfo.node2.Active 1356 ±122% +3343.6% 46721 ± 61% numa-meminfo.node2.Active(file) 100260 ± 60% -71.1% 28948 ± 78% numa-meminfo.node2.AnonPages 5047201 ± 7% -86.9% 661860 ± 6% numa-meminfo.node2.Writeback 399.75 ± 85% +2606.2% 10818 ± 14% numa-meminfo.node3.Active(anon) 283116 ±167% -97.2% 7930 ± 32% numa-meminfo.node3.Mapped 82365 ± 10% -23.8% 62787 ± 3% numa-meminfo.node3.SUnreclaim 4595302 ± 4% -86.1% 639095 ± 2% numa-meminfo.node3.Writeback 1021 ± 4% -40.2% 610.75 ± 8% slabinfo.biovec-max.active_objs 1114 ± 5% -35.4% 720.00 ± 7% slabinfo.biovec-max.num_objs 7746 ± 3% +19.7% 9269 ± 2% slabinfo.btrfs_extent_buffer.active_objs 7870 ± 2% +18.0% 9289 ± 2% slabinfo.btrfs_extent_buffer.num_objs 1694 ± 3% +7.9% 1828 ± 5% slabinfo.btrfs_ordered_extent.active_objs 1694 ± 3% +7.9% 1828 ± 5% slabinfo.btrfs_ordered_extent.num_objs 2033 ± 5% +39.5% 2835 ± 9% slabinfo.buffer_head.active_objs 2055 ± 5% +38.1% 2838 ± 9% slabinfo.buffer_head.num_objs 4933 ± 15% +125.3% 11117 ± 17% slabinfo.dmaengine-unmap-16.active_objs 117.50 ± 15% +125.1% 264.50 ± 17% slabinfo.dmaengine-unmap-16.active_slabs 4954 ± 15% +124.7% 11134 ± 17% slabinfo.dmaengine-unmap-16.num_objs 117.50 ± 15% +125.1% 264.50 ± 17% slabinfo.dmaengine-unmap-16.num_slabs 9462 ± 3% +14.1% 10795 ± 8% slabinfo.fsnotify_mark_connector.num_objs 588433 -8.5% 538563 ± 3% slabinfo.radix_tree_node.active_objs 10507 -8.5% 9616 ± 3% slabinfo.radix_tree_node.active_slabs 588440 -8.5% 538566 ± 3% slabinfo.radix_tree_node.num_objs 10507 -8.5% 9616 ± 3% slabinfo.radix_tree_node.num_slabs 3410 ± 3% +14.5% 3905 ± 3% slabinfo.trace_event_file.active_objs 3423 ± 3% +14.9% 3934 ± 4% slabinfo.trace_event_file.num_objs 20035 +23.6% 24756 slabinfo.vmap_area.active_objs 20100 +23.3% 24792 slabinfo.vmap_area.num_objs 295.50 ± 92% +2375.1% 7314 ± 38% numa-vmstat.node0.nr_active_file 22328 ± 33% +78.6% 39875 ± 36% numa-vmstat.node0.nr_anon_pages 1195604 ± 6% -86.8% 157291 ± 5% numa-vmstat.node0.nr_writeback 296.25 ± 93% +2369.0% 7314 ± 38% numa-vmstat.node0.nr_zone_active_file 1861506 ± 5% -57.6% 788811 ± 3% numa-vmstat.node0.nr_zone_write_pending 907498 ± 74% -100.0% 0.00 numa-vmstat.node0.numa_foreign 338047 ± 41% -99.3% 2248 ±114% numa-vmstat.node0.numa_miss 388008 ± 35% -89.1% 42425 ± 58% numa-vmstat.node0.numa_other 1252201 ± 8% -86.9% 164375 ± 6% numa-vmstat.node1.nr_writeback 1988661 ± 9% -59.4% 807812 ± 3% numa-vmstat.node1.nr_zone_write_pending 1478486 ± 93% -99.3% 10639 ±144% numa-vmstat.node1.numa_foreign 925105 ± 78% -100.0% 0.00 numa-vmstat.node1.numa_miss 1041053 ± 69% -89.6% 108743 ± 12% numa-vmstat.node1.numa_other 335.25 ±122% +3385.5% 11685 ± 61% numa-vmstat.node2.nr_active_file 25023 ± 60% -71.1% 7241 ± 78% numa-vmstat.node2.nr_anon_pages 1263373 ± 7% -86.9% 165494 ± 6% numa-vmstat.node2.nr_writeback 336.75 ±122% +3370.4% 11686 ± 61% numa-vmstat.node2.nr_zone_active_file 1945998 ± 8% -58.7% 804113 numa-vmstat.node2.nr_zone_write_pending 1347960 ± 88% -99.2% 10642 ±144% numa-vmstat.node2.numa_miss 1456839 ± 81% -92.3% 111463 ± 32% numa-vmstat.node2.numa_other 99.25 ± 85% +2629.7% 2709 ± 14% numa-vmstat.node3.nr_active_anon 20598 ± 10% -23.8% 15695 ± 3% numa-vmstat.node3.nr_slab_unreclaimable 1149987 ± 4% -86.1% 159664 ± 2% numa-vmstat.node3.nr_writeback 6127597 ± 7% +17.9% 7222770 ± 6% numa-vmstat.node3.nr_written 99.25 ± 85% +2629.7% 2709 ± 14% numa-vmstat.node3.nr_zone_active_anon 1830268 ± 3% -56.5% 796492 numa-vmstat.node3.nr_zone_write_pending 348039 ± 49% -99.4% 2248 ±114% numa-vmstat.node3.numa_foreign 590644 ± 34% -69.9% 177863 ± 9% proc-vmstat.compact_daemon_migrate_scanned 1079 ± 19% -57.2% 461.75 ± 4% proc-vmstat.compact_daemon_wake 257044 ± 42% -87.0% 33541 ± 21% proc-vmstat.compact_isolated 786837 ± 23% -58.4% 327500 ± 10% proc-vmstat.compact_migrate_scanned 554.00 ± 23% +118.5% 1210 ± 13% proc-vmstat.kswapd_high_wmark_hit_quickly 1612 ± 25% -45.0% 886.00 ± 18% proc-vmstat.kswapd_low_wmark_hit_quickly 973.25 +284.8% 3744 ± 4% proc-vmstat.nr_active_anon 28046 +11.2% 31197 ± 3% proc-vmstat.nr_active_file 89474 ± 4% -15.3% 75797 ± 4% proc-vmstat.nr_anon_pages 67188417 -5.4% 63554220 proc-vmstat.nr_dirtied 2765923 -7.8% 2550268 proc-vmstat.nr_dirty 32541918 -5.4% 30800890 ± 2% proc-vmstat.nr_file_pages 16346846 +11.0% 18143118 ± 4% proc-vmstat.nr_free_pages 367012 -2.6% 357316 proc-vmstat.nr_inactive_anon 31977972 -5.5% 30227089 ± 2% proc-vmstat.nr_inactive_file 283350 +1.3% 286901 proc-vmstat.nr_mapped 3194 ± 9% -20.6% 2534 ± 5% proc-vmstat.nr_page_table_pages 278659 +2.4% 285404 proc-vmstat.nr_shmem 105608 -6.0% 99255 ± 2% proc-vmstat.nr_slab_reclaimable 86209 -10.6% 77113 proc-vmstat.nr_slab_unreclaimable 4860968 -86.7% 646666 ± 4% proc-vmstat.nr_writeback 973.25 +284.8% 3744 ± 4% proc-vmstat.nr_zone_active_anon 28060 +11.2% 31201 ± 3% proc-vmstat.nr_zone_active_file 367029 -2.6% 357330 proc-vmstat.nr_zone_inactive_anon 31978036 -5.5% 30227107 ± 2% proc-vmstat.nr_zone_inactive_file 7627174 -58.1% 3197146 proc-vmstat.nr_zone_write_pending 8309580 ± 28% -96.6% 278711 ±103% proc-vmstat.numa_foreign 59708479 ± 3% +8.1% 64570979 proc-vmstat.numa_hit 59615003 ± 3% +8.2% 64477559 proc-vmstat.numa_local 8309580 ± 28% -96.6% 278711 ±103% proc-vmstat.numa_miss 8403056 ± 27% -95.6% 372131 ± 77% proc-vmstat.numa_other 13636 ± 2% +79.4% 24467 ± 2% proc-vmstat.pgactivate 68003378 -4.7% 64817961 proc-vmstat.pgalloc_normal 1087274 +58.2% 1719962 proc-vmstat.pgfault 20213698 -16.7% 16843776 ± 6% proc-vmstat.pgfree 123160 ± 41% -83.2% 20633 ± 21% proc-vmstat.pgmigrate_success 28853 ± 2% +126.8% 65426 ± 2% proc-vmstat.pgreuse 19006920 -19.6% 15280314 ± 7% proc-vmstat.pgscan_file 19001157 -19.6% 15280314 ± 7% proc-vmstat.pgscan_kswapd 19006880 -19.6% 15280274 ± 7% proc-vmstat.pgsteal_file 19001117 -19.6% 15280274 ± 7% proc-vmstat.pgsteal_kswapd 49968 ± 5% +17.5% 58735 proc-vmstat.slabs_scanned 52705 ± 4% -41.7% 30721 ± 14% proc-vmstat.workingset_nodes 1724 ± 37% +522.3% 10732 ± 13% sched_debug.cfs_rq:/.exec_clock.avg 26835 ± 28% +336.9% 117257 sched_debug.cfs_rq:/.exec_clock.max 71.42 ± 48% +86.3% 133.06 ± 14% sched_debug.cfs_rq:/.exec_clock.min 4531 ± 26% +522.7% 28217 ± 6% sched_debug.cfs_rq:/.exec_clock.stddev 41327 ± 75% +166.0% 109916 ± 15% sched_debug.cfs_rq:/.load.avg 707654 ± 16% +32.2% 935796 ± 9% sched_debug.cfs_rq:/.load.max 142326 ± 33% +84.2% 262137 ± 5% sched_debug.cfs_rq:/.load.stddev 82158 ± 14% +24.7% 102458 ± 8% sched_debug.cfs_rq:/.min_vruntime.avg 117042 ± 7% +85.2% 216774 ± 4% sched_debug.cfs_rq:/.min_vruntime.max 7933 ± 13% +275.2% 29765 ± 6% sched_debug.cfs_rq:/.min_vruntime.stddev 0.06 ± 48% +95.0% 0.12 ± 15% sched_debug.cfs_rq:/.nr_running.avg 0.23 ± 18% +32.7% 0.30 ± 6% sched_debug.cfs_rq:/.nr_running.stddev 77.84 ± 19% +67.5% 130.39 ± 13% sched_debug.cfs_rq:/.runnable_avg.avg 170.55 ± 12% +71.0% 291.57 ± 6% sched_debug.cfs_rq:/.runnable_avg.stddev 31274 ± 55% +261.8% 113161 ± 7% sched_debug.cfs_rq:/.spread0.max 7970 ± 13% +273.5% 29774 ± 6% sched_debug.cfs_rq:/.spread0.stddev 77.70 ± 19% +67.7% 130.32 ± 13% sched_debug.cfs_rq:/.util_avg.avg 170.33 ± 12% +71.1% 291.47 ± 6% sched_debug.cfs_rq:/.util_avg.stddev 20.44 ± 57% +372.5% 96.57 ± 15% sched_debug.cfs_rq:/.util_est_enqueued.avg 774.17 ± 4% +32.4% 1025 sched_debug.cfs_rq:/.util_est_enqueued.max 96.66 ± 21% +170.3% 261.27 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.stddev 2000666 ± 24% -29.4% 1411975 ± 7% sched_debug.cpu.avg_idle.max 28117 ± 22% +174.5% 77182 ± 23% sched_debug.cpu.avg_idle.min 245757 ± 13% -30.5% 170717 ± 9% sched_debug.cpu.avg_idle.stddev 82979 ± 18% +90.1% 157720 sched_debug.cpu.clock.avg 82995 ± 18% +90.1% 157733 sched_debug.cpu.clock.max 82963 ± 18% +90.1% 157706 sched_debug.cpu.clock.min 9.01 ± 7% -16.7% 7.50 ± 7% sched_debug.cpu.clock.stddev 82357 ± 17% +90.2% 156639 sched_debug.cpu.clock_task.avg 82617 ± 17% +89.9% 156912 sched_debug.cpu.clock_task.max 74586 ± 19% +96.2% 146369 ± 3% sched_debug.cpu.clock_task.min 163.49 ± 15% +28.2% 209.65 ± 2% sched_debug.cpu.curr->pid.avg 5098 ± 9% +46.8% 7485 sched_debug.cpu.curr->pid.max 764.83 ± 6% +24.3% 950.32 sched_debug.cpu.curr->pid.stddev 1367457 ± 42% -46.3% 734767 ± 10% sched_debug.cpu.max_idle_balance_cost.max 85422 ± 47% -63.5% 31174 ± 36% sched_debug.cpu.max_idle_balance_cost.stddev 185.21 ± 31% +147.5% 458.45 sched_debug.cpu.sched_count.min 68.12 ± 29% -47.6% 35.70 ± 5% sched_debug.cpu.sched_goidle.min 63.00 ± 31% +154.5% 160.35 sched_debug.cpu.ttwu_count.min 60.79 ± 32% +159.4% 157.70 sched_debug.cpu.ttwu_local.min 82964 ± 18% +90.1% 157707 sched_debug.cpu_clk 82090 ± 18% +91.0% 156833 sched_debug.ktime 83347 ± 17% +89.7% 158071 sched_debug.sched_clk 11.72 ± 5% -43.3% 6.64 ± 4% perf-stat.i.MPKI 2.276e+09 ± 2% -7.4% 2.107e+09 perf-stat.i.branch-instructions 0.80 ± 9% -0.3 0.47 ± 2% perf-stat.i.branch-miss-rate% 18472434 ± 7% -44.4% 10278875 ± 3% perf-stat.i.branch-misses 88521815 ± 4% -50.8% 43547583 ± 12% perf-stat.i.cache-misses 1.362e+08 ± 4% -49.1% 69364352 ± 3% perf-stat.i.cache-references 49713 -73.1% 13367 ± 2% perf-stat.i.context-switches 2.56 ± 2% +50.9% 3.86 ± 4% perf-stat.i.cpi 2.943e+10 ± 4% +29.0% 3.796e+10 ± 5% perf-stat.i.cpu-cycles 202.46 -2.0% 198.34 perf-stat.i.cpu-migrations 436.82 ± 3% +169.1% 1175 ± 25% perf-stat.i.cycles-between-cache-misses 0.02 ± 54% -0.0 0.00 ± 19% perf-stat.i.dTLB-load-miss-rate% 460559 ± 51% -86.2% 63647 ± 18% perf-stat.i.dTLB-load-misses 3.28e+09 ± 2% -24.1% 2.489e+09 perf-stat.i.dTLB-loads 0.01 ± 30% -0.0 0.00 ± 9% perf-stat.i.dTLB-store-miss-rate% 78206 ± 32% -69.9% 23542 ± 7% perf-stat.i.dTLB-store-misses 1.647e+09 ± 2% -42.4% 9.479e+08 ± 2% perf-stat.i.dTLB-stores 61.64 -7.4 54.19 ± 5% perf-stat.i.iTLB-load-miss-rate% 7442826 ± 5% -33.3% 4966408 ± 12% perf-stat.i.iTLB-load-misses 1.153e+10 ± 2% -13.1% 1.002e+10 perf-stat.i.instructions 1552 ± 6% +31.6% 2044 ± 13% perf-stat.i.instructions-per-iTLB-miss 0.40 ± 2% -34.5% 0.26 ± 4% perf-stat.i.ipc 2165 ± 2% -57.1% 929.17 perf-stat.i.major-faults 0.15 ± 4% +29.1% 0.20 ± 5% perf-stat.i.metric.GHz 38.37 ± 2% -23.6% 29.33 perf-stat.i.metric.M/sec 3814 -2.7% 3712 perf-stat.i.minor-faults 88.44 +3.2 91.67 perf-stat.i.node-load-miss-rate% 15156015 ± 5% -41.9% 8808887 ± 11% perf-stat.i.node-load-misses 2118968 ± 3% -54.3% 968301 ± 6% perf-stat.i.node-loads 5700463 ± 4% -43.7% 3211574 ± 10% perf-stat.i.node-store-misses 332356 ± 33% -44.9% 183110 ± 19% perf-stat.i.node-stores 5979 -22.4% 4641 perf-stat.i.page-faults 11.82 ± 4% -41.4% 6.93 ± 4% perf-stat.overall.MPKI 0.81 ± 8% -0.3 0.49 ± 2% perf-stat.overall.branch-miss-rate% 2.55 ± 2% +48.4% 3.79 ± 4% perf-stat.overall.cpi 332.53 ± 2% +168.4% 892.56 ± 18% perf-stat.overall.cycles-between-cache-misses 0.01 ± 52% -0.0 0.00 ± 19% perf-stat.overall.dTLB-load-miss-rate% 61.67 -7.4 54.23 ± 5% perf-stat.overall.iTLB-load-miss-rate% 1554 ± 6% +31.9% 2050 ± 13% perf-stat.overall.instructions-per-iTLB-miss 0.39 ± 2% -32.5% 0.26 ± 4% perf-stat.overall.ipc 87.71 +2.3 90.04 perf-stat.overall.node-load-miss-rate% 22307 +113.2% 47562 perf-stat.overall.path-length 2.259e+09 ± 2% -7.0% 2.101e+09 perf-stat.ps.branch-instructions 18345804 ± 7% -44.1% 10252843 ± 3% perf-stat.ps.branch-misses 87823932 ± 4% -50.5% 43430790 ± 12% perf-stat.ps.cache-misses 1.352e+08 ± 4% -48.8% 69168925 ± 3% perf-stat.ps.cache-references 49229 -72.9% 13340 ± 2% perf-stat.ps.context-switches 2.921e+10 ± 4% +29.6% 3.784e+10 ± 5% perf-stat.ps.cpu-cycles 200.82 -1.5% 197.89 perf-stat.ps.cpu-migrations 457356 ± 51% -86.1% 63626 ± 18% perf-stat.ps.dTLB-load-misses 3.256e+09 ± 2% -23.8% 2.482e+09 perf-stat.ps.dTLB-loads 77699 ± 31% -69.7% 23538 ± 7% perf-stat.ps.dTLB-store-misses 1.635e+09 ± 2% -42.2% 9.45e+08 ± 2% perf-stat.ps.dTLB-stores 7385117 ± 5% -33.0% 4950431 ± 12% perf-stat.ps.iTLB-load-misses 1.144e+10 ± 2% -12.7% 9.987e+09 perf-stat.ps.instructions 2151 ± 2% -56.7% 932.08 perf-stat.ps.major-faults 3789 -2.3% 3701 perf-stat.ps.minor-faults 15032331 ± 5% -41.6% 8782610 ± 11% perf-stat.ps.node-load-misses 2102459 ± 3% -54.0% 966082 ± 6% perf-stat.ps.node-loads 5653300 ± 4% -43.4% 3200900 ± 10% perf-stat.ps.node-store-misses 330281 ± 33% -44.7% 182598 ± 19% perf-stat.ps.node-stores 5940 -22.0% 4633 perf-stat.ps.page-faults 1.497e+12 +101.6% 3.018e+12 perf-stat.total.instructions 17198 ± 15% +147.2% 42516 ± 20% softirqs.CPU1.SCHED 18452 +100.8% 37059 ± 5% softirqs.CPU100.SCHED 18590 ± 2% +99.9% 37161 ± 6% softirqs.CPU101.SCHED 17981 ± 4% +80.3% 32428 ± 19% softirqs.CPU102.SCHED 18300 +103.8% 37293 ± 4% softirqs.CPU103.SCHED 18457 +98.5% 36642 ± 7% softirqs.CPU104.SCHED 18316 ± 2% +99.6% 36559 ± 7% softirqs.CPU105.SCHED 18193 +93.5% 35209 ± 14% softirqs.CPU106.SCHED 18445 +80.5% 33298 ± 19% softirqs.CPU107.SCHED 17224 ± 9% +114.4% 36931 ± 6% softirqs.CPU108.SCHED 18451 ± 2% +93.9% 35777 ± 10% softirqs.CPU110.SCHED 17642 ± 7% +110.2% 37077 ± 5% softirqs.CPU111.SCHED 18382 +98.0% 36400 ± 9% softirqs.CPU112.SCHED 18502 ± 2% +97.5% 36540 ± 7% softirqs.CPU114.SCHED 18201 ± 2% +99.6% 36338 ± 7% softirqs.CPU115.SCHED 17632 ± 4% +106.4% 36398 ± 7% softirqs.CPU116.SCHED 17117 ± 10% +106.4% 35336 ± 5% softirqs.CPU117.SCHED 14701 ± 23% +141.4% 35489 ± 8% softirqs.CPU12.SCHED 18161 ± 3% +106.2% 37457 ± 3% softirqs.CPU120.SCHED 17715 ± 4% +114.5% 38007 softirqs.CPU121.SCHED 18310 +87.9% 34397 ± 13% softirqs.CPU122.SCHED 18423 +110.5% 38790 softirqs.CPU123.SCHED 18362 ± 2% +104.8% 37613 ± 3% softirqs.CPU124.SCHED 17867 ± 5% +114.7% 38354 softirqs.CPU125.SCHED 18296 +111.4% 38684 softirqs.CPU126.SCHED 16289 ± 20% +133.0% 37961 softirqs.CPU127.SCHED 18342 ± 2% +108.6% 38256 softirqs.CPU128.SCHED 18310 ± 2% +109.6% 38387 softirqs.CPU129.SCHED 16041 ± 18% +124.5% 36017 ± 12% softirqs.CPU13.SCHED 17946 ± 2% +113.0% 38219 softirqs.CPU130.SCHED 16723 ± 18% +130.8% 38591 softirqs.CPU131.SCHED 18335 ± 2% +109.1% 38348 softirqs.CPU132.SCHED 18302 ± 2% +108.7% 38191 ± 2% softirqs.CPU133.SCHED 18283 ± 3% +108.3% 38088 softirqs.CPU134.SCHED 17555 ± 7% +119.5% 38541 softirqs.CPU135.SCHED 18432 ± 2% +108.9% 38507 softirqs.CPU136.SCHED 17918 ± 5% +115.1% 38542 softirqs.CPU137.SCHED 18413 ± 2% +109.8% 38629 softirqs.CPU138.SCHED 18330 ± 3% +109.0% 38316 softirqs.CPU139.SCHED 17624 ± 7% +108.4% 36734 ± 9% softirqs.CPU140.SCHED 18389 ± 3% +109.7% 38560 softirqs.CPU141.SCHED 16063 ± 19% +141.2% 38750 ± 2% softirqs.CPU142.SCHED 7704 ± 25% -35.2% 4995 ± 14% softirqs.CPU143.RCU 18426 ± 2% +108.8% 38474 softirqs.CPU143.SCHED 14487 ± 40% +128.9% 33157 ± 25% softirqs.CPU144.SCHED 17619 ± 4% +109.3% 36875 ± 4% softirqs.CPU145.SCHED 18063 ± 3% +106.2% 37239 ± 6% softirqs.CPU146.SCHED 18508 +103.1% 37591 ± 4% softirqs.CPU147.SCHED 18538 ± 2% +101.3% 37323 ± 5% softirqs.CPU148.SCHED 18428 +101.2% 37078 ± 5% softirqs.CPU149.SCHED 18358 +103.2% 37297 ± 3% softirqs.CPU15.SCHED 18347 ± 2% +102.4% 37130 ± 5% softirqs.CPU150.SCHED 18417 +100.3% 36896 ± 6% softirqs.CPU151.SCHED 7574 ± 25% -46.3% 4071 ± 6% softirqs.CPU152.RCU 16641 ± 17% +128.2% 37969 softirqs.CPU152.SCHED 18452 ± 2% +106.3% 38069 softirqs.CPU153.SCHED 18429 ± 2% +88.0% 34641 ± 17% softirqs.CPU154.SCHED 14804 ± 42% +156.0% 37898 ± 2% softirqs.CPU155.SCHED 18167 ± 3% +108.8% 37943 ± 2% softirqs.CPU156.SCHED 7767 ± 24% -53.2% 3634 ± 9% softirqs.CPU159.RCU 17298 ± 12% +110.8% 36472 ± 7% softirqs.CPU159.SCHED 18229 ± 2% +98.6% 36204 ± 9% softirqs.CPU16.SCHED 18552 ± 5% +100.0% 37099 ± 6% softirqs.CPU160.SCHED 17417 ± 13% +114.7% 37386 ± 4% softirqs.CPU161.SCHED 18732 +96.6% 36828 ± 6% softirqs.CPU162.SCHED 15648 ± 29% +133.7% 36562 ± 6% softirqs.CPU163.SCHED 18569 +99.5% 37042 ± 6% softirqs.CPU164.SCHED 18181 ± 4% +102.8% 36872 ± 6% softirqs.CPU165.SCHED 18464 ± 2% +102.5% 37389 ± 4% softirqs.CPU166.SCHED 18482 ± 2% +84.4% 34090 ± 22% softirqs.CPU167.SCHED 17758 ± 8% +115.5% 38265 softirqs.CPU168.SCHED 17148 ± 5% +122.2% 38101 ± 2% softirqs.CPU169.SCHED 18454 ± 2% +96.9% 36345 ± 8% softirqs.CPU17.SCHED 18560 +108.6% 38721 softirqs.CPU170.SCHED 18572 +107.0% 38445 softirqs.CPU171.SCHED 18659 ± 2% +107.7% 38757 softirqs.CPU172.SCHED 18620 +108.8% 38878 ± 2% softirqs.CPU173.SCHED 18586 +108.6% 38765 softirqs.CPU174.SCHED 18449 +109.9% 38731 softirqs.CPU175.SCHED 18327 +111.9% 38832 ± 2% softirqs.CPU176.SCHED 18441 ± 2% +110.1% 38749 ± 2% softirqs.CPU177.SCHED 18432 ± 2% +103.8% 37573 ± 7% softirqs.CPU178.SCHED 18490 +107.8% 38426 ± 2% softirqs.CPU179.SCHED 18543 ± 2% +98.3% 36764 ± 11% softirqs.CPU18.SCHED 18074 ± 2% +114.3% 38734 ± 3% softirqs.CPU180.SCHED 18122 +113.2% 38643 softirqs.CPU181.SCHED 18343 +97.7% 36258 ± 10% softirqs.CPU182.SCHED 18535 +110.1% 38941 softirqs.CPU183.SCHED 8342 ± 19% -57.5% 3544 ± 3% softirqs.CPU184.RCU 18534 +110.1% 38932 ± 2% softirqs.CPU184.SCHED 18099 ± 3% +113.7% 38671 softirqs.CPU186.SCHED 18487 +109.5% 38732 ± 2% softirqs.CPU187.SCHED 18448 ± 2% +108.6% 38492 softirqs.CPU189.SCHED 18351 ± 2% +83.2% 33618 ± 13% softirqs.CPU19.SCHED 18492 ± 2% +110.6% 38950 ± 2% softirqs.CPU190.SCHED 18507 ± 2% +108.5% 38591 softirqs.CPU191.SCHED 17632 ± 8% +112.7% 37511 ± 3% softirqs.CPU2.SCHED 18306 +97.9% 36237 ± 9% softirqs.CPU20.SCHED 17940 +101.0% 36057 ± 9% softirqs.CPU21.SCHED 18207 ± 2% +92.9% 35129 ± 8% softirqs.CPU23.SCHED 16452 ± 13% +109.3% 34434 ± 17% softirqs.CPU26.SCHED 17301 ± 8% +121.1% 38261 softirqs.CPU27.SCHED 18266 ± 3% +111.6% 38654 ± 2% softirqs.CPU28.SCHED 18597 +105.5% 38217 softirqs.CPU30.SCHED 16799 ± 8% +122.7% 37407 softirqs.CPU31.SCHED 7933 ± 18% -44.5% 4405 ± 11% softirqs.CPU32.RCU 17290 ± 5% +115.0% 37177 ± 2% softirqs.CPU32.SCHED 18270 ± 3% +108.0% 38001 softirqs.CPU33.SCHED 18134 ± 3% +107.3% 37594 softirqs.CPU34.SCHED 17815 ± 2% +115.4% 38371 softirqs.CPU35.SCHED 18263 ± 2% +108.7% 38121 softirqs.CPU36.SCHED 17606 ± 5% +115.8% 37994 softirqs.CPU37.SCHED 18241 ± 3% +109.0% 38129 softirqs.CPU38.SCHED 18273 ± 2% +108.0% 38015 softirqs.CPU39.SCHED 18457 ± 2% +102.3% 37342 ± 2% softirqs.CPU4.SCHED 18233 ± 3% +106.2% 37591 ± 2% softirqs.CPU40.SCHED 7891 ± 16% -43.5% 4462 ± 7% softirqs.CPU41.RCU 18223 ± 2% +101.5% 36727 ± 7% softirqs.CPU41.SCHED 8020 ± 18% -45.0% 4413 ± 8% softirqs.CPU42.RCU 18408 ± 2% +108.6% 38404 softirqs.CPU42.SCHED 7774 ± 19% -43.3% 4405 ± 8% softirqs.CPU43.RCU 18328 ± 2% +95.4% 35807 ± 13% softirqs.CPU43.SCHED 7730 ± 19% -44.6% 4280 ± 5% softirqs.CPU44.RCU 18008 ± 2% +112.6% 38278 softirqs.CPU44.SCHED 18309 ± 2% +108.3% 38145 softirqs.CPU45.SCHED 7622 ± 21% -41.4% 4468 ± 9% softirqs.CPU46.RCU 7856 ± 21% -40.7% 4662 ± 7% softirqs.CPU47.RCU 16775 ± 19% +124.6% 37682 ± 3% softirqs.CPU5.SCHED 16954 ± 5% +115.3% 36507 ± 8% softirqs.CPU50.SCHED 7834 ± 17% -44.7% 4333 ± 11% softirqs.CPU51.RCU 18341 ± 2% +102.0% 37052 ± 5% softirqs.CPU51.SCHED 7830 ± 19% -44.4% 4355 ± 9% softirqs.CPU52.RCU 18428 ± 2% +103.2% 37455 ± 4% softirqs.CPU52.SCHED 7815 ± 17% -44.7% 4319 ± 11% softirqs.CPU53.RCU 18193 ± 2% +101.7% 36690 ± 6% softirqs.CPU53.SCHED 7773 ± 18% -44.4% 4320 ± 11% softirqs.CPU54.RCU 18003 ± 5% +85.4% 33369 ± 20% softirqs.CPU54.SCHED 18378 +102.5% 37206 ± 4% softirqs.CPU55.SCHED 7636 ± 18% -44.9% 4205 ± 8% softirqs.CPU56.RCU 16546 ± 17% +121.7% 36683 ± 6% softirqs.CPU56.SCHED 7723 ± 18% -46.6% 4122 ± 10% softirqs.CPU57.RCU 16611 ± 16% +121.3% 36758 ± 6% softirqs.CPU57.SCHED 7655 ± 19% -41.2% 4503 ± 6% softirqs.CPU58.RCU 7799 ± 19% -44.9% 4299 ± 12% softirqs.CPU59.RCU 18398 +97.1% 36263 ± 9% softirqs.CPU59.SCHED 18178 ± 3% +102.9% 36893 ± 3% softirqs.CPU6.SCHED 7828 ± 18% -47.4% 4116 ± 12% softirqs.CPU60.RCU 18134 ± 2% +93.1% 35020 ± 15% softirqs.CPU60.SCHED 18396 ± 2% +95.5% 35961 ± 7% softirqs.CPU61.SCHED 18388 ± 2% +96.7% 36166 ± 9% softirqs.CPU63.SCHED 18427 ± 2% +105.9% 37937 ± 2% softirqs.CPU64.SCHED 18321 ± 2% +101.2% 36865 ± 6% softirqs.CPU65.SCHED 20470 ± 17% +79.7% 36780 ± 7% softirqs.CPU66.SCHED 18377 ± 2% +98.3% 36433 ± 7% softirqs.CPU67.SCHED 18340 ± 3% +101.1% 36884 ± 6% softirqs.CPU68.SCHED 18281 ± 3% +100.4% 36639 ± 7% softirqs.CPU69.SCHED 17542 ± 2% +108.7% 36615 ± 7% softirqs.CPU7.SCHED 18250 +103.0% 37048 ± 5% softirqs.CPU70.SCHED 17625 ± 4% +85.4% 32678 ± 22% softirqs.CPU71.SCHED 17594 ± 8% +120.9% 38859 ± 2% softirqs.CPU74.SCHED 18524 +96.7% 36437 ± 9% softirqs.CPU75.SCHED 17739 ± 3% +118.0% 38667 ± 2% softirqs.CPU76.SCHED 18491 +110.0% 38829 ± 2% softirqs.CPU77.SCHED 18478 ± 2% +109.0% 38614 softirqs.CPU78.SCHED 18501 +109.2% 38714 softirqs.CPU79.SCHED 18476 +110.6% 38906 ± 2% softirqs.CPU80.SCHED 18272 ± 2% +110.7% 38496 softirqs.CPU81.SCHED 18479 ± 2% +88.6% 34844 ± 21% softirqs.CPU82.SCHED 16347 ± 10% +135.6% 38510 softirqs.CPU84.SCHED 18329 ± 2% +113.2% 39083 ± 3% softirqs.CPU85.SCHED 18292 ± 2% +109.5% 38331 softirqs.CPU86.SCHED 18475 +109.8% 38757 softirqs.CPU87.SCHED 18527 +87.7% 34784 ± 18% softirqs.CPU88.SCHED 18477 +104.2% 37727 ± 3% softirqs.CPU89.SCHED 18279 +103.5% 37194 ± 4% softirqs.CPU9.SCHED 18466 +109.2% 38638 softirqs.CPU90.SCHED 8660 ± 21% -56.1% 3805 ± 3% softirqs.CPU91.RCU 18504 ± 2% +89.8% 35125 ± 19% softirqs.CPU91.SCHED 18295 ± 2% +69.8% 31071 ± 21% softirqs.CPU92.SCHED 18516 +108.1% 38525 softirqs.CPU93.SCHED 18607 ± 2% +108.5% 38786 ± 2% softirqs.CPU94.SCHED 18538 +45.4% 26948 ± 23% softirqs.CPU95.SCHED 17002 ± 12% +114.3% 36427 ± 10% softirqs.CPU98.SCHED 18239 ± 2% +92.8% 35158 ± 10% softirqs.CPU99.SCHED 2799 ± 44% +248.5% 9753 ± 44% softirqs.NET_RX 1405324 ± 15% -43.2% 797662 ± 6% softirqs.RCU 3436439 +101.2% 6914856 ± 2% softirqs.SCHED 37936 ± 2% +37.5% 52169 ± 2% softirqs.TIMER 262.50 ± 2% +130.9% 606.00 interrupts.9:IO-APIC.9-fasteoi.acpi 262080 ± 2% +116.0% 566222 ± 12% interrupts.CPU0.LOC:Local_timer_interrupts 0.75 ±173% +5.1e+05% 3842 ± 88% interrupts.CPU0.NMI:Non-maskable_interrupts 0.75 ±173% +5.1e+05% 3842 ± 88% interrupts.CPU0.PMI:Performance_monitoring_interrupts 65.00 ± 47% +443.5% 353.25 ± 45% interrupts.CPU0.RES:Rescheduling_interrupts 262.50 ± 2% +130.9% 606.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 1702 ± 3% +858.7% 16319 ±149% interrupts.CPU1.CAL:Function_call_interrupts 262519 ± 2% +117.7% 571624 ± 10% interrupts.CPU1.LOC:Local_timer_interrupts 262440 ± 2% +123.6% 586892 ± 5% interrupts.CPU10.LOC:Local_timer_interrupts 263041 ± 2% +114.6% 564434 ± 12% interrupts.CPU100.LOC:Local_timer_interrupts 0.25 ±173% +2.7e+05% 686.25 ± 68% interrupts.CPU100.NMI:Non-maskable_interrupts 0.25 ±173% +2.7e+05% 686.25 ± 68% interrupts.CPU100.PMI:Performance_monitoring_interrupts 262753 ± 2% +114.5% 563706 ± 12% interrupts.CPU101.LOC:Local_timer_interrupts 262541 ± 2% +115.1% 564758 ± 12% interrupts.CPU102.LOC:Local_timer_interrupts 263017 ± 3% +116.4% 569124 ± 11% interrupts.CPU103.LOC:Local_timer_interrupts 263682 ± 3% +114.7% 566056 ± 12% interrupts.CPU104.LOC:Local_timer_interrupts 263035 ± 3% +114.5% 564245 ± 12% interrupts.CPU105.LOC:Local_timer_interrupts 262394 ± 2% +115.3% 564927 ± 12% interrupts.CPU106.LOC:Local_timer_interrupts 262398 ± 2% +116.9% 569255 ± 11% interrupts.CPU107.LOC:Local_timer_interrupts 262388 ± 2% +115.9% 566437 ± 12% interrupts.CPU108.LOC:Local_timer_interrupts 262375 ± 2% +116.0% 566814 ± 11% interrupts.CPU109.LOC:Local_timer_interrupts 262435 ± 2% +116.1% 567228 ± 11% interrupts.CPU11.LOC:Local_timer_interrupts 262379 ± 2% +116.1% 566871 ± 11% interrupts.CPU110.LOC:Local_timer_interrupts 262336 ± 2% +115.2% 564540 ± 12% interrupts.CPU111.LOC:Local_timer_interrupts 262386 ± 2% +114.5% 562700 ± 13% interrupts.CPU112.LOC:Local_timer_interrupts 262354 ± 2% +114.7% 563289 ± 12% interrupts.CPU113.LOC:Local_timer_interrupts 262363 ± 2% +114.5% 562718 ± 13% interrupts.CPU114.LOC:Local_timer_interrupts 262399 ± 2% +114.3% 562370 ± 13% interrupts.CPU115.LOC:Local_timer_interrupts 262199 ± 2% +115.2% 564129 ± 12% interrupts.CPU116.LOC:Local_timer_interrupts 262422 ± 2% +115.9% 566482 ± 12% interrupts.CPU117.LOC:Local_timer_interrupts 262337 ± 2% +114.2% 561808 ± 13% interrupts.CPU118.LOC:Local_timer_interrupts 262366 ± 2% +114.6% 563121 ± 13% interrupts.CPU119.LOC:Local_timer_interrupts 0.25 ±173% +8.9e+05% 2214 ±162% interrupts.CPU119.NMI:Non-maskable_interrupts 0.25 ±173% +8.9e+05% 2214 ±162% interrupts.CPU119.PMI:Performance_monitoring_interrupts 262206 ± 2% +119.7% 576136 ± 12% interrupts.CPU12.LOC:Local_timer_interrupts 260078 ± 3% +110.4% 547239 ± 18% interrupts.CPU120.LOC:Local_timer_interrupts 261865 ± 2% +108.9% 547131 ± 18% interrupts.CPU121.LOC:Local_timer_interrupts 262335 ± 2% +112.5% 557486 ± 15% interrupts.CPU122.LOC:Local_timer_interrupts 260454 ± 3% +107.9% 541396 ± 20% interrupts.CPU123.LOC:Local_timer_interrupts 259867 ± 3% +110.2% 546236 ± 18% interrupts.CPU124.LOC:Local_timer_interrupts 262537 ± 2% +107.5% 544851 ± 19% interrupts.CPU125.LOC:Local_timer_interrupts 76.00 ± 16% +77.3% 134.75 ± 24% interrupts.CPU125.TLB:TLB_shootdowns 260299 ± 3% +109.3% 544923 ± 19% interrupts.CPU126.LOC:Local_timer_interrupts 262012 ± 2% +108.9% 547276 ± 18% interrupts.CPU127.LOC:Local_timer_interrupts 259654 ± 3% +111.3% 548548 ± 18% interrupts.CPU128.LOC:Local_timer_interrupts 262395 ± 2% +106.8% 542735 ± 20% interrupts.CPU129.LOC:Local_timer_interrupts 262340 ± 2% +121.6% 581356 ± 7% interrupts.CPU13.LOC:Local_timer_interrupts 262473 ± 2% +108.0% 546008 ± 19% interrupts.CPU130.LOC:Local_timer_interrupts 262413 ± 2% +106.2% 541040 ± 20% interrupts.CPU131.LOC:Local_timer_interrupts 262344 ± 2% +106.9% 542887 ± 20% interrupts.CPU132.LOC:Local_timer_interrupts 261664 ± 2% +107.8% 543712 ± 19% interrupts.CPU133.LOC:Local_timer_interrupts 262373 ± 2% +106.8% 542477 ± 19% interrupts.CPU134.LOC:Local_timer_interrupts 262387 ± 2% +104.8% 537456 ± 21% interrupts.CPU135.LOC:Local_timer_interrupts 261661 ± 2% +107.6% 543135 ± 19% interrupts.CPU136.LOC:Local_timer_interrupts 261989 ± 2% +106.3% 540444 ± 21% interrupts.CPU137.LOC:Local_timer_interrupts 262033 ± 2% +107.0% 542495 ± 20% interrupts.CPU138.LOC:Local_timer_interrupts 262582 ± 2% +106.6% 542623 ± 20% interrupts.CPU139.LOC:Local_timer_interrupts 262404 ± 2% +121.6% 581424 ± 7% interrupts.CPU14.LOC:Local_timer_interrupts 0.75 ±173% +3e+05% 2240 ±159% interrupts.CPU14.NMI:Non-maskable_interrupts 0.75 ±173% +3e+05% 2240 ±159% interrupts.CPU14.PMI:Performance_monitoring_interrupts 262495 ± 2% +106.8% 542886 ± 20% interrupts.CPU140.LOC:Local_timer_interrupts 73.00 ± 32% +81.5% 132.50 ± 26% interrupts.CPU140.TLB:TLB_shootdowns 262345 ± 2% +107.2% 543631 ± 19% interrupts.CPU141.LOC:Local_timer_interrupts 262474 ± 2% +106.1% 540980 ± 20% interrupts.CPU142.LOC:Local_timer_interrupts 262542 ± 2% +106.1% 540987 ± 20% interrupts.CPU143.LOC:Local_timer_interrupts 262511 ± 2% +108.0% 546124 ± 18% interrupts.CPU144.LOC:Local_timer_interrupts 262569 ± 2% +114.7% 563709 ± 13% interrupts.CPU145.LOC:Local_timer_interrupts 262480 ± 2% +111.4% 554998 ± 15% interrupts.CPU146.LOC:Local_timer_interrupts 262306 ± 2% +108.8% 547645 ± 18% interrupts.CPU147.LOC:Local_timer_interrupts 262459 ± 2% +107.8% 545453 ± 19% interrupts.CPU148.LOC:Local_timer_interrupts 0.25 ±173% +70600.0% 176.75 ± 29% interrupts.CPU148.NMI:Non-maskable_interrupts 0.25 ±173% +70600.0% 176.75 ± 29% interrupts.CPU148.PMI:Performance_monitoring_interrupts 262434 ± 2% +108.1% 546168 ± 18% interrupts.CPU149.LOC:Local_timer_interrupts 262325 ± 2% +117.6% 570853 ± 10% interrupts.CPU15.LOC:Local_timer_interrupts 2.25 ± 65% +1955.6% 46.25 ±122% interrupts.CPU15.RES:Rescheduling_interrupts 262382 ± 2% +108.1% 545997 ± 19% interrupts.CPU150.LOC:Local_timer_interrupts 263406 ± 2% +107.4% 546320 ± 18% interrupts.CPU151.LOC:Local_timer_interrupts 262629 ± 2% +112.3% 557638 ± 14% interrupts.CPU152.LOC:Local_timer_interrupts 263648 ± 2% +115.7% 568792 ± 11% interrupts.CPU153.LOC:Local_timer_interrupts 262475 ± 2% +110.0% 551072 ± 17% interrupts.CPU154.LOC:Local_timer_interrupts 1.50 ±173% +58900.0% 885.00 ±128% interrupts.CPU154.NMI:Non-maskable_interrupts 1.50 ±173% +58900.0% 885.00 ±128% interrupts.CPU154.PMI:Performance_monitoring_interrupts 262446 ± 2% +115.8% 566300 ± 11% interrupts.CPU155.LOC:Local_timer_interrupts 262464 ± 2% +116.1% 567266 ± 11% interrupts.CPU156.LOC:Local_timer_interrupts 262285 ± 2% +116.9% 568882 ± 11% interrupts.CPU157.LOC:Local_timer_interrupts 262472 ± 2% +110.9% 553580 ± 16% interrupts.CPU158.LOC:Local_timer_interrupts 262455 ± 2% +110.6% 552657 ± 16% interrupts.CPU159.LOC:Local_timer_interrupts 262628 ± 2% +115.2% 565202 ± 12% interrupts.CPU16.LOC:Local_timer_interrupts 0.25 ±173% +1.4e+05% 344.75 ± 60% interrupts.CPU16.NMI:Non-maskable_interrupts 0.25 ±173% +1.4e+05% 344.75 ± 60% interrupts.CPU16.PMI:Performance_monitoring_interrupts 262495 ± 2% +108.8% 548211 ± 18% interrupts.CPU160.LOC:Local_timer_interrupts 262484 ± 2% +113.6% 560649 ± 13% interrupts.CPU161.LOC:Local_timer_interrupts 0.25 ±173% +1.5e+05% 368.50 ± 84% interrupts.CPU161.NMI:Non-maskable_interrupts 0.25 ±173% +1.5e+05% 368.50 ± 84% interrupts.CPU161.PMI:Performance_monitoring_interrupts 262509 ± 2% +108.7% 547849 ± 18% interrupts.CPU162.LOC:Local_timer_interrupts 262508 ± 2% +109.5% 549885 ± 17% interrupts.CPU163.LOC:Local_timer_interrupts 262456 ± 2% +108.6% 547432 ± 18% interrupts.CPU164.LOC:Local_timer_interrupts 262463 ± 2% +108.2% 546382 ± 18% interrupts.CPU165.LOC:Local_timer_interrupts 0.25 ±173% +73900.0% 185.00 ± 20% interrupts.CPU165.NMI:Non-maskable_interrupts 0.25 ±173% +73900.0% 185.00 ± 20% interrupts.CPU165.PMI:Performance_monitoring_interrupts 262489 ± 2% +108.0% 546012 ± 18% interrupts.CPU166.LOC:Local_timer_interrupts 262487 ± 2% +111.8% 556017 ± 15% interrupts.CPU167.LOC:Local_timer_interrupts 262371 ± 2% +107.9% 545370 ± 19% interrupts.CPU168.LOC:Local_timer_interrupts 0.00 +2.2e+105% 2243 ±101% interrupts.CPU168.NMI:Non-maskable_interrupts 0.00 +2.2e+105% 2243 ±101% interrupts.CPU168.PMI:Performance_monitoring_interrupts 48.75 ± 83% -91.3% 4.25 ±105% interrupts.CPU168.RES:Rescheduling_interrupts 262437 ± 2% +106.5% 541884 ± 20% interrupts.CPU169.LOC:Local_timer_interrupts 262720 ± 2% +114.9% 564679 ± 12% interrupts.CPU17.LOC:Local_timer_interrupts 262496 ± 2% +105.5% 539521 ± 21% interrupts.CPU170.LOC:Local_timer_interrupts 262452 ± 2% +106.2% 541291 ± 20% interrupts.CPU171.LOC:Local_timer_interrupts 262437 ± 2% +106.8% 542846 ± 20% interrupts.CPU172.LOC:Local_timer_interrupts 262182 ± 2% +106.1% 540399 ± 21% interrupts.CPU173.LOC:Local_timer_interrupts 262276 ± 2% +105.7% 539393 ± 21% interrupts.CPU174.LOC:Local_timer_interrupts 1.50 ±173% +13650.0% 206.25 ± 19% interrupts.CPU174.NMI:Non-maskable_interrupts 1.50 ±173% +13650.0% 206.25 ± 19% interrupts.CPU174.PMI:Performance_monitoring_interrupts 262096 ± 2% +105.7% 539185 ± 21% interrupts.CPU175.LOC:Local_timer_interrupts 0.50 ±173% +40850.0% 204.75 ± 17% interrupts.CPU175.NMI:Non-maskable_interrupts 0.50 ±173% +40850.0% 204.75 ± 17% interrupts.CPU175.PMI:Performance_monitoring_interrupts 262080 ± 2% +106.1% 540169 ± 20% interrupts.CPU176.LOC:Local_timer_interrupts 260711 ± 2% +107.6% 541190 ± 20% interrupts.CPU177.LOC:Local_timer_interrupts 261545 ± 2% +106.4% 539702 ± 21% interrupts.CPU178.LOC:Local_timer_interrupts 260622 +107.4% 540514 ± 20% interrupts.CPU179.LOC:Local_timer_interrupts 262400 ± 2% +114.6% 563134 ± 13% interrupts.CPU18.LOC:Local_timer_interrupts 261789 ± 2% +111.8% 554460 ± 16% interrupts.CPU180.LOC:Local_timer_interrupts 260076 ± 2% +108.4% 541919 ± 20% interrupts.CPU181.LOC:Local_timer_interrupts 260765 +111.9% 552654 ± 16% interrupts.CPU182.LOC:Local_timer_interrupts 262033 ± 2% +106.6% 541360 ± 20% interrupts.CPU183.LOC:Local_timer_interrupts 262136 ± 2% +106.3% 540700 ± 20% interrupts.CPU184.LOC:Local_timer_interrupts 0.25 ±173% +80800.0% 202.25 ± 18% interrupts.CPU184.NMI:Non-maskable_interrupts 0.25 ±173% +80800.0% 202.25 ± 18% interrupts.CPU184.PMI:Performance_monitoring_interrupts 262220 ± 2% +116.9% 568857 ± 11% interrupts.CPU185.LOC:Local_timer_interrupts 262261 ± 2% +106.7% 541984 ± 20% interrupts.CPU186.LOC:Local_timer_interrupts 1.25 ±173% +13820.0% 174.00 ± 26% interrupts.CPU186.NMI:Non-maskable_interrupts 1.25 ±173% +13820.0% 174.00 ± 26% interrupts.CPU186.PMI:Performance_monitoring_interrupts 262443 ± 2% +106.0% 540544 ± 20% interrupts.CPU187.LOC:Local_timer_interrupts 261575 ± 2% +117.8% 569610 ± 11% interrupts.CPU188.LOC:Local_timer_interrupts 261522 ± 2% +106.9% 541106 ± 20% interrupts.CPU189.LOC:Local_timer_interrupts 262433 ± 2% +114.8% 563817 ± 12% interrupts.CPU19.LOC:Local_timer_interrupts 260718 +106.9% 539542 ± 21% interrupts.CPU190.LOC:Local_timer_interrupts 262427 ± 2% +108.9% 548096 ± 18% interrupts.CPU191.LOC:Local_timer_interrupts 1.25 ± 34% +13200.0% 166.25 ± 38% interrupts.CPU191.NMI:Non-maskable_interrupts 1.25 ± 34% +13200.0% 166.25 ± 38% interrupts.CPU191.PMI:Performance_monitoring_interrupts 262415 ± 2% +117.8% 571451 ± 10% interrupts.CPU2.LOC:Local_timer_interrupts 1.00 ±173% +17025.0% 171.25 ± 34% interrupts.CPU2.NMI:Non-maskable_interrupts 1.00 ±173% +17025.0% 171.25 ± 34% interrupts.CPU2.PMI:Performance_monitoring_interrupts 262388 ± 2% +115.5% 565435 ± 12% interrupts.CPU20.LOC:Local_timer_interrupts 262426 ± 2% +115.7% 566000 ± 12% interrupts.CPU21.LOC:Local_timer_interrupts 107.75 ±119% -98.4% 1.75 ± 47% interrupts.CPU21.RES:Rescheduling_interrupts 262428 ± 2% +115.7% 566087 ± 12% interrupts.CPU22.LOC:Local_timer_interrupts 262420 ± 2% +116.4% 567805 ± 11% interrupts.CPU23.LOC:Local_timer_interrupts 0.50 ±100% +3.1e+05% 1560 ±153% interrupts.CPU23.NMI:Non-maskable_interrupts 0.50 ±100% +3.1e+05% 1560 ±153% interrupts.CPU23.PMI:Performance_monitoring_interrupts 262493 ± 2% +122.4% 583804 ± 6% interrupts.CPU24.LOC:Local_timer_interrupts 71.00 ± 81% +215.5% 224.00 ± 39% interrupts.CPU24.RES:Rescheduling_interrupts 262614 ± 2% +111.9% 556548 ± 15% interrupts.CPU25.LOC:Local_timer_interrupts 262638 ± 2% +115.2% 565239 ± 12% interrupts.CPU26.LOC:Local_timer_interrupts 262641 ± 2% +107.1% 544048 ± 19% interrupts.CPU27.LOC:Local_timer_interrupts 262623 ± 2% +109.9% 551291 ± 17% interrupts.CPU28.LOC:Local_timer_interrupts 262610 ± 2% +107.2% 544162 ± 19% interrupts.CPU29.LOC:Local_timer_interrupts 262376 ± 2% +117.6% 570874 ± 10% interrupts.CPU3.LOC:Local_timer_interrupts 0.75 ±173% +18866.7% 142.25 ± 40% interrupts.CPU3.NMI:Non-maskable_interrupts 0.75 ±173% +18866.7% 142.25 ± 40% interrupts.CPU3.PMI:Performance_monitoring_interrupts 262652 ± 2% +108.1% 546468 ± 18% interrupts.CPU30.LOC:Local_timer_interrupts 262617 ± 2% +109.5% 550280 ± 17% interrupts.CPU31.LOC:Local_timer_interrupts 262628 ± 2% +109.7% 550671 ± 17% interrupts.CPU32.LOC:Local_timer_interrupts 262607 ± 2% +108.7% 548001 ± 18% interrupts.CPU33.LOC:Local_timer_interrupts 262627 ± 2% +107.5% 544959 ± 19% interrupts.CPU34.LOC:Local_timer_interrupts 262549 ± 2% +105.7% 540192 ± 21% interrupts.CPU35.LOC:Local_timer_interrupts 0.25 ±173% +49700.0% 124.50 ± 44% interrupts.CPU35.NMI:Non-maskable_interrupts 0.25 ±173% +49700.0% 124.50 ± 44% interrupts.CPU35.PMI:Performance_monitoring_interrupts 262618 ± 2% +106.3% 541838 ± 20% interrupts.CPU36.LOC:Local_timer_interrupts 262487 ± 2% +107.4% 544382 ± 19% interrupts.CPU37.LOC:Local_timer_interrupts 13.50 ±128% +514.8% 83.00 ± 70% interrupts.CPU37.TLB:TLB_shootdowns 262620 ± 2% +106.5% 542305 ± 20% interrupts.CPU38.LOC:Local_timer_interrupts 0.25 ±173% +63400.0% 158.75 ± 42% interrupts.CPU38.NMI:Non-maskable_interrupts 0.25 ±173% +63400.0% 158.75 ± 42% interrupts.CPU38.PMI:Performance_monitoring_interrupts 262649 ± 2% +107.4% 544844 ± 19% interrupts.CPU39.LOC:Local_timer_interrupts 262402 ± 2% +118.8% 574008 ± 11% interrupts.CPU4.LOC:Local_timer_interrupts 0.75 ±173% +1.6e+05% 1233 ±109% interrupts.CPU4.NMI:Non-maskable_interrupts 0.75 ±173% +1.6e+05% 1233 ±109% interrupts.CPU4.PMI:Performance_monitoring_interrupts 262634 ± 2% +107.0% 543548 ± 19% interrupts.CPU40.LOC:Local_timer_interrupts 262152 ± 2% +106.0% 539971 ± 21% interrupts.CPU41.LOC:Local_timer_interrupts 263740 ± 2% +105.8% 542816 ± 20% interrupts.CPU42.LOC:Local_timer_interrupts 67.50 ± 67% -96.7% 2.25 ± 72% interrupts.CPU42.RES:Rescheduling_interrupts 263061 ± 2% +106.2% 542431 ± 20% interrupts.CPU43.LOC:Local_timer_interrupts 263144 ± 2% +106.0% 542069 ± 20% interrupts.CPU44.LOC:Local_timer_interrupts 98.00 ±139% -99.5% 0.50 ±173% interrupts.CPU44.RES:Rescheduling_interrupts 31.00 ± 71% +213.7% 97.25 ± 34% interrupts.CPU44.TLB:TLB_shootdowns 262640 ± 2% +107.1% 543849 ± 19% interrupts.CPU45.LOC:Local_timer_interrupts 262653 ± 2% +105.7% 540247 ± 21% interrupts.CPU46.LOC:Local_timer_interrupts 261904 ± 2% +106.4% 540591 ± 20% interrupts.CPU47.LOC:Local_timer_interrupts 0.25 ±173% +9e+05% 2243 ±159% interrupts.CPU47.NMI:Non-maskable_interrupts 0.25 ±173% +9e+05% 2243 ±159% interrupts.CPU47.PMI:Performance_monitoring_interrupts 262378 ± 2% +108.9% 548209 ± 18% interrupts.CPU48.LOC:Local_timer_interrupts 55.00 ± 53% +351.8% 248.50 ± 35% interrupts.CPU48.RES:Rescheduling_interrupts 262257 ± 2% +113.0% 558578 ± 14% interrupts.CPU49.LOC:Local_timer_interrupts 262402 ± 2% +116.9% 569165 ± 11% interrupts.CPU5.LOC:Local_timer_interrupts 262531 ± 2% +109.0% 548622 ± 18% interrupts.CPU50.LOC:Local_timer_interrupts 0.00 +2e+104% 204.00 ± 14% interrupts.CPU50.NMI:Non-maskable_interrupts 0.00 +2e+104% 204.00 ± 14% interrupts.CPU50.PMI:Performance_monitoring_interrupts 262465 ± 2% +108.6% 547534 ± 18% interrupts.CPU51.LOC:Local_timer_interrupts 40.75 ±117% -81.6% 7.50 ± 98% interrupts.CPU51.RES:Rescheduling_interrupts 262492 ± 2% +108.5% 547260 ± 18% interrupts.CPU52.LOC:Local_timer_interrupts 262492 ± 2% +107.9% 545749 ± 19% interrupts.CPU53.LOC:Local_timer_interrupts 262435 ± 2% +108.5% 547082 ± 18% interrupts.CPU54.LOC:Local_timer_interrupts 262517 ± 2% +108.9% 548392 ± 18% interrupts.CPU55.LOC:Local_timer_interrupts 262573 ± 2% +108.4% 547312 ± 18% interrupts.CPU56.LOC:Local_timer_interrupts 262977 ± 2% +110.2% 552772 ± 16% interrupts.CPU57.LOC:Local_timer_interrupts 0.25 ±173% +1.9e+05% 470.75 ±106% interrupts.CPU57.NMI:Non-maskable_interrupts 0.25 ±173% +1.9e+05% 470.75 ±106% interrupts.CPU57.PMI:Performance_monitoring_interrupts 262378 ± 2% +121.7% 581611 ± 7% interrupts.CPU58.LOC:Local_timer_interrupts 262469 ± 2% +109.4% 549629 ± 17% interrupts.CPU59.LOC:Local_timer_interrupts 262426 ± 2% +118.6% 573635 ± 9% interrupts.CPU6.LOC:Local_timer_interrupts 262393 ± 2% +108.9% 548229 ± 18% interrupts.CPU60.LOC:Local_timer_interrupts 124.75 ± 34% -80.4% 24.50 ±104% interrupts.CPU60.RES:Rescheduling_interrupts 262324 ± 2% +109.6% 549933 ± 17% interrupts.CPU61.LOC:Local_timer_interrupts 262490 ± 2% +108.5% 547277 ± 18% interrupts.CPU62.LOC:Local_timer_interrupts 81.25 ±100% -97.2% 2.25 ± 65% interrupts.CPU62.RES:Rescheduling_interrupts 262468 ± 2% +108.9% 548272 ± 18% interrupts.CPU63.LOC:Local_timer_interrupts 262525 ± 2% +113.2% 559696 ± 14% interrupts.CPU64.LOC:Local_timer_interrupts 262486 ± 2% +111.3% 554745 ± 16% interrupts.CPU65.LOC:Local_timer_interrupts 262541 ± 2% +109.0% 548670 ± 18% interrupts.CPU66.LOC:Local_timer_interrupts 262413 ± 2% +109.9% 550793 ± 17% interrupts.CPU67.LOC:Local_timer_interrupts 262517 ± 2% +109.2% 549147 ± 17% interrupts.CPU68.LOC:Local_timer_interrupts 262508 ± 2% +108.9% 548499 ± 18% interrupts.CPU69.LOC:Local_timer_interrupts 0.25 ±173% +74500.0% 186.50 ± 20% interrupts.CPU69.NMI:Non-maskable_interrupts 0.25 ±173% +74500.0% 186.50 ± 20% interrupts.CPU69.PMI:Performance_monitoring_interrupts 262559 ± 2% +116.6% 568696 ± 11% interrupts.CPU7.LOC:Local_timer_interrupts 0.00 +1.4e+104% 145.00 ± 40% interrupts.CPU7.NMI:Non-maskable_interrupts 0.00 +1.4e+104% 145.00 ± 40% interrupts.CPU7.PMI:Performance_monitoring_interrupts 262390 ± 2% +108.4% 546885 ± 18% interrupts.CPU70.LOC:Local_timer_interrupts 262516 ± 2% +108.4% 547201 ± 18% interrupts.CPU71.LOC:Local_timer_interrupts 262409 ± 2% +107.4% 544158 ± 19% interrupts.CPU72.LOC:Local_timer_interrupts 262458 ± 2% +108.8% 548088 ± 18% interrupts.CPU73.LOC:Local_timer_interrupts 52.75 ± 57% +253.1% 186.25 ± 36% interrupts.CPU73.RES:Rescheduling_interrupts 262509 ± 2% +105.1% 538318 ± 21% interrupts.CPU74.LOC:Local_timer_interrupts 262532 ± 2% +109.5% 549979 ± 17% interrupts.CPU75.LOC:Local_timer_interrupts 262463 ± 2% +105.5% 539477 ± 21% interrupts.CPU76.LOC:Local_timer_interrupts 262484 ± 2% +105.6% 539784 ± 21% interrupts.CPU77.LOC:Local_timer_interrupts 0.25 ±173% +81300.0% 203.50 ± 19% interrupts.CPU77.NMI:Non-maskable_interrupts 0.25 ±173% +81300.0% 203.50 ± 19% interrupts.CPU77.PMI:Performance_monitoring_interrupts 262445 ± 2% +105.7% 539793 ± 21% interrupts.CPU78.LOC:Local_timer_interrupts 0.25 ±173% +82300.0% 206.00 ± 19% interrupts.CPU78.NMI:Non-maskable_interrupts 0.25 ±173% +82300.0% 206.00 ± 19% interrupts.CPU78.PMI:Performance_monitoring_interrupts 262433 ± 2% +105.7% 539708 ± 21% interrupts.CPU79.LOC:Local_timer_interrupts 262480 ± 2% +115.3% 565229 ± 12% interrupts.CPU8.LOC:Local_timer_interrupts 0.25 ±173% +1.9e+05% 470.75 ± 97% interrupts.CPU8.NMI:Non-maskable_interrupts 0.25 ±173% +1.9e+05% 470.75 ± 97% interrupts.CPU8.PMI:Performance_monitoring_interrupts 262432 ± 2% +105.8% 540115 ± 21% interrupts.CPU80.LOC:Local_timer_interrupts 262453 ± 2% +105.9% 540432 ± 20% interrupts.CPU81.LOC:Local_timer_interrupts 262472 ± 2% +105.3% 538774 ± 21% interrupts.CPU82.LOC:Local_timer_interrupts 262526 ± 2% +105.2% 538597 ± 21% interrupts.CPU83.LOC:Local_timer_interrupts 262579 ± 2% +105.6% 539844 ± 21% interrupts.CPU84.LOC:Local_timer_interrupts 0.00 +2.1e+104% 213.00 ± 21% interrupts.CPU84.NMI:Non-maskable_interrupts 0.00 +2.1e+104% 213.00 ± 21% interrupts.CPU84.PMI:Performance_monitoring_interrupts 262450 ± 2% +108.9% 548271 ± 18% interrupts.CPU85.LOC:Local_timer_interrupts 150.00 ± 98% -98.8% 1.75 ± 47% interrupts.CPU85.RES:Rescheduling_interrupts 1732 ± 5% +46.7% 2540 ± 43% interrupts.CPU86.CAL:Function_call_interrupts 262440 ± 2% +106.8% 542823 ± 20% interrupts.CPU86.LOC:Local_timer_interrupts 262459 ± 2% +108.2% 546320 ± 18% interrupts.CPU87.LOC:Local_timer_interrupts 261772 ± 2% +106.6% 540909 ± 20% interrupts.CPU88.LOC:Local_timer_interrupts 262191 ± 2% +108.1% 545731 ± 19% interrupts.CPU89.LOC:Local_timer_interrupts 262323 ± 2% +117.2% 569838 ± 11% interrupts.CPU9.LOC:Local_timer_interrupts 261786 ± 2% +107.1% 542265 ± 20% interrupts.CPU90.LOC:Local_timer_interrupts 262226 ± 2% +106.5% 541382 ± 20% interrupts.CPU91.LOC:Local_timer_interrupts 261703 ± 2% +108.7% 546199 ± 18% interrupts.CPU92.LOC:Local_timer_interrupts 262127 ± 2% +106.5% 541287 ± 20% interrupts.CPU93.LOC:Local_timer_interrupts 260263 ± 3% +107.7% 540500 ± 20% interrupts.CPU94.LOC:Local_timer_interrupts 261864 ± 2% +106.4% 540510 ± 20% interrupts.CPU95.LOC:Local_timer_interrupts 262438 ± 2% +114.1% 561887 ± 13% interrupts.CPU96.LOC:Local_timer_interrupts 0.25 ±173% +1.1e+06% 2648 ±127% interrupts.CPU96.NMI:Non-maskable_interrupts 0.25 ±173% +1.1e+06% 2648 ±127% interrupts.CPU96.PMI:Performance_monitoring_interrupts 262573 ± 2% +115.2% 565020 ± 12% interrupts.CPU97.LOC:Local_timer_interrupts 1.75 ±173% +11657.1% 205.75 ± 8% interrupts.CPU97.NMI:Non-maskable_interrupts 1.75 ±173% +11657.1% 205.75 ± 8% interrupts.CPU97.PMI:Performance_monitoring_interrupts 263173 ± 2% +114.2% 563719 ± 12% interrupts.CPU98.LOC:Local_timer_interrupts 0.50 ±100% +38950.0% 195.25 ± 22% interrupts.CPU98.NMI:Non-maskable_interrupts 0.50 ±100% +38950.0% 195.25 ± 22% interrupts.CPU98.PMI:Performance_monitoring_interrupts 263343 ± 2% +115.5% 567513 ± 14% interrupts.CPU99.LOC:Local_timer_interrupts 0.25 ±173% +49300.0% 123.50 ± 51% interrupts.CPU99.NMI:Non-maskable_interrupts 0.25 ±173% +49300.0% 123.50 ± 51% interrupts.CPU99.PMI:Performance_monitoring_interrupts 50365605 ± 2% +110.5% 1.06e+08 ± 16% interrupts.LOC:Local_timer_interrupts 0.00 +1.9e+104% 192.00 interrupts.MCP:Machine_check_polls 17.00 ± 16% +8.2e+05% 138886 ± 10% interrupts.NMI:Non-maskable_interrupts 17.00 ± 16% +8.2e+05% 138886 ± 10% interrupts.PMI:Performance_monitoring_interrupts 8445 ± 10% -24.0% 6416 ± 4% interrupts.RES:Rescheduling_interrupts 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode 17.20 ± 68% -17.2 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare 13.77 ±104% -13.8 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 13.77 ±104% -13.8 0.00 perf-profile.calltrace.cycles-pp.read 12.68 ±109% -12.7 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 11.59 ±117% -11.6 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 11.59 ±117% -11.6 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 10.19 ± 64% -10.2 0.00 perf-profile.calltrace.cycles-pp.task_work_run.do_exit.do_group_exit.get_signal.arch_do_signal 10.19 ± 64% -10.2 0.00 perf-profile.calltrace.cycles-pp.__fput.task_work_run.do_exit.do_group_exit.get_signal 9.82 ±107% -9.8 0.00 perf-profile.calltrace.cycles-pp.seq_read.proc_reg_read.vfs_read.ksys_read.do_syscall_64 9.82 ±107% -9.8 0.00 perf-profile.calltrace.cycles-pp.proc_reg_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.open64 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 9.03 ±107% -9.0 0.00 perf-profile.calltrace.cycles-pp.open64 8.66 ± 41% -8.7 0.00 perf-profile.calltrace.cycles-pp.__libc_start_main 8.66 ± 41% -8.7 0.00 perf-profile.calltrace.cycles-pp.main.__libc_start_main 8.66 ± 41% -8.7 0.00 perf-profile.calltrace.cycles-pp.run_builtin.main.__libc_start_main 8.66 ± 41% -8.7 0.00 perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.__libc_start_main 7.74 ±100% -7.7 0.00 perf-profile.calltrace.cycles-pp.show_interrupts.seq_read.proc_reg_read.vfs_read.ksys_read 7.02 ± 80% -7.0 0.00 perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.get_signal.arch_do_signal 7.02 ± 80% -7.0 0.00 perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.get_signal 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.record__mmap_read_evlist.cmd_record.run_builtin.main.__libc_start_main 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.perf_mmap__push.record__mmap_read_evlist.cmd_record.run_builtin.main 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.perf_mmap__read_head.perf_mmap__push.record__mmap_read_evlist.cmd_record.run_builtin 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.perf_mmap__read_head.perf_mmap__push.record__mmap_read_evlist.cmd_record 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.perf_mmap__read_head.perf_mmap__push.record__mmap_read_evlist 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.perf_mmap__read_head.perf_mmap__push 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.perf_mmap__read_head 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.perf_release.__fput.task_work_run.do_exit.do_group_exit 5.09 ± 64% -5.1 0.00 perf-profile.calltrace.cycles-pp.perf_event_release_kernel.perf_release.__fput.task_work_run.do_exit 4.26 ±100% -4.3 0.00 perf-profile.calltrace.cycles-pp.seq_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp._free_event.perf_event_release_kernel.perf_release.__fput.task_work_run 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.mutex_lock.swevent_hlist_put_cpu.sw_perf_event_destroy._free_event.perf_event_release_kernel 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release.__fput 4.00 ±100% -4.0 0.00 perf-profile.calltrace.cycles-pp.swevent_hlist_put_cpu.sw_perf_event_destroy._free_event.perf_event_release_kernel.perf_release 0.00 +0.6 0.58 ± 15% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter 0.00 +0.6 0.64 ± 13% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +0.6 0.64 ± 13% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +0.6 0.64 ± 13% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt 0.00 +0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write 0.00 +0.7 0.69 ± 9% perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter 0.00 +0.7 0.71 ± 10% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +0.7 0.73 ± 13% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +0.8 0.77 ± 11% perf-profile.calltrace.cycles-pp._find_next_bit.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes 0.00 +0.8 0.80 ± 16% perf-profile.calltrace.cycles-pp.pagecache_get_page.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +0.8 0.83 ± 17% perf-profile.calltrace.cycles-pp.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +0.8 0.84 ± 24% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 0.00 +0.9 0.88 ± 50% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 0.00 +0.9 0.89 ± 49% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 0.00 +1.2 1.16 ± 45% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.00 +1.3 1.28 ± 11% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 0.00 +1.3 1.30 ± 14% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +1.5 1.54 ± 36% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 0.00 +2.9 2.86 ± 37% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +2.9 2.89 ± 37% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +2.9 2.90 ± 37% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +4.5 4.49 ± 11% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 1.09 ±173% +6.2 7.24 ± 9% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.00 +6.4 6.41 ± 31% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +16.6 16.64 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 0.00 +16.9 16.92 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 0.00 +20.7 20.70 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 0.00 +20.9 20.93 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 0.00 +21.1 21.05 ± 10% perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +21.1 21.08 ± 10% perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 0.00 +22.0 21.99 ± 9% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 0.00 +22.0 22.01 ± 9% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 0.00 +22.3 22.30 ± 9% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 20.77 ± 35% +27.3 48.10 ± 10% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 3.57 ±173% +44.5 48.08 ± 10% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +47.8 47.77 ± 10% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 0.00 +47.9 47.87 ± 10% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.00 +47.9 47.89 ± 10% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +48.0 47.99 ± 10% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +48.0 48.03 ± 10% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 22.39 ± 62% -22.4 0.00 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 20.21 ± 68% -20.2 0.00 perf-profile.children.cycles-pp.exit_to_user_mode_prepare 17.20 ± 68% -17.2 0.00 perf-profile.children.cycles-pp.arch_do_signal 17.20 ± 68% -17.2 0.00 perf-profile.children.cycles-pp.get_signal 17.20 ± 68% -17.2 0.00 perf-profile.children.cycles-pp.do_group_exit 17.20 ± 68% -17.2 0.00 perf-profile.children.cycles-pp.do_exit 15.16 ± 78% -15.2 0.00 perf-profile.children.cycles-pp.ksys_read 15.16 ± 78% -15.2 0.00 perf-profile.children.cycles-pp.vfs_read 14.08 ± 86% -14.1 0.00 perf-profile.children.cycles-pp.seq_read 13.77 ±104% -13.8 0.00 perf-profile.children.cycles-pp.read 13.20 ± 63% -13.2 0.00 perf-profile.children.cycles-pp.task_work_run 13.20 ± 63% -13.2 0.00 perf-profile.children.cycles-pp.__fput 9.82 ±107% -9.8 0.00 perf-profile.children.cycles-pp.proc_reg_read 9.03 ±107% -9.0 0.00 perf-profile.children.cycles-pp.do_sys_open 9.03 ±107% -9.0 0.00 perf-profile.children.cycles-pp.do_sys_openat2 9.03 ±107% -9.0 0.00 perf-profile.children.cycles-pp.do_filp_open 9.03 ±107% -9.0 0.00 perf-profile.children.cycles-pp.path_openat 9.03 ±107% -9.0 0.00 perf-profile.children.cycles-pp.open64 8.66 ± 41% -8.7 0.00 perf-profile.children.cycles-pp.__libc_start_main 8.66 ± 41% -8.7 0.00 perf-profile.children.cycles-pp.main 8.66 ± 41% -8.7 0.00 perf-profile.children.cycles-pp.run_builtin 8.66 ± 41% -8.7 0.00 perf-profile.children.cycles-pp.cmd_record 8.66 ± 41% -8.7 0.00 perf-profile.children.cycles-pp.perf_mmap__push 7.74 ±100% -7.7 0.00 perf-profile.children.cycles-pp.show_interrupts 7.02 ± 80% -7.0 0.00 perf-profile.children.cycles-pp.mmput 7.02 ± 80% -7.0 0.00 perf-profile.children.cycles-pp.exit_mmap 5.18 ±106% -5.2 0.00 perf-profile.children.cycles-pp.refill_obj_stock 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.record__mmap_read_evlist 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.perf_mmap__read_head 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.asm_exc_page_fault 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.exc_page_fault 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.do_user_addr_fault 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.handle_mm_fault 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.__handle_mm_fault 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.perf_release 5.09 ± 64% -5.1 0.00 perf-profile.children.cycles-pp.perf_event_release_kernel 4.26 ±100% -4.3 0.00 perf-profile.children.cycles-pp.obj_cgroup_charge 4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.kfree 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.zap_pte_range 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp._free_event 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.mutex_lock 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.do_fault 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.__do_fault 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.unmap_vmas 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.unmap_page_range 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.sw_perf_event_destroy 4.00 ±100% -4.0 0.00 perf-profile.children.cycles-pp.swevent_hlist_put_cpu 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.rmqueue_bulk 0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.btrfs_delalloc_release_extents 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.rcu_idle_exit 0.00 +0.1 0.07 ± 19% perf-profile.children.cycles-pp.free_extent_state 0.00 +0.1 0.07 ± 24% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.00 +0.1 0.07 ± 15% perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size 0.00 +0.1 0.08 ± 22% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.1 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_write_check 0.00 +0.1 0.08 ± 15% perf-profile.children.cycles-pp.btrfs_transaction_in_commit 0.00 +0.1 0.08 ± 19% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.00 +0.1 0.08 ± 23% perf-profile.children.cycles-pp.merge_state 0.00 +0.1 0.09 ± 45% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.1 0.09 ± 14% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.00 +0.1 0.10 ± 30% perf-profile.children.cycles-pp.__mod_memcg_state 0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.rmqueue 0.00 +0.1 0.12 ± 39% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.12 ± 8% perf-profile.children.cycles-pp.irqtime_account_irq 0.00 +0.1 0.13 ± 14% perf-profile.children.cycles-pp.lock_extent_bits 0.00 +0.1 0.13 ± 20% perf-profile.children.cycles-pp.clear_state_bit 0.00 +0.1 0.13 ± 23% perf-profile.children.cycles-pp.update_sd_lb_stats 0.00 +0.1 0.13 ± 14% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.00 +0.1 0.13 ± 26% perf-profile.children.cycles-pp.find_busiest_group 0.00 +0.1 0.14 ± 10% perf-profile.children.cycles-pp.native_sched_clock 0.00 +0.1 0.14 ± 26% perf-profile.children.cycles-pp.account_page_dirtied 0.00 +0.1 0.14 ± 88% perf-profile.children.cycles-pp.arch_scale_freq_tick 0.00 +0.1 0.15 ± 15% perf-profile.children.cycles-pp.lru_cache_add 0.00 +0.1 0.15 ± 10% perf-profile.children.cycles-pp.sched_clock 0.00 +0.2 0.15 ± 10% perf-profile.children.cycles-pp.read_tsc 0.00 +0.2 0.16 ± 10% perf-profile.children.cycles-pp.sched_clock_cpu 0.00 +0.2 0.17 ± 17% perf-profile.children.cycles-pp.lapic_next_deadline 0.00 +0.2 0.17 ± 25% perf-profile.children.cycles-pp.get_page_from_freelist 0.00 +0.2 0.17 ± 15% perf-profile.children.cycles-pp.mem_cgroup_charge 0.00 +0.2 0.18 ± 21% perf-profile.children.cycles-pp.load_balance 0.00 +0.2 0.18 ± 40% perf-profile.children.cycles-pp.xas_load 0.00 +0.2 0.19 ± 19% perf-profile.children.cycles-pp.find_next_bit 0.00 +0.2 0.21 ± 14% perf-profile.children.cycles-pp.btrfs_drop_pages 0.00 +0.2 0.21 ±109% perf-profile.children.cycles-pp.update_rq_clock 0.00 +0.2 0.22 ± 21% perf-profile.children.cycles-pp.btrfs_get_alloc_profile 0.00 +0.2 0.22 ± 42% perf-profile.children.cycles-pp.btrfs_get_extent 0.00 +0.2 0.22 ±124% perf-profile.children.cycles-pp.update_blocked_averages 0.00 +0.2 0.23 ± 17% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need 0.00 +0.2 0.24 ± 12% perf-profile.children.cycles-pp._raw_spin_trylock 0.00 +0.2 0.24 ± 20% perf-profile.children.cycles-pp.calc_available_free_space 0.00 +0.2 0.25 ± 6% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.3 0.26 ± 17% perf-profile.children.cycles-pp.alloc_extent_state 0.00 +0.3 0.27 ± 21% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.3 0.29 ± 20% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.00 +0.3 0.33 ± 15% perf-profile.children.cycles-pp.clear_extent_bit 0.00 +0.4 0.36 ± 41% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.00 +0.4 0.40 ± 8% perf-profile.children.cycles-pp.btrfs_set_delalloc_extent 0.00 +0.4 0.40 ± 10% perf-profile.children.cycles-pp.btrfs_reserve_data_bytes 0.00 +0.4 0.41 ± 93% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.00 +0.4 0.41 ± 8% perf-profile.children.cycles-pp.set_state_bits 0.00 +0.4 0.42 ± 17% perf-profile.children.cycles-pp.__add_to_page_cache_locked 0.00 +0.5 0.47 ± 9% perf-profile.children.cycles-pp.btrfs_check_data_free_space 0.00 +0.5 0.47 ± 8% perf-profile.children.cycles-pp.set_extent_bit 0.00 +0.5 0.50 ± 12% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.5 0.52 ± 15% perf-profile.children.cycles-pp.__clear_extent_bit 0.00 +0.6 0.56 ± 27% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +0.6 0.56 ± 55% perf-profile.children.cycles-pp.scheduler_tick 0.00 +0.6 0.58 ± 15% perf-profile.children.cycles-pp.add_to_page_cache_lru 0.00 +0.6 0.60 ± 9% perf-profile.children.cycles-pp.__set_extent_bit 0.00 +0.6 0.62 ± 40% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.00 +0.6 0.62 ± 71% perf-profile.children.cycles-pp.tick_irq_enter 0.00 +0.7 0.68 ± 63% perf-profile.children.cycles-pp.irq_enter_rcu 0.00 +0.7 0.70 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.00 +0.7 0.70 ± 10% perf-profile.children.cycles-pp.copyin 0.00 +0.7 0.71 ± 10% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 0.00 +0.7 0.74 ± 10% perf-profile.children.cycles-pp.btrfs_copy_from_user 0.00 +0.8 0.83 ± 47% perf-profile.children.cycles-pp.__softirqentry_text_start 0.00 +0.8 0.83 ± 46% perf-profile.children.cycles-pp.do_softirq_own_stack 0.00 +0.8 0.83 ± 12% perf-profile.children.cycles-pp._find_next_bit 0.00 +0.8 0.83 ± 17% perf-profile.children.cycles-pp.prepare_pages 0.00 +0.9 0.90 ± 18% perf-profile.children.cycles-pp.pagecache_get_page 0.00 +0.9 0.90 ± 28% perf-profile.children.cycles-pp.clockevents_program_event 0.00 +0.9 0.93 ± 41% perf-profile.children.cycles-pp.irq_exit_rcu 0.00 +0.9 0.94 ± 38% perf-profile.children.cycles-pp.tick_nohz_next_event 0.00 +1.0 0.97 ± 47% perf-profile.children.cycles-pp.update_process_times 0.00 +1.0 0.98 ± 47% perf-profile.children.cycles-pp.tick_sched_handle 0.00 +1.0 1.02 ± 36% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +1.3 1.27 ± 44% perf-profile.children.cycles-pp.tick_sched_timer 0.00 +1.3 1.30 ± 14% perf-profile.children.cycles-pp.btrfs_dirty_pages 0.00 +1.4 1.44 ± 11% perf-profile.children.cycles-pp.cpumask_next 0.00 +1.7 1.69 ± 37% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.00 +3.1 3.12 ± 40% perf-profile.children.cycles-pp.hrtimer_interrupt 0.00 +3.2 3.16 ± 39% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 1.09 ±173% +3.9 5.02 ± 34% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.00 +4.5 4.52 ± 11% perf-profile.children.cycles-pp.__percpu_counter_sum 1.09 ±173% +5.3 6.43 ± 20% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.00 +6.4 6.43 ± 31% perf-profile.children.cycles-pp.menu_select 0.00 +21.1 21.07 ± 10% perf-profile.children.cycles-pp.btrfs_block_rsv_release 0.00 +21.1 21.08 ± 10% perf-profile.children.cycles-pp.btrfs_inode_rsv_release 26.37 ± 31% +21.8 48.19 ± 10% perf-profile.children.cycles-pp.do_syscall_64 0.00 +22.0 22.01 ± 9% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 0.00 +22.3 22.30 ± 9% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 0.00 +22.3 22.30 ± 9% perf-profile.children.cycles-pp.__reserve_bytes 0.00 +37.5 37.46 ± 9% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.00 +39.1 39.12 ± 10% perf-profile.children.cycles-pp._raw_spin_lock 0.00 +47.8 47.77 ± 10% perf-profile.children.cycles-pp.btrfs_buffered_write 0.00 +47.9 47.88 ± 10% perf-profile.children.cycles-pp.btrfs_file_write_iter 0.00 +47.9 47.91 ± 10% perf-profile.children.cycles-pp.new_sync_write 0.00 +48.0 48.01 ± 10% perf-profile.children.cycles-pp.vfs_write 0.00 +48.0 48.05 ± 10% perf-profile.children.cycles-pp.ksys_write 5.65 ±106% -5.7 0.00 perf-profile.self.cycles-pp.show_interrupts 4.00 ±100% -4.0 0.00 perf-profile.self.cycles-pp.zap_pte_range 4.00 ±100% -4.0 0.00 perf-profile.self.cycles-pp.mutex_lock 0.00 +0.1 0.05 ± 9% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.1 0.06 ± 22% perf-profile.self.cycles-pp.btrfs_buffered_write 0.00 +0.1 0.07 ± 25% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.00 +0.1 0.07 ± 26% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.00 +0.1 0.07 ± 12% perf-profile.self.cycles-pp.do_idle 0.00 +0.1 0.07 ± 19% perf-profile.self.cycles-pp.free_extent_state 0.00 +0.1 0.08 ± 24% perf-profile.self.cycles-pp.__add_to_page_cache_locked 0.00 +0.1 0.08 ± 21% perf-profile.self.cycles-pp.update_sd_lb_stats 0.00 +0.1 0.09 ± 24% perf-profile.self.cycles-pp.tick_sched_timer 0.00 +0.1 0.09 ± 25% perf-profile.self.cycles-pp.btrfs_get_alloc_profile 0.00 +0.1 0.09 ± 26% perf-profile.self.cycles-pp.btrfs_file_write_iter 0.00 +0.1 0.09 ± 19% perf-profile.self.cycles-pp.btrfs_reserve_data_bytes 0.00 +0.1 0.10 ± 11% perf-profile.self.cycles-pp.btrfs_block_rsv_release 0.00 +0.1 0.10 ± 11% perf-profile.self.cycles-pp.rebalance_domains 0.00 +0.1 0.10 ± 15% perf-profile.self.cycles-pp.btrfs_set_delalloc_extent 0.00 +0.1 0.10 ± 22% perf-profile.self.cycles-pp.__clear_extent_bit 0.00 +0.1 0.12 ± 27% perf-profile.self.cycles-pp.btrfs_dirty_pages 0.00 +0.1 0.12 ± 39% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.13 ± 39% perf-profile.self.cycles-pp.xas_load 0.00 +0.1 0.14 ± 10% perf-profile.self.cycles-pp.native_sched_clock 0.00 +0.1 0.14 ± 85% perf-profile.self.cycles-pp.arch_scale_freq_tick 0.00 +0.1 0.15 ± 12% perf-profile.self.cycles-pp.read_tsc 0.00 +0.2 0.15 ± 18% perf-profile.self.cycles-pp.alloc_extent_state 0.00 +0.2 0.16 ± 9% perf-profile.self.cycles-pp.update_process_times 0.00 +0.2 0.17 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline 0.00 +0.2 0.19 ± 18% perf-profile.self.cycles-pp.find_next_bit 0.00 +0.2 0.19 ± 16% perf-profile.self.cycles-pp.btrfs_drop_pages 0.00 +0.2 0.22 ± 42% perf-profile.self.cycles-pp.tick_nohz_next_event 0.00 +0.2 0.23 ± 13% perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.2 0.25 ± 6% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.3 0.32 ± 12% perf-profile.self.cycles-pp.__reserve_bytes 0.00 +0.3 0.33 ± 41% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.00 +0.4 0.38 ±102% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.00 +0.4 0.45 ± 10% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.5 0.52 ± 25% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.6 0.62 ± 40% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.00 +0.7 0.69 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.00 +0.8 0.80 ± 11% perf-profile.self.cycles-pp._find_next_bit 0.00 +1.7 1.68 ± 17% perf-profile.self.cycles-pp._raw_spin_lock 0.00 +2.8 2.80 ± 11% perf-profile.self.cycles-pp.__percpu_counter_sum 0.00 +4.6 4.59 ± 39% perf-profile.self.cycles-pp.cpuidle_enter_state 0.00 +5.4 5.35 ± 31% perf-profile.self.cycles-pp.menu_select 0.00 +37.3 37.29 ± 9% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath fio.write_bw_MBps 2200 +--------------------------------------------------------------------+ |+.++++.+++.++++.++++.+++. +++.+++.++++.++++.+++.++++.+ +.++++.++++. | 2000 |-+ + + +| 1800 |-+ | | | 1600 |-+ | | | 1400 |-+ | | | 1200 |-+ | 1000 |-+ | | | 800 |O+ OOO OOO OOOO OOOO OOO OOOO OOO OOOO OOOO OOO O | | O | 600 +--------------------------------------------------------------------+ fio.write_iops 550000 +------------------------------------------------------------------+ | +.++ + + ++++ +++.+ + ++++ ++++ + +.+++ +++ .++++.++ +.+ | 500000 |-+ + + +| | | 450000 |-+ | 400000 |-+ | | | 350000 |-+ | | | 300000 |-+ | 250000 |-+ | | | 200000 |O+ OOOO OOOO OOOO OOOO OOOO OOOO OOOOO OOOO OOO | | O | 150000 +------------------------------------------------------------------+ fio.write_clat_mean_us 45000 +-------------------------------------------------------------------+ | O | 40000 |-+ OO OO OO | |O O OOOO OO O O OOOO OOOO OOO OOOO OOOO OO | 35000 |-+ | | | 30000 |-+ | | | 25000 |-+ | | | 20000 |-+ | | | 15000 |+.++++.++++.++++.++++.++++.++++.+++.++++.++++.++++.++++.++++.++++.+| | | 10000 +-------------------------------------------------------------------+ fio.write_clat_stddev 400000 +------------------------------------------------------------------+ | + + +. + + + + :| 350000 |-+.+ : ++ ++ + ++ +++ ++ + ++: ++ +.+ + ++ +.+ +.+ | | : ++.+ : : : : : : : : :+ : : : : : +. + | 300000 |-: : : : : : : : : + : : : : : + + : | 250000 |:+ :: :: :: :: : :: :: + | |: : : : : : : : | 200000 |++ + + + + + + + | | | 150000 |-+ | 100000 |-+ | | | 50000 |-+ | | | 0 +------------------------------------------------------------------+ fio.write_clat_90__us 50000 +-------------------------------------------------------------------+ | | 45000 |-+ OO O O OOO OOOO OO O OOOO OOO O O O OO | 40000 |O+ O O O O O O O O OO | | | 35000 |-+ | | | 30000 |-+ | | | 25000 |-+ | 20000 |-+ | | | 15000 |+.+ .++ .++ ++ + .++ ++. + ++ ++. + .++++.++ | | +++ ++ ++.+ +.++ + ++.+ ++ +.+ +.++ ++ + ++.+| 10000 +-------------------------------------------------------------------+ fio.write_clat_95__us 55000 +-------------------------------------------------------------------+ | O | 50000 |-+ | 45000 |O+ OOO OOOO OOOO OOOO OOOO OOOO OOO OOOO OOOO OO | | | 40000 |-+ | 35000 |-+ | | | 30000 |-+ | 25000 |-+ | | | 20000 |+. .+ + .+ + + + .+ .+ | 15000 |-+++++.++++ +++.+ ++.++++ +++.+ +.++++.+ ++.++ +.++++ +++ +++. | | +| 10000 +-------------------------------------------------------------------+ fio.write_clat_99__us 60000 +-------------------------------------------------------------------+ | O | 55000 |O+ O O O O O | 50000 |-+ OOO OO O OOOO OOOO OO O OOOO OOO OOO O OO O | | | 45000 |-+ | 40000 |-+ | | | 35000 |-+ | 30000 |-+ | | | 25000 |-+ | 20000 |+.++ +.++++.+++ .++++. +++.+++ .+++.+ ++.++++. +++.+ ++.++++.++++. | | + + + + + + + +| 15000 +-------------------------------------------------------------------+ fio.latency_20us_ 80 +----------------------------------------------------------------------+ |+ + + + + + + : | 70 |:+ : : : : : : :: + | 60 |:: : :: : :: + :: :: :: :: | |:: :: : : + :: : : : : : :: : +. + : | 50 |-+: + : : : : ::: : : : : : : : : : +: + + | | ++ + +.+: +.+ : ++ + :: +.+ : ++ : :: ++ : ++ : + : | 40 |-+ :+ + :+ + : + :+ : ++ + + :: ++ | | + + + + + + + + | 30 |-+ +| 20 |-+ | | | 10 |-+ | | O | 0 +----------------------------------------------------------------------+ fio.latency_50us_ 100 +---------------------------------------------------------------------+ 90 |-+O O | | | 80 |-+ | 70 |-+ | | | 60 |-+ | 50 |-+ | 40 |-+ | | | 30 |-+ | 20 |-+ | | | 10 |-+ | 0 +---------------------------------------------------------------------+ fio.latency_100us_ 9 +-----------------------------------------------------------------------+ | O | 8 |-+ | 7 |-+ | | | 6 |-+ | 5 |-+ | | | 4 |-+ | 3 |-+ | | O O | 2 |O+ O O O O | 1 |-+ O O O OO OO OO OO OO OOO OOO OO O | | O O O O O O OO | 0 +-----------------------------------------------------------------------+ fio.workload 6.8e+07 +-----------------------------------------------------------------+ |++.++++.+++++.++++.+++++.++++.+++++.++++.+++++.++++.+++++.++++.++| 6.6e+07 |-+ | | O | 6.4e+07 |-+ O | | O O O OOO O OO O OOOO O OO | 6.2e+07 |O+ O OOOO O O O | | OO O OO O O O O | 6e+07 |-+ | | | 5.8e+07 |-+ | | | 5.6e+07 |-+ | | O | 5.4e+07 +-----------------------------------------------------------------+ fio.time.system_time 2400 +--------------------------------------------------------------------+ 2200 |-+ | | | 2000 |-+ | 1800 |-+ | | | 1600 |-+ | 1400 |-+ | 1200 |-+ | | | 1000 |-+ | 800 |+. + + + + +. + .+ | |: ++ .+++. :++. :+.+ +.+ :+.+ + ++ .+ : ++ :++. ++ +++.++ | 600 |-+ ++ + ++ + + ++ + ++ +.+ + ++.+| 400 +--------------------------------------------------------------------+ fio.time.elapsed_time 320 +---------------------------------------------------------------------+ 300 |O+OOOO OOO OOO OOOO OOO OOO OOOO OOO OOOO OOO OOO | | | 280 |-+ | 260 |-+ | | | 240 |-+ | 220 |-+ | 200 |-+ | | | 180 |-+ | 160 |-+ | | | 140 |-.+++ .+ +.+++ .+++.+++. +++.+ +.+++ .+ +.+++. +++.+++.+++.+++ .+| 120 +---------------------------------------------------------------------+ fio.time.elapsed_time.max 320 +---------------------------------------------------------------------+ 300 |O+OOOO OOO OOO OOOO OOO OOO OOOO OOO OOOO OOO OOO | | | 280 |-+ | 260 |-+ | | | 240 |-+ | 220 |-+ | 200 |-+ | | | 180 |-+ | 160 |-+ | | | 140 |-.+++ .+ +.+++ .+++.+++. +++.+ +.+++ .+ +.+++. +++.+++.+++.+++ .+| 120 +---------------------------------------------------------------------+ fio.time.involuntary_context_switches 9000 +--------------------------------------------------------------------+ | O | 8000 |-+ O O O O OOOO O | 7000 |-+ OO | | O OO O O O O O OO | 6000 |-+ O OOO O O OOO O O O O | |O OO | 5000 |-+ | | | 4000 |-+ | 3000 |-+ | |+. + + + .+ +. + .+ .+ | 2000 |-+++++.+++.+ ++.++ +.+++.++ +.+++ +++.+++ +++.+ ++.+++ +++ + +. | | + +| 1000 +--------------------------------------------------------------------+ fio.time.file_system_outputs 5.4e+08 +-----------------------------------------------------------------+ 5.3e+08 |-+ | | | 5.2e+08 |-+ O | 5.1e+08 |-+ O O | | O O O OO OO O O O OO | 5e+08 |O+ O OO OOOO O O O O O | 4.9e+08 |-+ OO O OO O O O O | 4.8e+08 |-+ | | | 4.7e+08 |-+ | 4.6e+08 |-+ | | | 4.5e+08 |-+ | 4.4e+08 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year, 9 months

1
0
0 / 0

[x86/io_apic] a27dca645d: Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: a27dca645d2c0f31abb7858aa0e10b2fa0f2f659 ("x86/io_apic: Cleanup trigger/polarity helpers") https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/apic in testcase: hwsim version: hwsim-x86_64-537ab94-1_20201012 with following parameters: group: hwsim-27 ucode: 0xdc on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 2.160102] DMAR: DRHD: handling fault status reg 2 [ 2.165059] DMAR: [INTR-REMAP] Request device [f0:1f.0] fault index 0 [fault reason 37] Blocked a compatibility format interrupt request [ 2.177383] Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC [ 2.185988] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I 5.10.0-rc1-00020-ga27dca645d2c #1 [ 2.195458] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.1.1 10/07/2015 [ 2.202937] Call Trace: [ 2.205476] dump_stack+0x57/0x6a [ 2.208878] panic+0x102/0x2d2 [ 2.212021] panic_if_irq_remap.cold+0x5/0x5 [ 2.216375] check_timer+0x1f6/0x694 [ 2.220038] ? printk+0x58/0x6f [ 2.223265] setup_IO_APIC+0x17b/0x1c3 [ 2.227099] x86_late_time_init+0x20/0x30 [ 2.231194] start_kernel+0x40c/0x4c7 [ 2.234940] secondary_startup_64_no_verify+0xb8/0xbb [ 2.240076] ACPI MEMORY or I/O RESET_REG. To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

1 year, 9 months

1
0
0 / 0

[cma] 1ea6c22c9b: page_allocation_failure:order:#,mode:#(__GFP_RECLAIMABLE),nodemask=(null)

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 1ea6c22c9b85ec176bb78d7076be06a4142f8bdd ("[PATCH 1/2] cma: redirect page allocation to CMA") url: https://github.com/0day-ci/linux/commits/Chris-Goldsworthy/Increasing-CMA... base: https://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git for-next in testcase: boot on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +---------------------------------------------------------------------------+------------+------------+ | | 67b6d2ce11 | 1ea6c22c9b | +---------------------------------------------------------------------------+------------+------------+ | boot_successes | 6 | 0 | | boot_failures | 0 | 8 | | page_allocation_failure:order:#,mode:#(__GFP_RECLAIMABLE),nodemask=(null) | 0 | 8 | | Mem-Info | 0 | 8 | | kernel_BUG_at_kernel/workqueue.c | 0 | 8 | | invalid_opcode:#[##] | 0 | 8 | | EIP:workqueue_init_early | 0 | 8 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 8 | +---------------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 2.108390] Memory: 8203240K/8388088K available (12819K kernel code, 6610K rwdata, 11292K rodata, 628K init, 12064K bss, 184848K reserved, 0K cma-reserved, 7481224K highmem) [ 2.111999] Checking if this processor honours the WP bit even in supervisor mode...Ok. [ 2.113918] random: get_random_u32 called from kmem_cache_open+0x1f/0x240 with crng_init=0 [ 2.114387] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 2.117656] Kernel/User page tables isolation: enabled [ 2.119610] swapper: page allocation failure: order:0, mode:0x10(__GFP_RECLAIMABLE), nodemask=(null) [ 2.121604] CPU: 0 PID: 0 Comm: swapper Not tainted 5.10.0-rc1-00365-g1ea6c22c9b85 #1 [ 2.123401] Call Trace: [ 2.123990] dump_stack+0x1b/0x1d [ 2.124787] warn_alloc+0x81/0xd9 [ 2.125599] __alloc_pages_slowpath+0x79c/0x7a9 [ 2.126690] ? get_page_from_freelist+0xb8/0x20d [ 2.127750] __alloc_pages_nodemask+0x107/0x188 [ 2.128795] __alloc_pages_node+0x17/0x1c [ 2.129757] alloc_slab_page+0x26/0x4e [ 2.130649] allocate_slab+0x80/0x27e [ 2.131521] ___slab_alloc+0x247/0x2ec [ 2.132605] ? radix_tree_node_alloc+0x5e/0x8e [ 2.133915] ? validate_chain+0x5a8/0x5c3 [ 2.134838] __slab_alloc+0x34/0x4d [ 2.135830] ? __slab_alloc+0x34/0x4d [ 2.136909] ? fs_reclaim_release+0x8/0x13 [ 2.137910] kmem_cache_alloc+0x46/0x157 [ 2.138854] ? radix_tree_node_alloc+0x5e/0x8e [ 2.140103] radix_tree_node_alloc+0x5e/0x8e [ 2.141369] idr_get_free+0xc1/0x21a [ 2.142230] idr_alloc_u32+0x4d/0x80 [ 2.143016] idr_alloc+0x30/0x3e [ 2.143751] worker_pool_assign_id+0x37/0x47 [ 2.144682] workqueue_init_early+0x9f/0x1f6 [ 2.145707] start_kernel+0x206/0x467 [ 2.146592] ? early_idt_handler_common+0x44/0x44 [ 2.147671] i386_start_kernel+0x42/0x44 [ 2.148607] startup_32_smp+0x164/0x168 [ 2.149567] Mem-Info: [ 2.150101] active_anon:0 inactive_anon:0 isolated_anon:0 [ 2.150101] active_file:0 inactive_file:0 isolated_file:0 [ 2.150101] unevictable:0 dirty:0 writeback:0 [ 2.150101] slab_reclaimable:0 slab_unreclaimable:88 [ 2.150101] mapped:0 shmem:0 pagetables:0 bounce:0 [ 2.150101] free:2050715 free_pcp:0 free_cma:0 [ 2.156588] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:0kB all_unreclaimable? no [ 2.162313] Normal free:721636kB min:0kB low:0kB high:0kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:906864kB managed:722016kB mlocked:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 2.162318] lowmem_reserve[]: 0 0 0 [ 2.169009] HighMem free:7481224kB min:0kB low:0kB high:0kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:7481224kB managed:7481224kB mlocked:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 2.169014] lowmem_reserve[]: 0 0 0 [ 2.175914] Normal: 1*4kB (M) 2*8kB (M) 1*16kB (U) 2*32kB (UM) 2*64kB (M) 2*128kB (UM) 1*256kB (M) 0*512kB 2*1024kB (UM) 1*2048kB (M) 175*4096kB (M) = 721636kB [ 2.179229] HighMem: 0*4kB 1*8kB (M) 0*16kB 0*32kB 0*64kB 1*128kB (M) 1*256kB (M) 1*512kB (M) 1*1024kB (M) 2*2048kB (M) 1825*4096kB (M) = 7481224kB [ 2.182343] 0 total pagecache pages [ 2.183068] 2097022 pages RAM [ 2.183799] 1870306 pages HighMem/MovableOnly [ 2.184853] 46212 pages reserved [ 2.185675] 0 pages cma reserved [ 2.186442] SLUB: Unable to allocate memory on node -1, gfp=0xcc0(GFP_KERNEL) [ 2.188136] cache: radix_tree_node, object size: 300, buffer size: 480, default order: 0, min order: 0 [ 2.190339] node 0: slabs: 0, objs: 0, free: 0 [ 2.191414] ------------[ cut here ]------------ [ 2.192442] kernel BUG at kernel/workqueue.c:5944! [ 2.193614] invalid opcode: 0000 [#1] PTI [ 2.194549] CPU: 0 PID: 0 Comm: swapper Not tainted 5.10.0-rc1-00365-g1ea6c22c9b85 #1 [ 2.196348] EIP: workqueue_init_early+0xa3/0x1f6 [ 2.197445] Code: 8b 83 fc 01 00 00 8b 54 bd e8 89 10 31 d2 b8 a0 40 7e c2 c7 43 30 00 00 00 00 e8 56 7a e4 fe 89 d8 e8 f1 64 21 fe 85 c0 74 02 <0f> 0b b8 a0 40 7e c2 81 c3 18 02 00 00 e8 db 76 e4 fe 83 ff 02 be [ 2.201824] EAX: fffffff4 EBX: c27e3bc0 ECX: 00000cc0 EDX: 07ffffff [ 2.203273] ESI: 00000000 EDI: 00000001 EBP: c278df7c ESP: c278df68 [ 2.204737] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00210082 [ 2.206288] CR0: 80050033 CR2: ffbff000 CR3: 02ec4000 CR4: 000006b0 [ 2.207717] Call Trace: [ 2.208256] start_kernel+0x206/0x467 [ 2.209038] ? early_idt_handler_common+0x44/0x44 [ 2.210192] i386_start_kernel+0x42/0x44 [ 2.211088] startup_32_smp+0x164/0x168 [ 2.211986] Modules linked in: [ 2.212723] random: get_random_bytes called from init_oops_id+0x23/0x3b with crng_init=0 [ 2.212729] ---[ end trace 7c274a1c59fae664 ]--- [ 2.215660] EIP: workqueue_init_early+0xa3/0x1f6 [ 2.216748] Code: 8b 83 fc 01 00 00 8b 54 bd e8 89 10 31 d2 b8 a0 40 7e c2 c7 43 30 00 00 00 00 e8 56 7a e4 fe 89 d8 e8 f1 64 21 fe 85 c0 74 02 <0f> 0b b8 a0 40 7e c2 81 c3 18 02 00 00 e8 db 76 e4 fe 83 ff 02 be [ 2.220942] EAX: fffffff4 EBX: c27e3bc0 ECX: 00000cc0 EDX: 07ffffff [ 2.222365] ESI: 00000000 EDI: 00000001 EBP: c278df7c ESP: c278df68 [ 2.223834] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00210082 [ 2.225389] CR0: 80050033 CR2: ffbff000 CR3: 02ec4000 CR4: 000006b0 [ 2.226784] Kernel panic - not syncing: Fatal exception Kboot worker: lkp-worker51 Elapsed time: 60 kvm=( To reproduce: # build kernel cd linux cp config-5.10.0-rc1-00365-g1ea6c22c9b85 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 9 months

2
1
0 / 0

Re: [perf metricgroup] fcc9c5243c: perf-sanity-tests.Parse_and_process_metrics.fail

by Ian Rogers

On Tue, Oct 20, 2020 at 1:56 AM kajoljain <kjain(a)linux.ibm.com> wrote: > > > > On 10/19/20 9:50 PM, Ian Rogers wrote: > > On Mon, Oct 19, 2020 at 2:51 AM John Garry <john.garry(a)huawei.com> wrote: > >> > >> On 19/10/2020 00:30, Ian Rogers wrote: > >>> On Sun, Oct 18, 2020 at 1:51 AM kernel test robot <rong.a.chen(a)intel.com> wrote: > >>>> > >>>> Greeting, > >>>> > >>>> FYI, we noticed the following commit (built with gcc-9): > >>>> > >>>> commit: fcc9c5243c478f104014daf4d23db86098d2aef0 ("perf metricgroup: Hack a fix for aliases when covering multiple PMUs") > >>>> url: https://github.com/0day-ci/linux/commits/John-Garry/perf-pmu-events-Suppo... > >>>> > >>>> > >>>> in testcase: perf-sanity-tests > >>>> version: perf-x86_64-c85fb28b6f99-1_20201008 > >>>> with following parameters: > >>>> > >>>> perf_compiler: gcc > >>>> ucode: 0xdc > >>>> > >>>> > >>>> > >>>> on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory > >>>> > >>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): > >>> > >>> I believe this is a Skylake and there is a known bug in the Skylake > >>> metric DRAM_Parallel_Reads as described here: > >>> https://lore.kernel.org/lkml/CAP-5=fXejVaQa9qfW66cY77qB962+jbe8tT5bsLoOOc... > >>> Fixing the bug needs more knowledge than what is available in manuals. > >>> Hopefully Intel can take a look. > >>> > >>> Thanks, > >>> Ian > >> > >> So this named patch ("perf metricgroup: Hack a fix for aliases...") is > >> breaking test #67 on my machine also, which is a broadwell. > > > > Thanks for taking a look John. If you want help you can send the > > output of "perf test 67 -vvv" to me. It is possible Broadwell has > > similar glitches in the json to Skylake. I tested the original test on > > server parts as I can access them as cloud machines. > > > >> I will have a look, but I was hoping that Ian would have a proper fix > >> for this on top of ("perf metricgroup: Fix uncore metric expressions"), > >> which now looks to be merged. > > > > I still have these changes to look at in my inbox but I'm assuming > > they're good :-) Sorry for not getting to them, but it's good they are > > merged. > > Hi Ian, > Checked in upstream kernel with your fix patch, in powerpc also test case 67 is passing. > But I am getting issue in test 10 for powerpc > > [command]# ./perf test 10 > 10: PMU events : > 10.1: PMU event table sanity : Ok > 10.2: PMU event map aliases : Ok > 10.3: Parsing of PMU event table metrics : Skip (some metrics failed) > 10.4: Parsing of PMU event table metrics with fake PMUs : FAILED! > > Was debugging it, issue is with commit e1c92a7fbbc5 perf tests: Add another metric parsing test. > > So, there we are passing different runtime parameter value in "expr__find_other and expr__parse" > in function `metric_parse_fake`. I believe we need to send same value. > I will send fix patch for the same. > > Thanks, > Kajol Jain Thanks, the fake support was done by Jiri. I do try to test on Power 8. The awesome thing, aside from the testing nit fixes, is that the metrics will actually work once the test is passing :-). They may of course report junk. Thanks, Ian > > > > Thanks, > > Ian > > > >> Thanks! > >> > >>> > >>>> > >>>> > >>>> If you fix the issue, kindly add following tag > >>>> Reported-by: kernel test robot <rong.a.chen(a)intel.com> > >>>> > >>>> > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 67 > >>>> 67: Parse and process metrics : FAILED! > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 68 > >>>> 68: x86 rdpmc : Ok > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 69 > >>>> 69: Convert perf time to TSC : Ok > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 70 > >>>> 70: DWARF unwind : Ok > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 71 > >>>> 71: x86 instruction decoder - new instructions : Ok > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 72 > >>>> 72: Intel PT packet decoder : Ok > >>>> 2020-10-16 19:31:52 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 73 > >>>> 73: x86 bp modify : Ok > >>>> 2020-10-16 19:31:53 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 74 > >>>> 74: probe libc's inet_pton & backtrace it with ping : Ok > >>>> 2020-10-16 19:31:54 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-fcc9c5243c478f104014daf4d23db86098d2aef0/tools/perf/perf test 75 > >>>> 75: Zstd perf.data compression/decompression : Ok > >>>> > >>>> > >>>> > >>>> To reproduce: > >>>> > >>>> git clone https://github.com/intel/lkp-tests.git > >>>> cd lkp-tests > >>>> bin/lkp install job.yaml # job file is attached in this email > >>>> bin/lkp run job.yaml > >>>> > >>>> > >>>> > >>>> Thanks, > >>>> Rong Chen > >>>> > >>> . > >>> > >>

1 year, 9 months

2
3
0 / 0

[x86/ioapic] b643128b91: Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC

by lkp

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: b643128b917ca8f1c8b1e14af64ebdc81147b2d1 ("x86/ioapic: Use irq_find_matching_fwspec() to find remapping irqdomain") https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/apic in testcase: nvml version: nvml-x86_64-9a558d859-1_20201008 with following parameters: test: pmem group: unicode nr_pmem: 1 fs: ext4 mount_option: dax bp_memmap: 32G!4G ucode: 0x7000019 on test machine: 16 threads Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz with 48G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 3.148819] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 3.154825] DMAR: DRHD: handling fault status reg 2 [ 3.159701] DMAR: [INTR-REMAP] Request device [f0:1f.7] fault index 0 [fault reason 37] Blocked a compatibility format interrupt request [ 3.173870] Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC [ 3.182381] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.10.0-rc1-00029-gb643128b917c #1 [ 3.190370] Hardware name: Supermicro SYS-5018D-FN4T/X10SDV-8C-TLN4F, BIOS 1.1 03/02/2016 [ 3.198534] Call Trace: [ 3.200983] dump_stack+0x57/0x6a [ 3.204298] panic+0x102/0x2d2 [ 3.207349] panic_if_irq_remap.cold+0x5/0x5 [ 3.211613] check_timer+0x1f6/0x694 [ 3.215184] ? printk+0x58/0x6f [ 3.218320] setup_IO_APIC+0x17b/0x1c3 [ 3.222067] x86_late_time_init+0x20/0x30 [ 3.226077] start_kernel+0x40c/0x4c7 [ 3.229734] secondary_startup_64_no_verify+0xb8/0xbb To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

1 year, 9 months

3
2
0 / 0

[resource] 22b17dc667: Kernel panic - not syncing: Fatal exception

by lkp

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 22b17dc667d36418ccabb9c668c4b489185fb40a ("[PATCH v5 13/15] resource: Move devmem revoke code to resource framework") url: https://github.com/0day-ci/linux/commits/Daniel-Vetter/follow_pfn-and-oth... base: git://linuxtv.org/media_tree.git master in testcase: fsmark version: fsmark-x86_64-3.3-1_20201007 with following parameters: iterations: 1x nr_threads: 1t disk: 1BRD_48G fs: f2fs fs2: nfsv4 filesize: 4M test_size: 40G sync_method: NoSync cpufreq_governor: performance ucode: 0x5002f01 test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload. test-url: https://sourceforge.net/projects/fsmark/ on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 28.644165] systemd[1]: RTC configured in localtime, applying delta of 0 minutes to system time. [ 28.699473] #PF: supervisor read access in kernel mode [ 28.704611] #PF: error_code(0x0000) - not-present page [ 28.709749] PGD 0 P4D 0 [ 28.712291] Oops: 0000 [#1] SMP NOPTI [ 28.715956] CPU: 0 PID: 1 Comm: systemd Not tainted 5.10.0-rc1-00015-g22b17dc667d3 #1 [ 28.723793] RIP: 0010:do_dentry_open+0x1c9/0x360 [ 28.728410] Code: 84 82 01 00 00 81 ca 00 00 04 00 89 53 44 48 8b 83 f0 00 00 00 81 63 40 3f fc ff ff 48 8d bb 98 00 00 00 c7 43 34 00 00 00 00 <48> 8b 00 48 8b 70 30 e8 2b cb f4 ff f6 43 41 40 74 5a 48 8b 83 f0 [ 28.747157] RSP: 0018:ffffc9000006fcc8 EFLAGS: 00010206 [ 28.752380] RAX: 0000000000000000 RBX: ffff8881502ad400 RCX: 0000000000000000 [ 28.759506] RDX: 00000000000a201d RSI: ffffffff8284d260 RDI: ffff8881502ad498 [ 28.766639] RBP: ffff88a485a06490 R08: 0000000000000000 R09: ffffffff8284d260 [ 28.773769] R10: ffffc9000006fcc8 R11: 756c6176006d656d R12: 0000000000000000 [ 28.780895] R13: ffffffff8133ddc0 R14: ffff8881502ad410 R15: ffff8881502ad400 [ 28.788028] FS: 00007ff54afa1940(0000) GS:ffff888c4f600000(0000) knlGS:0000000000000000 [ 28.796113] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 28.801858] CR2: 0000000000000000 CR3: 0000000100120003 CR4: 00000000007706f0 [ 28.808983] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 28.816114] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 28.823239] PKRU: 55555554 [ 28.825952] Call Trace: [ 28.828412] path_openat+0xaa8/0x10a0 [ 28.832073] do_filp_open+0x91/0x100 [ 28.835653] ? acpi_os_wait_semaphore+0x48/0x80 [ 28.840186] ? __check_object_size+0x136/0x160 [ 28.844631] do_sys_openat2+0x20d/0x2e0 [ 28.848470] do_sys_open+0x44/0x80 [ 28.851878] do_syscall_64+0x33/0x40 [ 28.855457] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 28.860509] RIP: 0033:0x7ff54c1521ae [ 28.864086] Code: 25 00 00 41 00 3d 00 00 41 00 74 48 48 8d 05 59 65 0d 00 8b 00 85 c0 75 69 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a6 00 00 00 48 8b 4c 24 28 64 48 33 0c 25 [ 28.882833] RSP: 002b:00007ffd1c9586d0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 [ 28.890399] RAX: ffffffffffffffda RBX: 00007ffd1c9587d0 RCX: 00007ff54c1521ae [ 28.897531] RDX: 0000000000080000 RSI: 00007ff54bfa0e5a RDI: 00000000ffffff9c [ 28.904662] RBP: 00007ffd1c9587d8 R08: 000000000000021f R09: 000055f837cf4290 [ 28.911796] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000056dd9000 [ 28.918927] R13: 00000000ffffffff R14: 00007ffd1c9587d0 R15: 0000000000000002 [ 28.926060] Modules linked in: ip_tables [ 28.929986] CR2: 0000000000000000 mDebian GNU/Linu [ 28.933416] ---[ end trace 94e4f9aa3df66098 ]--- [ 28.939355] RIP: 0010:do_dentry_open+0x1c9/0x360 [ 28.943975] Code: 84 82 01 00 00 81 ca 00 00 04 00 89 53 44 48 8b 83 f0 00 00 00 81 63 40 3f fc ff ff 48 8d bb 98 00 00 00 c7 43 34 00 00 00 00 <48> 8b 00 48 8b 70 30 e8 2b cb f4 ff f6 43 41 40 74 5a 48 8b 83 f0 [ 28.962721] RSP: 0018:ffffc9000006fcc8 EFLAGS: 00010206 [ 28.967948] RAX: 0000000000000000 RBX: ffff8881502ad400 RCX: 0000000000000000 [ 28.975079] RDX: 00000000000a201d RSI: ffffffff8284d260 RDI: ffff8881502ad498 [ 28.982211] RBP: ffff88a485a06490 R08: 0000000000000000 R09: ffffffff8284d260 [ 28.989337] R10: ffffc9000006fcc8 R11: 756c6176006d656d R12: 0000000000000000 [ 28.996467] R13: ffffffff8133ddc0 R14: ffff8881502ad410 R15: ffff8881502ad400 [ 29.003592] FS: 00007ff54afa1940(0000) GS:ffff888c4f600000(0000) knlGS:0000000000000000 [ 29.011668] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 29.017409] CR2: 0000000000000000 CR3: 0000000100120003 CR4: 00000000007706f0 [ 29.024539] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 29.031671] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 29.038804] PKRU: 55555554 [ 29.041508] Kernel panic - not syncing: Fatal exception ACPI MEMORY or I/O RESET_REG. To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, oliver.sang(a)intel.com

1 year, 9 months

3
2
0 / 0

[btrfs] c75e839414: aim7.jobs-per-min -9.1% regression

by kernel test robot

Greeting, FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit: commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the subvol_srcu") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: disk: 4BRD_12G md: RAID0 fs: btrfs test: disk_wrt load: 1500 cpufreq_governor: performance ucode: 0x500002c test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c commit: efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree lock") c75e839414 ("btrfs: kill the subvol_srcu") efc3453494af7818 c75e839414d3610e6487ae31451 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 3:9 -33% :8 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x %stddev %change %stddev \ | \ 29509 ± 2% -9.1% 26837 ± 2% aim7.jobs-per-min 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time.max 4883135 ± 10% +37.9% 6735464 ± 7% aim7.time.involuntary_context_switches 56288 ± 2% +10.5% 62202 ± 2% aim7.time.system_time 2344783 +6.5% 2497364 ± 2% aim7.time.voluntary_context_switches 62337721 ± 2% +9.8% 68456490 ± 2% turbostat.IRQ 431.56 ± 6% +22.3% 527.88 ± 4% vmstat.procs.r 27340 ± 2% +11.2% 30397 ± 2% vmstat.system.cs 226804 ± 6% +21.7% 276057 ± 4% meminfo.Active(file) 221309 ± 6% +22.3% 270668 ± 4% meminfo.Dirty 720.89 ±111% +49.3% 1076 ± 73% meminfo.Mlocked 14278 ± 2% -8.3% 13094 ± 2% meminfo.max_used_kB 57228 ± 6% +22.7% 70195 ± 5% numa-meminfo.node0.Active(file) 55433 ± 6% +21.6% 67431 ± 4% numa-meminfo.node0.Dirty 56152 ± 6% +21.4% 68180 ± 5% numa-meminfo.node1.Active(file) 55001 ± 6% +22.5% 67397 ± 4% numa-meminfo.node1.Dirty 56373 ± 6% +21.7% 68594 ± 4% numa-meminfo.node2.Active(file) 55222 ± 7% +22.6% 67726 ± 4% numa-meminfo.node2.Dirty 56671 ± 6% +20.5% 68317 ± 3% numa-meminfo.node3.Active(file) 55285 ± 6% +21.8% 67355 ± 4% numa-meminfo.node3.Dirty 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_active_file 55342 ± 6% +22.3% 67662 ± 4% proc-vmstat.nr_dirty 402316 +2.1% 410951 proc-vmstat.nr_file_pages 180.22 ±111% +49.4% 269.25 ± 73% proc-vmstat.nr_mlock 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_zone_active_file 54680 ± 6% +22.8% 67168 ± 4% proc-vmstat.nr_zone_write_pending 3144381 ± 2% +6.1% 3335275 proc-vmstat.pgactivate 1387558 ± 2% +7.9% 1496754 ± 2% proc-vmstat.pgfault 983.33 ± 4% +5.4% 1036 proc-vmstat.unevictable_pgs_culled 14331 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_active_file 13884 ± 6% +21.6% 16884 ± 4% numa-vmstat.node0.nr_dirty 14330 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_zone_active_file 13714 ± 6% +22.2% 16755 ± 4% numa-vmstat.node0.nr_zone_write_pending 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_active_file 13763 ± 6% +22.3% 16838 ± 4% numa-vmstat.node1.nr_dirty 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_zone_active_file 13599 ± 6% +23.0% 16726 ± 4% numa-vmstat.node1.nr_zone_write_pending 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_active_file 13786 ± 6% +22.6% 16906 ± 4% numa-vmstat.node2.nr_dirty 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_zone_active_file 13617 ± 7% +23.3% 16789 ± 4% numa-vmstat.node2.nr_zone_write_pending 14151 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_active_file 13790 ± 6% +22.1% 16832 ± 4% numa-vmstat.node3.nr_dirty 14152 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_zone_active_file 13627 ± 6% +22.5% 16696 ± 4% numa-vmstat.node3.nr_zone_write_pending 828631 ± 16% +49.9% 1241934 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 1.33 ± 30% +96.9% 2.61 ± 14% sched_debug.cfs_rq:/.nr_spread_over.avg 831152 ± 16% +49.7% 1244586 ± 8% sched_debug.cfs_rq:/.spread0.stddev 1.92 ± 14% +28.4% 2.47 ± 8% sched_debug.cpu.nr_running.avg 5.00 ± 9% +29.7% 6.48 ± 9% sched_debug.cpu.nr_running.max 0.90 ± 9% +16.0% 1.05 ± 3% sched_debug.cpu.nr_running.stddev 21790 ± 11% +16.7% 25440 ± 2% sched_debug.cpu.nr_switches.avg 19761 ± 11% +17.9% 23289 ± 2% sched_debug.cpu.nr_switches.min 20403 ± 12% +18.1% 24101 ± 2% sched_debug.cpu.sched_count.avg 27825 ± 10% +15.8% 32227 ± 4% sched_debug.cpu.sched_count.max 18995 ± 13% +18.3% 22463 ± 2% sched_debug.cpu.sched_count.min 1143 ± 11% +15.6% 1322 ± 4% sched_debug.cpu.sched_count.stddev 1404 ± 7% -31.0% 969.66 ± 11% sched_debug.cpu.sched_goidle.avg 1186 ± 8% -36.0% 759.25 ± 14% sched_debug.cpu.sched_goidle.min 16821 ± 14% +20.6% 20284 ± 6% sched_debug.cpu.ttwu_count.max 2790 ± 11% +21.7% 3395 ± 7% sched_debug.cpu.ttwu_count.stddev 398.19 ± 16% +28.3% 510.73 ± 10% sched_debug.cpu.ttwu_local.stddev 43238516 ± 5% +19.0% 51459028 ± 4% perf-stat.i.cache-misses 83870573 ± 4% +13.8% 95436555 ± 4% perf-stat.i.cache-references 27181 ± 2% +11.1% 30198 ± 2% perf-stat.i.context-switches 3008 ± 2% -7.1% 2794 perf-stat.i.cpu-migrations 12612 ± 5% -15.6% 10641 ± 3% perf-stat.i.cycles-between-cache-misses 1.1e+09 ± 2% -7.3% 1.019e+09 ± 2% perf-stat.i.dTLB-stores 10684629 ± 2% +10.8% 11842781 ± 2% perf-stat.i.node-load-misses 641429 ± 2% +24.4% 798190 ± 2% perf-stat.i.node-loads 5915955 ± 2% -6.3% 5543368 ± 2% perf-stat.i.node-store-misses 380971 -6.7% 355629 ± 2% perf-stat.i.node-stores 1.66 ± 4% +14.4% 1.90 ± 3% perf-stat.overall.MPKI 13064 ± 5% -15.8% 10999 ± 4% perf-stat.overall.cycles-between-cache-misses 43484558 ± 5% +19.2% 51837476 ± 4% perf-stat.ps.cache-misses 83278267 ± 4% +14.4% 95253129 ± 3% perf-stat.ps.cache-references 27414 ± 2% +11.2% 30486 ± 2% perf-stat.ps.context-switches 3028 ± 2% -7.0% 2817 perf-stat.ps.cpu-migrations 1.104e+09 ± 2% -7.3% 1.023e+09 ± 2% perf-stat.ps.dTLB-stores 4394 -1.8% 4315 perf-stat.ps.minor-faults 10766873 ± 2% +11.0% 11947268 ± 2% perf-stat.ps.node-load-misses 645940 ± 2% +24.7% 805392 ± 2% perf-stat.ps.node-loads 5965618 ± 2% -6.2% 5595562 ± 2% perf-stat.ps.node-store-misses 383092 -6.5% 358124 ± 2% perf-stat.ps.node-stores 4395 -1.8% 4315 perf-stat.ps.page-faults 1.538e+13 ± 2% +10.0% 1.692e+13 ± 2% perf-stat.total.instructions 27.39 -0.1 27.28 perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.13 +0.1 35.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 35.23 +0.1 35.34 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 35.48 +0.1 35.60 perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.08 +0.1 35.22 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 34.97 +0.1 35.12 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 35.57 +0.1 35.72 perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.10 +0.2 35.26 perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.11 +0.2 35.27 perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 98.71 +0.2 98.88 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.69 +0.2 98.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.74 +0.2 98.91 perf-profile.calltrace.cycles-pp.write 98.71 +0.2 98.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 98.68 +0.2 98.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.64 +0.2 98.82 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 98.63 +0.2 98.81 perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 98.58 +0.2 98.77 perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 27.39 -0.1 27.28 perf-profile.children.cycles-pp.btrfs_dirty_pages 0.56 ± 5% -0.1 0.46 ± 9% perf-profile.children.cycles-pp.btrfs_get_extent 0.51 ± 5% -0.1 0.41 ± 9% perf-profile.children.cycles-pp.btrfs_lookup_file_extent 0.52 ± 5% -0.1 0.43 ± 9% perf-profile.children.cycles-pp.btrfs_search_slot 0.44 ± 6% -0.1 0.35 ± 11% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 0.43 ± 6% -0.1 0.34 ± 11% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.19 ± 15% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.start_secondary 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter_state 0.41 ± 7% -0.1 0.33 ± 8% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.46 ± 7% -0.1 0.38 ± 8% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.secondary_startup_64 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.cpu_startup_entry 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.do_idle 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.children.cycles-pp.intel_idle 0.29 ± 9% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.osq_lock 0.19 ± 6% -0.0 0.15 ± 16% perf-profile.children.cycles-pp.finish_wait 0.39 ± 5% -0.0 0.35 ± 6% perf-profile.children.cycles-pp.do_unlinkat 0.39 ± 6% -0.0 0.35 ± 7% perf-profile.children.cycles-pp.unlink 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_open 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_openat2 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_filp_open 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.path_openat 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.creat 0.00 +0.0 0.05 perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size 99.65 +0.1 99.73 perf-profile.children.cycles-pp.do_syscall_64 99.66 +0.1 99.74 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 35.62 +0.1 35.74 perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 35.57 +0.1 35.72 perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 97.55 +0.2 97.72 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 98.74 +0.2 98.92 perf-profile.children.cycles-pp.write 98.70 +0.2 98.88 perf-profile.children.cycles-pp.ksys_write 98.69 +0.2 98.87 perf-profile.children.cycles-pp.vfs_write 98.65 +0.2 98.83 perf-profile.children.cycles-pp.new_sync_write 98.63 +0.2 98.81 perf-profile.children.cycles-pp.btrfs_file_write_iter 98.59 +0.2 98.78 perf-profile.children.cycles-pp.btrfs_buffered_write 97.60 +0.2 97.79 perf-profile.children.cycles-pp._raw_spin_lock 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.self.cycles-pp.intel_idle 0.28 ± 9% -0.1 0.21 ± 12% perf-profile.self.cycles-pp.osq_lock 0.45 ± 2% -0.0 0.41 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 135788 ± 2% +13.1% 153639 ± 3% softirqs.CPU0.RCU 115547 ± 3% +8.0% 124768 ± 4% softirqs.CPU0.TIMER 135243 ± 3% +14.0% 154188 ± 2% softirqs.CPU1.RCU 136056 ± 3% +12.4% 152960 ± 3% softirqs.CPU10.RCU 134253 ± 3% +14.2% 153260 ± 3% softirqs.CPU100.RCU 134668 ± 4% +13.7% 153157 ± 2% softirqs.CPU101.RCU 109995 ± 3% +7.9% 118637 ± 3% softirqs.CPU101.TIMER 134989 ± 4% +13.9% 153700 ± 2% softirqs.CPU102.RCU 109876 ± 3% +8.9% 119608 ± 2% softirqs.CPU102.TIMER 134868 ± 4% +13.8% 153427 ± 3% softirqs.CPU103.RCU 109933 ± 3% +8.2% 118946 ± 3% softirqs.CPU103.TIMER 134454 ± 3% +14.0% 153285 ± 3% softirqs.CPU104.RCU 109731 ± 2% +8.4% 118965 ± 3% softirqs.CPU104.TIMER 134878 ± 3% +13.6% 153283 ± 3% softirqs.CPU105.RCU 134096 ± 3% +14.1% 153022 ± 3% softirqs.CPU106.RCU 109821 ± 2% +8.2% 118845 ± 3% softirqs.CPU106.TIMER 133905 ± 3% +15.0% 154004 ± 3% softirqs.CPU107.RCU 134117 ± 3% +14.3% 153232 ± 3% softirqs.CPU108.RCU 109545 ± 2% +8.8% 119182 ± 3% softirqs.CPU108.TIMER 134574 ± 3% +13.8% 153202 ± 3% softirqs.CPU109.RCU 109804 ± 3% +8.2% 118760 ± 3% softirqs.CPU109.TIMER 135106 ± 3% +14.8% 155124 ± 3% softirqs.CPU11.RCU 111746 ± 4% +7.8% 120460 ± 3% softirqs.CPU11.TIMER 134482 ± 4% +14.3% 153687 ± 3% softirqs.CPU110.RCU 110314 ± 3% +8.1% 119241 ± 3% softirqs.CPU110.TIMER 134464 ± 3% +14.3% 153634 ± 3% softirqs.CPU111.RCU 110626 ± 3% +8.3% 119815 ± 3% softirqs.CPU111.TIMER 134895 ± 3% +14.4% 154257 ± 3% softirqs.CPU112.RCU 110040 ± 3% +14.6% 126058 ± 16% softirqs.CPU112.TIMER 134644 ± 3% +13.6% 152993 ± 3% softirqs.CPU113.RCU 109693 ± 3% +8.1% 118569 ± 3% softirqs.CPU113.TIMER 134313 ± 3% +13.9% 152993 ± 3% softirqs.CPU114.RCU 109711 ± 3% +8.6% 119143 ± 3% softirqs.CPU114.TIMER 134559 ± 3% +13.8% 153115 ± 3% softirqs.CPU115.RCU 110226 ± 3% +8.2% 119221 ± 3% softirqs.CPU115.TIMER 134087 ± 3% +14.2% 153077 ± 3% softirqs.CPU116.RCU 110225 ± 3% +9.0% 120184 ± 3% softirqs.CPU116.TIMER 134271 ± 3% +14.1% 153220 ± 3% softirqs.CPU117.RCU 109179 ± 2% +8.3% 118274 ± 3% softirqs.CPU117.TIMER 134281 ± 3% +14.1% 153230 ± 3% softirqs.CPU118.RCU 134415 ± 3% +14.0% 153215 ± 3% softirqs.CPU119.RCU 109514 ± 2% +8.3% 118643 ± 3% softirqs.CPU119.TIMER 135560 ± 3% +13.5% 153874 ± 3% softirqs.CPU12.RCU 110316 ± 2% +8.5% 119734 ± 3% softirqs.CPU12.TIMER 134177 ± 3% +14.6% 153815 ± 3% softirqs.CPU120.RCU 134367 ± 3% +15.0% 154493 ± 3% softirqs.CPU121.RCU 133972 ± 3% +14.3% 153173 ± 3% softirqs.CPU122.RCU 134801 ± 4% +13.6% 153136 ± 3% softirqs.CPU123.RCU 134709 ± 3% +13.9% 153440 ± 3% softirqs.CPU124.RCU 135185 ± 3% +13.1% 152925 ± 3% softirqs.CPU125.RCU 134977 ± 3% +13.8% 153608 ± 3% softirqs.CPU126.RCU 133862 ± 4% +14.7% 153558 ± 3% softirqs.CPU127.RCU 134554 ± 3% +13.9% 153237 ± 3% softirqs.CPU128.RCU 134254 ± 3% +14.1% 153130 ± 3% softirqs.CPU129.RCU 135133 ± 3% +14.0% 154103 ± 3% softirqs.CPU13.RCU 110389 ± 2% +8.1% 119382 ± 3% softirqs.CPU13.TIMER 134319 ± 3% +13.8% 152909 ± 3% softirqs.CPU130.RCU 133590 ± 3% +14.5% 152925 ± 3% softirqs.CPU131.RCU 134416 ± 4% +14.0% 153223 ± 3% softirqs.CPU132.RCU 133931 ± 3% +14.5% 153359 ± 3% softirqs.CPU133.RCU 134031 ± 3% +14.5% 153527 ± 3% softirqs.CPU134.RCU 133851 ± 4% +14.3% 153052 ± 3% softirqs.CPU135.RCU 134235 ± 4% +13.9% 152959 ± 3% softirqs.CPU136.RCU 134003 ± 3% +14.2% 153037 ± 3% softirqs.CPU137.RCU 133453 ± 3% +14.4% 152625 ± 3% softirqs.CPU138.RCU 133514 ± 3% +14.8% 153289 ± 3% softirqs.CPU139.RCU 134952 ± 3% +13.6% 153372 ± 3% softirqs.CPU14.RCU 134045 ± 3% +13.7% 152361 ± 3% softirqs.CPU140.RCU 133662 ± 3% +14.1% 152528 ± 3% softirqs.CPU141.RCU 133643 ± 3% +14.7% 153283 ± 2% softirqs.CPU142.RCU 133313 ± 3% +15.0% 153254 ± 3% softirqs.CPU143.RCU 133585 ± 3% +14.4% 152837 ± 3% softirqs.CPU144.RCU 108590 ± 3% +12.7% 122339 ± 5% softirqs.CPU144.TIMER 133471 ± 3% +14.5% 152883 ± 3% softirqs.CPU145.RCU 108095 ± 3% +14.5% 123726 ± 6% softirqs.CPU145.TIMER 133588 ± 3% +15.4% 154173 ± 3% softirqs.CPU146.RCU 108268 ± 3% +19.8% 129684 ± 16% softirqs.CPU146.TIMER 134060 ± 3% +14.3% 153270 ± 3% softirqs.CPU147.RCU 108731 ± 3% +13.0% 122902 ± 5% softirqs.CPU147.TIMER 133778 ± 3% +14.1% 152681 ± 3% softirqs.CPU148.RCU 108208 ± 3% +12.6% 121875 ± 5% softirqs.CPU148.TIMER 133514 ± 3% +14.2% 152460 ± 3% softirqs.CPU149.RCU 108054 ± 3% +15.6% 124938 ± 9% softirqs.CPU149.TIMER 134677 ± 3% +14.5% 154138 ± 2% softirqs.CPU15.RCU 133245 ± 3% +14.4% 152373 ± 3% softirqs.CPU150.RCU 107788 ± 3% +13.1% 121957 ± 5% softirqs.CPU150.TIMER 133747 ± 3% +14.0% 152473 ± 3% softirqs.CPU151.RCU 108258 ± 4% +12.4% 121662 ± 5% softirqs.CPU151.TIMER 133819 ± 3% +14.1% 152726 ± 3% softirqs.CPU152.RCU 108163 ± 3% +12.7% 121940 ± 5% softirqs.CPU152.TIMER 134057 ± 3% +14.0% 152844 ± 3% softirqs.CPU153.RCU 107680 ± 3% +13.1% 121787 ± 5% softirqs.CPU153.TIMER 133660 ± 3% +14.2% 152675 ± 3% softirqs.CPU154.RCU 107911 ± 3% +15.4% 124485 ± 9% softirqs.CPU154.TIMER 133349 ± 3% +14.6% 152811 ± 3% softirqs.CPU155.RCU 107513 ± 3% +15.1% 123737 ± 6% softirqs.CPU155.TIMER 133901 ± 3% +14.0% 152615 ± 3% softirqs.CPU156.RCU 107853 ± 3% +13.0% 121904 ± 5% softirqs.CPU156.TIMER 133621 ± 3% +14.2% 152615 ± 3% softirqs.CPU157.RCU 108040 ± 3% +12.6% 121684 ± 5% softirqs.CPU157.TIMER 133891 ± 3% +14.2% 152862 ± 3% softirqs.CPU158.RCU 108534 ± 3% +12.7% 122300 ± 5% softirqs.CPU158.TIMER 133925 ± 3% +14.5% 153347 ± 3% softirqs.CPU159.RCU 108099 ± 3% +12.9% 122015 ± 5% softirqs.CPU159.TIMER 134893 ± 3% +14.5% 154389 ± 3% softirqs.CPU16.RCU 134169 ± 3% +13.9% 152812 ± 3% softirqs.CPU160.RCU 108300 ± 3% +12.9% 122318 ± 5% softirqs.CPU160.TIMER 133818 ± 3% +15.1% 154048 ± 4% softirqs.CPU161.RCU 107919 ± 3% +13.6% 122606 ± 6% softirqs.CPU161.TIMER 133867 ± 3% +14.2% 152856 ± 3% softirqs.CPU162.RCU 107991 ± 3% +13.2% 122209 ± 5% softirqs.CPU162.TIMER 133579 ± 3% +14.3% 152654 ± 3% softirqs.CPU163.RCU 108317 ± 3% +12.9% 122271 ± 5% softirqs.CPU163.TIMER 133914 ± 3% +14.4% 153158 ± 3% softirqs.CPU164.RCU 133684 ± 3% +14.4% 152895 ± 3% softirqs.CPU165.RCU 107435 ± 3% +12.5% 120877 ± 5% softirqs.CPU165.TIMER 133983 ± 3% +13.9% 152582 ± 3% softirqs.CPU166.RCU 107432 ± 3% +13.2% 121641 ± 5% softirqs.CPU166.TIMER 133947 ± 3% +13.6% 152137 ± 3% softirqs.CPU167.RCU 134005 ± 3% +14.5% 153385 ± 4% softirqs.CPU168.RCU 115752 ± 4% +12.0% 129634 ± 4% softirqs.CPU168.TIMER 133557 ± 3% +15.2% 153865 ± 4% softirqs.CPU169.RCU 115884 ± 3% +11.7% 129481 ± 4% softirqs.CPU169.TIMER 135348 ± 3% +13.8% 153977 ± 3% softirqs.CPU17.RCU 110589 ± 3% +7.9% 119298 ± 3% softirqs.CPU17.TIMER 134463 ± 3% +14.0% 153229 ± 3% softirqs.CPU170.RCU 133740 ± 3% +14.1% 152630 ± 3% softirqs.CPU171.RCU 116524 ± 4% +10.9% 129237 ± 3% softirqs.CPU171.TIMER 133716 ± 3% +13.8% 152196 ± 3% softirqs.CPU172.RCU 116393 ± 4% +10.9% 129087 ± 4% softirqs.CPU172.TIMER 133564 ± 3% +13.9% 152165 ± 3% softirqs.CPU173.RCU 116237 ± 3% +11.4% 129460 ± 3% softirqs.CPU173.TIMER 133411 ± 3% +14.5% 152764 ± 3% softirqs.CPU174.RCU 116018 ± 3% +11.2% 129006 ± 4% softirqs.CPU174.TIMER 133147 ± 3% +15.0% 153104 ± 2% softirqs.CPU175.RCU 116052 ± 4% +11.7% 129664 ± 3% softirqs.CPU175.TIMER 133989 ± 3% +13.8% 152504 ± 3% softirqs.CPU176.RCU 116071 ± 4% +10.9% 128756 ± 3% softirqs.CPU176.TIMER 133902 ± 3% +14.1% 152811 ± 3% softirqs.CPU177.RCU 115994 ± 3% +11.1% 128849 ± 4% softirqs.CPU177.TIMER 133679 ± 3% +14.4% 152887 ± 3% softirqs.CPU178.RCU 115942 ± 4% +13.4% 131439 ± 5% softirqs.CPU178.TIMER 133985 ± 3% +14.0% 152762 ± 3% softirqs.CPU179.RCU 116347 ± 4% +11.2% 129422 ± 4% softirqs.CPU179.TIMER 134982 ± 3% +14.0% 153838 ± 3% softirqs.CPU18.RCU 110310 ± 3% +8.6% 119851 ± 3% softirqs.CPU18.TIMER 133236 ± 3% +14.7% 152870 ± 3% softirqs.CPU180.RCU 115901 ± 4% +11.3% 129054 ± 4% softirqs.CPU180.TIMER 133615 ± 3% +14.4% 152876 ± 3% softirqs.CPU181.RCU 115505 ± 4% +11.3% 128560 ± 3% softirqs.CPU181.TIMER 133171 ± 3% +14.7% 152810 ± 3% softirqs.CPU182.RCU 115955 ± 4% +11.0% 128687 ± 3% softirqs.CPU182.TIMER 133413 ± 3% +14.4% 152671 ± 3% softirqs.CPU183.RCU 116313 ± 4% +11.0% 129140 ± 3% softirqs.CPU183.TIMER 133598 ± 3% +14.8% 153388 ± 3% softirqs.CPU184.RCU 115462 ± 3% +13.0% 130430 ± 6% softirqs.CPU184.TIMER 133884 ± 3% +13.9% 152525 ± 3% softirqs.CPU185.RCU 115598 ± 4% +11.1% 128397 ± 4% softirqs.CPU185.TIMER 133262 ± 3% +14.5% 152646 ± 3% softirqs.CPU186.RCU 115745 ± 3% +11.1% 128612 ± 4% softirqs.CPU186.TIMER 133430 ± 3% +14.5% 152767 ± 3% softirqs.CPU187.RCU 115769 ± 4% +11.0% 128464 ± 3% softirqs.CPU187.TIMER 133259 ± 3% +14.8% 153036 ± 3% softirqs.CPU188.RCU 116108 ± 4% +11.3% 129232 ± 3% softirqs.CPU188.TIMER 133609 ± 3% +14.1% 152436 ± 3% softirqs.CPU189.RCU 115984 ± 4% +11.0% 128710 ± 3% softirqs.CPU189.TIMER 135090 ± 3% +13.9% 153802 ± 3% softirqs.CPU19.RCU 110865 ± 2% +8.2% 119936 ± 3% softirqs.CPU19.TIMER 133685 ± 3% +14.2% 152631 ± 3% softirqs.CPU190.RCU 115642 ± 3% +11.2% 128581 ± 4% softirqs.CPU190.TIMER 131031 ± 3% +14.6% 150126 ± 3% softirqs.CPU191.RCU 114275 ± 4% +11.4% 127249 ± 4% softirqs.CPU191.TIMER 134928 ± 3% +14.3% 154165 ± 3% softirqs.CPU2.RCU 135126 ± 3% +14.0% 154083 ± 3% softirqs.CPU20.RCU 111013 ± 3% +8.6% 120549 ± 3% softirqs.CPU20.TIMER 134663 ± 3% +14.5% 154251 ± 3% softirqs.CPU21.RCU 109671 ± 2% +8.2% 118708 ± 3% softirqs.CPU21.TIMER 135047 ± 3% +13.9% 153766 ± 3% softirqs.CPU22.RCU 134437 ± 3% +14.2% 153492 ± 3% softirqs.CPU23.RCU 109675 ± 2% +8.3% 118824 ± 3% softirqs.CPU23.TIMER 136114 ± 3% +13.3% 154277 ± 3% softirqs.CPU24.RCU 134825 ± 3% +14.2% 154007 ± 3% softirqs.CPU25.RCU 139234 ± 4% +13.8% 158515 ± 3% softirqs.CPU26.RCU 135056 ± 3% +14.1% 154062 ± 3% softirqs.CPU27.RCU 135307 ± 3% +13.7% 153897 ± 3% softirqs.CPU28.RCU 134686 ± 3% +14.7% 154497 ± 3% softirqs.CPU29.RCU 134688 ± 3% +14.4% 154115 ± 2% softirqs.CPU3.RCU 134642 ± 3% +14.5% 154226 ± 3% softirqs.CPU30.RCU 134773 ± 3% +14.2% 153887 ± 3% softirqs.CPU31.RCU 134523 ± 3% +14.3% 153733 ± 3% softirqs.CPU32.RCU 134488 ± 3% +14.4% 153801 ± 3% softirqs.CPU33.RCU 134483 ± 3% +14.1% 153380 ± 3% softirqs.CPU34.RCU 134886 ± 3% +13.6% 153180 ± 3% softirqs.CPU35.RCU 135021 ± 3% +13.6% 153340 ± 3% softirqs.CPU36.RCU 134064 ± 3% +15.1% 154277 ± 3% softirqs.CPU37.RCU 134166 ± 3% +14.8% 153967 ± 3% softirqs.CPU38.RCU 134488 ± 3% +14.4% 153871 ± 3% softirqs.CPU39.RCU 135294 ± 2% +14.6% 155025 ± 3% softirqs.CPU4.RCU 134598 ± 4% +14.3% 153848 ± 3% softirqs.CPU40.RCU 134255 ± 3% +14.8% 154099 ± 3% softirqs.CPU41.RCU 134334 ± 3% +14.3% 153491 ± 3% softirqs.CPU42.RCU 134201 ± 3% +14.5% 153654 ± 3% softirqs.CPU43.RCU 134260 ± 3% +14.2% 153359 ± 3% softirqs.CPU44.RCU 134005 ± 3% +14.4% 153367 ± 3% softirqs.CPU45.RCU 134236 ± 3% +14.3% 153419 ± 3% softirqs.CPU46.RCU 134552 ± 3% +13.8% 153056 ± 3% softirqs.CPU47.RCU 134623 ± 3% +14.4% 154028 ± 3% softirqs.CPU48.RCU 134348 ± 3% +14.7% 154080 ± 3% softirqs.CPU49.RCU 109020 ± 3% +18.6% 129279 ± 14% softirqs.CPU49.TIMER 134334 ± 3% +15.0% 154455 ± 3% softirqs.CPU5.RCU 110260 ± 2% +8.8% 119951 ± 4% softirqs.CPU5.TIMER 134377 ± 3% +13.8% 152954 ± 3% softirqs.CPU50.RCU 109169 ± 3% +15.0% 125548 ± 9% softirqs.CPU50.TIMER 134780 ± 3% +14.2% 153967 ± 3% softirqs.CPU51.RCU 109951 ± 3% +12.5% 123641 ± 5% softirqs.CPU51.TIMER 133929 ± 3% +15.0% 154012 ± 3% softirqs.CPU52.RCU 108665 ± 3% +12.8% 122613 ± 5% softirqs.CPU52.TIMER 133761 ± 3% +14.9% 153675 ± 3% softirqs.CPU53.RCU 108606 ± 3% +19.6% 129861 ± 16% softirqs.CPU53.TIMER 134369 ± 3% +14.1% 153255 ± 3% softirqs.CPU54.RCU 108917 ± 3% +12.5% 122585 ± 5% softirqs.CPU54.TIMER 133810 ± 3% +14.6% 153282 ± 3% softirqs.CPU55.RCU 108875 ± 3% +12.5% 122468 ± 5% softirqs.CPU55.TIMER 134038 ± 3% +14.3% 153248 ± 3% softirqs.CPU56.RCU 108569 ± 3% +12.8% 122502 ± 5% softirqs.CPU56.TIMER 133993 ± 4% +14.5% 153426 ± 3% softirqs.CPU57.RCU 108465 ± 3% +12.8% 122399 ± 5% softirqs.CPU57.TIMER 134049 ± 3% +14.7% 153708 ± 3% softirqs.CPU58.RCU 108634 ± 4% +18.8% 129107 ± 16% softirqs.CPU58.TIMER 134025 ± 3% +14.9% 154015 ± 3% softirqs.CPU59.RCU 108501 ± 3% +18.6% 128721 ± 13% softirqs.CPU59.TIMER 135247 ± 3% +14.5% 154804 ± 3% softirqs.CPU6.RCU 133745 ± 4% +14.7% 153452 ± 3% softirqs.CPU60.RCU 108526 ± 3% +13.0% 122688 ± 5% softirqs.CPU60.TIMER 133931 ± 3% +14.6% 153549 ± 3% softirqs.CPU61.RCU 108764 ± 3% +12.9% 122787 ± 5% softirqs.CPU61.TIMER 134682 ± 3% +13.8% 153266 ± 3% softirqs.CPU62.RCU 109188 ± 3% +12.5% 122806 ± 5% softirqs.CPU62.TIMER 134190 ± 3% +14.6% 153761 ± 2% softirqs.CPU63.RCU 108489 ± 3% +12.8% 122426 ± 5% softirqs.CPU63.TIMER 134280 ± 3% +14.3% 153538 ± 3% softirqs.CPU64.RCU 108493 ± 3% +13.3% 122953 ± 5% softirqs.CPU64.TIMER 134074 ± 3% +14.7% 153765 ± 3% softirqs.CPU65.RCU 108331 ± 3% +13.0% 122440 ± 5% softirqs.CPU65.TIMER 134424 ± 4% +14.3% 153692 ± 3% softirqs.CPU66.RCU 108874 ± 3% +12.5% 122490 ± 5% softirqs.CPU66.TIMER 133801 ± 3% +14.9% 153746 ± 3% softirqs.CPU67.RCU 108903 ± 3% +13.2% 123240 ± 5% softirqs.CPU67.TIMER 134085 ± 3% +14.4% 153371 ± 3% softirqs.CPU68.RCU 109310 ± 3% +12.3% 122810 ± 5% softirqs.CPU68.TIMER 134091 ± 3% +14.5% 153515 ± 3% softirqs.CPU69.RCU 108193 ± 3% +12.5% 121716 ± 5% softirqs.CPU69.TIMER 134883 ± 3% +13.6% 153181 ± 3% softirqs.CPU7.RCU 134392 ± 3% +14.8% 154240 ± 3% softirqs.CPU70.RCU 108056 ± 3% +13.4% 122578 ± 5% softirqs.CPU70.TIMER 133901 ± 3% +14.2% 152873 ± 3% softirqs.CPU71.RCU 134273 ± 3% +14.4% 153630 ± 3% softirqs.CPU72.RCU 116189 ± 3% +11.5% 129495 ± 3% softirqs.CPU72.TIMER 134231 ± 3% +14.5% 153682 ± 3% softirqs.CPU73.RCU 116348 ± 3% +11.3% 129453 ± 3% softirqs.CPU73.TIMER 134235 ± 3% +14.1% 153158 ± 3% softirqs.CPU74.RCU 133896 ± 3% +14.6% 153477 ± 3% softirqs.CPU75.RCU 116933 ± 4% +11.2% 129980 ± 3% softirqs.CPU75.TIMER 134014 ± 3% +14.3% 153198 ± 3% softirqs.CPU76.RCU 116889 ± 3% +11.0% 129768 ± 3% softirqs.CPU76.TIMER 134608 ± 3% +13.7% 152994 ± 3% softirqs.CPU77.RCU 117342 ± 3% +11.0% 130274 ± 3% softirqs.CPU77.TIMER 134551 ± 3% +13.9% 153253 ± 3% softirqs.CPU78.RCU 117229 ± 4% +10.5% 129549 ± 4% softirqs.CPU78.TIMER 134118 ± 3% +14.3% 153325 ± 3% softirqs.CPU79.RCU 116982 ± 3% +11.0% 129802 ± 3% softirqs.CPU79.TIMER 135303 ± 3% +13.9% 154129 ± 3% softirqs.CPU8.RCU 134699 ± 3% +13.6% 153047 ± 3% softirqs.CPU80.RCU 117058 ± 4% +10.6% 129470 ± 3% softirqs.CPU80.TIMER 134088 ± 3% +15.6% 155063 ± 3% softirqs.CPU81.RCU 116539 ± 4% +12.3% 130886 ± 5% softirqs.CPU81.TIMER 133989 ± 3% +14.6% 153526 ± 3% softirqs.CPU82.RCU 116614 ± 4% +16.8% 136238 ± 12% softirqs.CPU82.TIMER 133966 ± 3% +14.3% 153142 ± 3% softirqs.CPU83.RCU 117140 ± 3% +11.2% 130205 ± 4% softirqs.CPU83.TIMER 133561 ± 3% +15.0% 153661 ± 3% softirqs.CPU84.RCU 116648 ± 4% +11.3% 129844 ± 4% softirqs.CPU84.TIMER 134217 ± 3% +14.2% 153330 ± 3% softirqs.CPU85.RCU 116247 ± 4% +11.3% 129336 ± 3% softirqs.CPU85.TIMER 133956 ± 3% +14.5% 153338 ± 3% softirqs.CPU86.RCU 116654 ± 4% +11.1% 129560 ± 3% softirqs.CPU86.TIMER 133834 ± 3% +15.1% 154049 ± 2% softirqs.CPU87.RCU 116898 ± 4% +12.0% 130912 ± 3% softirqs.CPU87.TIMER 133951 ± 3% +15.1% 154211 ± 3% softirqs.CPU88.RCU 116139 ± 3% +16.0% 134676 ± 13% softirqs.CPU88.TIMER 134288 ± 3% +14.3% 153433 ± 2% softirqs.CPU89.RCU 116227 ± 3% +11.0% 129064 ± 3% softirqs.CPU89.TIMER 135289 ± 3% +13.6% 153744 ± 3% softirqs.CPU9.RCU 133598 ± 3% +14.5% 153019 ± 3% softirqs.CPU90.RCU 116395 ± 3% +10.9% 129084 ± 4% softirqs.CPU90.TIMER 133854 ± 4% +14.9% 153799 ± 3% softirqs.CPU91.RCU 116562 ± 4% +11.1% 129452 ± 3% softirqs.CPU91.TIMER 134107 ± 3% +14.1% 152958 ± 3% softirqs.CPU92.RCU 117107 ± 4% +10.6% 129559 ± 3% softirqs.CPU92.TIMER 133800 ± 3% +14.7% 153465 ± 3% softirqs.CPU93.RCU 116415 ± 3% +11.5% 129823 ± 4% softirqs.CPU93.TIMER 134092 ± 3% +14.0% 152835 ± 3% softirqs.CPU94.RCU 116063 ± 3% +11.4% 129252 ± 4% softirqs.CPU94.TIMER 134284 ± 3% +14.1% 153230 ± 4% softirqs.CPU95.RCU 117908 ± 3% +11.0% 130904 ± 4% softirqs.CPU95.TIMER 134264 ± 3% +14.3% 153455 ± 3% softirqs.CPU96.RCU 109819 ± 3% +8.6% 119258 ± 3% softirqs.CPU96.TIMER 135000 ± 3% +13.7% 153529 ± 3% softirqs.CPU97.RCU 134163 ± 3% +14.2% 153253 ± 3% softirqs.CPU98.RCU 109990 ± 3% +8.1% 118896 ± 3% softirqs.CPU98.TIMER 134966 ± 3% +13.5% 153245 ± 3% softirqs.CPU99.RCU 25776297 ± 3% +14.3% 29452390 ± 3% softirqs.RCU 21445010 ± 3% +9.5% 23488096 ± 3% softirqs.TIMER 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi 611854 ± 2% +10.1% 673945 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 611835 ± 2% +10.1% 673883 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts 611859 ± 2% +10.2% 673975 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts 7508 ± 5% +15.5% 8675 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts 611802 ± 2% +10.2% 673904 ± 2% interrupts.CPU100.LOC:Local_timer_interrupts 7323 ± 5% +16.1% 8503 ± 3% interrupts.CPU100.RES:Rescheduling_interrupts 611804 ± 2% +10.2% 673917 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts 7342 ± 3% +13.8% 8352 ± 3% interrupts.CPU101.RES:Rescheduling_interrupts 611780 ± 2% +10.2% 673877 ± 2% interrupts.CPU102.LOC:Local_timer_interrupts 7506 ± 5% +12.2% 8424 ± 4% interrupts.CPU102.RES:Rescheduling_interrupts 611734 ± 2% +10.2% 673851 ± 2% interrupts.CPU103.LOC:Local_timer_interrupts 7436 ± 3% +15.8% 8607 ± 4% interrupts.CPU103.RES:Rescheduling_interrupts 611800 ± 2% +10.2% 673908 ± 2% interrupts.CPU104.LOC:Local_timer_interrupts 7519 ± 5% +13.1% 8502 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts 611790 ± 2% +10.1% 673864 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts 7346 ± 5% +16.4% 8553 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673859 ± 2% interrupts.CPU106.LOC:Local_timer_interrupts 7352 ± 5% +16.5% 8567 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts 611797 ± 2% +10.1% 673890 ± 2% interrupts.CPU107.LOC:Local_timer_interrupts 7238 ± 6% +15.3% 8344 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673882 ± 2% interrupts.CPU108.LOC:Local_timer_interrupts 7310 ± 5% +16.3% 8503 ± 5% interrupts.CPU108.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673881 ± 2% interrupts.CPU109.LOC:Local_timer_interrupts 7324 ± 6% +13.4% 8307 interrupts.CPU109.RES:Rescheduling_interrupts 611827 ± 2% +10.2% 673960 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts 7598 ± 6% +12.6% 8557 ± 5% interrupts.CPU11.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673865 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts 7326 ± 4% +16.2% 8513 ± 4% interrupts.CPU110.RES:Rescheduling_interrupts 611804 ± 2% +10.1% 673890 ± 2% interrupts.CPU111.LOC:Local_timer_interrupts 7312 ± 3% +14.4% 8365 ± 3% interrupts.CPU111.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673890 ± 2% interrupts.CPU112.LOC:Local_timer_interrupts 7277 ± 2% +14.6% 8337 ± 5% interrupts.CPU112.RES:Rescheduling_interrupts 611789 ± 2% +10.2% 673896 ± 2% interrupts.CPU113.LOC:Local_timer_interrupts 7346 ± 4% +15.2% 8461 ± 4% interrupts.CPU113.RES:Rescheduling_interrupts 611759 ± 2% +10.2% 673861 ± 2% interrupts.CPU114.LOC:Local_timer_interrupts 7290 ± 4% +17.9% 8594 ± 4% interrupts.CPU114.RES:Rescheduling_interrupts 611783 ± 2% +10.2% 673883 ± 2% interrupts.CPU115.LOC:Local_timer_interrupts 7470 ± 4% +13.6% 8489 ± 5% interrupts.CPU115.RES:Rescheduling_interrupts 611807 ± 2% +10.2% 673925 ± 2% interrupts.CPU116.LOC:Local_timer_interrupts 7336 ± 4% +15.3% 8455 ± 2% interrupts.CPU116.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673890 ± 2% interrupts.CPU117.LOC:Local_timer_interrupts 7326 ± 5% +15.9% 8493 ± 3% interrupts.CPU117.RES:Rescheduling_interrupts 611782 ± 2% +10.1% 673849 ± 2% interrupts.CPU118.LOC:Local_timer_interrupts 7239 ± 4% +18.4% 8574 ± 5% interrupts.CPU118.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673881 ± 2% interrupts.CPU119.LOC:Local_timer_interrupts 7327 ± 4% +13.7% 8335 ± 6% interrupts.CPU119.RES:Rescheduling_interrupts 611853 ± 2% +10.1% 673944 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts 7516 ± 4% +17.0% 8798 ± 6% interrupts.CPU12.RES:Rescheduling_interrupts 611417 ± 2% +10.2% 673606 ± 2% interrupts.CPU120.LOC:Local_timer_interrupts 7392 ± 4% +14.2% 8444 ± 5% interrupts.CPU120.RES:Rescheduling_interrupts 611457 ± 2% +10.2% 673608 ± 2% interrupts.CPU121.LOC:Local_timer_interrupts 7376 ± 3% +15.6% 8525 ± 4% interrupts.CPU121.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673610 ± 2% interrupts.CPU122.LOC:Local_timer_interrupts 7466 ± 5% +11.7% 8336 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts 611384 ± 2% +10.2% 673577 ± 2% interrupts.CPU123.LOC:Local_timer_interrupts 7240 ± 5% +18.2% 8559 ± 3% interrupts.CPU123.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673597 ± 2% interrupts.CPU124.LOC:Local_timer_interrupts 7245 ± 5% +18.0% 8548 ± 5% interrupts.CPU124.RES:Rescheduling_interrupts 611391 ± 2% +10.2% 673567 ± 2% interrupts.CPU125.LOC:Local_timer_interrupts 7318 ± 3% +15.3% 8436 ± 5% interrupts.CPU125.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673570 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts 7258 ± 5% +16.4% 8448 ± 4% interrupts.CPU126.RES:Rescheduling_interrupts 611468 ± 2% +10.2% 673614 ± 2% interrupts.CPU127.LOC:Local_timer_interrupts 7276 ± 5% +16.9% 8508 ± 5% interrupts.CPU127.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673591 ± 2% interrupts.CPU128.LOC:Local_timer_interrupts 7278 ± 4% +17.0% 8518 ± 3% interrupts.CPU128.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673562 ± 2% interrupts.CPU129.LOC:Local_timer_interrupts 7284 ± 3% +17.0% 8519 ± 5% interrupts.CPU129.RES:Rescheduling_interrupts 611825 ± 2% +10.1% 673920 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts 7550 ± 4% +16.2% 8771 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 611428 ± 2% +10.2% 673571 ± 2% interrupts.CPU130.LOC:Local_timer_interrupts 7337 ± 5% +16.4% 8541 ± 5% interrupts.CPU130.RES:Rescheduling_interrupts 611406 ± 2% +10.2% 673582 ± 2% interrupts.CPU131.LOC:Local_timer_interrupts 7205 ± 5% +17.1% 8440 ± 6% interrupts.CPU131.RES:Rescheduling_interrupts 611449 ± 2% +10.2% 673586 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts 7431 ± 4% +12.7% 8372 ± 3% interrupts.CPU132.RES:Rescheduling_interrupts 611464 ± 2% +10.2% 673588 ± 2% interrupts.CPU133.LOC:Local_timer_interrupts 7289 ± 4% +15.9% 8446 ± 3% interrupts.CPU133.RES:Rescheduling_interrupts 611437 ± 2% +10.2% 673583 ± 2% interrupts.CPU134.LOC:Local_timer_interrupts 7321 ± 4% +17.7% 8616 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts 611401 ± 2% +10.2% 673595 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts 7300 ± 4% +15.7% 8446 ± 4% interrupts.CPU135.RES:Rescheduling_interrupts 611462 ± 2% +10.2% 673602 ± 2% interrupts.CPU136.LOC:Local_timer_interrupts 7389 ± 4% +13.9% 8417 ± 5% interrupts.CPU136.RES:Rescheduling_interrupts 611424 ± 2% +10.2% 673588 ± 2% interrupts.CPU137.LOC:Local_timer_interrupts 7126 ± 4% +20.0% 8550 ± 5% interrupts.CPU137.RES:Rescheduling_interrupts 611383 ± 2% +10.2% 673556 ± 2% interrupts.CPU138.LOC:Local_timer_interrupts 7297 ± 5% +16.1% 8475 ± 5% interrupts.CPU138.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673603 ± 2% interrupts.CPU139.LOC:Local_timer_interrupts 7160 ± 4% +17.6% 8423 ± 4% interrupts.CPU139.RES:Rescheduling_interrupts 611815 ± 2% +10.1% 673901 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts 7576 ± 5% +16.3% 8810 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673577 ± 2% interrupts.CPU140.LOC:Local_timer_interrupts 7342 ± 6% +12.9% 8293 ± 4% interrupts.CPU140.RES:Rescheduling_interrupts 611394 ± 2% +10.2% 673596 ± 2% interrupts.CPU141.LOC:Local_timer_interrupts 7257 ± 4% +16.1% 8423 ± 5% interrupts.CPU141.RES:Rescheduling_interrupts 611411 ± 2% +10.2% 673590 ± 2% interrupts.CPU142.LOC:Local_timer_interrupts 7300 ± 6% +16.1% 8475 ± 3% interrupts.CPU142.RES:Rescheduling_interrupts 611396 ± 2% +10.2% 673622 ± 2% interrupts.CPU143.LOC:Local_timer_interrupts 7399 ± 5% +14.0% 8435 ± 4% interrupts.CPU143.RES:Rescheduling_interrupts 610929 ± 2% +10.3% 673686 ± 2% interrupts.CPU144.LOC:Local_timer_interrupts 7281 ± 5% +13.4% 8257 ± 5% interrupts.CPU144.RES:Rescheduling_interrupts 610918 ± 2% +10.3% 673755 ± 2% interrupts.CPU145.LOC:Local_timer_interrupts 7343 ± 5% +14.5% 8408 ± 4% interrupts.CPU145.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673705 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts 7524 ± 4% +13.6% 8550 ± 6% interrupts.CPU146.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673769 ± 2% interrupts.CPU147.LOC:Local_timer_interrupts 7433 ± 4% +10.8% 8237 ± 3% interrupts.CPU147.RES:Rescheduling_interrupts 610872 ± 2% +10.3% 673727 ± 2% interrupts.CPU148.LOC:Local_timer_interrupts 7354 ± 3% +13.1% 8320 ± 4% interrupts.CPU148.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673750 ± 2% interrupts.CPU149.LOC:Local_timer_interrupts 7365 ± 3% +16.5% 8582 ± 3% interrupts.CPU149.RES:Rescheduling_interrupts 611818 ± 2% +10.2% 673937 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts 7632 ± 4% +12.5% 8587 ± 4% interrupts.CPU15.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673770 ± 2% interrupts.CPU150.LOC:Local_timer_interrupts 7369 ± 4% +12.4% 8282 ± 3% interrupts.CPU150.RES:Rescheduling_interrupts 610887 ± 2% +10.3% 673754 ± 2% interrupts.CPU151.LOC:Local_timer_interrupts 7322 ± 4% +15.3% 8445 ± 3% interrupts.CPU151.RES:Rescheduling_interrupts 610849 ± 2% +10.3% 673762 ± 2% interrupts.CPU152.LOC:Local_timer_interrupts 7305 ± 4% +15.5% 8439 ± 5% interrupts.CPU152.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673759 ± 2% interrupts.CPU153.LOC:Local_timer_interrupts 7363 ± 6% +14.5% 8432 ± 2% interrupts.CPU153.RES:Rescheduling_interrupts 610878 ± 2% +10.3% 673730 ± 2% interrupts.CPU154.LOC:Local_timer_interrupts 7373 ± 4% +14.1% 8414 ± 4% interrupts.CPU154.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673720 ± 2% interrupts.CPU155.LOC:Local_timer_interrupts 7385 ± 5% +11.8% 8260 ± 4% interrupts.CPU155.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673753 ± 2% interrupts.CPU156.LOC:Local_timer_interrupts 7377 ± 5% +14.0% 8410 ± 5% interrupts.CPU156.RES:Rescheduling_interrupts 610886 ± 2% +10.3% 673754 ± 2% interrupts.CPU157.LOC:Local_timer_interrupts 7399 ± 5% +13.1% 8370 ± 4% interrupts.CPU157.RES:Rescheduling_interrupts 610854 ± 2% +10.3% 673699 ± 2% interrupts.CPU158.LOC:Local_timer_interrupts 7489 ± 4% +10.5% 8278 ± 2% interrupts.CPU158.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673726 ± 2% interrupts.CPU159.LOC:Local_timer_interrupts 7281 ± 5% +15.4% 8402 ± 4% interrupts.CPU159.RES:Rescheduling_interrupts 611836 ± 2% +10.1% 673889 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts 7548 ± 4% +15.6% 8723 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 610870 ± 2% +10.3% 673742 ± 2% interrupts.CPU160.LOC:Local_timer_interrupts 610856 ± 2% +10.3% 673722 ± 2% interrupts.CPU161.LOC:Local_timer_interrupts 7472 ± 5% +13.4% 8475 ± 5% interrupts.CPU161.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673708 ± 2% interrupts.CPU162.LOC:Local_timer_interrupts 7177 ± 5% +14.8% 8239 ± 5% interrupts.CPU162.RES:Rescheduling_interrupts 610864 ± 2% +10.3% 673696 ± 2% interrupts.CPU163.LOC:Local_timer_interrupts 7337 ± 5% +12.7% 8266 ± 4% interrupts.CPU163.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673724 ± 2% interrupts.CPU164.LOC:Local_timer_interrupts 7269 ± 5% +15.2% 8376 ± 4% interrupts.CPU164.RES:Rescheduling_interrupts 610861 ± 2% +10.3% 673711 ± 2% interrupts.CPU165.LOC:Local_timer_interrupts 7431 ± 4% +9.4% 8131 ± 4% interrupts.CPU165.RES:Rescheduling_interrupts 610867 ± 2% +10.3% 673684 ± 2% interrupts.CPU166.LOC:Local_timer_interrupts 7339 ± 2% +14.1% 8374 ± 4% interrupts.CPU166.RES:Rescheduling_interrupts 610913 ± 2% +10.3% 673730 ± 2% interrupts.CPU167.LOC:Local_timer_interrupts 7174 ± 3% +17.5% 8427 ± 4% interrupts.CPU167.RES:Rescheduling_interrupts 611289 ± 2% +10.2% 673888 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts 7311 ± 5% +14.7% 8382 ± 3% interrupts.CPU168.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673887 ± 2% interrupts.CPU169.LOC:Local_timer_interrupts 7232 ± 2% +17.8% 8517 ± 5% interrupts.CPU169.RES:Rescheduling_interrupts 611838 ± 2% +10.1% 673879 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts 7539 ± 2% +17.7% 8873 ± 10% interrupts.CPU17.RES:Rescheduling_interrupts 611349 ± 2% +10.2% 673889 ± 2% interrupts.CPU170.LOC:Local_timer_interrupts 7374 ± 2% +14.2% 8424 ± 5% interrupts.CPU170.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673873 ± 2% interrupts.CPU171.LOC:Local_timer_interrupts 7438 ± 4% +13.9% 8471 ± 5% interrupts.CPU171.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673845 ± 2% interrupts.CPU172.LOC:Local_timer_interrupts 7317 ± 3% +16.6% 8529 ± 5% interrupts.CPU172.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673887 ± 2% interrupts.CPU173.LOC:Local_timer_interrupts 611309 ± 2% +10.2% 673858 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts 7369 ± 5% +15.8% 8531 ± 5% interrupts.CPU174.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673872 ± 2% interrupts.CPU175.LOC:Local_timer_interrupts 7352 ± 4% +15.1% 8465 ± 4% interrupts.CPU175.RES:Rescheduling_interrupts 611309 ± 2% +10.2% 673886 ± 2% interrupts.CPU176.LOC:Local_timer_interrupts 7395 ± 2% +14.0% 8434 ± 5% interrupts.CPU176.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673868 ± 2% interrupts.CPU177.LOC:Local_timer_interrupts 611248 ± 2% +10.2% 673805 ± 2% interrupts.CPU178.LOC:Local_timer_interrupts 7355 ± 5% +14.1% 8393 ± 4% interrupts.CPU178.RES:Rescheduling_interrupts 611319 ± 2% +10.2% 673906 ± 2% interrupts.CPU179.LOC:Local_timer_interrupts 7269 ± 4% +15.2% 8374 ± 6% interrupts.CPU179.RES:Rescheduling_interrupts 611799 ± 2% +10.1% 673864 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts 7585 ± 4% +14.5% 8683 ± 4% interrupts.CPU18.RES:Rescheduling_interrupts 611330 ± 2% +10.2% 673869 ± 2% interrupts.CPU180.LOC:Local_timer_interrupts 7404 ± 4% +14.0% 8442 ± 5% interrupts.CPU180.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673889 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts 7435 ± 2% +13.0% 8399 ± 3% interrupts.CPU181.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673874 ± 2% interrupts.CPU182.LOC:Local_timer_interrupts 7250 ± 4% +14.6% 8305 ± 4% interrupts.CPU182.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673852 ± 2% interrupts.CPU183.LOC:Local_timer_interrupts 7320 ± 4% +13.2% 8290 ± 5% interrupts.CPU183.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673877 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts 7377 ± 4% +12.5% 8299 ± 5% interrupts.CPU184.RES:Rescheduling_interrupts 611312 ± 2% +10.2% 673866 ± 2% interrupts.CPU185.LOC:Local_timer_interrupts 7333 ± 4% +13.9% 8351 ± 5% interrupts.CPU185.RES:Rescheduling_interrupts 611290 ± 2% +10.2% 673868 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts 7426 ± 4% +14.1% 8471 ± 3% interrupts.CPU186.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673853 ± 2% interrupts.CPU187.LOC:Local_timer_interrupts 7380 ± 3% +12.5% 8304 ± 5% interrupts.CPU187.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673881 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 7277 ± 5% +16.3% 8467 ± 6% interrupts.CPU188.RES:Rescheduling_interrupts 611299 ± 2% +10.2% 673879 ± 2% interrupts.CPU189.LOC:Local_timer_interrupts 7436 ± 3% +14.5% 8514 ± 6% interrupts.CPU189.RES:Rescheduling_interrupts 611761 ± 2% +10.2% 673928 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts 7515 ± 6% +12.7% 8467 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 611321 ± 2% +10.2% 673856 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts 7188 ± 3% +16.4% 8366 ± 5% interrupts.CPU190.RES:Rescheduling_interrupts 611544 ± 2% +10.2% 674120 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts 7207 ± 3% +15.3% 8312 ± 3% interrupts.CPU191.RES:Rescheduling_interrupts 611805 ± 2% +10.2% 673920 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts 611781 ± 2% +10.2% 673897 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts 7531 ± 5% +16.4% 8768 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts 611715 ± 2% +10.2% 673897 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts 7467 ± 3% +14.9% 8577 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts 611834 ± 2% +10.1% 673900 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts 7427 ± 6% +17.9% 8754 ± 4% interrupts.CPU22.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673869 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts 7592 ± 4% +12.2% 8518 ± 4% interrupts.CPU23.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673619 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts 7457 ± 4% +14.7% 8551 ± 3% interrupts.CPU24.RES:Rescheduling_interrupts 611439 ± 2% +10.2% 673636 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 611402 ± 2% +10.2% 673620 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 7463 ± 3% +18.4% 8838 ± 7% interrupts.CPU26.RES:Rescheduling_interrupts 611413 ± 2% +10.2% 673615 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 7412 ± 4% +16.1% 8607 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 611409 ± 2% +10.2% 673625 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts 7478 ± 5% +15.6% 8648 ± 3% interrupts.CPU28.RES:Rescheduling_interrupts 611415 ± 2% +10.2% 673620 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts 7431 ± 2% +15.5% 8585 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts 611807 ± 2% +10.1% 673896 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts 7659 ± 5% +20.9% 9264 ± 10% interrupts.CPU3.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673520 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts 7432 ± 3% +16.5% 8659 ± 4% interrupts.CPU30.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673604 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts 7490 ± 4% +14.6% 8585 ± 5% interrupts.CPU31.RES:Rescheduling_interrupts 611444 ± 2% +10.2% 673588 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 7393 ± 4% +19.0% 8799 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673606 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 7398 ± 4% +16.8% 8644 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts 611452 ± 2% +10.2% 673609 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts 7414 ± 5% +18.8% 8806 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673588 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 7486 ± 5% +18.6% 8878 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673603 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts 7503 ± 3% +15.0% 8627 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts 611465 ± 2% +10.2% 673611 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts 7416 ± 3% +17.6% 8719 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts 611445 ± 2% +10.2% 673575 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts 7477 ± 5% +14.2% 8540 ± 3% interrupts.CPU38.RES:Rescheduling_interrupts 611420 ± 2% +10.2% 673600 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts 7562 ± 5% +14.0% 8620 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts 611823 ± 2% +10.2% 673949 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts 7579 ± 4% +14.9% 8708 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 611390 ± 2% +10.2% 673618 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts 7444 ± 4% +15.9% 8626 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts 611429 ± 2% +10.2% 673612 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts 7464 ± 5% +15.3% 8605 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts 611418 ± 2% +10.2% 673597 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts 7394 ± 4% +18.6% 8772 ± 5% interrupts.CPU42.RES:Rescheduling_interrupts 611364 ± 2% +10.2% 673601 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts 7483 ± 5% +15.3% 8630 ± 3% interrupts.CPU43.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673593 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts 7463 ± 2% +16.0% 8656 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts 611423 ± 2% +10.2% 673593 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts 7455 ± 4% +14.8% 8558 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673616 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 7406 ± 3% +17.4% 8696 ± 3% interrupts.CPU46.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673605 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts 7482 ± 5% +15.7% 8660 ± 2% interrupts.CPU47.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673737 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts 7521 ± 6% +15.1% 8655 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673770 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts 7594 ± 5% +12.7% 8558 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673831 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts 610930 ± 2% +10.3% 673671 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts 7504 ± 2% +17.6% 8824 ± 4% interrupts.CPU50.RES:Rescheduling_interrupts 610882 ± 2% +10.3% 673738 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts 610884 ± 2% +10.3% 673731 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts 7436 ± 4% +18.1% 8783 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts 610885 ± 2% +10.3% 673774 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts 7544 ± 4% +15.3% 8700 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts 610902 ± 2% +10.3% 673763 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts 7534 ± 4% +15.6% 8709 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 610927 ± 2% +10.3% 673739 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts 610883 ± 2% +10.3% 673719 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts 610921 ± 2% +10.3% 673689 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts 7473 ± 5% +14.8% 8577 ± 4% interrupts.CPU57.RES:Rescheduling_interrupts 610893 ± 2% +10.3% 673781 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts 7407 ± 5% +16.6% 8635 ± 3% interrupts.CPU58.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673763 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts 7642 ± 5% +13.1% 8643 ± 3% interrupts.CPU59.RES:Rescheduling_interrupts 611823 ± 2% +10.1% 673914 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts 7542 ± 3% +17.3% 8845 ± 3% interrupts.CPU6.RES:Rescheduling_interrupts 610869 ± 2% +10.3% 673784 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts 7535 ± 4% +14.8% 8647 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673701 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts 7631 ± 4% +13.5% 8660 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 610863 ± 2% +10.3% 673705 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts 7634 ± 4% +11.8% 8532 ± 4% interrupts.CPU62.RES:Rescheduling_interrupts 610920 ± 2% +10.3% 673730 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts 7549 ± 5% +14.8% 8663 ± 5% interrupts.CPU63.RES:Rescheduling_interrupts 610884 ± 2% +10.3% 673727 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts 7644 ± 3% +13.4% 8672 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 610853 ± 2% +10.3% 673708 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts 7555 ± 5% +14.8% 8672 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 610950 ± 2% +10.3% 673804 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts 7457 ± 5% +16.8% 8710 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts 610888 ± 2% +10.3% 673735 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts 7550 ± 4% +11.5% 8415 ± 5% interrupts.CPU67.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673727 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts 7612 ± 4% +12.1% 8534 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 610901 ± 2% +10.3% 673659 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts 7493 ± 4% +14.8% 8601 ± 4% interrupts.CPU69.RES:Rescheduling_interrupts 611808 ± 2% +10.1% 673844 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts 7673 ± 5% +15.0% 8821 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673733 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts 7412 ± 3% +16.9% 8664 ± 4% interrupts.CPU70.RES:Rescheduling_interrupts 610871 ± 2% +10.3% 673671 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts 7519 ± 4% +13.8% 8557 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts 611325 ± 2% +10.2% 673906 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts 7720 ± 4% +9.6% 8459 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673908 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts 7535 ± 4% +14.6% 8634 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673883 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts 7526 ± 5% +14.7% 8635 ± 6% interrupts.CPU74.RES:Rescheduling_interrupts 611315 ± 2% +10.2% 673868 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 7712 ± 5% +9.8% 8469 ± 4% interrupts.CPU75.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673860 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts 7476 ± 5% +14.2% 8537 ± 5% interrupts.CPU76.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673834 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts 7670 ± 3% +12.5% 8630 ± 5% interrupts.CPU77.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673851 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 7417 ± 3% +15.6% 8578 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673820 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 7545 ± 4% +13.9% 8590 ± 4% interrupts.CPU79.RES:Rescheduling_interrupts 611842 ± 2% +10.1% 673928 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts 7673 ± 3% +14.9% 8818 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 611366 ± 2% +10.2% 673868 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts 7513 ± 3% +16.1% 8723 ± 5% interrupts.CPU80.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673860 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts 611299 ± 2% +10.2% 673868 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts 7523 ± 4% +13.8% 8562 ± 4% interrupts.CPU82.RES:Rescheduling_interrupts 611320 ± 2% +10.2% 673727 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts 7518 ± 5% +12.4% 8454 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673899 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts 7452 ± 6% +12.3% 8365 ± 5% interrupts.CPU84.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673885 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts 7399 ± 5% +11.1% 8218 ± 5% interrupts.CPU85.RES:Rescheduling_interrupts 611278 ± 2% +10.2% 673875 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts 7376 ± 4% +16.3% 8581 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts 611313 ± 2% +10.2% 673874 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts 7544 ± 4% +12.8% 8508 ± 4% interrupts.CPU87.RES:Rescheduling_interrupts 611351 ± 2% +10.2% 673928 ± 2% interrupts.CPU88.LOC:Local_timer_interrupts 7418 ± 6% +13.3% 8406 ± 4% interrupts.CPU88.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673795 ± 2% interrupts.CPU89.LOC:Local_timer_interrupts 7426 ± 4% +13.6% 8439 ± 5% interrupts.CPU89.RES:Rescheduling_interrupts 611832 ± 2% +10.2% 673968 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts 7538 ± 3% +15.6% 8715 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 611298 ± 2% +10.2% 673878 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts 7421 ± 3% +12.7% 8363 ± 4% interrupts.CPU90.RES:Rescheduling_interrupts 611305 ± 2% +10.2% 673874 ± 2% interrupts.CPU91.LOC:Local_timer_interrupts 7377 ± 5% +15.6% 8528 ± 5% interrupts.CPU91.RES:Rescheduling_interrupts 611288 ± 2% +10.2% 673876 ± 2% interrupts.CPU92.LOC:Local_timer_interrupts 7349 ± 4% +12.0% 8234 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673857 ± 2% interrupts.CPU93.LOC:Local_timer_interrupts 7438 ± 3% +14.7% 8530 ± 4% interrupts.CPU93.RES:Rescheduling_interrupts 611276 ± 2% +10.2% 673866 ± 2% interrupts.CPU94.LOC:Local_timer_interrupts 7363 ± 4% +12.7% 8300 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts 611381 ± 2% +10.2% 673922 ± 2% interrupts.CPU95.LOC:Local_timer_interrupts 10549 ± 3% +14.9% 12117 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 611824 ± 2% +10.1% 673847 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts 7416 ± 3% +12.9% 8375 ± 5% interrupts.CPU96.RES:Rescheduling_interrupts 611751 ± 2% +10.2% 673870 ± 2% interrupts.CPU97.LOC:Local_timer_interrupts 7437 ± 4% +14.0% 8477 ± 4% interrupts.CPU97.RES:Rescheduling_interrupts 611787 ± 2% +10.1% 673877 ± 2% interrupts.CPU98.LOC:Local_timer_interrupts 7555 ± 4% +13.0% 8534 ± 3% interrupts.CPU98.RES:Rescheduling_interrupts 611783 ± 2% +10.1% 673872 ± 2% interrupts.CPU99.LOC:Local_timer_interrupts 7452 ± 3% +12.8% 8404 ± 5% interrupts.CPU99.RES:Rescheduling_interrupts 1.174e+08 ± 2% +10.2% 1.294e+08 ± 2% interrupts.LOC:Local_timer_interrupts 1433875 ± 3% +14.6% 1643307 ± 3% interrupts.RES:Rescheduling_interrupts aim7.jobs-per-min 30500 +-------------------------------------------------------------------+ | ..+.... .| 30000 |-+ ...... ... ... | 29500 |................ ...... .. ... | | +.. ... .. | 29000 |-+ ... ... | 28500 |-+ . ... | | +. | 28000 |-+ O O | 27500 |-+ | | | 27000 |-+ | 26500 |-+ | | O | 26000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 9 months

2
3
0 / 0

[nbd] c514f02a2b: blktests.nbd.001.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: c514f02a2b5b2dedb96e25fec5a516d8fb711525 ("nbd: use set_capacity_revalidate_and_notify") git://git.infradead.org/users/hch/misc.git nbd-set-size-fix in testcase: blktests version: blktests-x86_64-20445c5-1_20201007 with following parameters: test: nbd-group1 ucode: 0xdc on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> 2020-11-02 20:28:51 sed "s:^:nbd/:" /lkp/benchmarks/blktests/tests/nbd-group1 2020-11-02 20:28:51 ./check nbd/001 nbd/002 nbd/003 nbd/001 (resize a connected nbd device) nbd/001 (resize a connected nbd device) [failed] runtime ... 1.095s --- tests/nbd/001.out 2020-10-07 22:29:16.000000000 +0000 +++ /lkp/benchmarks/blktests/results/nodev/nbd/001.out.bad 2020-11-02 20:28:52.308301248 +0000 @@ -1,5 +1,4 @@ Running nbd/001 -Disk /dev/nbd0: 10.7GB nbd0 43:0 0 10G 0 disk Setting size to 1gib nbd0 43:0 0 1G 0 disk nbd/002 (tests on partition handling for an nbd device) nbd/002 (tests on partition handling for an nbd device) [failed] runtime ... 0.341s --- tests/nbd/002.out 2020-10-07 22:29:16.000000000 +0000 +++ /lkp/benchmarks/blktests/results/nodev/nbd/002.out.bad 2020-11-02 20:28:52.684301245 +0000 @@ -1,4 +1,3 @@ Running nbd/002 Testing IOCTL path -Testing the netlink path -Test complete +Didn't have partition on ioctl path nbd/003 (mount/unmount concurrently with NBD_CLEAR_SOCK) nbd/003 (mount/unmount concurrently with NBD_CLEAR_SOCK) [passed] runtime ... 67.534s To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Rong Chen

1 year, 9 months

1
0
0 / 0

[mm/swap] aae466b005: vm-scalability.throughput 17.2% improvement

by kernel test robot

Greeting, FYI, we noticed a 17.2% improvement of vm-scalability.throughput due to commit: commit: aae466b0052e1888edd1d7f473d4310d64936196 ("mm/swap: implement workingset detection for anonymous LRU") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: vm-scalability on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory with following parameters: thp_enabled: never thp_defrag: never nr_task: 8 nr_pmem: 4 priority: 1 test: swap-w-seq cpufreq_governor: performance ucode: 0x16 test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/nr_pmem/nr_task/priority/rootfs/tbox_group/test/testcase/thp_defrag/thp_enabled/ucode: gcc-9/performance/x86_64-rhel-8.3/4/8/1/debian-10.4-x86_64-20200603.cgz/lkp-hsw-4ex1/swap-w-seq/vm-scalability/never/never/0x16 commit: 3852f6768e ("mm/swapcache: support to handle the shadow entries") aae466b005 ("mm/swap: implement workingset detection for anonymous LRU") 3852f6768ede542e aae466b0052e1888edd1d7f473d ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 2:4 1% 2:4 perf-profile.children.cycles-pp.error_entry 1:4 2% 2:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 23.24 ± 3% +8.6% 25.25 vm-scalability.free_time 451459 ± 2% +17.6% 531040 ± 2% vm-scalability.median 3613724 ± 2% +17.2% 4233483 ± 2% vm-scalability.throughput 147.90 ± 2% -11.1% 131.45 ± 2% vm-scalability.time.elapsed_time 147.90 ± 2% -11.1% 131.45 ± 2% vm-scalability.time.elapsed_time.max 126092 -40.2% 75396 ± 5% vm-scalability.time.involuntary_context_switches 709.75 +3.4% 733.75 vm-scalability.time.percent_of_cpu_this_job_got 858.01 ± 6% -12.7% 748.79 ± 3% vm-scalability.time.system_time 2555 ± 11% -86.5% 345.25 ± 19% vm-scalability.time.voluntary_context_switches 0.96 ± 22% +23.2% 1.18 ± 2% iostat.cpu.user 0.46 ± 2% -0.4 0.08 ± 6% mpstat.cpu.all.soft% 19058794 ± 73% -51.3% 9284205 ±108% numa-numastat.node2.numa_foreign 1166704 +28.5% 1499626 vmstat.memory.cache 2470874 ± 2% +12.5% 2779610 ± 2% vmstat.swap.so 5097 -25.6% 3792 ± 3% vmstat.system.cs 775916 ± 6% +15.8% 898759 vmstat.system.in 3.27e+09 ± 3% -9.3% 2.964e+09 ± 2% perf-node.node-load-misses 1.755e+09 ± 4% -16.1% 1.473e+09 ± 9% perf-node.node-loads 40.00 ± 4% -62.5% 15.00 perf-node.node-local-store-ratio 2.337e+09 ± 2% -39.6% 1.412e+09 ± 2% perf-node.node-store-misses 1.594e+09 ± 3% -83.9% 2.563e+08 ± 2% perf-node.node-stores 11725373 +15.5% 13547085 ± 6% meminfo.DirectMap2M 179916 +187.0% 516441 meminfo.KReclaimable 1116912 ± 5% +19.9% 1339332 ± 4% meminfo.MemAvailable 179916 +187.0% 516441 meminfo.SReclaimable 413707 +80.4% 746239 meminfo.Slab 218196 ± 2% +12.4% 245320 ± 2% meminfo.max_used_kB 11422 ± 11% +182.9% 32315 ± 25% numa-vmstat.node0.nr_slab_reclaimable 30123 +11.0% 33440 ± 4% numa-vmstat.node1.nr_page_table_pages 11318 ± 10% +187.6% 32549 ± 15% numa-vmstat.node1.nr_slab_reclaimable 14498502 +11.0% 16097678 ± 4% numa-vmstat.node1.nr_vmscan_write 14498504 +11.0% 16097680 ± 4% numa-vmstat.node1.nr_written 2099 ± 74% -67.5% 681.25 ± 58% numa-vmstat.node2.nr_active_anon 11274 ± 13% +172.3% 30698 ± 11% numa-vmstat.node2.nr_slab_reclaimable 2101 ± 74% -67.6% 681.50 ± 58% numa-vmstat.node2.nr_zone_active_anon 11491764 ± 72% -54.4% 5245129 ±107% numa-vmstat.node2.numa_foreign 10784 ± 7% +210.9% 33532 ± 6% numa-vmstat.node3.nr_slab_reclaimable 6117 ± 6% -15.4% 5175 ± 2% slabinfo.files_cache.active_objs 6124 ± 5% -15.5% 5175 ± 2% slabinfo.files_cache.num_objs 1269 ± 3% +21.3% 1539 ± 5% slabinfo.khugepaged_mm_slot.active_objs 1269 ± 3% +21.3% 1539 ± 5% slabinfo.khugepaged_mm_slot.num_objs 12779 ± 3% -15.2% 10839 ± 4% slabinfo.pde_opener.active_objs 12779 ± 3% -15.2% 10839 ± 4% slabinfo.pde_opener.num_objs 166610 ± 2% +352.8% 754410 slabinfo.radix_tree_node.active_objs 3337 ± 3% +318.4% 13961 slabinfo.radix_tree_node.active_slabs 167671 ± 2% +366.3% 781839 slabinfo.radix_tree_node.num_objs 3337 ± 3% +318.4% 13961 slabinfo.radix_tree_node.num_slabs 3109 ± 4% -10.4% 2785 slabinfo.sighand_cache.active_objs 3131 ± 4% -10.4% 2807 ± 2% slabinfo.sighand_cache.num_objs 8020 ± 4% -12.5% 7021 ± 2% slabinfo.task_delay_info.active_objs 8020 ± 4% -12.5% 7021 ± 2% slabinfo.task_delay_info.num_objs 45388 ± 11% +184.5% 129110 ± 25% numa-meminfo.node0.KReclaimable 45388 ± 11% +184.5% 129110 ± 25% numa-meminfo.node0.SReclaimable 114445 ± 14% +74.5% 199762 ± 18% numa-meminfo.node0.Slab 45359 ± 9% +187.0% 130162 ± 15% numa-meminfo.node1.KReclaimable 120458 +11.0% 133695 ± 4% numa-meminfo.node1.PageTables 45359 ± 9% +187.0% 130162 ± 15% numa-meminfo.node1.SReclaimable 109895 ± 16% +71.3% 188201 ± 13% numa-meminfo.node1.Slab 8457 ± 73% -67.5% 2751 ± 59% numa-meminfo.node2.Active 8389 ± 74% -67.5% 2726 ± 58% numa-meminfo.node2.Active(anon) 45582 ± 13% +169.3% 122741 ± 11% numa-meminfo.node2.KReclaimable 45582 ± 13% +169.3% 122741 ± 11% numa-meminfo.node2.SReclaimable 97401 ± 13% +73.7% 169199 ± 6% numa-meminfo.node2.Slab 43179 ± 6% +210.4% 134010 ± 6% numa-meminfo.node3.KReclaimable 43179 ± 6% +210.4% 134010 ± 6% numa-meminfo.node3.SReclaimable 91559 ± 9% +106.1% 188662 ± 8% numa-meminfo.node3.Slab 20729509 ± 16% -7.8e+06 12912354 ± 25% syscalls.sys_close.noise.100% 28960042 ± 11% -8.2e+06 20719899 ± 14% syscalls.sys_close.noise.2% 27766391 ± 11% -8.4e+06 19401559 ± 15% syscalls.sys_close.noise.25% 28908895 ± 11% -8.2e+06 20661833 ± 14% syscalls.sys_close.noise.5% 25426814 ± 13% -8.5e+06 16899949 ± 18% syscalls.sys_close.noise.50% 22869034 ± 14% -8e+06 14822779 ± 21% syscalls.sys_close.noise.75% 32447789 ± 25% -69.7% 9828013 ± 98% syscalls.sys_openat.max 1.654e+09 ± 39% -1.4e+09 2.337e+08 ± 39% syscalls.sys_openat.noise.100% 1.706e+09 ± 38% -1.4e+09 2.834e+08 ± 32% syscalls.sys_openat.noise.2% 1.693e+09 ± 38% -1.4e+09 2.712e+08 ± 34% syscalls.sys_openat.noise.25% 1.706e+09 ± 38% -1.4e+09 2.829e+08 ± 32% syscalls.sys_openat.noise.5% 1.678e+09 ± 38% -1.4e+09 2.548e+08 ± 36% syscalls.sys_openat.noise.50% 1.667e+09 ± 39% -1.4e+09 2.438e+08 ± 38% syscalls.sys_openat.noise.75% 1174 ± 22% +53.6% 1803 ± 5% syscalls.sys_write.med 5.911e+08 ± 64% +2e+09 2.555e+09 ± 2% syscalls.sys_write.noise.100% 6.002e+08 ± 63% +2e+09 2.56e+09 ± 2% syscalls.sys_write.noise.2% 5.985e+08 ± 64% +2e+09 2.559e+09 ± 2% syscalls.sys_write.noise.25% 6.002e+08 ± 63% +2e+09 2.56e+09 ± 2% syscalls.sys_write.noise.5% 5.953e+08 ± 64% +2e+09 2.558e+09 ± 2% syscalls.sys_write.noise.50% 5.93e+08 ± 64% +2e+09 2.557e+09 ± 2% syscalls.sys_write.noise.75% 52505 ± 2% +8.7% 57048 ± 4% proc-vmstat.allocstall_movable 1329701 ± 53% -76.6% 311376 ± 84% proc-vmstat.compact_daemon_migrate_scanned 3092 ±104% -98.4% 48.75 ± 73% proc-vmstat.compact_daemon_wake 91.25 ± 12% -83.0% 15.50 ± 39% proc-vmstat.compact_fail 1470217 ± 46% -68.7% 459729 ± 57% proc-vmstat.compact_migrate_scanned 97.50 ± 13% -81.3% 18.25 ± 32% proc-vmstat.compact_stall 3161 ± 98% -96.3% 117.50 ± 72% proc-vmstat.kswapd_low_wmark_hit_quickly 5191554 -2.4% 5065321 proc-vmstat.nr_anon_pages 5184927 -2.4% 5059469 proc-vmstat.nr_inactive_anon 187.75 ± 4% -9.1% 170.75 proc-vmstat.nr_isolated_anon 111810 +1.6% 113619 proc-vmstat.nr_page_table_pages 44657 +188.9% 129025 proc-vmstat.nr_slab_reclaimable 58446 -1.7% 57450 proc-vmstat.nr_slab_unreclaimable 5183621 -2.4% 5059220 proc-vmstat.nr_zone_inactive_anon 56750562 ± 10% -26.1% 41935745 ± 19% proc-vmstat.numa_pte_updates 1625054 ± 9% -100.0% 0.00 proc-vmstat.pgalloc_dma 8718205 ± 6% -40.3% 5207463 ± 26% proc-vmstat.pgalloc_dma32 1.346e+08 -30.0% 94227189 proc-vmstat.pgalloc_normal 1.451e+08 -31.4% 99498336 proc-vmstat.pgfree 63642 ± 3% -7.3% 59028 ± 5% proc-vmstat.slabs_scanned 456.00 ±127% +223.0% 1473 ± 21% proc-vmstat.swap_ra 345.00 ±165% +286.0% 1331 ± 20% proc-vmstat.swap_ra_hit 621120 ± 3% -12.0% 546816 ± 2% proc-vmstat.unevictable_pgs_scanned 5.92e+09 ± 3% +6.5% 6.304e+09 perf-stat.i.branch-instructions 61633368 ± 2% -24.6% 46490910 perf-stat.i.cache-misses 5041 -26.5% 3706 ± 3% perf-stat.i.context-switches 388.19 -23.9% 295.51 perf-stat.i.cpu-migrations 809.81 ± 8% +21.0% 980.06 ± 7% perf-stat.i.cycles-between-cache-misses 6.351e+09 +6.7% 6.776e+09 ± 2% perf-stat.i.dTLB-loads 0.15 ± 5% +0.0 0.18 ± 5% perf-stat.i.dTLB-store-miss-rate% 3.606e+09 ± 3% -7.4% 3.339e+09 ± 2% perf-stat.i.dTLB-stores 6823772 ± 5% +14.2% 7794812 ± 8% perf-stat.i.iTLB-load-misses 2.4e+10 ± 3% +4.7% 2.513e+10 perf-stat.i.instructions 112.11 +3.1% 115.57 perf-stat.i.metric.M/sec 666516 ± 2% +12.5% 749805 ± 2% perf-stat.i.minor-faults 61.38 ± 2% +20.6 82.01 perf-stat.i.node-store-miss-rate% 15267649 ± 4% -25.5% 11374283 ± 10% perf-stat.i.node-store-misses 10435945 ± 6% -79.8% 2104898 ± 10% perf-stat.i.node-stores 666572 ± 2% +12.5% 749872 ± 2% perf-stat.i.page-faults 659.86 ± 6% +28.5% 848.20 ± 6% perf-stat.overall.cycles-between-cache-misses 0.16 ± 3% +0.0 0.20 ± 3% perf-stat.overall.dTLB-store-miss-rate% 59.26 ± 2% +25.0 84.28 perf-stat.overall.node-store-miss-rate% 8026 -6.9% 7474 perf-stat.overall.path-length 5.902e+09 ± 3% +6.3% 6.274e+09 perf-stat.ps.branch-instructions 61374057 ± 2% -24.7% 46230779 perf-stat.ps.cache-misses 5019 -26.6% 3682 ± 3% perf-stat.ps.context-switches 386.19 -24.1% 293.29 perf-stat.ps.cpu-migrations 6.332e+09 +6.5% 6.744e+09 ± 2% perf-stat.ps.dTLB-loads 3.595e+09 ± 3% -7.6% 3.323e+09 ± 2% perf-stat.ps.dTLB-stores 6808353 ± 5% +14.1% 7769606 ± 8% perf-stat.ps.iTLB-load-misses 2.392e+10 ± 2% +4.6% 2.502e+10 perf-stat.ps.instructions 666070 ± 2% +12.2% 747635 ± 2% perf-stat.ps.minor-faults 15144045 ± 4% -25.7% 11246617 ± 10% perf-stat.ps.node-store-misses 10417852 ± 6% -79.9% 2096205 ± 10% perf-stat.ps.node-stores 666126 ± 2% +12.2% 747701 ± 2% perf-stat.ps.page-faults 3.555e+12 -6.8% 3.311e+12 perf-stat.total.instructions 12185 ± 8% +182.0% 34361 ± 8% sched_debug.cfs_rq:/.exec_clock.max 1188 ± 14% -98.8% 13.90 ± 53% sched_debug.cfs_rq:/.exec_clock.min 2713 ± 12% +171.7% 7373 ± 8% sched_debug.cfs_rq:/.exec_clock.stddev 158216 ± 20% -27.7% 114407 ± 17% sched_debug.cfs_rq:/.load.stddev 43.24 ± 15% -23.4% 33.11 ± 19% sched_debug.cfs_rq:/.load_avg.avg 1022 ± 3% -28.2% 733.92 ± 3% sched_debug.cfs_rq:/.load_avg.max 170.08 ± 8% -25.7% 126.42 ± 9% sched_debug.cfs_rq:/.load_avg.stddev 119278 ± 11% +110.4% 250918 ± 6% sched_debug.cfs_rq:/.min_vruntime.max 23859 ± 13% +115.1% 51319 ± 14% sched_debug.cfs_rq:/.min_vruntime.stddev 0.06 ± 48% -42.5% 0.04 ± 11% sched_debug.cfs_rq:/.nr_spread_over.avg 0.23 ± 26% -32.0% 0.16 ± 8% sched_debug.cfs_rq:/.nr_spread_over.stddev -50753 +175.6% -139895 sched_debug.cfs_rq:/.spread0.avg -83164 +122.8% -185253 sched_debug.cfs_rq:/.spread0.min 23868 ± 13% +115.0% 51318 ± 14% sched_debug.cfs_rq:/.spread0.stddev 140364 ± 14% +23.7% 173587 ± 4% sched_debug.cpu.avg_idle.stddev 678.88 ± 4% +20.0% 814.72 ± 5% sched_debug.cpu.clock_task.stddev 0.00 ± 26% +566.5% 0.00 ± 91% sched_debug.cpu.next_balance.stddev 3913 -17.4% 3233 ± 2% sched_debug.cpu.nr_switches.avg 14639 ± 12% +55.0% 22684 ± 19% sched_debug.cpu.nr_switches.max 1791 ± 8% -37.4% 1121 ± 7% sched_debug.cpu.nr_switches.min 2060 ± 8% +27.8% 2633 ± 6% sched_debug.cpu.nr_switches.stddev 2394 ± 2% -28.3% 1716 ± 5% sched_debug.cpu.sched_count.avg 10139 ± 24% +79.3% 18179 ± 24% sched_debug.cpu.sched_count.max 961.92 ± 11% -54.9% 434.25 sched_debug.cpu.sched_count.min 1375 ± 14% +47.3% 2025 ± 10% sched_debug.cpu.sched_count.stddev 626.15 ± 4% -19.1% 506.24 ± 5% sched_debug.cpu.sched_goidle.avg 4252 ± 25% +109.7% 8919 ± 24% sched_debug.cpu.sched_goidle.max 277.58 ± 11% -63.6% 101.00 ± 4% sched_debug.cpu.sched_goidle.min 554.97 ± 9% +71.0% 949.07 ± 11% sched_debug.cpu.sched_goidle.stddev 1105 ± 2% -30.2% 771.36 ± 4% sched_debug.cpu.ttwu_count.avg 4824 ± 23% +89.2% 9126 ± 21% sched_debug.cpu.ttwu_count.max 364.25 ± 13% -61.1% 141.83 sched_debug.cpu.ttwu_count.min 727.66 ± 8% +44.9% 1054 ± 8% sched_debug.cpu.ttwu_count.stddev 754.79 -25.7% 560.55 ± 5% sched_debug.cpu.ttwu_local.avg 1844 ± 5% +94.2% 3581 ± 25% sched_debug.cpu.ttwu_local.max 281.33 ± 10% -50.5% 139.25 sched_debug.cpu.ttwu_local.min 338.12 ± 10% +70.6% 576.68 ± 5% sched_debug.cpu.ttwu_local.stddev 4.68 ± 4% -4.0 0.73 ± 44% perf-profile.calltrace.cycles-pp.add_to_swap_cache.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec 6.00 ± 4% -3.2 2.75 ± 13% perf-profile.calltrace.cycles-pp.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node 1.82 ± 15% -1.5 0.28 ±100% perf-profile.calltrace.cycles-pp.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page 1.74 ± 5% -1.4 0.29 ±100% perf-profile.calltrace.cycles-pp.__delete_from_swap_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec 2.55 ± 11% -1.4 1.16 ± 12% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page.__handle_mm_fault 1.44 ± 15% -1.0 0.47 ±106% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 1.43 ± 16% -1.0 0.47 ±106% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt 1.43 ± 16% -1.0 0.47 ±106% perf-profile.calltrace.cycles-pp.asm_call_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 1.55 ± 16% -0.8 0.70 ± 68% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.10 ± 15% -0.5 1.61 ± 12% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 2.12 ± 8% +0.4 2.52 ± 10% perf-profile.calltrace.cycles-pp.end_page_writeback.pmem_rw_page.bdev_write_page.__swap_writepage.pageout 2.13 ± 5% +0.4 2.58 ± 11% perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_anon.try_to_unmap.shrink_page_list.shrink_inactive_list 0.95 ± 26% +0.5 1.47 ± 13% perf-profile.calltrace.cycles-pp.swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec 1.28 ± 4% +0.5 1.81 ± 14% perf-profile.calltrace.cycles-pp.page_counter_uncharge.mem_cgroup_swapout.__remove_mapping.shrink_page_list.shrink_inactive_list 1.27 ± 4% +0.5 1.80 ± 14% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.mem_cgroup_swapout.__remove_mapping.shrink_page_list 0.54 ± 71% +0.9 1.41 ± 10% perf-profile.calltrace.cycles-pp.__test_set_page_writeback.bdev_write_page.__swap_writepage.pageout.shrink_page_list 2.52 ± 5% +0.9 3.44 ± 14% perf-profile.calltrace.cycles-pp.mem_cgroup_swapout.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec 5.49 ± 4% -4.8 0.69 ± 44% perf-profile.children.cycles-pp.__softirqentry_text_start 4.76 ± 4% -3.7 1.09 ± 12% perf-profile.children.cycles-pp.add_to_swap_cache 3.74 ± 5% -3.6 0.16 ± 19% perf-profile.children.cycles-pp.xas_create_range 3.74 ± 5% -3.5 0.27 ± 15% perf-profile.children.cycles-pp.xas_create 6.06 ± 4% -3.3 2.77 ± 13% perf-profile.children.cycles-pp.add_to_swap 2.32 ± 11% -1.6 0.70 ± 44% perf-profile.children.cycles-pp.do_softirq_own_stack 2.61 ± 11% -1.6 1.04 ± 35% perf-profile.children.cycles-pp.irq_exit_rcu 5.20 ± 11% -1.5 3.68 ± 11% perf-profile.children.cycles-pp.get_page_from_freelist 3.78 ± 14% -1.3 2.45 ± 12% perf-profile.children.cycles-pp.prep_new_page 3.61 ± 14% -1.3 2.32 ± 12% perf-profile.children.cycles-pp.clear_page_erms 1.21 ± 7% -1.0 0.21 ± 20% perf-profile.children.cycles-pp.xas_store 1.75 ± 5% -0.9 0.86 ± 13% perf-profile.children.cycles-pp.__delete_from_swap_cache 2.17 ± 14% -0.5 1.63 ± 12% perf-profile.children.cycles-pp.mem_cgroup_charge 0.61 ± 17% -0.5 0.15 ± 25% perf-profile.children.cycles-pp.worker_thread 0.58 ± 19% -0.4 0.13 ± 24% perf-profile.children.cycles-pp.process_one_work 0.82 ± 7% -0.3 0.56 ± 10% perf-profile.children.cycles-pp.rmqueue 0.75 ± 13% -0.2 0.52 ± 8% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics 0.55 ± 6% -0.2 0.37 ± 11% perf-profile.children.cycles-pp.free_pcppages_bulk 0.77 ± 13% -0.2 0.59 ± 5% perf-profile.children.cycles-pp.__count_memcg_events 0.19 ± 30% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.__libc_fork 0.48 ± 13% -0.1 0.34 ± 13% perf-profile.children.cycles-pp.try_charge 0.36 ± 13% -0.1 0.23 ± 14% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.34 ± 12% -0.1 0.22 ± 13% perf-profile.children.cycles-pp.do_syscall_64 0.15 ± 22% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__do_sys_clone 0.15 ± 22% -0.1 0.03 ±100% perf-profile.children.cycles-pp._do_fork 0.21 ± 7% -0.1 0.11 ± 23% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work 0.35 ± 5% -0.1 0.27 ± 13% perf-profile.children.cycles-pp.rmqueue_bulk 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.drm_atomic_helper_dirtyfb 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.memcpy_toio 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.drm_atomic_helper_commit 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.commit_tail 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.drm_atomic_helper_commit_tail 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.drm_atomic_helper_commit_planes 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.mgag200_simple_display_pipe_update 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.mgag200_handle_damage 0.15 ± 14% -0.1 0.09 ± 23% perf-profile.children.cycles-pp.drm_fb_memcpy_dstclip 0.10 ± 8% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.schedule 0.08 ± 19% +0.0 0.11 ± 12% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.14 ± 3% +0.0 0.17 ± 10% perf-profile.children.cycles-pp.default_send_IPI_single_phys 0.12 ± 3% +0.0 0.16 ± 9% perf-profile.children.cycles-pp.scan_swap_map_try_ssd_cluster 0.10 ± 11% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.inc_node_page_state 0.03 ±102% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.arch_cpu_idle_enter 0.03 ±102% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust 0.13 ± 11% +0.0 0.18 ± 14% perf-profile.children.cycles-pp.cpumask_any_but 0.23 ± 8% +0.1 0.29 ± 10% perf-profile.children.cycles-pp.scan_swap_map_slots 0.15 ± 12% +0.1 0.23 ± 7% perf-profile.children.cycles-pp.lock_page_memcg 0.31 ± 5% +0.1 0.39 ± 8% perf-profile.children.cycles-pp.test_clear_page_writeback 0.29 ± 6% +0.1 0.38 ± 11% perf-profile.children.cycles-pp.get_swap_pages 0.37 ± 7% +0.1 0.46 ± 16% perf-profile.children.cycles-pp.llist_reverse_order 0.39 ± 8% +0.1 0.52 ± 11% perf-profile.children.cycles-pp.tlb_is_not_lazy 0.35 ± 8% +0.1 0.48 ± 14% perf-profile.children.cycles-pp.zswap_frontswap_store 0.45 ± 2% +0.2 0.63 ± 14% perf-profile.children.cycles-pp.page_swapcount 0.49 ± 3% +0.2 0.67 ± 14% perf-profile.children.cycles-pp.try_to_free_swap 0.82 ± 4% +0.2 1.01 ± 11% perf-profile.children.cycles-pp.put_swap_page 0.55 ± 9% +0.2 0.79 ± 14% perf-profile.children.cycles-pp.__frontswap_store 0.81 ± 6% +0.3 1.07 ± 13% perf-profile.children.cycles-pp.get_swap_page 0.62 ± 7% +0.3 0.89 ± 17% perf-profile.children.cycles-pp.page_swap_info 0.58 ± 8% +0.3 0.85 ± 17% perf-profile.children.cycles-pp.swap_set_page_dirty 2.18 ± 8% +0.4 2.56 ± 10% perf-profile.children.cycles-pp.end_page_writeback 1.05 ± 3% +0.4 1.45 ± 11% perf-profile.children.cycles-pp._swap_info_get 1.04 ± 5% +0.4 1.44 ± 11% perf-profile.children.cycles-pp.__test_set_page_writeback 1.07 ± 6% +0.4 1.48 ± 13% perf-profile.children.cycles-pp.swap_writepage 2.17 ± 5% +0.4 2.61 ± 11% perf-profile.children.cycles-pp.try_to_unmap_one 2.23 ± 5% +0.5 2.71 ± 15% perf-profile.children.cycles-pp.__sysvec_call_function_single 2.52 ± 6% +0.5 3.05 ± 15% perf-profile.children.cycles-pp.sysvec_call_function_single 1.30 ± 4% +0.5 1.83 ± 14% perf-profile.children.cycles-pp.page_counter_uncharge 1.29 ± 4% +0.5 1.82 ± 14% perf-profile.children.cycles-pp.page_counter_cancel 2.56 ± 5% +0.9 3.46 ± 14% perf-profile.children.cycles-pp.mem_cgroup_swapout 3.38 ± 13% -1.2 2.17 ± 12% perf-profile.self.cycles-pp.clear_page_erms 0.77 ± 13% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.__count_memcg_events 0.22 ± 8% -0.1 0.08 ± 15% perf-profile.self.cycles-pp.xas_store 0.32 ± 15% -0.1 0.18 ± 12% perf-profile.self.cycles-pp.try_charge 0.34 ± 9% -0.1 0.22 ± 9% perf-profile.self.cycles-pp.mem_cgroup_charge 0.35 ± 7% -0.1 0.24 ± 17% perf-profile.self.cycles-pp.xas_create 0.11 ± 10% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.smp_call_function_many_cond 0.18 ± 4% -0.0 0.13 ± 16% perf-profile.self.cycles-pp.rmqueue_bulk 0.13 ± 13% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.rmqueue 0.12 ± 15% -0.0 0.08 ± 14% perf-profile.self.cycles-pp.page_remove_rmap 0.15 ± 15% -0.0 0.12 ± 21% perf-profile.self.cycles-pp.prep_new_page 0.09 ± 13% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.inc_node_page_state 0.05 ± 58% +0.0 0.08 ± 8% perf-profile.self.cycles-pp.scan_swap_map_slots 0.01 ±173% +0.0 0.05 ± 9% perf-profile.self.cycles-pp.dec_zone_page_state 0.01 ±173% +0.0 0.05 ± 9% perf-profile.self.cycles-pp.mutex_unlock 0.01 ±173% +0.0 0.06 ± 20% perf-profile.self.cycles-pp.irqtime_account_process_tick 0.13 ± 11% +0.1 0.19 ± 16% perf-profile.self.cycles-pp.mem_cgroup_swapout 0.08 ± 15% +0.1 0.15 ± 14% perf-profile.self.cycles-pp.arch_tlbbatch_flush 0.15 ± 11% +0.1 0.21 ± 10% perf-profile.self.cycles-pp.lock_page_memcg 0.37 ± 7% +0.1 0.46 ± 16% perf-profile.self.cycles-pp.llist_reverse_order 0.18 ± 15% +0.1 0.28 ± 13% perf-profile.self.cycles-pp.__frontswap_store 0.57 +0.1 0.67 ± 11% perf-profile.self.cycles-pp.__delete_from_swap_cache 0.60 ± 5% +0.1 0.70 ± 12% perf-profile.self.cycles-pp.add_to_swap_cache 0.33 ± 7% +0.1 0.45 ± 13% perf-profile.self.cycles-pp.zswap_frontswap_store 0.45 ± 2% +0.1 0.57 ± 14% perf-profile.self.cycles-pp.flush_tlb_func_common 0.38 ± 7% +0.1 0.52 ± 13% perf-profile.self.cycles-pp.get_swap_page 0.74 ± 3% +0.2 0.91 ± 10% perf-profile.self.cycles-pp.try_to_unmap_one 0.57 ± 9% +0.2 0.81 ± 16% perf-profile.self.cycles-pp.page_swap_info 0.72 ± 3% +0.3 1.03 ± 11% perf-profile.self.cycles-pp.__test_set_page_writeback 1.68 ± 6% +0.3 2.00 ± 9% perf-profile.self.cycles-pp.end_page_writeback 0.98 ± 3% +0.4 1.35 ± 12% perf-profile.self.cycles-pp._swap_info_get 1.22 ± 5% +0.5 1.75 ± 14% perf-profile.self.cycles-pp.page_counter_cancel 52345 ± 17% -84.0% 8384 ± 41% softirqs.CPU0.RCU 19063 ± 7% -20.7% 15125 ± 6% softirqs.CPU0.SCHED 36466 ± 32% -86.4% 4972 ± 11% softirqs.CPU1.RCU 36608 ± 40% -87.4% 4628 ± 19% softirqs.CPU10.RCU 26212 ± 19% -86.4% 3553 ± 2% softirqs.CPU100.RCU 27448 ± 17% -86.5% 3706 ± 6% softirqs.CPU101.RCU 27842 ± 29% -87.2% 3562 ± 4% softirqs.CPU102.RCU 26457 ± 29% -87.1% 3409 ± 4% softirqs.CPU103.RCU 27085 ± 19% -86.7% 3602 ± 6% softirqs.CPU104.RCU 27395 ± 20% -86.8% 3611 ± 8% softirqs.CPU105.RCU 25853 ± 23% -86.6% 3457 ± 5% softirqs.CPU106.RCU 27375 ± 24% -86.5% 3696 ± 6% softirqs.CPU107.RCU 41902 ± 27% -90.1% 4168 ± 47% softirqs.CPU108.RCU 18997 ± 7% -15.2% 16115 ± 3% softirqs.CPU108.SCHED 35065 ± 28% -91.6% 2936 ± 11% softirqs.CPU109.RCU 35638 ± 34% -85.2% 5271 ± 41% softirqs.CPU11.RCU 34068 ± 26% -91.6% 2847 ± 12% softirqs.CPU110.RCU 19277 ± 6% -18.1% 15790 ± 9% softirqs.CPU110.SCHED 34270 ± 24% -89.9% 3470 ± 31% softirqs.CPU111.RCU 34839 ± 33% -90.4% 3339 ± 3% softirqs.CPU112.RCU 38104 ± 27% -91.3% 3308 ± 5% softirqs.CPU113.RCU 32552 ± 31% -90.9% 2966 ± 8% softirqs.CPU114.RCU 33623 ± 26% -90.2% 3303 ± 7% softirqs.CPU115.RCU 35550 ± 27% -90.4% 3417 ± 8% softirqs.CPU116.RCU 33356 ± 27% -89.8% 3415 ± 5% softirqs.CPU117.RCU 33993 ± 32% -90.0% 3405 ± 12% softirqs.CPU118.RCU 36253 ± 26% -90.3% 3519 ± 3% softirqs.CPU119.RCU 34824 ± 31% -87.1% 4494 ± 9% softirqs.CPU12.RCU 32768 ± 39% -88.8% 3661 ± 11% softirqs.CPU120.RCU 36094 ± 23% -90.6% 3401 ± 3% softirqs.CPU121.RCU 34497 ± 31% -89.0% 3807 ± 12% softirqs.CPU122.RCU 36262 ± 25% -88.4% 4193 ± 32% softirqs.CPU123.RCU 34252 ± 25% -90.1% 3394 ± 5% softirqs.CPU124.RCU 36674 ± 21% -88.3% 4301 ± 25% softirqs.CPU125.RCU 27951 ± 7% -88.9% 3101 ± 10% softirqs.CPU126.RCU 18871 ± 7% -14.7% 16104 ± 3% softirqs.CPU126.SCHED 23136 ± 14% -88.8% 2600 ± 4% softirqs.CPU127.RCU 19431 ± 5% -22.5% 15053 ± 14% softirqs.CPU127.SCHED 27391 ± 5% -88.3% 3205 softirqs.CPU128.RCU 25272 ± 14% -86.9% 3319 ± 4% softirqs.CPU129.RCU 35293 ± 39% -88.5% 4050 ± 11% softirqs.CPU13.RCU 26272 ± 10% -88.4% 3041 ± 10% softirqs.CPU130.RCU 25115 ± 5% -86.9% 3291 ± 3% softirqs.CPU131.RCU 23529 ± 6% -86.9% 3071 ± 10% softirqs.CPU132.RCU 27781 ± 13% -86.5% 3755 ± 15% softirqs.CPU133.RCU 26065 ± 16% -87.4% 3289 ± 10% softirqs.CPU134.RCU 23998 ± 10% -86.3% 3277 ± 3% softirqs.CPU135.RCU 26249 ± 18% -87.3% 3328 ± 5% softirqs.CPU136.RCU 21876 ± 13% -82.2% 3887 ± 14% softirqs.CPU137.RCU 22414 ± 10% -82.1% 4001 ± 18% softirqs.CPU138.RCU 24824 ± 5% -85.5% 3595 ± 9% softirqs.CPU139.RCU 34692 ± 44% -88.7% 3921 ± 6% softirqs.CPU14.RCU 22513 ± 12% -84.8% 3418 ± 7% softirqs.CPU140.RCU 23522 ± 5% -85.0% 3517 ± 7% softirqs.CPU141.RCU 22534 ± 11% -84.6% 3469 ± 8% softirqs.CPU142.RCU 25471 ± 22% -84.8% 3869 ± 18% softirqs.CPU143.RCU 33424 ± 43% -87.6% 4133 ± 9% softirqs.CPU15.RCU 32536 ± 25% -89.7% 3340 ± 4% softirqs.CPU16.RCU 31138 ± 29% -88.7% 3526 ± 7% softirqs.CPU17.RCU 51275 ± 11% -92.3% 3956 ± 11% softirqs.CPU18.RCU 16486 ± 6% -21.1% 13010 ± 12% softirqs.CPU18.SCHED 37422 ± 27% -86.3% 5144 ± 42% softirqs.CPU19.RCU 18953 ± 6% -29.2% 13419 ± 20% softirqs.CPU19.SCHED 38774 ± 41% -88.8% 4336 ± 11% softirqs.CPU2.RCU 19636 ± 2% -19.1% 15894 ± 7% softirqs.CPU2.SCHED 37496 ± 28% -90.0% 3733 ± 8% softirqs.CPU20.RCU 19096 ± 5% -19.5% 15372 ± 19% softirqs.CPU20.SCHED 36543 ± 22% -89.9% 3688 ± 5% softirqs.CPU21.RCU 33600 ± 22% -88.2% 3950 ± 16% softirqs.CPU22.RCU 38695 ± 21% -90.1% 3830 ± 4% softirqs.CPU23.RCU 35007 ± 22% -89.8% 3559 ± 7% softirqs.CPU24.RCU 19064 ± 4% -12.9% 16604 ± 8% softirqs.CPU24.SCHED 37485 ± 21% -90.1% 3728 ± 6% softirqs.CPU25.RCU 33627 ± 28% -88.8% 3763 ± 7% softirqs.CPU26.RCU 36026 ± 24% -89.4% 3829 ± 3% softirqs.CPU27.RCU 34690 ± 29% -88.4% 4028 ± 5% softirqs.CPU28.RCU 36943 ± 24% -89.5% 3882 ± 3% softirqs.CPU29.RCU 38484 ± 41% -85.7% 5499 ± 39% softirqs.CPU3.RCU 34943 ± 26% -89.3% 3743 ± 3% softirqs.CPU30.RCU 34022 ± 31% -88.5% 3927 ± 5% softirqs.CPU31.RCU 34577 ± 26% -90.2% 3392 ± 7% softirqs.CPU32.RCU 34324 ± 39% -90.2% 3379 ± 4% softirqs.CPU33.RCU 32528 ± 21% -89.1% 3560 ± 11% softirqs.CPU34.RCU 35054 ± 38% -86.3% 4802 ± 28% softirqs.CPU35.RCU 55830 ± 22% -91.4% 4827 ± 19% softirqs.CPU36.RCU 18131 ± 5% -32.8% 12178 ± 16% softirqs.CPU36.SCHED 44535 ± 25% -91.5% 3800 ± 6% softirqs.CPU37.RCU 18780 ± 6% -17.0% 15579 ± 12% softirqs.CPU37.SCHED 46422 ± 32% -90.5% 4423 ± 17% softirqs.CPU38.RCU 18645 ± 6% -27.3% 13557 ± 20% softirqs.CPU38.SCHED 43345 ± 22% -88.7% 4918 ± 36% softirqs.CPU39.RCU 32960 ± 30% -88.1% 3917 ± 4% softirqs.CPU4.RCU 44490 ± 29% -91.7% 3685 ± 2% softirqs.CPU40.RCU 46120 ± 29% -91.8% 3798 ± 2% softirqs.CPU41.RCU 42257 ± 32% -91.6% 3551 ± 4% softirqs.CPU42.RCU 46039 ± 20% -90.9% 4188 ± 21% softirqs.CPU43.RCU 44088 ± 29% -91.8% 3604 softirqs.CPU44.RCU 41672 ± 31% -91.2% 3647 softirqs.CPU45.RCU 44499 ± 25% -91.9% 3597 ± 8% softirqs.CPU46.RCU 47826 ± 27% -92.4% 3616 softirqs.CPU47.RCU 40295 ± 41% -91.3% 3518 softirqs.CPU48.RCU 41600 ± 24% -91.6% 3497 softirqs.CPU49.RCU 36523 ± 32% -88.8% 4102 ± 6% softirqs.CPU5.RCU 43678 ± 34% -92.3% 3347 ± 4% softirqs.CPU50.RCU 44579 ± 28% -90.7% 4125 ± 15% softirqs.CPU51.RCU 43503 ± 28% -92.2% 3396 ± 3% softirqs.CPU52.RCU 44468 ± 18% -90.5% 4219 ± 33% softirqs.CPU53.RCU 49302 ± 5% -90.4% 4747 ± 22% softirqs.CPU54.RCU 16826 ± 6% -23.7% 12844 ± 10% softirqs.CPU54.SCHED 34574 ± 13% -87.8% 4220 ± 18% softirqs.CPU55.RCU 18223 ± 8% -35.1% 11830 ± 25% softirqs.CPU55.SCHED 34444 ± 10% -88.1% 4102 ± 21% softirqs.CPU56.RCU 29792 ± 13% -87.5% 3728 ± 11% softirqs.CPU57.RCU 29979 ± 15% -86.7% 3997 ± 16% softirqs.CPU58.RCU 29522 ± 13% -87.5% 3687 ± 8% softirqs.CPU59.RCU 34227 ± 39% -88.9% 3815 ± 6% softirqs.CPU6.RCU 19618 ± 4% -10.2% 17615 ± 4% softirqs.CPU6.SCHED 29331 ± 13% -88.2% 3454 ± 8% softirqs.CPU60.RCU 30844 ± 15% -88.5% 3555 ± 4% softirqs.CPU61.RCU 30676 ± 16% -87.1% 3947 ± 18% softirqs.CPU62.RCU 31209 ± 20% -89.1% 3399 ± 8% softirqs.CPU63.RCU 32203 ± 16% -89.0% 3550 ± 12% softirqs.CPU64.RCU 26618 ± 18% -86.7% 3543 ± 4% softirqs.CPU65.RCU 28476 ± 20% -86.5% 3856 ± 11% softirqs.CPU66.RCU 29713 ± 12% -87.1% 3842 ± 14% softirqs.CPU67.RCU 28361 ± 25% -87.2% 3628 ± 6% softirqs.CPU68.RCU 30363 ± 18% -87.3% 3850 ± 9% softirqs.CPU69.RCU 40268 ± 49% -90.1% 3997 ± 6% softirqs.CPU7.RCU 29067 ± 11% -83.9% 4679 ± 38% softirqs.CPU70.RCU 30143 ± 14% -87.5% 3758 ± 4% softirqs.CPU71.RCU 33102 ± 37% -90.7% 3088 ± 6% softirqs.CPU72.RCU 29408 ± 29% -88.2% 3474 ± 5% softirqs.CPU73.RCU 19427 -18.5% 15841 ± 10% softirqs.CPU73.SCHED 31208 ± 34% -89.4% 3312 softirqs.CPU74.RCU 19059 ± 2% -12.2% 16733 ± 7% softirqs.CPU74.SCHED 29231 ± 42% -88.2% 3446 ± 2% softirqs.CPU75.RCU 27628 ± 26% -86.2% 3814 ± 21% softirqs.CPU76.RCU 26377 ± 27% -87.0% 3428 ± 2% softirqs.CPU77.RCU 26349 ± 50% -86.3% 3622 ± 18% softirqs.CPU78.RCU 30531 ± 48% -88.9% 3389 ± 7% softirqs.CPU79.RCU 35512 ± 42% -88.0% 4272 ± 4% softirqs.CPU8.RCU 25522 ± 36% -86.9% 3339 ± 16% softirqs.CPU80.RCU 27427 ± 48% -86.2% 3784 ± 16% softirqs.CPU81.RCU 25383 ± 40% -82.8% 4368 ± 33% softirqs.CPU82.RCU 26528 ± 40% -84.2% 4180 ± 15% softirqs.CPU83.RCU 26637 ± 33% -85.5% 3849 ± 9% softirqs.CPU84.RCU 27814 ± 30% -86.6% 3735 ± 8% softirqs.CPU85.RCU 27615 ± 39% -82.7% 4764 ± 19% softirqs.CPU86.RCU 27935 ± 40% -85.1% 4154 ± 14% softirqs.CPU87.RCU 27323 ± 26% -84.0% 4380 ± 15% softirqs.CPU88.RCU 26788 ± 28% -85.9% 3788 ± 2% softirqs.CPU89.RCU 34530 ± 43% -88.9% 3845 ± 11% softirqs.CPU9.RCU 34234 ± 12% -90.2% 3338 softirqs.CPU90.RCU 19353 ± 3% -24.2% 14671 ± 2% softirqs.CPU90.SCHED 27937 ± 28% -85.9% 3926 ± 15% softirqs.CPU91.RCU 19542 ± 6% -15.2% 16566 ± 3% softirqs.CPU91.SCHED 28497 ± 22% -87.9% 3441 ± 14% softirqs.CPU92.RCU 28320 ± 18% -89.1% 3074 ± 4% softirqs.CPU93.RCU 26306 ± 11% -85.4% 3840 ± 27% softirqs.CPU94.RCU 19738 ± 4% -12.9% 17184 ± 9% softirqs.CPU94.SCHED 26514 ± 8% -87.0% 3439 ± 23% softirqs.CPU95.RCU 25298 ± 14% -86.9% 3318 ± 10% softirqs.CPU96.RCU 19776 ± 3% -13.9% 17031 ± 6% softirqs.CPU96.SCHED 27549 ± 12% -87.5% 3438 ± 6% softirqs.CPU97.RCU 28550 ± 25% -88.7% 3235 ± 9% softirqs.CPU98.RCU 26684 ± 19% -87.0% 3473 ± 6% softirqs.CPU99.RCU 4756845 ± 5% -88.5% 544700 softirqs.RCU 30919 ± 3% -13.8% 26641 ± 2% softirqs.TIMER 145.50 ± 2% -56.0% 64.00 ±100% interrupts.175:PCI-MSI.512000-edge.ahci[0000:00:1f.2] 112.25 ± 3% -8.7% 102.50 ± 4% interrupts.45:PCI-MSI.1572864-edge.eth0-TxRx-0 78296627 +5.0% 82208891 interrupts.CAL:Function_call_interrupts 112.25 ± 3% -8.7% 102.50 ± 4% interrupts.CPU0.45:PCI-MSI.1572864-edge.eth0-TxRx-0 1231976 ± 21% +185.1% 3511906 ± 22% interrupts.CPU0.CAL:Function_call_interrupts 1343291 ± 20% +181.1% 3775855 ± 22% interrupts.CPU0.TLB:TLB_shootdowns 608271 ± 75% -99.3% 4379 ±170% interrupts.CPU10.CAL:Function_call_interrupts 457.50 ± 61% -79.9% 91.75 ± 38% interrupts.CPU10.NMI:Non-maskable_interrupts 457.50 ± 61% -79.9% 91.75 ± 38% interrupts.CPU10.PMI:Performance_monitoring_interrupts 89.00 ± 59% -89.9% 9.00 ± 79% interrupts.CPU10.RES:Rescheduling_interrupts 657079 ± 75% -99.3% 4336 ±173% interrupts.CPU10.TLB:TLB_shootdowns 411365 ± 29% -99.7% 1267 ± 64% interrupts.CPU100.CAL:Function_call_interrupts 444274 ± 30% -99.7% 1249 ± 69% interrupts.CPU100.TLB:TLB_shootdowns 659.50 ± 68% -74.6% 167.50 ± 22% interrupts.CPU102.NMI:Non-maskable_interrupts 659.50 ± 68% -74.6% 167.50 ± 22% interrupts.CPU102.PMI:Performance_monitoring_interrupts 302129 ± 60% -99.9% 166.75 ± 59% interrupts.CPU103.CAL:Function_call_interrupts 323.75 ± 53% -56.4% 141.00 ± 34% interrupts.CPU103.NMI:Non-maskable_interrupts 323.75 ± 53% -56.4% 141.00 ± 34% interrupts.CPU103.PMI:Performance_monitoring_interrupts 327333 ± 60% -100.0% 32.50 ±166% interrupts.CPU103.TLB:TLB_shootdowns 312284 ± 48% -53.9% 144043 ± 50% interrupts.CPU104.CAL:Function_call_interrupts 338648 ± 48% -54.3% 154674 ± 50% interrupts.CPU104.TLB:TLB_shootdowns 115.50 ± 72% -81.0% 22.00 ± 19% interrupts.CPU108.RES:Rescheduling_interrupts 626496 ± 83% -90.8% 57927 ±172% interrupts.CPU11.CAL:Function_call_interrupts 78.00 ± 60% -73.1% 21.00 ± 86% interrupts.CPU11.RES:Rescheduling_interrupts 679997 ± 83% -90.5% 64580 ±173% interrupts.CPU11.TLB:TLB_shootdowns 81.25 ± 51% -80.3% 16.00 ± 11% interrupts.CPU111.RES:Rescheduling_interrupts 87.50 ± 71% -89.7% 9.00 ± 40% interrupts.CPU112.RES:Rescheduling_interrupts 645406 ± 66% -89.8% 66133 ±103% interrupts.CPU113.CAL:Function_call_interrupts 103.50 ± 48% -89.6% 10.75 ± 47% interrupts.CPU113.RES:Rescheduling_interrupts 698561 ± 66% -89.8% 71091 ±104% interrupts.CPU113.TLB:TLB_shootdowns 78.00 ± 68% -86.2% 10.75 ± 41% interrupts.CPU114.RES:Rescheduling_interrupts 1141 ± 56% -72.3% 316.50 ±106% interrupts.CPU115.NMI:Non-maskable_interrupts 1141 ± 56% -72.3% 316.50 ±106% interrupts.CPU115.PMI:Performance_monitoring_interrupts 91.00 ± 62% -90.1% 9.00 ± 26% interrupts.CPU115.RES:Rescheduling_interrupts 757124 ± 58% -60.1% 302332 ±132% interrupts.CPU116.CAL:Function_call_interrupts 98.25 ± 46% -88.0% 11.75 ± 63% interrupts.CPU116.RES:Rescheduling_interrupts 823491 ± 58% -60.9% 322256 ±132% interrupts.CPU116.TLB:TLB_shootdowns 628006 ± 76% -73.8% 164708 ±146% interrupts.CPU117.CAL:Function_call_interrupts 987.50 ± 49% -85.9% 139.25 ± 30% interrupts.CPU117.NMI:Non-maskable_interrupts 987.50 ± 49% -85.9% 139.25 ± 30% interrupts.CPU117.PMI:Performance_monitoring_interrupts 109.25 ± 69% -93.1% 7.50 ± 52% interrupts.CPU117.RES:Rescheduling_interrupts 681610 ± 76% -74.4% 174707 ±146% interrupts.CPU117.TLB:TLB_shootdowns 104.00 ± 55% -87.7% 12.75 ± 78% interrupts.CPU119.RES:Rescheduling_interrupts 471.25 ± 54% -80.9% 90.00 ± 38% interrupts.CPU12.NMI:Non-maskable_interrupts 471.25 ± 54% -80.9% 90.00 ± 38% interrupts.CPU12.PMI:Performance_monitoring_interrupts 58.25 ± 79% -76.4% 13.75 ± 88% interrupts.CPU12.RES:Rescheduling_interrupts 878.50 ± 72% -88.3% 103.00 ± 43% interrupts.CPU120.NMI:Non-maskable_interrupts 878.50 ± 72% -88.3% 103.00 ± 43% interrupts.CPU120.PMI:Performance_monitoring_interrupts 64.00 ± 77% -85.5% 9.25 ± 68% interrupts.CPU120.RES:Rescheduling_interrupts 868.50 ± 54% -84.1% 138.50 ± 45% interrupts.CPU121.NMI:Non-maskable_interrupts 868.50 ± 54% -84.1% 138.50 ± 45% interrupts.CPU121.PMI:Performance_monitoring_interrupts 597802 ± 74% -99.8% 1291 ±164% interrupts.CPU122.CAL:Function_call_interrupts 804.25 ± 83% -85.9% 113.25 ± 65% interrupts.CPU122.NMI:Non-maskable_interrupts 804.25 ± 83% -85.9% 113.25 ± 65% interrupts.CPU122.PMI:Performance_monitoring_interrupts 95.50 ± 48% -92.7% 7.00 ± 56% interrupts.CPU122.RES:Rescheduling_interrupts 649164 ± 75% -99.8% 1290 ±173% interrupts.CPU122.TLB:TLB_shootdowns 114.50 ± 54% -93.7% 7.25 ± 48% interrupts.CPU123.RES:Rescheduling_interrupts 629610 ± 63% -69.6% 191537 ±168% interrupts.CPU124.CAL:Function_call_interrupts 1011 ± 58% -86.9% 132.50 ± 45% interrupts.CPU124.NMI:Non-maskable_interrupts 1011 ± 58% -86.9% 132.50 ± 45% interrupts.CPU124.PMI:Performance_monitoring_interrupts 70.25 ± 70% -85.4% 10.25 ± 44% interrupts.CPU124.RES:Rescheduling_interrupts 683705 ± 63% -69.7% 206886 ±168% interrupts.CPU124.TLB:TLB_shootdowns 739662 ± 49% -82.9% 126387 ±117% interrupts.CPU125.CAL:Function_call_interrupts 968.50 ± 63% -84.4% 151.00 ± 27% interrupts.CPU125.NMI:Non-maskable_interrupts 968.50 ± 63% -84.4% 151.00 ± 27% interrupts.CPU125.PMI:Performance_monitoring_interrupts 66.25 ± 66% -72.8% 18.00 ± 97% interrupts.CPU125.RES:Rescheduling_interrupts 804430 ± 50% -83.3% 134653 ±118% interrupts.CPU125.TLB:TLB_shootdowns 501165 ± 14% +221.8% 1612759 ± 13% interrupts.CPU126.CAL:Function_call_interrupts 545091 ± 14% +217.3% 1729798 ± 13% interrupts.CPU126.TLB:TLB_shootdowns 303.75 ± 35% +1609.7% 5193 ± 40% interrupts.CPU127.NMI:Non-maskable_interrupts 303.75 ± 35% +1609.7% 5193 ± 40% interrupts.CPU127.PMI:Performance_monitoring_interrupts 81.00 ± 59% -78.1% 17.75 ± 68% interrupts.CPU13.RES:Rescheduling_interrupts 221283 ± 18% -61.8% 84423 ± 90% interrupts.CPU132.CAL:Function_call_interrupts 240847 ± 18% -62.5% 90210 ± 89% interrupts.CPU132.TLB:TLB_shootdowns 228853 ± 19% -64.6% 81121 ±115% interrupts.CPU135.CAL:Function_call_interrupts 514.00 ± 87% -74.7% 130.25 ± 15% interrupts.CPU135.NMI:Non-maskable_interrupts 514.00 ± 87% -74.7% 130.25 ± 15% interrupts.CPU135.PMI:Performance_monitoring_interrupts 250600 ± 19% -65.4% 86702 ±116% interrupts.CPU135.TLB:TLB_shootdowns 319279 ± 47% -92.1% 25144 ±150% interrupts.CPU136.CAL:Function_call_interrupts 343.25 ± 76% -63.6% 125.00 ± 25% interrupts.CPU136.NMI:Non-maskable_interrupts 343.25 ± 76% -63.6% 125.00 ± 25% interrupts.CPU136.PMI:Performance_monitoring_interrupts 349192 ± 47% -92.4% 26548 ±150% interrupts.CPU136.TLB:TLB_shootdowns 270185 ± 37% -78.4% 58300 ±172% interrupts.CPU139.CAL:Function_call_interrupts 378.00 ± 56% -71.6% 107.25 ± 27% interrupts.CPU139.NMI:Non-maskable_interrupts 378.00 ± 56% -71.6% 107.25 ± 27% interrupts.CPU139.PMI:Performance_monitoring_interrupts 293913 ± 37% -78.9% 62022 ±173% interrupts.CPU139.TLB:TLB_shootdowns 78.00 ± 63% -76.3% 18.50 ±111% interrupts.CPU14.RES:Rescheduling_interrupts 162067 ± 56% -81.2% 30462 ±165% interrupts.CPU140.CAL:Function_call_interrupts 175459 ± 56% -82.0% 31590 ±166% interrupts.CPU140.TLB:TLB_shootdowns 253.25 ± 32% -50.4% 125.50 ± 35% interrupts.CPU143.NMI:Non-maskable_interrupts 253.25 ± 32% -50.4% 125.50 ± 35% interrupts.CPU143.PMI:Performance_monitoring_interrupts 66.75 ± 72% -82.4% 11.75 ± 49% interrupts.CPU15.RES:Rescheduling_interrupts 508268 ± 63% -100.0% 79.25 ± 22% interrupts.CPU16.CAL:Function_call_interrupts 1082 ± 98% -93.1% 75.25 ± 11% interrupts.CPU16.NMI:Non-maskable_interrupts 1082 ± 98% -93.1% 75.25 ± 11% interrupts.CPU16.PMI:Performance_monitoring_interrupts 58.50 ± 46% -86.3% 8.00 ± 81% interrupts.CPU16.RES:Rescheduling_interrupts 549265 ± 63% -100.0% 4.00 ±133% interrupts.CPU16.TLB:TLB_shootdowns 52.25 ± 65% -66.5% 17.50 ± 57% interrupts.CPU17.RES:Rescheduling_interrupts 1258584 ± 15% +208.1% 3878023 ± 32% interrupts.CPU18.CAL:Function_call_interrupts 1369478 ± 15% +204.9% 4175783 ± 31% interrupts.CPU18.TLB:TLB_shootdowns 629511 ± 88% +241.1% 2147207 ± 76% interrupts.CPU2.CAL:Function_call_interrupts 678734 ± 89% +237.7% 2292112 ± 75% interrupts.CPU2.TLB:TLB_shootdowns 620.75 ± 72% -82.7% 107.50 ± 33% interrupts.CPU21.NMI:Non-maskable_interrupts 620.75 ± 72% -82.7% 107.50 ± 33% interrupts.CPU21.PMI:Performance_monitoring_interrupts 50.50 ± 71% -71.8% 14.25 ± 84% interrupts.CPU21.RES:Rescheduling_interrupts 63.75 ± 43% -78.8% 13.50 ± 99% interrupts.CPU22.RES:Rescheduling_interrupts 66.75 ± 50% -80.5% 13.00 ± 88% interrupts.CPU23.RES:Rescheduling_interrupts 54.25 ± 62% -77.0% 12.50 ± 17% interrupts.CPU25.RES:Rescheduling_interrupts 323.25 ± 93% -74.2% 83.25 ± 8% interrupts.CPU28.NMI:Non-maskable_interrupts 323.25 ± 93% -74.2% 83.25 ± 8% interrupts.CPU28.PMI:Performance_monitoring_interrupts 71.75 ± 37% -83.6% 11.75 ±109% interrupts.CPU29.RES:Rescheduling_interrupts 657.25 ± 54% -83.5% 108.50 ± 30% interrupts.CPU3.NMI:Non-maskable_interrupts 657.25 ± 54% -83.5% 108.50 ± 30% interrupts.CPU3.PMI:Performance_monitoring_interrupts 81.75 ± 86% -78.0% 18.00 ± 42% interrupts.CPU3.RES:Rescheduling_interrupts 846.25 ± 65% -88.7% 96.00 ± 15% interrupts.CPU30.NMI:Non-maskable_interrupts 846.25 ± 65% -88.7% 96.00 ± 15% interrupts.CPU30.PMI:Performance_monitoring_interrupts 54.50 ± 54% -85.3% 8.00 ± 49% interrupts.CPU30.RES:Rescheduling_interrupts 545049 ± 65% -98.4% 8859 ±171% interrupts.CPU31.CAL:Function_call_interrupts 612.00 ± 63% -84.8% 92.75 ± 14% interrupts.CPU31.NMI:Non-maskable_interrupts 612.00 ± 63% -84.8% 92.75 ± 14% interrupts.CPU31.PMI:Performance_monitoring_interrupts 49.75 ± 83% -78.4% 10.75 ± 54% interrupts.CPU31.RES:Rescheduling_interrupts 593689 ± 65% -98.4% 9350 ±173% interrupts.CPU31.TLB:TLB_shootdowns 67.75 ± 84% -85.6% 9.75 ± 42% interrupts.CPU33.RES:Rescheduling_interrupts 138.75 ± 65% -78.7% 29.50 ± 25% interrupts.CPU36.RES:Rescheduling_interrupts 133.75 ± 81% -84.1% 21.25 ± 49% interrupts.CPU37.RES:Rescheduling_interrupts 108.75 ± 54% -82.3% 19.25 ± 45% interrupts.CPU38.RES:Rescheduling_interrupts 112.50 ± 75% -93.8% 7.00 ± 42% interrupts.CPU40.RES:Rescheduling_interrupts 117.75 ± 69% -89.0% 13.00 ± 44% interrupts.CPU41.RES:Rescheduling_interrupts 889287 ± 63% -69.2% 273776 ±142% interrupts.CPU42.CAL:Function_call_interrupts 917.25 ± 65% -84.8% 139.25 ± 48% interrupts.CPU42.NMI:Non-maskable_interrupts 917.25 ± 65% -84.8% 139.25 ± 48% interrupts.CPU42.PMI:Performance_monitoring_interrupts 110.75 ± 72% -91.0% 10.00 ± 50% interrupts.CPU42.RES:Rescheduling_interrupts 970285 ± 63% -69.8% 292582 ±142% interrupts.CPU42.TLB:TLB_shootdowns 945929 ± 47% -54.3% 432248 ±109% interrupts.CPU43.CAL:Function_call_interrupts 1431 ± 84% -74.5% 365.50 ±134% interrupts.CPU43.NMI:Non-maskable_interrupts 1431 ± 84% -74.5% 365.50 ±134% interrupts.CPU43.PMI:Performance_monitoring_interrupts 91.25 ± 46% -93.7% 5.75 ± 44% interrupts.CPU43.RES:Rescheduling_interrupts 1030879 ± 47% -54.7% 467326 ±108% interrupts.CPU43.TLB:TLB_shootdowns 120.25 ± 47% -93.6% 7.75 ± 23% interrupts.CPU44.RES:Rescheduling_interrupts 869076 ± 61% -79.1% 182037 ±126% interrupts.CPU45.CAL:Function_call_interrupts 1007 ± 52% -88.8% 113.00 ± 61% interrupts.CPU45.NMI:Non-maskable_interrupts 1007 ± 52% -88.8% 113.00 ± 61% interrupts.CPU45.PMI:Performance_monitoring_interrupts 119.00 ± 65% -88.4% 13.75 ± 37% interrupts.CPU45.RES:Rescheduling_interrupts 945574 ± 61% -79.6% 192547 ±127% interrupts.CPU45.TLB:TLB_shootdowns 121.00 ± 36% -87.6% 15.00 ± 48% interrupts.CPU46.RES:Rescheduling_interrupts 96.50 ± 69% -85.8% 13.75 ±116% interrupts.CPU47.RES:Rescheduling_interrupts 91.25 ± 70% -91.2% 8.00 ± 31% interrupts.CPU48.RES:Rescheduling_interrupts 113.00 ± 73% -93.1% 7.75 ±108% interrupts.CPU49.RES:Rescheduling_interrupts 471.25 ± 60% -81.3% 88.25 ± 22% interrupts.CPU5.NMI:Non-maskable_interrupts 471.25 ± 60% -81.3% 88.25 ± 22% interrupts.CPU5.PMI:Performance_monitoring_interrupts 72.25 ± 54% -60.2% 28.75 ± 87% interrupts.CPU5.RES:Rescheduling_interrupts 959091 ± 62% -94.9% 48533 ±172% interrupts.CPU50.CAL:Function_call_interrupts 137.75 ± 80% -95.5% 6.25 ± 6% interrupts.CPU50.RES:Rescheduling_interrupts 1039830 ± 62% -95.0% 52192 ±173% interrupts.CPU50.TLB:TLB_shootdowns 1333 ± 57% -88.1% 158.50 ± 30% interrupts.CPU51.NMI:Non-maskable_interrupts 1333 ± 57% -88.1% 158.50 ± 30% interrupts.CPU51.PMI:Performance_monitoring_interrupts 121.25 ± 51% -89.5% 12.75 ± 45% interrupts.CPU51.RES:Rescheduling_interrupts 939728 ± 55% -87.8% 114957 ±173% interrupts.CPU52.CAL:Function_call_interrupts 103.25 ± 63% -94.2% 6.00 ± 39% interrupts.CPU52.RES:Rescheduling_interrupts 1020727 ± 55% -88.1% 121660 ±173% interrupts.CPU52.TLB:TLB_shootdowns 1039918 ± 36% -89.8% 106181 ±122% interrupts.CPU53.CAL:Function_call_interrupts 1284 ± 78% -91.1% 114.00 ± 52% interrupts.CPU53.NMI:Non-maskable_interrupts 1284 ± 78% -91.1% 114.00 ± 52% interrupts.CPU53.PMI:Performance_monitoring_interrupts 104.00 ± 68% -94.7% 5.50 ± 27% interrupts.CPU53.RES:Rescheduling_interrupts 1134197 ± 36% -90.1% 112045 ±121% interrupts.CPU53.TLB:TLB_shootdowns 1168628 ± 8% +182.2% 3297943 ± 17% interrupts.CPU54.CAL:Function_call_interrupts 963.25 ± 47% +350.1% 4336 ± 34% interrupts.CPU54.NMI:Non-maskable_interrupts 963.25 ± 47% +350.1% 4336 ± 34% interrupts.CPU54.PMI:Performance_monitoring_interrupts 80.75 ± 40% -48.3% 41.75 ± 25% interrupts.CPU54.RES:Rescheduling_interrupts 1275853 ± 8% +177.1% 3535200 ± 18% interrupts.CPU54.TLB:TLB_shootdowns 392.25 ± 34% +1015.9% 4377 ± 29% interrupts.CPU55.NMI:Non-maskable_interrupts 392.25 ± 34% +1015.9% 4377 ± 29% interrupts.CPU55.PMI:Performance_monitoring_interrupts 84.75 ± 68% -72.6% 23.25 ± 94% interrupts.CPU6.RES:Rescheduling_interrupts 707.00 ± 42% -77.1% 162.25 ± 30% interrupts.CPU61.NMI:Non-maskable_interrupts 707.00 ± 42% -77.1% 162.25 ± 30% interrupts.CPU61.PMI:Performance_monitoring_interrupts 405740 ± 55% -76.0% 97283 ±115% interrupts.CPU63.CAL:Function_call_interrupts 684.50 ± 70% -77.1% 157.00 ± 37% interrupts.CPU63.NMI:Non-maskable_interrupts 684.50 ± 70% -77.1% 157.00 ± 37% interrupts.CPU63.PMI:Performance_monitoring_interrupts 440306 ± 55% -76.6% 102844 ±115% interrupts.CPU63.TLB:TLB_shootdowns 315353 ± 24% -62.0% 119836 ±106% interrupts.CPU65.CAL:Function_call_interrupts 340082 ± 24% -62.6% 127245 ±106% interrupts.CPU65.TLB:TLB_shootdowns 356445 ± 33% -88.5% 41068 ± 70% interrupts.CPU66.CAL:Function_call_interrupts 387163 ± 33% -88.6% 44312 ± 71% interrupts.CPU66.TLB:TLB_shootdowns 390653 ± 33% -91.5% 33124 ±172% interrupts.CPU67.CAL:Function_call_interrupts 461.25 ± 50% -72.1% 128.75 ± 43% interrupts.CPU67.NMI:Non-maskable_interrupts 461.25 ± 50% -72.1% 128.75 ± 43% interrupts.CPU67.PMI:Performance_monitoring_interrupts 424248 ± 33% -91.6% 35682 ±173% interrupts.CPU67.TLB:TLB_shootdowns 362929 ± 57% -67.3% 118702 ±125% interrupts.CPU68.CAL:Function_call_interrupts 393487 ± 57% -68.0% 125761 ±125% interrupts.CPU68.TLB:TLB_shootdowns 396812 ± 29% -91.7% 32913 ±168% interrupts.CPU71.CAL:Function_call_interrupts 407.00 ± 48% -71.3% 117.00 ± 35% interrupts.CPU71.NMI:Non-maskable_interrupts 407.00 ± 48% -71.3% 117.00 ± 35% interrupts.CPU71.PMI:Performance_monitoring_interrupts 432737 ± 29% -91.1% 38455 ±169% interrupts.CPU71.TLB:TLB_shootdowns 617468 ± 68% +224.7% 2004642 ± 58% interrupts.CPU72.CAL:Function_call_interrupts 670964 ± 68% +220.4% 2149931 ± 58% interrupts.CPU72.TLB:TLB_shootdowns 62.50 ± 63% -72.0% 17.50 ± 37% interrupts.CPU75.RES:Rescheduling_interrupts 58.50 ± 67% -68.4% 18.50 ± 84% interrupts.CPU77.RES:Rescheduling_interrupts 584390 ± 73% -92.3% 45050 ±111% interrupts.CPU8.CAL:Function_call_interrupts 498.50 ± 43% -82.6% 86.50 ± 18% interrupts.CPU8.NMI:Non-maskable_interrupts 498.50 ± 43% -82.6% 86.50 ± 18% interrupts.CPU8.PMI:Performance_monitoring_interrupts 71.25 ± 62% -94.0% 4.25 ± 67% interrupts.CPU8.RES:Rescheduling_interrupts 634390 ± 73% -92.5% 47709 ±111% interrupts.CPU8.TLB:TLB_shootdowns 305172 ±104% -98.9% 3263 ±136% interrupts.CPU80.CAL:Function_call_interrupts 332295 ±104% -99.0% 3256 ±144% interrupts.CPU80.TLB:TLB_shootdowns 306939 ± 97% -100.0% 77.25 ± 23% interrupts.CPU82.CAL:Function_call_interrupts 45.50 ± 79% -80.2% 9.00 ± 53% interrupts.CPU82.RES:Rescheduling_interrupts 332644 ± 98% -100.0% 0.75 ±110% interrupts.CPU82.TLB:TLB_shootdowns 626.25 ± 82% -78.1% 137.25 ± 44% interrupts.CPU84.NMI:Non-maskable_interrupts 626.25 ± 82% -78.1% 137.25 ± 44% interrupts.CPU84.PMI:Performance_monitoring_interrupts 145.50 ± 2% -56.0% 64.00 ±100% interrupts.CPU87.175:PCI-MSI.512000-edge.ahci[0000:00:1f.2] 330258 ±115% -100.0% 114.75 ± 59% interrupts.CPU88.CAL:Function_call_interrupts 936.25 ±104% -85.1% 139.50 ± 58% interrupts.CPU88.NMI:Non-maskable_interrupts 936.25 ±104% -85.1% 139.50 ± 58% interrupts.CPU88.PMI:Performance_monitoring_interrupts 360088 ±116% -100.0% 41.50 ±164% interrupts.CPU88.TLB:TLB_shootdowns 84.75 ± 77% -69.0% 26.25 ± 97% interrupts.CPU9.RES:Rescheduling_interrupts 673363 ± 17% +226.0% 2195380 ± 32% interrupts.CPU90.CAL:Function_call_interrupts 732427 ± 17% +223.3% 2367615 ± 32% interrupts.CPU90.TLB:TLB_shootdowns 8947 ± 18% -71.3% 2570 ± 19% interrupts.RES:Rescheduling_interrupts vm-scalability.throughput 4.5e+06 +-----------------------------------------------------------------+ 4.4e+06 |-+ O O O | | O O O | 4.3e+06 |-O O O O | 4.2e+06 |-+ O O O O O O O O O O O O | 4.1e+06 |-+ O O O O O O | 4e+06 |-+ O | | | 3.9e+06 |-+ | 3.8e+06 |-+ | 3.7e+06 |-+ .+. | 3.6e+06 |.+.+.. .+. .+. .+ .+. .+ .+. .+. .+. + | | + .+ +. .+.+.+. .+. + + .+. + + .+. + + +| 3.5e+06 |-+ + +. + + + | 3.4e+06 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 9 months

1
0
0 / 0

Re: [selftest/bpf] 472547778d: WARNING:at_kernel/bpf/verifier.c:#resolve_pseudo_ldimm64

by Hao Luo

Sure, let me take a look at it. On Thu, Oct 29, 2020 at 1:43 PM Andrii Nakryiko <andrii.nakryiko(a)gmail.com> wrote: > > Hao, > > This seems to be coming from resolve_pseudo_ldimm64(), could you > please take a look? Thanks! > > -- Andrii > > On Thu, Oct 29, 2020 at 5:58 AM kernel test robot <lkp(a)intel.com> wrote: > > > > Greeting, > > > > FYI, we noticed the following commit (built with gcc-9): > > > > commit: 472547778de24e2764ab325268dd5b77e6923939 ("selftest/bpf: Fix profiler test using CO-RE relocation for enums") > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > in testcase: kernel-selftests > > version: kernel-selftests-x86_64-b5a583fb-1_20201015 > > with following parameters: > > > > group: kselftests-bpf > > ucode: 0xd6 > > > > test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. > > test-url: https://www.kernel.org/doc/Documentation/kselftest.txt > > > > > > on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory > > > > caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): > > > > > > +------------------------------------------+------------+------------+ > > | | 435ccfa894 | 472547778d | > > +------------------------------------------+------------+------------+ > > | boot_successes | 10 | 0 | > > | boot_failures | 0 | 10 | > > | BUG:using__this_cpu_read()in_preemptible | 0 | 10 | > > +------------------------------------------+------------+------------+ > > > > > > If you fix the issue, kindly add following tag > > Reported-by: kernel test robot <lkp(a)intel.com> > > > > > > kern :warn : [ 325.530080] WARNING: CPU: 3 PID: 20214 at kernel/bpf/verifier.c:9718 resolve_pseudo_ldimm64+0x6b8/0x8e0 > > kern :warn : [ 325.530812] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver netconsole snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio btrfs blake2b_generic xor zstd_compress raid6_pq libcrc32c intel_rapl_msr intel_rapl_common sd_mod t10_pi sg i915 btusb wmi_bmof intel_wmi_thunderbolt x86_pkg_temp_thermal intel_powerclamp snd_soc_skl snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi snd_soc_core snd_compress coretemp crct10dif_pclmul snd_hda_intel crc32_pclmul btrtl crc32c_intel btbcm ghash_clmulni_intel btintel snd_intel_dspcfg snd_hda_codec snd_hda_core iwlwifi aesni_intel crypto_simd ahci snd_hwdep bluetooth snd_pcm libahci cryptd cfg80211 snd_timer glue_helper ir_rc6_decoder pcspkr ecdh_generic libata mei_me snd ecc rc_rc6_mce mei soundcore i2c_i801 rfkill i2c_smbus wmi ipmi_devintf ite_cir ipmi_msghandler rc_core acpi_pad video intel_pmc_core ip_tables > > user :notice: [ 325.533201] # #3/p valid map access into an array with a constant OK > > kern :warn : [ 325.537207] CPU: 3 PID: 20214 Comm: test_verifier Not tainted 5.9.0-13427-g472547778de2 #1 > > > > kern :warn : [ 325.538572] Hardware name: Intel Corporation NUC7i7BNH/NUC7i7BNB, BIOS BNKBL357.86A.0067.2018.0814.1500 08/14/2018 > > kern :warn : [ 325.539414] RIP: 0010:resolve_pseudo_ldimm64+0x6b8/0x8e0 > > kern :warn : [ 325.539835] Code: ff ff c7 44 24 28 ea ff ff ff e9 49 fc ff ff 48 c7 c7 58 a8 5e 82 89 0c 24 48 89 54 24 08 c6 05 21 71 ff 01 01 e8 46 38 aa 00 <0f> 0b 8b 0c 24 48 8b 54 24 08 e9 5d fd ff ff 48 c7 c6 d8 a7 5e 82 > > kern :warn : [ 325.541298] RSP: 0018:ffffc90003b0fc50 EFLAGS: 00010282 > > kern :warn : [ 325.541732] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000 > > kern :warn : [ 325.542274] RDX: 0000000000000001 RSI: ffffffff811d234f RDI: ffffffff811d234f > > kern :warn : [ 325.542847] RBP: ffffc90003b0fcb8 R08: 0000000000000001 R09: 0000000000000001 > > kern :warn : [ 325.543389] R10: 0000000000000001 R11: 0000000000000001 R12: ffff888100c3a000 > > user :notice: [ 325.543850] # #4/u valid map access into an array with a register OK > > kern :warn : [ 325.543996] R13: ffff8882c025dc00 R14: 0000000000000025 R15: ffffc9000108d058 > > > > kern :warn : [ 325.545045] FS: 00007fd26e745740(0000) GS:ffff88887ed80000(0000) knlGS:0000000000000000 > > kern :warn : [ 325.545893] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > kern :warn : [ 325.546334] CR2: 000055a9904a3e30 CR3: 00000002c0730002 CR4: 00000000003706e0 > > kern :warn : [ 325.546900] Call Trace: > > kern :warn : [ 325.547114] bpf_check+0x907/0x17c0 > > kern :warn : [ 325.547396] ? find_held_lock+0x2b/0x80 > > kern :warn : [ 325.547781] bpf_prog_load+0x48c/0x8c0 > > kern :warn : [ 325.548133] ? find_held_lock+0x2b/0x80 > > kern :warn : [ 325.548450] __do_sys_bpf+0x93e/0x1a60 > > kern :warn : [ 325.548791] do_syscall_64+0x33/0x40 > > kern :warn : [ 325.549076] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > kern :warn : [ 325.549468] RIP: 0033:0x7fd26e83df59 > > kern :warn : [ 325.549771] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 07 6f 0c 00 f7 d8 64 89 01 48 > > kern :warn : [ 325.551230] RSP: 002b:00007ffc55a5fc08 EFLAGS: 00000246 ORIG_RAX: 0000000000000141 > > user :notice: [ 325.551650] # #4/p valid map access into an array with a register OK > > kern :warn : [ 325.551853] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fd26e83df59 > > > > kern :warn : [ 325.552308] RDX: 0000000000000078 RSI: 00007ffc55a5fc30 RDI: 0000000000000005 > > kern :warn : [ 325.552309] RBP: 00007ffc55a5fc30 R08: 0000000000000004 R09: 0000000000000000 > > kern :warn : [ 325.552310] R10: 0000000000000025 R11: 0000000000000246 R12: 0000000000000005 > > kern :warn : [ 325.552311] R13: 0000000000000000 R14: 00007ffc55a5fda0 R15: 00007ffc55a5fda0 > > kern :warn : [ 325.552325] CPU: 3 PID: 20214 Comm: test_verifier Not tainted 5.9.0-13427-g472547778de2 #1 > > user :notice: [ 325.554366] # #5/u valid map access into an array with a variable OK > > kern :warn : [ 325.554456] Hardware name: Intel Corporation NUC7i7BNH/NUC7i7BNB, BIOS BNKBL357.86A.0067.2018.0814.1500 08/14/2018 > > kern :warn : [ 325.554457] Call Trace: > > kern :warn : [ 325.554461] dump_stack+0x8d/0xb5 > > kern :warn : [ 325.554464] ? resolve_pseudo_ldimm64+0x6b8/0x8e0 > > > > kern :warn : [ 325.555551] __warn.cold+0x24/0x4b > > kern :warn : [ 325.555554] ? resolve_pseudo_ldimm64+0x6b8/0x8e0 > > kern :warn : [ 325.555558] report_bug+0xd1/0x100 > > user :notice: [ 325.557680] # #5/p valid map access into an array with a variable OK > > kern :warn : [ 325.557869] ? tick_nohz_tick_stopped+0x12/0x40 > > kern :warn : [ 325.557873] handle_bug+0x3a/0xa0 > > kern :warn : [ 325.557876] exc_invalid_op+0x14/0x80 > > > > kern :warn : [ 325.558465] asm_exc_invalid_op+0x12/0x20 > > kern :warn : [ 325.560172] RIP: 0010:resolve_pseudo_ldimm64+0x6b8/0x8e0 > > user :notice: [ 325.560516] # #6/u valid map access into an array with a signed variable OK > > kern :warn : [ 325.560553] Code: ff ff c7 44 24 28 ea ff ff ff e9 49 fc ff ff 48 c7 c7 58 a8 5e 82 89 0c 24 48 89 54 24 08 c6 05 21 71 ff 01 01 e8 46 38 aa 00 <0f> 0b 8b 0c 24 48 8b 54 24 08 e9 5d fd ff ff 48 c7 c6 d8 a7 5e 82 > > kern :warn : [ 325.560554] RSP: 0018:ffffc90003b0fc50 EFLAGS: 00010282 > > > > > > kern :warn : [ 325.562354] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000 > > kern :warn : [ 325.562355] RDX: 0000000000000001 RSI: ffffffff811d234f RDI: ffffffff811d234f > > kern :warn : [ 325.562356] RBP: ffffc90003b0fcb8 R08: 0000000000000001 R09: 0000000000000001 > > kern :warn : [ 325.562357] R10: 0000000000000001 R11: 0000000000000001 R12: ffff888100c3a000 > > kern :warn : [ 325.562358] R13: ffff8882c025dc00 R14: 0000000000000025 R15: ffffc9000108d058 > > kern :warn : [ 325.562369] ? wake_up_klogd+0x4f/0x80 > > kern :warn : [ 325.562371] ? wake_up_klogd+0x4f/0x80 > > kern :warn : [ 325.562391] bpf_check+0x907/0x17c0 > > user :notice: [ 325.564634] # #6/p valid map access into an array with a signed variable OK > > kern :warn : [ 325.565045] ? find_held_lock+0x2b/0x80 > > kern :warn : [ 325.565057] bpf_prog_load+0x48c/0x8c0 > > > > kern :warn : [ 325.565875] ? find_held_lock+0x2b/0x80 > > kern :warn : [ 325.565889] __do_sys_bpf+0x93e/0x1a60 > > kern :warn : [ 325.565911] do_syscall_64+0x33/0x40 > > user :notice: [ 325.567592] # #7/u invalid map access into an array with a constant OK > > kern :warn : [ 325.567640] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > kern :warn : [ 325.568193] RIP: 0033:0x7fd26e83df59 > > kern :warn : [ 325.569659] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 07 6f 0c 00 f7 d8 64 89 01 48 > > user :notice: [ 325.569825] # #7/p invalid map access into an array with a constant OK > > kern :warn : [ 325.570965] RSP: 002b:00007ffc55a5fc08 EFLAGS: 00000246 ORIG_RAX: 0000000000000141 > > kern :warn : [ 325.570967] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fd26e83df59 > > kern :warn : [ 325.570968] RDX: 0000000000000078 RSI: 00007ffc55a5fc30 RDI: 0000000000000005 > > kern :warn : [ 325.570969] RBP: 00007ffc55a5fc30 R08: 0000000000000004 R09: 0000000000000000 > > kern :warn : [ 325.570970] R10: 0000000000000025 R11: 0000000000000246 R12: 0000000000000005 > > kern :warn : [ 325.570972] R13: 0000000000000000 R14: 00007ffc55a5fda0 R15: 00007ffc55a5fda0 > > > > kern :warn : [ 325.573227] irq event stamp: 93003 > > kern :warn : [ 325.575115] hardirqs last enabled at (93103): [<ffffffff811d30b3>] console_unlock+0x4d3/0x5c0 > > user :notice: [ 325.575571] # #8/u invalid map access into an array with a register OK > > > > kern :warn : [ 325.575873] hardirqs last disabled at (93130): [<ffffffff81ddfa21>] __schedule+0x6e1/0xaa0 > > kern :warn : [ 325.577041] softirqs last enabled at (93170): [<ffffffff8200034e>] __do_softirq+0x34e/0x49c > > kern :warn : [ 325.577666] softirqs last disabled at (93187): [<ffffffff81e010f2>] asm_call_irq_on_stack+0x12/0x20 > > kern :warn : [ 325.578385] ---[ end trace 54e7a2ba0948b528 ]--- > > > > > > To reproduce: > > > > git clone https://github.com/intel/lkp-tests.git > > cd lkp-tests > > bin/lkp install job.yaml # job file is attached in this email > > bin/lkp run job.yaml > > > > > > > > Thanks, > > lkp > >

1 year, 9 months

1
1
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP November 2020