[btrfs] 8d47a0d8f7: fio.write_bw_MBps -28.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -28.6% regression of fio.write_bw_MBps due to commit:
commit: 8d47a0d8f7947422dd359ac8e462687f81a7a137 ("btrfs: Do mandatory tree block check before submitting bio")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fio-basic
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
with following parameters:
disk: 2pmem
fs: btrfs
runtime: 200s
nr_task: 50%
time_based: tb
rw: randwrite
bs: 4k
ioengine: libaio
test_size: 100G
cpufreq_governor: performance
ucode: 0x3d
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
4k/gcc-7/performance/2pmem/btrfs/libaio/x86_64-rhel-7.6/50%/debian-x86_64-2018-04-03.cgz/200s/randwrite/lkp-hsw-ep2/100G/fio-basic/tb/0x3d
commit:
ff2ac107fa ("btrfs: tree-checker: Remove comprehensive root owner check")
8d47a0d8f7 ("btrfs: Do mandatory tree block check before submitting bio")
ff2ac107fae2440b 8d47a0d8f7947422dd359ac8e46
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.BUG:unable_to_handle_kernel
:4 25% 1:4 dmesg.Kernel_panic-not_syncing:Fatal_exception_in_interrupt
:4 25% 1:4 dmesg.Oops:#[##]
:4 25% 1:4 dmesg.RIP:cpuidle_enter_state
:4 25% 1:4 dmesg.RIP:native_write_msr
:4 25% 1:4 dmesg.RIP:perf_prepare_sample
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
0.01 ± 21% +0.3 0.28 ± 20% fio.latency_1000ms%
4.14 ± 13% +1.8 5.95 ± 9% fio.latency_10ms%
8.73 ± 15% +3.0 11.68 ± 8% fio.latency_20ms%
1.91 ± 5% -0.4 1.52 ± 3% fio.latency_250ms%
66.81 -5.9 60.90 ± 2% fio.latency_4ms%
0.18 ± 15% +0.1 0.29 ± 16% fio.latency_750us%
1.017e+08 -28.5% 72701525 fio.time.file_system_outputs
109.75 -26.8% 80.33 ± 2% fio.time.percent_of_cpu_this_job_got
197.97 -26.4% 145.79 ± 2% fio.time.system_time
38.66 ± 4% -24.0% 29.39 ± 3% fio.time.user_time
12388015 -28.5% 8858878 fio.time.voluntary_context_switches
12711059 -28.5% 9083954 fio.workload
247.91 -28.6% 176.96 fio.write_bw_MBps
309248 ± 2% +71.7% 531114 ± 4% fio.write_clat_99%_us
17574 +40.0% 24609 fio.write_clat_mean_us
57989 ± 4% +82.2% 105661 fio.write_clat_stddev
63463 -28.6% 45302 fio.write_iops
564.61 +40.2% 791.54 fio.write_slat_mean_us
9375 ± 7% +96.3% 18406 fio.write_slat_stddev
21.25 +8.5% 23.06 ± 9% boot-time.dhcp
93.32 +1.4% 94.58 iostat.cpu.idle
6.43 -18.6% 5.23 ± 2% iostat.cpu.system
6.48 -1.2 5.27 ± 2% mpstat.cpu.all.sys%
0.24 ± 5% -0.1 0.18 ± 2% mpstat.cpu.all.usr%
93.00 +1.1% 94.00 vmstat.cpu.id
591200 -30.0% 413921 vmstat.io.bo
31653776 ± 2% -10.9% 28196588 ± 2% vmstat.memory.cache
11927341 ± 7% +29.8% 15486805 ± 4% vmstat.memory.free
5.00 -26.7% 3.67 ± 12% vmstat.procs.r
635910 ± 2% -27.6% 460384 vmstat.system.cs
7822384 ± 2% -25.0% 5870498 numa-numastat.node0.local_node
7839861 ± 2% -24.8% 5893785 numa-numastat.node0.numa_hit
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node0.numa_miss
242465 ± 44% -84.3% 37964 ± 18% numa-numastat.node0.other_node
7762623 ± 12% -26.8% 5681358 ± 6% numa-numastat.node1.local_node
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node1.numa_foreign
7768618 ± 12% -26.9% 5681573 ± 6% numa-numastat.node1.numa_hit
9.429e+08 ± 10% -35.4% 6.094e+08 cpuidle.C1.time
53504154 ± 6% -31.8% 36504731 cpuidle.C1.usage
1.093e+09 ± 22% -50.3% 5.435e+08 ± 3% cpuidle.C1E.time
16541925 ± 13% -45.0% 9103775 cpuidle.C1E.usage
5.366e+09 ± 36% +71.7% 9.214e+09 ± 18% cpuidle.C3.time
16163664 ± 33% +57.0% 25379738 ± 5% cpuidle.C3.usage
9697055 ± 16% -35.8% 6225168 ± 19% cpuidle.C6.usage
19313595 ± 17% -16.3% 16166123 cpuidle.POLL.time
10291838 ± 9% -26.8% 7537045 ± 7% meminfo.Active
10034782 ± 9% -27.4% 7280710 ± 7% meminfo.Active(file)
31524905 ± 2% -10.9% 28082249 ± 2% meminfo.Cached
115157 ± 4% +61.1% 185560 ± 3% meminfo.CmaFree
6430720 ± 8% +19.1% 7660885 ± 6% meminfo.DirectMap2M
534230 ± 5% -12.8% 465932 ± 6% meminfo.Dirty
11737322 ± 7% +30.2% 15281189 ± 4% meminfo.MemFree
37604344 ± 2% -9.4% 34060483 meminfo.Memused
806.50 ± 37% +137.4% 1914 ± 8% meminfo.Writeback
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.active_objs
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.num_objs
15147 ± 5% -8.3% 13897 ± 2% slabinfo.blkdev_ioc.active_objs
17839 ± 6% -11.7% 15745 ± 2% slabinfo.blkdev_ioc.num_objs
12178 ± 4% -7.4% 11275 ± 2% slabinfo.kmalloc-512.num_objs
7794 ± 3% +25.4% 9774 slabinfo.proc_inode_cache.active_objs
8017 ± 2% +23.4% 9895 slabinfo.proc_inode_cache.num_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.active_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.num_objs
263.75 ± 2% -15.1% 224.00 ± 2% turbostat.Avg_MHz
9.46 ± 2% -1.4 8.07 ± 2% turbostat.Busy%
53502248 ± 6% -31.8% 36503898 turbostat.C1
5.86 ± 8% -2.0 3.82 turbostat.C1%
16540396 ± 13% -45.0% 9102733 turbostat.C1E
6.80 ± 22% -3.4 3.40 ± 4% turbostat.C1E%
16163332 ± 33% +57.0% 25379351 ± 5% turbostat.C3
33.55 ± 36% +24.1 57.61 ± 18% turbostat.C3%
9689445 ± 16% -35.8% 6221033 ± 19% turbostat.C6
18.04 ± 26% +49.1% 26.90 ± 9% turbostat.CPU%c3
53.25 ± 3% +10.8% 59.00 ± 4% turbostat.CoreTmp
4.67 ± 14% +62.7% 7.59 ± 26% turbostat.Pkg%pc2
56.50 +9.7% 62.00 ± 2% turbostat.PkgTmp
11.84 -4.8% 11.28 turbostat.RAMWatt
7232478 ± 10% -37.2% 4543037 ± 22% numa-meminfo.node0.Active
7072564 ± 10% -37.6% 4410135 ± 23% numa-meminfo.node0.Active(file)
308167 ± 6% -24.7% 232152 ± 10% numa-meminfo.node0.Dirty
8784028 ± 13% +21.7% 10686327 ± 2% numa-meminfo.node0.Inactive
8669548 ± 12% +20.7% 10465444 ± 2% numa-meminfo.node0.Inactive(file)
572.50 ± 22% +115.9% 1236 ± 25% numa-meminfo.node0.Writeback
15098140 ± 3% -17.7% 12425670 ± 3% numa-meminfo.node1.FilePages
11599549 ± 7% -22.3% 9018051 ± 4% numa-meminfo.node1.Inactive
105067 ± 92% -99.3% 725.33 ± 51% numa-meminfo.node1.Inactive(anon)
11494482 ± 7% -21.6% 9017325 ± 4% numa-meminfo.node1.Inactive(file)
5504 ± 9% -12.5% 4819 ± 2% numa-meminfo.node1.KernelStack
6709938 ± 11% +45.8% 9781880 ± 5% numa-meminfo.node1.MemFree
18029136 ± 4% -17.0% 14957200 ± 3% numa-meminfo.node1.MemUsed
4427 ± 69% -77.5% 996.00 ± 8% numa-meminfo.node1.PageTables
857660 ± 32% -37.2% 538646 ± 8% numa-meminfo.node1.SUnreclaim
109232 ± 88% -96.1% 4274 ± 10% numa-meminfo.node1.Shmem
986740 ± 35% -40.0% 592452 ± 10% numa-meminfo.node1.Slab
575.21 ± 15% +30.1% 748.58 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max
100.45 ± 15% +28.6% 129.19 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.stddev
830.19 ± 9% +15.7% 960.25 ± 6% sched_debug.cfs_rq:/.util_avg.max
29.97 ± 16% -25.2% 22.43 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.avg
591.17 ± 13% -22.2% 459.83 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max
107.52 ± 13% -23.0% 82.84 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock_task.stddev
24.51 ± 7% +21.5% 29.77 ± 8% sched_debug.cpu.cpu_load[0].avg
541.29 ± 15% +38.3% 748.58 ± 2% sched_debug.cpu.cpu_load[0].max
93.97 ± 7% +27.8% 120.09 ± 2% sched_debug.cpu.cpu_load[0].stddev
519.08 ± 6% +40.5% 729.42 ± 4% sched_debug.cpu.cpu_load[1].max
87.45 ± 5% +28.5% 112.39 ± 4% sched_debug.cpu.cpu_load[1].stddev
480.83 ± 7% +39.5% 670.92 ± 13% sched_debug.cpu.cpu_load[2].max
80.92 ± 5% +25.7% 101.75 ± 8% sched_debug.cpu.cpu_load[2].stddev
408.83 ± 4% +54.0% 629.58 ± 19% sched_debug.cpu.cpu_load[3].max
71.62 ± 2% +30.7% 93.61 ± 11% sched_debug.cpu.cpu_load[3].stddev
381.50 ± 8% +66.7% 636.00 ± 18% sched_debug.cpu.cpu_load[4].max
64.40 ± 6% +39.7% 89.96 ± 12% sched_debug.cpu.cpu_load[4].stddev
106.75 ± 20% -88.4% 12.33 ± 10% proc-vmstat.kswapd_high_wmark_hit_quickly
2508660 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_active_file
33315360 -30.4% 23192301 proc-vmstat.nr_dirtied
133447 ± 5% -12.6% 116580 ± 6% proc-vmstat.nr_dirty
7881043 ± 2% -10.9% 7019533 ± 2% proc-vmstat.nr_file_pages
28790 ± 4% +61.1% 46392 ± 3% proc-vmstat.nr_free_cma
2934503 ± 7% +30.2% 3821369 ± 4% proc-vmstat.nr_free_pages
5044168 -3.4% 4872965 proc-vmstat.nr_inactive_file
430282 -6.0% 404589 proc-vmstat.nr_slab_unreclaimable
197.75 ± 38% +136.5% 467.67 ± 10% proc-vmstat.nr_writeback
33243480 -30.4% 23143756 proc-vmstat.nr_written
2508665 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_zone_active_file
5044399 -3.4% 4873026 proc-vmstat.nr_zone_inactive_file
134972 ± 5% -12.4% 118262 ± 5% proc-vmstat.nr_zone_write_pending
15632301 ± 5% -25.8% 11598535 ± 3% proc-vmstat.numa_hit
15608826 ± 5% -25.8% 11575029 ± 3% proc-vmstat.numa_local
2300530 ± 10% -31.1% 1585363 ± 10% proc-vmstat.pgactivate
16955745 -26.3% 12499611 ± 2% proc-vmstat.pgalloc_normal
17488000 -25.9% 12959812 ± 2% proc-vmstat.pgfree
1.33e+08 -30.4% 92575440 proc-vmstat.pgpgout
5728177 ± 6% -78.5% 1232433 ± 25% proc-vmstat.pgscan_kswapd
5727967 ± 6% -78.5% 1232244 ± 25% proc-vmstat.pgsteal_kswapd
32760 -7.6% 30285 proc-vmstat.slabs_scanned
1768211 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_active_file
10410188 ± 7% -32.5% 7023939 ± 7% numa-vmstat.node0.nr_dirtied
76995 ± 6% -24.6% 58035 ± 10% numa-vmstat.node0.nr_dirty
2167457 ± 12% +20.7% 2616147 ± 2% numa-vmstat.node0.nr_inactive_file
142.00 ± 24% +120.4% 313.00 ± 24% numa-vmstat.node0.nr_writeback
10308758 ± 7% -32.6% 6949497 ± 7% numa-vmstat.node0.nr_written
1768216 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_zone_active_file
2167563 ± 12% +20.7% 2616189 ± 2% numa-vmstat.node0.nr_zone_inactive_file
77786 ± 7% -24.3% 58899 ± 10% numa-vmstat.node0.nr_zone_write_pending
5889240 ± 3% -14.3% 5046874 ± 3% numa-vmstat.node0.numa_hit
5837851 ± 4% -14.0% 5023120 ± 3% numa-vmstat.node0.numa_local
68376 ± 48% -96.9% 2149 ± 46% numa-vmstat.node0.numa_miss
119766 ± 42% -78.4% 25904 ± 3% numa-vmstat.node0.numa_other
30.50 ± 38% -100.0% 0.00 numa-vmstat.node0.workingset_nodes
6829126 ± 6% -22.8% 5269898 ± 10% numa-vmstat.node1.nr_dirtied
3774645 ± 3% -17.7% 3106081 ± 3% numa-vmstat.node1.nr_file_pages
28796 ± 4% +61.1% 46405 ± 3% numa-vmstat.node1.nr_free_cma
1677332 ± 11% +45.8% 2445778 ± 5% numa-vmstat.node1.nr_free_pages
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_inactive_anon
2873702 ± 7% -21.6% 2254116 ± 4% numa-vmstat.node1.nr_inactive_file
5506 ± 9% -12.5% 4820 ± 2% numa-vmstat.node1.nr_kernel_stack
29031 ± 85% -88.2% 3436 numa-vmstat.node1.nr_mapped
1106 ± 69% -77.5% 249.00 ± 8% numa-vmstat.node1.nr_page_table_pages
27308 ± 88% -96.1% 1068 ± 10% numa-vmstat.node1.nr_shmem
214418 ± 32% -37.2% 134663 ± 8% numa-vmstat.node1.nr_slab_unreclaimable
75.00 ± 35% +145.3% 184.00 ± 18% numa-vmstat.node1.nr_writeback
6761224 ± 6% -23.1% 5202586 ± 10% numa-vmstat.node1.nr_written
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_zone_inactive_anon
2873810 ± 7% -21.6% 2254135 ± 4% numa-vmstat.node1.nr_zone_inactive_file
68381 ± 48% -96.9% 2152 ± 46% numa-vmstat.node1.numa_foreign
5035283 ± 15% -26.9% 3679318 ± 7% numa-vmstat.node1.numa_hit
4905249 ± 15% -28.2% 3522229 ± 8% numa-vmstat.node1.numa_local
18.43 ± 6% -28.3% 13.21 ± 3% perf-stat.i.MPKI
2.205e+09 ± 3% +10.5% 2.437e+09 perf-stat.i.branch-instructions
1.83 ± 5% -0.6 1.24 ± 7% perf-stat.i.branch-miss-rate%
37019551 ± 3% -29.4% 26140844 ± 5% perf-stat.i.branch-misses
28932020 ± 6% -31.1% 19933819 ± 3% perf-stat.i.cache-misses
2.03e+08 ± 4% -24.8% 1.526e+08 perf-stat.i.cache-references
645248 ± 2% -27.4% 468301 perf-stat.i.context-switches
1.97 ± 7% -22.1% 1.53 ± 2% perf-stat.i.cpi
1.983e+10 ± 3% -15.1% 1.684e+10 perf-stat.i.cpu-cycles
1.03 ± 5% -0.4 0.64 ± 10% perf-stat.i.dTLB-load-miss-rate%
31879502 ± 6% -37.9% 19807598 ± 9% perf-stat.i.dTLB-load-misses
3.201e+09 ± 3% +18.8% 3.804e+09 perf-stat.i.dTLB-loads
0.13 ± 7% -0.0 0.09 ± 10% perf-stat.i.dTLB-store-miss-rate%
2516181 ± 10% -28.3% 1805272 ± 10% perf-stat.i.dTLB-store-misses
2.143e+09 ± 3% +19.8% 2.568e+09 ± 2% perf-stat.i.dTLB-stores
13781489 ± 3% -28.4% 9866239 ± 3% perf-stat.i.iTLB-loads
1.164e+10 ± 3% +21.6% 1.415e+10 perf-stat.i.instructions
8435 ± 17% +42.5% 12023 ± 14% perf-stat.i.instructions-per-iTLB-miss
0.57 ± 3% +43.6% 0.82 perf-stat.i.ipc
16050361 ± 9% -30.7% 11127721 ± 3% perf-stat.i.node-load-misses
8120610 ± 9% -31.3% 5580996 perf-stat.i.node-loads
2591649 ± 13% -31.1% 1785932 ± 6% perf-stat.i.node-store-misses
1809775 ± 12% -31.2% 1244360 ± 2% perf-stat.i.node-stores
17.45 ± 4% -38.2% 10.79 perf-stat.overall.MPKI
1.68 ± 3% -0.6 1.07 ± 5% perf-stat.overall.branch-miss-rate%
1.70 ± 3% -30.2% 1.19 perf-stat.overall.cpi
689.69 ± 9% +22.6% 845.48 ± 2% perf-stat.overall.cycles-between-cache-misses
0.99 ± 5% -0.5 0.52 ± 10% perf-stat.overall.dTLB-load-miss-rate%
0.12 ± 8% -0.0 0.07 ± 10% perf-stat.overall.dTLB-store-miss-rate%
10.81 ± 14% +2.3 13.09 ± 10% perf-stat.overall.iTLB-load-miss-rate%
7119 ± 14% +35.9% 9679 ± 13% perf-stat.overall.instructions-per-iTLB-miss
0.59 ± 3% +43.1% 0.84 perf-stat.overall.ipc
202912 ± 2% +68.8% 342615 ± 3% perf-stat.overall.path-length
2.195e+09 ± 3% +10.5% 2.425e+09 perf-stat.ps.branch-instructions
36847010 ± 3% -29.4% 26015686 ± 5% perf-stat.ps.branch-misses
28799092 ± 6% -31.1% 19837306 ± 3% perf-stat.ps.cache-misses
2.021e+08 ± 4% -24.8% 1.519e+08 perf-stat.ps.cache-references
642266 ± 2% -27.4% 465968 perf-stat.ps.context-switches
1.974e+10 ± 3% -15.1% 1.676e+10 perf-stat.ps.cpu-cycles
31729472 ± 6% -37.9% 19715594 ± 9% perf-stat.ps.dTLB-load-misses
3.186e+09 ± 3% +18.8% 3.785e+09 perf-stat.ps.dTLB-loads
2504443 ± 10% -28.2% 1797154 ± 10% perf-stat.ps.dTLB-store-misses
2.133e+09 ± 3% +19.8% 2.556e+09 ± 2% perf-stat.ps.dTLB-stores
13717699 ± 3% -28.4% 9816979 ± 3% perf-stat.ps.iTLB-loads
1.159e+10 ± 3% +21.5% 1.408e+10 perf-stat.ps.instructions
15979371 ± 9% -30.7% 11073468 ± 3% perf-stat.ps.node-load-misses
8081205 ± 9% -31.3% 5554114 perf-stat.ps.node-loads
2580198 ± 13% -31.1% 1777169 ± 6% perf-stat.ps.node-store-misses
1801590 ± 12% -31.3% 1238321 ± 2% perf-stat.ps.node-stores
2.579e+12 +20.6% 3.111e+12 ± 2% perf-stat.total.instructions
36149 ± 4% -26.2% 26686 ± 5% softirqs.CPU0.RCU
37255 ± 3% -26.8% 27256 ± 2% softirqs.CPU1.RCU
37834 ± 5% -26.5% 27801 ± 9% softirqs.CPU10.RCU
39075 ± 3% -24.5% 29494 ± 2% softirqs.CPU11.RCU
38137 ± 6% -23.1% 29340 ± 4% softirqs.CPU12.RCU
37480 ± 3% -20.5% 29797 ± 2% softirqs.CPU13.RCU
37690 ± 5% -28.7% 26861 ± 4% softirqs.CPU14.RCU
33863 ± 9% -29.5% 23885 ± 5% softirqs.CPU15.RCU
34306 ± 9% -31.1% 23641 ± 12% softirqs.CPU16.RCU
34374 ± 9% -29.0% 24402 ± 7% softirqs.CPU17.RCU
44013 ± 9% -31.7% 30059 softirqs.CPU18.RCU
43114 ± 8% -29.9% 30219 softirqs.CPU19.RCU
36276 ± 8% -23.3% 27825 ± 5% softirqs.CPU2.RCU
67049 ± 7% +39.1% 93267 ± 18% softirqs.CPU2.TIMER
39721 ± 13% -31.4% 27233 ± 3% softirqs.CPU20.RCU
42340 ± 5% -28.6% 30215 ± 6% softirqs.CPU21.RCU
40896 ± 7% -30.3% 28506 ± 2% softirqs.CPU22.RCU
42422 ± 7% -33.2% 28357 ± 2% softirqs.CPU23.RCU
42330 ± 9% -33.5% 28150 ± 5% softirqs.CPU24.RCU
41406 ± 9% -31.4% 28398 softirqs.CPU25.RCU
39754 ± 5% -29.4% 28063 softirqs.CPU26.RCU
40207 ± 6% -27.9% 28995 ± 12% softirqs.CPU27.RCU
40351 ± 9% -29.4% 28487 ± 3% softirqs.CPU28.RCU
41880 ± 5% -29.6% 29472 softirqs.CPU29.RCU
36293 ± 4% -17.7% 29864 ± 7% softirqs.CPU3.RCU
39341 ± 9% -34.3% 25847 softirqs.CPU30.RCU
39687 ± 9% -33.5% 26400 ± 2% softirqs.CPU31.RCU
38062 ± 14% -39.9% 22878 ± 10% softirqs.CPU32.RCU
39769 ± 13% -33.8% 26314 ± 5% softirqs.CPU33.RCU
39504 ± 10% -37.2% 24824 ± 6% softirqs.CPU34.RCU
39591 ± 11% -38.6% 24300 ± 5% softirqs.CPU35.RCU
40398 ± 8% -27.0% 29488 ± 12% softirqs.CPU36.RCU
39464 ± 12% -25.3% 29478 ± 10% softirqs.CPU37.RCU
41177 ± 9% -34.0% 27190 ± 19% softirqs.CPU38.RCU
38734 ± 13% -24.8% 29145 ± 5% softirqs.CPU39.RCU
36804 ± 5% -23.0% 28354 softirqs.CPU4.RCU
40421 ± 12% -28.8% 28776 ± 6% softirqs.CPU40.RCU
40371 ± 11% -31.1% 27834 ± 8% softirqs.CPU41.RCU
40633 ± 10% -29.9% 28469 ± 10% softirqs.CPU42.RCU
41018 ± 9% -33.6% 27252 ± 6% softirqs.CPU43.RCU
38181 ± 12% -27.0% 27856 ± 10% softirqs.CPU44.RCU
39605 ± 12% -32.3% 26803 ± 12% softirqs.CPU45.RCU
42686 ± 11% -29.7% 30016 ± 9% softirqs.CPU47.RCU
40433 ± 9% -31.8% 27586 ± 11% softirqs.CPU48.RCU
38371 ± 12% -23.0% 29528 ± 7% softirqs.CPU49.RCU
37172 ± 2% -25.9% 27562 ± 3% softirqs.CPU5.RCU
39660 ± 12% -27.5% 28752 ± 11% softirqs.CPU50.RCU
40375 ± 10% -26.7% 29582 ± 3% softirqs.CPU51.RCU
40568 ± 9% -28.8% 28886 ± 6% softirqs.CPU52.RCU
39967 ± 10% -27.3% 29073 ± 11% softirqs.CPU53.RCU
40874 ± 11% -37.1% 25725 ± 6% softirqs.CPU54.RCU
41227 ± 14% -37.9% 25606 ± 6% softirqs.CPU55.RCU
38856 ± 13% -34.7% 25380 ± 8% softirqs.CPU56.RCU
39618 ± 13% -39.1% 24119 ± 4% softirqs.CPU57.RCU
39412 ± 12% -36.6% 24970 ± 3% softirqs.CPU58.RCU
41572 ± 14% -41.3% 24399 ± 10% softirqs.CPU59.RCU
36927 ± 3% -24.4% 27923 ± 22% softirqs.CPU6.RCU
37057 ± 10% -31.8% 25261 ± 2% softirqs.CPU60.RCU
36584 ± 8% -27.8% 26419 ± 2% softirqs.CPU61.RCU
35859 ± 4% -28.2% 25729 ± 5% softirqs.CPU62.RCU
35849 ± 7% -33.2% 23933 ± 5% softirqs.CPU63.RCU
35379 ± 6% -26.4% 26041 ± 4% softirqs.CPU64.RCU
36908 ± 6% -26.5% 27110 ± 2% softirqs.CPU65.RCU
37595 ± 3% -26.1% 27783 ± 4% softirqs.CPU66.RCU
36327 ± 2% -27.0% 26533 ± 2% softirqs.CPU67.RCU
35136 ± 7% -31.7% 23998 ± 9% softirqs.CPU68.RCU
35638 ± 4% -28.6% 25435 ± 4% softirqs.CPU69.RCU
37268 ± 6% -25.4% 27786 ± 4% softirqs.CPU7.RCU
36693 ± 4% -29.2% 25986 ± 5% softirqs.CPU70.RCU
36859 ± 3% -28.8% 26260 ± 4% softirqs.CPU71.RCU
37539 ± 7% -27.7% 27144 ± 4% softirqs.CPU8.RCU
37584 ± 4% -26.0% 27802 ± 15% softirqs.CPU9.RCU
2795313 ± 7% -29.7% 1964363 ± 4% softirqs.RCU
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.41:PCI-MSI.1572870-edge.eth0-TxRx-6
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.42:PCI-MSI.1572871-edge.eth0-TxRx-7
150.50 ± 43% -28.0% 108.33 interrupts.84:PCI-MSI.1572913-edge.eth0-TxRx-49
10393 ± 10% -26.4% 7653 ± 10% interrupts.CPU1.RES:Rescheduling_interrupts
12518 ± 15% -31.0% 8642 ± 21% interrupts.CPU10.RES:Rescheduling_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.NMI:Non-maskable_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.PMI:Performance_monitoring_interrupts
13068 ± 18% -30.6% 9075 ± 23% interrupts.CPU11.RES:Rescheduling_interrupts
11949 ± 11% -49.8% 6001 ± 29% interrupts.CPU12.RES:Rescheduling_interrupts
13711 ± 8% -19.4% 11052 ± 24% interrupts.CPU13.RES:Rescheduling_interrupts
13361 ± 6% -38.8% 8170 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts
7628 ± 12% -29.2% 5401 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
7909 ± 25% -30.4% 5501 ± 37% interrupts.CPU19.RES:Rescheduling_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
2116 +23.2% 2606 ± 10% interrupts.CPU20.CAL:Function_call_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.NMI:Non-maskable_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.PMI:Performance_monitoring_interrupts
2096 ± 4% +18.0% 2474 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
11313 ± 20% -56.4% 4935 ± 23% interrupts.CPU21.RES:Rescheduling_interrupts
2111 ± 3% +20.1% 2534 ± 11% interrupts.CPU22.CAL:Function_call_interrupts
2109 ± 4% +22.6% 2585 ± 10% interrupts.CPU23.CAL:Function_call_interrupts
10305 ± 16% -40.7% 6115 ± 17% interrupts.CPU23.RES:Rescheduling_interrupts
2128 ± 2% +17.8% 2507 ± 10% interrupts.CPU24.CAL:Function_call_interrupts
12271 ± 20% -39.0% 7488 ± 29% interrupts.CPU24.RES:Rescheduling_interrupts
2123 +19.7% 2541 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
9780 ± 21% -32.9% 6563 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.NMI:Non-maskable_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.PMI:Performance_monitoring_interrupts
11163 ± 28% -49.8% 5605 ± 13% interrupts.CPU27.RES:Rescheduling_interrupts
11071 ± 12% -56.3% 4835 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
10599 ± 10% -38.7% 6501 ± 16% interrupts.CPU31.RES:Rescheduling_interrupts
2213 ± 4% +14.2% 2527 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
8862 ± 16% -39.6% 5354 ± 28% interrupts.CPU32.RES:Rescheduling_interrupts
2156 ± 6% +20.5% 2599 ± 9% interrupts.CPU33.CAL:Function_call_interrupts
2169 ± 5% +19.9% 2601 ± 9% interrupts.CPU34.CAL:Function_call_interrupts
10483 ± 18% -48.8% 5371 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts
2049 +24.4% 2549 ± 8% interrupts.CPU35.CAL:Function_call_interrupts
10929 ± 15% -49.0% 5578 ± 14% interrupts.CPU35.RES:Rescheduling_interrupts
1993 ± 4% +28.5% 2561 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.NMI:Non-maskable_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.PMI:Performance_monitoring_interrupts
1987 ± 4% +30.5% 2594 ± 13% interrupts.CPU37.CAL:Function_call_interrupts
1863 ± 10% +35.7% 2529 ± 14% interrupts.CPU38.CAL:Function_call_interrupts
1984 ± 6% +26.5% 2509 ± 12% interrupts.CPU41.CAL:Function_call_interrupts
12830 ± 30% -26.4% 9445 ± 9% interrupts.CPU43.RES:Rescheduling_interrupts
2067 ± 2% +27.4% 2632 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
2066 ± 3% +27.2% 2629 ± 9% interrupts.CPU45.CAL:Function_call_interrupts
2158 ± 3% +19.6% 2582 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
2036 ± 6% +30.3% 2652 ± 10% interrupts.CPU47.CAL:Function_call_interrupts
150.50 ± 43% -28.0% 108.33 interrupts.CPU49.84:PCI-MSI.1572913-edge.eth0-TxRx-49
11823 ± 12% -36.9% 7460 ± 18% interrupts.CPU49.RES:Rescheduling_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.NMI:Non-maskable_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.PMI:Performance_monitoring_interrupts
11748 ± 22% -40.0% 7048 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts
2102 ± 5% +19.6% 2515 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
2115 ± 6% +19.5% 2527 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
11318 ± 17% -20.4% 9004 ± 26% interrupts.CPU53.RES:Rescheduling_interrupts
2177 ± 5% +17.4% 2556 ± 9% interrupts.CPU54.CAL:Function_call_interrupts
10968 ± 23% -62.3% 4134 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts
2161 ± 5% +20.2% 2598 ± 9% interrupts.CPU55.CAL:Function_call_interrupts
9334 ± 19% -55.4% 4158 ± 34% interrupts.CPU55.RES:Rescheduling_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.NMI:Non-maskable_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.PMI:Performance_monitoring_interrupts
11377 ± 31% -56.9% 4907 ± 43% interrupts.CPU56.RES:Rescheduling_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts
8295 ± 31% -47.7% 4340 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts
9182 ± 37% -59.7% 3701 ± 46% interrupts.CPU58.RES:Rescheduling_interrupts
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.CPU6.41:PCI-MSI.1572870-edge.eth0-TxRx-6
12473 ± 27% -44.4% 6940 ± 13% interrupts.CPU6.RES:Rescheduling_interrupts
7886 ± 12% -42.2% 4555 ± 31% interrupts.CPU60.RES:Rescheduling_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts
9253 ± 26% -56.4% 4033 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts
2103 ± 5% +19.1% 2505 ± 11% interrupts.CPU63.CAL:Function_call_interrupts
2059 ± 2% +20.6% 2483 ± 10% interrupts.CPU64.CAL:Function_call_interrupts
8545 ± 17% -41.2% 5025 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts
7378 ± 23% -51.0% 3614 ± 30% interrupts.CPU68.RES:Rescheduling_interrupts
8573 ± 28% -50.1% 4277 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.CPU7.42:PCI-MSI.1572871-edge.eth0-TxRx-7
9910 ± 28% -51.2% 4839 ± 40% interrupts.CPU70.RES:Rescheduling_interrupts
8836 ± 26% -52.7% 4180 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.NMI:Non-maskable_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.PMI:Performance_monitoring_interrupts
12326 ± 15% -32.3% 8350 ± 10% interrupts.CPU8.RES:Rescheduling_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.NMI:Non-maskable_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.PMI:Performance_monitoring_interrupts
749419 -31.9% 510225 ± 3% interrupts.RES:Rescheduling_interrupts
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread
5.99 ± 8% -1.6 4.37 ± 6% perf-profile.calltrace.cycles-pp.io_submit
5.83 ± 8% -1.6 4.25 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.io_submit
5.82 ± 8% -1.6 4.24 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.calltrace.cycles-pp.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.65 ± 6% -1.6 4.08 ± 20% perf-profile.calltrace.cycles-pp.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
5.58 ± 8% -1.5 4.07 ± 6% perf-profile.calltrace.cycles-pp.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.50 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64
6.63 ± 8% -1.5 5.17 ± 10% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit
5.55 ± 8% -1.2 4.32 ± 10% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
4.98 ± 8% -1.1 3.86 ± 10% perf-profile.calltrace.cycles-pp.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages
5.00 ± 8% -1.1 3.88 ± 10% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages
2.80 ± 4% -1.0 1.78 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
2.32 ± 3% -0.6 1.72 ± 18% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.91 ± 8% -0.5 0.42 ± 73% perf-profile.calltrace.cycles-pp.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.88 ± 8% -0.5 0.41 ± 74% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
0.83 ± 5% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums
1.90 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
1.89 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.82 ± 4% -0.4 0.42 ± 71% perf-profile.calltrace.cycles-pp.run_one_async_done.normal_work_helper.process_one_work.worker_thread.kthread
0.81 ± 4% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.normal_work_helper.process_one_work.worker_thread
0.74 ± 6% -0.4 0.36 ± 70% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks
1.19 ± 3% -0.4 0.83 ± 16% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.calltrace.cycles-pp.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.25 ± 6% -0.3 0.90 ± 7% perf-profile.calltrace.cycles-pp.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.93 ± 8% -0.3 0.59 ± 7% perf-profile.calltrace.cycles-pp.__set_extent_bit.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter
0.94 ± 7% -0.3 0.60 ± 6% perf-profile.calltrace.cycles-pp.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.70 ± 8% -0.3 0.37 ± 70% perf-profile.calltrace.cycles-pp.__lookup_extent_mapping.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter
0.79 ± 6% -0.3 0.47 ± 71% perf-profile.calltrace.cycles-pp.setup_items_for_insert.btrfs_duplicate_item.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
0.71 ± 21% -0.3 0.43 ± 74% perf-profile.calltrace.cycles-pp.__btrfs_free_extent.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space
1.15 ± 11% -0.3 0.88 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.15 ± 11% -0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
1.43 ± 13% -0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.06 ± 5% -0.3 0.81 ± 7% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.30 ± 15% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write
1.03 ± 13% -0.2 0.79 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range
1.20 ± 4% -0.2 0.96 ± 15% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.24 ± 3% -0.2 1.01 ± 15% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.05 ± 8% -0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.calltrace.cycles-pp.wait_reserve_ticket.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
1.01 ± 9% -0.2 0.84 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.01 ± 10% -0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.72 ± 7% -0.2 0.56 ± 6% perf-profile.calltrace.cycles-pp.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.75 ± 8% -0.1 0.60 ± 10% perf-profile.calltrace.cycles-pp.submit_extent_page.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages
0.92 ± 15% +0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.01 ± 15% +0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
1.15 ± 15% +0.3 1.44 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.27 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
1.74 ± 9% +0.4 2.11 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.63 ± 10% +0.6 3.26 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
0.00 +0.8 0.75 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread
0.00 +0.9 0.90 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
4.71 ± 11% +1.0 5.74 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
5.04 ± 10% +1.2 6.19 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
0.00 +1.3 1.30 ± 19% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range
0.00 +1.4 1.36 ± 19% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents
0.29 ±101% +1.4 1.67 ± 18% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space
0.00 +1.5 1.52 ± 18% perf-profile.calltrace.cycles-pp.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
0.00 +2.2 2.20 ± 14% perf-profile.calltrace.cycles-pp.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work.worker_thread
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work
49.45 ± 2% +4.3 53.79 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64
1.07 ± 18% +5.0 6.06 ± 20% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
0.93 ± 18% +5.0 5.95 ± 20% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes
61.26 ± 2% +5.1 66.36 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
61.21 ± 2% +5.1 66.32 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.33 ±100% +5.3 5.68 ± 20% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode
55.45 ± 2% +5.4 60.88 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +6.4 6.39 ± 13% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page
0.00 +6.5 6.50 ± 13% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.children.cycles-pp.normal_work_helper
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.children.cycles-pp.btrfs_finish_ordered_io
7.84 ± 5% -2.4 5.39 ± 10% perf-profile.children.cycles-pp.btrfs_search_slot
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_writepages
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_write_cache_pages
7.03 ± 7% -1.6 5.38 ± 12% perf-profile.children.cycles-pp.__extent_writepage
6.19 ± 8% -1.6 4.56 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
6.17 ± 8% -1.6 4.55 ± 7% perf-profile.children.cycles-pp.do_syscall_64
6.01 ± 8% -1.6 4.39 ± 7% perf-profile.children.cycles-pp.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.children.cycles-pp.__x64_sys_io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.children.cycles-pp.io_submit_one
5.66 ± 5% -1.6 4.09 ± 20% perf-profile.children.cycles-pp.btrfs_mark_extent_written
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.add_pending_csums
5.59 ± 8% -1.5 4.07 ± 6% perf-profile.children.cycles-pp.aio_write
4.51 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.btrfs_csum_file_blocks
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.children.cycles-pp.btrfs_file_write_iter
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write
5.88 ± 7% -1.4 4.47 ± 11% perf-profile.children.cycles-pp.writepage_delalloc
5.30 ± 8% -1.3 4.01 ± 11% perf-profile.children.cycles-pp.btrfs_run_delalloc_range
5.28 ± 8% -1.3 3.99 ± 11% perf-profile.children.cycles-pp.run_delalloc_nocow
2.55 ± 2% -0.9 1.67 ± 16% perf-profile.children.cycles-pp.btrfs_cow_block
2.54 ± 2% -0.9 1.66 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block
2.15 ± 13% -0.8 1.31 ± 8% perf-profile.children.cycles-pp.__lookup_extent_mapping
1.87 ± 12% -0.8 1.04 ± 12% perf-profile.children.cycles-pp.__etree_search
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.printk
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.vprintk_emit
2.60 ± 2% -0.7 1.92 ± 14% perf-profile.children.cycles-pp.btrfs_insert_empty_items
3.00 ± 5% -0.7 2.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock
1.89 ± 2% -0.7 1.23 ± 7% perf-profile.children.cycles-pp.__set_extent_bit
2.78 ± 4% -0.6 2.14 ± 14% perf-profile.children.cycles-pp.__schedule
1.57 ± 2% -0.5 1.02 ± 9% perf-profile.children.cycles-pp.lock_extent_bits
1.19 ± 14% -0.5 0.70 ± 11% perf-profile.children.cycles-pp.__clear_extent_bit
1.51 ± 7% -0.5 1.04 ± 9% perf-profile.children.cycles-pp.read_block_for_search
2.08 ± 5% -0.5 1.62 ± 18% perf-profile.children.cycles-pp.setup_items_for_insert
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.btrfs_run_delayed_refs
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs
1.60 ± 6% -0.4 1.20 ± 12% perf-profile.children.cycles-pp.schedule
1.95 ± 11% -0.4 1.58 ± 12% perf-profile.children.cycles-pp.try_to_wake_up
1.31 ± 6% -0.4 0.94 ± 9% perf-profile.children.cycles-pp.create_io_em
0.99 ± 4% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need
1.27 ± 9% -0.4 0.92 ± 11% perf-profile.children.cycles-pp.__wake_up_common_lock
0.98 ± 3% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.btrfs_alloc_tree_block
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.children.cycles-pp.btrfs_dirty_pages
1.34 ± 7% -0.3 0.99 ± 10% perf-profile.children.cycles-pp.btrfs_map_bio
1.11 ± 9% -0.3 0.77 ± 11% perf-profile.children.cycles-pp.generic_bin_search
1.16 ± 6% -0.3 0.82 ± 9% perf-profile.children.cycles-pp.find_extent_buffer
0.91 ± 7% -0.3 0.57 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_csum
1.21 ± 11% -0.3 0.90 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_csums_range
1.21 ± 11% -0.3 0.91 ± 9% perf-profile.children.cycles-pp.csum_exist_in_range
1.10 ± 10% -0.3 0.80 ± 9% perf-profile.children.cycles-pp.__wake_up_common
0.37 ± 70% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.test_range_bit
1.61 ± 4% -0.3 1.32 ± 15% perf-profile.children.cycles-pp.push_leaf_right
0.60 ± 4% -0.3 0.31 ± 25% perf-profile.children.cycles-pp.btrfs_extend_item
1.02 ± 9% -0.3 0.73 ± 10% perf-profile.children.cycles-pp.autoremove_wake_function
1.12 ± 4% -0.3 0.84 ± 9% perf-profile.children.cycles-pp.btrfs_drop_extent_cache
0.82 ± 6% -0.3 0.55 ± 4% perf-profile.children.cycles-pp.btrfs_release_path
0.85 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_extent_buffer_full
0.99 ± 4% -0.3 0.73 ± 13% perf-profile.children.cycles-pp.btrfs_set_token_32
1.44 ± 13% -0.3 1.18 ± 5% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.84 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_page
0.87 ± 11% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.pagecache_get_page
0.84 ± 7% -0.2 0.60 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc
1.12 ± 7% -0.2 0.88 ± 17% perf-profile.children.cycles-pp.__extent_writepage_io
1.30 ± 15% -0.2 1.06 ± 5% perf-profile.children.cycles-pp.reserve_metadata_bytes
0.82 ± 4% -0.2 0.58 ± 13% perf-profile.children.cycles-pp.run_one_async_done
1.25 ± 3% -0.2 1.01 ± 15% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 53% -0.2 0.10 ± 28% perf-profile.children.cycles-pp.delay_tsc
0.70 ± 13% -0.2 0.47 ± 11% perf-profile.children.cycles-pp.prepare_pages
0.50 ± 22% -0.2 0.28 ± 13% perf-profile.children.cycles-pp.clear_state_bit
0.84 ± 5% -0.2 0.63 ± 6% perf-profile.children.cycles-pp.btrfs_get_extent
0.54 ± 11% -0.2 0.34 ± 4% perf-profile.children.cycles-pp.btrfs_free_path
1.16 ± 11% -0.2 0.96 ± 19% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.53 ± 9% -0.2 0.33 ± 18% perf-profile.children.cycles-pp.unpin_extent_cache
1.07 ± 10% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_file_extent
0.50 ± 4% -0.2 0.31 ± 15% perf-profile.children.cycles-pp.clear_extent_bit
0.45 ± 13% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.kmem_cache_free
0.40 ± 14% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.64 ± 7% -0.2 0.45 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair
0.50 ± 8% -0.2 0.33 ± 23% perf-profile.children.cycles-pp.btrfs_add_delayed_tree_ref
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.children.cycles-pp.wait_reserve_ticket
0.52 ± 9% -0.2 0.35 ± 7% perf-profile.children.cycles-pp.__radix_tree_lookup
0.52 ± 8% -0.2 0.36 ± 8% perf-profile.children.cycles-pp.xas_load
0.78 ± 19% -0.2 0.62 ± 23% perf-profile.children.cycles-pp.__btrfs_free_extent
0.56 ± 17% -0.1 0.42 ± 15% perf-profile.children.cycles-pp.btrfs_del_items
0.51 ± 5% -0.1 0.36 ± 9% perf-profile.children.cycles-pp.dequeue_entity
0.39 ± 7% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.82 ± 14% -0.1 0.68 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate
0.55 ± 8% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.btrfs_submit_bio_hook
0.53 ± 5% -0.1 0.40 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.40 ± 8% -0.1 0.27 ± 27% perf-profile.children.cycles-pp.add_delayed_ref_head
0.35 ± 13% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.set_extent_bit
0.52 ± 9% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.btrfs_wq_submit_bio
0.35 ± 3% -0.1 0.23 ± 32% perf-profile.children.cycles-pp.queued_write_lock_slowpath
0.33 ± 6% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.__slab_alloc
0.63 ± 19% -0.1 0.51 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode_fallback
0.57 ± 5% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.find_lock_delalloc_range
0.45 ± 5% -0.1 0.34 ± 9% perf-profile.children.cycles-pp.btrfs_cross_ref_exist
0.29 ± 3% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.free_extent_buffer
0.38 ± 3% -0.1 0.26 ± 8% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty
0.46 ± 5% -0.1 0.35 ± 13% perf-profile.children.cycles-pp.find_delalloc_range
0.41 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.btrfs_copy_from_user
0.40 ± 9% -0.1 0.29 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
0.38 ± 10% -0.1 0.27 perf-profile.children.cycles-pp.find_get_entry
0.31 ± 12% -0.1 0.20 ± 31% perf-profile.children.cycles-pp.btrfs_remove_ordered_extent
0.30 ± 6% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.___slab_alloc
0.26 ± 8% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.merge_state
0.38 ± 7% -0.1 0.28 ± 3% perf-profile.children.cycles-pp.copyin
0.26 ± 17% -0.1 0.15 ± 26% perf-profile.children.cycles-pp.btrfs_delayed_update_inode
0.25 ± 4% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.insert_state
0.40 ± 13% -0.1 0.30 ± 13% perf-profile.children.cycles-pp.extent_clear_unlock_delalloc
0.36 ± 9% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.alloc_extent_state
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.39 ± 3% -0.1 0.29 ± 14% perf-profile.children.cycles-pp.alloc_extent_buffer
0.31 ± 4% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.set_extent_buffer_dirty
0.26 ± 11% -0.1 0.17 ± 23% perf-profile.children.cycles-pp.unlock_up
0.32 ± 3% -0.1 0.24 ± 13% perf-profile.children.cycles-pp.__btrfs_map_block
0.23 ± 13% -0.1 0.15 ± 28% perf-profile.children.cycles-pp.btrfs_add_delayed_data_ref
0.25 ± 12% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free
0.47 ± 6% -0.1 0.39 ± 15% perf-profile.children.cycles-pp.__push_leaf_right
0.26 ± 9% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.__test_set_page_writeback
0.32 ± 11% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.btrfs_add_ordered_extent
0.15 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__xa_set_mark
0.23 ± 7% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.release_extent_buffer
0.18 ± 8% -0.1 0.11 ± 26% perf-profile.children.cycles-pp.alloc_extent_map
0.32 ± 10% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.__btrfs_add_ordered_extent
0.32 ± 17% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.btrfs_tree_read_lock_atomic
0.20 ± 13% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.btrfs_inc_extent_ref
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.find_ref_head
0.13 ± 32% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.rb_erase_cached
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_submit_bio_start
0.53 ± 12% -0.1 0.46 ± 16% perf-profile.children.cycles-pp.btrfs_tree_read_lock
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_csum_one_bio
0.17 ± 16% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.rb_erase
0.21 ± 7% -0.1 0.14 ± 16% perf-profile.children.cycles-pp.run_one_async_start
0.22 ± 3% -0.1 0.15 ± 22% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock
0.21 ± 9% -0.1 0.15 ± 21% perf-profile.children.cycles-pp.btrfs_leaf_free_space
0.15 ± 23% -0.1 0.09 ± 35% perf-profile.children.cycles-pp.free_extent_map
0.20 ± 9% -0.1 0.14 ± 20% perf-profile.children.cycles-pp.leaf_space_used
0.19 ± 8% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.check_delayed_ref
0.17 ± 13% -0.1 0.12 ± 31% perf-profile.children.cycles-pp.get_page_from_freelist
0.13 ± 8% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.__update_load_avg_se
0.17 ± 2% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.btrfs_set_range_writeback
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 11% -0.1 0.15 ± 35% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_unlock
0.14 ± 11% -0.1 0.09 ± 28% perf-profile.children.cycles-pp.btrfs_reserve_extent
0.10 ± 33% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__btrfs_release_delayed_node
0.22 ± 9% -0.0 0.17 ± 25% perf-profile.children.cycles-pp.mark_extent_buffer_accessed
0.09 ± 13% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_read_unlock_blocking
0.18 ± 8% -0.0 0.14 ± 24% perf-profile.children.cycles-pp.__list_del_entry_valid
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.children.cycles-pp.verify_parent_transid
0.11 ± 19% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.22 ± 5% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.check_committed_ref
0.11 ± 19% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.select_idle_sibling
0.14 ± 11% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.do_io_getevents
0.15 ± 7% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__x64_sys_io_getevents
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_release_extent_buffer_pages
0.17 ± 18% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.14 ± 5% -0.0 0.10 ± 25% perf-profile.children.cycles-pp.block_group_cache_tree_search
0.18 ± 4% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.btrfs_get_chunk_map
0.11 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.btrfs_buffer_uptodate
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.children.cycles-pp.xas_find_marked
0.12 ± 15% -0.0 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_unlock_up_safe
0.12 ± 14% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.btrfs_set_path_blocking
0.11 ± 11% -0.0 0.07 ± 28% perf-profile.children.cycles-pp.find_free_extent
0.24 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.btrfs_root_node
0.07 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.extent_mergeable
0.13 ± 12% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__switch_to
0.10 ± 18% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.rb_insert_color_cached
0.07 ± 14% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.rb_prev
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.get_io_u
0.10 ± 14% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.available_idle_cpu
0.17 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp._raw_write_lock
0.10 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__might_sleep
0.17 ± 7% -0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_read_lock
0.17 ± 10% -0.0 0.14 ± 12% perf-profile.children.cycles-pp.can_overcommit
0.10 ± 11% -0.0 0.07 perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.aio_read_events
0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.___perf_sw_event
0.11 ± 9% -0.0 0.09 ± 18% perf-profile.children.cycles-pp.get_alloc_profile
0.11 ± 12% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.pick_next_task_idle
0.10 ± 13% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.read_events
0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.replace_extent_mapping
0.10 ± 5% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.menu_reflect
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.05 ± 58% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.rcu_irq_enter
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ± 14% +0.1 0.27 ± 8% perf-profile.children.cycles-pp.btrfs_comp_cpu_keys
0.38 ± 9% +0.1 0.46 perf-profile.children.cycles-pp.load_balance
0.30 ± 8% +0.1 0.38 ± 15% perf-profile.children.cycles-pp.irq_enter
0.21 ± 24% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.22 ± 3% +0.1 0.32 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline
0.52 ± 12% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.rebalance_domains
1.90 ± 10% +0.3 2.23 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.23 ± 12% +0.4 0.67 ± 15% perf-profile.children.cycles-pp.read_extent_buffer
2.88 ± 11% +0.5 3.41 perf-profile.children.cycles-pp.hrtimer_interrupt
0.09 ± 12% +0.7 0.83 ± 15% perf-profile.children.cycles-pp.btrfs_get_token_8
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.children.cycles-pp.flush_space
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_marked_extents
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.42 ± 43% +1.3 1.67 ± 18% perf-profile.children.cycles-pp.btrfs_commit_transaction
0.25 ± 5% +1.9 2.17 ± 10% perf-profile.children.cycles-pp.btrfs_get_token_64
1.71 ± 5% +2.1 3.79 ± 5% perf-profile.children.cycles-pp.btrfs_get_token_32
0.43 ± 8% +2.4 2.84 ± 10% perf-profile.children.cycles-pp.map_private_extent_buffer
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_writeback
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.writeback_sb_inodes
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.__writeback_single_inode
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_workfn
50.06 ± 2% +3.8 53.82 ± 4% perf-profile.children.cycles-pp.intel_idle
61.99 ± 2% +4.4 66.39 ± 3% perf-profile.children.cycles-pp.do_idle
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry
8.51 ± 7% +4.5 13.04 ± 12% perf-profile.children.cycles-pp.do_writepages
56.43 ± 3% +4.9 61.30 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.children.cycles-pp.start_secondary
1.43 ± 7% +6.2 7.64 ± 13% perf-profile.children.cycles-pp.submit_extent_page
1.20 ± 17% +6.2 7.45 ± 13% perf-profile.children.cycles-pp.btree_write_cache_pages
1.05 ± 17% +6.3 7.31 ± 13% perf-profile.children.cycles-pp.write_one_eb
1.11 ± 7% +6.3 7.37 ± 13% perf-profile.children.cycles-pp.submit_one_bio
0.55 ± 16% +6.4 6.95 ± 13% perf-profile.children.cycles-pp.btree_submit_bio_hook
0.00 +6.5 6.48 ± 13% perf-profile.children.cycles-pp.check_leaf
0.00 +6.5 6.53 ± 13% perf-profile.children.cycles-pp.btree_csum_one_bio
2.12 ± 11% -0.8 1.29 ± 7% perf-profile.self.cycles-pp.__lookup_extent_mapping
1.83 ± 12% -0.8 1.03 ± 13% perf-profile.self.cycles-pp.__etree_search
2.06 ± 5% -0.5 1.59 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.81 ± 9% -0.3 0.52 ± 10% perf-profile.self.cycles-pp.generic_bin_search
0.84 ± 7% -0.3 0.58 ± 17% perf-profile.self.cycles-pp.copy_page
0.33 ± 54% -0.2 0.10 ± 28% perf-profile.self.cycles-pp.delay_tsc
0.89 ± 9% -0.2 0.69 ± 14% perf-profile.self.cycles-pp.btrfs_set_token_32
0.81 ± 4% -0.2 0.63 ± 14% perf-profile.self.cycles-pp.__schedule
0.51 ± 9% -0.2 0.35 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup
0.45 ± 8% -0.1 0.32 ± 10% perf-profile.self.cycles-pp.xas_load
0.43 ± 4% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.find_extent_buffer
0.77 ± 5% -0.1 0.66 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.31 ± 10% -0.1 0.20 ± 28% perf-profile.self.cycles-pp.add_delayed_ref_head
0.39 ± 16% -0.1 0.29 ± 16% perf-profile.self.cycles-pp.try_to_wake_up
0.31 ± 4% -0.1 0.20 ± 26% perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.20 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.kmem_cache_free
0.25 ± 11% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.__slab_free
0.16 ± 9% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.free_extent_buffer
0.28 ± 17% -0.1 0.20 ± 11% perf-profile.self.cycles-pp.btrfs_search_slot
0.31 ± 4% -0.1 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.find_ref_head
0.16 ± 10% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.insert_state
0.12 ± 8% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.__update_load_avg_se
0.16 ± 15% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.rb_erase
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_unlock
0.15 ± 13% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.___slab_alloc
0.28 ± 13% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.queued_read_lock_slowpath
0.15 ± 21% -0.1 0.09 ± 35% perf-profile.self.cycles-pp.free_extent_map
0.18 ± 6% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.update_load_avg
0.10 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.btrfs_extend_item
0.15 ± 23% -0.1 0.09 ± 9% perf-profile.self.cycles-pp.add_extent_mapping
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.self.cycles-pp.___might_sleep
0.18 ± 6% -0.0 0.13 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ± 4% -0.0 0.14 ± 22% perf-profile.self.cycles-pp.pick_next_task_fair
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.verify_parent_transid
0.09 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_read_unlock_blocking
0.16 ± 13% -0.0 0.12 ± 24% perf-profile.self.cycles-pp.mark_page_accessed
0.12 ± 14% -0.0 0.08 ± 20% perf-profile.self.cycles-pp.dequeue_task_fair
0.10 ± 19% -0.0 0.07 ± 18% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.08 ± 14% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.unlock_up
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.self.cycles-pp.xas_find_marked
0.24 ± 6% -0.0 0.20 ± 8% perf-profile.self.cycles-pp.btrfs_root_node
0.17 ± 6% -0.0 0.13 ± 10% perf-profile.self.cycles-pp._raw_read_lock
0.10 ± 13% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.btrfs_csum_one_bio
0.16 ± 7% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.07 -0.0 0.04 ± 71% perf-profile.self.cycles-pp.io_u_queued_complete
0.08 -0.0 0.05 ± 70% perf-profile.self.cycles-pp.mutex_lock
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.get_io_u
0.16 ± 9% -0.0 0.13 perf-profile.self.cycles-pp._raw_write_lock
0.07 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.btrfs_tree_read_lock
0.12 ± 8% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.__switch_to
0.10 ± 11% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.__clear_extent_bit
0.07 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.rb_prev
0.10 ± 15% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.find_get_pages_range_tag
0.10 ± 12% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.available_idle_cpu
0.10 ± 11% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 13% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__might_sleep
0.12 ± 17% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.release_extent_buffer
0.09 ± 17% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event
0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.__push_leaf_right
0.11 ± 7% -0.0 0.09 ± 9% perf-profile.self.cycles-pp.set_extent_buffer_dirty
0.11 ± 4% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.__set_extent_bit
0.09 ± 7% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.hrtimer_interrupt
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.load_balance
0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.1 0.06 ± 23% perf-profile.self.cycles-pp.find_next_and_bit
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.self.cycles-pp.native_irq_return_iret
0.19 ± 24% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.50 ± 8% +0.2 0.69 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state
0.07 ± 11% +0.4 0.46 ± 17% perf-profile.self.cycles-pp.btrfs_get_token_8
0.22 ± 13% +0.4 0.66 ± 16% perf-profile.self.cycles-pp.read_extent_buffer
0.00 +0.6 0.65 ± 14% perf-profile.self.cycles-pp.check_leaf
0.15 ± 11% +1.0 1.13 ± 11% perf-profile.self.cycles-pp.btrfs_get_token_64
1.48 ± 7% +1.0 2.51 ± 5% perf-profile.self.cycles-pp.btrfs_get_token_32
0.41 ± 8% +2.2 2.61 ± 9% perf-profile.self.cycles-pp.map_private_extent_buffer
49.99 ± 2% +3.7 53.74 ± 4% perf-profile.self.cycles-pp.intel_idle
fio.write_bw_MBps
300 +-+-------------------------------------------------------------------+
| |
250 +-++.+ +. .+.. .+..+..+.+.. .+..+.. .+ |
| : : +. +..+ + +..+ +. + |
| : : : : : : |
200 +-+ : : : : : : O O |
O O O: O :O O O O O O O O O O: O:O O: O: O O O O O
150 +-+ : : : : : : |
| : : : : : : |
100 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
50 +-+ :: :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_iops
70000 +-+-----------------------------------------------------------------+
|..+.+ +..+..+.+.. .+.+..+.+..+ +.. +..+.+..+..+ |
60000 +-+ : : +. : : + : |
| : : : : : : |
50000 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
40000 +-+ : O: : : : : |
| : : : : : : |
30000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
10000 +-+ : : : |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_mean_us
30000 +-+-----------------------------------------------------------------+
| |
25000 O-+O O O O O O O O O O O O O O O O O O O |
| O O O O O
| |
20000 +-+ |
|..+.+ +..+..+.+..+..+.+..+.+..+ +..+ +..+.+..+..+ |
15000 +-+ : : : : : : |
| : : : : : : |
10000 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
5000 +-+ : : : : :: |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_stddev
160000 +-+----------------------------------------------------------------+
| |
140000 O-+O O O O O O O O O O O O O O |
120000 +-+ O O |
| |
100000 +-+ O O O O O O O O
| |
80000 +-+ |
| .+.. |
60000 +-++.+ +..+.+. +.+..+.+..+..+ +.+ +..+..+.+..+ |
40000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
0 +-+-------------------------------------------O------------O-------+
fio.write_slat_mean_us
900 +-+-------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O |
800 +-+O O O O O O O
700 +-+ |
| |
600 +-++.+ +.+..+..+..+.+..+..+.+..+ +..+ +..+..+..+.+ |
500 +-+ : : : : : : |
| : : : : : : |
400 +-+ : : : : : : |
300 +-+ : : : : : : |
| : : : : : : |
200 +-+ : : : : : : |
100 +-+ : : :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_slat_stddev
25000 +-+-----O---------O-------------------------------------------------+
O O O O O O O O O O O O O O O |
| |
20000 +-+ |
| O O O O O O O O
| |
15000 +-+ |
| |
10000 +-++. .+.. .+.. .+..+ .+ .+..+ |
|. + +..+..+ +..+ + : +. : +..+.+. |
| : : : : : : |
5000 +-+ : : : : : : |
| : : : : : : |
| :: : :: |
0 +-+--------------------------------------------O------------O-------+
fio.latency_1000ms_
0.4 +-+------------------------------------------------------------------+
| |
0.35 +-+ O |
0.3 +-+ |
| O O |
0.25 +-+ O O |
| O O O
0.2 O-+ O O O O O O O O O O |
| O O |
0.15 +-+O O O O |
0.1 +-+ |
| |
0.05 +-+ |
| .+. .+..+..+.+..+.. .+. .+.+..+.. |
0 +-+---------------------------------------------O------------O-------+
fio.workload
1.4e+07 +-+---------------------------------------------------------------+
|..+.+ +..+.+..+.+..+.+..+.+..+ +.. +.+..+.+..+ |
1.2e+07 +-+ : : : : + : |
| : : : : : : |
1e+07 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
8e+06 +-+ : O: : : : : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| :: :: :: |
2e+06 +-+ : : : |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.voluntary_context_switches
1.4e+07 +-+---------------------------------------------------------------+
| |
1.2e+07 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
1e+07 +-+ : : : : : : |
| O O: : O O O : : :O : O O O O O O O
8e+06 O-+ : O:O O O O O O O: O:O O: : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| : : : : :: |
2e+06 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.file_system_outputs
1.2e+08 +-+---------------------------------------------------------------+
| |
1e+08 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
| : : : : : : |
8e+07 +-+ : : : : : : O O O O O
O O O: O:O O O O O O O O O O: O:O O:O : O O |
6e+07 +-+ : : : : : : |
| : : : : : : |
4e+07 +-+ : : : : : : |
| : : : : : : |
| : : : : :: |
2e+07 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 1 week
Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression
by Rong Chen
On 6/26/19 11:17 AM, Josef Bacik wrote:
> On Wed, Jun 26, 2019 at 10:39:36AM +0800, Rong Chen wrote:
>> On 6/25/19 10:22 PM, Josef Bacik wrote:
>>> On Fri, Jun 21, 2019 at 08:48:03AM +0800, Huang, Ying wrote:
>>>> "Huang, Ying" <ying.huang(a)intel.com> writes:
>>>>
>>>>> "Huang, Ying" <ying.huang(a)intel.com> writes:
>>>>>
>>>>>> Hi, Josef,
>>>>>>
>>>>>> kernel test robot <rong.a.chen(a)intel.com> writes:
>>>>>>
>>>>>>> Greeting,
>>>>>>>
>>>>>>> FYI, we noticed a -11.7% regression of aim7.jobs-per-min due to commit:
>>>>>>>
>>>>>>>
>>>>>>> commit: c8eaeac7b734347c3afba7008b7af62f37b9c140 ("btrfs: reserve
>>>>>>> delalloc metadata differently")
>>>>>>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>>>>>>
>>>>>>> in testcase: aim7
>>>>>>> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
>>>>>>> with following parameters:
>>>>>>>
>>>>>>> disk: 4BRD_12G
>>>>>>> md: RAID0
>>>>>>> fs: btrfs
>>>>>>> test: disk_rr
>>>>>>> load: 1500
>>>>>>> cpufreq_governor: performance
>>>>>>>
>>>>>>> test-description: AIM7 is a traditional UNIX system level benchmark
>>>>>>> suite which is used to test and measure the performance of multiuser
>>>>>>> system.
>>>>>>> test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
>>>>>> Here's another regression, do you have time to take a look at this?
>>>>> Ping
>>>> Ping again ...
>>>>
>>> Finally got time to look at this but I can't get the reproducer to work
>>>
>>> root@destiny ~/lkp-tests# bin/lkp run ~/job-aim.yaml
>>> Traceback (most recent call last):
>>> 11: from /root/lkp-tests/bin/run-local:18:in `<main>'
>>> 10: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 9: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 8: from /root/lkp-tests/lib/yaml.rb:5:in `<top (required)>'
>>> 7: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 6: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 5: from /root/lkp-tests/lib/common.rb:9:in `<top (required)>'
>>> 4: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 3: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 2: from /root/lkp-tests/lib/array_ext.rb:3:in `<top (required)>'
>>> 1: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- active_support/core_ext/enumerable (LoadError)
>> Hi Josef,
>>
>> I tried the latest lkp-tests, and didn't have the problem. Could you please
>> update the lkp-tests repo and run "lkp install" again?
>>
> I updated it this morning, and I just updated it now, my tree is on
>
> 2c5b1a95b08dbe81bba64419c482a877a3b424ac
>
> lkp install says everything is installed except
>
> No match for argument: libipc-run-perl
I've just fixed it. could you add "libipc-run-perl: perl-IPC-Run" to the
end of distro/adaptation/fedora?
Thanks,
Rong Chen
>
> and it still doesn't run properly. Thanks,
>
> Josef
1 year, 6 months
Re: [LKP] [SUNRPC] 0472e47660: fsmark.app_overhead 16.0% regression
by Xing Zhengjun
On 5/31/2019 3:10 AM, Trond Myklebust wrote:
> On Thu, 2019-05-30 at 15:20 +0800, Xing Zhengjun wrote:
>>
>> On 5/30/2019 10:00 AM, Trond Myklebust wrote:
>>> Hi Xing,
>>>
>>> On Thu, 2019-05-30 at 09:35 +0800, Xing Zhengjun wrote:
>>>> Hi Trond,
>>>>
>>>> On 5/20/2019 1:54 PM, kernel test robot wrote:
>>>>> Greeting,
>>>>>
>>>>> FYI, we noticed a 16.0% improvement of fsmark.app_overhead due
>>>>> to
>>>>> commit:
>>>>>
>>>>>
>>>>> commit: 0472e476604998c127f3c80d291113e77c5676ac ("SUNRPC:
>>>>> Convert
>>>>> socket page send code to use iov_iter()")
>>>>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git
>>>>> master
>>>>>
>>>>> in testcase: fsmark
>>>>> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @
>>>>> 3.00GHz with 384G memory
>>>>> with following parameters:
>>>>>
>>>>> iterations: 1x
>>>>> nr_threads: 64t
>>>>> disk: 1BRD_48G
>>>>> fs: xfs
>>>>> fs2: nfsv4
>>>>> filesize: 4M
>>>>> test_size: 40G
>>>>> sync_method: fsyncBeforeClose
>>>>> cpufreq_governor: performance
>>>>>
>>>>> test-description: The fsmark is a file system benchmark to test
>>>>> synchronous write workloads, for example, mail servers
>>>>> workload.
>>>>> test-url: https://sourceforge.net/projects/fsmark/
>>>>>
>>>>>
>>>>>
>>>>> Details are as below:
>>>>> -------------------------------------------------------------
>>>>> ----
>>>>> --------------------------------->
>>>>>
>>>>>
>>>>> To reproduce:
>>>>>
>>>>> git clone https://github.com/intel/lkp-tests.git
>>>>> cd lkp-tests
>>>>> bin/lkp install job.yaml # job file is attached in
>>>>> this
>>>>> email
>>>>> bin/lkp run job.yaml
>>>>>
>>>>> ===============================================================
>>>>> ====
>>>>> ======================
>>>>> compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconf
>>>>> ig/n
>>>>> r_threads/rootfs/sync_method/tbox_group/test_size/testcase:
>>>>> gcc-7/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel-
>>>>> 7.6/64t/debian-x86_64-2018-04-03.cgz/fsyncBeforeClose/lkp-ivb-
>>>>> ep01/40G/fsmark
>>>>>
>>>>> commit:
>>>>> e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use
>>>>> iov_iter_kvec()")
>>>>> 0472e47660 ("SUNRPC: Convert socket page send code to use
>>>>> iov_iter()")
>>>>>
>>>>> e791f8e9380d945e 0472e476604998c127f3c80d291
>>>>> ---------------- ---------------------------
>>>>> fail:runs %reproduction fail:runs
>>>>> | | |
>>>>> :4 50% 2:4 dmesg.WARNING:a
>>>>> t#for
>>>>> _ip_interrupt_entry/0x
>>>>> %stddev %change %stddev
>>>>> \ | \
>>>>> 15118573
>>>>> ± 2% +16.0% 17538083 fsmark.app_overhead
>>>>> 510.93 -
>>>>> 22.7% 395.12 fsmark.files_per_sec
>>>>> 24.90 +22.8% 30.57 fsmark.time.ela
>>>>> psed_
>>>>> time
>>>>> 24.90 +22.8% 30.57 fsmark.time.ela
>>>>> psed_
>>>>> time.max
>>>>> 288.00 ± 2% -
>>>>> 27.8% 208.00 fsmark.time.percent_of_cpu_this_job_got
>>>>> 70.03 ± 2% -
>>>>> 11.3% 62.14 fsmark.time.system_time
>>>>>
>>>>
>>>> Do you have time to take a look at this regression?
>>>
>>> From your stats, it looks to me as if the problem is increased
>>> NUMA
>>> overhead. Pretty much everything else appears to be the same or
>>> actually performing better than previously. Am I interpreting that
>>> correctly?
>> The real regression is the throughput(fsmark.files_per_sec) is
>> decreased
>> by 22.7%.
>
> Understood, but I'm trying to make sense of why. I'm not able to
> reproduce this, so I have to rely on your performance stats to
> understand where the 22.7% regression is coming from. As far as I can
> see, the only numbers in the stats you published that are showing a
> performance regression (other than the fsmark number itself), are the
> NUMA numbers. Is that a correct interpretation?
>
We re-test the case yesterday, the test result almost is the same.
we will do more test and also check the test case itself, if you need
more information, please let me know, thanks.
>>> If my interpretation above is correct, then I'm not seeing where
>>> this
>>> patch would be introducing new NUMA regressions. It is just
>>> converting
>>> from using one method of doing socket I/O to another. Could it
>>> perhaps
>>> be a memory artefact due to your running the NFS client and server
>>> on
>>> the same machine?
>>>
>>> Apologies for pushing back a little, but I just don't have the
>>> hardware available to test NUMA configurations, so I'm relying on
>>> external testing for the above kind of scenario.
>>>
>> Thanks for looking at this. If you need more information, please let
>> me
>> know.
>>> Thanks
>>> Trond
>>>
--
Zhengjun Xing
1 year, 6 months
07173c3ec2: stderr.fio:pid=#, err=#/file:filesetup.c:#, func=fsync, error=Input/output_error
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 07173c3ec276cbb18dc0e0687d37d310e98a1480 ("block: enable multipage bvecs")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fio-basic
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
with following parameters:
runtime: 300s
nr_task: 8t
disk: 1SSD
fs: xfs
rw: randread
bs: 2M
ioengine: sync
test_size: 512g
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
fio: pid=0, err=5/file:filesetup.c:190, func=fsync, error=Input/output error
fio: pid=0, err=5/file:filesetup.c:190, func=fsync, error=Input/output error
To reproduce:
git clonehttps://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
1 year, 6 months
[x86/hotplug] e1056a25da: WARNING:at_arch/x86/kernel/apic/apic.c:#setup_local_APIC
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: e1056a25daa6460c95e92d7d6853d05ad62458f7 ("x86/hotplug: Silence APIC and NMI when CPU is dead")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git WIP.x86/ipi
in testcase: locktorture
with following parameters:
runtime: 300s
test: cpuhotplug
test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors.
test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+----------------------------------------------------------+------------+------------+
| | c1257b49b9 | e1056a25da |
+----------------------------------------------------------+------------+------------+
| boot_successes | 8 | 6 |
| boot_failures | 51 | 48 |
| BUG:kernel_reboot-without-warning_in_test_stage | 50 | 22 |
| BUG:soft_lockup-CPU##stuck_for#s | 1 | |
| RIP:__memcpy | 1 | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 1 | |
| WARNING:at_arch/x86/kernel/apic/apic.c:#setup_local_APIC | 0 | 26 |
| RIP:setup_local_APIC | 0 | 26 |
+----------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 216.682229] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/apic/apic.c:1473 setup_local_APIC+0x250/0x3b0
[ 216.724190] Modules linked in: locktorture torture sr_mod cdrom ata_generic pata_acpi bochs_drm ttm ppdev drm_kms_helper ata_piix syscopyarea snd_pcm sysfillrect sysimgblt libata fb_sys_fops snd_timer snd drm crc32c_intel soundcore joydev serio_raw pcspkr i2c_piix4 parport_pc parport floppy
[ 216.771348] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.2.0-rc5-00003-ge1056a2 #1
[ 216.785027] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 216.800351] RIP: 0010:setup_local_APIC+0x250/0x3b0
[ 216.809104] Code: 30 8b 0d eb 78 7d 01 85 c9 74 26 0f 31 c1 e1 0a 48 c1 e2 20 41 89 ce 4c 03 34 24 48 09 d0 49 29 c6 4d 85 f6 0f 8f 74 fe ff ff <0f> 0b e9 22 ff ff ff 49 83 ee 01 eb ea a9 00 00 01 00 8b 15 2c ac
[ 216.843098] RSP: 0000:ffffffff89a03ee0 EFLAGS: 00010082
[ 216.852617] RAX: 00000076a573f693 RBX: 0000000000000020 RCX: 0000000089ff6800
[ 216.865848] RDX: 0000007600000000 RSI: 0000000000000020 RDI: ffffffff89a03ef0
[ 216.878971] RBP: 0000000000000000 R08: ffffffff8a348de2 R09: 0000000000000032
[ 216.892080] R10: ffffffff89a03ee8 R11: 0000000000000000 R12: 00000000000000f0
[ 216.905019] R13: 0000000000001000 R14: fffffffffffc1cfb R15: 0000000000000000
[ 216.918205] FS: 0000000000000000(0000) GS:ffff89af74800000(0000) knlGS:0000000000000000
[ 216.941317] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 216.951812] CR2: 0000000057567184 CR3: 000000000ba0a000 CR4: 00000000000006f0
[ 216.964997] Call Trace:
[ 216.969667] apic_ap_setup+0xa/0x20
[ 216.976266] start_secondary+0x78/0x1e0
[ 216.983423] secondary_startup_64+0xb6/0xc0
[ 216.991271] ---[ end trace 06172ec0274b1dfa ]---
To reproduce:
# build kernel
cd linux
cp config-5.2.0-rc5-00003-ge1056a2 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 6 months
fcc784be83 [ 150.952780] WARNING: held lock freed!
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
commit fcc784be837714a9173b372ff9fb9b514590dad9
Author: Steven Rostedt (VMware) <rostedt(a)goodmis.org>
AuthorDate: Wed Apr 4 14:06:30 2018 -0400
Commit: Ingo Molnar <mingo(a)kernel.org>
CommitDate: Thu Jun 21 18:19:01 2018 +0200
locking/lockdep: Do not record IRQ state within lockdep code
While debugging where things were going wrong with mapping
enabling/disabling interrupts with the lockdep state and actual real
enabling and disabling interrupts, I had to silent the IRQ
disabling/enabling in debug_check_no_locks_freed() because it was
always showing up as it was called before the splat was.
Use raw_local_irq_save/restore() for not only debug_check_no_locks_freed()
but for all internal lockdep functions, as they hide useful information
about where interrupts were used incorrectly last.
Signed-off-by: Steven Rostedt (VMware) <rostedt(a)goodmis.org>
Cc: Andrew Morton <akpm(a)linux-foundation.org>
Cc: Linus Torvalds <torvalds(a)linux-foundation.org>
Cc: Paul E. McKenney <paulmck(a)linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz(a)infradead.org>
Cc: Thomas Gleixner <tglx(a)linutronix.de>
Cc: Will Deacon <will.deacon(a)arm.com>
Link: https://lkml.kernel.org/lkml/20180404140630.3f4f4c7a@gandalf.local.home
Signed-off-by: Ingo Molnar <mingo(a)kernel.org>
03eeafdd9a locking/rwsem: Fix up_read_non_owner() warning with DEBUG_RWSEMS
fcc784be83 locking/lockdep: Do not record IRQ state within lockdep code
bed3c0d84e Merge tag 'for-5.2-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
1c6b40509d Add linux-next specific files for 20190618
+----------------------------------------------------------------------------------+------------+------------+------------+---------------+
| | 03eeafdd9a | fcc784be83 | bed3c0d84e | next-20190618 |
+----------------------------------------------------------------------------------+------------+------------+------------+---------------+
| boot_successes | 874 | 277 | 276 | 21 |
| boot_failures | 14 | 19 | 64 | 7 |
| BUG:soft_lockup-CPU##stuck_for#s | 11 | 2 | 4 | |
| EIP:smp_call_function_single | 1 | 1 | | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 11 | 2 | 4 | |
| EIP:_raw_spin_unlock_irqrestore | 3 | 0 | 1 | |
| EIP:__copy_user_ll | 2 | 0 | 1 | |
| invoked_oom-killer:gfp_mask=0x | 1 | 2 | 1 | |
| Mem-Info | 1 | 2 | 2 | |
| EIP:wp_page_copy | 3 | | | |
| BUG:kernel_hang_in_early-boot_stage | 1 | | | |
| EIP:shmem_getpage_gfp | 2 | 0 | 1 | |
| BUG:workqueue_lockup-pool | 1 | 0 | 1 | |
| BUG:kernel_hang_in_boot-around-mounting-root_stage | 0 | 1 | | |
| Out_of_memory:Kill_process | 0 | 1 | | |
| WARNING:held_lock_freed | 0 | 13 | 58 | 7 |
| is_freeing_memory#-#,with_a_lock_still_held_there | 0 | 13 | 58 | 7 |
| BUG:kernel_hang_in_early-boot_stage,last_printk:early_console_in_setup_code | 0 | 1 | | |
| EIP:rcu_is_watching | 0 | 1 | | |
| EIP:ring_buffer_consume | 0 | 0 | 1 | |
| page_allocation_failure:order:#,mode:#(GFP_KERNEL|__GFP_NORETRY),nodemask=(null) | 0 | 0 | 1 | |
| BUG:unable_to_handle_page_fault_for_address | 0 | 0 | 1 | |
| Oops:#[##] | 0 | 0 | 1 | |
| EIP:debug_check_no_locks_freed | 0 | 0 | 1 | |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 0 | 1 | |
+----------------------------------------------------------------------------------+------------+------------+------------+---------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 147.470176] init: tty3 main process (364) terminated with status 1
[ 147.497211] init: tty3 main process ended, respawning
[ 147.612415] init: tty6 main process (365) terminated with status 1
[ 147.687364] init: tty6 main process ended, respawning
[ 150.937355]
[ 150.941927] =========================
[ 150.952780] WARNING: held lock freed!
[ 150.963993] 4.17.0-09747-gfcc784b #1 Tainted: G T
[ 150.989489] -------------------------
[ 151.005112] trinity-main/356 is freeing memory ce93de20-ce93de47, with a lock still held there!
[ 151.040984] (ptrval) (&wq#3){....}, at: __wake_up_common_lock+0x5a/0x140
[ 151.069147] 1 lock held by trinity-main/356:
[ 151.087139] #0: (ptrval) (&sb->s_type->i_mutex_key){....}, at: iterate_dir+0x42/0x2a0
[ 151.117861]
[ 151.117861] stack backtrace:
[ 151.132116] CPU: 1 PID: 356 Comm: trinity-main Tainted: G T 4.17.0-09747-gfcc784b #1
[ 151.162025] Call Trace:
[ 151.170423] dump_stack+0x2bc/0x41a
[ 151.182441] debug_check_no_locks_freed+0x354/0x370
[ 151.199065] __raw_spin_lock_init+0x29/0x80
[ 151.214099] __init_waitqueue_head+0x2e/0x70
[ 151.228615] ? proc_tgid_base_lookup+0x40/0x40
[ 151.243819] proc_fill_cache+0x118/0x360
[ 151.256974] ? format_decode+0x1a6/0x890
[ 151.270037] ? format_decode+0x308/0x890
[ 151.283679] ? vsnprintf+0x622/0x860
[ 151.293724] proc_map_files_readdir+0x648/0x6a0
[ 151.306619] ? proc_tgid_base_lookup+0x40/0x40
[ 151.320722] iterate_dir+0x225/0x2a0
[ 151.331637] ? __fget_light+0x7f/0x100
[ 151.342010] ksys_getdents64+0x1ac/0x2f0
[ 151.353610] ? sys_old_readdir+0x160/0x160
[ 151.366017] sys_getdents64+0x24/0x40
[ 151.377653] do_fast_syscall_32+0x14b/0x780
[ 151.392209] entry_SYSENTER_32+0x53/0x86
[ 151.404786] EIP: 0xa7fc4cd9
[ 151.423581] Code: 08 8b 80 5c cd ff ff 85 d2 74 02 89 02 5d c3 8b 04 24 c3 8b 0c 24 c3 8b 1c 24 c3 8b 3c 24 c3 90 90 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d 76 00 58 b8 77 00 00 00 cd 80 90 8d 76
[ 151.547420] EAX: ffffffda EBX: 0000002b ECX: 08aefedc EDX: 00008000
[ 151.586666] ESI: 08aefedc EDI: ffffffd0 EBP: 00000026 ESP: af88aac4
[ 151.605753] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246
[main] Added 10680 filenames from /proc
[main] Added 13148 filenames from /proc
[main] Added 13151 filenames from /proc
[ 156.784347] init: tty4 main process (366) terminated with status 1
[ 156.849880] init: tty4 main process ended, respawning
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start v4.18 v4.17 --
git bisect bad c81b995f00c7a1c2ca9ad67f5bb4a50d02f98f84 # 23:06 B 6 1 0 0 Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 2a70ea5cda00214a1d573acf19fa0cd06d947e38 # 23:43 G 205 0 9 9 Merge tag 'hsi-for-4.18' of git://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-hsi
git bisect good e7655d2b25466c534ed1f539367dae595bb0bd20 # 00:13 G 209 0 15 15 Merge tag 'for-4.18-part2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
git bisect good 6d90eb7ba341b3eb035121eff0b69d370cbc251e # 00:44 G 206 0 18 18 Merge tag 'dma-rename-4.18' of git://git.infradead.org/users/hch/dma-mapping
git bisect good 5e2204832b20ef9db859dd6a2e955ac3e33eef27 # 01:07 G 212 0 9 9 Merge tag 'powerpc-4.18-2' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
git bisect good 6242258b6b472f8fdd8ed9b735cc1190c185d16d # 01:37 G 206 0 24 24 Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad d4e860eaf0584dfcc1375e06eeb34f85f43c8d34 # 02:10 B 36 1 4 4 Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 2da2ca24a38f0200111e3b8823c08d02cb59d362 # 03:07 B 90 1 7 7 Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good a43de489934cadcbc4cc08a6590fdcc833768461 # 04:05 G 302 0 21 21 Merge branch 'ras-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 7ea959c45769612aa92557fb6464679f5fec7d9e # 04:37 G 306 0 18 18 locking/refcounts: Implement refcount_dec_and_lock_irqsave()
git bisect bad fcc784be837714a9173b372ff9fb9b514590dad9 # 06:19 B 19 1 3 3 locking/lockdep: Do not record IRQ state within lockdep code
git bisect good 03eeafdd9ab06a770d42c2b264d50dff7e2f4eee # 06:47 G 305 0 18 18 locking/rwsem: Fix up_read_non_owner() warning with DEBUG_RWSEMS
# first bad commit: [fcc784be837714a9173b372ff9fb9b514590dad9] locking/lockdep: Do not record IRQ state within lockdep code
git bisect good 03eeafdd9ab06a770d42c2b264d50dff7e2f4eee # 07:23 G 904 0 40 58 locking/rwsem: Fix up_read_non_owner() warning with DEBUG_RWSEMS
# extra tests on HEAD of internal-eywa/master
git bisect bad b4c6b079156ebc029114a45812d5e5298f51fa01 # 07:23 B 8 3 0 2 Intel Next: Add release files for v5.2-rc5 2019-06-17
# extra tests on tree/branch linus/master
git bisect bad bed3c0d84e7e25c8e0964d297794f4c215b01f33 # 08:52 B 5 2 0 0 Merge tag 'for-5.2-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
# extra tests with first bad commit reverted
git bisect bad b0f031ad2fd0758b5bb7c58afe7f8bbaab65c265 # 09:42 B 126 2 10 10 Revert "locking/lockdep: Do not record IRQ state within lockdep code"
# extra tests on tree/branch linux-next/master
git bisect bad 1c6b40509daf5190b1fd2c758649f7df1da4827b # 10:20 B 9 2 0 0 Add linux-next specific files for 20190618
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 6 months
[locking/rwsem] 7d43f1ce9d: vm-scalability.median -16.5% regression
by kernel test robot
Greeting,
FYI, we noticed a -16.5% regression of vm-scalability.median due to commit:
commit: 7d43f1ce9dd075d8b2aa3ad1f3970ef386a5c358 ("locking/rwsem: Enable time-based spinning on reader-owned rwsem")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: vm-scalability
on test machine: 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory
with following parameters:
runtime: 300s
size: 8T
test: anon-w-seq-mt
cpufreq_governor: performance
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
In addition to that, the commit also has significant impact on the following tests:
+------------------+-----------------------------------------------------------------------+
| testcase: change | vm-scalability: vm-scalability.median 14.0% improvement |
| test machine | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory |
| test parameters | cpufreq_governor=performance |
| | runtime=300s |
| | test=small-allocs-mt |
| | ucode=0x21 |
+------------------+-----------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2019-05-14.cgz/300s/8T/lkp-csl-2ap1/anon-w-seq-mt/vm-scalability
commit:
94a9717b3c ("locking/rwsem: Make rwsem->owner an atomic_long_t")
7d43f1ce9d ("locking/rwsem: Enable time-based spinning on reader-owned rwsem")
94a9717b3c40e77a 7d43f1ce9dd075d8b2aa3ad1f39
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
0.01 ± 2% +166.9% 0.02 ± 6% vm-scalability.free_time
191759 -16.5% 160104 vm-scalability.median
0.04 ± 4% +163.7% 0.11 ± 8% vm-scalability.median_stddev
39109084 -7.1% 36338578 vm-scalability.throughput
306656 ± 2% -64.3% 109562 ± 5% vm-scalability.time.involuntary_context_switches
89394993 -8.2% 82031710 ± 4% vm-scalability.time.maximum_resident_set_size
13786907 -23.1% 10603356 ± 2% vm-scalability.time.minor_page_faults
15225 -29.2% 10777 ± 2% vm-scalability.time.percent_of_cpu_this_job_got
22047 -23.6% 16835 vm-scalability.time.system_time
23929 -34.3% 15715 ± 2% vm-scalability.time.user_time
1555880 -47.0% 825313 ± 2% vm-scalability.time.voluntary_context_switches
1.037e+10 -23.1% 7.98e+09 ± 2% vm-scalability.workload
37.83 +1.0% 38.20 boot-time.boot
18197810 ±138% -83.6% 2988363 ± 7% cpuidle.C1.time
21000317 ± 14% +109.1% 43903618 ± 21% cpuidle.C1E.usage
119868 ± 42% +54.1% 184715 ± 19% cpuidle.POLL.usage
20.00 ± 3% +113.8% 42.75 ± 4% vmstat.cpu.id
40.75 -34.4% 26.75 ± 3% vmstat.cpu.us
158.50 -31.1% 109.25 vmstat.procs.r
15578 -34.5% 10207 vmstat.system.cs
19.72 ± 2% +23.2 42.90 ± 3% mpstat.cpu.all.idle%
0.07 ± 5% -0.1 0.01 ± 36% mpstat.cpu.all.iowait%
0.00 ± 9% -0.0 0.00 ± 12% mpstat.cpu.all.soft%
38.55 -9.0 29.59 mpstat.cpu.all.sys%
41.67 -14.2 27.49 ± 2% mpstat.cpu.all.usr%
3990675 -22.9% 3074963 ± 2% numa-numastat.node1.local_node
4021819 -23.1% 3092032 ± 2% numa-numastat.node1.numa_hit
31144 -45.2% 17068 ± 64% numa-numastat.node1.other_node
4722883 ± 29% -33.2% 3155351 ± 3% numa-numastat.node2.local_node
4747771 ± 29% -32.9% 3186380 ± 3% numa-numastat.node2.numa_hit
2404 -28.1% 1728 ± 2% turbostat.Avg_MHz
80.61 -22.4 58.22 ± 2% turbostat.Busy%
20996179 ± 15% +109.1% 43902033 ± 21% turbostat.C1E
18.85 ± 2% +115.1% 40.56 ± 3% turbostat.CPU%c1
477.24 -11.9% 420.57 turbostat.PkgWatt
17150 ± 3% +13.7% 19503 ± 3% slabinfo.cred_jar.active_objs
17150 ± 3% +13.7% 19503 ± 3% slabinfo.cred_jar.num_objs
1757 ± 5% -11.9% 1547 ± 4% slabinfo.kmalloc-rcl-96.active_objs
1757 ± 5% -11.9% 1547 ± 4% slabinfo.kmalloc-rcl-96.num_objs
17269 -11.2% 15337 ± 2% slabinfo.task_delay_info.active_objs
17269 -11.2% 15337 ± 2% slabinfo.task_delay_info.num_objs
45090908 -30.3% 31436724 ± 2% meminfo.Active
45090677 -30.3% 31436484 ± 2% meminfo.Active(anon)
33671307 -30.5% 23416058 meminfo.AnonHugePages
44973813 -30.4% 31321842 ± 2% meminfo.AnonPages
90656336 -19.4% 73097429 meminfo.Committed_AS
568845 ± 10% -23.9% 432780 ± 4% meminfo.DirectMap4k
47516120 -28.9% 33772543 ± 2% meminfo.Memused
79828 -28.6% 56964 ± 2% meminfo.PageTables
303260 ± 2% -10.9% 270236 ± 4% meminfo.max_used_kB
11010213 ± 3% -27.8% 7949775 ± 22% numa-meminfo.node0.Active
11010184 ± 3% -27.8% 7949699 ± 22% numa-meminfo.node0.Active(anon)
10955056 ± 3% -27.7% 7918969 ± 22% numa-meminfo.node0.AnonPages
11634902 ± 3% -26.3% 8570261 ± 21% numa-meminfo.node0.MemUsed
11216510 ± 3% -40.5% 6673714 ± 5% numa-meminfo.node1.Active
11216510 ± 3% -40.5% 6673685 ± 5% numa-meminfo.node1.Active(anon)
8326268 ± 4% -40.7% 4934615 ± 5% numa-meminfo.node1.AnonHugePages
11198925 ± 3% -40.6% 6650406 ± 5% numa-meminfo.node1.AnonPages
37759405 +12.1% 42320486 numa-meminfo.node1.MemFree
11767954 ± 3% -38.8% 7206873 ± 5% numa-meminfo.node1.MemUsed
18759 ± 2% -37.9% 11650 ± 5% numa-meminfo.node1.PageTables
11526969 ± 9% -41.4% 6752594 ± 9% numa-meminfo.node2.Active
11526927 ± 9% -41.4% 6752552 ± 9% numa-meminfo.node2.Active(anon)
8576117 ± 10% -41.9% 4982950 ± 10% numa-meminfo.node2.AnonHugePages
11505263 ± 9% -41.5% 6735650 ± 9% numa-meminfo.node2.AnonPages
37426528 ± 2% +12.9% 42249731 numa-meminfo.node2.MemFree
12100840 ± 9% -39.9% 7277636 ± 8% numa-meminfo.node2.MemUsed
19912 ± 13% -37.6% 12427 ± 13% numa-meminfo.node2.PageTables
7507 ± 25% -25.9% 5563 ± 5% numa-meminfo.node3.Mapped
2653570 ± 3% -25.9% 1965712 ± 22% numa-vmstat.node0.nr_active_anon
2650970 ± 3% -25.9% 1964507 ± 22% numa-vmstat.node0.nr_zone_active_anon
2682266 ± 4% -37.6% 1672693 ± 5% numa-vmstat.node1.nr_active_anon
2677270 ± 4% -37.8% 1666244 ± 5% numa-vmstat.node1.nr_anon_pages
3905 ± 6% -38.4% 2405 ± 5% numa-vmstat.node1.nr_anon_transparent_hugepages
9564136 +10.6% 10577037 numa-vmstat.node1.nr_free_pages
4608 ± 3% -37.0% 2901 ± 5% numa-vmstat.node1.nr_page_table_pages
2680195 ± 4% -37.6% 1671790 ± 5% numa-vmstat.node1.nr_zone_active_anon
2295337 ± 5% -20.7% 1821302 ± 5% numa-vmstat.node1.numa_hit
2180885 ± 5% -21.1% 1720231 ± 5% numa-vmstat.node1.numa_local
2839093 ± 10% -39.9% 1707571 ± 11% numa-vmstat.node2.nr_active_anon
2833805 ± 10% -39.9% 1703266 ± 11% numa-vmstat.node2.nr_anon_pages
4158 ± 12% -40.7% 2465 ± 12% numa-vmstat.node2.nr_anon_transparent_hugepages
25.50 ±116% +204.9% 77.75 ± 80% numa-vmstat.node2.nr_dirtied
9402050 ± 3% +12.1% 10544232 numa-vmstat.node2.nr_free_pages
34.50 ±131% +145.7% 84.75 ± 76% numa-vmstat.node2.nr_inactive_file
4980 ± 14% -37.3% 3120 ± 12% numa-vmstat.node2.nr_page_table_pages
24.25 ±123% +193.8% 71.25 ± 79% numa-vmstat.node2.nr_written
2836803 ± 10% -39.8% 1706487 ± 11% numa-vmstat.node2.nr_zone_active_anon
34.50 ±131% +145.7% 84.75 ± 76% numa-vmstat.node2.nr_zone_inactive_file
2612725 ± 30% -27.7% 1887879 ± 6% numa-vmstat.node2.numa_hit
2504430 ± 31% -29.2% 1773066 ± 7% numa-vmstat.node2.numa_local
11388953 -31.7% 7777830 proc-vmstat.nr_active_anon
11358111 -31.8% 7748673 proc-vmstat.nr_anon_pages
16635 -31.6% 11372 proc-vmstat.nr_anon_transparent_hugepages
441.75 +20.8% 533.75 proc-vmstat.nr_dirtied
3718102 +9.8% 4082210 proc-vmstat.nr_dirty_background_threshold
7448721 +9.8% 8175520 proc-vmstat.nr_dirty_threshold
37434915 +9.7% 41069758 proc-vmstat.nr_free_pages
529.50 +8.3% 573.25 proc-vmstat.nr_inactive_file
28930 -1.0% 28637 proc-vmstat.nr_kernel_stack
9789 ± 2% -3.2% 9472 proc-vmstat.nr_mapped
20077 -29.2% 14204 proc-vmstat.nr_page_table_pages
46878 -5.0% 44552 ± 2% proc-vmstat.nr_shmem
416.00 +20.5% 501.25 proc-vmstat.nr_written
11388927 -31.7% 7777820 proc-vmstat.nr_zone_active_anon
529.50 +8.3% 573.25 proc-vmstat.nr_zone_inactive_file
19364148 -21.9% 15132755 ± 2% proc-vmstat.numa_hit
14619 ± 5% -43.5% 8257 ± 10% proc-vmstat.numa_huge_pte_updates
19270325 -22.0% 15039253 ± 2% proc-vmstat.numa_local
7544881 ± 5% -43.3% 4275462 ± 10% proc-vmstat.numa_pte_updates
27144 ± 5% -30.7% 18821 ± 3% proc-vmstat.pgactivate
2.311e+09 -23.1% 1.776e+09 ± 2% proc-vmstat.pgalloc_normal
14816778 -21.5% 11633019 ± 2% proc-vmstat.pgfault
2.308e+09 -23.4% 1.767e+09 ± 2% proc-vmstat.pgfree
4478662 -23.4% 3428568 ± 2% proc-vmstat.thp_deferred_split_page
4483698 -23.1% 3446054 ± 2% proc-vmstat.thp_fault_alloc
108799 ± 11% -36.7% 68886 ± 3% sched_debug.cfs_rq:/.exec_clock.avg
117668 ± 11% -22.3% 91468 ± 2% sched_debug.cfs_rq:/.exec_clock.max
98854 ± 11% -48.6% 50806 ± 4% sched_debug.cfs_rq:/.exec_clock.min
4923 ± 19% +124.0% 11028 ± 11% sched_debug.cfs_rq:/.exec_clock.stddev
285.55 ± 16% -17.5% 235.65 ± 20% sched_debug.cfs_rq:/.load_avg.max
1.28 ± 35% -100.0% 0.00 sched_debug.cfs_rq:/.load_avg.min
20824703 ± 11% -49.5% 10507532 ± 3% sched_debug.cfs_rq:/.min_vruntime.avg
22103859 ± 12% -43.6% 12472189 ± 3% sched_debug.cfs_rq:/.min_vruntime.max
18960410 ± 12% -58.0% 7969687 ± 5% sched_debug.cfs_rq:/.min_vruntime.min
649563 ± 30% +62.7% 1056596 ± 12% sched_debug.cfs_rq:/.min_vruntime.stddev
0.64 ± 13% -24.2% 0.48 ± 9% sched_debug.cfs_rq:/.nr_running.avg
0.31 ± 19% +28.6% 0.40 ± 8% sched_debug.cfs_rq:/.nr_running.stddev
9.18 ± 10% -33.1% 6.15 ± 4% sched_debug.cfs_rq:/.nr_spread_over.avg
65.36 ± 20% -35.4% 42.25 ± 33% sched_debug.cfs_rq:/.nr_spread_over.max
7.30 ± 15% -33.5% 4.85 ± 13% sched_debug.cfs_rq:/.nr_spread_over.stddev
1791102 ± 26% +35.2% 2420709 ± 10% sched_debug.cfs_rq:/.spread0.avg
3067181 ± 20% +42.7% 4377704 ± 7% sched_debug.cfs_rq:/.spread0.max
651962 ± 30% +62.0% 1055918 ± 12% sched_debug.cfs_rq:/.spread0.stddev
773.26 ± 7% -20.2% 617.26 ± 4% sched_debug.cfs_rq:/.util_avg.avg
188.92 ± 19% -99.4% 1.05 ±105% sched_debug.cfs_rq:/.util_avg.min
184.74 ± 23% +58.9% 293.65 ± 12% sched_debug.cfs_rq:/.util_avg.stddev
1166 ± 10% -17.1% 966.80 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.max
126.46 ± 15% -46.2% 67.98 ± 29% sched_debug.cpu.clock.stddev
126.46 ± 15% -46.2% 67.98 ± 29% sched_debug.cpu.clock_task.stddev
10644 ± 12% -37.8% 6622 ± 7% sched_debug.cpu.curr->pid.avg
15239 ± 9% -20.2% 12163 sched_debug.cpu.curr->pid.max
0.00 ± 15% -49.1% 0.00 ± 28% sched_debug.cpu.next_balance.stddev
144767 ± 10% -11.1% 128756 sched_debug.cpu.nr_load_updates.avg
125750 ± 9% -17.2% 104090 ± 10% sched_debug.cpu.nr_load_updates.min
0.64 ± 13% -28.8% 0.45 ± 7% sched_debug.cpu.nr_running.avg
1.51 ± 7% -23.8% 1.15 ± 14% sched_debug.cpu.nr_running.max
11047 ± 12% -41.7% 6438 sched_debug.cpu.nr_switches.avg
6783 ± 10% -44.3% 3775 ± 3% sched_debug.cpu.nr_switches.min
0.16 ± 51% +91.3% 0.30 ± 25% sched_debug.cpu.nr_uninterruptible.avg
-23.28 +27.6% -29.70 sched_debug.cpu.nr_uninterruptible.min
14.58 ± 2% +46.8% 21.41 sched_debug.cpu.nr_uninterruptible.stddev
10564 ± 12% -44.1% 5908 sched_debug.cpu.sched_count.avg
6543 ± 11% -45.8% 3546 ± 2% sched_debug.cpu.sched_count.min
4217 ± 11% -37.3% 2643 sched_debug.cpu.sched_goidle.avg
2673 ± 12% -47.3% 1409 ± 15% sched_debug.cpu.sched_goidle.min
5061 ± 12% -42.6% 2902 ± 2% sched_debug.cpu.ttwu_count.avg
2587 ± 14% -53.7% 1197 ± 3% sched_debug.cpu.ttwu_count.min
818.87 ± 18% -52.2% 391.10 ± 2% sched_debug.cpu.ttwu_local.avg
17144 ± 28% -64.4% 6106 ± 25% sched_debug.cpu.ttwu_local.max
228.22 ± 12% -20.1% 182.40 ± 2% sched_debug.cpu.ttwu_local.min
1796 ± 28% -72.7% 489.75 ± 17% sched_debug.cpu.ttwu_local.stddev
5.02 -4.6% 4.79 ± 2% perf-stat.i.MPKI
3.584e+10 -20.8% 2.839e+10 ± 2% perf-stat.i.branch-instructions
0.02 ± 5% +0.0 0.03 ± 25% perf-stat.i.branch-miss-rate%
70.15 -10.3 59.89 ± 2% perf-stat.i.cache-miss-rate%
3.988e+08 -30.7% 2.762e+08 ± 2% perf-stat.i.cache-misses
5.652e+08 -21.8% 4.419e+08 perf-stat.i.cache-references
15914 -35.2% 10304 perf-stat.i.context-switches
3.81 -14.1% 3.27 perf-stat.i.cpi
4.361e+11 -28.3% 3.126e+11 perf-stat.i.cpu-cycles
305.95 ± 3% -16.3% 255.98 ± 2% perf-stat.i.cpu-migrations
1080 +18.5% 1280 ± 6% perf-stat.i.cycles-between-cache-misses
2.472e+10 -18.0% 2.026e+10 perf-stat.i.dTLB-loads
0.00 ± 2% +0.0 0.00 ± 16% perf-stat.i.dTLB-store-miss-rate%
301820 ± 2% -15.5% 255096 ± 9% perf-stat.i.dTLB-store-misses
8.332e+09 -22.0% 6.497e+09 ± 2% perf-stat.i.dTLB-stores
77.38 -14.2 63.16 perf-stat.i.iTLB-load-miss-rate%
2037871 +6.8% 2177306 ± 2% perf-stat.i.iTLB-load-misses
762134 +85.5% 1413938 ± 4% perf-stat.i.iTLB-loads
1.129e+11 -19.1% 9.132e+10 ± 2% perf-stat.i.instructions
59923 -25.4% 44683 perf-stat.i.instructions-per-iTLB-miss
0.27 +22.8% 0.33 perf-stat.i.ipc
49321 -21.9% 38543 ± 2% perf-stat.i.minor-faults
18.01 +4.5 22.49 perf-stat.i.node-load-miss-rate%
1432415 -12.0% 1260792 perf-stat.i.node-load-misses
6775281 -28.9% 4818204 ± 2% perf-stat.i.node-loads
1.205e+08 -26.7% 88302418 ± 2% perf-stat.i.node-stores
49298 -21.8% 38548 ± 2% perf-stat.i.page-faults
5.00 -3.4% 4.84 perf-stat.overall.MPKI
0.02 ± 5% +0.0 0.03 ± 20% perf-stat.overall.branch-miss-rate%
70.65 -8.0 62.65 ± 2% perf-stat.overall.cache-miss-rate%
3.87 -11.1% 3.44 perf-stat.overall.cpi
1094 +3.8% 1135 perf-stat.overall.cycles-between-cache-misses
72.93 -12.1 60.85 perf-stat.overall.iTLB-load-miss-rate%
55490 -23.8% 42278 perf-stat.overall.instructions-per-iTLB-miss
0.26 +12.5% 0.29 perf-stat.overall.ipc
17.41 +3.4 20.83 perf-stat.overall.node-load-miss-rate%
3179 +6.5% 3387 perf-stat.overall.path-length
3.561e+10 -20.3% 2.836e+10 ± 2% perf-stat.ps.branch-instructions
3.966e+08 -30.3% 2.764e+08 ± 2% perf-stat.ps.cache-misses
5.614e+08 -21.4% 4.412e+08 perf-stat.ps.cache-references
15636 -35.3% 10116 perf-stat.ps.context-switches
4.339e+11 -27.7% 3.139e+11 perf-stat.ps.cpu-cycles
304.71 ± 3% -15.8% 256.64 ± 2% perf-stat.ps.cpu-migrations
2.457e+10 -17.6% 2.025e+10 perf-stat.ps.dTLB-loads
299980 ± 2% -15.6% 253225 ± 9% perf-stat.ps.dTLB-store-misses
8.285e+09 -21.6% 6.491e+09 ± 2% perf-stat.ps.dTLB-stores
2021880 +6.8% 2158636 ± 2% perf-stat.ps.iTLB-load-misses
750297 +85.3% 1389972 ± 4% perf-stat.ps.iTLB-loads
1.122e+11 -18.7% 9.126e+10 ± 2% perf-stat.ps.instructions
49039 -21.8% 38340 ± 2% perf-stat.ps.minor-faults
1411907 -11.6% 1248139 perf-stat.ps.node-load-misses
6699062 -29.2% 4743984 ± 2% perf-stat.ps.node-loads
1.2e+08 -26.2% 88495173 ± 2% perf-stat.ps.node-stores
49041 -21.8% 38339 ± 2% perf-stat.ps.page-faults
3.298e+13 -18.0% 2.704e+13 ± 2% perf-stat.total.instructions
69.87 ± 6% -11.9 57.98 ± 16% perf-profile.calltrace.cycles-pp.do_access
42.62 ± 6% -9.0 33.64 ± 17% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault.do_access
30.80 ± 6% -7.6 23.21 ± 18% perf-profile.calltrace.cycles-pp.do_rw_once
1.48 ± 6% -0.4 1.07 ± 28% perf-profile.calltrace.cycles-pp.rcu_all_qs._cond_resched.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
0.72 ± 4% -0.3 0.47 ± 59% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.do_rw_once
0.84 ± 6% -0.2 0.65 ± 11% perf-profile.calltrace.cycles-pp._cond_resched.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
0.86 ± 5% -0.2 0.70 ± 18% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.do_rw_once
0.00 +0.7 0.72 ± 20% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
0.00 +0.8 0.79 ± 21% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.00 +4.4 4.43 ± 12% perf-profile.calltrace.cycles-pp.rwsem_down_read_slowpath.__do_page_fault.do_page_fault.page_fault.do_access
0.00 +4.8 4.81 ± 22% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_read_slowpath.__do_page_fault.do_page_fault
0.00 +5.0 4.98 ± 23% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_read_slowpath.__do_page_fault.do_page_fault.page_fault
2.90 ± 9% +7.8 10.71 ± 16% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
3.22 ± 9% +8.3 11.56 ± 16% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
3.24 ± 9% +8.4 11.64 ± 16% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
3.43 ± 9% +8.6 12.08 ± 16% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
3.43 ± 9% +8.6 12.08 ± 16% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
3.43 ± 9% +8.7 12.08 ± 16% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
3.47 ± 9% +8.7 12.21 ± 16% perf-profile.calltrace.cycles-pp.secondary_startup_64
40.01 ± 6% -9.8 30.18 ± 18% perf-profile.children.cycles-pp.do_rw_once
0.24 ± 5% -0.0 0.20 ± 10% perf-profile.children.cycles-pp.do_anonymous_page
0.19 ± 6% -0.0 0.14 ± 9% perf-profile.children.cycles-pp.alloc_pages_vma
0.07 ± 17% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.update_sd_lb_stats
0.07 ± 17% +0.0 0.10 ± 5% perf-profile.children.cycles-pp.find_busiest_group
0.07 ± 10% +0.0 0.11 ± 28% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.08 ± 13% +0.0 0.12 ± 4% perf-profile.children.cycles-pp.load_balance
0.01 ±173% +0.0 0.06 ± 9% perf-profile.children.cycles-pp.unmap_vmas
0.01 ±173% +0.0 0.06 ± 9% perf-profile.children.cycles-pp.unmap_page_range
0.17 ± 14% +0.1 0.22 ± 21% perf-profile.children.cycles-pp.munmap
0.00 +0.1 0.06 ± 22% perf-profile.children.cycles-pp.lapic_next_deadline
0.06 ± 13% +0.1 0.12 ± 6% perf-profile.children.cycles-pp.rebalance_domains
0.22 ± 5% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.__softirqentry_text_start
0.13 ± 14% +0.1 0.22 ± 7% perf-profile.children.cycles-pp.__free_pages_ok
0.06 ± 11% +0.1 0.16 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event
0.17 ± 13% +0.1 0.27 ± 5% perf-profile.children.cycles-pp.release_pages
0.27 ± 5% +0.1 0.38 ± 8% perf-profile.children.cycles-pp.irq_exit
0.17 ± 13% +0.1 0.27 ± 7% perf-profile.children.cycles-pp.tlb_finish_mmu
0.17 ± 13% +0.1 0.27 ± 7% perf-profile.children.cycles-pp.tlb_flush_mmu
0.07 ± 10% +0.1 0.18 ± 17% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.21 ± 14% +0.1 0.33 ± 8% perf-profile.children.cycles-pp.__do_munmap
0.21 ± 15% +0.1 0.34 ± 7% perf-profile.children.cycles-pp.__vm_munmap
0.19 ± 13% +0.1 0.32 ± 7% perf-profile.children.cycles-pp.unmap_region
0.21 ± 14% +0.1 0.34 ± 7% perf-profile.children.cycles-pp.__x64_sys_munmap
0.23 ± 19% +0.2 0.40 ± 16% perf-profile.children.cycles-pp.update_cfs_group
0.11 ± 12% +0.2 0.29 ± 22% perf-profile.children.cycles-pp.menu_select
0.81 ± 13% +0.2 1.00 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.69 ± 5% +0.2 0.92 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.69 ± 5% +0.2 0.92 ± 7% perf-profile.children.cycles-pp.do_syscall_64
0.00 +4.8 4.82 ± 21% perf-profile.children.cycles-pp.osq_lock
0.00 +5.0 5.00 ± 23% perf-profile.children.cycles-pp.rwsem_optimistic_spin
0.10 ± 13% +5.0 5.12 ± 22% perf-profile.children.cycles-pp.rwsem_down_read_slowpath
2.95 ± 10% +7.9 10.83 ± 16% perf-profile.children.cycles-pp.intel_idle
3.29 ± 9% +8.5 11.76 ± 16% perf-profile.children.cycles-pp.cpuidle_enter
3.29 ± 9% +8.5 11.76 ± 16% perf-profile.children.cycles-pp.cpuidle_enter_state
3.43 ± 9% +8.6 12.08 ± 16% perf-profile.children.cycles-pp.start_secondary
3.47 ± 9% +8.7 12.21 ± 16% perf-profile.children.cycles-pp.do_idle
3.47 ± 9% +8.7 12.21 ± 16% perf-profile.children.cycles-pp.secondary_startup_64
3.47 ± 9% +8.7 12.21 ± 16% perf-profile.children.cycles-pp.cpu_startup_entry
38.36 ± 6% -9.6 28.80 ± 18% perf-profile.self.cycles-pp.do_rw_once
7.60 ± 6% -2.5 5.07 ± 18% perf-profile.self.cycles-pp.do_access
0.10 ± 5% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__handle_mm_fault
0.04 ± 59% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.update_sd_lb_stats
0.06 ± 13% +0.0 0.11 ± 25% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.11 ± 13% +0.1 0.19 ± 8% perf-profile.self.cycles-pp.__free_pages_ok
0.00 +0.1 0.09 ± 28% perf-profile.self.cycles-pp.menu_select
0.00 +0.1 0.10 ± 22% perf-profile.self.cycles-pp.cpuidle_enter_state
0.00 +0.2 0.15 ± 57% perf-profile.self.cycles-pp.rwsem_optimistic_spin
0.23 ± 19% +0.2 0.40 ± 16% perf-profile.self.cycles-pp.update_cfs_group
0.00 +4.8 4.77 ± 21% perf-profile.self.cycles-pp.osq_lock
2.94 ± 10% +7.9 10.83 ± 16% perf-profile.self.cycles-pp.intel_idle
30379 ± 4% -12.5% 26581 softirqs.CPU0.RCU
17842 ± 4% +62.7% 29021 ± 8% softirqs.CPU0.SCHED
12748 ± 10% +62.8% 20757 ± 13% softirqs.CPU1.SCHED
112729 ± 4% +8.4% 122207 softirqs.CPU1.TIMER
32056 ± 3% -14.0% 27582 ± 2% softirqs.CPU10.RCU
13372 ± 7% +48.7% 19891 ± 15% softirqs.CPU10.SCHED
13163 ± 13% +56.7% 20628 ± 17% softirqs.CPU100.SCHED
13220 ± 8% +56.5% 20689 ± 16% softirqs.CPU101.SCHED
111761 ± 5% +8.5% 121220 softirqs.CPU101.TIMER
13280 ± 10% +56.7% 20806 ± 14% softirqs.CPU102.SCHED
31322 ± 5% -11.2% 27817 softirqs.CPU103.RCU
13245 ± 8% +54.5% 20461 ± 14% softirqs.CPU103.SCHED
13470 ± 9% +55.2% 20910 ± 17% softirqs.CPU104.SCHED
31043 ± 5% -11.0% 27631 softirqs.CPU105.RCU
13171 ± 8% +57.6% 20753 ± 16% softirqs.CPU105.SCHED
32204 ± 5% -14.8% 27443 softirqs.CPU106.RCU
13560 ± 4% +51.8% 20590 ± 17% softirqs.CPU106.SCHED
31484 ± 5% -8.8% 28711 ± 4% softirqs.CPU107.RCU
13376 ± 9% +52.6% 20415 ± 17% softirqs.CPU107.SCHED
13179 ± 9% +53.2% 20196 ± 15% softirqs.CPU108.SCHED
30932 ± 5% -9.5% 28009 ± 3% softirqs.CPU109.RCU
13315 ± 9% +53.1% 20390 ± 16% softirqs.CPU109.SCHED
31762 ± 4% -10.3% 28485 softirqs.CPU11.RCU
13133 ± 8% +56.5% 20559 ± 15% softirqs.CPU11.SCHED
31055 ± 4% -11.0% 27630 softirqs.CPU110.RCU
13271 ± 9% +54.0% 20443 ± 17% softirqs.CPU110.SCHED
31417 ± 4% -12.9% 27357 softirqs.CPU111.RCU
13352 ± 9% +53.5% 20500 ± 17% softirqs.CPU111.SCHED
31743 ± 4% -14.0% 27295 softirqs.CPU112.RCU
13334 ± 9% +53.4% 20461 ± 16% softirqs.CPU112.SCHED
31649 ± 4% -12.2% 27796 softirqs.CPU113.RCU
13586 ± 9% +51.2% 20542 ± 16% softirqs.CPU113.SCHED
32529 ± 8% -13.3% 28213 ± 2% softirqs.CPU114.RCU
13423 ± 8% +51.9% 20388 ± 16% softirqs.CPU114.SCHED
31879 ± 6% -13.3% 27646 softirqs.CPU115.RCU
13357 ± 9% +53.0% 20442 ± 17% softirqs.CPU115.SCHED
31622 ± 4% -9.7% 28569 ± 5% softirqs.CPU116.RCU
13626 ± 9% +52.6% 20789 ± 14% softirqs.CPU116.SCHED
31089 ± 5% -11.8% 27412 softirqs.CPU117.RCU
12525 ± 14% +66.6% 20866 ± 17% softirqs.CPU117.SCHED
13348 ± 9% +55.9% 20815 ± 17% softirqs.CPU118.SCHED
112643 ± 4% +8.5% 122261 softirqs.CPU118.TIMER
31601 ± 4% -12.8% 27540 softirqs.CPU119.RCU
13363 ± 8% +49.6% 19995 ± 16% softirqs.CPU119.SCHED
33443 ± 12% -14.6% 28564 softirqs.CPU12.RCU
13720 ± 12% +49.4% 20492 ± 14% softirqs.CPU12.SCHED
31831 ± 5% -15.2% 26996 ± 2% softirqs.CPU120.RCU
12720 ± 9% +76.8% 22491 ± 2% softirqs.CPU120.SCHED
32010 ± 3% -15.3% 27101 ± 2% softirqs.CPU121.RCU
12802 ± 11% +75.2% 22431 ± 3% softirqs.CPU121.SCHED
30635 ± 5% -8.7% 27972 ± 6% softirqs.CPU122.RCU
12666 ± 12% +80.5% 22856 ± 4% softirqs.CPU122.SCHED
31969 ± 2% -15.4% 27041 ± 3% softirqs.CPU123.RCU
12565 ± 12% +79.2% 22518 ± 3% softirqs.CPU123.SCHED
31849 ± 3% -15.4% 26938 ± 2% softirqs.CPU124.RCU
12703 ± 10% +75.7% 22323 ± 3% softirqs.CPU124.SCHED
32084 ± 3% -15.6% 27087 ± 2% softirqs.CPU125.RCU
12639 ± 10% +76.3% 22280 ± 2% softirqs.CPU125.SCHED
31975 ± 3% -15.7% 26959 ± 2% softirqs.CPU126.RCU
12668 ± 9% +72.4% 21836 ± 5% softirqs.CPU126.SCHED
31527 ± 5% -14.3% 27020 ± 2% softirqs.CPU127.RCU
12542 ± 9% +79.4% 22495 ± 3% softirqs.CPU127.SCHED
31366 ± 2% -12.8% 27348 ± 2% softirqs.CPU128.RCU
12644 ± 11% +76.6% 22330 ± 3% softirqs.CPU128.SCHED
31189 ± 2% -11.8% 27499 ± 2% softirqs.CPU129.RCU
12696 ± 9% +77.1% 22480 softirqs.CPU129.SCHED
13020 ± 8% +56.6% 20385 ± 15% softirqs.CPU13.SCHED
31251 ± 3% -11.5% 27664 ± 2% softirqs.CPU130.RCU
12720 ± 9% +75.6% 22337 ± 3% softirqs.CPU130.SCHED
31795 -13.0% 27663 ± 2% softirqs.CPU131.RCU
12715 ± 11% +75.1% 22266 ± 2% softirqs.CPU131.SCHED
31456 ± 3% -12.1% 27662 ± 3% softirqs.CPU132.RCU
12661 ± 10% +76.7% 22370 ± 2% softirqs.CPU132.SCHED
30812 ± 4% -12.6% 26922 ± 3% softirqs.CPU133.RCU
12542 ± 12% +82.2% 22851 ± 5% softirqs.CPU133.SCHED
31150 ± 2% -12.7% 27197 ± 3% softirqs.CPU134.RCU
12669 ± 11% +74.1% 22063 ± 3% softirqs.CPU134.SCHED
31286 ± 3% -11.6% 27648 ± 2% softirqs.CPU135.RCU
12685 ± 9% +80.7% 22921 ± 3% softirqs.CPU135.SCHED
31466 ± 2% -12.3% 27610 ± 3% softirqs.CPU136.RCU
12643 ± 10% +78.4% 22560 ± 2% softirqs.CPU136.SCHED
31550 ± 2% -12.9% 27464 ± 2% softirqs.CPU137.RCU
12744 ± 9% +75.0% 22309 ± 2% softirqs.CPU137.SCHED
31343 ± 3% -12.5% 27418 ± 2% softirqs.CPU138.RCU
12741 ± 10% +68.8% 21506 ± 12% softirqs.CPU138.SCHED
31628 ± 2% -13.0% 27505 ± 2% softirqs.CPU139.RCU
12659 ± 11% +76.9% 22398 ± 6% softirqs.CPU139.SCHED
33226 ± 9% -16.0% 27898 softirqs.CPU14.RCU
13074 ± 7% +51.3% 19779 ± 16% softirqs.CPU14.SCHED
31554 ± 3% -11.3% 27993 ± 3% softirqs.CPU140.RCU
12730 ± 9% +78.0% 22663 ± 4% softirqs.CPU140.SCHED
12497 ± 13% +82.2% 22769 ± 3% softirqs.CPU141.SCHED
31195 ± 4% -12.3% 27362 ± 4% softirqs.CPU142.RCU
12640 ± 12% +81.9% 22989 ± 6% softirqs.CPU142.SCHED
31194 ± 4% -13.2% 27074 ± 4% softirqs.CPU143.RCU
12716 ± 11% +72.5% 21937 ± 13% softirqs.CPU143.SCHED
31035 ± 4% -9.9% 27958 ± 2% softirqs.CPU144.RCU
13145 ± 8% +72.0% 22605 ± 7% softirqs.CPU144.SCHED
31116 ± 3% -11.4% 27567 softirqs.CPU145.RCU
13086 ± 8% +70.4% 22297 ± 6% softirqs.CPU145.SCHED
30996 ± 3% -13.6% 26791 ± 2% softirqs.CPU146.RCU
13084 ± 8% +58.7% 20761 ± 19% softirqs.CPU146.SCHED
30902 ± 3% -10.1% 27769 softirqs.CPU147.RCU
13209 ± 8% +69.5% 22392 ± 6% softirqs.CPU147.SCHED
30816 ± 3% -9.3% 27938 softirqs.CPU148.RCU
13056 ± 7% +74.1% 22731 ± 7% softirqs.CPU148.SCHED
30876 ± 3% -9.6% 27926 softirqs.CPU149.RCU
13104 ± 7% +66.5% 21813 ± 11% softirqs.CPU149.SCHED
32201 ± 6% -13.2% 27952 ± 2% softirqs.CPU15.RCU
13135 ± 6% +57.2% 20646 ± 14% softirqs.CPU15.SCHED
30836 ± 3% -10.0% 27752 softirqs.CPU150.RCU
13288 ± 6% +65.0% 21919 ± 10% softirqs.CPU150.SCHED
31074 ± 3% -11.4% 27531 softirqs.CPU151.RCU
13178 ± 8% +70.8% 22506 ± 9% softirqs.CPU151.SCHED
30955 ± 3% -10.2% 27786 softirqs.CPU152.RCU
13182 ± 8% +75.6% 23155 ± 2% softirqs.CPU152.SCHED
31032 ± 3% -13.3% 26914 ± 4% softirqs.CPU153.RCU
13156 ± 8% +69.5% 22293 ± 7% softirqs.CPU153.SCHED
31190 ± 3% -11.5% 27609 softirqs.CPU154.RCU
13279 ± 7% +67.4% 22223 ± 7% softirqs.CPU154.SCHED
31230 ± 2% -10.7% 27889 softirqs.CPU155.RCU
13298 ± 5% +69.1% 22485 ± 6% softirqs.CPU155.SCHED
31098 ± 2% -10.9% 27722 softirqs.CPU156.RCU
13306 ± 6% +71.6% 22834 ± 7% softirqs.CPU156.SCHED
31711 ± 6% -16.7% 26402 ± 3% softirqs.CPU157.RCU
13212 ± 7% +74.3% 23031 ± 8% softirqs.CPU157.SCHED
30982 ± 3% -11.6% 27383 softirqs.CPU158.RCU
13159 ± 7% +71.6% 22585 ± 8% softirqs.CPU158.SCHED
30935 ± 3% -9.9% 27859 softirqs.CPU159.RCU
13305 ± 8% +71.5% 22816 ± 6% softirqs.CPU159.SCHED
32162 ± 4% -12.6% 28095 softirqs.CPU16.RCU
12923 ± 8% +54.4% 19953 ± 16% softirqs.CPU16.SCHED
31177 ± 3% -10.7% 27829 ± 4% softirqs.CPU160.RCU
13060 ± 8% +75.9% 22970 ± 7% softirqs.CPU160.SCHED
31499 ± 3% -12.5% 27562 ± 3% softirqs.CPU161.RCU
13111 ± 8% +76.9% 23190 ± 8% softirqs.CPU161.SCHED
30994 ± 2% -10.1% 27854 ± 4% softirqs.CPU162.RCU
13231 ± 9% +74.1% 23036 ± 8% softirqs.CPU162.SCHED
30973 ± 2% -11.4% 27428 ± 3% softirqs.CPU163.RCU
13239 ± 8% +73.0% 22902 ± 8% softirqs.CPU163.SCHED
30916 ± 3% -10.8% 27588 ± 4% softirqs.CPU164.RCU
13376 ± 9% +74.0% 23279 ± 8% softirqs.CPU164.SCHED
30922 ± 3% -11.9% 27245 ± 3% softirqs.CPU165.RCU
13124 ± 8% +72.9% 22694 ± 9% softirqs.CPU165.SCHED
30990 ± 2% -11.2% 27508 ± 3% softirqs.CPU166.RCU
13235 ± 8% +74.7% 23116 ± 8% softirqs.CPU166.SCHED
30940 ± 3% -11.4% 27423 ± 3% softirqs.CPU167.RCU
13241 ± 8% +68.6% 22319 ± 6% softirqs.CPU167.SCHED
31837 ± 7% -15.2% 27013 ± 2% softirqs.CPU168.RCU
11975 ± 8% +51.5% 18143 ± 19% softirqs.CPU168.SCHED
32905 ± 9% -17.5% 27162 ± 2% softirqs.CPU169.RCU
12075 ± 10% +46.4% 17674 ± 20% softirqs.CPU169.SCHED
32180 ± 4% -11.8% 28372 softirqs.CPU17.RCU
12701 ± 8% +58.9% 20186 ± 16% softirqs.CPU17.SCHED
32128 ± 8% -16.3% 26892 ± 3% softirqs.CPU170.RCU
12379 ± 11% +43.9% 17817 ± 19% softirqs.CPU170.SCHED
31840 ± 6% -15.0% 27057 ± 2% softirqs.CPU171.RCU
12030 ± 10% +48.8% 17901 ± 19% softirqs.CPU171.SCHED
31612 ± 7% -13.1% 27469 ± 4% softirqs.CPU172.RCU
12139 ± 10% +48.1% 17974 ± 20% softirqs.CPU172.SCHED
31799 ± 7% -14.4% 27217 ± 3% softirqs.CPU173.RCU
12137 ± 12% +47.7% 17923 ± 19% softirqs.CPU173.SCHED
31879 ± 6% -15.2% 27038 ± 3% softirqs.CPU174.RCU
12235 ± 12% +47.0% 17985 ± 20% softirqs.CPU174.SCHED
32162 ± 6% -14.4% 27533 ± 3% softirqs.CPU175.RCU
12298 ± 15% +46.7% 18043 ± 18% softirqs.CPU175.SCHED
30774 ± 6% -11.1% 27366 ± 2% softirqs.CPU176.RCU
12026 ± 11% +48.0% 17803 ± 19% softirqs.CPU176.SCHED
30679 ± 4% -11.5% 27156 ± 2% softirqs.CPU177.RCU
12176 ± 13% +45.5% 17714 ± 20% softirqs.CPU177.SCHED
12140 ± 11% +48.8% 18069 ± 19% softirqs.CPU178.SCHED
12211 ± 12% +47.4% 18003 ± 19% softirqs.CPU179.SCHED
32382 ± 4% -12.2% 28441 softirqs.CPU18.RCU
12902 ± 6% +57.4% 20303 ± 15% softirqs.CPU18.SCHED
31075 ± 5% -11.9% 27369 ± 3% softirqs.CPU180.RCU
12165 ± 12% +46.6% 17839 ± 20% softirqs.CPU180.SCHED
30964 ± 4% -12.0% 27238 ± 2% softirqs.CPU181.RCU
12248 ± 12% +47.8% 18098 ± 22% softirqs.CPU181.SCHED
31103 ± 4% -11.8% 27425 softirqs.CPU182.RCU
12424 ± 12% +43.3% 17805 ± 19% softirqs.CPU182.SCHED
30938 ± 5% -10.5% 27675 ± 2% softirqs.CPU183.RCU
12480 ± 16% +42.5% 17782 ± 18% softirqs.CPU183.SCHED
12169 ± 12% +47.7% 17972 ± 20% softirqs.CPU184.SCHED
30878 ± 5% -11.6% 27312 ± 2% softirqs.CPU185.RCU
12060 ± 13% +49.6% 18037 ± 20% softirqs.CPU185.SCHED
31185 ± 6% -13.0% 27132 ± 4% softirqs.CPU186.RCU
12442 ± 11% +44.3% 17958 ± 20% softirqs.CPU186.SCHED
30816 ± 6% -9.9% 27753 ± 2% softirqs.CPU187.RCU
11967 ± 13% +51.8% 18169 ± 19% softirqs.CPU187.SCHED
30741 ± 5% -10.3% 27590 softirqs.CPU188.RCU
12301 ± 11% +47.0% 18077 ± 18% softirqs.CPU188.SCHED
30878 ± 5% -11.1% 27437 softirqs.CPU189.RCU
12403 ± 11% +46.1% 18125 ± 19% softirqs.CPU189.SCHED
33549 ± 9% -15.5% 28337 softirqs.CPU19.RCU
13145 ± 8% +56.0% 20502 ± 15% softirqs.CPU19.SCHED
12217 ± 12% +45.2% 17734 ± 20% softirqs.CPU190.SCHED
31599 ± 4% -12.2% 27731 ± 2% softirqs.CPU191.RCU
12201 ± 13% +46.1% 17832 ± 20% softirqs.CPU191.SCHED
13107 ± 9% +57.7% 20673 ± 15% softirqs.CPU2.SCHED
32171 ± 4% -8.8% 29343 ± 4% softirqs.CPU20.RCU
13049 ± 7% +57.4% 20544 ± 15% softirqs.CPU20.SCHED
31950 ± 4% -12.1% 28099 softirqs.CPU21.RCU
13103 ± 7% +56.7% 20536 ± 16% softirqs.CPU21.SCHED
112050 ± 5% +9.1% 122248 softirqs.CPU21.TIMER
32375 ± 8% -12.1% 28462 softirqs.CPU22.RCU
13126 ± 8% +57.0% 20615 ± 15% softirqs.CPU22.SCHED
32092 ± 4% -11.6% 28371 softirqs.CPU23.RCU
12677 ± 12% +65.3% 20955 ± 17% softirqs.CPU23.SCHED
32554 ± 7% -12.0% 28659 softirqs.CPU24.RCU
12382 ± 10% +80.6% 22358 ± 3% softirqs.CPU24.SCHED
32092 ± 3% -13.9% 27628 ± 2% softirqs.CPU25.RCU
12271 ± 10% +81.6% 22280 ± 5% softirqs.CPU25.SCHED
31747 ± 4% -14.1% 27267 ± 4% softirqs.CPU26.RCU
12617 ± 7% +76.2% 22233 ± 4% softirqs.CPU26.SCHED
32013 ± 3% -13.6% 27673 ± 2% softirqs.CPU27.RCU
12501 ± 9% +80.0% 22496 ± 3% softirqs.CPU27.SCHED
32129 ± 3% -12.1% 28232 ± 2% softirqs.CPU28.RCU
12719 ± 9% +76.8% 22482 ± 4% softirqs.CPU28.SCHED
32415 ± 3% -14.2% 27797 ± 2% softirqs.CPU29.RCU
12529 ± 9% +77.8% 22284 ± 3% softirqs.CPU29.SCHED
31703 ± 5% -9.7% 28624 softirqs.CPU3.RCU
13071 ± 7% +56.5% 20460 ± 18% softirqs.CPU3.SCHED
32136 ± 3% -13.5% 27802 softirqs.CPU30.RCU
12408 ± 11% +82.4% 22632 ± 3% softirqs.CPU30.SCHED
32766 ± 2% -14.7% 27942 ± 2% softirqs.CPU31.RCU
12782 ± 10% +69.8% 21700 ± 3% softirqs.CPU31.SCHED
32114 ± 2% -13.2% 27885 ± 2% softirqs.CPU32.RCU
12565 ± 10% +76.9% 22227 ± 3% softirqs.CPU32.SCHED
31908 -10.9% 28428 ± 5% softirqs.CPU33.RCU
12470 ± 11% +81.9% 22679 softirqs.CPU33.SCHED
31862 ± 2% -12.2% 27972 ± 3% softirqs.CPU34.RCU
12441 ± 11% +79.4% 22324 ± 3% softirqs.CPU34.SCHED
32334 ± 2% -13.0% 28127 ± 2% softirqs.CPU35.RCU
12571 ± 10% +78.0% 22380 ± 2% softirqs.CPU35.SCHED
31808 ± 3% -11.3% 28213 ± 3% softirqs.CPU36.RCU
12634 ± 9% +76.1% 22245 ± 2% softirqs.CPU36.SCHED
31475 ± 4% -12.0% 27691 ± 2% softirqs.CPU37.RCU
12674 ± 9% +75.3% 22217 ± 2% softirqs.CPU37.SCHED
31654 -11.4% 28031 ± 3% softirqs.CPU38.RCU
12380 ± 9% +77.6% 21990 ± 2% softirqs.CPU38.SCHED
31846 ± 3% -11.1% 28313 ± 3% softirqs.CPU39.RCU
12426 ± 9% +80.6% 22446 ± 3% softirqs.CPU39.SCHED
12972 ± 6% +53.7% 19940 ± 17% softirqs.CPU4.SCHED
112980 ± 5% +7.3% 121215 ± 4% softirqs.CPU4.TIMER
31924 ± 2% -12.5% 27919 ± 3% softirqs.CPU40.RCU
12607 ± 8% +76.4% 22236 ± 4% softirqs.CPU40.SCHED
31979 ± 2% -13.2% 27768 ± 2% softirqs.CPU41.RCU
12515 ± 9% +83.2% 22925 ± 6% softirqs.CPU41.SCHED
31649 ± 2% -11.4% 28056 ± 2% softirqs.CPU42.RCU
12409 ± 10% +82.5% 22643 ± 2% softirqs.CPU42.SCHED
31892 ± 2% -12.3% 27980 ± 3% softirqs.CPU43.RCU
12459 ± 9% +77.6% 22123 ± 5% softirqs.CPU43.SCHED
31735 ± 2% -11.7% 28017 ± 2% softirqs.CPU44.RCU
12510 ± 9% +79.4% 22438 ± 2% softirqs.CPU44.SCHED
31322 ± 2% -11.7% 27656 ± 3% softirqs.CPU45.RCU
12601 ± 9% +73.3% 21837 ± 2% softirqs.CPU45.SCHED
31661 ± 3% -11.6% 27979 ± 4% softirqs.CPU46.RCU
12508 ± 9% +79.2% 22411 ± 3% softirqs.CPU46.SCHED
31532 ± 3% -11.8% 27802 ± 2% softirqs.CPU47.RCU
12568 ± 8% +78.5% 22440 ± 2% softirqs.CPU47.SCHED
32597 ± 3% -13.9% 28075 softirqs.CPU48.RCU
12838 ± 6% +74.5% 22404 ± 6% softirqs.CPU48.SCHED
31583 ± 3% -11.8% 27858 softirqs.CPU49.RCU
12826 ± 6% +74.7% 22407 ± 4% softirqs.CPU49.SCHED
32127 ± 7% -11.5% 28427 softirqs.CPU5.RCU
13560 ± 10% +50.6% 20420 ± 15% softirqs.CPU5.SCHED
31596 ± 3% -12.9% 27527 softirqs.CPU50.RCU
12964 ± 7% +73.7% 22515 ± 4% softirqs.CPU50.SCHED
31453 ± 3% -10.5% 28160 softirqs.CPU51.RCU
12739 ± 6% +73.9% 22153 ± 7% softirqs.CPU51.SCHED
12804 ± 6% +75.9% 22528 ± 6% softirqs.CPU52.SCHED
31450 ± 2% -10.4% 28167 softirqs.CPU53.RCU
12942 ± 6% +75.0% 22651 ± 7% softirqs.CPU53.SCHED
31441 ± 3% -10.1% 28251 softirqs.CPU54.RCU
13105 ± 3% +71.3% 22453 ± 7% softirqs.CPU54.SCHED
31603 ± 3% -11.0% 28132 softirqs.CPU55.RCU
12777 ± 7% +77.2% 22637 ± 8% softirqs.CPU55.SCHED
31555 ± 3% -11.1% 28066 softirqs.CPU56.RCU
12777 ± 6% +75.1% 22376 ± 6% softirqs.CPU56.SCHED
31436 ± 3% -12.0% 27671 ± 3% softirqs.CPU57.RCU
12781 ± 6% +77.2% 22649 ± 6% softirqs.CPU57.SCHED
31503 ± 3% -10.7% 28139 softirqs.CPU58.RCU
12954 ± 6% +75.3% 22709 ± 7% softirqs.CPU58.SCHED
31839 ± 3% -10.9% 28371 ± 2% softirqs.CPU59.RCU
13236 ± 7% +71.1% 22653 ± 6% softirqs.CPU59.SCHED
13223 ± 8% +52.4% 20148 ± 16% softirqs.CPU6.SCHED
32616 -13.7% 28163 softirqs.CPU60.RCU
13139 +73.8% 22841 ± 7% softirqs.CPU60.SCHED
31250 ± 3% -11.5% 27660 ± 3% softirqs.CPU61.RCU
12835 ± 6% +76.8% 22694 ± 7% softirqs.CPU61.SCHED
31480 ± 3% -11.3% 27922 softirqs.CPU62.RCU
12962 ± 6% +72.4% 22345 ± 6% softirqs.CPU62.SCHED
31567 ± 3% -10.2% 28342 ± 2% softirqs.CPU63.RCU
12909 ± 7% +74.7% 22556 ± 6% softirqs.CPU63.SCHED
30655 ± 5% -14.7% 26163 softirqs.CPU64.RCU
12793 ± 6% +75.5% 22452 ± 7% softirqs.CPU64.SCHED
30643 ± 5% -13.8% 26417 softirqs.CPU65.RCU
12839 ± 7% +73.3% 22251 ± 7% softirqs.CPU65.SCHED
30720 ± 4% -14.6% 26249 softirqs.CPU66.RCU
12754 ± 6% +75.7% 22415 ± 7% softirqs.CPU66.SCHED
30650 ± 4% -14.2% 26294 softirqs.CPU67.RCU
12760 ± 6% +77.6% 22659 ± 8% softirqs.CPU67.SCHED
30537 ± 5% -13.3% 26480 ± 2% softirqs.CPU68.RCU
12927 ± 7% +75.6% 22699 ± 7% softirqs.CPU68.SCHED
30702 ± 5% -14.5% 26253 ± 2% softirqs.CPU69.RCU
12814 ± 7% +79.8% 23040 ± 9% softirqs.CPU69.SCHED
12743 ± 10% +61.6% 20591 ± 13% softirqs.CPU7.SCHED
30578 ± 4% -13.8% 26345 softirqs.CPU70.RCU
12985 ± 8% +74.9% 22711 ± 8% softirqs.CPU70.SCHED
30596 ± 4% -14.2% 26259 ± 2% softirqs.CPU71.RCU
12903 ± 7% +75.1% 22588 ± 7% softirqs.CPU71.SCHED
31963 ± 10% -17.8% 26279 softirqs.CPU72.RCU
12353 ± 6% +52.5% 18835 ± 10% softirqs.CPU72.SCHED
32117 ± 12% -18.9% 26039 softirqs.CPU73.RCU
12342 ± 9% +45.4% 17944 ± 16% softirqs.CPU73.SCHED
32473 ± 12% -19.9% 26000 ± 2% softirqs.CPU74.RCU
12308 ± 8% +47.5% 18160 ± 16% softirqs.CPU74.SCHED
31632 ± 10% -16.9% 26275 ± 2% softirqs.CPU75.RCU
12134 ± 9% +46.8% 17808 ± 17% softirqs.CPU75.SCHED
31589 ± 10% -17.2% 26141 ± 2% softirqs.CPU76.RCU
11922 ± 11% +65.2% 19694 ± 33% softirqs.CPU76.SCHED
31630 ± 11% -17.2% 26191 ± 2% softirqs.CPU77.RCU
12062 ± 9% +48.6% 17923 ± 15% softirqs.CPU77.SCHED
32390 ± 9% -19.5% 26088 ± 2% softirqs.CPU78.RCU
12168 ± 9% +47.0% 17883 ± 16% softirqs.CPU78.SCHED
31792 ± 10% -16.7% 26488 ± 4% softirqs.CPU79.RCU
12295 ± 11% +44.8% 17807 ± 17% softirqs.CPU79.SCHED
31541 ± 5% -10.3% 28297 softirqs.CPU8.RCU
13076 ± 7% +58.5% 20721 ± 15% softirqs.CPU8.SCHED
12180 ± 10% +45.3% 17693 ± 17% softirqs.CPU80.SCHED
31450 ± 7% -12.7% 27447 ± 2% softirqs.CPU81.RCU
12339 ± 12% +42.2% 17544 ± 17% softirqs.CPU81.SCHED
12207 ± 9% +45.6% 17777 ± 18% softirqs.CPU82.SCHED
12064 ± 9% +48.7% 17943 ± 16% softirqs.CPU83.SCHED
12084 ± 8% +46.6% 17712 ± 18% softirqs.CPU84.SCHED
31146 ± 4% -10.9% 27756 ± 2% softirqs.CPU85.RCU
12156 ± 8% +47.3% 17911 ± 16% softirqs.CPU85.SCHED
31536 ± 4% -12.1% 27718 ± 3% softirqs.CPU86.RCU
12371 ± 8% +42.9% 17673 ± 18% softirqs.CPU86.SCHED
12263 ± 10% +44.0% 17662 ± 18% softirqs.CPU87.SCHED
12216 ± 9% +45.7% 17804 ± 18% softirqs.CPU88.SCHED
31424 ± 5% -11.8% 27724 ± 2% softirqs.CPU89.RCU
12379 ± 8% +44.2% 17853 ± 16% softirqs.CPU89.SCHED
31423 ± 5% -10.3% 28191 softirqs.CPU9.RCU
13188 ± 5% +58.2% 20864 ± 14% softirqs.CPU9.SCHED
113341 ± 4% +8.2% 122664 softirqs.CPU9.TIMER
31525 ± 5% -12.5% 27585 ± 4% softirqs.CPU90.RCU
12345 ± 8% +45.0% 17895 ± 20% softirqs.CPU90.SCHED
11971 ± 8% +49.7% 17922 ± 18% softirqs.CPU91.SCHED
12212 ± 9% +47.2% 17979 ± 16% softirqs.CPU92.SCHED
12305 ± 10% +46.2% 17988 ± 18% softirqs.CPU93.SCHED
31246 ± 5% -11.2% 27734 ± 2% softirqs.CPU94.RCU
12173 ± 9% +48.9% 18124 ± 19% softirqs.CPU94.SCHED
31258 ± 5% -11.0% 27824 ± 2% softirqs.CPU95.RCU
12187 ± 11% +47.6% 17988 ± 19% softirqs.CPU95.SCHED
31677 ± 4% -15.7% 26719 ± 2% softirqs.CPU96.RCU
13001 ± 7% +57.1% 20420 ± 15% softirqs.CPU96.SCHED
31379 ± 6% -11.8% 27678 ± 2% softirqs.CPU97.RCU
12869 ± 9% +62.5% 20909 ± 14% softirqs.CPU97.SCHED
110944 ± 5% +10.3% 122392 softirqs.CPU97.TIMER
33271 ± 7% -16.0% 27945 softirqs.CPU98.RCU
13545 ± 7% +56.5% 21193 ± 15% softirqs.CPU98.SCHED
31522 ± 5% -10.9% 28072 softirqs.CPU99.RCU
13760 ± 7% +50.1% 20650 ± 16% softirqs.CPU99.SCHED
6052777 ± 4% -12.3% 5310987 softirqs.RCU
2454586 +63.5% 4012658 ± 2% softirqs.SCHED
5164839 ± 4% -44.0% 2891677 ± 4% interrupts.CAL:Function_call_interrupts
23211 ± 5% -49.8% 11662 ± 14% interrupts.CPU0.CAL:Function_call_interrupts
19793 ± 4% -56.4% 8622 ± 20% interrupts.CPU0.TLB:TLB_shootdowns
25756 ± 6% -48.9% 13159 ± 6% interrupts.CPU1.CAL:Function_call_interrupts
6216 ± 6% -29.9% 4356 ± 21% interrupts.CPU1.NMI:Non-maskable_interrupts
6216 ± 6% -29.9% 4356 ± 21% interrupts.CPU1.PMI:Performance_monitoring_interrupts
22359 ± 5% -54.3% 10221 ± 8% interrupts.CPU1.TLB:TLB_shootdowns
25979 ± 5% -44.8% 14345 ± 10% interrupts.CPU10.CAL:Function_call_interrupts
6067 ± 5% -29.7% 4265 ± 36% interrupts.CPU10.NMI:Non-maskable_interrupts
6067 ± 5% -29.7% 4265 ± 36% interrupts.CPU10.PMI:Performance_monitoring_interrupts
22666 ± 4% -49.6% 11419 ± 11% interrupts.CPU10.TLB:TLB_shootdowns
26575 ± 9% -45.8% 14399 ± 6% interrupts.CPU100.CAL:Function_call_interrupts
23233 ± 11% -50.7% 11457 ± 8% interrupts.CPU100.TLB:TLB_shootdowns
27400 ± 6% -45.7% 14890 ± 8% interrupts.CPU101.CAL:Function_call_interrupts
24034 ± 7% -50.1% 11999 ± 10% interrupts.CPU101.TLB:TLB_shootdowns
27415 ± 9% -44.7% 15151 ± 6% interrupts.CPU102.CAL:Function_call_interrupts
6287 ± 4% -26.4% 4625 ± 16% interrupts.CPU102.NMI:Non-maskable_interrupts
6287 ± 4% -26.4% 4625 ± 16% interrupts.CPU102.PMI:Performance_monitoring_interrupts
24027 ± 11% -49.2% 12215 ± 8% interrupts.CPU102.TLB:TLB_shootdowns
27495 ± 7% -47.1% 14548 ± 7% interrupts.CPU103.CAL:Function_call_interrupts
24192 ± 6% -52.3% 11541 ± 9% interrupts.CPU103.TLB:TLB_shootdowns
27461 ± 7% -44.9% 15142 ± 9% interrupts.CPU104.CAL:Function_call_interrupts
6105 ± 6% -43.1% 3471 ± 39% interrupts.CPU104.NMI:Non-maskable_interrupts
6105 ± 6% -43.1% 3471 ± 39% interrupts.CPU104.PMI:Performance_monitoring_interrupts
24138 ± 6% -49.7% 12153 ± 11% interrupts.CPU104.TLB:TLB_shootdowns
26900 ± 7% -45.0% 14796 ± 7% interrupts.CPU105.CAL:Function_call_interrupts
23655 ± 7% -50.1% 11793 ± 8% interrupts.CPU105.TLB:TLB_shootdowns
26032 ± 9% -42.6% 14934 ± 10% interrupts.CPU106.CAL:Function_call_interrupts
6015 ± 6% -33.6% 3994 ± 18% interrupts.CPU106.NMI:Non-maskable_interrupts
6015 ± 6% -33.6% 3994 ± 18% interrupts.CPU106.PMI:Performance_monitoring_interrupts
22884 ± 8% -47.4% 12037 ± 12% interrupts.CPU106.TLB:TLB_shootdowns
27375 ± 5% -43.6% 15441 ± 9% interrupts.CPU107.CAL:Function_call_interrupts
24034 ± 6% -48.1% 12472 ± 12% interrupts.CPU107.TLB:TLB_shootdowns
27473 ± 6% -45.5% 14969 ± 8% interrupts.CPU108.CAL:Function_call_interrupts
24119 ± 6% -50.4% 11962 ± 10% interrupts.CPU108.TLB:TLB_shootdowns
27681 ± 9% -43.6% 15612 ± 8% interrupts.CPU109.CAL:Function_call_interrupts
24298 ± 9% -48.0% 12638 ± 10% interrupts.CPU109.TLB:TLB_shootdowns
27016 ± 7% -44.6% 14956 ± 10% interrupts.CPU11.CAL:Function_call_interrupts
6075 ± 6% -26.1% 4487 ± 22% interrupts.CPU11.NMI:Non-maskable_interrupts
6075 ± 6% -26.1% 4487 ± 22% interrupts.CPU11.PMI:Performance_monitoring_interrupts
23697 ± 8% -49.4% 12001 ± 13% interrupts.CPU11.TLB:TLB_shootdowns
27375 ± 6% -44.3% 15248 ± 9% interrupts.CPU110.CAL:Function_call_interrupts
6121 ± 5% -28.7% 4364 ± 34% interrupts.CPU110.NMI:Non-maskable_interrupts
6121 ± 5% -28.7% 4364 ± 34% interrupts.CPU110.PMI:Performance_monitoring_interrupts
24007 ± 7% -48.9% 12263 ± 12% interrupts.CPU110.TLB:TLB_shootdowns
27096 ± 5% -45.0% 14899 ± 9% interrupts.CPU111.CAL:Function_call_interrupts
23791 ± 5% -49.9% 11920 ± 12% interrupts.CPU111.TLB:TLB_shootdowns
27578 ± 7% -46.7% 14694 ± 10% interrupts.CPU112.CAL:Function_call_interrupts
6310 ± 7% -29.2% 4468 ± 17% interrupts.CPU112.NMI:Non-maskable_interrupts
6310 ± 7% -29.2% 4468 ± 17% interrupts.CPU112.PMI:Performance_monitoring_interrupts
24277 ± 7% -51.7% 11726 ± 12% interrupts.CPU112.TLB:TLB_shootdowns
28568 ± 4% -48.0% 14870 ± 7% interrupts.CPU113.CAL:Function_call_interrupts
6171 ± 7% -43.9% 3464 ± 39% interrupts.CPU113.NMI:Non-maskable_interrupts
6171 ± 7% -43.9% 3464 ± 39% interrupts.CPU113.PMI:Performance_monitoring_interrupts
25339 ± 5% -53.0% 11908 ± 9% interrupts.CPU113.TLB:TLB_shootdowns
27869 ± 7% -47.3% 14690 ± 11% interrupts.CPU114.CAL:Function_call_interrupts
24553 ± 6% -52.5% 11664 ± 14% interrupts.CPU114.TLB:TLB_shootdowns
27168 ± 5% -44.2% 15162 ± 9% interrupts.CPU115.CAL:Function_call_interrupts
23784 ± 5% -48.7% 12203 ± 10% interrupts.CPU115.TLB:TLB_shootdowns
27674 ± 7% -44.4% 15395 ± 10% interrupts.CPU116.CAL:Function_call_interrupts
24359 ± 7% -49.2% 12386 ± 13% interrupts.CPU116.TLB:TLB_shootdowns
25881 ± 15% -41.8% 15054 ± 11% interrupts.CPU117.CAL:Function_call_interrupts
22516 ± 18% -46.2% 12105 ± 13% interrupts.CPU117.TLB:TLB_shootdowns
27172 ± 6% -43.8% 15279 ± 7% interrupts.CPU118.CAL:Function_call_interrupts
23916 ± 6% -48.6% 12298 ± 9% interrupts.CPU118.TLB:TLB_shootdowns
27108 ± 7% -46.3% 14548 ± 13% interrupts.CPU119.CAL:Function_call_interrupts
6164 ± 4% -28.8% 4388 ± 21% interrupts.CPU119.NMI:Non-maskable_interrupts
6164 ± 4% -28.8% 4388 ± 21% interrupts.CPU119.PMI:Performance_monitoring_interrupts
23830 ± 7% -51.4% 11571 ± 17% interrupts.CPU119.TLB:TLB_shootdowns
26907 ± 5% -43.4% 15217 ± 8% interrupts.CPU12.CAL:Function_call_interrupts
6205 ± 5% -32.9% 4164 ± 34% interrupts.CPU12.NMI:Non-maskable_interrupts
6205 ± 5% -32.9% 4164 ± 34% interrupts.CPU12.PMI:Performance_monitoring_interrupts
23637 ± 6% -48.1% 12262 ± 11% interrupts.CPU12.TLB:TLB_shootdowns
25698 ± 9% -36.6% 16298 ± 5% interrupts.CPU120.CAL:Function_call_interrupts
22429 ± 9% -40.2% 13422 ± 6% interrupts.CPU120.TLB:TLB_shootdowns
25824 ± 9% -38.1% 15984 ± 2% interrupts.CPU121.CAL:Function_call_interrupts
6254 ± 4% -34.0% 4127 ± 24% interrupts.CPU121.NMI:Non-maskable_interrupts
6254 ± 4% -34.0% 4127 ± 24% interrupts.CPU121.PMI:Performance_monitoring_interrupts
22548 ± 8% -42.3% 13012 ± 3% interrupts.CPU121.TLB:TLB_shootdowns
25386 ± 12% -36.5% 16116 ± 4% interrupts.CPU122.CAL:Function_call_interrupts
21996 ± 11% -39.5% 13312 ± 4% interrupts.CPU122.TLB:TLB_shootdowns
25944 ± 11% -37.5% 16217 ± 5% interrupts.CPU123.CAL:Function_call_interrupts
6261 ± 3% -49.5% 3161 ± 42% interrupts.CPU123.NMI:Non-maskable_interrupts
6261 ± 3% -49.5% 3161 ± 42% interrupts.CPU123.PMI:Performance_monitoring_interrupts
22614 ± 10% -41.4% 13258 ± 7% interrupts.CPU123.TLB:TLB_shootdowns
26203 ± 9% -41.2% 15417 ± 5% interrupts.CPU124.CAL:Function_call_interrupts
22819 ± 9% -45.7% 12387 ± 6% interrupts.CPU124.TLB:TLB_shootdowns
26658 ± 9% -37.8% 16571 ± 3% interrupts.CPU125.CAL:Function_call_interrupts
6317 ± 4% -38.0% 3914 ± 35% interrupts.CPU125.NMI:Non-maskable_interrupts
6317 ± 4% -38.0% 3914 ± 35% interrupts.CPU125.PMI:Performance_monitoring_interrupts
23411 ± 9% -42.1% 13551 ± 4% interrupts.CPU125.TLB:TLB_shootdowns
26357 ± 11% -39.0% 16090 interrupts.CPU126.CAL:Function_call_interrupts
22964 ± 11% -42.7% 13155 ± 2% interrupts.CPU126.TLB:TLB_shootdowns
26279 ± 10% -37.8% 16357 ± 2% interrupts.CPU127.CAL:Function_call_interrupts
6414 ± 4% -31.6% 4389 ± 19% interrupts.CPU127.NMI:Non-maskable_interrupts
6414 ± 4% -31.6% 4389 ± 19% interrupts.CPU127.PMI:Performance_monitoring_interrupts
22928 ± 9% -42.0% 13304 ± 3% interrupts.CPU127.TLB:TLB_shootdowns
26403 ± 7% -38.7% 16195 ± 5% interrupts.CPU128.CAL:Function_call_interrupts
6320 ± 3% -48.7% 3241 ± 41% interrupts.CPU128.NMI:Non-maskable_interrupts
6320 ± 3% -48.7% 3241 ± 41% interrupts.CPU128.PMI:Performance_monitoring_interrupts
23017 ± 6% -42.9% 13151 ± 7% interrupts.CPU128.TLB:TLB_shootdowns
26062 ± 11% -38.4% 16041 ± 2% interrupts.CPU129.CAL:Function_call_interrupts
22712 ± 11% -42.6% 13045 ± 2% interrupts.CPU129.TLB:TLB_shootdowns
27023 ± 8% -46.6% 14438 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
23836 ± 9% -51.8% 11478 ± 11% interrupts.CPU13.TLB:TLB_shootdowns
25782 ± 11% -37.9% 16023 ± 4% interrupts.CPU130.CAL:Function_call_interrupts
22383 ± 11% -41.5% 13102 ± 6% interrupts.CPU130.TLB:TLB_shootdowns
25769 ± 10% -37.1% 16197 ± 3% interrupts.CPU131.CAL:Function_call_interrupts
22457 ± 9% -41.1% 13226 ± 3% interrupts.CPU131.TLB:TLB_shootdowns
25939 ± 8% -36.8% 16394 ± 3% interrupts.CPU132.CAL:Function_call_interrupts
6245 ± 3% -39.8% 3761 ± 35% interrupts.CPU132.NMI:Non-maskable_interrupts
6245 ± 3% -39.8% 3761 ± 35% interrupts.CPU132.PMI:Performance_monitoring_interrupts
22535 ± 8% -40.5% 13418 ± 5% interrupts.CPU132.TLB:TLB_shootdowns
24855 ± 17% -37.7% 15479 ± 8% interrupts.CPU133.CAL:Function_call_interrupts
21434 ± 17% -41.9% 12446 ± 10% interrupts.CPU133.TLB:TLB_shootdowns
26463 ± 12% -39.6% 15994 ± 5% interrupts.CPU134.CAL:Function_call_interrupts
6319 ± 2% -31.5% 4328 ± 18% interrupts.CPU134.NMI:Non-maskable_interrupts
6319 ± 2% -31.5% 4328 ± 18% interrupts.CPU134.PMI:Performance_monitoring_interrupts
23156 ± 11% -44.1% 12953 ± 7% interrupts.CPU134.TLB:TLB_shootdowns
25511 ± 10% -38.3% 15740 ± 5% interrupts.CPU135.CAL:Function_call_interrupts
22206 ± 9% -42.3% 12803 ± 6% interrupts.CPU135.TLB:TLB_shootdowns
26114 ± 12% -37.6% 16304 ± 2% interrupts.CPU136.CAL:Function_call_interrupts
6348 ± 4% -35.5% 4096 ± 17% interrupts.CPU136.NMI:Non-maskable_interrupts
6348 ± 4% -35.5% 4096 ± 17% interrupts.CPU136.PMI:Performance_monitoring_interrupts
22716 ± 12% -41.4% 13306 ± 3% interrupts.CPU136.TLB:TLB_shootdowns
25715 ± 11% -38.7% 15752 ± 4% interrupts.CPU137.CAL:Function_call_interrupts
6314 ± 3% -43.2% 3586 ± 38% interrupts.CPU137.NMI:Non-maskable_interrupts
6314 ± 3% -43.2% 3586 ± 38% interrupts.CPU137.PMI:Performance_monitoring_interrupts
22325 ± 10% -43.0% 12718 ± 5% interrupts.CPU137.TLB:TLB_shootdowns
25832 ± 14% -40.7% 15324 ± 10% interrupts.CPU138.CAL:Function_call_interrupts
6262 ± 5% -57.8% 2644 ± 36% interrupts.CPU138.NMI:Non-maskable_interrupts
6262 ± 5% -57.8% 2644 ± 36% interrupts.CPU138.PMI:Performance_monitoring_interrupts
22456 ± 14% -45.4% 12269 ± 13% interrupts.CPU138.TLB:TLB_shootdowns
26008 ± 10% -39.9% 15625 ± 7% interrupts.CPU139.CAL:Function_call_interrupts
22688 ± 10% -44.4% 12617 ± 9% interrupts.CPU139.TLB:TLB_shootdowns
27079 ± 7% -47.2% 14294 ± 6% interrupts.CPU14.CAL:Function_call_interrupts
23830 ± 7% -52.5% 11308 ± 7% interrupts.CPU14.TLB:TLB_shootdowns
25986 ± 9% -38.9% 15865 ± 5% interrupts.CPU140.CAL:Function_call_interrupts
6388 ± 4% -41.6% 3733 ± 34% interrupts.CPU140.NMI:Non-maskable_interrupts
6388 ± 4% -41.6% 3733 ± 34% interrupts.CPU140.PMI:Performance_monitoring_interrupts
22603 ± 9% -43.3% 12818 ± 6% interrupts.CPU140.TLB:TLB_shootdowns
25088 ± 17% -37.9% 15590 ± 5% interrupts.CPU141.CAL:Function_call_interrupts
6327 ± 4% -34.5% 4144 ± 34% interrupts.CPU141.NMI:Non-maskable_interrupts
6327 ± 4% -34.5% 4144 ± 34% interrupts.CPU141.PMI:Performance_monitoring_interrupts
21633 ± 17% -41.9% 12562 ± 6% interrupts.CPU141.TLB:TLB_shootdowns
25905 ± 15% -40.1% 15514 ± 8% interrupts.CPU142.CAL:Function_call_interrupts
22513 ± 16% -44.6% 12479 ± 10% interrupts.CPU142.TLB:TLB_shootdowns
26096 ± 12% -42.5% 15005 ± 5% interrupts.CPU143.CAL:Function_call_interrupts
22777 ± 11% -47.1% 12059 ± 6% interrupts.CPU143.TLB:TLB_shootdowns
27603 ± 3% -46.2% 14854 ± 8% interrupts.CPU144.CAL:Function_call_interrupts
24314 ± 5% -51.0% 11902 ± 10% interrupts.CPU144.TLB:TLB_shootdowns
28108 ± 3% -45.8% 15236 ± 6% interrupts.CPU145.CAL:Function_call_interrupts
24783 ± 5% -50.8% 12200 ± 8% interrupts.CPU145.TLB:TLB_shootdowns
27705 ± 2% -48.2% 14342 ± 7% interrupts.CPU146.CAL:Function_call_interrupts
24528 ± 4% -54.1% 11263 ± 10% interrupts.CPU146.TLB:TLB_shootdowns
27894 -44.5% 15469 ± 5% interrupts.CPU147.CAL:Function_call_interrupts
6127 ± 7% -37.3% 3843 ± 20% interrupts.CPU147.NMI:Non-maskable_interrupts
6127 ± 7% -37.3% 3843 ± 20% interrupts.CPU147.PMI:Performance_monitoring_interrupts
24626 ± 4% -49.5% 12436 ± 6% interrupts.CPU147.TLB:TLB_shootdowns
28319 ± 4% -44.3% 15773 ± 7% interrupts.CPU148.CAL:Function_call_interrupts
25004 ± 5% -48.9% 12776 ± 9% interrupts.CPU148.TLB:TLB_shootdowns
27991 ± 2% -44.4% 15576 ± 3% interrupts.CPU149.CAL:Function_call_interrupts
6236 ± 3% -30.5% 4336 ± 24% interrupts.CPU149.NMI:Non-maskable_interrupts
6236 ± 3% -30.5% 4336 ± 24% interrupts.CPU149.PMI:Performance_monitoring_interrupts
24647 ± 3% -49.2% 12518 ± 4% interrupts.CPU149.TLB:TLB_shootdowns
26702 ± 8% -43.9% 14975 ± 9% interrupts.CPU15.CAL:Function_call_interrupts
6223 ± 4% -26.3% 4587 ± 17% interrupts.CPU15.NMI:Non-maskable_interrupts
6223 ± 4% -26.3% 4587 ± 17% interrupts.CPU15.PMI:Performance_monitoring_interrupts
23511 ± 9% -48.8% 12038 ± 11% interrupts.CPU15.TLB:TLB_shootdowns
27200 ± 4% -42.7% 15580 ± 7% interrupts.CPU150.CAL:Function_call_interrupts
23912 ± 4% -47.7% 12512 ± 10% interrupts.CPU150.TLB:TLB_shootdowns
28217 ± 3% -45.3% 15433 ± 8% interrupts.CPU151.CAL:Function_call_interrupts
6289 ± 2% -34.3% 4132 ± 29% interrupts.CPU151.NMI:Non-maskable_interrupts
6289 ± 2% -34.3% 4132 ± 29% interrupts.CPU151.PMI:Performance_monitoring_interrupts
24912 ± 3% -50.3% 12379 ± 11% interrupts.CPU151.TLB:TLB_shootdowns
27892 ± 4% -43.5% 15747 ± 5% interrupts.CPU152.CAL:Function_call_interrupts
24655 ± 5% -48.4% 12730 ± 7% interrupts.CPU152.TLB:TLB_shootdowns
27985 ± 3% -43.6% 15781 ± 3% interrupts.CPU153.CAL:Function_call_interrupts
24700 ± 5% -48.2% 12785 ± 4% interrupts.CPU153.TLB:TLB_shootdowns
28258 -45.4% 15424 ± 6% interrupts.CPU154.CAL:Function_call_interrupts
24984 ± 2% -50.4% 12395 ± 9% interrupts.CPU154.TLB:TLB_shootdowns
28729 ± 4% -46.4% 15412 ± 5% interrupts.CPU155.CAL:Function_call_interrupts
25466 ± 3% -51.5% 12348 ± 7% interrupts.CPU155.TLB:TLB_shootdowns
28471 ± 2% -46.4% 15270 ± 8% interrupts.CPU156.CAL:Function_call_interrupts
25189 ± 4% -51.5% 12218 ± 10% interrupts.CPU156.TLB:TLB_shootdowns
28339 ± 2% -46.9% 15043 ± 2% interrupts.CPU157.CAL:Function_call_interrupts
25019 -51.6% 12097 interrupts.CPU157.TLB:TLB_shootdowns
27386 ± 3% -42.3% 15802 ± 3% interrupts.CPU158.CAL:Function_call_interrupts
24077 ± 6% -47.1% 12748 ± 3% interrupts.CPU158.TLB:TLB_shootdowns
28242 ± 3% -43.6% 15939 ± 4% interrupts.CPU159.CAL:Function_call_interrupts
6371 ± 2% -31.3% 4379 ± 22% interrupts.CPU159.NMI:Non-maskable_interrupts
6371 ± 2% -31.3% 4379 ± 22% interrupts.CPU159.PMI:Performance_monitoring_interrupts
24953 ± 5% -48.4% 12885 ± 6% interrupts.CPU159.TLB:TLB_shootdowns
27056 ± 7% -46.3% 14540 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
23823 ± 7% -51.5% 11562 ± 10% interrupts.CPU16.TLB:TLB_shootdowns
28362 -43.8% 15926 ± 6% interrupts.CPU160.CAL:Function_call_interrupts
6236 ± 4% -39.2% 3788 ± 38% interrupts.CPU160.NMI:Non-maskable_interrupts
6236 ± 4% -39.2% 3788 ± 38% interrupts.CPU160.PMI:Performance_monitoring_interrupts
25119 ± 4% -48.9% 12826 ± 8% interrupts.CPU160.TLB:TLB_shootdowns
28858 ± 3% -45.7% 15668 ± 9% interrupts.CPU161.CAL:Function_call_interrupts
6292 ± 3% -43.9% 3530 ± 21% interrupts.CPU161.NMI:Non-maskable_interrupts
6292 ± 3% -43.9% 3530 ± 21% interrupts.CPU161.PMI:Performance_monitoring_interrupts
25535 ± 4% -50.5% 12638 ± 11% interrupts.CPU161.TLB:TLB_shootdowns
28903 -47.3% 15233 ± 11% interrupts.CPU162.CAL:Function_call_interrupts
6140 ± 6% -41.1% 3616 ± 16% interrupts.CPU162.NMI:Non-maskable_interrupts
6140 ± 6% -41.1% 3616 ± 16% interrupts.CPU162.PMI:Performance_monitoring_interrupts
25603 ± 3% -52.5% 12159 ± 13% interrupts.CPU162.TLB:TLB_shootdowns
28874 ± 2% -48.4% 14894 ± 7% interrupts.CPU163.CAL:Function_call_interrupts
25490 ± 3% -53.5% 11857 ± 9% interrupts.CPU163.TLB:TLB_shootdowns
29304 ± 3% -48.4% 15117 ± 9% interrupts.CPU164.CAL:Function_call_interrupts
25985 ± 5% -53.5% 12092 ± 12% interrupts.CPU164.TLB:TLB_shootdowns
28915 ± 2% -49.4% 14641 ± 7% interrupts.CPU165.CAL:Function_call_interrupts
6260 ± 3% -37.2% 3933 ± 36% interrupts.CPU165.NMI:Non-maskable_interrupts
6260 ± 3% -37.2% 3933 ± 36% interrupts.CPU165.PMI:Performance_monitoring_interrupts
25555 ± 4% -54.8% 11548 ± 9% interrupts.CPU165.TLB:TLB_shootdowns
28013 ± 3% -47.5% 14719 ± 7% interrupts.CPU166.CAL:Function_call_interrupts
6299 -38.1% 3896 ± 37% interrupts.CPU166.NMI:Non-maskable_interrupts
6299 -38.1% 3896 ± 37% interrupts.CPU166.PMI:Performance_monitoring_interrupts
24656 ± 7% -52.4% 11733 ± 10% interrupts.CPU166.TLB:TLB_shootdowns
29233 ± 5% -47.6% 15321 ± 6% interrupts.CPU167.CAL:Function_call_interrupts
6154 ± 4% -27.6% 4454 ± 14% interrupts.CPU167.NMI:Non-maskable_interrupts
6154 ± 4% -27.6% 4454 ± 14% interrupts.CPU167.PMI:Performance_monitoring_interrupts
25980 ± 7% -52.6% 12311 ± 7% interrupts.CPU167.TLB:TLB_shootdowns
26911 ± 10% -45.7% 14624 ± 10% interrupts.CPU168.CAL:Function_call_interrupts
6336 ± 5% -32.3% 4287 ± 27% interrupts.CPU168.NMI:Non-maskable_interrupts
6336 ± 5% -32.3% 4287 ± 27% interrupts.CPU168.PMI:Performance_monitoring_interrupts
23567 ± 10% -50.9% 11568 ± 13% interrupts.CPU168.TLB:TLB_shootdowns
27296 ± 9% -46.2% 14687 ± 11% interrupts.CPU169.CAL:Function_call_interrupts
6293 ± 5% -41.3% 3694 ± 36% interrupts.CPU169.NMI:Non-maskable_interrupts
6293 ± 5% -41.3% 3694 ± 36% interrupts.CPU169.PMI:Performance_monitoring_interrupts
24002 ± 9% -51.4% 11676 ± 15% interrupts.CPU169.TLB:TLB_shootdowns
26859 ± 5% -46.1% 14479 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
6134 ± 6% -26.5% 4509 ± 18% interrupts.CPU17.NMI:Non-maskable_interrupts
6134 ± 6% -26.5% 4509 ± 18% interrupts.CPU17.PMI:Performance_monitoring_interrupts
23653 ± 6% -51.2% 11535 ± 9% interrupts.CPU17.TLB:TLB_shootdowns
27483 ± 10% -45.7% 14933 ± 13% interrupts.CPU170.CAL:Function_call_interrupts
24190 ± 10% -51.1% 11827 ± 16% interrupts.CPU170.TLB:TLB_shootdowns
27788 ± 8% -46.7% 14812 ± 12% interrupts.CPU171.CAL:Function_call_interrupts
24457 ± 8% -51.9% 11762 ± 15% interrupts.CPU171.TLB:TLB_shootdowns
27544 ± 9% -45.5% 15011 ± 8% interrupts.CPU172.CAL:Function_call_interrupts
6348 ± 7% -33.9% 4194 ± 26% interrupts.CPU172.NMI:Non-maskable_interrupts
6348 ± 7% -33.9% 4194 ± 26% interrupts.CPU172.PMI:Performance_monitoring_interrupts
24149 ± 9% -50.5% 11961 ± 10% interrupts.CPU172.TLB:TLB_shootdowns
27362 ± 9% -45.6% 14880 ± 8% interrupts.CPU173.CAL:Function_call_interrupts
23971 ± 10% -50.8% 11805 ± 10% interrupts.CPU173.TLB:TLB_shootdowns
27355 ± 10% -46.2% 14722 ± 8% interrupts.CPU174.CAL:Function_call_interrupts
6334 ± 6% -22.9% 4882 ± 26% interrupts.CPU174.NMI:Non-maskable_interrupts
6334 ± 6% -22.9% 4882 ± 26% interrupts.CPU174.PMI:Performance_monitoring_interrupts
24098 ± 10% -51.5% 11690 ± 10% interrupts.CPU174.TLB:TLB_shootdowns
27909 ± 10% -44.5% 15487 ± 9% interrupts.CPU175.CAL:Function_call_interrupts
6305 ± 6% -33.4% 4196 ± 31% interrupts.CPU175.NMI:Non-maskable_interrupts
6305 ± 6% -33.4% 4196 ± 31% interrupts.CPU175.PMI:Performance_monitoring_interrupts
24615 ± 10% -49.7% 12380 ± 12% interrupts.CPU175.TLB:TLB_shootdowns
27866 ± 8% -46.0% 15044 ± 8% interrupts.CPU176.CAL:Function_call_interrupts
24526 ± 8% -51.4% 11927 ± 11% interrupts.CPU176.TLB:TLB_shootdowns
27709 ± 11% -43.8% 15585 ± 11% interrupts.CPU177.CAL:Function_call_interrupts
24404 ± 12% -48.7% 12521 ± 15% interrupts.CPU177.TLB:TLB_shootdowns
28578 ± 8% -47.0% 15143 ± 9% interrupts.CPU178.CAL:Function_call_interrupts
25259 ± 8% -52.2% 12066 ± 11% interrupts.CPU178.TLB:TLB_shootdowns
27908 ± 8% -46.2% 15014 ± 11% interrupts.CPU179.CAL:Function_call_interrupts
24608 ± 8% -51.5% 11937 ± 15% interrupts.CPU179.TLB:TLB_shootdowns
27059 ± 7% -45.1% 14845 ± 9% interrupts.CPU18.CAL:Function_call_interrupts
23765 ± 6% -50.0% 11880 ± 11% interrupts.CPU18.TLB:TLB_shootdowns
27930 ± 9% -45.6% 15199 ± 13% interrupts.CPU180.CAL:Function_call_interrupts
24730 ± 7% -50.9% 12139 ± 16% interrupts.CPU180.TLB:TLB_shootdowns
27442 ± 6% -48.0% 14279 ± 11% interrupts.CPU181.CAL:Function_call_interrupts
24097 ± 6% -52.7% 11387 ± 15% interrupts.CPU181.TLB:TLB_shootdowns
27579 ± 7% -46.7% 14708 ± 10% interrupts.CPU182.CAL:Function_call_interrupts
24311 ± 6% -52.1% 11633 ± 14% interrupts.CPU182.TLB:TLB_shootdowns
27735 ± 9% -46.0% 14966 ± 11% interrupts.CPU183.CAL:Function_call_interrupts
24351 ± 9% -51.1% 11909 ± 13% interrupts.CPU183.TLB:TLB_shootdowns
27962 ± 10% -45.6% 15200 ± 12% interrupts.CPU184.CAL:Function_call_interrupts
24675 ± 10% -50.5% 12209 ± 15% interrupts.CPU184.TLB:TLB_shootdowns
27476 ± 6% -45.1% 15087 ± 12% interrupts.CPU185.CAL:Function_call_interrupts
24052 ± 7% -49.6% 12116 ± 15% interrupts.CPU185.TLB:TLB_shootdowns
27700 ± 7% -46.8% 14723 ± 10% interrupts.CPU186.CAL:Function_call_interrupts
24339 ± 8% -52.1% 11654 ± 13% interrupts.CPU186.TLB:TLB_shootdowns
27587 ± 8% -45.8% 14949 ± 11% interrupts.CPU187.CAL:Function_call_interrupts
6344 ± 5% -23.0% 4888 ± 25% interrupts.CPU187.NMI:Non-maskable_interrupts
6344 ± 5% -23.0% 4888 ± 25% interrupts.CPU187.PMI:Performance_monitoring_interrupts
24155 ± 10% -51.0% 11829 ± 15% interrupts.CPU187.TLB:TLB_shootdowns
28173 ± 10% -48.0% 14658 ± 12% interrupts.CPU188.CAL:Function_call_interrupts
24829 ± 10% -53.6% 11517 ± 15% interrupts.CPU188.TLB:TLB_shootdowns
28397 ± 8% -49.6% 14307 ± 13% interrupts.CPU189.CAL:Function_call_interrupts
25047 ± 8% -55.2% 11217 ± 17% interrupts.CPU189.TLB:TLB_shootdowns
27106 ± 8% -44.4% 15067 ± 9% interrupts.CPU19.CAL:Function_call_interrupts
23886 ± 7% -49.2% 12128 ± 11% interrupts.CPU19.TLB:TLB_shootdowns
28041 ± 10% -47.2% 14810 ± 12% interrupts.CPU190.CAL:Function_call_interrupts
24695 ± 10% -52.7% 11687 ± 16% interrupts.CPU190.TLB:TLB_shootdowns
28185 ± 9% -47.3% 14851 ± 10% interrupts.CPU191.CAL:Function_call_interrupts
6298 ± 6% -50.1% 3141 ± 49% interrupts.CPU191.NMI:Non-maskable_interrupts
6298 ± 6% -50.1% 3141 ± 49% interrupts.CPU191.PMI:Performance_monitoring_interrupts
24829 ± 9% -52.7% 11748 ± 13% interrupts.CPU191.TLB:TLB_shootdowns
25802 ± 6% -49.2% 13105 ± 5% interrupts.CPU2.CAL:Function_call_interrupts
22493 ± 6% -55.1% 10089 ± 7% interrupts.CPU2.TLB:TLB_shootdowns
26384 ± 9% -43.1% 15007 ± 8% interrupts.CPU20.CAL:Function_call_interrupts
23142 ± 8% -48.0% 12028 ± 10% interrupts.CPU20.TLB:TLB_shootdowns
26128 ± 9% -43.2% 14833 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
6251 ± 5% -33.4% 4162 ± 33% interrupts.CPU21.NMI:Non-maskable_interrupts
6251 ± 5% -33.4% 4162 ± 33% interrupts.CPU21.PMI:Performance_monitoring_interrupts
22823 ± 10% -48.1% 11854 ± 11% interrupts.CPU21.TLB:TLB_shootdowns
26900 ± 8% -45.7% 14604 ± 6% interrupts.CPU22.CAL:Function_call_interrupts
6279 ± 6% -26.0% 4643 ± 17% interrupts.CPU22.NMI:Non-maskable_interrupts
6279 ± 6% -26.0% 4643 ± 17% interrupts.CPU22.PMI:Performance_monitoring_interrupts
23712 ± 10% -50.8% 11667 ± 7% interrupts.CPU22.TLB:TLB_shootdowns
26228 ± 9% -44.3% 14618 ± 7% interrupts.CPU23.CAL:Function_call_interrupts
6251 ± 4% -40.9% 3697 ± 45% interrupts.CPU23.NMI:Non-maskable_interrupts
6251 ± 4% -40.9% 3697 ± 45% interrupts.CPU23.PMI:Performance_monitoring_interrupts
22955 ± 10% -48.5% 11824 ± 9% interrupts.CPU23.TLB:TLB_shootdowns
24749 ± 12% -42.9% 14142 ± 5% interrupts.CPU24.CAL:Function_call_interrupts
6406 ± 2% -39.9% 3847 ± 34% interrupts.CPU24.NMI:Non-maskable_interrupts
6406 ± 2% -39.9% 3847 ± 34% interrupts.CPU24.PMI:Performance_monitoring_interrupts
21366 ± 12% -47.8% 11161 ± 7% interrupts.CPU24.TLB:TLB_shootdowns
24735 ± 9% -41.2% 14545 ± 5% interrupts.CPU25.CAL:Function_call_interrupts
6006 ± 5% -32.6% 4045 ± 19% interrupts.CPU25.NMI:Non-maskable_interrupts
6006 ± 5% -32.6% 4045 ± 19% interrupts.CPU25.PMI:Performance_monitoring_interrupts
21376 ± 8% -45.8% 11588 ± 6% interrupts.CPU25.TLB:TLB_shootdowns
24910 ± 9% -39.7% 15012 ± 5% interrupts.CPU26.CAL:Function_call_interrupts
6311 ± 3% -34.7% 4123 ± 23% interrupts.CPU26.NMI:Non-maskable_interrupts
6311 ± 3% -34.7% 4123 ± 23% interrupts.CPU26.PMI:Performance_monitoring_interrupts
21534 ± 9% -43.7% 12131 ± 8% interrupts.CPU26.TLB:TLB_shootdowns
25255 ± 8% -41.0% 14892 ± 4% interrupts.CPU27.CAL:Function_call_interrupts
21957 ± 7% -45.5% 11959 ± 5% interrupts.CPU27.TLB:TLB_shootdowns
25496 ± 8% -41.6% 14890 ± 2% interrupts.CPU28.CAL:Function_call_interrupts
6340 ± 2% -44.1% 3546 ± 37% interrupts.CPU28.NMI:Non-maskable_interrupts
6340 ± 2% -44.1% 3546 ± 37% interrupts.CPU28.PMI:Performance_monitoring_interrupts
22200 ± 7% -46.0% 11986 ± 3% interrupts.CPU28.TLB:TLB_shootdowns
25768 ± 8% -41.9% 14977 ± 4% interrupts.CPU29.CAL:Function_call_interrupts
22356 ± 8% -46.5% 11961 ± 4% interrupts.CPU29.TLB:TLB_shootdowns
26674 ± 7% -46.9% 14173 ± 8% interrupts.CPU3.CAL:Function_call_interrupts
6329 ± 5% -26.1% 4680 ± 20% interrupts.CPU3.NMI:Non-maskable_interrupts
6329 ± 5% -26.1% 4680 ± 20% interrupts.CPU3.PMI:Performance_monitoring_interrupts
23313 ± 7% -52.0% 11198 ± 11% interrupts.CPU3.TLB:TLB_shootdowns
24951 ± 7% -37.8% 15520 ± 3% interrupts.CPU30.CAL:Function_call_interrupts
21567 ± 6% -41.9% 12538 ± 3% interrupts.CPU30.TLB:TLB_shootdowns
25942 ± 10% -38.6% 15925 ± 2% interrupts.CPU31.CAL:Function_call_interrupts
6355 ± 4% -31.7% 4343 ± 18% interrupts.CPU31.NMI:Non-maskable_interrupts
6355 ± 4% -31.7% 4343 ± 18% interrupts.CPU31.PMI:Performance_monitoring_interrupts
22629 ± 10% -42.8% 12954 ± 2% interrupts.CPU31.TLB:TLB_shootdowns
25946 ± 6% -40.2% 15516 ± 2% interrupts.CPU32.CAL:Function_call_interrupts
6418 ± 4% -41.7% 3744 ± 31% interrupts.CPU32.NMI:Non-maskable_interrupts
6418 ± 4% -41.7% 3744 ± 31% interrupts.CPU32.PMI:Performance_monitoring_interrupts
22602 ± 5% -44.1% 12631 ± 2% interrupts.CPU32.TLB:TLB_shootdowns
25218 ± 8% -40.5% 15014 interrupts.CPU33.CAL:Function_call_interrupts
21993 ± 6% -44.1% 12292 interrupts.CPU33.TLB:TLB_shootdowns
24915 ± 10% -36.8% 15737 ± 6% interrupts.CPU34.CAL:Function_call_interrupts
21629 ± 9% -41.0% 12751 ± 7% interrupts.CPU34.TLB:TLB_shootdowns
25492 ± 10% -39.2% 15504 ± 4% interrupts.CPU35.CAL:Function_call_interrupts
22134 ± 9% -43.2% 12579 ± 5% interrupts.CPU35.TLB:TLB_shootdowns
25612 ± 7% -38.6% 15723 ± 5% interrupts.CPU36.CAL:Function_call_interrupts
6202 ± 5% -56.3% 2713 ± 47% interrupts.CPU36.NMI:Non-maskable_interrupts
6202 ± 5% -56.3% 2713 ± 47% interrupts.CPU36.PMI:Performance_monitoring_interrupts
22216 ± 6% -42.6% 12741 ± 7% interrupts.CPU36.TLB:TLB_shootdowns
24492 ± 14% -35.7% 15750 ± 4% interrupts.CPU37.CAL:Function_call_interrupts
5455 ± 24% -46.0% 2945 ± 35% interrupts.CPU37.NMI:Non-maskable_interrupts
5455 ± 24% -46.0% 2945 ± 35% interrupts.CPU37.PMI:Performance_monitoring_interrupts
21124 ± 15% -39.2% 12849 ± 5% interrupts.CPU37.TLB:TLB_shootdowns
25436 ± 9% -37.7% 15841 ± 3% interrupts.CPU38.CAL:Function_call_interrupts
6297 ± 3% -31.3% 4327 ± 21% interrupts.CPU38.NMI:Non-maskable_interrupts
6297 ± 3% -31.3% 4327 ± 21% interrupts.CPU38.PMI:Performance_monitoring_interrupts
22093 ± 8% -41.7% 12888 ± 3% interrupts.CPU38.TLB:TLB_shootdowns
24543 ± 9% -35.4% 15843 ± 5% interrupts.CPU39.CAL:Function_call_interrupts
6360 ± 5% -43.0% 3623 ± 32% interrupts.CPU39.NMI:Non-maskable_interrupts
6360 ± 5% -43.0% 3623 ± 32% interrupts.CPU39.PMI:Performance_monitoring_interrupts
21208 ± 8% -39.1% 12907 ± 6% interrupts.CPU39.TLB:TLB_shootdowns
26453 ± 8% -47.2% 13959 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
23163 ± 8% -52.3% 11052 ± 10% interrupts.CPU4.TLB:TLB_shootdowns
24773 ± 12% -36.3% 15769 ± 3% interrupts.CPU40.CAL:Function_call_interrupts
21535 ± 11% -40.5% 12806 ± 4% interrupts.CPU40.TLB:TLB_shootdowns
25594 ± 10% -39.8% 15414 ± 2% interrupts.CPU41.CAL:Function_call_interrupts
22313 ± 8% -44.4% 12401 ± 2% interrupts.CPU41.TLB:TLB_shootdowns
25198 ± 13% -38.7% 15458 ± 4% interrupts.CPU42.CAL:Function_call_interrupts
21796 ± 13% -42.4% 12565 ± 5% interrupts.CPU42.TLB:TLB_shootdowns
25525 ± 7% -38.7% 15639 ± 6% interrupts.CPU43.CAL:Function_call_interrupts
6398 ± 3% -32.6% 4310 ± 21% interrupts.CPU43.NMI:Non-maskable_interrupts
6398 ± 3% -32.6% 4310 ± 21% interrupts.CPU43.PMI:Performance_monitoring_interrupts
22240 ± 6% -43.1% 12660 ± 7% interrupts.CPU43.TLB:TLB_shootdowns
25386 ± 12% -38.1% 15723 ± 5% interrupts.CPU44.CAL:Function_call_interrupts
6338 ± 5% -44.1% 3545 ± 24% interrupts.CPU44.NMI:Non-maskable_interrupts
6338 ± 5% -44.1% 3545 ± 24% interrupts.CPU44.PMI:Performance_monitoring_interrupts
22061 ± 11% -42.0% 12799 ± 7% interrupts.CPU44.TLB:TLB_shootdowns
24287 ± 13% -33.6% 16117 ± 2% interrupts.CPU45.CAL:Function_call_interrupts
6357 ± 5% -27.5% 4607 ± 21% interrupts.CPU45.NMI:Non-maskable_interrupts
6357 ± 5% -27.5% 4607 ± 21% interrupts.CPU45.PMI:Performance_monitoring_interrupts
20932 ± 12% -37.4% 13099 ± 2% interrupts.CPU45.TLB:TLB_shootdowns
25436 ± 10% -37.1% 15993 interrupts.CPU46.CAL:Function_call_interrupts
6270 ± 5% -30.6% 4350 ± 23% interrupts.CPU46.NMI:Non-maskable_interrupts
6270 ± 5% -30.6% 4350 ± 23% interrupts.CPU46.PMI:Performance_monitoring_interrupts
747.75 ± 30% -38.1% 462.50 ± 25% interrupts.CPU46.RES:Rescheduling_interrupts
22069 ± 9% -41.1% 12997 ± 2% interrupts.CPU46.TLB:TLB_shootdowns
25670 ± 10% -38.7% 15735 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
22454 ± 10% -43.3% 12733 ± 10% interrupts.CPU47.TLB:TLB_shootdowns
27053 ± 3% -47.8% 14113 ± 6% interrupts.CPU48.CAL:Function_call_interrupts
6279 ± 2% -34.2% 4134 ± 25% interrupts.CPU48.NMI:Non-maskable_interrupts
6279 ± 2% -34.2% 4134 ± 25% interrupts.CPU48.PMI:Performance_monitoring_interrupts
23836 ± 4% -53.3% 11123 ± 7% interrupts.CPU48.TLB:TLB_shootdowns
26789 -49.1% 13642 ± 7% interrupts.CPU49.CAL:Function_call_interrupts
6202 -35.4% 4006 ± 38% interrupts.CPU49.NMI:Non-maskable_interrupts
6202 -35.4% 4006 ± 38% interrupts.CPU49.PMI:Performance_monitoring_interrupts
23620 ± 4% -54.9% 10645 ± 9% interrupts.CPU49.TLB:TLB_shootdowns
26950 ± 4% -45.1% 14808 ± 7% interrupts.CPU5.CAL:Function_call_interrupts
6268 ± 5% -35.6% 4034 ± 32% interrupts.CPU5.NMI:Non-maskable_interrupts
6268 ± 5% -35.6% 4034 ± 32% interrupts.CPU5.PMI:Performance_monitoring_interrupts
23633 ± 5% -49.9% 11849 ± 9% interrupts.CPU5.TLB:TLB_shootdowns
26621 -46.0% 14377 ± 2% interrupts.CPU50.CAL:Function_call_interrupts
6283 ± 3% -31.5% 4304 ± 20% interrupts.CPU50.NMI:Non-maskable_interrupts
6283 ± 3% -31.5% 4304 ± 20% interrupts.CPU50.PMI:Performance_monitoring_interrupts
23351 ± 3% -51.4% 11359 ± 3% interrupts.CPU50.TLB:TLB_shootdowns
26585 -44.9% 14661 ± 5% interrupts.CPU51.CAL:Function_call_interrupts
6309 ± 4% -36.1% 4031 ± 38% interrupts.CPU51.NMI:Non-maskable_interrupts
6309 ± 4% -36.1% 4031 ± 38% interrupts.CPU51.PMI:Performance_monitoring_interrupts
23286 ± 5% -49.8% 11685 ± 6% interrupts.CPU51.TLB:TLB_shootdowns
26514 ± 4% -42.4% 15272 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
6147 ± 5% -48.6% 3159 ± 33% interrupts.CPU52.NMI:Non-maskable_interrupts
6147 ± 5% -48.6% 3159 ± 33% interrupts.CPU52.PMI:Performance_monitoring_interrupts
23414 ± 7% -47.6% 12270 ± 9% interrupts.CPU52.TLB:TLB_shootdowns
27328 ± 3% -46.5% 14624 ± 5% interrupts.CPU53.CAL:Function_call_interrupts
24033 ± 7% -51.6% 11632 ± 6% interrupts.CPU53.TLB:TLB_shootdowns
26150 ± 3% -43.3% 14824 ± 8% interrupts.CPU54.CAL:Function_call_interrupts
22887 ± 3% -48.1% 11879 ± 10% interrupts.CPU54.TLB:TLB_shootdowns
27196 ± 2% -45.0% 14952 ± 6% interrupts.CPU55.CAL:Function_call_interrupts
6356 ± 2% -32.9% 4263 ± 23% interrupts.CPU55.NMI:Non-maskable_interrupts
6356 ± 2% -32.9% 4263 ± 23% interrupts.CPU55.PMI:Performance_monitoring_interrupts
23944 ± 4% -49.7% 12051 ± 7% interrupts.CPU55.TLB:TLB_shootdowns
26840 ± 2% -43.1% 15283 ± 6% interrupts.CPU56.CAL:Function_call_interrupts
23570 ± 5% -47.7% 12319 ± 7% interrupts.CPU56.TLB:TLB_shootdowns
27239 ± 2% -43.1% 15511 ± 8% interrupts.CPU57.CAL:Function_call_interrupts
393.00 ± 12% +21.9% 479.25 ± 13% interrupts.CPU57.RES:Rescheduling_interrupts
23900 ± 5% -47.6% 12525 ± 10% interrupts.CPU57.TLB:TLB_shootdowns
27064 ± 2% -43.0% 15437 ± 5% interrupts.CPU58.CAL:Function_call_interrupts
23828 ± 4% -47.7% 12455 ± 7% interrupts.CPU58.TLB:TLB_shootdowns
27483 -43.8% 15433 ± 10% interrupts.CPU59.CAL:Function_call_interrupts
24266 ± 3% -48.6% 12463 ± 12% interrupts.CPU59.TLB:TLB_shootdowns
26361 ± 5% -46.2% 14179 ± 7% interrupts.CPU6.CAL:Function_call_interrupts
6265 ± 4% -24.6% 4723 ± 18% interrupts.CPU6.NMI:Non-maskable_interrupts
6265 ± 4% -24.6% 4723 ± 18% interrupts.CPU6.PMI:Performance_monitoring_interrupts
23127 ± 6% -51.5% 11222 ± 9% interrupts.CPU6.TLB:TLB_shootdowns
26757 ± 4% -43.3% 15169 ± 7% interrupts.CPU60.CAL:Function_call_interrupts
24077 ± 4% -49.3% 12205 ± 9% interrupts.CPU60.TLB:TLB_shootdowns
27530 -43.7% 15489 ± 6% interrupts.CPU61.CAL:Function_call_interrupts
24260 ± 2% -48.4% 12528 ± 9% interrupts.CPU61.TLB:TLB_shootdowns
27161 -44.6% 15044 ± 5% interrupts.CPU62.CAL:Function_call_interrupts
23894 ± 3% -49.2% 12132 ± 7% interrupts.CPU62.TLB:TLB_shootdowns
26635 ± 3% -42.9% 15214 ± 5% interrupts.CPU63.CAL:Function_call_interrupts
23534 ± 3% -47.9% 12263 ± 6% interrupts.CPU63.TLB:TLB_shootdowns
27482 ± 3% -45.4% 15004 ± 4% interrupts.CPU64.CAL:Function_call_interrupts
24240 ± 5% -50.5% 12010 ± 5% interrupts.CPU64.TLB:TLB_shootdowns
27551 ± 2% -43.2% 15661 ± 4% interrupts.CPU65.CAL:Function_call_interrupts
24288 ± 3% -47.8% 12668 ± 6% interrupts.CPU65.TLB:TLB_shootdowns
27620 ± 4% -44.8% 15257 ± 5% interrupts.CPU66.CAL:Function_call_interrupts
24313 ± 5% -49.6% 12248 ± 7% interrupts.CPU66.TLB:TLB_shootdowns
27420 ± 2% -43.7% 15438 ± 7% interrupts.CPU67.CAL:Function_call_interrupts
6064 ± 8% -27.8% 4378 ± 18% interrupts.CPU67.NMI:Non-maskable_interrupts
6064 ± 8% -27.8% 4378 ± 18% interrupts.CPU67.PMI:Performance_monitoring_interrupts
24187 ± 4% -48.5% 12455 ± 9% interrupts.CPU67.TLB:TLB_shootdowns
27980 ± 3% -46.0% 15105 ± 6% interrupts.CPU68.CAL:Function_call_interrupts
6223 ± 4% -30.8% 4303 ± 20% interrupts.CPU68.NMI:Non-maskable_interrupts
6223 ± 4% -30.8% 4303 ± 20% interrupts.CPU68.PMI:Performance_monitoring_interrupts
24702 ± 6% -50.9% 12120 ± 8% interrupts.CPU68.TLB:TLB_shootdowns
27578 ± 5% -45.0% 15181 ± 7% interrupts.CPU69.CAL:Function_call_interrupts
6112 ± 4% -40.5% 3638 ± 16% interrupts.CPU69.NMI:Non-maskable_interrupts
6112 ± 4% -40.5% 3638 ± 16% interrupts.CPU69.PMI:Performance_monitoring_interrupts
24269 ± 6% -49.7% 12205 ± 8% interrupts.CPU69.TLB:TLB_shootdowns
26146 ± 7% -44.8% 14424 ± 7% interrupts.CPU7.CAL:Function_call_interrupts
22825 ± 6% -49.7% 11483 ± 10% interrupts.CPU7.TLB:TLB_shootdowns
27118 ± 5% -46.2% 14601 ± 8% interrupts.CPU70.CAL:Function_call_interrupts
6198 ± 4% -39.0% 3780 ± 32% interrupts.CPU70.NMI:Non-maskable_interrupts
6198 ± 4% -39.0% 3780 ± 32% interrupts.CPU70.PMI:Performance_monitoring_interrupts
23866 ± 8% -51.3% 11619 ± 10% interrupts.CPU70.TLB:TLB_shootdowns
27347 ± 6% -45.7% 14855 ± 7% interrupts.CPU71.CAL:Function_call_interrupts
24244 ± 10% -51.0% 11883 ± 9% interrupts.CPU71.TLB:TLB_shootdowns
26505 ± 9% -45.8% 14367 ± 6% interrupts.CPU72.CAL:Function_call_interrupts
23166 ± 9% -50.8% 11391 ± 8% interrupts.CPU72.TLB:TLB_shootdowns
27462 ± 8% -47.5% 14410 ± 13% interrupts.CPU73.CAL:Function_call_interrupts
6234 ± 3% -32.7% 4197 ± 25% interrupts.CPU73.NMI:Non-maskable_interrupts
6234 ± 3% -32.7% 4197 ± 25% interrupts.CPU73.PMI:Performance_monitoring_interrupts
24197 ± 8% -52.9% 11396 ± 17% interrupts.CPU73.TLB:TLB_shootdowns
26916 ± 8% -47.1% 14237 ± 9% interrupts.CPU74.CAL:Function_call_interrupts
23636 ± 8% -52.1% 11321 ± 11% interrupts.CPU74.TLB:TLB_shootdowns
27194 ± 8% -46.6% 14528 ± 8% interrupts.CPU75.CAL:Function_call_interrupts
6285 ± 6% -33.9% 4153 ± 31% interrupts.CPU75.NMI:Non-maskable_interrupts
6285 ± 6% -33.9% 4153 ± 31% interrupts.CPU75.PMI:Performance_monitoring_interrupts
23889 ± 8% -51.9% 11497 ± 11% interrupts.CPU75.TLB:TLB_shootdowns
27127 ± 9% -45.3% 14833 ± 11% interrupts.CPU76.CAL:Function_call_interrupts
23874 ± 11% -50.4% 11841 ± 14% interrupts.CPU76.TLB:TLB_shootdowns
27266 ± 9% -46.1% 14706 ± 10% interrupts.CPU77.CAL:Function_call_interrupts
23940 ± 10% -51.3% 11653 ± 13% interrupts.CPU77.TLB:TLB_shootdowns
27483 ± 9% -46.9% 14595 ± 11% interrupts.CPU78.CAL:Function_call_interrupts
24199 ± 9% -52.0% 11605 ± 15% interrupts.CPU78.TLB:TLB_shootdowns
27843 ± 11% -48.8% 14261 ± 9% interrupts.CPU79.CAL:Function_call_interrupts
6445 ± 4% -24.5% 4868 ± 22% interrupts.CPU79.NMI:Non-maskable_interrupts
6445 ± 4% -24.5% 4868 ± 22% interrupts.CPU79.PMI:Performance_monitoring_interrupts
24643 ± 12% -54.0% 11330 ± 10% interrupts.CPU79.TLB:TLB_shootdowns
26776 ± 11% -46.1% 14430 ± 6% interrupts.CPU8.CAL:Function_call_interrupts
23541 ± 11% -51.4% 11431 ± 8% interrupts.CPU8.TLB:TLB_shootdowns
27139 ± 10% -45.4% 14826 ± 10% interrupts.CPU80.CAL:Function_call_interrupts
23925 ± 10% -50.4% 11866 ± 13% interrupts.CPU80.TLB:TLB_shootdowns
26416 ± 12% -45.7% 14354 ± 10% interrupts.CPU81.CAL:Function_call_interrupts
23185 ± 12% -50.8% 11399 ± 13% interrupts.CPU81.TLB:TLB_shootdowns
28161 ± 11% -47.9% 14665 ± 10% interrupts.CPU82.CAL:Function_call_interrupts
6338 ± 5% -34.3% 4164 ± 29% interrupts.CPU82.NMI:Non-maskable_interrupts
6338 ± 5% -34.3% 4164 ± 29% interrupts.CPU82.PMI:Performance_monitoring_interrupts
24870 ± 11% -52.7% 11756 ± 13% interrupts.CPU82.TLB:TLB_shootdowns
27751 ± 11% -46.6% 14821 ± 10% interrupts.CPU83.CAL:Function_call_interrupts
24497 ± 11% -51.7% 11823 ± 13% interrupts.CPU83.TLB:TLB_shootdowns
27812 ± 9% -47.2% 14675 ± 11% interrupts.CPU84.CAL:Function_call_interrupts
24536 ± 9% -52.5% 11650 ± 14% interrupts.CPU84.TLB:TLB_shootdowns
27537 ± 8% -44.6% 15254 ± 10% interrupts.CPU85.CAL:Function_call_interrupts
24260 ± 8% -49.0% 12380 ± 13% interrupts.CPU85.TLB:TLB_shootdowns
27193 ± 11% -47.4% 14313 ± 12% interrupts.CPU86.CAL:Function_call_interrupts
6416 ± 4% -45.7% 3485 ± 27% interrupts.CPU86.NMI:Non-maskable_interrupts
6416 ± 4% -45.7% 3485 ± 27% interrupts.CPU86.PMI:Performance_monitoring_interrupts
24032 ± 11% -52.7% 11370 ± 15% interrupts.CPU86.TLB:TLB_shootdowns
27092 ± 9% -45.6% 14750 ± 7% interrupts.CPU87.CAL:Function_call_interrupts
23820 ± 10% -50.7% 11754 ± 10% interrupts.CPU87.TLB:TLB_shootdowns
27593 ± 9% -45.8% 14961 ± 9% interrupts.CPU88.CAL:Function_call_interrupts
24289 ± 9% -50.8% 11950 ± 11% interrupts.CPU88.TLB:TLB_shootdowns
27852 ± 9% -47.1% 14746 ± 8% interrupts.CPU89.CAL:Function_call_interrupts
24606 ± 10% -52.4% 11718 ± 11% interrupts.CPU89.TLB:TLB_shootdowns
27251 ± 4% -46.2% 14670 ± 2% interrupts.CPU9.CAL:Function_call_interrupts
6241 ± 4% -40.6% 3710 ± 30% interrupts.CPU9.NMI:Non-maskable_interrupts
6241 ± 4% -40.6% 3710 ± 30% interrupts.CPU9.PMI:Performance_monitoring_interrupts
23911 ± 5% -50.8% 11756 ± 2% interrupts.CPU9.TLB:TLB_shootdowns
26653 ± 8% -44.2% 14876 ± 10% interrupts.CPU90.CAL:Function_call_interrupts
6351 ± 3% -31.2% 4370 ± 32% interrupts.CPU90.NMI:Non-maskable_interrupts
6351 ± 3% -31.2% 4370 ± 32% interrupts.CPU90.PMI:Performance_monitoring_interrupts
23402 ± 8% -49.3% 11858 ± 13% interrupts.CPU90.TLB:TLB_shootdowns
27116 ± 10% -46.6% 14491 ± 10% interrupts.CPU91.CAL:Function_call_interrupts
6467 ± 4% -33.5% 4301 ± 34% interrupts.CPU91.NMI:Non-maskable_interrupts
6467 ± 4% -33.5% 4301 ± 34% interrupts.CPU91.PMI:Performance_monitoring_interrupts
23824 ± 11% -51.6% 11536 ± 12% interrupts.CPU91.TLB:TLB_shootdowns
26961 ± 11% -46.1% 14532 ± 8% interrupts.CPU92.CAL:Function_call_interrupts
23679 ± 13% -51.4% 11497 ± 11% interrupts.CPU92.TLB:TLB_shootdowns
27548 ± 10% -46.2% 14817 ± 11% interrupts.CPU93.CAL:Function_call_interrupts
6512 ± 5% -30.4% 4531 ± 38% interrupts.CPU93.NMI:Non-maskable_interrupts
6512 ± 5% -30.4% 4531 ± 38% interrupts.CPU93.PMI:Performance_monitoring_interrupts
24280 ± 11% -51.2% 11841 ± 14% interrupts.CPU93.TLB:TLB_shootdowns
27800 ± 11% -47.4% 14610 ± 9% interrupts.CPU94.CAL:Function_call_interrupts
6433 ± 4% -41.1% 3789 ± 45% interrupts.CPU94.NMI:Non-maskable_interrupts
6433 ± 4% -41.1% 3789 ± 45% interrupts.CPU94.PMI:Performance_monitoring_interrupts
24567 ± 11% -52.6% 11637 ± 11% interrupts.CPU94.TLB:TLB_shootdowns
27488 ± 9% -44.3% 15305 ± 7% interrupts.CPU95.CAL:Function_call_interrupts
24215 ± 10% -48.8% 12388 ± 9% interrupts.CPU95.TLB:TLB_shootdowns
24706 ± 3% -47.9% 12861 ± 10% interrupts.CPU96.CAL:Function_call_interrupts
21352 ± 3% -54.2% 9779 ± 12% interrupts.CPU96.TLB:TLB_shootdowns
26692 ± 6% -46.5% 14274 ± 6% interrupts.CPU97.CAL:Function_call_interrupts
6129 ± 4% -35.1% 3975 ± 30% interrupts.CPU97.NMI:Non-maskable_interrupts
6129 ± 4% -35.1% 3975 ± 30% interrupts.CPU97.PMI:Performance_monitoring_interrupts
23379 ± 4% -51.5% 11344 ± 8% interrupts.CPU97.TLB:TLB_shootdowns
26419 ± 6% -46.1% 14244 ± 7% interrupts.CPU98.CAL:Function_call_interrupts
23076 ± 7% -50.9% 11339 ± 10% interrupts.CPU98.TLB:TLB_shootdowns
26972 ± 7% -44.4% 14989 ± 12% interrupts.CPU99.CAL:Function_call_interrupts
6269 ± 5% -46.7% 3344 ± 27% interrupts.CPU99.NMI:Non-maskable_interrupts
6269 ± 5% -46.7% 3344 ± 27% interrupts.CPU99.PMI:Performance_monitoring_interrupts
23671 ± 8% -49.2% 12023 ± 16% interrupts.CPU99.TLB:TLB_shootdowns
170.25 -44.5% 94.50 ± 47% interrupts.IWI:IRQ_work_interrupts
1103722 -28.1% 794110 ± 14% interrupts.NMI:Non-maskable_interrupts
1103722 -28.1% 794110 ± 14% interrupts.PMI:Performance_monitoring_interrupts
4531323 ± 2% -48.8% 2317840 ± 6% interrupts.TLB:TLB_shootdowns
vm-scalability.time.user_time
25000 +-+-----------------------------------------------------------------+
| +.+.+.+.+ +.+.+.+.+ +.+.+.+.+.+.+.+ +.+ +.+.+.+.+.+.|
| O |
20000 O-+ O O O O O O O |
| |
| O O O O |
15000 +-+ |
| |
10000 +-+ |
| |
| |
5000 +-+ |
| |
| |
0 +-O-O-O-O-----------------------------------------------------------+
vm-scalability.time.system_time
25000 +-+-----------------------------------------------------------------+
|. .+.+.+.+. .+. .+. .+. |
| + +.+.+.+.+.+.+.+ +.+.+.+.+.+.+ +.+.+ +.+.+.+.+.+.+.|
20000 O-+ O O O O O O O O |
| O O |
| O O |
15000 +-+ |
| |
10000 +-+ |
| |
| |
5000 +-+ |
| |
| |
0 +-O-O-O-O-----------------------------------------------------------+
vm-scalability.time.percent_of_cpu_this_job_got
16000 +-+-----------------------------------------------------------------+
|.+.+.+.+.+.+.+.+.+.+.+.+.+ +.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.|
14000 O-+ O O |
12000 +-+ O O O O O O |
| O O |
10000 +-+ O O |
| |
8000 +-+ |
| |
6000 +-+ |
4000 +-+ |
| |
2000 +-+ |
| |
0 +-O-O-O-O-----------------------------------------------------------+
vm-scalability.time.minor_page_faults
1.6e+07 +-+---------------------------------------------------------------+
| |
1.4e+07 +-+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.|
1.2e+07 O-+ O O O OO O O O |
| |
1e+07 +-+ O O O O |
| |
8e+06 +-+ |
| |
6e+06 +-+ |
4e+06 +-+ |
| |
2e+06 +-+ |
| |
0 +-O-O-O-O---------------------------------------------------------+
vm-scalability.time.voluntary_context_switches
1.6e+06 +-+---------------------------------------------------------------+
|.+. .+. .+. .++. .+. .+.+. .+. .+.+. .+ .+. + + |
1.4e+06 O-+ + +.O O O O + O O +.+.+ + +.+.+ + + |
| O O |
1.2e+06 +-+ |
1e+06 +-+ |
| |
800000 +-+ O O O O |
| |
600000 +-+ |
400000 +-+ |
| |
200000 +-+ |
| |
0 +-O-O-O-O---------------------------------------------------------+
vm-scalability.time.involuntary_context_switches
350000 +-+----------------------------------------------------------------+
| .+.+. .+. +. .+. .+.+. .+.+. .+. |
300000 +-+.+.+.+.+.+ +.+ +.+.+.+.+.+ + +.+.+ +.+.+ + +.|
| |
250000 +-+ |
| |
200000 +-+ |
O O O |
150000 +-+ O O O O O O |
| O |
100000 +-+ O O O |
| |
50000 +-+ |
| |
0 +-O-O-O-O----------------------------------------------------------+
vm-scalability.free_time
0.025 +-+-----------------------O-----O-----------------------------------+
| |
| O O |
0.02 +-+ |
| |
| |
0.015 +-+ |
| O O O O O |
0.01 O-+ O O O |
|.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.|
| |
0.005 +-+ |
| |
| |
0 +-O-O-O-O-----------------------------------------------------------+
vm-scalability.median
200000 +-+----------------------------------------------------------------+
180000 +-+ + +.+.O O O O + + + +.+ + + + |
O O O O O |
160000 +-+ O O O O |
140000 +-+ |
| |
120000 +-+ |
100000 +-+ |
80000 +-+ |
| |
60000 +-+ |
40000 +-+ |
| |
20000 +-+ |
0 +-O-O-O-O----------------------------------------------------------+
vm-scalability.median_stddev
0.12 +-+------------------------------------------------------------------+
| O O |
0.1 +-+ O |
| O |
O O O O O |
0.08 +-+ O O O |
| O |
0.06 +-+ |
| +. |
0.04 +-+ .+.+. .+.+. .+. .+.+.+. .+.. .+. .+. .+.+. .+. + +. .+.|
|.+.+ + + + +.+ + + + + +.+.+.+ + |
| |
0.02 +-+ |
| |
0 +-O-O-O-O------------------------------------------------------------+
vm-scalability.workload
1.2e+10 +-+---------------------------------------------------------------+
| |
1e+10 +-+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.|
O O O O OO O O O |
| |
8e+09 +-+ O O O O |
| |
6e+09 +-+ |
| |
4e+09 +-+ |
| |
| |
2e+09 +-+ |
| |
0 +-O-O-O-O---------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-ivb-ep01: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase:
gcc-7/performance/1BRD_48G/f2fs/x86_64-rhel-7.6/3000/debian-x86_64-2019-05-14.cgz/lkp-ivb-ep01/disk_src/aim7
commit:
94a9717b3c ("locking/rwsem: Make rwsem->owner an atomic_long_t")
7d43f1ce9d ("locking/rwsem: Enable time-based spinning on reader-owned rwsem")
94a9717b3c40e77a 7d43f1ce9dd075d8b2aa3ad1f39
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:2 50% 1:2 dmesg.WARNING:at_ip_do_filp_open/0x
***************************************************************************************************
lkp-ivb-d02: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2019-05-14.cgz/300s/lkp-ivb-d02/small-allocs-mt/vm-scalability/0x21
commit:
94a9717b3c ("locking/rwsem: Make rwsem->owner an atomic_long_t")
7d43f1ce9d ("locking/rwsem: Enable time-based spinning on reader-owned rwsem")
94a9717b3c40e77a 7d43f1ce9dd075d8b2aa3ad1f39
---------------- ---------------------------
%stddev %change %stddev
\ | \
793660 +14.0% 904895 vm-scalability.median
3173995 +14.0% 3617058 vm-scalability.throughput
151.84 -11.9% 133.72 vm-scalability.time.elapsed_time
151.84 -11.9% 133.72 vm-scalability.time.elapsed_time.max
713096 ± 2% +20.1% 856760 ± 2% vm-scalability.time.involuntary_context_switches
360.25 +4.9% 378.00 vm-scalability.time.percent_of_cpu_this_job_got
200.83 -19.5% 161.71 vm-scalability.time.system_time
7120853 ± 3% -97.5% 180425 vm-scalability.time.voluntary_context_switches
21.06 +5.8% 22.29 ± 4% boot-time.boot
54.91 +9.2% 59.95 ± 7% boot-time.idle
124534 ± 3% -9.1% 113147 meminfo.AnonHugePages
28262 +14.7% 32431 meminfo.max_used_kB
8.74 ± 4% -5.4 3.37 ± 3% mpstat.cpu.all.idle%
58.13 +7.7 65.86 mpstat.cpu.all.usr%
1207 ± 4% +14.8% 1385 ± 2% slabinfo.kmalloc-96.active_objs
1207 ± 4% +14.8% 1385 ± 2% slabinfo.kmalloc-96.num_objs
9.25 ± 4% -56.8% 4.00 vmstat.cpu.id
57.25 +12.7% 64.50 vmstat.cpu.us
101684 ± 2% -83.4% 16863 vmstat.system.cs
13871 +1.5% 14082 vmstat.system.in
5053 -1.2% 4991 proc-vmstat.nr_inactive_anon
5512 +1.4% 5588 proc-vmstat.nr_mapped
260063 +1.6% 264256 proc-vmstat.nr_slab_unreclaimable
5053 -1.2% 4991 proc-vmstat.nr_zone_inactive_anon
1113 -1.6% 1095 proc-vmstat.pgactivate
62845 ± 2% -74.7% 15909 ± 10% softirqs.CPU0.SCHED
61686 -72.7% 16834 ± 6% softirqs.CPU1.SCHED
62727 -76.0% 15049 ± 9% softirqs.CPU2.SCHED
60562 ± 2% -72.5% 16673 ± 4% softirqs.CPU3.SCHED
247822 -74.0% 64466 ± 2% softirqs.SCHED
30358831 ± 4% -94.8% 1581973 ± 4% cpuidle.C1.time
6764050 ± 3% -97.5% 169367 cpuidle.C1.usage
1648723 ± 27% +212.2% 5147756 ± 11% cpuidle.C1E.time
37741 ± 13% +132.6% 87804 ± 18% cpuidle.C1E.usage
104315 ± 7% +653.2% 785684 ± 76% cpuidle.C3.time
9526 ± 4% -71.0% 2761 ± 64% cpuidle.C3.usage
196095 ± 2% -98.8% 2432 ± 3% cpuidle.POLL.time
88406 ± 2% -97.4% 2325 ± 4% cpuidle.POLL.usage
3059 +3.8% 3176 turbostat.Avg_MHz
6764093 ± 3% -97.5% 169364 turbostat.C1
4.96 ± 3% -4.7 0.29 ± 3% turbostat.C1%
37886 ± 13% +131.7% 87799 ± 18% turbostat.C1E
0.27 ± 27% +0.7 0.95 ± 10% turbostat.C1E%
9529 ± 4% -71.1% 2754 ± 64% turbostat.C3
0.02 ± 24% +0.1 0.14 ± 72% turbostat.C3%
1.83 ± 3% +0.3 2.15 ± 2% turbostat.C6%
6.00 ± 3% -61.8% 2.29 ± 6% turbostat.CPU%c1
0.00 ±173% +25100.0% 0.63 ± 98% turbostat.CPU%c3
2138754 -10.3% 1918451 turbostat.IRQ
4014 -10.5% 3593 ± 2% interrupts.CAL:Function_call_interrupts
18427 ± 6% -95.1% 895.00 ± 11% interrupts.CPU0.RES:Rescheduling_interrupts
895.75 ±144% -100.0% 0.00 interrupts.CPU1.NMI:Non-maskable_interrupts
895.75 ±144% -100.0% 0.00 interrupts.CPU1.PMI:Performance_monitoring_interrupts
19299 ± 11% -94.1% 1139 ± 20% interrupts.CPU1.RES:Rescheduling_interrupts
58.00 ± 73% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
58.00 ± 73% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
19013 ± 8% -94.4% 1062 ± 24% interrupts.CPU2.RES:Rescheduling_interrupts
509962 ± 3% -20.9% 403302 ± 4% interrupts.CPU3.LOC:Local_timer_interrupts
631.75 ±163% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
631.75 ±163% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
18040 ± 12% -93.7% 1129 ± 14% interrupts.CPU3.RES:Rescheduling_interrupts
1607 ± 81% -99.6% 6.00 interrupts.NMI:Non-maskable_interrupts
1607 ± 81% -99.6% 6.00 interrupts.PMI:Performance_monitoring_interrupts
74780 ± 4% -94.3% 4226 ± 13% interrupts.RES:Rescheduling_interrupts
2.25 ± 23% +75.9% 3.96 ± 5% sched_debug.cfs_rq:/.nr_spread_over.avg
4.50 ± 44% +103.7% 9.17 ± 5% sched_debug.cfs_rq:/.nr_spread_over.max
1.64 ± 49% +101.5% 3.31 ± 3% sched_debug.cfs_rq:/.nr_spread_over.stddev
422.17 ± 14% +29.7% 547.50 ± 3% sched_debug.cfs_rq:/.runnable_load_avg.max
135.56 ± 13% +34.4% 182.15 ± 8% sched_debug.cfs_rq:/.runnable_load_avg.stddev
1047 ± 12% +33.7% 1400 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max
416.51 ± 11% +24.1% 516.73 sched_debug.cfs_rq:/.util_est_enqueued.stddev
273573 ± 4% +14.6% 313548 ± 9% sched_debug.cpu.avg_idle.avg
0.68 ± 22% +155.0% 1.73 ± 14% sched_debug.cpu.clock.stddev
0.68 ± 22% +155.0% 1.73 ± 14% sched_debug.cpu.clock_task.stddev
216.10 ± 10% +34.3% 290.33 ± 12% sched_debug.cpu.cpu_load[0].avg
401.50 ± 23% +40.5% 564.17 ± 4% sched_debug.cpu.cpu_load[0].max
112.08 ± 39% +65.0% 184.96 ± 7% sched_debug.cpu.cpu_load[0].stddev
211.88 ± 9% +17.9% 249.83 ± 4% sched_debug.cpu.cpu_load[1].avg
325.50 ± 15% +47.4% 479.83 ± 7% sched_debug.cpu.cpu_load[1].max
72.16 ± 25% +102.9% 146.40 ± 12% sched_debug.cpu.cpu_load[1].stddev
205.62 ± 6% +11.2% 228.71 ± 3% sched_debug.cpu.cpu_load[2].avg
273.50 ± 9% +47.5% 403.33 sched_debug.cpu.cpu_load[2].max
47.18 ± 17% +128.6% 107.86 sched_debug.cpu.cpu_load[2].stddev
202.62 ± 4% +7.5% 217.75 ± 3% sched_debug.cpu.cpu_load[3].avg
246.67 ± 5% +38.0% 340.50 ± 7% sched_debug.cpu.cpu_load[3].max
32.83 ± 7% +127.3% 74.63 ± 18% sched_debug.cpu.cpu_load[3].stddev
236.42 ± 4% +26.2% 298.33 ± 11% sched_debug.cpu.cpu_load[4].max
25.55 ± 13% +105.2% 52.42 ± 30% sched_debug.cpu.cpu_load[4].stddev
0.00 ± 14% -50.1% 0.00 sched_debug.cpu.next_balance.stddev
1905769 ± 2% -79.6% 387826 sched_debug.cpu.nr_switches.avg
1972844 ± 2% -73.6% 520051 ± 5% sched_debug.cpu.nr_switches.max
1744240 -84.6% 269070 ± 3% sched_debug.cpu.nr_switches.min
23.92 ± 55% +130.7% 55.17 ± 45% sched_debug.cpu.nr_uninterruptible.max
18.08 ± 60% +118.2% 39.45 ± 26% sched_debug.cpu.nr_uninterruptible.stddev
1895971 ± 2% -80.1% 377492 sched_debug.cpu.sched_count.avg
1962479 ± 2% -73.9% 512285 ± 5% sched_debug.cpu.sched_count.max
1734135 -84.9% 261092 ± 2% sched_debug.cpu.sched_count.min
794818 ± 2% -94.8% 41267 ± 8% sched_debug.cpu.sched_goidle.avg
825986 ± 3% -93.5% 54059 ± 12% sched_debug.cpu.sched_goidle.max
738516 ± 2% -96.2% 28020 sched_debug.cpu.sched_goidle.min
33586 ± 11% -71.6% 9536 ± 24% sched_debug.cpu.sched_goidle.stddev
986934 ± 2% -80.6% 191360 sched_debug.cpu.ttwu_count.avg
1048238 ± 2% -74.3% 269323 ± 5% sched_debug.cpu.ttwu_count.max
882027 ± 2% -86.0% 123850 ± 5% sched_debug.cpu.ttwu_count.min
179043 ± 2% +29.0% 230973 ± 6% sched_debug.cpu.ttwu_local.max
130127 ± 4% -22.1% 101364 ± 6% sched_debug.cpu.ttwu_local.min
18651 ± 12% +174.2% 51141 ± 9% sched_debug.cpu.ttwu_local.stddev
2.32 ± 16% -62.1% 0.88 perf-stat.i.MPKI
3.793e+09 +10.4% 4.187e+09 perf-stat.i.branch-instructions
2.64 ± 14% +2.8 5.39 ± 6% perf-stat.i.cache-miss-rate%
499435 ± 17% +37.6% 687301 ± 9% perf-stat.i.cache-misses
24288477 ± 3% -51.9% 11692134 perf-stat.i.cache-references
103479 ± 3% -83.4% 17136 perf-stat.i.context-switches
1.02 -6.7% 0.95 perf-stat.i.cpi
1.23e+10 +3.9% 1.278e+10 perf-stat.i.cpu-cycles
1998 ± 2% -96.2% 76.32 ± 10% perf-stat.i.cpu-migrations
3.219e+09 +9.8% 3.533e+09 perf-stat.i.dTLB-loads
0.28 +0.0 0.29 perf-stat.i.dTLB-store-miss-rate%
2078000 +4.5% 2171902 perf-stat.i.dTLB-store-misses
7.347e+08 +2.0% 7.492e+08 perf-stat.i.dTLB-stores
84.00 +14.2 98.21 perf-stat.i.iTLB-load-miss-rate%
1805387 ± 6% -31.7% 1232340 perf-stat.i.iTLB-load-misses
353772 ± 4% -95.3% 16649 perf-stat.i.iTLB-loads
1.226e+10 +9.6% 1.344e+10 perf-stat.i.instructions
8575 ± 12% +69.5% 14535 ± 3% perf-stat.i.instructions-per-iTLB-miss
1.00 +6.2% 1.06 perf-stat.i.ipc
690868 +13.2% 781827 perf-stat.i.minor-faults
690868 +13.2% 781827 perf-stat.i.page-faults
1.98 ± 3% -56.1% 0.87 perf-stat.overall.MPKI
2.06 ± 18% +3.8 5.89 ± 11% perf-stat.overall.cache-miss-rate%
1.00 -5.2% 0.95 perf-stat.overall.cpi
0.28 +0.0 0.29 perf-stat.overall.dTLB-store-miss-rate%
83.57 +15.1 98.67 perf-stat.overall.iTLB-load-miss-rate%
6818 ± 6% +59.9% 10905 perf-stat.overall.instructions-per-iTLB-miss
1.00 +5.5% 1.05 perf-stat.overall.ipc
3951 -3.2% 3826 perf-stat.overall.path-length
3.771e+09 +10.4% 4.164e+09 perf-stat.ps.branch-instructions
496503 ± 17% +37.7% 683817 ± 9% perf-stat.ps.cache-misses
24145521 ± 3% -51.8% 11629801 perf-stat.ps.cache-references
102871 ± 3% -83.4% 17045 perf-stat.ps.context-switches
1.223e+10 +3.9% 1.271e+10 perf-stat.ps.cpu-cycles
1986 ± 2% -96.2% 75.93 ± 10% perf-stat.ps.cpu-migrations
3.2e+09 +9.8% 3.514e+09 perf-stat.ps.dTLB-loads
2065804 +4.6% 2160396 perf-stat.ps.dTLB-store-misses
7.304e+08 +2.0% 7.453e+08 perf-stat.ps.dTLB-stores
1794803 ± 6% -31.7% 1225789 perf-stat.ps.iTLB-load-misses
351694 ± 4% -95.3% 16561 perf-stat.ps.iTLB-loads
1.219e+10 +9.7% 1.337e+10 perf-stat.ps.instructions
686822 +13.2% 777664 perf-stat.ps.minor-faults
686822 +13.2% 777664 perf-stat.ps.page-faults
1.861e+12 -3.2% 1.801e+12 perf-stat.total.instructions
56.68 ± 69% -56.7 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
56.68 ± 69% -56.7 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
56.32 ± 70% -56.3 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
56.32 ± 70% -56.3 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
56.32 ± 70% -56.3 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
56.11 ± 71% -56.1 0.00 perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
56.07 ± 71% -56.1 0.00 perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
36.23 ±101% -36.2 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64
35.47 ±105% -35.5 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
35.47 ±105% -35.5 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
35.47 ±105% -35.5 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
34.78 ±106% -34.8 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
34.75 ±106% -34.7 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
29.00 ±110% -29.0 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
19.43 ± 97% -19.4 0.00 perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit
18.70 ± 59% -18.7 0.00 perf-profile.calltrace.cycles-pp.remove_vma.exit_mmap.mmput.do_exit.do_group_exit
14.99 ± 96% -15.0 0.00 perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit
14.52 ±108% -14.5 0.00 perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
12.42 ±108% -12.4 0.00 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
10.42 ± 59% -10.4 0.00 perf-profile.calltrace.cycles-pp.kmem_cache_free.remove_vma.exit_mmap.mmput.do_exit
7.54 ± 95% -7.5 0.00 perf-profile.calltrace.cycles-pp.__rb_erase_color.unlink_file_vma.free_pgtables.exit_mmap.mmput
6.58 ± 61% -6.6 0.00 perf-profile.calltrace.cycles-pp.__slab_free.kmem_cache_free.remove_vma.exit_mmap.mmput
6.29 ±111% -6.3 0.00 perf-profile.calltrace.cycles-pp.con_scroll.lf.vt_console_print.console_unlock.vprintk_emit
6.29 ±111% -6.3 0.00 perf-profile.calltrace.cycles-pp.fbcon_scroll.con_scroll.lf.vt_console_print.console_unlock
6.29 ±111% -6.3 0.00 perf-profile.calltrace.cycles-pp.fbcon_redraw.fbcon_scroll.con_scroll.lf.vt_console_print
6.12 ±113% -6.1 0.00 perf-profile.calltrace.cycles-pp.fbcon_putcs.fbcon_redraw.fbcon_scroll.con_scroll.lf
6.12 ±113% -6.1 0.00 perf-profile.calltrace.cycles-pp.bit_putcs.fbcon_putcs.fbcon_redraw.fbcon_scroll.con_scroll
5.22 ±113% -5.2 0.00 perf-profile.calltrace.cycles-pp.drm_fb_helper_cfb_imageblit.bit_putcs.fbcon_putcs.fbcon_redraw.fbcon_scroll
5.22 ±113% -5.2 0.00 perf-profile.calltrace.cycles-pp.cfb_imageblit.drm_fb_helper_cfb_imageblit.bit_putcs.fbcon_putcs.fbcon_redraw
3.95 ±103% -3.9 0.00 perf-profile.calltrace.cycles-pp.vprintk_emit.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt
3.95 ±103% -3.9 0.00 perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.irq_work_run_list.irq_work_run
3.95 ±103% -3.9 0.00 perf-profile.calltrace.cycles-pp.vt_console_print.console_unlock.vprintk_emit.printk.irq_work_run_list
3.95 ±103% -3.9 0.00 perf-profile.calltrace.cycles-pp.lf.vt_console_print.console_unlock.vprintk_emit.printk
3.95 ±103% -3.9 0.00 perf-profile.calltrace.cycles-pp.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt.irq_work_interrupt
3.76 ± 95% -3.8 0.00 perf-profile.calltrace.cycles-pp.vma_interval_tree_remove.unlink_file_vma.free_pgtables.exit_mmap.mmput
3.00 ±101% -3.0 0.00 perf-profile.calltrace.cycles-pp.down_write.unlink_file_vma.free_pgtables.exit_mmap.mmput
61.00 ± 61% -61.0 0.00 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
60.99 ± 61% -61.0 0.00 perf-profile.children.cycles-pp.do_syscall_64
56.33 ± 70% -56.3 0.00 perf-profile.children.cycles-pp.do_group_exit
56.33 ± 70% -56.3 0.00 perf-profile.children.cycles-pp.do_exit
56.32 ± 70% -56.3 0.00 perf-profile.children.cycles-pp.__x64_sys_exit_group
56.12 ± 71% -56.1 0.00 perf-profile.children.cycles-pp.mmput
56.12 ± 71% -56.1 0.00 perf-profile.children.cycles-pp.exit_mmap
36.23 ±101% -36.2 0.00 perf-profile.children.cycles-pp.secondary_startup_64
36.23 ±101% -36.2 0.00 perf-profile.children.cycles-pp.cpu_startup_entry
36.23 ±101% -36.2 0.00 perf-profile.children.cycles-pp.do_idle
35.48 ±102% -35.5 0.00 perf-profile.children.cycles-pp.cpuidle_enter
35.48 ±102% -35.5 0.00 perf-profile.children.cycles-pp.cpuidle_enter_state
35.47 ±105% -35.5 0.00 perf-profile.children.cycles-pp.start_secondary
29.21 ±110% -29.2 0.00 perf-profile.children.cycles-pp.intel_idle
19.51 ± 97% -19.5 0.00 perf-profile.children.cycles-pp.free_pgtables
19.14 ± 59% -19.1 0.00 perf-profile.children.cycles-pp.remove_vma
15.08 ± 96% -15.1 0.00 perf-profile.children.cycles-pp.unlink_file_vma
14.52 ±108% -14.5 0.00 perf-profile.children.cycles-pp.unmap_vmas
12.67 ±107% -12.7 0.00 perf-profile.children.cycles-pp.unmap_page_range
10.58 ± 58% -10.6 0.00 perf-profile.children.cycles-pp.kmem_cache_free
7.55 ± 95% -7.6 0.00 perf-profile.children.cycles-pp.__rb_erase_color
7.39 ±102% -7.4 0.00 perf-profile.children.cycles-pp.console_unlock
7.36 ±103% -7.4 0.00 perf-profile.children.cycles-pp.vprintk_emit
7.17 ±101% -7.2 0.00 perf-profile.children.cycles-pp.vt_console_print
6.99 ±103% -7.0 0.00 perf-profile.children.cycles-pp.fbcon_putcs
6.99 ±103% -7.0 0.00 perf-profile.children.cycles-pp.bit_putcs
6.75 ± 97% -6.7 0.00 perf-profile.children.cycles-pp.lf
6.75 ± 97% -6.7 0.00 perf-profile.children.cycles-pp.con_scroll
6.75 ± 97% -6.7 0.00 perf-profile.children.cycles-pp.fbcon_scroll
6.74 ± 98% -6.7 0.00 perf-profile.children.cycles-pp.fbcon_redraw
6.63 ± 60% -6.6 0.00 perf-profile.children.cycles-pp.__slab_free
5.98 ±105% -6.0 0.00 perf-profile.children.cycles-pp.drm_fb_helper_cfb_imageblit
5.98 ±105% -6.0 0.00 perf-profile.children.cycles-pp.cfb_imageblit
4.29 ± 87% -4.3 0.00 perf-profile.children.cycles-pp.irq_work_interrupt
4.29 ± 87% -4.3 0.00 perf-profile.children.cycles-pp.smp_irq_work_interrupt
4.29 ± 87% -4.3 0.00 perf-profile.children.cycles-pp.irq_work_run_list
4.29 ± 87% -4.3 0.00 perf-profile.children.cycles-pp.irq_work_run
4.29 ± 87% -4.3 0.00 perf-profile.children.cycles-pp.printk
4.05 ± 55% -4.0 0.00 perf-profile.children.cycles-pp.___might_sleep
3.82 ± 95% -3.8 0.00 perf-profile.children.cycles-pp.vma_interval_tree_remove
3.19 ± 98% -3.2 0.00 perf-profile.children.cycles-pp.down_write
29.21 ±110% -29.2 0.00 perf-profile.self.cycles-pp.intel_idle
7.47 ±108% -7.5 0.00 perf-profile.self.cycles-pp.unmap_page_range
6.47 ± 59% -6.5 0.00 perf-profile.self.cycles-pp.__slab_free
5.98 ±105% -6.0 0.00 perf-profile.self.cycles-pp.cfb_imageblit
5.36 ± 97% -5.4 0.00 perf-profile.self.cycles-pp.__rb_erase_color
3.97 ± 55% -4.0 0.00 perf-profile.self.cycles-pp.___might_sleep
3.73 ± 95% -3.7 0.00 perf-profile.self.cycles-pp.vma_interval_tree_remove
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 6 months
9a23f5dbd7 ("Merge 'arm-soc/tegra/dt64' into .."): BUG: kernel reboot-without-warning in boot stage
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://internal_merge_and_test_tree devel-catchup-201906252112
commit 9a23f5dbd7bf84223f6afe70bd50c854c54ad3bc
Merge: 8569ea7437777 a586c88eab656
Author: 0day robot <lkp(a)intel.com>
AuthorDate: Tue Jun 25 21:12:57 2019 +0800
Commit: 0day robot <lkp(a)intel.com>
CommitDate: Tue Jun 25 21:12:57 2019 +0800
Merge 'arm-soc/tegra/dt64' into devel-catchup-201906252112
8569ea7437 Merge 'arm-soc/sunxi/dt' into devel-catchup-201906252112
a586c88eab arm64: tegra: Enable PCIe slots in P2972-0000 board
9a23f5dbd7 Merge 'arm-soc/tegra/dt64' into devel-catchup-201906252112
b65ee3b5ec 0day head guard for 'devel-catchup-201906252112'
+-------------------------------------------------------+------------+------------+------------+------------+
| | 8569ea7437 | a586c88eab | 9a23f5dbd7 | b65ee3b5ec |
+-------------------------------------------------------+------------+------------+------------+------------+
| boot_successes | 30 | 30 | 0 | 0 |
| boot_failures | 9 | 3 | 17 | 23 |
| BUG:kernel_timeout_in_boot_stage | 5 | | | |
| BUG:kernel_timeout_in_boot-around-mounting-root_stage | 2 | | | |
| BUG:kernel_hang_in_boot-around-mounting-root_stage | 2 | | | |
| invoked_oom-killer:gfp_mask=0x | 0 | 2 | | |
| Mem-Info | 0 | 3 | | |
| RIP:copy_user_generic_unrolled | 0 | 1 | | |
| BUG:kernel_reboot-without-warning_in_boot_stage | 0 | 0 | 17 | 23 |
+-------------------------------------------------------+------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 0.024775] check: Scanning 1 areas for low memory corruption
[ 0.025752] BRK [0x05201000, 0x05201fff] PGTABLE
[ 0.026538] BRK [0x05202000, 0x05202fff] PGTABLE
[ 0.027311] BRK [0x05203000, 0x05203fff] PGTABLE
[ 0.030561] BRK [0x05204000, 0x05204fff] PGTABLE
BUG: kernel reboot-without-warning in boot stage
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start b65ee3b5ecf5f1387642db259b7f39c279678315 9e0babf2c06c73cda2c0cd37a1653d823adb40ec --
git bisect good a81bdf82c8af97246edca3c5c4f4d023c914a6f5 # 12:48 G 10 0 1 1 Merge 'arm-soc/qcom/dt64' into devel-catchup-201906252112
git bisect bad 9a23f5dbd7bf84223f6afe70bd50c854c54ad3bc # 12:57 B 0 7 22 0 Merge 'arm-soc/tegra/dt64' into devel-catchup-201906252112
git bisect good 8569ea7437777addc37cd5e64f8f8f4d5dda7de6 # 13:47 G 10 0 1 1 Merge 'arm-soc/sunxi/dt' into devel-catchup-201906252112
git bisect good a4ff413b714d5b627bb81f5aca6d30e4714260e4 # 14:30 G 10 0 1 1 arm64: tegra: Add ID EEPROM for Jetson TX2 module
git bisect good ece6031ece2dd64d63708cfe1088016cee5b10c0 # 14:55 G 11 0 1 1 arm64: tegra: Update Jetson TX1 GPU regulator timings
git bisect good 9a182db4576d0f5d729a6900f16e4430f8fb1890 # 15:49 G 10 0 0 0 arm64: tegra: Enable ACONNECT, ADMA and AGIC
git bisect good 2602c32f15e71729f3d638da5ad1692099cacf97 # 16:35 G 10 0 1 1 arm64: tegra: Add P2U and PCIe controller nodes to Tegra194 DT
git bisect good a586c88eab65619f3654194dc90d46c98e712af2 # 17:04 G 10 0 3 3 arm64: tegra: Enable PCIe slots in P2972-0000 board
# first bad commit: [9a23f5dbd7bf84223f6afe70bd50c854c54ad3bc] Merge 'arm-soc/tegra/dt64' into devel-catchup-201906252112
git bisect good 8569ea7437777addc37cd5e64f8f8f4d5dda7de6 # 17:17 G 30 0 7 9 Merge 'arm-soc/sunxi/dt' into devel-catchup-201906252112
git bisect good a586c88eab65619f3654194dc90d46c98e712af2 # 17:36 G 30 0 2 5 arm64: tegra: Enable PCIe slots in P2972-0000 board
# extra tests on HEAD of linux-devel/devel-catchup-201906252112
git bisect bad b65ee3b5ecf5f1387642db259b7f39c279678315 # 17:37 B 0 23 51 0 0day head guard for 'devel-catchup-201906252112'
# extra tests on tree/branch linux-devel/devel-catchup-201906252112
git bisect bad b65ee3b5ecf5f1387642db259b7f39c279678315 # 17:37 B 0 23 51 0 0day head guard for 'devel-catchup-201906252112'
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 6 months
[bpf/tools] cd17d77705: kernel_selftests.bpf.test_sock_addr.sh.fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: cd17d77705780e2270937fb3cbd2b985adab3edc ("bpf/tools: sync bpf.h")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-00
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
# selftests: bpf: test_sock_addr.sh
# Wait for testing IPv4/IPv6 to become available ... OK
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int connect_v4_prog(struct bpf_sock_addr *ctx)
# 0: (bf) r6 = r1
# 1: (b7) r1 = 1544617984
# ; memset(&tuple.ipv4.sport, 0, sizeof(tuple.ipv4.sport));
# 2: (7b) *(u64 *)(r10 -32) = r1
# 3: (18) r1 = 0x100007f00000000
# ; memset(&tuple.ipv4.saddr, 0, sizeof(tuple.ipv4.saddr));
# 5: (7b) *(u64 *)(r10 -40) = r1
# 6: (b7) r7 = 0
# ; struct bpf_sock_tuple tuple = {};
# 7: (63) *(u32 *)(r10 -8) = r7
# 8: (63) *(u32 *)(r10 -12) = r7
# 9: (63) *(u32 *)(r10 -16) = r7
# 10: (63) *(u32 *)(r10 -20) = r7
# 11: (63) *(u32 *)(r10 -24) = r7
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 12: (61) r1 = *(u32 *)(r6 +32)
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 13: (bf) r2 = r1
# 14: (07) r2 += -1
# 15: (67) r2 <<= 32
# 16: (77) r2 >>= 32
# 17: (25) if r2 > 0x1 goto pc+24
# R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm
# ; else if (ctx->type == SOCK_STREAM)
# 18: (55) if r1 != 0x1 goto pc+8
# R1=inv1 R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm
# 19: (bf) r2 = r10
# ; sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv4),
# 20: (07) r2 += -40
# 21: (bf) r1 = r6
# 22: (b7) r3 = 12
# 23: (b7) r4 = -1
# 24: (b7) r5 = 0
# 25: (85) call bpf_sk_lookup_tcp#84
# 26: (05) goto pc+7
# ; if (!sk)
# 34: (15) if r0 == 0x0 goto pc+7
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; if (sk->src_ip4 != tuple.ipv4.daddr ||
# 35: (61) r1 = *(u32 *)(r0 +24)
# ; if (sk->src_ip4 != tuple.ipv4.daddr ||
# 36: (61) r2 = *(u32 *)(r10 -36)
# ; if (sk->src_ip4 != tuple.ipv4.daddr ||
# 37: (5d) if r1 != r2 goto pc+2
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_port != DST_REWRITE_PORT4) {
# 38: (61) r1 = *(u32 *)(r0 +44)
# ; if (sk->src_ip4 != tuple.ipv4.daddr ||
# 39: (15) if r1 == 0x115c goto pc+4
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 40: (bf) r1 = r0
# 41: (85) call bpf_sk_release#86
# ; }
# 42: (bf) r0 = r7
# 43: (95) exit
#
# from 39 to 44: R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv4444 R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????0000 fp-16=00000000 fp-24=00000000 fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 44: (bf) r1 = r0
# 45: (85) call bpf_sk_release#86
# 46: (b7) r1 = 23569
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT4);
# 47: (63) *(u32 *)(r6 +24) = r1
# 48: (b7) r1 = 16777343
# ; ctx->user_ip4 = bpf_htonl(DST_REWRITE_IP4);
# 49: (63) *(u32 *)(r6 +4) = r1
# invalid bpf_context access off=4 size=4
# processed 42 insns (limit 1000000) max_states_per_insn 0 total_states 10 peak_states 10 mark_read 8
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/connect4'
# libbpf: failed to load object './connect4_prog.o'
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int connect_v6_prog(struct bpf_sock_addr *ctx)
# 0: (bf) r6 = r1
# 1: (18) r1 = 0x100000000000000
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 3: (7b) *(u64 *)(r10 -16) = r1
# 4: (b7) r1 = 169476096
# ; memset(&tuple.ipv6.sport, 0, sizeof(tuple.ipv6.sport));
# 5: (63) *(u32 *)(r10 -8) = r1
# 6: (b7) r7 = 0
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 7: (7b) *(u64 *)(r10 -24) = r7
# 8: (7b) *(u64 *)(r10 -32) = r7
# 9: (7b) *(u64 *)(r10 -40) = r7
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 10: (61) r1 = *(u32 *)(r6 +32)
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 11: (bf) r2 = r1
# 12: (07) r2 += -1
# 13: (67) r2 <<= 32
# 14: (77) r2 >>= 32
# 15: (25) if r2 > 0x1 goto pc+33
# R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# ; else if (ctx->type == SOCK_STREAM)
# 16: (55) if r1 != 0x1 goto pc+8
# R1=inv1 R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# 17: (bf) r2 = r10
# ; sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv6),
# 18: (07) r2 += -40
# 19: (bf) r1 = r6
# 20: (b7) r3 = 36
# 21: (b7) r4 = -1
# 22: (b7) r5 = 0
# 23: (85) call bpf_sk_lookup_tcp#84
# 24: (05) goto pc+7
# ; if (!sk)
# 32: (15) if r0 == 0x0 goto pc+16
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 33: (61) r1 = *(u32 *)(r0 +28)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 34: (61) r2 = *(u32 *)(r10 -24)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 35: (5d) if r1 != r2 goto pc+11
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 36: (61) r1 = *(u32 *)(r0 +32)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 37: (61) r2 = *(u32 *)(r10 -20)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 38: (5d) if r1 != r2 goto pc+8
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 39: (61) r1 = *(u32 *)(r0 +36)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 40: (61) r2 = *(u32 *)(r10 -16)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 41: (5d) if r1 != r2 goto pc+5
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 42: (61) r1 = *(u32 *)(r0 +40)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 43: (61) r2 = *(u32 *)(r10 -12)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 44: (5d) if r1 != r2 goto pc+2
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_port != DST_REWRITE_PORT6) {
# 45: (61) r1 = *(u32 *)(r0 +44)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 46: (15) if r1 == 0x1a0a goto pc+4
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 47: (bf) r1 = r0
# 48: (85) call bpf_sk_release#86
# ; }
# 49: (bf) r0 = r7
# 50: (95) exit
#
# from 46 to 51: R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv6666 R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 51: (bf) r1 = r0
# 52: (85) call bpf_sk_release#86
# 53: (b7) r1 = 2586
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
# 54: (63) *(u32 *)(r6 +24) = r1
# 55: (18) r1 = 0x100000000000000
# ; ctx->user_ip6[2] = bpf_htonl(DST_REWRITE_IP6_2);
# 57: (7b) *(u64 *)(r6 +16) = r1
# invalid bpf_context access off=16 size=8
# processed 49 insns (limit 1000000) max_states_per_insn 0 total_states 13 peak_states 13 mark_read 11
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/connect6'
# libbpf: failed to load object './connect6_prog.o'
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int connect_v6_prog(struct bpf_sock_addr *ctx)
# 0: (bf) r6 = r1
# 1: (18) r1 = 0x100000000000000
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 3: (7b) *(u64 *)(r10 -16) = r1
# 4: (b7) r1 = 169476096
# ; memset(&tuple.ipv6.sport, 0, sizeof(tuple.ipv6.sport));
# 5: (63) *(u32 *)(r10 -8) = r1
# 6: (b7) r7 = 0
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 7: (7b) *(u64 *)(r10 -24) = r7
# 8: (7b) *(u64 *)(r10 -32) = r7
# 9: (7b) *(u64 *)(r10 -40) = r7
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 10: (61) r1 = *(u32 *)(r6 +32)
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 11: (bf) r2 = r1
# 12: (07) r2 += -1
# 13: (67) r2 <<= 32
# 14: (77) r2 >>= 32
# 15: (25) if r2 > 0x1 goto pc+33
# R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# ; else if (ctx->type == SOCK_STREAM)
# 16: (55) if r1 != 0x1 goto pc+8
# R1=inv1 R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# 17: (bf) r2 = r10
# ; sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv6),
# 18: (07) r2 += -40
# 19: (bf) r1 = r6
# 20: (b7) r3 = 36
# 21: (b7) r4 = -1
# 22: (b7) r5 = 0
# 23: (85) call bpf_sk_lookup_tcp#84
# 24: (05) goto pc+7
# ; if (!sk)
# 32: (15) if r0 == 0x0 goto pc+16
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 33: (61) r1 = *(u32 *)(r0 +28)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 34: (61) r2 = *(u32 *)(r10 -24)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 35: (5d) if r1 != r2 goto pc+11
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 36: (61) r1 = *(u32 *)(r0 +32)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 37: (61) r2 = *(u32 *)(r10 -20)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 38: (5d) if r1 != r2 goto pc+8
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 39: (61) r1 = *(u32 *)(r0 +36)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 40: (61) r2 = *(u32 *)(r10 -16)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 41: (5d) if r1 != r2 goto pc+5
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 42: (61) r1 = *(u32 *)(r0 +40)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 43: (61) r2 = *(u32 *)(r10 -12)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 44: (5d) if r1 != r2 goto pc+2
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_port != DST_REWRITE_PORT6) {
# 45: (61) r1 = *(u32 *)(r0 +44)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 46: (15) if r1 == 0x1a0a goto pc+4
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 47: (bf) r1 = r0
# 48: (85) call bpf_sk_release#86
# ; }
# 49: (bf) r0 = r7
# 50: (95) exit
#
# from 46 to 51: R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv6666 R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 51: (bf) r1 = r0
# 52: (85) call bpf_sk_release#86
# 53: (b7) r1 = 2586
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
# 54: (63) *(u32 *)(r6 +24) = r1
# 55: (18) r1 = 0x100000000000000
# ; ctx->user_ip6[2] = bpf_htonl(DST_REWRITE_IP6_2);
# 57: (7b) *(u64 *)(r6 +16) = r1
# invalid bpf_context access off=16 size=8
# processed 49 insns (limit 1000000) max_states_per_insn 0 total_states 13 peak_states 13 mark_read 11
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/connect6'
# libbpf: failed to load object './connect6_prog.o'
# (test_sock_addr.c:752: errno: Bad file descriptor) >>> Loading program (./connect6_prog.o) error.
#
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int connect_v6_prog(struct bpf_sock_addr *ctx)
# 0: (bf) r6 = r1
# 1: (18) r1 = 0x100000000000000
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 3: (7b) *(u64 *)(r10 -16) = r1
# 4: (b7) r1 = 169476096
# ; memset(&tuple.ipv6.sport, 0, sizeof(tuple.ipv6.sport));
# 5: (63) *(u32 *)(r10 -8) = r1
# 6: (b7) r7 = 0
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 7: (7b) *(u64 *)(r10 -24) = r7
# 8: (7b) *(u64 *)(r10 -32) = r7
# 9: (7b) *(u64 *)(r10 -40) = r7
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 10: (61) r1 = *(u32 *)(r6 +32)
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 11: (bf) r2 = r1
# 12: (07) r2 += -1
# 13: (67) r2 <<= 32
# 14: (77) r2 >>= 32
# 15: (25) if r2 > 0x1 goto pc+33
# R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# ; else if (ctx->type == SOCK_STREAM)
# 16: (55) if r1 != 0x1 goto pc+8
# R1=inv1 R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# 17: (bf) r2 = r10
# ; sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv6),
# 18: (07) r2 += -40
# 19: (bf) r1 = r6
# 20: (b7) r3 = 36
# 21: (b7) r4 = -1
# 22: (b7) r5 = 0
# 23: (85) call bpf_sk_lookup_tcp#84
# 24: (05) goto pc+7
# ; if (!sk)
# 32: (15) if r0 == 0x0 goto pc+16
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 33: (61) r1 = *(u32 *)(r0 +28)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 34: (61) r2 = *(u32 *)(r10 -24)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 35: (5d) if r1 != r2 goto pc+11
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 36: (61) r1 = *(u32 *)(r0 +32)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 37: (61) r2 = *(u32 *)(r10 -20)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 38: (5d) if r1 != r2 goto pc+8
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 39: (61) r1 = *(u32 *)(r0 +36)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 40: (61) r2 = *(u32 *)(r10 -16)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 41: (5d) if r1 != r2 goto pc+5
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 42: (61) r1 = *(u32 *)(r0 +40)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 43: (61) r2 = *(u32 *)(r10 -12)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 44: (5d) if r1 != r2 goto pc+2
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_port != DST_REWRITE_PORT6) {
# 45: (61) r1 = *(u32 *)(r0 +44)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 46: (15) if r1 == 0x1a0a goto pc+4
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 47: (bf) r1 = r0
# 48: (85) call bpf_sk_release#86
# ; }
# 49: (bf) r0 = r7
# 50: (95) exit
#
# from 46 to 51: R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv6666 R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 51: (bf) r1 = r0
# 52: (85) call bpf_sk_release#86
# 53: (b7) r1 = 2586
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
# 54: (63) *(u32 *)(r6 +24) = r1
# 55: (18) r1 = 0x100000000000000
# ; ctx->user_ip6[2] = bpf_htonl(DST_REWRITE_IP6_2);
# 57: (7b) *(u64 *)(r6 +16) = r1
# invalid bpf_context access off=16 size=8
# processed 49 insns (limit 1000000) max_states_per_insn 0 total_states 13 peak_states 13 mark_read 11
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/connect6'
# libbpf: failed to load object './connect6_prog.o'
# (test_sock_addr.c:752: errno: Bad file descriptor) >>> Loading program (./connect6_prog.o) error.
#
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int connect_v6_prog(struct bpf_sock_addr *ctx)
# 0: (bf) r6 = r1
# 1: (18) r1 = 0x100000000000000
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 3: (7b) *(u64 *)(r10 -16) = r1
# 4: (b7) r1 = 169476096
# ; memset(&tuple.ipv6.sport, 0, sizeof(tuple.ipv6.sport));
# 5: (63) *(u32 *)(r10 -8) = r1
# 6: (b7) r7 = 0
# ; tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
# 7: (7b) *(u64 *)(r10 -24) = r7
# 8: (7b) *(u64 *)(r10 -32) = r7
# 9: (7b) *(u64 *)(r10 -40) = r7
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 10: (61) r1 = *(u32 *)(r6 +32)
# ; if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
# 11: (bf) r2 = r1
# 12: (07) r2 += -1
# 13: (67) r2 <<= 32
# 14: (77) r2 >>= 32
# 15: (25) if r2 > 0x1 goto pc+33
# R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# ; else if (ctx->type == SOCK_STREAM)
# 16: (55) if r1 != 0x1 goto pc+8
# R1=inv1 R2=inv(id=0,umax_value=1,var_off=(0x0; 0x1)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=00000000 fp-32=00000000 fp-40=00000000
# 17: (bf) r2 = r10
# ; sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv6),
# 18: (07) r2 += -40
# 19: (bf) r1 = r6
# 20: (b7) r3 = 36
# 21: (b7) r4 = -1
# 22: (b7) r5 = 0
# 23: (85) call bpf_sk_lookup_tcp#84
# 24: (05) goto pc+7
# ; if (!sk)
# 32: (15) if r0 == 0x0 goto pc+16
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 33: (61) r1 = *(u32 *)(r0 +28)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 34: (61) r2 = *(u32 *)(r10 -24)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 35: (5d) if r1 != r2 goto pc+11
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 36: (61) r1 = *(u32 *)(r0 +32)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 37: (61) r2 = *(u32 *)(r10 -20)
# ; sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
# 38: (5d) if r1 != r2 goto pc+8
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 39: (61) r1 = *(u32 *)(r0 +36)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 40: (61) r2 = *(u32 *)(r10 -16)
# ; sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
# 41: (5d) if r1 != r2 goto pc+5
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 42: (61) r1 = *(u32 *)(r0 +40)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 43: (61) r2 = *(u32 *)(r10 -12)
# ; sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
# 44: (5d) if r1 != r2 goto pc+2
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; sk->src_port != DST_REWRITE_PORT6) {
# 45: (61) r1 = *(u32 *)(r0 +44)
# ; if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
# 46: (15) if r1 == 0x1a0a goto pc+4
# R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 47: (bf) r1 = r0
# 48: (85) call bpf_sk_release#86
# ; }
# 49: (bf) r0 = r7
# 50: (95) exit
#
# from 46 to 51: R0=sock(id=0,ref_obj_id=2,off=0,imm=0) R1=inv6666 R2=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R10=fp0,call_-1 fp-8=????mmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm refs=2
# ; bpf_sk_release(sk);
# 51: (bf) r1 = r0
# 52: (85) call bpf_sk_release#86
# 53: (b7) r1 = 2586
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
# 54: (63) *(u32 *)(r6 +24) = r1
# 55: (18) r1 = 0x100000000000000
# ; ctx->user_ip6[2] = bpf_htonl(DST_REWRITE_IP6_2);
# 57: (7b) *(u64 *)(r6 +16) = r1
# invalid bpf_context access off=16 size=8
# processed 49 insns (limit 1000000) max_states_per_insn 0 total_states 13 peak_states 13 mark_read 11
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/connect6'
# libbpf: failed to load object './connect6_prog.o'
# (test_sock_addr.c:752: errno: Bad file descriptor) >>> Loading program (./connect6_prog.o) error.
#
# (test_sock_addr.c:1122: errno: Operation not permitted) Fail to send message to server
# libbpf: load bpf program failed: Permission denied
# libbpf: -- BEGIN DUMP LOG ---
# libbpf:
# ; int sendmsg_v6_prog(struct bpf_sock_addr *ctx)
# 0: (b7) r0 = 0
# ; if (ctx->type != SOCK_DGRAM)
# 1: (61) r2 = *(u32 *)(r1 +32)
# ; if (ctx->type != SOCK_DGRAM)
# 2: (55) if r2 != 0x2 goto pc+20
# R0=inv0 R1=ctx(id=0,off=0,imm=0) R2=inv2 R10=fp0,call_-1
# ; if (ctx->msg_src_ip6[3] == bpf_htonl(1) ||
# 3: (61) r2 = *(u32 *)(r1 +56)
# ; if (ctx->msg_src_ip6[3] == bpf_htonl(1) ||
# 4: (47) r2 |= 16777216
# 5: (15) if r2 == 0x1000000 goto pc+1
# R0=inv0 R1=ctx(id=0,off=0,imm=0) R2=inv(id=0,umin_value=16777216,umax_value=4294967295,var_off=(0x1000000; 0xfeffffff)) R10=fp0,call_-1
# 6: (05) goto pc+16
# ; }
# 23: (95) exit
#
# from 5 to 7: R0=inv0 R1=ctx(id=0,off=0,imm=0) R2=inv16777216 R10=fp0,call_-1
# ; if (ctx->msg_src_ip6[3] == bpf_htonl(1) ||
# 7: (b7) r2 = 100663296
# ; ctx->msg_src_ip6[3] = bpf_htonl(SRC_REWRITE_IP6_3);
# 8: (63) *(u32 *)(r1 +56) = r2
# 9: (b7) r0 = 0
# ; ctx->msg_src_ip6[2] = bpf_htonl(SRC_REWRITE_IP6_2);
# 10: (63) *(u32 *)(r1 +52) = r0
# ; ctx->msg_src_ip6[1] = bpf_htonl(SRC_REWRITE_IP6_1);
# 11: (63) *(u32 *)(r1 +48) = r0
# ; ctx->msg_src_ip6[0] = bpf_htonl(SRC_REWRITE_IP6_0);
# 12: (63) *(u32 *)(r1 +44) = r0
# ; if ((ctx->user_ip6[0] & 0xFFFF) == bpf_htons(0xFACE) &&
# 13: (61) r2 = *(u32 *)(r1 +8)
# ; if ((ctx->user_ip6[0] & 0xFFFF) == bpf_htons(0xFACE) &&
# 14: (55) if r2 != 0xcb0cefa goto pc+8
# R0=inv0 R1=ctx(id=0,off=0,imm=0) R2=inv212913914 R10=fp0,call_-1
# 15: (b7) r2 = 2586
# ; ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
# 16: (63) *(u32 *)(r1 +24) = r2
# 17: (18) r2 = 0x100000000000000
# ; ctx->user_ip6[2] = bpf_htonl(DST_REWRITE_IP6_2);
# 19: (7b) *(u64 *)(r1 +16) = r2
# invalid bpf_context access off=16 size=8
# processed 20 insns (limit 1000000) max_states_per_insn 0 total_states 5 peak_states 5 mark_read 3
#
# libbpf: -- END LOG --
# libbpf: failed to load program 'cgroup/sendmsg6'
# libbpf: failed to load object './sendmsg6_prog.o'
# (test_sock_addr.c:752: errno: Bad file descriptor) >>> Loading program (./sendmsg6_prog.o) error.
#
# (test_sock_addr.c:1122: errno: Unknown error 524) Fail to send message to server
# (test_sock_addr.c:1122: errno: Operation not permitted) Fail to send message to server
# Test case: bind4: load prog with wrong expected attach type .. [PASS]
# Test case: bind4: attach prog with wrong attach type .. [PASS]
# Test case: bind4: rewrite IP & TCP port in .. [PASS]
# Test case: bind4: rewrite IP & UDP port in .. [PASS]
# Test case: bind6: load prog with wrong expected attach type .. [PASS]
# Test case: bind6: attach prog with wrong attach type .. [PASS]
# Test case: bind6: rewrite IP & TCP port in .. [PASS]
# Test case: bind6: rewrite IP & UDP port in .. [PASS]
# Test case: connect4: load prog with wrong expected attach type .. [PASS]
# Test case: connect4: attach prog with wrong attach type .. [PASS]
# Test case: connect4: rewrite IP & TCP port .. [PASS]
# Test case: connect4: rewrite IP & UDP port .. [PASS]
# Test case: connect6: load prog with wrong expected attach type .. [PASS]
# Test case: connect6: attach prog with wrong attach type .. [FAIL]
# Test case: connect6: rewrite IP & TCP port .. [FAIL]
# Test case: connect6: rewrite IP & UDP port .. [FAIL]
# Test case: sendmsg4: load prog with wrong expected attach type .. [PASS]
# Test case: sendmsg4: attach prog with wrong attach type .. [PASS]
# Test case: sendmsg4: rewrite IP & port (asm) .. [PASS]
# Test case: sendmsg4: rewrite IP & port (C) .. [PASS]
# Test case: sendmsg4: deny call .. [PASS]
# Test case: sendmsg6: load prog with wrong expected attach type .. [PASS]
# Test case: sendmsg6: attach prog with wrong attach type .. [PASS]
# Test case: sendmsg6: rewrite IP & port (asm) .. [PASS]
# Test case: sendmsg6: rewrite IP & port (C) .. [FAIL]
# Test case: sendmsg6: IPv4-mapped IPv6 .. [PASS]
# Test case: sendmsg6: set dst IP = [::] (BSD'ism) .. [PASS]
# Test case: sendmsg6: preserve dst IP = [::] (BSD'ism) .. [PASS]
# Test case: sendmsg6: deny call .. [PASS]
# Summary: 25 PASSED, 4 FAILED
not ok 34 selftests: bpf: test_sock_addr.sh
To reproduce:
# build kernel
cd linux
cp config-5.2.0-rc2-00597-gcd17d77 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 6 months
[nl80211] d8572a564d: assertion_failed
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: d8572a564d8aa17cb726a2aed04d0bbe52403249 ("[PATCH 2/3] nl80211: Limit certain commands to interface owner")
url: https://github.com/0day-ci/linux/commits/Denis-Kenzior/nl80211-Update-uap...
base: https://git.kernel.org/cgit/linux/kernel/git/jberg/mac80211-next.git master
in testcase: hwsim
with following parameters:
group: hwsim-13
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+---------------------------------------------------------------------------------+------------+------------+
| | 97b91db67a | d8572a564d |
+---------------------------------------------------------------------------------+------------+------------+
| boot_successes | 2 | 0 |
| boot_failures | 18 | 20 |
| BUG:kernel_reboot-without-warning_in_test_stage | 18 | 4 |
| assertion_failed | 0 | 16 |
| WARNING:at_net/wireless/reg.c:#reset_regdomains[cfg80211] | 0 | 16 |
| RIP:reset_regdomains[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/reg.c:#update_all_wiphy_regulatory[cfg80211] | 0 | 16 |
| RIP:update_all_wiphy_regulatory[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/mlme.c:#cfg80211_process_mlme_unregistrations[cfg80211] | 0 | 16 |
| RIP:cfg80211_process_mlme_unregistrations[cfg80211] | 0 | 16 |
| WARNING:at_net/core/dev.c:#call_netdevice_notifiers_info | 0 | 11 |
| RIP:call_netdevice_notifiers_info | 0 | 11 |
| WARNING:at_net/wireless/nl80211.c:#__cfg80211_wdev_from_attrs[cfg80211] | 0 | 16 |
| RIP:__cfg80211_wdev_from_attrs[cfg80211] | 0 | 16 |
| WARNING:at_net/sched/sch_api.c:#qdisc_hash_add | 0 | 10 |
| RIP:qdisc_hash_add | 0 | 10 |
| WARNING:at_net/core/rtnetlink.c:#rtnl_fill_ifinfo | 0 | 12 |
| RIP:rtnl_fill_ifinfo | 0 | 12 |
| WARNING:at_net/core/devlink.c:#devlink_compat_phys_port_name_get | 0 | 12 |
| RIP:devlink_compat_phys_port_name_get | 0 | 12 |
| WARNING:at_net/core/devlink.c:#devlink_compat_switch_id_get | 0 | 12 |
| RIP:devlink_compat_switch_id_get | 0 | 12 |
| WARNING:at_net/core/rtnetlink.c:#rtnl_xdp_prog_skb | 0 | 12 |
| RIP:rtnl_xdp_prog_skb | 0 | 12 |
| WARNING:at_net/core/fib_rules.c:#fib_rules_event | 0 | 11 |
| RIP:fib_rules_event | 0 | 11 |
| WARNING:at_net/ipv4/devinet.c:#inetdev_event | 0 | 11 |
| RIP:inetdev_event | 0 | 11 |
| WARNING:at_net/wireless/nl80211.c:#__cfg80211_rdev_from_attrs[cfg80211] | 0 | 16 |
| RIP:__cfg80211_rdev_from_attrs[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/core.c:#cfg80211_rdev_by_wiphy_idx[cfg80211] | 0 | 16 |
| RIP:cfg80211_rdev_by_wiphy_idx[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/util.c:#cfg80211_change_iface[cfg80211] | 0 | 16 |
| RIP:cfg80211_change_iface[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/util.c:#cfg80211_process_rdev_events[cfg80211] | 0 | 16 |
| RIP:cfg80211_process_rdev_events[cfg80211] | 0 | 16 |
| WARNING:at_net/mac80211/iface.c:#ieee80211_if_change_type[mac80211] | 0 | 16 |
| RIP:ieee80211_if_change_type[mac80211] | 0 | 16 |
| WARNING:at_net/wireless/nl80211.c:#nl80211_set_wiphy[cfg80211] | 0 | 16 |
| RIP:nl80211_set_wiphy[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/chan.c:#cfg80211_reg_can_beacon_relax[cfg80211] | 0 | 16 |
| RIP:cfg80211_reg_can_beacon_relax[cfg80211] | 0 | 16 |
| WARNING:at_net/wireless/reg.c:#restore_regulatory_settings[cfg80211] | 0 | 2 |
| RIP:restore_regulatory_settings[cfg80211] | 0 | 2 |
| WARNING:at_net/wireless/core.c:#wiphy_idx_to_wiphy[cfg80211] | 0 | 8 |
| RIP:wiphy_idx_to_wiphy[cfg80211] | 0 | 8 |
| WARNING:at_net/wireless/core.c:#__cfg80211_leave[cfg80211] | 0 | 3 |
| RIP:__cfg80211_leave[cfg80211] | 0 | 3 |
| BUG:unable_to_handle_page_fault_for_address | 0 | 1 |
| Oops:#[##] | 0 | 1 |
| RIP:netpoll_poll_disable | 0 | 1 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 2 |
| WARNING:at_net/ipv4/igmp.c:#ip_mc_up | 0 | 6 |
| RIP:ip_mc_up | 0 | 6 |
| WARNING:at_net/ipv4/igmp.c:#____ip_mc_inc_group | 0 | 6 |
| RIP:____ip_mc_inc_group | 0 | 6 |
| WARNING:at_net/wireless/core.c:#cfg80211_update_iface_num[cfg80211] | 0 | 7 |
| RIP:cfg80211_update_iface_num[cfg80211] | 0 | 7 |
| WARNING:at_net/wireless/wext-sme.c:#cfg80211_mgd_wext_connect[cfg80211] | 0 | 6 |
| RIP:cfg80211_mgd_wext_connect[cfg80211] | 0 | 6 |
| WARNING:at_net/ipv6/addrconf.c:#addrconf_dev_config | 0 | 4 |
| RIP:addrconf_dev_config | 0 | 4 |
| WARNING:at_net/ipv6/addrconf.c:#addrconf_add_dev | 0 | 4 |
| RIP:addrconf_add_dev | 0 | 4 |
| WARNING:at_net/ipv6/addrconf.c:#ipv6_find_idev | 0 | 4 |
| RIP:ipv6_find_idev | 0 | 4 |
| WARNING:at_net/mac80211/iface.c:#ieee80211_check_concurrent_iface[mac80211] | 0 | 4 |
| RIP:ieee80211_check_concurrent_iface[mac80211] | 0 | 4 |
| WARNING:at_net/wireless/reg.c:#set_regdom[cfg80211] | 0 | 7 |
| RIP:set_regdom[cfg80211] | 0 | 7 |
| general_protection_fault:#[##] | 0 | 1 |
| RIP:dev_deactivate_many | 0 | 1 |
| WARNING:at_net/wireless/chan.c:#cfg80211_any_wiphy_oper_chan[cfg80211] | 0 | 1 |
| RIP:cfg80211_any_wiphy_oper_chan[cfg80211] | 0 | 1 |
| WARNING:at_net/wireless/core.c:#cfg80211_destroy_ifaces[cfg80211] | 0 | 2 |
| RIP:cfg80211_destroy_ifaces[cfg80211] | 0 | 2 |
| WARNING:at_net/mac80211/iface.c:#ieee80211_if_add[mac80211] | 0 | 2 |
| RIP:ieee80211_if_add[mac80211] | 0 | 2 |
| WARNING:at_net/core/dev.c:#register_netdevice | 0 | 2 |
| RIP:register_netdevice | 0 | 2 |
| WARNING:at_net/core/dev.c:#__netdev_update_features | 0 | 2 |
| RIP:__netdev_update_features | 0 | 2 |
| WARNING:at_net/core/dev.c:#list_netdevice | 0 | 2 |
| RIP:list_netdevice | 0 | 2 |
| WARNING:at_net/ipv4/devinet.c:#inetdev_init | 0 | 2 |
| RIP:inetdev_init | 0 | 2 |
| WARNING:at_net/ipv4/igmp.c:#ip_mc_init_dev | 0 | 2 |
| RIP:ip_mc_init_dev | 0 | 2 |
| WARNING:at_net/ipv6/addrconf.c:#ipv6_add_dev | 0 | 2 |
| RIP:ipv6_add_dev | 0 | 2 |
| WARNING:at_net/ipv6/mcast.c:#__ipv6_dev_mc_inc | 0 | 2 |
| RIP:__ipv6_dev_mc_inc | 0 | 2 |
| WARNING:at_net/mac80211/iface.c:#ieee80211_if_remove[mac80211] | 0 | 2 |
| RIP:ieee80211_if_remove[mac80211] | 0 | 2 |
| WARNING:at_net/core/dev.c:#unregister_netdevice_queue | 0 | 2 |
| RIP:unregister_netdevice_queue | 0 | 2 |
| WARNING:at_net/ipv4/igmp.c:#ip_mc_down | 0 | 2 |
| RIP:ip_mc_down | 0 | 2 |
| WARNING:at_net/ipv4/igmp.c:#__ip_mc_dec_group | 0 | 2 |
| RIP:__ip_mc_dec_group | 0 | 2 |
| WARNING:at_net/ipv6/addrconf.c:#addrconf_ifdown | 0 | 2 |
| RIP:addrconf_ifdown | 0 | 2 |
| WARNING:at_net/ipv6/addrconf.c:#__ipv6_ifa_notify | 0 | 3 |
| RIP:__ipv6_ifa_notify | 0 | 3 |
| WARNING:at_net/ipv6/mcast.c:#__ipv6_dev_mc_dec | 0 | 2 |
| RIP:__ipv6_dev_mc_dec | 0 | 2 |
| WARNING:at_net/core/dev.c:#unlist_netdevice | 0 | 1 |
| RIP:unlist_netdevice | 0 | 1 |
| WARNING:at_net/sched/sch_api.c:#qdisc_hash_del | 0 | 1 |
| RIP:qdisc_hash_del | 0 | 1 |
| WARNING:at_net/ipv4/igmp.c:#ip_mc_destroy_dev | 0 | 1 |
| RIP:ip_mc_destroy_dev | 0 | 1 |
| WARNING:at_net/wireless/core.c:#__cfg80211_unregister_wdev[cfg80211] | 0 | 1 |
| RIP:__cfg80211_unregister_wdev[cfg80211] | 0 | 1 |
| WARNING:at_net/core/dev.c:#netdev_has_any_upper_dev | 0 | 1 |
| RIP:netdev_has_any_upper_dev | 0 | 1 |
| WARNING:at_net/core/dev.c:#rollback_registered_many | 0 | 2 |
| RIP:rollback_registered_many | 0 | 2 |
| WARNING:at_net/core/dev.c:#__dev_close_many | 0 | 2 |
| RIP:__dev_close_many | 0 | 2 |
| BUG:soft_lockup-CPU##stuck_for#s | 0 | 1 |
| RIP:find_next_bit | 0 | 1 |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 0 | 1 |
+---------------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 186.333346] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[ 186.336318] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[ 186.339612] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ 186.343633] cfg80211: failed to load regulatory.db
[ 186.359277] ------------[ cut here ]------------
[ 186.361527] RTNL: assertion failed at net/wireless/reg.c (306)
[ 186.364166] WARNING: CPU: 0 PID: 4804 at net/wireless/reg.c:306 reset_regdomains+0xd5/0xe0 [cfg80211]
[ 186.367388] Modules linked in: cfg80211 rfkill bochs_drm ttm sr_mod cdrom crct10dif_pclmul drm_kms_helper crc32_pclmul sg crc32c_intel ata_generic pata_acpi ghash_clmulni_intel ppdev syscopyarea sysfillrect sysimgblt fb_sys_fops snd_pcm drm ata_piix aesni_intel snd_timer snd crypto_simd libata cryptd glue_helper soundcore joydev pcspkr serio_raw parport_pc i2c_piix4 floppy parport ip_tables
[ 186.378827] CPU: 0 PID: 4804 Comm: crda Not tainted 5.2.0-rc3-01111-gd8572a5 #1
[ 186.381529] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 186.384383] RIP: 0010:reset_regdomains+0xd5/0xe0 [cfg80211]
[ 186.386829] Code: 33 07 09 00 00 0f 85 63 ff ff ff ba 32 01 00 00 48 c7 c6 40 9d 55 c0 48 c7 c7 68 9d 55 c0 c6 05 13 07 09 00 01 e8 eb fa 79 ea <0f> 0b e9 3d ff ff ff 0f 1f 40 00 66 66 66 66 90 48 8b 05 d4 26 08
[ 186.392729] RSP: 0018:ffffb103c09bba38 EFLAGS: 00010282
[ 186.395104] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[ 186.397792] RDX: ffff95233fc26d80 RSI: ffff95233fc17778 RDI: ffff95233fc17778
[ 186.400389] RBP: ffff95227e894400 R08: 0000000000000459 R09: 0000000000aaaaaa
[ 186.403046] R10: 0000000000000000 R11: ffff95227dd39e30 R12: ffff95227e894414
[ 186.405696] R13: ffffffffc0556140 R14: 0000000000000000 R15: ffff95227e7ef9c8
[ 186.408337] FS: 00007f157ef5cb40(0000) GS:ffff95233fc00000(0000) knlGS:0000000000000000
[ 186.411470] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 186.413856] CR2: 00005623bc744028 CR3: 000000008861e000 CR4: 00000000000406f0
[ 186.416457] Call Trace:
[ 186.418174] set_regdom+0x2b0/0x430 [cfg80211]
[ 186.420255] nl80211_set_reg+0x290/0x2c0 [cfg80211]
[ 186.422395] genl_family_rcv_msg+0x203/0x400
[ 186.424986] ? __alloc_pages_nodemask+0x157/0x340
[ 186.427159] genl_rcv_msg+0x47/0x90
[ 186.429001] ? __kmalloc_node_track_caller+0x59/0x2a0
[ 186.431140] ? genl_family_rcv_msg+0x400/0x400
[ 186.433207] netlink_rcv_skb+0x4a/0x110
[ 186.435092] genl_rcv+0x24/0x40
[ 186.436918] netlink_unicast+0x193/0x230
[ 186.438784] netlink_sendmsg+0x2c1/0x3c0
[ 186.440809] sock_sendmsg+0x4c/0x50
[ 186.442756] ___sys_sendmsg+0x289/0x300
[ 186.444825] ? mem_cgroup_throttle_swaprate+0x20/0x160
[ 186.447196] ? do_anonymous_page+0x2f3/0x3f0
[ 186.449197] ? __seccomp_filter+0x96/0x6c0
[ 186.451348] ? __sys_sendmsg+0x5e/0xa0
[ 186.453442] __sys_sendmsg+0x5e/0xa0
[ 186.455394] do_syscall_64+0x5b/0x1e0
[ 186.457454] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 186.459677] RIP: 0033:0x7f157e463dc7
[ 186.461625] Code: d8 64 89 02 48 c7 c0 ff ff ff ff eb cd 66 0f 1f 44 00 00 8b 05 4a 49 2b 00 85 c0 75 2e 48 63 ff 48 63 d2 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 a1 f0 2a 00 f7 d8 64 89 02 48
[ 186.467465] RSP: 002b:00007ffd7f9d92d8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 186.470182] RAX: ffffffffffffffda RBX: 00005623bc756010 RCX: 00007f157e463dc7
[ 186.472725] RDX: 0000000000000000 RSI: 00007ffd7f9d9360 RDI: 0000000000000000
[ 186.475275] RBP: 00005623bc755ef0 R08: 0000000000000000 R09: 00000000000000f0
[ 186.477814] R10: 00000000000000ea R11: 0000000000000246 R12: 00005623bc756060
[ 186.480547] R13: 00007ffd7f9d9360 R14: 00005623bc755c20 R15: 0000000000000008
[ 186.484264] ---[ end trace 9a39044671a6e95d ]---
[ 186.486521] ------------[ cut here ]------------
[ 186.488449] RTNL: assertion failed at net/wireless/reg.c (2252)
[ 186.491060] WARNING: CPU: 0 PID: 4804 at net/wireless/reg.c:2252 update_all_wiphy_regulatory+0x8a/0x90 [cfg80211]
[ 186.495546] Modules linked in: cfg80211 rfkill bochs_drm ttm sr_mod cdrom crct10dif_pclmul drm_kms_helper crc32_pclmul sg crc32c_intel ata_generic pata_acpi ghash_clmulni_intel ppdev syscopyarea sysfillrect sysimgblt fb_sys_fops snd_pcm drm ata_piix aesni_intel snd_timer snd crypto_simd libata cryptd glue_helper soundcore joydev pcspkr serio_raw parport_pc i2c_piix4 floppy parport ip_tables
[ 186.506468] CPU: 0 PID: 4804 Comm: crda Tainted: G W 5.2.0-rc3-01111-gd8572a5 #1
[ 186.509258] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 186.512042] RIP: 0010:update_all_wiphy_regulatory+0x8a/0x90 [cfg80211]
[ 186.514585] Code: 7b ea 80 3d d5 e3 08 00 00 75 a7 ba cc 08 00 00 48 c7 c6 40 9d 55 c0 48 c7 c7 68 9d 55 c0 c6 05 b9 e3 08 00 01 e8 96 d7 79 ea <0f> 0b eb 84 66 90 66 66 66 66 90 41 55 41 54 55 53 48 89 fb 8b 6f
[ 186.520426] RSP: 0018:ffffb103c09bba38 EFLAGS: 00010282
[ 186.522651] RAX: 0000000000000000 RBX: ffff95227e894400 RCX: 0000000000000000
[ 186.525202] RDX: ffff95233fc26d80 RSI: ffff95233fc17778 RDI: ffff95233fc17778
[ 186.527808] RBP: 0000000000000000 R08: 0000000000000488 R09: 0000000000aaaaaa
[ 186.530318] R10: 0000000000000000 R11: ffff95227dd38b70 R12: 0000000000000000
[ 186.533247] R13: ffffffffc0556140 R14: 0000000000000000 R15: ffff95227e7ef9c8
[ 186.535818] FS: 00007f157ef5cb40(0000) GS:ffff95233fc00000(0000) knlGS:0000000000000000
[ 186.538591] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 186.540959] CR2: 00005623bc744028 CR3: 000000008861e000 CR4: 00000000000406f0
[ 186.543642] Call Trace:
[ 186.545460] set_regdom+0x149/0x430 [cfg80211]
[ 186.547596] nl80211_set_reg+0x290/0x2c0 [cfg80211]
[ 186.549758] genl_family_rcv_msg+0x203/0x400
[ 186.551913] ? __alloc_pages_nodemask+0x157/0x340
[ 186.554053] genl_rcv_msg+0x47/0x90
[ 186.556085] ? __kmalloc_node_track_caller+0x59/0x2a0
[ 186.558417] ? genl_family_rcv_msg+0x400/0x400
[ 186.560609] netlink_rcv_skb+0x4a/0x110
[ 186.562710] genl_rcv+0x24/0x40
[ 186.564712] netlink_unicast+0x193/0x230
[ 186.566835] netlink_sendmsg+0x2c1/0x3c0
[ 186.568847] sock_sendmsg+0x4c/0x50
[ 186.570935] ___sys_sendmsg+0x289/0x300
[ 186.572994] ? mem_cgroup_throttle_swaprate+0x20/0x160
[ 186.575196] ? do_anonymous_page+0x2f3/0x3f0
[ 186.577419] ? __seccomp_filter+0x96/0x6c0
[ 186.579439] ? __sys_sendmsg+0x5e/0xa0
[ 186.581528] __sys_sendmsg+0x5e/0xa0
[ 186.583526] do_syscall_64+0x5b/0x1e0
[ 186.585418] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 186.587625] RIP: 0033:0x7f157e463dc7
[ 186.589610] Code: d8 64 89 02 48 c7 c0 ff ff ff ff eb cd 66 0f 1f 44 00 00 8b 05 4a 49 2b 00 85 c0 75 2e 48 63 ff 48 63 d2 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 a1 f0 2a 00 f7 d8 64 89 02 48
[ 186.595368] RSP: 002b:00007ffd7f9d92d8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 186.598376] RAX: ffffffffffffffda RBX: 00005623bc756010 RCX: 00007f157e463dc7
[ 186.601001] RDX: 0000000000000000 RSI: 00007ffd7f9d9360 RDI: 0000000000000000
[ 186.603656] RBP: 00005623bc755ef0 R08: 0000000000000000 R09: 00000000000000f0
[ 186.606129] R10: 00000000000000ea R11: 0000000000000246 R12: 00005623bc756060
[ 186.608590] R13: 00007ffd7f9d9360 R14: 00005623bc755c20 R15: 0000000000000008
[ 186.614598] ---[ end trace 9a39044671a6e95e ]---
[ 186.749954] mac80211_hwsim: initializing netlink
[ 186.777079] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
To reproduce:
# build kernel
cd linux
cp config-5.2.0-rc3-01111-gd8572a5 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 6 months