[SUNRPC] 0472e47660: fsmark.app_overhead 16.0% regression
by kernel test robot
Greeting,
FYI, we noticed a 16.0% improvement of fsmark.app_overhead due to commit:
commit: 0472e476604998c127f3c80d291113e77c5676ac ("SUNRPC: Convert socket page send code to use iov_iter()")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fsmark
on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
with following parameters:
iterations: 1x
nr_threads: 64t
disk: 1BRD_48G
fs: xfs
fs2: nfsv4
filesize: 4M
test_size: 40G
sync_method: fsyncBeforeClose
cpufreq_governor: performance
test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload.
test-url: https://sourceforge.net/projects/fsmark/
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase:
gcc-7/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel-7.6/64t/debian-x86_64-2018-04-03.cgz/fsyncBeforeClose/lkp-ivb-ep01/40G/fsmark
commit:
e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use iov_iter_kvec()")
0472e47660 ("SUNRPC: Convert socket page send code to use iov_iter()")
e791f8e9380d945e 0472e476604998c127f3c80d291
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 50% 2:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
15118573 ± 2% +16.0% 17538083 fsmark.app_overhead
510.93 -22.7% 395.12 fsmark.files_per_sec
24.90 +22.8% 30.57 fsmark.time.elapsed_time
24.90 +22.8% 30.57 fsmark.time.elapsed_time.max
288.00 ± 2% -27.8% 208.00 fsmark.time.percent_of_cpu_this_job_got
70.03 ± 2% -11.3% 62.14 fsmark.time.system_time
4391964 -16.7% 3658341 meminfo.max_used_kB
6.10 ± 4% +1.9 7.97 ± 3% mpstat.cpu.all.iowait%
0.27 -0.0 0.24 ± 3% mpstat.cpu.all.soft%
13668070 ± 40% +118.0% 29801846 ± 19% numa-numastat.node0.local_node
13677774 ± 40% +117.9% 29810258 ± 19% numa-numastat.node0.numa_hit
5.70 ± 3% +32.1% 7.53 ± 3% iostat.cpu.iowait
16.42 ± 2% -5.8% 15.47 iostat.cpu.system
2.57 -4.1% 2.46 iostat.cpu.user
1406781 ± 2% -15.5% 1188498 vmstat.io.bo
251792 ± 3% -16.6% 209928 vmstat.system.cs
84841 -1.9% 83239 vmstat.system.in
97374502 ± 20% +66.1% 1.617e+08 ± 17% cpuidle.C1E.time
573934 ± 19% +44.6% 829662 ± 26% cpuidle.C1E.usage
5.892e+08 ± 8% +15.3% 6.796e+08 ± 2% cpuidle.C6.time
1968016 ± 3% -15.1% 1670867 ± 3% cpuidle.POLL.time
106420 ± 47% +86.2% 198108 ± 35% numa-meminfo.node0.Active
106037 ± 48% +86.2% 197395 ± 35% numa-meminfo.node0.Active(anon)
105052 ± 48% +86.6% 196037 ± 35% numa-meminfo.node0.AnonPages
212876 ± 24% -41.5% 124572 ± 56% numa-meminfo.node1.Active
211801 ± 24% -41.5% 123822 ± 56% numa-meminfo.node1.Active(anon)
208559 ± 24% -42.2% 120547 ± 57% numa-meminfo.node1.AnonPages
9955 +1.6% 10116 proc-vmstat.nr_kernel_stack
452.25 ± 59% +280.9% 1722 ±100% proc-vmstat.numa_hint_faults_local
33817303 +55.0% 52421773 ± 5% proc-vmstat.numa_hit
33804286 +55.0% 52408807 ± 5% proc-vmstat.numa_local
33923002 +81.8% 61663426 ± 5% proc-vmstat.pgalloc_normal
184765 +9.3% 201985 proc-vmstat.pgfault
12840986 +216.0% 40581327 ± 7% proc-vmstat.pgfree
31447 ± 11% -26.1% 23253 ± 13% sched_debug.cfs_rq:/.min_vruntime.max
4241 ± 3% -12.2% 3724 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev
20631 ± 11% -36.7% 13069 ± 29% sched_debug.cfs_rq:/.spread0.max
4238 ± 4% -12.1% 3724 ± 11% sched_debug.cfs_rq:/.spread0.stddev
497105 ± 19% -16.0% 417777 ± 4% sched_debug.cpu.avg_idle.avg
21199 ± 10% -12.0% 18650 ± 3% sched_debug.cpu.nr_load_updates.max
2229 ± 10% -15.0% 1895 ± 4% sched_debug.cpu.nr_load_updates.stddev
4.86 ± 5% -23.6% 3.72 ± 5% sched_debug.cpu.nr_uninterruptible.stddev
524.75 ± 2% -10.7% 468.50 turbostat.Avg_MHz
5.26 ± 41% -1.6 3.66 ± 2% turbostat.C1%
573633 ± 19% +44.6% 829267 ± 26% turbostat.C1E
8.53 ± 20% +3.4 11.88 ± 17% turbostat.C1E%
76.75 -6.4% 71.86 turbostat.CorWatt
2534071 ± 2% +16.2% 2943968 turbostat.IRQ
2.89 ± 24% +52.9% 4.42 ± 22% turbostat.Pkg%pc2
104.28 -4.6% 99.51 turbostat.PkgWatt
2289 +18.8% 2720 turbostat.SMI
26496 ± 47% +86.2% 49347 ± 35% numa-vmstat.node0.nr_active_anon
26251 ± 48% +86.7% 49008 ± 35% numa-vmstat.node0.nr_anon_pages
26496 ± 47% +86.2% 49347 ± 35% numa-vmstat.node0.nr_zone_active_anon
8272770 ± 35% +99.6% 16515977 ± 17% numa-vmstat.node0.numa_hit
8199681 ± 36% +100.9% 16474282 ± 17% numa-vmstat.node0.numa_local
52953 ± 24% -41.5% 30956 ± 56% numa-vmstat.node1.nr_active_anon
52144 ± 24% -42.2% 30136 ± 57% numa-vmstat.node1.nr_anon_pages
2899 ± 2% -19.4% 2336 ± 10% numa-vmstat.node1.nr_writeback
52953 ± 24% -41.5% 30956 ± 56% numa-vmstat.node1.nr_zone_active_anon
4549 ± 8% -18.9% 3689 ± 7% numa-vmstat.node1.nr_zone_write_pending
43.25 -1.6 41.69 perf-stat.i.cache-miss-rate%
286377 ± 2% -19.1% 231819 ± 2% perf-stat.i.context-switches
2.258e+10 ± 4% -16.2% 1.893e+10 ± 10% perf-stat.i.cpu-cycles
4084 -34.0% 2696 ± 2% perf-stat.i.cpu-migrations
21563675 ± 24% -30.8% 14929445 ± 5% perf-stat.i.dTLB-load-misses
1996341 ± 16% -25.7% 1482826 ± 6% perf-stat.i.dTLB-store-misses
2.313e+09 ± 4% -11.7% 2.042e+09 ± 6% perf-stat.i.dTLB-stores
75.93 -1.3 74.66 perf-stat.i.iTLB-load-miss-rate%
1561381 ± 3% -13.4% 1351731 perf-stat.i.iTLB-load-misses
549643 ± 4% -10.1% 494255 ± 3% perf-stat.i.iTLB-loads
6423 -10.0% 5779 perf-stat.i.minor-faults
31.72 ± 2% +4.4 36.14 perf-stat.i.node-load-miss-rate%
23492890 +23.5% 29020305 ± 3% perf-stat.i.node-load-misses
15.13 ± 8% +7.9 23.02 ± 3% perf-stat.i.node-store-miss-rate%
11517615 +71.3% 19734054 ± 2% perf-stat.i.node-store-misses
6423 -10.0% 5779 perf-stat.i.page-faults
16.88 ± 14% +33.4% 22.52 ± 23% perf-stat.overall.MPKI
46.67 -2.4 44.31 perf-stat.overall.cache-miss-rate%
31.96 +4.8 36.72 perf-stat.overall.node-load-miss-rate%
13.85 +8.0 21.87 perf-stat.overall.node-store-miss-rate%
2.266e+08 +2.8% 2.331e+08 ± 2% perf-stat.ps.cache-references
275634 ± 2% -18.5% 224718 ± 2% perf-stat.ps.context-switches
2.174e+10 ± 4% -15.6% 1.835e+10 ± 10% perf-stat.ps.cpu-cycles
3931 -33.5% 2614 ± 2% perf-stat.ps.cpu-migrations
20746040 ± 24% -30.2% 14476157 ± 5% perf-stat.ps.dTLB-load-misses
1921077 ± 16% -25.2% 1437750 ± 6% perf-stat.ps.dTLB-store-misses
2.227e+09 ± 4% -11.1% 1.979e+09 ± 6% perf-stat.ps.dTLB-stores
1503433 ± 3% -12.8% 1310741 perf-stat.ps.iTLB-load-misses
529058 ± 4% -9.4% 479204 ± 3% perf-stat.ps.iTLB-loads
6200 -9.4% 5620 perf-stat.ps.minor-faults
22613159 +24.4% 28133123 ± 3% perf-stat.ps.node-load-misses
11085254 +72.6% 19131576 ± 2% perf-stat.ps.node-store-misses
6200 -9.4% 5620 perf-stat.ps.page-faults
7008 ± 13% +55.2% 10876 ± 31% softirqs.CPU1.NET_RX
6509 ± 4% +51.8% 9883 ± 27% softirqs.CPU1.RCU
7294 ± 13% +36.7% 9974 ± 8% softirqs.CPU10.RCU
7800 ± 14% +85.5% 14469 ± 23% softirqs.CPU12.NET_RX
5697 ± 43% +77.5% 10110 ± 24% softirqs.CPU13.NET_RX
15944 ± 9% +14.6% 18278 ± 12% softirqs.CPU14.TIMER
6064 ± 19% +68.6% 10223 ± 31% softirqs.CPU15.NET_RX
7796 ± 14% +80.3% 14059 ± 25% softirqs.CPU16.NET_RX
15934 ± 10% +22.1% 19452 ± 11% softirqs.CPU18.TIMER
6725 ± 7% +40.0% 9413 ± 18% softirqs.CPU19.NET_RX
5710 ± 3% +53.4% 8756 ± 12% softirqs.CPU20.RCU
7018 ± 14% +65.5% 11616 ± 40% softirqs.CPU21.NET_RX
6389 ± 18% +66.9% 10666 ± 31% softirqs.CPU23.NET_RX
7259 ± 7% +36.1% 9881 ± 6% softirqs.CPU24.RCU
6491 ± 20% +58.5% 10289 ± 33% softirqs.CPU25.NET_RX
7090 ± 10% +58.7% 11256 ± 29% softirqs.CPU27.NET_RX
6333 ± 27% +70.3% 10786 ± 27% softirqs.CPU29.NET_RX
5813 ± 20% +84.2% 10706 ± 36% softirqs.CPU3.NET_RX
7041 ± 23% +105.7% 14483 ± 21% softirqs.CPU30.NET_RX
6654 ± 7% +64.1% 10918 ± 29% softirqs.CPU31.NET_RX
18019 ± 10% +11.1% 20016 ± 7% softirqs.CPU31.TIMER
4666 ± 8% +104.0% 9518 ± 23% softirqs.CPU32.RCU
15721 ± 16% +35.9% 21371 ± 11% softirqs.CPU32.TIMER
15684 ± 13% +40.0% 21959 ± 18% softirqs.CPU34.TIMER
6489 ± 15% +69.7% 11013 ± 37% softirqs.CPU35.NET_RX
7930 ± 7% +82.1% 14442 ± 29% softirqs.CPU36.NET_RX
15744 ± 14% +24.3% 19563 ± 6% softirqs.CPU36.TIMER
7028 ± 13% +29.3% 9085 ± 11% softirqs.CPU39.NET_RX
7491 ± 22% +100.4% 15011 ± 24% softirqs.CPU4.NET_RX
6119 ± 13% +58.7% 9710 ± 37% softirqs.CPU5.NET_RX
6980 ± 8% +47.8% 10318 ± 42% softirqs.CPU7.NET_RX
285674 +65.7% 473395 softirqs.NET_RX
267950 ± 5% +21.1% 324597 softirqs.RCU
238298 ± 2% +15.1% 274371 ± 2% softirqs.SCHED
689305 ± 3% +14.1% 786236 ± 3% softirqs.TIMER
56196 ± 2% +19.9% 67389 interrupts.CPU0.LOC:Local_timer_interrupts
55971 ± 2% +19.7% 67005 interrupts.CPU1.LOC:Local_timer_interrupts
5265 ± 17% -32.2% 3568 ± 29% interrupts.CPU1.RES:Rescheduling_interrupts
56163 ± 2% +19.2% 66960 interrupts.CPU10.LOC:Local_timer_interrupts
56178 ± 2% +19.5% 67139 interrupts.CPU11.LOC:Local_timer_interrupts
4669 ± 15% -30.3% 3253 ± 29% interrupts.CPU11.RES:Rescheduling_interrupts
56081 +19.7% 67142 interrupts.CPU12.LOC:Local_timer_interrupts
55999 ± 2% +19.5% 66893 interrupts.CPU13.LOC:Local_timer_interrupts
4746 ± 23% -30.0% 3324 ± 29% interrupts.CPU13.RES:Rescheduling_interrupts
55991 ± 2% +19.5% 66898 interrupts.CPU14.LOC:Local_timer_interrupts
55855 ± 2% +20.0% 67031 interrupts.CPU15.LOC:Local_timer_interrupts
55943 ± 2% +19.9% 67087 interrupts.CPU16.LOC:Local_timer_interrupts
56164 +18.9% 66802 interrupts.CPU17.LOC:Local_timer_interrupts
4579 ± 20% -45.6% 2492 ± 16% interrupts.CPU17.RES:Rescheduling_interrupts
55913 ± 2% +19.6% 66860 interrupts.CPU18.LOC:Local_timer_interrupts
55820 ± 2% +19.7% 66822 interrupts.CPU19.LOC:Local_timer_interrupts
4733 ± 17% -35.5% 3052 ± 21% interrupts.CPU19.RES:Rescheduling_interrupts
55989 ± 2% +20.0% 67177 interrupts.CPU2.LOC:Local_timer_interrupts
55891 ± 2% +20.3% 67258 interrupts.CPU20.LOC:Local_timer_interrupts
55966 ± 2% +19.6% 66921 interrupts.CPU21.LOC:Local_timer_interrupts
4920 ± 21% -33.4% 3278 ± 31% interrupts.CPU21.RES:Rescheduling_interrupts
55945 ± 2% +19.9% 67098 interrupts.CPU22.LOC:Local_timer_interrupts
55945 ± 2% +19.9% 67073 interrupts.CPU23.LOC:Local_timer_interrupts
4972 ± 24% -34.1% 3277 ± 36% interrupts.CPU23.RES:Rescheduling_interrupts
56093 ± 2% +19.8% 67185 interrupts.CPU24.LOC:Local_timer_interrupts
55910 ± 2% +19.7% 66914 interrupts.CPU25.LOC:Local_timer_interrupts
4660 ± 25% -28.9% 3313 ± 34% interrupts.CPU25.RES:Rescheduling_interrupts
56105 ± 2% +18.8% 66631 interrupts.CPU26.LOC:Local_timer_interrupts
55928 ± 2% +19.5% 66827 interrupts.CPU27.LOC:Local_timer_interrupts
55934 ± 2% +19.3% 66740 interrupts.CPU28.LOC:Local_timer_interrupts
55918 ± 2% +19.5% 66812 interrupts.CPU29.LOC:Local_timer_interrupts
4825 ± 22% -30.3% 3362 ± 30% interrupts.CPU29.RES:Rescheduling_interrupts
55920 ± 2% +19.8% 67004 interrupts.CPU3.LOC:Local_timer_interrupts
56007 ± 2% +19.5% 66917 interrupts.CPU30.LOC:Local_timer_interrupts
56238 +18.8% 66838 interrupts.CPU31.LOC:Local_timer_interrupts
4859 ± 21% -33.0% 3257 ± 36% interrupts.CPU31.RES:Rescheduling_interrupts
56099 +19.3% 66921 interrupts.CPU32.LOC:Local_timer_interrupts
55963 ± 2% +19.5% 66866 interrupts.CPU33.LOC:Local_timer_interrupts
55992 ± 2% +19.2% 66767 interrupts.CPU34.LOC:Local_timer_interrupts
55964 ± 2% +19.6% 66939 interrupts.CPU35.LOC:Local_timer_interrupts
4797 ± 26% -33.6% 3185 ± 38% interrupts.CPU35.RES:Rescheduling_interrupts
55770 ± 2% +20.2% 67016 interrupts.CPU36.LOC:Local_timer_interrupts
55889 +20.0% 67095 interrupts.CPU37.LOC:Local_timer_interrupts
56140 ± 2% +18.9% 66743 interrupts.CPU38.LOC:Local_timer_interrupts
56040 ± 2% +19.2% 66828 interrupts.CPU39.LOC:Local_timer_interrupts
5126 ± 23% -40.5% 3052 ± 11% interrupts.CPU39.RES:Rescheduling_interrupts
56022 ± 2% +19.9% 67149 interrupts.CPU4.LOC:Local_timer_interrupts
55850 ± 2% +20.1% 67066 interrupts.CPU5.LOC:Local_timer_interrupts
4741 ± 23% -31.9% 3227 ± 29% interrupts.CPU5.RES:Rescheduling_interrupts
55912 +19.5% 66798 interrupts.CPU6.LOC:Local_timer_interrupts
56062 ± 2% +19.6% 67052 interrupts.CPU7.LOC:Local_timer_interrupts
56118 +19.6% 67142 interrupts.CPU8.LOC:Local_timer_interrupts
56110 ± 2% +19.6% 67121 interrupts.CPU9.LOC:Local_timer_interrupts
2216 ± 43% -69.4% 677.50 ±112% interrupts.CPU9.NMI:Non-maskable_interrupts
2216 ± 43% -69.4% 677.50 ±112% interrupts.CPU9.PMI:Performance_monitoring_interrupts
2240009 ± 2% +19.6% 2678940 interrupts.LOC:Local_timer_interrupts
168296 ± 3% -10.1% 151314 interrupts.RES:Rescheduling_interrupts
20.37 -3.3 17.04 ± 5% perf-profile.calltrace.cycles-pp.svc_process_common.svc_process.nfsd.kthread.ret_from_fork
20.38 -3.3 17.04 ± 5% perf-profile.calltrace.cycles-pp.svc_process.nfsd.kthread.ret_from_fork
20.27 -3.3 16.95 ± 5% perf-profile.calltrace.cycles-pp.nfsd_dispatch.svc_process_common.svc_process.nfsd.kthread
20.21 -3.3 16.91 ± 5% perf-profile.calltrace.cycles-pp.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process.nfsd
26.02 -3.1 22.91 ± 4% perf-profile.calltrace.cycles-pp.nfsd.kthread.ret_from_fork
9.04 ± 2% -1.9 7.18 ± 7% perf-profile.calltrace.cycles-pp.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common
9.02 ± 2% -1.9 7.17 ± 7% perf-profile.calltrace.cycles-pp.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound
9.02 ± 2% -1.9 7.16 ± 7% perf-profile.calltrace.cycles-pp.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write.nfsd4_write
9.03 ± 2% -1.9 7.18 ± 7% perf-profile.calltrace.cycles-pp.do_iter_write.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch
9.07 ± 2% -1.9 7.22 ± 7% perf-profile.calltrace.cycles-pp.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process
10.50 ± 2% -1.4 9.06 ± 4% perf-profile.calltrace.cycles-pp.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common
10.51 ± 2% -1.4 9.08 ± 4% perf-profile.calltrace.cycles-pp.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process
10.45 ± 2% -1.4 9.02 ± 4% perf-profile.calltrace.cycles-pp.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound.nfsd_dispatch
9.82 ± 3% -1.4 8.45 ± 3% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit.nfsd4_proc_compound
9.82 ± 3% -1.4 8.45 ± 3% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync.nfsd_commit
9.82 ± 3% -1.4 8.45 ± 3% perf-profile.calltrace.cycles-pp.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range.xfs_file_fsync
10.61 ± 5% -1.1 9.46 ± 4% perf-profile.calltrace.cycles-pp.write
7.54 -1.1 6.41 ± 6% perf-profile.calltrace.cycles-pp.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write
7.54 -1.1 6.41 ± 6% perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write
10.27 ± 5% -1.1 9.15 ± 4% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
7.49 -1.1 6.37 ± 6% perf-profile.calltrace.cycles-pp.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write.do_iter_readv_writev
10.23 ± 5% -1.1 9.11 ± 4% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
10.35 ± 5% -1.1 9.23 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
10.08 ± 5% -1.1 8.96 ± 4% perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.33 ± 5% -1.1 9.22 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
10.05 ± 5% -1.1 8.95 ± 4% perf-profile.calltrace.cycles-pp.nfs_file_write.__vfs_write.vfs_write.ksys_write.do_syscall_64
9.80 ± 5% -1.1 8.73 ± 4% perf-profile.calltrace.cycles-pp.generic_perform_write.nfs_file_write.__vfs_write.vfs_write.ksys_write
5.31 ± 5% -1.0 4.27 ± 5% perf-profile.calltrace.cycles-pp.rpc_free_task.rpc_async_release.process_one_work.worker_thread.kthread
5.31 ± 5% -1.0 4.27 ± 5% perf-profile.calltrace.cycles-pp.rpc_async_release.process_one_work.worker_thread.kthread.ret_from_fork
4.38 ± 5% -1.0 3.35 ± 2% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.svc_recvfrom.svc_tcp_recvfrom.svc_recv
4.38 ± 6% -1.0 3.36 ± 2% perf-profile.calltrace.cycles-pp.inet_recvmsg.svc_recvfrom.svc_tcp_recvfrom.svc_recv.nfsd
4.39 ± 6% -1.0 3.40 ± 2% perf-profile.calltrace.cycles-pp.svc_recvfrom.svc_tcp_recvfrom.svc_recv.nfsd.kthread
3.53 ± 5% -0.9 2.62 ± 3% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.svc_recvfrom.svc_tcp_recvfrom
3.52 ± 5% -0.9 2.62 ± 3% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.svc_recvfrom
5.01 ± 6% -0.9 4.14 ± 3% perf-profile.calltrace.cycles-pp.fsync
4.09 ± 7% -0.8 3.24 ± 5% perf-profile.calltrace.cycles-pp.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work.worker_thread
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.fsync
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.calltrace.cycles-pp.nfs_file_fsync.do_fsync.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.calltrace.cycles-pp.do_fsync.__x64_sys_fsync.do_syscall_64.entry_SYSCALL_64_after_hwframe.fsync
5.84 ± 2% -0.8 5.01 ± 4% perf-profile.calltrace.cycles-pp.write_cache_pages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range
1.15 ± 7% -0.8 0.36 ±100% perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.xfs_ilock
4.38 ± 7% -0.8 3.59 ± 3% perf-profile.calltrace.cycles-pp.filemap_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync.do_syscall_64
5.17 ± 2% -0.8 4.39 ± 5% perf-profile.calltrace.cycles-pp.xfs_do_writepage.write_cache_pages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range
4.89 ± 4% -0.7 4.18 ± 5% perf-profile.calltrace.cycles-pp.brd_insert_page.brd_do_bvec.brd_make_request.generic_make_request.submit_bio
1.38 ± 9% -0.7 0.68 ± 22% perf-profile.calltrace.cycles-pp.down_write.xfs_ilock.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write
1.38 ± 9% -0.7 0.68 ± 22% perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.xfs_ilock.xfs_file_buffered_aio_write.do_iter_readv_writev
1.38 ± 9% -0.7 0.68 ± 22% perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.xfs_ilock.xfs_file_buffered_aio_write
1.38 ± 9% -0.7 0.68 ± 22% perf-profile.calltrace.cycles-pp.xfs_ilock.xfs_file_buffered_aio_write.do_iter_readv_writev.do_iter_write.nfsd_vfs_write
4.01 ± 3% -0.6 3.40 ± 5% perf-profile.calltrace.cycles-pp.brd_make_request.generic_make_request.submit_bio.xfs_add_to_ioend.xfs_do_writepage
4.01 ± 3% -0.6 3.40 ± 5% perf-profile.calltrace.cycles-pp.submit_bio.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages.xfs_vm_writepages
4.01 ± 3% -0.6 3.40 ± 5% perf-profile.calltrace.cycles-pp.generic_make_request.submit_bio.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages
3.99 ± 3% -0.6 3.38 ± 5% perf-profile.calltrace.cycles-pp.brd_do_bvec.brd_make_request.generic_make_request.submit_bio.xfs_add_to_ioend
4.11 ± 3% -0.6 3.51 ± 4% perf-profile.calltrace.cycles-pp.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages.xfs_vm_writepages.do_writepages
3.97 ± 5% -0.5 3.43 ± 4% perf-profile.calltrace.cycles-pp.submit_bio.xfs_submit_ioend.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range
3.97 ± 5% -0.5 3.43 ± 4% perf-profile.calltrace.cycles-pp.generic_make_request.submit_bio.xfs_submit_ioend.xfs_vm_writepages.do_writepages
3.98 ± 5% -0.5 3.44 ± 4% perf-profile.calltrace.cycles-pp.xfs_submit_ioend.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.file_write_and_wait_range
3.96 ± 5% -0.5 3.43 ± 4% perf-profile.calltrace.cycles-pp.brd_make_request.generic_make_request.submit_bio.xfs_submit_ioend.xfs_vm_writepages
3.92 ± 5% -0.5 3.40 ± 4% perf-profile.calltrace.cycles-pp.brd_do_bvec.brd_make_request.generic_make_request.submit_bio.xfs_submit_ioend
2.89 ± 6% -0.5 2.38 ± 6% perf-profile.calltrace.cycles-pp.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release.process_one_work
2.77 ± 10% -0.5 2.28 ± 2% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.filemap_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync
2.77 ± 10% -0.5 2.28 ± 2% perf-profile.calltrace.cycles-pp.nfs_writepages.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range.nfs_file_fsync
2.77 ± 10% -0.5 2.28 ± 2% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range.nfs_file_fsync.do_fsync
2.74 ± 10% -0.5 2.25 ± 2% perf-profile.calltrace.cycles-pp.write_cache_pages.nfs_writepages.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range
3.12 ± 7% -0.5 2.64 ± 4% perf-profile.calltrace.cycles-pp.nfs_write_begin.generic_perform_write.nfs_file_write.__vfs_write.vfs_write
2.93 ± 7% -0.5 2.47 ± 3% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write.__vfs_write
3.37 ± 4% -0.5 2.91 ± 4% perf-profile.calltrace.cycles-pp.nfs_write_end.generic_perform_write.nfs_file_write.__vfs_write.vfs_write
2.51 ± 4% -0.4 2.06 ± 7% perf-profile.calltrace.cycles-pp.clear_page_erms.get_page_from_freelist.__alloc_pages_nodemask.brd_insert_page.brd_do_bvec
3.12 ± 5% -0.4 2.68 ± 7% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.brd_insert_page.brd_do_bvec.brd_make_request.generic_make_request
2.91 ± 5% -0.4 2.48 ± 3% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg
3.02 ± 5% -0.4 2.59 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.brd_insert_page.brd_do_bvec.brd_make_request
2.59 ± 4% -0.4 2.18 ± 5% perf-profile.calltrace.cycles-pp.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write.__vfs_write
2.84 ± 7% -0.4 2.43 ± 3% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write
3.27 -0.4 2.86 ± 6% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
3.20 -0.4 2.79 ± 6% perf-profile.calltrace.cycles-pp.memcpy_erms.iov_iter_copy_from_user_atomic.iomap_write_actor.iomap_apply.iomap_file_buffered_write
2.85 ± 5% -0.4 2.44 ± 3% perf-profile.calltrace.cycles-pp.memcpy_erms._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg
2.50 ± 4% -0.4 2.14 ± 7% perf-profile.calltrace.cycles-pp.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
2.41 ± 4% -0.4 2.06 ± 7% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply.iomap_file_buffered_write
2.02 ± 11% -0.3 1.67 ± 3% perf-profile.calltrace.cycles-pp.nfs_writepages_callback.write_cache_pages.nfs_writepages.do_writepages.__filemap_fdatawrite_range
0.60 ± 8% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.iomap_set_page_dirty.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write
1.96 ± 11% -0.3 1.62 ± 3% perf-profile.calltrace.cycles-pp.nfs_do_writepage.nfs_writepages_callback.write_cache_pages.nfs_writepages.do_writepages
1.55 ± 2% -0.3 1.22 ± 5% perf-profile.calltrace.cycles-pp.iomap_write_end.iomap_write_actor.iomap_apply.iomap_file_buffered_write.xfs_file_buffered_aio_write
2.35 ± 4% -0.3 2.03 ± 7% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor.iomap_apply
1.60 ± 2% -0.3 1.32 ± 4% perf-profile.calltrace.cycles-pp.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_file_fsync.do_fsync.__x64_sys_fsync
1.49 ± 6% -0.2 1.24 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.brd_insert_page.brd_do_bvec.brd_make_request.generic_make_request
1.03 ± 6% -0.2 0.80 ± 7% perf-profile.calltrace.cycles-pp.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release
0.97 ± 7% -0.2 0.74 ± 9% perf-profile.calltrace.cycles-pp.test_clear_page_writeback.end_page_writeback.nfs_end_page_writeback.nfs_write_completion.rpc_free_task
1.45 ± 6% -0.2 1.23 ± 4% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write
0.97 ± 4% -0.2 0.77 ± 3% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
1.22 ± 6% -0.2 1.03 ± 6% perf-profile.calltrace.cycles-pp.nfs_commit_release_pages.nfs_commit_release.rpc_free_task.rpc_async_release.process_one_work
1.22 ± 6% -0.2 1.03 ± 5% perf-profile.calltrace.cycles-pp.nfs_commit_release.rpc_free_task.rpc_async_release.process_one_work.worker_thread
1.25 ± 5% -0.2 1.06 ± 8% perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.pagecache_get_page.grab_cache_page_write_begin.iomap_write_begin.iomap_write_actor
1.06 ± 2% -0.2 0.87 ± 4% perf-profile.calltrace.cycles-pp.wait_on_page_bit_common.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_file_fsync.do_fsync
1.27 ± 5% -0.2 1.11 ± 6% perf-profile.calltrace.cycles-pp.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task.rpc_async_release
0.78 ± 8% -0.2 0.63 ± 6% perf-profile.calltrace.cycles-pp.__set_page_dirty_nobuffers.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write
1.11 ± 7% -0.1 0.99 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion.rpc_free_task
0.68 -0.1 0.57 ± 5% perf-profile.calltrace.cycles-pp.nfs_create_request.nfs_updatepage.nfs_write_end.generic_perform_write.nfs_file_write
1.04 ± 7% -0.1 0.93 ± 5% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback.nfs_write_completion
1.02 ± 8% -0.1 0.92 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.wake_up_page_bit.nfs_end_page_writeback
0.71 ± 2% -0.1 0.60 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit_common.__filemap_fdatawait_range
0.72 ± 2% -0.1 0.61 ± 8% perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit_common.__filemap_fdatawait_range.filemap_write_and_wait_range
0.72 -0.1 0.62 ± 7% perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit_common.__filemap_fdatawait_range.filemap_write_and_wait_range.nfs_file_fsync
0.93 ± 5% -0.1 0.83 ± 5% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write
0.73 -0.1 0.67 ± 5% perf-profile.calltrace.cycles-pp.end_page_writeback.xfs_destroy_ioend.process_one_work.worker_thread.kthread
4.86 ± 6% +0.5 5.39 perf-profile.calltrace.cycles-pp.svc_recv.nfsd.kthread.ret_from_fork
0.00 +0.5 0.54 ± 3% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.svc_recv.nfsd.kthread
0.00 +0.6 0.59 ± 6% perf-profile.calltrace.cycles-pp.ip6_xmit.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked
0.00 +0.6 0.63 ± 6% perf-profile.calltrace.cycles-pp.inet6_csk_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg
0.00 +0.7 0.65 ± 4% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.svc_recv.nfsd.kthread.ret_from_fork
0.00 +0.7 0.74 ± 8% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
0.00 +0.9 0.86 ± 8% perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages
9.98 ± 4% +2.5 12.44 ± 4% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
9.61 ± 4% +2.5 12.15 ± 4% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
47.02 ± 3% +2.7 49.68 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64
0.00 +3.5 3.54 ± 3% perf-profile.calltrace.cycles-pp.memcpy_erms.memcpy_from_page._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg
0.00 +3.6 3.56 ± 3% perf-profile.calltrace.cycles-pp.memcpy_from_page._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
0.00 +3.7 3.67 ± 3% perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages
2.29 ± 6% +3.8 6.05 ± 3% perf-profile.calltrace.cycles-pp.rpc_async_schedule.process_one_work.worker_thread.kthread.ret_from_fork
2.28 ± 6% +3.8 6.05 ± 3% perf-profile.calltrace.cycles-pp.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread.kthread
1.86 ± 3% +3.8 5.64 ± 4% perf-profile.calltrace.cycles-pp.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread
1.85 ± 3% +3.8 5.64 ± 4% perf-profile.calltrace.cycles-pp.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work
1.80 ± 4% +3.8 5.60 ± 4% perf-profile.calltrace.cycles-pp.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule
1.79 ± 3% +3.8 5.60 ± 4% perf-profile.calltrace.cycles-pp.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute
0.00 +5.5 5.47 ± 4% perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.xs_sendpages.xs_tcp_send_request
0.00 +5.6 5.56 ± 4% perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.xs_sendpages.xs_tcp_send_request.xprt_transmit
0.00 +5.6 5.57 ± 4% perf-profile.calltrace.cycles-pp.sock_sendmsg.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit
20.37 -3.3 17.04 ± 5% perf-profile.children.cycles-pp.svc_process_common
20.38 -3.3 17.04 ± 5% perf-profile.children.cycles-pp.svc_process
20.27 -3.3 16.95 ± 5% perf-profile.children.cycles-pp.nfsd_dispatch
20.21 -3.3 16.91 ± 5% perf-profile.children.cycles-pp.nfsd4_proc_compound
26.02 -3.1 22.91 ± 4% perf-profile.children.cycles-pp.nfsd
15.88 ± 5% -1.9 13.95 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
15.87 ± 5% -1.9 13.95 ± 4% perf-profile.children.cycles-pp.do_syscall_64
12.61 ± 3% -1.9 10.73 ± 3% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
12.60 ± 3% -1.9 10.73 ± 3% perf-profile.children.cycles-pp.do_writepages
9.04 ± 2% -1.9 7.18 ± 7% perf-profile.children.cycles-pp.nfsd_vfs_write
9.02 ± 2% -1.9 7.17 ± 7% perf-profile.children.cycles-pp.do_iter_readv_writev
9.02 ± 2% -1.9 7.16 ± 7% perf-profile.children.cycles-pp.xfs_file_buffered_aio_write
9.03 ± 2% -1.9 7.18 ± 7% perf-profile.children.cycles-pp.do_iter_write
9.07 ± 2% -1.9 7.22 ± 7% perf-profile.children.cycles-pp.nfsd4_write
2.21 ± 4% -1.8 0.41 ± 4% perf-profile.children.cycles-pp.inet_sendpage
2.17 ± 4% -1.8 0.40 ± 5% perf-profile.children.cycles-pp.tcp_sendpage
10.50 ± 2% -1.4 9.06 ± 4% perf-profile.children.cycles-pp.xfs_file_fsync
10.51 ± 2% -1.4 9.08 ± 4% perf-profile.children.cycles-pp.nfsd_commit
10.45 ± 2% -1.4 9.02 ± 4% perf-profile.children.cycles-pp.file_write_and_wait_range
9.82 ± 3% -1.4 8.45 ± 3% perf-profile.children.cycles-pp.xfs_vm_writepages
8.59 ± 4% -1.3 7.27 ± 4% perf-profile.children.cycles-pp.write_cache_pages
8.02 ± 4% -1.2 6.87 ± 4% perf-profile.children.cycles-pp.submit_bio
8.02 ± 4% -1.2 6.87 ± 4% perf-profile.children.cycles-pp.generic_make_request
8.00 ± 4% -1.1 6.85 ± 4% perf-profile.children.cycles-pp.brd_make_request
7.94 ± 4% -1.1 6.79 ± 4% perf-profile.children.cycles-pp.brd_do_bvec
10.63 ± 5% -1.1 9.49 ± 4% perf-profile.children.cycles-pp.write
7.55 -1.1 6.41 ± 5% perf-profile.children.cycles-pp.iomap_apply
7.54 -1.1 6.41 ± 6% perf-profile.children.cycles-pp.iomap_file_buffered_write
10.28 ± 5% -1.1 9.16 ± 4% perf-profile.children.cycles-pp.ksys_write
7.50 -1.1 6.37 ± 6% perf-profile.children.cycles-pp.iomap_write_actor
4.68 ± 5% -1.1 3.56 ± 2% perf-profile.children.cycles-pp.inet_recvmsg
4.68 ± 5% -1.1 3.56 ± 2% perf-profile.children.cycles-pp.tcp_recvmsg
10.24 ± 5% -1.1 9.12 ± 4% perf-profile.children.cycles-pp.vfs_write
10.08 ± 5% -1.1 8.98 ± 4% perf-profile.children.cycles-pp.__vfs_write
10.05 ± 5% -1.1 8.95 ± 4% perf-profile.children.cycles-pp.nfs_file_write
5.51 ± 5% -1.1 4.43 ± 5% perf-profile.children.cycles-pp.rpc_free_task
9.82 ± 5% -1.1 8.74 ± 4% perf-profile.children.cycles-pp.generic_perform_write
1.39 ± 5% -1.1 0.32 ± 4% perf-profile.children.cycles-pp.tcp_sendpage_locked
1.36 ± 5% -1.1 0.31 ± 5% perf-profile.children.cycles-pp.do_tcp_sendpages
5.31 ± 5% -1.0 4.27 ± 5% perf-profile.children.cycles-pp.rpc_async_release
4.39 ± 6% -1.0 3.40 ± 2% perf-profile.children.cycles-pp.svc_recvfrom
3.55 ± 5% -0.9 2.64 ± 2% perf-profile.children.cycles-pp.__skb_datagram_iter
3.55 ± 5% -0.9 2.64 ± 2% perf-profile.children.cycles-pp.skb_copy_datagram_iter
5.01 ± 6% -0.9 4.15 ± 3% perf-profile.children.cycles-pp.fsync
4.09 ± 7% -0.9 3.24 ± 5% perf-profile.children.cycles-pp.nfs_write_completion
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.children.cycles-pp.__x64_sys_fsync
4.91 ± 6% -0.8 4.07 ± 3% perf-profile.children.cycles-pp.nfs_file_fsync
4.90 ± 6% -0.8 4.05 ± 3% perf-profile.children.cycles-pp.do_fsync
5.36 ± 4% -0.8 4.53 ± 5% perf-profile.children.cycles-pp.grab_cache_page_write_begin
4.39 ± 7% -0.8 3.60 ± 2% perf-profile.children.cycles-pp.filemap_write_and_wait_range
5.18 ± 2% -0.8 4.40 ± 5% perf-profile.children.cycles-pp.xfs_do_writepage
5.21 ± 4% -0.7 4.47 ± 5% perf-profile.children.cycles-pp.pagecache_get_page
4.90 ± 4% -0.7 4.19 ± 5% perf-profile.children.cycles-pp.brd_insert_page
1.38 ± 9% -0.7 0.68 ± 22% perf-profile.children.cycles-pp.xfs_ilock
1.41 ± 8% -0.7 0.71 ± 22% perf-profile.children.cycles-pp.call_rwsem_down_write_failed
1.41 ± 8% -0.7 0.71 ± 22% perf-profile.children.cycles-pp.rwsem_down_write_failed
1.44 ± 7% -0.7 0.74 ± 21% perf-profile.children.cycles-pp.down_write
4.12 ± 3% -0.6 3.52 ± 4% perf-profile.children.cycles-pp.xfs_add_to_ioend
1.18 ± 6% -0.6 0.61 ± 21% perf-profile.children.cycles-pp.rwsem_spin_on_owner
3.98 ± 5% -0.5 3.44 ± 4% perf-profile.children.cycles-pp.xfs_submit_ioend
6.25 -0.5 5.72 ± 4% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
2.89 ± 6% -0.5 2.38 ± 6% perf-profile.children.cycles-pp.nfs_end_page_writeback
2.78 ± 10% -0.5 2.29 ± 2% perf-profile.children.cycles-pp.nfs_writepages
3.12 ± 7% -0.5 2.64 ± 4% perf-profile.children.cycles-pp.nfs_write_begin
3.37 ± 4% -0.5 2.91 ± 4% perf-profile.children.cycles-pp.nfs_write_end
2.54 ± 4% -0.5 2.08 ± 7% perf-profile.children.cycles-pp.clear_page_erms
2.92 ± 5% -0.4 2.48 ± 3% perf-profile.children.cycles-pp._copy_to_iter
2.60 ± 4% -0.4 2.18 ± 5% perf-profile.children.cycles-pp.nfs_updatepage
2.71 ± 4% -0.4 2.29 ± 6% perf-profile.children.cycles-pp.add_to_page_cache_lru
0.75 ± 3% -0.4 0.37 ± 6% perf-profile.children.cycles-pp.release_sock
2.51 ± 4% -0.4 2.15 ± 7% perf-profile.children.cycles-pp.iomap_write_begin
2.02 ± 10% -0.3 1.67 ± 3% perf-profile.children.cycles-pp.nfs_writepages_callback
2.23 ± 2% -0.3 1.89 ± 4% perf-profile.children.cycles-pp.__filemap_fdatawait_range
0.82 ± 7% -0.3 0.47 ± 7% perf-profile.children.cycles-pp.__tcp_push_pending_frames
1.96 ± 11% -0.3 1.63 ± 3% perf-profile.children.cycles-pp.nfs_do_writepage
1.55 ± 2% -0.3 1.22 ± 5% perf-profile.children.cycles-pp.iomap_write_end
0.78 ± 7% -0.3 0.48 ± 6% perf-profile.children.cycles-pp.svc_send
1.78 ± 3% -0.3 1.48 ± 4% perf-profile.children.cycles-pp.end_page_writeback
1.73 ± 3% -0.3 1.45 ± 4% perf-profile.children.cycles-pp.test_clear_page_writeback
0.90 ± 5% -0.3 0.63 ± 8% perf-profile.children.cycles-pp.xas_load
0.35 ± 14% -0.2 0.10 ± 14% perf-profile.children.cycles-pp.simple_copy_to_iter
1.78 ± 4% -0.2 1.54 ± 7% perf-profile.children.cycles-pp.__wake_up_common
0.43 ± 6% -0.2 0.19 ± 7% perf-profile.children.cycles-pp.lock_sock_nested
1.41 ± 5% -0.2 1.18 ± 4% perf-profile.children.cycles-pp.nfs_commit_release
1.41 ± 5% -0.2 1.18 ± 4% perf-profile.children.cycles-pp.nfs_commit_release_pages
1.95 ± 4% -0.2 1.72 ± 5% perf-profile.children.cycles-pp.try_to_wake_up
0.64 ± 8% -0.2 0.41 ± 4% perf-profile.children.cycles-pp.svc_tcp_sendto
0.63 ± 8% -0.2 0.41 ± 5% perf-profile.children.cycles-pp.svc_sendto
0.62 ± 8% -0.2 0.41 ± 5% perf-profile.children.cycles-pp.svc_send_common
0.62 ± 8% -0.2 0.41 ± 5% perf-profile.children.cycles-pp.kernel_sendpage
1.65 ± 5% -0.2 1.44 ± 7% perf-profile.children.cycles-pp.autoremove_wake_function
1.34 ± 3% -0.2 1.14 ± 3% perf-profile.children.cycles-pp.wait_on_page_bit_common
0.98 ± 4% -0.2 0.79 ± 2% perf-profile.children.cycles-pp.poll_idle
0.26 ± 8% -0.2 0.08 ± 13% perf-profile.children.cycles-pp._raw_spin_lock_bh
1.59 ± 5% -0.2 1.42 ± 5% perf-profile.children.cycles-pp.schedule
0.79 ± 5% -0.2 0.62 ± 4% perf-profile.children.cycles-pp.tcp_v6_do_rcv
0.64 ± 5% -0.2 0.47 ± 8% perf-profile.children.cycles-pp.tcp_v6_rcv
0.34 ± 12% -0.2 0.18 ± 20% perf-profile.children.cycles-pp.xas_start
0.24 ± 26% -0.2 0.08 ± 40% perf-profile.children.cycles-pp.osq_lock
0.76 ± 5% -0.2 0.61 ± 5% perf-profile.children.cycles-pp.tcp_rcv_established
0.49 ± 26% -0.2 0.34 ± 14% perf-profile.children.cycles-pp.nfs_request_add_commit_list
0.66 ± 5% -0.1 0.52 ± 8% perf-profile.children.cycles-pp.ip6_input
0.66 ± 4% -0.1 0.51 ± 8% perf-profile.children.cycles-pp.ip6_input_finish
0.65 ± 4% -0.1 0.51 ± 9% perf-profile.children.cycles-pp.ip6_protocol_deliver_rcu
0.79 ± 8% -0.1 0.64 ± 7% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.83 ± 6% -0.1 0.69 ± 8% perf-profile.children.cycles-pp.__nfs_commit_inode
0.42 ± 4% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.xs_stream_data_receive_workfn
0.42 ± 4% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.xs_stream_data_receive
0.70 ± 4% -0.1 0.56 ± 6% perf-profile.children.cycles-pp.percpu_counter_add_batch
0.71 ± 4% -0.1 0.58 ± 8% perf-profile.children.cycles-pp.ipv6_rcv
0.86 ± 6% -0.1 0.73 ± 6% perf-profile.children.cycles-pp.__local_bh_enable_ip
0.78 ± 5% -0.1 0.65 ± 8% perf-profile.children.cycles-pp.process_backlog
0.45 ± 14% -0.1 0.32 ± 14% perf-profile.children.cycles-pp.__mutex_lock
1.06 ± 4% -0.1 0.93 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.86 ± 6% -0.1 0.73 ± 9% perf-profile.children.cycles-pp.pagevec_lru_move_fn
0.75 ± 5% -0.1 0.63 ± 8% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
0.61 ± 8% -0.1 0.49 ± 9% perf-profile.children.cycles-pp.iomap_set_page_dirty
0.94 ± 5% -0.1 0.81 ± 8% perf-profile.children.cycles-pp.__lru_cache_add
0.82 ± 6% -0.1 0.70 ± 2% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.80 ± 6% -0.1 0.68 ± 7% perf-profile.children.cycles-pp.net_rx_action
0.94 ± 2% -0.1 0.82 ± 3% perf-profile.children.cycles-pp.io_schedule
0.81 ± 7% -0.1 0.70 ± 7% perf-profile.children.cycles-pp.do_softirq_own_stack
0.69 -0.1 0.58 ± 5% perf-profile.children.cycles-pp.nfs_create_request
0.62 ± 4% -0.1 0.51 ± 5% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.32 ± 4% -0.1 0.21 ± 11% perf-profile.children.cycles-pp.xs_sock_recvmsg
0.49 ± 12% -0.1 0.38 ± 7% perf-profile.children.cycles-pp.sched_ttwu_pending
0.82 ± 6% -0.1 0.72 ± 7% perf-profile.children.cycles-pp.do_softirq
0.32 ± 10% -0.1 0.22 ± 7% perf-profile.children.cycles-pp.xfs_map_blocks
0.40 ± 13% -0.1 0.30 ± 12% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.47 ± 2% -0.1 0.38 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc
0.26 ± 4% -0.1 0.17 ± 8% perf-profile.children.cycles-pp.__wake_up_common_lock
0.52 ± 6% -0.1 0.43 ± 4% perf-profile.children.cycles-pp.account_page_dirtied
0.22 ± 16% -0.1 0.13 ± 9% perf-profile.children.cycles-pp.__check_object_size
0.50 ± 8% -0.1 0.42 ± 7% perf-profile.children.cycles-pp.iomap_set_range_uptodate
0.41 ± 3% -0.1 0.33 ± 14% perf-profile.children.cycles-pp.__set_page_dirty
0.31 ± 11% -0.1 0.24 ± 11% perf-profile.children.cycles-pp.nfs_io_completion_release
0.41 ± 8% -0.1 0.33 ± 7% perf-profile.children.cycles-pp.nfs_inode_remove_request
0.25 ± 9% -0.1 0.18 ± 8% perf-profile.children.cycles-pp.xfs_iomap_write_allocate
0.33 ± 8% -0.1 0.26 ± 4% perf-profile.children.cycles-pp.___might_sleep
0.25 ± 5% -0.1 0.18 ± 15% perf-profile.children.cycles-pp.__xa_set_mark
0.31 ± 9% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.nfs_scan_commit_list
0.25 ± 5% -0.1 0.18 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.45 ± 6% -0.1 0.39 ± 4% perf-profile.children.cycles-pp.nfs_lock_and_join_requests
0.50 ± 4% -0.1 0.45 ± 5% perf-profile.children.cycles-pp.nfs_page_group_destroy
0.19 ± 11% -0.1 0.14 ± 8% perf-profile.children.cycles-pp.xfs_bmap_btalloc
0.21 ± 2% -0.1 0.15 ± 14% perf-profile.children.cycles-pp.__lock_sock
0.17 ± 8% -0.1 0.12 ± 7% perf-profile.children.cycles-pp.xfs_alloc_ag_vextent
0.18 ± 9% -0.1 0.13 ± 6% perf-profile.children.cycles-pp.xfs_alloc_vextent
0.20 ± 12% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.xfs_bmapi_write
0.30 ± 9% -0.0 0.25 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair
0.16 ± 8% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.xfs_alloc_ag_vextent_near
0.16 ± 15% -0.0 0.12 ± 17% perf-profile.children.cycles-pp.__generic_write_end
0.48 ± 2% -0.0 0.44 ± 7% perf-profile.children.cycles-pp.dequeue_task_fair
0.31 ± 2% -0.0 0.27 ± 10% perf-profile.children.cycles-pp.release_pages
0.20 ± 4% -0.0 0.17 ± 10% perf-profile.children.cycles-pp.nfs_initiate_commit
0.17 ± 10% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.__might_sleep
0.29 ± 6% -0.0 0.26 ± 4% perf-profile.children.cycles-pp.dec_zone_page_state
0.23 ± 7% -0.0 0.20 ± 7% perf-profile.children.cycles-pp.__pagevec_release
0.19 ± 9% -0.0 0.15 ± 9% perf-profile.children.cycles-pp.nfs_get_lock_context
0.17 ± 13% -0.0 0.13 ± 8% perf-profile.children.cycles-pp.check_preempt_curr
0.09 ± 17% -0.0 0.06 ± 20% perf-profile.children.cycles-pp.memset_erms
0.14 ± 8% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.nfs_request_remove_commit_list
0.07 ± 10% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.xfs_defer_finish_noroll
0.15 ± 5% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.__nfs_find_lock_context
0.19 ± 4% -0.0 0.16 ± 11% perf-profile.children.cycles-pp.mem_cgroup_try_charge
0.17 ± 8% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.vfs_create
0.16 ± 2% -0.0 0.14 ± 8% perf-profile.children.cycles-pp.__fprop_inc_percpu_max
0.16 ± 8% -0.0 0.13 ± 8% perf-profile.children.cycles-pp.xfs_create
0.10 ± 4% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.xfs_file_aio_write_checks
0.11 ± 10% -0.0 0.08 ± 15% perf-profile.children.cycles-pp.nfs_pageio_doio
0.11 ± 10% -0.0 0.08 ± 15% perf-profile.children.cycles-pp.nfs_generic_pg_pgios
0.19 ± 4% -0.0 0.17 ± 7% perf-profile.children.cycles-pp.update_rq_clock
0.15 ± 4% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.__xfs_trans_commit
0.08 ± 15% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.06 ± 15% +0.0 0.07 ± 14% perf-profile.children.cycles-pp.selinux_ip_postroute
0.08 ± 16% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.nf_hook_slow
0.04 ± 60% +0.0 0.08 ± 19% perf-profile.children.cycles-pp.__inc_numa_state
0.03 ±100% +0.0 0.07 ± 12% perf-profile.children.cycles-pp.get_task_policy
0.41 ± 6% +0.0 0.46 perf-profile.children.cycles-pp.__release_sock
0.14 ± 11% +0.1 0.19 ± 8% perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 17% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.svc_xprt_do_enqueue
0.14 ± 10% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.free_unref_page_commit
0.10 ± 11% +0.1 0.17 ± 6% perf-profile.children.cycles-pp.iov_iter_advance
0.17 ± 9% +0.1 0.24 ± 2% perf-profile.children.cycles-pp.tcp_ack
0.11 ± 21% +0.1 0.18 ± 2% perf-profile.children.cycles-pp.schedule_timeout
0.00 +0.1 0.08 ± 17% perf-profile.children.cycles-pp.free_one_page
0.23 ± 12% +0.1 0.32 ± 4% perf-profile.children.cycles-pp.__kfree_skb
0.10 ± 4% +0.1 0.19 ± 6% perf-profile.children.cycles-pp.xas_create
0.12 ± 17% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.skb_release_data
0.00 +0.1 0.14 ± 5% perf-profile.children.cycles-pp.__free_pages_ok
0.00 +0.2 0.21 ± 9% perf-profile.children.cycles-pp.skb_page_frag_refill
0.00 +0.2 0.22 ± 9% perf-profile.children.cycles-pp.sk_page_frag_refill
0.00 +0.2 0.25 ± 5% perf-profile.children.cycles-pp.__sk_flush_backlog
0.03 ±100% +0.4 0.39 ± 7% perf-profile.children.cycles-pp.free_pcppages_bulk
0.07 ± 29% +0.5 0.54 ± 6% perf-profile.children.cycles-pp.free_unref_page
4.86 ± 6% +0.5 5.40 ± 2% perf-profile.children.cycles-pp.svc_recv
9.98 ± 4% +2.5 12.44 ± 4% perf-profile.children.cycles-pp.worker_thread
9.61 ± 4% +2.5 12.15 ± 4% perf-profile.children.cycles-pp.process_one_work
47.02 ± 3% +2.7 49.68 ± 4% perf-profile.children.cycles-pp.secondary_startup_64
47.02 ± 3% +2.7 49.68 ± 4% perf-profile.children.cycles-pp.cpu_startup_entry
47.02 ± 3% +2.7 49.69 ± 4% perf-profile.children.cycles-pp.do_idle
6.12 ± 2% +2.8 8.88 ± 3% perf-profile.children.cycles-pp.memcpy_erms
0.00 +3.6 3.58 ± 3% perf-profile.children.cycles-pp.memcpy_from_page
0.00 +3.7 3.68 ± 3% perf-profile.children.cycles-pp._copy_from_iter_full
2.43 ± 6% +3.8 6.20 ± 3% perf-profile.children.cycles-pp.__rpc_execute
2.29 ± 6% +3.8 6.05 ± 3% perf-profile.children.cycles-pp.rpc_async_schedule
1.93 ± 3% +3.8 5.71 ± 4% perf-profile.children.cycles-pp.call_transmit
1.92 ± 4% +3.8 5.71 ± 4% perf-profile.children.cycles-pp.xprt_transmit
1.87 ± 4% +3.8 5.67 ± 4% perf-profile.children.cycles-pp.xs_tcp_send_request
1.86 ± 4% +3.8 5.67 ± 4% perf-profile.children.cycles-pp.xs_sendpages
0.21 ± 7% +5.3 5.54 ± 4% perf-profile.children.cycles-pp.tcp_sendmsg_locked
0.24 ± 8% +5.4 5.63 ± 4% perf-profile.children.cycles-pp.tcp_sendmsg
0.25 ± 9% +5.4 5.64 ± 4% perf-profile.children.cycles-pp.sock_sendmsg
1.17 ± 7% -0.6 0.61 ± 21% perf-profile.self.cycles-pp.rwsem_spin_on_owner
2.50 ± 4% -0.5 2.05 ± 7% perf-profile.self.cycles-pp.clear_page_erms
2.46 ± 4% -0.4 2.06 ± 5% perf-profile.self.cycles-pp.brd_do_bvec
0.22 ± 10% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__skb_datagram_iter
1.53 ± 6% -0.2 1.34 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.91 ± 4% -0.2 0.74 ± 2% perf-profile.self.cycles-pp.poll_idle
0.24 ± 25% -0.2 0.08 ± 40% perf-profile.self.cycles-pp.osq_lock
0.74 ± 14% -0.2 0.59 ± 10% perf-profile.self.cycles-pp.nfs_do_writepage
0.32 ± 13% -0.1 0.17 ± 18% perf-profile.self.cycles-pp.xas_start
0.97 ± 3% -0.1 0.84 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.57 ± 6% -0.1 0.45 ± 9% perf-profile.self.cycles-pp.xas_load
0.55 ± 10% -0.1 0.43 ± 7% perf-profile.self.cycles-pp.__schedule
0.14 ± 12% -0.1 0.03 ±100% perf-profile.self.cycles-pp._raw_spin_lock_bh
0.53 ± 8% -0.1 0.43 ± 9% perf-profile.self.cycles-pp.percpu_counter_add_batch
0.40 ± 13% -0.1 0.30 ± 12% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.33 ± 8% -0.1 0.24 ± 12% perf-profile.self.cycles-pp.nfs_end_page_writeback
0.74 ± 6% -0.1 0.65 ± 8% perf-profile.self.cycles-pp.nfs_updatepage
0.40 ± 8% -0.1 0.31 ± 11% perf-profile.self.cycles-pp.__test_set_page_writeback
0.50 ± 8% -0.1 0.41 ± 7% perf-profile.self.cycles-pp.iomap_set_range_uptodate
0.56 ± 5% -0.1 0.48 ± 8% perf-profile.self.cycles-pp.try_to_wake_up
0.38 ± 8% -0.1 0.30 ± 5% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.28 ± 9% -0.1 0.20 ± 6% perf-profile.self.cycles-pp.iomap_write_end
0.45 ± 7% -0.1 0.38 ± 6% perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.15 ± 14% -0.1 0.08 ± 27% perf-profile.self.cycles-pp.add_to_page_cache_lru
0.43 ± 6% -0.1 0.36 perf-profile.self.cycles-pp.clear_page_dirty_for_io
0.51 ± 8% -0.1 0.45 ± 7% perf-profile.self.cycles-pp.test_clear_page_writeback
0.20 ± 9% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.xas_store
0.31 ± 8% -0.1 0.25 ± 4% perf-profile.self.cycles-pp.___might_sleep
0.15 ± 20% -0.1 0.09 ± 12% perf-profile.self.cycles-pp.__check_object_size
0.22 ± 6% -0.1 0.17 ± 10% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.08 ± 11% -0.0 0.03 ±100% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.28 ± 4% -0.0 0.24 ± 10% perf-profile.self.cycles-pp.release_pages
0.27 ± 8% -0.0 0.23 ± 2% perf-profile.self.cycles-pp.dec_zone_page_state
0.19 ± 6% -0.0 0.15 ± 12% perf-profile.self.cycles-pp.kmem_cache_alloc
0.15 ± 10% -0.0 0.11 ± 7% perf-profile.self.cycles-pp.__might_sleep
0.25 ± 10% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.wait_on_page_bit_common
0.07 ± 12% -0.0 0.03 ±100% perf-profile.self.cycles-pp.___slab_alloc
0.23 ± 7% -0.0 0.20 ± 10% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.18 ± 5% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.select_task_rq_fair
0.14 ± 5% -0.0 0.11 ± 17% perf-profile.self.cycles-pp.alloc_pages_current
0.09 ± 16% -0.0 0.06 ± 20% perf-profile.self.cycles-pp.memset_erms
0.10 ± 8% -0.0 0.07 perf-profile.self.cycles-pp._cond_resched
0.15 ± 4% -0.0 0.12 ± 6% perf-profile.self.cycles-pp.__nfs_find_lock_context
0.11 ± 13% -0.0 0.09 ± 14% perf-profile.self.cycles-pp.nfs_request_add_commit_list_locked
0.09 ± 4% -0.0 0.07 perf-profile.self.cycles-pp.nfs_request_remove_commit_list
0.08 ± 10% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.08 ± 10% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.nfs_request_add_commit_list
0.10 ± 5% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
0.08 ± 13% +0.0 0.11 ± 10% perf-profile.self.cycles-pp.mem_cgroup_commit_charge
0.03 ±100% +0.0 0.07 ± 12% perf-profile.self.cycles-pp.run_timer_softirq
0.03 ±100% +0.0 0.07 ± 12% perf-profile.self.cycles-pp.get_task_policy
0.03 ±100% +0.0 0.07 ± 17% perf-profile.self.cycles-pp.__inc_numa_state
0.03 ±102% +0.1 0.08 ± 8% perf-profile.self.cycles-pp.svc_recv
0.11 ± 13% +0.1 0.17 ± 12% perf-profile.self.cycles-pp.__list_add_valid
0.09 ± 14% +0.1 0.16 ± 9% perf-profile.self.cycles-pp.iov_iter_advance
0.06 ± 13% +0.1 0.15 ± 8% perf-profile.self.cycles-pp.xas_create
0.00 +0.1 0.15 ± 14% perf-profile.self.cycles-pp.tcp_sendmsg_locked
0.00 +0.2 0.20 ± 11% perf-profile.self.cycles-pp.free_pcppages_bulk
0.04 ± 60% +0.2 0.28 ± 7% perf-profile.self.cycles-pp.svc_tcp_recvfrom
1.25 +0.3 1.52 ± 5% perf-profile.self.cycles-pp.get_page_from_freelist
6.04 ± 2% +2.7 8.74 ± 3% perf-profile.self.cycles-pp.memcpy_erms
fsmark.time.percent_of_cpu_this_job_got
320 +-+-------------------------------------------------------------------+
| |
300 +-++. + |
|. +.. .+. +.. +.. +. .. + .+.|
280 +-+ +.+. +.. : + .+..+. .+. .+. .+.+.. .. + +. |
| : + + +. +. +. .+ |
260 +-+ + + |
| |
240 +-+ |
| |
220 +-+ |
| O O O O O O O |
200 O-+ O O O O O O O O |
| O O O O O O O O O O |
180 +-+-------------------------------------------------------------------+
fsmark.time.elapsed_time
32 +-+O-------------------------------------------------------------------+
O O O O O O O O O O O O |
31 +-+ O O O O O O O O O O O O |
30 +-+ O |
| |
29 +-+ |
28 +-+ |
| |
27 +-+ |
26 +-+ .+ |
| + + +.+.. .+. + .+. .+. |
25 +-+ .. + .+ + : +..+.+..+. .. + +. +. +..|
24 +-+ .+.+..+.+ +. + + : .. + |
| +.+. + + |
23 +-+--------------------------------------------------------------------+
fsmark.time.elapsed_time.max
32 +-+O-------------------------------------------------------------------+
O O O O O O O O O O O O |
31 +-+ O O O O O O O O O O O O |
30 +-+ O |
| |
29 +-+ |
28 +-+ |
| |
27 +-+ |
26 +-+ .+ |
| + + +.+.. .+. + .+. .+. |
25 +-+ .. + .+ + : +..+.+..+. .. + +. +. +..|
24 +-+ .+.+..+.+ +. + + : .. + |
| +.+. + + |
23 +-+--------------------------------------------------------------------+
fsmark.files_per_sec
520 +-+-------------------------------------------------------------------+
|.. +.+. + +.. .+.. : .+. .+.. .+.+..+ |
500 +-+ +.. + + : +.+..+.+. +..+ .+.+. |
480 +-+ + + +.+. |
| |
460 +-+ |
| |
440 +-+ |
| |
420 +-+ |
400 +-+ O |
| O O O O O O O O O |
380 O-+ O O O O O O O O O |
| O O O O O O |
360 +-+-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 7 months
Re: [LKP] [net] 9dd6d07682: kernel_BUG_at_drivers/net/phy/mdio_bus.c
by Andrew Lunn
> phydev = dsa_port_get_phy_device(dp);
> if (!phydev)
> return 0;
>
> I don't see how this method is sane either.
If i remember correctly, the thinking was, you can get away without
the PHY, at least during bring up of a new switch/board. So don't
consider this a fatal error. Now that it is fatal, in the sense of an
Opps, maybe we should change this.
Andrew
1 year, 7 months
[ext4] 079f9927c7: ltp.mmap16.fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 079f9927c7bfa026d963db1455197159ebe5b534 ("ext4: gracefully handle ext4_break_layouts() failure during truncate")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: ltp
with following parameters:
disk: 1HDD
fs: xfs
test: syscalls_part2
test-description: The LTP testsuite contains a collection of tools for testing the Linux kernel and related features.
test-url: http://linux-test-project.github.io/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
<<<test_start>>>
tag=mmap16 stime=1559078706
cmdline="mmap16"
contacts=""
analysis=exit
<<<test_output>>>
mke2fs 1.43.4 (31-Jan-2017)
mmap16 0 TINFO : Using test device LTP_DEV='/dev/loop0'
mmap16 0 TINFO : Formatting /dev/loop0 with ext4 opts='-b 1024' extra opts='10240'
mmap16 1 TFAIL : mmap16.c:85: Bug is reproduced!
<<<execution_status>>>
initiation_status="ok"
duration=8 termination_type=exited termination_id=1 corefile=no
cutime=11 cstime=345
<<<test_end>>>
To reproduce:
# build kernel
cd linux
cp config-5.1.0-rc3-00039-g079f992 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 7 months
[net] 9dd6d07682: kernel_BUG_at_drivers/net/phy/mdio_bus.c
by kernel test robot
FYI, we noticed the following commit (built with gcc-6):
commit: 9dd6d07682b10a55d1f49d495b85f7b945ff75ca ("[PATCH 10/11] net: dsa: Use PHYLINK for the CPU/DSA ports")
url: https://github.com/0day-ci/linux/commits/Ioana-Ciornei/Decoupling-PHYLINK...
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------+------------+------------+
| | 3a2573f868 | 9dd6d07682 |
+------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 6 |
| kernel_BUG_at_drivers/net/phy/mdio_bus.c | 0 | 6 |
| invalid_opcode:#[##] | 0 | 6 |
| EIP:mdiobus_free | 0 | 6 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 6 |
+------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 39.031781] kernel BUG at drivers/net/phy/mdio_bus.c:503!
[ 39.049792] invalid opcode: 0000 [#1] PREEMPT
[ 39.058345] CPU: 0 PID: 152 Comm: kworker/0:2 Tainted: G T 5.2.0-rc1-00321-g9dd6d07 #1
[ 39.076106] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 39.091893] Workqueue: events deferred_probe_work_func
[ 39.101903] EIP: mdiobus_free+0x21/0x39
[ 39.109323] Code: 18 5f ed ff 5b 5e 5f 5d c3 55 89 e5 52 89 45 fc 8b 45 fc 8b 90 9c 00 00 00 83 fa 01 75 07 e8 9b fa 99 ff eb 1b 83 fa 03 74 02 <0f> 0b c7 80 9c 00 00 00 04 00 00 00 05 a0 00 00 00 e8 94 58 ed ff
[ 39.144715] EAX: eff3e008 EBX: eff3a020 ECX: c23a9de9 EDX: 00000002
[ 39.156773] ESI: f0403560 EDI: efffbe24 EBP: efffbdf8 ESP: efffbdf4
[ 39.168825] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00010297
[ 39.182048] CR0: 80050033 CR2: 00000000 CR3: 02781000 CR4: 000006b0
[ 39.194136] Call Trace:
[ 39.198932] _devm_mdiobus_free+0xd/0x10
[ 39.206573] release_nodes+0x194/0x1ad
[ 39.213804] devres_release_all+0x37/0x3d
[ 39.221709] really_probe+0x2b4/0x3a3
[ 39.228808] driver_probe_device+0x110/0x14b
[ 39.237206] __device_attach_driver+0x9d/0xa5
[ 39.245622] bus_for_each_drv+0x65/0x77
[ 39.253081] __device_attach+0x8f/0x104
[ 39.260507] ? driver_allows_async_probing+0x26/0x26
[ 39.270161] device_initial_probe+0x14/0x16
[ 39.278223] bus_probe_device+0x22/0x64
[ 39.285645] deferred_probe_work_func+0x7b/0xa1
[ 39.294428] process_one_work+0x1bc/0x2eb
[ 39.302332] ? process_one_work+0x164/0x2eb
[ 39.310539] process_scheduled_works+0x1e/0x24
[ 39.319218] worker_thread+0x1cb/0x268
[ 39.326604] kthread+0xeb/0xf0
[ 39.332607] ? process_scheduled_works+0x24/0x24
[ 39.341641] ? __kthread_create_on_node+0x128/0x128
[ 39.350985] ret_from_fork+0x1e/0x28
[ 39.369743] ---[ end trace 2d9c21baf7b99d11 ]---
To reproduce:
# build kernel
cd linux
cp config-5.2.0-rc1-00321-g9dd6d07 .config
make HOSTCC=gcc-6 CC=gcc-6 ARCH=i386 olddefconfig
make HOSTCC=gcc-6 CC=gcc-6 ARCH=i386 prepare
make HOSTCC=gcc-6 CC=gcc-6 ARCH=i386 modules_prepare
make HOSTCC=gcc-6 CC=gcc-6 ARCH=i386 SHELL=/bin/bash
make HOSTCC=gcc-6 CC=gcc-6 ARCH=i386 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 7 months
[mm] 42a3003535: will-it-scale.per_process_ops -25.9% regression
by kernel test robot
Greeting,
FYI, we noticed a -25.9% regression of will-it-scale.per_process_ops due to commit:
commit: 42a300353577ccc17ecc627b8570a89fa1678bec ("mm: memcontrol: fix recursive statistics correctness & scalabilty")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads Skylake-SP with 256G memory
with following parameters:
nr_task: 100%
mode: process
test: page_fault3
cpufreq_governor: performance
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-2018-04-03.cgz/lkp-skl-4sp1/page_fault3/will-it-scale
commit:
db9adbcbe7 ("mm: memcontrol: move stat/event counting functions out-of-line")
42a3003535 ("mm: memcontrol: fix recursive statistics correctness & scalabilty")
db9adbcbe740e098 42a300353577ccc17ecc627b857
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
44:4 -283% 33:4 perf-profile.calltrace.cycles-pp.error_entry.testcase
23:4 -151% 17:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry.testcase
46:4 -294% 34:4 perf-profile.children.cycles-pp.error_entry
0:4 -2% 0:4 perf-profile.children.cycles-pp.error_exit
21:4 -138% 16:4 perf-profile.self.cycles-pp.error_entry
0:4 -2% 0:4 perf-profile.self.cycles-pp.error_exit
%stddev %change %stddev
\ | \
457395 -25.9% 339092 will-it-scale.per_process_ops
87819982 -25.9% 65105780 will-it-scale.workload
14796 ± 54% +95.8% 28964 ± 24% numa-meminfo.node1.Inactive(anon)
38.02 +6.1% 40.35 ± 5% boot-time.boot
7275 +4.7% 7618 ± 5% boot-time.idle
58843 ± 96% -93.8% 3646 ± 12% cpuidle.C1.usage
735515 ± 55% -77.2% 167576 ± 4% cpuidle.C6.usage
269480 ± 10% +15.7% 311822 ± 3% sched_debug.cfs_rq:/.min_vruntime.stddev
268385 ± 10% +15.7% 310417 ± 2% sched_debug.cfs_rq:/.spread0.stddev
79.00 +3.2% 81.50 vmstat.cpu.sy
17.75 ± 2% -8.5% 16.25 ± 2% vmstat.cpu.us
7938 -2.5% 7741 vmstat.system.cs
462347 -4.4% 441932 vmstat.system.in
60227822 -22.5% 46651173 proc-vmstat.numa_hit
60082905 -22.6% 46506548 proc-vmstat.numa_local
60343904 -22.5% 46759361 proc-vmstat.pgalloc_normal
2.646e+10 -25.9% 1.962e+10 proc-vmstat.pgfault
60301946 -22.5% 46714581 proc-vmstat.pgfree
57689 ± 99% -96.0% 2309 ± 15% turbostat.C1
692737 ± 61% -83.4% 114720 ± 10% turbostat.C6
0.30 ± 67% -68.1% 0.10 ± 22% turbostat.Pkg%pc2
524.21 -6.8% 488.32 turbostat.PkgWatt
122.63 -7.6% 113.33 turbostat.RAMWatt
15022686 -18.4% 12260958 ± 2% numa-numastat.node0.local_node
15051138 -18.4% 12280459 ± 2% numa-numastat.node0.numa_hit
14900790 -24.7% 11213127 numa-numastat.node1.local_node
14941317 -24.7% 11258221 numa-numastat.node1.numa_hit
14987489 ± 2% -24.5% 11320760 ± 2% numa-numastat.node2.local_node
15022530 ± 2% -24.3% 11368910 ± 2% numa-numastat.node2.numa_hit
15145257 -22.8% 11685459 numa-numastat.node3.local_node
15186130 -22.8% 11717320 numa-numastat.node3.numa_hit
1.04 +11.4% 1.16 irq_exception_noise.__do_page_fault.60th
1.07 +22.5% 1.32 irq_exception_noise.__do_page_fault.70th
1.11 +40.9% 1.57 irq_exception_noise.__do_page_fault.80th
1.19 +65.2% 1.96 irq_exception_noise.__do_page_fault.90th
1.29 +93.5% 2.50 ± 2% irq_exception_noise.__do_page_fault.95th
1.64 +202.5% 4.96 ± 4% irq_exception_noise.__do_page_fault.99th
1.07 +28.0% 1.37 irq_exception_noise.__do_page_fault.avg
214103 +28.0% 274033 irq_exception_noise.__do_page_fault.sum
372.75 ± 3% +17.8% 439.25 ± 2% irq_exception_noise.softirq_nr
188.50 ± 11% +26.7% 238.75 ± 10% irq_exception_noise.softirq_time
8826971 ± 3% -15.6% 7452852 ± 3% numa-vmstat.node0.numa_hit
8798559 ± 3% -15.5% 7433083 ± 4% numa-vmstat.node0.numa_local
3634 ± 55% +100.6% 7291 ± 24% numa-vmstat.node1.nr_inactive_anon
3638 ± 55% +100.4% 7291 ± 24% numa-vmstat.node1.nr_zone_inactive_anon
8654591 -21.8% 6764571 numa-vmstat.node1.numa_hit
8529741 -22.2% 6635474 numa-vmstat.node1.numa_local
8789111 ± 2% -22.2% 6838852 ± 3% numa-vmstat.node2.numa_hit
8669691 ± 3% -22.6% 6706417 ± 3% numa-vmstat.node2.numa_local
8870528 ± 2% -21.2% 6989990 numa-vmstat.node3.numa_hit
8745356 ± 2% -21.4% 6873917 numa-vmstat.node3.numa_local
5.1e+10 -25.3% 3.807e+10 perf-stat.i.branch-instructions
4.024e+08 -24.3% 3.046e+08 perf-stat.i.branch-misses
70.00 -8.4 61.59 perf-stat.i.cache-miss-rate%
2.932e+08 -6.7% 2.737e+08 perf-stat.i.cache-misses
4.174e+08 +6.3% 4.439e+08 perf-stat.i.cache-references
7791 -3.4% 7529 perf-stat.i.context-switches
1.69 ± 2% +32.2% 2.23 perf-stat.i.cpi
1435 +6.9% 1534 perf-stat.i.cycles-between-cache-misses
7.381e+10 -25.2% 5.518e+10 perf-stat.i.dTLB-loads
1.447e+09 -26.0% 1.072e+09 perf-stat.i.dTLB-store-misses
4.045e+10 -25.1% 3.031e+10 perf-stat.i.dTLB-stores
68.74 -27.5 41.27 perf-stat.i.iTLB-load-miss-rate%
1.882e+08 -24.9% 1.413e+08 perf-stat.i.iTLB-load-misses
94586813 ± 5% +147.3% 2.339e+08 perf-stat.i.iTLB-loads
2.512e+11 -25.2% 1.879e+11 perf-stat.i.instructions
0.60 -25.5% 0.45 perf-stat.i.ipc
87236715 -25.5% 64948201 perf-stat.i.minor-faults
1.27 ± 34% +3.5 4.75 ± 5% perf-stat.i.node-load-miss-rate%
371727 ± 8% +291.3% 1454574 ± 2% perf-stat.i.node-load-misses
40821938 -27.1% 29771040 perf-stat.i.node-loads
7.55 ± 5% +8.2 15.80 perf-stat.i.node-store-miss-rate%
6885297 +79.9% 12389289 perf-stat.i.node-store-misses
89036136 -25.5% 66289356 perf-stat.i.node-stores
87237611 -25.5% 64948420 perf-stat.i.page-faults
1.66 +42.2% 2.36 perf-stat.overall.MPKI
0.79 +0.0 0.80 perf-stat.overall.branch-miss-rate%
70.23 -8.6 61.65 perf-stat.overall.cache-miss-rate%
1.65 +34.8% 2.22 perf-stat.overall.cpi
1414 +8.0% 1527 perf-stat.overall.cycles-between-cache-misses
0.01 ± 13% +0.0 0.01 ± 16% perf-stat.overall.dTLB-load-miss-rate%
3.45 -0.0 3.41 perf-stat.overall.dTLB-store-miss-rate%
66.57 ± 2% -28.9 37.66 perf-stat.overall.iTLB-load-miss-rate%
0.61 -25.8% 0.45 perf-stat.overall.ipc
0.90 ± 9% +3.8 4.66 ± 2% perf-stat.overall.node-load-miss-rate%
7.18 +8.6 15.75 perf-stat.overall.node-store-miss-rate%
5.079e+10 -25.3% 3.793e+10 perf-stat.ps.branch-instructions
4.011e+08 -24.3% 3.037e+08 perf-stat.ps.branch-misses
2.92e+08 -6.6% 2.726e+08 perf-stat.ps.cache-misses
4.158e+08 +6.4% 4.423e+08 perf-stat.ps.cache-references
7755 -3.2% 7509 perf-stat.ps.context-switches
7.351e+10 -25.2% 5.498e+10 perf-stat.ps.dTLB-loads
1.441e+09 -25.9% 1.068e+09 perf-stat.ps.dTLB-store-misses
4.028e+10 -25.0% 3.02e+10 perf-stat.ps.dTLB-stores
1.874e+08 -24.9% 1.407e+08 perf-stat.ps.iTLB-load-misses
94208530 ± 5% +147.3% 2.33e+08 perf-stat.ps.iTLB-loads
2.502e+11 -25.2% 1.872e+11 perf-stat.ps.instructions
86898473 -25.5% 64708711 perf-stat.ps.minor-faults
371049 ± 8% +290.7% 1449665 ± 2% perf-stat.ps.node-load-misses
40664993 -27.1% 29662127 perf-stat.ps.node-loads
6859094 +80.0% 12344092 perf-stat.ps.node-store-misses
88694004 -25.5% 66047901 perf-stat.ps.node-stores
86899384 -25.5% 64708780 perf-stat.ps.page-faults
7.618e+13 -25.5% 5.673e+13 perf-stat.total.instructions
13965 ± 7% -12.3% 12250 ± 3% softirqs.CPU104.RCU
13818 ± 6% -11.9% 12167 ± 4% softirqs.CPU105.RCU
13944 ± 6% -12.3% 12231 ± 4% softirqs.CPU106.RCU
13932 ± 8% -12.2% 12237 ± 4% softirqs.CPU107.RCU
13838 ± 6% -10.9% 12330 ± 4% softirqs.CPU109.RCU
13524 ± 6% -10.0% 12174 ± 4% softirqs.CPU111.RCU
15248 ± 10% -14.3% 13068 ± 8% softirqs.CPU112.RCU
14832 ± 7% -13.7% 12799 ± 4% softirqs.CPU113.RCU
14741 ± 8% -11.0% 13117 ± 6% softirqs.CPU114.RCU
14729 ± 8% -14.3% 12616 ± 3% softirqs.CPU115.RCU
15234 ± 6% -16.8% 12676 ± 5% softirqs.CPU116.RCU
14968 ± 7% -15.4% 12656 ± 5% softirqs.CPU117.RCU
15220 ± 8% -17.8% 12515 ± 4% softirqs.CPU118.RCU
15680 ± 7% -13.5% 13559 ± 4% softirqs.CPU120.RCU
14621 ± 8% -12.3% 12821 ± 4% softirqs.CPU123.RCU
16075 ± 11% -19.2% 12983 ± 4% softirqs.CPU13.RCU
13799 ± 10% -14.3% 11823 ± 3% softirqs.CPU133.RCU
13752 ± 9% -15.2% 11659 ± 5% softirqs.CPU134.RCU
13855 ± 9% -14.0% 11915 ± 4% softirqs.CPU138.RCU
13601 ± 9% -15.5% 11499 ± 4% softirqs.CPU142.RCU
13840 ± 11% -19.4% 11154 ± 14% softirqs.CPU145.RCU
13154 ± 7% -12.5% 11505 ± 3% softirqs.CPU147.RCU
13396 ± 7% -13.2% 11622 ± 3% softirqs.CPU148.RCU
13392 ± 7% -13.4% 11595 ± 4% softirqs.CPU150.RCU
13335 ± 6% -13.0% 11607 ± 4% softirqs.CPU151.RCU
13446 ± 6% -18.1% 11013 ± 13% softirqs.CPU152.RCU
13447 ± 6% -16.9% 11179 ± 3% softirqs.CPU153.RCU
13557 ± 9% -14.3% 11617 ± 3% softirqs.CPU154.RCU
13282 ± 8% -13.1% 11547 ± 3% softirqs.CPU155.RCU
13098 ± 8% -12.2% 11502 ± 3% softirqs.CPU158.RCU
13096 ± 8% -12.0% 11528 ± 3% softirqs.CPU159.RCU
14572 ± 9% -13.0% 12683 ± 5% softirqs.CPU160.RCU
14327 ± 5% -12.7% 12510 ± 5% softirqs.CPU161.RCU
14348 ± 6% -11.1% 12751 ± 4% softirqs.CPU162.RCU
14213 ± 6% -10.7% 12692 ± 4% softirqs.CPU163.RCU
14249 ± 6% -12.1% 12524 ± 5% softirqs.CPU164.RCU
14234 ± 6% -12.0% 12523 ± 6% softirqs.CPU165.RCU
14315 ± 5% -12.2% 12561 ± 5% softirqs.CPU166.RCU
13999 ± 6% -10.2% 12574 ± 5% softirqs.CPU167.RCU
15604 ± 6% -12.7% 13627 ± 3% softirqs.CPU17.RCU
15087 ± 10% -15.8% 12706 ± 4% softirqs.CPU174.RCU
13327 ± 8% -11.6% 11776 ± 7% softirqs.CPU185.RCU
13851 ± 9% -19.4% 11162 ± 4% softirqs.CPU187.RCU
15629 ± 6% -12.4% 13685 ± 4% softirqs.CPU20.RCU
15723 ± 6% -11.9% 13857 ± 6% softirqs.CPU21.RCU
16114 ± 7% -17.6% 13275 ± 3% softirqs.CPU22.RCU
15767 ± 6% -17.7% 12972 ± 8% softirqs.CPU23.RCU
16746 ± 8% -13.9% 14417 ± 3% softirqs.CPU24.RCU
14317 ± 5% -11.5% 12667 ± 3% softirqs.CPU3.RCU
14121 ± 9% -12.9% 12299 ± 4% softirqs.CPU32.RCU
14184 ± 8% -13.7% 12242 ± 4% softirqs.CPU38.RCU
14239 ± 9% -13.0% 12394 ± 4% softirqs.CPU40.RCU
15062 ± 8% -17.0% 12497 ± 4% softirqs.CPU41.RCU
14218 ± 8% -13.5% 12299 ± 4% softirqs.CPU42.RCU
14305 ± 8% -11.8% 12623 softirqs.CPU45.RCU
14295 ± 8% -15.5% 12080 ± 8% softirqs.CPU46.RCU
14199 ± 9% -12.7% 12397 ± 4% softirqs.CPU47.RCU
16386 ± 20% -21.3% 12892 ± 2% softirqs.CPU5.RCU
14976 ± 5% -13.0% 13026 ± 5% softirqs.CPU65.RCU
14809 ± 5% -12.6% 12942 ± 4% softirqs.CPU66.RCU
15051 ± 5% -14.5% 12865 ± 4% softirqs.CPU68.RCU
15246 ± 4% -15.4% 12900 ± 3% softirqs.CPU69.RCU
14766 ± 5% -14.6% 12617 ± 7% softirqs.CPU70.RCU
14623 ± 6% -11.4% 12949 ± 5% softirqs.CPU71.RCU
15793 ± 11% -17.6% 13009 ± 3% softirqs.CPU8.RCU
13773 ± 7% -12.6% 12032 ± 4% softirqs.CPU82.RCU
14870 ± 6% -13.3% 12894 ± 3% softirqs.CPU9.RCU
14727 ± 10% -17.6% 12135 ± 4% softirqs.CPU91.RCU
14321 ± 5% -14.0% 12313 ± 5% softirqs.CPU92.RCU
81.90 -4.5 77.37 perf-profile.calltrace.cycles-pp.page_fault.testcase
9.87 -1.1 8.80 perf-profile.calltrace.cycles-pp.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
9.50 -1.0 8.52 perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
8.17 -0.7 7.49 perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault.handle_mm_fault
5.31 -0.6 4.68 perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault
7.06 -0.6 6.45 perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode.testcase
2.44 -0.6 1.83 perf-profile.calltrace.cycles-pp.xas_load.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault
2.60 -0.5 2.10 perf-profile.calltrace.cycles-pp.fault_dirty_shared_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
1.82 -0.5 1.34 perf-profile.calltrace.cycles-pp.__do_page_fault.return_to_handler.testcase
1.82 -0.5 1.34 perf-profile.calltrace.cycles-pp.ftrace_graph_caller.__do_page_fault.return_to_handler.testcase
1.82 -0.5 1.34 perf-profile.calltrace.cycles-pp.return_to_handler.testcase
1.81 -0.5 1.33 perf-profile.calltrace.cycles-pp.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault.return_to_handler.testcase
1.75 -0.5 1.28 perf-profile.calltrace.cycles-pp.function_graph_enter.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault.return_to_handler
1.76 -0.4 1.35 perf-profile.calltrace.cycles-pp.__perf_sw_event.__do_page_fault.do_page_fault.page_fault.testcase
7.17 -0.4 6.75 perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.__handle_mm_fault
5.85 -0.4 5.43 perf-profile.calltrace.cycles-pp.ftrace_graph_caller.__do_page_fault.do_page_fault.page_fault.testcase
1.36 -0.3 1.04 perf-profile.calltrace.cycles-pp.___perf_sw_event.__perf_sw_event.__do_page_fault.do_page_fault.page_fault
0.84 -0.3 0.53 perf-profile.calltrace.cycles-pp.set_page_dirty.fault_dirty_shared_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
1.22 -0.3 0.92 perf-profile.calltrace.cycles-pp.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode.testcase
1.11 -0.3 0.81 perf-profile.calltrace.cycles-pp.trace_clock_local.function_graph_enter.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault
5.35 -0.3 5.07 perf-profile.calltrace.cycles-pp.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault.do_page_fault.page_fault
0.98 -0.3 0.71 perf-profile.calltrace.cycles-pp.sched_clock.trace_clock_local.function_graph_enter.prepare_ftrace_return.ftrace_graph_caller
0.99 -0.3 0.73 perf-profile.calltrace.cycles-pp.native_sched_clock.sched_clock.trace_clock_local.function_graph_enter.prepare_ftrace_return
1.23 ± 2% -0.2 0.98 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__perf_sw_event.do_page_fault.page_fault.testcase
1.02 -0.2 0.81 perf-profile.calltrace.cycles-pp.vmacache_find.find_vma.__do_page_fault.do_page_fault.page_fault
5.03 -0.2 4.83 perf-profile.calltrace.cycles-pp.function_graph_enter.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault.do_page_fault
1.12 -0.2 0.94 perf-profile.calltrace.cycles-pp.find_vma.__do_page_fault.do_page_fault.page_fault.testcase
1.20 -0.2 1.01 perf-profile.calltrace.cycles-pp.file_update_time.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
0.85 -0.2 0.67 perf-profile.calltrace.cycles-pp.__set_page_dirty_no_writeback.fault_dirty_shared_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
0.71 -0.2 0.54 perf-profile.calltrace.cycles-pp.__set_page_dirty_no_writeback.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
1.73 -0.1 1.58 perf-profile.calltrace.cycles-pp.__perf_sw_event.do_page_fault.page_fault.testcase
0.76 -0.1 0.64 ± 2% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
4.64 -0.1 4.55 perf-profile.calltrace.cycles-pp.trace_graph_entry.function_graph_enter.prepare_ftrace_return.ftrace_graph_caller.__do_page_fault
0.68 ± 3% -0.1 0.61 perf-profile.calltrace.cycles-pp.current_time.file_update_time.__handle_mm_fault.handle_mm_fault.__do_page_fault
0.55 +0.1 0.64 perf-profile.calltrace.cycles-pp.unlock_page.fault_dirty_shared_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
0.70 +0.3 0.97 perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault
0.61 +0.4 1.03 perf-profile.calltrace.cycles-pp.up_read.__do_page_fault.do_page_fault.page_fault.testcase
0.00 +0.8 0.82 perf-profile.calltrace.cycles-pp.lock_page_memcg.page_remove_rmap.unmap_page_range.unmap_vmas.unmap_region
0.59 +1.2 1.75 perf-profile.calltrace.cycles-pp.down_read_trylock.__do_page_fault.do_page_fault.page_fault.testcase
79.30 +1.2 80.50 perf-profile.calltrace.cycles-pp.testcase
0.00 +2.7 2.67 perf-profile.calltrace.cycles-pp.lock_page_memcg.page_add_file_rmap.alloc_set_pte.finish_fault.__handle_mm_fault
22.92 +3.4 26.37 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
0.54 +3.7 4.26 perf-profile.calltrace.cycles-pp.__count_memcg_events.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
1.30 +3.9 5.17 perf-profile.calltrace.cycles-pp.__mod_lruvec_state.page_add_file_rmap.alloc_set_pte.finish_fault.__handle_mm_fault
7.93 +4.0 11.88 perf-profile.calltrace.cycles-pp.munmap
7.88 +4.0 11.84 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
7.88 +4.0 11.84 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
7.89 +4.0 11.85 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap
7.89 +4.0 11.85 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
7.88 +4.0 11.84 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.88 +4.0 11.84 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
7.86 +4.0 11.82 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
7.78 +4.0 11.77 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
1.22 +4.0 5.26 perf-profile.calltrace.cycles-pp.__mod_lruvec_state.page_remove_rmap.unmap_page_range.unmap_vmas.unmap_region
0.53 +4.1 4.58 ± 2% perf-profile.calltrace.cycles-pp.__mod_memcg_state.__mod_lruvec_state.page_remove_rmap.unmap_page_range.unmap_vmas
0.00 +4.4 4.44 ± 2% perf-profile.calltrace.cycles-pp.__mod_memcg_state.__mod_lruvec_state.page_add_file_rmap.alloc_set_pte.finish_fault
2.47 +4.8 7.29 perf-profile.calltrace.cycles-pp.page_remove_rmap.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
5.32 +6.1 11.46 perf-profile.calltrace.cycles-pp.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
5.01 +6.2 11.24 perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault.__do_page_fault
40.44 +6.3 46.71 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.testcase
2.52 +6.3 8.84 perf-profile.calltrace.cycles-pp.page_add_file_rmap.alloc_set_pte.finish_fault.__handle_mm_fault.handle_mm_fault
38.29 +6.5 44.81 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault.testcase
25.85 +6.6 32.41 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault.testcase
92.00 -4.0 88.05 perf-profile.children.cycles-pp.testcase
6.03 -1.6 4.48 perf-profile.children.cycles-pp.sync_regs
9.89 -1.1 8.81 perf-profile.children.cycles-pp.__do_fault
9.53 -1.0 8.54 perf-profile.children.cycles-pp.shmem_fault
7.71 -0.9 6.80 perf-profile.children.cycles-pp.ftrace_graph_caller
7.21 -0.8 6.43 perf-profile.children.cycles-pp.prepare_ftrace_return
8.18 -0.7 7.50 perf-profile.children.cycles-pp.shmem_getpage_gfp
6.81 -0.7 6.13 perf-profile.children.cycles-pp.function_graph_enter
5.36 -0.6 4.72 perf-profile.children.cycles-pp.find_get_entry
2.53 -0.6 1.91 perf-profile.children.cycles-pp.xas_load
7.07 -0.6 6.45 perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
2.65 -0.6 2.08 perf-profile.children.cycles-pp.___perf_sw_event
3.51 -0.6 2.94 perf-profile.children.cycles-pp.__perf_sw_event
2.69 -0.5 2.16 perf-profile.children.cycles-pp.fault_dirty_shared_page
1.48 -0.5 0.97 perf-profile.children.cycles-pp.set_page_dirty
1.82 -0.5 1.34 perf-profile.children.cycles-pp.return_to_handler
7.24 -0.4 6.82 perf-profile.children.cycles-pp.find_lock_entry
1.46 -0.4 1.08 perf-profile.children.cycles-pp.page_mapping
1.60 -0.4 1.24 perf-profile.children.cycles-pp.__set_page_dirty_no_writeback
1.25 -0.3 0.94 perf-profile.children.cycles-pp.prepare_exit_to_usermode
1.13 -0.3 0.83 perf-profile.children.cycles-pp.trace_clock_local
1.03 -0.3 0.75 perf-profile.children.cycles-pp.sched_clock
1.00 -0.3 0.73 perf-profile.children.cycles-pp.native_sched_clock
1.01 -0.2 0.76 ± 2% perf-profile.children.cycles-pp.__mod_node_page_state
1.05 -0.2 0.82 perf-profile.children.cycles-pp.vmacache_find
1.23 -0.2 1.04 perf-profile.children.cycles-pp.file_update_time
1.15 -0.2 0.97 perf-profile.children.cycles-pp.find_vma
7.79 -0.2 7.61 perf-profile.children.cycles-pp.native_irq_return_iret
0.65 -0.1 0.50 perf-profile.children.cycles-pp.pmd_devmap_trans_unstable
4.79 -0.1 4.66 perf-profile.children.cycles-pp.trace_graph_entry
0.54 ± 2% -0.1 0.41 ± 3% perf-profile.children.cycles-pp.xas_start
0.47 -0.1 0.35 perf-profile.children.cycles-pp.__trace_graph_entry
0.77 -0.1 0.65 ± 2% perf-profile.children.cycles-pp.tlb_flush_mmu
0.23 ± 9% -0.1 0.12 ± 5% perf-profile.children.cycles-pp.retint_user
0.45 -0.1 0.34 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.35 -0.1 0.25 perf-profile.children.cycles-pp.trace_buffer_lock_reserve
0.70 -0.1 0.60 perf-profile.children.cycles-pp.___might_sleep
0.37 -0.1 0.28 perf-profile.children.cycles-pp.mark_page_accessed
0.27 ± 18% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.mem_cgroup_from_task
0.39 ± 2% -0.1 0.30 perf-profile.children.cycles-pp._cond_resched
0.35 ± 2% -0.1 0.27 perf-profile.children.cycles-pp.ftrace_lookup_ip
0.23 ± 3% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.__tlb_remove_page_size
0.72 ± 3% -0.1 0.65 perf-profile.children.cycles-pp.current_time
0.28 -0.1 0.20 ± 2% perf-profile.children.cycles-pp.__might_sleep
0.20 -0.1 0.14 ± 3% perf-profile.children.cycles-pp._vm_normal_page
0.56 -0.1 0.50 ± 2% perf-profile.children.cycles-pp.release_pages
0.22 -0.1 0.16 ± 4% perf-profile.children.cycles-pp.ring_buffer_lock_reserve
0.23 -0.1 0.18 ± 2% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.18 ± 2% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.pmd_pfn
0.22 ± 3% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.perf_exclude_event
0.21 ± 2% -0.0 0.17 ± 3% perf-profile.children.cycles-pp.pmd_page_vaddr
0.20 ± 2% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.page_rmapping
0.20 -0.0 0.15 ± 2% perf-profile.children.cycles-pp.free_pages_and_swap_cache
0.15 ± 2% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.PageHuge
0.13 ± 3% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.native_iret
0.17 -0.0 0.14 perf-profile.children.cycles-pp.rcu_all_qs
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.perf_swevent_event
0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.pte_alloc_one
0.07 ± 5% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.get_page_from_freelist
0.08 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.56 +0.1 0.64 perf-profile.children.cycles-pp.unlock_page
0.12 ± 4% +0.1 0.27 perf-profile.children.cycles-pp.__unlock_page_memcg
0.74 +0.3 1.01 perf-profile.children.cycles-pp._raw_spin_lock
0.61 +0.4 1.03 perf-profile.children.cycles-pp.up_read
61.67 +0.7 62.41 perf-profile.children.cycles-pp.page_fault
0.61 +1.2 1.77 perf-profile.children.cycles-pp.down_read_trylock
0.35 +3.2 3.51 perf-profile.children.cycles-pp.lock_page_memcg
23.05 +3.4 26.47 perf-profile.children.cycles-pp.__handle_mm_fault
0.54 +3.7 4.26 perf-profile.children.cycles-pp.__count_memcg_events
7.93 +4.0 11.88 perf-profile.children.cycles-pp.munmap
7.93 +4.0 11.89 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
7.93 +4.0 11.89 perf-profile.children.cycles-pp.do_syscall_64
7.88 +4.0 11.84 perf-profile.children.cycles-pp.__x64_sys_munmap
7.88 +4.0 11.84 perf-profile.children.cycles-pp.__vm_munmap
7.88 +4.0 11.84 perf-profile.children.cycles-pp.__do_munmap
7.88 +4.0 11.84 perf-profile.children.cycles-pp.unmap_region
7.86 +4.0 11.82 perf-profile.children.cycles-pp.unmap_vmas
7.86 +4.0 11.82 perf-profile.children.cycles-pp.unmap_page_range
2.50 +4.8 7.33 perf-profile.children.cycles-pp.page_remove_rmap
40.28 +6.0 46.28 perf-profile.children.cycles-pp.__do_page_fault
5.36 +6.1 11.49 perf-profile.children.cycles-pp.finish_fault
5.13 +6.2 11.32 perf-profile.children.cycles-pp.alloc_set_pte
40.60 +6.2 46.82 perf-profile.children.cycles-pp.do_page_fault
2.55 +6.3 8.87 perf-profile.children.cycles-pp.page_add_file_rmap
26.02 +6.5 32.55 perf-profile.children.cycles-pp.handle_mm_fault
2.61 +7.8 10.45 perf-profile.children.cycles-pp.__mod_lruvec_state
1.01 +8.0 9.03 ± 2% perf-profile.children.cycles-pp.__mod_memcg_state
22.09 -5.7 16.41 perf-profile.self.cycles-pp.testcase
5.95 -1.5 4.42 perf-profile.self.cycles-pp.sync_regs
3.44 -0.8 2.62 perf-profile.self.cycles-pp.__handle_mm_fault
2.15 -0.6 1.59 perf-profile.self.cycles-pp.handle_mm_fault
2.35 -0.5 1.85 perf-profile.self.cycles-pp.___perf_sw_event
2.03 -0.5 1.53 perf-profile.self.cycles-pp.xas_load
1.58 -0.4 1.15 perf-profile.self.cycles-pp.__do_page_fault
1.40 -0.4 1.04 perf-profile.self.cycles-pp.page_mapping
1.45 -0.3 1.13 perf-profile.self.cycles-pp.__set_page_dirty_no_writeback
1.16 -0.3 0.84 perf-profile.self.cycles-pp.alloc_set_pte
1.30 -0.3 0.99 perf-profile.self.cycles-pp.shmem_fault
5.81 -0.3 5.51 perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.99 -0.3 0.72 perf-profile.self.cycles-pp.function_graph_enter
0.97 -0.3 0.71 perf-profile.self.cycles-pp.native_sched_clock
0.90 -0.2 0.67 ± 6% perf-profile.self.cycles-pp.shmem_getpage_gfp
1.01 -0.2 0.79 perf-profile.self.cycles-pp.vmacache_find
2.39 -0.2 2.17 perf-profile.self.cycles-pp.unmap_page_range
0.92 -0.2 0.70 perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.53 -0.2 0.31 perf-profile.self.cycles-pp.set_page_dirty
0.96 -0.2 0.75 ± 2% perf-profile.self.cycles-pp.__mod_node_page_state
7.76 -0.2 7.60 perf-profile.self.cycles-pp.native_irq_return_iret
0.59 -0.1 0.45 perf-profile.self.cycles-pp.page_fault
0.60 -0.1 0.47 perf-profile.self.cycles-pp.pmd_devmap_trans_unstable
0.49 -0.1 0.36 ± 3% perf-profile.self.cycles-pp.ftrace_graph_caller
0.52 ± 2% -0.1 0.40 perf-profile.self.cycles-pp.file_update_time
0.47 ± 2% -0.1 0.35 ± 4% perf-profile.self.cycles-pp.xas_start
0.23 ± 10% -0.1 0.12 ± 5% perf-profile.self.cycles-pp.retint_user
0.42 -0.1 0.32 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.69 -0.1 0.59 perf-profile.self.cycles-pp.___might_sleep
0.35 -0.1 0.26 perf-profile.self.cycles-pp.prepare_ftrace_return
0.32 -0.1 0.23 perf-profile.self.cycles-pp.do_page_fault
0.26 ± 19% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.mem_cgroup_from_task
0.34 ± 2% -0.1 0.25 perf-profile.self.cycles-pp.ftrace_lookup_ip
0.34 -0.1 0.26 perf-profile.self.cycles-pp.mark_page_accessed
0.27 -0.1 0.20 ± 2% perf-profile.self.cycles-pp.fault_dirty_shared_page
0.21 ± 3% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.__tlb_remove_page_size
0.55 -0.1 0.48 ± 2% perf-profile.self.cycles-pp.release_pages
0.25 ± 2% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.32 ± 2% -0.1 0.26 ± 4% perf-profile.self.cycles-pp.current_time
0.21 ± 2% -0.1 0.15 ± 2% perf-profile.self.cycles-pp._cond_resched
0.26 -0.1 0.20 ± 3% perf-profile.self.cycles-pp.__do_fault
0.24 ± 4% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.finish_fault
0.22 -0.1 0.16 ± 4% perf-profile.self.cycles-pp.ring_buffer_lock_reserve
0.18 -0.1 0.12 ± 4% perf-profile.self.cycles-pp._vm_normal_page
0.22 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.16 ± 2% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.pmd_pfn
0.20 ± 2% -0.0 0.15 perf-profile.self.cycles-pp.free_pages_and_swap_cache
0.20 ± 2% -0.0 0.15 ± 4% perf-profile.self.cycles-pp.pmd_page_vaddr
0.18 ± 2% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.perf_exclude_event
0.16 ± 2% -0.0 0.12 ± 5% perf-profile.self.cycles-pp.page_rmapping
0.13 ± 3% -0.0 0.09 perf-profile.self.cycles-pp.native_iret
0.15 ± 2% -0.0 0.11 perf-profile.self.cycles-pp.__trace_graph_entry
0.12 ± 3% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.PageHuge
0.11 ± 4% -0.0 0.08 perf-profile.self.cycles-pp.trace_buffer_lock_reserve
0.10 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.trace_clock_local
0.10 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.perf_swevent_event
0.13 ± 3% -0.0 0.11 perf-profile.self.cycles-pp.rcu_all_qs
0.12 ± 3% +0.0 0.17 perf-profile.self.cycles-pp.find_vma
3.94 +0.1 4.03 perf-profile.self.cycles-pp.trace_graph_entry
0.53 +0.1 0.62 perf-profile.self.cycles-pp.unlock_page
0.11 ± 4% +0.1 0.25 ± 3% perf-profile.self.cycles-pp.__unlock_page_memcg
1.05 +0.1 1.19 ± 2% perf-profile.self.cycles-pp.page_remove_rmap
0.71 +0.3 0.99 perf-profile.self.cycles-pp._raw_spin_lock
0.88 ± 2% +0.4 1.28 perf-profile.self.cycles-pp.find_lock_entry
0.59 +0.4 1.01 ± 2% perf-profile.self.cycles-pp.up_read
0.59 +1.1 1.74 perf-profile.self.cycles-pp.down_read_trylock
0.31 +3.2 3.46 perf-profile.self.cycles-pp.lock_page_memcg
0.52 +3.7 4.25 perf-profile.self.cycles-pp.__count_memcg_events
0.97 +8.0 8.96 ± 2% perf-profile.self.cycles-pp.__mod_memcg_state
871.50 ± 42% -49.3% 441.75 ± 48% interrupts.32:PCI-MSI.26738689-edge.eth0-TxRx-0
1306534 ± 7% -11.5% 1156735 interrupts.CAL:Function_call_interrupts
21692 ± 5% -99.9% 24.75 ± 78% interrupts.CPU0.TRM:Thermal_event_interrupts
6825 ± 7% -12.1% 5997 interrupts.CPU1.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU1.TRM:Thermal_event_interrupts
6794 ± 7% -11.0% 6047 interrupts.CPU10.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU10.TRM:Thermal_event_interrupts
6847 ± 7% -12.1% 6017 interrupts.CPU100.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU100.TRM:Thermal_event_interrupts
6736 ± 8% -10.7% 6018 interrupts.CPU101.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU101.TRM:Thermal_event_interrupts
6871 ± 7% -11.8% 6059 interrupts.CPU102.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU102.TRM:Thermal_event_interrupts
6849 ± 7% -11.7% 6047 interrupts.CPU103.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU103.TRM:Thermal_event_interrupts
6847 ± 7% -11.5% 6060 interrupts.CPU104.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU104.TRM:Thermal_event_interrupts
6869 ± 7% -11.8% 6058 interrupts.CPU105.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU105.TRM:Thermal_event_interrupts
6820 ± 7% -11.0% 6071 interrupts.CPU106.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU106.TRM:Thermal_event_interrupts
6868 ± 7% -11.6% 6074 interrupts.CPU107.CAL:Function_call_interrupts
21693 ± 5% -99.9% 14.25 ±108% interrupts.CPU107.TRM:Thermal_event_interrupts
6787 ± 7% -10.6% 6070 interrupts.CPU108.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU108.TRM:Thermal_event_interrupts
6830 ± 7% -11.2% 6065 interrupts.CPU109.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU109.TRM:Thermal_event_interrupts
6836 ± 8% -11.4% 6055 interrupts.CPU11.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU11.TRM:Thermal_event_interrupts
6809 ± 7% -11.1% 6054 interrupts.CPU110.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU110.TRM:Thermal_event_interrupts
6808 ± 7% -11.1% 6054 interrupts.CPU111.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU111.TRM:Thermal_event_interrupts
6819 ± 7% -11.2% 6054 interrupts.CPU112.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU112.TRM:Thermal_event_interrupts
6806 ± 7% -11.0% 6055 interrupts.CPU113.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU113.TRM:Thermal_event_interrupts
6793 ± 7% -17.1% 5631 ± 13% interrupts.CPU114.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU114.TRM:Thermal_event_interrupts
6802 ± 7% -11.6% 6014 interrupts.CPU115.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU115.TRM:Thermal_event_interrupts
6791 ± 7% -10.8% 6060 interrupts.CPU116.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU116.TRM:Thermal_event_interrupts
6801 ± 7% -11.0% 6054 interrupts.CPU117.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU117.TRM:Thermal_event_interrupts
6842 ± 7% -11.5% 6057 interrupts.CPU118.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU118.TRM:Thermal_event_interrupts
6813 ± 7% -10.9% 6067 interrupts.CPU119.CAL:Function_call_interrupts
12.50 ± 66% +1090.0% 148.75 ± 74% interrupts.CPU119.RES:Rescheduling_interrupts
21692 ± 5% -99.9% 24.75 ± 78% interrupts.CPU119.TRM:Thermal_event_interrupts
6823 ± 8% -11.4% 6048 interrupts.CPU12.CAL:Function_call_interrupts
533.00 ± 77% +171.0% 1444 ± 36% interrupts.CPU12.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU12.TRM:Thermal_event_interrupts
6733 ± 7% -10.1% 6055 interrupts.CPU120.CAL:Function_call_interrupts
6815 ± 7% -11.1% 6061 interrupts.CPU121.CAL:Function_call_interrupts
6817 ± 7% -10.9% 6072 interrupts.CPU122.CAL:Function_call_interrupts
6806 ± 7% -10.8% 6070 interrupts.CPU123.CAL:Function_call_interrupts
6829 ± 7% -11.0% 6075 interrupts.CPU124.CAL:Function_call_interrupts
6842 ± 6% -11.3% 6070 interrupts.CPU125.CAL:Function_call_interrupts
6847 ± 7% -11.5% 6059 interrupts.CPU126.CAL:Function_call_interrupts
6844 ± 7% -11.4% 6065 interrupts.CPU127.CAL:Function_call_interrupts
6839 ± 7% -11.4% 6056 interrupts.CPU128.CAL:Function_call_interrupts
6820 ± 6% -11.2% 6054 interrupts.CPU129.CAL:Function_call_interrupts
3852 ± 34% +61.3% 6214 interrupts.CPU129.NMI:Non-maskable_interrupts
3852 ± 34% +61.3% 6214 interrupts.CPU129.PMI:Performance_monitoring_interrupts
122.75 ±121% -97.6% 3.00 ± 23% interrupts.CPU129.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU13.TRM:Thermal_event_interrupts
6845 ± 7% -11.5% 6058 interrupts.CPU130.CAL:Function_call_interrupts
6835 ± 7% -11.5% 6046 interrupts.CPU131.CAL:Function_call_interrupts
6826 ± 7% -11.4% 6047 interrupts.CPU132.CAL:Function_call_interrupts
6839 ± 7% -11.5% 6050 interrupts.CPU133.CAL:Function_call_interrupts
6832 ± 7% -11.3% 6057 interrupts.CPU134.CAL:Function_call_interrupts
6835 ± 7% -11.5% 6051 interrupts.CPU135.CAL:Function_call_interrupts
6813 ± 7% -11.4% 6037 interrupts.CPU137.CAL:Function_call_interrupts
6846 ± 7% -11.8% 6035 interrupts.CPU138.CAL:Function_call_interrupts
6823 ± 7% -11.4% 6048 interrupts.CPU139.CAL:Function_call_interrupts
6852 ± 8% -11.7% 6050 interrupts.CPU14.CAL:Function_call_interrupts
245.50 ± 74% +162.5% 644.50 ± 65% interrupts.CPU14.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU14.TRM:Thermal_event_interrupts
6787 ± 7% -10.9% 6044 interrupts.CPU140.CAL:Function_call_interrupts
6802 ± 7% -11.2% 6042 interrupts.CPU141.CAL:Function_call_interrupts
6839 ± 7% -11.6% 6046 interrupts.CPU142.CAL:Function_call_interrupts
6823 ± 7% -11.3% 6051 interrupts.CPU143.CAL:Function_call_interrupts
6693 ± 8% -11.1% 5954 ± 2% interrupts.CPU144.CAL:Function_call_interrupts
6807 ± 7% -11.5% 6025 interrupts.CPU145.CAL:Function_call_interrupts
6825 ± 7% -11.4% 6045 interrupts.CPU147.CAL:Function_call_interrupts
6861 ± 7% -11.9% 6042 interrupts.CPU148.CAL:Function_call_interrupts
6855 ± 7% -11.8% 6050 interrupts.CPU149.CAL:Function_call_interrupts
6846 ± 8% -11.8% 6037 interrupts.CPU15.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU15.TRM:Thermal_event_interrupts
6832 ± 7% -11.5% 6046 interrupts.CPU150.CAL:Function_call_interrupts
6651 ± 6% -9.3% 6034 interrupts.CPU151.CAL:Function_call_interrupts
6860 ± 7% -12.0% 6034 interrupts.CPU153.CAL:Function_call_interrupts
6843 ± 7% -11.8% 6034 interrupts.CPU154.CAL:Function_call_interrupts
6865 ± 7% -11.9% 6050 interrupts.CPU155.CAL:Function_call_interrupts
6862 ± 7% -12.2% 6026 interrupts.CPU156.CAL:Function_call_interrupts
6882 ± 8% -12.3% 6035 interrupts.CPU157.CAL:Function_call_interrupts
6854 ± 7% -12.1% 6022 interrupts.CPU158.CAL:Function_call_interrupts
6856 ± 7% -11.8% 6046 interrupts.CPU159.CAL:Function_call_interrupts
6838 ± 8% -11.8% 6033 interrupts.CPU16.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU16.TRM:Thermal_event_interrupts
6860 ± 7% -11.9% 6044 interrupts.CPU160.CAL:Function_call_interrupts
6849 ± 7% -11.8% 6042 interrupts.CPU161.CAL:Function_call_interrupts
6836 ± 7% -11.4% 6054 interrupts.CPU162.CAL:Function_call_interrupts
6850 ± 7% -13.6% 5917 ± 3% interrupts.CPU163.CAL:Function_call_interrupts
6840 ± 7% -11.5% 6056 interrupts.CPU164.CAL:Function_call_interrupts
6843 ± 7% -11.7% 6043 interrupts.CPU165.CAL:Function_call_interrupts
6858 ± 7% -11.7% 6053 interrupts.CPU166.CAL:Function_call_interrupts
6855 ± 7% -11.8% 6043 interrupts.CPU167.CAL:Function_call_interrupts
6855 ± 7% -11.1% 6095 interrupts.CPU169.CAL:Function_call_interrupts
6826 ± 8% -11.3% 6056 interrupts.CPU17.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU17.TRM:Thermal_event_interrupts
6866 ± 7% -15.8% 5782 ± 8% interrupts.CPU170.CAL:Function_call_interrupts
6867 ± 7% -11.9% 6049 interrupts.CPU171.CAL:Function_call_interrupts
6866 ± 7% -11.9% 6052 interrupts.CPU172.CAL:Function_call_interrupts
6869 ± 7% -11.9% 6053 interrupts.CPU173.CAL:Function_call_interrupts
6884 ± 7% -12.1% 6051 interrupts.CPU174.CAL:Function_call_interrupts
6858 ± 7% -13.0% 5963 ± 3% interrupts.CPU175.CAL:Function_call_interrupts
6818 ± 7% -11.2% 6056 interrupts.CPU176.CAL:Function_call_interrupts
6887 ± 7% -12.2% 6049 interrupts.CPU177.CAL:Function_call_interrupts
6849 ± 7% -11.7% 6047 interrupts.CPU179.CAL:Function_call_interrupts
6855 ± 7% -14.3% 5877 ± 4% interrupts.CPU18.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU18.TRM:Thermal_event_interrupts
6879 ± 8% -12.1% 6044 interrupts.CPU181.CAL:Function_call_interrupts
6879 ± 8% -12.0% 6054 interrupts.CPU182.CAL:Function_call_interrupts
6915 ± 7% -12.6% 6046 interrupts.CPU183.CAL:Function_call_interrupts
6892 ± 8% -12.4% 6039 interrupts.CPU184.CAL:Function_call_interrupts
6877 ± 7% -12.1% 6042 interrupts.CPU185.CAL:Function_call_interrupts
6887 ± 8% -12.5% 6024 interrupts.CPU186.CAL:Function_call_interrupts
6851 ± 8% -11.9% 6038 interrupts.CPU187.CAL:Function_call_interrupts
6855 ± 7% -11.7% 6053 interrupts.CPU188.CAL:Function_call_interrupts
6882 ± 7% -14.5% 5887 ± 5% interrupts.CPU189.CAL:Function_call_interrupts
6843 ± 8% -19.3% 5520 ± 16% interrupts.CPU19.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU19.TRM:Thermal_event_interrupts
6852 ± 7% -11.6% 6060 interrupts.CPU190.CAL:Function_call_interrupts
6876 ± 7% -11.9% 6056 interrupts.CPU191.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU2.TRM:Thermal_event_interrupts
6848 ± 8% -12.0% 6030 interrupts.CPU20.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU20.TRM:Thermal_event_interrupts
6820 ± 8% -11.3% 6046 interrupts.CPU21.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU21.TRM:Thermal_event_interrupts
6824 ± 8% -11.3% 6050 interrupts.CPU22.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU22.TRM:Thermal_event_interrupts
6828 ± 8% -11.4% 6047 interrupts.CPU23.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU23.TRM:Thermal_event_interrupts
6856 ± 7% -13.2% 5951 ± 2% interrupts.CPU24.CAL:Function_call_interrupts
6817 ± 7% -11.3% 6046 interrupts.CPU25.CAL:Function_call_interrupts
6840 ± 7% -11.7% 6039 interrupts.CPU26.CAL:Function_call_interrupts
6824 ± 7% -12.9% 5946 ± 2% interrupts.CPU27.CAL:Function_call_interrupts
137.00 ± 74% -71.7% 38.75 ±150% interrupts.CPU27.RES:Rescheduling_interrupts
6851 ± 7% -12.2% 6018 interrupts.CPU28.CAL:Function_call_interrupts
6833 ± 7% -11.3% 6062 interrupts.CPU29.CAL:Function_call_interrupts
97.50 ± 92% -92.1% 7.75 ± 69% interrupts.CPU29.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU3.TRM:Thermal_event_interrupts
6814 ± 8% -11.2% 6052 interrupts.CPU30.CAL:Function_call_interrupts
234.00 ±103% -82.2% 41.75 ±120% interrupts.CPU30.RES:Rescheduling_interrupts
6811 ± 8% -11.1% 6052 interrupts.CPU31.CAL:Function_call_interrupts
6813 ± 8% -11.1% 6059 interrupts.CPU32.CAL:Function_call_interrupts
6808 ± 8% -11.0% 6059 interrupts.CPU33.CAL:Function_call_interrupts
6842 ± 8% -11.4% 6062 interrupts.CPU34.CAL:Function_call_interrupts
6823 ± 8% -11.3% 6049 interrupts.CPU35.CAL:Function_call_interrupts
6838 ± 8% -10.9% 6092 interrupts.CPU36.CAL:Function_call_interrupts
194.50 ± 41% -80.3% 38.25 ± 93% interrupts.CPU36.RES:Rescheduling_interrupts
6835 ± 8% -11.4% 6057 interrupts.CPU37.CAL:Function_call_interrupts
6802 ± 7% -11.0% 6052 interrupts.CPU38.CAL:Function_call_interrupts
211.25 ± 74% -89.7% 21.75 ± 82% interrupts.CPU38.RES:Rescheduling_interrupts
6812 ± 7% -11.1% 6055 interrupts.CPU39.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU4.TRM:Thermal_event_interrupts
6791 ± 7% -10.8% 6057 interrupts.CPU40.CAL:Function_call_interrupts
112.25 ± 21% -75.1% 28.00 ± 82% interrupts.CPU40.RES:Rescheduling_interrupts
6798 ± 7% -11.0% 6048 interrupts.CPU42.CAL:Function_call_interrupts
6829 ± 7% -11.3% 6057 interrupts.CPU44.CAL:Function_call_interrupts
586.50 ±141% -93.6% 37.50 ± 71% interrupts.CPU45.RES:Rescheduling_interrupts
6816 ± 7% -11.2% 6051 interrupts.CPU46.CAL:Function_call_interrupts
6833 ± 7% -11.4% 6055 interrupts.CPU47.CAL:Function_call_interrupts
493.00 ± 87% -92.8% 35.50 ± 92% interrupts.CPU48.RES:Rescheduling_interrupts
6800 ± 7% -23.1% 5230 ± 23% interrupts.CPU49.CAL:Function_call_interrupts
6712 ± 8% -9.8% 6057 interrupts.CPU5.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU5.TRM:Thermal_event_interrupts
6858 ± 8% -11.6% 6066 interrupts.CPU53.CAL:Function_call_interrupts
6839 ± 8% -11.4% 6060 interrupts.CPU54.CAL:Function_call_interrupts
6838 ± 8% -11.3% 6063 interrupts.CPU55.CAL:Function_call_interrupts
6825 ± 7% -11.2% 6060 interrupts.CPU56.CAL:Function_call_interrupts
6866 ± 7% -11.7% 6061 interrupts.CPU58.CAL:Function_call_interrupts
6846 ± 8% -11.5% 6062 interrupts.CPU59.CAL:Function_call_interrupts
6852 ± 7% -12.3% 6007 interrupts.CPU6.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU6.TRM:Thermal_event_interrupts
6847 ± 8% -11.5% 6062 interrupts.CPU60.CAL:Function_call_interrupts
6842 ± 7% -11.4% 6061 interrupts.CPU63.CAL:Function_call_interrupts
6870 ± 7% -11.8% 6058 interrupts.CPU64.CAL:Function_call_interrupts
6819 ± 7% -18.7% 5542 ± 16% interrupts.CPU67.CAL:Function_call_interrupts
6819 ± 7% -11.2% 6054 interrupts.CPU68.CAL:Function_call_interrupts
6862 ± 6% -11.7% 6062 interrupts.CPU69.CAL:Function_call_interrupts
6812 ± 7% -19.8% 5466 ± 18% interrupts.CPU7.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU7.TRM:Thermal_event_interrupts
6836 ± 7% -11.3% 6064 interrupts.CPU70.CAL:Function_call_interrupts
6814 ± 7% -11.1% 6060 interrupts.CPU71.CAL:Function_call_interrupts
6816 ± 8% -11.1% 6061 interrupts.CPU72.CAL:Function_call_interrupts
6843 ± 8% -11.4% 6062 interrupts.CPU73.CAL:Function_call_interrupts
6826 ± 8% -14.7% 5820 ± 7% interrupts.CPU74.CAL:Function_call_interrupts
6820 ± 8% -11.2% 6054 interrupts.CPU75.CAL:Function_call_interrupts
164.50 ± 34% +261.2% 594.25 ± 35% interrupts.CPU75.RES:Rescheduling_interrupts
6812 ± 8% -11.1% 6053 interrupts.CPU76.CAL:Function_call_interrupts
6814 ± 8% -11.3% 6045 interrupts.CPU77.CAL:Function_call_interrupts
6818 ± 8% -11.2% 6058 interrupts.CPU78.CAL:Function_call_interrupts
6791 ± 8% -10.8% 6057 interrupts.CPU79.CAL:Function_call_interrupts
871.50 ± 42% -49.3% 441.75 ± 48% interrupts.CPU8.32:PCI-MSI.26738689-edge.eth0-TxRx-0
6729 ± 8% -10.1% 6050 interrupts.CPU8.CAL:Function_call_interrupts
316.50 ± 89% +444.2% 1722 ± 67% interrupts.CPU8.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU8.TRM:Thermal_event_interrupts
6814 ± 8% -11.1% 6058 interrupts.CPU80.CAL:Function_call_interrupts
6826 ± 7% -11.1% 6069 interrupts.CPU81.CAL:Function_call_interrupts
6832 ± 7% -11.3% 6062 interrupts.CPU88.CAL:Function_call_interrupts
6860 ± 7% -11.8% 6050 interrupts.CPU89.CAL:Function_call_interrupts
6820 ± 7% -10.6% 6097 interrupts.CPU9.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU9.TRM:Thermal_event_interrupts
6843 ± 7% -11.4% 6062 interrupts.CPU90.CAL:Function_call_interrupts
6803 ± 7% -10.8% 6066 interrupts.CPU91.CAL:Function_call_interrupts
6877 ± 8% -11.8% 6066 interrupts.CPU92.CAL:Function_call_interrupts
6880 ± 7% -17.8% 5658 ± 12% interrupts.CPU93.CAL:Function_call_interrupts
6858 ± 7% -11.7% 6058 interrupts.CPU94.CAL:Function_call_interrupts
367.50 ± 77% -58.2% 153.50 ±119% interrupts.CPU94.RES:Rescheduling_interrupts
6813 ± 7% -11.7% 6018 interrupts.CPU95.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU96.TRM:Thermal_event_interrupts
6855 ± 7% -12.2% 6017 interrupts.CPU97.CAL:Function_call_interrupts
113.25 ±105% -86.5% 15.25 ± 57% interrupts.CPU97.RES:Rescheduling_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU97.TRM:Thermal_event_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU98.TRM:Thermal_event_interrupts
6845 ± 7% -11.6% 6049 interrupts.CPU99.CAL:Function_call_interrupts
21693 ± 5% -99.9% 24.75 ± 78% interrupts.CPU99.TRM:Thermal_event_interrupts
52257 ± 4% +12.0% 58541 ± 6% interrupts.RES:Rescheduling_interrupts
1041285 ± 5% -99.9% 1177 ± 77% interrupts.TRM:Thermal_event_interrupts
irq_exception_noise.__do_page_fault.70th
1.5 +-+------------------------------------------------------------------+
| O O O |
1.45 +-+ O OO O O |
1.4 OO+O OO O O O O OO O O O |
| O O O O O O |
1.35 +-+ O OO OO |
1.3 +-+ OO OO |
| |
1.25 +-+ |
1.2 +-+ |
| |
1.15 +-+ .++ .+ |
1.1 +-+++++.++ .++++. +++ +.++++.+++.+ ++.++ +.+++.++++.+++.++++ + |
|+ + + + + ++. |
1.05 +-+------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.80th
1.8 +-+-------------------------------------------------------------------+
| |
1.7 +-+ O O O |
OO O O O OO OOOO O O O O O |
1.6 +-+ O OO O OO OO O O OOOO O O |
| O O |
1.5 +-+ |
| |
1.4 +-+ |
| |
1.3 +-+ |
| |
1.2 +-+ ++. |
|+.++++.+++.+++.++++.+ +++.++++.+++.++++.+++.+++.++++.+++.+++.++ |
1.1 +-+-------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.90th
2.2 +-+-O-----------------------------------------------------------------+
2.1 +-+ O O |
OO O O O OO OOOO O O O O O |
2 +-+ O OO O OO OO O O OOOO OOO O |
1.9 +-+ |
1.8 +-+ |
1.7 +-+ |
| |
1.6 +-+ |
1.5 +-+ |
1.4 +-+ |
1.3 +-+ |
|+.++++.+++.+++.++++.+++.+++.++++.+++.++++.+++.+++.++++.+++.+++.++ |
1.2 +-+ ++.+|
1.1 +-+-------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.95th
3 +-+-------------------------------------------------------------------+
| O |
2.8 +-+ O O |
2.6 O-+O O O O O O O O |
|O OO O O OOOO O O O O O O O O OO |
2.4 +-+ O O OO O O |
2.2 +-+ |
| |
2 +-+ |
1.8 +-+ |
| |
1.6 +-+ |
1.4 +-+ + |
|+.++++.+++.+++.++++.+ +.+++.++++.+++.++++.+++.+++.++++.+++.+++.++++.+|
1.2 +-+-------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.99th
6 +-+-------------------------------------------------------------------+
| O |
5.5 +-+ O |
5 +-+ O O O OO O |
O O OO O O OO O O O O O |
4.5 +-+ O O O O O O O O O |
4 +O+ O O O O |
| O O |
3.5 +-+ |
3 +-+ |
| |
2.5 +-+ |
2 +-+ |
| ++.+ +. ++ ++. +++.+++.+++.++ |
1.5 +-+-------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.avg
1.5 +-+-O---------------------O-O----------------------------------------+
|O O OO O |
1.45 O-+ OO OO O OO O O |
1.4 +-+O OO O O O O O O O |
| O OO OOOO OO |
1.35 +-+ |
1.3 +-+ |
| |
1.25 +-+ |
1.2 +-+ |
| + |
1.15 +-+ + + + .+ +.++++.++ |
1.1 +-+ +++.++ +++. ++ +.++++.+++.+ +.++ .+++.++ + + : |
|+.+ +.+ ++ ++ ++ ++. |
1.05 +-+------------------------------------------------------------------+
irq_exception_noise.__do_page_fault.sum
300000 +-+----------------------O-O---------------------------------------+
|O O O O O |
290000 O-+ O O OOO O O O O O |
280000 +-O OO OO OO OO O |
| O OOO OOOO O |
270000 +-+ |
260000 +-+ |
| |
250000 +-+ |
240000 +-+ |
| + |
230000 +-+ :+. + .+ ++.+++ |
220000 +-+.++++.+ +.++ .++ ++++.++++.+ ++.+ +.++++.++ + + : |
|++ ++ ++ ++ ++ +.+ |
210000 +-+----------------------------------------------------------------+
will-it-scale.per_process_ops
460000 +-+----------------------------------------------------------------+
|++.++++.++++.++++.++++.++++.++++.+++++.++++.+ +.++++.++++.+++ |
440000 +-+ |
| |
420000 +-+ |
| |
400000 +-+ |
| |
380000 +-+ |
| |
360000 +-+ |
| OO O O OO |
340000 OOO OO O O OOOO OOOO OOOO OO O OOO OOOO O |
| O |
320000 +-+----------------------------------------------------------------+
will-it-scale.workload
9e+07 +-+---------------------------------------------------------------+
|++.++++.+++++.++++.+ ++.++++.+++++.++++.+++++.+ +.++|
8.5e+07 +-+ ++ +++.+++++.+++ |
| |
| |
8e+07 +-+ |
| |
7.5e+07 +-+ |
| |
7e+07 +-+ |
| |
|O OO OOOO OOOO OOO O OOO O O O |
6.5e+07 O-O OO O O O OOOOO O O OO |
| |
6e+07 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 7 months
b915ebe6d9: kernel_selftests.bpf.make_fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: b915ebe6d9c8c6b5427e606c0ecee53df921382b ("bpf, selftest: test global data/bss/rodata sections")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-00
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url:https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot<rong.a.chen(a)intel.com>
2019-04-12 21:28:33 make run_tests -C bpf
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf'
make -C ../../../lib/bpf OUTPUT=/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/
make[1]: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/lib/bpf'
Auto-detecting system features:
... libelf: [ ^[[32mon^[[m ]
... bpf: [ ^[[32mon^[[m ]
HOSTCC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/fixdep.o
HOSTLD /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/fixdep-in.o
LINK /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/fixdep
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/bpf.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/nlattr.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/btf.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf_errno.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/str_error.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/netlink.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/bpf_prog_linfo.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf_probes.o
CC /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/xsk.o
LD /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf-in.o
LINK /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a
LINK /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.so.0.0.3
GEN /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.pc
make[1]: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/lib/bpf'
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf -Iverifier test_verifier.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_verifier
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_tag.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_tag
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_maps.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_maps
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_lru_map.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_lru_map
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_lpm_map.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_lpm_map
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include -I. -I/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf test_progs.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a trace_helpers.c prog_tests/stacktrace_map.c prog_tests/xdp_noinline.c prog_tests/stacktrace_map_raw_tp.c prog_tests/bpf_verif_scale.c prog_tests/signal_pending.c prog_tests/reference_tracking.c prog_tests/prog_run_xattr.c prog_tests/task_fd_query_tp.c prog_tests/tp_attach_query.c prog_tests/get_stack_raw_tp.c prog_tests/pkt_md_access.c prog_tests/xdp_adjust_tail.c prog_tests/stacktrace_build_id.c prog_tests/bpf_obj_id.c prog_tests/pkt_access.c prog_tests/spinlock.c prog_tests/task_fd_query_rawtp.c prog_tests/xdp.c prog_tests/queue_stack_map.c prog_tests/flow_dissector.c prog_tests/tcp_estats.c prog_tests/obj_name.c prog_tests/map_lock.c prog_tests/global_data.c prog_tests/l4lb_all.c prog_tests/stacktrace_build_id_nmi.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_progs
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_align.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_align
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_verifier_log.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_verifier_log
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_dev_cgroup.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_dev_cgroup
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_tcpbpf_user.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_tcpbpf_user
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_sock.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_sock
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_btf.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_btf
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_sockmap.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_sockmap
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include get_cgroup_id_user.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/get_cgroup_id_user
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_socket_cookie.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_socket_cookie
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_cgroup_storage.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_cgroup_storage
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_select_reuseport.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_select_reuseport
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_section_names.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_section_names
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_netcnt.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_netcnt
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_tcpnotify_user.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c trace_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_tcpnotify_user
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_sock_fields.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_sock_fields
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_libbpf_open.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_libbpf_open
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_sock_addr.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_sock_addr
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_skb_cgroup_id_user.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a cgroup_helpers.c -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_skb_cgroup_id_user
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include flow_dissector_load.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/flow_dissector_load
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_flow_dissector.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_flow_dissector
gcc -Wall -O2 -I../../../include/uapi -I../../../lib -I../../../lib/bpf -I../../../../include/generated -I../../../include test_tcp_check_syncookie_user.c /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/libbpf.a -lcap -lelf -lrt -lpthread -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_tcp_check_syncookie_user
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/test_stack_map.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_stack_map.o
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/test_tunnel_kern.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_tunnel_kern.o
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/dev_cgroup.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/dev_cgroup.o
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/test_lwt_ip_encap.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_lwt_ip_encap.o
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/test_obj_id.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_obj_id.o
clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /usr/lib/llvm-7/lib/clang/7.0.1/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -Wno-compare-distinct-pointer-types \
-O2 -target bpf -emit-llvm -c progs/test_global_data.c -o - | \
llc -march=bpf -mcpu=probe -filetype=obj -o /usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_global_data.o
LLVM ERROR: Unsupported relocation: try to compile with -O2 or above, or check your static variable usage
Makefile:204: recipe for target '/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_global_data.o' failed
make: *** [/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf/test_global_data.o] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-b915ebe6d9c8c6b5427e606c0ecee53df921382b/tools/testing/selftests/bpf'
To reproduce:
# build kernel
cd linux
cp config-5.1.0-rc7-02038-g71ab3fd .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clonehttps://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 7 months
eb9d1bf079 [ 88.881528] EIP: _random_read
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
commit eb9d1bf079bb438d1a066d72337092935fc770f6
Author: Theodore Ts'o <tytso(a)mit.edu>
AuthorDate: Wed Feb 20 16:06:38 2019 -0500
Commit: Theodore Ts'o <tytso(a)mit.edu>
CommitDate: Wed Apr 17 10:30:21 2019 -0400
random: only read from /dev/random after its pool has received 128 bits
Immediately after boot, we allow reads from /dev/random before its
entropy pool has been fully initialized. Fix this so that we don't
allow this until the blocking pool has received 128 bits.
We do this by repurposing the initialized flag in the entropy pool
struct, and use the initialized flag in the blocking pool to indicate
whether it is safe to pull from the blocking pool.
To do this, we needed to rework when we decide to push entropy from the
input pool to the blocking pool, since the initialized flag for the
input pool was used for this purpose. To simplify things, we no
longer use the initialized flag for that purpose, nor do we use the
entropy_total field any more.
Signed-off-by: Theodore Ts'o <tytso(a)mit.edu>
764ed189c8 drivers/char/random.c: make primary_crng static
eb9d1bf079 random: only read from /dev/random after its pool has received 128 bits
814137768b Add gitignore file for samples/vfs/ generated files
+---------------------------------------------------------+------------+------------+------------+
| | 764ed189c8 | eb9d1bf079 | 814137768b |
+---------------------------------------------------------+------------+------------+------------+
| boot_successes | 3035 | 2712 | 2036 |
| boot_failures | 915 | 887 | 656 |
| BUG:kernel_hang_in_boot-around-mounting-root_stage | 818 | 720 | 457 |
| BUG:kernel_timeout_in_boot-around-mounting-root_stage | 55 | 51 | 63 |
| BUG:soft_lockup-CPU##stuck_for#s | 26 | 23 | 24 |
| EIP:separate_adjacent_colors | 14 | 15 | 5 |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 25 | 101 | 119 |
| EIP:rb_next | 7 | 3 | 5 |
| EIP:igt_color | 1 | | |
| EIP:rb_prev | 2 | 3 | 2 |
| Mem-Info | 17 | 15 | 15 |
| EIP:drm_mm_insert_node_in_range | 2 | 1 | 4 |
| invoked_oom-killer:gfp_mask=0x | 1 | | |
| Out_of_memory_and_no_killable_processes | 1 | | |
| Kernel_panic-not_syncing:System_is_deadlocked_on_memory | 1 | | |
| EIP:__slab_alloc | 0 | 1 | |
| EIP:debug_lockdep_rcu_enabled | 0 | 8 | 16 |
| EIP:lock_is_held_type | 0 | 41 | 63 |
| EIP:_random_read | 0 | 14 | 12 |
| EIP:___might_sleep | 0 | 11 | 7 |
| EIP:xfer_secondary_pool | 0 | 1 | |
| EIP:rcu_read_lock_sched_held | 0 | 3 | 3 |
| BUG:kernel_reboot-without-warning_in_test_stage | 0 | 0 | 2 |
| EIP:__might_sleep | 0 | 0 | 2 |
+---------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
01 00 00 00 60 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 ff ef ff ff 00 00 00 00 00 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 47 31 f7 0f 00 00 00 00 1b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 ac 88 ea 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
02 00 00 00 70 00 00 00 37 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 33 c5 01 00 00 00 00 00 00 00 20 00 00 00 00 00 ff ff 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 80 00 00 ff ff 00 00
00 00 00 00 00 00 00 00 00 00
[ 88.862983] irq event stamp: 294320
[ 88.864335] hardirqs last enabled at (294319): [<c2801237>] trace_hardirqs_on_thunk+0xc/0x10
[ 88.867241] hardirqs last disabled at (294320): [<c2801247>] trace_hardirqs_off_thunk+0xc/0x10
[ 88.870159] softirqs last enabled at (290628): [<c2f59cc7>] __do_softirq+0x287/0x2b5
[ 88.872933] softirqs last disabled at (290617): [<c2813e32>] do_softirq_own_stack+0x1c/0x22
[ 88.875826] CPU: 0 PID: 904 Comm: trinity-c0 Not tainted 5.1.0-rc5-00004-geb9d1bf #196
[ 88.878665] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 88.881528] EIP: _random_read+0xa7/0x31f
[ 88.882986] Code: 75 e9 ff 0d 4c 32 30 c3 ff 05 4c 32 30 c3 a1 dc 8c 51 c3 e8 99 30 c2 ff 85 c0 74 2d 80 3d 01 30 51 c3 00 75 24 e8 94 3e c2 ff <85> c0 75 1b b9 7c b8 1c c3 ba 05 01 00 00 b8 90 6c 24 c3 c6 05 01
[ 88.888325] EAX: 00000001 EBX: 00000000 ECX: 00000000 EDX: dc525140
[ 88.890136] ESI: ffc02100 EDI: c2c591aa EBP: dc365f68 ESP: dc365f1c
[ 88.892016] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00000202
[ 88.893927] CR0: 80050033 CR2: 08c99024 CR3: 1c355000 CR4: 003406d0
[ 88.895775] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 88.897651] DR6: fffe0ff0 DR7: 00000400
[ 88.899080] Call Trace:
[ 88.900279] ? __wake_up_locked_key+0x17/0x17
[ 88.901827] sys_getrandom+0x35/0x70
[ 88.903244] ? lockdep_hardirqs_on+0x148/0x163
[ 88.904810] do_int80_syscall_32+0x4f/0x116
[ 88.906344] entry_INT80_32+0xe1/0xe1
[ 88.907738] EIP: 0x809af42
[ 88.908995] Code: 89 c8 c3 90 8d 74 26 00 85 c0 c7 01 01 00 00 00 75 d8 a1 ec bd a7 08 eb d1 66 90 66 90 66 90 66 90 66 90 66 90 66 90 90 cd 80 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 8b 10 a3 14 be a7 08 85
[ 88.914344] EAX: ffffffda EBX: b6f59000 ECX: 001d2000 EDX: 00000002
[ 88.916234] ESI: 484b29b7 EDI: ffffff5c EBP: 00200000 ESP: bf88ed58
[ 88.918112] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000292
[ 88.920091] Kernel panic - not syncing: softlockup: hung tasks
[ 88.921900] CPU: 0 PID: 904 Comm: trinity-c0 Tainted: G L 5.1.0-rc5-00004-geb9d1bf #196
[ 88.925004] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 88.927916] Call Trace:
[ 88.929120] <IRQ>
[ 88.930261] dump_stack+0x16/0x18
[ 88.931622] panic+0x9f/0x20e
[ 88.932931] watchdog_timer_fn+0x1db/0x200
[ 88.934461] __hrtimer_run_queues+0x1a0/0x297
[ 88.936198] ? __touch_watchdog+0x15/0x15
[ 88.937690] hrtimer_run_queues+0xd1/0xe6
[ 88.939177] run_local_timers+0x8/0x2e
[ 88.940582] update_process_times+0x1b/0x43
[ 88.942084] tick_nohz_handler+0xb2/0xfe
[ 88.943558] timer_interrupt+0xd/0x14
[ 88.944965] __handle_irq_event_percpu+0x9f/0x1ad
[ 88.946577] ? trace_hardirqs_off+0xa3/0xa9
[ 88.948079] handle_irq_event_percpu+0x17/0x3d
[ 88.949584] handle_irq_event+0x29/0x42
[ 88.951018] ? irq_set_chip_and_handler_name+0x24/0x24
[ 88.952706] handle_level_irq+0x82/0xae
[ 88.954156] handle_irq+0x73/0x95
[ 88.955492] </IRQ>
[ 88.956648] do_IRQ+0x36/0x92
[ 88.957913] ? sys_getrandom+0x35/0x70
[ 88.959343] common_interrupt+0xeb/0xf0
[ 88.960769] EIP: _random_read+0xa7/0x31f
[ 88.962202] Code: 75 e9 ff 0d 4c 32 30 c3 ff 05 4c 32 30 c3 a1 dc 8c 51 c3 e8 99 30 c2 ff 85 c0 74 2d 80 3d 01 30 51 c3 00 75 24 e8 94 3e c2 ff <85> c0 75 1b b9 7c b8 1c c3 ba 05 01 00 00 b8 90 6c 24 c3 c6 05 01
[ 88.967549] EAX: 00000001 EBX: 00000000 ECX: 00000000 EDX: dc525140
[ 88.969410] ESI: ffc02100 EDI: c2c591aa EBP: dc365f68 ESP: dc365f1c
[ 88.971305] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00000202
[ 88.973236] ? sys_getrandom+0x35/0x70
[ 88.974648] ? autoconfig_16550a+0x241/0x4bd
[ 88.976201] ? __wake_up_locked_key+0x17/0x17
[ 88.977724] sys_getrandom+0x35/0x70
[ 88.979094] ? lockdep_hardirqs_on+0x148/0x163
[ 88.980534] do_int80_syscall_32+0x4f/0x116
[ 88.981918] entry_INT80_32+0xe1/0xe1
[ 88.983196] EIP: 0x809af42
[ 88.984358] Code: 89 c8 c3 90 8d 74 26 00 85 c0 c7 01 01 00 00 00 75 d8 a1 ec bd a7 08 eb d1 66 90 66 90 66 90 66 90 66 90 66 90 66 90 90 cd 80 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 8b 10 a3 14 be a7 08 85
[ 88.989303] EAX: ffffffda EBX: b6f59000 ECX: 001d2000 EDX: 00000002
[ 88.990988] ESI: 484b29b7 EDI: ffffff5c EBP: 00200000 ESP: bf88ed58
[ 88.992737] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000292
[ 88.994519] Kernel Offset: 0x1800000 from 0xc1000000 (relocation range: 0xc0000000-0xe07dffff)
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 724086ae5682c612e822727bae3efe01ae13cc41 e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd --
git bisect bad 87b53080376c675a80ea63b58b9f22670dbb970d # 01:07 B 200 3 0 88 Merge 'drm-drm-misc/drm-misc-next-fixes' into devel-hourly-2019051401
git bisect bad 35d593de37a3dc1bb758fa091b5f318cb9fb22e3 # 01:07 B 178 5 0 131 Merge 'jpirko-mlxsw/shalomt_ptp' into devel-hourly-2019051401
git bisect bad a9fe71b64b806000437b97ead2b436cd194e7449 # 01:07 B 195 5 0 234 Merge 'linux-review/Dan-Carpenter/drm-i915-gvt-Fix-an-error-code-in-ppgtt_populate_spt_by_guest_entry/20190514-001803' into devel-hourly-2019051401
git bisect good 9381a094625c169860695287c827468a18031b43 # 01:38 G 903 0 206 306 Merge 'ljones-mfd/ib-mfd-pinctrl-5.2-2' into devel-hourly-2019051401
git bisect bad 947a45eff9cae803662bb9e7126e94689ff742eb # 01:38 B 169 4 0 268 Merge 'block/io_uring-next' into devel-hourly-2019051401
git bisect bad 80f232121b69cc69a31ccb2b38c1665d770b0710 # 01:39 B 167 5 0 293 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
git bisect good e0dccbdf5ac7ccb9da5612100dedba302f3ebcfe # 02:08 G 903 0 174 255 Merge tag 'staging-5.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
git bisect good 5f0d736e7f7db586141f974821b6ca6c1d906d5b # 02:42 G 900 0 176 247 Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
git bisect good 5abe37954e9a315c35c9490f78d55f307c3c636b # 03:17 G 901 0 190 313 Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4
git bisect good 6ffe0acc935f344eb0b35da07c034d5122222e77 # 03:50 G 901 0 195 299 Merge tag 'wireless-drivers-next-for-davem-2019-05-03' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next
git bisect good 54516da1ea859dd4f56ebba2e483d2df9d7c8a32 # 04:18 G 905 0 166 255 Merge branch 'r8169-replace-some-magic-with-more-speaking-functions'
git bisect good 23bfaa594002f4bba085e0a1ae3c9847b988d816 # 04:54 G 904 0 196 297 net: phy: improve pause mode reporting in phy_print_status
git bisect bad 82efe439599439a5e1e225ce5740e6cfb777a7dd # 04:54 B 167 7 0 136 Merge tag 'devicetree-for-5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
git bisect good 6f7dc9a37f2b325fc870d1e1ada6169185f8852c # 05:27 G 906 0 199 299 of: irq: Remove WARN_ON() for kzalloc() failure
git bisect good a9fbcd6728837268784439ad0b02ede2c024c516 # 05:40 G 900 0 50 353 Merge tag 'fscrypt_for_linus' of git://git.kernel.org/pub/scm/fs/fscrypt/fscrypt
git bisect bad d55535232c3dbde9a523a9d10d68670f5fe5dec3 # 05:44 B 223 1 8 145 random: move rand_initialize() earlier
git bisect good 3bd0b5bf7dc3ea02070fcbcd682ecf628269e8ef # 07:13 G 910 0 311 726 drivers/char/random.c: remove unused stuct poolinfo::poolbits
git bisect bad eb9d1bf079bb438d1a066d72337092935fc770f6 # 07:35 B 14 1 5 5 random: only read from /dev/random after its pool has received 128 bits
git bisect good 764ed189c82090c1d85f0e30636156736d8f09a8 # 08:59 G 902 0 276 630 drivers/char/random.c: make primary_crng static
# first bad commit: [eb9d1bf079bb438d1a066d72337092935fc770f6] random: only read from /dev/random after its pool has received 128 bits
git bisect good 764ed189c82090c1d85f0e30636156736d8f09a8 # 09:17 G 1001 0 85 754 drivers/char/random.c: make primary_crng static
# extra tests with debug options
git bisect bad eb9d1bf079bb438d1a066d72337092935fc770f6 # 09:33 B 1 1 0 0 random: only read from /dev/random after its pool has received 128 bits
# extra tests on HEAD of linux-devel/devel-hourly-2019051401
git bisect bad 724086ae5682c612e822727bae3efe01ae13cc41 # 09:33 B 77 2 0 35 0day head guard for 'devel-hourly-2019051401'
# extra tests on tree/branch linus/master
git bisect bad 814137768b5a9504f758aa760e7b1ac355539783 # 11:55 B 115 1 43 345 Add gitignore file for samples/vfs/ generated files
# extra tests with first bad commit reverted
git bisect good 63fc3aff16bb767e89202696f7be3f56252fc091 # 18:34 G 901 0 233 586 Revert "random: only read from /dev/random after its pool has received 128 bits"
# extra tests on tree/branch linux-next/master
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 7 months
27d21449d3 ("sched: Add task_struct pointer to .."): WARNING: CPU: 1 PID: 15 at kernel/sched/sched.h:1719 pick_next_task_fair
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://github.com/digitalocean/linux-coresched coresched
commit 27d21449d31b065ebaf1be593f2874096f48eaef
Author: Peter Zijlstra <peterz(a)infradead.org>
AuthorDate: Mon Feb 18 17:56:25 2019 +0100
Commit: Vineeth Remanan Pillai <vpillai(a)digitalocean.com>
CommitDate: Tue May 21 13:55:24 2019 +0000
sched: Add task_struct pointer to sched_class::set_curr_task
In preparation of further separating pick_next_task() and
set_curr_task() we have to pass the actual task into it, while there,
rename the thing to better pair with put_prev_task().
Signed-off-by: Peter Zijlstra (Intel) <peterz(a)infradead.org>
1b3e82fe5a sched/{rt,deadline}: Fix set_next_task vs pick_next_task
27d21449d3 sched: Add task_struct pointer to sched_class::set_curr_task
8d75dc6554 prio_less refactoring
+------------------------------------------------------+------------+------------+------------+
| | 1b3e82fe5a | 27d21449d3 | 8d75dc6554 |
+------------------------------------------------------+------------+------------+------------+
| boot_successes | 41 | 0 | 2 |
| boot_failures | 1 | 10 | 11 |
| BUG:soft_lockup-CPU##stuck_for#s | 1 | 1 | |
| EIP:prep_new_page | 1 | | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 1 | 1 | |
| WARNING:at_kernel/sched/sched.h:#pick_next_task_fair | 0 | 9 | |
| EIP:pick_next_task_fair | 0 | 9 | |
| WARNING:at_kernel/sched/sched.h:#put_prev_task | 0 | 9 | 11 |
| EIP:put_prev_task | 0 | 9 | 11 |
| EIP:__drm_mm_interval_first | 0 | 1 | |
+------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
Please press Enter to activate this console.
[ 69.039734] random: ubusd: uninitialized urandom read (4 bytes read)
[ 69.043732] random: ubusd: uninitialized urandom read (4 bytes read)
[ 69.044626] random: ubusd: uninitialized urandom read (4 bytes read)
[ 70.024617] Unregister pv shared memory for cpu 1
[ 70.027227] WARNING: CPU: 1 PID: 15 at kernel/sched/sched.h:1719 pick_next_task_fair+0x33/0x3b5
[ 70.028193] CPU: 1 PID: 15 Comm: migration/1 Not tainted 5.2.0-rc1-00103-g27d2144 #1
[ 70.028938] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 70.029704] EIP: pick_next_task_fair+0x33/0x3b5
[ 70.030161] Code: 56 53 89 c3 83 ec 2c 89 55 d8 89 4d dc b8 40 e0 05 82 89 45 c8 83 7b 6c 00 0f 84 13 01 00 00 8b 45 d8 3b 83 44 05 00 00 74 02 <0f> 0b 8d 43 60 89 45 e8 8b 45 d8 8b 48 50 89 c2 89 d8 ff 51 1c 8d
[ 70.031910] EAX: 81d0d800 EBX: 9f81c040 ECX: 9f27becc EDX: 81d0d800
[ 70.032507] ESI: 81d0d800 EDI: 8186f120 EBP: 9f27be94 ESP: 9f27be5c
[ 70.033117] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010087
[ 70.033775] CR0: 80050033 CR2: 77f6053c CR3: 1d511000 CR4: 001406d0
[ 70.034403] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 70.035000] DR6: fffe0ff0 DR7: 00000400
[ 70.035365] Call Trace:
[ 70.035600] ? update_irq_load_avg+0x2df/0x2ff
[ 70.036007] ? pick_next_task+0x4c/0x74
[ 70.036386] ? sched_cpu_dying+0x92/0x1ab
[ 70.036838] ? sched_cpu_starting+0x2e/0x2e
[ 70.037349] ? cpuhp_invoke_callback+0xab/0x215
[ 70.037903] ? sched_cpu_starting+0x2e/0x2e
[ 70.038429] ? preempt_count_sub+0x14/0x1e
[ 70.038924] ? take_cpu_down+0x56/0x7f
[ 70.039381] ? multi_cpu_stop+0x72/0xd6
[ 70.039858] ? cpumask_test_cpu+0x15/0x15
[ 70.040364] ? cpu_stopper_thread+0x78/0xd6
[ 70.040806] ? smpboot_thread_fn+0x1df/0x1f3
[ 70.041337] ? kthread+0xd8/0xda
[ 70.041648] ? smpboot_destroy_threads+0x67/0x67
[ 70.042186] ? __kthread_create_on_node+0x125/0x125
[ 70.042707] ? ret_from_fork+0x1e/0x30
[ 70.043122] ---[ end trace 2c1f854fe6492c7f ]---
[ 70.043604] WARNING: CPU: 1 PID: 15 at kernel/sched/sched.h:1719 put_prev_task+0x1a/0x24
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 2fa2a748b230dca1b0664a6ed78068bb86e002aa f49aa1de98363b6c5fba4637678d6b0ba3d18065 --
git bisect bad b0655ef051a4d1cea71effa1258285ee770916ac # 23:47 B 1 17 0 0 sched/fair: Export newidle_balance()
git bisect good 4ef3eb925e2bd3eb4bcf7484bd3804d5b526b3c3 # 23:48 G 18 0 0 0 stop_machine: Fix stop_cpus_in_progress ordering
git bisect good 811221193b809099f9c4d6cf37b85981c9e7a635 # 23:48 G 11 0 0 0 sched: Wrap rq::lock access
git bisect good 1b3e82fe5a4fc1e07d8b71d785ac808238365429 # 23:48 G 11 0 0 1 sched/{rt,deadline}: Fix set_next_task vs pick_next_task
git bisect bad 27d21449d31b065ebaf1be593f2874096f48eaef # 00:17 B 1 5 1 1 sched: Add task_struct pointer to sched_class::set_curr_task
# first bad commit: [27d21449d31b065ebaf1be593f2874096f48eaef] sched: Add task_struct pointer to sched_class::set_curr_task
git bisect good 1b3e82fe5a4fc1e07d8b71d785ac808238365429 # 00:35 G 31 0 0 1 sched/{rt,deadline}: Fix set_next_task vs pick_next_task
# extra tests on HEAD of digitalocean-linux-coresched/coresched
git bisect bad 2fa2a748b230dca1b0664a6ed78068bb86e002aa # 00:35 B 0 21 40 0 sched: Debug bits...
# extra tests on tree/branch digitalocean-linux-coresched/coresched
git bisect bad 8d75dc65548b1dbccb9e3480c5130b9a6c7e626f # 00:38 B 0 11 36 0 prio_less refactoring
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 7 months
Re: [LKP] CKI hackfest @Plumbers invite
by Veronika Kabatova
----- Original Message -----
> From: "Tim Bird" <Tim.Bird(a)sony.com>
> To: vkabatov(a)redhat.com, automated-testing(a)yoctoproject.org, info(a)kernelci.org, khilamn(a)baylibre.org,
> syzkaller(a)googlegroups.com, lkp(a)lists.01.org, stable(a)vger.kernel.org, labbott(a)redhat.com
> Cc: eslobodo(a)redhat.com, cki-project(a)redhat.com
> Sent: Friday, May 24, 2019 10:17:04 PM
> Subject: RE: CKI hackfest @Plumbers invite
>
>
>
> > -----Original Message-----
> > From: Veronika Kabatova
> >
> > Hi,
> >
> > as some of you have heard, CKI Project is planning hackfest CI meetings
> > after
> > Plumbers conference this year (Sept. 12-13). We would like to invite
> > everyone
> > who has interest in CI for kernel to come and join us.
> >
> > The early agenda with summary is at the end of the email. If you think
> > there's
> > something important missing let us know! Also let us know in case you'd
> > want to
> > lead any of the sessions, we'd be happy to delegate out some work :)
> >
> >
> > Please send us an email as soon as you decide to come and feel free to
> > invite
> > other people who should be present. We are not planning to cap the
> > attendance
> > right now but need to solve the logistics based on the interest. The event
> > is
> > free to attend, no additional registration except letting us know is
> > needed.
> >
> > Feel free to contact us if you have any questions,
>
> I plan to come to the event.
>
> > -----------------------------------------------------------
> > Here is an early agenda we put together:
> > - Introductions
> > - Common place for upstream results, result publishing in general
> > - The discussion on the mailing list is going strong so we might be able
> > to
> > substitute this session for a different one in case everything is
> > solved by
> > September.
> > - Test result interpretation and bug detection
> > - How to autodetect infrastructure failures, regressions/new bugs and
> > test
> > bugs? How to handle continuous failures due to known bugs in both tests
> > and
> > kernel? What's your solution? Can people always trust the results they
> > receive?
> > - Getting results to developers/maintainers
> > - Aimed at kernel developers and maintainers, share your feedback and
> > expectations.
> > - How much data should be sent in the initial communication vs. a click
> > away
> > in a dashboard? Do you want incremental emails with new results as they
> > come
> > in?
> > - What about adding checks to tested patches in Patchwork when patch
> > series
> > are being tested?
> > - Providing enough data/script to reproduce the failure. What if special
> > HW
> > is needed?
> > - Onboarding new kernel trees to test
> > - Aimed at kernel developers and maintainers.
> > - Which trees are most prone to bring in new problems? Which are the most
> > critical ones? Do you want them to be tested? Which tests do you feel
> > are
> > most beneficial for specific trees or in general?
> > - Security when testing untrusted patches
> > - How do we merge, compile, and test patches that have untrusted code in
> > them
> > and have not yet been reviewed? How do we avoid abuse of systems,
> > information theft, or other damage?
> > - Check out the original patch that sparked the discussion at
> > https://patchwork.ozlabs.org/patch/862123/
> > - Avoiding effort duplication
> > - Food for thought by GregKH
> > - X different CI systems running ${TEST} on latest stable kernel on
> > x86_64
> > might look useless on the first look but is it? AMD/Intel CPUs,
> > different
> > network cards, different graphic drivers, compilers, kernel
> > configuration...
> > How do we distribute the workload to avoid doing the same thing all
> > over
> > again while still running in enough different environments to get the
> > most
> > coverage?
> > - Common hardware pools
> > - Is this something people are interested in? Would be helpful especially
> > for
> > HW that's hard to access, eg. ppc64le or s390x systems. Companies could
> > also
> > sing up to share their HW for testing to ensure kernel works with their
> > products.
>
> I have strong opinions on some of these, but maybe only useful experience
> in a few areas. Fuego has 2 separate notions, which we call "skiplists"
> and "pass criteria", which have to do with this bullet:
>
> - How to autodetect infrastructure failures, regressions/new bugs and test
> bugs? How to handle continuous failures due to known bugs in both
> tests and kernel? What's your solution? Can people always trust the
> results they
> receive?
>
> I'd be happy to discuss this, if it's desired.
>
> Otherwise, I've recently been working on standards for "test definition",
> which defines the data and meta-data associated with a test. I could talk
> about where I'm at with that, if people are interested.
>
Sounds great! I added both your points to the agenda as I do think they have
a place here. The list of items is growing so I hope we can still fit
everything into the two days we planned :)
See you there!
Veronika
> Let me know what you think.
> -- Tim
>
>
1 year, 7 months
[x86/speculation/mds] 8ed0c139ba: reaim.jobs_per_min -12.7% regression
by kernel test robot
Greeting,
FYI, we noticed a -12.7% regression of reaim.jobs_per_min due to commit:
commit: 8ed0c139ba01f0a85af2b95b88d24a6cbf27fba1 ("x86/speculation/mds: Add mitigation control for MDS")
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-5.1.y
in testcase: reaim
on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
with following parameters:
nr_task: 1600%
iterations: 20
test: aim9
ucode: 0x14
cpufreq_governor: performance
test-description: REAIM is an updated and improved version of AIM 7 benchmark.
test-url: https://sourceforge.net/projects/re-aim-7/
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/iterations/kconfig/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/20/x86_64-rhel-7.6/1600%/debian-x86_64-2019-05-14.cgz/lkp-hsx04/aim9/reaim/0x14
commit:
7014e3277e ("x86/speculation/mds: Conditionally clear CPU buffers on idle entry")
8ed0c139ba ("x86/speculation/mds: Add mitigation control for MDS")
7014e3277eb54077 8ed0c139ba01f0a85af2b95b88d
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:4 25% 1:4 dmesg.WARNING:at_ip__slab_free/0x
1:4 -22% 0:4 perf-profile.children.cycles-pp.error_entry
1:4 -17% 0:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
903.67 ± 2% +4.6% 945.53 reaim.child_systime
1485 +13.7% 1689 reaim.child_utime
792819 -12.7% 692160 reaim.jobs_per_min
344.11 -12.7% 300.42 reaim.jobs_per_min_child
88.34 +2.0% 90.12 reaim.jti
830662 -12.3% 728269 reaim.max_jobs_per_min
20.95 +14.5% 23.98 reaim.parent_time
11.12 -16.0% 9.34 reaim.std_dev_percent
1.84 -7.1% 1.71 reaim.std_dev_time
464.42 +13.1% 525.19 reaim.time.elapsed_time
464.42 +13.1% 525.19 reaim.time.elapsed_time.max
11935255 +4.3% 12445748 reaim.time.involuntary_context_switches
10308 -2.5% 10054 reaim.time.percent_of_cpu_this_job_got
18141 ± 2% +4.6% 18980 reaim.time.system_time
29735 +13.7% 33823 reaim.time.user_time
34200 ± 2% -10.4% 30631 ± 4% meminfo.Mapped
9132 ±168% +259.7% 32848 ± 18% numa-numastat.node0.other_node
36314 -64.9% 12740 ±115% numa-numastat.node1.other_node
969.78 -2.9% 942.02 pmeter.Average_Active_Power
817.53 -10.1% 734.76 pmeter.performance_per_watt
15308 -9.3% 13881 slabinfo.skbuff_ext_cache.active_objs
15308 -9.3% 13881 slabinfo.skbuff_ext_cache.num_objs
28.00 +6.2% 29.75 vmstat.cpu.id
171.25 ± 4% -8.8% 156.25 ± 2% vmstat.procs.r
178504 -11.6% 157792 vmstat.system.cs
1.923e+08 ± 5% -10.9% 1.713e+08 ± 2% cpuidle.C1.time
2823412 ± 4% -11.3% 2503490 cpuidle.C1.usage
5.014e+09 ± 78% +113.4% 1.07e+10 ± 40% cpuidle.C3.time
18706494 ± 48% +73.7% 32501580 ± 29% cpuidle.C3.usage
13241 ± 11% -14.2% 11356 numa-meminfo.node0.Mapped
33590 ± 15% -26.1% 24817 ± 4% numa-meminfo.node1.KernelStack
77783 ± 20% -27.5% 56380 ± 4% numa-meminfo.node1.PageTables
136435 ± 10% -18.3% 111490 ± 4% numa-meminfo.node1.SUnreclaim
6648 -9.8% 5996 ± 3% numa-meminfo.node3.Mapped
122025 ± 3% -7.2% 113289 ± 3% numa-meminfo.node3.SUnreclaim
54591 ± 3% +12.0% 61152 ± 4% numa-meminfo.node3.Shmem
2092 -2.3% 2044 turbostat.Avg_MHz
2821435 ± 4% -11.3% 2501497 turbostat.C1
0.29 ± 5% -0.1 0.23 ± 3% turbostat.C1%
18705354 ± 48% +73.7% 32498147 ± 29% turbostat.C3
1.384e+08 +12.6% 1.558e+08 turbostat.IRQ
568.21 -3.2% 550.04 turbostat.PkgWatt
82.88 -3.1% 80.33 turbostat.RAMWatt
3306 ± 10% -14.1% 2841 numa-vmstat.node0.nr_mapped
33509 ± 15% -25.9% 24824 ± 4% numa-vmstat.node1.nr_kernel_stack
19376 ± 20% -27.4% 14072 ± 4% numa-vmstat.node1.nr_page_table_pages
34065 ± 10% -18.2% 27865 ± 4% numa-vmstat.node1.nr_slab_unreclaimable
56.75 ± 59% -91.6% 4.75 ± 80% numa-vmstat.node3.nr_dirtied
1664 -9.8% 1502 ± 3% numa-vmstat.node3.nr_mapped
13804 ± 3% +10.9% 15308 ± 4% numa-vmstat.node3.nr_shmem
30498 ± 3% -7.1% 28323 ± 3% numa-vmstat.node3.nr_slab_unreclaimable
56.50 ± 59% -91.6% 4.75 ± 80% numa-vmstat.node3.nr_written
511653 -3.6% 493339 proc-vmstat.nr_active_anon
423620 -2.7% 412121 proc-vmstat.nr_anon_pages
6437 ± 2% -7.0% 5983 ± 2% proc-vmstat.nr_inactive_anon
124890 -6.8% 116431 proc-vmstat.nr_kernel_stack
8603 ± 3% -11.1% 7652 ± 3% proc-vmstat.nr_mapped
70653 -6.7% 65939 ± 2% proc-vmstat.nr_page_table_pages
93884 ± 3% -7.6% 86708 ± 5% proc-vmstat.nr_shmem
129820 -3.1% 125803 proc-vmstat.nr_slab_unreclaimable
511654 -3.6% 493339 proc-vmstat.nr_zone_active_anon
6437 ± 2% -7.0% 5983 ± 2% proc-vmstat.nr_zone_inactive_anon
40184046 ± 9% +9.1e+06 49290652 ± 2% syscalls.sys_brk.noise.2%
33689320 ± 10% +9.9e+06 43600228 ± 2% syscalls.sys_brk.noise.25%
39565110 ± 10% +9.2e+06 48782038 ± 3% syscalls.sys_brk.noise.5%
24892 ± 58% +305.6% 100960 ± 58% syscalls.sys_close.max
3535790 ± 58% +3.9e+06 7404209 ± 7% syscalls.sys_close.noise.100%
9986532 ± 57% +8.6e+06 18603077 ± 4% syscalls.sys_close.noise.2%
8590639 ± 57% +8.3e+06 16870365 ± 4% syscalls.sys_close.noise.25%
9841124 ± 57% +8.7e+06 18491817 ± 4% syscalls.sys_close.noise.5%
6509750 ± 57% +7.6e+06 14090291 ± 4% syscalls.sys_close.noise.50%
4809001 ± 57% +5.6e+06 10442476 ± 5% syscalls.sys_close.noise.75%
8887151 ± 7% +2.6e+06 11534840 ± 3% syscalls.sys_lseek.noise.2%
6947923 ± 11% +2.7e+06 9625213 ± 3% syscalls.sys_lseek.noise.25%
8685347 ± 7% +2.7e+06 11353787 ± 3% syscalls.sys_lseek.noise.5%
3660039 ± 20% +3.2e+06 6902092 ± 7% syscalls.sys_lseek.noise.50%
3111006 ± 20% +2.2e+06 5342453 ± 3% syscalls.sys_lseek.noise.75%
148013 ± 6% +14.9% 170093 sched_debug.cfs_rq:/.exec_clock.avg
150150 ± 6% +14.7% 172257 sched_debug.cfs_rq:/.exec_clock.max
142601 ± 6% +14.6% 163491 sched_debug.cfs_rq:/.exec_clock.min
755.57 ± 3% +13.9% 860.46 ± 8% sched_debug.cfs_rq:/.exec_clock.stddev
24951497 ± 6% +14.2% 28494838 sched_debug.cfs_rq:/.min_vruntime.avg
25967318 ± 6% +14.4% 29704490 sched_debug.cfs_rq:/.min_vruntime.max
455911 ± 15% +47.2% 670898 ± 13% sched_debug.cfs_rq:/.min_vruntime.stddev
460975 ± 15% +45.7% 671794 ± 12% sched_debug.cfs_rq:/.spread0.stddev
242224 ± 5% +15.8% 280540 sched_debug.cpu.clock.avg
242417 ± 5% +15.8% 280723 sched_debug.cpu.clock.max
242022 ± 5% +15.8% 280331 sched_debug.cpu.clock.min
242224 ± 5% +15.8% 280540 sched_debug.cpu.clock_task.avg
242417 ± 5% +15.8% 280723 sched_debug.cpu.clock_task.max
242022 ± 5% +15.8% 280331 sched_debug.cpu.clock_task.min
39414 ± 3% +35.6% 53428 ± 6% sched_debug.cpu.curr->pid.avg
208789 ± 6% +18.1% 246519 sched_debug.cpu.nr_load_updates.avg
218490 ± 5% +16.5% 254524 sched_debug.cpu.nr_load_updates.max
207163 ± 6% +18.3% 245016 sched_debug.cpu.nr_load_updates.min
242018 ± 5% +15.8% 280327 sched_debug.cpu_clk
237905 ± 5% +16.1% 276214 sched_debug.ktime
0.00 ± 5% -35.6% 0.00 ± 57% sched_debug.rt_rq:/.rt_nr_migratory.avg
0.13 ± 5% -35.6% 0.08 ± 57% sched_debug.rt_rq:/.rt_nr_migratory.max
0.01 ± 5% -35.6% 0.01 ± 57% sched_debug.rt_rq:/.rt_nr_migratory.stddev
0.00 ± 5% -35.6% 0.00 ± 57% sched_debug.rt_rq:/.rt_nr_running.avg
0.13 ± 5% -35.6% 0.08 ± 57% sched_debug.rt_rq:/.rt_nr_running.max
0.01 ± 5% -35.6% 0.01 ± 57% sched_debug.rt_rq:/.rt_nr_running.stddev
246724 ± 5% +15.5% 285035 sched_debug.sched_clk
14.23 ± 4% +17.0% 16.66 ± 5% perf-stat.i.MPKI
2.901e+10 -11.9% 2.556e+10 perf-stat.i.branch-instructions
4.486e+08 -11.1% 3.988e+08 perf-stat.i.branch-misses
5.91 -0.2 5.74 perf-stat.i.cache-miss-rate%
1.31e+08 -10.3% 1.176e+08 perf-stat.i.cache-misses
1.896e+09 -10.8% 1.692e+09 perf-stat.i.cache-references
180130 -11.7% 159033 perf-stat.i.context-switches
2.06 ± 2% +17.7% 2.42 ± 3% perf-stat.i.cpi
3.013e+11 -2.3% 2.943e+11 perf-stat.i.cpu-cycles
55416 -9.8% 50004 perf-stat.i.cpu-migrations
5650 ± 5% +34.0% 7571 ± 6% perf-stat.i.cycles-between-cache-misses
0.38 ± 6% +0.1 0.47 ± 9% perf-stat.i.dTLB-load-miss-rate%
5.744e+10 -11.5% 5.081e+10 perf-stat.i.dTLB-loads
0.06 ± 5% +0.1 0.12 ± 8% perf-stat.i.dTLB-store-miss-rate%
14251959 ± 8% +136.2% 33658179 ± 9% perf-stat.i.dTLB-store-misses
3.692e+10 -9.0% 3.36e+10 perf-stat.i.dTLB-stores
52307356 -7.0% 48639861 ± 3% perf-stat.i.iTLB-load-misses
31102681 -4.4% 29744957 perf-stat.i.iTLB-loads
2.062e+11 -11.7% 1.82e+11 perf-stat.i.instructions
3640 -7.7% 3360 ± 2% perf-stat.i.instructions-per-iTLB-miss
0.63 -11.9% 0.55 perf-stat.i.ipc
1984416 -11.3% 1759513 perf-stat.i.minor-faults
73548093 -11.8% 64884069 perf-stat.i.node-load-misses
24715517 -6.7% 23049460 perf-stat.i.node-loads
50.32 -1.2 49.10 perf-stat.i.node-store-miss-rate%
16538796 -13.1% 14376186 perf-stat.i.node-store-misses
14924381 -5.7% 14066356 perf-stat.i.node-stores
1984421 -11.3% 1759532 perf-stat.i.page-faults
9.20 +1.1% 9.29 perf-stat.overall.MPKI
1.46 +10.7% 1.62 perf-stat.overall.cpi
2299 +8.9% 2504 perf-stat.overall.cycles-between-cache-misses
0.04 ± 9% +0.1 0.10 ± 9% perf-stat.overall.dTLB-store-miss-rate%
3942 -4.9% 3747 ± 3% perf-stat.overall.instructions-per-iTLB-miss
0.68 -9.6% 0.62 perf-stat.overall.ipc
74.84 -1.1 73.79 perf-stat.overall.node-load-miss-rate%
52.56 -2.0 50.54 perf-stat.overall.node-store-miss-rate%
2.888e+10 -11.8% 2.547e+10 perf-stat.ps.branch-instructions
4.467e+08 -11.0% 3.974e+08 perf-stat.ps.branch-misses
1.305e+08 -10.2% 1.171e+08 perf-stat.ps.cache-misses
1.888e+09 -10.7% 1.685e+09 perf-stat.ps.cache-references
179352 -11.7% 158431 perf-stat.ps.context-switches
3e+11 -2.3% 2.933e+11 perf-stat.ps.cpu-cycles
55170 -9.7% 49809 perf-stat.ps.cpu-migrations
5.72e+10 -11.5% 5.063e+10 perf-stat.ps.dTLB-loads
14193221 ± 8% +136.3% 33533439 ± 9% perf-stat.ps.dTLB-store-misses
3.676e+10 -8.9% 3.348e+10 perf-stat.ps.dTLB-stores
52085823 -7.0% 48459938 ± 3% perf-stat.ps.iTLB-load-misses
30971275 -4.3% 29634598 perf-stat.ps.iTLB-loads
2.053e+11 -11.7% 1.813e+11 perf-stat.ps.instructions
1975970 -11.3% 1753003 perf-stat.ps.minor-faults
73227904 -11.7% 64636757 perf-stat.ps.node-load-misses
24610969 -6.7% 22965444 perf-stat.ps.node-loads
16466487 -13.0% 14321079 perf-stat.ps.node-store-misses
14861678 -5.7% 14015331 perf-stat.ps.node-stores
1975970 -11.3% 1753004 perf-stat.ps.page-faults
158617 +11.5% 176913 softirqs.CPU0.TIMER
149136 ± 2% +10.8% 165168 softirqs.CPU1.RCU
50931 +11.2% 56644 softirqs.CPU1.SCHED
156520 +12.8% 176545 softirqs.CPU1.TIMER
151584 +9.8% 166412 ± 2% softirqs.CPU10.RCU
50296 +9.9% 55253 softirqs.CPU10.SCHED
153715 +12.1% 172273 softirqs.CPU10.TIMER
149095 +11.1% 165651 softirqs.CPU100.RCU
148753 +11.8% 166309 softirqs.CPU100.TIMER
151571 +11.7% 169334 softirqs.CPU101.TIMER
50377 +8.8% 54808 softirqs.CPU102.SCHED
152312 +11.2% 169371 softirqs.CPU102.TIMER
151897 +9.3% 166051 ± 2% softirqs.CPU103.RCU
152442 +29.6% 197496 ± 23% softirqs.CPU103.TIMER
148991 +11.2% 165672 softirqs.CPU104.RCU
145127 ± 5% +14.6% 166347 softirqs.CPU104.TIMER
151470 +11.3% 168576 ± 2% softirqs.CPU105.RCU
50131 +10.4% 55369 softirqs.CPU105.SCHED
151829 +11.4% 169208 softirqs.CPU105.TIMER
151462 +10.5% 167390 softirqs.CPU106.RCU
50502 +8.4% 54751 ± 2% softirqs.CPU106.SCHED
151896 +27.8% 194065 ± 21% softirqs.CPU107.TIMER
144410 +10.1% 158984 softirqs.CPU108.RCU
154465 +11.9% 172844 softirqs.CPU108.TIMER
154051 +11.9% 172394 softirqs.CPU109.TIMER
155497 +11.5% 173391 softirqs.CPU11.TIMER
150334 +11.3% 167364 softirqs.CPU110.TIMER
150071 +12.6% 168932 softirqs.CPU111.TIMER
151277 +9.8% 166119 ± 3% softirqs.CPU112.RCU
149417 +9.1% 163081 ± 4% softirqs.CPU112.TIMER
153377 +11.9% 171628 softirqs.CPU113.TIMER
148379 +9.9% 163088 softirqs.CPU114.RCU
152461 +11.8% 170476 softirqs.CPU114.TIMER
151190 +12.3% 169738 softirqs.CPU115.TIMER
150699 +9.5% 164989 ± 2% softirqs.CPU116.RCU
150860 +11.4% 168128 softirqs.CPU116.TIMER
148382 +11.5% 165443 softirqs.CPU117.RCU
149552 +12.1% 167587 softirqs.CPU117.TIMER
149711 +12.1% 167819 softirqs.CPU118.TIMER
151787 +11.7% 169601 softirqs.CPU119.TIMER
157218 +11.3% 175038 softirqs.CPU12.TIMER
151638 +10.6% 167682 ± 2% softirqs.CPU120.RCU
152235 +12.1% 170727 softirqs.CPU120.TIMER
151750 +12.3% 170436 softirqs.CPU121.TIMER
149648 +13.5% 169857 ± 2% softirqs.CPU122.TIMER
152481 +11.3% 169719 ± 2% softirqs.CPU123.RCU
151021 +12.5% 169952 softirqs.CPU123.TIMER
151867 +12.2% 170462 softirqs.CPU124.TIMER
151920 +11.7% 169757 softirqs.CPU125.TIMER
158268 +11.3% 176232 softirqs.CPU126.TIMER
156045 +12.5% 175515 softirqs.CPU127.TIMER
147226 +10.1% 162070 softirqs.CPU128.RCU
152981 +12.6% 172196 softirqs.CPU128.TIMER
147892 +10.6% 163518 softirqs.CPU129.RCU
153576 +12.4% 172660 softirqs.CPU129.TIMER
156926 +11.2% 174510 softirqs.CPU13.TIMER
152566 +11.9% 170689 softirqs.CPU130.TIMER
150938 +10.1% 166198 softirqs.CPU131.RCU
157817 +11.3% 175666 softirqs.CPU131.TIMER
149443 +8.7% 162471 ± 2% softirqs.CPU132.RCU
155709 +11.8% 174090 softirqs.CPU132.TIMER
149948 ± 5% +14.6% 171844 softirqs.CPU133.TIMER
152790 +12.2% 171367 softirqs.CPU134.TIMER
151985 +12.0% 170221 softirqs.CPU135.TIMER
150310 +10.5% 166055 ± 2% softirqs.CPU136.RCU
151786 +12.1% 170110 softirqs.CPU136.TIMER
150771 +9.9% 165708 softirqs.CPU137.RCU
154444 +11.4% 172069 softirqs.CPU137.TIMER
151005 +11.6% 168503 softirqs.CPU138.RCU
158155 ± 2% +9.0% 172427 softirqs.CPU138.TIMER
150613 +12.1% 168767 softirqs.CPU139.RCU
155323 +11.1% 172521 softirqs.CPU139.TIMER
49928 ± 2% +9.7% 54779 ± 2% softirqs.CPU14.SCHED
152544 +11.4% 169981 softirqs.CPU140.TIMER
150299 +11.2% 167138 softirqs.CPU141.RCU
154922 +11.4% 172589 softirqs.CPU141.TIMER
155832 +10.7% 172496 softirqs.CPU142.TIMER
151216 +10.5% 167034 softirqs.CPU143.RCU
154896 +11.3% 172369 softirqs.CPU143.TIMER
155381 +12.2% 174261 softirqs.CPU15.TIMER
155240 +12.1% 173962 softirqs.CPU16.TIMER
50446 +10.1% 55555 ± 2% softirqs.CPU17.SCHED
154787 +12.0% 173339 softirqs.CPU17.TIMER
150694 +11.7% 168322 softirqs.CPU18.RCU
49727 +10.6% 54997 ± 2% softirqs.CPU18.SCHED
156488 ± 3% +12.8% 176517 ± 3% softirqs.CPU18.TIMER
154320 +11.2% 171612 softirqs.CPU19.TIMER
146145 +10.1% 160934 softirqs.CPU2.RCU
49878 +9.8% 54745 ± 2% softirqs.CPU2.SCHED
152704 +11.7% 170553 softirqs.CPU2.TIMER
147262 +11.4% 164094 softirqs.CPU20.RCU
149916 +12.1% 168013 softirqs.CPU20.TIMER
150013 +10.3% 165500 softirqs.CPU21.RCU
148972 +12.0% 166841 softirqs.CPU22.RCU
49747 +10.8% 55144 ± 2% softirqs.CPU22.SCHED
150268 +11.5% 167572 softirqs.CPU22.TIMER
151175 +10.4% 166900 softirqs.CPU23.RCU
154542 +11.6% 172438 softirqs.CPU23.TIMER
149083 +10.3% 164424 softirqs.CPU24.RCU
153232 +27.9% 195916 ± 21% softirqs.CPU24.TIMER
50491 +8.4% 54734 ± 2% softirqs.CPU25.SCHED
151037 +11.7% 168650 softirqs.CPU25.TIMER
150051 +11.5% 167361 softirqs.CPU26.TIMER
151641 +9.8% 166496 softirqs.CPU27.RCU
149974 +11.8% 167724 softirqs.CPU27.TIMER
151097 +10.2% 166571 softirqs.CPU28.RCU
150223 +11.2% 167064 softirqs.CPU28.TIMER
153159 +10.9% 169863 softirqs.CPU29.TIMER
49824 +9.5% 54559 ± 2% softirqs.CPU3.SCHED
153900 +12.6% 173307 softirqs.CPU3.TIMER
152479 +10.7% 168838 softirqs.CPU30.RCU
50332 +9.8% 55249 ± 2% softirqs.CPU30.SCHED
153129 +11.6% 170857 softirqs.CPU30.TIMER
50760 +8.8% 55240 softirqs.CPU33.SCHED
152441 +11.2% 169476 softirqs.CPU33.TIMER
154274 +10.7% 170762 ± 2% softirqs.CPU34.TIMER
152156 +9.8% 167014 softirqs.CPU36.RCU
154677 +11.3% 172219 softirqs.CPU36.TIMER
150641 +10.3% 166153 softirqs.CPU37.RCU
154609 +11.7% 172679 softirqs.CPU37.TIMER
151577 +11.6% 169104 softirqs.CPU38.TIMER
150231 +10.8% 166482 softirqs.CPU39.RCU
151547 +12.2% 170112 softirqs.CPU39.TIMER
152938 +12.1% 171515 softirqs.CPU4.TIMER
150331 +28.2% 192772 ± 22% softirqs.CPU40.TIMER
151613 +10.7% 167772 ± 2% softirqs.CPU41.RCU
154568 +11.7% 172634 softirqs.CPU41.TIMER
153080 +11.6% 170788 softirqs.CPU42.TIMER
150039 +10.6% 165960 softirqs.CPU43.RCU
152755 +11.7% 170573 softirqs.CPU43.TIMER
151408 +11.9% 169361 softirqs.CPU44.TIMER
150510 +11.8% 168221 softirqs.CPU45.TIMER
152010 +9.9% 167061 ± 2% softirqs.CPU46.RCU
150699 +11.9% 168575 softirqs.CPU46.TIMER
150850 +10.0% 165986 ± 2% softirqs.CPU47.RCU
152110 +12.6% 171223 softirqs.CPU47.TIMER
147820 +10.2% 162843 softirqs.CPU48.RCU
153480 +12.0% 171898 softirqs.CPU48.TIMER
152664 +11.7% 170477 softirqs.CPU49.TIMER
50517 +9.9% 55494 softirqs.CPU5.SCHED
157310 +12.1% 176342 softirqs.CPU5.TIMER
150065 +12.4% 168642 softirqs.CPU50.TIMER
152588 +12.4% 171486 softirqs.CPU51.TIMER
153016 +11.5% 170649 softirqs.CPU52.TIMER
152005 +12.4% 170887 softirqs.CPU53.TIMER
158270 +11.5% 176485 softirqs.CPU54.TIMER
148200 +10.0% 162965 softirqs.CPU55.RCU
156974 +11.9% 175652 softirqs.CPU55.TIMER
153111 +12.3% 171923 softirqs.CPU56.TIMER
154191 +12.3% 173193 softirqs.CPU57.TIMER
149187 +10.4% 164754 softirqs.CPU58.RCU
153682 +11.3% 170998 softirqs.CPU58.TIMER
157862 +11.8% 176447 softirqs.CPU59.TIMER
49925 +11.0% 55435 ± 2% softirqs.CPU6.SCHED
155729 +12.4% 174966 softirqs.CPU6.TIMER
156860 ± 2% +11.8% 175313 softirqs.CPU60.TIMER
149225 +9.8% 163867 softirqs.CPU62.RCU
153553 +11.9% 171793 softirqs.CPU62.TIMER
153004 +11.7% 170928 softirqs.CPU63.TIMER
152618 +11.9% 170756 softirqs.CPU64.TIMER
155303 +10.9% 172209 softirqs.CPU65.TIMER
149220 +10.1% 164284 softirqs.CPU66.RCU
156226 +10.9% 173242 softirqs.CPU66.TIMER
155481 +11.5% 173324 softirqs.CPU67.TIMER
147399 +9.3% 161056 softirqs.CPU68.RCU
152592 +12.1% 171029 softirqs.CPU68.TIMER
155202 +11.2% 172535 softirqs.CPU69.TIMER
154332 +12.2% 173107 softirqs.CPU7.TIMER
155970 +11.3% 173587 softirqs.CPU70.TIMER
155261 +11.2% 172701 softirqs.CPU71.TIMER
160692 ± 2% +11.3% 178926 softirqs.CPU72.TIMER
50371 +8.7% 54767 ± 2% softirqs.CPU73.SCHED
154801 +12.7% 174527 softirqs.CPU73.TIMER
49670 +10.8% 55040 softirqs.CPU74.SCHED
150733 +13.8% 171487 softirqs.CPU74.TIMER
49751 +9.7% 54585 ± 2% softirqs.CPU75.SCHED
152451 +13.1% 172450 softirqs.CPU75.TIMER
152163 +12.8% 171582 softirqs.CPU76.TIMER
156187 +12.5% 175659 softirqs.CPU77.TIMER
49977 ± 2% +10.3% 55106 ± 2% softirqs.CPU78.SCHED
155226 +12.3% 174355 softirqs.CPU78.TIMER
153275 +12.5% 172410 softirqs.CPU79.TIMER
152664 +13.1% 172701 softirqs.CPU8.TIMER
146607 +10.3% 161679 softirqs.CPU80.RCU
49939 ± 2% +10.8% 55351 softirqs.CPU80.SCHED
152287 +12.7% 171685 softirqs.CPU80.TIMER
151838 +12.5% 170760 softirqs.CPU81.TIMER
153102 +12.0% 171540 softirqs.CPU82.TIMER
50311 +9.7% 55170 ± 2% softirqs.CPU83.SCHED
153984 +11.9% 172320 softirqs.CPU83.TIMER
154228 +12.0% 172754 softirqs.CPU84.TIMER
154791 +12.4% 173939 softirqs.CPU85.TIMER
49360 +11.2% 54877 ± 2% softirqs.CPU86.SCHED
147129 ± 5% +16.5% 171397 softirqs.CPU86.TIMER
153826 +12.3% 172689 softirqs.CPU87.TIMER
50240 +10.2% 55382 ± 2% softirqs.CPU88.SCHED
154161 +12.0% 172715 softirqs.CPU88.TIMER
149706 +11.0% 166134 softirqs.CPU89.RCU
153252 +12.3% 172171 softirqs.CPU89.TIMER
153485 +12.0% 171829 softirqs.CPU9.TIMER
145151 +10.7% 160642 softirqs.CPU90.RCU
155450 +11.2% 172862 softirqs.CPU90.TIMER
153033 +11.6% 170784 softirqs.CPU91.TIMER
142998 +11.0% 158739 softirqs.CPU92.RCU
49353 +10.7% 54653 softirqs.CPU92.SCHED
149710 +11.9% 167514 softirqs.CPU92.TIMER
50229 +9.3% 54903 softirqs.CPU93.SCHED
146305 ± 5% +14.9% 168086 softirqs.CPU93.TIMER
144940 +11.7% 161877 softirqs.CPU94.RCU
50017 +9.7% 54850 softirqs.CPU94.SCHED
149446 +12.1% 167502 softirqs.CPU94.TIMER
50831 +7.9% 54849 ± 2% softirqs.CPU95.SCHED
155012 +11.1% 172171 softirqs.CPU95.TIMER
150147 +11.9% 167969 softirqs.CPU97.TIMER
148169 +10.4% 163536 softirqs.CPU98.RCU
49894 +9.4% 54574 ± 2% softirqs.CPU98.SCHED
149108 +12.3% 167439 softirqs.CPU98.TIMER
149250 +11.6% 166538 softirqs.CPU99.TIMER
22138760 +11.8% 24760659 softirqs.TIMER
248.25 ± 11% +48.5% 368.75 ± 41% interrupts.111:PCI-MSI.1574913-edge.eth1-TxRx-1
237.50 ± 4% +86.1% 442.00 ± 37% interrupts.118:PCI-MSI.1574920-edge.eth1-TxRx-8
283.75 ± 27% +126.5% 642.75 ± 47% interrupts.121:PCI-MSI.1574923-edge.eth1-TxRx-11
250.25 ± 5% +33.8% 334.75 ± 23% interrupts.124:PCI-MSI.1574926-edge.eth1-TxRx-14
233.50 ± 2% +11.3% 260.00 interrupts.126:PCI-MSI.1574928-edge.eth1-TxRx-16
230.50 +12.4% 259.00 interrupts.128:PCI-MSI.1574930-edge.eth1-TxRx-18
231.00 ± 2% +12.9% 260.75 interrupts.129:PCI-MSI.1574931-edge.eth1-TxRx-19
230.00 +13.0% 260.00 interrupts.130:PCI-MSI.1574932-edge.eth1-TxRx-20
232.75 +10.7% 257.75 interrupts.132:PCI-MSI.1574934-edge.eth1-TxRx-22
232.25 ± 2% +12.7% 261.75 interrupts.133:PCI-MSI.1574935-edge.eth1-TxRx-23
229.75 +13.4% 260.50 interrupts.134:PCI-MSI.1574936-edge.eth1-TxRx-24
230.25 +13.4% 261.00 interrupts.135:PCI-MSI.1574937-edge.eth1-TxRx-25
231.50 ± 3% +61.1% 373.00 ± 52% interrupts.136:PCI-MSI.1574938-edge.eth1-TxRx-26
233.25 +11.0% 259.00 interrupts.137:PCI-MSI.1574939-edge.eth1-TxRx-27
231.25 +16.2% 268.75 ± 5% interrupts.140:PCI-MSI.1574942-edge.eth1-TxRx-30
229.25 +16.0% 266.00 ± 3% interrupts.141:PCI-MSI.1574943-edge.eth1-TxRx-31
231.50 +12.6% 260.75 ± 2% interrupts.142:PCI-MSI.1574944-edge.eth1-TxRx-32
229.25 +13.8% 261.00 interrupts.143:PCI-MSI.1574945-edge.eth1-TxRx-33
232.25 +11.9% 260.00 interrupts.144:PCI-MSI.1574946-edge.eth1-TxRx-34
229.50 +12.9% 259.00 interrupts.145:PCI-MSI.1574947-edge.eth1-TxRx-35
228.75 +13.9% 260.50 interrupts.146:PCI-MSI.1574948-edge.eth1-TxRx-36
229.00 +13.3% 259.50 interrupts.147:PCI-MSI.1574949-edge.eth1-TxRx-37
230.25 ± 2% +12.6% 259.25 interrupts.148:PCI-MSI.1574950-edge.eth1-TxRx-38
233.00 +11.2% 259.00 interrupts.149:PCI-MSI.1574951-edge.eth1-TxRx-39
229.00 +14.5% 262.25 interrupts.150:PCI-MSI.1574952-edge.eth1-TxRx-40
231.50 +12.4% 260.25 interrupts.151:PCI-MSI.1574953-edge.eth1-TxRx-41
230.50 +84.7% 425.75 ± 55% interrupts.152:PCI-MSI.1574954-edge.eth1-TxRx-42
229.00 +13.3% 259.50 interrupts.153:PCI-MSI.1574955-edge.eth1-TxRx-43
230.50 +11.9% 258.00 interrupts.154:PCI-MSI.1574956-edge.eth1-TxRx-44
228.25 +45.0% 331.00 ± 36% interrupts.155:PCI-MSI.1574957-edge.eth1-TxRx-45
229.50 +13.5% 260.50 interrupts.156:PCI-MSI.1574958-edge.eth1-TxRx-46
234.00 +11.2% 260.25 interrupts.157:PCI-MSI.1574959-edge.eth1-TxRx-47
229.50 +13.9% 261.50 interrupts.158:PCI-MSI.1574960-edge.eth1-TxRx-48
231.75 +26.3% 292.75 ± 17% interrupts.159:PCI-MSI.1574961-edge.eth1-TxRx-49
228.75 +14.9% 262.75 interrupts.160:PCI-MSI.1574962-edge.eth1-TxRx-50
232.50 +12.4% 261.25 interrupts.161:PCI-MSI.1574963-edge.eth1-TxRx-51
229.50 +12.6% 258.50 interrupts.162:PCI-MSI.1574964-edge.eth1-TxRx-52
228.50 +14.2% 261.00 interrupts.163:PCI-MSI.1574965-edge.eth1-TxRx-53
229.75 +13.4% 260.50 interrupts.166:PCI-MSI.1574968-edge.eth1-TxRx-56
232.25 +11.9% 260.00 interrupts.167:PCI-MSI.1574969-edge.eth1-TxRx-57
229.75 +65.3% 379.75 ± 54% interrupts.169:PCI-MSI.1574971-edge.eth1-TxRx-59
231.25 +12.1% 259.25 interrupts.170:PCI-MSI.1574972-edge.eth1-TxRx-60
229.00 +14.1% 261.25 interrupts.171:PCI-MSI.1574973-edge.eth1-TxRx-61
233.00 ± 2% +12.1% 261.25 interrupts.172:PCI-MSI.1574974-edge.eth1-TxRx-62
831701 +5.6% 878531 interrupts.CAL:Function_call_interrupts
5687 ± 2% +8.2% 6152 ± 4% interrupts.CPU0.CAL:Function_call_interrupts
933510 +13.0% 1054517 interrupts.CPU0.LOC:Local_timer_interrupts
248.25 ± 11% +48.5% 368.75 ± 41% interrupts.CPU1.111:PCI-MSI.1574913-edge.eth1-TxRx-1
932762 +13.1% 1054602 interrupts.CPU1.LOC:Local_timer_interrupts
5605 ± 2% +10.1% 6172 ± 4% interrupts.CPU10.CAL:Function_call_interrupts
933149 +13.0% 1054160 interrupts.CPU10.LOC:Local_timer_interrupts
14743 +12.4% 16566 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts
932749 +13.0% 1054363 interrupts.CPU100.LOC:Local_timer_interrupts
14791 ± 3% +13.3% 16762 ± 3% interrupts.CPU100.RES:Rescheduling_interrupts
932624 +13.1% 1054603 interrupts.CPU101.LOC:Local_timer_interrupts
4355 ± 66% -89.3% 468.00 ±173% interrupts.CPU101.NMI:Non-maskable_interrupts
4355 ± 66% -89.3% 468.00 ±173% interrupts.CPU101.PMI:Performance_monitoring_interrupts
14543 ± 4% +14.2% 16606 ± 3% interrupts.CPU101.RES:Rescheduling_interrupts
933432 +13.0% 1054384 interrupts.CPU102.LOC:Local_timer_interrupts
14424 ± 3% +13.9% 16425 ± 4% interrupts.CPU102.RES:Rescheduling_interrupts
932994 +13.0% 1054598 interrupts.CPU103.LOC:Local_timer_interrupts
4553 ± 61% -93.2% 309.75 ±149% interrupts.CPU103.NMI:Non-maskable_interrupts
4553 ± 61% -93.2% 309.75 ±149% interrupts.CPU103.PMI:Performance_monitoring_interrupts
14751 ± 2% +11.6% 16455 ± 3% interrupts.CPU103.RES:Rescheduling_interrupts
933041 +13.0% 1054372 interrupts.CPU104.LOC:Local_timer_interrupts
15056 ± 2% +10.3% 16613 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts
932769 +13.1% 1054681 interrupts.CPU105.LOC:Local_timer_interrupts
14238 ± 3% +15.5% 16439 interrupts.CPU105.RES:Rescheduling_interrupts
932683 +13.1% 1054548 interrupts.CPU106.LOC:Local_timer_interrupts
14425 ± 4% +13.4% 16362 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts
932723 +13.0% 1054209 interrupts.CPU107.LOC:Local_timer_interrupts
14903 ± 3% +12.0% 16688 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts
933017 +12.9% 1053675 interrupts.CPU108.LOC:Local_timer_interrupts
4207 ± 71% -94.9% 213.00 ±150% interrupts.CPU108.NMI:Non-maskable_interrupts
4207 ± 71% -94.9% 213.00 ±150% interrupts.CPU108.PMI:Performance_monitoring_interrupts
14975 ± 5% +10.6% 16560 ± 2% interrupts.CPU108.RES:Rescheduling_interrupts
932757 +13.0% 1054271 interrupts.CPU109.LOC:Local_timer_interrupts
14945 ± 2% +13.9% 17015 ± 5% interrupts.CPU109.RES:Rescheduling_interrupts
1085 ± 5% -18.2% 888.50 ± 10% interrupts.CPU109.TLB:TLB_shootdowns
283.75 ± 27% +126.5% 642.75 ± 47% interrupts.CPU11.121:PCI-MSI.1574923-edge.eth1-TxRx-11
933364 +13.0% 1054235 interrupts.CPU11.LOC:Local_timer_interrupts
932761 +13.0% 1054035 interrupts.CPU110.LOC:Local_timer_interrupts
15113 ± 3% +12.0% 16932 ± 2% interrupts.CPU110.RES:Rescheduling_interrupts
1055 ± 11% -19.2% 852.50 ± 7% interrupts.CPU110.TLB:TLB_shootdowns
5801 ± 4% +9.8% 6367 interrupts.CPU111.CAL:Function_call_interrupts
932448 +13.0% 1053989 interrupts.CPU111.LOC:Local_timer_interrupts
1093 ±150% +333.7% 4743 ± 18% interrupts.CPU111.NMI:Non-maskable_interrupts
1093 ±150% +333.7% 4743 ± 18% interrupts.CPU111.PMI:Performance_monitoring_interrupts
14746 ± 4% +16.3% 17146 ± 5% interrupts.CPU111.RES:Rescheduling_interrupts
933296 +12.9% 1054110 interrupts.CPU112.LOC:Local_timer_interrupts
933293 +13.0% 1054268 interrupts.CPU113.LOC:Local_timer_interrupts
5363 ± 31% -34.6% 3506 ± 30% interrupts.CPU113.NMI:Non-maskable_interrupts
5363 ± 31% -34.6% 3506 ± 30% interrupts.CPU113.PMI:Performance_monitoring_interrupts
15420 +9.7% 16912 ± 3% interrupts.CPU113.RES:Rescheduling_interrupts
933069 +13.0% 1054371 interrupts.CPU114.LOC:Local_timer_interrupts
15154 +11.4% 16874 ± 2% interrupts.CPU114.RES:Rescheduling_interrupts
933299 +13.0% 1054454 interrupts.CPU115.LOC:Local_timer_interrupts
15352 +8.8% 16702 ± 4% interrupts.CPU115.RES:Rescheduling_interrupts
5721 ± 3% +9.1% 6242 ± 2% interrupts.CPU116.CAL:Function_call_interrupts
933277 +13.0% 1054288 interrupts.CPU116.LOC:Local_timer_interrupts
994.75 ±168% +274.2% 3722 ± 57% interrupts.CPU116.NMI:Non-maskable_interrupts
994.75 ±168% +274.2% 3722 ± 57% interrupts.CPU116.PMI:Performance_monitoring_interrupts
15040 ± 2% +11.5% 16766 ± 5% interrupts.CPU116.RES:Rescheduling_interrupts
933308 +12.9% 1054068 interrupts.CPU117.LOC:Local_timer_interrupts
14959 +11.9% 16745 ± 2% interrupts.CPU117.RES:Rescheduling_interrupts
933306 +13.0% 1054329 interrupts.CPU118.LOC:Local_timer_interrupts
14979 ± 3% +11.3% 16676 ± 3% interrupts.CPU118.RES:Rescheduling_interrupts
933276 +12.9% 1054023 interrupts.CPU119.LOC:Local_timer_interrupts
14712 +14.5% 16852 ± 4% interrupts.CPU119.RES:Rescheduling_interrupts
933040 +13.0% 1054232 interrupts.CPU12.LOC:Local_timer_interrupts
14718 ± 2% +11.1% 16349 ± 3% interrupts.CPU12.RES:Rescheduling_interrupts
933018 +13.0% 1054009 interrupts.CPU120.LOC:Local_timer_interrupts
15006 +12.6% 16897 ± 3% interrupts.CPU120.RES:Rescheduling_interrupts
933265 +12.9% 1053822 interrupts.CPU121.LOC:Local_timer_interrupts
6339 ± 26% -60.2% 2521 ± 86% interrupts.CPU121.NMI:Non-maskable_interrupts
6339 ± 26% -60.2% 2521 ± 86% interrupts.CPU121.PMI:Performance_monitoring_interrupts
14977 ± 2% +12.9% 16905 ± 2% interrupts.CPU121.RES:Rescheduling_interrupts
933305 +13.0% 1054317 interrupts.CPU122.LOC:Local_timer_interrupts
14869 ± 4% +13.9% 16931 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts
933300 +13.0% 1054331 interrupts.CPU123.LOC:Local_timer_interrupts
933342 +13.0% 1054404 interrupts.CPU124.LOC:Local_timer_interrupts
5669 ± 4% +10.4% 6261 interrupts.CPU125.CAL:Function_call_interrupts
933327 +13.0% 1054597 interrupts.CPU125.LOC:Local_timer_interrupts
14647 +14.3% 16735 ± 2% interrupts.CPU125.RES:Rescheduling_interrupts
1026 ± 11% -15.5% 866.75 ± 11% interrupts.CPU125.TLB:TLB_shootdowns
933078 +13.0% 1054153 interrupts.CPU126.LOC:Local_timer_interrupts
14773 ± 4% +15.9% 17123 ± 3% interrupts.CPU126.RES:Rescheduling_interrupts
933301 +12.9% 1053895 interrupts.CPU127.LOC:Local_timer_interrupts
5283 ± 27% -53.2% 2474 ±102% interrupts.CPU127.NMI:Non-maskable_interrupts
5283 ± 27% -53.2% 2474 ±102% interrupts.CPU127.PMI:Performance_monitoring_interrupts
933466 +13.0% 1054389 interrupts.CPU128.LOC:Local_timer_interrupts
14854 ± 3% +14.3% 16981 ± 4% interrupts.CPU128.RES:Rescheduling_interrupts
933496 +13.0% 1054402 interrupts.CPU129.LOC:Local_timer_interrupts
14776 ± 3% +12.5% 16629 ± 3% interrupts.CPU129.RES:Rescheduling_interrupts
5666 ± 4% +7.8% 6110 ± 4% interrupts.CPU13.CAL:Function_call_interrupts
932876 +13.0% 1054428 interrupts.CPU13.LOC:Local_timer_interrupts
14693 +11.9% 16438 interrupts.CPU13.RES:Rescheduling_interrupts
933379 +12.9% 1053895 interrupts.CPU130.LOC:Local_timer_interrupts
14859 ± 3% +13.1% 16810 ± 3% interrupts.CPU130.RES:Rescheduling_interrupts
1081 ± 9% -23.1% 832.00 ± 9% interrupts.CPU130.TLB:TLB_shootdowns
933216 +13.0% 1054297 interrupts.CPU131.LOC:Local_timer_interrupts
933195 +13.0% 1054157 interrupts.CPU132.LOC:Local_timer_interrupts
15104 +11.7% 16869 ± 2% interrupts.CPU132.RES:Rescheduling_interrupts
933501 +12.9% 1053599 interrupts.CPU133.LOC:Local_timer_interrupts
5464 ± 37% -79.1% 1141 ±165% interrupts.CPU133.NMI:Non-maskable_interrupts
5464 ± 37% -79.1% 1141 ±165% interrupts.CPU133.PMI:Performance_monitoring_interrupts
14969 ± 3% +14.1% 17075 ± 2% interrupts.CPU133.RES:Rescheduling_interrupts
1049 ± 12% -19.5% 845.25 ± 12% interrupts.CPU133.TLB:TLB_shootdowns
933523 +12.9% 1054160 interrupts.CPU134.LOC:Local_timer_interrupts
14764 +13.3% 16732 interrupts.CPU134.RES:Rescheduling_interrupts
933384 +12.9% 1053922 interrupts.CPU135.LOC:Local_timer_interrupts
14892 ± 3% +12.6% 16764 interrupts.CPU135.RES:Rescheduling_interrupts
1122 ± 8% -24.7% 845.25 ± 7% interrupts.CPU135.TLB:TLB_shootdowns
933167 +12.9% 1053907 interrupts.CPU136.LOC:Local_timer_interrupts
1074 ± 12% -19.2% 868.50 ± 10% interrupts.CPU136.TLB:TLB_shootdowns
933230 +12.9% 1053922 interrupts.CPU137.LOC:Local_timer_interrupts
1048 ± 8% -21.5% 822.75 ± 8% interrupts.CPU137.TLB:TLB_shootdowns
933346 +13.0% 1054394 interrupts.CPU138.LOC:Local_timer_interrupts
14331 ± 2% +14.8% 16448 ± 3% interrupts.CPU138.RES:Rescheduling_interrupts
933500 +12.9% 1054257 interrupts.CPU139.LOC:Local_timer_interrupts
14452 ± 4% +18.0% 17050 interrupts.CPU139.RES:Rescheduling_interrupts
1053 ± 6% -15.1% 893.75 ± 7% interrupts.CPU139.TLB:TLB_shootdowns
250.25 ± 5% +33.8% 334.75 ± 23% interrupts.CPU14.124:PCI-MSI.1574926-edge.eth1-TxRx-14
5706 ± 4% +9.6% 6251 ± 3% interrupts.CPU14.CAL:Function_call_interrupts
933299 +13.0% 1054257 interrupts.CPU14.LOC:Local_timer_interrupts
933501 +13.0% 1054427 interrupts.CPU140.LOC:Local_timer_interrupts
1199 ±134% +301.3% 4812 ± 18% interrupts.CPU140.NMI:Non-maskable_interrupts
1199 ±134% +301.3% 4812 ± 18% interrupts.CPU140.PMI:Performance_monitoring_interrupts
14885 +13.7% 16919 ± 2% interrupts.CPU140.RES:Rescheduling_interrupts
1085 ± 11% -15.8% 914.00 ± 14% interrupts.CPU140.TLB:TLB_shootdowns
933483 +12.9% 1053975 interrupts.CPU141.LOC:Local_timer_interrupts
14614 ± 3% +13.5% 16590 ± 2% interrupts.CPU141.RES:Rescheduling_interrupts
1022 ± 5% -11.9% 900.00 ± 7% interrupts.CPU141.TLB:TLB_shootdowns
933523 +12.9% 1053718 interrupts.CPU142.LOC:Local_timer_interrupts
14587 ± 2% +15.8% 16897 ± 3% interrupts.CPU142.RES:Rescheduling_interrupts
933563 +13.0% 1054466 interrupts.CPU143.LOC:Local_timer_interrupts
4802 ± 66% -72.7% 1310 ±171% interrupts.CPU143.NMI:Non-maskable_interrupts
4802 ± 66% -72.7% 1310 ±171% interrupts.CPU143.PMI:Performance_monitoring_interrupts
1099 ± 6% -20.4% 875.75 ± 7% interrupts.CPU143.TLB:TLB_shootdowns
932630 +13.1% 1054525 interrupts.CPU15.LOC:Local_timer_interrupts
233.50 ± 2% +11.3% 260.00 interrupts.CPU16.126:PCI-MSI.1574928-edge.eth1-TxRx-16
933042 +13.0% 1054291 interrupts.CPU16.LOC:Local_timer_interrupts
932989 +13.0% 1054305 interrupts.CPU17.LOC:Local_timer_interrupts
230.50 +12.4% 259.00 interrupts.CPU18.128:PCI-MSI.1574930-edge.eth1-TxRx-18
5712 ± 5% +9.4% 6249 ± 4% interrupts.CPU18.CAL:Function_call_interrupts
933388 +13.0% 1054443 interrupts.CPU18.LOC:Local_timer_interrupts
15010 ± 3% +16.8% 17530 ± 4% interrupts.CPU18.RES:Rescheduling_interrupts
231.00 ± 2% +12.9% 260.75 interrupts.CPU19.129:PCI-MSI.1574931-edge.eth1-TxRx-19
933340 +13.0% 1054461 interrupts.CPU19.LOC:Local_timer_interrupts
14575 ± 4% +15.2% 16789 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts
5516 ± 2% +14.3% 6307 ± 4% interrupts.CPU2.CAL:Function_call_interrupts
932491 +13.1% 1054258 interrupts.CPU2.LOC:Local_timer_interrupts
20.25 ±161% +20727.2% 4217 ± 29% interrupts.CPU2.NMI:Non-maskable_interrupts
20.25 ±161% +20727.2% 4217 ± 29% interrupts.CPU2.PMI:Performance_monitoring_interrupts
14902 ± 2% +11.0% 16539 ± 3% interrupts.CPU2.RES:Rescheduling_interrupts
230.00 +13.0% 260.00 interrupts.CPU20.130:PCI-MSI.1574932-edge.eth1-TxRx-20
933558 +13.0% 1054660 interrupts.CPU20.LOC:Local_timer_interrupts
15052 ± 2% +12.7% 16961 ± 5% interrupts.CPU20.RES:Rescheduling_interrupts
933674 +12.9% 1054527 interrupts.CPU21.LOC:Local_timer_interrupts
14970 +12.6% 16849 ± 3% interrupts.CPU21.RES:Rescheduling_interrupts
232.75 +10.7% 257.75 interrupts.CPU22.132:PCI-MSI.1574934-edge.eth1-TxRx-22
933573 +13.0% 1054550 interrupts.CPU22.LOC:Local_timer_interrupts
14657 ± 3% +20.4% 17652 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts
1093 ± 9% -17.1% 906.00 ± 7% interrupts.CPU22.TLB:TLB_shootdowns
232.25 ± 2% +12.7% 261.75 interrupts.CPU23.133:PCI-MSI.1574935-edge.eth1-TxRx-23
933094 +13.0% 1053936 interrupts.CPU23.LOC:Local_timer_interrupts
15122 ± 2% +11.3% 16831 ± 6% interrupts.CPU23.RES:Rescheduling_interrupts
229.75 +13.4% 260.50 interrupts.CPU24.134:PCI-MSI.1574936-edge.eth1-TxRx-24
933273 +13.0% 1054828 interrupts.CPU24.LOC:Local_timer_interrupts
14742 +15.8% 17072 ± 5% interrupts.CPU24.RES:Rescheduling_interrupts
230.25 +13.4% 261.00 interrupts.CPU25.135:PCI-MSI.1574937-edge.eth1-TxRx-25
933445 +13.0% 1054725 interrupts.CPU25.LOC:Local_timer_interrupts
14917 +15.2% 17180 ± 2% interrupts.CPU25.RES:Rescheduling_interrupts
231.50 ± 3% +61.1% 373.00 ± 52% interrupts.CPU26.136:PCI-MSI.1574938-edge.eth1-TxRx-26
5726 +9.7% 6282 ± 4% interrupts.CPU26.CAL:Function_call_interrupts
932972 +13.0% 1054132 interrupts.CPU26.LOC:Local_timer_interrupts
14597 ± 2% +16.1% 16952 ± 5% interrupts.CPU26.RES:Rescheduling_interrupts
233.25 +11.0% 259.00 interrupts.CPU27.137:PCI-MSI.1574939-edge.eth1-TxRx-27
932905 +13.0% 1054147 interrupts.CPU27.LOC:Local_timer_interrupts
14658 ± 2% +15.4% 16922 ± 3% interrupts.CPU27.RES:Rescheduling_interrupts
932938 +13.0% 1054344 interrupts.CPU28.LOC:Local_timer_interrupts
14506 ± 4% +15.6% 16776 ± 4% interrupts.CPU28.RES:Rescheduling_interrupts
5655 ± 3% +10.4% 6244 ± 4% interrupts.CPU29.CAL:Function_call_interrupts
933193 +13.0% 1054488 interrupts.CPU29.LOC:Local_timer_interrupts
14421 ± 2% +18.1% 17032 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts
932354 +13.1% 1054241 interrupts.CPU3.LOC:Local_timer_interrupts
231.25 +16.2% 268.75 ± 5% interrupts.CPU30.140:PCI-MSI.1574942-edge.eth1-TxRx-30
933392 +13.0% 1054655 interrupts.CPU30.LOC:Local_timer_interrupts
14299 ± 2% +17.3% 16775 ± 2% interrupts.CPU30.RES:Rescheduling_interrupts
229.25 +16.0% 266.00 ± 3% interrupts.CPU31.141:PCI-MSI.1574943-edge.eth1-TxRx-31
5645 ± 4% +8.8% 6142 ± 2% interrupts.CPU31.CAL:Function_call_interrupts
933540 +12.9% 1053989 interrupts.CPU31.LOC:Local_timer_interrupts
14593 +17.0% 17071 ± 3% interrupts.CPU31.RES:Rescheduling_interrupts
231.50 +12.6% 260.75 ± 2% interrupts.CPU32.142:PCI-MSI.1574944-edge.eth1-TxRx-32
933162 +13.0% 1054157 interrupts.CPU32.LOC:Local_timer_interrupts
14742 +12.3% 16550 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts
229.25 +13.8% 261.00 interrupts.CPU33.143:PCI-MSI.1574945-edge.eth1-TxRx-33
933019 +13.0% 1054366 interrupts.CPU33.LOC:Local_timer_interrupts
15074 ± 2% +11.0% 16740 ± 5% interrupts.CPU33.RES:Rescheduling_interrupts
232.25 +11.9% 260.00 interrupts.CPU34.144:PCI-MSI.1574946-edge.eth1-TxRx-34
933204 +13.0% 1054167 interrupts.CPU34.LOC:Local_timer_interrupts
14861 ± 2% +13.9% 16929 ± 2% interrupts.CPU34.RES:Rescheduling_interrupts
229.50 +12.9% 259.00 interrupts.CPU35.145:PCI-MSI.1574947-edge.eth1-TxRx-35
932935 +13.0% 1054326 interrupts.CPU35.LOC:Local_timer_interrupts
14487 ± 2% +16.0% 16799 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts
228.75 +13.9% 260.50 interrupts.CPU36.146:PCI-MSI.1574948-edge.eth1-TxRx-36
932984 +13.0% 1054434 interrupts.CPU36.LOC:Local_timer_interrupts
15596 ± 2% +12.0% 17460 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts
229.00 +13.3% 259.50 interrupts.CPU37.147:PCI-MSI.1574949-edge.eth1-TxRx-37
932625 +13.0% 1054016 interrupts.CPU37.LOC:Local_timer_interrupts
230.25 ± 2% +12.6% 259.25 interrupts.CPU38.148:PCI-MSI.1574950-edge.eth1-TxRx-38
933326 +13.0% 1054212 interrupts.CPU38.LOC:Local_timer_interrupts
15403 ± 2% +10.7% 17049 ± 4% interrupts.CPU38.RES:Rescheduling_interrupts
233.00 +11.2% 259.00 interrupts.CPU39.149:PCI-MSI.1574951-edge.eth1-TxRx-39
933166 +13.0% 1054312 interrupts.CPU39.LOC:Local_timer_interrupts
4360 ± 64% -71.6% 1236 ±173% interrupts.CPU39.NMI:Non-maskable_interrupts
4360 ± 64% -71.6% 1236 ±173% interrupts.CPU39.PMI:Performance_monitoring_interrupts
15089 +12.1% 16917 ± 3% interrupts.CPU39.RES:Rescheduling_interrupts
1049 ± 8% -16.8% 873.00 ± 11% interrupts.CPU39.TLB:TLB_shootdowns
932615 +13.0% 1054201 interrupts.CPU4.LOC:Local_timer_interrupts
15061 ± 2% +10.6% 16652 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts
229.00 +14.5% 262.25 interrupts.CPU40.150:PCI-MSI.1574952-edge.eth1-TxRx-40
5544 +8.9% 6037 ± 2% interrupts.CPU40.CAL:Function_call_interrupts
933148 +12.9% 1053915 interrupts.CPU40.LOC:Local_timer_interrupts
14933 ± 2% +13.5% 16954 ± 5% interrupts.CPU40.RES:Rescheduling_interrupts
1047 ± 12% -14.8% 892.00 ± 13% interrupts.CPU40.TLB:TLB_shootdowns
231.50 +12.4% 260.25 interrupts.CPU41.151:PCI-MSI.1574953-edge.eth1-TxRx-41
932899 +13.0% 1054472 interrupts.CPU41.LOC:Local_timer_interrupts
14844 ± 3% +16.4% 17277 ± 4% interrupts.CPU41.RES:Rescheduling_interrupts
1053 ± 11% -12.8% 918.50 ± 14% interrupts.CPU41.TLB:TLB_shootdowns
230.50 +84.7% 425.75 ± 55% interrupts.CPU42.152:PCI-MSI.1574954-edge.eth1-TxRx-42
5589 ± 2% +10.0% 6148 ± 5% interrupts.CPU42.CAL:Function_call_interrupts
932826 +13.0% 1054343 interrupts.CPU42.LOC:Local_timer_interrupts
15146 ± 2% +13.4% 17168 ± 2% interrupts.CPU42.RES:Rescheduling_interrupts
229.00 +13.3% 259.50 interrupts.CPU43.153:PCI-MSI.1574955-edge.eth1-TxRx-43
933185 +13.0% 1054835 interrupts.CPU43.LOC:Local_timer_interrupts
15126 ± 3% +11.1% 16810 ± 6% interrupts.CPU43.RES:Rescheduling_interrupts
230.50 +11.9% 258.00 interrupts.CPU44.154:PCI-MSI.1574956-edge.eth1-TxRx-44
933020 +13.0% 1054676 interrupts.CPU44.LOC:Local_timer_interrupts
228.25 +45.0% 331.00 ± 36% interrupts.CPU45.155:PCI-MSI.1574957-edge.eth1-TxRx-45
5576 ± 3% +8.7% 6059 ± 4% interrupts.CPU45.CAL:Function_call_interrupts
933248 +12.9% 1053999 interrupts.CPU45.LOC:Local_timer_interrupts
14935 ± 3% +11.8% 16693 ± 5% interrupts.CPU45.RES:Rescheduling_interrupts
229.50 +13.5% 260.50 interrupts.CPU46.156:PCI-MSI.1574958-edge.eth1-TxRx-46
933239 +13.0% 1054323 interrupts.CPU46.LOC:Local_timer_interrupts
15119 ± 2% +11.7% 16881 ± 2% interrupts.CPU46.RES:Rescheduling_interrupts
234.00 +11.2% 260.25 interrupts.CPU47.157:PCI-MSI.1574959-edge.eth1-TxRx-47
933007 +13.0% 1054248 interrupts.CPU47.LOC:Local_timer_interrupts
229.50 +13.9% 261.50 interrupts.CPU48.158:PCI-MSI.1574960-edge.eth1-TxRx-48
933017 +12.9% 1053780 interrupts.CPU48.LOC:Local_timer_interrupts
14825 +14.2% 16927 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts
1122 ± 11% -23.7% 856.00 ± 11% interrupts.CPU48.TLB:TLB_shootdowns
231.75 +26.3% 292.75 ± 17% interrupts.CPU49.159:PCI-MSI.1574961-edge.eth1-TxRx-49
932494 +13.0% 1054135 interrupts.CPU49.LOC:Local_timer_interrupts
15003 ± 3% +14.8% 17223 ± 5% interrupts.CPU49.RES:Rescheduling_interrupts
932950 +13.0% 1054649 interrupts.CPU5.LOC:Local_timer_interrupts
14907 ± 2% +11.2% 16580 interrupts.CPU5.RES:Rescheduling_interrupts
228.75 +14.9% 262.75 interrupts.CPU50.160:PCI-MSI.1574962-edge.eth1-TxRx-50
932793 +13.0% 1054474 interrupts.CPU50.LOC:Local_timer_interrupts
15003 +14.1% 17118 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts
1049 ± 12% -18.1% 859.25 ± 13% interrupts.CPU50.TLB:TLB_shootdowns
232.50 +12.4% 261.25 interrupts.CPU51.161:PCI-MSI.1574963-edge.eth1-TxRx-51
933219 +13.0% 1054268 interrupts.CPU51.LOC:Local_timer_interrupts
14775 ± 2% +14.2% 16868 ± 5% interrupts.CPU51.RES:Rescheduling_interrupts
229.50 +12.6% 258.50 interrupts.CPU52.162:PCI-MSI.1574964-edge.eth1-TxRx-52
5625 ± 3% +7.8% 6061 ± 5% interrupts.CPU52.CAL:Function_call_interrupts
933072 +12.9% 1053784 interrupts.CPU52.LOC:Local_timer_interrupts
15068 ± 2% +12.1% 16884 ± 6% interrupts.CPU52.RES:Rescheduling_interrupts
228.50 +14.2% 261.00 interrupts.CPU53.163:PCI-MSI.1574965-edge.eth1-TxRx-53
932725 +13.0% 1054236 interrupts.CPU53.LOC:Local_timer_interrupts
15191 ± 2% +11.1% 16876 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts
933397 +13.0% 1054488 interrupts.CPU54.LOC:Local_timer_interrupts
14903 ± 3% +16.3% 17336 interrupts.CPU54.RES:Rescheduling_interrupts
5581 +10.1% 6146 ± 5% interrupts.CPU55.CAL:Function_call_interrupts
933220 +12.9% 1053932 interrupts.CPU55.LOC:Local_timer_interrupts
14944 +15.1% 17194 ± 3% interrupts.CPU55.RES:Rescheduling_interrupts
229.75 +13.4% 260.50 interrupts.CPU56.166:PCI-MSI.1574968-edge.eth1-TxRx-56
5694 ± 3% +7.2% 6102 ± 4% interrupts.CPU56.CAL:Function_call_interrupts
933451 +12.9% 1054267 interrupts.CPU56.LOC:Local_timer_interrupts
14798 ± 3% +16.6% 17253 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts
1053 ± 7% -14.2% 904.00 ± 6% interrupts.CPU56.TLB:TLB_shootdowns
232.25 +11.9% 260.00 interrupts.CPU57.167:PCI-MSI.1574969-edge.eth1-TxRx-57
932920 +13.0% 1054029 interrupts.CPU57.LOC:Local_timer_interrupts
1108 ± 9% -20.3% 883.25 ± 10% interrupts.CPU57.TLB:TLB_shootdowns
5714 ± 2% +7.6% 6148 ± 3% interrupts.CPU58.CAL:Function_call_interrupts
933488 +12.9% 1053578 interrupts.CPU58.LOC:Local_timer_interrupts
1032 ± 10% -14.4% 883.75 ± 12% interrupts.CPU58.TLB:TLB_shootdowns
229.75 +65.3% 379.75 ± 54% interrupts.CPU59.169:PCI-MSI.1574971-edge.eth1-TxRx-59
933184 +13.0% 1054265 interrupts.CPU59.LOC:Local_timer_interrupts
14602 ± 4% +15.9% 16918 interrupts.CPU59.RES:Rescheduling_interrupts
933212 +13.0% 1054468 interrupts.CPU6.LOC:Local_timer_interrupts
14821 ± 2% +9.6% 16240 ± 4% interrupts.CPU6.RES:Rescheduling_interrupts
231.25 +12.1% 259.25 interrupts.CPU60.170:PCI-MSI.1574972-edge.eth1-TxRx-60
933465 +13.0% 1054550 interrupts.CPU60.LOC:Local_timer_interrupts
14816 ± 3% +14.2% 16914 ± 4% interrupts.CPU60.RES:Rescheduling_interrupts
229.00 +14.1% 261.25 interrupts.CPU61.171:PCI-MSI.1574973-edge.eth1-TxRx-61
5547 +11.8% 6204 ± 2% interrupts.CPU61.CAL:Function_call_interrupts
933633 +12.9% 1053949 interrupts.CPU61.LOC:Local_timer_interrupts
14666 ± 4% +15.4% 16931 ± 2% interrupts.CPU61.RES:Rescheduling_interrupts
1067 ± 8% -18.0% 875.00 ± 6% interrupts.CPU61.TLB:TLB_shootdowns
233.00 ± 2% +12.1% 261.25 interrupts.CPU62.172:PCI-MSI.1574974-edge.eth1-TxRx-62
933655 +12.9% 1053657 interrupts.CPU62.LOC:Local_timer_interrupts
14612 ± 4% +19.2% 17417 interrupts.CPU62.RES:Rescheduling_interrupts
5619 ± 4% +8.0% 6067 ± 3% interrupts.CPU63.CAL:Function_call_interrupts
933528 +12.9% 1053981 interrupts.CPU63.LOC:Local_timer_interrupts
14550 +18.0% 17174 ± 2% interrupts.CPU63.RES:Rescheduling_interrupts
1081 ± 6% -15.0% 919.50 ± 12% interrupts.CPU63.TLB:TLB_shootdowns
5660 ± 4% +8.5% 6143 ± 4% interrupts.CPU64.CAL:Function_call_interrupts
933555 +12.9% 1054348 interrupts.CPU64.LOC:Local_timer_interrupts
14786 ± 2% +14.7% 16953 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts
1099 ± 10% -21.7% 860.25 ± 11% interrupts.CPU64.TLB:TLB_shootdowns
933511 +12.9% 1054283 interrupts.CPU65.LOC:Local_timer_interrupts
14820 ± 4% +15.7% 17147 ± 2% interrupts.CPU65.RES:Rescheduling_interrupts
933691 +12.9% 1054320 interrupts.CPU66.LOC:Local_timer_interrupts
3496 ±100% -56.7% 1512 ±173% interrupts.CPU66.NMI:Non-maskable_interrupts
3496 ±100% -56.7% 1512 ±173% interrupts.CPU66.PMI:Performance_monitoring_interrupts
15004 ± 5% +11.3% 16707 ± 2% interrupts.CPU66.RES:Rescheduling_interrupts
933485 +12.9% 1054287 interrupts.CPU67.LOC:Local_timer_interrupts
7195 ± 8% -69.7% 2182 ±101% interrupts.CPU67.NMI:Non-maskable_interrupts
7195 ± 8% -69.7% 2182 ±101% interrupts.CPU67.PMI:Performance_monitoring_interrupts
15132 ± 4% +13.1% 17114 ± 3% interrupts.CPU67.RES:Rescheduling_interrupts
933563 +12.9% 1053996 interrupts.CPU68.LOC:Local_timer_interrupts
5283 ± 58% -87.8% 646.50 ±173% interrupts.CPU68.NMI:Non-maskable_interrupts
5283 ± 58% -87.8% 646.50 ±173% interrupts.CPU68.PMI:Performance_monitoring_interrupts
15090 ± 2% +11.2% 16773 ± 2% interrupts.CPU68.RES:Rescheduling_interrupts
933471 +12.9% 1054072 interrupts.CPU69.LOC:Local_timer_interrupts
14966 ± 3% +14.5% 17140 interrupts.CPU69.RES:Rescheduling_interrupts
5647 ± 6% +8.7% 6141 ± 4% interrupts.CPU7.CAL:Function_call_interrupts
933235 +13.0% 1054288 interrupts.CPU7.LOC:Local_timer_interrupts
14832 ± 2% +11.6% 16557 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts
933181 +12.9% 1053888 interrupts.CPU70.LOC:Local_timer_interrupts
14660 +15.9% 16996 interrupts.CPU70.RES:Rescheduling_interrupts
933262 +12.9% 1054025 interrupts.CPU71.LOC:Local_timer_interrupts
19057 ± 5% +14.3% 21779 ± 4% interrupts.CPU71.RES:Rescheduling_interrupts
1029 ± 7% -11.2% 914.00 ± 7% interrupts.CPU71.TLB:TLB_shootdowns
933118 +13.0% 1054528 interrupts.CPU72.LOC:Local_timer_interrupts
15000 ± 2% +7.7% 16149 ± 3% interrupts.CPU72.RES:Rescheduling_interrupts
1040 ± 8% -17.3% 860.50 ± 11% interrupts.CPU72.TLB:TLB_shootdowns
932820 +13.0% 1054552 interrupts.CPU73.LOC:Local_timer_interrupts
989.25 ± 8% -13.5% 855.50 ± 10% interrupts.CPU73.TLB:TLB_shootdowns
932466 +13.1% 1054607 interrupts.CPU74.LOC:Local_timer_interrupts
5508 ± 38% -84.8% 835.00 ±156% interrupts.CPU74.NMI:Non-maskable_interrupts
5508 ± 38% -84.8% 835.00 ±156% interrupts.CPU74.PMI:Performance_monitoring_interrupts
932477 +13.1% 1054632 interrupts.CPU75.LOC:Local_timer_interrupts
932464 +13.1% 1054284 interrupts.CPU76.LOC:Local_timer_interrupts
1800 ± 94% -96.9% 56.50 ± 90% interrupts.CPU76.NMI:Non-maskable_interrupts
1800 ± 94% -96.9% 56.50 ± 90% interrupts.CPU76.PMI:Performance_monitoring_interrupts
932602 +13.1% 1054379 interrupts.CPU77.LOC:Local_timer_interrupts
932942 +13.0% 1054248 interrupts.CPU78.LOC:Local_timer_interrupts
933455 +13.0% 1054533 interrupts.CPU79.LOC:Local_timer_interrupts
237.50 ± 4% +86.1% 442.00 ± 37% interrupts.CPU8.118:PCI-MSI.1574920-edge.eth1-TxRx-8
933128 +13.0% 1054152 interrupts.CPU8.LOC:Local_timer_interrupts
933216 +13.0% 1054462 interrupts.CPU80.LOC:Local_timer_interrupts
15009 +10.3% 16547 ± 2% interrupts.CPU80.RES:Rescheduling_interrupts
932995 +13.0% 1053937 interrupts.CPU81.LOC:Local_timer_interrupts
14860 +10.1% 16360 ± 3% interrupts.CPU81.RES:Rescheduling_interrupts
932787 +13.0% 1054279 interrupts.CPU82.LOC:Local_timer_interrupts
932883 +13.0% 1054564 interrupts.CPU83.LOC:Local_timer_interrupts
1008 ± 4% -14.5% 862.25 ± 11% interrupts.CPU83.TLB:TLB_shootdowns
933029 +13.0% 1054221 interrupts.CPU84.LOC:Local_timer_interrupts
932738 +13.1% 1054620 interrupts.CPU85.LOC:Local_timer_interrupts
1051 ± 4% -18.1% 861.00 ± 14% interrupts.CPU85.TLB:TLB_shootdowns
932660 +13.1% 1054600 interrupts.CPU86.LOC:Local_timer_interrupts
7240 ± 8% -91.8% 591.50 ±156% interrupts.CPU86.NMI:Non-maskable_interrupts
7240 ± 8% -91.8% 591.50 ±156% interrupts.CPU86.PMI:Performance_monitoring_interrupts
932547 +13.1% 1054581 interrupts.CPU87.LOC:Local_timer_interrupts
932777 +13.1% 1054506 interrupts.CPU88.LOC:Local_timer_interrupts
932383 +13.1% 1054766 interrupts.CPU89.LOC:Local_timer_interrupts
6289 ± 26% -72.5% 1730 ± 56% interrupts.CPU89.NMI:Non-maskable_interrupts
6289 ± 26% -72.5% 1730 ± 56% interrupts.CPU89.PMI:Performance_monitoring_interrupts
933395 +13.0% 1054471 interrupts.CPU9.LOC:Local_timer_interrupts
14486 +11.4% 16132 interrupts.CPU9.RES:Rescheduling_interrupts
933333 +13.0% 1054511 interrupts.CPU90.LOC:Local_timer_interrupts
1032 ± 9% -19.0% 836.50 ± 11% interrupts.CPU90.TLB:TLB_shootdowns
932514 +13.1% 1054478 interrupts.CPU91.LOC:Local_timer_interrupts
14641 ± 2% +14.2% 16726 ± 3% interrupts.CPU91.RES:Rescheduling_interrupts
933477 +13.0% 1054715 interrupts.CPU92.LOC:Local_timer_interrupts
14412 ± 4% +15.4% 16625 ± 5% interrupts.CPU92.RES:Rescheduling_interrupts
932788 +13.0% 1054391 interrupts.CPU93.LOC:Local_timer_interrupts
14622 ± 2% +15.1% 16835 ± 4% interrupts.CPU93.RES:Rescheduling_interrupts
933241 +13.0% 1054261 interrupts.CPU94.LOC:Local_timer_interrupts
14488 ± 3% +14.3% 16554 ± 4% interrupts.CPU94.RES:Rescheduling_interrupts
5712 ± 4% +9.1% 6232 ± 4% interrupts.CPU95.CAL:Function_call_interrupts
932767 +13.0% 1054282 interrupts.CPU95.LOC:Local_timer_interrupts
14327 ± 3% +15.9% 16606 ± 5% interrupts.CPU95.RES:Rescheduling_interrupts
932882 +13.0% 1054055 interrupts.CPU96.LOC:Local_timer_interrupts
14845 ± 2% +12.6% 16710 ± 5% interrupts.CPU96.RES:Rescheduling_interrupts
933112 +13.0% 1054371 interrupts.CPU97.LOC:Local_timer_interrupts
14313 +17.4% 16810 ± 3% interrupts.CPU97.RES:Rescheduling_interrupts
932828 +13.0% 1053941 interrupts.CPU98.LOC:Local_timer_interrupts
14624 ± 2% +13.3% 16574 ± 4% interrupts.CPU98.RES:Rescheduling_interrupts
933182 +12.9% 1054014 interrupts.CPU99.LOC:Local_timer_interrupts
14557 ± 3% +13.3% 16495 ± 2% interrupts.CPU99.RES:Rescheduling_interrupts
253.50 ± 11% -51.4% 123.25 ± 36% interrupts.IWI:IRQ_work_interrupts
1.344e+08 +13.0% 1.518e+08 interrupts.LOC:Local_timer_interrupts
468331 ± 9% -33.1% 313473 ± 15% interrupts.NMI:Non-maskable_interrupts
468331 ± 9% -33.1% 313473 ± 15% interrupts.PMI:Performance_monitoring_interrupts
2147272 +12.3% 2411556 interrupts.RES:Rescheduling_interrupts
148081 ± 9% -12.3% 129914 ± 8% interrupts.TLB:TLB_shootdowns
59.87 -52.8 7.11 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
59.53 -52.4 7.10 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.20 ± 2% -15.2 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
14.65 ± 2% -14.6 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.32 ± 3% -7.3 0.00 perf-profile.calltrace.cycles-pp._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.84 ± 3% -6.8 0.00 perf-profile.calltrace.cycles-pp.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.97 -6.5 4.49 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.97 -6.5 4.49 ± 4% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.96 -6.5 4.49 ± 4% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.68 ± 2% -6.4 6.24 ± 3% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.91 ± 2% -6.0 5.88 ± 3% perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
11.00 ± 2% -5.6 5.44 ± 3% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
9.08 -5.3 3.80 ± 5% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
9.05 -5.3 3.79 ± 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
10.37 ± 2% -5.2 5.15 ± 3% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
14.44 ± 8% -4.7 9.71 ± 22% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
7.15 ± 3% -4.7 2.45 ± 2% perf-profile.calltrace.cycles-pp.page_fault
7.11 ± 3% -4.7 2.40 ± 2% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
6.97 ± 3% -4.6 2.36 ± 2% perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
6.27 ± 3% -4.1 2.14 ± 2% perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
6.11 ± 3% -4.0 2.09 ± 2% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
4.86 ± 3% -2.5 2.32 ± 4% perf-profile.calltrace.cycles-pp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
4.66 ± 3% -2.4 2.22 ± 3% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
3.47 -2.1 1.34 ± 4% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit
3.23 ± 7% -2.1 1.13 ± 3% perf-profile.calltrace.cycles-pp.filemap_map_pages.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
3.36 -2.1 1.29 ± 4% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit
3.30 -1.6 1.69 ± 3% perf-profile.calltrace.cycles-pp.shmem_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
2.98 ± 2% -1.5 1.49 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.48 ± 2% -1.5 1.00 ± 4% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
2.48 ± 2% -1.5 1.00 ± 4% perf-profile.calltrace.cycles-pp.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
2.33 -1.5 0.86 ± 4% perf-profile.calltrace.cycles-pp.anon_vma_fork.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.46 ± 2% -1.5 0.98 ± 4% perf-profile.calltrace.cycles-pp.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap.mmput
2.65 ± 2% -1.4 1.30 ± 2% perf-profile.calltrace.cycles-pp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
2.60 ± 7% -1.4 1.25 ± 7% perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit
2.26 ± 2% -1.2 1.03 ± 4% perf-profile.calltrace.cycles-pp.search_binary_handler.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.22 ± 2% -1.2 1.02 ± 4% perf-profile.calltrace.cycles-pp.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve.do_syscall_64
1.93 ± 2% -1.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu_free.arch_tlb_finish_mmu.tlb_finish_mmu.exit_mmap
1.40 ± 2% -1.1 0.27 ±100% perf-profile.calltrace.cycles-pp.do_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
1.48 ± 5% -1.0 0.46 ± 57% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.58 ± 2% -1.0 0.60 ± 3% perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.copy_process._do_fork.do_syscall_64
1.43 ± 4% -0.8 0.67 ± 5% perf-profile.calltrace.cycles-pp.shmem_alloc_and_acct_page.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter
1.33 ± 3% -0.7 0.59 ± 3% perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.mmput.do_exit
1.36 ± 5% -0.7 0.62 ± 5% perf-profile.calltrace.cycles-pp.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve
0.95 ± 13% -0.7 0.26 ±100% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.27 ± 2% -0.6 0.64 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read
1.07 ± 20% -0.6 0.44 ± 58% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit
1.00 -0.6 0.38 ± 57% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.shmem_file_read_iter.new_sync_read
1.20 ± 22% -0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler
1.00 ± 11% -0.6 0.40 ± 57% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.21 ± 22% -0.6 0.61 ± 5% perf-profile.calltrace.cycles-pp.mmput.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file
0.99 ± 12% -0.6 0.40 ± 57% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.13 ± 2% -0.6 0.55 ± 4% perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_write_begin.generic_perform_write.__generic_file_write_iter
1.21 -0.6 0.65 ± 5% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
0.96 ± 2% -0.6 0.40 ± 57% perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.91 ± 2% -0.5 0.39 ± 57% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.generic_perform_write.__generic_file_write_iter
1.01 ± 2% -0.5 0.51 perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.shmem_file_read_iter.new_sync_read.vfs_read
0.00 +0.6 0.58 ± 4% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
0.00 +0.6 0.58 ± 6% perf-profile.calltrace.cycles-pp.div_int
0.00 +0.6 0.59 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink
0.00 +0.6 0.59 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink
0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.close
0.00 +0.6 0.60 ± 3% perf-profile.calltrace.cycles-pp.unlink
0.00 +0.6 0.61 ± 5% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.write
0.00 +0.6 0.63 ± 6% perf-profile.calltrace.cycles-pp.div_short
0.00 +0.7 0.73 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.write
0.00 +0.7 0.74 ± 18% perf-profile.calltrace.cycles-pp.creat
0.00 +0.7 0.75 ± 18% perf-profile.calltrace.cycles-pp.ftrace_syscall_exit.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +0.8 0.84 ± 5% perf-profile.calltrace.cycles-pp.add_short.add_short
0.00 +0.8 0.84 ± 17% perf-profile.calltrace.cycles-pp.ftrace_syscall_enter.syscall_trace_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +0.9 0.87 ± 16% perf-profile.calltrace.cycles-pp.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.20 ±173% +0.9 1.11 ± 34% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
0.21 ±173% +1.0 1.20 ± 34% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
0.00 +1.1 1.05 ± 14% perf-profile.calltrace.cycles-pp.syscall_trace_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +1.1 1.15 ± 5% perf-profile.calltrace.cycles-pp.add_long.add_long
0.00 +1.2 1.22 ± 5% perf-profile.calltrace.cycles-pp.add_int.add_int
0.00 +1.3 1.30 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.lseek64
0.00 +1.3 1.32 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.lseek64
0.00 +1.4 1.42 ± 7% perf-profile.calltrace.cycles-pp.do_brk_flags.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
0.66 ± 6% +1.7 2.35 ± 4% perf-profile.calltrace.cycles-pp.copy_process._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
0.00 +1.8 1.78 perf-profile.calltrace.cycles-pp.lseek64
0.00 +1.8 1.82 ± 5% perf-profile.calltrace.cycles-pp.string_rtns_1
0.00 +1.9 1.86 ± 4% perf-profile.calltrace.cycles-pp.div_long
0.00 +1.9 1.87 ± 3% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +2.0 1.98 ± 3% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +2.0 2.00 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
0.00 +2.5 2.51 ± 5% perf-profile.calltrace.cycles-pp.div_float
0.00 +2.5 2.52 ± 4% perf-profile.calltrace.cycles-pp._do_fork.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.fork_test
0.00 +2.5 2.54 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.fork_test
0.00 +2.5 2.54 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork.fork_test
0.00 +2.7 2.67 ± 4% perf-profile.calltrace.cycles-pp.div_double
0.00 +2.9 2.89 ± 4% perf-profile.calltrace.cycles-pp.__libc_fork.fork_test
0.00 +3.1 3.08 ± 4% perf-profile.calltrace.cycles-pp.fork_test
0.00 +3.3 3.25 ± 4% perf-profile.calltrace.cycles-pp.add_float.add_float
0.00 +3.7 3.65 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
0.00 +3.7 3.67 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +3.7 3.69 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.brk
0.00 +3.7 3.71 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
0.00 +3.8 3.76 ± 4% perf-profile.calltrace.cycles-pp.mul_double.mul_double
0.00 +4.0 4.05 ± 6% perf-profile.calltrace.cycles-pp.array_rtns
0.00 +4.1 4.13 ± 6% perf-profile.calltrace.cycles-pp.mul_float.mul_float
0.00 +4.4 4.38 perf-profile.calltrace.cycles-pp.read
0.00 +4.4 4.45 ± 3% perf-profile.calltrace.cycles-pp.brk
0.00 +4.8 4.84 ± 6% perf-profile.calltrace.cycles-pp.add_double.add_double
0.00 +6.1 6.11 ± 5% perf-profile.calltrace.cycles-pp.sieve
0.00 +7.2 7.16 ± 3% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +7.4 7.42 ± 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +11.3 11.28 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +11.4 11.36 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
0.00 +13.0 12.97 perf-profile.calltrace.cycles-pp.write
64.48 -28.1 36.37 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
64.12 -27.9 36.18 perf-profile.children.cycles-pp.do_syscall_64
15.23 ± 2% -7.7 7.50 ± 2% perf-profile.children.cycles-pp.ksys_write
14.69 ± 2% -7.4 7.24 ± 2% perf-profile.children.cycles-pp.vfs_write
11.01 -6.5 4.54 ± 4% perf-profile.children.cycles-pp.__x64_sys_exit_group
11.01 -6.5 4.54 ± 4% perf-profile.children.cycles-pp.do_group_exit
11.00 -6.5 4.54 ± 4% perf-profile.children.cycles-pp.do_exit
12.70 ± 2% -6.4 6.30 ± 2% perf-profile.children.cycles-pp.new_sync_write
10.78 ± 2% -6.2 4.58 ± 5% perf-profile.children.cycles-pp.mmput
10.75 ± 2% -6.2 4.57 ± 5% perf-profile.children.cycles-pp.exit_mmap
11.94 ± 2% -6.0 5.94 ± 2% perf-profile.children.cycles-pp.generic_file_write_iter
11.02 ± 2% -5.5 5.50 ± 2% perf-profile.children.cycles-pp.__generic_file_write_iter
10.43 ± 2% -5.2 5.22 ± 2% perf-profile.children.cycles-pp.generic_perform_write
14.51 ± 8% -4.7 9.78 ± 22% perf-profile.children.cycles-pp.intel_idle
8.55 ± 3% -4.5 4.05 ± 2% perf-profile.children.cycles-pp.do_page_fault
8.67 ± 3% -4.4 4.24 perf-profile.children.cycles-pp.page_fault
8.09 ± 3% -4.4 3.67 ± 3% perf-profile.children.cycles-pp._do_fork
8.40 ± 3% -4.4 3.98 ± 2% perf-profile.children.cycles-pp.__do_page_fault
7.50 ± 3% -4.1 3.39 ± 4% perf-profile.children.cycles-pp.copy_process
7.63 ± 4% -4.0 3.62 ± 2% perf-profile.children.cycles-pp.handle_mm_fault
7.43 ± 4% -3.9 3.53 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault
5.53 ± 2% -2.9 2.67 ± 2% perf-profile.children.cycles-pp.shmem_getpage_gfp
4.71 ± 2% -2.7 2.04 ± 4% perf-profile.children.cycles-pp.__x64_sys_brk
4.26 -2.6 1.66 ± 4% perf-profile.children.cycles-pp.unmap_vmas
4.87 ± 3% -2.5 2.35 ± 2% perf-profile.children.cycles-pp.shmem_write_begin
4.12 -2.5 1.60 ± 4% perf-profile.children.cycles-pp.unmap_page_range
4.74 ± 14% -2.4 2.35 ± 8% perf-profile.children.cycles-pp.down_write
4.09 ± 8% -2.1 1.95 ± 2% perf-profile.children.cycles-pp.filemap_map_pages
4.16 -2.1 2.05 ± 2% perf-profile.children.cycles-pp.ksys_read
3.96 -2.0 1.96 ± 2% perf-profile.children.cycles-pp.vfs_read
2.87 ± 6% -2.0 0.89 ± 3% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
3.70 ± 2% -2.0 1.74 ± 3% perf-profile.children.cycles-pp.syscall_return_via_sysret
3.70 ± 2% -2.0 1.74 ± 3% perf-profile.children.cycles-pp.__x64_sys_execve
3.67 ± 2% -1.9 1.73 ± 3% perf-profile.children.cycles-pp.__do_execve_file
3.19 -1.9 1.31 ± 4% perf-profile.children.cycles-pp.tlb_finish_mmu
3.18 -1.9 1.31 ± 4% perf-profile.children.cycles-pp.arch_tlb_finish_mmu
3.32 ± 2% -1.9 1.45 ± 5% perf-profile.children.cycles-pp.do_brk_flags
3.26 ± 2% -1.9 1.41 ± 3% perf-profile.children.cycles-pp.release_pages
3.53 ± 9% -1.8 1.72 ± 7% perf-profile.children.cycles-pp.free_pgtables
2.91 ± 2% -1.7 1.20 ± 4% perf-profile.children.cycles-pp.tlb_flush_mmu_free
3.57 ± 18% -1.7 1.86 ± 9% perf-profile.children.cycles-pp.call_rwsem_down_write_failed
3.56 ± 18% -1.7 1.86 ± 9% perf-profile.children.cycles-pp.rwsem_down_write_failed
3.32 -1.6 1.72 perf-profile.children.cycles-pp.shmem_write_end
3.12 ± 2% -1.6 1.54 ± 2% perf-profile.children.cycles-pp.new_sync_read
3.35 ± 3% -1.5 1.80 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64
2.85 ± 2% -1.5 1.32 ± 3% perf-profile.children.cycles-pp.search_binary_handler
2.82 ± 2% -1.5 1.31 ± 3% perf-profile.children.cycles-pp.load_elf_binary
2.67 ± 2% -1.3 1.32 perf-profile.children.cycles-pp.shmem_file_read_iter
2.38 -1.3 1.12 ± 3% perf-profile.children.cycles-pp.anon_vma_fork
2.11 ± 3% -1.1 0.98 ± 13% perf-profile.children.cycles-pp.do_sys_open
2.04 ± 3% -1.1 0.94 ± 12% perf-profile.children.cycles-pp.do_filp_open
2.19 ± 3% -1.1 1.10 ± 4% perf-profile.children.cycles-pp.__alloc_pages_nodemask
2.02 ± 3% -1.1 0.93 ± 13% perf-profile.children.cycles-pp.path_openat
2.30 ± 22% -1.1 1.23 ± 12% perf-profile.children.cycles-pp.osq_lock
1.65 ± 5% -1.1 0.59 ± 3% perf-profile.children.cycles-pp.__do_sys_wait4
1.65 ± 5% -1.1 0.59 ± 3% perf-profile.children.cycles-pp.kernel_wait4
1.63 ± 5% -1.0 0.58 ± 3% perf-profile.children.cycles-pp.do_wait
1.68 ± 2% -1.0 0.66 ± 6% perf-profile.children.cycles-pp.copy_page_range
1.63 ± 2% -1.0 0.63 ± 5% perf-profile.children.cycles-pp.copy_p4d_range
1.88 -1.0 0.92 ± 2% perf-profile.children.cycles-pp.find_lock_entry
1.76 ± 4% -1.0 0.80 ± 4% perf-profile.children.cycles-pp.flush_old_exec
1.98 -0.9 1.04 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.68 ± 2% -0.9 0.75 ± 3% perf-profile.children.cycles-pp.unlink_anon_vmas
1.19 ± 13% -0.9 0.29 ± 4% perf-profile.children.cycles-pp.queued_write_lock_slowpath
1.58 ± 4% -0.9 0.69 ± 5% perf-profile.children.cycles-pp.__vma_adjust
1.84 ± 4% -0.9 0.95 ± 4% perf-profile.children.cycles-pp.get_page_from_freelist
1.75 ± 3% -0.9 0.86 ± 2% perf-profile.children.cycles-pp.evict
1.75 ± 3% -0.9 0.86 ± 2% perf-profile.children.cycles-pp.shmem_evict_inode
1.74 ± 3% -0.9 0.85 ± 2% perf-profile.children.cycles-pp.shmem_truncate_range
1.74 ± 3% -0.9 0.85 ± 2% perf-profile.children.cycles-pp.shmem_undo_range
1.66 -0.9 0.80 ± 2% perf-profile.children.cycles-pp.anon_vma_clone
1.54 ± 2% -0.8 0.70 ± 2% perf-profile.children.cycles-pp.do_wp_page
1.47 ± 5% -0.8 0.68 ± 3% perf-profile.children.cycles-pp.__do_munmap
1.54 ± 3% -0.8 0.76 ± 5% perf-profile.children.cycles-pp.alloc_pages_vma
1.60 ± 20% -0.8 0.83 ± 11% perf-profile.children.cycles-pp.unlink_file_vma
1.44 ± 4% -0.8 0.68 ± 4% perf-profile.children.cycles-pp.shmem_alloc_and_acct_page
1.24 -0.8 0.48 ± 2% perf-profile.children.cycles-pp.page_remove_rmap
1.39 ± 2% -0.8 0.64 ± 2% perf-profile.children.cycles-pp.wp_page_copy
1.51 ± 8% -0.7 0.77 ± 3% perf-profile.children.cycles-pp.vm_mmap_pgoff
1.31 ± 4% -0.7 0.58 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
1.37 -0.7 0.67 ± 2% perf-profile.children.cycles-pp.find_get_entry
1.43 ± 8% -0.7 0.73 ± 3% perf-profile.children.cycles-pp.do_mmap
1.29 -0.7 0.60 ± 5% perf-profile.children.cycles-pp.perf_event_mmap
1.28 -0.7 0.61 ± 3% perf-profile.children.cycles-pp.security_file_permission
1.31 ± 2% -0.6 0.67 perf-profile.children.cycles-pp.copy_page_to_iter
1.12 ± 5% -0.6 0.49 ± 5% perf-profile.children.cycles-pp.ret_from_fork
1.30 ± 9% -0.6 0.67 ± 4% perf-profile.children.cycles-pp.mmap_region
1.25 ± 5% -0.6 0.62 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop
1.10 ± 2% -0.6 0.48 ± 3% perf-profile.children.cycles-pp.___might_sleep
1.27 ± 9% -0.6 0.66 ± 3% perf-profile.children.cycles-pp.ksys_mmap_pgoff
0.92 -0.6 0.33 ± 5% perf-profile.children.cycles-pp.vma_merge
1.17 -0.6 0.58 ± 3% perf-profile.children.cycles-pp.do_unlinkat
1.16 -0.6 0.58 perf-profile.children.cycles-pp.__schedule
1.21 -0.6 0.65 ± 4% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
1.06 ± 2% -0.6 0.51 ± 4% perf-profile.children.cycles-pp.selinux_file_permission
0.79 ± 7% -0.5 0.24 ± 2% perf-profile.children.cycles-pp.queued_read_lock_slowpath
1.05 ± 4% -0.5 0.51 ± 2% perf-profile.children.cycles-pp.dput
1.05 ± 5% -0.5 0.51 ± 4% perf-profile.children.cycles-pp.task_work_run
0.96 -0.5 0.44 ± 3% perf-profile.children.cycles-pp.pagevec_lru_move_fn
1.04 ± 2% -0.5 0.53 perf-profile.children.cycles-pp.copyout
0.99 ± 5% -0.5 0.47 ± 3% perf-profile.children.cycles-pp.__fput
0.95 ± 6% -0.5 0.44 ± 4% perf-profile.children.cycles-pp.unmap_region
0.94 -0.5 0.45 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.91 ± 6% -0.5 0.43 ± 4% perf-profile.children.cycles-pp.free_unref_page_list
0.99 ± 8% -0.5 0.51 ± 6% perf-profile.children.cycles-pp.__split_vma
0.85 ± 2% -0.5 0.37 perf-profile.children.cycles-pp.kmem_cache_free
0.92 ± 6% -0.5 0.44 ± 6% perf-profile.children.cycles-pp.shmem_alloc_page
0.79 ± 7% -0.5 0.32 ± 10% perf-profile.children.cycles-pp.kthread
0.88 ± 4% -0.5 0.40 ± 2% perf-profile.children.cycles-pp.security_vm_enough_memory_mm
0.98 -0.4 0.53 ± 4% perf-profile.children.cycles-pp.copyin
0.88 -0.4 0.43 ± 2% perf-profile.children.cycles-pp.do_anonymous_page
0.86 ± 4% -0.4 0.42 ± 3% perf-profile.children.cycles-pp.__pagevec_release
0.85 ± 6% -0.4 0.42 ± 2% perf-profile.children.cycles-pp.dentry_kill
0.84 ± 6% -0.4 0.41 ± 3% perf-profile.children.cycles-pp.__dentry_kill
0.94 ± 6% -0.4 0.51 ± 5% perf-profile.children.cycles-pp.up_write
0.86 ± 2% -0.4 0.44 ± 2% perf-profile.children.cycles-pp.anon_vma_interval_tree_insert
0.78 -0.4 0.36 ± 3% perf-profile.children.cycles-pp.rcu_core
0.85 ± 7% -0.4 0.45 ± 4% perf-profile.children.cycles-pp.__x64_sys_mprotect
0.85 ± 7% -0.4 0.45 ± 4% perf-profile.children.cycles-pp.do_mprotect_pkey
0.84 ± 7% -0.4 0.44 ± 4% perf-profile.children.cycles-pp.mprotect_fixup
1.05 ± 3% -0.4 0.65 ± 12% perf-profile.children.cycles-pp.__softirqentry_text_start
0.60 ± 6% -0.4 0.21 ± 8% perf-profile.children.cycles-pp.wait_consider_task
0.72 -0.4 0.33 ± 4% perf-profile.children.cycles-pp.__lru_cache_add
0.78 ± 6% -0.4 0.39 ± 5% perf-profile.children.cycles-pp.rwsem_spin_on_owner
0.72 -0.4 0.34 ± 3% perf-profile.children.cycles-pp.perf_iterate_sb
0.66 ± 3% -0.4 0.29 perf-profile.children.cycles-pp.__slab_free
0.72 -0.4 0.35 ± 5% perf-profile.children.cycles-pp.xas_load
0.69 ± 7% -0.4 0.32 ± 3% perf-profile.children.cycles-pp.__fdget_pos
0.69 ± 9% -0.4 0.32 ± 5% perf-profile.children.cycles-pp.free_pcppages_bulk
0.72 -0.4 0.36 perf-profile.children.cycles-pp.select_task_rq_fair
0.59 ± 2% -0.3 0.24 ± 5% perf-profile.children.cycles-pp.free_pages_and_swap_cache
0.73 ± 3% -0.3 0.39 ± 3% perf-profile.children.cycles-pp.task_tick_fair
1.05 ± 7% -0.3 0.71 ± 9% perf-profile.children.cycles-pp.update_process_times
0.63 ± 8% -0.3 0.29 ± 6% perf-profile.children.cycles-pp.__list_del_entry_valid
0.65 -0.3 0.31 ± 3% perf-profile.children.cycles-pp._cond_resched
0.62 ± 3% -0.3 0.29 ± 5% perf-profile.children.cycles-pp.load_script
0.65 ± 3% -0.3 0.31 ± 3% perf-profile.children.cycles-pp.alloc_set_pte
0.64 -0.3 0.30 ± 4% perf-profile.children.cycles-pp.find_vma
0.64 ± 2% -0.3 0.30 ± 3% perf-profile.children.cycles-pp.truncate_inode_page
0.89 ± 4% -0.3 0.55 ± 6% perf-profile.children.cycles-pp.scheduler_tick
0.74 ± 2% -0.3 0.41 ± 2% perf-profile.children.cycles-pp.prep_new_page
0.59 ± 8% -0.3 0.26 ± 4% perf-profile.children.cycles-pp.selinux_vm_enough_memory
1.07 ± 7% -0.3 0.74 ± 10% perf-profile.children.cycles-pp.tick_sched_handle
0.62 ± 3% -0.3 0.30 ± 6% perf-profile.children.cycles-pp.link_path_walk
0.71 ± 4% -0.3 0.38 ± 4% perf-profile.children.cycles-pp.try_to_wake_up
0.51 ± 5% -0.3 0.18 ± 7% perf-profile.children.cycles-pp.release_task
0.57 ± 2% -0.3 0.24 ± 4% perf-profile.children.cycles-pp.flush_tlb_mm_range
0.62 -0.3 0.30 ± 5% perf-profile.children.cycles-pp.unlock_page
0.58 ± 3% -0.3 0.27 ± 4% perf-profile.children.cycles-pp.__might_sleep
1.11 ± 8% -0.3 0.79 ± 11% perf-profile.children.cycles-pp.tick_sched_timer
0.69 -0.3 0.37 perf-profile.children.cycles-pp.clear_page_erms
0.59 ± 4% -0.3 0.28 ± 7% perf-profile.children.cycles-pp.kmem_cache_alloc
0.60 ± 3% -0.3 0.30 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair
0.56 ± 7% -0.3 0.26 ± 4% perf-profile.children.cycles-pp.__fget_light
1.38 ± 9% -0.3 1.08 ± 13% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.68 ± 3% -0.3 0.39 ± 2% perf-profile.children.cycles-pp.setlocale
0.55 ± 2% -0.3 0.27 ± 3% perf-profile.children.cycles-pp.wake_up_new_task
0.57 ± 3% -0.3 0.29 ± 4% perf-profile.children.cycles-pp.filename_lookup
0.55 ± 2% -0.3 0.27 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret
0.56 ± 3% -0.3 0.28 ± 5% perf-profile.children.cycles-pp.path_lookupat
0.49 ± 3% -0.3 0.22 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock
0.53 ± 3% -0.3 0.26 ± 4% perf-profile.children.cycles-pp.copy_page
0.51 ± 2% -0.3 0.24 ± 4% perf-profile.children.cycles-pp.get_unmapped_area
0.46 ± 12% -0.3 0.20 ± 7% perf-profile.children.cycles-pp.cred_has_capability
0.53 ± 8% -0.3 0.27 ± 4% perf-profile.children.cycles-pp.vma_link
0.45 ± 2% -0.3 0.19 ± 4% perf-profile.children.cycles-pp.flush_tlb_func_common
0.59 -0.3 0.34 ± 6% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.47 ± 4% -0.3 0.21 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate
0.49 ± 2% -0.3 0.24 ± 3% perf-profile.children.cycles-pp.delete_from_page_cache
0.48 ± 2% -0.3 0.23 ± 2% perf-profile.children.cycles-pp.mark_page_accessed
0.48 ± 3% -0.3 0.23 ± 8% perf-profile.children.cycles-pp.fsnotify
0.49 -0.2 0.24 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair
0.48 ± 2% -0.2 0.23 ± 3% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.42 ± 2% -0.2 0.18 ± 2% perf-profile.children.cycles-pp.remove_vma
0.43 ± 3% -0.2 0.18 ± 4% perf-profile.children.cycles-pp.anon_vma_interval_tree_remove
0.40 ± 4% -0.2 0.15 ± 8% perf-profile.children.cycles-pp.smpboot_thread_fn
0.42 ± 3% -0.2 0.17 ± 6% perf-profile.children.cycles-pp.do_task_dead
0.44 ± 3% -0.2 0.20 ± 3% perf-profile.children.cycles-pp.__wake_up_common
0.42 ± 5% -0.2 0.18 ± 4% perf-profile.children.cycles-pp.__sb_start_write
0.45 ± 3% -0.2 0.21 ± 7% perf-profile.children.cycles-pp.elf_map
0.47 ± 3% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.pte_alloc_one
0.41 ± 3% -0.2 0.18 ± 2% perf-profile.children.cycles-pp.native_flush_tlb_one_user
0.39 ± 11% -0.2 0.16 ± 14% perf-profile.children.cycles-pp.worker_thread
0.40 ± 15% -0.2 0.16 ± 5% perf-profile.children.cycles-pp.avc_has_perm_noaudit
0.47 ± 3% -0.2 0.24 perf-profile.children.cycles-pp.vma_interval_tree_insert
0.45 ± 5% -0.2 0.22 ± 3% perf-profile.children.cycles-pp.walk_component
0.43 -0.2 0.21 ± 4% perf-profile.children.cycles-pp.__x64_sys_getdents
0.41 ± 2% -0.2 0.20 ± 7% perf-profile.children.cycles-pp.enqueue_entity
0.43 -0.2 0.21 ± 4% perf-profile.children.cycles-pp.iterate_dir
0.41 ± 2% -0.2 0.20 ± 2% perf-profile.children.cycles-pp.dcache_readdir
0.32 ± 14% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.free_work
0.39 ± 2% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.__vm_enough_memory
0.40 -0.2 0.19 ± 2% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.34 ± 12% -0.2 0.13 ± 19% perf-profile.children.cycles-pp.process_one_work
0.31 ± 15% -0.2 0.11 ± 4% perf-profile.children.cycles-pp.__vunmap
0.50 -0.2 0.30 ± 3% perf-profile.children.cycles-pp.schedule
0.39 ± 4% -0.2 0.19 ± 4% perf-profile.children.cycles-pp.__delete_from_page_cache
0.37 -0.2 0.17 ± 2% perf-profile.children.cycles-pp._dl_addr
0.38 -0.2 0.18 ± 6% perf-profile.children.cycles-pp.ttwu_do_activate
0.37 -0.2 0.18 ± 2% perf-profile.children.cycles-pp.file_update_time
0.48 ± 4% -0.2 0.29 ± 6% perf-profile.children.cycles-pp.load_balance
0.35 ± 3% -0.2 0.16 perf-profile.children.cycles-pp.do_notify_parent
0.38 ± 2% -0.2 0.18 ± 2% perf-profile.children.cycles-pp.__pte_alloc
0.35 ± 5% -0.2 0.16 ± 2% perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay
0.37 ± 5% -0.2 0.19 ± 2% perf-profile.children.cycles-pp.page_mapping
0.39 ± 5% -0.2 0.21 ± 3% perf-profile.children.cycles-pp.update_cfs_group
0.31 ± 3% -0.2 0.13 ± 5% perf-profile.children.cycles-pp.wait4
0.30 ± 5% -0.2 0.12 ± 10% perf-profile.children.cycles-pp.put_cred_rcu
0.33 ± 2% -0.2 0.15 ± 2% perf-profile.children.cycles-pp.filldir
0.30 ± 13% -0.2 0.12 ± 4% perf-profile.children.cycles-pp.__vmalloc_node_range
0.35 ± 2% -0.2 0.18 ± 4% perf-profile.children.cycles-pp.__vm_munmap
0.35 -0.2 0.18 ± 6% perf-profile.children.cycles-pp.vfs_statx
0.29 -0.2 0.13 ± 6% perf-profile.children.cycles-pp.vma_interval_tree_remove
0.32 ± 4% -0.2 0.15 ± 5% perf-profile.children.cycles-pp.schedule_tail
0.29 -0.2 0.13 ± 13% perf-profile.children.cycles-pp.security_inode_permission
0.31 ± 2% -0.2 0.15 perf-profile.children.cycles-pp.perf_event_mmap_output
0.29 ± 3% -0.2 0.13 ± 12% perf-profile.children.cycles-pp.alloc_empty_file
0.28 -0.2 0.12 ± 6% perf-profile.children.cycles-pp.ptep_clear_flush
0.33 ± 3% -0.2 0.17 ± 4% perf-profile.children.cycles-pp.copy_strings
0.30 ± 2% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.rcu_all_qs
0.23 ± 18% -0.2 0.08 ± 6% perf-profile.children.cycles-pp.find_vmap_area
0.28 -0.2 0.12 ± 10% perf-profile.children.cycles-pp.selinux_inode_permission
0.27 ± 3% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__alloc_file
0.23 ± 4% -0.2 0.07 ± 17% perf-profile.children.cycles-pp.run_ksoftirqd
0.29 ± 3% -0.2 0.13 ± 3% perf-profile.children.cycles-pp.lookup_fast
0.27 ± 3% -0.2 0.11 ± 3% perf-profile.children.cycles-pp.vma_compute_subtree_gap
0.30 ± 3% -0.2 0.15 ± 4% perf-profile.children.cycles-pp.page_add_file_rmap
0.29 ± 2% -0.2 0.14 ± 6% perf-profile.children.cycles-pp.current_time
0.26 ± 2% -0.2 0.11 ± 6% perf-profile.children.cycles-pp.pipe_write
0.28 ± 4% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.__perf_sw_event
0.27 ± 6% -0.1 0.13 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair
0.27 ± 5% -0.1 0.13 ± 6% perf-profile.children.cycles-pp.lru_add_drain_cpu
0.28 -0.1 0.13 ± 3% perf-profile.children.cycles-pp.__check_object_size
0.27 ± 5% -0.1 0.13 ± 5% perf-profile.children.cycles-pp.lru_add_drain
0.28 ± 5% -0.1 0.14 ± 6% perf-profile.children.cycles-pp.do_faccessat
0.29 -0.1 0.15 ± 8% perf-profile.children.cycles-pp.__do_sys_newstat
0.26 ± 7% -0.1 0.12 ± 5% perf-profile.children.cycles-pp.ksys_lseek
0.27 ± 3% -0.1 0.14 ± 3% perf-profile.children.cycles-pp.xas_store
0.27 ± 4% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.___perf_sw_event
0.26 ± 5% -0.1 0.13 ± 10% perf-profile.children.cycles-pp.xas_start
0.25 ± 4% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.iov_iter_fault_in_readable
0.21 ± 20% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.__get_vm_area_node
0.23 ± 3% -0.1 0.10 perf-profile.children.cycles-pp.tlb_flush_mmu
0.26 ± 3% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.update_curr
0.24 ± 3% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.finish_task_switch
0.24 ± 2% -0.1 0.11 ± 7% perf-profile.children.cycles-pp.__might_fault
0.23 ± 9% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.mem_cgroup_commit_charge
0.29 ± 6% -0.1 0.17 ± 8% perf-profile.children.cycles-pp.find_busiest_group
0.24 ± 3% -0.1 0.12 ± 5% perf-profile.children.cycles-pp.vm_area_dup
0.23 ± 3% -0.1 0.11 ± 7% perf-profile.children.cycles-pp.sched_exec
0.28 ± 5% -0.1 0.17 ± 6% perf-profile.children.cycles-pp.update_sd_lb_stats
0.21 ± 4% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.dequeue_entity
0.24 ± 3% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.sock_write_iter
0.24 ± 3% -0.1 0.12 perf-profile.children.cycles-pp.sock_sendmsg
0.22 ± 3% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
0.22 -0.1 0.10 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent
0.18 ± 18% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.alloc_vmap_area
0.22 ± 4% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.__account_scheduler_latency
0.21 ± 3% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.security_mmap_addr
0.19 ± 5% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.do_truncate
0.30 ± 3% -0.1 0.19 ± 3% perf-profile.children.cycles-pp.mmap64
0.23 -0.1 0.12 ± 9% perf-profile.children.cycles-pp.vmacache_find
0.23 ± 6% -0.1 0.11 ± 3% perf-profile.children.cycles-pp.mem_cgroup_try_charge
0.23 ± 5% -0.1 0.11 ± 3% perf-profile.children.cycles-pp.touch_atime
0.21 ± 5% -0.1 0.10 ± 9% perf-profile.children.cycles-pp.do_signal
0.19 ± 3% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.refcount_dec_not_one
0.21 ± 3% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.down_write_killable
0.22 ± 3% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.__put_user_4
0.21 ± 3% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.__rb_insert_augmented
0.17 ± 8% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.schedule_idle
0.23 ± 6% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.set_page_dirty
0.21 ± 7% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.getname_flags
0.18 ± 7% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.do_dentry_open
0.17 ± 4% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.refcount_dec_and_lock_irqsave
0.18 ± 4% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.sched_move_task
0.18 ± 2% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.update_load_avg
0.23 ± 8% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.__get_free_pages
0.20 ± 4% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.cap_vm_enough_memory
0.19 ± 4% -0.1 0.09 ± 8% perf-profile.children.cycles-pp.__mod_node_page_state
0.20 ± 9% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.strlcpy
0.18 ± 4% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.save_stack_trace_tsk
0.17 ± 4% -0.1 0.07 ± 15% perf-profile.children.cycles-pp.free_uid
0.18 ± 3% -0.1 0.08 ± 13% perf-profile.children.cycles-pp.__set_page_dirty_no_writeback
0.15 ± 12% -0.1 0.05 perf-profile.children.cycles-pp._vm_normal_page
0.18 ± 6% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.__put_anon_vma
0.18 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.autoremove_wake_function
0.18 ± 7% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.atime_needs_update
0.18 ± 4% -0.1 0.08 perf-profile.children.cycles-pp.__save_stack_trace
0.17 ± 4% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.__x64_sys_kill
0.20 ± 10% -0.1 0.11 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.17 ± 3% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.__x64_sys_write
0.11 ± 7% -0.1 0.03 ±100% perf-profile.children.cycles-pp.rcu_cblist_dequeue
0.17 ± 6% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.__clear_user
0.11 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__tlb_remove_page_size
0.13 ± 5% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.notify_change
0.16 ± 4% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.__list_add_valid
0.15 ± 4% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.kill_something_info
0.17 ± 3% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.mm_init
0.14 ± 3% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.__send_signal
0.15 ± 13% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.__x64_sys_link
0.12 ± 6% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.unlazy_walk
0.17 ± 9% -0.1 0.09 ± 11% perf-profile.children.cycles-pp.lookup_slow
0.17 ± 2% -0.1 0.09 perf-profile.children.cycles-pp.__x64_sys_munmap
0.15 ± 4% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.__d_lookup_rcu
0.14 ± 7% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.strnlen_user
0.15 ± 14% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.do_linkat
0.14 -0.1 0.06 ± 6% perf-profile.children.cycles-pp.do_open_execat
0.10 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.osq_unlock
0.16 ± 8% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.iov_iter_advance
0.16 ± 5% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.ip_output
0.12 ± 5% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.free_pgd_range
0.14 ± 5% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.pgd_alloc
0.12 ± 7% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.legitimize_path
0.13 ± 5% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.kill_pid_info
0.10 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.try_charge
0.10 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.xas_clear_mark
0.14 ± 5% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.ip_finish_output2
0.14 ± 3% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__rb_erase_color
0.14 ± 7% -0.1 0.07 ± 14% perf-profile.children.cycles-pp.xas_find
0.10 ± 5% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.10 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.d_alloc
0.15 ± 2% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.strncpy_from_user
0.14 ± 9% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.time
0.12 ± 8% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__vma_link_rb
0.13 ± 9% -0.1 0.06 ± 13% perf-profile.children.cycles-pp.__lookup_slow
0.11 ± 4% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.truncate_cleanup_page
0.11 ± 4% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.create_elf_tables
0.13 ± 13% -0.1 0.07 ± 19% perf-profile.children.cycles-pp.__slab_alloc
0.12 ± 6% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.12 ± 4% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.terminate_walk
0.13 ± 5% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.xas_init_marks
0.10 ± 4% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.free_p4d_range
0.11 ± 4% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.memset_erms
0.11 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.11 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.unwind_next_frame
0.12 ± 3% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.generic_write_check_limits
0.10 ± 4% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.perf_event_task
0.10 -0.1 0.04 ± 57% perf-profile.children.cycles-pp.percpu_counter_add_batch
0.12 ± 7% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.__x64_sys_sync
0.12 ± 7% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.ksys_sync
0.12 ± 7% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.lock_page_memcg
0.09 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__x64_sys_connect
0.12 ± 8% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.strlen
0.14 ± 7% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.find_get_entries
0.12 ± 12% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.d_alloc_parallel
0.10 ± 7% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.tcp_sendmsg_locked
0.13 ± 9% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.__tcp_transmit_skb
0.12 ± 7% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.sync
0.12 ± 10% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.__ip_queue_xmit
0.12 ± 3% -0.1 0.06 ± 7% perf-profile.children.cycles-pp._copy_to_user
0.12 ± 13% -0.1 0.07 ± 13% perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited
0.12 ± 3% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.12 -0.1 0.06 ± 11% perf-profile.children.cycles-pp.change_protection
0.10 ± 4% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.do_send_sig_info
0.08 ± 5% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__sys_connect
0.11 ± 7% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.free_unref_page_commit
0.12 ± 4% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.__pmd_alloc
0.12 ± 11% -0.1 0.06 ± 13% perf-profile.children.cycles-pp.___slab_alloc
0.12 ± 11% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.sock_read_iter
0.10 ± 5% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.inode_permission
0.10 ± 5% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.process_backlog
0.11 ± 8% -0.1 0.05 perf-profile.children.cycles-pp.iterate_supers
0.11 ± 4% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.rw_verify_area
0.10 ± 5% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.copy_user_generic_unrolled
0.08 -0.1 0.03 ±100% perf-profile.children.cycles-pp.udp_sendmsg
0.11 ± 7% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.put_task_stack
0.11 ± 6% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.tcp_sendmsg
0.11 ± 4% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.do_softirq_own_stack
0.10 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.net_rx_action
0.09 ± 7% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
0.16 ± 5% -0.1 0.11 ± 10% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.11 ± 6% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.__local_bh_enable_ip
0.09 -0.1 0.04 ± 57% perf-profile.children.cycles-pp.pipe_read
0.11 ± 4% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.do_softirq
0.08 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp.available_idle_cpu
0.10 ± 8% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.arch_dup_task_struct
0.09 ± 16% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.inet_recvmsg
0.12 ± 11% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.__pud_alloc
0.12 ± 3% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.get_user_pages_remote
0.11 ± 4% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.__get_user_pages
0.08 ± 5% -0.0 0.04 ± 57% perf-profile.children.cycles-pp.copy_strings_kernel
0.08 ± 5% -0.0 0.04 ± 57% perf-profile.children.cycles-pp.preempt_schedule_common
0.10 ± 4% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.setup_arg_pages
0.12 ± 3% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.update_rq_clock
0.10 ± 5% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.select_idle_sibling
0.09 ± 7% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.getenv
0.08 ± 10% -0.0 0.05 perf-profile.children.cycles-pp.__strcasecmp
0.09 ± 4% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__nss_lookup_function
0.10 ± 11% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.__fxstat64
0.07 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.connect
0.17 +0.0 0.21 ± 6% perf-profile.children.cycles-pp.__xstat64
0.01 ±173% +0.0 0.06 ± 7% perf-profile.children.cycles-pp._exit
0.01 ±173% +0.0 0.06 perf-profile.children.cycles-pp.__lxstat64
0.01 ±173% +0.0 0.06 perf-profile.children.cycles-pp.open64
0.06 ± 17% +0.1 0.11 ± 40% perf-profile.children.cycles-pp.rebalance_domains
0.03 ±102% +0.1 0.09 ± 17% perf-profile.children.cycles-pp.ring_buffer_event_data
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__waitpid
0.09 ± 37% +0.1 0.16 ± 25% perf-profile.children.cycles-pp.ktime_get
0.00 +0.1 0.07 perf-profile.children.cycles-pp.semop
0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.link
0.01 ±173% +0.1 0.09 ± 35% perf-profile.children.cycles-pp.tick_irq_enter
0.04 ±110% +0.1 0.12 ± 32% perf-profile.children.cycles-pp.irq_enter
0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.exit
0.01 ±173% +0.1 0.11 ± 36% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.sighandler
0.07 ±105% +0.1 0.18 ± 23% perf-profile.children.cycles-pp.tracing_generic_entry_update
0.00 +0.1 0.14 ± 3% perf-profile.children.cycles-pp.expm1
0.00 +0.1 0.14 ± 7% perf-profile.children.cycles-pp.aim_system
0.00 +0.1 0.14 ± 5% perf-profile.children.cycles-pp.mem_rtns_1
0.02 ±173% +0.2 0.18 ± 34% perf-profile.children.cycles-pp.tick_nohz_next_event
0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.fcal2
0.14 ± 77% +0.2 0.33 ± 28% perf-profile.children.cycles-pp.rb_commit
0.00 +0.2 0.20 ± 6% perf-profile.children.cycles-pp.fcal1
0.03 ±173% +0.2 0.23 ± 35% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.2 0.21 ± 12% perf-profile.children.cycles-pp.fcal15
0.42 ± 2% +0.2 0.63 ± 2% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.14 ± 77% +0.2 0.37 ± 25% perf-profile.children.cycles-pp.__rb_reserve_next
0.00 +0.2 0.23 ± 9% perf-profile.children.cycles-pp.fcal0
0.19 ± 71% +0.3 0.45 ± 27% perf-profile.children.cycles-pp.trace_clock_x86_tsc
0.00 +0.3 0.26 ± 4% perf-profile.children.cycles-pp.exec_test
0.00 +0.3 0.29 ± 3% perf-profile.children.cycles-pp.kill
0.14 ± 70% +0.3 0.43 ± 35% perf-profile.children.cycles-pp.menu_select
0.21 ± 78% +0.3 0.52 ± 27% perf-profile.children.cycles-pp.ring_buffer_unlock_commit
0.26 ± 78% +0.4 0.65 ± 26% perf-profile.children.cycles-pp.trace_buffer_unlock_commit_regs
0.04 ± 58% +0.4 0.46 perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.05 ± 62% +0.5 0.51 ± 5% perf-profile.children.cycles-pp.wait
0.00 +0.5 0.47 ± 5% perf-profile.children.cycles-pp.page_test
0.00 +0.5 0.48 ± 3% perf-profile.children.cycles-pp.fun_cal15
0.00 +0.6 0.58 ± 6% perf-profile.children.cycles-pp.div_int
0.00 +0.6 0.59 ± 7% perf-profile.children.cycles-pp.close
0.00 +0.6 0.60 ± 3% perf-profile.children.cycles-pp.unlink
0.00 +0.6 0.63 ± 6% perf-profile.children.cycles-pp.div_short
0.00 +0.8 0.75 ± 18% perf-profile.children.cycles-pp.creat
0.00 +0.8 0.84 ± 5% perf-profile.children.cycles-pp.add_short
0.00 +1.1 1.15 ± 5% perf-profile.children.cycles-pp.add_long
0.00 +1.2 1.22 ± 5% perf-profile.children.cycles-pp.add_int
0.00 +1.8 1.82 perf-profile.children.cycles-pp.lseek64
0.00 +1.8 1.84 ± 5% perf-profile.children.cycles-pp.string_rtns_1
0.00 +1.9 1.86 ± 4% perf-profile.children.cycles-pp.div_long
0.00 +2.5 2.52 ± 5% perf-profile.children.cycles-pp.div_float
0.00 +2.7 2.67 ± 4% perf-profile.children.cycles-pp.div_double
0.93 ± 5% +2.8 3.70 ± 4% perf-profile.children.cycles-pp.__libc_fork
0.00 +3.1 3.10 ± 4% perf-profile.children.cycles-pp.fork_test
0.00 +3.3 3.25 ± 4% perf-profile.children.cycles-pp.add_float
0.00 +3.8 3.77 ± 4% perf-profile.children.cycles-pp.mul_double
0.00 +4.0 4.05 ± 6% perf-profile.children.cycles-pp.array_rtns
0.00 +4.1 4.13 ± 6% perf-profile.children.cycles-pp.mul_float
0.14 ± 3% +4.3 4.43 perf-profile.children.cycles-pp.read
0.04 ± 57% +4.5 4.50 ± 3% perf-profile.children.cycles-pp.brk
0.00 +4.8 4.85 ± 6% perf-profile.children.cycles-pp.add_double
0.00 +6.1 6.12 ± 5% perf-profile.children.cycles-pp.sieve
0.00 +13.1 13.10 perf-profile.children.cycles-pp.write
14.49 ± 8% -4.7 9.77 ± 22% perf-profile.self.cycles-pp.intel_idle
3.70 ± 2% -2.0 1.73 ± 3% perf-profile.self.cycles-pp.syscall_return_via_sysret
2.85 ± 6% -2.0 0.89 ± 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
3.33 ± 3% -1.8 1.57 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64
3.07 ± 10% -1.6 1.47 ± 2% perf-profile.self.cycles-pp.filemap_map_pages
2.43 ± 2% -1.5 0.92 ± 5% perf-profile.self.cycles-pp.unmap_page_range
2.58 -1.2 1.35 ± 2% perf-profile.self.cycles-pp.shmem_write_end
2.27 ± 21% -1.1 1.21 ± 12% perf-profile.self.cycles-pp.osq_lock
1.66 -1.0 0.69 ± 4% perf-profile.self.cycles-pp.release_pages
1.94 -0.9 1.02 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.11 -0.7 0.43 ± 2% perf-profile.self.cycles-pp.page_remove_rmap
1.00 ± 4% -0.7 0.33 ± 6% perf-profile.self.cycles-pp.copy_p4d_range
1.06 ± 3% -0.6 0.47 ± 5% perf-profile.self.cycles-pp.___might_sleep
1.21 ± 3% -0.5 0.67 ± 2% perf-profile.self.cycles-pp.ftrace_syscall_enter
1.06 -0.5 0.55 ± 3% perf-profile.self.cycles-pp.ftrace_syscall_exit
0.79 ± 4% -0.5 0.33 ± 7% perf-profile.self.cycles-pp.down_write
0.85 ± 2% -0.4 0.44 ± 2% perf-profile.self.cycles-pp.anon_vma_interval_tree_insert
0.80 -0.4 0.39 ± 3% perf-profile.self.cycles-pp.find_get_entry
0.72 ± 3% -0.4 0.31 ± 5% perf-profile.self.cycles-pp.up_write
0.76 ± 7% -0.4 0.38 ± 6% perf-profile.self.cycles-pp.rwsem_spin_on_owner
0.65 ± 3% -0.4 0.28 perf-profile.self.cycles-pp.__slab_free
0.70 ± 3% -0.3 0.35 perf-profile.self.cycles-pp._raw_spin_lock
0.57 ± 2% -0.3 0.24 ± 6% perf-profile.self.cycles-pp.free_pages_and_swap_cache
0.62 ± 8% -0.3 0.29 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid
0.63 ± 4% -0.3 0.30 ± 3% perf-profile.self.cycles-pp.shmem_getpage_gfp
0.62 -0.3 0.31 ± 5% perf-profile.self.cycles-pp.selinux_file_permission
0.60 ± 2% -0.3 0.29 ± 4% perf-profile.self.cycles-pp.unlock_page
0.67 -0.3 0.37 perf-profile.self.cycles-pp.clear_page_erms
0.77 ± 4% -0.3 0.47 ± 9% perf-profile.self.cycles-pp.syscall_trace_enter
0.53 ± 8% -0.3 0.25 ± 3% perf-profile.self.cycles-pp.__fget_light
0.55 ± 2% -0.3 0.27 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret
0.52 ± 3% -0.3 0.24 ± 3% perf-profile.self.cycles-pp.__might_sleep
0.53 -0.3 0.25 ± 3% perf-profile.self.cycles-pp.select_task_rq_fair
0.52 ± 3% -0.3 0.26 ± 5% perf-profile.self.cycles-pp.copy_page
0.51 ± 5% -0.3 0.26 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist
0.45 ± 2% -0.2 0.20 ± 2% perf-profile.self.cycles-pp.__handle_mm_fault
0.47 ± 4% -0.2 0.23 perf-profile.self.cycles-pp.xas_load
0.46 ± 3% -0.2 0.22 ± 8% perf-profile.self.cycles-pp.fsnotify
0.41 ± 3% -0.2 0.18 ± 2% perf-profile.self.cycles-pp.native_flush_tlb_one_user
0.46 ± 3% -0.2 0.23 ± 3% perf-profile.self.cycles-pp.vma_interval_tree_insert
0.52 -0.2 0.29 ± 6% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.38 ± 15% -0.2 0.16 ± 5% perf-profile.self.cycles-pp.avc_has_perm_noaudit
0.40 ± 2% -0.2 0.17 ± 4% perf-profile.self.cycles-pp.anon_vma_interval_tree_remove
0.43 ± 7% -0.2 0.21 ± 3% perf-profile.self.cycles-pp.free_pcppages_bulk
0.40 ± 2% -0.2 0.18 ± 2% perf-profile.self.cycles-pp.mark_page_accessed
0.48 ± 7% -0.2 0.27 ± 5% perf-profile.self.cycles-pp.syscall_slow_exit_work
0.36 ± 2% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.find_vma
0.36 -0.2 0.17 ± 4% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.36 -0.2 0.17 ± 2% perf-profile.self.cycles-pp._dl_addr
0.35 ± 19% -0.2 0.17 ± 4% perf-profile.self.cycles-pp.rwsem_down_write_failed
0.33 -0.2 0.14 ± 5% perf-profile.self.cycles-pp.copy_process
0.39 ± 5% -0.2 0.21 ± 3% perf-profile.self.cycles-pp.update_cfs_group
0.33 ± 4% -0.2 0.15 ± 5% perf-profile.self.cycles-pp.__vm_enough_memory
0.39 ± 4% -0.2 0.21 ± 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.34 ± 6% -0.2 0.16 ± 2% perf-profile.self.cycles-pp.page_mapping
0.33 ± 5% -0.2 0.15 ± 5% perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.30 -0.2 0.12 ± 6% perf-profile.self.cycles-pp.__vma_adjust
0.33 -0.2 0.16 ± 5% perf-profile.self.cycles-pp.kmem_cache_alloc
0.31 -0.2 0.14 ± 5% perf-profile.self.cycles-pp.perf_iterate_sb
0.31 ± 3% -0.2 0.15 ± 10% perf-profile.self.cycles-pp.perf_event_mmap
0.29 -0.2 0.12 ± 8% perf-profile.self.cycles-pp.vma_interval_tree_remove
0.32 ± 2% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.generic_perform_write
0.23 ± 8% -0.2 0.08 perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.30 ± 5% -0.2 0.14 ± 3% perf-profile.self.cycles-pp.generic_file_write_iter
0.29 ± 3% -0.2 0.13 ± 8% perf-profile.self.cycles-pp.vfs_write
0.25 ± 4% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.vma_compute_subtree_gap
0.27 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.shmem_file_read_iter
0.26 ± 3% -0.1 0.12 ± 5% perf-profile.self.cycles-pp._cond_resched
0.25 ± 3% -0.1 0.11 ± 6% perf-profile.self.cycles-pp.__alloc_pages_nodemask
0.26 ± 6% -0.1 0.12 ± 10% perf-profile.self.cycles-pp.do_brk_flags
0.26 -0.1 0.12 ± 3% perf-profile.self.cycles-pp.find_lock_entry
0.24 ± 4% -0.1 0.11 ± 3% perf-profile.self.cycles-pp.iov_iter_fault_in_readable
0.23 -0.1 0.11 ± 4% perf-profile.self.cycles-pp.alloc_set_pte
0.21 ± 11% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.__sb_start_write
0.24 ± 5% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.xas_start
0.23 -0.1 0.11 ± 3% perf-profile.self.cycles-pp.perf_event_mmap_output
0.26 ± 4% -0.1 0.15 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.24 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.iov_iter_copy_from_user_atomic
0.23 ± 3% -0.1 0.12 ± 7% perf-profile.self.cycles-pp.page_add_file_rmap
0.23 ± 2% -0.1 0.11 ± 6% perf-profile.self.cycles-pp.new_sync_write
0.19 ± 3% -0.1 0.08 ± 10% perf-profile.self.cycles-pp.refcount_dec_not_one
0.23 ± 3% -0.1 0.11 ± 9% perf-profile.self.cycles-pp.vmacache_find
0.22 ± 3% -0.1 0.11 ± 3% perf-profile.self.cycles-pp.___perf_sw_event
0.24 ± 3% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.update_sd_lb_stats
0.21 ± 4% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.rcu_all_qs
0.20 ± 2% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent
0.19 ± 4% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.__rb_insert_augmented
0.19 ± 6% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.__x64_sys_brk
0.19 ± 3% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.__mod_node_page_state
0.19 ± 10% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.__do_page_fault
0.19 ± 4% -0.1 0.09 perf-profile.self.cycles-pp.anon_vma_clone
0.24 -0.1 0.14 ± 5% perf-profile.self.cycles-pp.trace_buffer_lock_reserve
0.17 ± 4% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.anon_vma_fork
0.18 ± 2% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.unlink_anon_vmas
0.18 ± 6% -0.1 0.09 perf-profile.self.cycles-pp.update_curr
0.13 ± 9% -0.1 0.04 ± 57% perf-profile.self.cycles-pp._vm_normal_page
0.17 ± 3% -0.1 0.07 ± 15% perf-profile.self.cycles-pp.__set_page_dirty_no_writeback
0.17 ± 3% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.file_update_time
0.17 ± 4% -0.1 0.08 perf-profile.self.cycles-pp.kmem_cache_free
0.18 ± 7% -0.1 0.09 ± 8% perf-profile.self.cycles-pp.shmem_write_begin
0.11 ± 7% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rcu_cblist_dequeue
0.17 ± 4% -0.1 0.08 perf-profile.self.cycles-pp.__inode_security_revalidate
0.16 ± 4% -0.1 0.07 ± 11% perf-profile.self.cycles-pp.queued_read_lock_slowpath
0.17 ± 4% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.handle_mm_fault
0.17 ± 6% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.__generic_file_write_iter
0.15 ± 2% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.selinux_inode_permission
0.15 ± 8% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.__fdget_pos
0.11 ± 6% -0.1 0.03 ±100% perf-profile.self.cycles-pp.lock_page_memcg
0.17 ± 6% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.ksys_write
0.12 -0.1 0.04 ± 57% perf-profile.self.cycles-pp.shmem_alloc_and_acct_page
0.17 ± 8% -0.1 0.09 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.15 ± 4% -0.1 0.07 ± 11% perf-profile.self.cycles-pp.__d_lookup_rcu
0.14 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.arch_get_unmapped_area_topdown
0.15 ± 4% -0.1 0.07 ± 11% perf-profile.self.cycles-pp.__alloc_file
0.15 ± 2% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.__list_add_valid
0.10 ± 12% -0.1 0.03 ±100% perf-profile.self.cycles-pp.osq_unlock
0.14 ± 5% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.current_time
0.13 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.__x64_sys_write
0.11 -0.1 0.04 ± 57% perf-profile.self.cycles-pp.shmem_add_to_page_cache
0.15 ± 5% -0.1 0.08 ± 10% perf-profile.self.cycles-pp.task_tick_fair
0.14 ± 5% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.security_file_permission
0.13 ± 9% -0.1 0.06 ± 11% perf-profile.self.cycles-pp.new_sync_read
0.10 ± 5% -0.1 0.03 ±100% perf-profile.self.cycles-pp.down_write_killable
0.10 ± 11% -0.1 0.03 ±100% perf-profile.self.cycles-pp.try_charge
0.11 ± 6% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.generic_write_check_limits
0.10 ± 5% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rw_verify_area
0.15 ± 8% -0.1 0.08 perf-profile.self.cycles-pp.iov_iter_advance
0.13 ± 3% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.xas_store
0.13 ± 6% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.selinux_vm_enough_memory
0.12 ± 5% -0.1 0.06 ± 9% perf-profile.self.cycles-pp.strnlen_user
0.10 ± 8% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.memset_erms
0.09 ± 8% -0.1 0.03 ±100% perf-profile.self.cycles-pp.strlen
0.13 ± 6% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.cred_has_capability
0.12 ± 5% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.vm_area_dup
0.12 ± 5% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.__rb_erase_color
0.09 ± 9% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.12 ± 4% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.vma_merge
0.12 ± 3% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.cap_vm_enough_memory
0.12 ± 3% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.mem_cgroup_charge_statistics
0.12 ± 12% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited
0.11 ± 6% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.vfs_read
0.13 ± 9% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.find_get_entries
0.11 ± 4% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.__schedule
0.11 ± 10% -0.1 0.05 ± 9% perf-profile.self.cycles-pp.copy_page_to_iter
0.08 ± 6% -0.1 0.03 ±100% perf-profile.self.cycles-pp.available_idle_cpu
0.10 ± 8% -0.0 0.05 perf-profile.self.cycles-pp.atime_needs_update
0.10 ± 8% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.set_page_dirty
0.10 ± 5% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.update_rq_clock
0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.ring_buffer_event_data
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.read
0.04 ±102% +0.1 0.11 ± 24% perf-profile.self.cycles-pp.ktime_get
0.05 ±114% +0.1 0.13 ± 39% perf-profile.self.cycles-pp.menu_select
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.page_test
0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.brk
0.04 ±104% +0.1 0.13 ± 29% perf-profile.self.cycles-pp.trace_buffer_unlock_commit_regs
0.02 ±173% +0.1 0.14 ± 46% perf-profile.self.cycles-pp.cpuidle_enter_state
0.06 ±106% +0.1 0.19 ± 23% perf-profile.self.cycles-pp.ring_buffer_unlock_commit
0.00 +0.1 0.14 ± 3% perf-profile.self.cycles-pp.expm1
0.00 +0.1 0.14 ± 7% perf-profile.self.cycles-pp.mem_rtns_1
0.00 +0.1 0.14 ± 3% perf-profile.self.cycles-pp.write
0.00 +0.2 0.16 ± 13% perf-profile.self.cycles-pp.fcal2
0.00 +0.2 0.17 ± 5% perf-profile.self.cycles-pp.fcal1
0.00 +0.2 0.18 ± 10% perf-profile.self.cycles-pp.fcal15
0.13 ± 76% +0.2 0.32 ± 29% perf-profile.self.cycles-pp.rb_commit
0.00 +0.2 0.20 ± 11% perf-profile.self.cycles-pp.fcal0
0.14 ± 76% +0.2 0.36 ± 25% perf-profile.self.cycles-pp.__rb_reserve_next
0.18 ± 72% +0.2 0.42 ± 26% perf-profile.self.cycles-pp.trace_clock_x86_tsc
0.00 +0.4 0.43 ± 2% perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.00 +0.5 0.46 ± 4% perf-profile.self.cycles-pp.fun_cal15
0.00 +0.6 0.57 ± 5% perf-profile.self.cycles-pp.div_int
0.00 +0.6 0.62 ± 5% perf-profile.self.cycles-pp.div_short
0.00 +0.8 0.83 ± 5% perf-profile.self.cycles-pp.add_short
0.00 +1.1 1.13 ± 5% perf-profile.self.cycles-pp.add_long
0.00 +1.2 1.20 ± 6% perf-profile.self.cycles-pp.add_int
0.00 +1.8 1.80 ± 5% perf-profile.self.cycles-pp.string_rtns_1
0.00 +1.8 1.84 ± 4% perf-profile.self.cycles-pp.div_long
0.00 +2.5 2.48 ± 5% perf-profile.self.cycles-pp.div_float
0.00 +2.6 2.63 ± 4% perf-profile.self.cycles-pp.div_double
0.00 +3.2 3.21 ± 4% perf-profile.self.cycles-pp.add_float
0.00 +3.7 3.72 ± 4% perf-profile.self.cycles-pp.mul_double
0.82 ± 3% +3.8 4.64 ± 3% perf-profile.self.cycles-pp.do_syscall_64
0.00 +4.0 4.00 ± 6% perf-profile.self.cycles-pp.array_rtns
0.00 +4.1 4.08 ± 6% perf-profile.self.cycles-pp.mul_float
0.00 +4.8 4.78 ± 6% perf-profile.self.cycles-pp.add_double
0.00 +6.0 6.03 ± 5% perf-profile.self.cycles-pp.sieve
reaim.time.user_time
34000 +-+-------------------------O-----O----------------O----------------+
O O O O O O O O O O
33500 +-+ |
33000 +-+ |
| |
32500 +-+ |
32000 +-+ |
| |
31500 +-+ |
31000 +-+ |
| |
30500 +-+ |
30000 +-+ |
| ...+....+.....+.....+....+.....+.....+....+.....+.....+....+.....|
29500 +-+-----------------------------------------------------------------+
reaim.time.elapsed_time
600 +-+-------------------------------------------------------------------+
| O |
580 O-+ |
560 +-+ |
| |
540 +-+ |
| O O O O O O
520 +-+ O O O O O |
| |
500 +-+ |
480 +-+ .+.. |
|..... ... .. ...+..... |
460 +-+ +. . ...+..... ...+.. +....+.....+.....|
| +....+.. +.. |
440 +-+-------------------------------------------------------------------+
reaim.time.elapsed_time.max
600 +-+-------------------------------------------------------------------+
| O |
580 O-+ |
560 +-+ |
| |
540 +-+ |
| O O O O O O
520 +-+ O O O O O |
| |
500 +-+ |
480 +-+ .+.. |
|..... ... .. ...+..... |
460 +-+ +. . ...+..... ...+.. +....+.....+.....|
| +....+.. +.. |
440 +-+-------------------------------------------------------------------+
reaim.parent_time
26 O-+--------------------------------------------------------------------+
| |
25 +-+ |
| O |
| O O O
24 +-+ O O O O O O O O |
| |
23 +-+ |
| |
22 +-+ |
| .+.. |
| ...+.... ... .. ...+.....|
21 +-+...+.. . ...+..... ..+. . ...+.. |
| +.....+.. +.. +.. |
20 +-+--------------------------------------------------------------------+
reaim.child_utime
1700 +-+--------------------------O----------------------O----------------+
O O O O O O O O O O O
| |
1650 +-+ |
| |
| |
1600 +-+ |
| |
1550 +-+ |
| |
| |
1500 +-+ |
|.....+.....+....+.....+.....+.....+....+.....+.....+.....+....+.....|
| |
1450 +-+------------------------------------------------------------------+
reaim.jobs_per_min
820000 +-+----------------------------------------------------------------+
| .+.. . ...+... +.... |
800000 +-+...+.... ... +.. . .. . |
780000 +-+ +. +..... .. +....+.....|
| + |
760000 +-+ |
| |
740000 +-+ |
| |
720000 +-+ |
700000 +-+ O O |
| O O O O O O O |
680000 +-+ O O
| O |
660000 O-+----------------------------------------------------------------+
reaim.jobs_per_min_child
360 +-+-------------------------------------------------------------------+
| .+....+.... |
350 +-+ .. . ...+..... .+.... |
340 +-+...+..... .. +.. +..... ... +.....+.....|
| + +. |
330 +-+ |
| |
320 +-+ |
| |
310 +-+ |
300 +-+ O O O O O O O |
| O O O O
290 +-+ O |
O |
280 +-+-------------------------------------------------------------------+
reaim.max_jobs_per_min
860000 +-+----------------------------------------------------------------+
|..... .+....+..... |
840000 +-+ +... .. +.....+.... ...+....+..... ..+.....|
820000 +-+ . .. +.. +.. |
| + |
800000 +-+ |
| |
780000 +-+ |
| |
760000 O-+ |
740000 +-+ O O O |
| O O O |
720000 +-+ O O O O O
| O |
700000 +-+----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 7 months