Greeting, FYI, we noticed a -33.6% regression of will-it-scale.per_process_ops due to commit: commit: 0f12156dff2862ac54235fc72703f18770769042 ("memcg: enable accounting for file lock caches") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 104 threads 2 sockets Skylake with 192G memory with following parameters: nr_task: 50% mode: process test: lock1 cpufreq_governor: performance ucode: 0x2006a0a test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -40.9% regression | | test machine | 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=50% | | | test=lock1 | | | ucode=0xd000280 | +------------------+---------------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com&gt; Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run bin/lkp run generated-yaml-file ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/lock1/will-it-scale/0x2006a0a commit: b655843444 ("memcg: enable accounting for pollfd and select bits arrays") 0f12156dff ("memcg: enable accounting for file lock caches") b655843444152c0a 0f12156dff2862ac54235fc7270 ---------------- --------------------------- %stddev %change %stddev \ | \ 65855410 -33.6% 43722413 ± 5% will-it-scale.52.processes 1266449 -33.6% 840815 ± 5% will-it-scale.per_process_ops 65855410 -33.6% 43722413 ± 5% will-it-scale.workload 141099 ± 3% +8.2% 152680 ± 2% meminfo.Active 140875 ± 3% +8.2% 152456 ± 2% meminfo.Active(anon) 28.79 +6.7 35.47 mpstat.cpu.all.sys% 20.49 -6.7 13.76 ± 5% mpstat.cpu.all.usr% 138801 ± 3% +8.4% 150456 ± 2% numa-meminfo.node1.Active 138689 ± 3% +8.4% 150381 ± 2% numa-meminfo.node1.Active(anon) 34681 ± 3% +8.3% 37570 ± 2% numa-vmstat.node1.nr_active_anon 34681 ± 3% +8.3% 37570 ± 2% numa-vmstat.node1.nr_zone_active_anon 4480 ± 3% -85.9% 632.67 ± 12% slabinfo.Acpi-Parse.active_objs 4480 ± 3% -85.9% 632.67 ± 12% slabinfo.Acpi-Parse.num_objs 2957456 ±214% -96.5% 104939 ± 5% turbostat.C1 103.99 +5.8% 110.04 turbostat.RAMWatt 19.83 -34.5% 13.00 ± 4% vmstat.cpu.us 2487 ± 2% +4.4% 2596 vmstat.system.cs 35219 ± 3% +8.2% 38114 ± 2% proc-vmstat.nr_active_anon 10977 +1.8% 11171 proc-vmstat.nr_mapped 40640 ± 3% +7.6% 43745 ± 2% proc-vmstat.nr_shmem 35219 ± 3% +8.2% 38114 ± 2% proc-vmstat.nr_zone_active_anon 53061 ± 3% +6.5% 56488 ± 3% proc-vmstat.pgactivate 0.02 ± 19% -24.0% 0.01 ± 18% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 9.26 ± 21% -39.8% 5.58 ± 14% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 172.83 ± 11% -29.4% 122.00 ± 5% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 127.50 ± 4% +47.8% 188.50 ± 7% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 1080 ± 19% +66.6% 1799 ± 13% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 7204 ± 3% -12.0% 6337 perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7204 ± 3% -12.0% 6337 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7209 ± 3% -12.0% 6342 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 6825 ± 4% -11.5% 6036 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 9.26 ± 21% -39.8% 5.57 ± 14% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 7204 ± 3% -12.0% 6337 perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7204 ± 3% -12.0% 6337 perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 0.05 ± 7% +751.3% 0.40 ±180% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 7209 ± 3% -12.0% 6342 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 6825 ± 4% -11.5% 6036 perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 827.67 ± 9% -16.3% 692.50 ± 4% interrupts.CPU13.CAL:Function_call_interrupts 3623 ± 54% +66.8% 6045 ± 33% interrupts.CPU26.NMI:Non-maskable_interrupts 3623 ± 54% +66.8% 6045 ± 33% interrupts.CPU26.PMI:Performance_monitoring_interrupts 239.50 ± 18% -33.1% 160.33 ± 38% interrupts.CPU41.RES:Rescheduling_interrupts 261.33 ± 18% -54.4% 119.17 ± 49% interrupts.CPU42.RES:Rescheduling_interrupts 7894 -30.2% 5511 ± 37% interrupts.CPU45.NMI:Non-maskable_interrupts 7894 -30.2% 5511 ± 37% interrupts.CPU45.PMI:Performance_monitoring_interrupts 3374 ± 40% +90.6% 6432 ± 23% interrupts.CPU58.NMI:Non-maskable_interrupts 3374 ± 40% +90.6% 6432 ± 23% interrupts.CPU58.PMI:Performance_monitoring_interrupts 272.83 ± 8% -36.4% 173.50 ± 40% interrupts.CPU6.RES:Rescheduling_interrupts 2890 ± 37% +61.0% 4652 ± 37% interrupts.CPU76.NMI:Non-maskable_interrupts 2890 ± 37% +61.0% 4652 ± 37% interrupts.CPU76.PMI:Performance_monitoring_interrupts 272.67 ± 9% -29.0% 193.67 ± 27% interrupts.CPU81.RES:Rescheduling_interrupts 3475 ± 47% +109.3% 7273 ± 12% interrupts.CPU93.NMI:Non-maskable_interrupts 3475 ± 47% +109.3% 7273 ± 12% interrupts.CPU93.PMI:Performance_monitoring_interrupts 3054 ± 37% +91.7% 5857 ± 38% interrupts.CPU96.NMI:Non-maskable_interrupts 3054 ± 37% +91.7% 5857 ± 38% interrupts.CPU96.PMI:Performance_monitoring_interrupts 0.05 ± 5% +1601.9% 0.79 ± 20% perf-stat.i.MPKI 12.52 ± 2% +20.9 33.42 ± 4% perf-stat.i.cache-miss-rate% 647128 ± 2% +4855.0% 32065121 ± 8% perf-stat.i.cache-misses 5130146 ± 2% +1777.3% 96305803 ± 9% perf-stat.i.cache-references 2302 +6.1% 2443 perf-stat.i.context-switches 259518 ± 2% -98.1% 4827 ± 6% perf-stat.i.cycles-between-cache-misses 0.17 -0.1 0.11 perf-stat.i.dTLB-load-miss-rate% 65644478 -33.6% 43563819 ± 5% perf-stat.i.dTLB-load-misses 51892 -27.4% 37675 ± 3% perf-stat.i.dTLB-store-misses 68181011 -33.4% 45405593 ± 5% perf-stat.i.iTLB-load-misses 1933 +58.2% 3058 ± 3% perf-stat.i.instructions-per-iTLB-miss 81.87 ± 12% +782.2% 722.30 ± 16% perf-stat.i.metric.K/sec 126326 ± 3% +6119.4% 7856755 ± 2% perf-stat.i.node-load-misses 26893 ± 6% +3382.5% 936567 ± 3% perf-stat.i.node-loads 87.04 +12.8 99.79 perf-stat.i.node-store-miss-rate% 33838 ± 3% +28351.8% 9627739 perf-stat.i.node-store-misses 0.04 ± 2% +1709.7% 0.71 ± 14% perf-stat.overall.MPKI 0.52 -0.0 0.49 ± 3% perf-stat.overall.branch-miss-rate% 12.65 ± 2% +20.7 33.35 ± 4% perf-stat.overall.cache-miss-rate% 222708 ± 2% -98.0% 4560 ± 8% perf-stat.overall.cycles-between-cache-misses 0.17 -0.1 0.10 perf-stat.overall.dTLB-load-miss-rate% 0.00 -0.0 0.00 ± 2% perf-stat.overall.dTLB-store-miss-rate% 1929 +56.0% 3010 perf-stat.overall.instructions-per-iTLB-miss 82.22 +7.1 89.34 perf-stat.overall.node-load-miss-rate% 78.75 +21.2 99.91 perf-stat.overall.node-store-miss-rate% 603681 +56.8% 946452 perf-stat.overall.path-length 649626 ± 2% +4816.5% 31938893 ± 8% perf-stat.ps.cache-misses 5137402 ± 2% +1767.5% 95941923 ± 9% perf-stat.ps.cache-references 2313 +5.8% 2448 perf-stat.ps.context-switches 65424262 -33.6% 43419292 ± 5% perf-stat.ps.dTLB-load-misses 51749 -27.4% 37573 ± 3% perf-stat.ps.dTLB-store-misses 67951468 -33.4% 45252180 ± 5% perf-stat.ps.iTLB-load-misses 126213 ± 3% +6102.7% 7828694 ± 2% perf-stat.ps.node-load-misses 27281 ± 6% +3321.9% 933553 ± 3% perf-stat.ps.node-loads 33783 ± 3% +28302.8% 9595407 perf-stat.ps.node-store-misses 15207 ± 10% -34.0% 10039 ± 12% softirqs.CPU0.RCU 16968 ± 17% -36.3% 10810 ± 19% softirqs.CPU1.RCU 17526 ± 19% -36.4% 11142 ± 14% softirqs.CPU10.RCU 17232 ± 16% -48.1% 8946 ± 13% softirqs.CPU100.RCU 15770 ± 16% -43.7% 8874 ± 28% softirqs.CPU101.RCU 14396 ± 15% -44.6% 7969 ± 13% softirqs.CPU102.RCU 18909 ± 5% -46.4% 10130 ± 17% softirqs.CPU103.RCU 17822 ± 23% -45.3% 9744 ± 22% softirqs.CPU11.RCU 18056 ± 22% -52.5% 8582 ± 22% softirqs.CPU12.RCU 14576 ± 17% -33.3% 9723 ± 15% softirqs.CPU13.RCU 18836 ± 15% -47.1% 9961 ± 8% softirqs.CPU14.RCU 20542 ± 17% -54.3% 9394 ± 11% softirqs.CPU15.RCU 17485 ± 19% -42.0% 10141 ± 19% softirqs.CPU16.RCU 17387 ± 17% -37.8% 10807 ± 13% softirqs.CPU17.RCU 18473 ± 15% -49.9% 9250 ± 18% softirqs.CPU18.RCU 19232 ± 19% -44.1% 10751 ± 10% softirqs.CPU19.RCU 17052 ± 21% -42.9% 9744 ± 22% softirqs.CPU2.RCU 17330 ± 17% -43.5% 9797 ± 16% softirqs.CPU20.RCU 18624 ± 15% -43.9% 10445 ± 24% softirqs.CPU21.RCU 18206 ± 18% -44.4% 10123 ± 26% softirqs.CPU22.RCU 18047 ± 12% -44.5% 10022 ± 23% softirqs.CPU23.RCU 19351 ± 18% -47.7% 10115 ± 21% softirqs.CPU24.RCU 18629 ± 17% -51.4% 9057 ± 18% softirqs.CPU25.RCU 15355 ± 14% -44.8% 8480 ± 13% softirqs.CPU26.RCU 15129 ± 14% -39.7% 9117 ± 16% softirqs.CPU27.RCU 14744 ± 10% -39.0% 8996 ± 21% softirqs.CPU28.RCU 13973 ± 11% -39.4% 8470 ± 18% softirqs.CPU29.RCU 35779 ± 10% -27.5% 25930 ± 30% softirqs.CPU29.SCHED 18703 ± 16% -43.4% 10577 ± 19% softirqs.CPU3.RCU 17000 ± 19% -40.8% 10057 ± 21% softirqs.CPU30.RCU 18602 ± 16% -40.6% 11040 ± 15% softirqs.CPU31.RCU 17242 ± 23% -44.0% 9662 ± 19% softirqs.CPU32.RCU 17841 ± 15% -43.1% 10144 ± 21% softirqs.CPU33.RCU 17867 ± 14% -44.6% 9890 ± 21% softirqs.CPU34.RCU 19083 ± 16% -46.7% 10177 ± 15% softirqs.CPU35.RCU 19161 ± 11% -49.8% 9616 ± 24% softirqs.CPU36.RCU 19335 ± 15% -47.7% 10106 ± 16% softirqs.CPU37.RCU 20460 ± 11% -54.3% 9341 ± 17% softirqs.CPU38.RCU 10641 ± 58% +117.0% 23087 ± 43% softirqs.CPU38.SCHED 20165 ± 8% -53.5% 9381 ± 17% softirqs.CPU39.RCU 20158 ± 9% -47.3% 10630 ± 20% softirqs.CPU4.RCU 18503 ± 20% -40.7% 10980 ± 20% softirqs.CPU40.RCU 18917 ± 11% -51.1% 9241 ± 17% softirqs.CPU41.RCU 19989 ± 9% -57.3% 8544 ± 12% softirqs.CPU42.RCU 10393 ± 58% +155.8% 26585 ± 30% softirqs.CPU42.SCHED 16499 ± 11% -45.5% 8991 ± 19% softirqs.CPU43.RCU 18599 ± 16% -49.3% 9433 ± 24% softirqs.CPU44.RCU 20721 ± 8% -56.4% 9035 ± 14% softirqs.CPU45.RCU 6210 ± 26% +213.4% 19464 ± 52% softirqs.CPU45.SCHED 16912 ± 21% -43.6% 9547 ± 24% softirqs.CPU46.RCU 19085 ± 14% -49.9% 9571 ± 15% softirqs.CPU47.RCU 16524 ± 21% -46.7% 8815 ± 20% softirqs.CPU48.RCU 18590 ± 13% -52.0% 8921 ± 11% softirqs.CPU49.RCU 19473 ± 15% -47.6% 10198 ± 12% softirqs.CPU5.RCU 19807 ± 16% -51.7% 9574 ± 20% softirqs.CPU50.RCU 14621 ± 7% -44.6% 8100 ± 14% softirqs.CPU51.RCU 21579 ± 9% -49.4% 10911 ± 16% softirqs.CPU52.RCU 17758 ± 25% -50.6% 8778 ± 25% softirqs.CPU53.RCU 18495 ± 15% -47.1% 9776 ± 18% softirqs.CPU54.RCU 16729 ± 20% -46.4% 8963 ± 20% softirqs.CPU55.RCU 15031 ± 10% -44.4% 8358 ± 18% softirqs.CPU56.RCU 15721 ± 17% -42.2% 9080 ± 30% softirqs.CPU57.RCU 14757 ± 3% -32.1% 10022 ± 23% softirqs.CPU58.RCU 37022 ± 9% -41.4% 21679 ± 41% softirqs.CPU58.SCHED 16077 ± 6% -41.1% 9466 ± 32% softirqs.CPU59.RCU 20064 ± 11% -54.4% 9150 ± 18% softirqs.CPU6.RCU 14811 ± 18% -46.8% 7880 ± 15% softirqs.CPU60.RCU 15191 ± 8% -49.8% 7623 ± 14% softirqs.CPU61.RCU 16592 ± 13% -49.0% 8455 ± 12% softirqs.CPU62.RCU 15392 ± 21% -44.3% 8566 ± 17% softirqs.CPU63.RCU 18606 ± 16% -52.5% 8837 ± 19% softirqs.CPU65.RCU 14770 ± 22% -40.4% 8802 ± 29% softirqs.CPU66.RCU 14465 ± 11% -32.3% 9797 ± 24% softirqs.CPU67.RCU 16993 ± 16% -48.1% 8818 ± 12% softirqs.CPU68.RCU 18034 ± 14% -52.9% 8497 ± 18% softirqs.CPU69.RCU 18628 ± 16% -47.9% 9700 ± 18% softirqs.CPU7.RCU 16564 ± 17% -42.4% 9545 ± 15% softirqs.CPU70.RCU 16333 ± 19% -45.3% 8933 ± 25% softirqs.CPU71.RCU 17985 ± 19% -52.0% 8624 ± 25% softirqs.CPU72.RCU 16575 ± 17% -50.7% 8172 ± 15% softirqs.CPU73.RCU 17243 ± 18% -48.5% 8875 ± 15% softirqs.CPU74.RCU 17293 ± 12% -44.8% 9547 ± 15% softirqs.CPU75.RCU 16269 ± 20% -46.7% 8673 ± 24% softirqs.CPU76.RCU 17326 ± 18% -45.8% 9383 ± 22% softirqs.CPU77.RCU 18826 ± 18% -50.6% 9292 ± 24% softirqs.CPU78.RCU 19893 ± 8% -52.4% 9460 ± 18% softirqs.CPU79.RCU 18812 ± 19% -43.6% 10616 ± 19% softirqs.CPU8.RCU 20001 ± 12% -53.0% 9393 ± 14% softirqs.CPU80.RCU 20756 ± 5% -53.9% 9562 ± 18% softirqs.CPU81.RCU 18840 ± 15% -53.5% 8761 ± 17% softirqs.CPU82.RCU 17159 ± 21% -52.3% 8189 ± 20% softirqs.CPU83.RCU 18671 ± 16% -52.4% 8881 ± 19% softirqs.CPU84.RCU 18929 ± 11% -55.5% 8421 ± 21% softirqs.CPU85.RCU 17559 ± 15% -51.0% 8604 ± 13% softirqs.CPU86.RCU 17343 ± 14% -53.0% 8154 ± 18% softirqs.CPU87.RCU 15481 ± 13% -43.6% 8727 ± 14% softirqs.CPU88.RCU 16364 ± 14% -43.9% 9187 ± 24% softirqs.CPU89.RCU 18492 ± 19% -43.5% 10441 ± 14% softirqs.CPU9.RCU 14056 ± 7% -38.3% 8666 ± 21% softirqs.CPU90.RCU 14189 ± 6% -37.9% 8815 ± 24% softirqs.CPU91.RCU 15154 ± 14% -48.1% 7867 ± 15% softirqs.CPU92.RCU 15589 ± 14% -42.4% 8979 ± 25% softirqs.CPU93.RCU 14492 ± 10% -34.4% 9504 ± 19% softirqs.CPU94.RCU 34198 ± 16% -52.1% 16384 ± 44% softirqs.CPU94.SCHED 17203 ± 7% -46.2% 9249 ± 16% softirqs.CPU95.RCU 15367 ± 11% -44.0% 8601 ± 12% softirqs.CPU96.RCU 14328 ± 13% -37.8% 8911 ± 27% softirqs.CPU97.RCU 16134 ± 16% -46.3% 8672 ± 19% softirqs.CPU98.RCU 15282 ± 17% -44.5% 8486 ± 18% softirqs.CPU99.RCU 1808149 ± 6% -46.3% 971467 ± 15% softirqs.RCU 36250 ± 5% +36.3% 49404 ± 2% softirqs.TIMER 17.47 ± 9% -5.9 11.61 ± 9% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 9.44 ± 9% -3.2 6.23 ± 9% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64 8.57 ± 9% -2.9 5.64 ± 9% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64 2.27 ± 9% -1.0 1.27 ± 29% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.__libc_fcntl64 2.68 ± 9% -1.0 1.70 ± 9% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 2.65 ± 8% -0.9 1.78 ± 9% perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 2.52 ± 9% -0.9 1.65 ± 9% perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.80 ± 8% -0.5 0.28 ±100% perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.34 ± 9% -0.5 0.83 ± 9% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.62 ± 9% -0.5 1.12 ± 10% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl 0.82 ± 10% -0.4 0.38 ± 71% perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.66 ± 11% -0.4 0.28 ±100% perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.04 ± 9% -0.3 0.69 ± 9% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.86 ± 9% -0.3 0.59 ± 8% perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +0.6 0.59 ± 11% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.06 ± 9% +0.9 1.92 ± 12% perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 0.00 +0.9 0.92 ± 10% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 0.67 ± 9% +1.0 1.68 ± 14% perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.4 1.38 ± 8% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +1.8 1.79 ± 59% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk 0.00 +2.2 2.18 ± 58% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl 0.00 +2.2 2.21 ± 58% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +2.4 2.35 ± 43% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode 0.00 +2.8 2.80 ± 59% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 0.00 +2.8 2.84 ± 41% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait 0.00 +2.9 2.86 ± 41% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 1.10 ± 9% +2.9 3.98 ± 32% perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.00 +3.0 2.99 ± 55% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.69 ± 9% +3.8 5.49 ± 21% perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 3.39 ± 8% +4.0 7.42 ± 22% perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 3.00 ± 8% +4.2 7.17 ± 23% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +4.6 4.61 ± 29% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 0.00 +5.0 5.00 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 6.48 ± 9% +6.3 12.76 ± 12% perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 5.71 ± 9% +6.5 12.21 ± 13% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +7.3 7.35 ± 35% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock 17.27 ± 9% +8.5 25.78 ± 11% perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 13.80 ± 9% +9.6 23.35 ± 12% perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 28.29 ± 9% +13.3 41.58 ± 12% perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 26.73 ± 9% +13.8 40.53 ± 12% perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 23.74 ± 9% +14.8 38.56 ± 13% perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.55 ± 9% -5.9 11.68 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 9.74 ± 9% -3.3 6.41 ± 9% perf-profile.children.cycles-pp.syscall_return_via_sysret 9.02 ± 8% -3.1 5.96 ± 9% perf-profile.children.cycles-pp.__entry_text_start 4.02 ± 9% -1.5 2.53 ± 9% perf-profile.children.cycles-pp.memset_erms 2.58 ± 9% -0.9 1.69 ± 9% perf-profile.children.cycles-pp._copy_from_user 2.68 ± 8% -0.9 1.80 ± 9% perf-profile.children.cycles-pp.security_file_lock 2.02 ± 7% -0.7 1.33 ± 9% perf-profile.children.cycles-pp._raw_spin_lock 1.40 ± 8% -0.6 0.85 ± 10% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 9% -0.5 1.16 ± 10% perf-profile.children.cycles-pp.common_file_perm 1.48 ± 9% -0.5 0.98 ± 9% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 1.23 ± 9% -0.4 0.81 ± 9% perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.04 ± 9% -0.4 0.67 ± 9% perf-profile.children.cycles-pp.__might_sleep 0.86 ± 10% -0.3 0.56 ± 9% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.86 ± 9% -0.3 0.59 ± 8% perf-profile.children.cycles-pp.__fget_light 0.76 ± 7% -0.3 0.50 ± 8% perf-profile.children.cycles-pp.apparmor_file_lock 0.77 ± 8% -0.3 0.51 ± 9% perf-profile.children.cycles-pp.__cond_resched 0.59 ± 8% -0.2 0.38 ± 9% perf-profile.children.cycles-pp.__might_fault 0.66 ± 9% -0.2 0.46 ± 9% perf-profile.children.cycles-pp.locks_release_private 0.56 ± 9% -0.2 0.36 ± 9% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.58 ± 9% -0.2 0.44 ± 10% perf-profile.children.cycles-pp.locks_insert_lock_ctx 0.69 ± 11% -0.1 0.54 ± 9% perf-profile.children.cycles-pp.locks_delete_lock_ctx 0.56 ± 10% -0.1 0.44 ± 8% perf-profile.children.cycles-pp.locks_unlink_lock_ctx 0.38 ± 8% -0.1 0.26 ± 10% perf-profile.children.cycles-pp.rcu_all_qs 0.33 ± 11% -0.1 0.21 ± 9% perf-profile.children.cycles-pp.flock64_to_posix_lock 0.35 ± 8% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.aa_file_perm 0.27 ± 21% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.23 ± 9% -0.1 0.13 ± 8% perf-profile.children.cycles-pp.__init_waitqueue_head 0.31 ± 8% -0.1 0.21 ± 9% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.24 ± 9% -0.1 0.17 ± 6% perf-profile.children.cycles-pp.testcase 0.19 ± 11% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.should_failslab 0.15 ± 10% -0.1 0.09 ± 10% perf-profile.children.cycles-pp.fcntl@plt 0.16 ± 7% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.13 ± 9% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.security_file_fcntl 0.12 ± 12% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.__list_del_entry_valid 0.14 ± 9% -0.0 0.10 ± 10% perf-profile.children.cycles-pp.locks_get_lock_context 0.08 ± 10% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.vfs_lock_file 0.13 ± 8% -0.0 0.10 ± 7% perf-profile.children.cycles-pp.locks_delete_block 0.10 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__list_add_valid 0.10 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.memset 0.00 +0.1 0.08 ± 9% perf-profile.children.cycles-pp.obj_cgroup_uncharge 0.00 +0.2 0.21 ± 18% perf-profile.children.cycles-pp.mem_cgroup_from_task 0.00 +0.6 0.55 ± 9% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.00 +0.6 0.60 ± 9% perf-profile.children.cycles-pp.refill_obj_stock 1.10 ± 9% +0.9 1.96 ± 12% perf-profile.children.cycles-pp.locks_dispose_list 0.00 +1.7 1.67 ± 10% perf-profile.children.cycles-pp.mod_objcg_state 0.00 +2.4 2.42 ± 35% perf-profile.children.cycles-pp.propagate_protected_usage 0.00 +2.5 2.47 ± 9% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.00 +4.7 4.74 ± 36% perf-profile.children.cycles-pp.page_counter_cancel 0.00 +5.8 5.77 ± 35% perf-profile.children.cycles-pp.page_counter_uncharge 0.00 +5.8 5.82 ± 35% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages 0.00 +7.4 7.36 ± 35% perf-profile.children.cycles-pp.page_counter_try_charge 0.00 +7.4 7.42 ± 35% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 3.47 ± 9% +7.8 11.27 ± 18% perf-profile.children.cycles-pp.kmem_cache_free 0.00 +8.1 8.05 ± 32% perf-profile.children.cycles-pp.obj_cgroup_charge 17.35 ± 9% +8.5 25.81 ± 11% perf-profile.children.cycles-pp.do_lock_file_wait 14.01 ± 9% +9.5 23.56 ± 12% perf-profile.children.cycles-pp.posix_lock_inode 10.00 ± 8% +10.2 20.24 ± 14% perf-profile.children.cycles-pp.locks_alloc_lock 9.00 ± 8% +10.7 19.67 ± 14% perf-profile.children.cycles-pp.kmem_cache_alloc 28.35 ± 9% +13.3 41.61 ± 12% perf-profile.children.cycles-pp.__x64_sys_fcntl 26.89 ± 9% +13.7 40.63 ± 12% perf-profile.children.cycles-pp.do_fcntl 23.92 ± 9% +14.7 38.67 ± 13% perf-profile.children.cycles-pp.fcntl_setlk 17.12 ± 9% -5.7 11.40 ± 9% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 9.72 ± 9% -3.3 6.39 ± 9% perf-profile.self.cycles-pp.syscall_return_via_sysret 7.81 ± 8% -2.6 5.16 ± 9% perf-profile.self.cycles-pp.__entry_text_start 3.90 ± 9% -1.4 2.45 ± 9% perf-profile.self.cycles-pp.memset_erms 1.97 ± 7% -0.7 1.32 ± 10% perf-profile.self.cycles-pp._raw_spin_lock 1.78 ± 9% -0.6 1.21 ± 9% perf-profile.self.cycles-pp.__libc_fcntl64 1.34 ± 8% -0.5 0.82 ± 11% perf-profile.self.cycles-pp.___might_sleep 1.35 ± 9% -0.4 0.94 ± 10% perf-profile.self.cycles-pp.common_file_perm 1.19 ± 9% -0.4 0.79 ± 9% perf-profile.self.cycles-pp.copy_user_generic_unrolled 1.30 ± 8% -0.3 0.96 ± 8% perf-profile.self.cycles-pp.posix_lock_inode 0.91 ± 9% -0.3 0.58 ± 9% perf-profile.self.cycles-pp.__might_sleep 1.08 ± 9% -0.3 0.76 ± 8% perf-profile.self.cycles-pp.fcntl_setlk 0.74 ± 8% -0.3 0.43 ± 10% perf-profile.self.cycles-pp.locks_alloc_lock 0.79 ± 9% -0.3 0.51 ± 10% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.76 ± 9% -0.3 0.49 ± 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.83 ± 9% -0.3 0.56 ± 8% perf-profile.self.cycles-pp.__fget_light 0.70 ± 9% -0.2 0.46 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.66 ± 7% -0.2 0.43 ± 7% perf-profile.self.cycles-pp.apparmor_file_lock 0.64 ± 9% -0.2 0.42 ± 9% perf-profile.self.cycles-pp.locks_release_private 0.52 ± 9% -0.2 0.34 ± 9% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.47 ± 10% -0.2 0.30 ± 9% perf-profile.self.cycles-pp.do_lock_file_wait 0.54 ± 9% -0.2 0.38 ± 9% perf-profile.self.cycles-pp.do_fcntl 0.46 ± 9% -0.2 0.30 ± 8% perf-profile.self.cycles-pp.__x64_sys_fcntl 0.40 ± 11% -0.2 0.25 ± 11% perf-profile.self.cycles-pp.do_syscall_64 0.33 ± 11% -0.1 0.19 ± 10% perf-profile.self.cycles-pp.flock64_to_posix_lock 0.38 ± 9% -0.1 0.25 ± 9% perf-profile.self.cycles-pp.__cond_resched 0.25 ± 7% -0.1 0.14 ± 10% perf-profile.self.cycles-pp.locks_dispose_list 0.24 ± 9% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.testcase 0.28 ± 7% -0.1 0.19 ± 11% perf-profile.self.cycles-pp.aa_file_perm 0.22 ± 24% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.28 ± 8% -0.1 0.18 ± 10% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.25 ± 9% -0.1 0.16 ± 10% perf-profile.self.cycles-pp._copy_from_user 0.26 ± 8% -0.1 0.17 ± 10% perf-profile.self.cycles-pp.rcu_all_qs 0.18 ± 9% -0.1 0.10 ± 10% perf-profile.self.cycles-pp.__init_waitqueue_head 0.15 ± 8% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.14 ± 9% -0.1 0.08 ± 11% perf-profile.self.cycles-pp.fcntl@plt 0.12 ± 12% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.18 ± 12% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.security_file_lock 0.08 ± 12% -0.0 0.03 ±100% perf-profile.self.cycles-pp.__list_add_valid 0.12 ± 9% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.locks_get_lock_context 0.10 ± 10% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__might_fault 0.10 ± 10% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.memset 0.10 ± 8% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.security_file_fcntl 0.11 ± 8% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.locks_delete_block 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.obj_cgroup_uncharge 0.00 +0.2 0.16 ± 24% perf-profile.self.cycles-pp.mem_cgroup_from_task 0.00 +0.3 0.34 ± 9% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.00 +0.6 0.56 ± 9% perf-profile.self.cycles-pp.obj_cgroup_charge 0.00 +0.6 0.58 ± 10% perf-profile.self.cycles-pp.refill_obj_stock 3.14 ± 8% +1.3 4.46 ± 10% perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +1.7 1.65 ± 10% perf-profile.self.cycles-pp.mod_objcg_state 0.00 +2.2 2.17 ± 9% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.00 +2.4 2.40 ± 35% perf-profile.self.cycles-pp.propagate_protected_usage 0.00 +4.7 4.70 ± 36% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +5.9 5.92 ± 35% perf-profile.self.cycles-pp.page_counter_try_charge will-it-scale.52.processes 7e+07 +-----------------------------------------------------------------+ | .+ + + + + | 6.5e+07 |+++ ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++ | | | 6e+07 |-+ | | | 5.5e+07 |-+ | | | 5e+07 |-+ O | | OO O O OOO OO O O OOO O | 4.5e+07 |O+O OOOO O O O O OOO O OOOOO O O O O OO O OOOO O | | O O O OO O O O| 4e+07 |-+ O O | | | 3.5e+07 +-----------------------------------------------------------------+ will-it-scale.per_process_ops 1.3e+06 +-----------------------------------------------------------------+ |+++.+++++++.++++++++.+++++++.+++++++.+++++++.++++++++.+++++++ | 1.2e+06 |-+ | | | | | 1.1e+06 |-+ | | | 1e+06 |-+ | | O | 900000 |-+ O O O O O O O O O O | |O O OOO O O OO O O O O O O OOOO O O O OO O O OOOO | | O O O O OO OO O OO O| 800000 |-O O O O | | | 700000 +-----------------------------------------------------------------+ will-it-scale.workload 7e+07 +-----------------------------------------------------------------+ | .+ + + + + | 6.5e+07 |+++ ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++ | | | 6e+07 |-+ | | | 5.5e+07 |-+ | | | 5e+07 |-+ O | | OO O O OOO OO O O OOO O | 4.5e+07 |O+O OOOO O O O O OOO O OOOOO O O O O OO O OOOO O | | O O O OO O O O| 4e+07 |-+ O O | | | 3.5e+07 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-icl-2sp2: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/lock1/will-it-scale/0xd000280 commit: b655843444 ("memcg: enable accounting for pollfd and select bits arrays") 0f12156dff ("memcg: enable accounting for file lock caches") b655843444152c0a 0f12156dff2862ac54235fc7270 ---------------- --------------------------- %stddev %change %stddev \ | \ 1.785e+08 -40.9% 1.055e+08 ± 2% will-it-scale.64.processes 2788911 -40.9% 1649196 ± 2% will-it-scale.per_process_ops 1.785e+08 -40.9% 1.055e+08 ± 2% will-it-scale.workload 366571 ± 7% +15.2% 422206 ± 9% numa-numastat.node0.local_node 2807 +2.4% 2874 vmstat.system.cs 0.02 ± 2% -0.0 0.02 ± 2% mpstat.cpu.all.soft% 5.55 -2.4 3.15 ± 7% mpstat.cpu.all.usr% 5364 ± 2% -85.5% 778.67 ± 13% slabinfo.Acpi-Parse.active_objs 5364 ± 2% -85.5% 778.67 ± 13% slabinfo.Acpi-Parse.num_objs 49038 +5.1% 51541 proc-vmstat.nr_active_anon 55209 +5.1% 58037 proc-vmstat.nr_shmem 49038 +5.1% 51541 proc-vmstat.nr_zone_active_anon 183.83 ± 28% +320.7% 773.33 ±101% interrupts.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5 836.50 ± 10% +46.3% 1223 ± 30% interrupts.CPU103.CAL:Function_call_interrupts 1348 ± 83% +208.2% 4155 ± 50% interrupts.CPU2.RES:Rescheduling_interrupts 1089 ± 77% +158.6% 2818 ± 53% interrupts.CPU4.RES:Rescheduling_interrupts 183.83 ± 28% +320.7% 773.33 ±101% interrupts.CPU5.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5 806.17 ± 14% +31.2% 1058 ± 30% interrupts.CPU55.CAL:Function_call_interrupts 723.17 ± 74% +417.0% 3738 ± 82% interrupts.CPU55.RES:Rescheduling_interrupts 4847 ± 94% -97.1% 138.17 ± 19% interrupts.CPU86.NMI:Non-maskable_interrupts 4847 ± 94% -97.1% 138.17 ± 19% interrupts.CPU86.PMI:Performance_monitoring_interrupts 8069 ± 63% -80.7% 1561 ±200% interrupts.CPU99.NMI:Non-maskable_interrupts 8069 ± 63% -80.7% 1561 ±200% interrupts.CPU99.PMI:Performance_monitoring_interrupts 2345 -12.0% 2063 perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 2345 -12.0% 2063 perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 262.68 ± 7% -42.8% 150.13 ± 12% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 561.17 ± 9% -16.5% 468.33 ± 5% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe 183.33 ± 7% +43.2% 262.50 ± 8% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 113.33 ± 11% +78.7% 202.50 ± 4% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 7031 -12.0% 6185 perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7031 -12.0% 6185 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7035 -12.0% 6189 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 2345 -12.0% 2063 perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 2345 -12.0% 2063 perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 2.50 ± 7% -47.6% 1.31 ± 18% perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 262.68 ± 7% -42.8% 150.13 ± 12% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 7031 -12.0% 6185 perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7031 -12.0% 6185 perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7035 -12.0% 6189 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.04 ± 22% +1112.5% 0.50 ±128% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.02 ± 3% +1574.6% 0.27 ± 6% perf-stat.i.MPKI 7.076e+10 -6.5% 6.615e+10 ± 2% perf-stat.i.branch-instructions 0.05 +0.0 0.08 ± 5% perf-stat.i.branch-miss-rate% 35955313 ± 2% +45.6% 52354202 ± 5% perf-stat.i.branch-misses 10.31 +18.0 28.33 ± 8% perf-stat.i.cache-miss-rate% 492492 +4777.7% 24022051 ± 4% perf-stat.i.cache-misses 4620608 +1768.8% 86349315 ± 4% perf-stat.i.cache-references 2651 +2.5% 2716 perf-stat.i.context-switches 0.47 +8.5% 0.51 ± 2% perf-stat.i.cpi 425867 -98.4% 6992 ± 5% perf-stat.i.cycles-between-cache-misses 0.00 ± 4% +0.0 0.00 ± 11% perf-stat.i.dTLB-load-miss-rate% 1.03e+11 -3.8% 9.907e+10 ± 2% perf-stat.i.dTLB-loads 0.00 +0.0 0.00 ± 8% perf-stat.i.dTLB-store-miss-rate% 6.534e+10 -11.3% 5.796e+10 ± 2% perf-stat.i.dTLB-stores 3.516e+11 -7.2% 3.263e+11 ± 2% perf-stat.i.instructions 2.11 -7.1% 1.96 ± 2% perf-stat.i.ipc 37.50 ± 2% +2090.1% 821.35 ± 3% perf-stat.i.metric.K/sec 1867 -6.7% 1743 ± 2% perf-stat.i.metric.M/sec 104632 ± 2% +6707.0% 7122311 ± 3% perf-stat.i.node-load-misses 14377 ± 21% +3209.4% 475820 ± 29% perf-stat.i.node-loads 45.30 ± 16% +51.5 96.80 perf-stat.i.node-store-miss-rate% 48602 ± 14% +22917.3% 11186859 ± 6% perf-stat.i.node-store-misses 65851 ± 9% +432.6% 350742 ± 9% perf-stat.i.node-stores 0.01 +1902.5% 0.27 ± 6% perf-stat.overall.MPKI 0.05 +0.0 0.08 ± 6% perf-stat.overall.branch-miss-rate% 10.65 +17.3 27.92 ± 8% perf-stat.overall.cache-miss-rate% 0.47 +7.7% 0.51 ± 2% perf-stat.overall.cpi 336119 -97.9% 6940 ± 4% perf-stat.overall.cycles-between-cache-misses 0.00 ± 4% +0.0 0.00 ± 10% perf-stat.overall.dTLB-load-miss-rate% 0.00 +0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 2.11 -7.1% 1.96 ± 2% perf-stat.overall.ipc 87.73 ± 2% +6.0 93.76 perf-stat.overall.node-load-miss-rate% 42.34 ± 13% +54.6 96.95 perf-stat.overall.node-store-miss-rate% 594415 +56.9% 932831 perf-stat.overall.path-length 7.05e+10 -6.5% 6.592e+10 ± 2% perf-stat.ps.branch-instructions 35867377 +45.6% 52233576 ± 5% perf-stat.ps.branch-misses 493933 +4747.8% 23944727 ± 4% perf-stat.ps.cache-misses 4639810 +1756.0% 86115077 ± 4% perf-stat.ps.cache-references 2653 +2.6% 2721 perf-stat.ps.context-switches 1.027e+11 -3.8% 9.872e+10 ± 2% perf-stat.ps.dTLB-loads 6.51e+10 -11.3% 5.775e+10 ± 2% perf-stat.ps.dTLB-stores 3.504e+11 -7.2% 3.252e+11 ± 2% perf-stat.ps.instructions 104869 ± 2% +6668.1% 7097622 ± 3% perf-stat.ps.node-load-misses 14698 ± 21% +3134.0% 475371 ± 29% perf-stat.ps.node-loads 48497 ± 14% +22887.1% 11148161 ± 6% perf-stat.ps.node-store-misses 65971 ± 9% +430.7% 350139 ± 8% perf-stat.ps.node-stores 1.061e+14 -7.2% 9.845e+13 ± 2% perf-stat.total.instructions 11738 ± 11% -32.5% 7926 ± 19% softirqs.CPU1.RCU 12768 ± 12% -39.4% 7741 ± 11% softirqs.CPU10.RCU 8853 ± 10% -33.9% 5851 ± 8% softirqs.CPU100.RCU 9695 ± 6% -40.1% 5811 ± 5% softirqs.CPU101.RCU 9277 ± 7% -38.0% 5752 ± 7% softirqs.CPU102.RCU 9585 ± 8% -33.5% 6376 ± 17% softirqs.CPU103.RCU 9838 ± 7% -40.5% 5857 ± 9% softirqs.CPU104.RCU 9315 ± 8% -40.6% 5529 ± 7% softirqs.CPU105.RCU 9661 ± 10% -40.5% 5749 ± 5% softirqs.CPU106.RCU 9531 ± 6% -35.6% 6138 ± 7% softirqs.CPU108.RCU 9472 ± 7% -34.4% 6210 ± 16% softirqs.CPU109.RCU 11502 ± 13% -41.5% 6726 ± 14% softirqs.CPU11.RCU 9003 ± 11% -34.3% 5916 ± 12% softirqs.CPU111.RCU 9739 ± 7% -36.5% 6187 ± 10% softirqs.CPU113.RCU 9453 ± 9% -34.2% 6220 ± 13% softirqs.CPU114.RCU 9891 ± 7% -31.3% 6798 ± 18% softirqs.CPU115.RCU 10008 ± 9% -39.1% 6090 ± 12% softirqs.CPU116.RCU 10132 ± 17% -41.7% 5912 ± 15% softirqs.CPU117.RCU 9765 ± 8% -37.5% 6103 ± 8% softirqs.CPU118.RCU 8960 ± 11% -33.9% 5923 ± 8% softirqs.CPU119.RCU 10910 ± 17% -34.0% 7197 ± 6% softirqs.CPU12.RCU 9577 ± 6% -37.6% 5973 ± 12% softirqs.CPU120.RCU 9565 ± 10% -42.9% 5457 ± 7% softirqs.CPU121.RCU 9137 ± 10% -38.7% 5604 ± 11% softirqs.CPU124.RCU 9418 ± 13% -34.3% 6186 ± 13% softirqs.CPU125.RCU 9151 ± 15% -37.2% 5745 ± 11% softirqs.CPU126.RCU 10800 ± 7% -34.7% 7057 ± 10% softirqs.CPU127.RCU 9815 ± 9% -29.1% 6958 ± 6% softirqs.CPU13.RCU 11463 ± 15% -39.2% 6967 ± 12% softirqs.CPU14.RCU 10757 ± 15% -37.6% 6716 ± 10% softirqs.CPU15.RCU 10809 ± 17% -38.5% 6648 ± 16% softirqs.CPU16.RCU 11169 ± 14% -42.4% 6435 ± 13% softirqs.CPU17.RCU 12215 ± 11% -40.1% 7319 ± 13% softirqs.CPU19.RCU 11500 ± 12% -39.2% 6996 ± 10% softirqs.CPU2.RCU 11027 ± 18% -32.7% 7416 ± 9% softirqs.CPU20.RCU 11155 ± 17% -39.5% 6745 ± 5% softirqs.CPU21.RCU 11083 ± 16% -32.8% 7451 ± 12% softirqs.CPU22.RCU 11841 ± 15% -44.9% 6523 ± 13% softirqs.CPU23.RCU 11415 ± 16% -38.7% 6997 ± 15% softirqs.CPU24.RCU 11325 ± 5% -37.7% 7061 ± 8% softirqs.CPU25.RCU 12111 ± 11% -44.5% 6715 ± 11% softirqs.CPU26.RCU 11627 ± 11% -43.1% 6613 ± 11% softirqs.CPU27.RCU 11113 ± 3% -40.1% 6658 ± 10% softirqs.CPU28.RCU 10333 ± 13% -33.2% 6907 ± 8% softirqs.CPU29.RCU 12191 ± 13% -42.5% 7009 ± 13% softirqs.CPU3.RCU 10265 ± 11% -31.7% 7012 ± 9% softirqs.CPU30.RCU 11440 ± 13% -39.6% 6910 ± 17% softirqs.CPU31.RCU 10389 ± 12% -33.8% 6874 ± 11% softirqs.CPU32.RCU 11224 ± 10% -37.3% 7032 ± 19% softirqs.CPU33.RCU 10886 ± 13% -32.8% 7317 ± 16% softirqs.CPU34.RCU 10588 ± 9% -32.7% 7125 ± 6% softirqs.CPU35.RCU 9719 ± 16% -31.8% 6625 ± 14% softirqs.CPU36.RCU 11099 ± 12% -33.2% 7414 ± 12% softirqs.CPU37.RCU 10338 ± 9% -25.6% 7690 ± 14% softirqs.CPU38.RCU 10238 ± 11% -27.1% 7461 ± 9% softirqs.CPU39.RCU 11660 ± 15% -35.5% 7516 ± 12% softirqs.CPU4.RCU 10552 ± 13% -31.6% 7220 ± 12% softirqs.CPU40.RCU 10552 ± 10% -31.1% 7273 ± 7% softirqs.CPU41.RCU 10103 ± 13% -31.8% 6891 ± 9% softirqs.CPU42.RCU 10974 ± 8% -36.9% 6921 ± 10% softirqs.CPU43.RCU 10730 ± 13% -37.2% 6739 ± 17% softirqs.CPU44.RCU 11205 ± 12% -35.4% 7233 ± 11% softirqs.CPU45.RCU 11136 ± 12% -39.4% 6751 ± 12% softirqs.CPU46.RCU 10649 ± 11% -33.6% 7073 ± 8% softirqs.CPU47.RCU 9988 ± 10% -36.5% 6346 ± 8% softirqs.CPU48.RCU 9801 ± 9% -36.9% 6180 ± 6% softirqs.CPU49.RCU 12102 ± 9% -37.5% 7563 ± 17% softirqs.CPU5.RCU 9884 ± 12% -36.2% 6303 ± 7% softirqs.CPU50.RCU 9347 ± 9% -34.8% 6096 ± 7% softirqs.CPU51.RCU 10164 ± 11% -35.3% 6575 ± 7% softirqs.CPU52.RCU 10518 ± 15% -37.2% 6609 ± 10% softirqs.CPU53.RCU 9857 ± 12% -32.5% 6655 ± 5% softirqs.CPU54.RCU 10795 ± 13% -44.0% 6047 ± 5% softirqs.CPU55.RCU 10397 ± 12% -38.9% 6350 ± 7% softirqs.CPU56.RCU 9513 ± 6% -30.5% 6615 ± 8% softirqs.CPU57.RCU 10400 ± 10% -37.6% 6492 ± 5% softirqs.CPU58.RCU 10434 ± 8% -39.1% 6351 ± 9% softirqs.CPU59.RCU 12055 ± 14% -32.8% 8102 ± 22% softirqs.CPU6.RCU 10580 ± 12% -40.1% 6340 ± 11% softirqs.CPU60.RCU 10099 ± 11% -41.4% 5915 ± 7% softirqs.CPU61.RCU 10021 ± 10% -35.8% 6436 ± 3% softirqs.CPU62.RCU 9856 ± 7% -31.3% 6774 ± 9% softirqs.CPU64.RCU 9963 ± 12% -36.0% 6372 ± 8% softirqs.CPU65.RCU 9664 ± 8% -30.1% 6752 ± 5% softirqs.CPU66.RCU 9673 ± 5% -33.2% 6463 ± 10% softirqs.CPU67.RCU 9910 ± 8% -36.4% 6307 ± 6% softirqs.CPU68.RCU 9589 ± 5% -34.2% 6310 ± 7% softirqs.CPU69.RCU 11733 ± 9% -42.0% 6809 ± 12% softirqs.CPU7.RCU 9603 ± 42% +105.6% 19745 ± 32% softirqs.CPU7.SCHED 9881 ± 9% -34.0% 6521 ± 7% softirqs.CPU70.RCU 9620 ± 6% -33.2% 6424 ± 2% softirqs.CPU71.RCU 35921 ± 13% -39.4% 21764 ± 30% softirqs.CPU71.SCHED 10659 ± 13% -38.8% 6524 ± 7% softirqs.CPU72.RCU 9917 ± 7% -38.9% 6055 ± 7% softirqs.CPU73.RCU 9572 ± 7% -40.0% 5741 ± 9% softirqs.CPU74.RCU 9683 ± 9% -36.2% 6176 ± 6% softirqs.CPU75.RCU 10099 ± 7% -38.4% 6220 ± 10% softirqs.CPU76.RCU 9726 ± 5% -28.6% 6946 ± 14% softirqs.CPU78.RCU 9463 ± 6% -37.1% 5956 ± 12% softirqs.CPU79.RCU 10789 ± 12% -34.1% 7105 ± 20% softirqs.CPU8.RCU 9917 ± 7% -35.1% 6439 ± 11% softirqs.CPU80.RCU 9821 ± 4% -34.5% 6430 ± 3% softirqs.CPU81.RCU 9938 ± 12% -38.6% 6098 ± 10% softirqs.CPU82.RCU 9838 ± 4% -35.2% 6370 ± 7% softirqs.CPU83.RCU 10401 ± 8% -40.1% 6229 ± 6% softirqs.CPU84.RCU 10051 ± 6% -36.4% 6396 ± 9% softirqs.CPU85.RCU 9792 ± 4% -35.3% 6339 ± 14% softirqs.CPU86.RCU 10059 ± 9% -36.3% 6408 ± 8% softirqs.CPU87.RCU 9972 ± 8% -39.2% 6059 ± 3% softirqs.CPU88.RCU 9447 ± 9% -30.8% 6534 ± 12% softirqs.CPU89.RCU 11511 ± 24% -37.0% 7246 ± 11% softirqs.CPU9.RCU 9563 ± 6% -30.9% 6612 ± 6% softirqs.CPU90.RCU 36244 ± 11% -29.7% 25496 ± 25% softirqs.CPU90.SCHED 9671 ± 8% -34.4% 6340 ± 7% softirqs.CPU91.RCU 9848 ± 7% -33.7% 6531 ± 5% softirqs.CPU92.RCU 9578 ± 8% -32.4% 6479 ± 5% softirqs.CPU93.RCU 9519 ± 5% -36.3% 6064 ± 7% softirqs.CPU94.RCU 9480 ± 9% -31.0% 6544 ± 12% softirqs.CPU95.RCU 9025 ± 8% -35.2% 5845 ± 5% softirqs.CPU96.RCU 9656 ± 19% -39.8% 5814 ± 8% softirqs.CPU97.RCU 10008 ± 7% -39.4% 6064 ± 9% softirqs.CPU98.RCU 9619 ± 6% -41.1% 5661 ± 3% softirqs.CPU99.RCU 1308899 ± 3% -36.2% 835466 ± 3% softirqs.RCU 47568 ± 2% +20.6% 57361 ± 4% softirqs.TIMER 10.02 ± 3% -4.9 5.08 ± 18% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64 8.20 -3.8 4.38 ± 6% perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.28 -2.7 2.58 ± 5% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 4.78 -2.3 2.48 ± 7% perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 3.68 -1.8 1.88 ± 7% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl 3.36 -1.4 1.93 ± 6% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 2.98 -1.2 1.75 ± 6% perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 2.51 -1.2 1.30 ± 7% perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 2.49 -1.1 1.36 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 2.02 -1.0 1.05 ± 6% perf-profile.calltrace.cycles-pp.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.02 -1.0 1.06 ± 7% perf-profile.calltrace.cycles-pp.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.14 -0.8 0.37 ± 70% perf-profile.calltrace.cycles-pp.locks_release_private.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.70 -0.7 0.98 ± 7% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 1.24 -0.6 0.64 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.15 -0.6 1.57 ± 8% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.34 -0.6 0.77 ± 6% perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.32 -0.6 0.75 ± 7% perf-profile.calltrace.cycles-pp.__might_fault._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 1.22 -0.5 0.68 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait 1.21 -0.5 0.68 ± 7% perf-profile.calltrace.cycles-pp.aa_file_perm.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk 1.41 -0.5 0.90 ± 9% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 1.21 -0.5 0.70 ± 6% perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 1.08 -0.5 0.61 ± 7% perf-profile.calltrace.cycles-pp.flock64_to_posix_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.76 -0.4 0.35 ± 70% perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.27 -0.4 0.86 ± 5% perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.04 -0.4 0.66 ± 10% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64 0.76 -0.1 0.63 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 97.66 +0.4 98.07 perf-profile.calltrace.cycles-pp.__libc_fcntl64 0.00 +0.5 0.54 ± 4% perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +0.8 0.83 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +0.9 0.90 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 0.00 +0.9 0.92 ± 5% perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.3 1.26 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.15 +1.4 3.54 ± 14% perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 0.00 +1.6 1.59 ± 11% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.14 +1.8 2.90 ± 18% perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.8 1.83 ± 6% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +2.1 2.13 ± 29% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc 0.00 +2.2 2.16 ± 29% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk 0.00 +2.4 2.42 ± 7% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +2.6 2.60 ± 28% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl 0.00 +2.7 2.66 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +3.0 3.04 ± 37% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode 2.86 ± 2% +3.6 6.42 ± 10% perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.00 +3.6 3.59 ± 33% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 0.00 +3.6 3.65 ± 36% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait 0.00 +3.7 3.72 ± 36% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +4.3 4.29 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 3.76 +5.7 9.47 ± 12% perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 8.70 +6.2 14.88 ± 6% perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 7.40 +6.3 13.74 ± 7% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl 83.98 +6.8 90.74 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 82.41 +7.4 89.82 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +7.8 7.75 ± 29% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 46.22 +7.9 54.10 ± 3% perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 78.68 +8.9 87.56 perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +9.2 9.15 ± 23% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 73.64 +11.0 84.60 perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +11.2 11.21 ± 25% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock 38.04 +11.7 49.70 ± 4% perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 16.21 +11.8 28.04 ± 5% perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 13.60 +12.1 25.71 ± 6% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk 63.80 +15.2 78.96 perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.76 -3.4 4.33 ± 6% perf-profile.children.cycles-pp.memset_erms 8.05 -3.4 4.62 ± 7% perf-profile.children.cycles-pp._copy_from_user 5.70 ± 2% -2.8 2.89 ± 18% perf-profile.children.cycles-pp.__entry_text_start 6.00 -2.5 3.47 ± 6% perf-profile.children.cycles-pp._raw_spin_lock 5.10 -2.4 2.66 ± 6% perf-profile.children.cycles-pp.security_file_lock 5.00 -2.2 2.75 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret 3.90 -1.9 2.01 ± 7% perf-profile.children.cycles-pp.common_file_perm 3.94 -1.7 2.28 ± 6% perf-profile.children.cycles-pp.copy_user_generic_unrolled 3.74 -1.7 2.08 ± 5% perf-profile.children.cycles-pp.___might_sleep 3.95 -1.6 2.40 ± 6% perf-profile.children.cycles-pp.__might_sleep 2.72 -1.3 1.41 ± 6% perf-profile.children.cycles-pp.locks_delete_lock_ctx 2.58 -1.3 1.29 ± 5% perf-profile.children.cycles-pp.locks_release_private 3.10 -1.3 1.81 ± 6% perf-profile.children.cycles-pp.__fget_light 2.18 -1.0 1.14 ± 6% perf-profile.children.cycles-pp.locks_unlink_lock_ctx 2.18 -1.0 1.15 ± 7% perf-profile.children.cycles-pp.locks_insert_lock_ctx 2.03 -0.9 1.17 ± 7% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.19 -0.8 1.34 ± 5% perf-profile.children.cycles-pp.__cond_resched 1.54 -0.7 0.86 ± 7% perf-profile.children.cycles-pp.aa_file_perm 1.54 -0.7 0.87 ± 7% perf-profile.children.cycles-pp.__might_fault 1.68 -0.6 1.07 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 1.32 -0.6 0.76 ± 6% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 1.19 -0.5 0.67 ± 6% perf-profile.children.cycles-pp.flock64_to_posix_lock 1.10 -0.4 0.65 ± 5% perf-profile.children.cycles-pp.rcu_all_qs 0.81 -0.4 0.45 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.94 -0.3 0.60 ± 6% perf-profile.children.cycles-pp.__init_waitqueue_head 0.74 -0.3 0.41 ± 4% perf-profile.children.cycles-pp.locks_delete_block 0.75 -0.3 0.42 ± 5% perf-profile.children.cycles-pp.locks_get_lock_context 0.70 -0.3 0.39 ± 7% perf-profile.children.cycles-pp.locks_copy_lock 0.53 -0.3 0.23 ± 5% perf-profile.children.cycles-pp.__list_del_entry_valid 0.78 ± 2% -0.3 0.49 ± 9% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.61 -0.3 0.35 ± 8% perf-profile.children.cycles-pp.security_file_fcntl 0.60 -0.2 0.35 ± 11% perf-profile.children.cycles-pp.memset 0.53 -0.2 0.29 ± 6% perf-profile.children.cycles-pp.apparmor_file_lock 0.43 ± 2% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.43 ± 2% -0.2 0.25 ± 7% perf-profile.children.cycles-pp.vfs_lock_file 0.50 -0.2 0.33 ± 8% perf-profile.children.cycles-pp.testcase 0.37 -0.2 0.21 ± 6% perf-profile.children.cycles-pp.locks_copy_conflock 0.32 ± 2% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.__fdget_raw 0.36 -0.1 0.24 ± 7% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.22 -0.1 0.12 ± 6% perf-profile.children.cycles-pp.locks_move_blocks 0.41 -0.1 0.32 ± 6% perf-profile.children.cycles-pp.should_failslab 0.20 ± 2% -0.1 0.13 ± 10% perf-profile.children.cycles-pp.fcntl@plt 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.08 ± 11% perf-profile.children.cycles-pp.refill_stock 98.10 +0.1 98.23 perf-profile.children.cycles-pp.__libc_fcntl64 0.00 +0.3 0.30 ± 6% perf-profile.children.cycles-pp.obj_cgroup_uncharge 0.00 +0.5 0.53 ± 7% perf-profile.children.cycles-pp.mem_cgroup_from_task 2.41 +1.3 3.69 ± 14% perf-profile.children.cycles-pp.locks_dispose_list 0.00 +2.0 1.99 ± 5% perf-profile.children.cycles-pp.refill_obj_stock 0.32 ± 2% +2.1 2.40 ± 6% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.00 +3.4 3.39 ± 21% perf-profile.children.cycles-pp.propagate_protected_usage 0.00 +4.9 4.90 ± 8% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.00 +5.6 5.60 ± 6% perf-profile.children.cycles-pp.mod_objcg_state 0.00 +5.9 5.94 ± 26% perf-profile.children.cycles-pp.page_counter_cancel 84.29 +6.7 90.98 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.00 +7.1 7.12 ± 24% perf-profile.children.cycles-pp.page_counter_uncharge 83.10 +7.2 90.30 perf-profile.children.cycles-pp.do_syscall_64 0.00 +7.3 7.27 ± 24% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages 46.87 +7.6 54.45 ± 3% perf-profile.children.cycles-pp.do_lock_file_wait 79.26 +8.6 87.90 perf-profile.children.cycles-pp.__x64_sys_fcntl 74.02 +10.8 84.84 perf-profile.children.cycles-pp.do_fcntl 39.59 +11.0 50.57 ± 4% perf-profile.children.cycles-pp.posix_lock_inode 0.00 +11.2 11.23 ± 25% perf-profile.children.cycles-pp.page_counter_try_charge 0.00 +11.4 11.37 ± 24% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 8.04 +11.5 19.54 ± 6% perf-profile.children.cycles-pp.kmem_cache_free 0.00 +13.6 13.63 ± 19% perf-profile.children.cycles-pp.obj_cgroup_charge 64.58 +14.8 79.39 perf-profile.children.cycles-pp.fcntl_setlk 25.73 +17.7 43.45 ± 2% perf-profile.children.cycles-pp.locks_alloc_lock 22.71 +18.8 41.55 ± 3% perf-profile.children.cycles-pp.kmem_cache_alloc 7.42 -3.3 4.14 ± 6% perf-profile.self.cycles-pp.memset_erms 6.55 -3.2 3.35 ± 8% perf-profile.self.cycles-pp.__libc_fcntl64 5.72 -2.4 3.30 ± 6% perf-profile.self.cycles-pp._raw_spin_lock 5.14 -2.4 2.76 ± 5% perf-profile.self.cycles-pp.posix_lock_inode 5.00 -2.2 2.75 ± 8% perf-profile.self.cycles-pp.syscall_return_via_sysret 3.49 -1.7 1.77 ± 4% perf-profile.self.cycles-pp.fcntl_setlk 3.61 -1.5 2.09 ± 6% perf-profile.self.cycles-pp.copy_user_generic_unrolled 3.20 -1.4 1.77 ± 5% perf-profile.self.cycles-pp.___might_sleep 2.97 -1.2 1.74 ± 6% perf-profile.self.cycles-pp.__fget_light 2.35 -1.2 1.14 ± 7% perf-profile.self.cycles-pp.common_file_perm 3.02 -1.2 1.84 ± 6% perf-profile.self.cycles-pp.__might_sleep 2.25 -1.1 1.14 ± 5% perf-profile.self.cycles-pp.locks_release_private 2.38 -0.9 1.47 ± 8% perf-profile.self.cycles-pp.locks_alloc_lock 1.80 -0.8 1.04 ± 7% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.39 -0.7 0.71 ± 21% perf-profile.self.cycles-pp.__entry_text_start 1.61 -0.7 0.94 ± 7% perf-profile.self.cycles-pp.__x64_sys_fcntl 1.60 -0.6 0.95 ± 8% perf-profile.self.cycles-pp.do_fcntl 1.43 -0.6 0.79 ± 6% perf-profile.self.cycles-pp.do_lock_file_wait 1.40 -0.6 0.81 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.31 -0.6 0.73 ± 7% perf-profile.self.cycles-pp.aa_file_perm 1.16 -0.5 0.67 ± 6% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 1.08 -0.5 0.60 ± 6% perf-profile.self.cycles-pp.flock64_to_posix_lock 0.97 -0.4 0.56 ± 6% perf-profile.self.cycles-pp._copy_from_user 1.02 -0.4 0.61 ± 8% perf-profile.self.cycles-pp.do_syscall_64 1.04 -0.4 0.65 ± 4% perf-profile.self.cycles-pp.__cond_resched 0.81 -0.4 0.45 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.76 -0.4 0.42 ± 5% perf-profile.self.cycles-pp.security_file_lock 0.71 -0.3 0.38 ± 8% perf-profile.self.cycles-pp.locks_insert_lock_ctx 0.64 -0.3 0.32 ± 6% perf-profile.self.cycles-pp.locks_unlink_lock_ctx 0.75 ± 2% -0.3 0.44 ± 6% perf-profile.self.cycles-pp.rcu_all_qs 0.70 -0.3 0.42 ± 10% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.58 ± 2% -0.3 0.32 ± 5% perf-profile.self.cycles-pp.locks_dispose_list 0.43 -0.3 0.18 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.52 -0.2 0.29 ± 5% perf-profile.self.cycles-pp.locks_delete_block 0.53 -0.2 0.30 ± 5% perf-profile.self.cycles-pp.locks_get_lock_context 0.63 ± 2% -0.2 0.42 ± 5% perf-profile.self.cycles-pp.__init_waitqueue_head 0.43 -0.2 0.23 ± 9% perf-profile.self.cycles-pp.locks_delete_lock_ctx 0.56 ± 2% -0.2 0.36 ± 11% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.43 -0.2 0.25 ± 9% perf-profile.self.cycles-pp.security_file_fcntl 0.32 ± 2% -0.2 0.17 ± 7% perf-profile.self.cycles-pp.__list_add_valid 0.33 -0.2 0.18 ± 9% perf-profile.self.cycles-pp.locks_copy_lock 0.33 -0.1 0.18 ± 7% perf-profile.self.cycles-pp.__might_fault 0.42 ± 2% -0.1 0.27 ± 9% perf-profile.self.cycles-pp.testcase 0.33 ± 2% -0.1 0.18 ± 5% perf-profile.self.cycles-pp.vfs_lock_file 0.32 -0.1 0.18 ± 7% perf-profile.self.cycles-pp.apparmor_file_lock 0.32 ± 2% -0.1 0.18 ± 6% perf-profile.self.cycles-pp.locks_copy_conflock 0.30 ± 3% -0.1 0.18 ± 10% perf-profile.self.cycles-pp.memset 0.26 ± 3% -0.1 0.18 ± 7% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.17 ± 2% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.locks_move_blocks 0.11 ± 4% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.__fdget_raw 0.20 ± 3% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.should_failslab 0.10 ± 4% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.fcntl@plt 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.08 ± 11% perf-profile.self.cycles-pp.refill_stock 0.00 +0.1 0.14 ± 10% perf-profile.self.cycles-pp.obj_cgroup_uncharge 0.00 +0.4 0.36 ± 7% perf-profile.self.cycles-pp.mem_cgroup_from_task 0.11 ± 3% +0.8 0.92 ± 5% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.00 +1.8 1.85 ± 5% perf-profile.self.cycles-pp.refill_obj_stock 0.00 +2.0 2.05 ± 7% perf-profile.self.cycles-pp.obj_cgroup_charge 9.35 +3.0 12.32 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +3.4 3.36 ± 21% perf-profile.self.cycles-pp.propagate_protected_usage 0.00 +4.0 4.01 ± 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.00 +5.2 5.23 ± 7% perf-profile.self.cycles-pp.mod_objcg_state 0.00 +5.9 5.90 ± 26% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +9.0 8.97 ± 25% perf-profile.self.cycles-pp.page_counter_try_charge Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang