January 2022 - LKP - Ml01.01.Org

[memcg] 0f12156dff: will-it-scale.per_process_ops -33.6% regression

by kernel test robot

Greeting, FYI, we noticed a -33.6% regression of will-it-scale.per_process_ops due to commit: commit: 0f12156dff2862ac54235fc72703f18770769042 ("memcg: enable accounting for file lock caches") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 104 threads 2 sockets Skylake with 192G memory with following parameters: nr_task: 50% mode: process test: lock1 cpufreq_governor: performance ucode: 0x2006a0a test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -40.9% regression | | test machine | 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=50% | | | test=lock1 | | | ucode=0xd000280 | +------------------+---------------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run bin/lkp run generated-yaml-file ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/lock1/will-it-scale/0x2006a0a commit: b655843444 ("memcg: enable accounting for pollfd and select bits arrays") 0f12156dff ("memcg: enable accounting for file lock caches") b655843444152c0a 0f12156dff2862ac54235fc7270 ---------------- --------------------------- %stddev %change %stddev \ | \ 65855410 -33.6% 43722413 ± 5% will-it-scale.52.processes 1266449 -33.6% 840815 ± 5% will-it-scale.per_process_ops 65855410 -33.6% 43722413 ± 5% will-it-scale.workload 141099 ± 3% +8.2% 152680 ± 2% meminfo.Active 140875 ± 3% +8.2% 152456 ± 2% meminfo.Active(anon) 28.79 +6.7 35.47 mpstat.cpu.all.sys% 20.49 -6.7 13.76 ± 5% mpstat.cpu.all.usr% 138801 ± 3% +8.4% 150456 ± 2% numa-meminfo.node1.Active 138689 ± 3% +8.4% 150381 ± 2% numa-meminfo.node1.Active(anon) 34681 ± 3% +8.3% 37570 ± 2% numa-vmstat.node1.nr_active_anon 34681 ± 3% +8.3% 37570 ± 2% numa-vmstat.node1.nr_zone_active_anon 4480 ± 3% -85.9% 632.67 ± 12% slabinfo.Acpi-Parse.active_objs 4480 ± 3% -85.9% 632.67 ± 12% slabinfo.Acpi-Parse.num_objs 2957456 ±214% -96.5% 104939 ± 5% turbostat.C1 103.99 +5.8% 110.04 turbostat.RAMWatt 19.83 -34.5% 13.00 ± 4% vmstat.cpu.us 2487 ± 2% +4.4% 2596 vmstat.system.cs 35219 ± 3% +8.2% 38114 ± 2% proc-vmstat.nr_active_anon 10977 +1.8% 11171 proc-vmstat.nr_mapped 40640 ± 3% +7.6% 43745 ± 2% proc-vmstat.nr_shmem 35219 ± 3% +8.2% 38114 ± 2% proc-vmstat.nr_zone_active_anon 53061 ± 3% +6.5% 56488 ± 3% proc-vmstat.pgactivate 0.02 ± 19% -24.0% 0.01 ± 18% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 9.26 ± 21% -39.8% 5.58 ± 14% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 172.83 ± 11% -29.4% 122.00 ± 5% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 127.50 ± 4% +47.8% 188.50 ± 7% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 1080 ± 19% +66.6% 1799 ± 13% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 7204 ± 3% -12.0% 6337 perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7204 ± 3% -12.0% 6337 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7209 ± 3% -12.0% 6342 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 6825 ± 4% -11.5% 6036 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 9.26 ± 21% -39.8% 5.57 ± 14% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 7204 ± 3% -12.0% 6337 perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7204 ± 3% -12.0% 6337 perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 0.05 ± 7% +751.3% 0.40 ±180% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 7209 ± 3% -12.0% 6342 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 6825 ± 4% -11.5% 6036 perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 827.67 ± 9% -16.3% 692.50 ± 4% interrupts.CPU13.CAL:Function_call_interrupts 3623 ± 54% +66.8% 6045 ± 33% interrupts.CPU26.NMI:Non-maskable_interrupts 3623 ± 54% +66.8% 6045 ± 33% interrupts.CPU26.PMI:Performance_monitoring_interrupts 239.50 ± 18% -33.1% 160.33 ± 38% interrupts.CPU41.RES:Rescheduling_interrupts 261.33 ± 18% -54.4% 119.17 ± 49% interrupts.CPU42.RES:Rescheduling_interrupts 7894 -30.2% 5511 ± 37% interrupts.CPU45.NMI:Non-maskable_interrupts 7894 -30.2% 5511 ± 37% interrupts.CPU45.PMI:Performance_monitoring_interrupts 3374 ± 40% +90.6% 6432 ± 23% interrupts.CPU58.NMI:Non-maskable_interrupts 3374 ± 40% +90.6% 6432 ± 23% interrupts.CPU58.PMI:Performance_monitoring_interrupts 272.83 ± 8% -36.4% 173.50 ± 40% interrupts.CPU6.RES:Rescheduling_interrupts 2890 ± 37% +61.0% 4652 ± 37% interrupts.CPU76.NMI:Non-maskable_interrupts 2890 ± 37% +61.0% 4652 ± 37% interrupts.CPU76.PMI:Performance_monitoring_interrupts 272.67 ± 9% -29.0% 193.67 ± 27% interrupts.CPU81.RES:Rescheduling_interrupts 3475 ± 47% +109.3% 7273 ± 12% interrupts.CPU93.NMI:Non-maskable_interrupts 3475 ± 47% +109.3% 7273 ± 12% interrupts.CPU93.PMI:Performance_monitoring_interrupts 3054 ± 37% +91.7% 5857 ± 38% interrupts.CPU96.NMI:Non-maskable_interrupts 3054 ± 37% +91.7% 5857 ± 38% interrupts.CPU96.PMI:Performance_monitoring_interrupts 0.05 ± 5% +1601.9% 0.79 ± 20% perf-stat.i.MPKI 12.52 ± 2% +20.9 33.42 ± 4% perf-stat.i.cache-miss-rate% 647128 ± 2% +4855.0% 32065121 ± 8% perf-stat.i.cache-misses 5130146 ± 2% +1777.3% 96305803 ± 9% perf-stat.i.cache-references 2302 +6.1% 2443 perf-stat.i.context-switches 259518 ± 2% -98.1% 4827 ± 6% perf-stat.i.cycles-between-cache-misses 0.17 -0.1 0.11 perf-stat.i.dTLB-load-miss-rate% 65644478 -33.6% 43563819 ± 5% perf-stat.i.dTLB-load-misses 51892 -27.4% 37675 ± 3% perf-stat.i.dTLB-store-misses 68181011 -33.4% 45405593 ± 5% perf-stat.i.iTLB-load-misses 1933 +58.2% 3058 ± 3% perf-stat.i.instructions-per-iTLB-miss 81.87 ± 12% +782.2% 722.30 ± 16% perf-stat.i.metric.K/sec 126326 ± 3% +6119.4% 7856755 ± 2% perf-stat.i.node-load-misses 26893 ± 6% +3382.5% 936567 ± 3% perf-stat.i.node-loads 87.04 +12.8 99.79 perf-stat.i.node-store-miss-rate% 33838 ± 3% +28351.8% 9627739 perf-stat.i.node-store-misses 0.04 ± 2% +1709.7% 0.71 ± 14% perf-stat.overall.MPKI 0.52 -0.0 0.49 ± 3% perf-stat.overall.branch-miss-rate% 12.65 ± 2% +20.7 33.35 ± 4% perf-stat.overall.cache-miss-rate% 222708 ± 2% -98.0% 4560 ± 8% perf-stat.overall.cycles-between-cache-misses 0.17 -0.1 0.10 perf-stat.overall.dTLB-load-miss-rate% 0.00 -0.0 0.00 ± 2% perf-stat.overall.dTLB-store-miss-rate% 1929 +56.0% 3010 perf-stat.overall.instructions-per-iTLB-miss 82.22 +7.1 89.34 perf-stat.overall.node-load-miss-rate% 78.75 +21.2 99.91 perf-stat.overall.node-store-miss-rate% 603681 +56.8% 946452 perf-stat.overall.path-length 649626 ± 2% +4816.5% 31938893 ± 8% perf-stat.ps.cache-misses 5137402 ± 2% +1767.5% 95941923 ± 9% perf-stat.ps.cache-references 2313 +5.8% 2448 perf-stat.ps.context-switches 65424262 -33.6% 43419292 ± 5% perf-stat.ps.dTLB-load-misses 51749 -27.4% 37573 ± 3% perf-stat.ps.dTLB-store-misses 67951468 -33.4% 45252180 ± 5% perf-stat.ps.iTLB-load-misses 126213 ± 3% +6102.7% 7828694 ± 2% perf-stat.ps.node-load-misses 27281 ± 6% +3321.9% 933553 ± 3% perf-stat.ps.node-loads 33783 ± 3% +28302.8% 9595407 perf-stat.ps.node-store-misses 15207 ± 10% -34.0% 10039 ± 12% softirqs.CPU0.RCU 16968 ± 17% -36.3% 10810 ± 19% softirqs.CPU1.RCU 17526 ± 19% -36.4% 11142 ± 14% softirqs.CPU10.RCU 17232 ± 16% -48.1% 8946 ± 13% softirqs.CPU100.RCU 15770 ± 16% -43.7% 8874 ± 28% softirqs.CPU101.RCU 14396 ± 15% -44.6% 7969 ± 13% softirqs.CPU102.RCU 18909 ± 5% -46.4% 10130 ± 17% softirqs.CPU103.RCU 17822 ± 23% -45.3% 9744 ± 22% softirqs.CPU11.RCU 18056 ± 22% -52.5% 8582 ± 22% softirqs.CPU12.RCU 14576 ± 17% -33.3% 9723 ± 15% softirqs.CPU13.RCU 18836 ± 15% -47.1% 9961 ± 8% softirqs.CPU14.RCU 20542 ± 17% -54.3% 9394 ± 11% softirqs.CPU15.RCU 17485 ± 19% -42.0% 10141 ± 19% softirqs.CPU16.RCU 17387 ± 17% -37.8% 10807 ± 13% softirqs.CPU17.RCU 18473 ± 15% -49.9% 9250 ± 18% softirqs.CPU18.RCU 19232 ± 19% -44.1% 10751 ± 10% softirqs.CPU19.RCU 17052 ± 21% -42.9% 9744 ± 22% softirqs.CPU2.RCU 17330 ± 17% -43.5% 9797 ± 16% softirqs.CPU20.RCU 18624 ± 15% -43.9% 10445 ± 24% softirqs.CPU21.RCU 18206 ± 18% -44.4% 10123 ± 26% softirqs.CPU22.RCU 18047 ± 12% -44.5% 10022 ± 23% softirqs.CPU23.RCU 19351 ± 18% -47.7% 10115 ± 21% softirqs.CPU24.RCU 18629 ± 17% -51.4% 9057 ± 18% softirqs.CPU25.RCU 15355 ± 14% -44.8% 8480 ± 13% softirqs.CPU26.RCU 15129 ± 14% -39.7% 9117 ± 16% softirqs.CPU27.RCU 14744 ± 10% -39.0% 8996 ± 21% softirqs.CPU28.RCU 13973 ± 11% -39.4% 8470 ± 18% softirqs.CPU29.RCU 35779 ± 10% -27.5% 25930 ± 30% softirqs.CPU29.SCHED 18703 ± 16% -43.4% 10577 ± 19% softirqs.CPU3.RCU 17000 ± 19% -40.8% 10057 ± 21% softirqs.CPU30.RCU 18602 ± 16% -40.6% 11040 ± 15% softirqs.CPU31.RCU 17242 ± 23% -44.0% 9662 ± 19% softirqs.CPU32.RCU 17841 ± 15% -43.1% 10144 ± 21% softirqs.CPU33.RCU 17867 ± 14% -44.6% 9890 ± 21% softirqs.CPU34.RCU 19083 ± 16% -46.7% 10177 ± 15% softirqs.CPU35.RCU 19161 ± 11% -49.8% 9616 ± 24% softirqs.CPU36.RCU 19335 ± 15% -47.7% 10106 ± 16% softirqs.CPU37.RCU 20460 ± 11% -54.3% 9341 ± 17% softirqs.CPU38.RCU 10641 ± 58% +117.0% 23087 ± 43% softirqs.CPU38.SCHED 20165 ± 8% -53.5% 9381 ± 17% softirqs.CPU39.RCU 20158 ± 9% -47.3% 10630 ± 20% softirqs.CPU4.RCU 18503 ± 20% -40.7% 10980 ± 20% softirqs.CPU40.RCU 18917 ± 11% -51.1% 9241 ± 17% softirqs.CPU41.RCU 19989 ± 9% -57.3% 8544 ± 12% softirqs.CPU42.RCU 10393 ± 58% +155.8% 26585 ± 30% softirqs.CPU42.SCHED 16499 ± 11% -45.5% 8991 ± 19% softirqs.CPU43.RCU 18599 ± 16% -49.3% 9433 ± 24% softirqs.CPU44.RCU 20721 ± 8% -56.4% 9035 ± 14% softirqs.CPU45.RCU 6210 ± 26% +213.4% 19464 ± 52% softirqs.CPU45.SCHED 16912 ± 21% -43.6% 9547 ± 24% softirqs.CPU46.RCU 19085 ± 14% -49.9% 9571 ± 15% softirqs.CPU47.RCU 16524 ± 21% -46.7% 8815 ± 20% softirqs.CPU48.RCU 18590 ± 13% -52.0% 8921 ± 11% softirqs.CPU49.RCU 19473 ± 15% -47.6% 10198 ± 12% softirqs.CPU5.RCU 19807 ± 16% -51.7% 9574 ± 20% softirqs.CPU50.RCU 14621 ± 7% -44.6% 8100 ± 14% softirqs.CPU51.RCU 21579 ± 9% -49.4% 10911 ± 16% softirqs.CPU52.RCU 17758 ± 25% -50.6% 8778 ± 25% softirqs.CPU53.RCU 18495 ± 15% -47.1% 9776 ± 18% softirqs.CPU54.RCU 16729 ± 20% -46.4% 8963 ± 20% softirqs.CPU55.RCU 15031 ± 10% -44.4% 8358 ± 18% softirqs.CPU56.RCU 15721 ± 17% -42.2% 9080 ± 30% softirqs.CPU57.RCU 14757 ± 3% -32.1% 10022 ± 23% softirqs.CPU58.RCU 37022 ± 9% -41.4% 21679 ± 41% softirqs.CPU58.SCHED 16077 ± 6% -41.1% 9466 ± 32% softirqs.CPU59.RCU 20064 ± 11% -54.4% 9150 ± 18% softirqs.CPU6.RCU 14811 ± 18% -46.8% 7880 ± 15% softirqs.CPU60.RCU 15191 ± 8% -49.8% 7623 ± 14% softirqs.CPU61.RCU 16592 ± 13% -49.0% 8455 ± 12% softirqs.CPU62.RCU 15392 ± 21% -44.3% 8566 ± 17% softirqs.CPU63.RCU 18606 ± 16% -52.5% 8837 ± 19% softirqs.CPU65.RCU 14770 ± 22% -40.4% 8802 ± 29% softirqs.CPU66.RCU 14465 ± 11% -32.3% 9797 ± 24% softirqs.CPU67.RCU 16993 ± 16% -48.1% 8818 ± 12% softirqs.CPU68.RCU 18034 ± 14% -52.9% 8497 ± 18% softirqs.CPU69.RCU 18628 ± 16% -47.9% 9700 ± 18% softirqs.CPU7.RCU 16564 ± 17% -42.4% 9545 ± 15% softirqs.CPU70.RCU 16333 ± 19% -45.3% 8933 ± 25% softirqs.CPU71.RCU 17985 ± 19% -52.0% 8624 ± 25% softirqs.CPU72.RCU 16575 ± 17% -50.7% 8172 ± 15% softirqs.CPU73.RCU 17243 ± 18% -48.5% 8875 ± 15% softirqs.CPU74.RCU 17293 ± 12% -44.8% 9547 ± 15% softirqs.CPU75.RCU 16269 ± 20% -46.7% 8673 ± 24% softirqs.CPU76.RCU 17326 ± 18% -45.8% 9383 ± 22% softirqs.CPU77.RCU 18826 ± 18% -50.6% 9292 ± 24% softirqs.CPU78.RCU 19893 ± 8% -52.4% 9460 ± 18% softirqs.CPU79.RCU 18812 ± 19% -43.6% 10616 ± 19% softirqs.CPU8.RCU 20001 ± 12% -53.0% 9393 ± 14% softirqs.CPU80.RCU 20756 ± 5% -53.9% 9562 ± 18% softirqs.CPU81.RCU 18840 ± 15% -53.5% 8761 ± 17% softirqs.CPU82.RCU 17159 ± 21% -52.3% 8189 ± 20% softirqs.CPU83.RCU 18671 ± 16% -52.4% 8881 ± 19% softirqs.CPU84.RCU 18929 ± 11% -55.5% 8421 ± 21% softirqs.CPU85.RCU 17559 ± 15% -51.0% 8604 ± 13% softirqs.CPU86.RCU 17343 ± 14% -53.0% 8154 ± 18% softirqs.CPU87.RCU 15481 ± 13% -43.6% 8727 ± 14% softirqs.CPU88.RCU 16364 ± 14% -43.9% 9187 ± 24% softirqs.CPU89.RCU 18492 ± 19% -43.5% 10441 ± 14% softirqs.CPU9.RCU 14056 ± 7% -38.3% 8666 ± 21% softirqs.CPU90.RCU 14189 ± 6% -37.9% 8815 ± 24% softirqs.CPU91.RCU 15154 ± 14% -48.1% 7867 ± 15% softirqs.CPU92.RCU 15589 ± 14% -42.4% 8979 ± 25% softirqs.CPU93.RCU 14492 ± 10% -34.4% 9504 ± 19% softirqs.CPU94.RCU 34198 ± 16% -52.1% 16384 ± 44% softirqs.CPU94.SCHED 17203 ± 7% -46.2% 9249 ± 16% softirqs.CPU95.RCU 15367 ± 11% -44.0% 8601 ± 12% softirqs.CPU96.RCU 14328 ± 13% -37.8% 8911 ± 27% softirqs.CPU97.RCU 16134 ± 16% -46.3% 8672 ± 19% softirqs.CPU98.RCU 15282 ± 17% -44.5% 8486 ± 18% softirqs.CPU99.RCU 1808149 ± 6% -46.3% 971467 ± 15% softirqs.RCU 36250 ± 5% +36.3% 49404 ± 2% softirqs.TIMER 17.47 ± 9% -5.9 11.61 ± 9% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 9.44 ± 9% -3.2 6.23 ± 9% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64 8.57 ± 9% -2.9 5.64 ± 9% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64 2.27 ± 9% -1.0 1.27 ± 29% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.__libc_fcntl64 2.68 ± 9% -1.0 1.70 ± 9% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 2.65 ± 8% -0.9 1.78 ± 9% perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 2.52 ± 9% -0.9 1.65 ± 9% perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.80 ± 8% -0.5 0.28 ±100% perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.34 ± 9% -0.5 0.83 ± 9% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.62 ± 9% -0.5 1.12 ± 10% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl 0.82 ± 10% -0.4 0.38 ± 71% perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.66 ± 11% -0.4 0.28 ±100% perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.04 ± 9% -0.3 0.69 ± 9% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.86 ± 9% -0.3 0.59 ± 8% perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +0.6 0.59 ± 11% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.06 ± 9% +0.9 1.92 ± 12% perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 0.00 +0.9 0.92 ± 10% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 0.67 ± 9% +1.0 1.68 ± 14% perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.4 1.38 ± 8% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +1.8 1.79 ± 59% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk 0.00 +2.2 2.18 ± 58% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl 0.00 +2.2 2.21 ± 58% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +2.4 2.35 ± 43% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode 0.00 +2.8 2.80 ± 59% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 0.00 +2.8 2.84 ± 41% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait 0.00 +2.9 2.86 ± 41% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 1.10 ± 9% +2.9 3.98 ± 32% perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.00 +3.0 2.99 ± 55% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.69 ± 9% +3.8 5.49 ± 21% perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 3.39 ± 8% +4.0 7.42 ± 22% perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 3.00 ± 8% +4.2 7.17 ± 23% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +4.6 4.61 ± 29% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 0.00 +5.0 5.00 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 6.48 ± 9% +6.3 12.76 ± 12% perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 5.71 ± 9% +6.5 12.21 ± 13% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +7.3 7.35 ± 35% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock 17.27 ± 9% +8.5 25.78 ± 11% perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 13.80 ± 9% +9.6 23.35 ± 12% perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 28.29 ± 9% +13.3 41.58 ± 12% perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 26.73 ± 9% +13.8 40.53 ± 12% perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 23.74 ± 9% +14.8 38.56 ± 13% perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.55 ± 9% -5.9 11.68 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 9.74 ± 9% -3.3 6.41 ± 9% perf-profile.children.cycles-pp.syscall_return_via_sysret 9.02 ± 8% -3.1 5.96 ± 9% perf-profile.children.cycles-pp.__entry_text_start 4.02 ± 9% -1.5 2.53 ± 9% perf-profile.children.cycles-pp.memset_erms 2.58 ± 9% -0.9 1.69 ± 9% perf-profile.children.cycles-pp._copy_from_user 2.68 ± 8% -0.9 1.80 ± 9% perf-profile.children.cycles-pp.security_file_lock 2.02 ± 7% -0.7 1.33 ± 9% perf-profile.children.cycles-pp._raw_spin_lock 1.40 ± 8% -0.6 0.85 ± 10% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 9% -0.5 1.16 ± 10% perf-profile.children.cycles-pp.common_file_perm 1.48 ± 9% -0.5 0.98 ± 9% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 1.23 ± 9% -0.4 0.81 ± 9% perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.04 ± 9% -0.4 0.67 ± 9% perf-profile.children.cycles-pp.__might_sleep 0.86 ± 10% -0.3 0.56 ± 9% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.86 ± 9% -0.3 0.59 ± 8% perf-profile.children.cycles-pp.__fget_light 0.76 ± 7% -0.3 0.50 ± 8% perf-profile.children.cycles-pp.apparmor_file_lock 0.77 ± 8% -0.3 0.51 ± 9% perf-profile.children.cycles-pp.__cond_resched 0.59 ± 8% -0.2 0.38 ± 9% perf-profile.children.cycles-pp.__might_fault 0.66 ± 9% -0.2 0.46 ± 9% perf-profile.children.cycles-pp.locks_release_private 0.56 ± 9% -0.2 0.36 ± 9% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.58 ± 9% -0.2 0.44 ± 10% perf-profile.children.cycles-pp.locks_insert_lock_ctx 0.69 ± 11% -0.1 0.54 ± 9% perf-profile.children.cycles-pp.locks_delete_lock_ctx 0.56 ± 10% -0.1 0.44 ± 8% perf-profile.children.cycles-pp.locks_unlink_lock_ctx 0.38 ± 8% -0.1 0.26 ± 10% perf-profile.children.cycles-pp.rcu_all_qs 0.33 ± 11% -0.1 0.21 ± 9% perf-profile.children.cycles-pp.flock64_to_posix_lock 0.35 ± 8% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.aa_file_perm 0.27 ± 21% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.23 ± 9% -0.1 0.13 ± 8% perf-profile.children.cycles-pp.__init_waitqueue_head 0.31 ± 8% -0.1 0.21 ± 9% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.24 ± 9% -0.1 0.17 ± 6% perf-profile.children.cycles-pp.testcase 0.19 ± 11% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.should_failslab 0.15 ± 10% -0.1 0.09 ± 10% perf-profile.children.cycles-pp.fcntl@plt 0.16 ± 7% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.13 ± 9% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.security_file_fcntl 0.12 ± 12% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.__list_del_entry_valid 0.14 ± 9% -0.0 0.10 ± 10% perf-profile.children.cycles-pp.locks_get_lock_context 0.08 ± 10% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.vfs_lock_file 0.13 ± 8% -0.0 0.10 ± 7% perf-profile.children.cycles-pp.locks_delete_block 0.10 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__list_add_valid 0.10 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.memset 0.00 +0.1 0.08 ± 9% perf-profile.children.cycles-pp.obj_cgroup_uncharge 0.00 +0.2 0.21 ± 18% perf-profile.children.cycles-pp.mem_cgroup_from_task 0.00 +0.6 0.55 ± 9% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.00 +0.6 0.60 ± 9% perf-profile.children.cycles-pp.refill_obj_stock 1.10 ± 9% +0.9 1.96 ± 12% perf-profile.children.cycles-pp.locks_dispose_list 0.00 +1.7 1.67 ± 10% perf-profile.children.cycles-pp.mod_objcg_state 0.00 +2.4 2.42 ± 35% perf-profile.children.cycles-pp.propagate_protected_usage 0.00 +2.5 2.47 ± 9% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.00 +4.7 4.74 ± 36% perf-profile.children.cycles-pp.page_counter_cancel 0.00 +5.8 5.77 ± 35% perf-profile.children.cycles-pp.page_counter_uncharge 0.00 +5.8 5.82 ± 35% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages 0.00 +7.4 7.36 ± 35% perf-profile.children.cycles-pp.page_counter_try_charge 0.00 +7.4 7.42 ± 35% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 3.47 ± 9% +7.8 11.27 ± 18% perf-profile.children.cycles-pp.kmem_cache_free 0.00 +8.1 8.05 ± 32% perf-profile.children.cycles-pp.obj_cgroup_charge 17.35 ± 9% +8.5 25.81 ± 11% perf-profile.children.cycles-pp.do_lock_file_wait 14.01 ± 9% +9.5 23.56 ± 12% perf-profile.children.cycles-pp.posix_lock_inode 10.00 ± 8% +10.2 20.24 ± 14% perf-profile.children.cycles-pp.locks_alloc_lock 9.00 ± 8% +10.7 19.67 ± 14% perf-profile.children.cycles-pp.kmem_cache_alloc 28.35 ± 9% +13.3 41.61 ± 12% perf-profile.children.cycles-pp.__x64_sys_fcntl 26.89 ± 9% +13.7 40.63 ± 12% perf-profile.children.cycles-pp.do_fcntl 23.92 ± 9% +14.7 38.67 ± 13% perf-profile.children.cycles-pp.fcntl_setlk 17.12 ± 9% -5.7 11.40 ± 9% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 9.72 ± 9% -3.3 6.39 ± 9% perf-profile.self.cycles-pp.syscall_return_via_sysret 7.81 ± 8% -2.6 5.16 ± 9% perf-profile.self.cycles-pp.__entry_text_start 3.90 ± 9% -1.4 2.45 ± 9% perf-profile.self.cycles-pp.memset_erms 1.97 ± 7% -0.7 1.32 ± 10% perf-profile.self.cycles-pp._raw_spin_lock 1.78 ± 9% -0.6 1.21 ± 9% perf-profile.self.cycles-pp.__libc_fcntl64 1.34 ± 8% -0.5 0.82 ± 11% perf-profile.self.cycles-pp.___might_sleep 1.35 ± 9% -0.4 0.94 ± 10% perf-profile.self.cycles-pp.common_file_perm 1.19 ± 9% -0.4 0.79 ± 9% perf-profile.self.cycles-pp.copy_user_generic_unrolled 1.30 ± 8% -0.3 0.96 ± 8% perf-profile.self.cycles-pp.posix_lock_inode 0.91 ± 9% -0.3 0.58 ± 9% perf-profile.self.cycles-pp.__might_sleep 1.08 ± 9% -0.3 0.76 ± 8% perf-profile.self.cycles-pp.fcntl_setlk 0.74 ± 8% -0.3 0.43 ± 10% perf-profile.self.cycles-pp.locks_alloc_lock 0.79 ± 9% -0.3 0.51 ± 10% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.76 ± 9% -0.3 0.49 ± 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.83 ± 9% -0.3 0.56 ± 8% perf-profile.self.cycles-pp.__fget_light 0.70 ± 9% -0.2 0.46 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.66 ± 7% -0.2 0.43 ± 7% perf-profile.self.cycles-pp.apparmor_file_lock 0.64 ± 9% -0.2 0.42 ± 9% perf-profile.self.cycles-pp.locks_release_private 0.52 ± 9% -0.2 0.34 ± 9% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.47 ± 10% -0.2 0.30 ± 9% perf-profile.self.cycles-pp.do_lock_file_wait 0.54 ± 9% -0.2 0.38 ± 9% perf-profile.self.cycles-pp.do_fcntl 0.46 ± 9% -0.2 0.30 ± 8% perf-profile.self.cycles-pp.__x64_sys_fcntl 0.40 ± 11% -0.2 0.25 ± 11% perf-profile.self.cycles-pp.do_syscall_64 0.33 ± 11% -0.1 0.19 ± 10% perf-profile.self.cycles-pp.flock64_to_posix_lock 0.38 ± 9% -0.1 0.25 ± 9% perf-profile.self.cycles-pp.__cond_resched 0.25 ± 7% -0.1 0.14 ± 10% perf-profile.self.cycles-pp.locks_dispose_list 0.24 ± 9% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.testcase 0.28 ± 7% -0.1 0.19 ± 11% perf-profile.self.cycles-pp.aa_file_perm 0.22 ± 24% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.28 ± 8% -0.1 0.18 ± 10% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.25 ± 9% -0.1 0.16 ± 10% perf-profile.self.cycles-pp._copy_from_user 0.26 ± 8% -0.1 0.17 ± 10% perf-profile.self.cycles-pp.rcu_all_qs 0.18 ± 9% -0.1 0.10 ± 10% perf-profile.self.cycles-pp.__init_waitqueue_head 0.15 ± 8% -0.1 0.07 ± 14% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.14 ± 9% -0.1 0.08 ± 11% perf-profile.self.cycles-pp.fcntl@plt 0.12 ± 12% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.18 ± 12% -0.1 0.13 ± 8% perf-profile.self.cycles-pp.security_file_lock 0.08 ± 12% -0.0 0.03 ±100% perf-profile.self.cycles-pp.__list_add_valid 0.12 ± 9% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.locks_get_lock_context 0.10 ± 10% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__might_fault 0.10 ± 10% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.memset 0.10 ± 8% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.security_file_fcntl 0.11 ± 8% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.locks_delete_block 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.obj_cgroup_uncharge 0.00 +0.2 0.16 ± 24% perf-profile.self.cycles-pp.mem_cgroup_from_task 0.00 +0.3 0.34 ± 9% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.00 +0.6 0.56 ± 9% perf-profile.self.cycles-pp.obj_cgroup_charge 0.00 +0.6 0.58 ± 10% perf-profile.self.cycles-pp.refill_obj_stock 3.14 ± 8% +1.3 4.46 ± 10% perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +1.7 1.65 ± 10% perf-profile.self.cycles-pp.mod_objcg_state 0.00 +2.2 2.17 ± 9% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.00 +2.4 2.40 ± 35% perf-profile.self.cycles-pp.propagate_protected_usage 0.00 +4.7 4.70 ± 36% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +5.9 5.92 ± 35% perf-profile.self.cycles-pp.page_counter_try_charge will-it-scale.52.processes 7e+07 +-----------------------------------------------------------------+ | .+ + + + + | 6.5e+07 |+++ ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++ | | | 6e+07 |-+ | | | 5.5e+07 |-+ | | | 5e+07 |-+ O | | OO O O OOO OO O O OOO O | 4.5e+07 |O+O OOOO O O O O OOO O OOOOO O O O O OO O OOOO O | | O O O OO O O O| 4e+07 |-+ O O | | | 3.5e+07 +-----------------------------------------------------------------+ will-it-scale.per_process_ops 1.3e+06 +-----------------------------------------------------------------+ |+++.+++++++.++++++++.+++++++.+++++++.+++++++.++++++++.+++++++ | 1.2e+06 |-+ | | | | | 1.1e+06 |-+ | | | 1e+06 |-+ | | O | 900000 |-+ O O O O O O O O O O | |O O OOO O O OO O O O O O O OOOO O O O OO O O OOOO | | O O O O OO OO O OO O| 800000 |-O O O O | | | 700000 +-----------------------------------------------------------------+ will-it-scale.workload 7e+07 +-----------------------------------------------------------------+ | .+ + + + + | 6.5e+07 |+++ ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++ | | | 6e+07 |-+ | | | 5.5e+07 |-+ | | | 5e+07 |-+ O | | OO O O OOO OO O O OOO O | 4.5e+07 |O+O OOOO O O O O OOO O OOOOO O O O O OO O OOOO O | | O O O OO O O O| 4e+07 |-+ O O | | | 3.5e+07 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-icl-2sp2: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/lock1/will-it-scale/0xd000280 commit: b655843444 ("memcg: enable accounting for pollfd and select bits arrays") 0f12156dff ("memcg: enable accounting for file lock caches") b655843444152c0a 0f12156dff2862ac54235fc7270 ---------------- --------------------------- %stddev %change %stddev \ | \ 1.785e+08 -40.9% 1.055e+08 ± 2% will-it-scale.64.processes 2788911 -40.9% 1649196 ± 2% will-it-scale.per_process_ops 1.785e+08 -40.9% 1.055e+08 ± 2% will-it-scale.workload 366571 ± 7% +15.2% 422206 ± 9% numa-numastat.node0.local_node 2807 +2.4% 2874 vmstat.system.cs 0.02 ± 2% -0.0 0.02 ± 2% mpstat.cpu.all.soft% 5.55 -2.4 3.15 ± 7% mpstat.cpu.all.usr% 5364 ± 2% -85.5% 778.67 ± 13% slabinfo.Acpi-Parse.active_objs 5364 ± 2% -85.5% 778.67 ± 13% slabinfo.Acpi-Parse.num_objs 49038 +5.1% 51541 proc-vmstat.nr_active_anon 55209 +5.1% 58037 proc-vmstat.nr_shmem 49038 +5.1% 51541 proc-vmstat.nr_zone_active_anon 183.83 ± 28% +320.7% 773.33 ±101% interrupts.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5 836.50 ± 10% +46.3% 1223 ± 30% interrupts.CPU103.CAL:Function_call_interrupts 1348 ± 83% +208.2% 4155 ± 50% interrupts.CPU2.RES:Rescheduling_interrupts 1089 ± 77% +158.6% 2818 ± 53% interrupts.CPU4.RES:Rescheduling_interrupts 183.83 ± 28% +320.7% 773.33 ±101% interrupts.CPU5.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5 806.17 ± 14% +31.2% 1058 ± 30% interrupts.CPU55.CAL:Function_call_interrupts 723.17 ± 74% +417.0% 3738 ± 82% interrupts.CPU55.RES:Rescheduling_interrupts 4847 ± 94% -97.1% 138.17 ± 19% interrupts.CPU86.NMI:Non-maskable_interrupts 4847 ± 94% -97.1% 138.17 ± 19% interrupts.CPU86.PMI:Performance_monitoring_interrupts 8069 ± 63% -80.7% 1561 ±200% interrupts.CPU99.NMI:Non-maskable_interrupts 8069 ± 63% -80.7% 1561 ±200% interrupts.CPU99.PMI:Performance_monitoring_interrupts 2345 -12.0% 2063 perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 2345 -12.0% 2063 perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 262.68 ± 7% -42.8% 150.13 ± 12% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 561.17 ± 9% -16.5% 468.33 ± 5% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe 183.33 ± 7% +43.2% 262.50 ± 8% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 113.33 ± 11% +78.7% 202.50 ± 4% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 7031 -12.0% 6185 perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7031 -12.0% 6185 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7035 -12.0% 6189 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 2345 -12.0% 2063 perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 2345 -12.0% 2063 perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read 2.50 ± 7% -47.6% 1.31 ± 18% perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 262.68 ± 7% -42.8% 150.13 ± 12% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 7031 -12.0% 6185 perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 7031 -12.0% 6185 perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 7035 -12.0% 6189 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.04 ± 22% +1112.5% 0.50 ±128% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.02 ± 3% +1574.6% 0.27 ± 6% perf-stat.i.MPKI 7.076e+10 -6.5% 6.615e+10 ± 2% perf-stat.i.branch-instructions 0.05 +0.0 0.08 ± 5% perf-stat.i.branch-miss-rate% 35955313 ± 2% +45.6% 52354202 ± 5% perf-stat.i.branch-misses 10.31 +18.0 28.33 ± 8% perf-stat.i.cache-miss-rate% 492492 +4777.7% 24022051 ± 4% perf-stat.i.cache-misses 4620608 +1768.8% 86349315 ± 4% perf-stat.i.cache-references 2651 +2.5% 2716 perf-stat.i.context-switches 0.47 +8.5% 0.51 ± 2% perf-stat.i.cpi 425867 -98.4% 6992 ± 5% perf-stat.i.cycles-between-cache-misses 0.00 ± 4% +0.0 0.00 ± 11% perf-stat.i.dTLB-load-miss-rate% 1.03e+11 -3.8% 9.907e+10 ± 2% perf-stat.i.dTLB-loads 0.00 +0.0 0.00 ± 8% perf-stat.i.dTLB-store-miss-rate% 6.534e+10 -11.3% 5.796e+10 ± 2% perf-stat.i.dTLB-stores 3.516e+11 -7.2% 3.263e+11 ± 2% perf-stat.i.instructions 2.11 -7.1% 1.96 ± 2% perf-stat.i.ipc 37.50 ± 2% +2090.1% 821.35 ± 3% perf-stat.i.metric.K/sec 1867 -6.7% 1743 ± 2% perf-stat.i.metric.M/sec 104632 ± 2% +6707.0% 7122311 ± 3% perf-stat.i.node-load-misses 14377 ± 21% +3209.4% 475820 ± 29% perf-stat.i.node-loads 45.30 ± 16% +51.5 96.80 perf-stat.i.node-store-miss-rate% 48602 ± 14% +22917.3% 11186859 ± 6% perf-stat.i.node-store-misses 65851 ± 9% +432.6% 350742 ± 9% perf-stat.i.node-stores 0.01 +1902.5% 0.27 ± 6% perf-stat.overall.MPKI 0.05 +0.0 0.08 ± 6% perf-stat.overall.branch-miss-rate% 10.65 +17.3 27.92 ± 8% perf-stat.overall.cache-miss-rate% 0.47 +7.7% 0.51 ± 2% perf-stat.overall.cpi 336119 -97.9% 6940 ± 4% perf-stat.overall.cycles-between-cache-misses 0.00 ± 4% +0.0 0.00 ± 10% perf-stat.overall.dTLB-load-miss-rate% 0.00 +0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 2.11 -7.1% 1.96 ± 2% perf-stat.overall.ipc 87.73 ± 2% +6.0 93.76 perf-stat.overall.node-load-miss-rate% 42.34 ± 13% +54.6 96.95 perf-stat.overall.node-store-miss-rate% 594415 +56.9% 932831 perf-stat.overall.path-length 7.05e+10 -6.5% 6.592e+10 ± 2% perf-stat.ps.branch-instructions 35867377 +45.6% 52233576 ± 5% perf-stat.ps.branch-misses 493933 +4747.8% 23944727 ± 4% perf-stat.ps.cache-misses 4639810 +1756.0% 86115077 ± 4% perf-stat.ps.cache-references 2653 +2.6% 2721 perf-stat.ps.context-switches 1.027e+11 -3.8% 9.872e+10 ± 2% perf-stat.ps.dTLB-loads 6.51e+10 -11.3% 5.775e+10 ± 2% perf-stat.ps.dTLB-stores 3.504e+11 -7.2% 3.252e+11 ± 2% perf-stat.ps.instructions 104869 ± 2% +6668.1% 7097622 ± 3% perf-stat.ps.node-load-misses 14698 ± 21% +3134.0% 475371 ± 29% perf-stat.ps.node-loads 48497 ± 14% +22887.1% 11148161 ± 6% perf-stat.ps.node-store-misses 65971 ± 9% +430.7% 350139 ± 8% perf-stat.ps.node-stores 1.061e+14 -7.2% 9.845e+13 ± 2% perf-stat.total.instructions 11738 ± 11% -32.5% 7926 ± 19% softirqs.CPU1.RCU 12768 ± 12% -39.4% 7741 ± 11% softirqs.CPU10.RCU 8853 ± 10% -33.9% 5851 ± 8% softirqs.CPU100.RCU 9695 ± 6% -40.1% 5811 ± 5% softirqs.CPU101.RCU 9277 ± 7% -38.0% 5752 ± 7% softirqs.CPU102.RCU 9585 ± 8% -33.5% 6376 ± 17% softirqs.CPU103.RCU 9838 ± 7% -40.5% 5857 ± 9% softirqs.CPU104.RCU 9315 ± 8% -40.6% 5529 ± 7% softirqs.CPU105.RCU 9661 ± 10% -40.5% 5749 ± 5% softirqs.CPU106.RCU 9531 ± 6% -35.6% 6138 ± 7% softirqs.CPU108.RCU 9472 ± 7% -34.4% 6210 ± 16% softirqs.CPU109.RCU 11502 ± 13% -41.5% 6726 ± 14% softirqs.CPU11.RCU 9003 ± 11% -34.3% 5916 ± 12% softirqs.CPU111.RCU 9739 ± 7% -36.5% 6187 ± 10% softirqs.CPU113.RCU 9453 ± 9% -34.2% 6220 ± 13% softirqs.CPU114.RCU 9891 ± 7% -31.3% 6798 ± 18% softirqs.CPU115.RCU 10008 ± 9% -39.1% 6090 ± 12% softirqs.CPU116.RCU 10132 ± 17% -41.7% 5912 ± 15% softirqs.CPU117.RCU 9765 ± 8% -37.5% 6103 ± 8% softirqs.CPU118.RCU 8960 ± 11% -33.9% 5923 ± 8% softirqs.CPU119.RCU 10910 ± 17% -34.0% 7197 ± 6% softirqs.CPU12.RCU 9577 ± 6% -37.6% 5973 ± 12% softirqs.CPU120.RCU 9565 ± 10% -42.9% 5457 ± 7% softirqs.CPU121.RCU 9137 ± 10% -38.7% 5604 ± 11% softirqs.CPU124.RCU 9418 ± 13% -34.3% 6186 ± 13% softirqs.CPU125.RCU 9151 ± 15% -37.2% 5745 ± 11% softirqs.CPU126.RCU 10800 ± 7% -34.7% 7057 ± 10% softirqs.CPU127.RCU 9815 ± 9% -29.1% 6958 ± 6% softirqs.CPU13.RCU 11463 ± 15% -39.2% 6967 ± 12% softirqs.CPU14.RCU 10757 ± 15% -37.6% 6716 ± 10% softirqs.CPU15.RCU 10809 ± 17% -38.5% 6648 ± 16% softirqs.CPU16.RCU 11169 ± 14% -42.4% 6435 ± 13% softirqs.CPU17.RCU 12215 ± 11% -40.1% 7319 ± 13% softirqs.CPU19.RCU 11500 ± 12% -39.2% 6996 ± 10% softirqs.CPU2.RCU 11027 ± 18% -32.7% 7416 ± 9% softirqs.CPU20.RCU 11155 ± 17% -39.5% 6745 ± 5% softirqs.CPU21.RCU 11083 ± 16% -32.8% 7451 ± 12% softirqs.CPU22.RCU 11841 ± 15% -44.9% 6523 ± 13% softirqs.CPU23.RCU 11415 ± 16% -38.7% 6997 ± 15% softirqs.CPU24.RCU 11325 ± 5% -37.7% 7061 ± 8% softirqs.CPU25.RCU 12111 ± 11% -44.5% 6715 ± 11% softirqs.CPU26.RCU 11627 ± 11% -43.1% 6613 ± 11% softirqs.CPU27.RCU 11113 ± 3% -40.1% 6658 ± 10% softirqs.CPU28.RCU 10333 ± 13% -33.2% 6907 ± 8% softirqs.CPU29.RCU 12191 ± 13% -42.5% 7009 ± 13% softirqs.CPU3.RCU 10265 ± 11% -31.7% 7012 ± 9% softirqs.CPU30.RCU 11440 ± 13% -39.6% 6910 ± 17% softirqs.CPU31.RCU 10389 ± 12% -33.8% 6874 ± 11% softirqs.CPU32.RCU 11224 ± 10% -37.3% 7032 ± 19% softirqs.CPU33.RCU 10886 ± 13% -32.8% 7317 ± 16% softirqs.CPU34.RCU 10588 ± 9% -32.7% 7125 ± 6% softirqs.CPU35.RCU 9719 ± 16% -31.8% 6625 ± 14% softirqs.CPU36.RCU 11099 ± 12% -33.2% 7414 ± 12% softirqs.CPU37.RCU 10338 ± 9% -25.6% 7690 ± 14% softirqs.CPU38.RCU 10238 ± 11% -27.1% 7461 ± 9% softirqs.CPU39.RCU 11660 ± 15% -35.5% 7516 ± 12% softirqs.CPU4.RCU 10552 ± 13% -31.6% 7220 ± 12% softirqs.CPU40.RCU 10552 ± 10% -31.1% 7273 ± 7% softirqs.CPU41.RCU 10103 ± 13% -31.8% 6891 ± 9% softirqs.CPU42.RCU 10974 ± 8% -36.9% 6921 ± 10% softirqs.CPU43.RCU 10730 ± 13% -37.2% 6739 ± 17% softirqs.CPU44.RCU 11205 ± 12% -35.4% 7233 ± 11% softirqs.CPU45.RCU 11136 ± 12% -39.4% 6751 ± 12% softirqs.CPU46.RCU 10649 ± 11% -33.6% 7073 ± 8% softirqs.CPU47.RCU 9988 ± 10% -36.5% 6346 ± 8% softirqs.CPU48.RCU 9801 ± 9% -36.9% 6180 ± 6% softirqs.CPU49.RCU 12102 ± 9% -37.5% 7563 ± 17% softirqs.CPU5.RCU 9884 ± 12% -36.2% 6303 ± 7% softirqs.CPU50.RCU 9347 ± 9% -34.8% 6096 ± 7% softirqs.CPU51.RCU 10164 ± 11% -35.3% 6575 ± 7% softirqs.CPU52.RCU 10518 ± 15% -37.2% 6609 ± 10% softirqs.CPU53.RCU 9857 ± 12% -32.5% 6655 ± 5% softirqs.CPU54.RCU 10795 ± 13% -44.0% 6047 ± 5% softirqs.CPU55.RCU 10397 ± 12% -38.9% 6350 ± 7% softirqs.CPU56.RCU 9513 ± 6% -30.5% 6615 ± 8% softirqs.CPU57.RCU 10400 ± 10% -37.6% 6492 ± 5% softirqs.CPU58.RCU 10434 ± 8% -39.1% 6351 ± 9% softirqs.CPU59.RCU 12055 ± 14% -32.8% 8102 ± 22% softirqs.CPU6.RCU 10580 ± 12% -40.1% 6340 ± 11% softirqs.CPU60.RCU 10099 ± 11% -41.4% 5915 ± 7% softirqs.CPU61.RCU 10021 ± 10% -35.8% 6436 ± 3% softirqs.CPU62.RCU 9856 ± 7% -31.3% 6774 ± 9% softirqs.CPU64.RCU 9963 ± 12% -36.0% 6372 ± 8% softirqs.CPU65.RCU 9664 ± 8% -30.1% 6752 ± 5% softirqs.CPU66.RCU 9673 ± 5% -33.2% 6463 ± 10% softirqs.CPU67.RCU 9910 ± 8% -36.4% 6307 ± 6% softirqs.CPU68.RCU 9589 ± 5% -34.2% 6310 ± 7% softirqs.CPU69.RCU 11733 ± 9% -42.0% 6809 ± 12% softirqs.CPU7.RCU 9603 ± 42% +105.6% 19745 ± 32% softirqs.CPU7.SCHED 9881 ± 9% -34.0% 6521 ± 7% softirqs.CPU70.RCU 9620 ± 6% -33.2% 6424 ± 2% softirqs.CPU71.RCU 35921 ± 13% -39.4% 21764 ± 30% softirqs.CPU71.SCHED 10659 ± 13% -38.8% 6524 ± 7% softirqs.CPU72.RCU 9917 ± 7% -38.9% 6055 ± 7% softirqs.CPU73.RCU 9572 ± 7% -40.0% 5741 ± 9% softirqs.CPU74.RCU 9683 ± 9% -36.2% 6176 ± 6% softirqs.CPU75.RCU 10099 ± 7% -38.4% 6220 ± 10% softirqs.CPU76.RCU 9726 ± 5% -28.6% 6946 ± 14% softirqs.CPU78.RCU 9463 ± 6% -37.1% 5956 ± 12% softirqs.CPU79.RCU 10789 ± 12% -34.1% 7105 ± 20% softirqs.CPU8.RCU 9917 ± 7% -35.1% 6439 ± 11% softirqs.CPU80.RCU 9821 ± 4% -34.5% 6430 ± 3% softirqs.CPU81.RCU 9938 ± 12% -38.6% 6098 ± 10% softirqs.CPU82.RCU 9838 ± 4% -35.2% 6370 ± 7% softirqs.CPU83.RCU 10401 ± 8% -40.1% 6229 ± 6% softirqs.CPU84.RCU 10051 ± 6% -36.4% 6396 ± 9% softirqs.CPU85.RCU 9792 ± 4% -35.3% 6339 ± 14% softirqs.CPU86.RCU 10059 ± 9% -36.3% 6408 ± 8% softirqs.CPU87.RCU 9972 ± 8% -39.2% 6059 ± 3% softirqs.CPU88.RCU 9447 ± 9% -30.8% 6534 ± 12% softirqs.CPU89.RCU 11511 ± 24% -37.0% 7246 ± 11% softirqs.CPU9.RCU 9563 ± 6% -30.9% 6612 ± 6% softirqs.CPU90.RCU 36244 ± 11% -29.7% 25496 ± 25% softirqs.CPU90.SCHED 9671 ± 8% -34.4% 6340 ± 7% softirqs.CPU91.RCU 9848 ± 7% -33.7% 6531 ± 5% softirqs.CPU92.RCU 9578 ± 8% -32.4% 6479 ± 5% softirqs.CPU93.RCU 9519 ± 5% -36.3% 6064 ± 7% softirqs.CPU94.RCU 9480 ± 9% -31.0% 6544 ± 12% softirqs.CPU95.RCU 9025 ± 8% -35.2% 5845 ± 5% softirqs.CPU96.RCU 9656 ± 19% -39.8% 5814 ± 8% softirqs.CPU97.RCU 10008 ± 7% -39.4% 6064 ± 9% softirqs.CPU98.RCU 9619 ± 6% -41.1% 5661 ± 3% softirqs.CPU99.RCU 1308899 ± 3% -36.2% 835466 ± 3% softirqs.RCU 47568 ± 2% +20.6% 57361 ± 4% softirqs.TIMER 10.02 ± 3% -4.9 5.08 ± 18% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64 8.20 -3.8 4.38 ± 6% perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.28 -2.7 2.58 ± 5% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 4.78 -2.3 2.48 ± 7% perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 3.68 -1.8 1.88 ± 7% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl 3.36 -1.4 1.93 ± 6% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 2.98 -1.2 1.75 ± 6% perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 2.51 -1.2 1.30 ± 7% perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 2.49 -1.1 1.36 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 2.02 -1.0 1.05 ± 6% perf-profile.calltrace.cycles-pp.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.02 -1.0 1.06 ± 7% perf-profile.calltrace.cycles-pp.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.14 -0.8 0.37 ± 70% perf-profile.calltrace.cycles-pp.locks_release_private.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 1.70 -0.7 0.98 ± 7% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 1.24 -0.6 0.64 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.15 -0.6 1.57 ± 8% perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.34 -0.6 0.77 ± 6% perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.32 -0.6 0.75 ± 7% perf-profile.calltrace.cycles-pp.__might_fault._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64 1.22 -0.5 0.68 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait 1.21 -0.5 0.68 ± 7% perf-profile.calltrace.cycles-pp.aa_file_perm.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk 1.41 -0.5 0.90 ± 9% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 1.21 -0.5 0.70 ± 6% perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 1.08 -0.5 0.61 ± 7% perf-profile.calltrace.cycles-pp.flock64_to_posix_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.76 -0.4 0.35 ± 70% perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.27 -0.4 0.86 ± 5% perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 1.04 -0.4 0.66 ± 10% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64 0.76 -0.1 0.63 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 97.66 +0.4 98.07 perf-profile.calltrace.cycles-pp.__libc_fcntl64 0.00 +0.5 0.54 ± 4% perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +0.8 0.83 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +0.9 0.90 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 0.00 +0.9 0.92 ± 5% perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.3 1.26 ± 7% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 2.15 +1.4 3.54 ± 14% perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 0.00 +1.6 1.59 ± 11% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 1.14 +1.8 2.90 ± 18% perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +1.8 1.83 ± 6% perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +2.1 2.13 ± 29% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc 0.00 +2.2 2.16 ± 29% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk 0.00 +2.4 2.42 ± 7% perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 0.00 +2.6 2.60 ± 28% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl 0.00 +2.7 2.66 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl 0.00 +3.0 3.04 ± 37% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode 2.86 ± 2% +3.6 6.42 ± 10% perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 0.00 +3.6 3.59 ± 33% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk 0.00 +3.6 3.65 ± 36% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait 0.00 +3.7 3.72 ± 36% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk 0.00 +4.3 4.29 ± 27% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl 3.76 +5.7 9.47 ± 12% perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 8.70 +6.2 14.88 ± 6% perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 7.40 +6.3 13.74 ± 7% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl 83.98 +6.8 90.74 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 82.41 +7.4 89.82 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +7.8 7.75 ± 29% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode 46.22 +7.9 54.10 ± 3% perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64 78.68 +8.9 87.56 perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +9.2 9.15 ± 23% perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait 73.64 +11.0 84.60 perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64 0.00 +11.2 11.21 ± 25% perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock 38.04 +11.7 49.70 ± 4% perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl 16.21 +11.8 28.04 ± 5% perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl 13.60 +12.1 25.71 ± 6% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk 63.80 +15.2 78.96 perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.76 -3.4 4.33 ± 6% perf-profile.children.cycles-pp.memset_erms 8.05 -3.4 4.62 ± 7% perf-profile.children.cycles-pp._copy_from_user 5.70 ± 2% -2.8 2.89 ± 18% perf-profile.children.cycles-pp.__entry_text_start 6.00 -2.5 3.47 ± 6% perf-profile.children.cycles-pp._raw_spin_lock 5.10 -2.4 2.66 ± 6% perf-profile.children.cycles-pp.security_file_lock 5.00 -2.2 2.75 ± 8% perf-profile.children.cycles-pp.syscall_return_via_sysret 3.90 -1.9 2.01 ± 7% perf-profile.children.cycles-pp.common_file_perm 3.94 -1.7 2.28 ± 6% perf-profile.children.cycles-pp.copy_user_generic_unrolled 3.74 -1.7 2.08 ± 5% perf-profile.children.cycles-pp.___might_sleep 3.95 -1.6 2.40 ± 6% perf-profile.children.cycles-pp.__might_sleep 2.72 -1.3 1.41 ± 6% perf-profile.children.cycles-pp.locks_delete_lock_ctx 2.58 -1.3 1.29 ± 5% perf-profile.children.cycles-pp.locks_release_private 3.10 -1.3 1.81 ± 6% perf-profile.children.cycles-pp.__fget_light 2.18 -1.0 1.14 ± 6% perf-profile.children.cycles-pp.locks_unlink_lock_ctx 2.18 -1.0 1.15 ± 7% perf-profile.children.cycles-pp.locks_insert_lock_ctx 2.03 -0.9 1.17 ± 7% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.19 -0.8 1.34 ± 5% perf-profile.children.cycles-pp.__cond_resched 1.54 -0.7 0.86 ± 7% perf-profile.children.cycles-pp.aa_file_perm 1.54 -0.7 0.87 ± 7% perf-profile.children.cycles-pp.__might_fault 1.68 -0.6 1.07 ± 9% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 1.32 -0.6 0.76 ± 6% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 1.19 -0.5 0.67 ± 6% perf-profile.children.cycles-pp.flock64_to_posix_lock 1.10 -0.4 0.65 ± 5% perf-profile.children.cycles-pp.rcu_all_qs 0.81 -0.4 0.45 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.94 -0.3 0.60 ± 6% perf-profile.children.cycles-pp.__init_waitqueue_head 0.74 -0.3 0.41 ± 4% perf-profile.children.cycles-pp.locks_delete_block 0.75 -0.3 0.42 ± 5% perf-profile.children.cycles-pp.locks_get_lock_context 0.70 -0.3 0.39 ± 7% perf-profile.children.cycles-pp.locks_copy_lock 0.53 -0.3 0.23 ± 5% perf-profile.children.cycles-pp.__list_del_entry_valid 0.78 ± 2% -0.3 0.49 ± 9% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.61 -0.3 0.35 ± 8% perf-profile.children.cycles-pp.security_file_fcntl 0.60 -0.2 0.35 ± 11% perf-profile.children.cycles-pp.memset 0.53 -0.2 0.29 ± 6% perf-profile.children.cycles-pp.apparmor_file_lock 0.43 ± 2% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.43 ± 2% -0.2 0.25 ± 7% perf-profile.children.cycles-pp.vfs_lock_file 0.50 -0.2 0.33 ± 8% perf-profile.children.cycles-pp.testcase 0.37 -0.2 0.21 ± 6% perf-profile.children.cycles-pp.locks_copy_conflock 0.32 ± 2% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.__fdget_raw 0.36 -0.1 0.24 ± 7% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.22 -0.1 0.12 ± 6% perf-profile.children.cycles-pp.locks_move_blocks 0.41 -0.1 0.32 ± 6% perf-profile.children.cycles-pp.should_failslab 0.20 ± 2% -0.1 0.13 ± 10% perf-profile.children.cycles-pp.fcntl@plt 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.08 ± 11% perf-profile.children.cycles-pp.refill_stock 98.10 +0.1 98.23 perf-profile.children.cycles-pp.__libc_fcntl64 0.00 +0.3 0.30 ± 6% perf-profile.children.cycles-pp.obj_cgroup_uncharge 0.00 +0.5 0.53 ± 7% perf-profile.children.cycles-pp.mem_cgroup_from_task 2.41 +1.3 3.69 ± 14% perf-profile.children.cycles-pp.locks_dispose_list 0.00 +2.0 1.99 ± 5% perf-profile.children.cycles-pp.refill_obj_stock 0.32 ± 2% +2.1 2.40 ± 6% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.00 +3.4 3.39 ± 21% perf-profile.children.cycles-pp.propagate_protected_usage 0.00 +4.9 4.90 ± 8% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.00 +5.6 5.60 ± 6% perf-profile.children.cycles-pp.mod_objcg_state 0.00 +5.9 5.94 ± 26% perf-profile.children.cycles-pp.page_counter_cancel 84.29 +6.7 90.98 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.00 +7.1 7.12 ± 24% perf-profile.children.cycles-pp.page_counter_uncharge 83.10 +7.2 90.30 perf-profile.children.cycles-pp.do_syscall_64 0.00 +7.3 7.27 ± 24% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages 46.87 +7.6 54.45 ± 3% perf-profile.children.cycles-pp.do_lock_file_wait 79.26 +8.6 87.90 perf-profile.children.cycles-pp.__x64_sys_fcntl 74.02 +10.8 84.84 perf-profile.children.cycles-pp.do_fcntl 39.59 +11.0 50.57 ± 4% perf-profile.children.cycles-pp.posix_lock_inode 0.00 +11.2 11.23 ± 25% perf-profile.children.cycles-pp.page_counter_try_charge 0.00 +11.4 11.37 ± 24% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 8.04 +11.5 19.54 ± 6% perf-profile.children.cycles-pp.kmem_cache_free 0.00 +13.6 13.63 ± 19% perf-profile.children.cycles-pp.obj_cgroup_charge 64.58 +14.8 79.39 perf-profile.children.cycles-pp.fcntl_setlk 25.73 +17.7 43.45 ± 2% perf-profile.children.cycles-pp.locks_alloc_lock 22.71 +18.8 41.55 ± 3% perf-profile.children.cycles-pp.kmem_cache_alloc 7.42 -3.3 4.14 ± 6% perf-profile.self.cycles-pp.memset_erms 6.55 -3.2 3.35 ± 8% perf-profile.self.cycles-pp.__libc_fcntl64 5.72 -2.4 3.30 ± 6% perf-profile.self.cycles-pp._raw_spin_lock 5.14 -2.4 2.76 ± 5% perf-profile.self.cycles-pp.posix_lock_inode 5.00 -2.2 2.75 ± 8% perf-profile.self.cycles-pp.syscall_return_via_sysret 3.49 -1.7 1.77 ± 4% perf-profile.self.cycles-pp.fcntl_setlk 3.61 -1.5 2.09 ± 6% perf-profile.self.cycles-pp.copy_user_generic_unrolled 3.20 -1.4 1.77 ± 5% perf-profile.self.cycles-pp.___might_sleep 2.97 -1.2 1.74 ± 6% perf-profile.self.cycles-pp.__fget_light 2.35 -1.2 1.14 ± 7% perf-profile.self.cycles-pp.common_file_perm 3.02 -1.2 1.84 ± 6% perf-profile.self.cycles-pp.__might_sleep 2.25 -1.1 1.14 ± 5% perf-profile.self.cycles-pp.locks_release_private 2.38 -0.9 1.47 ± 8% perf-profile.self.cycles-pp.locks_alloc_lock 1.80 -0.8 1.04 ± 7% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.39 -0.7 0.71 ± 21% perf-profile.self.cycles-pp.__entry_text_start 1.61 -0.7 0.94 ± 7% perf-profile.self.cycles-pp.__x64_sys_fcntl 1.60 -0.6 0.95 ± 8% perf-profile.self.cycles-pp.do_fcntl 1.43 -0.6 0.79 ± 6% perf-profile.self.cycles-pp.do_lock_file_wait 1.40 -0.6 0.81 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.31 -0.6 0.73 ± 7% perf-profile.self.cycles-pp.aa_file_perm 1.16 -0.5 0.67 ± 6% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 1.08 -0.5 0.60 ± 6% perf-profile.self.cycles-pp.flock64_to_posix_lock 0.97 -0.4 0.56 ± 6% perf-profile.self.cycles-pp._copy_from_user 1.02 -0.4 0.61 ± 8% perf-profile.self.cycles-pp.do_syscall_64 1.04 -0.4 0.65 ± 4% perf-profile.self.cycles-pp.__cond_resched 0.81 -0.4 0.45 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.76 -0.4 0.42 ± 5% perf-profile.self.cycles-pp.security_file_lock 0.71 -0.3 0.38 ± 8% perf-profile.self.cycles-pp.locks_insert_lock_ctx 0.64 -0.3 0.32 ± 6% perf-profile.self.cycles-pp.locks_unlink_lock_ctx 0.75 ± 2% -0.3 0.44 ± 6% perf-profile.self.cycles-pp.rcu_all_qs 0.70 -0.3 0.42 ± 10% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.58 ± 2% -0.3 0.32 ± 5% perf-profile.self.cycles-pp.locks_dispose_list 0.43 -0.3 0.18 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.52 -0.2 0.29 ± 5% perf-profile.self.cycles-pp.locks_delete_block 0.53 -0.2 0.30 ± 5% perf-profile.self.cycles-pp.locks_get_lock_context 0.63 ± 2% -0.2 0.42 ± 5% perf-profile.self.cycles-pp.__init_waitqueue_head 0.43 -0.2 0.23 ± 9% perf-profile.self.cycles-pp.locks_delete_lock_ctx 0.56 ± 2% -0.2 0.36 ± 11% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.43 -0.2 0.25 ± 9% perf-profile.self.cycles-pp.security_file_fcntl 0.32 ± 2% -0.2 0.17 ± 7% perf-profile.self.cycles-pp.__list_add_valid 0.33 -0.2 0.18 ± 9% perf-profile.self.cycles-pp.locks_copy_lock 0.33 -0.1 0.18 ± 7% perf-profile.self.cycles-pp.__might_fault 0.42 ± 2% -0.1 0.27 ± 9% perf-profile.self.cycles-pp.testcase 0.33 ± 2% -0.1 0.18 ± 5% perf-profile.self.cycles-pp.vfs_lock_file 0.32 -0.1 0.18 ± 7% perf-profile.self.cycles-pp.apparmor_file_lock 0.32 ± 2% -0.1 0.18 ± 6% perf-profile.self.cycles-pp.locks_copy_conflock 0.30 ± 3% -0.1 0.18 ± 10% perf-profile.self.cycles-pp.memset 0.26 ± 3% -0.1 0.18 ± 7% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.17 ± 2% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.locks_move_blocks 0.11 ± 4% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.__fdget_raw 0.20 ± 3% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.should_failslab 0.10 ± 4% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.fcntl@plt 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.08 ± 11% perf-profile.self.cycles-pp.refill_stock 0.00 +0.1 0.14 ± 10% perf-profile.self.cycles-pp.obj_cgroup_uncharge 0.00 +0.4 0.36 ± 7% perf-profile.self.cycles-pp.mem_cgroup_from_task 0.11 ± 3% +0.8 0.92 ± 5% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.00 +1.8 1.85 ± 5% perf-profile.self.cycles-pp.refill_obj_stock 0.00 +2.0 2.05 ± 7% perf-profile.self.cycles-pp.obj_cgroup_charge 9.35 +3.0 12.32 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +3.4 3.36 ± 21% perf-profile.self.cycles-pp.propagate_protected_usage 0.00 +4.0 4.01 ± 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.00 +5.2 5.23 ± 7% perf-profile.self.cycles-pp.mod_objcg_state 0.00 +5.9 5.90 ± 26% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +9.0 8.97 ± 25% perf-profile.self.cycles-pp.page_counter_try_charge Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

2 weeks, 4 days

8
18
0 / 0

[xfs] 6191cf3ad5: stress-ng.rename.ops_per_sec -73.5% regression

by kernel test robot

Greeting, FYI, we noticed a -73.5% regression of stress-ng.rename.ops_per_sec due to commit: commit: 6191cf3ad59fda5901160633fef8e41b064a5246 ("xfs: flush inodegc workqueue tasks before cancel") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: stress-ng on test machine: 96 threads 2 sockets Ice Lake with 256G memory with following parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: xfs class: filesystem test: rename cpufreq_governor: performance ucode: 0xb000280 In addition to that, the commit also has significant impact on the following tests: If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: filesystem/gcc-9/performance/1HDD/xfs/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp1/rename/stress-ng/60s/0xb000280 commit: a8e422af69 ("xfs: remove unused xfs_ioctl32.h declarations") 6191cf3ad5 ("xfs: flush inodegc workqueue tasks before cancel") a8e422af69613300 6191cf3ad59fda5901160633fef ---------------- --------------------------- %stddev %change %stddev \ | \ 13809323 -73.5% 3655053 ± 2% stress-ng.rename.ops 230150 -73.5% 60915 ± 2% stress-ng.rename.ops_per_sec 68641 ± 6% -71.5% 19530 ± 18% stress-ng.time.involuntary_context_switches 762.40 +1.7% 775.00 stress-ng.time.percent_of_cpu_this_job_got 463.07 +3.3% 478.34 stress-ng.time.system_time 10.61 ± 5% -71.3% 3.04 ± 3% stress-ng.time.user_time 152587 ± 6% -72.6% 41861 ± 14% stress-ng.time.voluntary_context_switches 280117 ± 13% -22.7% 216667 ± 19% numa-numastat.node0.numa_hit 10272 ± 4% -48.7% 5273 ± 6% vmstat.system.cs 0.06 ± 21% -0.0 0.04 ± 4% mpstat.cpu.all.soft% 0.20 ± 3% -0.1 0.07 ± 2% mpstat.cpu.all.usr% 2703 ± 4% +5.6% 2854 ± 3% proc-vmstat.nr_active_anon 7988 ± 2% +2.5% 8185 proc-vmstat.nr_shmem 2703 ± 4% +5.6% 2854 ± 3% proc-vmstat.nr_zone_active_anon 522094 -10.7% 466068 proc-vmstat.numa_hit 436762 ± 2% -13.2% 379232 proc-vmstat.numa_local 4853 ± 6% +12.1% 5438 ± 3% proc-vmstat.pgactivate 525596 -10.4% 470724 proc-vmstat.pgalloc_normal 476799 -11.2% 423300 proc-vmstat.pgfree 61932 -2.6% 60328 proc-vmstat.pgpgout 3.60 ± 40% +102.9% 7.30 perf-stat.i.MPKI 4.774e+09 -71.1% 1.381e+09 perf-stat.i.branch-instructions 16114311 ± 32% -72.6% 4418605 ± 4% perf-stat.i.branch-misses 30.00 ± 24% +8.4 38.44 ± 4% perf-stat.i.cache-miss-rate% 20101026 ± 3% -8.7% 18358375 ± 3% perf-stat.i.cache-misses 73234568 ± 37% -34.7% 47830132 ± 2% perf-stat.i.cache-references 10261 ± 4% -50.5% 5079 ± 7% perf-stat.i.context-switches 1.29 ± 3% +227.7% 4.23 perf-stat.i.cpi 123.06 -8.1% 113.09 perf-stat.i.cpu-migrations 1474 ± 4% +9.8% 1619 ± 2% perf-stat.i.cycles-between-cache-misses 7.692e+11 ±199% -100.0% 202244 ± 46% perf-stat.i.dTLB-load-misses 5.182e+09 -70.1% 1.551e+09 perf-stat.i.dTLB-loads 107292 ± 95% -66.8% 35623 ± 23% perf-stat.i.dTLB-store-misses 2.564e+09 -76.9% 5.931e+08 ± 2% perf-stat.i.dTLB-stores 2.194e+10 -70.1% 6.557e+09 perf-stat.i.instructions 0.78 ± 2% -68.4% 0.25 perf-stat.i.ipc 130.67 -71.9% 36.71 perf-stat.i.metric.M/sec 93.21 +4.0 97.26 perf-stat.i.node-load-miss-rate% 3910237 ± 5% +52.1% 5948845 ± 3% perf-stat.i.node-load-misses 215305 ± 35% -69.9% 64873 ± 4% perf-stat.i.node-loads 55.52 ± 4% +14.4 69.90 perf-stat.i.node-store-miss-rate% 3175249 ± 9% +151.9% 7999504 ± 3% perf-stat.i.node-store-misses 2444045 ± 4% +32.3% 3234491 ± 2% perf-stat.i.node-stores 3.33 ± 36% +119.4% 7.29 perf-stat.overall.MPKI 30.29 ± 25% +8.1 38.42 ± 4% perf-stat.overall.cache-miss-rate% 1.28 ± 2% +232.8% 4.26 perf-stat.overall.cpi 1399 ± 5% +8.8% 1522 ± 3% perf-stat.overall.cycles-between-cache-misses 0.78 ± 2% -70.0% 0.23 perf-stat.overall.ipc 94.87 +4.0 98.92 perf-stat.overall.node-load-miss-rate% 56.43 ± 4% +14.8 71.21 perf-stat.overall.node-store-miss-rate% 4.699e+09 -71.1% 1.359e+09 perf-stat.ps.branch-instructions 15853475 ± 32% -72.6% 4344260 ± 4% perf-stat.ps.branch-misses 19790250 ± 3% -8.7% 18072625 ± 3% perf-stat.ps.cache-misses 72066501 ± 37% -34.7% 47079258 ± 2% perf-stat.ps.cache-references 10100 ± 4% -50.5% 4999 ± 7% perf-stat.ps.context-switches 120.98 -8.1% 111.24 perf-stat.ps.cpu-migrations 7.486e+11 ±199% -100.0% 198976 ± 46% perf-stat.ps.dTLB-load-misses 5.102e+09 -70.1% 1.526e+09 perf-stat.ps.dTLB-loads 105425 ± 95% -66.8% 34991 ± 23% perf-stat.ps.dTLB-store-misses 2.524e+09 -76.9% 5.837e+08 ± 2% perf-stat.ps.dTLB-stores 2.16e+10 -70.1% 6.454e+09 perf-stat.ps.instructions 3850372 ± 5% +52.1% 5857127 ± 3% perf-stat.ps.node-load-misses 211823 ± 35% -69.9% 63770 ± 4% perf-stat.ps.node-loads 3125462 ± 9% +152.0% 7875415 ± 3% perf-stat.ps.node-store-misses 2404750 ± 4% +32.4% 3183125 ± 2% perf-stat.ps.node-stores 1.364e+12 -70.1% 4.079e+11 perf-stat.total.instructions 18334 ± 40% -45.8% 9942 ± 27% softirqs.CPU0.RCU 25205 ± 59% -74.2% 6513 ± 35% softirqs.CPU1.RCU 11337 ± 24% -64.5% 4025 ± 16% softirqs.CPU10.RCU 3467 ±128% -93.0% 243.40 ± 45% softirqs.CPU11.NET_RX 9325 ± 16% -53.9% 4303 ± 31% softirqs.CPU12.RCU 18711 ± 69% -76.9% 4323 ± 42% softirqs.CPU14.RCU 11424 ± 24% -68.3% 3616 ± 23% softirqs.CPU16.RCU 10296 ± 10% -64.4% 3663 ± 15% softirqs.CPU17.RCU 16686 ± 80% -73.0% 4497 ± 48% softirqs.CPU18.RCU 22801 ± 52% -75.3% 5620 ± 51% softirqs.CPU19.RCU 19856 ± 66% -76.0% 4775 ± 32% softirqs.CPU2.RCU 17692 ± 63% -67.3% 5787 ± 51% softirqs.CPU23.RCU 10098 ± 11% -24.2% 7656 ± 19% softirqs.CPU24.SCHED 28140 ± 38% -79.2% 5848 ± 77% softirqs.CPU26.RCU 25221 ± 51% -76.7% 5889 ± 64% softirqs.CPU27.RCU 19503 ± 56% -61.0% 7614 ± 53% softirqs.CPU28.RCU 21536 ± 32% -85.6% 3093 ± 13% softirqs.CPU29.RCU 22529 ± 40% -70.4% 6667 ± 64% softirqs.CPU3.RCU 16623 ± 48% -81.0% 3155 ± 14% softirqs.CPU30.RCU 12194 ± 36% -72.9% 3299 ± 3% softirqs.CPU31.RCU 14779 ± 66% -68.1% 4710 ± 41% softirqs.CPU35.RCU 15014 ± 48% -78.2% 3276 ± 13% softirqs.CPU39.RCU 13379 ± 53% -73.9% 3493 ± 28% softirqs.CPU41.RCU 13192 ± 44% -70.0% 3964 ± 45% softirqs.CPU42.RCU 8842 ± 27% -55.8% 3907 ± 44% softirqs.CPU44.RCU 10412 ± 21% -61.1% 4054 ± 24% softirqs.CPU47.RCU 24935 ± 43% -67.6% 8068 ± 75% softirqs.CPU49.RCU 14355 ± 23% -63.9% 5179 ± 77% softirqs.CPU5.RCU 24979 ± 74% -83.7% 4082 ± 28% softirqs.CPU50.RCU 16596 ± 35% -72.4% 4583 ± 37% softirqs.CPU51.RCU 14924 ± 37% -69.3% 4587 ± 45% softirqs.CPU52.RCU 10496 ± 17% -67.7% 3391 ± 23% softirqs.CPU54.RCU 11523 ± 27% -63.5% 4210 ± 37% softirqs.CPU58.RCU 10856 ± 23% -69.6% 3300 ± 14% softirqs.CPU6.RCU 9567 ± 12% -55.3% 4278 ± 34% softirqs.CPU60.RCU 9623 ± 13% -65.3% 3339 ± 20% softirqs.CPU61.RCU 17635 ± 52% -76.6% 4123 ± 38% softirqs.CPU62.RCU 11383 ± 23% -67.1% 3748 ± 16% softirqs.CPU64.RCU 12070 ± 16% -66.9% 3989 ± 9% softirqs.CPU65.RCU 13676 ± 50% -66.3% 4608 ± 54% softirqs.CPU66.RCU 25554 ± 51% -80.7% 4932 ± 38% softirqs.CPU67.RCU 19115 ± 76% -68.3% 6051 ± 34% softirqs.CPU68.RCU 22577 ± 40% -74.8% 5681 ± 50% softirqs.CPU72.RCU 28142 ± 51% -78.7% 6003 ± 56% softirqs.CPU73.RCU 25500 ± 37% -86.6% 3423 ± 28% softirqs.CPU74.RCU 17326 ± 38% -73.1% 4669 ± 36% softirqs.CPU75.RCU 20427 ± 44% -76.4% 4825 ± 34% softirqs.CPU76.RCU 22440 ± 35% -86.0% 3148 ± 14% softirqs.CPU77.RCU 15102 ± 31% -79.1% 3155 ± 12% softirqs.CPU78.RCU 12795 ± 27% -75.0% 3204 ± 11% softirqs.CPU79.RCU 15211 ± 66% -75.0% 3805 ± 23% softirqs.CPU83.RCU 17019 ± 73% -81.4% 3162 ± 30% softirqs.CPU86.RCU 14440 ± 49% -78.2% 3154 ± 21% softirqs.CPU87.RCU 15354 ± 60% -74.2% 3957 ± 61% softirqs.CPU88.RCU 16994 ± 65% -79.9% 3411 ± 32% softirqs.CPU89.RCU 13048 ± 47% -69.2% 4016 ± 59% softirqs.CPU90.RCU 13869 ± 57% -75.8% 3360 ± 29% softirqs.CPU91.RCU 11657 ± 57% -71.4% 3328 ± 32% softirqs.CPU92.RCU 9599 ± 20% -56.5% 4175 ± 43% softirqs.CPU93.RCU 9690 ± 12% -69.3% 2974 ± 15% softirqs.CPU94.RCU 9935 ± 13% -56.1% 4358 ± 48% softirqs.CPU95.RCU 1526333 ± 5% -67.9% 489535 ± 7% softirqs.RCU 59.46 ± 3% -56.6 2.84 ± 4% perf-profile.calltrace.cycles-pp.rename 59.31 ± 3% -56.5 2.80 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.rename 59.28 ± 3% -56.5 2.80 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.rename 59.20 ± 3% -56.4 2.78 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe.rename 58.75 ± 3% -56.1 2.66 ± 4% perf-profile.calltrace.cycles-pp.do_renameat2.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe.rename 47.67 ± 3% -47.7 0.00 perf-profile.calltrace.cycles-pp.lock_rename.do_renameat2.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe 46.96 ± 3% -47.0 0.00 perf-profile.calltrace.cycles-pp.__mutex_lock.lock_rename.do_renameat2.__x64_sys_rename.do_syscall_64 36.08 ± 3% -36.1 0.00 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.lock_rename.do_renameat2.__x64_sys_rename 11.08 ± 9% -11.1 0.00 perf-profile.calltrace.cycles-pp.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs 9.71 ± 2% -9.7 0.00 perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.lock_rename.do_renameat2.__x64_sys_rename 11.05 ± 4% -8.6 2.43 ± 4% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs 6.60 ± 15% -6.6 0.00 perf-profile.calltrace.cycles-pp.__flush_work.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 7.71 ± 2% -5.9 1.86 ± 4% perf-profile.calltrace.cycles-pp.vfs_rename.do_renameat2.__x64_sys_rename.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.47 ± 2% -4.9 1.53 ± 4% perf-profile.calltrace.cycles-pp.xfs_vn_rename.vfs_rename.do_renameat2.__x64_sys_rename.do_syscall_64 6.43 ± 2% -4.9 1.53 ± 4% perf-profile.calltrace.cycles-pp.xfs_rename.xfs_vn_rename.vfs_rename.do_renameat2.__x64_sys_rename 3.87 ± 2% -2.9 0.97 ± 23% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 2.49 ± 3% -1.9 0.62 ± 3% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_rename.xfs_vn_rename.vfs_rename.do_renameat2 2.28 ± 2% -1.7 0.59 ± 4% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_rename.xfs_vn_rename.vfs_rename 0.00 +9.1 9.15 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.flush_workqueue_prep_pwqs.flush_workqueue.xfs_fs_statfs.statfs_by_dentry 0.00 +10.1 10.14 ± 2% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.flush_workqueue.xfs_fs_statfs.statfs_by_dentry 0.00 +10.2 10.21 ± 2% perf-profile.calltrace.cycles-pp.flush_workqueue_prep_pwqs.flush_workqueue.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 13.53 ± 2% +27.9 41.41 perf-profile.calltrace.cycles-pp.statvfs64 13.47 ± 2% +27.9 41.39 perf-profile.calltrace.cycles-pp.__statfs.statvfs64 13.44 ± 2% +27.9 41.38 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 13.44 ± 2% +27.9 41.38 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 13.41 ± 2% +28.0 41.37 perf-profile.calltrace.cycles-pp.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 13.41 +30.4 43.83 perf-profile.calltrace.cycles-pp.__statfs 13.21 +30.6 43.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__statfs 13.18 +30.6 43.77 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 13.09 +30.6 43.74 perf-profile.calltrace.cycles-pp.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 26.34 +58.7 85.07 perf-profile.calltrace.cycles-pp.user_statfs.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 0.00 +59.8 59.78 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.flush_workqueue.xfs_fs_statfs.statfs_by_dentry 23.44 ± 2% +61.0 84.49 perf-profile.calltrace.cycles-pp.vfs_statfs.user_statfs.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe 23.40 ± 2% +61.1 84.47 perf-profile.calltrace.cycles-pp.statfs_by_dentry.vfs_statfs.user_statfs.__do_sys_statfs.do_syscall_64 23.25 ± 2% +61.2 84.44 perf-profile.calltrace.cycles-pp.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs.__do_sys_statfs 0.00 +70.5 70.49 perf-profile.calltrace.cycles-pp.__mutex_lock.flush_workqueue.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 0.00 +81.1 81.11 perf-profile.calltrace.cycles-pp.flush_workqueue.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs 59.50 ± 3% -56.6 2.85 ± 4% perf-profile.children.cycles-pp.rename 59.21 ± 3% -56.4 2.78 ± 4% perf-profile.children.cycles-pp.__x64_sys_rename 58.78 ± 3% -56.1 2.67 ± 4% perf-profile.children.cycles-pp.do_renameat2 47.67 ± 3% -47.6 0.10 ± 21% perf-profile.children.cycles-pp.lock_rename 11.20 ± 9% -10.5 0.75 ± 4% perf-profile.children.cycles-pp.xfs_inodegc_flush 11.50 ± 4% -9.0 2.55 ± 4% perf-profile.children.cycles-pp.__percpu_counter_sum 7.15 ± 14% -7.2 0.00 perf-profile.children.cycles-pp.__flush_work 7.73 ± 2% -5.9 1.87 ± 4% perf-profile.children.cycles-pp.vfs_rename 7.34 -5.6 1.75 ± 4% perf-profile.children.cycles-pp.cpumask_next 6.47 ± 2% -4.9 1.54 ± 4% perf-profile.children.cycles-pp.xfs_vn_rename 6.45 ± 2% -4.9 1.54 ± 4% perf-profile.children.cycles-pp.xfs_rename 4.65 -3.5 1.12 ± 3% perf-profile.children.cycles-pp._find_next_bit 2.71 ± 2% -2.0 0.75 ± 4% perf-profile.children.cycles-pp.xfs_inodegc_queue_all 2.51 ± 2% -1.9 0.63 ± 4% perf-profile.children.cycles-pp.__xfs_trans_commit 2.12 ± 7% -1.7 0.39 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 2.31 ± 2% -1.7 0.61 ± 4% perf-profile.children.cycles-pp.xlog_cil_commit 2.08 ± 3% -1.7 0.41 perf-profile.children.cycles-pp._raw_spin_lock 1.78 ± 6% -1.5 0.30 ± 3% perf-profile.children.cycles-pp.user_path_at_empty 1.40 ± 3% -1.3 0.10 ± 9% perf-profile.children.cycles-pp.__might_resched 1.66 ± 5% -1.3 0.39 ± 10% perf-profile.children.cycles-pp.dput 1.33 ± 7% -1.1 0.18 ± 4% perf-profile.children.cycles-pp.filename_lookup 1.44 ± 4% -1.1 0.32 ± 7% perf-profile.children.cycles-pp.xfs_trans_alloc 1.29 ± 7% -1.1 0.17 ± 4% perf-profile.children.cycles-pp.path_lookupat 1.35 ± 4% -1.1 0.29 ± 3% perf-profile.children.cycles-pp.__lookup_hash 1.28 ± 3% -1.0 0.29 ± 7% perf-profile.children.cycles-pp.xfs_trans_reserve 0.99 ± 4% -1.0 0.04 ± 50% perf-profile.children.cycles-pp.__cond_resched 1.01 ± 9% -0.9 0.10 ± 7% perf-profile.children.cycles-pp.complete_walk 1.16 ± 6% -0.9 0.27 ± 12% perf-profile.children.cycles-pp.lockref_put_return 0.97 ± 10% -0.9 0.10 ± 10% perf-profile.children.cycles-pp.try_to_unlazy 0.92 ± 10% -0.8 0.08 ± 9% perf-profile.children.cycles-pp.__legitimize_path 0.90 ± 4% -0.8 0.07 ± 9% perf-profile.children.cycles-pp.__might_sleep 1.09 ± 5% -0.8 0.27 ± 13% perf-profile.children.cycles-pp.path_put 0.97 ± 6% -0.8 0.19 ± 9% perf-profile.children.cycles-pp.xfs_log_reserve 0.83 ± 11% -0.8 0.05 ± 9% perf-profile.children.cycles-pp.lockref_get_not_dead 0.98 ± 5% -0.7 0.28 ± 4% perf-profile.children.cycles-pp.xlog_cil_insert_items 0.88 ± 3% -0.6 0.23 ± 5% perf-profile.children.cycles-pp.filename_parentat 0.83 ± 2% -0.6 0.22 ± 6% perf-profile.children.cycles-pp.path_parentat 0.84 -0.6 0.23 ± 4% perf-profile.children.cycles-pp.xfs_trans_log_inode 0.80 -0.6 0.22 ± 6% perf-profile.children.cycles-pp.getname_flags 0.66 ± 2% -0.5 0.15 ± 7% perf-profile.children.cycles-pp.kmem_cache_alloc 0.70 ± 2% -0.5 0.19 ± 8% perf-profile.children.cycles-pp.xfs_dir_createname 0.65 ± 3% -0.5 0.17 ± 13% perf-profile.children.cycles-pp.__vsnprintf_chk 0.59 ± 8% -0.5 0.11 ± 7% perf-profile.children.cycles-pp.xfs_vn_lookup 0.60 ± 4% -0.4 0.17 ± 4% perf-profile.children.cycles-pp.link_path_walk 0.54 ± 2% -0.4 0.15 ± 7% perf-profile.children.cycles-pp.strncpy_from_user 0.54 ± 4% -0.4 0.15 ± 10% perf-profile.children.cycles-pp.xfs_dir2_sf_addname 0.53 ± 6% -0.4 0.14 ± 6% perf-profile.children.cycles-pp.d_move 0.53 ± 2% -0.4 0.16 ± 4% perf-profile.children.cycles-pp.xfs_log_ticket_ungrant 0.49 ± 5% -0.4 0.12 ± 11% perf-profile.children.cycles-pp.vfprintf 1.22 ± 38% -0.3 0.87 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.47 ± 5% -0.3 0.14 ± 4% perf-profile.children.cycles-pp.xfs_inode_item_format 0.42 ± 4% -0.3 0.10 ± 11% perf-profile.children.cycles-pp.path_init 0.76 ± 11% -0.3 0.46 ± 5% perf-profile.children.cycles-pp.__softirqentry_text_start 0.40 ± 4% -0.3 0.10 ± 11% perf-profile.children.cycles-pp.lookup_dcache 0.39 ± 4% -0.3 0.10 ± 9% perf-profile.children.cycles-pp.d_lookup 0.39 ± 12% -0.3 0.10 ± 8% perf-profile.children.cycles-pp.xfs_mod_fdblocks 0.36 ± 13% -0.3 0.09 ± 13% perf-profile.children.cycles-pp.xfs_lookup 0.35 ± 3% -0.3 0.08 ± 5% perf-profile.children.cycles-pp.d_alloc 0.35 ± 13% -0.3 0.09 ± 13% perf-profile.children.cycles-pp.xfs_dir_lookup 0.76 ± 19% -0.3 0.50 ± 7% perf-profile.children.cycles-pp.irq_exit_rcu 0.34 ± 6% -0.3 0.09 perf-profile.children.cycles-pp.xfs_dir_removename 0.72 ± 43% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.tick_sched_handle 0.32 ± 5% -0.2 0.08 ± 12% perf-profile.children.cycles-pp._IO_default_xsputn 0.31 ± 10% -0.2 0.08 ± 7% perf-profile.children.cycles-pp.rcu_core 0.28 ± 3% -0.2 0.05 perf-profile.children.cycles-pp.__d_alloc 0.31 ± 6% -0.2 0.08 ± 5% perf-profile.children.cycles-pp.inode_permission 0.69 ± 40% -0.2 0.47 ± 3% perf-profile.children.cycles-pp.update_process_times 0.31 ± 6% -0.2 0.10 ± 9% perf-profile.children.cycles-pp.xlog_grant_add_space 0.28 ± 13% -0.2 0.07 ± 10% perf-profile.children.cycles-pp.down_read 0.24 ± 7% -0.2 0.07 ± 11% perf-profile.children.cycles-pp.__d_move 0.29 ± 10% -0.2 0.12 ± 8% perf-profile.children.cycles-pp.kthread 0.21 ± 7% -0.2 0.05 ± 50% perf-profile.children.cycles-pp.rcu_do_batch 0.29 ± 10% -0.2 0.13 ± 6% perf-profile.children.cycles-pp.ret_from_fork 0.19 ± 6% -0.2 0.03 ± 81% perf-profile.children.cycles-pp.xlog_space_left 0.19 ± 7% -0.2 0.03 ± 82% perf-profile.children.cycles-pp.xlog_grant_push_ail 0.23 ± 6% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.kmem_cache_free 0.21 ± 4% -0.2 0.05 ± 9% perf-profile.children.cycles-pp.__kmalloc 0.19 ± 6% -0.2 0.03 ± 82% perf-profile.children.cycles-pp.xlog_grant_push_threshold 0.22 ± 4% -0.2 0.06 ± 12% perf-profile.children.cycles-pp.__check_object_size 0.21 ± 4% -0.2 0.06 ± 6% perf-profile.children.cycles-pp.kmem_alloc 0.21 ± 9% -0.2 0.06 ± 6% perf-profile.children.cycles-pp.xfs_dir2_sf_removename 0.19 ± 6% -0.1 0.05 perf-profile.children.cycles-pp.make_kuid 0.20 ± 7% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.map_id_range_down 0.19 ± 6% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.xfs_lock_inodes 0.29 ± 7% -0.1 0.15 ± 12% perf-profile.children.cycles-pp.mutex_unlock 0.24 ± 8% -0.1 0.10 ± 9% perf-profile.children.cycles-pp.__d_lookup 0.17 ± 7% -0.1 0.03 ± 81% perf-profile.children.cycles-pp.__entry_text_start 0.20 ± 5% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.__schedule 0.19 ± 7% -0.1 0.05 ± 7% perf-profile.children.cycles-pp.generic_permission 0.18 ± 9% -0.1 0.04 ± 51% perf-profile.children.cycles-pp.schedule 0.18 ± 6% -0.1 0.05 ± 52% perf-profile.children.cycles-pp.fsnotify_get_cookie 0.35 ± 4% -0.1 0.23 ± 6% perf-profile.children.cycles-pp.mutex_lock 0.17 ± 6% -0.1 0.05 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.40 ± 24% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.scheduler_tick 0.23 ± 9% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.load_balance 0.20 ± 5% -0.1 0.10 ± 7% perf-profile.children.cycles-pp.find_busiest_group 0.19 ± 5% -0.1 0.10 ± 7% perf-profile.children.cycles-pp.update_sd_lb_stats 0.16 ± 34% -0.1 0.09 ± 11% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.12 ± 61% -0.1 0.05 ± 52% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.17 ± 11% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.osq_unlock 0.13 ± 18% -0.1 0.06 ± 56% perf-profile.children.cycles-pp.start_kernel 0.10 ± 28% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.process_one_work 0.12 ± 26% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.worker_thread 0.12 ± 17% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.update_rq_clock 0.07 ± 31% -0.0 0.03 ± 81% perf-profile.children.cycles-pp.update_blocked_averages 0.07 ± 37% -0.0 0.04 ± 50% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.1 0.08 ± 15% perf-profile.children.cycles-pp._IO_setb 9.72 ± 2% +0.4 10.16 perf-profile.children.cycles-pp.mutex_spin_on_owner 86.17 ± 2% +2.0 88.15 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 86.13 ± 2% +2.0 88.13 perf-profile.children.cycles-pp.do_syscall_64 2.35 ± 40% +6.9 9.23 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.00 +10.3 10.26 ± 2% perf-profile.children.cycles-pp.flush_workqueue_prep_pwqs 46.97 ± 3% +23.5 70.52 perf-profile.children.cycles-pp.__mutex_lock 36.10 ± 3% +23.7 59.81 perf-profile.children.cycles-pp.osq_lock 13.54 ± 2% +27.9 41.41 perf-profile.children.cycles-pp.statvfs64 26.95 +58.3 85.24 perf-profile.children.cycles-pp.__statfs 26.51 +58.6 85.12 perf-profile.children.cycles-pp.__do_sys_statfs 26.34 +58.7 85.07 perf-profile.children.cycles-pp.user_statfs 23.44 ± 2% +61.0 84.49 perf-profile.children.cycles-pp.vfs_statfs 23.40 ± 2% +61.1 84.47 perf-profile.children.cycles-pp.statfs_by_dentry 23.26 ± 2% +61.2 84.44 perf-profile.children.cycles-pp.xfs_fs_statfs 0.00 +81.1 81.12 perf-profile.children.cycles-pp.flush_workqueue 5.07 ± 6% -4.0 1.04 ± 4% perf-profile.self.cycles-pp.__percpu_counter_sum 3.96 -3.0 0.96 ± 3% perf-profile.self.cycles-pp._find_next_bit 2.75 ± 2% -2.1 0.65 ± 6% perf-profile.self.cycles-pp.cpumask_next 2.04 ± 3% -1.6 0.40 perf-profile.self.cycles-pp._raw_spin_lock 1.28 ± 3% -1.2 0.10 ± 14% perf-profile.self.cycles-pp.__might_resched 1.44 ± 6% -1.1 0.38 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.15 ± 6% -0.9 0.27 ± 12% perf-profile.self.cycles-pp.lockref_put_return 1.21 ± 2% -0.9 0.34 ± 5% perf-profile.self.cycles-pp.xfs_inodegc_queue_all 0.82 ± 11% -0.8 0.05 ± 9% perf-profile.self.cycles-pp.lockref_get_not_dead 0.73 ± 5% -0.7 0.06 ± 6% perf-profile.self.cycles-pp.__might_sleep 0.62 ± 4% -0.4 0.17 ± 8% perf-profile.self.cycles-pp.xfs_trans_log_inode 0.86 ± 2% -0.4 0.43 ± 3% perf-profile.self.cycles-pp.__mutex_lock 0.52 ± 2% -0.4 0.16 ± 4% perf-profile.self.cycles-pp.xfs_log_ticket_ungrant 0.47 ± 5% -0.4 0.11 ± 10% perf-profile.self.cycles-pp.vfprintf 0.41 ± 4% -0.3 0.10 ± 10% perf-profile.self.cycles-pp.path_init 0.30 ± 6% -0.2 0.09 ± 8% perf-profile.self.cycles-pp.xfs_inode_item_format 0.31 ± 6% -0.2 0.10 ± 9% perf-profile.self.cycles-pp.xlog_grant_add_space 0.28 ± 6% -0.2 0.08 ± 12% perf-profile.self.cycles-pp._IO_default_xsputn 0.26 ± 2% -0.2 0.07 ± 15% perf-profile.self.cycles-pp.strncpy_from_user 0.24 ± 14% -0.2 0.06 ± 12% perf-profile.self.cycles-pp.down_read 0.23 ± 3% -0.2 0.06 ± 12% perf-profile.self.cycles-pp.kmem_cache_alloc 0.22 ± 3% -0.2 0.06 ± 15% perf-profile.self.cycles-pp.link_path_walk 0.19 ± 5% -0.2 0.03 ± 81% perf-profile.self.cycles-pp.xlog_space_left 0.29 ± 6% -0.1 0.15 ± 12% perf-profile.self.cycles-pp.mutex_unlock 0.18 ± 5% -0.1 0.05 ± 52% perf-profile.self.cycles-pp.fsnotify_get_cookie 0.18 ± 6% -0.1 0.05 perf-profile.self.cycles-pp.map_id_range_down 0.22 ± 8% -0.1 0.09 ± 12% perf-profile.self.cycles-pp.__d_lookup 0.17 ± 6% -0.1 0.05 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.33 ± 4% -0.1 0.21 ± 4% perf-profile.self.cycles-pp.mutex_lock 0.17 ± 11% -0.1 0.10 ± 11% perf-profile.self.cycles-pp.osq_unlock 0.14 ± 7% -0.1 0.07 ± 9% perf-profile.self.cycles-pp.update_sd_lb_stats 0.15 ± 34% -0.1 0.09 ± 11% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.13 ± 8% -0.0 0.11 ± 8% perf-profile.self.cycles-pp.native_sched_clock 0.00 +0.1 0.08 ± 12% perf-profile.self.cycles-pp._IO_setb 9.67 ± 2% +0.5 10.13 ± 2% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.00 +1.1 1.07 ± 4% perf-profile.self.cycles-pp.flush_workqueue_prep_pwqs 2.33 ± 40% +6.9 9.18 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irq 35.94 ± 3% +23.7 59.65 perf-profile.self.cycles-pp.osq_lock Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

2 months, 2 weeks

2
1
0 / 0

[PATCH v2 0/4] Fix softlockup when adding inotify watch

by Stephen Brennan

Hello, I'd like to request re-testing this series on LKP, before posting my v2 more broadly. The 0-day bot detected a warning on my v1 posting[1]. I believe v2 should address the warning but I'm still working on reproducing it via the information included in the email. According to the LKP website [2] it's possible to request testing by directly mailing this list, so I wanted to try that in case I can't reproduce the original. [1]: https://lore.kernel.org/linux-fsdevel/20211218081736.GA1071@xsang-OptiPle... [2]: https://01.org/lkp/documentation/0-day-test-service Thanks, Stephen (series summary below) When a system with large amounts of memory has several millions of negative dentries in a single directory, a softlockup can occur while adding an inotify watch: watchdog: BUG: soft lockup - CPU#20 stuck for 9s! [inotifywait:9528] CPU: 20 PID: 9528 Comm: inotifywait Kdump: loaded Not tainted 5.16.0-rc4.20211208.el8uek.rc1.x86_64 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.4.1 12/03/2020 RIP: 0010:__fsnotify_update_child_dentry_flags+0xad/0x120 Call Trace: <TASK> fsnotify_add_mark_locked+0x113/0x160 inotify_new_watch+0x130/0x190 inotify_update_watch+0x11a/0x140 __x64_sys_inotify_add_watch+0xef/0x140 do_syscall_64+0x3b/0x90 entry_SYSCALL_64_after_hwframe+0x44/0xae v1 of this series can be found at: https://lore.kernel.org/linux-fsdevel/20211214005337.161885-1-stephen.s.b... v1 itself is based on a prior series: https://lore.kernel.org/linux-fsdevel/1611235185-1685-1-git-send-email-ga... The strategy employed by this series is to move negative dentries to the end of the d_subdirs list, and mark them with a flag as "tail negative". Then, readers of the d_subdirs list, which are only interested in positive dentries, can stop reading once they reach the first tail negative dentry. By applying this patch, I'm able to avoid the above softlockup caused by 200 million negative dentries on my test system. Inotify watches are set up nearly instantly. Previously, Al expressed concern for: 1. Possible memory corruption due to use of lock_parent() in sweep_negative(), see patch 01 for fix. 2. The previous patch didn't catch all ways a negative dentry could become positive (d_add, d_instantiate_new), see patch 01. 3. The previous series contained a new negative dentry limit, which capped the negative dentry count at around 3 per hash bucket. I've dropped this patch from the series. Patches 2-4 are unmodified from the previous posting. Konstantin Khlebnikov (3): fsnotify: stop walking child dentries if remaining tail is negative dcache: add action D_WALK_SKIP_SIBLINGS to d_walk() dcache: stop walking siblings if remaining dentries all negative Stephen Brennan (1): dcache: sweep cached negative dentries to the end of list of siblings fs/dcache.c | 96 ++++++++++++++++++++++++++++++++++++++++-- fs/libfs.c | 3 ++ fs/notify/fsnotify.c | 6 ++- include/linux/dcache.h | 6 +++ 4 files changed, 106 insertions(+), 5 deletions(-) -- 2.30.2

3 months

2
7
0 / 0

[ocfs2] c42ff46f97: sysctl_table_check_failed

by kernel test robot

(please be noted we reported "[ocfs2] 46e33fd45a: sysctl_table_check_failed" on https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/KQ2F6TPJWMDV... when this commit on: commit: 46e33fd45a52bf03769906e64d8a8a1ab317777d ("ocfs2: simplify subdirectory registration with register_sysctl()") https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git 20211118-sysctl-cleanups-set-04 we resend this report as a reminder the issue still exists on maineline) Greeting, FYI, we noticed the following commit (built with clang-14): commit: c42ff46f97c1c25577a84fbfb111710d25a129e0 ("ocfs2: simplify subdirectory registration with register_sysctl()") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu Icelake-Server -smp 4 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +---------------------------+------------+------------+ | | e99f5e7479 | c42ff46f97 | +---------------------------+------------+------------+ | sysctl_table_check_failed | 0 | 10 | +---------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 187.975448][ T1] ksmbd: The ksmbd server is experimental, use at your own risk. [ 187.977238][ T1] QNX4 filesystem 0.2.3 registered. [ 187.979872][ T1] orangefs_debugfs_init: called with debug mask: :none: :0: [ 187.981449][ T1] orangefs_init: module version upstream loaded [ 187.983277][ T1] JFS: nTxBlock = 8192, nTxLock = 65536 [ 188.000617][ T1] sysctl table check failed: fs/ocfs2/nm Not a file [ 188.001528][ T1] sysctl table check failed: fs/ocfs2/nm No proc_handler [ 188.002483][ T1] sysctl table check failed: fs/ocfs2/nm bogus .mode 0555 [ 188.003437][ T1] CPU: 0 PID: 1 Comm: swapper Not tainted 5.16.0-11534-gc42ff46f97c1 #1 843f34bc11d03a69a557abb9c2a31a40f9711bc8 [ 188.005020][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 188.006289][ T1] Call Trace: [ 188.006805][ T1] <TASK> [ 188.007273][ T1] dump_stack_lvl (??:?) [ 188.007965][ T1] dump_stack (??:?) [ 188.008592][ T1] __register_sysctl_table (??:?) [ 188.009381][ T1] ? ocfs2_stack_glue_init (stackglue.c:?) To reproduce: # build kernel cd linux cp config-5.16.0-11534-gc42ff46f97c1 .config make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

3 months, 2 weeks

5
5
0 / 0

[tools headers UAPI] e2bcbd7769: kernel-selftests.ir.make_fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: e2bcbd7769ee8f05e1b3d10848aace98973844e4 ("tools headers UAPI: remove stale lirc.h") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: kernel-selftests version: kernel-selftests-x86_64-db530529-1_20220124 with following parameters: group: group-01 ucode: 0xe2 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: 8 threads Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz with 16G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2022-01-27 18:57:29 make -C ir make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir' gcc -Wall -O2 -I../../../include/uapi ir_loopback.c -o /usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir/ir_loopback ir_loopback.c: In function ‘main’: ir_loopback.c:147:20: error: ‘RC_PROTO_RCMM32’ undeclared (first use in this function); did you mean ‘RC_PROTO_RC6_MCE’? if (rc_proto == RC_PROTO_RCMM32 && ^~~~~~~~~~~~~~~ RC_PROTO_RC6_MCE ir_loopback.c:147:20: note: each undeclared identifier is reported only once for each function it appears in make: *** [../lib.mk:146: /usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir/ir_loopback] Error 1 make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir' 2022-01-27 18:57:29 make run_tests -C ir make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir' gcc -Wall -O2 -I../../../include/uapi ir_loopback.c -o /usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir/ir_loopback ir_loopback.c: In function ‘main’: ir_loopback.c:147:20: error: ‘RC_PROTO_RCMM32’ undeclared (first use in this function); did you mean ‘RC_PROTO_RC6_MCE’? if (rc_proto == RC_PROTO_RCMM32 && ^~~~~~~~~~~~~~~ RC_PROTO_RC6_MCE ir_loopback.c:147:20: note: each undeclared identifier is reported only once for each function it appears in make: *** [../lib.mk:146: /usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir/ir_loopback] Error 1 make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-8.3-kselftests-e2bcbd7769ee8f05e1b3d10848aace98973844e4/tools/testing/selftests/ir' To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

3 months, 2 weeks

5
10
0 / 0

[fs/exec] 80bd5afdd8: xfstests.generic.633.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 80bd5afdd8568e41fc3a75c695bb179e0d9eee4d ("[PATCH v3] fs/exec: require argv[0] presence in do_execveat_common()") url: https://github.com/0day-ci/linux/commits/Ariadne-Conill/fs-exec-require-a... base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git 2c271fe77d52a0555161926c232cd5bc07178b39 patch link: https://lore.kernel.org/lkml/20220127000724.15106-1-ariadne@dereferenced.org in testcase: xfstests version: xfstests-x86_64-972d710-1_20220127 with following parameters: disk: 4HDD fs: f2fs test: generic-group-31 ucode: 0xe2 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> user :warn : [ 208.077271] run fstests generic/633 at 2022-01-30 04:50:49 kern :warn : [ 208.529090] Attempted to run process '/dev/fd/5/file1' with NULL argv user :notice: [ 208.806756] generic/633 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//generic/633.out.bad) user :notice: [ 208.826454] --- tests/generic/633.out 2022-01-27 11:54:16.000000000 +0000 user :notice: [ 208.842458] +++ /lkp/benchmarks/xfstests/results//generic/633.out.bad 2022-01-30 04:50:49.769538285 +0000 user :notice: [ 208.859622] @@ -1,2 +1,4 @@ user :warn : [ 208.860623] run fstests generic/634 at 2022-01-30 04:50:49 user :notice: [ 208.866037] QA output created by 633 user :notice: [ 208.889262] Silence is golden user :notice: [ 208.901240] +idmapped-mounts.c: 3608: setid_binaries - Invalid argument - failure: sys_execveat user :notice: [ 208.918907] +idmapped-mounts.c: 13953: run_test - Success - failure: setid binaries on regular mounts user :notice: [ 208.935261] ... user :notice: [ 208.949376] (Run 'diff -u /lkp/benchmarks/xfstests/tests/generic/633.out /lkp/benchmarks/xfstests/results//generic/633.out.bad' to see the entire diff) To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

3 months, 2 weeks

5
10
0 / 0

I'll call back in an hour

by Vlada

Hi, I like a lot of things.. I just cannot list all of them here! But most of all.. My name is Vlada. http://earworks.com/?wptouch_switch=desktop&redirect=https%3A%2F%2Fyour-d... I like that I'm a woman, beautiful, shy and sometimes whimsical! I do believe that I would meet a reliable, loving and wise man with whom I share all the moments in my life! Is said that I am a tender, smart, with good sense of humor and honest girl. I am modest, balanced and loving. I respect people and their personal freedom. I'd like to have a nice family of full value, but unfortunately I can't find a partner among people around me Goodbye, Vlada.

3 months, 2 weeks

1
0
0 / 0

Most likely

by Zinaida

Hi. All my life I have been dreaming about a caring husband and happy children, everybody knows that for every woman the most important is family happiness. I would like to meet an honest. I am a very nice, sensual girl, so I would love my man to be kind and sympathetic too. My Profiles:374074 - https://slackliner.de/wiki/?wptouch_switch=desktop&redirect=https%3A%2F%2... My friends say that it is very difficult to resist my charm! I like jogging, it helps me to keep in a good form. I am able to listen and to find the mutual language with different people. I am waiting for you! I'm just a lady who wants to find love! Regards, Zinaida.

3 months, 2 weeks

1
0
0 / 0

[io_uring] 811b398582: WARNING:possible_recursive_locking_detected

by kernel test robot

Greeting, FYI, we noticed the following commit (built with clang-14): commit: 811b3985828e422a3759cf07a848fa75c17c1db4 ("io_uring: support for user allocated memory for rings/sqes") https://github.com/ammarfaizi2/linux-block axboe/linux-block/perf-wip in testcase: trinity version: trinity-static-x86_64-x86_64-1c734c75-1_2020-01-06 with following parameters: runtime: 300s group: group-02 test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 411.985606][ T3987] WARNING: possible recursive locking detected [ 411.986603][ T3987] 5.17.0-rc1-00119-g811b3985828e #1 Not tainted [ 411.987466][ T3987] -------------------------------------------- [ 411.988512][ T3987] trinity-c2/3987 is trying to acquire lock: [ 411.989352][ T3987] ffff888103bc1160 (&mm->mmap_lock#2){++++}-{3:3}, at: internal_get_user_pages_fast (gup.c:?) [ 411.991790][ T3987] [ 411.991790][ T3987] but task is already holding lock: [ 411.992859][ T3987] ffff888103bc1160 (&mm->mmap_lock#2){++++}-{3:3}, at: __io_uaddr_map (io_uring.c:?) [ 411.994141][ T3987] [ 411.994141][ T3987] other info that might help us debug this: [ 411.995262][ T3987] Possible unsafe locking scenario: [ 411.995262][ T3987] [ 411.996391][ T3987] CPU0 [ 411.996947][ T3987] ---- [ 411.997487][ T3987] lock(&mm->mmap_lock#2); [ 411.998169][ T3987] lock(&mm->mmap_lock#2); [ 411.998857][ T3987] [ 411.998857][ T3987] *** DEADLOCK *** [ 411.998857][ T3987] [ 412.000128][ T3987] May be due to missing lock nesting notation [ 412.000128][ T3987] [ 412.001283][ T3987] 1 lock held by trinity-c2/3987: [ 412.002016][ T3987] #0: ffff888103bc1160 (&mm->mmap_lock#2){++++}-{3:3}, at: __io_uaddr_map (io_uring.c:?) [ 412.003336][ T3987] [ 412.003336][ T3987] stack backtrace: [ 412.004261][ T3987] CPU: 0 PID: 3987 Comm: trinity-c2 Not tainted 5.17.0-rc1-00119-g811b3985828e #1 [ 412.005522][ T3987] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 412.006766][ T3987] Call Trace: [ 412.007338][ T3987] <TASK> [ 412.007880][ T3987] validate_chain (lockdep.c:?) [ 412.008646][ T3987] ? validate_chain (lockdep.c:?) [ 412.009361][ T3987] ? __io_uaddr_map (io_uring.c:?) [ 412.012103][ T3987] ? __se_sys_io_uring_setup (io_uring.c:?) [ 412.012912][ T3987] ? do_syscall_64 (??:?) [ 412.013651][ T3987] ? entry_SYSCALL_64_after_hwframe (??:?) [ 412.014519][ T3987] ? mark_lock (lockdep.c:?) [ 412.015181][ T3987] __lock_acquire (lockdep.c:?) [ 412.015875][ T3987] lock_acquire (??:?) [ 412.016538][ T3987] ? internal_get_user_pages_fast (gup.c:?) [ 412.017441][ T3987] internal_get_user_pages_fast (gup.c:?) [ 412.018272][ T3987] ? internal_get_user_pages_fast (gup.c:?) [ 412.019119][ T3987] ? pin_user_pages_fast (??:?) [ 412.019856][ T3987] __io_uaddr_map (io_uring.c:?) [ 412.020561][ T3987] io_allocate_scq_urings (io_uring.c:?) [ 412.021346][ T3987] io_uring_create (io_uring.c:?) [ 412.022084][ T3987] __se_sys_io_uring_setup (io_uring.c:?) [ 412.022885][ T3987] do_syscall_64 (??:?) [ 412.023559][ T3987] entry_SYSCALL_64_after_hwframe (??:?) [ 412.024393][ T3987] RIP: 0033:0x463519 [ 412.025011][ T3987] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db 59 00 00 c3 66 2e 0f 1f 84 00 00 00 00 All code ======== 0: 00 f3 add %dh,%bl 2: c3 retq 3: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) a: 00 00 00 d: 0f 1f 40 00 nopl 0x0(%rax) 11: 48 89 f8 mov %rdi,%rax 14: 48 89 f7 mov %rsi,%rdi 17: 48 89 d6 mov %rdx,%rsi 1a: 48 89 ca mov %rcx,%rdx 1d: 4d 89 c2 mov %r8,%r10 20: 4d 89 c8 mov %r9,%r8 23: 4c 8b 4c 24 08 mov 0x8(%rsp),%r9 28: 0f 05 syscall 2a:* 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax <-- trapping instruction 30: 0f 83 db 59 00 00 jae 0x5a11 36: c3 retq 37: 66 data16 38: 2e cs 39: 0f .byte 0xf 3a: 1f (bad) 3b: 84 00 test %al,(%rax) 3d: 00 00 add %al,(%rax) ... Code starting with the faulting instruction =========================================== 0: 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax 6: 0f 83 db 59 00 00 jae 0x59e7 c: c3 retq d: 66 data16 e: 2e cs f: 0f .byte 0xf 10: 1f (bad) 11: 84 00 test %al,(%rax) 13: 00 00 add %al,(%rax) ... [ 412.027422][ T3987] RSP: 002b:00007fff36077c88 EFLAGS: 00000246 ORIG_RAX: 00000000000001a9 [ 412.028649][ T3987] RAX: ffffffffffffffda RBX: 00000000000001a9 RCX: 0000000000463519 [ 412.029783][ T3987] RDX: 000000a437863b79 RSI: 0000000000000004 RDI: 00000000a4a4a4a4 [ 412.030907][ T3987] RBP: 00007faea05c0000 R08: fffffffffffffff6 R09: 004af9db521a5050 [ 412.032066][ T3987] R10: 00000000fafafafa R11: 0000000000000246 R12: 0000000000000002 [ 412.033184][ T3987] R13: 00007faea05c0058 R14: 000000000109a850 R15: 00007faea05c0000 [ 412.034309][ T3987] </TASK> [ 624.729797][ T417] sysrq: Emergency Sync [ 624.730827][ T10] Emergency Sync complete [ 624.731705][ T417] sysrq: Resetting Kboot worker: lkp-worker53 Elapsed time: 660 kvm=( qemu-system-x86_64 -enable-kvm -cpu SandyBridge -kernel $kernel -initrd initrd-vm-snb-45.cgz -m 16384 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::32032-:22 -boot order=nc -no-reboot -watchdog i6300esb -watchdog-action debug -rtc base=localtime -serial stdio -display none -monitor null ) append=( ip=::::vm-snb-45::dhcp root=/dev/ram0 RESULT_ROOT=/result/trinity/group-02-300s/vm-snb/yocto-x86_64-minimal-20190520.cgz/x86_64-randconfig-a015-20220124/clang-14/811b3985828e422a3759cf07a848fa75c17c1db4/9 BOOT_IMAGE=/pkg/linux/x86_64-randconfig-a015-20220124/clang-14/811b3985828e422a3759cf07a848fa75c17c1db4/vmlinuz-5.17.0-rc1-00119-g811b3985828e branch=ammarfaizi2-block/axboe/linux-block/perf-wip job=/job-script user=lkp ARCH=x86_64 kconfig=x86_64-randconfig-a015-20220124 commit=811b3985828e422a3759cf07a848fa75c17c1db4 vmalloc=128M initramfs_async=0 page_owner=on max_uptime=2100 result_service=tmpfs selinux=0 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 net.ifnames=0 printk.devkmsg=on panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 To reproduce: # build kernel cd linux cp config-5.17.0-rc1-00119-g811b3985828e .config make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

3 months, 2 weeks

2
1
0 / 0

[net] 85540a47fc: BUG:unable_to_handle_page_fault_for_address

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 85540a47fc03a568669f02a09a1ee223f9d8a98a ("[PATCH net] net: dev: Detect dev_hold() after netdev_wait_allrefs()") url: https://github.com/0day-ci/linux/commits/Jann-Horn/net-dev-Detect-dev_hol... patch link: https://lore.kernel.org/netdev/20220128014303.2334568-1-jannh@google.com in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu Icelake-Server -smp 4 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------------+------------+------------+ | | 23a46422c5 | 85540a47fc | +-------------------------------------------------------------+------------+------------+ | BUG:unable_to_handle_page_fault_for_address | 0 | 16 | | Oops:#[##] | 0 | 16 | | EIP:netdev_refcnt_read | 0 | 16 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 16 | +-------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 17.479496][ T1] BUG: unable to handle page fault for address: 3ef81000 [ 17.479636][ T1] #PF: supervisor read access in kernel mode [ 17.479636][ T1] #PF: error_code(0x0000) - not-present page [ 17.479636][ T1] *pde = 00000000 [ 17.479636][ T1] Oops: 0000 [#1] SMP [ 17.479636][ T1] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 5.17.0-rc1-00187-g85540a47fc03 #1 [ 17.479636][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 17.479636][ T1] EIP: netdev_refcnt_read (net/core/dev.c:9803) [ 17.479636][ T1] Code: ff ff ff ff 83 ec 08 8b 15 c0 7a 6f b3 89 55 f0 eb 1b 8d b6 00 00 00 00 8b 8e 50 03 00 00 83 fb 07 77 27 8b 04 9d 80 64 02 b3 <03> 3c 08 89 d8 ba 68 79 6f b3 e8 b7 fd 3b ff 89 c3 3b 45 f0 72 d8 All code ======== 0: ff (bad) 1: ff (bad) 2: ff (bad) 3: ff 83 ec 08 8b 15 incl 0x158b08ec(%rbx) 9: c0 7a 6f b3 sarb $0xb3,0x6f(%rdx) d: 89 55 f0 mov %edx,-0x10(%rbp) 10: eb 1b jmp 0x2d 12: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi 18: 8b 8e 50 03 00 00 mov 0x350(%rsi),%ecx 1e: 83 fb 07 cmp $0x7,%ebx 21: 77 27 ja 0x4a 23: 8b 04 9d 80 64 02 b3 mov -0x4cfd9b80(,%rbx,4),%eax 2a:* 03 3c 08 add (%rax,%rcx,1),%edi <-- trapping instruction 2d: 89 d8 mov %ebx,%eax 2f: ba 68 79 6f b3 mov $0xb36f7968,%edx 34: e8 b7 fd 3b ff callq 0xffffffffff3bfdf0 39: 89 c3 mov %eax,%ebx 3b: 3b 45 f0 cmp -0x10(%rbp),%eax 3e: 72 d8 jb 0x18 Code starting with the faulting instruction =========================================== 0: 03 3c 08 add (%rax,%rcx,1),%edi 3: 89 d8 mov %ebx,%eax 5: ba 68 79 6f b3 mov $0xb36f7968,%edx a: e8 b7 fd 3b ff callq 0xffffffffff3bfdc6 f: 89 c3 mov %eax,%ebx 11: 3b 45 f0 cmp -0x10(%rbp),%eax 14: 72 d8 jb 0xffffffffffffffee [ 17.479636][ T1] EAX: 3ef81000 EBX: 00000000 ECX: 00000000 EDX: 00000000 [ 17.479636][ T1] ESI: b579a000 EDI: 00000000 EBP: b46a5ed0 ESP: b46a5ebc [ 17.479636][ T1] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010297 [ 17.479636][ T1] CR0: 80050033 CR2: 3ef81000 CR3: 039ab000 CR4: 000406d0 [ 17.479636][ T1] Call Trace: [ 17.479636][ T1] free_netdev (net/core/dev.c:10309) [ 17.479636][ T1] tc515_probe (drivers/net/ethernet/3com/3c515.c:430) [ 17.479636][ T1] net_olddevs_init (drivers/net/Space.c:192 drivers/net/Space.c:242 drivers/net/Space.c:251) [ 17.479636][ T1] ? ether_boot_setup (drivers/net/Space.c:247) [ 17.479636][ T1] do_one_initcall (init/main.c:1300) [ 17.479636][ T1] ? rdinit_setup (init/main.c:1357) [ 17.479636][ T1] ? rcu_read_lock_sched_held (include/linux/lockdep.h:283 kernel/rcu/update.c:125) [ 17.479636][ T1] kernel_init_freeable (init/main.c:1372 init/main.c:1389 init/main.c:1408 init/main.c:1613) [ 17.479636][ T1] ? rest_init (init/main.c:1494) [ 17.479636][ T1] kernel_init (init/main.c:1504) [ 17.479636][ T1] ret_from_fork (arch/x86/entry/entry_32.S:772) [ 17.479636][ T1] Modules linked in: [ 17.479636][ T1] CR2: 000000003ef81000 [ 17.479636][ T1] ---[ end trace 0000000000000000 ]--- [ 17.479636][ T1] EIP: netdev_refcnt_read (net/core/dev.c:9803) [ 17.479636][ T1] Code: ff ff ff ff 83 ec 08 8b 15 c0 7a 6f b3 89 55 f0 eb 1b 8d b6 00 00 00 00 8b 8e 50 03 00 00 83 fb 07 77 27 8b 04 9d 80 64 02 b3 <03> 3c 08 89 d8 ba 68 79 6f b3 e8 b7 fd 3b ff 89 c3 3b 45 f0 72 d8 All code ======== 0: ff (bad) 1: ff (bad) 2: ff (bad) 3: ff 83 ec 08 8b 15 incl 0x158b08ec(%rbx) 9: c0 7a 6f b3 sarb $0xb3,0x6f(%rdx) d: 89 55 f0 mov %edx,-0x10(%rbp) 10: eb 1b jmp 0x2d 12: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi 18: 8b 8e 50 03 00 00 mov 0x350(%rsi),%ecx 1e: 83 fb 07 cmp $0x7,%ebx 21: 77 27 ja 0x4a 23: 8b 04 9d 80 64 02 b3 mov -0x4cfd9b80(,%rbx,4),%eax 2a:* 03 3c 08 add (%rax,%rcx,1),%edi <-- trapping instruction 2d: 89 d8 mov %ebx,%eax 2f: ba 68 79 6f b3 mov $0xb36f7968,%edx 34: e8 b7 fd 3b ff callq 0xffffffffff3bfdf0 39: 89 c3 mov %eax,%ebx 3b: 3b 45 f0 cmp -0x10(%rbp),%eax 3e: 72 d8 jb 0x18 Code starting with the faulting instruction =========================================== 0: 03 3c 08 add (%rax,%rcx,1),%edi 3: 89 d8 mov %ebx,%eax 5: ba 68 79 6f b3 mov $0xb36f7968,%edx a: e8 b7 fd 3b ff callq 0xffffffffff3bfdc6 f: 89 c3 mov %eax,%ebx 11: 3b 45 f0 cmp -0x10(%rbp),%eax 14: 72 d8 jb 0xffffffffffffffee To reproduce: # build kernel cd linux cp config-5.17.0-rc1-00187-g85540a47fc03 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

3 months, 2 weeks

1
0
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP January 2022