Greeting, FYI, we noticed a -73.2% regression of vm-scalability.throughput due to commit: commit: a067b3644bc70f2f715227bc3d22f5bc4a951dff ("shmem: Convert shmem_add_to_page_cache to XArray") git://git.infradead.org/users/willy/linux-dax.git xarray in testcase: vm-scalability on test machine: qemu-system-x86_64 -enable-kvm -cpu qemu64,+ssse3 -smp 4 -m 4G with following parameters: runtime: 300s size: 2T test: shm-pread-seq test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email ========================================================================================= compiler/kconfig/rootfs/runtime/size/tbox_group/test/testcase: gcc-7/x86_64-allyesdebian/debian-x86_64-2018-04-03.cgz/300s/2T/vm-kbuild-4G/shm-pread-seq/vm-scalability commit: 108ba81fe3 ("shmem: Convert find_swap_entry to XArray") a067b3644b ("shmem: Convert shmem_add_to_page_cache to XArray") 108ba81fe3134638 a067b3644bc70f2f715227bc3d ---------------- -------------------------- %stddev %change %stddev \ | \ 973694 ± 5% -73.2% 260463 ± 2% vm-scalability.throughput 0.18 -57.9% 0.08 ± 4% vm-scalability.free_time 243461 ± 5% +7.0% 260463 ± 2% vm-scalability.median 62943 ± 15% -86.6% 8405 ± 9% vm-scalability.time.involuntary_context_switches 41550585 ± 11% -63.7% 15073341 ± 3% vm-scalability.time.minor_page_faults 282.50 ± 3% -65.0% 99.00 vm-scalability.time.percent_of_cpu_this_job_got 653.30 ± 4% -61.9% 249.02 vm-scalability.time.system_time 222.16 ± 2% -74.2% 57.36 vm-scalability.time.user_time 3921839 ± 17% -100.0% 1366 ± 3% vm-scalability.time.voluntary_context_switches 2.553e+08 ± 5% -73.5% 67626963 ± 3% vm-scalability.workload 283.00 ± 9% +159.1% 733.25 ± 6% interrupts.CAL:Function_call_interrupts 3202 -46.3% 1721 slabinfo.page->ptl.active_objs 3927 -41.8% 2286 slabinfo.page->ptl.num_objs 83771 -10.8% 74711 softirqs.RCU 111439 ± 5% -58.7% 45980 ± 3% softirqs.SCHED 135036 ± 3% -18.9% 109566 ± 2% softirqs.TIMER 3.00 -66.7% 1.00 vmstat.procs.r 27602 ± 16% -94.0% 1651 ± 13% vmstat.system.cs 5979 ± 21% -78.4% 1294 ± 10% vmstat.system.in 403546 -36.1% 258064 meminfo.Active 403475 -36.1% 257752 meminfo.Active(anon) 1016425 +12.4% 1142559 meminfo.Inactive 1015445 +12.4% 1141822 meminfo.Inactive(anon) 11637 -51.8% 5605 meminfo.PageTables 18.92 ± 7% +46.6 65.47 mpstat.cpu.idle% 0.43 ± 35% -0.4 0.00 ±120% mpstat.cpu.iowait% 15.02 ± 28% -8.5 6.51 ± 10% mpstat.cpu.steal% 48.39 ± 9% -26.8 21.61 ± 2% mpstat.cpu.sys% 17.15 ± 6% -10.9 6.25 mpstat.cpu.usr% 101963 -36.7% 64496 proc-vmstat.nr_active_anon 385408 -1.6% 379417 proc-vmstat.nr_file_pages 253679 +12.4% 285190 proc-vmstat.nr_inactive_anon 4149 -1.5% 4086 proc-vmstat.nr_kernel_stack 2898 -51.6% 1403 proc-vmstat.nr_page_table_pages 295656 -2.0% 289677 proc-vmstat.nr_shmem 101967 -36.7% 64500 proc-vmstat.nr_zone_active_anon 253681 +12.4% 285191 proc-vmstat.nr_zone_inactive_anon 19639935 ± 5% -21.2% 15468488 ± 3% proc-vmstat.numa_hit 19639935 ± 5% -21.2% 15468488 ± 3% proc-vmstat.numa_local 14187284 ± 5% -99.9% 17509 ± 13% proc-vmstat.pgactivate 14133756 ± 4% -21.6% 11084067 ± 5% proc-vmstat.pgalloc_dma32 5499321 ± 9% -20.4% 4376536 ± 3% proc-vmstat.pgalloc_normal 41936064 ± 11% -63.1% 15463252 ± 3% proc-vmstat.pgfault 19505793 ± 5% -21.1% 15389143 ± 3% proc-vmstat.pgfree 117160 ± 3% -53.6% 54389 sched_debug.cfs_rq:/.exec_clock.avg 118940 ± 3% -39.3% 72184 ± 5% sched_debug.cfs_rq:/.exec_clock.max 114940 ± 3% -65.5% 39702 ± 5% sched_debug.cfs_rq:/.exec_clock.min 1557 ± 27% +736.8% 13030 ± 13% sched_debug.cfs_rq:/.exec_clock.stddev 126995 ± 33% -63.7% 46135 ± 96% sched_debug.cfs_rq:/.load.min 294.54 ± 4% +19.6% 352.41 ± 9% sched_debug.cfs_rq:/.load_avg.avg 139.08 ± 8% +31.9% 183.48 ± 16% sched_debug.cfs_rq:/.load_avg.stddev 455686 ± 4% -70.6% 133860 ± 5% sched_debug.cfs_rq:/.min_vruntime.avg 466686 ± 3% -65.7% 160093 ± 2% sched_debug.cfs_rq:/.min_vruntime.max 444165 ± 4% -75.2% 110113 ± 6% sched_debug.cfs_rq:/.min_vruntime.min 8771 ± 25% +128.4% 20036 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev 0.92 ± 3% -22.7% 0.71 ± 8% sched_debug.cfs_rq:/.nr_running.avg 0.62 ± 29% -60.0% 0.25 ± 57% sched_debug.cfs_rq:/.nr_running.min 0.19 ± 60% +97.2% 0.37 ± 20% sched_debug.cfs_rq:/.nr_running.stddev 90.92 ± 29% -60.6% 35.79 ±110% sched_debug.cfs_rq:/.runnable_load_avg.min 134.24 ± 15% +41.9% 190.46 ± 19% sched_debug.cfs_rq:/.runnable_load_avg.stddev 121098 ± 31% -69.9% 36507 ±109% sched_debug.cfs_rq:/.runnable_weight.min 8770 ± 25% +128.5% 20037 ± 11% sched_debug.cfs_rq:/.spread0.stddev 806.85 ± 12% -24.7% 607.84 ± 7% sched_debug.cfs_rq:/.util_avg.avg 592.62 ± 21% -55.9% 261.21 ± 28% sched_debug.cfs_rq:/.util_avg.min 163.80 ± 19% +82.8% 299.38 ± 8% sched_debug.cfs_rq:/.util_avg.stddev 544.27 ± 26% -77.6% 121.94 ± 17% sched_debug.cfs_rq:/.util_est_enqueued.avg 817.46 ± 28% -47.0% 432.96 ± 17% sched_debug.cfs_rq:/.util_est_enqueued.max 229.71 ± 50% -99.9% 0.33 ± 50% sched_debug.cfs_rq:/.util_est_enqueued.min 1179733 ± 48% +771.7% 10283446 ± 12% sched_debug.cpu.avg_idle.avg 2186782 ± 39% +655.9% 16529998 ± 14% sched_debug.cpu.avg_idle.max 436002 ± 80% +1048.0% 5005135 ± 19% sched_debug.cpu.avg_idle.min 697066 ± 27% +539.8% 4459732 ± 23% sched_debug.cpu.avg_idle.stddev 401.04 ± 2% +31.8% 528.42 ± 13% sched_debug.cpu.cpu_load[1].max 130.00 ± 27% -48.9% 66.42 ± 14% sched_debug.cpu.cpu_load[1].min 103.93 ± 16% +73.3% 180.16 ± 14% sched_debug.cpu.cpu_load[1].stddev 366.04 ± 6% +62.6% 595.33 ± 12% sched_debug.cpu.cpu_load[2].max 139.92 ± 20% -60.0% 55.92 ± 9% sched_debug.cpu.cpu_load[2].min 87.49 ± 26% +142.0% 211.73 ± 14% sched_debug.cpu.cpu_load[2].stddev 353.79 ± 10% +81.7% 642.87 ± 11% sched_debug.cpu.cpu_load[3].max 151.04 ± 18% -72.6% 41.38 ± 20% sched_debug.cpu.cpu_load[3].min 79.58 ± 31% +205.8% 243.35 ± 14% sched_debug.cpu.cpu_load[3].stddev 337.00 ± 13% +100.0% 674.17 ± 11% sched_debug.cpu.cpu_load[4].max 158.04 ± 17% -78.4% 34.12 ± 35% sched_debug.cpu.cpu_load[4].min 71.84 ± 36% +269.3% 265.33 ± 13% sched_debug.cpu.cpu_load[4].stddev 2696 ± 9% -27.7% 1948 ± 7% sched_debug.cpu.curr->pid.avg 1372 ± 64% -82.3% 243.29 ±126% sched_debug.cpu.curr->pid.min 921.23 ± 41% +69.1% 1558 ± 8% sched_debug.cpu.curr->pid.stddev 487128 ± 14% +30.0% 633229 ± 19% sched_debug.cpu.load.max 144979 ± 24% +69.4% 245569 ± 16% sched_debug.cpu.load.stddev 3298283 ± 31% +118.2% 7198331 ± 14% sched_debug.cpu.max_idle_balance_cost.avg 5959825 ± 30% +85.5% 11054227 ± 19% sched_debug.cpu.max_idle_balance_cost.max 1278454 ± 39% +212.9% 4000812 ± 18% sched_debug.cpu.max_idle_balance_cost.min 41082 -25.0% 30809 ± 2% sched_debug.cpu.nr_load_updates.avg 41797 -14.8% 35619 ± 2% sched_debug.cpu.nr_load_updates.max 40213 -34.8% 26209 ± 3% sched_debug.cpu.nr_load_updates.min 629.61 ± 9% +489.3% 3710 ± 4% sched_debug.cpu.nr_load_updates.stddev 0.58 ± 31% -57.1% 0.25 ± 57% sched_debug.cpu.nr_running.min 953962 ± 21% -92.9% 67812 ± 10% sched_debug.cpu.nr_switches.avg 996979 ± 19% -90.9% 90867 ± 17% sched_debug.cpu.nr_switches.max 913679 ± 21% -94.9% 46191 ± 5% sched_debug.cpu.nr_switches.min 32843 ± 27% -47.3% 17308 ± 42% sched_debug.cpu.nr_switches.stddev 942452 ± 21% -94.0% 56255 ± 13% sched_debug.cpu.sched_count.avg 987126 ± 20% -92.2% 77110 ± 22% sched_debug.cpu.sched_count.max 900694 ± 21% -96.0% 36326 ± 8% sched_debug.cpu.sched_count.min 33839 ± 26% -53.1% 15870 ± 46% sched_debug.cpu.sched_count.stddev 416113 ± 22% -97.1% 11890 ± 15% sched_debug.cpu.sched_goidle.avg 429842 ± 23% -96.2% 16204 ± 20% sched_debug.cpu.sched_goidle.max 402313 ± 21% -98.2% 7433 ± 16% sched_debug.cpu.sched_goidle.min 10768 ± 48% -69.0% 3337 ± 30% sched_debug.cpu.sched_goidle.stddev 478266 ± 21% -94.1% 28209 ± 13% sched_debug.cpu.ttwu_count.avg 501683 ± 20% -92.0% 39900 ± 24% sched_debug.cpu.ttwu_count.max 453413 ± 22% -95.9% 18446 ± 3% sched_debug.cpu.ttwu_count.min 18884 ± 16% -54.4% 8612 ± 48% sched_debug.cpu.ttwu_count.stddev 37415 ± 20% -48.4% 19288 ± 26% sched_debug.cpu.ttwu_local.avg 57109 ± 13% -52.0% 27419 ± 43% sched_debug.cpu.ttwu_local.max 455859 ± 34% -39.7% 275085 ± 5% lock_stat.&(&adapter->stats_lock)->rlock.holdtime-total 16210 ± 60% -86.7% 2151 ±160% lock_stat.&(&dn_rt_hash_table[i].lock)->rlock.holdtime-max 68614 ± 23% -37.5% 42887 ± 7% lock_stat.&(&dn_rt_hash_table[i].lock)->rlock.holdtime-total 153567 ± 2% +17.9% 181023 ± 5% lock_stat.&(&dsp_lock)->rlock.holdtime-total 81334 ± 53% +89.8% 154340 ± 44% lock_stat.&(&ep->lock)->rlock.holdtime-total 13027 ± 24% +78.8% 23291 ± 12% lock_stat.&(&ep->lock)->rlock.waittime-max 74988 ± 66% +153.9% 190420 ± 38% lock_stat.&(&ep->lock)->rlock.waittime-total 34932 ± 3% +28.1% 44763 ± 17% lock_stat.&(&gc_work->dwork)->timer.holdtime-total 708791 ± 11% -99.9% 405.00 ± 7% lock_stat.&(&info->lock)->rlock.acq-bounces 4475 ± 8% +22.8% 5494 lock_stat.&(&ipvs->defense_work)->timer.holdtime-total 157167 ± 2% -27.4% 114177 ± 3% lock_stat.&(&parent->list_lock)->rlock.acq-bounces 163974 ± 4% -16.7% 136547 ± 5% lock_stat.&(&parent->list_lock)->rlock.holdtime-total 1482495 ± 7% -97.2% 42182 ± 4% lock_stat.&(&pgdat->lru_lock)->rlock.acq-bounces 3216117 ± 5% -35.9% 2060805 ± 3% lock_stat.&(&pgdat->lru_lock)->rlock.acquisitions 737686 ± 9% -99.8% 1798 ± 5% lock_stat.&(&pgdat->lru_lock)->rlock.con-bounces 737788 ± 9% -99.8% 1798 ± 5% lock_stat.&(&pgdat->lru_lock)->rlock.contentions 1474000 ± 9% -99.8% 2478 ± 5% lock_stat.&(&pgdat->lru_lock)->rlock.contentions.pagevec_lru_move_fn 12663440 ± 3% -29.1% 8973526 lock_stat.&(&pgdat->lru_lock)->rlock.holdtime-total 2071580 ± 10% -85.6% 298272 ± 25% lock_stat.&(&pgdat->lru_lock)->rlock.waittime-total 115219 ± 16% +293.6% 453481 ± 15% lock_stat.&(&pool->lock)->rlock.holdtime-total 11095 ±102% +164.7% 29370 ± 8% lock_stat.&(&pool->lock)->rlock.waittime-max 16178 ± 95% +2231.4% 377188 ± 21% lock_stat.&(&pool->lock)->rlock.waittime-total 545696 ± 22% +337.2% 2385686 ± 3% lock_stat.&(&sighand->siglock)->rlock.holdtime-total 9209588 ± 7% -100.0% 3443 ± 3% lock_stat.&(&xa->xa_lock)->rlock.acq-bounces 33330684 ± 5% -9.8% 30075070 ± 3% lock_stat.&(&xa->xa_lock)->rlock.acquisitions 750708 ± 10% -100.0% 12.75 ± 30% lock_stat.&(&xa->xa_lock)->rlock.con-bounces 751838 ± 10% -100.0% 12.75 ± 30% lock_stat.&(&xa->xa_lock)->rlock.contentions 1503651 ± 10% -100.0% 23.75 ± 27% lock_stat.&(&xa->xa_lock)->rlock.contentions.shmem_add_to_page_cache 20138731 ± 3% +37.9% 27772112 lock_stat.&(&xa->xa_lock)->rlock.holdtime-total 21047 ± 10% -71.1% 6073 ± 61% lock_stat.&(&xa->xa_lock)->rlock.waittime-max 2826405 ± 29% -99.7% 8548 ± 66% lock_stat.&(&xa->xa_lock)->rlock.waittime-total 263136 ± 5% -91.4% 22741 ± 3% lock_stat.&(&zone->lock)->rlock.acq-bounces 910134 ± 51% -65.1% 317918 ± 9% lock_stat.&(&zone->lock)->rlock.waittime-total 15925 ± 16% -21.6% 12486 ± 10% lock_stat.&(({do{constvoid*__vpp_verify=.holdtime-total 109133 ± 4% -61.8% 41693 ± 7% lock_stat.&(ptlock_ptr(page))->rlock#2.acq-bounces 68196728 ± 9% -55.9% 30053960 ± 3% lock_stat.&(ptlock_ptr(page))->rlock#2.acquisitions 172010 ±122% -85.4% 25129 ± 35% lock_stat.&(ptlock_ptr(page))->rlock#2.holdtime-max 33983810 ± 4% -70.4% 10071152 lock_stat.&(ptlock_ptr(page))->rlock#2.holdtime-total 189456 ± 3% -42.2% 109466 lock_stat.&(ptlock_ptr(page))->rlock.acquisitions 106129 ± 7% -52.1% 50795 ± 4% lock_stat.&(ptlock_ptr(page))->rlock.holdtime-total 4109768 ± 14% -82.1% 736924 lock_stat.&base->lock.acquisitions 13669 ± 18% -73.3% 3646 ± 37% lock_stat.&base->lock.holdtime-max 2209337 ± 9% -80.5% 431561 ± 4% lock_stat.&base->lock.holdtime-total 1910 ±156% +338.4% 8375 ± 43% lock_stat.&ctx->wqh.holdtime-max 1200480 ± 5% +56.4% 1877354 lock_stat.&dup_mmap_sem-R.holdtime-total 336791 ± 7% +39.0% 468161 ± 6% lock_stat.&f->f_pos_lock.holdtime-avg 74615 ± 8% +65.9% 123751 ± 33% lock_stat.&group->notification_waitq.holdtime-total 80849 ± 19% +96.4% 158807 ± 33% lock_stat.&group->notification_waitq.waittime-total 47155458 ± 10% -56.3% 20625775 ± 2% lock_stat.&mm->mmap_sem-R.acquisitions 248011 ± 20% -74.0% 64461 ± 47% lock_stat.&mm->mmap_sem-R.holdtime-max 8.109e+08 -76.2% 1.929e+08 lock_stat.&mm->mmap_sem-R.holdtime-total 386738 ± 36% -68.7% 121115 ± 15% lock_stat.&mm->mmap_sem-W.holdtime-max 20781707 ± 3% -75.2% 5146865 ± 2% lock_stat.&mm->mmap_sem-W.holdtime-total 172243 ± 3% -7.6% 159110 lock_stat.&nf_conntrack_generation-R.holdtime-total 2005217 ± 17% -98.8% 23380 lock_stat.&p->pi_lock.acq-bounces 4514659 ± 16% -90.7% 420676 ± 10% lock_stat.&p->pi_lock.acquisitions 43481206 ± 7% -40.7% 25775609 ± 9% lock_stat.&p->pi_lock.holdtime-total 35054 ± 15% -31.4% 24032 ± 10% lock_stat.&p->pi_lock.waittime-total 12830201 ± 14% -100.0% 192.00 ± 14% lock_stat.&page_wait_table[i].acq-bounces 17227249 ± 13% -100.0% 339.00 ± 9% lock_stat.&page_wait_table[i].acquisitions 4229005 ± 16% -100.0% 14.00 ± 34% lock_stat.&page_wait_table[i].con-bounces 4233464 ± 16% -100.0% 14.00 ± 34% lock_stat.&page_wait_table[i].contentions 986102 ± 13% -100.0% 9.50 ± 41% lock_stat.&page_wait_table[i].contentions.finish_wait 870129 ± 17% -100.0% 0.75 ±110% lock_stat.&page_wait_table[i].contentions.wait_on_page_bit_common 6610697 ± 16% -100.0% 17.75 ± 36% lock_stat.&page_wait_table[i].contentions.wake_up_page_bit 21461 ± 14% -77.2% 4900 ± 95% lock_stat.&page_wait_table[i].holdtime-max 44324794 ± 3% -99.9% 27092 ±130% lock_stat.&page_wait_table[i].holdtime-total 13.53 ± 46% +20884.1% 2839 ±105% lock_stat.&page_wait_table[i].waittime-avg 29849 ± 14% -60.8% 11704 ± 79% lock_stat.&page_wait_table[i].waittime-max 53128316 ± 24% -99.9% 52419 ±133% lock_stat.&page_wait_table[i].waittime-total 289983 ± 19% -50.3% 144255 ± 24% lock_stat.&pipe->mutex/1.waittime-total 16126 ± 15% +44.2% 23247 ± 15% lock_stat.&pipe->wait.holdtime-max 985.70 ± 59% +352.1% 4456 ± 19% lock_stat.&pipe->wait.waittime-avg 23510 ± 17% +100.7% 47181 ± 12% lock_stat.&pipe->wait.waittime-max 531976 ± 48% +607.7% 3764886 ± 24% lock_stat.&pipe->wait.waittime-total 4145 ± 91% +225.4% 13490 ± 10% lock_stat.&pool->lock/1.holdtime-max 8992 ± 55% +397.1% 44699 ± 37% lock_stat.&pool->lock/1.holdtime-total 1275 ±109% +431.5% 6780 ± 11% lock_stat.&pool->lock/1.waittime-avg 3026 ±118% +477.6% 17480 ± 34% lock_stat.&pool->lock/1.waittime-max 3419 ±122% +1283.1% 47299 ± 51% lock_stat.&pool->lock/1.waittime-total 3756368 ± 8% -98.8% 46374 ± 5% lock_stat.&rq->lock.acq-bounces 24049672 ± 14% -93.1% 1652462 ± 11% lock_stat.&rq->lock.acquisitions 1879491 ± 7% -99.5% 9184 ± 13% lock_stat.&rq->lock.con-bounces 1879660 ± 7% -99.5% 9189 ± 13% lock_stat.&rq->lock.contentions 1878333 ± 7% -99.6% 7513 ± 17% lock_stat.&rq->lock.contentions.__task_rq_lock 1863315 ± 8% -99.5% 8537 ± 15% lock_stat.&rq->lock.contentions.rq_lock 21165 ± 18% +29.2% 27353 ± 8% lock_stat.&rq->lock.holdtime-max 59512381 ± 13% -86.6% 7958968 ± 6% lock_stat.&rq->lock.holdtime-total 15399445 ± 22% -70.0% 4619267 ± 6% lock_stat.&rq->lock.waittime-total 21045 ± 15% +50.8% 31733 ± 11% lock_stat.&rsp->gp_wq.holdtime-max 1292559 ± 23% +1042.7% 14770713 ± 14% lock_stat.&rsp->gp_wq.holdtime-total 7115 ± 12% +70.9% 12158 ± 2% lock_stat.&rsp->gp_wq.waittime-avg 1706488 ± 26% +1149.1% 21315655 ± 13% lock_stat.&rsp->gp_wq.waittime-total 8481 ± 5% -18.4% 6918 ± 3% lock_stat.&sb->s_type->i_lock_key#1.holdtime-total 7290 ± 72% -58.1% 3053 ± 49% lock_stat.&sb->s_type->i_mutex_key#1.holdtime-total 7456886 ± 10% -33.0% 4996474 ± 9% lock_stat.&sig->cred_guard_mutex.holdtime-total 522.32 ± 36% +2030.3% 11126 ± 94% lock_stat.&sighand->signalfd_wqh.holdtime-total 12353 ± 5% -84.8% 1878 ± 4% lock_stat.&stopper->lock.acquisitions 4994 ± 2% -85.6% 718.25 ± 3% lock_stat.&stopper->lock.holdtime-total 1974810 ± 17% -100.0% 38.75 ± 23% lock_stat.&tsk->delays->lock.acq-bounces 3920656 ± 17% -100.0% 129.00 ± 13% lock_stat.&tsk->delays->lock.acquisitions 8996 ± 10% -99.4% 53.16 ± 12% lock_stat.&tsk->delays->lock.holdtime-max 982328 ± 14% -100.0% 116.84 ± 22% lock_stat.&tsk->delays->lock.holdtime-total 0.19 ±100% +3.7e+06% 7011 ± 95% lock_stat.&txwq.waittime-avg 0.22 ±101% +4.4e+06% 9686 ± 98% lock_stat.&txwq.waittime-max 0.16 ±100% +2e+06% 3138 ±119% lock_stat.&txwq.waittime-min 0.38 ±100% +5.9e+06% 22240 ±118% lock_stat.&txwq.waittime-total 3104 ± 10% -85.4% 452.72 ± 18% lock_stat.&x->wait#3.holdtime-total 20473 ± 68% -86.2% 2823 ±117% lock_stat.(&dn_route_timer).holdtime-max 106379 ± 29% -37.5% 66495 ± 4% lock_stat.(&dn_route_timer).holdtime-total 204022 ± 2% +41.4% 288477 ± 35% lock_stat.(&dsp_spl_tl).holdtime-total 14154 ± 37% -79.8% 2859 ± 49% lock_stat.(&n->timer).holdtime-total 1128 ± 6% +1423.7% 17197 ±148% lock_stat.(&net->can.can_stattimer).holdtime-total 55207 +13.7% 62786 lock_stat.(&timer.timer).acquisitions 56.56 ± 84% +1903.7% 1133 ± 19% lock_stat.(shepherd).work.holdtime-avg 12719 ± 94% +130.5% 29323 ± 6% lock_stat.(shepherd).work.holdtime-max 17528 ± 84% +1905.4% 351513 ± 19% lock_stat.(shepherd).work.holdtime-total 3486 ± 29% -44.3% 1942 ± 4% lock_stat.(work_completion)(&(&adapter->watchdo.holdtime-avg 535756 ± 28% -44.2% 298691 ± 4% lock_stat.(work_completion)(&(&adapter->watchdo.holdtime-total 13139 ± 15% +328.5% 56302 ±119% lock_stat.(work_completion)(&(&gc_work->dwork)-.holdtime-max 457.76 ± 11% +2237.9% 10702 ± 56% lock_stat.(work_completion)(&(&wb->dwork)->work.holdtime-max 3236 ± 88% +853.2% 30849 ± 54% lock_stat.(work_completion)(&(reap_work)->work).holdtime-max 101649 ± 8% +26.0% 128028 ± 12% lock_stat.(work_completion)(&(reap_work)->work).holdtime-total 10752 ± 57% -48.4% 5551 ± 93% lock_stat.(work_completion)(&task->u.tk_work).holdtime-max 29789 ± 26% -30.7% 20637 ± 21% lock_stat.(work_completion)(&task->u.tk_work).holdtime-total 13140 ± 15% +328.5% 56303 ±119% lock_stat.(wq_completion)"events_power_efficien.holdtime-max 10753 ± 57% -48.4% 5552 ± 93% lock_stat.(wq_completion)"rpciod".holdtime-max 29817 ± 26% -30.7% 20663 ± 21% lock_stat.(wq_completion)"rpciod".holdtime-total 458.81 ± 10% +2232.8% 10702 ± 56% lock_stat.(wq_completion)"writeback".holdtime-max 8932 ± 6% -11.0% 7953 ± 2% lock_stat.binfmt_lock-R.holdtime-total 25365 ± 14% +43.4% 36368 ± 30% lock_stat.console_owner.holdtime-max 1551024 ± 11% -24.2% 1176166 ± 12% lock_stat.console_owner.holdtime-total 12724 ± 94% +130.4% 29320 ± 6% lock_stat.cpu_hotplug_lock.rw_sem-R.holdtime-max 23096 ± 63% +1450.7% 358151 ± 18% lock_stat.cpu_hotplug_lock.rw_sem-R.holdtime-total 3203 ± 12% -100.0% 0.00 lock_stat.drivers/atm/iph.holdtime-total 0.00 +9e+105% 8984 ±121% lock_stat.drivers/atm/iphase.holdtime-max 0.00 +1.3e+106% 12660 ± 91% lock_stat.drivers/atm/iphase.holdtime-total 18559 -100.0% 0.00 lock_stat.drivers/tty/isi.acquisitions 11255 ± 85% -100.0% 0.00 lock_stat.drivers/tty/isi.holdtime-max 68448 ± 31% -100.0% 0.00 lock_stat.drivers/tty/isi.holdtime-total 0.00 +1.9e+106% 18787 lock_stat.drivers/tty/isicom.acquisitions 0.00 +5.2e+106% 51897 ± 16% lock_stat.drivers/tty/isicom.holdtime-total 20598124 ± 5% -20.6% 16345527 ± 2% lock_stat.fs_reclaim.acquisitions 14366 ± 20% +95.7% 28113 ± 58% lock_stat.fs_reclaim.holdtime-max 3651960 ± 2% -20.5% 2902504 lock_stat.fs_reclaim.holdtime-total 10515834 ± 14% -94.1% 620597 ± 3% lock_stat.hrtimer_bases.lock.acquisitions 23336318 ± 13% -95.7% 1000877 ± 3% lock_stat.hrtimer_bases.lock.holdtime-total 41455 ± 4% +9.8% 45529 ± 3% lock_stat.iclock_lock-R.holdtime-total 3784656 ± 17% -95.6% 167136 ± 7% lock_stat.jiffies_lock#2-R.acquisitions 9650 ± 12% -98.9% 106.17 ± 11% lock_stat.jiffies_lock#2-R.holdtime-max 697919 ± 6% -94.9% 35417 ± 6% lock_stat.jiffies_lock#2-R.holdtime-total 31117 ± 3% -38.9% 19010 ± 7% lock_stat.jiffies_lock.acq-bounces 234334 ± 5% -99.9% 319.50 ± 7% lock_stat.key#2.acq-bounces 129919 ± 5% -19.2% 104951 ± 3% lock_stat.key#2.holdtime-total 926.38 ± 82% -100.0% 0.00 lock_stat.key#2.waittime-avg 5293 ±122% -100.0% 0.00 lock_stat.key#2.waittime-max 12783 ±121% -100.0% 0.00 lock_stat.key#2.waittime-total 10681 ± 4% -96.6% 363.00 ± 2% lock_stat.key#3.acq-bounces 3278573 ± 20% -99.5% 17651 ± 3% lock_stat.latency_lock.acq-bounces 4330299 ± 16% -94.5% 238341 ± 15% lock_stat.latency_lock.acquisitions 28394 ± 31% -99.3% 199.75 ± 15% lock_stat.latency_lock.con-bounces 28437 ± 31% -99.3% 200.75 ± 15% lock_stat.latency_lock.contentions 56839 ± 31% -99.3% 399.25 ± 15% lock_stat.latency_lock.contentions.__account_scheduler_latency 2085235 ± 19% -95.1% 102278 ± 11% lock_stat.latency_lock.holdtime-total 126589 ± 21% -90.7% 11764 ± 66% lock_stat.latency_lock.waittime-total 178625 ± 19% +98.8% 355148 ± 34% lock_stat.log_wait.lock.holdtime-total 201393 ± 39% +125.0% 453153 ± 42% lock_stat.log_wait.lock.waittime-total 8387 ±106% -96.6% 284.26 ±146% lock_stat.logbuf_lock.waittime-total 2885 ± 3% -100.0% 0.00 lock_stat.mm/vmstat.c:187.holdtime-total 0.00 +4e+105% 4039 ± 41% lock_stat.mm/vmstat.c:1875.holdtime-total 29566 ± 21% -18.7% 24025 ± 6% lock_stat.mount_lock#2-R.holdtime-total 19986 ± 22% -22.1% 15571 ± 3% lock_stat.pgd_lock.holdtime-total 17894 ± 16% +61.6% 28914 ± 33% lock_stat.printing_lock.holdtime-max 10991 ± 18% +170.1% 29683 ± 71% lock_stat.rcu_callback-R.holdtime-max 143791 ± 2% -10.0% 129434 ± 2% lock_stat.rcu_node_0.acquisitions 147374 ± 13% -14.7% 125665 lock_stat.rcu_node_0.holdtime-total 6949 ± 35% -91.9% 561.12 ±112% lock_stat.rcu_node_0.waittime-max 16821 ± 29% -81.9% 3039 ± 41% lock_stat.rcu_node_0.waittime-total 1.241e+08 ± 9% -58.0% 52076183 ± 2% lock_stat.rcu_read_lock-R.acquisitions 2.913e+08 ± 4% -71.2% 84041884 lock_stat.rcu_read_lock-R.holdtime-total 17953 ± 31% -23.2% 13787 lock_stat.rename_lock#2-R.holdtime-total 9057 ± 38% +239.2% 30722 ± 55% lock_stat.slab_mutex.holdtime-max 1018 ± 94% +714.6% 8298 ± 49% lock_stat.slock-AF_INET-RPC/1.holdtime-max 8395 ± 9% -31.8% 5727 ± 5% lock_stat.speakup_info.spinlock.holdtime-total 402906 ± 36% +496.4% 2402835 ± 3% lock_stat.tasklist_lock-R.waittime-total 501739 ± 23% +365.5% 2335605 ± 4% lock_stat.tasklist_lock-W.holdtime-total 1527 ± 45% +104.4% 3121 ± 30% lock_stat.tasklist_lock-W.waittime-avg 29476 ± 3% -38.7% 18056 ± 7% lock_stat.timekeeper_lock.acq-bounces 2729 ± 79% +253.1% 9637 ± 42% lock_stat.timekeeper_lock.holdtime-max 31404402 ± 13% -83.3% 5258508 ± 3% lock_stat.tk_core-R.acquisitions 4978505 ± 10% -82.8% 853856 ± 6% lock_stat.tk_core-R.holdtime-total 2728 ± 79% +234.6% 9129 ± 49% lock_stat.tk_core-W.holdtime-max vm-scalability.throughput 1.4e+06 +-+---------------------------------------------------------------+ | +..+ | 1.2e+06 +-+. : + .+.+. .+.+. | | +..+.+. : +. .+.+. +.+.+. +.+.+.. | | +.+ + .+. +. | 1e+06 +-+ + +.+. .. +.| | + | 800000 +-+ | | | 600000 +-+ | | | | | 400000 +-+ | O O O O O O O O O O O O O O O O O O O O O O | 200000 +-+-----------------O---------------------------------------------+ vm-scalability.free_time 0.2 +-+------------------------------------------------------------------+ | .. + +.+.+..+. .+.. | 0.18 +-+..+.+.+ +. .. +.+..+.+.+.+..+.+.+..+.+ +.+.+.+..+.| | + | 0.16 +-+ | | | 0.14 +-+ | | | 0.12 +-+ | | | 0.1 +-+ | | | 0.08 +-+ O O O O O O O O O O O | O O O O O O O O O O O O | 0.06 +-+------------------------------------------------------------------+ vm-scalability.workload 3.5e+08 +-+---------------------------------------------------------------+ |.+. +. : .+. .+. | 3e+08 +-+ +..+.+. : : .+.+..+ +.+.+..+ +.+.+.. | | +. : +.+ | | + +.+. .+.+.| 2.5e+08 +-+ +.+.+. | | | 2e+08 +-+ | | | 1.5e+08 +-+ | | | | | 1e+08 +-+ | O O O O O O O O O O O O O O O O O O O O O O | 5e+07 +-+-----------------O---------------------------------------------+ vm-scalability.time.user_time 300 +-+-------------------------------------------------------------------+ | + +..+. | |.+..+. .. + + +. .+.+..+.+. .+.+..+. | 250 +-+ +.+ + +..+ +..+ +.+.. .+ | | + + .+.+..+.| | +..+ | 200 +-+ | | | 150 +-+ | | | | | 100 +-+ | | | | O | 50 O-O--O-O-O--O-O-O--O-O----O-O-O--O-O-O--O-O-O--O-O-O------------------+ vm-scalability.time.system_time 700 +-+-------------------------------------------------------------------+ 650 +-+.. + +.. +.+ +..+.| |+ .+ +.+. .. + + + + + | 600 +-+ +. .+. + .. + +. .+ +. + +..+.+ | 550 +-+ +.+..+.+ +.+ +..+ +.+ | | | 500 +-+ | 450 +-+ | 400 +-+ | | | 350 +-+ | 300 +-+ | | O O O | 250 O-O O O O O O O O O O O O O O O O O O O | 200 +-+-------------------------------------------------------------------+ vm-scalability.time.percent_of_cpu_this_job_got 350 +-+-------------------------------------------------------------------+ | | 300 +-+ .+ | |.+..+. .+. + .+.+.+..+.+. .+.+..+. .+.+. .+..+.| | +.+..+.+ +.+. +..+ +.+. +..+.+ | 250 +-+ | | | 200 +-+ | | | 150 +-+ | | | | | 100 O-O O O O O O O O O O O O O O O O O O O O O O | | | 50 +-+-------------------------------------------------------------------+ vm-scalability.time.minor_page_faults 6.5e+07 +-+---------------------------------------------------------------+ 6e+07 +-+ + | | + ..: + + | 5.5e+07 +-+: + : +.+.. + : + : | 5e+07 +-+: : : : + : .+ : + | 4.5e+07 +-+ +..+.+ : : : +.+.+. +.+.+.. .+ : + | 4e+07 +-+ + : +. : + + : +.| | +.+ + +.+. : | 3.5e+07 +-+ + | 3e+07 +-+ | 2.5e+07 +-+ | 2e+07 +-+ | O O O O | 1.5e+07 +-+ O O O O O O O O O O O O O O O O O O O | 1e+07 +-+---------------------------------------------------------------+ vm-scalability.time.voluntary_context_switches 7e+06 +-+-----------------------------------------------------------------+ | | 6e+06 +-+ + + | |+ + .+ .+. .. : + : | 5e+06 +-+ + .+.+ + : + + : .+..+.+ : .+.+ | | + : : : : + +. + +| 4e+06 +-+ : : : : +.+.. +..+ | | +..+ +..+ + | 3e+06 +-+ +.+.+ | | | 2e+06 +-+ | | | 1e+06 +-+ | | | 0 O-O--O-O-O-O--O-O-O-O--O-O-O-O--O-O-O--O-O-O-O--O-O-----------------+ vm-scalability.time.involuntary_context_switches 100000 +-+----------------------------------------------------------------+ 90000 +-++ .+ : : | |: + + + +. +. : + + : + + | 80000 +-+ : : +.. + + : + : : :: : : :+ + | 70000 +-+ : : + : : +: : : :: : : +..+ + :| | : : : : + :: + :: + + :| 60000 +-+ + + + + +.+. .. | 50000 +-+ + | 40000 +-+ | | | 30000 +-+ | 20000 +-+ | | | 10000 O-O O O O O O O O O O O O O O O O O O O O O O | 0 +-+----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Xiaolong