FYI, we noticed a -7.7% regression of fsmark.files_per_sec due to commit: commit 818e8e4109d58cc55c884223a1d93426024d23d8 ("SUNRPC: RPC transport queue must be low latency") https://github.com/0day-ci/linux Trond-Myklebust/SUNRPC-Small-optimisation-of-client-receive/20160615-035043 in testcase: fsmark on test machine: ivb44: 48 threads Ivytown Ivy Bridge-EP with 64G memory with following parameters: cpufreq_governor=performance/disk=1BRD_48G/filesize=4M/fs2=nfsv4/fs=xfs/iterations=1x/nr_threads=64t/sync_method=NoSync/test_size=40G Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase: gcc-4.9/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel/64t/debian-x86_64-2015-02-07.cgz/NoSync/ivb44/40G/fsmark commit: 6b25579f3a0c53fc63884d1487e3e8434670def4 818e8e4109d58cc55c884223a1d93426024d23d8 6b25579f3a0c53fc 818e8e4109d58cc55c884223a1 ---------------- -------------------------- %stddev %change %stddev \ | \ 10245512 ± 2% -54.4% 4675370 ± 2% fsmark.app_overhead 83.20 ± 0% -7.7% 76.80 ± 0% fsmark.files_per_sec 123.40 ± 0% +6.2% 131.06 ± 0% fsmark.time.elapsed_time 123.40 ± 0% +6.2% 131.06 ± 0% fsmark.time.elapsed_time.max 45128 ± 0% -68.3% 14323 ± 1% fsmark.time.involuntary_context_switches 37.50 ± 1% -17.3% 31.00 ± 0% fsmark.time.percent_of_cpu_this_job_got 46.66 ± 0% -11.5% 41.30 ± 0% fsmark.time.system_time 1488713 ± 0% -6.6% 1390819 ± 0% fsmark.time.voluntary_context_switches 280541 ± 2% +62.5% 455972 ± 0% softirqs.NET_RX 334642 ± 0% -5.7% 315581 ± 0% vmstat.io.bo 5821075 ± 0% +21.6% 7079435 ± 0% vmstat.memory.cache 46892 ± 0% -5.7% 44240 ± 0% vmstat.system.cs 21396 ± 28% -67.4% 6979 ± 11% latency_stats.max.call_rwsem_down_write_failed.filename_create.SyS_mkdir.entry_SYSCALL_64_fastpath 10905 ± 16% +474.2% 62621 ±165% latency_stats.max.rpc_wait_bit_killable.__rpc_execute.rpc_execute.rpc_run_task.nfs4_call_sync_sequence.[nfsv4]._nfs4_proc_lookup.[nfsv4].nfs4_proc_lookup_common.[nfsv4].nfs4_proc_lookup.[nfsv4].nfs_lookup_revalidate.nfs4_lookup_revalidate.lookup_dcache.__lookup_hash 3782146 ± 6% -70.4% 1120937 ± 2% latency_stats.sum.call_rwsem_down_write_failed.filename_create.SyS_mkdir.entry_SYSCALL_64_fastpath 208165 ± 10% -75.5% 51062 ± 13% latency_stats.sum.rpc_wait_bit_killable.__rpc_execute.rpc_execute.rpc_run_task.nfs4_call_sync_sequence.[nfsv4]._nfs4_proc_access.[nfsv4].nfs4_proc_access.[nfsv4].nfs_do_access.nfs_permission.__inode_permission.inode_permission.link_path_walk 7.46 ± 0% -6.5% 6.98 ± 0% turbostat.%Busy 158.75 ± 0% -5.8% 149.50 ± 0% turbostat.Avg_MHz 0.31 ± 6% -35.0% 0.20 ± 3% turbostat.CPU%c3 4.73 ± 0% -1.5% 4.65 ± 0% turbostat.RAMWatt 37004192 ± 2% -73.8% 9682574 ± 1% cpuidle.C1E-IVT.time 120363 ± 1% -52.9% 56683 ± 1% cpuidle.C1E-IVT.usage 23302776 ± 3% -24.2% 17659372 ± 2% cpuidle.C3-IVT.time 82331 ± 1% -21.0% 65040 ± 0% cpuidle.C3-IVT.usage 36963 ± 1% -25.0% 27736 ± 2% cpuidle.POLL.usage 5420047 ± 1% +23.2% 6677171 ± 0% meminfo.Cached 3081 ± 6% -46.5% 1648 ± 12% meminfo.Dirty 5358458 ± 1% +23.3% 6605440 ± 0% meminfo.Inactive 5349323 ± 1% +23.3% 6596293 ± 0% meminfo.Inactive(file) 3093 ± 13% -56.6% 1344 ± 1% meminfo.NFS_Unstable 145243 ± 1% +15.9% 168335 ± 0% meminfo.SReclaimable 44381 ± 3% +22.4% 54339 ± 2% meminfo.Unevictable 2696 ± 6% -49.5% 1362 ± 7% meminfo.Writeback 615701 ± 1% +25.7% 773764 ± 0% slabinfo.buffer_head.active_objs 15787 ± 1% +25.7% 19840 ± 0% slabinfo.buffer_head.active_slabs 615714 ± 1% +25.7% 773776 ± 0% slabinfo.buffer_head.num_objs 15787 ± 1% +25.7% 19840 ± 0% slabinfo.buffer_head.num_slabs 57899 ± 2% +21.8% 70515 ± 1% slabinfo.radix_tree_node.active_objs 1037 ± 2% +21.3% 1258 ± 1% slabinfo.radix_tree_node.active_slabs 58118 ± 2% +21.3% 70515 ± 1% slabinfo.radix_tree_node.num_objs 1037 ± 2% +21.3% 1258 ± 1% slabinfo.radix_tree_node.num_slabs 1266 ± 2% -10.9% 1128 ± 3% slabinfo.scsi_data_buffer.active_objs 1266 ± 2% -10.9% 1128 ± 3% slabinfo.scsi_data_buffer.num_objs 760.25 ± 2% +22.2% 929.00 ± 1% proc-vmstat.kswapd_low_wmark_hit_quickly 760.00 ± 4% -44.9% 418.50 ± 8% proc-vmstat.nr_dirty 1355408 ± 1% +23.2% 1669485 ± 0% proc-vmstat.nr_file_pages 1337536 ± 1% +23.3% 1649068 ± 0% proc-vmstat.nr_inactive_file 36313 ± 1% +15.9% 42082 ± 0% proc-vmstat.nr_slab_reclaimable 11094 ± 3% +22.4% 13584 ± 2% proc-vmstat.nr_unevictable 756.00 ± 13% -55.8% 334.25 ± 2% proc-vmstat.nr_unstable 681.75 ± 9% -48.2% 353.25 ± 5% proc-vmstat.nr_writeback 1488 ± 34% -90.8% 136.25 ±173% proc-vmstat.pgscan_direct_dma32 34551 ± 22% -91.0% 3113 ±173% proc-vmstat.pgscan_direct_normal 1004 ± 49% -86.4% 136.25 ±173% proc-vmstat.pgsteal_direct_dma32 7921 ± 64% -83.3% 1322 ±173% proc-vmstat.pgsteal_direct_normal 9.06e+08 ± 2% -10.8% 8.078e+08 ± 4% perf-stat.L1-dcache-prefetch-misses 4.514e+09 ± 0% +5.6% 4.766e+09 ± 1% perf-stat.L1-icache-load-misses 1.53e+09 ± 5% -20.4% 1.218e+09 ± 0% perf-stat.LLC-load-misses 2.351e+09 ± 3% +10.2% 2.592e+09 ± 2% perf-stat.LLC-prefetches 2.48e+09 ± 3% +10.7% 2.746e+09 ± 2% perf-stat.LLC-store-misses 3.651e+09 ± 1% +5.6% 3.855e+09 ± 2% perf-stat.LLC-stores 6.288e+10 ± 4% +11.5% 7.01e+10 ± 0% perf-stat.branch-instructions 1.176e+09 ± 2% +11.5% 1.311e+09 ± 4% perf-stat.branch-load-misses 5.597e+10 ± 10% +13.1% 6.332e+10 ± 6% perf-stat.branch-loads 1.241e+09 ± 2% +11.6% 1.386e+09 ± 4% perf-stat.branch-misses 1.038e+11 ± 2% +9.3% 1.134e+11 ± 2% perf-stat.dTLB-loads 92930638 ± 1% +4.9% 97490872 ± 2% perf-stat.iTLB-load-misses 277756 ± 0% +5.3% 292450 ± 0% perf-stat.minor-faults 1.138e+09 ± 10% -26.1% 8.414e+08 ± 5% perf-stat.node-loads 6.039e+08 ± 9% -18.9% 4.896e+08 ± 4% perf-stat.node-prefetches 5.084e+08 ± 2% +25.3% 6.372e+08 ± 5% perf-stat.node-store-misses 277802 ± 0% +5.3% 292431 ± 0% perf-stat.page-faults 1398 ± 12% -49.8% 701.75 ± 12% numa-meminfo.node0.Dirty 2673371 ± 2% +21.0% 3233672 ± 6% numa-meminfo.node0.FilePages 2642305 ± 2% +21.0% 3197794 ± 6% numa-meminfo.node0.Inactive 2639728 ± 1% +21.1% 3197516 ± 6% numa-meminfo.node0.Inactive(file) 4995759 ± 2% +21.7% 6082060 ± 2% numa-meminfo.node0.MemUsed 1521 ± 5% -59.6% 614.25 ± 11% numa-meminfo.node0.NFS_Unstable 21760 ± 3% +23.2% 26805 ± 0% numa-meminfo.node0.Unevictable 1438 ± 17% -51.5% 698.00 ± 13% numa-meminfo.node0.Writeback 3517 ± 61% +56.7% 5511 ± 15% numa-meminfo.node1.AnonHugePages 1634 ± 9% -46.2% 879.00 ± 9% numa-meminfo.node1.Dirty 2726731 ± 1% +25.8% 3429717 ± 5% numa-meminfo.node1.FilePages 2696295 ± 1% +25.9% 3393847 ± 5% numa-meminfo.node1.Inactive 2689711 ± 1% +25.8% 3384959 ± 5% numa-meminfo.node1.Inactive(file) 4917935 ± 2% +23.5% 6072040 ± 2% numa-meminfo.node1.MemUsed 1384 ± 2% -48.5% 713.75 ± 21% numa-meminfo.node1.NFS_Unstable 69455 ± 1% +22.6% 85179 ± 7% numa-meminfo.node1.SReclaimable 120279 ± 3% +14.6% 137894 ± 8% numa-meminfo.node1.Slab 21762 ± 3% +23.3% 26831 ± 0% numa-meminfo.node1.Unevictable 1702 ± 9% -58.3% 709.50 ± 11% numa-meminfo.node1.Writeback 975493 ± 2% +21.0% 1180348 ± 5% numa-vmstat.node0.nr_dirtied 380.50 ± 9% -55.2% 170.50 ± 13% numa-vmstat.node0.nr_dirty 668385 ± 2% +20.9% 808309 ± 6% numa-vmstat.node0.nr_file_pages 659973 ± 1% +21.1% 799269 ± 6% numa-vmstat.node0.nr_inactive_file 5439 ± 3% +23.2% 6701 ± 0% numa-vmstat.node0.nr_unevictable 369.25 ± 7% -59.4% 150.00 ± 9% numa-vmstat.node0.nr_unstable 379.00 ± 15% -56.6% 164.50 ± 11% numa-vmstat.node0.nr_writeback 974763 ± 2% +21.1% 1180042 ± 5% numa-vmstat.node0.nr_written 1655473 ± 2% +21.5% 2011897 ± 6% numa-vmstat.node0.numa_hit 1655473 ± 2% +21.5% 2011897 ± 6% numa-vmstat.node0.numa_local 967615 ± 3% +28.0% 1238876 ± 5% numa-vmstat.node1.nr_dirtied 367.00 ± 0% -40.3% 219.00 ± 9% numa-vmstat.node1.nr_dirty 681556 ± 1% +25.8% 857306 ± 5% numa-vmstat.node1.nr_file_pages 672299 ± 1% +25.9% 846118 ± 5% numa-vmstat.node1.nr_inactive_file 17358 ± 1% +22.7% 21291 ± 7% numa-vmstat.node1.nr_slab_reclaimable 5440 ± 3% +23.3% 6708 ± 0% numa-vmstat.node1.nr_unevictable 348.75 ± 10% -45.3% 190.75 ± 17% numa-vmstat.node1.nr_unstable 389.75 ± 13% -48.5% 200.75 ± 4% numa-vmstat.node1.nr_writeback 966871 ± 3% +28.1% 1238490 ± 5% numa-vmstat.node1.nr_written 1594506 ± 4% +27.2% 2028330 ± 5% numa-vmstat.node1.numa_hit 1594505 ± 4% +27.2% 2028329 ± 5% numa-vmstat.node1.numa_local 976.82 ± 1% -26.3% 719.87 ± 1% sched_debug.cfs_rq:/.exec_clock.avg 2411 ± 3% -12.9% 2101 ± 3% sched_debug.cfs_rq:/.exec_clock.max 58.45 ± 11% -60.9% 22.88 ± 8% sched_debug.cfs_rq:/.exec_clock.min 887.20 ± 1% -20.3% 706.71 ± 0% sched_debug.cfs_rq:/.exec_clock.stddev 330.47 ± 5% -60.8% 129.43 ± 6% sched_debug.cfs_rq:/.load_avg.avg 1765 ± 7% -56.7% 763.92 ± 9% sched_debug.cfs_rq:/.load_avg.max 456.70 ± 7% -56.4% 199.20 ± 7% sched_debug.cfs_rq:/.load_avg.stddev 49.53 ± 14% -33.3% 33.04 ± 10% sched_debug.cfs_rq:/.runnable_load_avg.avg 140.97 ± 23% -26.0% 104.31 ± 4% sched_debug.cfs_rq:/.runnable_load_avg.stddev 746.08 ± 5% -12.9% 649.58 ± 2% sched_debug.cfs_rq:/.util_avg.max 153.35 ± 6% -16.3% 128.31 ± 3% sched_debug.cfs_rq:/.util_avg.stddev 36.90 ± 11% -21.6% 28.93 ± 13% sched_debug.cpu.cpu_load[0].avg 108.16 ± 5% -10.7% 96.61 ± 6% sched_debug.cpu.cpu_load[0].stddev 67.83 ± 6% -22.6% 52.51 ± 6% sched_debug.cpu.cpu_load[1].avg 133.54 ± 7% -17.7% 109.90 ± 12% sched_debug.cpu.cpu_load[1].stddev 62.46 ± 7% -23.9% 47.54 ± 3% sched_debug.cpu.cpu_load[2].avg 632.17 ± 11% -14.3% 541.67 ± 9% sched_debug.cpu.cpu_load[2].max 119.72 ± 7% -18.3% 97.77 ± 7% sched_debug.cpu.cpu_load[2].stddev 55.54 ± 7% -24.7% 41.80 ± 2% sched_debug.cpu.cpu_load[3].avg 589.17 ± 6% -17.2% 488.00 ± 4% sched_debug.cpu.cpu_load[3].max 107.89 ± 5% -19.1% 87.30 ± 4% sched_debug.cpu.cpu_load[3].stddev 50.93 ± 8% -26.0% 37.71 ± 2% sched_debug.cpu.cpu_load[4].avg 543.75 ± 4% -16.7% 453.08 ± 4% sched_debug.cpu.cpu_load[4].max 99.09 ± 3% -18.6% 80.63 ± 3% sched_debug.cpu.cpu_load[4].stddev 0.00 ± 44% -43.6% 0.00 ± 13% sched_debug.cpu.next_balance.stddev 2415 ± 6% -11.7% 2133 ± 4% sched_debug.cpu.nr_load_updates.stddev 37473 ± 1% -27.5% 27179 ± 0% sched_debug.cpu.nr_switches.avg 87510 ± 7% -23.2% 67204 ± 6% sched_debug.cpu.nr_switches.max 1512 ± 15% -58.6% 625.33 ± 8% sched_debug.cpu.nr_switches.min 35187 ± 1% -24.7% 26504 ± 1% sched_debug.cpu.nr_switches.stddev 12.59 ± 10% -27.2% 9.16 ± 8% sched_debug.cpu.nr_uninterruptible.stddev 36793 ± 1% -28.0% 26494 ± 0% sched_debug.cpu.sched_count.avg 90630 ± 4% -22.0% 70672 ± 9% sched_debug.cpu.sched_count.max 1189 ± 16% -71.9% 333.92 ± 13% sched_debug.cpu.sched_count.min 35241 ± 1% -24.8% 26503 ± 1% sched_debug.cpu.sched_count.stddev 16975 ± 1% -28.4% 12159 ± 1% sched_debug.cpu.sched_goidle.avg 39073 ± 6% -22.9% 30133 ± 6% sched_debug.cpu.sched_goidle.max 517.50 ± 17% -75.5% 126.83 ± 13% sched_debug.cpu.sched_goidle.min 16091 ± 1% -25.3% 12019 ± 1% sched_debug.cpu.sched_goidle.stddev 18581 ± 1% -27.9% 13391 ± 1% sched_debug.cpu.ttwu_count.avg 43020 ± 5% -25.7% 31949 ± 7% sched_debug.cpu.ttwu_count.max 919.08 ± 15% -77.3% 208.75 ± 2% sched_debug.cpu.ttwu_count.min 16996 ± 1% -23.8% 12956 ± 1% sched_debug.cpu.ttwu_count.stddev 1647 ± 0% -18.3% 1347 ± 0% sched_debug.cpu.ttwu_local.avg 3739 ± 3% -14.2% 3207 ± 3% sched_debug.cpu.ttwu_local.max 120.75 ± 4% -37.2% 75.83 ± 3% sched_debug.cpu.ttwu_local.min 1476 ± 1% -13.7% 1273 ± 1% sched_debug.cpu.ttwu_local.stddev 1.05 ± 4% -12.8% 0.92 ± 2% perf-profile.cycles-pp.__filemap_fdatawait_range.filemap_fdatawait_range.filemap_write_and_wait_range.xfs_file_fsync.vfs_fsync_range 12.69 ± 1% -25.3% 9.47 ± 1% perf-profile.cycles-pp.__filemap_fdatawrite_range.filemap_write_and_wait_range.xfs_file_fsync.vfs_fsync_range.nfsd_commit 10.17 ± 5% -17.3% 8.41 ± 5% perf-profile.cycles-pp.__generic_file_write_iter.generic_file_write_iter.nfs_file_write.__vfs_write.vfs_write 1.70 ± 2% +17.8% 2.00 ± 5% perf-profile.cycles-pp.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread.kthread 10.41 ± 5% -18.1% 8.52 ± 4% perf-profile.cycles-pp.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath 1.25 ± 6% +52.7% 1.90 ± 5% perf-profile.cycles-pp.apic_timer_interrupt.cpuidle_enter.call_cpuidle.cpu_startup_entry.start_secondary 7.69 ± 2% -30.9% 5.31 ± 3% perf-profile.cycles-pp.brd_do_bvec.brd_make_request.generic_make_request.submit_bio.xfs_add_to_ioend 2.61 ± 2% -28.3% 1.87 ± 5% perf-profile.cycles-pp.brd_do_bvec.brd_make_request.generic_make_request.submit_bio.xfs_submit_ioend.isra.17 4.68 ± 1% -19.5% 3.77 ± 2% perf-profile.cycles-pp.brd_insert_page.part.4.brd_do_bvec.brd_make_request.generic_make_request.submit_bio 7.80 ± 2% -30.5% 5.42 ± 3% perf-profile.cycles-pp.brd_make_request.generic_make_request.submit_bio.xfs_add_to_ioend.xfs_do_writepage 2.64 ± 2% -26.3% 1.94 ± 2% perf-profile.cycles-pp.brd_make_request.generic_make_request.submit_bio.xfs_submit_ioend.isra.17.xfs_vm_writepages 3.15 ± 7% +121.1% 6.96 ± 2% perf-profile.cycles-pp.call_rwsem_down_write_failed.down_write.xfs_file_buffered_aio_write.xfs_file_write_iter.do_iter_readv_writev 1.31 ± 4% +26.0% 1.66 ± 5% perf-profile.cycles-pp.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work.worker_thread 3.58 ± 7% -16.2% 3.00 ± 3% perf-profile.cycles-pp.copy_page_to_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.sock_recvmsg 3.89 ± 5% -13.6% 3.36 ± 5% perf-profile.cycles-pp.copy_user_enhanced_fast_string.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.nfs_file_write 13.17 ± 6% +26.5% 16.66 ± 1% perf-profile.cycles-pp.do_iter_readv_writev.do_readv_writev.vfs_writev.nfsd_vfs_write.nfsd4_write 13.21 ± 6% +26.3% 16.69 ± 1% perf-profile.cycles-pp.do_readv_writev.vfs_writev.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound 12.68 ± 1% -25.3% 9.47 ± 1% perf-profile.cycles-pp.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range.xfs_file_fsync.vfs_fsync_range 3.15 ± 7% +120.8% 6.96 ± 2% perf-profile.cycles-pp.down_write.xfs_file_buffered_aio_write.xfs_file_write_iter.do_iter_readv_writev.do_readv_writev 16.75 ± 3% -13.9% 14.42 ± 4% perf-profile.cycles-pp.entry_SYSCALL_64_fastpath 1.07 ± 4% -12.9% 0.93 ± 3% perf-profile.cycles-pp.filemap_fdatawait_range.filemap_write_and_wait_range.xfs_file_fsync.vfs_fsync_range.nfsd_commit 3.51 ± 4% -7.4% 3.25 ± 5% perf-profile.cycles-pp.filemap_write_and_wait_range.nfs_file_fsync.vfs_fsync_range.vfs_fsync.nfs4_file_flush 13.75 ± 1% -24.3% 10.41 ± 1% perf-profile.cycles-pp.filemap_write_and_wait_range.xfs_file_fsync.vfs_fsync_range.nfsd_commit.nfsd4_commit 10.23 ± 5% -17.4% 8.46 ± 5% perf-profile.cycles-pp.generic_file_write_iter.nfs_file_write.__vfs_write.vfs_write.sys_write 7.84 ± 1% -30.3% 5.47 ± 3% perf-profile.cycles-pp.generic_make_request.submit_bio.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages 2.64 ± 2% -26.0% 1.95 ± 2% perf-profile.cycles-pp.generic_make_request.submit_bio.xfs_submit_ioend.isra.17.xfs_vm_writepages.do_writepages 10.05 ± 5% -17.8% 8.26 ± 5% perf-profile.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.nfs_file_write.__vfs_write 4.65 ± 7% -13.6% 4.02 ± 2% perf-profile.cycles-pp.inet_recvmsg.sock_recvmsg.kernel_recvmsg.svc_recvfrom.isra.9.svc_tcp_recvfrom 1.00 ± 4% +27.6% 1.28 ± 8% perf-profile.cycles-pp.inet_sendpage.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit 4.66 ± 7% -13.3% 4.04 ± 2% perf-profile.cycles-pp.kernel_recvmsg.svc_recvfrom.isra.9.svc_tcp_recvfrom.svc_recv.nfsd 3.45 ± 6% -16.4% 2.89 ± 4% perf-profile.cycles-pp.memcpy_erms.copy_page_to_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg 10.39 ± 5% -18.0% 8.52 ± 5% perf-profile.cycles-pp.nfs_file_write.__vfs_write.vfs_write.sys_write.entry_SYSCALL_64_fastpath 1.85 ± 3% -15.4% 1.57 ± 6% perf-profile.cycles-pp.nfs_updatepage.nfs_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 3.80 ± 4% -24.3% 2.88 ± 7% perf-profile.cycles-pp.nfs_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.nfs_file_write 13.81 ± 1% -24.0% 10.50 ± 1% perf-profile.cycles-pp.nfsd4_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 13.25 ± 6% +26.2% 16.73 ± 1% perf-profile.cycles-pp.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common.svc_process 13.81 ± 1% -24.0% 10.50 ± 1% perf-profile.cycles-pp.nfsd_commit.nfsd4_commit.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 13.22 ± 6% +26.3% 16.70 ± 1% perf-profile.cycles-pp.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch.svc_process_common 0.00 ± -1% +Inf% 1.56 ± 3% perf-profile.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write 1.71 ± 3% +17.8% 2.02 ± 4% perf-profile.cycles-pp.rpc_async_schedule.process_one_work.worker_thread.kthread.ret_from_fork 3.15 ± 7% +121.0% 6.96 ± 2% perf-profile.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.xfs_file_buffered_aio_write.xfs_file_write_iter 3.15 ± 7% +121.0% 6.96 ± 2% perf-profile.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.xfs_file_buffered_aio_write 2.73 ± 6% +93.7% 5.29 ± 2% perf-profile.cycles-pp.rwsem_spin_on_owner.rwsem_optimistic_spin.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write 3.81 ± 6% -15.5% 3.22 ± 3% perf-profile.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.sock_recvmsg.kernel_recvmsg 1.19 ± 7% +54.9% 1.85 ± 5% perf-profile.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter.call_cpuidle.cpu_startup_entry 4.66 ± 7% -13.3% 4.04 ± 2% perf-profile.cycles-pp.sock_recvmsg.kernel_recvmsg.svc_recvfrom.isra.9.svc_tcp_recvfrom.svc_recv 7.84 ± 1% -30.2% 5.47 ± 3% perf-profile.cycles-pp.submit_bio.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages.xfs_vm_writepages 2.64 ± 2% -26.0% 1.95 ± 2% perf-profile.cycles-pp.submit_bio.xfs_submit_ioend.isra.17.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range 4.67 ± 7% -13.3% 4.05 ± 2% perf-profile.cycles-pp.svc_recvfrom.isra.9.svc_tcp_recvfrom.svc_recv.nfsd.kthread 10.51 ± 5% -17.7% 8.65 ± 5% perf-profile.cycles-pp.sys_write.entry_SYSCALL_64_fastpath 4.64 ± 7% -13.8% 4.00 ± 2% perf-profile.cycles-pp.tcp_recvmsg.inet_recvmsg.sock_recvmsg.kernel_recvmsg.svc_recvfrom.isra.9 0.96 ± 4% +29.1% 1.24 ± 7% perf-profile.cycles-pp.tcp_sendpage.inet_sendpage.xs_sendpages.xs_tcp_send_request.xprt_transmit 13.81 ± 1% -24.1% 10.48 ± 1% perf-profile.cycles-pp.vfs_fsync_range.nfsd_commit.nfsd4_commit.nfsd4_proc_compound.nfsd_dispatch 10.49 ± 5% -17.9% 8.62 ± 5% perf-profile.cycles-pp.vfs_write.sys_write.entry_SYSCALL_64_fastpath 13.21 ± 6% +26.3% 16.69 ± 1% perf-profile.cycles-pp.vfs_writev.nfsd_vfs_write.nfsd4_write.nfsd4_proc_compound.nfsd_dispatch 10.03 ± 1% -25.1% 7.51 ± 1% perf-profile.cycles-pp.write_cache_pages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range 8.03 ± 2% -30.2% 5.61 ± 3% perf-profile.cycles-pp.xfs_add_to_ioend.xfs_do_writepage.write_cache_pages.xfs_vm_writepages.do_writepages 9.43 ± 1% -26.4% 6.93 ± 2% perf-profile.cycles-pp.xfs_do_writepage.write_cache_pages.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range 13.17 ± 6% +26.5% 16.65 ± 1% perf-profile.cycles-pp.xfs_file_buffered_aio_write.xfs_file_write_iter.do_iter_readv_writev.do_readv_writev.vfs_writev 13.80 ± 0% -24.1% 10.48 ± 1% perf-profile.cycles-pp.xfs_file_fsync.vfs_fsync_range.nfsd_commit.nfsd4_commit.nfsd4_proc_compound 13.17 ± 6% +26.5% 16.66 ± 1% perf-profile.cycles-pp.xfs_file_write_iter.do_iter_readv_writev.do_readv_writev.vfs_writev.nfsd_vfs_write 2.65 ± 2% -26.0% 1.96 ± 2% perf-profile.cycles-pp.xfs_submit_ioend.isra.17.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range 12.68 ± 1% -25.3% 9.47 ± 1% perf-profile.cycles-pp.xfs_vm_writepages.do_writepages.__filemap_fdatawrite_range.filemap_write_and_wait_range.xfs_file_fsync 1.26 ± 4% +27.8% 1.61 ± 5% perf-profile.cycles-pp.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule.process_one_work 1.24 ± 3% +28.0% 1.59 ± 5% perf-profile.cycles-pp.xs_sendpages.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute 1.24 ± 3% +28.4% 1.59 ± 5% perf-profile.cycles-pp.xs_tcp_send_request.xprt_transmit.call_transmit.__rpc_execute.rpc_async_schedule fsmark.time.system_time 48 ++---------------------------------------------------------------------+ | *. * *. *.. | 47 ++ + * .. + .* .. * + * .*.. .* 46 ++ .* + .* *. .*..* + .* + .* + .* * | *.*. *..*.* * *..*.* *.*..* * | 45 ++ | | | 44 ++ | | | 43 ++ | 42 ++ | | O | 41 ++ O O O O O O O O O O O O O O O O O O O | O O O O O O | 40 ++-----O---------------------------------------------------------------+ fsmark.time.percent_of_cpu_this_job_got 38 ++-----*-*-----------*----------*-----------*-*----------*--*-----*----* | + + .. + + + .. + + + + +| 37 *+*..* *..*.*.* *.*.*..* *..*.*.* *.*..*.* *.* * | 36 ++ | | | 35 ++ | | | 34 ++ | | | 33 ++ | 32 ++ | | | 31 ++ O O O O O O O O O O O O O O O O O O O O O O | | | 30 O+O--O-O-O-------------------------------------------------------------+ fsmark.time.elapsed_time 134 ++---------O----------------------------------------------------------+ O O O O O O | 132 ++ | | O O | | O O O O O O O O O O O O O O O O O | 130 ++ O | | | 128 ++ | | | 126 ++ | | | | *.. *.. *. | 124 ++ *.*. .*.. .*..*. + *.*. + *.*.. .*.*.. .* * *.*..*.*.* *.*.* *.* *.*..* *.* * | 122 ++--------------------------------------------------------------------+ fsmark.time.elapsed_time.max 134 ++---------O----------------------------------------------------------+ O O O O O O | 132 ++ | | O O | | O O O O O O O O O O O O O O O O O | 130 ++ O | | | 128 ++ | | | 126 ++ | | | | *.. *.. *. | 124 ++ *.*. .*.. .*..*. + *.*. + *.*.. .*.*.. .* * *.*..*.*.* *.*.* *.* *.*..* *.* * | 122 ++--------------------------------------------------------------------+ fsmark.time.involuntary_context_switches 50000 ++------------------------------------------------------------------+ | | 45000 *+*.*..*.*.*.*.*..*.*.*.*.*..*.*.*.*.*.*..*.*.*.*.*..*.*.*.*.*..*.*.* 40000 ++ | | | 35000 ++ | | | 30000 ++ | | | 25000 ++ | 20000 ++ | | | 15000 ++ O O O O O O O O O O O O O O O O O O O O O | O O O O O O | 10000 ++------------------------------------------------------------------+ fsmark.files_per_sec 84 ++---------------------------------------------------------------------+ *.*..*.*.*.*..*.*.*..*.*.*.*..*.*.*..*.*.*..*.*.*.*..*.*.*..*.*.*.*..*.* 83 ++ | 82 ++ | | | 81 ++ | | | 80 ++ | | | 79 ++ | 78 ++ | | | 77 ++ | O O O O O O O O O O O O O O O O O O O O O O O O O O O | 76 ++---------------------------------------------------------------------+ fsmark.app_overhead 1.6e+07 ++----------------------------------------------------------------+ | * | 1.4e+07 ++ : | | : : | | : : | 1.2e+07 ++ : : | |.*.*. .* : .*. .*. .*.*. .*. .*.*. .*.. .*.| 1e+07 *+ *..* *.* *.*..*.*.* * *..* * *.* *.* * | | 8e+06 ++ | | | | | 6e+06 ++ | O O O O O O O O O O O O O O O O O O O O O O O O O O O | 4e+06 ++----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Xiaolong