May 2020 - LKP - Ml01.01.Org

d13fee049f ("Default enable RCU list lockdep debugging with .."): WARNING: suspicious RCU usage

by kernel test robot

Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git rcu/next commit d13fee049fa8e94649aa9914fd1ab5d07da51da4 Author: Madhuparna Bhowmik <madhuparnabhowmik10(a)gmail.com> AuthorDate: Fri Feb 28 14:54:51 2020 +0530 Commit: Paul E. McKenney <paulmck(a)kernel.org> CommitDate: Thu May 7 10:24:43 2020 -0700 Default enable RCU list lockdep debugging with PROVE_RCU This patch default enables CONFIG_PROVE_RCU_LIST option with CONFIG_PROVE_RCU for RCU list lockdep debugging. With this change, RCU list lockdep debugging will be default enabled in CONFIG_PROVE_RCU=y kernels. Most of the RCU users (in core kernel/, drivers/, and net/ subsystem) have already been modified to include lockdep expressions hence RCU list debugging can be enabled by default. However, there are still chances of enountering false-positive lockdep splats because not everything is converted, in case RCU list primitives are used in non-RCU read-side critical section but under the protection of a lock. It would be okay to have a few false-positives, as long as bugs are identified, since this patch only affects debugging kernels. Co-developed-by: Amol Grover <frextrite(a)gmail.com> Signed-off-by: Amol Grover <frextrite(a)gmail.com> Signed-off-by: Madhuparna Bhowmik <madhuparnabhowmik10(a)gmail.com> Acked-by: Joel Fernandes (Google) <joel(a)joelfernandes.org> Signed-off-by: Paul E. McKenney <paulmck(a)kernel.org> c1fe153b2a torture: Add --allcpus argument to the kvm.sh script d13fee049f Default enable RCU list lockdep debugging with PROVE_RCU ec06fa2b14 doc: Tasks RCU must protect instructions before trampoline +--------------------------------------------------------------------------------------+------------+------------+------------+ | | c1fe153b2a | d13fee049f | ec06fa2b14 | +--------------------------------------------------------------------------------------+------------+------------+------------+ | boot_successes | 31 | 0 | 0 | | boot_failures | 2 | 11 | 11 | | BUG:kernel_hang_in_early-boot_stage,last_printk:Probing_EDD(edd=off_to_disable)...ok | 2 | | | | WARNING:suspicious_RCU_usage | 0 | 11 | 11 | | net/ipv6/ip6mr.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | 11 | | BUG:kernel_hang_in_test_stage | 0 | 1 | | +--------------------------------------------------------------------------------------+------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 71.408539] Performance counters on [ 71.409303] input device check on [ 71.473366] NET: Registered protocol family 10 [ 71.479116] [ 71.479916] ============================= [ 71.480827] WARNING: suspicious RCU usage [ 71.481663] 5.7.0-rc2-00234-gd13fee049fa8e #1 Not tainted [ 71.482791] ----------------------------- [ 71.483839] net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!! [ 71.485271] [ 71.485271] other info that might help us debug this: [ 71.485271] [ 71.487039] [ 71.487039] rcu_scheduler_active = 2, debug_locks = 1 [ 71.488607] 1 lock held by swapper/1: [ 71.489445] #0: ffffffffa9d313f0 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x1d/0x60 [ 71.491830] [ 71.491830] stack backtrace: [ 71.492769] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00234-gd13fee049fa8e #1 [ 71.494422] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 71.496343] Call Trace: [ 71.496909] dump_stack+0x23/0x38 [ 71.497671] lockdep_rcu_suspicious+0x10c/0x11a [ 71.498683] ip6mr_get_table+0x117/0x130 [ 71.499491] ip6mr_new_table+0x19/0x60 [ 71.500021] ip6mr_net_init+0xf6/0x267 [ 71.500472] ops_init+0x60/0x2b0 [ 71.501054] register_pernet_operations+0xad/0x1e0 [ 71.501899] ? unix_sysctl_register+0xe4/0xe4 [ 71.502727] register_pernet_subsys+0x34/0x60 [ 71.503661] ip6_mr_init+0x58/0x15e [ 71.504290] inet6_init+0x1a1/0x68f [ 71.505024] do_one_initcall+0x102/0x500 [ 71.505811] ? perf_trace_initcall_level+0x2c0/0x2c0 [ 71.506759] ? kasan_unpoison_shadow+0x38/0x60 [ 71.507766] ? rcu_read_lock_sched_held+0x85/0xd0 [ 71.508607] do_basic_setup+0x1a9/0x1fd [ 71.509327] kernel_init_freeable+0x115/0x183 [ 71.510131] ? rest_init+0x22e/0x22e [ 71.510854] kernel_init+0x13/0x1d2 [ 71.511763] ? rest_init+0x22e/0x22e [ 71.512572] ret_from_fork+0x24/0x30 modprobe: FATAL: Could not load /lib/modules/5.7.0-rc2-00234-gd13fee049fa8e/modules.dep: No such file or directory [ 71.746089] Segment Routing with IPv6 [ 71.746959] RPL Segment Routing with IPv6 [ 71.772698] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 71.803782] can: controller area network core (rev 20170425 abi 9) # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD git bisect start a41e54ce803b45b284e10a4e231dba5fec292064 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8 -- git bisect bad 2f7991448f9c479f7ce0e7007b697555df328b46 # 23:56 B 0 2 18 0 Merge 'linux-review/Pavel-Begunkov/io_uring-fix-zero-len-do_splice/20200505-171034' into devel-hourly-2020051219 git bisect bad 3edfcaf0e7672c1e6c7bbfe8db1ceca43f5e7c85 # 00:28 B 0 2 18 0 Merge 'linux-review/Dan-Williams/ACPI-Drop-rcu-usage-for-MMIO-mappings/20200509-013208' into devel-hourly-2020051219 git bisect good 3f1db4c8a03e1ab931248c00ae4c4559e3b5406b # 01:17 G 10 0 1 1 Merge 'ezequielg/rkvdec-vp9-upstream' into devel-hourly-2020051219 git bisect good 5579a78821bdf86725db495878dd9c6245ed2bb3 # 01:41 G 10 0 1 1 Merge 'linux-review/Takashi-Sakamoto/ALSA-firewire-motu-fulfill-missing-entries-in-Kconfig/20200511-103548' into devel-hourly-2020051219 git bisect good e3689f9e910f8fd4b1f565359664e4a72b112034 # 02:13 G 10 0 0 0 Merge 'linux-review/UPDATE-20200512-031403/Lee-Shawn-C/drm-i915-mst-filter-out-the-display-mode-exceed-sink-s-capability/20200420-213842' into devel-hourly-2020051219 git bisect bad d69b0f11241256c60dbf757ac3dd5a96ed91882e # 02:33 B 0 2 18 0 Merge 'rcu/rcu/dev' into devel-hourly-2020051219 git bisect good 0144187eb59fafe035c65eff7a3935a80da27eb0 # 03:12 G 10 0 0 0 Merge 'linux-review/Maninder-Singh/scripts-checkstack-pl-don-t-display-dre-as-different-entity/20200509-051219' into devel-hourly-2020051219 git bisect good f4311b8d31874d2bde6adabf8ac1d335ea35b35f # 03:52 G 10 0 0 0 Merge 'linux-review/Saeed-Mahameed/Mellanox-mlx5-and-bonding-updates-2020-05-09/20200509-191959' into devel-hourly-2020051219 git bisect good 4a49fd5e59ac0f31ef28ebb545a73845e7943416 # 04:17 G 10 0 1 1 Merge 'nf/master' into devel-hourly-2020051219 git bisect good 071972c7badf15fd8fbb804c6b5d727a48d16ed7 # 04:44 G 10 0 1 1 Merge 'zen-kernel-zen-kernel/5.6/fsgsbase' into devel-hourly-2020051219 git bisect good 3123dcd3ef7f98482dd0a459e85a750448161b38 # 05:27 G 10 0 1 1 fork: Annotate a data race in vm_area_dup() git bisect good 8d6d79f04175a9ce8091601a9e5eefc1dfead091 # 06:04 G 11 0 2 2 kcsan: Add test suite git bisect good 3ff7cf6d6d90ec5a196b31ef03777e756807e92a # 06:24 G 10 0 0 0 torture: Add script to smoke-test commits in a branch git bisect bad 91f877ab4b94c0d463a60b8c77d529096a8268c1 # 06:40 B 0 1 17 0 rcu: Priority-boost-related sleeps to idle priority git bisect good c1fe153b2a011a7d5f156f12ee518b1b0dc53fa1 # 07:20 G 10 0 0 0 torture: Add --allcpus argument to the kvm.sh script git bisect bad 3d9e98763d855022fef5285b3fd779238f9ab554 # 07:35 B 0 1 17 0 rcu: Grace-period-kthread related sleeps to idle priority git bisect bad d13fee049fa8e94649aa9914fd1ab5d07da51da4 # 07:58 B 0 2 18 0 Default enable RCU list lockdep debugging with PROVE_RCU # first bad commit: [d13fee049fa8e94649aa9914fd1ab5d07da51da4] Default enable RCU list lockdep debugging with PROVE_RCU git bisect good c1fe153b2a011a7d5f156f12ee518b1b0dc53fa1 # 08:12 G 30 0 0 1 torture: Add --allcpus argument to the kvm.sh script # extra tests with debug options git bisect bad d13fee049fa8e94649aa9914fd1ab5d07da51da4 # 08:30 B 0 2 18 0 Default enable RCU list lockdep debugging with PROVE_RCU # extra tests on head commit of rcu/rcu/next git bisect bad ec06fa2b148836f3f50082eb502b8f0aadb8970a # 08:42 B 0 1 17 0 doc: Tasks RCU must protect instructions before trampoline # bad: [ec06fa2b148836f3f50082eb502b8f0aadb8970a] doc: Tasks RCU must protect instructions before trampoline # extra tests on revert first bad commit git bisect good 92587bc102c54408efd66e5b9212bbd3641967c1 # 09:27 G 10 0 1 1 Revert "Default enable RCU list lockdep debugging with PROVE_RCU" # good: [92587bc102c54408efd66e5b9212bbd3641967c1] Revert "Default enable RCU list lockdep debugging with PROVE_RCU" --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/lkp@lists.01.org

8 months, 2 weeks

1
0
0 / 0

0902bb3bb8: vm-scalability.median -86.2% regression

by kernel test robot

Greeting, FYI, we noticed a -86.2% regression of vm-scalability.median due to commit: commit: 0902bb3bb8fdb69f956f4c3ee8157fe5d1c1e44d ("[PATCH RFC tip/core/rcu] Add shrinker to shift to fast/inefficient GP mode") url: https://github.com/0day-ci/linux/commits/Paul-E-McKenney/Add-shrinker-to-... in testcase: vm-scalability on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory with following parameters: runtime: 300 thp_enabled: never thp_defrag: always nr_task: 8 nr_pmem: 1 test: swap-w-seq-mt bp_memmap: 96G!18G cpufreq_governor: performance ucode: 0x500002c test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= bp_memmap/compiler/cpufreq_governor/kconfig/nr_pmem/nr_task/rootfs/runtime/tbox_group/test/testcase/thp_defrag/thp_enabled/ucode: 96G!18G/gcc-7/performance/x86_64-rhel-7.6/1/8/debian-x86_64-20191114.cgz/300/lkp-csl-2sp6/swap-w-seq-mt/vm-scalability/always/never/0x500002c commit: baf5fe7618 ("Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu") 0902bb3bb8 ("Add shrinker to shift to fast/inefficient GP mode") baf5fe7618468151 0902bb3bb8fdb69f956f4c3ee81 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at_ip__slab_free/0x 1:4 -33% 0:4 perf-profile.children.cycles-pp.error_entry %stddev %change %stddev \ | \ 1.60 +11.1% 1.78 ± 4% vm-scalability.free_time 716935 ± 3% -86.2% 99160 ± 9% vm-scalability.median 46.71 ± 14% -35.3 11.43 ± 28% vm-scalability.stddev% 5619989 ± 2% -86.0% 788970 ± 9% vm-scalability.throughput 44.93 +635.3% 330.35 ± 10% vm-scalability.time.elapsed_time 44.93 +635.3% 330.35 ± 10% vm-scalability.time.elapsed_time.max 91727 ± 7% -49.8% 46077 ± 8% vm-scalability.time.involuntary_context_switches 198.06 ± 3% +927.4% 2034 ± 8% vm-scalability.time.system_time 102.53 +6.3% 108.96 ± 2% vm-scalability.time.user_time 9919 ± 10% -65.2% 3455 ± 9% vm-scalability.time.voluntary_context_switches 92.25 -3.7% 88.81 iostat.cpu.idle 5.24 ± 2% +106.5% 10.82 ± 2% iostat.cpu.system 2.51 -85.6% 0.36 ± 11% iostat.cpu.user 0.00 ±100% +0.0 0.00 ± 37% mpstat.cpu.all.iowait% 5.41 ± 2% +5.4 10.86 ± 2% mpstat.cpu.all.sys% 2.60 -2.2 0.36 ± 11% mpstat.cpu.all.usr% 918871 ± 24% +49.6% 1374492 ± 11% cpuidle.C1.time 21145 ± 18% +240.5% 72004 ± 9% cpuidle.C1.usage 1.541e+09 ± 95% +1137.7% 1.908e+10 ± 30% cpuidle.C1E.time 5241728 ± 34% +748.9% 44495355 ± 16% cpuidle.C1E.usage 34825 ± 7% +2744.5% 990603 ± 11% cpuidle.POLL.time 12508 ± 4% +3114.1% 402020 ± 9% cpuidle.POLL.usage 92.00 -3.6% 88.67 vmstat.cpu.id 1031 ± 3% -87.7% 126.33 ± 40% vmstat.memory.buff 61018140 ± 2% -48.5% 31447956 ± 11% vmstat.memory.free 6453338 ± 17% +53.4% 9897990 ± 4% vmstat.memory.swpd 7.25 ± 5% +83.9% 13.33 ± 3% vmstat.procs.r 909.25 ± 2% -86.3% 124.33 ± 3% vmstat.swap.si 611160 ± 9% -87.7% 74985 ± 2% vmstat.swap.so 7293 ± 7% -31.7% 4981 vmstat.system.cs 785179 ± 6% -70.1% 234556 vmstat.system.in 99837026 +18.5% 1.183e+08 ± 3% meminfo.Active 99835926 +18.5% 1.183e+08 ± 3% meminfo.Active(anon) 1100 ± 2% -67.1% 362.33 ± 12% meminfo.Active(file) 1.017e+08 ± 2% +20.0% 1.221e+08 ± 3% meminfo.AnonPages 1063 ± 2% -85.4% 155.33 ± 41% meminfo.Buffers 129506 ± 3% -56.6% 56267 ± 6% meminfo.CmaFree 1.704e+08 -7.7% 1.572e+08 ± 3% meminfo.Committed_AS 8189726 ± 10% -6.0% 7701277 ± 6% meminfo.DirectMap2M 1950380 ± 6% +95.5% 3813862 ± 3% meminfo.Inactive 1949183 ± 6% +95.6% 3813016 ± 3% meminfo.Inactive(anon) 1196 ± 3% -29.3% 845.33 ± 21% meminfo.Inactive(file) 83888 ± 2% +16.4% 97645 meminfo.KReclaimable 29604 +44.9% 42884 ± 4% meminfo.Mapped 58640993 ± 3% -34.5% 38418196 ± 11% meminfo.MemAvailable 59089646 ± 3% -34.3% 38806088 ± 11% meminfo.MemFree 1.056e+08 +19.2% 1.259e+08 ± 3% meminfo.Memused 218191 ± 2% +19.3% 260390 ± 3% meminfo.PageTables 83888 ± 2% +16.4% 97645 meminfo.SReclaimable 3557167 -82.4% 626370 ± 10% meminfo.max_used_kB 1041 ± 7% -82.7% 180.00 ± 69% numa-meminfo.node0.Active(file) 713269 ± 3% +73.5% 1237284 ± 2% numa-meminfo.node0.Inactive 712476 ± 3% +73.6% 1237076 ± 2% numa-meminfo.node0.Inactive(anon) 792.25 ± 27% -73.8% 207.33 ± 48% numa-meminfo.node0.Inactive(file) 8539 ± 2% -14.0% 7342 numa-meminfo.node0.KernelStack 15778 ± 13% +23.6% 19500 ± 8% numa-meminfo.node0.Mapped 61936 -11.1% 55075 ± 2% numa-meminfo.node0.PageTables 77142184 ± 3% +22.9% 94803620 ± 3% numa-meminfo.node1.Active 77142121 ± 3% +22.9% 94803431 ± 3% numa-meminfo.node1.Active(anon) 78351034 ± 3% +24.2% 97283455 ± 3% numa-meminfo.node1.AnonPages 1238824 ± 12% +104.0% 2526732 ± 2% numa-meminfo.node1.Inactive 1238414 ± 12% +104.0% 2526081 ± 2% numa-meminfo.node1.Inactive(anon) 39485 ± 5% +30.0% 51344 ± 7% numa-meminfo.node1.KReclaimable 6764 ± 4% +14.9% 7776 numa-meminfo.node1.KernelStack 13907 ± 13% +68.8% 23470 ± 15% numa-meminfo.node1.Mapped 52452027 ± 5% -36.1% 33505419 ± 9% numa-meminfo.node1.MemFree 79628231 ± 3% +23.8% 98574840 ± 3% numa-meminfo.node1.MemUsed 158507 ± 4% +28.1% 203008 ± 3% numa-meminfo.node1.PageTables 39485 ± 5% +30.0% 51344 ± 7% numa-meminfo.node1.SReclaimable 221.00 ± 15% +131.4% 511.33 ± 31% slabinfo.biovec-128.active_objs 221.00 ± 15% +131.4% 511.33 ± 31% slabinfo.biovec-128.num_objs 82252 ± 2% +12.3% 92376 slabinfo.dentry.active_objs 1993 ± 2% +13.5% 2262 slabinfo.dentry.active_slabs 1993 ± 2% +13.5% 2262 slabinfo.dentry.num_slabs 3972 ± 2% +11.4% 4425 ± 2% slabinfo.files_cache.active_objs 3972 ± 2% +11.4% 4425 ± 2% slabinfo.files_cache.num_objs 22678 ± 5% +14.3% 25913 slabinfo.filp.active_objs 717.25 ± 5% +17.6% 843.33 slabinfo.filp.active_slabs 22968 ± 5% +13.5% 26077 slabinfo.filp.num_objs 717.25 ± 5% +17.6% 843.33 slabinfo.filp.num_slabs 81.25 ± 9% +1136.1% 1004 ± 43% slabinfo.nfs_commit_data.active_objs 81.25 ± 9% +1136.1% 1004 ± 43% slabinfo.nfs_commit_data.num_objs 65.75 ± 19% +941.3% 684.67 ± 53% slabinfo.nfs_read_data.active_objs 65.75 ± 19% +941.3% 684.67 ± 53% slabinfo.nfs_read_data.num_objs 41710 ± 3% +62.0% 67563 slabinfo.radix_tree_node.active_objs 751.75 ± 3% +63.7% 1230 slabinfo.radix_tree_node.active_slabs 42119 ± 3% +63.6% 68894 slabinfo.radix_tree_node.num_objs 751.75 ± 3% +63.7% 1230 slabinfo.radix_tree_node.num_slabs 1587 ± 7% +14.6% 1819 ± 8% slabinfo.skbuff_ext_cache.active_objs 1587 ± 7% +14.7% 1821 ± 8% slabinfo.skbuff_ext_cache.num_objs 357.00 ± 9% +171.5% 969.33 ± 22% slabinfo.skbuff_fclone_cache.active_objs 357.00 ± 9% +171.5% 969.33 ± 22% slabinfo.skbuff_fclone_cache.num_objs 9323 +21.0% 11281 ± 3% slabinfo.vmap_area.active_objs 9339 +20.8% 11282 ± 3% slabinfo.vmap_area.num_objs 257.75 ± 6% -82.5% 45.00 ± 69% numa-vmstat.node0.nr_active_file 41.25 ± 39% +1416.8% 625.67 ± 48% numa-vmstat.node0.nr_dirtied 176027 ± 2% +73.7% 305747 ± 3% numa-vmstat.node0.nr_inactive_anon 194.75 ± 25% -73.6% 51.33 ± 49% numa-vmstat.node0.nr_inactive_file 8540 ± 2% -14.0% 7342 numa-vmstat.node0.nr_kernel_stack 3951 ± 13% +26.0% 4977 ± 8% numa-vmstat.node0.nr_mapped 15365 -11.3% 13635 ± 2% numa-vmstat.node0.nr_page_table_pages 257.75 ± 6% -82.7% 44.67 ± 69% numa-vmstat.node0.nr_zone_active_file 176136 ± 2% +73.7% 305881 ± 3% numa-vmstat.node0.nr_zone_inactive_anon 194.00 ± 25% -73.5% 51.33 ± 49% numa-vmstat.node0.nr_zone_inactive_file 7706557 ± 6% +59.4% 12280502 ± 3% numa-vmstat.node0.numa_foreign 7881976 +15.1% 9074037 ± 2% numa-vmstat.node0.numa_hit 7744537 +15.2% 8920222 ± 2% numa-vmstat.node0.numa_local 227000 ± 52% +85.0% 419979 ± 15% numa-vmstat.node0.numa_miss 364448 ± 32% +57.4% 573797 ± 12% numa-vmstat.node0.numa_other 19143467 ± 3% +22.7% 23496200 ± 3% numa-vmstat.node1.nr_active_anon 19443323 ± 3% +24.0% 24107902 ± 3% numa-vmstat.node1.nr_anon_pages 11.25 ±168% +6092.6% 696.67 ± 23% numa-vmstat.node1.nr_dirtied 32665 ± 3% -54.3% 14934 ± 7% numa-vmstat.node1.nr_free_cma 13257702 ± 4% -35.2% 8589733 ± 10% numa-vmstat.node1.nr_free_pages 307118 ± 12% +102.9% 623237 ± 3% numa-vmstat.node1.nr_inactive_anon 32.75 ± 23% +289.8% 127.67 ± 2% numa-vmstat.node1.nr_isolated_anon 6765 ± 4% +14.9% 7773 numa-vmstat.node1.nr_kernel_stack 3627 ± 14% +61.5% 5858 ± 15% numa-vmstat.node1.nr_mapped 39352 ± 4% +27.7% 50243 ± 3% numa-vmstat.node1.nr_page_table_pages 9945 ± 5% +28.7% 12796 ± 7% numa-vmstat.node1.nr_slab_reclaimable 1441846 ± 21% +56.3% 2254231 ± 9% numa-vmstat.node1.nr_vmscan_write 1441849 ± 21% +56.4% 2254915 ± 9% numa-vmstat.node1.nr_written 19143456 ± 3% +22.7% 23496195 ± 3% numa-vmstat.node1.nr_zone_active_anon 307119 ± 12% +102.9% 623237 ± 3% numa-vmstat.node1.nr_zone_inactive_anon 227115 ± 52% +85.0% 420194 ± 15% numa-vmstat.node1.numa_foreign 17315459 ± 5% +26.1% 21840232 ± 3% numa-vmstat.node1.numa_hit 17256929 ± 5% +26.3% 21794912 ± 3% numa-vmstat.node1.numa_local 7707341 ± 6% +59.3% 12281115 ± 3% numa-vmstat.node1.numa_miss 7765889 ± 6% +58.7% 12326444 ± 3% numa-vmstat.node1.numa_other 1.25 ± 87% +3126.7% 40.33 ±108% numa-vmstat.node1.workingset_nodes 10353 ±123% +219.7% 33104 ± 2% proc-vmstat.compact_daemon_migrate_scanned 1888 ± 49% +325.1% 8028 ± 8% proc-vmstat.compact_fail 10353 ±123% +219.7% 33104 ± 2% proc-vmstat.compact_migrate_scanned 1888 ± 49% +325.2% 8030 ± 8% proc-vmstat.compact_stall 25092321 ± 2% +16.8% 29316619 ± 3% proc-vmstat.nr_active_anon 276.00 ± 3% -66.9% 91.33 ± 11% proc-vmstat.nr_active_file 25568624 ± 2% +18.3% 30244462 ± 3% proc-vmstat.nr_anon_pages 65.00 ± 24% +2683.6% 1809 ± 12% proc-vmstat.nr_dirtied 1449552 ± 3% -32.0% 986067 ± 10% proc-vmstat.nr_dirty_background_threshold 2902650 ± 3% -32.0% 1974547 ± 10% proc-vmstat.nr_dirty_threshold 276121 +2.8% 283935 proc-vmstat.nr_file_pages 32040 ± 4% -54.9% 14439 ± 6% proc-vmstat.nr_free_cma 14635098 ± 3% -31.8% 9975596 ± 10% proc-vmstat.nr_free_pages 490377 ± 6% +92.1% 941776 ± 2% proc-vmstat.nr_inactive_anon 300.25 ± 3% -28.9% 213.33 ± 21% proc-vmstat.nr_inactive_file 55.25 ± 16% +265.6% 202.00 ± 3% proc-vmstat.nr_isolated_anon 15301 -1.1% 15126 proc-vmstat.nr_kernel_stack 7563 +43.4% 10848 ± 4% proc-vmstat.nr_mapped 55169 ± 3% +17.0% 64572 ± 3% proc-vmstat.nr_page_table_pages 20862 +17.7% 24545 proc-vmstat.nr_slab_reclaimable 40904 +3.1% 42163 proc-vmstat.nr_slab_unreclaimable 261601 +3.0% 269429 proc-vmstat.nr_unevictable 2405495 ± 10% +46.5% 3524586 ± 9% proc-vmstat.nr_vmscan_write 25092249 ± 2% +16.8% 29316482 ± 3% proc-vmstat.nr_zone_active_anon 275.75 ± 3% -67.0% 91.00 ± 12% proc-vmstat.nr_zone_active_file 490475 ± 6% +92.0% 941924 ± 2% proc-vmstat.nr_zone_inactive_anon 299.75 ± 2% -28.8% 213.33 ± 21% proc-vmstat.nr_zone_inactive_file 261601 +3.0% 269430 proc-vmstat.nr_zone_unevictable 13602156 ± 4% +7.3% 14596636 ± 3% proc-vmstat.numa_foreign 35948444 -3.5% 34680186 proc-vmstat.numa_hit 35917370 -3.5% 34648547 proc-vmstat.numa_local 13602156 ± 4% +7.3% 14596636 ± 3% proc-vmstat.numa_miss 13633230 ± 4% +7.3% 14628275 ± 3% proc-vmstat.numa_other 5588835 ± 27% +336.2% 24380925 ± 10% proc-vmstat.numa_pte_updates 5204 ± 4% +201.5% 15693 proc-vmstat.pgactivate 49648820 +3.5% 51363583 proc-vmstat.pgfree 2677 ± 4% +63.5% 4378 ± 44% proc-vmstat.pgmajfault 0.25 ±173% +4.8e+05% 1195 ±141% proc-vmstat.pgmigrate_fail 2939659 ± 10% -44.1% 1641880 ± 10% proc-vmstat.pgscan_kswapd 2937571 ± 10% -44.2% 1638222 ± 10% proc-vmstat.pgsteal_kswapd 36379 +26416.6% 9646476 ± 25% proc-vmstat.slabs_scanned 733.25 ± 7% +86.8% 1370 ± 17% proc-vmstat.swap_ra 576.25 ± 8% +64.7% 949.00 ± 15% proc-vmstat.swap_ra_hit 16.00 ± 4% +37393.8% 5999 ± 24% proc-vmstat.unevictable_pgs_culled 6.76 ± 3% -7.4% 6.26 ± 6% perf-stat.i.MPKI 6.754e+09 -73.1% 1.814e+09 ± 7% perf-stat.i.branch-instructions 23049236 -74.2% 5947053 ± 3% perf-stat.i.branch-misses 37.40 ± 3% -11.4 26.01 ± 3% perf-stat.i.cache-miss-rate% 55113848 ± 4% -82.2% 9792343 ± 9% perf-stat.i.cache-misses 1.419e+08 ± 3% -75.4% 34878738 ± 5% perf-stat.i.cache-references 7468 ± 8% -31.1% 5145 ± 2% perf-stat.i.context-switches 1.14 +621.9% 8.23 perf-stat.i.cpi 2.592e+10 +48.0% 3.835e+10 ± 3% perf-stat.i.cpu-cycles 131.22 +10.4% 144.87 perf-stat.i.cpu-migrations 568.39 ± 4% +880.0% 5570 ± 3% perf-stat.i.cycles-between-cache-misses 0.02 ± 12% -0.0 0.01 ± 45% perf-stat.i.dTLB-load-miss-rate% 619825 ± 7% -88.7% 69818 ± 17% perf-stat.i.dTLB-load-misses 6.232e+09 -71.8% 1.755e+09 ± 6% perf-stat.i.dTLB-loads 0.21 ± 2% -0.1 0.08 ± 6% perf-stat.i.dTLB-store-miss-rate% 5684367 -83.6% 929886 ± 11% perf-stat.i.dTLB-store-misses 2.382e+09 -79.1% 4.974e+08 ± 6% perf-stat.i.dTLB-stores 59.54 -9.7 49.81 perf-stat.i.iTLB-load-miss-rate% 4835144 ± 6% -65.4% 1674858 ± 2% perf-stat.i.iTLB-load-misses 2.449e+10 -71.3% 7.019e+09 ± 6% perf-stat.i.instructions 13139 -66.9% 4353 ± 7% perf-stat.i.instructions-per-iTLB-miss 0.98 -77.3% 0.22 ± 6% perf-stat.i.ipc 60.11 ± 5% -84.2% 9.47 ± 6% perf-stat.i.major-faults 0.27 +52.5% 0.41 ± 3% perf-stat.i.metric.GHz 0.19 ± 24% +301.6% 0.74 ± 3% perf-stat.i.metric.K/sec 161.46 -73.1% 43.49 ± 6% perf-stat.i.metric.M/sec 1102490 -83.6% 180444 ± 11% perf-stat.i.minor-faults 77.68 +4.9 82.55 perf-stat.i.node-load-miss-rate% 7350049 ± 5% -82.0% 1325602 ± 8% perf-stat.i.node-load-misses 1863257 ± 4% -84.3% 293219 ± 8% perf-stat.i.node-loads 59.91 ± 7% +10.2 70.12 ± 2% perf-stat.i.node-store-miss-rate% 4561753 ± 7% -82.0% 823146 ± 7% perf-stat.i.node-store-misses 2697430 ± 15% -79.2% 561007 ± 8% perf-stat.i.node-stores 1102550 -83.6% 180454 ± 11% perf-stat.i.page-faults 5.80 ± 3% -11.7% 5.12 ± 2% perf-stat.overall.MPKI 38.81 -11.3 27.49 ± 4% perf-stat.overall.cache-miss-rate% 1.06 +440.0% 5.72 ± 4% perf-stat.overall.cpi 471.06 ± 4% +763.4% 4067 ± 6% perf-stat.overall.cycles-between-cache-misses 0.01 ± 7% -0.0 0.01 ± 44% perf-stat.overall.dTLB-load-miss-rate% 0.24 -0.1 0.18 ± 5% perf-stat.overall.dTLB-store-miss-rate% 73.85 -23.8 50.03 perf-stat.overall.iTLB-load-miss-rate% 5082 ± 7% -22.4% 3945 ± 9% perf-stat.overall.instructions-per-iTLB-miss 0.94 -81.4% 0.18 ± 4% perf-stat.overall.ipc 79.75 +2.1 81.87 perf-stat.overall.node-load-miss-rate% 4953 +88.6% 9340 ± 6% perf-stat.overall.path-length 6.606e+09 -74.7% 1.668e+09 ± 10% perf-stat.ps.branch-instructions 22608775 -75.2% 5605292 ± 5% perf-stat.ps.branch-misses 53930981 ± 4% -83.0% 9162780 ± 11% perf-stat.ps.cache-misses 1.389e+08 ± 3% -76.1% 33217475 ± 7% perf-stat.ps.cache-references 7311 ± 8% -31.2% 5026 perf-stat.ps.context-switches 93863 +1.9% 95670 perf-stat.ps.cpu-clock 2.536e+10 +45.8% 3.697e+10 ± 4% perf-stat.ps.cpu-cycles 128.53 +7.9% 138.70 ± 4% perf-stat.ps.cpu-migrations 607037 ± 7% -86.5% 81773 ± 36% perf-stat.ps.dTLB-load-misses 6.096e+09 -73.3% 1.625e+09 ± 9% perf-stat.ps.dTLB-loads 5558677 -85.3% 817456 ± 15% perf-stat.ps.dTLB-store-misses 2.33e+09 -80.4% 4.572e+08 ± 9% perf-stat.ps.dTLB-stores 4734527 ± 6% -65.2% 1646334 perf-stat.ps.iTLB-load-misses 2.395e+10 -72.9% 6.498e+09 ± 9% perf-stat.ps.instructions 58.86 ± 5% -84.7% 9.02 ± 5% perf-stat.ps.major-faults 1078073 -85.3% 158082 ± 16% perf-stat.ps.minor-faults 7194716 ± 5% -82.5% 1257009 ± 10% perf-stat.ps.node-load-misses 1824407 ± 4% -84.7% 278752 ± 11% perf-stat.ps.node-loads 4460797 ± 7% -83.0% 756410 ± 11% perf-stat.ps.node-store-misses 2640492 ± 15% -80.9% 504536 ± 11% perf-stat.ps.node-stores 1078132 -85.3% 158091 ± 16% perf-stat.ps.page-faults 93863 +1.9% 95670 perf-stat.ps.task-clock 1.09e+12 +85.9% 2.026e+12 ± 4% perf-stat.total.instructions 0.02 ± 81% +1.2e+08% 21164 ± 16% sched_debug.cfs_rq:/.exec_clock.avg 0.92 ± 77% +1.4e+07% 127531 ± 18% sched_debug.cfs_rq:/.exec_clock.max 0.11 ± 76% +1.5e+07% 16164 ± 24% sched_debug.cfs_rq:/.exec_clock.stddev 17284 ± 23% +326.3% 73680 ± 14% sched_debug.cfs_rq:/.load.avg 135741 ± 33% +66.7% 226288 ± 9% sched_debug.cfs_rq:/.load.stddev 1028 +9.7% 1128 ± 4% sched_debug.cfs_rq:/.load_avg.max 14965 ± 17% +1150.1% 187086 ± 13% sched_debug.cfs_rq:/.min_vruntime.avg 31992 ± 13% +1045.6% 366502 ± 10% sched_debug.cfs_rq:/.min_vruntime.max 8981 ± 24% +1017.6% 100380 ± 22% sched_debug.cfs_rq:/.min_vruntime.min 3372 ± 4% +1501.5% 54013 ± 9% sched_debug.cfs_rq:/.min_vruntime.stddev 0.14 ± 7% +38.2% 0.19 ± 5% sched_debug.cfs_rq:/.nr_running.avg 24.12 ± 73% -92.3% 1.87 ± 11% sched_debug.cfs_rq:/.removed.load_avg.avg 1018 -83.2% 171.17 ± 12% sched_debug.cfs_rq:/.removed.load_avg.max 144.98 ± 35% -87.7% 17.78 ± 11% sched_debug.cfs_rq:/.removed.load_avg.stddev 1112 ± 73% -92.2% 86.36 ± 11% sched_debug.cfs_rq:/.removed.runnable_sum.avg 46957 -83.1% 7914 ± 12% sched_debug.cfs_rq:/.removed.runnable_sum.max 6679 ± 35% -87.7% 821.98 ± 11% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 8.16 ± 83% -91.7% 0.68 ± 33% sched_debug.cfs_rq:/.removed.util_avg.avg 395.25 ± 40% -84.1% 62.96 ± 36% sched_debug.cfs_rq:/.removed.util_avg.max 50.76 ± 55% -87.2% 6.51 ± 34% sched_debug.cfs_rq:/.removed.util_avg.stddev 2.20 ± 8% +2562.8% 58.63 ± 18% sched_debug.cfs_rq:/.runnable_load_avg.avg 40.75 ± 68% +2053.4% 877.49 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.max 6.71 ± 27% +2795.2% 194.19 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.stddev 15922 ± 29% +361.8% 73525 ± 14% sched_debug.cfs_rq:/.runnable_weight.avg 134704 ± 34% +67.7% 225909 ± 9% sched_debug.cfs_rq:/.runnable_weight.stddev 2199 ± 64% -3699.3% -79153 sched_debug.cfs_rq:/.spread0.avg 19225 ± 27% +422.8% 100520 ± 49% sched_debug.cfs_rq:/.spread0.max -3785 +4284.8% -165992 sched_debug.cfs_rq:/.spread0.min 3372 ± 4% +1503.5% 54080 ± 9% sched_debug.cfs_rq:/.spread0.stddev 266.83 ± 3% -22.3% 207.34 ± 7% sched_debug.cfs_rq:/.util_avg.avg 1452 ± 17% -23.1% 1116 ± 7% sched_debug.cfs_rq:/.util_avg.max 23.15 ± 24% +351.8% 104.60 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.avg 101.20 ± 18% +158.9% 262.04 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.stddev 723046 +28.8% 931633 sched_debug.cpu.avg_idle.avg 8921 ±136% +3367.2% 309320 ± 6% sched_debug.cpu.avg_idle.min 314848 ± 6% -58.0% 132096 ± 5% sched_debug.cpu.avg_idle.stddev 28760 +533.3% 182149 ± 13% sched_debug.cpu.clock.avg 28764 +533.6% 182260 ± 13% sched_debug.cpu.clock.max 28756 +533.1% 182049 ± 13% sched_debug.cpu.clock.min 2.36 ± 9% +2660.5% 65.27 ± 41% sched_debug.cpu.clock.stddev 28760 +533.3% 182149 ± 13% sched_debug.cpu.clock_task.avg 28764 +533.6% 182260 ± 13% sched_debug.cpu.clock_task.max 28756 +533.1% 182049 ± 13% sched_debug.cpu.clock_task.min 2.36 ± 9% +2660.4% 65.27 ± 41% sched_debug.cpu.clock_task.stddev 230.67 ± 12% +23.0% 283.61 ± 4% sched_debug.cpu.curr->pid.avg 2078 +142.3% 5035 ± 9% sched_debug.cpu.curr->pid.max 619.98 ± 6% +36.5% 846.49 ± 6% sched_debug.cpu.curr->pid.stddev 0.00 ± 5% +173.2% 0.00 ± 31% sched_debug.cpu.next_balance.stddev 1666 +492.9% 9880 ± 11% sched_debug.cpu.nr_switches.avg 8085 ± 22% +577.2% 54755 ± 9% sched_debug.cpu.nr_switches.max 470.25 ± 12% +448.2% 2577 ± 10% sched_debug.cpu.nr_switches.min 1060 ± 8% +813.3% 9687 ± 9% sched_debug.cpu.nr_switches.stddev -58.50 -27.4% -42.47 sched_debug.cpu.nr_uninterruptible.min 0.81 ±100% +1e+06% 8263 ± 14% sched_debug.cpu.sched_count.avg 18.50 ±130% +2.8e+05% 52205 ± 9% sched_debug.cpu.sched_count.max 3.67 ±120% +2.6e+05% 9512 ± 9% sched_debug.cpu.sched_count.stddev 0.41 ± 97% +8.8e+05% 3601 ± 14% sched_debug.cpu.sched_goidle.avg 9.25 ±130% +2.8e+05% 25571 ± 9% sched_debug.cpu.sched_goidle.max 1.84 ±119% +2.6e+05% 4754 ± 9% sched_debug.cpu.sched_goidle.stddev 0.36 ±123% +1.1e+06% 3997 ± 14% sched_debug.cpu.ttwu_count.avg 7.25 ±134% +4.1e+05% 29797 ± 2% sched_debug.cpu.ttwu_count.max 1.44 ±125% +3.5e+05% 5022 ± 6% sched_debug.cpu.ttwu_count.stddev 0.01 ±173% +8.8e+06% 782.74 ± 17% sched_debug.cpu.ttwu_local.avg 0.25 ±173% +8.7e+05% 2176 ± 28% sched_debug.cpu.ttwu_local.max 0.05 ±173% +6.9e+05% 319.05 ± 22% sched_debug.cpu.ttwu_local.stddev 28757 +533.0% 182050 ± 13% sched_debug.cpu_clk 28251 +542.5% 181508 ± 13% sched_debug.ktime 29119 +526.5% 182429 ± 13% sched_debug.sched_clk 36688 ± 39% -60.6% 14451 ± 18% softirqs.CPU0.RCU 12193 ± 4% +305.1% 49394 ± 7% softirqs.CPU0.SCHED 22020 ± 6% +465.9% 124620 ± 12% softirqs.CPU0.TIMER 6532 ± 32% +458.7% 36495 ± 9% softirqs.CPU1.SCHED 22332 ± 7% +445.4% 121803 ± 11% softirqs.CPU1.TIMER 5556 ± 13% +601.2% 38958 ± 9% softirqs.CPU10.SCHED 21976 ± 5% +545.3% 141801 ± 18% softirqs.CPU10.TIMER 6146 ± 21% +513.9% 37735 ± 10% softirqs.CPU11.SCHED 21510 ± 2% +474.1% 123501 ± 11% softirqs.CPU11.TIMER 6910 ± 40% +442.5% 37486 ± 12% softirqs.CPU12.SCHED 21979 ± 3% +460.8% 123261 ± 10% softirqs.CPU12.TIMER 7494 ± 17% +400.1% 37479 ± 10% softirqs.CPU13.SCHED 21998 ± 6% +460.2% 123230 ± 11% softirqs.CPU13.TIMER 7486 ± 31% +69.0% 12649 ± 15% softirqs.CPU14.RCU 7354 ± 21% +415.3% 37902 ± 10% softirqs.CPU14.SCHED 22857 ± 4% +433.4% 121921 ± 10% softirqs.CPU14.TIMER 5572 ± 33% +584.7% 38156 ± 10% softirqs.CPU15.SCHED 22447 ± 4% +447.9% 122977 ± 10% softirqs.CPU15.TIMER 6945 ± 2% +430.3% 36828 ± 12% softirqs.CPU16.SCHED 22646 ± 7% +439.4% 122151 ± 11% softirqs.CPU16.TIMER 7728 ± 26% +74.9% 13518 ± 18% softirqs.CPU17.RCU 6935 +438.4% 37342 ± 13% softirqs.CPU17.SCHED 22983 ± 8% +434.9% 122933 ± 11% softirqs.CPU17.TIMER 6346 ± 11% +508.7% 38628 ± 10% softirqs.CPU18.SCHED 22324 ± 7% +451.8% 123194 ± 11% softirqs.CPU18.TIMER 20405 ± 27% -38.2% 12618 ± 16% softirqs.CPU19.RCU 6280 ± 8% +528.9% 39500 ± 6% softirqs.CPU19.SCHED 22052 ± 5% +453.7% 122092 ± 11% softirqs.CPU19.TIMER 5597 ± 19% +548.2% 36280 ± 7% softirqs.CPU2.SCHED 23466 ± 14% +419.7% 121952 ± 10% softirqs.CPU2.TIMER 38152 ± 29% -62.6% 14267 ± 11% softirqs.CPU20.RCU 8398 ± 35% +342.7% 37176 ± 12% softirqs.CPU20.SCHED 22273 ± 7% +443.2% 120993 ± 11% softirqs.CPU20.TIMER 6796 ± 3% +446.2% 37120 ± 11% softirqs.CPU21.SCHED 22092 ± 7% +453.1% 122183 ± 11% softirqs.CPU21.TIMER 6590 ± 11% +115.2% 14179 ± 22% softirqs.CPU22.RCU 8729 ± 18% +336.0% 38058 ± 11% softirqs.CPU22.SCHED 22540 ± 7% +448.8% 123706 ± 12% softirqs.CPU22.TIMER 6104 ± 17% +517.0% 37661 ± 9% softirqs.CPU23.SCHED 22902 ± 6% +446.1% 125080 ± 12% softirqs.CPU23.TIMER 5850 ± 14% +493.9% 34745 ± 13% softirqs.CPU24.SCHED 22197 +426.9% 116964 ± 10% softirqs.CPU24.TIMER 5649 ± 14% +564.8% 37554 ± 15% softirqs.CPU25.SCHED 22145 ± 5% +441.1% 119822 ± 10% softirqs.CPU25.TIMER 23123 ± 3% +511.5% 141395 ± 9% softirqs.CPU26.TIMER 4401 ± 16% +716.2% 35925 ± 14% softirqs.CPU27.SCHED 22894 ± 5% +414.3% 117743 ± 7% softirqs.CPU27.TIMER 6128 ± 15% +504.5% 37044 ± 11% softirqs.CPU28.SCHED 23322 ± 7% +409.5% 118819 ± 10% softirqs.CPU28.TIMER 6491 ± 10% +473.1% 37203 ± 10% softirqs.CPU29.SCHED 23701 ± 6% +406.3% 120008 ± 10% softirqs.CPU29.TIMER 5690 ± 20% +583.1% 38873 ± 8% softirqs.CPU3.SCHED 22621 ± 6% +441.8% 122560 ± 11% softirqs.CPU3.TIMER 6782 ± 4% +441.5% 36725 ± 15% softirqs.CPU30.SCHED 23426 ± 7% +399.1% 116914 ± 13% softirqs.CPU30.TIMER 6655 ± 4% +449.2% 36548 ± 9% softirqs.CPU31.SCHED 22760 ± 6% +427.1% 119968 ± 7% softirqs.CPU31.TIMER 6456 +498.8% 38661 ± 11% softirqs.CPU32.SCHED 23092 ± 5% +420.0% 120077 ± 10% softirqs.CPU32.TIMER 6304 ± 11% +485.0% 36879 ± 12% softirqs.CPU33.SCHED 22838 ± 6% +420.5% 118880 ± 10% softirqs.CPU33.TIMER 6685 ± 8% +440.3% 36119 ± 11% softirqs.CPU34.SCHED 23305 ± 7% +401.8% 116950 ± 10% softirqs.CPU34.TIMER 6730 ± 5% +471.1% 38439 ± 10% softirqs.CPU35.SCHED 23368 ± 7% +410.9% 119389 ± 9% softirqs.CPU35.TIMER 6157 +98.3% 12210 ± 18% softirqs.CPU36.RCU 7044 ± 4% +435.9% 37749 ± 11% softirqs.CPU36.SCHED 23157 ± 8% +411.8% 118523 ± 9% softirqs.CPU36.TIMER 6393 ± 13% +490.4% 37744 ± 11% softirqs.CPU37.SCHED 22945 ± 9% +415.9% 118379 ± 9% softirqs.CPU37.TIMER 6797 ± 4% +450.3% 37410 ± 11% softirqs.CPU38.SCHED 23095 ± 8% +421.7% 120485 ± 11% softirqs.CPU38.TIMER 6462 ± 4% +494.1% 38397 ± 12% softirqs.CPU39.SCHED 23003 ± 8% +416.3% 118775 ± 9% softirqs.CPU39.TIMER 6658 ± 13% +461.7% 37400 ± 8% softirqs.CPU4.SCHED 22898 ± 6% +436.0% 122724 ± 11% softirqs.CPU4.TIMER 6663 ± 5% +463.6% 37555 ± 11% softirqs.CPU40.SCHED 23428 ± 6% +405.9% 118523 ± 10% softirqs.CPU40.TIMER 23172 ± 7% +479.8% 134363 ± 22% softirqs.CPU41.TIMER 6681 ± 29% +95.0% 13027 ± 19% softirqs.CPU42.RCU 6761 ± 2% +425.8% 35552 ± 6% softirqs.CPU42.SCHED 22874 ± 8% +419.7% 118879 ± 8% softirqs.CPU42.TIMER 7112 ± 3% +422.5% 37157 ± 10% softirqs.CPU43.SCHED 25900 ± 13% +357.4% 118464 ± 9% softirqs.CPU43.TIMER 6067 ± 5% +111.6% 12839 ± 17% softirqs.CPU44.RCU 7078 ± 2% +432.4% 37685 ± 10% softirqs.CPU44.SCHED 23467 ± 7% +409.3% 119517 ± 10% softirqs.CPU44.TIMER 6417 ± 15% +481.3% 37303 ± 12% softirqs.CPU45.SCHED 23074 ± 8% +508.5% 140417 ± 26% softirqs.CPU45.TIMER 7519 ± 26% +65.2% 12423 ± 13% softirqs.CPU46.RCU 6956 ± 2% +437.2% 37373 ± 14% softirqs.CPU46.SCHED 23184 ± 7% +411.5% 118586 ± 9% softirqs.CPU46.TIMER 6439 ± 11% +101.4% 12965 ± 13% softirqs.CPU47.RCU 5486 ± 3% +572.2% 36874 ± 9% softirqs.CPU47.SCHED 23475 ± 6% +407.4% 119116 ± 10% softirqs.CPU47.TIMER 7008 ± 35% +88.7% 13222 ± 16% softirqs.CPU48.RCU 5881 ± 14% +545.4% 37959 ± 12% softirqs.CPU48.SCHED 21852 ± 6% +473.9% 125408 ± 12% softirqs.CPU48.TIMER 6708 ± 6% +471.3% 38329 ± 11% softirqs.CPU49.SCHED 21856 ± 6% +468.8% 124326 ± 12% softirqs.CPU49.TIMER 8618 ± 36% +343.5% 38219 ± 10% softirqs.CPU5.SCHED 24849 ± 12% +395.1% 123026 ± 11% softirqs.CPU5.TIMER 6985 ± 22% +91.7% 13393 ± 17% softirqs.CPU50.RCU 6749 ± 5% +468.1% 38343 ± 9% softirqs.CPU50.SCHED 21851 ± 6% +475.0% 125647 ± 10% softirqs.CPU50.TIMER 6478 ± 11% +502.4% 39026 ± 10% softirqs.CPU51.SCHED 22332 ± 5% +467.5% 126738 ± 11% softirqs.CPU51.TIMER 7628 ± 26% +65.0% 12585 ± 13% softirqs.CPU52.RCU 6744 ± 12% +489.4% 39748 ± 11% softirqs.CPU52.SCHED 22287 ± 6% +465.9% 126113 ± 12% softirqs.CPU52.TIMER 6361 ± 13% +93.1% 12283 ± 12% softirqs.CPU53.RCU 7273 +438.2% 39150 ± 11% softirqs.CPU53.SCHED 22745 ± 5% +457.9% 126889 ± 11% softirqs.CPU53.TIMER 6768 ± 10% +469.3% 38533 ± 11% softirqs.CPU54.SCHED 22198 ± 6% +470.5% 126633 ± 11% softirqs.CPU54.TIMER 6671 ± 8% +93.0% 12879 ± 17% softirqs.CPU55.RCU 6345 ± 15% +492.7% 37606 ± 14% softirqs.CPU55.SCHED 22178 ± 6% +465.2% 125361 ± 12% softirqs.CPU55.TIMER 6098 ± 13% +115.3% 13130 ± 11% softirqs.CPU56.RCU 6372 ± 14% +510.8% 38918 ± 8% softirqs.CPU56.SCHED 22829 ± 6% +447.5% 124995 ± 10% softirqs.CPU56.TIMER 6253 ± 20% +512.4% 38293 ± 9% softirqs.CPU57.SCHED 22136 ± 8% +472.0% 126610 ± 11% softirqs.CPU57.TIMER 6949 ± 31% +72.3% 11971 ± 15% softirqs.CPU58.RCU 6970 +451.0% 38404 ± 11% softirqs.CPU58.SCHED 22076 ± 6% +468.8% 125570 ± 11% softirqs.CPU58.TIMER 6546 ± 23% +103.9% 13346 ± 13% softirqs.CPU59.RCU 7124 +442.9% 38680 ± 12% softirqs.CPU59.SCHED 22292 ± 5% +469.6% 126980 ± 11% softirqs.CPU59.TIMER 6929 ± 6% +452.9% 38316 ± 14% softirqs.CPU6.SCHED 22690 ± 5% +442.9% 123184 ± 11% softirqs.CPU6.TIMER 6101 ± 5% +112.2% 12946 ± 13% softirqs.CPU60.RCU 6947 ± 4% +457.3% 38718 ± 11% softirqs.CPU60.SCHED 22785 ± 9% +451.7% 125714 ± 11% softirqs.CPU60.TIMER 6909 ± 2% +574.0% 46572 ± 25% softirqs.CPU61.SCHED 22245 ± 5% +464.5% 125576 ± 12% softirqs.CPU61.TIMER 6073 ± 15% +545.4% 39196 ± 9% softirqs.CPU62.SCHED 22429 ± 8% +458.0% 125143 ± 9% softirqs.CPU62.TIMER 6085 ± 24% +98.9% 12103 ± 16% softirqs.CPU63.RCU 6672 ± 7% +482.2% 38847 ± 9% softirqs.CPU63.SCHED 22255 ± 5% +469.1% 126660 ± 10% softirqs.CPU63.TIMER 6906 ± 3% +459.5% 38640 ± 11% softirqs.CPU64.SCHED 22023 ± 6% +472.1% 125995 ± 12% softirqs.CPU64.TIMER 6456 ± 5% +503.6% 38969 ± 10% softirqs.CPU65.SCHED 23126 ± 10% +448.6% 126869 ± 11% softirqs.CPU65.TIMER 6727 ± 4% +513.3% 41259 ± 17% softirqs.CPU66.SCHED 22465 ± 5% +467.5% 127482 ± 11% softirqs.CPU66.TIMER 6641 ± 10% +489.5% 39150 ± 12% softirqs.CPU67.SCHED 22547 ± 5% +446.2% 123143 ± 11% softirqs.CPU67.TIMER 7569 ± 25% +68.6% 12762 ± 13% softirqs.CPU68.RCU 6913 ± 3% +475.1% 39758 ± 11% softirqs.CPU68.SCHED 22058 ± 7% +480.0% 127942 ± 12% softirqs.CPU68.TIMER 7077 ± 9% +448.1% 38792 ± 11% softirqs.CPU69.SCHED 23097 ± 9% +444.0% 125637 ± 11% softirqs.CPU69.TIMER 6610 ± 14% +105.6% 13594 ± 18% softirqs.CPU7.RCU 7036 ± 4% +433.5% 37541 ± 11% softirqs.CPU7.SCHED 22654 ± 5% +441.4% 122652 ± 11% softirqs.CPU7.TIMER 6307 ± 8% +95.1% 12305 ± 17% softirqs.CPU70.RCU 7178 ± 4% +447.8% 39319 ± 10% softirqs.CPU70.SCHED 22725 ± 9% +459.8% 127222 ± 11% softirqs.CPU70.TIMER 6902 ± 4% +456.7% 38426 ± 9% softirqs.CPU71.SCHED 23173 ± 6% +439.5% 125013 ± 11% softirqs.CPU71.TIMER 6936 ± 2% +440.3% 37478 ± 10% softirqs.CPU72.SCHED 22718 ± 7% +427.3% 119795 ± 11% softirqs.CPU72.TIMER 6248 ± 14% +492.2% 37002 ± 9% softirqs.CPU73.SCHED 22736 ± 8% +415.1% 117122 ± 9% softirqs.CPU73.TIMER 22828 ± 7% +518.2% 141117 ± 7% softirqs.CPU74.TIMER 5932 ± 13% +552.1% 38681 ± 11% softirqs.CPU75.SCHED 22651 ± 6% +434.1% 120982 ± 10% softirqs.CPU75.TIMER 5989 ± 12% +551.0% 38989 ± 7% softirqs.CPU76.SCHED 22679 ± 4% +514.7% 139409 ± 5% softirqs.CPU76.TIMER 6342 ± 14% +504.5% 38341 ± 11% softirqs.CPU77.SCHED 22680 ± 7% +448.9% 124493 ± 11% softirqs.CPU77.TIMER 6180 ± 16% +541.6% 39656 ± 6% softirqs.CPU78.SCHED 22959 ± 9% +443.7% 124843 ± 9% softirqs.CPU78.TIMER 15836 ± 26% -24.8% 11911 ± 16% softirqs.CPU79.RCU 6431 ± 11% +493.9% 38195 ± 10% softirqs.CPU79.SCHED 22471 ± 8% +436.6% 120572 ± 9% softirqs.CPU79.TIMER 6116 ± 8% +123.0% 13639 ± 8% softirqs.CPU8.RCU 7066 ± 3% +422.6% 36930 ± 8% softirqs.CPU8.SCHED 23222 ± 7% +425.3% 121998 ± 11% softirqs.CPU8.TIMER 5679 ± 7% +102.9% 11523 ± 18% softirqs.CPU80.RCU 6918 ± 3% +470.6% 39480 ± 7% softirqs.CPU80.SCHED 22973 ± 6% +444.5% 125094 ± 10% softirqs.CPU80.TIMER 7395 ± 28% +64.4% 12155 ± 18% softirqs.CPU81.RCU 6876 ± 3% +456.0% 38229 ± 11% softirqs.CPU81.SCHED 23024 ± 6% +428.9% 121786 ± 12% softirqs.CPU81.TIMER 6278 ± 17% +518.7% 38843 ± 10% softirqs.CPU82.SCHED 23005 ± 6% +436.6% 123442 ± 11% softirqs.CPU82.TIMER 7106 ± 20% +66.5% 11833 ± 16% softirqs.CPU83.RCU 6996 ± 4% +456.4% 38930 ± 9% softirqs.CPU83.SCHED 22885 ± 7% +436.0% 122678 ± 9% softirqs.CPU83.TIMER 6469 ± 12% +499.7% 38794 ± 10% softirqs.CPU84.SCHED 22723 ± 8% +437.5% 122129 ± 11% softirqs.CPU84.TIMER 6769 ± 5% +467.8% 38437 ± 10% softirqs.CPU85.SCHED 22753 ± 10% +433.7% 121443 ± 10% softirqs.CPU85.TIMER 6908 +457.7% 38530 ± 10% softirqs.CPU86.SCHED 22687 ± 7% +440.0% 122516 ± 10% softirqs.CPU86.TIMER 6530 ± 6% +482.1% 38008 ± 11% softirqs.CPU87.SCHED 22574 ± 8% +435.7% 120927 ± 10% softirqs.CPU87.TIMER 6739 +471.2% 38491 ± 11% softirqs.CPU88.SCHED 22813 ± 6% +440.3% 123263 ± 10% softirqs.CPU88.TIMER 6861 ± 7% +303.4% 27680 ± 43% softirqs.CPU89.SCHED 22714 ± 7% +504.2% 137243 ± 21% softirqs.CPU89.TIMER 6086 ± 23% +515.1% 37437 ± 9% softirqs.CPU9.SCHED 27118 ± 20% +356.1% 123681 ± 10% softirqs.CPU9.TIMER 6827 ± 22% +68.7% 11520 ± 18% softirqs.CPU90.RCU 6846 ± 2% +448.7% 37560 ± 8% softirqs.CPU90.SCHED 22693 ± 8% +432.5% 120850 ± 9% softirqs.CPU90.TIMER 5636 ± 9% +98.8% 11207 ± 21% softirqs.CPU91.RCU 6865 ± 3% +454.3% 38054 ± 10% softirqs.CPU91.SCHED 22663 ± 7% +429.1% 119908 ± 10% softirqs.CPU91.TIMER 5713 ± 5% +111.0% 12055 ± 18% softirqs.CPU92.RCU 7080 ± 4% +436.3% 37976 ± 12% softirqs.CPU92.SCHED 23117 ± 6% +434.7% 123599 ± 10% softirqs.CPU92.TIMER 6754 ± 2% +446.7% 36928 ± 8% softirqs.CPU93.SCHED 22554 ± 8% +437.1% 121143 ± 10% softirqs.CPU93.TIMER 6007 ± 19% +86.4% 11197 ± 18% softirqs.CPU94.RCU 6982 +454.1% 38688 ± 10% softirqs.CPU94.SCHED 22658 ± 7% +441.2% 122629 ± 10% softirqs.CPU94.TIMER 6128 ± 18% +97.3% 12093 ± 24% softirqs.CPU95.RCU 6669 ± 6% +471.8% 38136 ± 8% softirqs.CPU95.SCHED 22538 ± 7% +442.8% 122343 ± 10% softirqs.CPU95.TIMER 641599 ± 2% +462.4% 3608346 ± 10% softirqs.SCHED 2187113 ± 5% +443.2% 11881220 ± 10% softirqs.TIMER 0.50 ±173% +40633.3% 203.67 ± 78% interrupts.113:PCI-MSI.31981646-edge.i40e-eth0-TxRx-77 0.00 +1.9e+104% 190.67 ± 99% interrupts.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78 0.00 +1.8e+104% 183.00 ± 63% interrupts.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79 0.00 +7.1e+103% 71.33 ± 62% interrupts.117:PCI-MSI.31981650-edge.i40e-eth0-TxRx-81 0.50 ±173% +10766.7% 54.33 ± 85% interrupts.120:PCI-MSI.31981653-edge.i40e-eth0-TxRx-84 36.75 ± 8% +672.8% 284.00 interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc 28331569 ± 11% -48.9% 14483923 ± 7% interrupts.CAL:Function_call_interrupts 1597643 ± 37% -85.7% 228000 ± 5% interrupts.CPU0.CAL:Function_call_interrupts 91505 ± 2% +617.7% 656759 ± 10% interrupts.CPU0.LOC:Local_timer_interrupts 3990 ± 53% +271.5% 14822 ± 16% interrupts.CPU0.RES:Rescheduling_interrupts 3293773 ± 36% -76.5% 775632 ± 5% interrupts.CPU0.TLB:TLB_shootdowns 91444 ± 2% +618.8% 657273 ± 10% interrupts.CPU1.LOC:Local_timer_interrupts 2970 ± 53% -65.0% 1041 ± 25% interrupts.CPU1.RES:Rescheduling_interrupts 91427 ± 2% +619.1% 657486 ± 10% interrupts.CPU10.LOC:Local_timer_interrupts 91397 ± 2% +618.7% 656886 ± 10% interrupts.CPU11.LOC:Local_timer_interrupts 91424 ± 2% +618.1% 656560 ± 10% interrupts.CPU12.LOC:Local_timer_interrupts 91436 ± 2% +619.4% 657799 ± 10% interrupts.CPU13.LOC:Local_timer_interrupts 91438 ± 2% +619.9% 658306 ± 10% interrupts.CPU14.LOC:Local_timer_interrupts 295.00 ± 60% +360.9% 1359 ± 42% interrupts.CPU14.NMI:Non-maskable_interrupts 295.00 ± 60% +360.9% 1359 ± 42% interrupts.CPU14.PMI:Performance_monitoring_interrupts 91423 ± 2% +619.5% 657765 ± 10% interrupts.CPU15.LOC:Local_timer_interrupts 91473 ± 2% +616.9% 655776 ± 10% interrupts.CPU16.LOC:Local_timer_interrupts 91420 ± 2% +617.9% 656334 ± 9% interrupts.CPU17.LOC:Local_timer_interrupts 579.50 ±104% +176.3% 1601 ± 32% interrupts.CPU17.NMI:Non-maskable_interrupts 579.50 ±104% +176.3% 1601 ± 32% interrupts.CPU17.PMI:Performance_monitoring_interrupts 125.00 ±126% +515.7% 769.67 ± 22% interrupts.CPU17.RES:Rescheduling_interrupts 91432 ± 2% +618.8% 657187 ± 10% interrupts.CPU18.LOC:Local_timer_interrupts 91348 ± 2% +620.3% 658014 ± 10% interrupts.CPU19.LOC:Local_timer_interrupts 91442 ± 2% +617.7% 656305 ± 10% interrupts.CPU2.LOC:Local_timer_interrupts 1607994 ± 30% -88.3% 187593 ± 16% interrupts.CPU20.CAL:Function_call_interrupts 91411 ± 2% +606.9% 646203 ± 10% interrupts.CPU20.LOC:Local_timer_interrupts 2901 ± 19% -77.1% 665.33 ± 58% interrupts.CPU20.NMI:Non-maskable_interrupts 2901 ± 19% -77.1% 665.33 ± 58% interrupts.CPU20.PMI:Performance_monitoring_interrupts 7329 ± 93% -84.2% 1159 ± 67% interrupts.CPU20.RES:Rescheduling_interrupts 3241724 ± 32% -80.3% 640070 ± 13% interrupts.CPU20.TLB:TLB_shootdowns 91406 ± 2% +619.5% 657687 ± 10% interrupts.CPU21.LOC:Local_timer_interrupts 34527 ± 99% +368.6% 161789 ± 19% interrupts.CPU22.CAL:Function_call_interrupts 91424 ± 2% +619.3% 657647 ± 10% interrupts.CPU22.LOC:Local_timer_interrupts 172.75 ± 57% +799.2% 1553 ± 30% interrupts.CPU22.NMI:Non-maskable_interrupts 172.75 ± 57% +799.2% 1553 ± 30% interrupts.CPU22.PMI:Performance_monitoring_interrupts 67878 ±101% +672.9% 524610 ± 19% interrupts.CPU22.TLB:TLB_shootdowns 91444 ± 2% +619.1% 657560 ± 10% interrupts.CPU23.LOC:Local_timer_interrupts 91322 ± 2% +616.4% 654231 ± 10% interrupts.CPU24.LOC:Local_timer_interrupts 91360 ± 2% +609.2% 647901 ± 8% interrupts.CPU25.LOC:Local_timer_interrupts 2895 ± 91% -63.4% 1059 ±100% interrupts.CPU25.NMI:Non-maskable_interrupts 2895 ± 91% -63.4% 1059 ±100% interrupts.CPU25.PMI:Performance_monitoring_interrupts 91300 ± 2% +624.3% 661277 ± 9% interrupts.CPU26.LOC:Local_timer_interrupts 482.00 ± 80% +350.6% 2171 ± 61% interrupts.CPU26.NMI:Non-maskable_interrupts 482.00 ± 80% +350.6% 2171 ± 61% interrupts.CPU26.PMI:Performance_monitoring_interrupts 91312 ± 2% +604.8% 643563 ± 10% interrupts.CPU27.LOC:Local_timer_interrupts 5890 ±116% +2954.1% 179892 ± 20% interrupts.CPU28.CAL:Function_call_interrupts 91293 ± 2% +620.6% 657852 ± 10% interrupts.CPU28.LOC:Local_timer_interrupts 10660 ±122% +5430.5% 589551 ± 20% interrupts.CPU28.TLB:TLB_shootdowns 91201 ± 2% +619.7% 656371 ± 10% interrupts.CPU29.LOC:Local_timer_interrupts 91435 ± 2% +618.6% 657029 ± 10% interrupts.CPU3.LOC:Local_timer_interrupts 91315 ± 2% +599.6% 638802 ± 14% interrupts.CPU30.LOC:Local_timer_interrupts 91237 ± 2% +620.9% 657775 ± 10% interrupts.CPU31.LOC:Local_timer_interrupts 91320 ± 2% +620.7% 658106 ± 10% interrupts.CPU32.LOC:Local_timer_interrupts 91328 ± 2% +620.2% 657720 ± 10% interrupts.CPU33.LOC:Local_timer_interrupts 91329 ± 2% +604.7% 643577 ± 11% interrupts.CPU34.LOC:Local_timer_interrupts 489.50 ± 78% +304.9% 1982 ± 62% interrupts.CPU34.NMI:Non-maskable_interrupts 489.50 ± 78% +304.9% 1982 ± 62% interrupts.CPU34.PMI:Performance_monitoring_interrupts 91330 ± 2% +620.5% 658042 ± 10% interrupts.CPU35.LOC:Local_timer_interrupts 257.50 ± 34% +411.6% 1317 ± 53% interrupts.CPU35.NMI:Non-maskable_interrupts 257.50 ± 34% +411.6% 1317 ± 53% interrupts.CPU35.PMI:Performance_monitoring_interrupts 11062 ±166% +1103.0% 133081 ± 18% interrupts.CPU36.CAL:Function_call_interrupts 91325 ± 2% +620.2% 657759 ± 10% interrupts.CPU36.LOC:Local_timer_interrupts 73.25 ±101% +1184.2% 940.67 ± 65% interrupts.CPU36.RES:Rescheduling_interrupts 21201 ±173% +1910.5% 426249 ± 18% interrupts.CPU36.TLB:TLB_shootdowns 91322 ± 2% +621.6% 658973 ± 10% interrupts.CPU37.LOC:Local_timer_interrupts 91317 ± 2% +619.8% 657278 ± 10% interrupts.CPU38.LOC:Local_timer_interrupts 91311 ± 2% +620.3% 657721 ± 10% interrupts.CPU39.LOC:Local_timer_interrupts 91452 ± 2% +618.2% 656807 ± 10% interrupts.CPU4.LOC:Local_timer_interrupts 91287 ± 2% +619.9% 657174 ± 10% interrupts.CPU40.LOC:Local_timer_interrupts 323.50 ±100% +207.6% 995.00 ± 48% interrupts.CPU40.NMI:Non-maskable_interrupts 323.50 ±100% +207.6% 995.00 ± 48% interrupts.CPU40.PMI:Performance_monitoring_interrupts 115822 ±173% +373.0% 547848 ± 11% interrupts.CPU40.TLB:TLB_shootdowns 91314 ± 2% +621.4% 658707 ± 10% interrupts.CPU41.LOC:Local_timer_interrupts 91323 ± 2% +618.7% 656381 ± 10% interrupts.CPU42.LOC:Local_timer_interrupts 119.75 ± 21% +1207.2% 1565 ± 26% interrupts.CPU42.NMI:Non-maskable_interrupts 119.75 ± 21% +1207.2% 1565 ± 26% interrupts.CPU42.PMI:Performance_monitoring_interrupts 76.50 ±139% +2120.9% 1699 ± 81% interrupts.CPU42.RES:Rescheduling_interrupts 135052 ±159% +286.8% 522441 ± 23% interrupts.CPU42.TLB:TLB_shootdowns 91315 ± 2% +619.8% 657282 ± 10% interrupts.CPU43.LOC:Local_timer_interrupts 308.00 ± 53% +263.3% 1119 ± 28% interrupts.CPU43.NMI:Non-maskable_interrupts 308.00 ± 53% +263.3% 1119 ± 28% interrupts.CPU43.PMI:Performance_monitoring_interrupts 21303 ±160% +635.8% 156751 ± 7% interrupts.CPU44.CAL:Function_call_interrupts 91333 ± 2% +620.0% 657591 ± 10% interrupts.CPU44.LOC:Local_timer_interrupts 132.00 ± 19% +775.0% 1155 ± 17% interrupts.CPU44.NMI:Non-maskable_interrupts 132.00 ± 19% +775.0% 1155 ± 17% interrupts.CPU44.PMI:Performance_monitoring_interrupts 36.25 ± 97% +4102.3% 1523 ± 81% interrupts.CPU44.RES:Rescheduling_interrupts 35668 ±160% +1294.0% 497211 ± 9% interrupts.CPU44.TLB:TLB_shootdowns 91324 ± 2% +619.1% 656671 ± 10% interrupts.CPU45.LOC:Local_timer_interrupts 171.25 ± 47% +1160.5% 2158 ± 43% interrupts.CPU45.NMI:Non-maskable_interrupts 171.25 ± 47% +1160.5% 2158 ± 43% interrupts.CPU45.PMI:Performance_monitoring_interrupts 91319 ± 2% +620.5% 657981 ± 10% interrupts.CPU46.LOC:Local_timer_interrupts 259.25 ± 74% +386.8% 1262 ± 74% interrupts.CPU46.NMI:Non-maskable_interrupts 259.25 ± 74% +386.8% 1262 ± 74% interrupts.CPU46.PMI:Performance_monitoring_interrupts 152.25 ± 93% +605.2% 1073 ± 73% interrupts.CPU46.RES:Rescheduling_interrupts 179949 ±102% +138.9% 429973 ± 16% interrupts.CPU46.TLB:TLB_shootdowns 22158 ±103% +636.1% 163103 ± 15% interrupts.CPU47.CAL:Function_call_interrupts 91316 ± 2% +619.3% 656852 ± 10% interrupts.CPU47.LOC:Local_timer_interrupts 258.00 ± 41% +491.0% 1524 ± 14% interrupts.CPU47.NMI:Non-maskable_interrupts 258.00 ± 41% +491.0% 1524 ± 14% interrupts.CPU47.PMI:Performance_monitoring_interrupts 110.25 ±144% +501.7% 663.33 ± 14% interrupts.CPU47.RES:Rescheduling_interrupts 44623 ±106% +1072.9% 523397 ± 15% interrupts.CPU47.TLB:TLB_shootdowns 91426 ± 2% +618.7% 657099 ± 10% interrupts.CPU48.LOC:Local_timer_interrupts 91442 ± 2% +619.4% 657810 ± 10% interrupts.CPU49.LOC:Local_timer_interrupts 91475 ± 2% +619.8% 658437 ± 10% interrupts.CPU5.LOC:Local_timer_interrupts 91447 ± 2% +619.1% 657562 ± 10% interrupts.CPU50.LOC:Local_timer_interrupts 133055 ± 87% +274.6% 498477 ± 13% interrupts.CPU50.TLB:TLB_shootdowns 91446 ± 2% +619.6% 658011 ± 10% interrupts.CPU51.LOC:Local_timer_interrupts 91442 ± 2% +619.5% 657910 ± 10% interrupts.CPU52.LOC:Local_timer_interrupts 33202 ±137% +230.0% 109566 ± 5% interrupts.CPU53.CAL:Function_call_interrupts 91470 ± 2% +619.8% 658431 ± 10% interrupts.CPU53.LOC:Local_timer_interrupts 381.00 ± 73% +272.5% 1419 ± 27% interrupts.CPU53.NMI:Non-maskable_interrupts 381.00 ± 73% +272.5% 1419 ± 27% interrupts.CPU53.PMI:Performance_monitoring_interrupts 110.25 ±111% +383.7% 533.33 ± 2% interrupts.CPU53.RES:Rescheduling_interrupts 62130 ±139% +481.0% 361006 ± 6% interrupts.CPU53.TLB:TLB_shootdowns 7571 ±117% +1548.1% 124780 ± 24% interrupts.CPU54.CAL:Function_call_interrupts 91450 ± 2% +620.1% 658506 ± 10% interrupts.CPU54.LOC:Local_timer_interrupts 11528 ±117% +3376.7% 400807 ± 22% interrupts.CPU54.TLB:TLB_shootdowns 50740 ± 62% +228.9% 166883 ± 26% interrupts.CPU55.CAL:Function_call_interrupts 91421 ± 2% +620.2% 658453 ± 10% interrupts.CPU55.LOC:Local_timer_interrupts 270.75 ± 48% +372.4% 1279 ± 24% interrupts.CPU55.NMI:Non-maskable_interrupts 270.75 ± 48% +372.4% 1279 ± 24% interrupts.CPU55.PMI:Performance_monitoring_interrupts 180.75 ± 73% +267.5% 664.33 ± 6% interrupts.CPU55.RES:Rescheduling_interrupts 98857 ± 63% +440.1% 533878 ± 27% interrupts.CPU55.TLB:TLB_shootdowns 2726 ±145% +4959.8% 137968 ± 18% interrupts.CPU56.CAL:Function_call_interrupts 91596 ± 2% +618.4% 658045 ± 10% interrupts.CPU56.LOC:Local_timer_interrupts 162.50 ± 48% +668.2% 1248 ± 57% interrupts.CPU56.NMI:Non-maskable_interrupts 162.50 ± 48% +668.2% 1248 ± 57% interrupts.CPU56.PMI:Performance_monitoring_interrupts 30.75 ±146% +4917.9% 1543 ± 77% interrupts.CPU56.RES:Rescheduling_interrupts 4435 ±173% +10216.4% 457608 ± 20% interrupts.CPU56.TLB:TLB_shootdowns 37241 ±171% +310.1% 152741 ± 4% interrupts.CPU57.CAL:Function_call_interrupts 91394 ± 2% +618.1% 656291 ± 10% interrupts.CPU57.LOC:Local_timer_interrupts 70192 ±173% +624.1% 508288 ± 6% interrupts.CPU57.TLB:TLB_shootdowns 91415 ± 2% +619.3% 657592 ± 10% interrupts.CPU58.LOC:Local_timer_interrupts 91463 ± 2% +619.9% 658458 ± 10% interrupts.CPU59.LOC:Local_timer_interrupts 613.25 ± 85% +149.3% 1528 ± 24% interrupts.CPU59.NMI:Non-maskable_interrupts 613.25 ± 85% +149.3% 1528 ± 24% interrupts.CPU59.PMI:Performance_monitoring_interrupts 65.50 ±126% +865.9% 632.67 ± 30% interrupts.CPU59.RES:Rescheduling_interrupts 94936 ±173% +411.2% 485333 ± 31% interrupts.CPU59.TLB:TLB_shootdowns 41173 ±106% +324.2% 174664 ± 15% interrupts.CPU6.CAL:Function_call_interrupts 91443 ± 2% +619.7% 658131 ± 10% interrupts.CPU6.LOC:Local_timer_interrupts 84022 ±105% +596.4% 585147 ± 13% interrupts.CPU6.TLB:TLB_shootdowns 13559 ±163% +1052.6% 156280 ± 25% interrupts.CPU60.CAL:Function_call_interrupts 91424 ± 2% +619.8% 658027 ± 10% interrupts.CPU60.LOC:Local_timer_interrupts 82.50 ±103% +508.1% 501.67 ± 24% interrupts.CPU60.RES:Rescheduling_interrupts 24650 ±168% +1992.2% 515724 ± 26% interrupts.CPU60.TLB:TLB_shootdowns 91451 ± 2% +620.1% 658530 ± 10% interrupts.CPU61.LOC:Local_timer_interrupts 91276 ± 2% +622.2% 659231 ± 10% interrupts.CPU62.LOC:Local_timer_interrupts 91566 +619.4% 658761 ± 10% interrupts.CPU63.LOC:Local_timer_interrupts 79456 ±169% +433.2% 423698 ± 31% interrupts.CPU63.TLB:TLB_shootdowns 91248 +621.0% 657891 ± 10% interrupts.CPU64.LOC:Local_timer_interrupts 694.75 ± 52% +198.5% 2074 ± 16% interrupts.CPU64.NMI:Non-maskable_interrupts 694.75 ± 52% +198.5% 2074 ± 16% interrupts.CPU64.PMI:Performance_monitoring_interrupts 91417 ± 2% +619.6% 657844 ± 9% interrupts.CPU65.LOC:Local_timer_interrupts 329.00 ± 61% +273.0% 1227 ± 33% interrupts.CPU65.NMI:Non-maskable_interrupts 329.00 ± 61% +273.0% 1227 ± 33% interrupts.CPU65.PMI:Performance_monitoring_interrupts 23082 ± 67% +556.6% 151549 ± 41% interrupts.CPU66.CAL:Function_call_interrupts 91407 ± 2% +618.5% 656754 ± 10% interrupts.CPU66.LOC:Local_timer_interrupts 41149 ± 63% +1078.4% 484899 ± 39% interrupts.CPU66.TLB:TLB_shootdowns 91433 ± 2% +620.3% 658575 ± 10% interrupts.CPU67.LOC:Local_timer_interrupts 91411 ± 2% +620.5% 658627 ± 10% interrupts.CPU68.LOC:Local_timer_interrupts 91423 ± 2% +620.4% 658572 ± 10% interrupts.CPU69.LOC:Local_timer_interrupts 36.75 ± 8% +672.8% 284.00 interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc 659.50 ± 57% +22709.4% 150428 ± 33% interrupts.CPU7.CAL:Function_call_interrupts 91406 ± 2% +619.9% 658001 ± 10% interrupts.CPU7.LOC:Local_timer_interrupts 209.75 ± 30% +598.1% 1464 ± 37% interrupts.CPU7.NMI:Non-maskable_interrupts 209.75 ± 30% +598.1% 1464 ± 37% interrupts.CPU7.PMI:Performance_monitoring_interrupts 69.50 ± 57% +833.3% 648.67 ± 19% interrupts.CPU7.RES:Rescheduling_interrupts 480.00 ±170% +1e+05% 492449 ± 33% interrupts.CPU7.TLB:TLB_shootdowns 27310 ±112% +400.2% 136617 ± 15% interrupts.CPU70.CAL:Function_call_interrupts 91446 ± 2% +620.6% 658978 ± 10% interrupts.CPU70.LOC:Local_timer_interrupts 185.50 ± 15% +906.1% 1866 ± 19% interrupts.CPU70.NMI:Non-maskable_interrupts 185.50 ± 15% +906.1% 1866 ± 19% interrupts.CPU70.PMI:Performance_monitoring_interrupts 33.75 ± 82% +1382.5% 500.33 ± 18% interrupts.CPU70.RES:Rescheduling_interrupts 48996 ±120% +794.6% 438323 ± 15% interrupts.CPU70.TLB:TLB_shootdowns 91429 ± 2% +619.0% 657394 ± 10% interrupts.CPU71.LOC:Local_timer_interrupts 91329 ± 2% +618.8% 656467 ± 10% interrupts.CPU72.LOC:Local_timer_interrupts 1388 ± 28% +109.1% 2902 ± 33% interrupts.CPU72.NMI:Non-maskable_interrupts 1388 ± 28% +109.1% 2902 ± 33% interrupts.CPU72.PMI:Performance_monitoring_interrupts 91296 ± 2% +587.7% 627805 ± 6% interrupts.CPU73.LOC:Local_timer_interrupts 9516 ± 98% +629.3% 69405 ± 49% interrupts.CPU74.CAL:Function_call_interrupts 91352 ± 2% +623.3% 660713 ± 10% interrupts.CPU74.LOC:Local_timer_interrupts 353.25 ± 81% +350.4% 1591 ± 53% interrupts.CPU74.NMI:Non-maskable_interrupts 353.25 ± 81% +350.4% 1591 ± 53% interrupts.CPU74.PMI:Performance_monitoring_interrupts 79.25 ±148% +292.0% 310.67 ± 65% interrupts.CPU74.RES:Rescheduling_interrupts 14031 ± 98% +1540.9% 230243 ± 54% interrupts.CPU74.TLB:TLB_shootdowns 91317 ± 2% +618.0% 655625 ± 10% interrupts.CPU75.LOC:Local_timer_interrupts 91334 ± 2% +621.1% 658628 ± 10% interrupts.CPU76.LOC:Local_timer_interrupts 0.25 ±173% +81366.7% 203.67 ± 78% interrupts.CPU77.113:PCI-MSI.31981646-edge.i40e-eth0-TxRx-77 91337 ± 2% +619.9% 657559 ± 10% interrupts.CPU77.LOC:Local_timer_interrupts 0.00 +1.9e+104% 190.67 ± 99% interrupts.CPU78.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78 91338 ± 2% +620.5% 658069 ± 10% interrupts.CPU78.LOC:Local_timer_interrupts 0.00 +1.8e+104% 182.67 ± 63% interrupts.CPU79.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79 531436 ± 49% -73.6% 140073 ± 10% interrupts.CPU79.CAL:Function_call_interrupts 91338 ± 2% +620.6% 658180 ± 10% interrupts.CPU79.LOC:Local_timer_interrupts 1279 ± 57% -61.4% 494.00 ± 5% interrupts.CPU79.RES:Rescheduling_interrupts 1024465 ± 49% -56.6% 444652 ± 9% interrupts.CPU79.TLB:TLB_shootdowns 438.25 +37483.9% 164711 ± 17% interrupts.CPU8.CAL:Function_call_interrupts 91423 ± 2% +618.9% 657275 ± 10% interrupts.CPU8.LOC:Local_timer_interrupts 150.00 ± 12% +1021.3% 1682 ± 63% interrupts.CPU8.NMI:Non-maskable_interrupts 150.00 ± 12% +1021.3% 1682 ± 63% interrupts.CPU8.PMI:Performance_monitoring_interrupts 1.50 ±110% +3.8e+07% 565425 ± 17% interrupts.CPU8.TLB:TLB_shootdowns 8744 ± 98% +1181.2% 112026 ± 25% interrupts.CPU80.CAL:Function_call_interrupts 91335 ± 2% +621.2% 658668 ± 10% interrupts.CPU80.LOC:Local_timer_interrupts 70.75 ± 70% +546.9% 457.67 ± 11% interrupts.CPU80.RES:Rescheduling_interrupts 16253 ±104% +2070.0% 352696 ± 26% interrupts.CPU80.TLB:TLB_shootdowns 0.00 +7.1e+103% 71.00 ± 62% interrupts.CPU81.117:PCI-MSI.31981650-edge.i40e-eth0-TxRx-81 91337 ± 2% +620.6% 658169 ± 10% interrupts.CPU81.LOC:Local_timer_interrupts 193.75 ± 95% +234.5% 648.00 ± 16% interrupts.CPU81.RES:Rescheduling_interrupts 91342 ± 2% +616.5% 654464 ± 10% interrupts.CPU82.LOC:Local_timer_interrupts 180.00 ±107% +244.1% 619.33 ± 16% interrupts.CPU82.RES:Rescheduling_interrupts 91344 ± 2% +621.8% 659347 ± 10% interrupts.CPU83.LOC:Local_timer_interrupts 350.00 ± 75% +222.7% 1129 ± 19% interrupts.CPU83.NMI:Non-maskable_interrupts 350.00 ± 75% +222.7% 1129 ± 19% interrupts.CPU83.PMI:Performance_monitoring_interrupts 183.25 ±120% +486.6% 1075 ± 68% interrupts.CPU83.RES:Rescheduling_interrupts 0.25 ±173% +21233.3% 53.33 ± 87% interrupts.CPU84.120:PCI-MSI.31981653-edge.i40e-eth0-TxRx-84 91331 ± 2% +621.6% 659054 ± 10% interrupts.CPU84.LOC:Local_timer_interrupts 91309 ± 2% +621.7% 658964 ± 10% interrupts.CPU85.LOC:Local_timer_interrupts 91327 ± 2% +621.7% 659107 ± 10% interrupts.CPU86.LOC:Local_timer_interrupts 122.75 ± 68% +302.4% 494.00 ± 16% interrupts.CPU86.RES:Rescheduling_interrupts 464.75 +25290.8% 118003 ± 31% interrupts.CPU87.CAL:Function_call_interrupts 91350 ± 2% +620.7% 658353 ± 10% interrupts.CPU87.LOC:Local_timer_interrupts 3.75 ± 60% +1e+07% 376468 ± 32% interrupts.CPU87.TLB:TLB_shootdowns 510.75 ± 15% +29935.6% 153406 ± 11% interrupts.CPU88.CAL:Function_call_interrupts 91333 ± 2% +620.8% 658330 ± 10% interrupts.CPU88.LOC:Local_timer_interrupts 109.00 ±169% +4.4e+05% 481712 ± 8% interrupts.CPU88.TLB:TLB_shootdowns 91343 ± 2% +621.3% 658818 ± 10% interrupts.CPU89.LOC:Local_timer_interrupts 91400 ± 2% +618.3% 656547 ± 10% interrupts.CPU9.LOC:Local_timer_interrupts 20036 ± 84% +583.1% 136879 ± 3% interrupts.CPU90.CAL:Function_call_interrupts 91525 ± 2% +619.4% 658431 ± 10% interrupts.CPU90.LOC:Local_timer_interrupts 221.25 ± 60% +440.4% 1195 ± 25% interrupts.CPU90.NMI:Non-maskable_interrupts 221.25 ± 60% +440.4% 1195 ± 25% interrupts.CPU90.PMI:Performance_monitoring_interrupts 34753 ± 82% +1197.6% 450948 ± 3% interrupts.CPU90.TLB:TLB_shootdowns 20216 ±100% +513.0% 123921 ± 30% interrupts.CPU91.CAL:Function_call_interrupts 91058 ± 2% +622.4% 657853 ± 10% interrupts.CPU91.LOC:Local_timer_interrupts 266.00 ± 28% +289.2% 1035 ± 35% interrupts.CPU91.NMI:Non-maskable_interrupts 266.00 ± 28% +289.2% 1035 ± 35% interrupts.CPU91.PMI:Performance_monitoring_interrupts 78.25 ± 89% +666.8% 600.00 ± 14% interrupts.CPU91.RES:Rescheduling_interrupts 40088 ±103% +923.6% 410355 ± 30% interrupts.CPU91.TLB:TLB_shootdowns 14262 ±168% +809.1% 129653 ± 17% interrupts.CPU92.CAL:Function_call_interrupts 91323 ± 2% +621.5% 658884 ± 10% interrupts.CPU92.LOC:Local_timer_interrupts 168.25 ± 22% +541.9% 1080 ± 78% interrupts.CPU92.NMI:Non-maskable_interrupts 168.25 ± 22% +541.9% 1080 ± 78% interrupts.CPU92.PMI:Performance_monitoring_interrupts 22.00 ± 98% +2825.8% 643.67 ± 10% interrupts.CPU92.RES:Rescheduling_interrupts 26830 ±173% +1461.8% 419031 ± 14% interrupts.CPU92.TLB:TLB_shootdowns 91329 ± 2% +621.3% 658747 ± 10% interrupts.CPU93.LOC:Local_timer_interrupts 171.75 ± 18% +1043.3% 1963 ± 25% interrupts.CPU93.NMI:Non-maskable_interrupts 171.75 ± 18% +1043.3% 1963 ± 25% interrupts.CPU93.PMI:Performance_monitoring_interrupts 91336 ± 2% +621.9% 659365 ± 10% interrupts.CPU94.LOC:Local_timer_interrupts 190.25 ± 44% +329.1% 816.33 ± 86% interrupts.CPU94.NMI:Non-maskable_interrupts 190.25 ± 44% +329.1% 816.33 ± 86% interrupts.CPU94.PMI:Performance_monitoring_interrupts 46.75 ±165% +1341.0% 673.67 ± 35% interrupts.CPU94.RES:Rescheduling_interrupts 70182 ±173% +390.0% 343907 ± 20% interrupts.CPU94.TLB:TLB_shootdowns 91411 ± 2% +619.9% 658036 ± 10% interrupts.CPU95.LOC:Local_timer_interrupts 219.50 ± 25% +614.5% 1568 ± 9% interrupts.CPU95.NMI:Non-maskable_interrupts 219.50 ± 25% +614.5% 1568 ± 9% interrupts.CPU95.PMI:Performance_monitoring_interrupts 131.75 ±160% +745.0% 1113 ± 46% interrupts.CPU95.RES:Rescheduling_interrupts 64224 ±172% +623.4% 464571 ± 47% interrupts.CPU95.TLB:TLB_shootdowns 8772137 ± 2% +618.8% 63050381 ± 10% interrupts.LOC:Local_timer_interrupts 115013 ± 5% +25.5% 144352 ± 2% interrupts.NMI:Non-maskable_interrupts 115013 ± 5% +25.5% 144352 ± 2% interrupts.PMI:Performance_monitoring_interrupts 45.16 ± 7% -7.3 37.81 perf-profile.calltrace.cycles-pp.do_access 36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 36.67 ± 16% -6.0 30.63 perf-profile.calltrace.cycles-pp.secondary_startup_64 5.66 ± 10% -5.7 0.00 perf-profile.calltrace.cycles-pp.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node 33.76 ± 18% -5.3 28.51 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 33.39 ± 18% -5.2 28.21 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 4.78 ± 10% -4.8 0.00 perf-profile.calltrace.cycles-pp.__swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec 5.43 ± 8% -3.7 1.71 ± 11% perf-profile.calltrace.cycles-pp.do_rw_once 1.83 ± 9% -1.1 0.75 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault 2.42 ± 8% -0.9 1.49 ± 3% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 2.74 ± 11% -0.8 1.89 ± 2% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.36 ± 12% -0.8 1.59 ± 2% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 1.05 ± 9% -0.4 0.63 ± 9% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault 1.26 ± 8% -0.3 0.92 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.__lookup_slow.walk_component.link_path_walk.path_parentat.filename_parentat 0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.path_parentat 0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.link_path_walk.path_parentat.filename_parentat.filename_create.do_mkdirat 0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_parentat.filename_parentat.filename_create 0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.path_parentat.filename_parentat.filename_create.do_mkdirat.do_syscall_64 0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.filename_create.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.filename_parentat.filename_create.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mkdir 0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 0.00 +0.7 0.68 ± 5% perf-profile.calltrace.cycles-pp.mkdir 0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.page_fault.__libc_fork.forkshell 0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.__libc_fork.forkshell 0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault.__libc_fork.forkshell 0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault.__libc_fork 0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat 0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc.__alloc_file.alloc_empty_file 0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc.__alloc_file 0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open 0.00 +1.1 1.08 ± 19% perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +1.1 1.08 ± 19% perf-profile.calltrace.cycles-pp.__alloc_file.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2 0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write.new_sync_write.vfs_write 0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write.new_sync_write 0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write 0.00 +1.2 1.25 ± 36% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc.__d_alloc.d_alloc.d_alloc_parallel 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc.__d_alloc.d_alloc 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc.__d_alloc 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__d_alloc.d_alloc.d_alloc_parallel.__lookup_slow 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.d_alloc.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk 0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.__d_alloc.d_alloc.d_alloc_parallel.__lookup_slow.walk_component 0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__vmalloc_node_range.copy_process._do_fork.__x64_sys_clone 0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range.copy_process._do_fork 0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range.copy_process 0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range 0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__vmalloc_node_range.copy_process._do_fork.__x64_sys_clone.do_syscall_64 0.00 +1.4 1.42 ± 21% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write 0.00 +1.4 1.42 ± 21% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin 0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write 0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write 0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write.vfs_write 0.00 +1.4 1.43 ± 21% perf-profile.calltrace.cycles-pp.generic_perform_write.nfs_file_write.new_sync_write.vfs_write.ksys_write 0.00 +1.4 1.43 ± 21% perf-profile.calltrace.cycles-pp.nfs_file_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin.generic_perform_write 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin.generic_perform_write.__generic_file_write_iter 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write 0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.__GI___libc_write 0.00 +1.8 1.84 ± 18% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +1.8 1.84 ± 18% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell 0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell 0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell 0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork 0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.copy_process._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.8 1.85 ± 18% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +1.8 1.85 ± 18% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.00 +1.9 1.85 ± 18% perf-profile.calltrace.cycles-pp.__GI___libc_open 0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.page_fault 0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault 0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault 0.00 +2.4 2.37 ± 24% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.4 2.37 ± 24% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc 0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab.___slab_alloc.__slab_alloc 0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab.___slab_alloc 0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab 0.00 +2.5 2.54 ± 8% perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write 0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.write 0.52 ±173% +2.8 3.32 ± 11% perf-profile.calltrace.cycles-pp.__libc_fork.forkshell 0.00 +2.8 2.82 ± 9% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 0.00 +2.8 2.82 ± 9% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 0.97 ± 17% +3.1 4.05 ± 25% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 0.90 ± 16% +3.1 4.03 ± 25% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +3.2 3.23 ± 6% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin 0.00 +3.2 3.23 ± 6% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page 0.00 +3.2 3.25 ± 9% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk 0.52 ±173% +3.3 3.85 ± 9% perf-profile.calltrace.cycles-pp.forkshell 0.00 +3.4 3.41 ± 9% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.rcu_oom_scan 0.00 +3.5 3.49 ± 8% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.rcu_oom_scan.do_shrink_slab 0.49 ± 59% +3.5 4.03 ± 5% perf-profile.calltrace.cycles-pp.shrink_slab.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 0.47 ± 59% +3.6 4.03 ± 5% perf-profile.calltrace.cycles-pp.do_shrink_slab.shrink_slab.shrink_node.do_try_to_free_pages.try_to_free_pages 0.00 +3.6 3.61 ± 25% perf-profile.calltrace.cycles-pp.memcpy_erms.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread 0.00 +3.7 3.73 ± 25% perf-profile.calltrace.cycles-pp.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread.ret_from_fork 33.79 ± 6% +3.9 37.64 ± 2% perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault 0.00 +3.9 3.92 ± 6% perf-profile.calltrace.cycles-pp.rcu_oom_scan.do_shrink_slab.shrink_slab.shrink_node.do_try_to_free_pages 0.00 +4.4 4.35 ± 3% perf-profile.calltrace.cycles-pp.printk.rcu_oom_scan.do_shrink_slab.shrink_slab.shrink_node 0.00 +4.4 4.35 ± 3% perf-profile.calltrace.cycles-pp.vprintk_emit.printk.rcu_oom_scan.do_shrink_slab.shrink_slab 0.00 +4.5 4.49 ± 8% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 19.77 ± 16% +4.9 24.63 ± 11% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault 19.60 ± 16% +5.4 25.02 ± 12% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma 24.02 ± 7% +13.2 37.20 ± 3% perf-profile.calltrace.cycles-pp.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath 22.00 ± 7% +14.7 36.68 ± 3% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages 21.63 ± 7% +15.0 36.64 ± 3% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages 24.80 ± 6% +17.1 41.89 ± 3% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask 14.09 ± 10% +29.5 43.58 ± 4% perf-profile.calltrace.cycles-pp.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node 14.05 ± 10% +29.5 43.57 ± 4% perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec 13.55 ± 10% +30.0 43.55 ± 4% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list 12.92 ± 10% +30.6 43.47 ± 4% perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list 9.99 ± 6% -9.5 0.47 ± 32% perf-profile.children.cycles-pp.call_function_interrupt 7.22 ± 6% -6.9 0.34 ± 33% perf-profile.children.cycles-pp.smp_call_function_interrupt 45.05 ± 7% -6.8 38.25 perf-profile.children.cycles-pp.do_access 7.03 ± 6% -6.7 0.33 ± 31% perf-profile.children.cycles-pp.flush_smp_call_function_queue 36.34 ± 16% -6.2 30.13 perf-profile.children.cycles-pp.start_secondary 36.69 ± 16% -6.1 30.63 perf-profile.children.cycles-pp.do_idle 36.67 ± 16% -6.0 30.63 perf-profile.children.cycles-pp.secondary_startup_64 36.67 ± 16% -6.0 30.63 perf-profile.children.cycles-pp.cpu_startup_entry 5.97 ± 10% -5.5 0.48 ± 26% perf-profile.children.cycles-pp.pageout 5.81 ± 6% -5.1 0.70 ± 23% perf-profile.children.cycles-pp.rmap_walk_anon 34.08 ± 18% -5.1 28.98 perf-profile.children.cycles-pp.cpuidle_enter_state 34.08 ± 18% -5.1 28.98 perf-profile.children.cycles-pp.cpuidle_enter 5.05 ± 10% -4.7 0.39 ± 25% perf-profile.children.cycles-pp.__swap_writepage 4.97 ± 10% -4.6 0.38 ± 26% perf-profile.children.cycles-pp.bdev_write_page 6.29 ± 9% -4.3 1.97 ± 12% perf-profile.children.cycles-pp.do_rw_once 4.42 ± 10% -4.1 0.33 ± 24% perf-profile.children.cycles-pp.pmem_rw_page 3.39 ± 8% -3.1 0.26 ± 36% perf-profile.children.cycles-pp.try_to_unmap 2.98 ± 9% -2.8 0.21 ± 37% perf-profile.children.cycles-pp.try_to_unmap_one 2.93 ± 11% -2.7 0.22 ± 21% perf-profile.children.cycles-pp.__remove_mapping 2.82 ± 14% -2.7 0.15 ± 25% perf-profile.children.cycles-pp.llist_add_batch 2.78 ± 6% -2.6 0.13 ± 30% perf-profile.children.cycles-pp.flush_tlb_func_common 3.78 ± 7% -2.6 1.13 ± 13% perf-profile.children.cycles-pp._raw_spin_lock 2.77 ± 11% -2.6 0.19 ± 23% perf-profile.children.cycles-pp.pmem_do_bvec 2.75 ± 11% -2.6 0.19 ± 23% perf-profile.children.cycles-pp.write_pmem 2.73 ± 11% -2.5 0.19 ± 23% perf-profile.children.cycles-pp.__memcpy_flushcache 2.58 ± 9% -2.4 0.19 ± 28% perf-profile.children.cycles-pp.add_to_swap 2.46 ± 6% -2.3 0.13 ± 29% perf-profile.children.cycles-pp.llist_reverse_order 2.72 ± 5% -2.2 0.49 ± 15% perf-profile.children.cycles-pp.page_referenced 2.30 ± 8% -2.1 0.19 ± 22% perf-profile.children.cycles-pp.default_send_IPI_mask_sequence_phys 2.85 ± 9% -2.1 0.75 ± 12% perf-profile.children.cycles-pp.get_page_from_freelist 2.12 ± 11% -1.7 0.41 ± 15% perf-profile.children.cycles-pp.__softirqentry_text_start 1.81 ± 8% -1.7 0.14 ± 28% perf-profile.children.cycles-pp.add_to_swap_cache 2.19 ± 5% -1.6 0.57 ± 10% perf-profile.children.cycles-pp.shrink_active_list 1.75 ± 12% -1.6 0.17 ± 38% perf-profile.children.cycles-pp.rcu_core 1.95 ± 8% -1.6 0.37 ± 11% perf-profile.children.cycles-pp.native_irq_return_iret 1.72 ± 9% -1.6 0.15 ± 21% perf-profile.children.cycles-pp.__default_send_IPI_dest_field 1.74 ± 8% -1.5 0.26 ± 23% perf-profile.children.cycles-pp.page_vma_mapped_walk 1.62 ± 9% -1.5 0.14 ± 29% perf-profile.children.cycles-pp.end_page_writeback 1.60 ± 11% -1.5 0.15 ± 44% perf-profile.children.cycles-pp.rcu_do_batch 3.21 ± 9% -1.3 1.86 perf-profile.children.cycles-pp.smp_apic_timer_interrupt 1.61 ± 6% -1.3 0.26 ± 19% perf-profile.children.cycles-pp.page_referenced_one 3.63 ± 9% -1.3 2.32 ± 2% perf-profile.children.cycles-pp.apic_timer_interrupt 1.23 ± 6% -1.2 0.05 ± 72% perf-profile.children.cycles-pp.native_flush_tlb 1.23 ± 11% -1.1 0.11 ± 42% perf-profile.children.cycles-pp.kmem_cache_free 1.16 ± 17% -1.1 0.06 ± 71% perf-profile.children.cycles-pp.smpboot_thread_fn 1.13 ± 17% -1.1 0.05 ± 71% perf-profile.children.cycles-pp.run_ksoftirqd 1.54 ± 10% -1.0 0.51 ± 12% perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay 1.06 ± 11% -1.0 0.09 ± 41% perf-profile.children.cycles-pp.__slab_free 1.36 ± 14% -1.0 0.39 ± 13% perf-profile.children.cycles-pp.down_read_trylock 2.43 ± 8% -0.9 1.50 ± 3% perf-profile.children.cycles-pp.menu_select 0.99 ± 11% -0.9 0.09 ± 28% perf-profile.children.cycles-pp.__delete_from_swap_cache 1.22 ± 7% -0.9 0.32 ± 22% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.36 ± 7% -0.9 0.46 ± 13% perf-profile.children.cycles-pp.prep_new_page 0.99 ± 9% -0.9 0.09 ± 26% perf-profile.children.cycles-pp.xas_create 0.98 ± 9% -0.9 0.10 ± 29% perf-profile.children.cycles-pp.xas_create_range 1.25 ± 9% -0.8 0.43 ± 14% perf-profile.children.cycles-pp.mem_cgroup_try_charge 1.30 ± 3% -0.8 0.52 ± 6% perf-profile.children.cycles-pp.irq_exit 1.20 ± 8% -0.8 0.42 ± 12% perf-profile.children.cycles-pp.clear_page_erms 0.84 ± 7% -0.8 0.08 ± 12% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.82 ± 14% -0.8 0.07 ± 23% perf-profile.children.cycles-pp.mem_cgroup_swapout 0.83 ± 16% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_local_pages_wq 0.83 ± 16% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_pages 0.82 ± 15% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_pages_zone 0.81 ± 8% -0.7 0.08 ± 29% perf-profile.children.cycles-pp.xas_alloc 0.83 ± 11% -0.7 0.11 ± 4% perf-profile.children.cycles-pp._find_next_bit 0.80 ± 16% -0.7 0.09 ± 71% perf-profile.children.cycles-pp.free_pcppages_bulk 0.81 ± 5% -0.7 0.16 ± 21% perf-profile.children.cycles-pp.page_lock_anon_vma_read 0.69 ± 12% -0.6 0.05 ± 72% perf-profile.children.cycles-pp.xas_store 0.64 ± 9% -0.6 0.04 ± 71% perf-profile.children.cycles-pp.swap_writepage 0.66 ± 7% -0.6 0.08 ± 17% perf-profile.children.cycles-pp.cpumask_next 0.75 ± 9% -0.5 0.25 ± 11% perf-profile.children.cycles-pp.__lru_cache_add 0.72 ± 8% -0.5 0.22 ± 11% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.74 ± 11% -0.5 0.25 ± 15% perf-profile.children.cycles-pp.lru_cache_add_active_or_unevictable 0.68 ± 13% -0.5 0.21 ± 7% perf-profile.children.cycles-pp.up_read 0.53 ± 15% -0.4 0.10 ± 16% perf-profile.children.cycles-pp.count_shadow_nodes 1.57 ± 8% -0.4 1.15 perf-profile.children.cycles-pp.hrtimer_interrupt 0.44 ± 11% -0.4 0.04 ± 73% perf-profile.children.cycles-pp.call_rcu 0.40 ± 8% -0.4 0.04 ± 71% perf-profile.children.cycles-pp.radix_tree_node_ctor 0.43 ± 10% -0.3 0.08 ± 17% perf-profile.children.cycles-pp.isolate_lru_pages 0.41 ± 15% -0.3 0.06 ± 13% perf-profile.children.cycles-pp.__list_del_entry_valid 0.46 ± 9% -0.3 0.13 ± 12% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.99 ± 19% -0.3 0.66 ± 6% perf-profile.children.cycles-pp.ktime_get 0.37 ± 10% -0.3 0.04 ± 76% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm 0.40 ± 9% -0.2 0.15 ± 12% perf-profile.children.cycles-pp.sync_regs 0.39 ± 6% -0.2 0.15 ± 11% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.79 ± 5% -0.2 0.57 perf-profile.children.cycles-pp.__hrtimer_run_queues 0.28 ± 6% -0.2 0.07 ± 20% perf-profile.children.cycles-pp.__mod_lruvec_state 0.59 ± 4% -0.2 0.40 ± 2% perf-profile.children.cycles-pp.tick_sched_timer 0.31 ± 13% -0.2 0.13 ± 14% perf-profile.children.cycles-pp.try_charge 0.29 ± 16% -0.2 0.11 ± 19% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.24 ± 10% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.move_pages_to_lru 0.59 ± 12% -0.2 0.44 ± 3% perf-profile.children.cycles-pp.clockevents_program_event 0.20 ± 7% -0.1 0.06 ± 8% perf-profile.children.cycles-pp.__perf_sw_event 0.20 ± 17% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate 0.43 ± 6% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.update_process_times 0.43 ± 6% -0.1 0.30 ± 5% perf-profile.children.cycles-pp.tick_sched_handle 0.51 ± 14% -0.1 0.39 ± 14% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.15 ± 15% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.mem_cgroup_commit_charge 0.17 ± 19% -0.1 0.07 ± 25% perf-profile.children.cycles-pp.__sched_text_start 0.20 ± 9% -0.1 0.12 ± 23% perf-profile.children.cycles-pp.irq_work_run_list 0.11 ± 28% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.schedule 0.18 ± 6% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.scheduler_tick 0.20 ± 4% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.18 ± 21% -0.0 0.13 perf-profile.children.cycles-pp.rebalance_domains 0.15 ± 6% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.native_write_msr 0.12 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.lapic_next_deadline 0.14 ± 5% -0.0 0.11 ± 8% perf-profile.children.cycles-pp.__next_timer_interrupt 0.09 ± 24% -0.0 0.06 ± 16% perf-profile.children.cycles-pp._raw_spin_trylock 0.09 ± 9% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.read_tsc 0.08 ± 10% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.sched_clock_cpu 0.07 ± 22% -0.0 0.06 ± 16% perf-profile.children.cycles-pp.run_local_timers 0.00 +0.1 0.09 ± 36% perf-profile.children.cycles-pp.fbcon_putcs 0.00 +0.1 0.09 ± 36% perf-profile.children.cycles-pp.bit_putcs 0.00 +0.1 0.09 ± 39% perf-profile.children.cycles-pp.fbcon_redraw 0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.lf 0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.con_scroll 0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.fbcon_scroll 0.00 +0.1 0.11 ± 34% perf-profile.children.cycles-pp.vt_console_print 0.00 +0.1 0.14 ± 40% perf-profile.children.cycles-pp.ksys_read 0.00 +0.1 0.14 ± 40% perf-profile.children.cycles-pp.vfs_read 0.00 +0.1 0.14 ± 42% perf-profile.children.cycles-pp.read 0.00 +0.2 0.16 ± 38% perf-profile.children.cycles-pp.sk_page_frag_refill 0.00 +0.2 0.16 ± 38% perf-profile.children.cycles-pp.skb_page_frag_refill 0.00 +0.2 0.16 ± 21% perf-profile.children.cycles-pp.force_qs_rnp 0.00 +0.2 0.17 ± 17% perf-profile.children.cycles-pp.rcu_gp_kthread 0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.setup_arg_pages 0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.shift_arg_pages 0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.move_page_tables 0.00 +0.2 0.19 ± 39% perf-profile.children.cycles-pp.schedule_tail 0.00 +0.2 0.19 ± 39% perf-profile.children.cycles-pp.__put_user_4 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.call_transmit 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xprt_transmit 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xs_tcp_send_request 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xs_sendpages 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.sock_sendmsg 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.tcp_sendmsg 0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.tcp_sendmsg_locked 0.00 +0.2 0.19 ± 71% perf-profile.children.cycles-pp.__vmalloc_node 0.00 +0.2 0.19 ± 28% perf-profile.children.cycles-pp.rpc_async_schedule 0.00 +0.2 0.19 ± 28% perf-profile.children.cycles-pp.__rpc_execute 0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.copy_strings 0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.get_user_pages_remote 0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.__get_user_pages 0.01 ±173% +0.2 0.24 ± 48% perf-profile.children.cycles-pp.search_binary_handler 0.01 ±173% +0.2 0.24 ± 48% perf-profile.children.cycles-pp.load_elf_binary 0.00 +0.3 0.26 ± 53% perf-profile.children.cycles-pp.shmem_alloc_page 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.posix_fallocate 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.__x64_sys_fallocate 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.ksys_fallocate 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.vfs_fallocate 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.shmem_fallocate 0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.shmem_alloc_and_acct_page 0.00 +0.3 0.27 ± 28% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.00 +0.3 0.28 ± 33% perf-profile.children.cycles-pp.__pmd_alloc 0.12 ± 13% +0.3 0.45 ± 9% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.perf_poll 0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.start_thread 0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.__pollwait 0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.poll 0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.__x64_sys_poll 0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.do_sys_poll 0.08 ± 23% +0.4 0.44 ± 9% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.00 +0.4 0.39 ± 9% perf-profile.children.cycles-pp.delay_tsc 0.00 +0.4 0.41 ± 9% perf-profile.children.cycles-pp.flush_tlb_batched_pending 0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.task_numa_work 0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.change_prot_numa 0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.change_p4d_range 0.01 ±173% +0.4 0.43 ± 9% perf-profile.children.cycles-pp.change_protection 0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.kvmalloc_node 0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.__kmalloc_node 0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.kmalloc_large_node 0.01 ±173% +0.4 0.44 ± 10% perf-profile.children.cycles-pp.task_work_run 0.00 +0.4 0.43 ± 12% perf-profile.children.cycles-pp.flush_tlb_mm_range 0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_fault 0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_swapin_page 0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_swapin 0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.swap_cluster_readahead 0.00 +0.5 0.52 ± 41% perf-profile.children.cycles-pp.__read_swap_cache_async 0.00 +0.5 0.54 ± 6% perf-profile.children.cycles-pp.io_serial_out 0.00 +0.6 0.55 ± 47% perf-profile.children.cycles-pp.__do_fault 0.04 ± 59% +0.6 0.63 ± 29% perf-profile.children.cycles-pp.exit_to_usermode_loop 0.00 +0.6 0.59 ± 7% perf-profile.children.cycles-pp.do_swap_page 0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.do_dentry_open 0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.open64 0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.proc_reg_open 0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.single_open_size 0.00 +0.7 0.66 ± 4% perf-profile.children.cycles-pp.path_parentat 0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.filename_create 0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.filename_parentat 0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.do_mkdirat 0.00 +0.7 0.68 ± 5% perf-profile.children.cycles-pp.mkdir 0.00 +0.8 0.77 ± 28% perf-profile.children.cycles-pp.shmem_getpage_gfp 0.00 +1.1 1.08 ± 19% perf-profile.children.cycles-pp.alloc_empty_file 0.00 +1.1 1.08 ± 19% perf-profile.children.cycles-pp.__alloc_file 0.00 +1.2 1.16 ± 52% perf-profile.children.cycles-pp.do_wp_page 0.00 +1.2 1.16 ± 52% perf-profile.children.cycles-pp.wp_page_copy 0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.__lookup_slow 0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.d_alloc_parallel 0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.d_alloc 0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.__d_alloc 0.00 +1.4 1.41 ± 45% perf-profile.children.cycles-pp.pipe_write 0.00 +1.4 1.42 ± 35% perf-profile.children.cycles-pp.walk_component 0.00 +1.4 1.42 ± 20% perf-profile.children.cycles-pp.nfs_write_begin 0.00 +1.4 1.43 ± 34% perf-profile.children.cycles-pp.link_path_walk 0.00 +1.4 1.43 ± 21% perf-profile.children.cycles-pp.nfs_file_write 0.03 ±102% +1.6 1.61 ± 27% perf-profile.children.cycles-pp.__vmalloc_node_range 0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp.__x64_sys_clone 0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp._do_fork 0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp.copy_process 0.00 +1.8 1.81 ± 27% perf-profile.children.cycles-pp.simple_write_begin 0.00 +1.8 1.82 ± 27% perf-profile.children.cycles-pp.generic_file_write_iter 0.00 +1.8 1.82 ± 27% perf-profile.children.cycles-pp.__generic_file_write_iter 0.00 +1.8 1.83 ± 27% perf-profile.children.cycles-pp.__GI___libc_write 0.00 +1.9 1.86 ± 18% perf-profile.children.cycles-pp.__GI___libc_open 0.77 ± 8% +1.9 2.66 ± 16% perf-profile.children.cycles-pp.kmem_cache_alloc 0.62 ± 7% +2.3 2.92 ± 17% perf-profile.children.cycles-pp.__slab_alloc 0.62 ± 7% +2.3 2.92 ± 17% perf-profile.children.cycles-pp.___slab_alloc 0.56 ± 7% +2.4 2.92 ± 17% perf-profile.children.cycles-pp.new_slab 0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_filp_open 0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.path_openat 0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_sys_open 0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_sys_openat2 0.68 ±119% +2.6 3.32 ± 11% perf-profile.children.cycles-pp.__libc_fork 0.00 +2.8 2.85 ± 30% perf-profile.children.cycles-pp.write 0.97 ± 17% +3.1 4.05 ± 25% perf-profile.children.cycles-pp.worker_thread 0.90 ± 16% +3.1 4.03 ± 25% perf-profile.children.cycles-pp.process_one_work 0.69 ±116% +3.2 3.85 ± 9% perf-profile.children.cycles-pp.forkshell 0.09 ± 7% +3.2 3.30 ± 4% perf-profile.children.cycles-pp.io_serial_in 0.00 +3.2 3.24 ± 6% perf-profile.children.cycles-pp.grab_cache_page_write_begin 0.00 +3.2 3.25 ± 6% perf-profile.children.cycles-pp.generic_perform_write 0.00 +3.3 3.27 ± 7% perf-profile.children.cycles-pp.pagecache_get_page 0.11 ± 12% +3.4 3.52 ± 5% perf-profile.children.cycles-pp.serial8250_console_putchar 0.12 ± 12% +3.6 3.68 ± 5% perf-profile.children.cycles-pp.wait_for_xmitr 0.00 +3.7 3.73 ± 25% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work 0.00 +3.7 3.73 ± 25% perf-profile.children.cycles-pp.memcpy_erms 0.12 ± 12% +3.9 4.04 ± 5% perf-profile.children.cycles-pp.uart_console_write 0.12 ± 12% +4.1 4.23 ± 5% perf-profile.children.cycles-pp.serial8250_console_write 0.12 ± 14% +4.2 4.35 ± 4% perf-profile.children.cycles-pp.console_unlock 35.04 ± 6% +4.4 39.41 ± 2% perf-profile.children.cycles-pp.handle_mm_fault 0.78 ± 16% +4.6 5.38 ± 3% perf-profile.children.cycles-pp.shrink_slab 34.72 ± 6% +4.6 39.33 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault 0.74 ± 16% +4.6 5.38 ± 3% perf-profile.children.cycles-pp.do_shrink_slab 0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.ksys_write 0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.vfs_write 0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.new_sync_write 34.00 ± 6% +5.0 39.02 ± 2% perf-profile.children.cycles-pp.handle_pte_fault 0.00 +5.2 5.21 ± 3% perf-profile.children.cycles-pp.rcu_oom_scan 0.08 ± 27% +5.3 5.43 ± 2% perf-profile.children.cycles-pp.printk 0.08 ± 27% +5.3 5.43 ± 2% perf-profile.children.cycles-pp.vprintk_emit 1.05 ±104% +10.7 11.71 ± 4% perf-profile.children.cycles-pp.do_syscall_64 1.05 ±104% +10.7 11.71 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 34.02 ± 9% +16.9 50.93 ± 4% perf-profile.children.cycles-pp.shrink_lruvec 31.76 ± 9% +18.6 50.34 ± 4% perf-profile.children.cycles-pp.shrink_inactive_list 31.30 ± 9% +19.0 50.29 ± 4% perf-profile.children.cycles-pp.shrink_page_list 29.28 ± 7% +19.0 48.27 ± 3% perf-profile.children.cycles-pp.__alloc_pages_nodemask 28.07 ± 7% +19.5 47.59 ± 3% perf-profile.children.cycles-pp.__alloc_pages_slowpath 34.84 ± 9% +21.5 56.33 ± 3% perf-profile.children.cycles-pp.shrink_node 25.45 ± 7% +22.0 47.43 ± 4% perf-profile.children.cycles-pp.try_to_free_pages 25.21 ± 7% +22.2 47.41 ± 4% perf-profile.children.cycles-pp.do_try_to_free_pages 14.33 ± 10% +34.6 48.95 ± 5% perf-profile.children.cycles-pp.try_to_unmap_flush_dirty 14.31 ± 10% +34.6 48.95 ± 5% perf-profile.children.cycles-pp.arch_tlbbatch_flush 13.81 ± 10% +35.1 48.94 ± 5% perf-profile.children.cycles-pp.on_each_cpu_cond_mask 13.46 ± 11% +35.9 49.31 ± 5% perf-profile.children.cycles-pp.smp_call_function_many_cond 5.33 ± 9% -3.4 1.90 ± 10% perf-profile.self.cycles-pp.do_rw_once 4.88 ± 9% -3.1 1.75 ± 11% perf-profile.self.cycles-pp.do_access 2.46 ± 6% -2.3 0.13 ± 29% perf-profile.self.cycles-pp.llist_reverse_order 2.29 ± 15% -2.2 0.13 ± 26% perf-profile.self.cycles-pp.llist_add_batch 2.26 ± 12% -2.1 0.17 ± 23% perf-profile.self.cycles-pp.__memcpy_flushcache 2.15 ± 8% -2.1 0.09 ± 39% perf-profile.self.cycles-pp.flush_smp_call_function_queue 2.49 ± 9% -1.6 0.90 ± 10% perf-profile.self.cycles-pp._raw_spin_lock 1.95 ± 8% -1.6 0.37 ± 11% perf-profile.self.cycles-pp.native_irq_return_iret 1.72 ± 9% -1.6 0.15 ± 21% perf-profile.self.cycles-pp.__default_send_IPI_dest_field 1.55 ± 6% -1.5 0.08 ± 26% perf-profile.self.cycles-pp.flush_tlb_func_common 1.25 ± 10% -1.2 0.06 ± 75% perf-profile.self.cycles-pp.try_to_unmap_one 1.23 ± 6% -1.2 0.05 ± 72% perf-profile.self.cycles-pp.native_flush_tlb 1.10 ± 11% -1.0 0.12 ± 28% perf-profile.self.cycles-pp.end_page_writeback 1.02 ± 12% -0.9 0.09 ± 41% perf-profile.self.cycles-pp.__slab_free 1.15 ± 9% -0.8 0.31 ± 22% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 1.19 ± 15% -0.8 0.37 ± 14% perf-profile.self.cycles-pp.down_read_trylock 1.90 ± 13% -0.8 1.09 ± 4% perf-profile.self.cycles-pp.menu_select 0.84 ± 17% -0.8 0.07 ± 30% perf-profile.self.cycles-pp.page_vma_mapped_walk 0.85 ± 10% -0.6 0.21 ± 10% perf-profile.self.cycles-pp.get_page_from_freelist 1.05 ± 7% -0.6 0.42 ± 13% perf-profile.self.cycles-pp.clear_page_erms 0.67 ± 10% -0.6 0.10 ± 4% perf-profile.self.cycles-pp._find_next_bit 0.62 ± 15% -0.4 0.19 ± 18% perf-profile.self.cycles-pp.__handle_mm_fault 0.67 ± 11% -0.4 0.25 ± 15% perf-profile.self.cycles-pp.lru_cache_add_active_or_unevictable 0.60 ± 12% -0.4 0.20 ± 8% perf-profile.self.cycles-pp.up_read 1.03 ± 11% -0.4 0.64 ± 4% perf-profile.self.cycles-pp.cpuidle_enter_state 0.39 ± 8% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.radix_tree_node_ctor 0.39 ± 16% -0.3 0.06 ± 13% perf-profile.self.cycles-pp.__list_del_entry_valid 0.92 ± 20% -0.3 0.61 ± 7% perf-profile.self.cycles-pp.ktime_get 0.31 ± 10% -0.3 0.04 ± 73% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm 0.30 ± 6% -0.3 0.04 ± 73% perf-profile.self.cycles-pp.page_lock_anon_vma_read 0.49 ± 11% -0.3 0.24 ± 12% perf-profile.self.cycles-pp.mem_cgroup_try_charge 0.40 ± 9% -0.2 0.15 ± 12% perf-profile.self.cycles-pp.sync_regs 0.34 ± 17% -0.2 0.11 ± 22% perf-profile.self.cycles-pp.handle_pte_fault 0.27 ± 16% -0.2 0.04 ± 71% perf-profile.self.cycles-pp.count_shadow_nodes 0.32 ± 11% -0.2 0.09 ± 15% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.24 ± 13% -0.2 0.04 ± 71% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.22 ± 19% -0.2 0.04 ± 73% perf-profile.self.cycles-pp.handle_mm_fault 0.28 ± 12% -0.2 0.13 ± 13% perf-profile.self.cycles-pp.try_charge 0.23 ± 6% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.__pagevec_lru_add_fn 0.15 ± 15% -0.1 0.06 ± 8% perf-profile.self.cycles-pp.mem_cgroup_throttle_swaprate 0.15 ± 6% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.native_write_msr 0.07 ± 22% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.run_local_timers 0.09 ± 24% -0.0 0.05 ± 8% perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.4 0.39 ± 9% perf-profile.self.cycles-pp.delay_tsc 0.00 +0.5 0.54 ± 6% perf-profile.self.cycles-pp.io_serial_out 0.00 +1.1 1.09 ± 17% perf-profile.self.cycles-pp.vprintk_emit 0.09 ± 7% +3.2 3.30 ± 4% perf-profile.self.cycles-pp.io_serial_in 0.00 +3.7 3.71 ± 25% perf-profile.self.cycles-pp.memcpy_erms 6.37 ± 12% +42.3 48.62 ± 5% perf-profile.self.cycles-pp.smp_call_function_many_cond vm-scalability.time.system_time 2500 +--------------------------------------------------------------------+ | O O O | | O O O O O O | 2000 |-+ O O | | O O | | | 1500 |-+ | | | 1000 |-+ | | | | | 500 |-+ | | | | .+.+.+..+.+. .+..+.+. .+.+.+.. .+.+.+..+.+.+..+.+.+..+.+.+.+..+.| 0 +--------------------------------------------------------------------+ vm-scalability.time.elapsed_time 400 +---------------------------------------------------------------------+ | O O O O | 350 |-+ O O O | 300 |-O O O | | O O O | 250 |-+ | | | 200 |-+ | | | 150 |-+ | 100 |-+ | | | 50 |-+ +.+.+..+.+ .+.. +.+.. +..+.+.+..+.+.+..+.+.+..+.+.+..+.| | .. + .+.+ + +. + | 0 +---------------------------------------------------------------------+ vm-scalability.time.elapsed_time.max 400 +---------------------------------------------------------------------+ | O O O O | 350 |-+ O O O | 300 |-O O O | | O O O | 250 |-+ | | | 200 |-+ | | | 150 |-+ | 100 |-+ | | | 50 |-+ +.+.+..+.+ .+.. +.+.. +..+.+.+..+.+.+..+.+.+..+.+.+..+.| | .. + .+.+ + +. + | 0 +---------------------------------------------------------------------+ vm-scalability.throughput 7e+06 +-------------------------------------------------------------------+ | | 6e+06 |-+ .+.+ +.+.. +..+ .+. | | + + : + : : +..+.+.+.+..+.+.+.+..+.+ +..+.| 5e+06 |-+ : +..+ : : : : : | | : : : : + : : | 4e+06 |-+ : : : : : : : | | : : : : : : : | 3e+06 |-+ : : : : : : : | | : : : : : : : | 2e+06 |-+: : : : : : : | | : : : : : :: | 1e+06 |-O: O : O : O : O O O | | : O : O O : O O : O | 0 +-------------------------------------------------------------------+ vm-scalability.median 800000 +------------------------------------------------------------------+ | .+ : +.+. +.+ +.+. .+. .+.. | 700000 |-+ +. : : + : : : +.+..+.+ +..+.+.+ +.+.| | : +.+ : : : : : | 600000 |-+ : : : : + : : | 500000 |-+ : : : : : : : | | : : : : : : : | 400000 |-+: : : : : : : | | : : : : : : : | 300000 |-+: : : : : : : | 200000 |-+: : : : : : : | | : :: : : :: | 100000 |-: O O :: O O O : O O O : O O O O | | : : : : | 0 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

8 months, 2 weeks

2
1
0 / 0

[treewide] 9cd745027b: stack_segment:#[##]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 9cd745027be08348374ddf42eeb3b94b785aba3f ("treewide: Replace one-element array with flexible-array") https://git.kernel.org/cgit/linux/kernel/git/gustavoars/linux.git testing/fam1 in testcase: locktorture with following parameters: runtime: 300s test: cpuhotplug test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors. test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 2.141748] ACPI: Interpreter enabled [ 2.157189] ACPI: (supports S0 S3 S4 S5) [ 2.164064] ACPI: Using IOAPIC for interrupt routing [ 2.170175] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 2.189565] ACPI: Enabled 2 GPEs in block 00 to 0F [ 2.194436] stack segment: 0000 [#1] SMP PTI [ 2.195001] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3-00001-g9cd745027be08 #1 [ 2.195001] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 2.195001] RIP: 0010:__kmalloc_track_caller+0x98/0x270 [ 2.195001] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 77 9a d6 60 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b [ 2.195001] RSP: 0000:ffffc1f7c0013a38 EFLAGS: 00010206 [ 2.195001] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000077 [ 2.195001] RDX: 0000000000000076 RSI: 0000000000000cc0 RDI: ffff9c6a47c03a40 [ 2.195001] RBP: 0035304130504e50 R08: 0000000000031060 R09: 0000000000000000 [ 2.195001] R10: 0000000000000000 R11: ffff9c6a4786f0c8 R12: 0000000000000cc0 [ 2.195001] R13: 000000000000000c R14: ffff9c6a47c03a40 R15: ffff9c6a47c03a40 [ 2.195001] FS: 0000000000000000(0000) GS:ffff9c6b7fd00000(0000) knlGS:0000000000000000 [ 2.195001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2.195001] CR2: 00000000ffffffff CR3: 000000023a00a000 CR4: 00000000000406e0 [ 2.195001] Call Trace: [ 2.195001] ? __kernfs_new_node+0x40/0x1e0 [ 2.195001] kstrdup+0x2d/0x60 [ 2.195001] __kernfs_new_node+0x40/0x1e0 [ 2.195001] ? kernfs_create_dir_ns+0x50/0x80 [ 2.195001] ? _cond_resched+0x19/0x30 [ 2.195001] ? kernfs_activate+0x63/0x80 [ 2.195001] kernfs_new_node+0x34/0x60 [ 2.195001] kernfs_create_link+0x35/0xb0 [ 2.195001] sysfs_do_create_link_sd+0x67/0xc0 [ 2.195001] bus_add_device+0x6e/0x150 [ 2.195001] device_add+0x38b/0x840 [ 2.195001] ? kmem_cache_alloc_trace+0x21e/0x230 [ 2.195001] acpi_device_add+0x1dc/0x3f0 [ 2.195001] ? acpi_free_pnp_ids+0x50/0x50 [ 2.195001] acpi_add_single_object+0x25c/0x5f0 [ 2.195001] ? up+0x12/0x60 [ 2.195001] ? acpi_ut_acquire_mutex+0x45/0x89 [ 2.195001] ? acpi_ut_release_mutex+0x6f/0x74 [ 2.195001] ? acpi_get_data_full+0x7e/0x90 [ 2.195001] ? rdinit_setup+0x2b/0x2b [ 2.195001] acpi_bus_check_add+0xd0/0x260 [ 2.195001] ? acpi_ev_gpe_detect+0xd0/0xe2 [ 2.195001] ? acpi_sleep_proc_init+0x24/0x24 [ 2.195001] acpi_bus_scan+0x31/0x90 [ 2.195001] acpi_scan_init+0xf1/0x235 [ 2.195001] ? acpi_sleep_proc_init+0x24/0x24 [ 2.195001] acpi_init+0x2f4/0x354 [ 2.195001] do_one_initcall+0x46/0x220 [ 2.195001] kernel_init_freeable+0x206/0x280 [ 2.195001] ? rest_init+0xd0/0xd0 [ 2.195001] kernel_init+0xa/0x110 [ 2.195001] ret_from_fork+0x35/0x40 [ 2.195001] Modules linked in: [ 2.469071] ---[ end trace a63967e0ecd68f27 ]--- [ 2.473044] RIP: 0010:__kmalloc_track_caller+0x98/0x270 [ 2.478035] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 77 9a d6 60 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b [ 2.494032] RSP: 0000:ffffc1f7c0013a38 EFLAGS: 00010206 [ 2.499032] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000077 [ 2.505029] RDX: 0000000000000076 RSI: 0000000000000cc0 RDI: ffff9c6a47c03a40 [ 2.521078] RBP: 0035304130504e50 R08: 0000000000031060 R09: 0000000000000000 [ 2.531050] R10: 0000000000000000 R11: ffff9c6a4786f0c8 R12: 0000000000000cc0 [ 2.537046] R13: 000000000000000c R14: ffff9c6a47c03a40 R15: ffff9c6a47c03a40 [ 2.543053] FS: 0000000000000000(0000) GS:ffff9c6b7fd00000(0000) knlGS:0000000000000000 [ 2.550044] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2.555032] CR2: 00000000ffffffff CR3: 000000023a00a000 CR4: 00000000000406e0 [ 2.561038] Kernel panic - not syncing: Fatal exception Elapsed time: 60 qemu-img create -f qcow2 disk-vm-snb-152-0 256G qemu-img create -f qcow2 disk-vm-snb-152-1 256G To reproduce: # build kernel cd linux cp config-5.7.0-rc3-00001-g9cd745027be08 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

c088afc0d1: BUG:KASAN:slab-out-of-bounds_in__ptrace_may_access

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: c088afc0d1ec4660184a675c1c0c3d035f37c8b5 (" iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCXrFbowAKCRCRxhvAZXjc") https://git.kernel.org/cgit/linux/kernel/git/brauner/linux.git setns_pidfd in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 29.718859] BUG: KASAN: slab-out-of-bounds in __ptrace_may_access+0x4d/0x230 [ 29.719652] Read of size 8 at addr ffff8881d2d365d0 by task trinity-c0/882 [ 29.720412] [ 29.720595] CPU: 0 PID: 882 Comm: trinity-c0 Not tainted 5.7.0-rc2-00002-gc088afc0d1ec4 #1 [ 29.721497] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 29.722411] Call Trace: [ 29.722701] dump_stack+0x19/0x25 [ 29.723093] print_address_description+0x20/0x320 [ 29.723768] ? __ptrace_may_access+0x4d/0x230 [ 29.724271] ? __ptrace_may_access+0x4d/0x230 [ 29.724770] __kasan_report+0xf2/0x11d [ 29.725213] ? pidns_for_children_get+0x60/0x120 [ 29.725747] ? __ptrace_may_access+0x4d/0x230 [ 29.726251] kasan_report+0x38/0x50 [ 29.726662] __asan_load8+0x69/0x90 [ 29.727076] __ptrace_may_access+0x4d/0x230 [ 29.727552] ptrace_may_access+0x27/0x60 [ 29.727995] __x64_sys_setns+0x4b8/0x950 [ 29.728440] ? exit_task_namespaces+0x20/0x20 [ 29.728929] do_syscall_64+0xbb/0x660 [ 29.729350] ? syscall_return_slowpath+0x230/0x230 [ 29.729889] ? fpregs_assert_state_consistent+0x63/0x70 [ 29.730478] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 29.732003] RIP: 0033:0x453b29 [ 29.732358] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00 [ 29.734458] RSP: 002b:00007ffddc2ad678 EFLAGS: 00000246 ORIG_RAX: 0000000000000134 [ 29.735292] RAX: ffffffffffffffda RBX: 0000000000000134 RCX: 0000000000453b29 [ 29.736146] RDX: 000000014522b083 RSI: 0000000040000000 RDI: 0000000000000014 [ 29.736954] RBP: 00007ffddc2ad720 R08: 0000000000000012 R09: 27b18fbd5200e56b [ 29.737826] R10: 000000000000009d R11: 0000000000000246 R12: 0000000000000002 [ 29.738630] R13: 00007fbc53011058 R14: 00000000010a2830 R15: 00007fbc53011000 [ 29.739420] [ 29.739646] Allocated by task 589: [ 29.740059] save_stack+0x21/0x50 [ 29.740439] __kasan_kmalloc+0xa6/0xd0 [ 29.740979] kasan_kmalloc+0x9/0x10 [ 29.741381] kmem_cache_alloc_trace+0x108/0x250 [ 29.741950] do_eventfd+0x46/0x120 [ 29.742341] __x64_sys_eventfd2+0x31/0x40 [ 29.742824] do_syscall_64+0xbb/0x660 [ 29.743245] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 29.743874] [ 29.744059] Freed by task 0: [ 29.744376] (stack is not available) [ 29.744765] [ 29.744943] The buggy address belongs to the object at ffff8881d2d36440 [ 29.744943] which belongs to the cache kmalloc-64 of size 64 [ 29.746330] The buggy address is located 336 bytes to the right of [ 29.746330] 64-byte region [ffff8881d2d36440, ffff8881d2d36480) [ 29.747710] The buggy address belongs to the page: [ 29.748262] page:ffffea00074b4d80 refcount:1 mapcount:0 mapping:(____ptrval____) index:0x0 [ 29.749193] flags: 0x8000000000000200(slab) [ 29.749745] raw: 8000000000000200 ffffea00073bc3c8 ffffea000788fdc8 ffff8881f68036c0 [ 29.750616] raw: 0000000000000000 0000000000080008 00000001ffffffff 0000000000000000 [ 29.751458] page dumped because: kasan: bad access detected [ 29.752073] [ 29.752249] Memory state around the buggy address: [ 29.752782] ffff8881d2d36480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 29.753570] ffff8881d2d36500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 29.754360] >ffff8881d2d36580: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 29.755180] ^ [ 29.755813] ffff8881d2d36600: fc fc fc fc fc fc fc fc 00 00 00 00 00 fc fc fc [ 29.756593] ffff8881d2d36680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 29.757386] ================================================================== [ 29.758171] Disabling lock debugging due to kernel taint [ 29.758818] BUG: kernel NULL pointer dereference, address: 0000000000000014 [ 29.759586] #PF: supervisor read access in kernel mode [ 29.760173] #PF: error_code(0x0000) - not-present page [ 29.760754] PGD 1cea22067 P4D 1cea22067 PUD 1db0d4067 PMD 0 [ 29.761397] Oops: 0000 [#1] PREEMPT KASAN [ 29.761858] CPU: 0 PID: 882 Comm: trinity-c0 Tainted: G B 5.7.0-rc2-00002-gc088afc0d1ec4 #1 [ 29.762900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 29.763503] RIP: 0010:__ptrace_may_access+0xc9/0x230 [ 29.763864] Code: 75 04 e8 9a 89 26 00 41 8b 45 08 89 45 d4 48 8d bb 50 05 00 00 e8 a7 8a 26 00 4c 8b bb 50 05 00 00 49 8d 7f 14 e8 77 89 26 00 <45> 3b 77 14 75 13 49 8d 7f 0c e8 68 89 26 00 45 39 77 0c 0f 84 c0 [ 29.765191] RSP: 0018:ffff8881e64a7d68 EFLAGS: 00010246 [ 29.765568] RAX: 0000000000000000 RBX: ffff8881d2d36020 RCX: ffffffff810bbfd9 [ 29.766076] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000014 [ 29.766585] RBP: ffff8881e64a7d98 R08: fffffbfff0676939 R09: fffffbfff0676939 [ 29.767107] R10: ffffffff833b49c7 R11: 0000000000000001 R12: 0000000000000011 [ 29.767616] R13: ffff8881e1d31480 R14: 000000000000fffe R15: 0000000000000000 [ 29.768128] FS: 00000000010a2880(0000) GS:ffffffff82c9c000(0000) knlGS:0000000000000000 [ 29.768708] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 29.769123] CR2: 0000000000000014 CR3: 00000001ceaf6000 CR4: 00000000000406b0 [ 29.769636] DR0: 00007fbc526f3000 DR1: 0000000000000000 DR2: 0000000000000000 [ 29.770146] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600 [ 29.770656] Call Trace: [ 29.770855] ptrace_may_access+0x27/0x60 [ 29.771150] __x64_sys_setns+0x4b8/0x950 [ 29.771437] ? exit_task_namespaces+0x20/0x20 [ 29.771755] do_syscall_64+0xbb/0x660 [ 29.772023] ? syscall_return_slowpath+0x230/0x230 [ 29.772372] ? fpregs_assert_state_consistent+0x63/0x70 [ 29.772749] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 29.773116] RIP: 0033:0x453b29 [ 29.773343] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00 [ 29.774666] RSP: 002b:00007ffddc2ad678 EFLAGS: 00000246 ORIG_RAX: 0000000000000134 [ 29.775221] RAX: ffffffffffffffda RBX: 0000000000000134 RCX: 0000000000453b29 [ 29.775729] RDX: 000000014522b083 RSI: 0000000040000000 RDI: 0000000000000014 [ 29.776237] RBP: 00007ffddc2ad720 R08: 0000000000000012 R09: 27b18fbd5200e56b [ 29.776745] R10: 000000000000009d R11: 0000000000000246 R12: 0000000000000002 [ 29.777252] R13: 00007fbc53011058 R14: 00000000010a2830 R15: 00007fbc53011000 [ 29.777759] Modules linked in: input_leds serio_raw evdev qemu_fw_cfg processor tiny_power_button button [ 29.778435] CR2: 0000000000000014 [ 29.778678] ---[ end trace 9561f8100fce2866 ]--- To reproduce: # build kernel cd linux cp config-5.7.0-rc2-00002-gc088afc0d1ec4 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

[fs] 05c5a0273b: netperf.Throughput_total_tps -71.8% regression

by kernel test robot

Greeting, FYI, we noticed a -71.8% regression of netperf.Throughput_total_tps due to commit: commit: 05c5a0273bc25627b9e9bbebc858b74e88f49c5b ("[PATCH v2] fs: jfs: fix a possible data race in txBegin()") url: https://github.com/0day-ci/linux/commits/Jia-Ju-Bai/fs-jfs-fix-a-possible... base: https://github.com/kleikamp/linux-shaggy jfs-next in testcase: netperf on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory with following parameters: ip: ipv4 runtime: 300s nr_threads: 1 cluster: cs-localhost test: TCP_CRR cpufreq_governor: performance ucode: 0x2000065 test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance. test-url: http://www.netperf.org/netperf/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode: cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.6/1/debian-x86_64-20191114.cgz/300s/lkp-skl-2sp7/TCP_CRR/netperf/0x2000065 commit: 7aba5dcc23 ("jfs: Replace zero-length array with flexible-array member") 05c5a0273b ("fs: jfs: fix a possible data race in txBegin()") 7aba5dcc234635b4 05c5a0273bc25627b9e9bbebc85 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 100% 4:4 kmsg.IP-Config:Failed_to_open_erspan0 :4 100% 4:4 kmsg.IP-Config:Failed_to_open_gretap0 :4 100% 4:4 kmsg.kvm:already_loaded_the_other_module %stddev %change %stddev \ | \ 32033 -71.8% 9048 netperf.Throughput_total_tps 32033 -71.8% 9048 netperf.Throughput_tps 34270 ± 19% -32.2% 23234 ± 11% netperf.time.involuntary_context_switches 87.50 -1.7% 86.00 netperf.time.percent_of_cpu_this_job_got 237.76 +7.1% 254.71 netperf.time.system_time 30.66 -66.6% 10.25 ± 3% netperf.time.user_time 17926206 -69.8% 5405643 netperf.time.voluntary_context_switches 9610194 -71.8% 2714625 netperf.workload 0.25 -0.1 0.12 ± 3% mpstat.cpu.all.usr% 181085 -60.2% 72003 vmstat.system.cs 24.19 ± 2% +8.7% 26.30 boot-time.boot 15.27 ± 2% +12.6% 17.18 boot-time.dhcp 1449 ± 2% +10.6% 1602 boot-time.idle 1.589e+08 -16.1% 1.334e+08 cpuidle.C1.time 10004352 -12.6% 8738852 cpuidle.C1.usage 24458189 ± 3% -76.6% 5718567 ± 15% cpuidle.POLL.time 18015660 -95.3% 840897 ± 17% cpuidle.POLL.usage 42120 ± 2% +9.9% 46309 ± 5% softirqs.CPU10.SCHED 41095 ± 4% +10.9% 45593 ± 5% softirqs.CPU27.SCHED 25223 ± 54% +70.0% 42867 softirqs.CPU36.SCHED 166155 ± 8% -10.1% 149301 softirqs.CPU65.RCU 28947742 -71.8% 8152772 softirqs.NET_RX 12415372 -9.2% 11268919 softirqs.RCU 8089600 ± 11% -23.4% 6200320 ± 8% meminfo.DirectMap2M 503148 ± 8% -41.3% 295276 ± 8% meminfo.DirectMap4k 75713 +34.4% 101777 meminfo.KReclaimable 22720 +11.6% 25360 meminfo.Percpu 75713 +34.4% 101777 meminfo.SReclaimable 152955 +16.4% 178056 meminfo.SUnreclaim 228668 +22.4% 279834 meminfo.Slab 137288 +17.8% 161678 meminfo.VmallocUsed 33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.KReclaimable 33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.SReclaimable 74713 ± 16% +34.8% 100736 ± 11% numa-meminfo.node0.SUnreclaim 108029 ± 16% +47.4% 159219 ± 11% numa-meminfo.node0.Slab 101746 ± 44% +68.1% 171066 ± 12% numa-meminfo.node1.Active 101684 ± 44% +68.1% 170963 ± 12% numa-meminfo.node1.Active(anon) 60012 ± 62% +94.5% 116727 ± 18% numa-meminfo.node1.AnonHugePages 98606 ± 46% +69.1% 166702 ± 12% numa-meminfo.node1.AnonPages 5648 +3.4% 5842 proc-vmstat.nr_shmem 18920 +34.5% 25446 proc-vmstat.nr_slab_reclaimable 38238 +16.4% 44513 proc-vmstat.nr_slab_unreclaimable 895359 ± 5% -17.9% 734777 proc-vmstat.numa_hit 872027 ± 5% -18.4% 711460 proc-vmstat.numa_local 1080 ± 9% +27.4% 1375 ± 4% proc-vmstat.pgactivate 1530529 ± 15% -42.6% 878573 ± 2% proc-vmstat.pgalloc_normal 1505342 ± 15% -43.5% 849856 ± 2% proc-vmstat.pgfree 8335 ± 18% +75.4% 14620 ± 12% numa-vmstat.node0.nr_slab_reclaimable 18678 ± 16% +34.8% 25183 ± 11% numa-vmstat.node0.nr_slab_unreclaimable 647500 ± 10% -21.7% 507083 ± 8% numa-vmstat.node0.numa_hit 494986 ± 13% -27.6% 358327 ± 13% numa-vmstat.node0.numa_local 25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_active_anon 24661 ± 46% +69.0% 41679 ± 12% numa-vmstat.node1.nr_anon_pages 25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_zone_active_anon 725209 ± 8% -16.4% 606495 ± 8% numa-vmstat.node1.numa_hit 702021 ± 8% -17.9% 576047 ± 10% numa-vmstat.node1.numa_local 49724 ± 16% -42.5% 28572 ± 16% sched_debug.cfs_rq:/.load.avg 113065 ± 11% -24.3% 85601 ± 10% sched_debug.cfs_rq:/.load.stddev 57662 ± 7% -47.5% 30301 ± 9% sched_debug.cfs_rq:/.min_vruntime.max 6576 ± 8% -34.6% 4300 ± 19% sched_debug.cfs_rq:/.min_vruntime.min 10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.min_vruntime.stddev 0.16 ± 8% -29.7% 0.11 ± 8% sched_debug.cfs_rq:/.nr_running.avg 0.36 ± 2% -13.7% 0.31 ± 4% sched_debug.cfs_rq:/.nr_running.stddev 35.45 ± 20% -52.9% 16.69 sched_debug.cfs_rq:/.runnable_load_avg.avg 95.13 ± 14% -32.7% 63.99 sched_debug.cfs_rq:/.runnable_load_avg.stddev 50290 ± 14% -43.7% 28330 ± 15% sched_debug.cfs_rq:/.runnable_weight.avg 113143 ± 10% -24.9% 84975 ± 10% sched_debug.cfs_rq:/.runnable_weight.stddev 33703 ± 43% -68.0% 10796 ± 42% sched_debug.cfs_rq:/.spread0.max 10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.spread0.stddev 140.66 ± 10% -17.8% 115.68 ± 7% sched_debug.cfs_rq:/.util_avg.avg 247.04 ± 4% -11.6% 218.36 ± 4% sched_debug.cfs_rq:/.util_avg.stddev 51.66 ± 11% -41.4% 30.29 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.avg 179.93 ± 3% -26.3% 132.58 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev 1.26 ± 2% +10.1% 1.39 ± 3% sched_debug.cpu.clock.stddev 1.26 ± 2% +10.2% 1.39 ± 3% sched_debug.cpu.clock_task.stddev 365656 ± 9% -57.9% 154017 sched_debug.cpu.nr_switches.avg 3644073 ± 21% -81.6% 671977 ± 17% sched_debug.cpu.nr_switches.max 1315 ± 5% +863.7% 12673 ± 91% sched_debug.cpu.nr_switches.min 592717 ± 15% -75.6% 144437 ± 18% sched_debug.cpu.nr_switches.stddev 341.50 ± 96% +267.0% 1253 ± 7% interrupts.CPU12.RES:Rescheduling_interrupts 145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.NMI:Non-maskable_interrupts 145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.PMI:Performance_monitoring_interrupts 168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.NMI:Non-maskable_interrupts 168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.PMI:Performance_monitoring_interrupts 316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.NMI:Non-maskable_interrupts 316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3796 ± 83% -82.0% 685.00 ±119% interrupts.CPU18.RES:Rescheduling_interrupts 2353 ± 49% -84.5% 364.00 ±134% interrupts.CPU20.RES:Rescheduling_interrupts 107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.NMI:Non-maskable_interrupts 107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.PMI:Performance_monitoring_interrupts 89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.NMI:Non-maskable_interrupts 89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.PMI:Performance_monitoring_interrupts 93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.NMI:Non-maskable_interrupts 93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.PMI:Performance_monitoring_interrupts 87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.NMI:Non-maskable_interrupts 87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.PMI:Performance_monitoring_interrupts 104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.NMI:Non-maskable_interrupts 104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1951 ± 71% -80.2% 387.00 ±113% interrupts.CPU33.RES:Rescheduling_interrupts 117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.NMI:Non-maskable_interrupts 117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.PMI:Performance_monitoring_interrupts 142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.NMI:Non-maskable_interrupts 142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.PMI:Performance_monitoring_interrupts 320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.NMI:Non-maskable_interrupts 320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.PMI:Performance_monitoring_interrupts 106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.NMI:Non-maskable_interrupts 106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.PMI:Performance_monitoring_interrupts 87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.NMI:Non-maskable_interrupts 87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.PMI:Performance_monitoring_interrupts 23466 +23.8% 29040 ± 5% interrupts.NMI:Non-maskable_interrupts 23466 +23.8% 29040 ± 5% interrupts.PMI:Performance_monitoring_interrupts 13.69 ± 25% -71.4% 3.92 ± 48% perf-stat.i.MPKI 8.78e+08 +90.0% 1.668e+09 perf-stat.i.branch-instructions 2.30 ± 14% -1.5 0.83 ± 33% perf-stat.i.branch-miss-rate% 20600243 ± 12% -31.0% 14224189 ± 29% perf-stat.i.branch-misses 2.48 ± 18% +1.7 4.22 ± 14% perf-stat.i.cache-miss-rate% 57749473 ± 23% -44.2% 32231609 ± 44% perf-stat.i.cache-references 182988 -60.2% 72782 perf-stat.i.context-switches 1.65 ± 5% -39.4% 1.00 ± 5% perf-stat.i.cpi 7.078e+09 ± 3% +18.6% 8.393e+09 ± 3% perf-stat.i.cpu-cycles 85.15 -8.1% 78.28 perf-stat.i.cpu-migrations 0.09 ± 32% -0.1 0.02 ±106% perf-stat.i.dTLB-load-miss-rate% 1.284e+09 +89.1% 2.428e+09 perf-stat.i.dTLB-loads 0.01 ± 51% -0.0 0.00 ±133% perf-stat.i.dTLB-store-miss-rate% 7.676e+08 +81.0% 1.39e+09 perf-stat.i.dTLB-stores 28.71 ± 22% -19.9 8.79 ± 72% perf-stat.i.iTLB-load-miss-rate% 661655 ± 18% -65.2% 230579 ± 60% perf-stat.i.iTLB-load-misses 1672859 ± 15% +53.1% 2560787 ± 13% perf-stat.i.iTLB-loads 4.377e+09 +92.6% 8.432e+09 perf-stat.i.instructions 7111 ± 22% +568.3% 47528 ± 35% perf-stat.i.instructions-per-iTLB-miss 0.61 ± 4% +63.4% 1.00 ± 4% perf-stat.i.ipc 0.10 ± 3% +18.6% 0.12 ± 3% perf-stat.i.metric.GHz 41.51 +84.7% 76.66 perf-stat.i.metric.M/sec 131741 ± 7% +48.3% 195398 ± 7% perf-stat.i.node-load-misses 36778 ± 31% -55.6% 16347 ± 16% perf-stat.i.node-stores 13.22 ± 23% -70.9% 3.85 ± 46% perf-stat.overall.MPKI 2.35 ± 12% -1.5 0.86 ± 30% perf-stat.overall.branch-miss-rate% 2.34 ± 18% +1.9 4.20 ± 14% perf-stat.overall.cache-miss-rate% 1.62 ± 4% -38.4% 1.00 ± 5% perf-stat.overall.cpi 0.08 ± 30% -0.1 0.02 ±103% perf-stat.overall.dTLB-load-miss-rate% 0.01 ± 50% -0.0 0.00 ±132% perf-stat.overall.dTLB-store-miss-rate% 28.68 ± 22% -19.9 8.73 ± 71% perf-stat.overall.iTLB-load-miss-rate% 6894 ± 21% +579.7% 46861 ± 36% perf-stat.overall.instructions-per-iTLB-miss 0.62 ± 4% +62.5% 1.01 ± 4% perf-stat.overall.ipc 138819 +581.5% 946109 perf-stat.overall.path-length 8.751e+08 +90.0% 1.662e+09 perf-stat.ps.branch-instructions 20537517 ± 12% -30.9% 14182248 ± 29% perf-stat.ps.branch-misses 57557495 ± 23% -44.2% 32125500 ± 44% perf-stat.ps.cache-references 182374 -60.2% 72538 perf-stat.ps.context-switches 7.055e+09 ± 3% +18.6% 8.366e+09 ± 3% perf-stat.ps.cpu-cycles 84.88 -8.1% 78.04 perf-stat.ps.cpu-migrations 1.28e+09 +89.1% 2.42e+09 perf-stat.ps.dTLB-loads 7.651e+08 +81.0% 1.385e+09 perf-stat.ps.dTLB-stores 659450 ± 18% -65.2% 229817 ± 60% perf-stat.ps.iTLB-load-misses 1667327 ± 15% +53.1% 2552296 ± 13% perf-stat.ps.iTLB-loads 4.363e+09 +92.6% 8.405e+09 perf-stat.ps.instructions 131326 ± 7% +48.3% 194773 ± 7% perf-stat.ps.node-load-misses 36665 ± 31% -55.5% 16301 ± 16% perf-stat.ps.node-stores 1.334e+12 +92.5% 2.568e+12 perf-stat.total.instructions 1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.active_objs 1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.num_objs 172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.active_objs 172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.num_objs 1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.active_objs 1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.num_objs 497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.active_objs 497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.num_objs 3990 -100.0% 0.00 slabinfo.cred_jar.active_objs 3990 -100.0% 0.00 slabinfo.cred_jar.num_objs 1917 -26.9% 1400 slabinfo.dentry.active_slabs 1917 -26.9% 1400 slabinfo.dentry.num_slabs 2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.active_objs 2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.num_objs 761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.active_objs 761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.num_objs 3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.active_objs 3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.num_objs 19419 ± 6% +25.2% 24310 ± 8% slabinfo.filp.active_objs 613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.active_slabs 19628 ± 6% +24.4% 24408 ± 8% slabinfo.filp.num_objs 613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.num_slabs 2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.active_objs 2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.num_objs 1007 +58.3% 1594 slabinfo.inode_cache.active_slabs 1007 +58.3% 1594 slabinfo.inode_cache.num_slabs 2517 -35.4% 1625 slabinfo.kernfs_node_cache.active_slabs 2517 -35.4% 1625 slabinfo.kernfs_node_cache.num_slabs 4927 ± 2% -13.8% 4248 ± 4% slabinfo.kmalloc-1k.active_objs 5040 ± 2% -15.1% 4280 ± 4% slabinfo.kmalloc-1k.num_objs 4837 ± 4% +28.0% 6193 slabinfo.kmalloc-2k.active_objs 4870 ± 5% +28.1% 6237 slabinfo.kmalloc-2k.num_objs 1191 +19.1% 1419 slabinfo.kmalloc-4k.active_objs 1219 +18.3% 1442 slabinfo.kmalloc-4k.num_objs 3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.active_objs 3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.num_objs 387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.active_objs 387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.num_objs 759.00 ± 10% -36.2% 484.00 ± 5% slabinfo.kmem_cache_node.active_objs 816.00 ± 10% -39.2% 496.00 ± 5% slabinfo.kmem_cache_node.num_objs 12875 -100.0% 0.00 slabinfo.lsm_file_cache.active_objs 12875 -100.0% 0.00 slabinfo.lsm_file_cache.num_objs 2271 -27.9% 1639 slabinfo.mm_struct.active_objs 2271 -27.9% 1639 slabinfo.mm_struct.num_objs 790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.active_objs 790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.num_objs 123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.active_objs 123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.num_objs 294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.active_objs 294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.num_objs 3128 +73.2% 5418 slabinfo.pid.active_objs 3128 +73.2% 5418 slabinfo.pid.num_objs 10638 -13.0% 9252 slabinfo.proc_inode_cache.active_objs 10638 -12.9% 9266 slabinfo.proc_inode_cache.num_objs 5651 -15.6% 4771 slabinfo.shmem_inode_cache.active_objs 5651 -15.6% 4771 slabinfo.shmem_inode_cache.num_objs 2794 -11.6% 2469 slabinfo.signal_cache.active_objs 2794 -11.6% 2469 slabinfo.signal_cache.num_objs 13766 ± 10% -46.0% 7439 ± 16% slabinfo.sock_inode_cache.active_objs 13865 ± 10% -45.9% 7507 ± 16% slabinfo.sock_inode_cache.num_objs 4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.active_objs 4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.num_objs 940.25 +13.2% 1064 slabinfo.task_struct.active_objs 947.50 -43.4% 536.00 slabinfo.task_struct.active_slabs 947.50 +13.2% 1072 slabinfo.task_struct.num_objs 947.50 -43.4% 536.00 slabinfo.task_struct.num_slabs 1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.active_objs 1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.num_objs 1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.active_slabs 1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.num_slabs 38.42 ± 3% -6.9 31.49 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 38.56 ± 3% -6.9 31.66 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 13.48 ± 3% -2.6 10.86 ± 5% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack 13.07 ± 3% -2.5 10.59 ± 5% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start 14.11 ± 3% -2.3 11.79 ± 5% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip 12.53 ± 3% -2.3 10.22 ± 5% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core 12.39 ± 3% -2.3 10.12 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv 13.88 ± 3% -2.2 11.64 ± 5% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq 12.77 ± 3% -2.2 10.53 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action 14.41 ± 3% -2.2 12.18 ± 5% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb 14.41 ± 3% -2.2 12.21 ± 5% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit 14.29 ± 3% -2.1 12.15 ± 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2 12.58 ± 3% -2.1 10.46 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog 14.30 ± 3% -2.1 12.18 ± 5% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output 8.47 ± 3% -1.7 6.73 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.43 ± 3% -1.7 6.72 perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.31 ± 3% -1.6 6.68 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.00 ± 3% -1.6 7.39 ± 3% perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64 9.22 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.20 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.21 ± 3% -1.6 6.63 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64 9.07 ± 3% -1.6 7.51 ± 3% perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.81 -1.3 3.51 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.77 ± 2% -1.3 3.50 ± 3% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.98 ± 3% -1.2 5.73 perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 9.35 ± 3% -1.2 8.11 ± 4% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames 9.27 ± 3% -1.2 8.06 ± 5% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit 4.46 ± 2% -1.1 3.37 ± 3% perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.45 -1.1 3.37 ± 4% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect 4.49 ± 2% -1.1 3.42 ± 4% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect 4.05 -1.1 2.99 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect 4.52 ± 2% -1.0 3.48 ± 4% perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 4.40 ± 2% -1.0 3.36 ± 3% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64 6.03 ± 3% -0.9 5.18 ± 2% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto 6.02 ± 3% -0.8 5.17 ± 2% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 5.18 ± 4% -0.8 4.39 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release 7.17 ± 2% -0.8 6.38 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 1.03 ± 2% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg 4.58 ± 4% -0.7 3.85 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg 0.98 ± 2% -0.7 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg 3.51 ± 3% -0.7 2.80 ± 4% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock 3.77 ± 6% -0.7 3.07 ± 3% perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect 5.87 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release.sock_close 5.86 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release 4.98 ± 4% -0.6 4.33 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close 3.29 ± 3% -0.6 2.64 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process 3.42 ± 3% -0.6 2.78 ± 4% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock 3.35 ± 3% -0.6 2.71 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv 0.93 ± 6% -0.6 0.28 ±100% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable 4.42 ± 5% -0.6 3.81 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked 7.44 ± 4% -0.6 6.85 ± 6% perf-profile.calltrace.cycles-pp.tcp_close.inet_release.__sock_release.sock_close.__fput 9.05 ± 4% -0.6 8.46 ± 5% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.33 ± 5% -0.6 3.74 ± 2% perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 3.25 ± 5% -0.6 2.70 ± 3% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect 7.74 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64 7.61 ± 4% -0.5 7.07 ± 6% perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run 7.72 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_usermode_loop 0.93 ± 7% -0.5 0.40 ± 57% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.94 ± 6% -0.5 0.42 ± 58% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process 0.95 ± 7% -0.5 0.43 ± 58% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv 1.55 ± 5% -0.5 1.03 perf-profile.calltrace.cycles-pp.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 4.02 -0.5 3.50 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 2.68 ± 5% -0.5 2.20 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect 1.10 ± 3% -0.5 0.62 ± 5% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto 2.69 ± 5% -0.5 2.21 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect 1.11 ± 7% -0.5 0.64 ± 9% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 1.03 ± 5% -0.5 0.56 ± 3% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 1.15 ± 3% -0.4 0.72 ± 7% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 3.07 ± 4% -0.4 2.64 ± 3% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect 1.03 ± 6% -0.4 0.60 ± 9% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu 0.99 ± 4% -0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.93 ± 6% -0.3 0.61 ± 6% perf-profile.calltrace.cycles-pp.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv 0.88 ± 5% -0.3 0.56 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established 1.40 ± 5% -0.3 1.12 ± 3% perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 1.52 ± 10% -0.2 1.28 ± 6% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.64 ± 2% -0.1 0.53 ± 3% perf-profile.calltrace.cycles-pp.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file.__sys_accept4 0.58 ± 5% -0.1 0.53 ± 2% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file 2.08 ± 2% +0.1 2.21 ± 3% perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 0.64 ± 4% +0.3 0.93 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 1.15 ± 3% +0.3 1.44 ± 7% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.96 ± 4% +0.3 1.26 ± 10% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv 0.98 ± 4% +0.3 1.30 ± 10% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.34 ±101% +0.5 0.81 ± 18% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.35 ±101% +0.5 0.85 ± 18% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.00 +0.6 0.57 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_get_next_event.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select 0.00 +0.6 0.59 ± 12% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv 0.13 ±173% +0.6 0.72 ± 5% perf-profile.calltrace.cycles-pp.run_timer_softirq.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +0.6 0.61 ± 6% perf-profile.calltrace.cycles-pp.hrtimer_next_event_without.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.13 ±173% +0.6 0.75 ± 7% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv 0.00 +0.6 0.62 ± 4% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established 0.00 +0.6 0.62 ± 12% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.16 ±173% +0.7 0.82 ± 36% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt 0.89 ± 11% +0.7 1.58 ± 4% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle 0.13 ±173% +0.7 0.83 ± 20% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 1.42 ± 18% +0.7 2.15 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt 0.58 ± 61% +0.7 1.33 ± 35% perf-profile.calltrace.cycles-pp.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 1.05 ± 16% +1.0 2.08 ± 25% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 1.12 ± 18% +1.0 2.15 ± 27% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 0.00 +1.1 1.15 ± 13% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 2.12 ± 11% +1.2 3.28 ± 2% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 1.41 ± 13% +1.3 2.71 ± 24% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.39 ±101% +1.4 1.77 ± 38% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +1.5 1.48 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start 0.00 +1.5 1.49 ± 30% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 3.14 ± 13% +1.6 4.75 ± 11% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 1.35 ± 25% +1.8 3.14 ± 15% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +1.9 1.86 ± 6% perf-profile.calltrace.cycles-pp.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit 1.62 ± 20% +2.2 3.83 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 3.29 ± 11% +2.4 5.72 ± 4% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 3.98 ± 7% +2.9 6.83 ± 2% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.40 ± 14% +3.9 6.30 ± 24% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 4.14 ± 4% +6.5 10.63 ± 11% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 48.02 ± 4% +7.9 55.92 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 47.19 ± 4% +8.0 55.23 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 56.36 ± 2% +8.1 64.47 perf-profile.calltrace.cycles-pp.secondary_startup_64 54.68 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 54.66 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 54.62 ± 3% +8.7 63.31 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 10.59 ± 2% +10.3 20.89 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 8.93 ± 2% +10.6 19.50 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 38.73 ± 3% -6.7 32.02 ± 2% perf-profile.children.cycles-pp.do_syscall_64 38.86 ± 3% -6.7 32.19 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 13.52 ± 3% -2.7 10.87 ± 5% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 13.11 ± 3% -2.5 10.60 ± 5% perf-profile.children.cycles-pp.ip_rcv 17.80 ± 3% -2.5 15.33 ± 4% perf-profile.children.cycles-pp.__tcp_transmit_skb 13.39 ± 2% -2.4 10.95 ± 4% perf-profile.children.cycles-pp.tcp_v4_do_rcv 2.72 -2.4 0.35 ± 4% perf-profile.children.cycles-pp.poll_idle 14.15 ± 3% -2.3 11.81 ± 5% perf-profile.children.cycles-pp.net_rx_action 12.56 ± 3% -2.3 10.22 ± 5% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu 12.46 ± 3% -2.3 10.12 ± 5% perf-profile.children.cycles-pp.tcp_v4_rcv 12.80 ± 3% -2.3 10.53 ± 5% perf-profile.children.cycles-pp.ip_local_deliver 13.93 ± 3% -2.3 11.66 ± 5% perf-profile.children.cycles-pp.process_backlog 14.46 ± 3% -2.2 12.24 ± 5% perf-profile.children.cycles-pp.do_softirq 12.61 ± 3% -2.1 10.46 ± 5% perf-profile.children.cycles-pp.ip_local_deliver_finish 14.34 ± 3% -2.1 12.20 ± 5% perf-profile.children.cycles-pp.do_softirq_own_stack 16.07 ± 3% -1.8 14.25 ± 4% perf-profile.children.cycles-pp.ip_finish_output2 16.42 ± 3% -1.8 14.64 ± 4% perf-profile.children.cycles-pp.ip_output 16.86 ± 3% -1.8 15.08 ± 4% perf-profile.children.cycles-pp.__ip_queue_xmit 8.47 ± 3% -1.7 6.73 perf-profile.children.cycles-pp.__x64_sys_sendto 14.60 ± 3% -1.7 12.89 ± 4% perf-profile.children.cycles-pp.__local_bh_enable_ip 8.43 ± 3% -1.7 6.72 perf-profile.children.cycles-pp.__sys_sendto 8.32 ± 3% -1.6 6.68 perf-profile.children.cycles-pp.sock_sendmsg 9.00 ± 3% -1.6 7.39 ± 3% perf-profile.children.cycles-pp.__inet_stream_connect 9.22 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__x64_sys_connect 8.22 ± 3% -1.6 6.63 perf-profile.children.cycles-pp.tcp_sendmsg 9.20 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__sys_connect 9.08 ± 3% -1.6 7.51 ± 3% perf-profile.children.cycles-pp.inet_stream_connect 6.35 ± 2% -1.5 4.81 ± 3% perf-profile.children.cycles-pp.__release_sock 11.90 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.__tcp_push_pending_frames 11.88 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.tcp_write_xmit 4.82 ± 2% -1.3 3.51 ± 3% perf-profile.children.cycles-pp.__x64_sys_recvfrom 4.77 ± 2% -1.3 3.50 ± 3% perf-profile.children.cycles-pp.__sys_recvfrom 8.17 ± 3% -1.3 6.92 ± 5% perf-profile.children.cycles-pp.tcp_rcv_state_process 6.99 ± 3% -1.3 5.73 perf-profile.children.cycles-pp.tcp_sendmsg_locked 6.10 ± 2% -1.2 4.88 ± 3% perf-profile.children.cycles-pp.release_sock 1.47 ± 5% -1.2 0.32 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64 1.45 ± 7% -1.1 0.35 ± 10% perf-profile.children.cycles-pp.syscall_return_via_sysret 5.09 -1.1 3.99 ± 4% perf-profile.children.cycles-pp.tcp_rcv_established 4.46 ± 2% -1.1 3.37 ± 3% perf-profile.children.cycles-pp.inet_recvmsg 4.41 ± 2% -1.0 3.36 ± 3% perf-profile.children.cycles-pp.tcp_recvmsg 1.83 ± 8% -1.0 0.83 ± 14% perf-profile.children.cycles-pp.tcp_ack 0.91 ± 8% -0.7 0.20 ± 18% perf-profile.children.cycles-pp.__inet_lookup_established 3.77 ± 6% -0.7 3.08 ± 3% perf-profile.children.cycles-pp.tcp_connect 2.63 ± 2% -0.6 2.00 ± 6% perf-profile.children.cycles-pp.try_to_wake_up 2.75 ± 2% -0.6 2.13 ± 6% perf-profile.children.cycles-pp.__wake_up_common 1.19 ± 6% -0.6 0.59 ± 11% perf-profile.children.cycles-pp.read_tsc 7.45 ± 4% -0.6 6.86 ± 6% perf-profile.children.cycles-pp.tcp_close 9.06 ± 4% -0.6 8.47 ± 5% perf-profile.children.cycles-pp.exit_to_usermode_loop 4.33 ± 5% -0.6 3.74 ± 2% perf-profile.children.cycles-pp.tcp_v4_connect 3.03 ± 2% -0.6 2.45 ± 6% perf-profile.children.cycles-pp.__wake_up_common_lock 7.75 ± 4% -0.6 7.19 ± 5% perf-profile.children.cycles-pp.sock_close 7.72 ± 4% -0.5 7.19 ± 5% perf-profile.children.cycles-pp.__sock_release 7.61 ± 4% -0.5 7.07 ± 6% perf-profile.children.cycles-pp.inet_release 1.56 ± 6% -0.5 1.04 perf-profile.children.cycles-pp.tcp_child_process 0.98 ± 6% -0.4 0.53 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair 0.95 ± 6% -0.4 0.54 ± 11% perf-profile.children.cycles-pp.autoremove_wake_function 0.68 ± 5% -0.4 0.27 ± 11% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 2.34 ± 3% -0.4 1.94 ± 6% perf-profile.children.cycles-pp.sock_def_readable 0.90 ± 4% -0.4 0.52 ± 7% perf-profile.children.cycles-pp.__kfree_skb 0.98 ± 5% -0.4 0.60 ± 9% perf-profile.children.cycles-pp.activate_task 0.96 ± 8% -0.4 0.58 ± 17% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 0.99 ± 5% -0.4 0.64 ± 9% perf-profile.children.cycles-pp.ttwu_do_activate 0.78 ± 5% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.enqueue_entity 1.06 ± 5% -0.3 0.73 ± 6% perf-profile.children.cycles-pp.sock_def_wakeup 0.39 ± 10% -0.3 0.11 ± 7% perf-profile.children.cycles-pp.skb_release_all 1.40 ± 5% -0.3 1.12 ± 3% perf-profile.children.cycles-pp.tcp_conn_request 0.53 ± 9% -0.3 0.27 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.36 ± 10% -0.3 0.10 ± 10% perf-profile.children.cycles-pp.skb_release_head_state 0.35 ± 12% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.skb_release_data 0.48 ± 13% -0.2 0.24 ± 13% perf-profile.children.cycles-pp.__skb_datagram_iter 0.29 ± 10% -0.2 0.06 ± 63% perf-profile.children.cycles-pp.dst_release 1.59 ± 9% -0.2 1.35 ± 4% perf-profile.children.cycles-pp.schedule_idle 0.34 ± 11% -0.2 0.11 ± 17% perf-profile.children.cycles-pp.__switch_to 0.48 ± 12% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.skb_copy_datagram_iter 0.27 ± 9% -0.2 0.04 ± 58% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.55 ± 9% -0.2 0.32 ± 11% perf-profile.children.cycles-pp.set_next_entity 0.69 ± 4% -0.2 0.47 ± 6% perf-profile.children.cycles-pp.native_write_msr 0.29 ± 20% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.ipv4_mtu 0.27 ± 6% -0.2 0.06 ± 14% perf-profile.children.cycles-pp.call_rcu 0.45 ± 11% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.sched_clock 1.13 ± 4% -0.2 0.93 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_bh 0.51 ± 7% -0.2 0.30 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair 0.71 ± 3% -0.2 0.52 ± 3% perf-profile.children.cycles-pp.sk_reset_timer 0.50 ± 5% -0.2 0.31 ± 6% perf-profile.children.cycles-pp.tcp_event_new_data_sent 0.29 ± 5% -0.2 0.10 ± 23% perf-profile.children.cycles-pp.__slab_free 0.90 ± 4% -0.2 0.71 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair 0.24 ± 11% -0.2 0.06 ± 26% perf-profile.children.cycles-pp.ipv4_dst_check 0.67 ± 13% -0.2 0.50 ± 2% perf-profile.children.cycles-pp.__next_timer_interrupt 0.22 ± 6% -0.2 0.05 ± 58% perf-profile.children.cycles-pp.__ksize 0.40 ± 14% -0.2 0.23 ± 8% perf-profile.children.cycles-pp.native_sched_clock 0.19 ± 15% -0.2 0.03 ±102% perf-profile.children.cycles-pp.nf_hook_slow 0.20 ± 18% -0.2 0.04 ± 60% perf-profile.children.cycles-pp.sock_rfree 0.22 ± 3% -0.2 0.07 ± 19% perf-profile.children.cycles-pp.sockfd_lookup_light 0.23 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.__switch_to_asm 0.52 ± 5% -0.1 0.38 ± 2% perf-profile.children.cycles-pp.lapic_next_deadline 0.21 ± 9% -0.1 0.07 ± 16% perf-profile.children.cycles-pp.__check_object_size 0.47 ± 15% -0.1 0.33 ± 7% perf-profile.children.cycles-pp._find_next_bit 0.17 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp._cond_resched 0.25 ± 20% -0.1 0.12 ± 23% perf-profile.children.cycles-pp.__x64_sys_close 0.27 ± 18% -0.1 0.14 ± 38% perf-profile.children.cycles-pp.update_ts_time_stats 0.50 ± 13% -0.1 0.36 ± 13% perf-profile.children.cycles-pp.tcp_create_openreq_child 0.17 ± 9% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.validate_xmit_skb 0.48 ± 9% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.update_rq_clock 0.18 ± 16% -0.1 0.05 ± 58% perf-profile.children.cycles-pp.__might_sleep 0.21 ± 14% -0.1 0.08 ± 59% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit 0.23 ± 10% -0.1 0.10 ± 19% perf-profile.children.cycles-pp.rcu_eqs_exit 0.25 ± 11% -0.1 0.13 ± 22% perf-profile.children.cycles-pp.nr_iowait_cpu 0.19 ± 11% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.__fget_light 0.21 ± 15% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.run_ksoftirqd 0.20 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.select_idle_sibling 0.20 ± 24% -0.1 0.08 ± 74% perf-profile.children.cycles-pp.wake_up_klogd_work_func 0.15 ± 10% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.update_cfs_group 0.21 ± 23% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.36 ± 6% -0.1 0.25 ± 16% perf-profile.children.cycles-pp.__netif_receive_skb_core 0.23 ± 19% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.smpboot_thread_fn 0.44 ± 7% -0.1 0.34 ± 4% perf-profile.children.cycles-pp.tcp_send_fin 1.51 ± 5% -0.1 1.41 ± 5% perf-profile.children.cycles-pp.wait_woken 0.14 ± 19% -0.1 0.03 ±100% perf-profile.children.cycles-pp.simple_copy_to_iter 0.16 ± 29% -0.1 0.06 ± 66% perf-profile.children.cycles-pp.rb_next 0.36 ± 12% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.inet_csk_destroy_sock 0.27 ± 13% -0.1 0.17 ± 16% perf-profile.children.cycles-pp.__sk_dst_check 0.76 ± 7% -0.1 0.67 ± 6% perf-profile.children.cycles-pp.dequeue_entity 0.15 ± 12% -0.1 0.06 ± 58% perf-profile.children.cycles-pp.sk_forced_mem_schedule 0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.inet_reqsk_alloc 0.27 ± 6% -0.1 0.18 ± 15% perf-profile.children.cycles-pp.memcpy_erms 0.37 ± 11% -0.1 0.28 ± 15% perf-profile.children.cycles-pp.inet_csk_clone_lock 0.14 ± 18% -0.1 0.05 ± 64% perf-profile.children.cycles-pp.inet_sk_rebuild_header 0.42 ± 8% -0.1 0.34 ± 16% perf-profile.children.cycles-pp.native_irq_return_iret 0.18 ± 10% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.rcu_eqs_enter 0.11 ± 24% -0.1 0.04 ±107% perf-profile.children.cycles-pp.native_apic_mem_write 0.15 ± 25% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.tcp_current_mss 0.19 ± 10% -0.1 0.12 ± 17% perf-profile.children.cycles-pp._copy_to_iter 0.11 ± 14% -0.1 0.04 ± 60% perf-profile.children.cycles-pp.reweight_entity 0.16 ± 23% -0.1 0.09 ± 37% perf-profile.children.cycles-pp.menu_reflect 0.12 ± 8% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.sock_recvmsg 0.12 ± 29% -0.1 0.05 ± 67% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust 0.12 ± 19% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.tcp_release_cb 0.10 ± 23% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.run_local_timers 0.25 ± 12% -0.1 0.19 ± 14% perf-profile.children.cycles-pp.__tcp_send_ack 0.09 ± 25% -0.1 0.03 ±100% perf-profile.children.cycles-pp.rb_insert_color 0.26 ± 5% -0.1 0.20 ± 10% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.12 ± 3% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.security_socket_recvmsg 0.12 ± 28% -0.1 0.07 ± 31% perf-profile.children.cycles-pp.arch_cpu_idle_enter 0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.tcp_add_backlog 0.12 ± 13% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.security_sk_alloc 0.11 ± 13% -0.1 0.06 ± 20% perf-profile.children.cycles-pp.apparmor_sk_alloc_security 0.19 ± 13% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.tcp_make_synack 0.07 ± 11% -0.0 0.03 ±102% perf-profile.children.cycles-pp.security_socket_getsockopt 0.20 ± 6% -0.0 0.16 ± 10% perf-profile.children.cycles-pp.sk_prot_alloc 0.10 ± 27% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.tcp_get_metrics 0.11 ± 17% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__update_load_avg_se 0.11 ± 14% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.pick_next_entity 0.11 ± 9% -0.0 0.08 ± 23% perf-profile.children.cycles-pp.inode_init_always 0.15 ± 12% -0.0 0.12 ± 15% perf-profile.children.cycles-pp.remove_wait_queue 0.07 ± 16% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.read 0.06 ± 14% +0.0 0.09 ± 17% perf-profile.children.cycles-pp.forkshell 0.07 ± 13% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.security_sk_free 0.06 +0.0 0.10 ± 15% perf-profile.children.cycles-pp.inet_csk_reqsk_queue_drop 0.14 ± 7% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.evict 0.11 ± 6% +0.0 0.15 ± 12% perf-profile.children.cycles-pp.sock_alloc_inode 0.06 ± 66% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.add_wait_queue 0.03 ±100% +0.1 0.08 ± 16% perf-profile.children.cycles-pp.tick_sched_do_timer 0.08 ± 17% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.try_to_del_timer_sync 0.04 ± 58% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.down_write 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_group_exit 0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_exit 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.alloc_set_pte 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.bpf_sk_storage_free 0.09 ± 38% +0.1 0.15 ± 19% perf-profile.children.cycles-pp.execve 0.01 ±173% +0.1 0.07 ± 10% perf-profile.children.cycles-pp.update_rt_rq_load_avg 0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.sock_owned_by_me 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.clear_inode 0.01 ±173% +0.1 0.07 ± 22% perf-profile.children.cycles-pp.__sk_free 0.00 +0.1 0.06 ± 17% perf-profile.children.cycles-pp.__fd_install 0.18 ± 15% +0.1 0.25 ± 12% perf-profile.children.cycles-pp.del_timer 0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.__libc_start_main 0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.main 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.update_dl_rq_load_avg 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.rcu_read_lock_bh_held 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.lockdep_init_map 0.00 +0.1 0.07 ± 25% perf-profile.children.cycles-pp.netlink_has_listeners 0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.ksys_read 0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.vfs_read 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.register_lock_class 0.00 +0.1 0.07 ± 21% perf-profile.children.cycles-pp.inet_twsk_hashdance 0.32 ± 9% +0.1 0.39 ± 5% perf-profile.children.cycles-pp.alloc_empty_file 0.01 ±173% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.cmd_stat 0.00 +0.1 0.07 ± 15% perf-profile.children.cycles-pp.inet_csk_init_xmit_timers 0.32 ± 6% +0.1 0.39 ± 9% perf-profile.children.cycles-pp.inet_create 0.31 ± 11% +0.1 0.38 ± 6% perf-profile.children.cycles-pp.__alloc_file 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.do_filp_open 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.path_openat 0.01 ±173% +0.1 0.09 ± 7% perf-profile.children.cycles-pp.__note_gp_changes 0.00 +0.1 0.08 ± 20% perf-profile.children.cycles-pp.trace_hardirqs_off_caller 0.33 ± 6% +0.1 0.41 ± 9% perf-profile.children.cycles-pp.tcp_done 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.inet_sk_state_store 0.14 ± 7% +0.1 0.22 ± 13% perf-profile.children.cycles-pp.sk_alloc 0.06 ± 13% +0.1 0.14 ± 32% perf-profile.children.cycles-pp.rcu_gp_kthread 0.03 ±100% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.tcp_update_metrics 0.04 ± 59% +0.1 0.12 ± 27% perf-profile.children.cycles-pp._copy_from_user 0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.inet_put_port 0.16 ± 17% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__x64_sys_setsockopt 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.detach_if_pending 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.cgroup_sk_alloc 0.01 ±173% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.enqueue_timer 0.21 ± 8% +0.1 0.30 ± 7% perf-profile.children.cycles-pp.__inet_bind 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_open 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_openat2 0.15 ± 14% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__sys_setsockopt 0.03 ±100% +0.1 0.12 ± 11% perf-profile.children.cycles-pp.filemap_map_pages 0.01 ±173% +0.1 0.10 ± 4% perf-profile.children.cycles-pp.__update_idle_core 0.00 +0.1 0.09 ± 27% perf-profile.children.cycles-pp.ret_from_intr 0.17 ± 10% +0.1 0.26 ± 9% perf-profile.children.cycles-pp.newidle_balance 0.00 +0.1 0.10 ± 11% perf-profile.children.cycles-pp.init_timer_key 0.03 ±100% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.pick_next_task_idle 0.00 +0.1 0.10 ± 21% perf-profile.children.cycles-pp.do_raw_spin_trylock 0.00 +0.1 0.10 ± 18% perf-profile.children.cycles-pp._raw_write_lock_bh 0.01 ±173% +0.1 0.12 ± 3% perf-profile.children.cycles-pp.set_next_task_idle 0.00 +0.1 0.11 ± 20% perf-profile.children.cycles-pp.restore_regs_and_return_to_kernel 0.15 ± 10% +0.1 0.26 ± 16% perf-profile.children.cycles-pp.inet_csk_route_req 0.09 ± 8% +0.1 0.21 ± 5% perf-profile.children.cycles-pp.sock_setsockopt 0.24 ± 10% +0.1 0.36 ± 6% perf-profile.children.cycles-pp.__kmalloc_reserve 0.22 ± 8% +0.1 0.33 ± 9% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.00 +0.1 0.12 ± 32% perf-profile.children.cycles-pp.__local_bh_enable 2.08 ± 2% +0.1 2.21 ± 3% perf-profile.children.cycles-pp.sk_wait_data 0.39 ± 7% +0.1 0.51 ± 9% perf-profile.children.cycles-pp.tcp_mstamp_refresh 0.09 ± 20% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.tcp_set_state 0.09 ± 26% +0.1 0.22 ± 11% perf-profile.children.cycles-pp.del_timer_sync 0.03 ±100% +0.1 0.15 ± 14% perf-profile.children.cycles-pp.cpuacct_charge 0.07 ± 62% +0.1 0.20 ± 6% perf-profile.children.cycles-pp.__handle_mm_fault 0.06 ± 61% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.handle_pte_fault 0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch 0.00 +0.1 0.14 ± 7% perf-profile.children.cycles-pp.trace_softirqs_on 0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.lock_unpin_lock 0.34 ± 19% +0.1 0.48 ± 10% perf-profile.children.cycles-pp.enqueue_to_backlog 0.09 ± 8% +0.1 0.24 ± 11% perf-profile.children.cycles-pp.dput 0.20 ± 13% +0.1 0.35 ± 7% perf-profile.children.cycles-pp.__kmalloc_node_track_caller 0.00 +0.1 0.15 ± 29% perf-profile.children.cycles-pp.trace_softirqs_off 0.00 +0.2 0.15 ± 12% perf-profile.children.cycles-pp.lockdep_rtnl_is_held 0.00 +0.2 0.15 ± 14% perf-profile.children.cycles-pp.trace_hardirqs_on_thunk 0.23 ± 12% +0.2 0.39 ± 11% perf-profile.children.cycles-pp.__sk_destruct 0.07 ± 64% +0.2 0.22 ± 4% perf-profile.children.cycles-pp.handle_mm_fault 0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.__raw_spin_lock_init 0.00 +0.2 0.16 ± 15% perf-profile.children.cycles-pp._raw_spin_unlock_bh 0.00 +0.2 0.16 ± 13% perf-profile.children.cycles-pp.lock_pin_lock 0.06 ± 11% +0.2 0.22 ± 14% perf-profile.children.cycles-pp.trigger_load_balance 0.10 ± 28% +0.2 0.26 ± 4% perf-profile.children.cycles-pp.do_page_fault 0.22 ± 21% +0.2 0.39 ± 15% perf-profile.children.cycles-pp.fib_table_lookup 0.11 ± 31% +0.2 0.28 ± 6% perf-profile.children.cycles-pp.page_fault 0.16 ± 7% +0.2 0.33 ± 9% perf-profile.children.cycles-pp.sk_filter_trim_cap 0.16 ± 49% +0.2 0.34 ± 29% perf-profile.children.cycles-pp.enqueue_hrtimer 0.00 +0.2 0.18 ± 14% perf-profile.children.cycles-pp._raw_spin_lock_nested 0.19 ± 26% +0.2 0.38 ± 25% perf-profile.children.cycles-pp.finish_task_switch 0.09 ± 21% +0.2 0.29 ± 9% perf-profile.children.cycles-pp.___perf_sw_event 0.19 ± 14% +0.2 0.40 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.00 +0.2 0.21 ± 31% perf-profile.children.cycles-pp.trace_hardirqs_off_thunk 0.00 +0.2 0.21 ± 11% perf-profile.children.cycles-pp.tracer_hardirqs_on 0.11 ± 15% +0.2 0.33 ± 12% perf-profile.children.cycles-pp.raise_softirq 0.18 ± 4% +0.2 0.40 ± 8% perf-profile.children.cycles-pp.kfree 0.31 +0.2 0.53 ± 14% perf-profile.children.cycles-pp.ktime_get_with_offset 1.40 ± 5% +0.2 1.63 ± 2% perf-profile.children.cycles-pp.dev_hard_start_xmit 0.39 ± 9% +0.3 0.65 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc 0.22 ± 37% +0.3 0.49 ± 9% perf-profile.children.cycles-pp.run_rebalance_domains 0.00 +0.3 0.27 ± 12% perf-profile.children.cycles-pp.tracer_hardirqs_off 0.18 ± 26% +0.3 0.46 ± 8% perf-profile.children.cycles-pp.update_blocked_averages 0.27 ± 8% +0.3 0.54 ± 3% perf-profile.children.cycles-pp.update_curr 0.00 +0.3 0.28 ± 6% perf-profile.children.cycles-pp.debug_check_no_locks_freed 0.00 +0.3 0.29 ± 14% perf-profile.children.cycles-pp.do_raw_spin_unlock 0.00 +0.3 0.32 ± 10% perf-profile.children.cycles-pp.reacquire_held_locks 0.39 ± 10% +0.4 0.74 ± 13% perf-profile.children.cycles-pp.kmem_cache_free 0.31 ± 7% +0.4 0.67 ± 5% perf-profile.children.cycles-pp.ip_route_output_flow 0.39 ± 12% +0.4 0.75 ± 7% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu 0.48 ± 51% +0.4 0.85 ± 18% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.00 +0.4 0.38 ± 13% perf-profile.children.cycles-pp.__local_bh_disable_ip 0.62 ± 11% +0.4 1.01 ± 4% perf-profile.children.cycles-pp.netif_rx_internal 0.00 +0.4 0.40 ± 10% perf-profile.children.cycles-pp.trace_hardirqs_on 2.03 ± 6% +0.4 2.44 ± 4% perf-profile.children.cycles-pp.__dev_queue_xmit 0.33 ± 28% +0.4 0.75 ± 40% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.4 0.42 ± 7% perf-profile.children.cycles-pp.fs_reclaim_acquire 0.20 ± 6% +0.4 0.63 ± 6% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.00 +0.4 0.43 ± 21% perf-profile.children.cycles-pp._raw_spin_unlock_irq 0.43 ± 11% +0.4 0.87 ± 6% perf-profile.children.cycles-pp.ip_route_output_key_hash 0.47 ± 15% +0.4 0.91 ± 20% perf-profile.children.cycles-pp.scheduler_tick 2.42 +0.4 2.87 ± 7% perf-profile.children.cycles-pp.tcp_data_queue 0.13 ± 20% +0.4 0.58 ± 10% perf-profile.children.cycles-pp.__might_fault 0.13 ± 18% +0.5 0.58 ± 8% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.28 ± 65% +0.5 0.74 ± 5% perf-profile.children.cycles-pp.run_timer_softirq 0.00 +0.6 0.58 ± 5% perf-profile.children.cycles-pp.rcu_is_watching 0.64 ± 10% +0.6 1.23 ± 4% perf-profile.children.cycles-pp.netif_rx 0.70 ± 34% +0.7 1.36 ± 34% perf-profile.children.cycles-pp.irq_enter 0.92 ± 11% +0.7 1.60 ± 4% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.49 ± 10% +0.8 1.25 ± 3% perf-profile.children.cycles-pp.___might_sleep 0.00 +0.8 0.79 ± 5% perf-profile.children.cycles-pp.trace_hardirqs_off 0.00 +0.8 0.82 ± 5% perf-profile.children.cycles-pp.find_held_lock 0.22 ± 8% +0.9 1.08 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq 1.29 ± 16% +0.9 2.23 ± 25% perf-profile.children.cycles-pp.update_process_times 1.35 ± 16% +0.9 2.29 ± 26% perf-profile.children.cycles-pp.tick_sched_handle 0.26 ± 31% +0.9 1.20 ± 13% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.00 +1.0 1.03 ± 2% perf-profile.children.cycles-pp.mark_held_locks 0.00 +1.0 1.04 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock 0.70 ± 14% +1.1 1.76 ± 4% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.49 ± 11% +1.1 1.58 ± 30% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.00 +1.1 1.15 ± 11% perf-profile.children.cycles-pp.lockdep_hardirqs_off 2.52 ± 10% +1.2 3.70 ± 2% perf-profile.children.cycles-pp.rcu_core 0.00 +1.2 1.19 ± 3% perf-profile.children.cycles-pp.match_held_lock 1.65 ± 14% +1.3 2.90 ± 23% perf-profile.children.cycles-pp.tick_sched_timer 0.00 +1.4 1.37 ± 11% perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online 3.23 ± 13% +1.6 4.83 ± 11% perf-profile.children.cycles-pp.menu_select 0.00 +1.7 1.68 ± 3% perf-profile.children.cycles-pp.rcu_read_lock_held 2.81 ± 9% +1.7 4.56 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +1.8 1.76 ± 4% perf-profile.children.cycles-pp.lockdep_hardirqs_on 0.00 +1.8 1.79 ± 12% perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled 1.40 ± 24% +1.8 3.21 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event 0.00 +2.2 2.19 perf-profile.children.cycles-pp.mark_lock 1.68 ± 20% +2.2 3.88 ± 13% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +2.5 2.46 ± 9% perf-profile.children.cycles-pp.__lock_text_start 0.00 +2.7 2.65 ± 4% perf-profile.children.cycles-pp.rcu_read_lock_held_common 4.25 ± 6% +2.9 7.11 ± 2% perf-profile.children.cycles-pp.irq_exit 0.00 +3.0 2.97 ± 4% perf-profile.children.cycles-pp.do_raw_spin_lock 2.18 ± 33% +3.4 5.57 ± 4% perf-profile.children.cycles-pp.ktime_get 2.75 ± 16% +3.9 6.65 ± 25% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.00 +4.7 4.71 ± 3% perf-profile.children.cycles-pp.lock_release 0.00 +4.7 4.74 perf-profile.children.cycles-pp.rcu_read_lock_sched_held 0.00 +6.2 6.18 ± 2% perf-profile.children.cycles-pp.lock_is_held_type 4.58 ± 8% +6.6 11.14 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt 49.50 ± 3% +7.4 56.87 perf-profile.children.cycles-pp.cpuidle_enter 49.48 ± 3% +7.4 56.85 perf-profile.children.cycles-pp.cpuidle_enter_state 56.39 ± 2% +8.1 64.48 perf-profile.children.cycles-pp.do_idle 56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.secondary_startup_64 56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.cpu_startup_entry 54.68 ± 3% +8.7 63.34 perf-profile.children.cycles-pp.start_secondary 10.58 ± 4% +10.5 21.06 ± 9% perf-profile.children.cycles-pp.apic_timer_interrupt 9.66 ± 4% +10.6 20.30 ± 9% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 0.00 +10.8 10.77 ± 3% perf-profile.children.cycles-pp.__lock_acquire 0.00 +14.2 14.15 ± 2% perf-profile.children.cycles-pp.lock_acquire 2.65 ± 2% -2.3 0.32 ± 8% perf-profile.self.cycles-pp.poll_idle 2.35 ± 8% -1.8 0.58 ± 6% perf-profile.self.cycles-pp.do_syscall_64 1.62 ± 6% -1.5 0.12 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.47 ± 6% -1.1 0.32 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64 1.45 ± 7% -1.1 0.35 ± 10% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.81 ± 9% -0.6 0.17 ± 16% perf-profile.self.cycles-pp.__inet_lookup_established 1.14 ± 5% -0.6 0.56 ± 13% perf-profile.self.cycles-pp.read_tsc 1.26 ± 15% -0.6 0.70 ± 21% perf-profile.self.cycles-pp.menu_select 0.81 ± 8% -0.5 0.32 ± 4% perf-profile.self.cycles-pp.__sched_text_start 0.56 ± 10% -0.4 0.17 ± 15% perf-profile.self.cycles-pp.tcp_recvmsg 0.63 ± 4% -0.4 0.25 ± 14% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.50 ± 10% -0.4 0.14 ± 19% perf-profile.self.cycles-pp.__tcp_transmit_skb 0.46 ± 10% -0.4 0.11 ± 7% perf-profile.self.cycles-pp.tcp_ack 0.42 ± 12% -0.3 0.11 ± 12% perf-profile.self.cycles-pp.__dev_queue_xmit 0.47 ± 6% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.tcp_v4_rcv 0.50 ± 14% -0.3 0.22 ± 7% perf-profile.self.cycles-pp.do_idle 0.57 ± 7% -0.3 0.29 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.48 ± 12% -0.3 0.21 ± 4% perf-profile.self.cycles-pp.___might_sleep 0.35 ± 12% -0.3 0.09 ± 18% perf-profile.self.cycles-pp.skb_release_data 0.36 ± 5% -0.3 0.10 ± 11% perf-profile.self.cycles-pp.__netif_receive_skb_core 0.38 ± 11% -0.3 0.12 ± 9% perf-profile.self.cycles-pp.set_next_entity 0.29 ± 17% -0.2 0.04 ± 60% perf-profile.self.cycles-pp.sock_def_readable 0.31 ± 12% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.tcp_clean_rtx_queue 0.26 ± 11% -0.2 0.03 ±100% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.29 ± 12% -0.2 0.06 ± 64% perf-profile.self.cycles-pp.dst_release 0.35 ± 10% -0.2 0.12 ± 5% perf-profile.self.cycles-pp.ip_finish_output2 0.69 ± 4% -0.2 0.47 ± 6% perf-profile.self.cycles-pp.native_write_msr 0.32 ± 9% -0.2 0.10 ± 19% perf-profile.self.cycles-pp.__switch_to 0.35 ± 14% -0.2 0.14 ± 21% perf-profile.self.cycles-pp.update_rq_clock 0.29 ± 8% -0.2 0.09 ± 12% perf-profile.self.cycles-pp.__ip_queue_xmit 0.27 ± 20% -0.2 0.07 ± 10% perf-profile.self.cycles-pp.ipv4_mtu 1.03 ± 7% -0.2 0.83 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state 0.24 ± 8% -0.2 0.05 ± 60% perf-profile.self.cycles-pp.tcp_sendmsg_locked 0.28 ± 5% -0.2 0.10 ± 23% perf-profile.self.cycles-pp.__slab_free 0.27 ± 7% -0.2 0.09 ± 21% perf-profile.self.cycles-pp.select_task_rq_fair 0.22 ± 6% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.__ksize 0.20 ± 20% -0.2 0.03 ±105% perf-profile.self.cycles-pp.net_rx_action 0.39 ± 15% -0.2 0.22 ± 9% perf-profile.self.cycles-pp.native_sched_clock 0.21 ± 8% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.aa_sk_perm 0.24 ± 9% -0.2 0.07 ± 15% perf-profile.self.cycles-pp.loopback_xmit 0.22 ± 12% -0.2 0.06 ± 16% perf-profile.self.cycles-pp.enqueue_entity 0.25 ± 10% -0.2 0.09 ± 17% perf-profile.self.cycles-pp.process_backlog 0.22 ± 21% -0.2 0.06 ± 20% perf-profile.self.cycles-pp.fib_table_lookup 0.19 ± 16% -0.2 0.04 ±107% perf-profile.self.cycles-pp.ipv4_dst_check 0.23 ± 14% -0.2 0.08 ± 21% perf-profile.self.cycles-pp.tcp_rcv_established 0.19 ± 23% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.enqueue_to_backlog 0.23 ± 11% -0.1 0.08 ± 23% perf-profile.self.cycles-pp.__switch_to_asm 0.19 ± 16% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.sock_rfree 0.35 ± 14% -0.1 0.20 ± 17% perf-profile.self.cycles-pp.note_gp_changes 0.46 ± 14% -0.1 0.32 ± 6% perf-profile.self.cycles-pp._find_next_bit 0.21 ± 18% -0.1 0.08 ± 27% perf-profile.self.cycles-pp.kmem_cache_alloc 0.21 ± 23% -0.1 0.07 ± 17% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.21 ± 14% -0.1 0.08 ± 59% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit 0.19 ± 11% -0.1 0.07 ± 13% perf-profile.self.cycles-pp.__fget_light 0.16 ± 16% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.__might_sleep 0.24 ± 12% -0.1 0.13 ± 22% perf-profile.self.cycles-pp.nr_iowait_cpu 0.16 ± 2% -0.1 0.05 ± 60% perf-profile.self.cycles-pp.tcp_rcv_state_process 0.14 ± 24% -0.1 0.03 ±100% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.15 ± 10% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.update_cfs_group 0.17 ± 8% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.tcp_write_xmit 0.14 ± 13% -0.1 0.04 ±100% perf-profile.self.cycles-pp.__kmalloc_node_track_caller 0.19 ± 18% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.17 ± 11% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.mod_timer 0.17 ± 17% -0.1 0.08 ± 20% perf-profile.self.cycles-pp.ip_local_deliver 0.14 ± 15% -0.1 0.05 ± 58% perf-profile.self.cycles-pp.sk_forced_mem_schedule 0.27 ± 6% -0.1 0.18 ± 13% perf-profile.self.cycles-pp.memcpy_erms 0.18 ± 8% -0.1 0.09 ± 17% perf-profile.self.cycles-pp.ip_output 0.14 ± 27% -0.1 0.06 ± 66% perf-profile.self.cycles-pp.rb_next 0.18 ± 10% -0.1 0.10 ± 15% perf-profile.self.cycles-pp.rcu_eqs_enter 0.42 ± 8% -0.1 0.34 ± 16% perf-profile.self.cycles-pp.native_irq_return_iret 0.30 ± 14% -0.1 0.22 ± 11% perf-profile.self.cycles-pp.__softirqentry_text_start 0.14 ± 8% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.kmem_cache_alloc_node 0.11 ± 14% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.reweight_entity 0.18 ± 6% -0.1 0.11 ± 21% perf-profile.self.cycles-pp.kfree 0.12 ± 20% -0.1 0.05 ± 67% perf-profile.self.cycles-pp.menu_reflect 0.12 ± 12% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.dequeue_entity 0.12 ± 13% -0.1 0.07 ± 25% perf-profile.self.cycles-pp.newidle_balance 0.08 ± 23% -0.1 0.03 ±100% perf-profile.self.cycles-pp.run_local_timers 0.08 ± 19% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rb_insert_color 0.24 ± 12% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.__next_timer_interrupt 0.12 ± 18% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.resched_curr 0.11 ± 14% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__update_load_avg_se 0.12 ± 24% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.update_curr 0.10 ± 14% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu 0.09 ± 15% -0.0 0.04 ± 59% perf-profile.self.cycles-pp.rcu_eqs_exit 0.07 ± 22% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.dev_hard_start_xmit 0.11 ± 4% -0.0 0.08 ± 22% perf-profile.self.cycles-pp.finish_task_switch 0.10 ± 15% -0.0 0.07 ± 21% perf-profile.self.cycles-pp.inet_csk_accept 0.08 ± 8% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.release_sock 0.08 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_idle_exit 0.05 ± 62% +0.0 0.08 ± 15% perf-profile.self.cycles-pp.__wake_up_common 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.register_lock_class 0.15 ± 27% +0.1 0.22 ± 15% perf-profile.self.cycles-pp.rcu_core 0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.reacquire_held_locks 0.03 ±100% +0.1 0.12 ± 18% perf-profile.self.cycles-pp.smp_apic_timer_interrupt 0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp._raw_spin_unlock 0.00 +0.1 0.10 ± 25% perf-profile.self.cycles-pp.__lock_text_start 0.00 +0.1 0.10 ± 21% perf-profile.self.cycles-pp.do_raw_spin_trylock 0.00 +0.1 0.11 ± 10% perf-profile.self.cycles-pp.lock_pin_lock 0.00 +0.1 0.12 ± 14% perf-profile.self.cycles-pp.lock_unpin_lock 0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.trace_softirqs_on 0.00 +0.1 0.14 ± 31% perf-profile.self.cycles-pp.trace_softirqs_off 0.00 +0.2 0.18 ± 11% perf-profile.self.cycles-pp.rcu_read_lock_held 0.00 +0.2 0.18 ± 17% perf-profile.self.cycles-pp.tracer_hardirqs_on 0.00 +0.2 0.21 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_on 0.00 +0.2 0.24 ± 10% perf-profile.self.cycles-pp.tracer_hardirqs_off 0.00 +0.3 0.28 ± 13% perf-profile.self.cycles-pp.do_raw_spin_unlock 0.00 +0.3 0.28 ± 6% perf-profile.self.cycles-pp.debug_check_no_locks_freed 0.00 +0.3 0.28 ± 3% perf-profile.self.cycles-pp.__local_bh_disable_ip 0.00 +0.4 0.39 ± 14% perf-profile.self.cycles-pp.rcu_read_lock_sched_held 0.00 +0.4 0.43 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_off 0.00 +0.5 0.48 ± 6% perf-profile.self.cycles-pp.rcu_is_watching 0.00 +0.5 0.51 ± 9% perf-profile.self.cycles-pp.find_held_lock 0.00 +0.5 0.53 ± 3% perf-profile.self.cycles-pp.mark_held_locks 0.00 +0.7 0.68 ± 8% perf-profile.self.cycles-pp.rcu_read_lock_held_common 0.00 +0.8 0.75 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_on 0.00 +0.9 0.88 ± 6% perf-profile.self.cycles-pp.match_held_lock 0.00 +1.1 1.13 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_off 0.00 +1.3 1.30 ± 5% perf-profile.self.cycles-pp.do_raw_spin_lock 0.00 +1.3 1.31 ± 11% perf-profile.self.cycles-pp.rcu_lockdep_current_cpu_online 0.00 +1.5 1.55 ± 12% perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled 0.00 +2.1 2.07 perf-profile.self.cycles-pp.mark_lock 0.00 +3.3 3.29 ± 2% perf-profile.self.cycles-pp.lock_release 0.00 +3.3 3.30 ± 2% perf-profile.self.cycles-pp.lock_acquire 0.00 +5.4 5.41 perf-profile.self.cycles-pp.lock_is_held_type 0.00 +9.1 9.07 ± 4% perf-profile.self.cycles-pp.__lock_acquire netperf.Throughput_tps 35000 +-------------------------------------------------------------------+ | +.+..+.+..+..+.+ +.+..+.+..+..+ +.+..+..+.+..+..+.+..+.+..| 30000 |-+: : : : : | | : : : : : | 25000 |-: : : : : | | : : : : : | 20000 |-: : : : : | | :O O O O O O: O :O O : : | 15000 |:+ : : : : | |: : : : : | 10000 |:+ : : O O:O: O O O O O | |: : : O O :: | 5000 |-+ : : | | : : | 0 +-------------------------------------------------------------------+ netperf.Throughput_total_tps 35000 +-------------------------------------------------------------------+ | +.+..+.+..+..+.+ +.+..+.+..+..+ +.+..+..+.+..+..+.+..+.+..| 30000 |-+: : : : : | | : : : : : | 25000 |-: : : : : | | : : : : : | 20000 |-: : : : : | | :O O O O O O: O :O O : : | 15000 |:+ : : : : | |: : : : : | 10000 |:+ : : O O:O: O O O O O | |: : : O O :: | 5000 |-+ : : | | : : | 0 +-------------------------------------------------------------------+ netperf.workload 1e+07 +-------------------------------------------------------------------+ 9e+06 |-+: : : +.+..+..+ + +.+..+..+.+..+.+..| | : : : : : | 8e+06 |-: : : : : | 7e+06 |-: : : : : | | : : : : : | 6e+06 |-: : : : : | 5e+06 |-:O O O O O O :O :O O : : | 4e+06 |:+ : : : : | |: : : : : | 3e+06 |:+ : : O O O O:O: O O O O O | 2e+06 |:+ : : :: | | : : | 1e+06 |-+ : : | 0 +-------------------------------------------------------------------+ netperf.time.user_time 40 +----------------------------------------------------------------------+ | | 35 |-+ + .+.. .+.. | 30 |-++.+..+..+..+.+..+ .+..+.. +: +.+..+. + +..+..+.+..| | : : +.+. + : : | 25 |-: : : : : | | : : : : : | 20 |-: O O O : :O O : : | | :O O O: O : : : | 15 |:+ : : : : | 10 |:+ : : O O O O :O: O O O | |: : : : : O O | 5 |:+ : : :: | | : : | 0 +----------------------------------------------------------------------+ netperf.time.voluntary_context_switches 2e+07 +-----------------------------------------------------------------+ 1.8e+07 |-+ .+.+..+..+ +.+..+.+..+.+..+.+..+.+..| | +.+..+.+.. .+..+ +.+. : : | 1.6e+07 |-+: + : : : : | 1.4e+07 |-+: : : : : | | : : : : : | 1.2e+07 |-: : : : : | 1e+07 |-:O O O O O O:O :O O : : | 8e+06 |-: : : : : | |: : : : : | 6e+06 |:+ : : O O O O:O: O O O O O | 4e+06 |:+ :: :: | |: :: : | 2e+06 |-+ : : | 0 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

[RDMA/core] 7b491b3fb2: BUG:kernel_hang_in_test_stage

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 7b491b3fb2f4f5749432aba03a6ee31731446ea6 ("[PATCH 2/4] RDMA/core: Introduce shared CQ pool API") url: https://github.com/0day-ci/linux/commits/Yamin-Friedman/Introducing-RDMA-... base: https://git.kernel.org/cgit/linux/kernel/git/rdma/rdma.git for-next in testcase: rcuperf with following parameters: runtime: 300s perf_type: rcu on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------+------------+------------+ | | 3b3f80b960 | 7b491b3fb2 | +-------------------------------+------------+------------+ | boot_successes | 4 | 1 | | boot_failures | 0 | 5 | | BUG:kernel_hang_in_test_stage | 0 | 5 | +-------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 14.746983] random: systemd: uninitialized urandom read (16 bytes read) [ 14.758869] watchdog: watchdog0: watchdog did not stop! [ 454.101407] random: fast init done [ 867.382064] random: crng init done BUG: kernel hang in test stage To reproduce: # build kernel cd linux cp config-5.7.0-rc1-00152-g7b491b3fb2f4f .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

8 months, 2 weeks

1
0
0 / 0

[KVM] 6b6a864bd7: kernel-selftests.kvm.vmx_tsc_adjust_test.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 6b6a864bd7d7eb12a82ec6da1c2dd97a43f79449 ("[PATCH v2] KVM: nVMX: Tweak handling of failure code for nested VM-Enter failure") url: https://github.com/0day-ci/linux/commits/Sean-Christopherson/KVM-nVMX-Twe... base: https://git.kernel.org/cgit/virt/kvm/kvm.git linux-next in testcase: kernel-selftests with following parameters: group: kselftests-kvm ucode: 0x500002c test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> # selftests: kvm: vmx_tsc_adjust_test # ==== Test Assertion Failure ==== # x86_64/vmx_tsc_adjust_test.c:153: false # pid=12157 tid=12157 - Interrupted system call # 1 0x000000000040116a: main at vmx_tsc_adjust_test.c:153 # 2 0x00007fafd54bce0a: ?? ??:0 # 3 0x00000000004011e9: _start at ??:? # Failed guest assert: (vmreadz(VM_EXIT_REASON) == (0x80000000 | 33)) # IA32_TSC_ADJUST is -4294972240 (-1 * TSC_ADJUST_VALUE + -4944). not ok 14 selftests: kvm: vmx_tsc_adjust_test # exit=254 To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Rong Chen

8 months, 2 weeks

2
1
0 / 0

[selftests] 5c39e08fc4: kernel-selftests.netfilter.nft_nat.sh.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: 5c39e08fc417b91d24d243bc6d1a305e1c38fb9f ("selftests: netfilter: add simple masq/redirect test cases") https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-4.19.y in testcase: kernel-selftests with following parameters: group: kselftests-02 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> selftests: netfilter: nft_nat.sh ======================================== /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: Operation not supported map nsincounter { ^^^^^^^^^^^ /dev/stdin:18:6-16: Error: Could not process rule: No such file or directory map nsincounter { ^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:26:6-17: Error: Could not process rule: No such file or directory map nsincounter6 { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:34:6-17: Error: Could not process rule: No such file or directory map nsoutcounter { ^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:42:6-18: Error: Could not process rule: No such file or directory map nsoutcounter6 { ^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:15-46: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:52:3-40: Error: Could not process rule: No such file or directory counter name ip saddr map @nsincounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:53:3-87: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:57:3-41: Error: Could not process rule: No such file or directory counter name ip daddr map @nsoutcounter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /dev/stdin:58:3-88: Error: Could not process rule: No such file or directory icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1 ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2 ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2 ^^^^^^ /dev/stdin:2:8-13: Error: Could not process rule: No such file or directory chain output { ^^^^^^ /dev/stdin:4:3-38: Error: Could not process rule: No such file or directory ip daddr 10.0.1.99 dnat to 10.0.2.99 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip nat output ^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns0 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns0 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns2in counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns2out counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns2out ^^^^^^ /dev/stdin:2:8-13: Error: Could not process rule: No such file or directory chain output { ^^^^^^ /dev/stdin:4:3-41: Error: Could not process rule: No such file or directory ip6 daddr dead:1::99 dnat to dead:2::99 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SKIP: Could not add add ip6 dnat hook Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ /dev/stdin:2:8-18: Error: Could not process rule: No such file or directory chain postrouting { ^^^^^^^^^^^ /dev/stdin:4:3-27: Error: Could not process rule: No such file or directory meta oif veth0 masquerade ^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns0out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip nat postrouting ^^^ ERROR: Could not flush nat postrouting Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ /dev/stdin:2:8-18: Error: Could not process rule: No such file or directory chain postrouting { ^^^^^^^^^^^ /dev/stdin:4:3-27: Error: Could not process rule: No such file or directory meta oif veth0 masquerade ^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ Error: Could not process rule: No such file or directory flush chain ip6 nat postrouting ^^^ ERROR: Could not flush ip6 nat postrouting Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2in ^^^^^^ Error: No such file or directory list counter inet filter ns1in ^^^^^^ ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns2out ^^^^^^ Error: No such file or directory list counter inet filter ns1out ^^^^^^ ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns1out ^^^^^^ /dev/stdin:2:8-17: Error: Could not process rule: No such file or directory chain prerouting { ^^^^^^^^^^ /dev/stdin:4:3-80: Error: Could not process rule: No such file or directory meta iif veth1 ip protocol icmp ip saddr 10.0.2.99 ip daddr 10.0.1.99 redirect ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: No such file or directory list counter inet filter ns2in ^^^^^^ ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0in ^^^^^^ Error: No such file or directory list counter inet filter ns2out ^^^^^^ ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84) Error: No such file or directory list counter inet filter ns0out ^^^^^^ Error: Could not process rule: No such file or directory delete table ip nat ^^^ ERROR: Could not delete nat table Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns1out6 ^^^^^^ /dev/stdin:2:8-17: Error: Could not process rule: No such file or directory chain prerouting { ^^^^^^^^^^ /dev/stdin:4:3-87: Error: Could not process rule: No such file or directory meta iif veth1 meta l4proto icmpv6 ip6 saddr dead:2::99 ip6 daddr dead:1::99 redirect ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: No such file or directory list counter inet filter ns2in6 ^^^^^^ ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0in6 ^^^^^^ Error: No such file or directory list counter inet filter ns2out6 ^^^^^^ ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104) Error: No such file or directory list counter inet filter ns0out6 ^^^^^^ Error: Could not process rule: No such file or directory delete table ip6 nat ^^^ ERROR: Could not delete ip6 nat table not ok 1..2 selftests: netfilter: nft_nat.sh [FAIL] To reproduce: # build kernel cd linux cp config-4.19.28-00103-g5c39e08fc417b .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 2 weeks

1
0
0 / 0

[pipe] f2af7d90e2: xfstests.btrfs.052.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-7): commit: f2af7d90e25ce1cf03318bf3290fcfc1e762d8f9 ("pipe: read/write_iter() handler should check for IOCB_NOWAIT") url: https://github.com/0day-ci/linux/commits/UPDATE-20200501-052027/Jens-Axbo... in testcase: xfstests with following parameters: disk: 6HDD fs: btrfs test: btrfs-group00 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> 2020-05-06 21:03:14 export TEST_DIR=/fs/vda 2020-05-06 21:03:14 export TEST_DEV=/dev/vda 2020-05-06 21:03:14 export FSTYP=btrfs 2020-05-06 21:03:14 export SCRATCH_MNT=/fs/scratch 2020-05-06 21:03:14 mkdir /fs/scratch -p 2020-05-06 21:03:15 export SCRATCH_DEV_POOL="/dev/vdb /dev/vdc /dev/vdd /dev/vde /dev/vdf" 2020-05-06 21:03:15 sed "s:^:btrfs/:" //lkp/benchmarks/xfstests/tests/btrfs-group00 2020-05-06 21:03:15 ./check btrfs/001 btrfs/004 btrfs/007 btrfs/010 btrfs/013 btrfs/016 btrfs/019 btrfs/022 btrfs/025 btrfs/028 btrfs/031 btrfs/034 btrfs/037 btrfs/040 btrfs/043 btrfs/046 btrfs/049 btrfs/052 btrfs/055 btrfs/058 btrfs/061 btrfs/064 btrfs/067 btrfs/071 btrfs/074 btrfs/077 btrfs/080 btrfs/083 btrfs/086 btrfs/089 btrfs/092 btrfs/095 btrfs/098 btrfs/101 btrfs/104 btrfs/107 btrfs/110 btrfs/113 btrfs/116 btrfs/119 btrfs/122 btrfs/125 btrfs/128 btrfs/131 btrfs/134 btrfs/137 btrfs/140 btrfs/143 btrfs/146 btrfs/149 btrfs/152 btrfs/155 btrfs/158 btrfs/161 btrfs/164 btrfs/167 btrfs/170 btrfs/173 btrfs/176 btrfs/179 btrfs/182 btrfs/185 btrfs/188 btrfs/191 btrfs/194 btrfs/197 btrfs/200 btrfs/203 btrfs/206 btrfs/209 FSTYP -- btrfs PLATFORM -- Linux/x86_64 vm-snb-30 5.7.0-rc3-00044-gf2af7d90e25ce #1 SMP Wed May 6 04:20:50 CST 2020 MKFS_OPTIONS -- /dev/vdb MOUNT_OPTIONS -- /dev/vdb /fs/scratch btrfs/001 3s btrfs/004 73s btrfs/007 1s btrfs/010 129s btrfs/013 11s btrfs/016 2s btrfs/019 0s btrfs/022 5s btrfs/025 1s btrfs/028 31s btrfs/031 0s btrfs/034 6s btrfs/037 1s btrfs/040 1s btrfs/043 1s btrfs/046 4s btrfs/049 6s btrfs/052 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/052.out.bad) --- tests/btrfs/052.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/052.out.bad 2020-05-06 21:07:55.779485703 +0800 @@ -1,643 +1,139 @@ QA output created by 052 Testing with a cow file (default) -Blocks modified: [0 - 1] -Blocks modified: [2 - 3] -Blocks modified: [4 - 5] -Blocks modified: [6 - 7] -Blocks modified: [8 - 23] ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/052.out /lkp/benchmarks/xfstests/results//btrfs/052.out.bad' to see the entire diff) btrfs/055 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/055.out.bad) --- tests/btrfs/055.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/055.out.bad 2020-05-06 21:07:56.994485703 +0800 @@ -1,337 +1,59 @@ QA output created by 055 Testing without the NO_HOLES feature -Blocks modified: [0 - 1] -Blocks modified: [2 - 3] -Blocks modified: [6 - 7] -Blocks modified: [8 - 9] -Blocks modified: [0 - 23] ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/055.out /lkp/benchmarks/xfstests/results//btrfs/055.out.bad' to see the entire diff) btrfs/058 3s btrfs/061 97s btrfs/064 88s btrfs/067 51s btrfs/071 22s btrfs/074 45s btrfs/077 0s btrfs/080 43s btrfs/083 1s btrfs/086 1s btrfs/089 1s btrfs/092 1s btrfs/095 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/095.out.bad) --- tests/btrfs/095.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/095.out.bad 2020-05-06 21:13:51.276485703 +0800 @@ -1,35 +1,9 @@ QA output created by 095 -Blocks modified: [135 - 164] -Blocks modified: [768 - 792] +awk: line 19: function strtonum never defined +awk: line 19: function strtonum never defined +awk: line 19: function strtonum never defined +awk: line 19: function strtonum never defined ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/095.out /lkp/benchmarks/xfstests/results//btrfs/095.out.bad' to see the entire diff) btrfs/098 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/098.out.bad) --- tests/btrfs/098.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/098.out.bad 2020-05-06 21:13:52.311485703 +0800 @@ -1,22 +1,7 @@ QA output created by 098 -Blocks modified: [200 - 224] +awk: line 19: function strtonum never defined +awk: line 19: function strtonum never defined File contents before power failure: -0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -* ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/098.out /lkp/benchmarks/xfstests/results//btrfs/098.out.bad' to see the entire diff) btrfs/101 17s btrfs/104 1s btrfs/107 1s btrfs/110 1s btrfs/113 1s btrfs/116 [not run] FITRIM not supported on /fs/scratch btrfs/119 1s btrfs/122 6s btrfs/125 19s btrfs/128 2s btrfs/131 2s btrfs/134 1s btrfs/137 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/137.out.bad) --- tests/btrfs/137.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/137.out.bad 2020-05-06 21:14:46.418485703 +0800 @@ -17,22 +17,15 @@ d3dc847171f9081bd75d7a2d3b53d322 SCRATCH_MNT/snap2/bar File snap1/foo fiemap results in the original filesystem: -0: [0..7]: data File snap1/bar fiemap results in the original filesystem: -0: [0..7]: data ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/137.out /lkp/benchmarks/xfstests/results//btrfs/137.out.bad' to see the entire diff) btrfs/140 1s btrfs/143 2s btrfs/146 2s btrfs/149 2s btrfs/152 1s btrfs/155 1s btrfs/158 1s btrfs/161 0s btrfs/164 2s btrfs/167 1s btrfs/170 1s btrfs/173 0s btrfs/176 2s btrfs/179 133s btrfs/182 3s btrfs/185 0s btrfs/188 1s btrfs/191 1s btrfs/194 20s btrfs/197 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/197.out.bad) --- tests/btrfs/197.out 2020-04-09 10:45:28.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//btrfs/197.out.bad 2020-05-06 21:17:44.450485703 +0800 @@ -3,23 +3,19 @@ Label: none uuid: <UUID> Total devices <NUM> FS bytes used <SIZE> devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV - *** Some devices missing raid5 Label: none uuid: <UUID> ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/197.out /lkp/benchmarks/xfstests/results//btrfs/197.out.bad' to see the entire diff) btrfs/200 1s btrfs/203 1s btrfs/206 [not run] This test requires a valid $LOGWRITES_DEV btrfs/209 0s Ran: btrfs/001 btrfs/004 btrfs/007 btrfs/010 btrfs/013 btrfs/016 btrfs/019 btrfs/022 btrfs/025 btrfs/028 btrfs/031 btrfs/034 btrfs/037 btrfs/040 btrfs/043 btrfs/046 btrfs/049 btrfs/052 btrfs/055 btrfs/058 btrfs/061 btrfs/064 btrfs/067 btrfs/071 btrfs/074 btrfs/077 btrfs/080 btrfs/083 btrfs/086 btrfs/089 btrfs/092 btrfs/095 btrfs/098 btrfs/101 btrfs/104 btrfs/107 btrfs/110 btrfs/113 btrfs/116 btrfs/119 btrfs/122 btrfs/125 btrfs/128 btrfs/131 btrfs/134 btrfs/137 btrfs/140 btrfs/143 btrfs/146 btrfs/149 btrfs/152 btrfs/155 btrfs/158 btrfs/161 btrfs/164 btrfs/167 btrfs/170 btrfs/173 btrfs/176 btrfs/179 btrfs/182 btrfs/185 btrfs/188 btrfs/191 btrfs/194 btrfs/197 btrfs/200 btrfs/203 btrfs/206 btrfs/209 Not run: btrfs/116 btrfs/206 Failures: btrfs/052 btrfs/055 btrfs/095 btrfs/098 btrfs/137 btrfs/197 Failed 6 of 70 tests To reproduce: # build kernel cd linux cp config-5.7.0-rc3-00044-gf2af7d90e25ce .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, Rong Chen

8 months, 2 weeks

4
3
0 / 0

9ef0b22bca ("fs: call file_{start,end}_write from .."): [ 45.496622] WARNING: possible recursive locking detected

by kernel test robot

Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.infradead.org/users/hch/misc.git cleanup-kernel-rw-helpers commit 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1 Author: Christoph Hellwig <hch(a)lst.de> AuthorDate: Thu May 7 19:27:06 2020 +0200 Commit: Christoph Hellwig <hch(a)lst.de> CommitDate: Fri May 8 11:11:54 2020 +0200 fs: call file_{start,end}_write from __kernel_write We always need to take a reference on the file system we are writing to. Signed-off-by: Christoph Hellwig <hch(a)lst.de> 79dede78c0 Merge branch 'for-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 9ef0b22bca fs: call file_{start,end}_write from __kernel_write c2c302133b fs: don't change the address limit for ->read_iter in __kernel_read +-------------------------------------------------+------------+------------+------------+ | | 79dede78c0 | 9ef0b22bca | c2c302133b | +-------------------------------------------------+------------+------------+------------+ | boot_successes | 151 | 39 | 0 | | boot_failures | 2 | 13 | 1 | | INFO:rcu_sched_self-detected_stall_on_CPU | 1 | | | | RIP:iov_iter_copy_from_user_atomic | 1 | | | | BUG:soft_lockup-CPU##stuck_for#s![trinity-c2:#] | 2 | | | | Kernel_panic-not_syncing:softlockup:hung_tasks | 2 | | | | RIP:smp_call_function_single | 1 | | | | WARNING:possible_recursive_locking_detected | 0 | 13 | 1 | | BUG:unable_to_handle_page_fault_for_address | 0 | 1 | | | Oops:#[##] | 0 | 1 | | | Kernel_panic-not_syncing:Fatal_exception | 0 | 1 | | +-------------------------------------------------+------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory [ 45.219908] audit: type=1326 audit(1589183752.519:33): auid=4294967295 uid=65534 gid=65534 ses=4294967295 subj=kernel pid=3333 comm="trinity-c1" exe="/bin/trinity" sig=9 arch=c000003e syscall=8 compat=0 ip=0x4587a7 code=0x0 modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory [ 45.495619] [ 45.495976] ============================================ [ 45.496622] WARNING: possible recursive locking detected [ 45.497266] 5.7.0-rc4-00209-g9ef0b22bca82d #1 Not tainted [ 45.497882] -------------------------------------------- [ 45.498489] trinity-c7/3230 is trying to acquire lock: [ 45.499081] ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: __kernel_write+0x12f/0x180 [ 45.500032] [ 45.500032] but task is already holding lock: [ 45.500791] ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: do_sendfile+0x370/0x3c0 [ 45.501761] [ 45.501761] other info that might help us debug this: [ 45.502615] Possible unsafe locking scenario: [ 45.502615] [ 45.503383] CPU0 [ 45.503756] ---- [ 45.504128] lock(sb_writers#6); [ 45.504568] lock(sb_writers#6); [ 45.505008] [ 45.505008] *** DEADLOCK *** [ 45.505008] [ 45.505896] May be due to missing lock nesting notation [ 45.505896] [ 45.506738] 1 lock held by trinity-c7/3230: [ 45.507259] #0: ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: do_sendfile+0x370/0x3c0 [ 45.508242] [ 45.508242] stack backtrace: [ 45.508903] CPU: 1 PID: 3230 Comm: trinity-c7 Not tainted 5.7.0-rc4-00209-g9ef0b22bca82d #1 [ 45.509922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 45.510919] Call Trace: [ 45.511292] dump_stack+0x8f/0xcb [ 45.511735] validate_chain+0x1228/0x1240 [ 45.512235] __lock_acquire+0x86f/0xbc0 [ 45.512720] lock_acquire+0xac/0x390 [ 45.513188] ? __kernel_write+0x12f/0x180 [ 45.513699] __sb_start_write+0x137/0x220 [ 45.514198] ? __kernel_write+0x12f/0x180 [ 45.514699] ? do_splice_direct+0xd0/0xd0 [ 45.515202] __kernel_write+0x12f/0x180 [ 45.515693] ? do_splice_direct+0xd0/0xd0 [ 45.516192] write_pipe_buf+0x76/0xa0 [ 45.516664] __splice_from_pipe+0xc3/0x1c0 [ 45.517171] ? do_splice_direct+0xd0/0xd0 [ 45.521759] splice_from_pipe+0x5d/0x90 [ 45.522250] default_file_splice_write+0x15/0x20 [ 45.522810] direct_splice_actor+0x35/0x40 [ 45.523342] splice_direct_to_actor+0x110/0x250 [ 45.523907] ? generic_pipe_buf_nosteal+0x10/0x10 [ 45.524485] do_splice_direct+0xa0/0xd0 [ 45.524976] do_sendfile+0x1d5/0x3c0 [ 45.525451] __x64_sys_sendfile64+0x5c/0xd0 [ 45.525968] do_syscall_64+0x60/0x280 [ 45.526449] entry_SYSCALL_64_after_hwframe+0x49/0xb3 [ 45.527042] RIP: 0033:0x457729 [ 45.527468] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00 [ 45.529442] RSP: 002b:00007fffa97ea1f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028 [ 45.530378] RAX: ffffffffffffffda RBX: 0000000000000028 RCX: 0000000000457729 [ 45.531178] RDX: 00007fcc78f87000 RSI: 00000000000000ba RDI: 00000000000000ba [ 45.531942] RBP: 00007fffa97ea2a0 R08: 0000000000100000 R09: 0000000000000048 [ 45.532703] R10: 0000000000001000 R11: 0000000000000246 R12: 0000000000000002 [ 45.533470] R13: 00007fcc7988b058 R14: 000000000277b830 R15: 00007fcc7988b000 modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory [child2:3389] read (0) returned ENOSYS, marking as inactive. modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD git bisect start 18c86474c709bd4e3317a8d0dddb160bdae94996 0e698dfa282211e414076f9dc7e83c1c288314fd -- git bisect bad f06471c67ef6929f7e054de7651a47fdfa00c8e8 # 00:05 B 7 3 0 0 Merge 'linux-review/Yonghong-Song/bpf-implement-bpf-iterator-for-kernel-data/20200510-033248' into devel-hourly-2020051021 git bisect good 459e15afbe338077afbba837212c1c1a7527b202 # 00:26 G 26 0 1 1 Merge 'linux-review/Steven-Rostedt/tools-bootconfig-Fix-apply_xbc-to-return-zero-on-success/20200509-070610' into devel-hourly-2020051021 git bisect good df47a754d7e6c065c132eb060a6824c1e99276d9 # 00:44 G 26 0 0 0 Merge 'linux-review/Dan-Murphy/net-ethernet-ti-cpts-Fix-linker-issue-when-TI_CPTS-is-defined/20200508-230627' into devel-hourly-2020051021 git bisect good cfbd4971d03fefcc627e8aeacc4d305964114535 # 01:05 G 24 0 0 0 Merge 'linux-review/Mike-Marciniszyn/IB-hfi1-Fix-another-case-where-pq-is-left-on-waitlist/20200505-070157' into devel-hourly-2020051021 git bisect good da8cd247cb72b5e588884a4b745b32f32740c95c # 01:33 G 24 0 0 0 Merge 'linux-review/Charan-Teja-Reddy/dma-buf-fix-use-after-free-in-dmabuffs_dname/20200509-032857' into devel-hourly-2020051021 git bisect bad d6bae61b2c781390851cf4182ea2fdc74ec03b5f # 01:49 B 4 2 0 0 Merge 'linux-review/Enric-Balletbo-i-Serra/Convert-mtk-dpi-to-drm_bridge-API/20200505-072853' into devel-hourly-2020051021 git bisect good 18f312ea74b4185bc135eccc8fc8a161c0402ceb # 02:31 G 25 0 0 0 Merge 'linux-review/Thierry-Reding/clk-tegra-Fix-initial-rate-for-pll_a-on-Tegra124/20200506-024253' into devel-hourly-2020051021 git bisect bad 9ed636bcf097595b4946aa746ce3b541becb0e71 # 03:02 B 16 1 0 0 Merge 'linux-review/Christophe-JAILLET/clk-clk-xgene-Fix-a-typo-in-Kconfig/20200505-034755' into devel-hourly-2020051021 git bisect good 365c8030c03fd5109d1191fdf9aab067663b8c54 # 03:32 G 51 0 1 1 Merge 'linux-review/Yong-Zhi/ASoC-max98373-reorder-max98373_reset-in-resume/20200502-074911' into devel-hourly-2020051021 git bisect bad 9fd97e3bc948c40e14fb27d9ac3fc788b7cae193 # 03:47 B 2 1 0 0 Merge 'hch-misc/cleanup-kernel-rw-helpers' into devel-hourly-2020051021 git bisect bad dd0b01a50f3de2d8e71b4865f50bbe49810e061a # 04:12 B 8 1 0 0 fs: don't change the address limit for ->write_iter in __kernel_write git bisect bad bcdb099498e6c632d1363b6a1bf83898bdd9015d # 04:38 B 4 1 0 0 fs: remove the call_{read,write}_iter functions git bisect bad 31a9c7ee10bc2bef9f623a11f77bf7e2d72aba5c # 05:00 B 5 1 0 0 fs: check FMODE_WRITE in __kernel_write git bisect bad 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1 # 05:38 B 7 1 0 0 fs: call file_{start,end}_write from __kernel_write # first bad commit: [9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1] fs: call file_{start,end}_write from __kernel_write git bisect good 79dede78c0573618e3137d3d8cbf78c84e25fabd # 06:34 G 150 0 3 3 Merge branch 'for-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security # extra tests with debug options git bisect bad 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1 # 06:55 B 0 1 17 0 fs: call file_{start,end}_write from __kernel_write # extra tests on head commit of hch-misc/cleanup-kernel-rw-helpers git bisect bad c2c302133bed00699eb52d5516f45d79c4f4ee85 # 07:35 B 0 1 17 0 fs: don't change the address limit for ->read_iter in __kernel_read # bad: [c2c302133bed00699eb52d5516f45d79c4f4ee85] fs: don't change the address limit for ->read_iter in __kernel_read --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/lkp@lists.01.org

8 months, 2 weeks

1
0
0 / 0

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP May 2020