d13fee049f ("Default enable RCU list lockdep debugging with .."): WARNING: suspicious RCU usage
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git rcu/next
commit d13fee049fa8e94649aa9914fd1ab5d07da51da4
Author: Madhuparna Bhowmik <madhuparnabhowmik10(a)gmail.com>
AuthorDate: Fri Feb 28 14:54:51 2020 +0530
Commit: Paul E. McKenney <paulmck(a)kernel.org>
CommitDate: Thu May 7 10:24:43 2020 -0700
Default enable RCU list lockdep debugging with PROVE_RCU
This patch default enables CONFIG_PROVE_RCU_LIST option with
CONFIG_PROVE_RCU for RCU list lockdep debugging.
With this change, RCU list lockdep debugging will be default
enabled in CONFIG_PROVE_RCU=y kernels.
Most of the RCU users (in core kernel/, drivers/, and net/
subsystem) have already been modified to include lockdep
expressions hence RCU list debugging can be enabled by
default.
However, there are still chances of enountering
false-positive lockdep splats because not everything is converted,
in case RCU list primitives are used in non-RCU read-side critical
section but under the protection of a lock. It would be okay to
have a few false-positives, as long as bugs are identified, since this
patch only affects debugging kernels.
Co-developed-by: Amol Grover <frextrite(a)gmail.com>
Signed-off-by: Amol Grover <frextrite(a)gmail.com>
Signed-off-by: Madhuparna Bhowmik <madhuparnabhowmik10(a)gmail.com>
Acked-by: Joel Fernandes (Google) <joel(a)joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck(a)kernel.org>
c1fe153b2a torture: Add --allcpus argument to the kvm.sh script
d13fee049f Default enable RCU list lockdep debugging with PROVE_RCU
ec06fa2b14 doc: Tasks RCU must protect instructions before trampoline
+--------------------------------------------------------------------------------------+------------+------------+------------+
| | c1fe153b2a | d13fee049f | ec06fa2b14 |
+--------------------------------------------------------------------------------------+------------+------------+------------+
| boot_successes | 31 | 0 | 0 |
| boot_failures | 2 | 11 | 11 |
| BUG:kernel_hang_in_early-boot_stage,last_printk:Probing_EDD(edd=off_to_disable)...ok | 2 | | |
| WARNING:suspicious_RCU_usage | 0 | 11 | 11 |
| net/ipv6/ip6mr.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | 11 |
| BUG:kernel_hang_in_test_stage | 0 | 1 | |
+--------------------------------------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 71.408539] Performance counters on
[ 71.409303] input device check on
[ 71.473366] NET: Registered protocol family 10
[ 71.479116]
[ 71.479916] =============================
[ 71.480827] WARNING: suspicious RCU usage
[ 71.481663] 5.7.0-rc2-00234-gd13fee049fa8e #1 Not tainted
[ 71.482791] -----------------------------
[ 71.483839] net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!!
[ 71.485271]
[ 71.485271] other info that might help us debug this:
[ 71.485271]
[ 71.487039]
[ 71.487039] rcu_scheduler_active = 2, debug_locks = 1
[ 71.488607] 1 lock held by swapper/1:
[ 71.489445] #0: ffffffffa9d313f0 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x1d/0x60
[ 71.491830]
[ 71.491830] stack backtrace:
[ 71.492769] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00234-gd13fee049fa8e #1
[ 71.494422] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 71.496343] Call Trace:
[ 71.496909] dump_stack+0x23/0x38
[ 71.497671] lockdep_rcu_suspicious+0x10c/0x11a
[ 71.498683] ip6mr_get_table+0x117/0x130
[ 71.499491] ip6mr_new_table+0x19/0x60
[ 71.500021] ip6mr_net_init+0xf6/0x267
[ 71.500472] ops_init+0x60/0x2b0
[ 71.501054] register_pernet_operations+0xad/0x1e0
[ 71.501899] ? unix_sysctl_register+0xe4/0xe4
[ 71.502727] register_pernet_subsys+0x34/0x60
[ 71.503661] ip6_mr_init+0x58/0x15e
[ 71.504290] inet6_init+0x1a1/0x68f
[ 71.505024] do_one_initcall+0x102/0x500
[ 71.505811] ? perf_trace_initcall_level+0x2c0/0x2c0
[ 71.506759] ? kasan_unpoison_shadow+0x38/0x60
[ 71.507766] ? rcu_read_lock_sched_held+0x85/0xd0
[ 71.508607] do_basic_setup+0x1a9/0x1fd
[ 71.509327] kernel_init_freeable+0x115/0x183
[ 71.510131] ? rest_init+0x22e/0x22e
[ 71.510854] kernel_init+0x13/0x1d2
[ 71.511763] ? rest_init+0x22e/0x22e
[ 71.512572] ret_from_fork+0x24/0x30
modprobe: FATAL: Could not load /lib/modules/5.7.0-rc2-00234-gd13fee049fa8e/modules.dep: No such file or directory
[ 71.746089] Segment Routing with IPv6
[ 71.746959] RPL Segment Routing with IPv6
[ 71.772698] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 71.803782] can: controller area network core (rev 20170425 abi 9)
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start a41e54ce803b45b284e10a4e231dba5fec292064 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8 --
git bisect bad 2f7991448f9c479f7ce0e7007b697555df328b46 # 23:56 B 0 2 18 0 Merge 'linux-review/Pavel-Begunkov/io_uring-fix-zero-len-do_splice/20200505-171034' into devel-hourly-2020051219
git bisect bad 3edfcaf0e7672c1e6c7bbfe8db1ceca43f5e7c85 # 00:28 B 0 2 18 0 Merge 'linux-review/Dan-Williams/ACPI-Drop-rcu-usage-for-MMIO-mappings/20200509-013208' into devel-hourly-2020051219
git bisect good 3f1db4c8a03e1ab931248c00ae4c4559e3b5406b # 01:17 G 10 0 1 1 Merge 'ezequielg/rkvdec-vp9-upstream' into devel-hourly-2020051219
git bisect good 5579a78821bdf86725db495878dd9c6245ed2bb3 # 01:41 G 10 0 1 1 Merge 'linux-review/Takashi-Sakamoto/ALSA-firewire-motu-fulfill-missing-entries-in-Kconfig/20200511-103548' into devel-hourly-2020051219
git bisect good e3689f9e910f8fd4b1f565359664e4a72b112034 # 02:13 G 10 0 0 0 Merge 'linux-review/UPDATE-20200512-031403/Lee-Shawn-C/drm-i915-mst-filter-out-the-display-mode-exceed-sink-s-capability/20200420-213842' into devel-hourly-2020051219
git bisect bad d69b0f11241256c60dbf757ac3dd5a96ed91882e # 02:33 B 0 2 18 0 Merge 'rcu/rcu/dev' into devel-hourly-2020051219
git bisect good 0144187eb59fafe035c65eff7a3935a80da27eb0 # 03:12 G 10 0 0 0 Merge 'linux-review/Maninder-Singh/scripts-checkstack-pl-don-t-display-dre-as-different-entity/20200509-051219' into devel-hourly-2020051219
git bisect good f4311b8d31874d2bde6adabf8ac1d335ea35b35f # 03:52 G 10 0 0 0 Merge 'linux-review/Saeed-Mahameed/Mellanox-mlx5-and-bonding-updates-2020-05-09/20200509-191959' into devel-hourly-2020051219
git bisect good 4a49fd5e59ac0f31ef28ebb545a73845e7943416 # 04:17 G 10 0 1 1 Merge 'nf/master' into devel-hourly-2020051219
git bisect good 071972c7badf15fd8fbb804c6b5d727a48d16ed7 # 04:44 G 10 0 1 1 Merge 'zen-kernel-zen-kernel/5.6/fsgsbase' into devel-hourly-2020051219
git bisect good 3123dcd3ef7f98482dd0a459e85a750448161b38 # 05:27 G 10 0 1 1 fork: Annotate a data race in vm_area_dup()
git bisect good 8d6d79f04175a9ce8091601a9e5eefc1dfead091 # 06:04 G 11 0 2 2 kcsan: Add test suite
git bisect good 3ff7cf6d6d90ec5a196b31ef03777e756807e92a # 06:24 G 10 0 0 0 torture: Add script to smoke-test commits in a branch
git bisect bad 91f877ab4b94c0d463a60b8c77d529096a8268c1 # 06:40 B 0 1 17 0 rcu: Priority-boost-related sleeps to idle priority
git bisect good c1fe153b2a011a7d5f156f12ee518b1b0dc53fa1 # 07:20 G 10 0 0 0 torture: Add --allcpus argument to the kvm.sh script
git bisect bad 3d9e98763d855022fef5285b3fd779238f9ab554 # 07:35 B 0 1 17 0 rcu: Grace-period-kthread related sleeps to idle priority
git bisect bad d13fee049fa8e94649aa9914fd1ab5d07da51da4 # 07:58 B 0 2 18 0 Default enable RCU list lockdep debugging with PROVE_RCU
# first bad commit: [d13fee049fa8e94649aa9914fd1ab5d07da51da4] Default enable RCU list lockdep debugging with PROVE_RCU
git bisect good c1fe153b2a011a7d5f156f12ee518b1b0dc53fa1 # 08:12 G 30 0 0 1 torture: Add --allcpus argument to the kvm.sh script
# extra tests with debug options
git bisect bad d13fee049fa8e94649aa9914fd1ab5d07da51da4 # 08:30 B 0 2 18 0 Default enable RCU list lockdep debugging with PROVE_RCU
# extra tests on head commit of rcu/rcu/next
git bisect bad ec06fa2b148836f3f50082eb502b8f0aadb8970a # 08:42 B 0 1 17 0 doc: Tasks RCU must protect instructions before trampoline
# bad: [ec06fa2b148836f3f50082eb502b8f0aadb8970a] doc: Tasks RCU must protect instructions before trampoline
# extra tests on revert first bad commit
git bisect good 92587bc102c54408efd66e5b9212bbd3641967c1 # 09:27 G 10 0 1 1 Revert "Default enable RCU list lockdep debugging with PROVE_RCU"
# good: [92587bc102c54408efd66e5b9212bbd3641967c1] Revert "Default enable RCU list lockdep debugging with PROVE_RCU"
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/lkp@lists.01.org
8 months, 2 weeks
0902bb3bb8: vm-scalability.median -86.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -86.2% regression of vm-scalability.median due to commit:
commit: 0902bb3bb8fdb69f956f4c3ee8157fe5d1c1e44d ("[PATCH RFC tip/core/rcu] Add shrinker to shift to fast/inefficient GP mode")
url: https://github.com/0day-ci/linux/commits/Paul-E-McKenney/Add-shrinker-to-...
in testcase: vm-scalability
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:
runtime: 300
thp_enabled: never
thp_defrag: always
nr_task: 8
nr_pmem: 1
test: swap-w-seq-mt
bp_memmap: 96G!18G
cpufreq_governor: performance
ucode: 0x500002c
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bp_memmap/compiler/cpufreq_governor/kconfig/nr_pmem/nr_task/rootfs/runtime/tbox_group/test/testcase/thp_defrag/thp_enabled/ucode:
96G!18G/gcc-7/performance/x86_64-rhel-7.6/1/8/debian-x86_64-20191114.cgz/300/lkp-csl-2sp6/swap-w-seq-mt/vm-scalability/always/never/0x500002c
commit:
baf5fe7618 ("Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu")
0902bb3bb8 ("Add shrinker to shift to fast/inefficient GP mode")
baf5fe7618468151 0902bb3bb8fdb69f956f4c3ee81
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at_ip__slab_free/0x
1:4 -33% 0:4 perf-profile.children.cycles-pp.error_entry
%stddev %change %stddev
\ | \
1.60 +11.1% 1.78 ± 4% vm-scalability.free_time
716935 ± 3% -86.2% 99160 ± 9% vm-scalability.median
46.71 ± 14% -35.3 11.43 ± 28% vm-scalability.stddev%
5619989 ± 2% -86.0% 788970 ± 9% vm-scalability.throughput
44.93 +635.3% 330.35 ± 10% vm-scalability.time.elapsed_time
44.93 +635.3% 330.35 ± 10% vm-scalability.time.elapsed_time.max
91727 ± 7% -49.8% 46077 ± 8% vm-scalability.time.involuntary_context_switches
198.06 ± 3% +927.4% 2034 ± 8% vm-scalability.time.system_time
102.53 +6.3% 108.96 ± 2% vm-scalability.time.user_time
9919 ± 10% -65.2% 3455 ± 9% vm-scalability.time.voluntary_context_switches
92.25 -3.7% 88.81 iostat.cpu.idle
5.24 ± 2% +106.5% 10.82 ± 2% iostat.cpu.system
2.51 -85.6% 0.36 ± 11% iostat.cpu.user
0.00 ±100% +0.0 0.00 ± 37% mpstat.cpu.all.iowait%
5.41 ± 2% +5.4 10.86 ± 2% mpstat.cpu.all.sys%
2.60 -2.2 0.36 ± 11% mpstat.cpu.all.usr%
918871 ± 24% +49.6% 1374492 ± 11% cpuidle.C1.time
21145 ± 18% +240.5% 72004 ± 9% cpuidle.C1.usage
1.541e+09 ± 95% +1137.7% 1.908e+10 ± 30% cpuidle.C1E.time
5241728 ± 34% +748.9% 44495355 ± 16% cpuidle.C1E.usage
34825 ± 7% +2744.5% 990603 ± 11% cpuidle.POLL.time
12508 ± 4% +3114.1% 402020 ± 9% cpuidle.POLL.usage
92.00 -3.6% 88.67 vmstat.cpu.id
1031 ± 3% -87.7% 126.33 ± 40% vmstat.memory.buff
61018140 ± 2% -48.5% 31447956 ± 11% vmstat.memory.free
6453338 ± 17% +53.4% 9897990 ± 4% vmstat.memory.swpd
7.25 ± 5% +83.9% 13.33 ± 3% vmstat.procs.r
909.25 ± 2% -86.3% 124.33 ± 3% vmstat.swap.si
611160 ± 9% -87.7% 74985 ± 2% vmstat.swap.so
7293 ± 7% -31.7% 4981 vmstat.system.cs
785179 ± 6% -70.1% 234556 vmstat.system.in
99837026 +18.5% 1.183e+08 ± 3% meminfo.Active
99835926 +18.5% 1.183e+08 ± 3% meminfo.Active(anon)
1100 ± 2% -67.1% 362.33 ± 12% meminfo.Active(file)
1.017e+08 ± 2% +20.0% 1.221e+08 ± 3% meminfo.AnonPages
1063 ± 2% -85.4% 155.33 ± 41% meminfo.Buffers
129506 ± 3% -56.6% 56267 ± 6% meminfo.CmaFree
1.704e+08 -7.7% 1.572e+08 ± 3% meminfo.Committed_AS
8189726 ± 10% -6.0% 7701277 ± 6% meminfo.DirectMap2M
1950380 ± 6% +95.5% 3813862 ± 3% meminfo.Inactive
1949183 ± 6% +95.6% 3813016 ± 3% meminfo.Inactive(anon)
1196 ± 3% -29.3% 845.33 ± 21% meminfo.Inactive(file)
83888 ± 2% +16.4% 97645 meminfo.KReclaimable
29604 +44.9% 42884 ± 4% meminfo.Mapped
58640993 ± 3% -34.5% 38418196 ± 11% meminfo.MemAvailable
59089646 ± 3% -34.3% 38806088 ± 11% meminfo.MemFree
1.056e+08 +19.2% 1.259e+08 ± 3% meminfo.Memused
218191 ± 2% +19.3% 260390 ± 3% meminfo.PageTables
83888 ± 2% +16.4% 97645 meminfo.SReclaimable
3557167 -82.4% 626370 ± 10% meminfo.max_used_kB
1041 ± 7% -82.7% 180.00 ± 69% numa-meminfo.node0.Active(file)
713269 ± 3% +73.5% 1237284 ± 2% numa-meminfo.node0.Inactive
712476 ± 3% +73.6% 1237076 ± 2% numa-meminfo.node0.Inactive(anon)
792.25 ± 27% -73.8% 207.33 ± 48% numa-meminfo.node0.Inactive(file)
8539 ± 2% -14.0% 7342 numa-meminfo.node0.KernelStack
15778 ± 13% +23.6% 19500 ± 8% numa-meminfo.node0.Mapped
61936 -11.1% 55075 ± 2% numa-meminfo.node0.PageTables
77142184 ± 3% +22.9% 94803620 ± 3% numa-meminfo.node1.Active
77142121 ± 3% +22.9% 94803431 ± 3% numa-meminfo.node1.Active(anon)
78351034 ± 3% +24.2% 97283455 ± 3% numa-meminfo.node1.AnonPages
1238824 ± 12% +104.0% 2526732 ± 2% numa-meminfo.node1.Inactive
1238414 ± 12% +104.0% 2526081 ± 2% numa-meminfo.node1.Inactive(anon)
39485 ± 5% +30.0% 51344 ± 7% numa-meminfo.node1.KReclaimable
6764 ± 4% +14.9% 7776 numa-meminfo.node1.KernelStack
13907 ± 13% +68.8% 23470 ± 15% numa-meminfo.node1.Mapped
52452027 ± 5% -36.1% 33505419 ± 9% numa-meminfo.node1.MemFree
79628231 ± 3% +23.8% 98574840 ± 3% numa-meminfo.node1.MemUsed
158507 ± 4% +28.1% 203008 ± 3% numa-meminfo.node1.PageTables
39485 ± 5% +30.0% 51344 ± 7% numa-meminfo.node1.SReclaimable
221.00 ± 15% +131.4% 511.33 ± 31% slabinfo.biovec-128.active_objs
221.00 ± 15% +131.4% 511.33 ± 31% slabinfo.biovec-128.num_objs
82252 ± 2% +12.3% 92376 slabinfo.dentry.active_objs
1993 ± 2% +13.5% 2262 slabinfo.dentry.active_slabs
1993 ± 2% +13.5% 2262 slabinfo.dentry.num_slabs
3972 ± 2% +11.4% 4425 ± 2% slabinfo.files_cache.active_objs
3972 ± 2% +11.4% 4425 ± 2% slabinfo.files_cache.num_objs
22678 ± 5% +14.3% 25913 slabinfo.filp.active_objs
717.25 ± 5% +17.6% 843.33 slabinfo.filp.active_slabs
22968 ± 5% +13.5% 26077 slabinfo.filp.num_objs
717.25 ± 5% +17.6% 843.33 slabinfo.filp.num_slabs
81.25 ± 9% +1136.1% 1004 ± 43% slabinfo.nfs_commit_data.active_objs
81.25 ± 9% +1136.1% 1004 ± 43% slabinfo.nfs_commit_data.num_objs
65.75 ± 19% +941.3% 684.67 ± 53% slabinfo.nfs_read_data.active_objs
65.75 ± 19% +941.3% 684.67 ± 53% slabinfo.nfs_read_data.num_objs
41710 ± 3% +62.0% 67563 slabinfo.radix_tree_node.active_objs
751.75 ± 3% +63.7% 1230 slabinfo.radix_tree_node.active_slabs
42119 ± 3% +63.6% 68894 slabinfo.radix_tree_node.num_objs
751.75 ± 3% +63.7% 1230 slabinfo.radix_tree_node.num_slabs
1587 ± 7% +14.6% 1819 ± 8% slabinfo.skbuff_ext_cache.active_objs
1587 ± 7% +14.7% 1821 ± 8% slabinfo.skbuff_ext_cache.num_objs
357.00 ± 9% +171.5% 969.33 ± 22% slabinfo.skbuff_fclone_cache.active_objs
357.00 ± 9% +171.5% 969.33 ± 22% slabinfo.skbuff_fclone_cache.num_objs
9323 +21.0% 11281 ± 3% slabinfo.vmap_area.active_objs
9339 +20.8% 11282 ± 3% slabinfo.vmap_area.num_objs
257.75 ± 6% -82.5% 45.00 ± 69% numa-vmstat.node0.nr_active_file
41.25 ± 39% +1416.8% 625.67 ± 48% numa-vmstat.node0.nr_dirtied
176027 ± 2% +73.7% 305747 ± 3% numa-vmstat.node0.nr_inactive_anon
194.75 ± 25% -73.6% 51.33 ± 49% numa-vmstat.node0.nr_inactive_file
8540 ± 2% -14.0% 7342 numa-vmstat.node0.nr_kernel_stack
3951 ± 13% +26.0% 4977 ± 8% numa-vmstat.node0.nr_mapped
15365 -11.3% 13635 ± 2% numa-vmstat.node0.nr_page_table_pages
257.75 ± 6% -82.7% 44.67 ± 69% numa-vmstat.node0.nr_zone_active_file
176136 ± 2% +73.7% 305881 ± 3% numa-vmstat.node0.nr_zone_inactive_anon
194.00 ± 25% -73.5% 51.33 ± 49% numa-vmstat.node0.nr_zone_inactive_file
7706557 ± 6% +59.4% 12280502 ± 3% numa-vmstat.node0.numa_foreign
7881976 +15.1% 9074037 ± 2% numa-vmstat.node0.numa_hit
7744537 +15.2% 8920222 ± 2% numa-vmstat.node0.numa_local
227000 ± 52% +85.0% 419979 ± 15% numa-vmstat.node0.numa_miss
364448 ± 32% +57.4% 573797 ± 12% numa-vmstat.node0.numa_other
19143467 ± 3% +22.7% 23496200 ± 3% numa-vmstat.node1.nr_active_anon
19443323 ± 3% +24.0% 24107902 ± 3% numa-vmstat.node1.nr_anon_pages
11.25 ±168% +6092.6% 696.67 ± 23% numa-vmstat.node1.nr_dirtied
32665 ± 3% -54.3% 14934 ± 7% numa-vmstat.node1.nr_free_cma
13257702 ± 4% -35.2% 8589733 ± 10% numa-vmstat.node1.nr_free_pages
307118 ± 12% +102.9% 623237 ± 3% numa-vmstat.node1.nr_inactive_anon
32.75 ± 23% +289.8% 127.67 ± 2% numa-vmstat.node1.nr_isolated_anon
6765 ± 4% +14.9% 7773 numa-vmstat.node1.nr_kernel_stack
3627 ± 14% +61.5% 5858 ± 15% numa-vmstat.node1.nr_mapped
39352 ± 4% +27.7% 50243 ± 3% numa-vmstat.node1.nr_page_table_pages
9945 ± 5% +28.7% 12796 ± 7% numa-vmstat.node1.nr_slab_reclaimable
1441846 ± 21% +56.3% 2254231 ± 9% numa-vmstat.node1.nr_vmscan_write
1441849 ± 21% +56.4% 2254915 ± 9% numa-vmstat.node1.nr_written
19143456 ± 3% +22.7% 23496195 ± 3% numa-vmstat.node1.nr_zone_active_anon
307119 ± 12% +102.9% 623237 ± 3% numa-vmstat.node1.nr_zone_inactive_anon
227115 ± 52% +85.0% 420194 ± 15% numa-vmstat.node1.numa_foreign
17315459 ± 5% +26.1% 21840232 ± 3% numa-vmstat.node1.numa_hit
17256929 ± 5% +26.3% 21794912 ± 3% numa-vmstat.node1.numa_local
7707341 ± 6% +59.3% 12281115 ± 3% numa-vmstat.node1.numa_miss
7765889 ± 6% +58.7% 12326444 ± 3% numa-vmstat.node1.numa_other
1.25 ± 87% +3126.7% 40.33 ±108% numa-vmstat.node1.workingset_nodes
10353 ±123% +219.7% 33104 ± 2% proc-vmstat.compact_daemon_migrate_scanned
1888 ± 49% +325.1% 8028 ± 8% proc-vmstat.compact_fail
10353 ±123% +219.7% 33104 ± 2% proc-vmstat.compact_migrate_scanned
1888 ± 49% +325.2% 8030 ± 8% proc-vmstat.compact_stall
25092321 ± 2% +16.8% 29316619 ± 3% proc-vmstat.nr_active_anon
276.00 ± 3% -66.9% 91.33 ± 11% proc-vmstat.nr_active_file
25568624 ± 2% +18.3% 30244462 ± 3% proc-vmstat.nr_anon_pages
65.00 ± 24% +2683.6% 1809 ± 12% proc-vmstat.nr_dirtied
1449552 ± 3% -32.0% 986067 ± 10% proc-vmstat.nr_dirty_background_threshold
2902650 ± 3% -32.0% 1974547 ± 10% proc-vmstat.nr_dirty_threshold
276121 +2.8% 283935 proc-vmstat.nr_file_pages
32040 ± 4% -54.9% 14439 ± 6% proc-vmstat.nr_free_cma
14635098 ± 3% -31.8% 9975596 ± 10% proc-vmstat.nr_free_pages
490377 ± 6% +92.1% 941776 ± 2% proc-vmstat.nr_inactive_anon
300.25 ± 3% -28.9% 213.33 ± 21% proc-vmstat.nr_inactive_file
55.25 ± 16% +265.6% 202.00 ± 3% proc-vmstat.nr_isolated_anon
15301 -1.1% 15126 proc-vmstat.nr_kernel_stack
7563 +43.4% 10848 ± 4% proc-vmstat.nr_mapped
55169 ± 3% +17.0% 64572 ± 3% proc-vmstat.nr_page_table_pages
20862 +17.7% 24545 proc-vmstat.nr_slab_reclaimable
40904 +3.1% 42163 proc-vmstat.nr_slab_unreclaimable
261601 +3.0% 269429 proc-vmstat.nr_unevictable
2405495 ± 10% +46.5% 3524586 ± 9% proc-vmstat.nr_vmscan_write
25092249 ± 2% +16.8% 29316482 ± 3% proc-vmstat.nr_zone_active_anon
275.75 ± 3% -67.0% 91.00 ± 12% proc-vmstat.nr_zone_active_file
490475 ± 6% +92.0% 941924 ± 2% proc-vmstat.nr_zone_inactive_anon
299.75 ± 2% -28.8% 213.33 ± 21% proc-vmstat.nr_zone_inactive_file
261601 +3.0% 269430 proc-vmstat.nr_zone_unevictable
13602156 ± 4% +7.3% 14596636 ± 3% proc-vmstat.numa_foreign
35948444 -3.5% 34680186 proc-vmstat.numa_hit
35917370 -3.5% 34648547 proc-vmstat.numa_local
13602156 ± 4% +7.3% 14596636 ± 3% proc-vmstat.numa_miss
13633230 ± 4% +7.3% 14628275 ± 3% proc-vmstat.numa_other
5588835 ± 27% +336.2% 24380925 ± 10% proc-vmstat.numa_pte_updates
5204 ± 4% +201.5% 15693 proc-vmstat.pgactivate
49648820 +3.5% 51363583 proc-vmstat.pgfree
2677 ± 4% +63.5% 4378 ± 44% proc-vmstat.pgmajfault
0.25 ±173% +4.8e+05% 1195 ±141% proc-vmstat.pgmigrate_fail
2939659 ± 10% -44.1% 1641880 ± 10% proc-vmstat.pgscan_kswapd
2937571 ± 10% -44.2% 1638222 ± 10% proc-vmstat.pgsteal_kswapd
36379 +26416.6% 9646476 ± 25% proc-vmstat.slabs_scanned
733.25 ± 7% +86.8% 1370 ± 17% proc-vmstat.swap_ra
576.25 ± 8% +64.7% 949.00 ± 15% proc-vmstat.swap_ra_hit
16.00 ± 4% +37393.8% 5999 ± 24% proc-vmstat.unevictable_pgs_culled
6.76 ± 3% -7.4% 6.26 ± 6% perf-stat.i.MPKI
6.754e+09 -73.1% 1.814e+09 ± 7% perf-stat.i.branch-instructions
23049236 -74.2% 5947053 ± 3% perf-stat.i.branch-misses
37.40 ± 3% -11.4 26.01 ± 3% perf-stat.i.cache-miss-rate%
55113848 ± 4% -82.2% 9792343 ± 9% perf-stat.i.cache-misses
1.419e+08 ± 3% -75.4% 34878738 ± 5% perf-stat.i.cache-references
7468 ± 8% -31.1% 5145 ± 2% perf-stat.i.context-switches
1.14 +621.9% 8.23 perf-stat.i.cpi
2.592e+10 +48.0% 3.835e+10 ± 3% perf-stat.i.cpu-cycles
131.22 +10.4% 144.87 perf-stat.i.cpu-migrations
568.39 ± 4% +880.0% 5570 ± 3% perf-stat.i.cycles-between-cache-misses
0.02 ± 12% -0.0 0.01 ± 45% perf-stat.i.dTLB-load-miss-rate%
619825 ± 7% -88.7% 69818 ± 17% perf-stat.i.dTLB-load-misses
6.232e+09 -71.8% 1.755e+09 ± 6% perf-stat.i.dTLB-loads
0.21 ± 2% -0.1 0.08 ± 6% perf-stat.i.dTLB-store-miss-rate%
5684367 -83.6% 929886 ± 11% perf-stat.i.dTLB-store-misses
2.382e+09 -79.1% 4.974e+08 ± 6% perf-stat.i.dTLB-stores
59.54 -9.7 49.81 perf-stat.i.iTLB-load-miss-rate%
4835144 ± 6% -65.4% 1674858 ± 2% perf-stat.i.iTLB-load-misses
2.449e+10 -71.3% 7.019e+09 ± 6% perf-stat.i.instructions
13139 -66.9% 4353 ± 7% perf-stat.i.instructions-per-iTLB-miss
0.98 -77.3% 0.22 ± 6% perf-stat.i.ipc
60.11 ± 5% -84.2% 9.47 ± 6% perf-stat.i.major-faults
0.27 +52.5% 0.41 ± 3% perf-stat.i.metric.GHz
0.19 ± 24% +301.6% 0.74 ± 3% perf-stat.i.metric.K/sec
161.46 -73.1% 43.49 ± 6% perf-stat.i.metric.M/sec
1102490 -83.6% 180444 ± 11% perf-stat.i.minor-faults
77.68 +4.9 82.55 perf-stat.i.node-load-miss-rate%
7350049 ± 5% -82.0% 1325602 ± 8% perf-stat.i.node-load-misses
1863257 ± 4% -84.3% 293219 ± 8% perf-stat.i.node-loads
59.91 ± 7% +10.2 70.12 ± 2% perf-stat.i.node-store-miss-rate%
4561753 ± 7% -82.0% 823146 ± 7% perf-stat.i.node-store-misses
2697430 ± 15% -79.2% 561007 ± 8% perf-stat.i.node-stores
1102550 -83.6% 180454 ± 11% perf-stat.i.page-faults
5.80 ± 3% -11.7% 5.12 ± 2% perf-stat.overall.MPKI
38.81 -11.3 27.49 ± 4% perf-stat.overall.cache-miss-rate%
1.06 +440.0% 5.72 ± 4% perf-stat.overall.cpi
471.06 ± 4% +763.4% 4067 ± 6% perf-stat.overall.cycles-between-cache-misses
0.01 ± 7% -0.0 0.01 ± 44% perf-stat.overall.dTLB-load-miss-rate%
0.24 -0.1 0.18 ± 5% perf-stat.overall.dTLB-store-miss-rate%
73.85 -23.8 50.03 perf-stat.overall.iTLB-load-miss-rate%
5082 ± 7% -22.4% 3945 ± 9% perf-stat.overall.instructions-per-iTLB-miss
0.94 -81.4% 0.18 ± 4% perf-stat.overall.ipc
79.75 +2.1 81.87 perf-stat.overall.node-load-miss-rate%
4953 +88.6% 9340 ± 6% perf-stat.overall.path-length
6.606e+09 -74.7% 1.668e+09 ± 10% perf-stat.ps.branch-instructions
22608775 -75.2% 5605292 ± 5% perf-stat.ps.branch-misses
53930981 ± 4% -83.0% 9162780 ± 11% perf-stat.ps.cache-misses
1.389e+08 ± 3% -76.1% 33217475 ± 7% perf-stat.ps.cache-references
7311 ± 8% -31.2% 5026 perf-stat.ps.context-switches
93863 +1.9% 95670 perf-stat.ps.cpu-clock
2.536e+10 +45.8% 3.697e+10 ± 4% perf-stat.ps.cpu-cycles
128.53 +7.9% 138.70 ± 4% perf-stat.ps.cpu-migrations
607037 ± 7% -86.5% 81773 ± 36% perf-stat.ps.dTLB-load-misses
6.096e+09 -73.3% 1.625e+09 ± 9% perf-stat.ps.dTLB-loads
5558677 -85.3% 817456 ± 15% perf-stat.ps.dTLB-store-misses
2.33e+09 -80.4% 4.572e+08 ± 9% perf-stat.ps.dTLB-stores
4734527 ± 6% -65.2% 1646334 perf-stat.ps.iTLB-load-misses
2.395e+10 -72.9% 6.498e+09 ± 9% perf-stat.ps.instructions
58.86 ± 5% -84.7% 9.02 ± 5% perf-stat.ps.major-faults
1078073 -85.3% 158082 ± 16% perf-stat.ps.minor-faults
7194716 ± 5% -82.5% 1257009 ± 10% perf-stat.ps.node-load-misses
1824407 ± 4% -84.7% 278752 ± 11% perf-stat.ps.node-loads
4460797 ± 7% -83.0% 756410 ± 11% perf-stat.ps.node-store-misses
2640492 ± 15% -80.9% 504536 ± 11% perf-stat.ps.node-stores
1078132 -85.3% 158091 ± 16% perf-stat.ps.page-faults
93863 +1.9% 95670 perf-stat.ps.task-clock
1.09e+12 +85.9% 2.026e+12 ± 4% perf-stat.total.instructions
0.02 ± 81% +1.2e+08% 21164 ± 16% sched_debug.cfs_rq:/.exec_clock.avg
0.92 ± 77% +1.4e+07% 127531 ± 18% sched_debug.cfs_rq:/.exec_clock.max
0.11 ± 76% +1.5e+07% 16164 ± 24% sched_debug.cfs_rq:/.exec_clock.stddev
17284 ± 23% +326.3% 73680 ± 14% sched_debug.cfs_rq:/.load.avg
135741 ± 33% +66.7% 226288 ± 9% sched_debug.cfs_rq:/.load.stddev
1028 +9.7% 1128 ± 4% sched_debug.cfs_rq:/.load_avg.max
14965 ± 17% +1150.1% 187086 ± 13% sched_debug.cfs_rq:/.min_vruntime.avg
31992 ± 13% +1045.6% 366502 ± 10% sched_debug.cfs_rq:/.min_vruntime.max
8981 ± 24% +1017.6% 100380 ± 22% sched_debug.cfs_rq:/.min_vruntime.min
3372 ± 4% +1501.5% 54013 ± 9% sched_debug.cfs_rq:/.min_vruntime.stddev
0.14 ± 7% +38.2% 0.19 ± 5% sched_debug.cfs_rq:/.nr_running.avg
24.12 ± 73% -92.3% 1.87 ± 11% sched_debug.cfs_rq:/.removed.load_avg.avg
1018 -83.2% 171.17 ± 12% sched_debug.cfs_rq:/.removed.load_avg.max
144.98 ± 35% -87.7% 17.78 ± 11% sched_debug.cfs_rq:/.removed.load_avg.stddev
1112 ± 73% -92.2% 86.36 ± 11% sched_debug.cfs_rq:/.removed.runnable_sum.avg
46957 -83.1% 7914 ± 12% sched_debug.cfs_rq:/.removed.runnable_sum.max
6679 ± 35% -87.7% 821.98 ± 11% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
8.16 ± 83% -91.7% 0.68 ± 33% sched_debug.cfs_rq:/.removed.util_avg.avg
395.25 ± 40% -84.1% 62.96 ± 36% sched_debug.cfs_rq:/.removed.util_avg.max
50.76 ± 55% -87.2% 6.51 ± 34% sched_debug.cfs_rq:/.removed.util_avg.stddev
2.20 ± 8% +2562.8% 58.63 ± 18% sched_debug.cfs_rq:/.runnable_load_avg.avg
40.75 ± 68% +2053.4% 877.49 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.max
6.71 ± 27% +2795.2% 194.19 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.stddev
15922 ± 29% +361.8% 73525 ± 14% sched_debug.cfs_rq:/.runnable_weight.avg
134704 ± 34% +67.7% 225909 ± 9% sched_debug.cfs_rq:/.runnable_weight.stddev
2199 ± 64% -3699.3% -79153 sched_debug.cfs_rq:/.spread0.avg
19225 ± 27% +422.8% 100520 ± 49% sched_debug.cfs_rq:/.spread0.max
-3785 +4284.8% -165992 sched_debug.cfs_rq:/.spread0.min
3372 ± 4% +1503.5% 54080 ± 9% sched_debug.cfs_rq:/.spread0.stddev
266.83 ± 3% -22.3% 207.34 ± 7% sched_debug.cfs_rq:/.util_avg.avg
1452 ± 17% -23.1% 1116 ± 7% sched_debug.cfs_rq:/.util_avg.max
23.15 ± 24% +351.8% 104.60 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.avg
101.20 ± 18% +158.9% 262.04 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.stddev
723046 +28.8% 931633 sched_debug.cpu.avg_idle.avg
8921 ±136% +3367.2% 309320 ± 6% sched_debug.cpu.avg_idle.min
314848 ± 6% -58.0% 132096 ± 5% sched_debug.cpu.avg_idle.stddev
28760 +533.3% 182149 ± 13% sched_debug.cpu.clock.avg
28764 +533.6% 182260 ± 13% sched_debug.cpu.clock.max
28756 +533.1% 182049 ± 13% sched_debug.cpu.clock.min
2.36 ± 9% +2660.5% 65.27 ± 41% sched_debug.cpu.clock.stddev
28760 +533.3% 182149 ± 13% sched_debug.cpu.clock_task.avg
28764 +533.6% 182260 ± 13% sched_debug.cpu.clock_task.max
28756 +533.1% 182049 ± 13% sched_debug.cpu.clock_task.min
2.36 ± 9% +2660.4% 65.27 ± 41% sched_debug.cpu.clock_task.stddev
230.67 ± 12% +23.0% 283.61 ± 4% sched_debug.cpu.curr->pid.avg
2078 +142.3% 5035 ± 9% sched_debug.cpu.curr->pid.max
619.98 ± 6% +36.5% 846.49 ± 6% sched_debug.cpu.curr->pid.stddev
0.00 ± 5% +173.2% 0.00 ± 31% sched_debug.cpu.next_balance.stddev
1666 +492.9% 9880 ± 11% sched_debug.cpu.nr_switches.avg
8085 ± 22% +577.2% 54755 ± 9% sched_debug.cpu.nr_switches.max
470.25 ± 12% +448.2% 2577 ± 10% sched_debug.cpu.nr_switches.min
1060 ± 8% +813.3% 9687 ± 9% sched_debug.cpu.nr_switches.stddev
-58.50 -27.4% -42.47 sched_debug.cpu.nr_uninterruptible.min
0.81 ±100% +1e+06% 8263 ± 14% sched_debug.cpu.sched_count.avg
18.50 ±130% +2.8e+05% 52205 ± 9% sched_debug.cpu.sched_count.max
3.67 ±120% +2.6e+05% 9512 ± 9% sched_debug.cpu.sched_count.stddev
0.41 ± 97% +8.8e+05% 3601 ± 14% sched_debug.cpu.sched_goidle.avg
9.25 ±130% +2.8e+05% 25571 ± 9% sched_debug.cpu.sched_goidle.max
1.84 ±119% +2.6e+05% 4754 ± 9% sched_debug.cpu.sched_goidle.stddev
0.36 ±123% +1.1e+06% 3997 ± 14% sched_debug.cpu.ttwu_count.avg
7.25 ±134% +4.1e+05% 29797 ± 2% sched_debug.cpu.ttwu_count.max
1.44 ±125% +3.5e+05% 5022 ± 6% sched_debug.cpu.ttwu_count.stddev
0.01 ±173% +8.8e+06% 782.74 ± 17% sched_debug.cpu.ttwu_local.avg
0.25 ±173% +8.7e+05% 2176 ± 28% sched_debug.cpu.ttwu_local.max
0.05 ±173% +6.9e+05% 319.05 ± 22% sched_debug.cpu.ttwu_local.stddev
28757 +533.0% 182050 ± 13% sched_debug.cpu_clk
28251 +542.5% 181508 ± 13% sched_debug.ktime
29119 +526.5% 182429 ± 13% sched_debug.sched_clk
36688 ± 39% -60.6% 14451 ± 18% softirqs.CPU0.RCU
12193 ± 4% +305.1% 49394 ± 7% softirqs.CPU0.SCHED
22020 ± 6% +465.9% 124620 ± 12% softirqs.CPU0.TIMER
6532 ± 32% +458.7% 36495 ± 9% softirqs.CPU1.SCHED
22332 ± 7% +445.4% 121803 ± 11% softirqs.CPU1.TIMER
5556 ± 13% +601.2% 38958 ± 9% softirqs.CPU10.SCHED
21976 ± 5% +545.3% 141801 ± 18% softirqs.CPU10.TIMER
6146 ± 21% +513.9% 37735 ± 10% softirqs.CPU11.SCHED
21510 ± 2% +474.1% 123501 ± 11% softirqs.CPU11.TIMER
6910 ± 40% +442.5% 37486 ± 12% softirqs.CPU12.SCHED
21979 ± 3% +460.8% 123261 ± 10% softirqs.CPU12.TIMER
7494 ± 17% +400.1% 37479 ± 10% softirqs.CPU13.SCHED
21998 ± 6% +460.2% 123230 ± 11% softirqs.CPU13.TIMER
7486 ± 31% +69.0% 12649 ± 15% softirqs.CPU14.RCU
7354 ± 21% +415.3% 37902 ± 10% softirqs.CPU14.SCHED
22857 ± 4% +433.4% 121921 ± 10% softirqs.CPU14.TIMER
5572 ± 33% +584.7% 38156 ± 10% softirqs.CPU15.SCHED
22447 ± 4% +447.9% 122977 ± 10% softirqs.CPU15.TIMER
6945 ± 2% +430.3% 36828 ± 12% softirqs.CPU16.SCHED
22646 ± 7% +439.4% 122151 ± 11% softirqs.CPU16.TIMER
7728 ± 26% +74.9% 13518 ± 18% softirqs.CPU17.RCU
6935 +438.4% 37342 ± 13% softirqs.CPU17.SCHED
22983 ± 8% +434.9% 122933 ± 11% softirqs.CPU17.TIMER
6346 ± 11% +508.7% 38628 ± 10% softirqs.CPU18.SCHED
22324 ± 7% +451.8% 123194 ± 11% softirqs.CPU18.TIMER
20405 ± 27% -38.2% 12618 ± 16% softirqs.CPU19.RCU
6280 ± 8% +528.9% 39500 ± 6% softirqs.CPU19.SCHED
22052 ± 5% +453.7% 122092 ± 11% softirqs.CPU19.TIMER
5597 ± 19% +548.2% 36280 ± 7% softirqs.CPU2.SCHED
23466 ± 14% +419.7% 121952 ± 10% softirqs.CPU2.TIMER
38152 ± 29% -62.6% 14267 ± 11% softirqs.CPU20.RCU
8398 ± 35% +342.7% 37176 ± 12% softirqs.CPU20.SCHED
22273 ± 7% +443.2% 120993 ± 11% softirqs.CPU20.TIMER
6796 ± 3% +446.2% 37120 ± 11% softirqs.CPU21.SCHED
22092 ± 7% +453.1% 122183 ± 11% softirqs.CPU21.TIMER
6590 ± 11% +115.2% 14179 ± 22% softirqs.CPU22.RCU
8729 ± 18% +336.0% 38058 ± 11% softirqs.CPU22.SCHED
22540 ± 7% +448.8% 123706 ± 12% softirqs.CPU22.TIMER
6104 ± 17% +517.0% 37661 ± 9% softirqs.CPU23.SCHED
22902 ± 6% +446.1% 125080 ± 12% softirqs.CPU23.TIMER
5850 ± 14% +493.9% 34745 ± 13% softirqs.CPU24.SCHED
22197 +426.9% 116964 ± 10% softirqs.CPU24.TIMER
5649 ± 14% +564.8% 37554 ± 15% softirqs.CPU25.SCHED
22145 ± 5% +441.1% 119822 ± 10% softirqs.CPU25.TIMER
23123 ± 3% +511.5% 141395 ± 9% softirqs.CPU26.TIMER
4401 ± 16% +716.2% 35925 ± 14% softirqs.CPU27.SCHED
22894 ± 5% +414.3% 117743 ± 7% softirqs.CPU27.TIMER
6128 ± 15% +504.5% 37044 ± 11% softirqs.CPU28.SCHED
23322 ± 7% +409.5% 118819 ± 10% softirqs.CPU28.TIMER
6491 ± 10% +473.1% 37203 ± 10% softirqs.CPU29.SCHED
23701 ± 6% +406.3% 120008 ± 10% softirqs.CPU29.TIMER
5690 ± 20% +583.1% 38873 ± 8% softirqs.CPU3.SCHED
22621 ± 6% +441.8% 122560 ± 11% softirqs.CPU3.TIMER
6782 ± 4% +441.5% 36725 ± 15% softirqs.CPU30.SCHED
23426 ± 7% +399.1% 116914 ± 13% softirqs.CPU30.TIMER
6655 ± 4% +449.2% 36548 ± 9% softirqs.CPU31.SCHED
22760 ± 6% +427.1% 119968 ± 7% softirqs.CPU31.TIMER
6456 +498.8% 38661 ± 11% softirqs.CPU32.SCHED
23092 ± 5% +420.0% 120077 ± 10% softirqs.CPU32.TIMER
6304 ± 11% +485.0% 36879 ± 12% softirqs.CPU33.SCHED
22838 ± 6% +420.5% 118880 ± 10% softirqs.CPU33.TIMER
6685 ± 8% +440.3% 36119 ± 11% softirqs.CPU34.SCHED
23305 ± 7% +401.8% 116950 ± 10% softirqs.CPU34.TIMER
6730 ± 5% +471.1% 38439 ± 10% softirqs.CPU35.SCHED
23368 ± 7% +410.9% 119389 ± 9% softirqs.CPU35.TIMER
6157 +98.3% 12210 ± 18% softirqs.CPU36.RCU
7044 ± 4% +435.9% 37749 ± 11% softirqs.CPU36.SCHED
23157 ± 8% +411.8% 118523 ± 9% softirqs.CPU36.TIMER
6393 ± 13% +490.4% 37744 ± 11% softirqs.CPU37.SCHED
22945 ± 9% +415.9% 118379 ± 9% softirqs.CPU37.TIMER
6797 ± 4% +450.3% 37410 ± 11% softirqs.CPU38.SCHED
23095 ± 8% +421.7% 120485 ± 11% softirqs.CPU38.TIMER
6462 ± 4% +494.1% 38397 ± 12% softirqs.CPU39.SCHED
23003 ± 8% +416.3% 118775 ± 9% softirqs.CPU39.TIMER
6658 ± 13% +461.7% 37400 ± 8% softirqs.CPU4.SCHED
22898 ± 6% +436.0% 122724 ± 11% softirqs.CPU4.TIMER
6663 ± 5% +463.6% 37555 ± 11% softirqs.CPU40.SCHED
23428 ± 6% +405.9% 118523 ± 10% softirqs.CPU40.TIMER
23172 ± 7% +479.8% 134363 ± 22% softirqs.CPU41.TIMER
6681 ± 29% +95.0% 13027 ± 19% softirqs.CPU42.RCU
6761 ± 2% +425.8% 35552 ± 6% softirqs.CPU42.SCHED
22874 ± 8% +419.7% 118879 ± 8% softirqs.CPU42.TIMER
7112 ± 3% +422.5% 37157 ± 10% softirqs.CPU43.SCHED
25900 ± 13% +357.4% 118464 ± 9% softirqs.CPU43.TIMER
6067 ± 5% +111.6% 12839 ± 17% softirqs.CPU44.RCU
7078 ± 2% +432.4% 37685 ± 10% softirqs.CPU44.SCHED
23467 ± 7% +409.3% 119517 ± 10% softirqs.CPU44.TIMER
6417 ± 15% +481.3% 37303 ± 12% softirqs.CPU45.SCHED
23074 ± 8% +508.5% 140417 ± 26% softirqs.CPU45.TIMER
7519 ± 26% +65.2% 12423 ± 13% softirqs.CPU46.RCU
6956 ± 2% +437.2% 37373 ± 14% softirqs.CPU46.SCHED
23184 ± 7% +411.5% 118586 ± 9% softirqs.CPU46.TIMER
6439 ± 11% +101.4% 12965 ± 13% softirqs.CPU47.RCU
5486 ± 3% +572.2% 36874 ± 9% softirqs.CPU47.SCHED
23475 ± 6% +407.4% 119116 ± 10% softirqs.CPU47.TIMER
7008 ± 35% +88.7% 13222 ± 16% softirqs.CPU48.RCU
5881 ± 14% +545.4% 37959 ± 12% softirqs.CPU48.SCHED
21852 ± 6% +473.9% 125408 ± 12% softirqs.CPU48.TIMER
6708 ± 6% +471.3% 38329 ± 11% softirqs.CPU49.SCHED
21856 ± 6% +468.8% 124326 ± 12% softirqs.CPU49.TIMER
8618 ± 36% +343.5% 38219 ± 10% softirqs.CPU5.SCHED
24849 ± 12% +395.1% 123026 ± 11% softirqs.CPU5.TIMER
6985 ± 22% +91.7% 13393 ± 17% softirqs.CPU50.RCU
6749 ± 5% +468.1% 38343 ± 9% softirqs.CPU50.SCHED
21851 ± 6% +475.0% 125647 ± 10% softirqs.CPU50.TIMER
6478 ± 11% +502.4% 39026 ± 10% softirqs.CPU51.SCHED
22332 ± 5% +467.5% 126738 ± 11% softirqs.CPU51.TIMER
7628 ± 26% +65.0% 12585 ± 13% softirqs.CPU52.RCU
6744 ± 12% +489.4% 39748 ± 11% softirqs.CPU52.SCHED
22287 ± 6% +465.9% 126113 ± 12% softirqs.CPU52.TIMER
6361 ± 13% +93.1% 12283 ± 12% softirqs.CPU53.RCU
7273 +438.2% 39150 ± 11% softirqs.CPU53.SCHED
22745 ± 5% +457.9% 126889 ± 11% softirqs.CPU53.TIMER
6768 ± 10% +469.3% 38533 ± 11% softirqs.CPU54.SCHED
22198 ± 6% +470.5% 126633 ± 11% softirqs.CPU54.TIMER
6671 ± 8% +93.0% 12879 ± 17% softirqs.CPU55.RCU
6345 ± 15% +492.7% 37606 ± 14% softirqs.CPU55.SCHED
22178 ± 6% +465.2% 125361 ± 12% softirqs.CPU55.TIMER
6098 ± 13% +115.3% 13130 ± 11% softirqs.CPU56.RCU
6372 ± 14% +510.8% 38918 ± 8% softirqs.CPU56.SCHED
22829 ± 6% +447.5% 124995 ± 10% softirqs.CPU56.TIMER
6253 ± 20% +512.4% 38293 ± 9% softirqs.CPU57.SCHED
22136 ± 8% +472.0% 126610 ± 11% softirqs.CPU57.TIMER
6949 ± 31% +72.3% 11971 ± 15% softirqs.CPU58.RCU
6970 +451.0% 38404 ± 11% softirqs.CPU58.SCHED
22076 ± 6% +468.8% 125570 ± 11% softirqs.CPU58.TIMER
6546 ± 23% +103.9% 13346 ± 13% softirqs.CPU59.RCU
7124 +442.9% 38680 ± 12% softirqs.CPU59.SCHED
22292 ± 5% +469.6% 126980 ± 11% softirqs.CPU59.TIMER
6929 ± 6% +452.9% 38316 ± 14% softirqs.CPU6.SCHED
22690 ± 5% +442.9% 123184 ± 11% softirqs.CPU6.TIMER
6101 ± 5% +112.2% 12946 ± 13% softirqs.CPU60.RCU
6947 ± 4% +457.3% 38718 ± 11% softirqs.CPU60.SCHED
22785 ± 9% +451.7% 125714 ± 11% softirqs.CPU60.TIMER
6909 ± 2% +574.0% 46572 ± 25% softirqs.CPU61.SCHED
22245 ± 5% +464.5% 125576 ± 12% softirqs.CPU61.TIMER
6073 ± 15% +545.4% 39196 ± 9% softirqs.CPU62.SCHED
22429 ± 8% +458.0% 125143 ± 9% softirqs.CPU62.TIMER
6085 ± 24% +98.9% 12103 ± 16% softirqs.CPU63.RCU
6672 ± 7% +482.2% 38847 ± 9% softirqs.CPU63.SCHED
22255 ± 5% +469.1% 126660 ± 10% softirqs.CPU63.TIMER
6906 ± 3% +459.5% 38640 ± 11% softirqs.CPU64.SCHED
22023 ± 6% +472.1% 125995 ± 12% softirqs.CPU64.TIMER
6456 ± 5% +503.6% 38969 ± 10% softirqs.CPU65.SCHED
23126 ± 10% +448.6% 126869 ± 11% softirqs.CPU65.TIMER
6727 ± 4% +513.3% 41259 ± 17% softirqs.CPU66.SCHED
22465 ± 5% +467.5% 127482 ± 11% softirqs.CPU66.TIMER
6641 ± 10% +489.5% 39150 ± 12% softirqs.CPU67.SCHED
22547 ± 5% +446.2% 123143 ± 11% softirqs.CPU67.TIMER
7569 ± 25% +68.6% 12762 ± 13% softirqs.CPU68.RCU
6913 ± 3% +475.1% 39758 ± 11% softirqs.CPU68.SCHED
22058 ± 7% +480.0% 127942 ± 12% softirqs.CPU68.TIMER
7077 ± 9% +448.1% 38792 ± 11% softirqs.CPU69.SCHED
23097 ± 9% +444.0% 125637 ± 11% softirqs.CPU69.TIMER
6610 ± 14% +105.6% 13594 ± 18% softirqs.CPU7.RCU
7036 ± 4% +433.5% 37541 ± 11% softirqs.CPU7.SCHED
22654 ± 5% +441.4% 122652 ± 11% softirqs.CPU7.TIMER
6307 ± 8% +95.1% 12305 ± 17% softirqs.CPU70.RCU
7178 ± 4% +447.8% 39319 ± 10% softirqs.CPU70.SCHED
22725 ± 9% +459.8% 127222 ± 11% softirqs.CPU70.TIMER
6902 ± 4% +456.7% 38426 ± 9% softirqs.CPU71.SCHED
23173 ± 6% +439.5% 125013 ± 11% softirqs.CPU71.TIMER
6936 ± 2% +440.3% 37478 ± 10% softirqs.CPU72.SCHED
22718 ± 7% +427.3% 119795 ± 11% softirqs.CPU72.TIMER
6248 ± 14% +492.2% 37002 ± 9% softirqs.CPU73.SCHED
22736 ± 8% +415.1% 117122 ± 9% softirqs.CPU73.TIMER
22828 ± 7% +518.2% 141117 ± 7% softirqs.CPU74.TIMER
5932 ± 13% +552.1% 38681 ± 11% softirqs.CPU75.SCHED
22651 ± 6% +434.1% 120982 ± 10% softirqs.CPU75.TIMER
5989 ± 12% +551.0% 38989 ± 7% softirqs.CPU76.SCHED
22679 ± 4% +514.7% 139409 ± 5% softirqs.CPU76.TIMER
6342 ± 14% +504.5% 38341 ± 11% softirqs.CPU77.SCHED
22680 ± 7% +448.9% 124493 ± 11% softirqs.CPU77.TIMER
6180 ± 16% +541.6% 39656 ± 6% softirqs.CPU78.SCHED
22959 ± 9% +443.7% 124843 ± 9% softirqs.CPU78.TIMER
15836 ± 26% -24.8% 11911 ± 16% softirqs.CPU79.RCU
6431 ± 11% +493.9% 38195 ± 10% softirqs.CPU79.SCHED
22471 ± 8% +436.6% 120572 ± 9% softirqs.CPU79.TIMER
6116 ± 8% +123.0% 13639 ± 8% softirqs.CPU8.RCU
7066 ± 3% +422.6% 36930 ± 8% softirqs.CPU8.SCHED
23222 ± 7% +425.3% 121998 ± 11% softirqs.CPU8.TIMER
5679 ± 7% +102.9% 11523 ± 18% softirqs.CPU80.RCU
6918 ± 3% +470.6% 39480 ± 7% softirqs.CPU80.SCHED
22973 ± 6% +444.5% 125094 ± 10% softirqs.CPU80.TIMER
7395 ± 28% +64.4% 12155 ± 18% softirqs.CPU81.RCU
6876 ± 3% +456.0% 38229 ± 11% softirqs.CPU81.SCHED
23024 ± 6% +428.9% 121786 ± 12% softirqs.CPU81.TIMER
6278 ± 17% +518.7% 38843 ± 10% softirqs.CPU82.SCHED
23005 ± 6% +436.6% 123442 ± 11% softirqs.CPU82.TIMER
7106 ± 20% +66.5% 11833 ± 16% softirqs.CPU83.RCU
6996 ± 4% +456.4% 38930 ± 9% softirqs.CPU83.SCHED
22885 ± 7% +436.0% 122678 ± 9% softirqs.CPU83.TIMER
6469 ± 12% +499.7% 38794 ± 10% softirqs.CPU84.SCHED
22723 ± 8% +437.5% 122129 ± 11% softirqs.CPU84.TIMER
6769 ± 5% +467.8% 38437 ± 10% softirqs.CPU85.SCHED
22753 ± 10% +433.7% 121443 ± 10% softirqs.CPU85.TIMER
6908 +457.7% 38530 ± 10% softirqs.CPU86.SCHED
22687 ± 7% +440.0% 122516 ± 10% softirqs.CPU86.TIMER
6530 ± 6% +482.1% 38008 ± 11% softirqs.CPU87.SCHED
22574 ± 8% +435.7% 120927 ± 10% softirqs.CPU87.TIMER
6739 +471.2% 38491 ± 11% softirqs.CPU88.SCHED
22813 ± 6% +440.3% 123263 ± 10% softirqs.CPU88.TIMER
6861 ± 7% +303.4% 27680 ± 43% softirqs.CPU89.SCHED
22714 ± 7% +504.2% 137243 ± 21% softirqs.CPU89.TIMER
6086 ± 23% +515.1% 37437 ± 9% softirqs.CPU9.SCHED
27118 ± 20% +356.1% 123681 ± 10% softirqs.CPU9.TIMER
6827 ± 22% +68.7% 11520 ± 18% softirqs.CPU90.RCU
6846 ± 2% +448.7% 37560 ± 8% softirqs.CPU90.SCHED
22693 ± 8% +432.5% 120850 ± 9% softirqs.CPU90.TIMER
5636 ± 9% +98.8% 11207 ± 21% softirqs.CPU91.RCU
6865 ± 3% +454.3% 38054 ± 10% softirqs.CPU91.SCHED
22663 ± 7% +429.1% 119908 ± 10% softirqs.CPU91.TIMER
5713 ± 5% +111.0% 12055 ± 18% softirqs.CPU92.RCU
7080 ± 4% +436.3% 37976 ± 12% softirqs.CPU92.SCHED
23117 ± 6% +434.7% 123599 ± 10% softirqs.CPU92.TIMER
6754 ± 2% +446.7% 36928 ± 8% softirqs.CPU93.SCHED
22554 ± 8% +437.1% 121143 ± 10% softirqs.CPU93.TIMER
6007 ± 19% +86.4% 11197 ± 18% softirqs.CPU94.RCU
6982 +454.1% 38688 ± 10% softirqs.CPU94.SCHED
22658 ± 7% +441.2% 122629 ± 10% softirqs.CPU94.TIMER
6128 ± 18% +97.3% 12093 ± 24% softirqs.CPU95.RCU
6669 ± 6% +471.8% 38136 ± 8% softirqs.CPU95.SCHED
22538 ± 7% +442.8% 122343 ± 10% softirqs.CPU95.TIMER
641599 ± 2% +462.4% 3608346 ± 10% softirqs.SCHED
2187113 ± 5% +443.2% 11881220 ± 10% softirqs.TIMER
0.50 ±173% +40633.3% 203.67 ± 78% interrupts.113:PCI-MSI.31981646-edge.i40e-eth0-TxRx-77
0.00 +1.9e+104% 190.67 ± 99% interrupts.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
0.00 +1.8e+104% 183.00 ± 63% interrupts.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
0.00 +7.1e+103% 71.33 ± 62% interrupts.117:PCI-MSI.31981650-edge.i40e-eth0-TxRx-81
0.50 ±173% +10766.7% 54.33 ± 85% interrupts.120:PCI-MSI.31981653-edge.i40e-eth0-TxRx-84
36.75 ± 8% +672.8% 284.00 interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
28331569 ± 11% -48.9% 14483923 ± 7% interrupts.CAL:Function_call_interrupts
1597643 ± 37% -85.7% 228000 ± 5% interrupts.CPU0.CAL:Function_call_interrupts
91505 ± 2% +617.7% 656759 ± 10% interrupts.CPU0.LOC:Local_timer_interrupts
3990 ± 53% +271.5% 14822 ± 16% interrupts.CPU0.RES:Rescheduling_interrupts
3293773 ± 36% -76.5% 775632 ± 5% interrupts.CPU0.TLB:TLB_shootdowns
91444 ± 2% +618.8% 657273 ± 10% interrupts.CPU1.LOC:Local_timer_interrupts
2970 ± 53% -65.0% 1041 ± 25% interrupts.CPU1.RES:Rescheduling_interrupts
91427 ± 2% +619.1% 657486 ± 10% interrupts.CPU10.LOC:Local_timer_interrupts
91397 ± 2% +618.7% 656886 ± 10% interrupts.CPU11.LOC:Local_timer_interrupts
91424 ± 2% +618.1% 656560 ± 10% interrupts.CPU12.LOC:Local_timer_interrupts
91436 ± 2% +619.4% 657799 ± 10% interrupts.CPU13.LOC:Local_timer_interrupts
91438 ± 2% +619.9% 658306 ± 10% interrupts.CPU14.LOC:Local_timer_interrupts
295.00 ± 60% +360.9% 1359 ± 42% interrupts.CPU14.NMI:Non-maskable_interrupts
295.00 ± 60% +360.9% 1359 ± 42% interrupts.CPU14.PMI:Performance_monitoring_interrupts
91423 ± 2% +619.5% 657765 ± 10% interrupts.CPU15.LOC:Local_timer_interrupts
91473 ± 2% +616.9% 655776 ± 10% interrupts.CPU16.LOC:Local_timer_interrupts
91420 ± 2% +617.9% 656334 ± 9% interrupts.CPU17.LOC:Local_timer_interrupts
579.50 ±104% +176.3% 1601 ± 32% interrupts.CPU17.NMI:Non-maskable_interrupts
579.50 ±104% +176.3% 1601 ± 32% interrupts.CPU17.PMI:Performance_monitoring_interrupts
125.00 ±126% +515.7% 769.67 ± 22% interrupts.CPU17.RES:Rescheduling_interrupts
91432 ± 2% +618.8% 657187 ± 10% interrupts.CPU18.LOC:Local_timer_interrupts
91348 ± 2% +620.3% 658014 ± 10% interrupts.CPU19.LOC:Local_timer_interrupts
91442 ± 2% +617.7% 656305 ± 10% interrupts.CPU2.LOC:Local_timer_interrupts
1607994 ± 30% -88.3% 187593 ± 16% interrupts.CPU20.CAL:Function_call_interrupts
91411 ± 2% +606.9% 646203 ± 10% interrupts.CPU20.LOC:Local_timer_interrupts
2901 ± 19% -77.1% 665.33 ± 58% interrupts.CPU20.NMI:Non-maskable_interrupts
2901 ± 19% -77.1% 665.33 ± 58% interrupts.CPU20.PMI:Performance_monitoring_interrupts
7329 ± 93% -84.2% 1159 ± 67% interrupts.CPU20.RES:Rescheduling_interrupts
3241724 ± 32% -80.3% 640070 ± 13% interrupts.CPU20.TLB:TLB_shootdowns
91406 ± 2% +619.5% 657687 ± 10% interrupts.CPU21.LOC:Local_timer_interrupts
34527 ± 99% +368.6% 161789 ± 19% interrupts.CPU22.CAL:Function_call_interrupts
91424 ± 2% +619.3% 657647 ± 10% interrupts.CPU22.LOC:Local_timer_interrupts
172.75 ± 57% +799.2% 1553 ± 30% interrupts.CPU22.NMI:Non-maskable_interrupts
172.75 ± 57% +799.2% 1553 ± 30% interrupts.CPU22.PMI:Performance_monitoring_interrupts
67878 ±101% +672.9% 524610 ± 19% interrupts.CPU22.TLB:TLB_shootdowns
91444 ± 2% +619.1% 657560 ± 10% interrupts.CPU23.LOC:Local_timer_interrupts
91322 ± 2% +616.4% 654231 ± 10% interrupts.CPU24.LOC:Local_timer_interrupts
91360 ± 2% +609.2% 647901 ± 8% interrupts.CPU25.LOC:Local_timer_interrupts
2895 ± 91% -63.4% 1059 ±100% interrupts.CPU25.NMI:Non-maskable_interrupts
2895 ± 91% -63.4% 1059 ±100% interrupts.CPU25.PMI:Performance_monitoring_interrupts
91300 ± 2% +624.3% 661277 ± 9% interrupts.CPU26.LOC:Local_timer_interrupts
482.00 ± 80% +350.6% 2171 ± 61% interrupts.CPU26.NMI:Non-maskable_interrupts
482.00 ± 80% +350.6% 2171 ± 61% interrupts.CPU26.PMI:Performance_monitoring_interrupts
91312 ± 2% +604.8% 643563 ± 10% interrupts.CPU27.LOC:Local_timer_interrupts
5890 ±116% +2954.1% 179892 ± 20% interrupts.CPU28.CAL:Function_call_interrupts
91293 ± 2% +620.6% 657852 ± 10% interrupts.CPU28.LOC:Local_timer_interrupts
10660 ±122% +5430.5% 589551 ± 20% interrupts.CPU28.TLB:TLB_shootdowns
91201 ± 2% +619.7% 656371 ± 10% interrupts.CPU29.LOC:Local_timer_interrupts
91435 ± 2% +618.6% 657029 ± 10% interrupts.CPU3.LOC:Local_timer_interrupts
91315 ± 2% +599.6% 638802 ± 14% interrupts.CPU30.LOC:Local_timer_interrupts
91237 ± 2% +620.9% 657775 ± 10% interrupts.CPU31.LOC:Local_timer_interrupts
91320 ± 2% +620.7% 658106 ± 10% interrupts.CPU32.LOC:Local_timer_interrupts
91328 ± 2% +620.2% 657720 ± 10% interrupts.CPU33.LOC:Local_timer_interrupts
91329 ± 2% +604.7% 643577 ± 11% interrupts.CPU34.LOC:Local_timer_interrupts
489.50 ± 78% +304.9% 1982 ± 62% interrupts.CPU34.NMI:Non-maskable_interrupts
489.50 ± 78% +304.9% 1982 ± 62% interrupts.CPU34.PMI:Performance_monitoring_interrupts
91330 ± 2% +620.5% 658042 ± 10% interrupts.CPU35.LOC:Local_timer_interrupts
257.50 ± 34% +411.6% 1317 ± 53% interrupts.CPU35.NMI:Non-maskable_interrupts
257.50 ± 34% +411.6% 1317 ± 53% interrupts.CPU35.PMI:Performance_monitoring_interrupts
11062 ±166% +1103.0% 133081 ± 18% interrupts.CPU36.CAL:Function_call_interrupts
91325 ± 2% +620.2% 657759 ± 10% interrupts.CPU36.LOC:Local_timer_interrupts
73.25 ±101% +1184.2% 940.67 ± 65% interrupts.CPU36.RES:Rescheduling_interrupts
21201 ±173% +1910.5% 426249 ± 18% interrupts.CPU36.TLB:TLB_shootdowns
91322 ± 2% +621.6% 658973 ± 10% interrupts.CPU37.LOC:Local_timer_interrupts
91317 ± 2% +619.8% 657278 ± 10% interrupts.CPU38.LOC:Local_timer_interrupts
91311 ± 2% +620.3% 657721 ± 10% interrupts.CPU39.LOC:Local_timer_interrupts
91452 ± 2% +618.2% 656807 ± 10% interrupts.CPU4.LOC:Local_timer_interrupts
91287 ± 2% +619.9% 657174 ± 10% interrupts.CPU40.LOC:Local_timer_interrupts
323.50 ±100% +207.6% 995.00 ± 48% interrupts.CPU40.NMI:Non-maskable_interrupts
323.50 ±100% +207.6% 995.00 ± 48% interrupts.CPU40.PMI:Performance_monitoring_interrupts
115822 ±173% +373.0% 547848 ± 11% interrupts.CPU40.TLB:TLB_shootdowns
91314 ± 2% +621.4% 658707 ± 10% interrupts.CPU41.LOC:Local_timer_interrupts
91323 ± 2% +618.7% 656381 ± 10% interrupts.CPU42.LOC:Local_timer_interrupts
119.75 ± 21% +1207.2% 1565 ± 26% interrupts.CPU42.NMI:Non-maskable_interrupts
119.75 ± 21% +1207.2% 1565 ± 26% interrupts.CPU42.PMI:Performance_monitoring_interrupts
76.50 ±139% +2120.9% 1699 ± 81% interrupts.CPU42.RES:Rescheduling_interrupts
135052 ±159% +286.8% 522441 ± 23% interrupts.CPU42.TLB:TLB_shootdowns
91315 ± 2% +619.8% 657282 ± 10% interrupts.CPU43.LOC:Local_timer_interrupts
308.00 ± 53% +263.3% 1119 ± 28% interrupts.CPU43.NMI:Non-maskable_interrupts
308.00 ± 53% +263.3% 1119 ± 28% interrupts.CPU43.PMI:Performance_monitoring_interrupts
21303 ±160% +635.8% 156751 ± 7% interrupts.CPU44.CAL:Function_call_interrupts
91333 ± 2% +620.0% 657591 ± 10% interrupts.CPU44.LOC:Local_timer_interrupts
132.00 ± 19% +775.0% 1155 ± 17% interrupts.CPU44.NMI:Non-maskable_interrupts
132.00 ± 19% +775.0% 1155 ± 17% interrupts.CPU44.PMI:Performance_monitoring_interrupts
36.25 ± 97% +4102.3% 1523 ± 81% interrupts.CPU44.RES:Rescheduling_interrupts
35668 ±160% +1294.0% 497211 ± 9% interrupts.CPU44.TLB:TLB_shootdowns
91324 ± 2% +619.1% 656671 ± 10% interrupts.CPU45.LOC:Local_timer_interrupts
171.25 ± 47% +1160.5% 2158 ± 43% interrupts.CPU45.NMI:Non-maskable_interrupts
171.25 ± 47% +1160.5% 2158 ± 43% interrupts.CPU45.PMI:Performance_monitoring_interrupts
91319 ± 2% +620.5% 657981 ± 10% interrupts.CPU46.LOC:Local_timer_interrupts
259.25 ± 74% +386.8% 1262 ± 74% interrupts.CPU46.NMI:Non-maskable_interrupts
259.25 ± 74% +386.8% 1262 ± 74% interrupts.CPU46.PMI:Performance_monitoring_interrupts
152.25 ± 93% +605.2% 1073 ± 73% interrupts.CPU46.RES:Rescheduling_interrupts
179949 ±102% +138.9% 429973 ± 16% interrupts.CPU46.TLB:TLB_shootdowns
22158 ±103% +636.1% 163103 ± 15% interrupts.CPU47.CAL:Function_call_interrupts
91316 ± 2% +619.3% 656852 ± 10% interrupts.CPU47.LOC:Local_timer_interrupts
258.00 ± 41% +491.0% 1524 ± 14% interrupts.CPU47.NMI:Non-maskable_interrupts
258.00 ± 41% +491.0% 1524 ± 14% interrupts.CPU47.PMI:Performance_monitoring_interrupts
110.25 ±144% +501.7% 663.33 ± 14% interrupts.CPU47.RES:Rescheduling_interrupts
44623 ±106% +1072.9% 523397 ± 15% interrupts.CPU47.TLB:TLB_shootdowns
91426 ± 2% +618.7% 657099 ± 10% interrupts.CPU48.LOC:Local_timer_interrupts
91442 ± 2% +619.4% 657810 ± 10% interrupts.CPU49.LOC:Local_timer_interrupts
91475 ± 2% +619.8% 658437 ± 10% interrupts.CPU5.LOC:Local_timer_interrupts
91447 ± 2% +619.1% 657562 ± 10% interrupts.CPU50.LOC:Local_timer_interrupts
133055 ± 87% +274.6% 498477 ± 13% interrupts.CPU50.TLB:TLB_shootdowns
91446 ± 2% +619.6% 658011 ± 10% interrupts.CPU51.LOC:Local_timer_interrupts
91442 ± 2% +619.5% 657910 ± 10% interrupts.CPU52.LOC:Local_timer_interrupts
33202 ±137% +230.0% 109566 ± 5% interrupts.CPU53.CAL:Function_call_interrupts
91470 ± 2% +619.8% 658431 ± 10% interrupts.CPU53.LOC:Local_timer_interrupts
381.00 ± 73% +272.5% 1419 ± 27% interrupts.CPU53.NMI:Non-maskable_interrupts
381.00 ± 73% +272.5% 1419 ± 27% interrupts.CPU53.PMI:Performance_monitoring_interrupts
110.25 ±111% +383.7% 533.33 ± 2% interrupts.CPU53.RES:Rescheduling_interrupts
62130 ±139% +481.0% 361006 ± 6% interrupts.CPU53.TLB:TLB_shootdowns
7571 ±117% +1548.1% 124780 ± 24% interrupts.CPU54.CAL:Function_call_interrupts
91450 ± 2% +620.1% 658506 ± 10% interrupts.CPU54.LOC:Local_timer_interrupts
11528 ±117% +3376.7% 400807 ± 22% interrupts.CPU54.TLB:TLB_shootdowns
50740 ± 62% +228.9% 166883 ± 26% interrupts.CPU55.CAL:Function_call_interrupts
91421 ± 2% +620.2% 658453 ± 10% interrupts.CPU55.LOC:Local_timer_interrupts
270.75 ± 48% +372.4% 1279 ± 24% interrupts.CPU55.NMI:Non-maskable_interrupts
270.75 ± 48% +372.4% 1279 ± 24% interrupts.CPU55.PMI:Performance_monitoring_interrupts
180.75 ± 73% +267.5% 664.33 ± 6% interrupts.CPU55.RES:Rescheduling_interrupts
98857 ± 63% +440.1% 533878 ± 27% interrupts.CPU55.TLB:TLB_shootdowns
2726 ±145% +4959.8% 137968 ± 18% interrupts.CPU56.CAL:Function_call_interrupts
91596 ± 2% +618.4% 658045 ± 10% interrupts.CPU56.LOC:Local_timer_interrupts
162.50 ± 48% +668.2% 1248 ± 57% interrupts.CPU56.NMI:Non-maskable_interrupts
162.50 ± 48% +668.2% 1248 ± 57% interrupts.CPU56.PMI:Performance_monitoring_interrupts
30.75 ±146% +4917.9% 1543 ± 77% interrupts.CPU56.RES:Rescheduling_interrupts
4435 ±173% +10216.4% 457608 ± 20% interrupts.CPU56.TLB:TLB_shootdowns
37241 ±171% +310.1% 152741 ± 4% interrupts.CPU57.CAL:Function_call_interrupts
91394 ± 2% +618.1% 656291 ± 10% interrupts.CPU57.LOC:Local_timer_interrupts
70192 ±173% +624.1% 508288 ± 6% interrupts.CPU57.TLB:TLB_shootdowns
91415 ± 2% +619.3% 657592 ± 10% interrupts.CPU58.LOC:Local_timer_interrupts
91463 ± 2% +619.9% 658458 ± 10% interrupts.CPU59.LOC:Local_timer_interrupts
613.25 ± 85% +149.3% 1528 ± 24% interrupts.CPU59.NMI:Non-maskable_interrupts
613.25 ± 85% +149.3% 1528 ± 24% interrupts.CPU59.PMI:Performance_monitoring_interrupts
65.50 ±126% +865.9% 632.67 ± 30% interrupts.CPU59.RES:Rescheduling_interrupts
94936 ±173% +411.2% 485333 ± 31% interrupts.CPU59.TLB:TLB_shootdowns
41173 ±106% +324.2% 174664 ± 15% interrupts.CPU6.CAL:Function_call_interrupts
91443 ± 2% +619.7% 658131 ± 10% interrupts.CPU6.LOC:Local_timer_interrupts
84022 ±105% +596.4% 585147 ± 13% interrupts.CPU6.TLB:TLB_shootdowns
13559 ±163% +1052.6% 156280 ± 25% interrupts.CPU60.CAL:Function_call_interrupts
91424 ± 2% +619.8% 658027 ± 10% interrupts.CPU60.LOC:Local_timer_interrupts
82.50 ±103% +508.1% 501.67 ± 24% interrupts.CPU60.RES:Rescheduling_interrupts
24650 ±168% +1992.2% 515724 ± 26% interrupts.CPU60.TLB:TLB_shootdowns
91451 ± 2% +620.1% 658530 ± 10% interrupts.CPU61.LOC:Local_timer_interrupts
91276 ± 2% +622.2% 659231 ± 10% interrupts.CPU62.LOC:Local_timer_interrupts
91566 +619.4% 658761 ± 10% interrupts.CPU63.LOC:Local_timer_interrupts
79456 ±169% +433.2% 423698 ± 31% interrupts.CPU63.TLB:TLB_shootdowns
91248 +621.0% 657891 ± 10% interrupts.CPU64.LOC:Local_timer_interrupts
694.75 ± 52% +198.5% 2074 ± 16% interrupts.CPU64.NMI:Non-maskable_interrupts
694.75 ± 52% +198.5% 2074 ± 16% interrupts.CPU64.PMI:Performance_monitoring_interrupts
91417 ± 2% +619.6% 657844 ± 9% interrupts.CPU65.LOC:Local_timer_interrupts
329.00 ± 61% +273.0% 1227 ± 33% interrupts.CPU65.NMI:Non-maskable_interrupts
329.00 ± 61% +273.0% 1227 ± 33% interrupts.CPU65.PMI:Performance_monitoring_interrupts
23082 ± 67% +556.6% 151549 ± 41% interrupts.CPU66.CAL:Function_call_interrupts
91407 ± 2% +618.5% 656754 ± 10% interrupts.CPU66.LOC:Local_timer_interrupts
41149 ± 63% +1078.4% 484899 ± 39% interrupts.CPU66.TLB:TLB_shootdowns
91433 ± 2% +620.3% 658575 ± 10% interrupts.CPU67.LOC:Local_timer_interrupts
91411 ± 2% +620.5% 658627 ± 10% interrupts.CPU68.LOC:Local_timer_interrupts
91423 ± 2% +620.4% 658572 ± 10% interrupts.CPU69.LOC:Local_timer_interrupts
36.75 ± 8% +672.8% 284.00 interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
659.50 ± 57% +22709.4% 150428 ± 33% interrupts.CPU7.CAL:Function_call_interrupts
91406 ± 2% +619.9% 658001 ± 10% interrupts.CPU7.LOC:Local_timer_interrupts
209.75 ± 30% +598.1% 1464 ± 37% interrupts.CPU7.NMI:Non-maskable_interrupts
209.75 ± 30% +598.1% 1464 ± 37% interrupts.CPU7.PMI:Performance_monitoring_interrupts
69.50 ± 57% +833.3% 648.67 ± 19% interrupts.CPU7.RES:Rescheduling_interrupts
480.00 ±170% +1e+05% 492449 ± 33% interrupts.CPU7.TLB:TLB_shootdowns
27310 ±112% +400.2% 136617 ± 15% interrupts.CPU70.CAL:Function_call_interrupts
91446 ± 2% +620.6% 658978 ± 10% interrupts.CPU70.LOC:Local_timer_interrupts
185.50 ± 15% +906.1% 1866 ± 19% interrupts.CPU70.NMI:Non-maskable_interrupts
185.50 ± 15% +906.1% 1866 ± 19% interrupts.CPU70.PMI:Performance_monitoring_interrupts
33.75 ± 82% +1382.5% 500.33 ± 18% interrupts.CPU70.RES:Rescheduling_interrupts
48996 ±120% +794.6% 438323 ± 15% interrupts.CPU70.TLB:TLB_shootdowns
91429 ± 2% +619.0% 657394 ± 10% interrupts.CPU71.LOC:Local_timer_interrupts
91329 ± 2% +618.8% 656467 ± 10% interrupts.CPU72.LOC:Local_timer_interrupts
1388 ± 28% +109.1% 2902 ± 33% interrupts.CPU72.NMI:Non-maskable_interrupts
1388 ± 28% +109.1% 2902 ± 33% interrupts.CPU72.PMI:Performance_monitoring_interrupts
91296 ± 2% +587.7% 627805 ± 6% interrupts.CPU73.LOC:Local_timer_interrupts
9516 ± 98% +629.3% 69405 ± 49% interrupts.CPU74.CAL:Function_call_interrupts
91352 ± 2% +623.3% 660713 ± 10% interrupts.CPU74.LOC:Local_timer_interrupts
353.25 ± 81% +350.4% 1591 ± 53% interrupts.CPU74.NMI:Non-maskable_interrupts
353.25 ± 81% +350.4% 1591 ± 53% interrupts.CPU74.PMI:Performance_monitoring_interrupts
79.25 ±148% +292.0% 310.67 ± 65% interrupts.CPU74.RES:Rescheduling_interrupts
14031 ± 98% +1540.9% 230243 ± 54% interrupts.CPU74.TLB:TLB_shootdowns
91317 ± 2% +618.0% 655625 ± 10% interrupts.CPU75.LOC:Local_timer_interrupts
91334 ± 2% +621.1% 658628 ± 10% interrupts.CPU76.LOC:Local_timer_interrupts
0.25 ±173% +81366.7% 203.67 ± 78% interrupts.CPU77.113:PCI-MSI.31981646-edge.i40e-eth0-TxRx-77
91337 ± 2% +619.9% 657559 ± 10% interrupts.CPU77.LOC:Local_timer_interrupts
0.00 +1.9e+104% 190.67 ± 99% interrupts.CPU78.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
91338 ± 2% +620.5% 658069 ± 10% interrupts.CPU78.LOC:Local_timer_interrupts
0.00 +1.8e+104% 182.67 ± 63% interrupts.CPU79.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
531436 ± 49% -73.6% 140073 ± 10% interrupts.CPU79.CAL:Function_call_interrupts
91338 ± 2% +620.6% 658180 ± 10% interrupts.CPU79.LOC:Local_timer_interrupts
1279 ± 57% -61.4% 494.00 ± 5% interrupts.CPU79.RES:Rescheduling_interrupts
1024465 ± 49% -56.6% 444652 ± 9% interrupts.CPU79.TLB:TLB_shootdowns
438.25 +37483.9% 164711 ± 17% interrupts.CPU8.CAL:Function_call_interrupts
91423 ± 2% +618.9% 657275 ± 10% interrupts.CPU8.LOC:Local_timer_interrupts
150.00 ± 12% +1021.3% 1682 ± 63% interrupts.CPU8.NMI:Non-maskable_interrupts
150.00 ± 12% +1021.3% 1682 ± 63% interrupts.CPU8.PMI:Performance_monitoring_interrupts
1.50 ±110% +3.8e+07% 565425 ± 17% interrupts.CPU8.TLB:TLB_shootdowns
8744 ± 98% +1181.2% 112026 ± 25% interrupts.CPU80.CAL:Function_call_interrupts
91335 ± 2% +621.2% 658668 ± 10% interrupts.CPU80.LOC:Local_timer_interrupts
70.75 ± 70% +546.9% 457.67 ± 11% interrupts.CPU80.RES:Rescheduling_interrupts
16253 ±104% +2070.0% 352696 ± 26% interrupts.CPU80.TLB:TLB_shootdowns
0.00 +7.1e+103% 71.00 ± 62% interrupts.CPU81.117:PCI-MSI.31981650-edge.i40e-eth0-TxRx-81
91337 ± 2% +620.6% 658169 ± 10% interrupts.CPU81.LOC:Local_timer_interrupts
193.75 ± 95% +234.5% 648.00 ± 16% interrupts.CPU81.RES:Rescheduling_interrupts
91342 ± 2% +616.5% 654464 ± 10% interrupts.CPU82.LOC:Local_timer_interrupts
180.00 ±107% +244.1% 619.33 ± 16% interrupts.CPU82.RES:Rescheduling_interrupts
91344 ± 2% +621.8% 659347 ± 10% interrupts.CPU83.LOC:Local_timer_interrupts
350.00 ± 75% +222.7% 1129 ± 19% interrupts.CPU83.NMI:Non-maskable_interrupts
350.00 ± 75% +222.7% 1129 ± 19% interrupts.CPU83.PMI:Performance_monitoring_interrupts
183.25 ±120% +486.6% 1075 ± 68% interrupts.CPU83.RES:Rescheduling_interrupts
0.25 ±173% +21233.3% 53.33 ± 87% interrupts.CPU84.120:PCI-MSI.31981653-edge.i40e-eth0-TxRx-84
91331 ± 2% +621.6% 659054 ± 10% interrupts.CPU84.LOC:Local_timer_interrupts
91309 ± 2% +621.7% 658964 ± 10% interrupts.CPU85.LOC:Local_timer_interrupts
91327 ± 2% +621.7% 659107 ± 10% interrupts.CPU86.LOC:Local_timer_interrupts
122.75 ± 68% +302.4% 494.00 ± 16% interrupts.CPU86.RES:Rescheduling_interrupts
464.75 +25290.8% 118003 ± 31% interrupts.CPU87.CAL:Function_call_interrupts
91350 ± 2% +620.7% 658353 ± 10% interrupts.CPU87.LOC:Local_timer_interrupts
3.75 ± 60% +1e+07% 376468 ± 32% interrupts.CPU87.TLB:TLB_shootdowns
510.75 ± 15% +29935.6% 153406 ± 11% interrupts.CPU88.CAL:Function_call_interrupts
91333 ± 2% +620.8% 658330 ± 10% interrupts.CPU88.LOC:Local_timer_interrupts
109.00 ±169% +4.4e+05% 481712 ± 8% interrupts.CPU88.TLB:TLB_shootdowns
91343 ± 2% +621.3% 658818 ± 10% interrupts.CPU89.LOC:Local_timer_interrupts
91400 ± 2% +618.3% 656547 ± 10% interrupts.CPU9.LOC:Local_timer_interrupts
20036 ± 84% +583.1% 136879 ± 3% interrupts.CPU90.CAL:Function_call_interrupts
91525 ± 2% +619.4% 658431 ± 10% interrupts.CPU90.LOC:Local_timer_interrupts
221.25 ± 60% +440.4% 1195 ± 25% interrupts.CPU90.NMI:Non-maskable_interrupts
221.25 ± 60% +440.4% 1195 ± 25% interrupts.CPU90.PMI:Performance_monitoring_interrupts
34753 ± 82% +1197.6% 450948 ± 3% interrupts.CPU90.TLB:TLB_shootdowns
20216 ±100% +513.0% 123921 ± 30% interrupts.CPU91.CAL:Function_call_interrupts
91058 ± 2% +622.4% 657853 ± 10% interrupts.CPU91.LOC:Local_timer_interrupts
266.00 ± 28% +289.2% 1035 ± 35% interrupts.CPU91.NMI:Non-maskable_interrupts
266.00 ± 28% +289.2% 1035 ± 35% interrupts.CPU91.PMI:Performance_monitoring_interrupts
78.25 ± 89% +666.8% 600.00 ± 14% interrupts.CPU91.RES:Rescheduling_interrupts
40088 ±103% +923.6% 410355 ± 30% interrupts.CPU91.TLB:TLB_shootdowns
14262 ±168% +809.1% 129653 ± 17% interrupts.CPU92.CAL:Function_call_interrupts
91323 ± 2% +621.5% 658884 ± 10% interrupts.CPU92.LOC:Local_timer_interrupts
168.25 ± 22% +541.9% 1080 ± 78% interrupts.CPU92.NMI:Non-maskable_interrupts
168.25 ± 22% +541.9% 1080 ± 78% interrupts.CPU92.PMI:Performance_monitoring_interrupts
22.00 ± 98% +2825.8% 643.67 ± 10% interrupts.CPU92.RES:Rescheduling_interrupts
26830 ±173% +1461.8% 419031 ± 14% interrupts.CPU92.TLB:TLB_shootdowns
91329 ± 2% +621.3% 658747 ± 10% interrupts.CPU93.LOC:Local_timer_interrupts
171.75 ± 18% +1043.3% 1963 ± 25% interrupts.CPU93.NMI:Non-maskable_interrupts
171.75 ± 18% +1043.3% 1963 ± 25% interrupts.CPU93.PMI:Performance_monitoring_interrupts
91336 ± 2% +621.9% 659365 ± 10% interrupts.CPU94.LOC:Local_timer_interrupts
190.25 ± 44% +329.1% 816.33 ± 86% interrupts.CPU94.NMI:Non-maskable_interrupts
190.25 ± 44% +329.1% 816.33 ± 86% interrupts.CPU94.PMI:Performance_monitoring_interrupts
46.75 ±165% +1341.0% 673.67 ± 35% interrupts.CPU94.RES:Rescheduling_interrupts
70182 ±173% +390.0% 343907 ± 20% interrupts.CPU94.TLB:TLB_shootdowns
91411 ± 2% +619.9% 658036 ± 10% interrupts.CPU95.LOC:Local_timer_interrupts
219.50 ± 25% +614.5% 1568 ± 9% interrupts.CPU95.NMI:Non-maskable_interrupts
219.50 ± 25% +614.5% 1568 ± 9% interrupts.CPU95.PMI:Performance_monitoring_interrupts
131.75 ±160% +745.0% 1113 ± 46% interrupts.CPU95.RES:Rescheduling_interrupts
64224 ±172% +623.4% 464571 ± 47% interrupts.CPU95.TLB:TLB_shootdowns
8772137 ± 2% +618.8% 63050381 ± 10% interrupts.LOC:Local_timer_interrupts
115013 ± 5% +25.5% 144352 ± 2% interrupts.NMI:Non-maskable_interrupts
115013 ± 5% +25.5% 144352 ± 2% interrupts.PMI:Performance_monitoring_interrupts
45.16 ± 7% -7.3 37.81 perf-profile.calltrace.cycles-pp.do_access
36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
36.34 ± 16% -6.2 30.13 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
36.67 ± 16% -6.0 30.63 perf-profile.calltrace.cycles-pp.secondary_startup_64
5.66 ± 10% -5.7 0.00 perf-profile.calltrace.cycles-pp.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
33.76 ± 18% -5.3 28.51 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
33.39 ± 18% -5.2 28.21 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
4.78 ± 10% -4.8 0.00 perf-profile.calltrace.cycles-pp.__swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec
5.43 ± 8% -3.7 1.71 ± 11% perf-profile.calltrace.cycles-pp.do_rw_once
1.83 ± 9% -1.1 0.75 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
2.42 ± 8% -0.9 1.49 ± 3% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
2.74 ± 11% -0.8 1.89 ± 2% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
2.36 ± 12% -0.8 1.59 ± 2% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
1.05 ± 9% -0.4 0.63 ± 9% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault
1.26 ± 8% -0.3 0.92 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.__lookup_slow.walk_component.link_path_walk.path_parentat.filename_parentat
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.path_parentat
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.link_path_walk.path_parentat.filename_parentat.filename_create.do_mkdirat
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_parentat.filename_parentat.filename_create
0.00 +0.7 0.66 ± 4% perf-profile.calltrace.cycles-pp.path_parentat.filename_parentat.filename_create.do_mkdirat.do_syscall_64
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.filename_create.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.filename_parentat.filename_create.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mkdir
0.00 +0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir
0.00 +0.7 0.68 ± 5% perf-profile.calltrace.cycles-pp.mkdir
0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.page_fault.__libc_fork.forkshell
0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault.__libc_fork.forkshell
0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault.__libc_fork.forkshell
0.00 +0.8 0.77 ± 41% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault.__libc_fork
0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat
0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc.__alloc_file.alloc_empty_file
0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc.__alloc_file
0.00 +1.1 1.07 ± 18% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open
0.00 +1.1 1.08 ± 19% perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2.do_sys_open
0.00 +1.1 1.08 ± 19% perf-profile.calltrace.cycles-pp.__alloc_file.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2
0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write.new_sync_write.vfs_write
0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write.new_sync_write
0.00 +1.2 1.22 ± 37% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pipe_write
0.00 +1.2 1.25 ± 36% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.__slab_alloc.kmem_cache_alloc.__d_alloc.d_alloc.d_alloc_parallel
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.___slab_alloc.__slab_alloc.kmem_cache_alloc.__d_alloc.d_alloc
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc.__d_alloc
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__d_alloc.d_alloc.d_alloc_parallel.__lookup_slow
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.d_alloc.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk
0.00 +1.4 1.37 ± 39% perf-profile.calltrace.cycles-pp.__d_alloc.d_alloc.d_alloc_parallel.__lookup_slow.walk_component
0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.__vmalloc_node_range.copy_process._do_fork.__x64_sys_clone
0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range.copy_process._do_fork
0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range.copy_process
0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.__vmalloc_node_range
0.00 +1.4 1.42 ± 38% perf-profile.calltrace.cycles-pp.__vmalloc_node_range.copy_process._do_fork.__x64_sys_clone.do_syscall_64
0.00 +1.4 1.42 ± 21% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write
0.00 +1.4 1.42 ± 21% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin
0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write
0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.nfs_write_begin.generic_perform_write.nfs_file_write
0.00 +1.4 1.42 ± 20% perf-profile.calltrace.cycles-pp.nfs_write_begin.generic_perform_write.nfs_file_write.new_sync_write.vfs_write
0.00 +1.4 1.43 ± 21% perf-profile.calltrace.cycles-pp.generic_perform_write.nfs_file_write.new_sync_write.vfs_write.ksys_write
0.00 +1.4 1.43 ± 21% perf-profile.calltrace.cycles-pp.nfs_file_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin.generic_perform_write
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.grab_cache_page_write_begin.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.pagecache_get_page.grab_cache_page_write_begin.simple_write_begin.generic_perform_write.__generic_file_write_iter
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write
0.00 +1.8 1.81 ± 27% perf-profile.calltrace.cycles-pp.simple_write_begin.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write
0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.82 ± 27% perf-profile.calltrace.cycles-pp.__GI___libc_write
0.00 +1.8 1.84 ± 18% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open
0.00 +1.8 1.84 ± 18% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open
0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell
0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell
0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork.forkshell
0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
0.51 ±173% +1.8 2.35 ± 8% perf-profile.calltrace.cycles-pp.copy_process._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.8 1.85 ± 18% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_open
0.00 +1.8 1.85 ± 18% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open
0.00 +1.9 1.85 ± 18% perf-profile.calltrace.cycles-pp.__GI___libc_open
0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.page_fault
0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault
0.00 +1.9 1.88 ± 18% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
0.00 +2.4 2.37 ± 24% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +2.4 2.37 ± 24% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.new_slab.___slab_alloc.__slab_alloc.kmem_cache_alloc
0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab.___slab_alloc.__slab_alloc
0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab.___slab_alloc
0.00 +2.4 2.44 ± 15% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.new_slab
0.00 +2.5 2.54 ± 8% perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +2.7 2.68 ± 27% perf-profile.calltrace.cycles-pp.write
0.52 ±173% +2.8 3.32 ± 11% perf-profile.calltrace.cycles-pp.__libc_fork.forkshell
0.00 +2.8 2.82 ± 9% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
0.00 +2.8 2.82 ± 9% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
0.97 ± 17% +3.1 4.05 ± 25% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
0.90 ± 16% +3.1 4.03 ± 25% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +3.2 3.23 ± 6% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page.grab_cache_page_write_begin
0.00 +3.2 3.23 ± 6% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.pagecache_get_page
0.00 +3.2 3.25 ± 9% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
0.52 ±173% +3.3 3.85 ± 9% perf-profile.calltrace.cycles-pp.forkshell
0.00 +3.4 3.41 ± 9% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.rcu_oom_scan
0.00 +3.5 3.49 ± 8% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.rcu_oom_scan.do_shrink_slab
0.49 ± 59% +3.5 4.03 ± 5% perf-profile.calltrace.cycles-pp.shrink_slab.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
0.47 ± 59% +3.6 4.03 ± 5% perf-profile.calltrace.cycles-pp.do_shrink_slab.shrink_slab.shrink_node.do_try_to_free_pages.try_to_free_pages
0.00 +3.6 3.61 ± 25% perf-profile.calltrace.cycles-pp.memcpy_erms.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread
0.00 +3.7 3.73 ± 25% perf-profile.calltrace.cycles-pp.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread.ret_from_fork
33.79 ± 6% +3.9 37.64 ± 2% perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault
0.00 +3.9 3.92 ± 6% perf-profile.calltrace.cycles-pp.rcu_oom_scan.do_shrink_slab.shrink_slab.shrink_node.do_try_to_free_pages
0.00 +4.4 4.35 ± 3% perf-profile.calltrace.cycles-pp.printk.rcu_oom_scan.do_shrink_slab.shrink_slab.shrink_node
0.00 +4.4 4.35 ± 3% perf-profile.calltrace.cycles-pp.vprintk_emit.printk.rcu_oom_scan.do_shrink_slab.shrink_slab
0.00 +4.5 4.49 ± 8% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
19.77 ± 16% +4.9 24.63 ± 11% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault
19.60 ± 16% +5.4 25.02 ± 12% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma
24.02 ± 7% +13.2 37.20 ± 3% perf-profile.calltrace.cycles-pp.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
22.00 ± 7% +14.7 36.68 ± 3% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages
21.63 ± 7% +15.0 36.64 ± 3% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
24.80 ± 6% +17.1 41.89 ± 3% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask
14.09 ± 10% +29.5 43.58 ± 4% perf-profile.calltrace.cycles-pp.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
14.05 ± 10% +29.5 43.57 ± 4% perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec
13.55 ± 10% +30.0 43.55 ± 4% perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list
12.92 ± 10% +30.6 43.47 ± 4% perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list
9.99 ± 6% -9.5 0.47 ± 32% perf-profile.children.cycles-pp.call_function_interrupt
7.22 ± 6% -6.9 0.34 ± 33% perf-profile.children.cycles-pp.smp_call_function_interrupt
45.05 ± 7% -6.8 38.25 perf-profile.children.cycles-pp.do_access
7.03 ± 6% -6.7 0.33 ± 31% perf-profile.children.cycles-pp.flush_smp_call_function_queue
36.34 ± 16% -6.2 30.13 perf-profile.children.cycles-pp.start_secondary
36.69 ± 16% -6.1 30.63 perf-profile.children.cycles-pp.do_idle
36.67 ± 16% -6.0 30.63 perf-profile.children.cycles-pp.secondary_startup_64
36.67 ± 16% -6.0 30.63 perf-profile.children.cycles-pp.cpu_startup_entry
5.97 ± 10% -5.5 0.48 ± 26% perf-profile.children.cycles-pp.pageout
5.81 ± 6% -5.1 0.70 ± 23% perf-profile.children.cycles-pp.rmap_walk_anon
34.08 ± 18% -5.1 28.98 perf-profile.children.cycles-pp.cpuidle_enter_state
34.08 ± 18% -5.1 28.98 perf-profile.children.cycles-pp.cpuidle_enter
5.05 ± 10% -4.7 0.39 ± 25% perf-profile.children.cycles-pp.__swap_writepage
4.97 ± 10% -4.6 0.38 ± 26% perf-profile.children.cycles-pp.bdev_write_page
6.29 ± 9% -4.3 1.97 ± 12% perf-profile.children.cycles-pp.do_rw_once
4.42 ± 10% -4.1 0.33 ± 24% perf-profile.children.cycles-pp.pmem_rw_page
3.39 ± 8% -3.1 0.26 ± 36% perf-profile.children.cycles-pp.try_to_unmap
2.98 ± 9% -2.8 0.21 ± 37% perf-profile.children.cycles-pp.try_to_unmap_one
2.93 ± 11% -2.7 0.22 ± 21% perf-profile.children.cycles-pp.__remove_mapping
2.82 ± 14% -2.7 0.15 ± 25% perf-profile.children.cycles-pp.llist_add_batch
2.78 ± 6% -2.6 0.13 ± 30% perf-profile.children.cycles-pp.flush_tlb_func_common
3.78 ± 7% -2.6 1.13 ± 13% perf-profile.children.cycles-pp._raw_spin_lock
2.77 ± 11% -2.6 0.19 ± 23% perf-profile.children.cycles-pp.pmem_do_bvec
2.75 ± 11% -2.6 0.19 ± 23% perf-profile.children.cycles-pp.write_pmem
2.73 ± 11% -2.5 0.19 ± 23% perf-profile.children.cycles-pp.__memcpy_flushcache
2.58 ± 9% -2.4 0.19 ± 28% perf-profile.children.cycles-pp.add_to_swap
2.46 ± 6% -2.3 0.13 ± 29% perf-profile.children.cycles-pp.llist_reverse_order
2.72 ± 5% -2.2 0.49 ± 15% perf-profile.children.cycles-pp.page_referenced
2.30 ± 8% -2.1 0.19 ± 22% perf-profile.children.cycles-pp.default_send_IPI_mask_sequence_phys
2.85 ± 9% -2.1 0.75 ± 12% perf-profile.children.cycles-pp.get_page_from_freelist
2.12 ± 11% -1.7 0.41 ± 15% perf-profile.children.cycles-pp.__softirqentry_text_start
1.81 ± 8% -1.7 0.14 ± 28% perf-profile.children.cycles-pp.add_to_swap_cache
2.19 ± 5% -1.6 0.57 ± 10% perf-profile.children.cycles-pp.shrink_active_list
1.75 ± 12% -1.6 0.17 ± 38% perf-profile.children.cycles-pp.rcu_core
1.95 ± 8% -1.6 0.37 ± 11% perf-profile.children.cycles-pp.native_irq_return_iret
1.72 ± 9% -1.6 0.15 ± 21% perf-profile.children.cycles-pp.__default_send_IPI_dest_field
1.74 ± 8% -1.5 0.26 ± 23% perf-profile.children.cycles-pp.page_vma_mapped_walk
1.62 ± 9% -1.5 0.14 ± 29% perf-profile.children.cycles-pp.end_page_writeback
1.60 ± 11% -1.5 0.15 ± 44% perf-profile.children.cycles-pp.rcu_do_batch
3.21 ± 9% -1.3 1.86 perf-profile.children.cycles-pp.smp_apic_timer_interrupt
1.61 ± 6% -1.3 0.26 ± 19% perf-profile.children.cycles-pp.page_referenced_one
3.63 ± 9% -1.3 2.32 ± 2% perf-profile.children.cycles-pp.apic_timer_interrupt
1.23 ± 6% -1.2 0.05 ± 72% perf-profile.children.cycles-pp.native_flush_tlb
1.23 ± 11% -1.1 0.11 ± 42% perf-profile.children.cycles-pp.kmem_cache_free
1.16 ± 17% -1.1 0.06 ± 71% perf-profile.children.cycles-pp.smpboot_thread_fn
1.13 ± 17% -1.1 0.05 ± 71% perf-profile.children.cycles-pp.run_ksoftirqd
1.54 ± 10% -1.0 0.51 ± 12% perf-profile.children.cycles-pp.mem_cgroup_try_charge_delay
1.06 ± 11% -1.0 0.09 ± 41% perf-profile.children.cycles-pp.__slab_free
1.36 ± 14% -1.0 0.39 ± 13% perf-profile.children.cycles-pp.down_read_trylock
2.43 ± 8% -0.9 1.50 ± 3% perf-profile.children.cycles-pp.menu_select
0.99 ± 11% -0.9 0.09 ± 28% perf-profile.children.cycles-pp.__delete_from_swap_cache
1.22 ± 7% -0.9 0.32 ± 22% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
1.36 ± 7% -0.9 0.46 ± 13% perf-profile.children.cycles-pp.prep_new_page
0.99 ± 9% -0.9 0.09 ± 26% perf-profile.children.cycles-pp.xas_create
0.98 ± 9% -0.9 0.10 ± 29% perf-profile.children.cycles-pp.xas_create_range
1.25 ± 9% -0.8 0.43 ± 14% perf-profile.children.cycles-pp.mem_cgroup_try_charge
1.30 ± 3% -0.8 0.52 ± 6% perf-profile.children.cycles-pp.irq_exit
1.20 ± 8% -0.8 0.42 ± 12% perf-profile.children.cycles-pp.clear_page_erms
0.84 ± 7% -0.8 0.08 ± 12% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.82 ± 14% -0.8 0.07 ± 23% perf-profile.children.cycles-pp.mem_cgroup_swapout
0.83 ± 16% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_local_pages_wq
0.83 ± 16% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_pages
0.82 ± 15% -0.7 0.08 ± 72% perf-profile.children.cycles-pp.drain_pages_zone
0.81 ± 8% -0.7 0.08 ± 29% perf-profile.children.cycles-pp.xas_alloc
0.83 ± 11% -0.7 0.11 ± 4% perf-profile.children.cycles-pp._find_next_bit
0.80 ± 16% -0.7 0.09 ± 71% perf-profile.children.cycles-pp.free_pcppages_bulk
0.81 ± 5% -0.7 0.16 ± 21% perf-profile.children.cycles-pp.page_lock_anon_vma_read
0.69 ± 12% -0.6 0.05 ± 72% perf-profile.children.cycles-pp.xas_store
0.64 ± 9% -0.6 0.04 ± 71% perf-profile.children.cycles-pp.swap_writepage
0.66 ± 7% -0.6 0.08 ± 17% perf-profile.children.cycles-pp.cpumask_next
0.75 ± 9% -0.5 0.25 ± 11% perf-profile.children.cycles-pp.__lru_cache_add
0.72 ± 8% -0.5 0.22 ± 11% perf-profile.children.cycles-pp.pagevec_lru_move_fn
0.74 ± 11% -0.5 0.25 ± 15% perf-profile.children.cycles-pp.lru_cache_add_active_or_unevictable
0.68 ± 13% -0.5 0.21 ± 7% perf-profile.children.cycles-pp.up_read
0.53 ± 15% -0.4 0.10 ± 16% perf-profile.children.cycles-pp.count_shadow_nodes
1.57 ± 8% -0.4 1.15 perf-profile.children.cycles-pp.hrtimer_interrupt
0.44 ± 11% -0.4 0.04 ± 73% perf-profile.children.cycles-pp.call_rcu
0.40 ± 8% -0.4 0.04 ± 71% perf-profile.children.cycles-pp.radix_tree_node_ctor
0.43 ± 10% -0.3 0.08 ± 17% perf-profile.children.cycles-pp.isolate_lru_pages
0.41 ± 15% -0.3 0.06 ± 13% perf-profile.children.cycles-pp.__list_del_entry_valid
0.46 ± 9% -0.3 0.13 ± 12% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.99 ± 19% -0.3 0.66 ± 6% perf-profile.children.cycles-pp.ktime_get
0.37 ± 10% -0.3 0.04 ± 76% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.40 ± 9% -0.2 0.15 ± 12% perf-profile.children.cycles-pp.sync_regs
0.39 ± 6% -0.2 0.15 ± 11% perf-profile.children.cycles-pp.__pagevec_lru_add_fn
0.79 ± 5% -0.2 0.57 perf-profile.children.cycles-pp.__hrtimer_run_queues
0.28 ± 6% -0.2 0.07 ± 20% perf-profile.children.cycles-pp.__mod_lruvec_state
0.59 ± 4% -0.2 0.40 ± 2% perf-profile.children.cycles-pp.tick_sched_timer
0.31 ± 13% -0.2 0.13 ± 14% perf-profile.children.cycles-pp.try_charge
0.29 ± 16% -0.2 0.11 ± 19% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.24 ± 10% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.move_pages_to_lru
0.59 ± 12% -0.2 0.44 ± 3% perf-profile.children.cycles-pp.clockevents_program_event
0.20 ± 7% -0.1 0.06 ± 8% perf-profile.children.cycles-pp.__perf_sw_event
0.20 ± 17% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.mem_cgroup_throttle_swaprate
0.43 ± 6% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.update_process_times
0.43 ± 6% -0.1 0.30 ± 5% perf-profile.children.cycles-pp.tick_sched_handle
0.51 ± 14% -0.1 0.39 ± 14% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.15 ± 15% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.mem_cgroup_commit_charge
0.17 ± 19% -0.1 0.07 ± 25% perf-profile.children.cycles-pp.__sched_text_start
0.20 ± 9% -0.1 0.12 ± 23% perf-profile.children.cycles-pp.irq_work_run_list
0.11 ± 28% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.schedule
0.18 ± 6% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.scheduler_tick
0.20 ± 4% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.18 ± 21% -0.0 0.13 perf-profile.children.cycles-pp.rebalance_domains
0.15 ± 6% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.native_write_msr
0.12 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.lapic_next_deadline
0.14 ± 5% -0.0 0.11 ± 8% perf-profile.children.cycles-pp.__next_timer_interrupt
0.09 ± 24% -0.0 0.06 ± 16% perf-profile.children.cycles-pp._raw_spin_trylock
0.09 ± 9% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.read_tsc
0.08 ± 10% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.sched_clock_cpu
0.07 ± 22% -0.0 0.06 ± 16% perf-profile.children.cycles-pp.run_local_timers
0.00 +0.1 0.09 ± 36% perf-profile.children.cycles-pp.fbcon_putcs
0.00 +0.1 0.09 ± 36% perf-profile.children.cycles-pp.bit_putcs
0.00 +0.1 0.09 ± 39% perf-profile.children.cycles-pp.fbcon_redraw
0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.lf
0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.con_scroll
0.00 +0.1 0.10 ± 37% perf-profile.children.cycles-pp.fbcon_scroll
0.00 +0.1 0.11 ± 34% perf-profile.children.cycles-pp.vt_console_print
0.00 +0.1 0.14 ± 40% perf-profile.children.cycles-pp.ksys_read
0.00 +0.1 0.14 ± 40% perf-profile.children.cycles-pp.vfs_read
0.00 +0.1 0.14 ± 42% perf-profile.children.cycles-pp.read
0.00 +0.2 0.16 ± 38% perf-profile.children.cycles-pp.sk_page_frag_refill
0.00 +0.2 0.16 ± 38% perf-profile.children.cycles-pp.skb_page_frag_refill
0.00 +0.2 0.16 ± 21% perf-profile.children.cycles-pp.force_qs_rnp
0.00 +0.2 0.17 ± 17% perf-profile.children.cycles-pp.rcu_gp_kthread
0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.setup_arg_pages
0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.shift_arg_pages
0.00 +0.2 0.18 ± 69% perf-profile.children.cycles-pp.move_page_tables
0.00 +0.2 0.19 ± 39% perf-profile.children.cycles-pp.schedule_tail
0.00 +0.2 0.19 ± 39% perf-profile.children.cycles-pp.__put_user_4
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.call_transmit
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xprt_transmit
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xs_tcp_send_request
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.xs_sendpages
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.sock_sendmsg
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.tcp_sendmsg
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.tcp_sendmsg_locked
0.00 +0.2 0.19 ± 71% perf-profile.children.cycles-pp.__vmalloc_node
0.00 +0.2 0.19 ± 28% perf-profile.children.cycles-pp.rpc_async_schedule
0.00 +0.2 0.19 ± 28% perf-profile.children.cycles-pp.__rpc_execute
0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.copy_strings
0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.get_user_pages_remote
0.00 +0.2 0.20 ± 14% perf-profile.children.cycles-pp.__get_user_pages
0.01 ±173% +0.2 0.24 ± 48% perf-profile.children.cycles-pp.search_binary_handler
0.01 ±173% +0.2 0.24 ± 48% perf-profile.children.cycles-pp.load_elf_binary
0.00 +0.3 0.26 ± 53% perf-profile.children.cycles-pp.shmem_alloc_page
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.posix_fallocate
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.__x64_sys_fallocate
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.ksys_fallocate
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.vfs_fallocate
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.shmem_fallocate
0.00 +0.3 0.26 ± 51% perf-profile.children.cycles-pp.shmem_alloc_and_acct_page
0.00 +0.3 0.27 ± 28% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.00 +0.3 0.28 ± 33% perf-profile.children.cycles-pp.__pmd_alloc
0.12 ± 13% +0.3 0.45 ± 9% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.perf_poll
0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.start_thread
0.00 +0.3 0.33 ± 37% perf-profile.children.cycles-pp.__pollwait
0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.poll
0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.__x64_sys_poll
0.00 +0.3 0.33 ± 36% perf-profile.children.cycles-pp.do_sys_poll
0.08 ± 23% +0.4 0.44 ± 9% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.00 +0.4 0.39 ± 9% perf-profile.children.cycles-pp.delay_tsc
0.00 +0.4 0.41 ± 9% perf-profile.children.cycles-pp.flush_tlb_batched_pending
0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.task_numa_work
0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.change_prot_numa
0.01 ±173% +0.4 0.43 ± 10% perf-profile.children.cycles-pp.change_p4d_range
0.01 ±173% +0.4 0.43 ± 9% perf-profile.children.cycles-pp.change_protection
0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.kvmalloc_node
0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.__kmalloc_node
0.00 +0.4 0.42 ± 51% perf-profile.children.cycles-pp.kmalloc_large_node
0.01 ±173% +0.4 0.44 ± 10% perf-profile.children.cycles-pp.task_work_run
0.00 +0.4 0.43 ± 12% perf-profile.children.cycles-pp.flush_tlb_mm_range
0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_fault
0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_swapin_page
0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.shmem_swapin
0.00 +0.5 0.50 ± 39% perf-profile.children.cycles-pp.swap_cluster_readahead
0.00 +0.5 0.52 ± 41% perf-profile.children.cycles-pp.__read_swap_cache_async
0.00 +0.5 0.54 ± 6% perf-profile.children.cycles-pp.io_serial_out
0.00 +0.6 0.55 ± 47% perf-profile.children.cycles-pp.__do_fault
0.04 ± 59% +0.6 0.63 ± 29% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.00 +0.6 0.59 ± 7% perf-profile.children.cycles-pp.do_swap_page
0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.do_dentry_open
0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.open64
0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.proc_reg_open
0.00 +0.6 0.61 ± 53% perf-profile.children.cycles-pp.single_open_size
0.00 +0.7 0.66 ± 4% perf-profile.children.cycles-pp.path_parentat
0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.filename_create
0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.filename_parentat
0.00 +0.7 0.67 ± 4% perf-profile.children.cycles-pp.do_mkdirat
0.00 +0.7 0.68 ± 5% perf-profile.children.cycles-pp.mkdir
0.00 +0.8 0.77 ± 28% perf-profile.children.cycles-pp.shmem_getpage_gfp
0.00 +1.1 1.08 ± 19% perf-profile.children.cycles-pp.alloc_empty_file
0.00 +1.1 1.08 ± 19% perf-profile.children.cycles-pp.__alloc_file
0.00 +1.2 1.16 ± 52% perf-profile.children.cycles-pp.do_wp_page
0.00 +1.2 1.16 ± 52% perf-profile.children.cycles-pp.wp_page_copy
0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.__lookup_slow
0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.d_alloc_parallel
0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.d_alloc
0.00 +1.4 1.40 ± 35% perf-profile.children.cycles-pp.__d_alloc
0.00 +1.4 1.41 ± 45% perf-profile.children.cycles-pp.pipe_write
0.00 +1.4 1.42 ± 35% perf-profile.children.cycles-pp.walk_component
0.00 +1.4 1.42 ± 20% perf-profile.children.cycles-pp.nfs_write_begin
0.00 +1.4 1.43 ± 34% perf-profile.children.cycles-pp.link_path_walk
0.00 +1.4 1.43 ± 21% perf-profile.children.cycles-pp.nfs_file_write
0.03 ±102% +1.6 1.61 ± 27% perf-profile.children.cycles-pp.__vmalloc_node_range
0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp.__x64_sys_clone
0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp._do_fork
0.64 ±127% +1.7 2.35 ± 8% perf-profile.children.cycles-pp.copy_process
0.00 +1.8 1.81 ± 27% perf-profile.children.cycles-pp.simple_write_begin
0.00 +1.8 1.82 ± 27% perf-profile.children.cycles-pp.generic_file_write_iter
0.00 +1.8 1.82 ± 27% perf-profile.children.cycles-pp.__generic_file_write_iter
0.00 +1.8 1.83 ± 27% perf-profile.children.cycles-pp.__GI___libc_write
0.00 +1.9 1.86 ± 18% perf-profile.children.cycles-pp.__GI___libc_open
0.77 ± 8% +1.9 2.66 ± 16% perf-profile.children.cycles-pp.kmem_cache_alloc
0.62 ± 7% +2.3 2.92 ± 17% perf-profile.children.cycles-pp.__slab_alloc
0.62 ± 7% +2.3 2.92 ± 17% perf-profile.children.cycles-pp.___slab_alloc
0.56 ± 7% +2.4 2.92 ± 17% perf-profile.children.cycles-pp.new_slab
0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_filp_open
0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.path_openat
0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_sys_open
0.00 +2.5 2.45 ± 22% perf-profile.children.cycles-pp.do_sys_openat2
0.68 ±119% +2.6 3.32 ± 11% perf-profile.children.cycles-pp.__libc_fork
0.00 +2.8 2.85 ± 30% perf-profile.children.cycles-pp.write
0.97 ± 17% +3.1 4.05 ± 25% perf-profile.children.cycles-pp.worker_thread
0.90 ± 16% +3.1 4.03 ± 25% perf-profile.children.cycles-pp.process_one_work
0.69 ±116% +3.2 3.85 ± 9% perf-profile.children.cycles-pp.forkshell
0.09 ± 7% +3.2 3.30 ± 4% perf-profile.children.cycles-pp.io_serial_in
0.00 +3.2 3.24 ± 6% perf-profile.children.cycles-pp.grab_cache_page_write_begin
0.00 +3.2 3.25 ± 6% perf-profile.children.cycles-pp.generic_perform_write
0.00 +3.3 3.27 ± 7% perf-profile.children.cycles-pp.pagecache_get_page
0.11 ± 12% +3.4 3.52 ± 5% perf-profile.children.cycles-pp.serial8250_console_putchar
0.12 ± 12% +3.6 3.68 ± 5% perf-profile.children.cycles-pp.wait_for_xmitr
0.00 +3.7 3.73 ± 25% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
0.00 +3.7 3.73 ± 25% perf-profile.children.cycles-pp.memcpy_erms
0.12 ± 12% +3.9 4.04 ± 5% perf-profile.children.cycles-pp.uart_console_write
0.12 ± 12% +4.1 4.23 ± 5% perf-profile.children.cycles-pp.serial8250_console_write
0.12 ± 14% +4.2 4.35 ± 4% perf-profile.children.cycles-pp.console_unlock
35.04 ± 6% +4.4 39.41 ± 2% perf-profile.children.cycles-pp.handle_mm_fault
0.78 ± 16% +4.6 5.38 ± 3% perf-profile.children.cycles-pp.shrink_slab
34.72 ± 6% +4.6 39.33 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault
0.74 ± 16% +4.6 5.38 ± 3% perf-profile.children.cycles-pp.do_shrink_slab
0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.ksys_write
0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.vfs_write
0.00 +4.7 4.67 ± 12% perf-profile.children.cycles-pp.new_sync_write
34.00 ± 6% +5.0 39.02 ± 2% perf-profile.children.cycles-pp.handle_pte_fault
0.00 +5.2 5.21 ± 3% perf-profile.children.cycles-pp.rcu_oom_scan
0.08 ± 27% +5.3 5.43 ± 2% perf-profile.children.cycles-pp.printk
0.08 ± 27% +5.3 5.43 ± 2% perf-profile.children.cycles-pp.vprintk_emit
1.05 ±104% +10.7 11.71 ± 4% perf-profile.children.cycles-pp.do_syscall_64
1.05 ±104% +10.7 11.71 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
34.02 ± 9% +16.9 50.93 ± 4% perf-profile.children.cycles-pp.shrink_lruvec
31.76 ± 9% +18.6 50.34 ± 4% perf-profile.children.cycles-pp.shrink_inactive_list
31.30 ± 9% +19.0 50.29 ± 4% perf-profile.children.cycles-pp.shrink_page_list
29.28 ± 7% +19.0 48.27 ± 3% perf-profile.children.cycles-pp.__alloc_pages_nodemask
28.07 ± 7% +19.5 47.59 ± 3% perf-profile.children.cycles-pp.__alloc_pages_slowpath
34.84 ± 9% +21.5 56.33 ± 3% perf-profile.children.cycles-pp.shrink_node
25.45 ± 7% +22.0 47.43 ± 4% perf-profile.children.cycles-pp.try_to_free_pages
25.21 ± 7% +22.2 47.41 ± 4% perf-profile.children.cycles-pp.do_try_to_free_pages
14.33 ± 10% +34.6 48.95 ± 5% perf-profile.children.cycles-pp.try_to_unmap_flush_dirty
14.31 ± 10% +34.6 48.95 ± 5% perf-profile.children.cycles-pp.arch_tlbbatch_flush
13.81 ± 10% +35.1 48.94 ± 5% perf-profile.children.cycles-pp.on_each_cpu_cond_mask
13.46 ± 11% +35.9 49.31 ± 5% perf-profile.children.cycles-pp.smp_call_function_many_cond
5.33 ± 9% -3.4 1.90 ± 10% perf-profile.self.cycles-pp.do_rw_once
4.88 ± 9% -3.1 1.75 ± 11% perf-profile.self.cycles-pp.do_access
2.46 ± 6% -2.3 0.13 ± 29% perf-profile.self.cycles-pp.llist_reverse_order
2.29 ± 15% -2.2 0.13 ± 26% perf-profile.self.cycles-pp.llist_add_batch
2.26 ± 12% -2.1 0.17 ± 23% perf-profile.self.cycles-pp.__memcpy_flushcache
2.15 ± 8% -2.1 0.09 ± 39% perf-profile.self.cycles-pp.flush_smp_call_function_queue
2.49 ± 9% -1.6 0.90 ± 10% perf-profile.self.cycles-pp._raw_spin_lock
1.95 ± 8% -1.6 0.37 ± 11% perf-profile.self.cycles-pp.native_irq_return_iret
1.72 ± 9% -1.6 0.15 ± 21% perf-profile.self.cycles-pp.__default_send_IPI_dest_field
1.55 ± 6% -1.5 0.08 ± 26% perf-profile.self.cycles-pp.flush_tlb_func_common
1.25 ± 10% -1.2 0.06 ± 75% perf-profile.self.cycles-pp.try_to_unmap_one
1.23 ± 6% -1.2 0.05 ± 72% perf-profile.self.cycles-pp.native_flush_tlb
1.10 ± 11% -1.0 0.12 ± 28% perf-profile.self.cycles-pp.end_page_writeback
1.02 ± 12% -0.9 0.09 ± 41% perf-profile.self.cycles-pp.__slab_free
1.15 ± 9% -0.8 0.31 ± 22% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.19 ± 15% -0.8 0.37 ± 14% perf-profile.self.cycles-pp.down_read_trylock
1.90 ± 13% -0.8 1.09 ± 4% perf-profile.self.cycles-pp.menu_select
0.84 ± 17% -0.8 0.07 ± 30% perf-profile.self.cycles-pp.page_vma_mapped_walk
0.85 ± 10% -0.6 0.21 ± 10% perf-profile.self.cycles-pp.get_page_from_freelist
1.05 ± 7% -0.6 0.42 ± 13% perf-profile.self.cycles-pp.clear_page_erms
0.67 ± 10% -0.6 0.10 ± 4% perf-profile.self.cycles-pp._find_next_bit
0.62 ± 15% -0.4 0.19 ± 18% perf-profile.self.cycles-pp.__handle_mm_fault
0.67 ± 11% -0.4 0.25 ± 15% perf-profile.self.cycles-pp.lru_cache_add_active_or_unevictable
0.60 ± 12% -0.4 0.20 ± 8% perf-profile.self.cycles-pp.up_read
1.03 ± 11% -0.4 0.64 ± 4% perf-profile.self.cycles-pp.cpuidle_enter_state
0.39 ± 8% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.radix_tree_node_ctor
0.39 ± 16% -0.3 0.06 ± 13% perf-profile.self.cycles-pp.__list_del_entry_valid
0.92 ± 20% -0.3 0.61 ± 7% perf-profile.self.cycles-pp.ktime_get
0.31 ± 10% -0.3 0.04 ± 73% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.30 ± 6% -0.3 0.04 ± 73% perf-profile.self.cycles-pp.page_lock_anon_vma_read
0.49 ± 11% -0.3 0.24 ± 12% perf-profile.self.cycles-pp.mem_cgroup_try_charge
0.40 ± 9% -0.2 0.15 ± 12% perf-profile.self.cycles-pp.sync_regs
0.34 ± 17% -0.2 0.11 ± 22% perf-profile.self.cycles-pp.handle_pte_fault
0.27 ± 16% -0.2 0.04 ± 71% perf-profile.self.cycles-pp.count_shadow_nodes
0.32 ± 11% -0.2 0.09 ± 15% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.24 ± 13% -0.2 0.04 ± 71% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.22 ± 19% -0.2 0.04 ± 73% perf-profile.self.cycles-pp.handle_mm_fault
0.28 ± 12% -0.2 0.13 ± 13% perf-profile.self.cycles-pp.try_charge
0.23 ± 6% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.__pagevec_lru_add_fn
0.15 ± 15% -0.1 0.06 ± 8% perf-profile.self.cycles-pp.mem_cgroup_throttle_swaprate
0.15 ± 6% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.native_write_msr
0.07 ± 22% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.run_local_timers
0.09 ± 24% -0.0 0.05 ± 8% perf-profile.self.cycles-pp._raw_spin_trylock
0.00 +0.4 0.39 ± 9% perf-profile.self.cycles-pp.delay_tsc
0.00 +0.5 0.54 ± 6% perf-profile.self.cycles-pp.io_serial_out
0.00 +1.1 1.09 ± 17% perf-profile.self.cycles-pp.vprintk_emit
0.09 ± 7% +3.2 3.30 ± 4% perf-profile.self.cycles-pp.io_serial_in
0.00 +3.7 3.71 ± 25% perf-profile.self.cycles-pp.memcpy_erms
6.37 ± 12% +42.3 48.62 ± 5% perf-profile.self.cycles-pp.smp_call_function_many_cond
vm-scalability.time.system_time
2500 +--------------------------------------------------------------------+
| O O O |
| O O O O O O |
2000 |-+ O O |
| O O |
| |
1500 |-+ |
| |
1000 |-+ |
| |
| |
500 |-+ |
| |
| .+.+.+..+.+. .+..+.+. .+.+.+.. .+.+.+..+.+.+..+.+.+..+.+.+.+..+.|
0 +--------------------------------------------------------------------+
vm-scalability.time.elapsed_time
400 +---------------------------------------------------------------------+
| O O O O |
350 |-+ O O O |
300 |-O O O |
| O O O |
250 |-+ |
| |
200 |-+ |
| |
150 |-+ |
100 |-+ |
| |
50 |-+ +.+.+..+.+ .+.. +.+.. +..+.+.+..+.+.+..+.+.+..+.+.+..+.|
| .. + .+.+ + +. + |
0 +---------------------------------------------------------------------+
vm-scalability.time.elapsed_time.max
400 +---------------------------------------------------------------------+
| O O O O |
350 |-+ O O O |
300 |-O O O |
| O O O |
250 |-+ |
| |
200 |-+ |
| |
150 |-+ |
100 |-+ |
| |
50 |-+ +.+.+..+.+ .+.. +.+.. +..+.+.+..+.+.+..+.+.+..+.+.+..+.|
| .. + .+.+ + +. + |
0 +---------------------------------------------------------------------+
vm-scalability.throughput
7e+06 +-------------------------------------------------------------------+
| |
6e+06 |-+ .+.+ +.+.. +..+ .+. |
| + + : + : : +..+.+.+.+..+.+.+.+..+.+ +..+.|
5e+06 |-+ : +..+ : : : : : |
| : : : : + : : |
4e+06 |-+ : : : : : : : |
| : : : : : : : |
3e+06 |-+ : : : : : : : |
| : : : : : : : |
2e+06 |-+: : : : : : : |
| : : : : : :: |
1e+06 |-O: O : O : O : O O O |
| : O : O O : O O : O |
0 +-------------------------------------------------------------------+
vm-scalability.median
800000 +------------------------------------------------------------------+
| .+ : +.+. +.+ +.+. .+. .+.. |
700000 |-+ +. : : + : : : +.+..+.+ +..+.+.+ +.+.|
| : +.+ : : : : : |
600000 |-+ : : : : + : : |
500000 |-+ : : : : : : : |
| : : : : : : : |
400000 |-+: : : : : : : |
| : : : : : : : |
300000 |-+: : : : : : : |
200000 |-+: : : : : : : |
| : :: : : :: |
100000 |-: O O :: O O O : O O O : O O O O |
| : : : : |
0 +------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
8 months, 2 weeks
[treewide] 9cd745027b: stack_segment:#[##]
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: 9cd745027be08348374ddf42eeb3b94b785aba3f ("treewide: Replace one-element array with flexible-array")
https://git.kernel.org/cgit/linux/kernel/git/gustavoars/linux.git testing/fam1
in testcase: locktorture
with following parameters:
runtime: 300s
test: cpuhotplug
test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors.
test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 2.141748] ACPI: Interpreter enabled
[ 2.157189] ACPI: (supports S0 S3 S4 S5)
[ 2.164064] ACPI: Using IOAPIC for interrupt routing
[ 2.170175] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 2.189565] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 2.194436] stack segment: 0000 [#1] SMP PTI
[ 2.195001] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3-00001-g9cd745027be08 #1
[ 2.195001] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 2.195001] RIP: 0010:__kmalloc_track_caller+0x98/0x270
[ 2.195001] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 77 9a d6 60 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[ 2.195001] RSP: 0000:ffffc1f7c0013a38 EFLAGS: 00010206
[ 2.195001] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000077
[ 2.195001] RDX: 0000000000000076 RSI: 0000000000000cc0 RDI: ffff9c6a47c03a40
[ 2.195001] RBP: 0035304130504e50 R08: 0000000000031060 R09: 0000000000000000
[ 2.195001] R10: 0000000000000000 R11: ffff9c6a4786f0c8 R12: 0000000000000cc0
[ 2.195001] R13: 000000000000000c R14: ffff9c6a47c03a40 R15: ffff9c6a47c03a40
[ 2.195001] FS: 0000000000000000(0000) GS:ffff9c6b7fd00000(0000) knlGS:0000000000000000
[ 2.195001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2.195001] CR2: 00000000ffffffff CR3: 000000023a00a000 CR4: 00000000000406e0
[ 2.195001] Call Trace:
[ 2.195001] ? __kernfs_new_node+0x40/0x1e0
[ 2.195001] kstrdup+0x2d/0x60
[ 2.195001] __kernfs_new_node+0x40/0x1e0
[ 2.195001] ? kernfs_create_dir_ns+0x50/0x80
[ 2.195001] ? _cond_resched+0x19/0x30
[ 2.195001] ? kernfs_activate+0x63/0x80
[ 2.195001] kernfs_new_node+0x34/0x60
[ 2.195001] kernfs_create_link+0x35/0xb0
[ 2.195001] sysfs_do_create_link_sd+0x67/0xc0
[ 2.195001] bus_add_device+0x6e/0x150
[ 2.195001] device_add+0x38b/0x840
[ 2.195001] ? kmem_cache_alloc_trace+0x21e/0x230
[ 2.195001] acpi_device_add+0x1dc/0x3f0
[ 2.195001] ? acpi_free_pnp_ids+0x50/0x50
[ 2.195001] acpi_add_single_object+0x25c/0x5f0
[ 2.195001] ? up+0x12/0x60
[ 2.195001] ? acpi_ut_acquire_mutex+0x45/0x89
[ 2.195001] ? acpi_ut_release_mutex+0x6f/0x74
[ 2.195001] ? acpi_get_data_full+0x7e/0x90
[ 2.195001] ? rdinit_setup+0x2b/0x2b
[ 2.195001] acpi_bus_check_add+0xd0/0x260
[ 2.195001] ? acpi_ev_gpe_detect+0xd0/0xe2
[ 2.195001] ? acpi_sleep_proc_init+0x24/0x24
[ 2.195001] acpi_bus_scan+0x31/0x90
[ 2.195001] acpi_scan_init+0xf1/0x235
[ 2.195001] ? acpi_sleep_proc_init+0x24/0x24
[ 2.195001] acpi_init+0x2f4/0x354
[ 2.195001] do_one_initcall+0x46/0x220
[ 2.195001] kernel_init_freeable+0x206/0x280
[ 2.195001] ? rest_init+0xd0/0xd0
[ 2.195001] kernel_init+0xa/0x110
[ 2.195001] ret_from_fork+0x35/0x40
[ 2.195001] Modules linked in:
[ 2.469071] ---[ end trace a63967e0ecd68f27 ]---
[ 2.473044] RIP: 0010:__kmalloc_track_caller+0x98/0x270
[ 2.478035] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 77 9a d6 60 49 8b 28 48 85 ed 0f 84 9d 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[ 2.494032] RSP: 0000:ffffc1f7c0013a38 EFLAGS: 00010206
[ 2.499032] RAX: 0000000000000008 RBX: 0000000000000cc0 RCX: 0000000000000077
[ 2.505029] RDX: 0000000000000076 RSI: 0000000000000cc0 RDI: ffff9c6a47c03a40
[ 2.521078] RBP: 0035304130504e50 R08: 0000000000031060 R09: 0000000000000000
[ 2.531050] R10: 0000000000000000 R11: ffff9c6a4786f0c8 R12: 0000000000000cc0
[ 2.537046] R13: 000000000000000c R14: ffff9c6a47c03a40 R15: ffff9c6a47c03a40
[ 2.543053] FS: 0000000000000000(0000) GS:ffff9c6b7fd00000(0000) knlGS:0000000000000000
[ 2.550044] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2.555032] CR2: 00000000ffffffff CR3: 000000023a00a000 CR4: 00000000000406e0
[ 2.561038] Kernel panic - not syncing: Fatal exception
Elapsed time: 60
qemu-img create -f qcow2 disk-vm-snb-152-0 256G
qemu-img create -f qcow2 disk-vm-snb-152-1 256G
To reproduce:
# build kernel
cd linux
cp config-5.7.0-rc3-00001-g9cd745027be08 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
8 months, 2 weeks
c088afc0d1: BUG:KASAN:slab-out-of-bounds_in__ptrace_may_access
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: c088afc0d1ec4660184a675c1c0c3d035f37c8b5 (" iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCXrFbowAKCRCRxhvAZXjc")
https://git.kernel.org/cgit/linux/kernel/git/brauner/linux.git setns_pidfd
in testcase: trinity
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 29.718859] BUG: KASAN: slab-out-of-bounds in __ptrace_may_access+0x4d/0x230
[ 29.719652] Read of size 8 at addr ffff8881d2d365d0 by task trinity-c0/882
[ 29.720412]
[ 29.720595] CPU: 0 PID: 882 Comm: trinity-c0 Not tainted 5.7.0-rc2-00002-gc088afc0d1ec4 #1
[ 29.721497] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 29.722411] Call Trace:
[ 29.722701] dump_stack+0x19/0x25
[ 29.723093] print_address_description+0x20/0x320
[ 29.723768] ? __ptrace_may_access+0x4d/0x230
[ 29.724271] ? __ptrace_may_access+0x4d/0x230
[ 29.724770] __kasan_report+0xf2/0x11d
[ 29.725213] ? pidns_for_children_get+0x60/0x120
[ 29.725747] ? __ptrace_may_access+0x4d/0x230
[ 29.726251] kasan_report+0x38/0x50
[ 29.726662] __asan_load8+0x69/0x90
[ 29.727076] __ptrace_may_access+0x4d/0x230
[ 29.727552] ptrace_may_access+0x27/0x60
[ 29.727995] __x64_sys_setns+0x4b8/0x950
[ 29.728440] ? exit_task_namespaces+0x20/0x20
[ 29.728929] do_syscall_64+0xbb/0x660
[ 29.729350] ? syscall_return_slowpath+0x230/0x230
[ 29.729889] ? fpregs_assert_state_consistent+0x63/0x70
[ 29.730478] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 29.732003] RIP: 0033:0x453b29
[ 29.732358] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00
[ 29.734458] RSP: 002b:00007ffddc2ad678 EFLAGS: 00000246 ORIG_RAX: 0000000000000134
[ 29.735292] RAX: ffffffffffffffda RBX: 0000000000000134 RCX: 0000000000453b29
[ 29.736146] RDX: 000000014522b083 RSI: 0000000040000000 RDI: 0000000000000014
[ 29.736954] RBP: 00007ffddc2ad720 R08: 0000000000000012 R09: 27b18fbd5200e56b
[ 29.737826] R10: 000000000000009d R11: 0000000000000246 R12: 0000000000000002
[ 29.738630] R13: 00007fbc53011058 R14: 00000000010a2830 R15: 00007fbc53011000
[ 29.739420]
[ 29.739646] Allocated by task 589:
[ 29.740059] save_stack+0x21/0x50
[ 29.740439] __kasan_kmalloc+0xa6/0xd0
[ 29.740979] kasan_kmalloc+0x9/0x10
[ 29.741381] kmem_cache_alloc_trace+0x108/0x250
[ 29.741950] do_eventfd+0x46/0x120
[ 29.742341] __x64_sys_eventfd2+0x31/0x40
[ 29.742824] do_syscall_64+0xbb/0x660
[ 29.743245] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 29.743874]
[ 29.744059] Freed by task 0:
[ 29.744376] (stack is not available)
[ 29.744765]
[ 29.744943] The buggy address belongs to the object at ffff8881d2d36440
[ 29.744943] which belongs to the cache kmalloc-64 of size 64
[ 29.746330] The buggy address is located 336 bytes to the right of
[ 29.746330] 64-byte region [ffff8881d2d36440, ffff8881d2d36480)
[ 29.747710] The buggy address belongs to the page:
[ 29.748262] page:ffffea00074b4d80 refcount:1 mapcount:0 mapping:(____ptrval____) index:0x0
[ 29.749193] flags: 0x8000000000000200(slab)
[ 29.749745] raw: 8000000000000200 ffffea00073bc3c8 ffffea000788fdc8 ffff8881f68036c0
[ 29.750616] raw: 0000000000000000 0000000000080008 00000001ffffffff 0000000000000000
[ 29.751458] page dumped because: kasan: bad access detected
[ 29.752073]
[ 29.752249] Memory state around the buggy address:
[ 29.752782] ffff8881d2d36480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 29.753570] ffff8881d2d36500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 29.754360] >ffff8881d2d36580: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 29.755180] ^
[ 29.755813] ffff8881d2d36600: fc fc fc fc fc fc fc fc 00 00 00 00 00 fc fc fc
[ 29.756593] ffff8881d2d36680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 29.757386] ==================================================================
[ 29.758171] Disabling lock debugging due to kernel taint
[ 29.758818] BUG: kernel NULL pointer dereference, address: 0000000000000014
[ 29.759586] #PF: supervisor read access in kernel mode
[ 29.760173] #PF: error_code(0x0000) - not-present page
[ 29.760754] PGD 1cea22067 P4D 1cea22067 PUD 1db0d4067 PMD 0
[ 29.761397] Oops: 0000 [#1] PREEMPT KASAN
[ 29.761858] CPU: 0 PID: 882 Comm: trinity-c0 Tainted: G B 5.7.0-rc2-00002-gc088afc0d1ec4 #1
[ 29.762900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 29.763503] RIP: 0010:__ptrace_may_access+0xc9/0x230
[ 29.763864] Code: 75 04 e8 9a 89 26 00 41 8b 45 08 89 45 d4 48 8d bb 50 05 00 00 e8 a7 8a 26 00 4c 8b bb 50 05 00 00 49 8d 7f 14 e8 77 89 26 00 <45> 3b 77 14 75 13 49 8d 7f 0c e8 68 89 26 00 45 39 77 0c 0f 84 c0
[ 29.765191] RSP: 0018:ffff8881e64a7d68 EFLAGS: 00010246
[ 29.765568] RAX: 0000000000000000 RBX: ffff8881d2d36020 RCX: ffffffff810bbfd9
[ 29.766076] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000014
[ 29.766585] RBP: ffff8881e64a7d98 R08: fffffbfff0676939 R09: fffffbfff0676939
[ 29.767107] R10: ffffffff833b49c7 R11: 0000000000000001 R12: 0000000000000011
[ 29.767616] R13: ffff8881e1d31480 R14: 000000000000fffe R15: 0000000000000000
[ 29.768128] FS: 00000000010a2880(0000) GS:ffffffff82c9c000(0000) knlGS:0000000000000000
[ 29.768708] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 29.769123] CR2: 0000000000000014 CR3: 00000001ceaf6000 CR4: 00000000000406b0
[ 29.769636] DR0: 00007fbc526f3000 DR1: 0000000000000000 DR2: 0000000000000000
[ 29.770146] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 29.770656] Call Trace:
[ 29.770855] ptrace_may_access+0x27/0x60
[ 29.771150] __x64_sys_setns+0x4b8/0x950
[ 29.771437] ? exit_task_namespaces+0x20/0x20
[ 29.771755] do_syscall_64+0xbb/0x660
[ 29.772023] ? syscall_return_slowpath+0x230/0x230
[ 29.772372] ? fpregs_assert_state_consistent+0x63/0x70
[ 29.772749] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 29.773116] RIP: 0033:0x453b29
[ 29.773343] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00
[ 29.774666] RSP: 002b:00007ffddc2ad678 EFLAGS: 00000246 ORIG_RAX: 0000000000000134
[ 29.775221] RAX: ffffffffffffffda RBX: 0000000000000134 RCX: 0000000000453b29
[ 29.775729] RDX: 000000014522b083 RSI: 0000000040000000 RDI: 0000000000000014
[ 29.776237] RBP: 00007ffddc2ad720 R08: 0000000000000012 R09: 27b18fbd5200e56b
[ 29.776745] R10: 000000000000009d R11: 0000000000000246 R12: 0000000000000002
[ 29.777252] R13: 00007fbc53011058 R14: 00000000010a2830 R15: 00007fbc53011000
[ 29.777759] Modules linked in: input_leds serio_raw evdev qemu_fw_cfg processor tiny_power_button button
[ 29.778435] CR2: 0000000000000014
[ 29.778678] ---[ end trace 9561f8100fce2866 ]---
To reproduce:
# build kernel
cd linux
cp config-5.7.0-rc2-00002-gc088afc0d1ec4 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
8 months, 2 weeks
[fs] 05c5a0273b: netperf.Throughput_total_tps -71.8% regression
by kernel test robot
Greeting,
FYI, we noticed a -71.8% regression of netperf.Throughput_total_tps due to commit:
commit: 05c5a0273bc25627b9e9bbebc858b74e88f49c5b ("[PATCH v2] fs: jfs: fix a possible data race in txBegin()")
url: https://github.com/0day-ci/linux/commits/Jia-Ju-Bai/fs-jfs-fix-a-possible...
base: https://github.com/kleikamp/linux-shaggy jfs-next
in testcase: netperf
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:
ip: ipv4
runtime: 300s
nr_threads: 1
cluster: cs-localhost
test: TCP_CRR
cpufreq_governor: performance
ucode: 0x2000065
test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
cs-localhost/gcc-7/performance/ipv4/x86_64-rhel-7.6/1/debian-x86_64-20191114.cgz/300s/lkp-skl-2sp7/TCP_CRR/netperf/0x2000065
commit:
7aba5dcc23 ("jfs: Replace zero-length array with flexible-array member")
05c5a0273b ("fs: jfs: fix a possible data race in txBegin()")
7aba5dcc234635b4 05c5a0273bc25627b9e9bbebc85
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:4 100% 4:4 kmsg.IP-Config:Failed_to_open_erspan0
:4 100% 4:4 kmsg.IP-Config:Failed_to_open_gretap0
:4 100% 4:4 kmsg.kvm:already_loaded_the_other_module
%stddev %change %stddev
\ | \
32033 -71.8% 9048 netperf.Throughput_total_tps
32033 -71.8% 9048 netperf.Throughput_tps
34270 ± 19% -32.2% 23234 ± 11% netperf.time.involuntary_context_switches
87.50 -1.7% 86.00 netperf.time.percent_of_cpu_this_job_got
237.76 +7.1% 254.71 netperf.time.system_time
30.66 -66.6% 10.25 ± 3% netperf.time.user_time
17926206 -69.8% 5405643 netperf.time.voluntary_context_switches
9610194 -71.8% 2714625 netperf.workload
0.25 -0.1 0.12 ± 3% mpstat.cpu.all.usr%
181085 -60.2% 72003 vmstat.system.cs
24.19 ± 2% +8.7% 26.30 boot-time.boot
15.27 ± 2% +12.6% 17.18 boot-time.dhcp
1449 ± 2% +10.6% 1602 boot-time.idle
1.589e+08 -16.1% 1.334e+08 cpuidle.C1.time
10004352 -12.6% 8738852 cpuidle.C1.usage
24458189 ± 3% -76.6% 5718567 ± 15% cpuidle.POLL.time
18015660 -95.3% 840897 ± 17% cpuidle.POLL.usage
42120 ± 2% +9.9% 46309 ± 5% softirqs.CPU10.SCHED
41095 ± 4% +10.9% 45593 ± 5% softirqs.CPU27.SCHED
25223 ± 54% +70.0% 42867 softirqs.CPU36.SCHED
166155 ± 8% -10.1% 149301 softirqs.CPU65.RCU
28947742 -71.8% 8152772 softirqs.NET_RX
12415372 -9.2% 11268919 softirqs.RCU
8089600 ± 11% -23.4% 6200320 ± 8% meminfo.DirectMap2M
503148 ± 8% -41.3% 295276 ± 8% meminfo.DirectMap4k
75713 +34.4% 101777 meminfo.KReclaimable
22720 +11.6% 25360 meminfo.Percpu
75713 +34.4% 101777 meminfo.SReclaimable
152955 +16.4% 178056 meminfo.SUnreclaim
228668 +22.4% 279834 meminfo.Slab
137288 +17.8% 161678 meminfo.VmallocUsed
33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.KReclaimable
33315 ± 18% +75.5% 58482 ± 12% numa-meminfo.node0.SReclaimable
74713 ± 16% +34.8% 100736 ± 11% numa-meminfo.node0.SUnreclaim
108029 ± 16% +47.4% 159219 ± 11% numa-meminfo.node0.Slab
101746 ± 44% +68.1% 171066 ± 12% numa-meminfo.node1.Active
101684 ± 44% +68.1% 170963 ± 12% numa-meminfo.node1.Active(anon)
60012 ± 62% +94.5% 116727 ± 18% numa-meminfo.node1.AnonHugePages
98606 ± 46% +69.1% 166702 ± 12% numa-meminfo.node1.AnonPages
5648 +3.4% 5842 proc-vmstat.nr_shmem
18920 +34.5% 25446 proc-vmstat.nr_slab_reclaimable
38238 +16.4% 44513 proc-vmstat.nr_slab_unreclaimable
895359 ± 5% -17.9% 734777 proc-vmstat.numa_hit
872027 ± 5% -18.4% 711460 proc-vmstat.numa_local
1080 ± 9% +27.4% 1375 ± 4% proc-vmstat.pgactivate
1530529 ± 15% -42.6% 878573 ± 2% proc-vmstat.pgalloc_normal
1505342 ± 15% -43.5% 849856 ± 2% proc-vmstat.pgfree
8335 ± 18% +75.4% 14620 ± 12% numa-vmstat.node0.nr_slab_reclaimable
18678 ± 16% +34.8% 25183 ± 11% numa-vmstat.node0.nr_slab_unreclaimable
647500 ± 10% -21.7% 507083 ± 8% numa-vmstat.node0.numa_hit
494986 ± 13% -27.6% 358327 ± 13% numa-vmstat.node0.numa_local
25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_active_anon
24661 ± 46% +69.0% 41679 ± 12% numa-vmstat.node1.nr_anon_pages
25431 ± 44% +68.1% 42742 ± 12% numa-vmstat.node1.nr_zone_active_anon
725209 ± 8% -16.4% 606495 ± 8% numa-vmstat.node1.numa_hit
702021 ± 8% -17.9% 576047 ± 10% numa-vmstat.node1.numa_local
49724 ± 16% -42.5% 28572 ± 16% sched_debug.cfs_rq:/.load.avg
113065 ± 11% -24.3% 85601 ± 10% sched_debug.cfs_rq:/.load.stddev
57662 ± 7% -47.5% 30301 ± 9% sched_debug.cfs_rq:/.min_vruntime.max
6576 ± 8% -34.6% 4300 ± 19% sched_debug.cfs_rq:/.min_vruntime.min
10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.min_vruntime.stddev
0.16 ± 8% -29.7% 0.11 ± 8% sched_debug.cfs_rq:/.nr_running.avg
0.36 ± 2% -13.7% 0.31 ± 4% sched_debug.cfs_rq:/.nr_running.stddev
35.45 ± 20% -52.9% 16.69 sched_debug.cfs_rq:/.runnable_load_avg.avg
95.13 ± 14% -32.7% 63.99 sched_debug.cfs_rq:/.runnable_load_avg.stddev
50290 ± 14% -43.7% 28330 ± 15% sched_debug.cfs_rq:/.runnable_weight.avg
113143 ± 10% -24.9% 84975 ± 10% sched_debug.cfs_rq:/.runnable_weight.stddev
33703 ± 43% -68.0% 10796 ± 42% sched_debug.cfs_rq:/.spread0.max
10636 ± 5% -39.1% 6478 ± 15% sched_debug.cfs_rq:/.spread0.stddev
140.66 ± 10% -17.8% 115.68 ± 7% sched_debug.cfs_rq:/.util_avg.avg
247.04 ± 4% -11.6% 218.36 ± 4% sched_debug.cfs_rq:/.util_avg.stddev
51.66 ± 11% -41.4% 30.29 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.avg
179.93 ± 3% -26.3% 132.58 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev
1.26 ± 2% +10.1% 1.39 ± 3% sched_debug.cpu.clock.stddev
1.26 ± 2% +10.2% 1.39 ± 3% sched_debug.cpu.clock_task.stddev
365656 ± 9% -57.9% 154017 sched_debug.cpu.nr_switches.avg
3644073 ± 21% -81.6% 671977 ± 17% sched_debug.cpu.nr_switches.max
1315 ± 5% +863.7% 12673 ± 91% sched_debug.cpu.nr_switches.min
592717 ± 15% -75.6% 144437 ± 18% sched_debug.cpu.nr_switches.stddev
341.50 ± 96% +267.0% 1253 ± 7% interrupts.CPU12.RES:Rescheduling_interrupts
145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.NMI:Non-maskable_interrupts
145.00 ± 42% +316.6% 604.00 ± 78% interrupts.CPU15.PMI:Performance_monitoring_interrupts
168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.NMI:Non-maskable_interrupts
168.25 ± 70% +219.9% 538.25 ± 63% interrupts.CPU16.PMI:Performance_monitoring_interrupts
316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.NMI:Non-maskable_interrupts
316.25 ± 69% +303.7% 1276 ± 83% interrupts.CPU17.PMI:Performance_monitoring_interrupts
3796 ± 83% -82.0% 685.00 ±119% interrupts.CPU18.RES:Rescheduling_interrupts
2353 ± 49% -84.5% 364.00 ±134% interrupts.CPU20.RES:Rescheduling_interrupts
107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.NMI:Non-maskable_interrupts
107.50 ± 26% +199.8% 322.25 ± 80% interrupts.CPU23.PMI:Performance_monitoring_interrupts
89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.NMI:Non-maskable_interrupts
89.00 ± 14% +90.2% 169.25 ± 23% interrupts.CPU24.PMI:Performance_monitoring_interrupts
93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.NMI:Non-maskable_interrupts
93.25 ± 19% +101.6% 188.00 ± 35% interrupts.CPU31.PMI:Performance_monitoring_interrupts
87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.NMI:Non-maskable_interrupts
87.25 ± 14% +88.3% 164.25 ± 13% interrupts.CPU32.PMI:Performance_monitoring_interrupts
104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.NMI:Non-maskable_interrupts
104.50 ± 34% +106.9% 216.25 ± 46% interrupts.CPU33.PMI:Performance_monitoring_interrupts
1951 ± 71% -80.2% 387.00 ±113% interrupts.CPU33.RES:Rescheduling_interrupts
117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.NMI:Non-maskable_interrupts
117.25 ± 42% +295.7% 464.00 ± 68% interrupts.CPU51.PMI:Performance_monitoring_interrupts
142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.NMI:Non-maskable_interrupts
142.50 ± 52% +186.7% 408.50 ± 55% interrupts.CPU52.PMI:Performance_monitoring_interrupts
320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.NMI:Non-maskable_interrupts
320.00 ± 65% +246.8% 1109 ± 57% interrupts.CPU53.PMI:Performance_monitoring_interrupts
106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.NMI:Non-maskable_interrupts
106.75 ± 30% +111.2% 225.50 ± 56% interrupts.CPU67.PMI:Performance_monitoring_interrupts
87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.NMI:Non-maskable_interrupts
87.00 ± 13% +123.6% 194.50 ± 33% interrupts.CPU68.PMI:Performance_monitoring_interrupts
23466 +23.8% 29040 ± 5% interrupts.NMI:Non-maskable_interrupts
23466 +23.8% 29040 ± 5% interrupts.PMI:Performance_monitoring_interrupts
13.69 ± 25% -71.4% 3.92 ± 48% perf-stat.i.MPKI
8.78e+08 +90.0% 1.668e+09 perf-stat.i.branch-instructions
2.30 ± 14% -1.5 0.83 ± 33% perf-stat.i.branch-miss-rate%
20600243 ± 12% -31.0% 14224189 ± 29% perf-stat.i.branch-misses
2.48 ± 18% +1.7 4.22 ± 14% perf-stat.i.cache-miss-rate%
57749473 ± 23% -44.2% 32231609 ± 44% perf-stat.i.cache-references
182988 -60.2% 72782 perf-stat.i.context-switches
1.65 ± 5% -39.4% 1.00 ± 5% perf-stat.i.cpi
7.078e+09 ± 3% +18.6% 8.393e+09 ± 3% perf-stat.i.cpu-cycles
85.15 -8.1% 78.28 perf-stat.i.cpu-migrations
0.09 ± 32% -0.1 0.02 ±106% perf-stat.i.dTLB-load-miss-rate%
1.284e+09 +89.1% 2.428e+09 perf-stat.i.dTLB-loads
0.01 ± 51% -0.0 0.00 ±133% perf-stat.i.dTLB-store-miss-rate%
7.676e+08 +81.0% 1.39e+09 perf-stat.i.dTLB-stores
28.71 ± 22% -19.9 8.79 ± 72% perf-stat.i.iTLB-load-miss-rate%
661655 ± 18% -65.2% 230579 ± 60% perf-stat.i.iTLB-load-misses
1672859 ± 15% +53.1% 2560787 ± 13% perf-stat.i.iTLB-loads
4.377e+09 +92.6% 8.432e+09 perf-stat.i.instructions
7111 ± 22% +568.3% 47528 ± 35% perf-stat.i.instructions-per-iTLB-miss
0.61 ± 4% +63.4% 1.00 ± 4% perf-stat.i.ipc
0.10 ± 3% +18.6% 0.12 ± 3% perf-stat.i.metric.GHz
41.51 +84.7% 76.66 perf-stat.i.metric.M/sec
131741 ± 7% +48.3% 195398 ± 7% perf-stat.i.node-load-misses
36778 ± 31% -55.6% 16347 ± 16% perf-stat.i.node-stores
13.22 ± 23% -70.9% 3.85 ± 46% perf-stat.overall.MPKI
2.35 ± 12% -1.5 0.86 ± 30% perf-stat.overall.branch-miss-rate%
2.34 ± 18% +1.9 4.20 ± 14% perf-stat.overall.cache-miss-rate%
1.62 ± 4% -38.4% 1.00 ± 5% perf-stat.overall.cpi
0.08 ± 30% -0.1 0.02 ±103% perf-stat.overall.dTLB-load-miss-rate%
0.01 ± 50% -0.0 0.00 ±132% perf-stat.overall.dTLB-store-miss-rate%
28.68 ± 22% -19.9 8.73 ± 71% perf-stat.overall.iTLB-load-miss-rate%
6894 ± 21% +579.7% 46861 ± 36% perf-stat.overall.instructions-per-iTLB-miss
0.62 ± 4% +62.5% 1.01 ± 4% perf-stat.overall.ipc
138819 +581.5% 946109 perf-stat.overall.path-length
8.751e+08 +90.0% 1.662e+09 perf-stat.ps.branch-instructions
20537517 ± 12% -30.9% 14182248 ± 29% perf-stat.ps.branch-misses
57557495 ± 23% -44.2% 32125500 ± 44% perf-stat.ps.cache-references
182374 -60.2% 72538 perf-stat.ps.context-switches
7.055e+09 ± 3% +18.6% 8.366e+09 ± 3% perf-stat.ps.cpu-cycles
84.88 -8.1% 78.04 perf-stat.ps.cpu-migrations
1.28e+09 +89.1% 2.42e+09 perf-stat.ps.dTLB-loads
7.651e+08 +81.0% 1.385e+09 perf-stat.ps.dTLB-stores
659450 ± 18% -65.2% 229817 ± 60% perf-stat.ps.iTLB-load-misses
1667327 ± 15% +53.1% 2552296 ± 13% perf-stat.ps.iTLB-loads
4.363e+09 +92.6% 8.405e+09 perf-stat.ps.instructions
131326 ± 7% +48.3% 194773 ± 7% perf-stat.ps.node-load-misses
36665 ± 31% -55.5% 16301 ± 16% perf-stat.ps.node-stores
1.334e+12 +92.5% 2.568e+12 perf-stat.total.instructions
1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.active_objs
1551 ± 13% -100.0% 0.00 slabinfo.Acpi-Parse.num_objs
172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.active_objs
172.50 ± 6% -100.0% 0.00 slabinfo.UDPv6.num_objs
1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.active_objs
1648 ± 9% -42.1% 954.25 ± 2% slabinfo.UNIX.num_objs
497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.active_objs
497.25 ± 3% -38.7% 304.75 ± 6% slabinfo.bdev_cache.num_objs
3990 -100.0% 0.00 slabinfo.cred_jar.active_objs
3990 -100.0% 0.00 slabinfo.cred_jar.num_objs
1917 -26.9% 1400 slabinfo.dentry.active_slabs
1917 -26.9% 1400 slabinfo.dentry.num_slabs
2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.active_objs
2225 ± 8% -100.0% 0.00 slabinfo.dmaengine-unmap-16.num_objs
761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.active_objs
761.25 ± 11% -100.0% 0.00 slabinfo.file_lock_cache.num_objs
3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.active_objs
3185 ± 2% -12.2% 2795 ± 2% slabinfo.files_cache.num_objs
19419 ± 6% +25.2% 24310 ± 8% slabinfo.filp.active_objs
613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.active_slabs
19628 ± 6% +24.4% 24408 ± 8% slabinfo.filp.num_objs
613.00 ± 6% +24.4% 762.50 ± 8% slabinfo.filp.num_slabs
2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.active_objs
2080 ± 11% -100.0% 0.00 slabinfo.fsnotify_mark_connector.num_objs
1007 +58.3% 1594 slabinfo.inode_cache.active_slabs
1007 +58.3% 1594 slabinfo.inode_cache.num_slabs
2517 -35.4% 1625 slabinfo.kernfs_node_cache.active_slabs
2517 -35.4% 1625 slabinfo.kernfs_node_cache.num_slabs
4927 ± 2% -13.8% 4248 ± 4% slabinfo.kmalloc-1k.active_objs
5040 ± 2% -15.1% 4280 ± 4% slabinfo.kmalloc-1k.num_objs
4837 ± 4% +28.0% 6193 slabinfo.kmalloc-2k.active_objs
4870 ± 5% +28.1% 6237 slabinfo.kmalloc-2k.num_objs
1191 +19.1% 1419 slabinfo.kmalloc-4k.active_objs
1219 +18.3% 1442 slabinfo.kmalloc-4k.num_objs
3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.active_objs
3935 ± 4% +14.6% 4511 ± 5% slabinfo.kmalloc-rcl-64.num_objs
387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.active_objs
387.00 ± 12% -13.2% 336.00 ± 8% slabinfo.kmem_cache.num_objs
759.00 ± 10% -36.2% 484.00 ± 5% slabinfo.kmem_cache_node.active_objs
816.00 ± 10% -39.2% 496.00 ± 5% slabinfo.kmem_cache_node.num_objs
12875 -100.0% 0.00 slabinfo.lsm_file_cache.active_objs
12875 -100.0% 0.00 slabinfo.lsm_file_cache.num_objs
2271 -27.9% 1639 slabinfo.mm_struct.active_objs
2271 -27.9% 1639 slabinfo.mm_struct.num_objs
790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.active_objs
790.50 ± 7% -100.0% 0.00 slabinfo.mnt_cache.num_objs
123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.active_objs
123.75 ± 8% -100.0% 0.00 slabinfo.nfs_read_data.num_objs
294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.active_objs
294.50 ± 17% +360.4% 1355 ± 15% slabinfo.numa_policy.num_objs
3128 +73.2% 5418 slabinfo.pid.active_objs
3128 +73.2% 5418 slabinfo.pid.num_objs
10638 -13.0% 9252 slabinfo.proc_inode_cache.active_objs
10638 -12.9% 9266 slabinfo.proc_inode_cache.num_objs
5651 -15.6% 4771 slabinfo.shmem_inode_cache.active_objs
5651 -15.6% 4771 slabinfo.shmem_inode_cache.num_objs
2794 -11.6% 2469 slabinfo.signal_cache.active_objs
2794 -11.6% 2469 slabinfo.signal_cache.num_objs
13766 ± 10% -46.0% 7439 ± 16% slabinfo.sock_inode_cache.active_objs
13865 ± 10% -45.9% 7507 ± 16% slabinfo.sock_inode_cache.num_objs
4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.active_objs
4036 ± 2% +16.8% 4713 ± 2% slabinfo.task_delay_info.num_objs
940.25 +13.2% 1064 slabinfo.task_struct.active_objs
947.50 -43.4% 536.00 slabinfo.task_struct.active_slabs
947.50 +13.2% 1072 slabinfo.task_struct.num_objs
947.50 -43.4% 536.00 slabinfo.task_struct.num_slabs
1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.active_objs
1909 ± 2% +34.3% 2564 slabinfo.trace_event_file.num_objs
1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.active_slabs
1966 -41.0% 1159 ± 3% slabinfo.tw_sock_TCP.num_slabs
38.42 ± 3% -6.9 31.49 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
38.56 ± 3% -6.9 31.66 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
13.48 ± 3% -2.6 10.86 ± 5% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack
13.07 ± 3% -2.5 10.59 ± 5% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start
14.11 ± 3% -2.3 11.79 ± 5% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip
12.53 ± 3% -2.3 10.22 ± 5% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core
12.39 ± 3% -2.3 10.12 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv
13.88 ± 3% -2.2 11.64 ± 5% perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq
12.77 ± 3% -2.2 10.53 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action
14.41 ± 3% -2.2 12.18 ± 5% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
14.41 ± 3% -2.2 12.21 ± 5% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit
14.29 ± 3% -2.1 12.15 ± 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2
12.58 ± 3% -2.1 10.46 ± 5% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog
14.30 ± 3% -2.1 12.18 ± 5% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
8.47 ± 3% -1.7 6.73 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.43 ± 3% -1.7 6.72 perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.31 ± 3% -1.6 6.68 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.00 ± 3% -1.6 7.39 ± 3% perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64
9.22 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.20 ± 3% -1.6 7.62 ± 3% perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.21 ± 3% -1.6 6.63 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
9.07 ± 3% -1.6 7.51 ± 3% perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.81 -1.3 3.51 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.77 ± 2% -1.3 3.50 ± 3% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.98 ± 3% -1.2 5.73 perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
9.35 ± 3% -1.2 8.11 ± 4% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
9.27 ± 3% -1.2 8.06 ± 5% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
4.46 ± 2% -1.1 3.37 ± 3% perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.45 -1.1 3.37 ± 4% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect
4.49 ± 2% -1.1 3.42 ± 4% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect
4.05 -1.1 2.99 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect
4.52 ± 2% -1.0 3.48 ± 4% perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect
4.40 ± 2% -1.0 3.36 ± 3% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
6.03 ± 3% -0.9 5.18 ± 2% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
6.02 ± 3% -0.8 5.17 ± 2% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
5.18 ± 4% -0.8 4.39 ± 6% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release
7.17 ± 2% -0.8 6.38 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver
1.03 ± 2% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
4.58 ± 4% -0.7 3.85 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
0.98 ± 2% -0.7 0.26 ±100% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
3.51 ± 3% -0.7 2.80 ± 4% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock
3.77 ± 6% -0.7 3.07 ± 3% perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect
5.87 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release.sock_close
5.86 ± 4% -0.7 5.21 ± 8% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_close.inet_release.__sock_release
4.98 ± 4% -0.6 4.33 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_close
3.29 ± 3% -0.6 2.64 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process
3.42 ± 3% -0.6 2.78 ± 4% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock
3.35 ± 3% -0.6 2.71 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_state_process.tcp_v4_do_rcv
0.93 ± 6% -0.6 0.28 ±100% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable
4.42 ± 5% -0.6 3.81 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
7.44 ± 4% -0.6 6.85 ± 6% perf-profile.calltrace.cycles-pp.tcp_close.inet_release.__sock_release.sock_close.__fput
9.05 ± 4% -0.6 8.46 ± 5% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.33 ± 5% -0.6 3.74 ± 2% perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect
3.25 ± 5% -0.6 2.70 ± 3% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect
7.74 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.sock_close.__fput.task_work_run.exit_to_usermode_loop.do_syscall_64
7.61 ± 4% -0.5 7.07 ± 6% perf-profile.calltrace.cycles-pp.inet_release.__sock_release.sock_close.__fput.task_work_run
7.72 ± 4% -0.5 7.19 ± 5% perf-profile.calltrace.cycles-pp.__sock_release.sock_close.__fput.task_work_run.exit_to_usermode_loop
0.93 ± 7% -0.5 0.40 ± 57% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
0.94 ± 6% -0.5 0.42 ± 58% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process
0.95 ± 7% -0.5 0.43 ± 58% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv
1.55 ± 5% -0.5 1.03 perf-profile.calltrace.cycles-pp.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver
4.02 -0.5 3.50 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
2.68 ± 5% -0.5 2.20 ± 3% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect
1.10 ± 3% -0.5 0.62 ± 5% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
2.69 ± 5% -0.5 2.21 ± 3% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect
1.11 ± 7% -0.5 0.64 ± 9% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
1.03 ± 5% -0.5 0.56 ± 3% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
1.15 ± 3% -0.4 0.72 ± 7% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
3.07 ± 4% -0.4 2.64 ± 3% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect
1.03 ± 6% -0.4 0.60 ± 9% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu
0.99 ± 4% -0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
0.93 ± 6% -0.3 0.61 ± 6% perf-profile.calltrace.cycles-pp.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv
0.88 ± 5% -0.3 0.56 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established
1.40 ± 5% -0.3 1.12 ± 3% perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
1.52 ± 10% -0.2 1.28 ± 6% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.64 ± 2% -0.1 0.53 ± 3% perf-profile.calltrace.cycles-pp.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file.__sys_accept4
0.58 ± 5% -0.1 0.53 ± 2% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file
2.08 ± 2% +0.1 2.21 ± 3% perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
0.64 ± 4% +0.3 0.93 ± 6% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
1.15 ± 3% +0.3 1.44 ± 7% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
0.96 ± 4% +0.3 1.26 ± 10% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv
0.98 ± 4% +0.3 1.30 ± 10% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
0.34 ±101% +0.5 0.81 ± 18% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
0.35 ±101% +0.5 0.85 ± 18% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
0.00 +0.6 0.57 ± 9% perf-profile.calltrace.cycles-pp.hrtimer_get_next_event.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select
0.00 +0.6 0.59 ± 12% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv
0.13 ±173% +0.6 0.72 ± 5% perf-profile.calltrace.cycles-pp.run_timer_softirq.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
0.00 +0.6 0.61 ± 6% perf-profile.calltrace.cycles-pp.hrtimer_next_event_without.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
0.13 ±173% +0.6 0.75 ± 7% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv
0.00 +0.6 0.62 ± 4% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established
0.00 +0.6 0.62 ± 12% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
0.16 ±173% +0.7 0.82 ± 36% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt
0.89 ± 11% +0.7 1.58 ± 4% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
0.13 ±173% +0.7 0.83 ± 20% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
1.42 ± 18% +0.7 2.15 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt
0.58 ± 61% +0.7 1.33 ± 35% perf-profile.calltrace.cycles-pp.irq_enter.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
1.05 ± 16% +1.0 2.08 ± 25% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.12 ± 18% +1.0 2.15 ± 27% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
0.00 +1.1 1.15 ± 13% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.12 ± 11% +1.2 3.28 ± 2% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
1.41 ± 13% +1.3 2.71 ± 24% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.39 ±101% +1.4 1.77 ± 38% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.00 +1.5 1.48 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start
0.00 +1.5 1.49 ± 30% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
3.14 ± 13% +1.6 4.75 ± 11% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.35 ± 25% +1.8 3.14 ± 15% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
0.00 +1.9 1.86 ± 6% perf-profile.calltrace.cycles-pp.do_raw_spin_lock._raw_spin_lock_irqsave.rcu_core.__softirqentry_text_start.irq_exit
1.62 ± 20% +2.2 3.83 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
3.29 ± 11% +2.4 5.72 ± 4% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
3.98 ± 7% +2.9 6.83 ± 2% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
2.40 ± 14% +3.9 6.30 ± 24% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
4.14 ± 4% +6.5 10.63 ± 11% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
48.02 ± 4% +7.9 55.92 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
47.19 ± 4% +8.0 55.23 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
56.36 ± 2% +8.1 64.47 perf-profile.calltrace.cycles-pp.secondary_startup_64
54.68 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
54.66 ± 3% +8.7 63.34 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
54.62 ± 3% +8.7 63.31 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
10.59 ± 2% +10.3 20.89 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
8.93 ± 2% +10.6 19.50 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
38.73 ± 3% -6.7 32.02 ± 2% perf-profile.children.cycles-pp.do_syscall_64
38.86 ± 3% -6.7 32.19 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
13.52 ± 3% -2.7 10.87 ± 5% perf-profile.children.cycles-pp.__netif_receive_skb_one_core
13.11 ± 3% -2.5 10.60 ± 5% perf-profile.children.cycles-pp.ip_rcv
17.80 ± 3% -2.5 15.33 ± 4% perf-profile.children.cycles-pp.__tcp_transmit_skb
13.39 ± 2% -2.4 10.95 ± 4% perf-profile.children.cycles-pp.tcp_v4_do_rcv
2.72 -2.4 0.35 ± 4% perf-profile.children.cycles-pp.poll_idle
14.15 ± 3% -2.3 11.81 ± 5% perf-profile.children.cycles-pp.net_rx_action
12.56 ± 3% -2.3 10.22 ± 5% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
12.46 ± 3% -2.3 10.12 ± 5% perf-profile.children.cycles-pp.tcp_v4_rcv
12.80 ± 3% -2.3 10.53 ± 5% perf-profile.children.cycles-pp.ip_local_deliver
13.93 ± 3% -2.3 11.66 ± 5% perf-profile.children.cycles-pp.process_backlog
14.46 ± 3% -2.2 12.24 ± 5% perf-profile.children.cycles-pp.do_softirq
12.61 ± 3% -2.1 10.46 ± 5% perf-profile.children.cycles-pp.ip_local_deliver_finish
14.34 ± 3% -2.1 12.20 ± 5% perf-profile.children.cycles-pp.do_softirq_own_stack
16.07 ± 3% -1.8 14.25 ± 4% perf-profile.children.cycles-pp.ip_finish_output2
16.42 ± 3% -1.8 14.64 ± 4% perf-profile.children.cycles-pp.ip_output
16.86 ± 3% -1.8 15.08 ± 4% perf-profile.children.cycles-pp.__ip_queue_xmit
8.47 ± 3% -1.7 6.73 perf-profile.children.cycles-pp.__x64_sys_sendto
14.60 ± 3% -1.7 12.89 ± 4% perf-profile.children.cycles-pp.__local_bh_enable_ip
8.43 ± 3% -1.7 6.72 perf-profile.children.cycles-pp.__sys_sendto
8.32 ± 3% -1.6 6.68 perf-profile.children.cycles-pp.sock_sendmsg
9.00 ± 3% -1.6 7.39 ± 3% perf-profile.children.cycles-pp.__inet_stream_connect
9.22 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__x64_sys_connect
8.22 ± 3% -1.6 6.63 perf-profile.children.cycles-pp.tcp_sendmsg
9.20 ± 3% -1.6 7.62 ± 3% perf-profile.children.cycles-pp.__sys_connect
9.08 ± 3% -1.6 7.51 ± 3% perf-profile.children.cycles-pp.inet_stream_connect
6.35 ± 2% -1.5 4.81 ± 3% perf-profile.children.cycles-pp.__release_sock
11.90 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.__tcp_push_pending_frames
11.88 ± 3% -1.5 10.38 ± 5% perf-profile.children.cycles-pp.tcp_write_xmit
4.82 ± 2% -1.3 3.51 ± 3% perf-profile.children.cycles-pp.__x64_sys_recvfrom
4.77 ± 2% -1.3 3.50 ± 3% perf-profile.children.cycles-pp.__sys_recvfrom
8.17 ± 3% -1.3 6.92 ± 5% perf-profile.children.cycles-pp.tcp_rcv_state_process
6.99 ± 3% -1.3 5.73 perf-profile.children.cycles-pp.tcp_sendmsg_locked
6.10 ± 2% -1.2 4.88 ± 3% perf-profile.children.cycles-pp.release_sock
1.47 ± 5% -1.2 0.32 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64
1.45 ± 7% -1.1 0.35 ± 10% perf-profile.children.cycles-pp.syscall_return_via_sysret
5.09 -1.1 3.99 ± 4% perf-profile.children.cycles-pp.tcp_rcv_established
4.46 ± 2% -1.1 3.37 ± 3% perf-profile.children.cycles-pp.inet_recvmsg
4.41 ± 2% -1.0 3.36 ± 3% perf-profile.children.cycles-pp.tcp_recvmsg
1.83 ± 8% -1.0 0.83 ± 14% perf-profile.children.cycles-pp.tcp_ack
0.91 ± 8% -0.7 0.20 ± 18% perf-profile.children.cycles-pp.__inet_lookup_established
3.77 ± 6% -0.7 3.08 ± 3% perf-profile.children.cycles-pp.tcp_connect
2.63 ± 2% -0.6 2.00 ± 6% perf-profile.children.cycles-pp.try_to_wake_up
2.75 ± 2% -0.6 2.13 ± 6% perf-profile.children.cycles-pp.__wake_up_common
1.19 ± 6% -0.6 0.59 ± 11% perf-profile.children.cycles-pp.read_tsc
7.45 ± 4% -0.6 6.86 ± 6% perf-profile.children.cycles-pp.tcp_close
9.06 ± 4% -0.6 8.47 ± 5% perf-profile.children.cycles-pp.exit_to_usermode_loop
4.33 ± 5% -0.6 3.74 ± 2% perf-profile.children.cycles-pp.tcp_v4_connect
3.03 ± 2% -0.6 2.45 ± 6% perf-profile.children.cycles-pp.__wake_up_common_lock
7.75 ± 4% -0.6 7.19 ± 5% perf-profile.children.cycles-pp.sock_close
7.72 ± 4% -0.5 7.19 ± 5% perf-profile.children.cycles-pp.__sock_release
7.61 ± 4% -0.5 7.07 ± 6% perf-profile.children.cycles-pp.inet_release
1.56 ± 6% -0.5 1.04 perf-profile.children.cycles-pp.tcp_child_process
0.98 ± 6% -0.4 0.53 ± 7% perf-profile.children.cycles-pp.enqueue_task_fair
0.95 ± 6% -0.4 0.54 ± 11% perf-profile.children.cycles-pp.autoremove_wake_function
0.68 ± 5% -0.4 0.27 ± 11% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
2.34 ± 3% -0.4 1.94 ± 6% perf-profile.children.cycles-pp.sock_def_readable
0.90 ± 4% -0.4 0.52 ± 7% perf-profile.children.cycles-pp.__kfree_skb
0.98 ± 5% -0.4 0.60 ± 9% perf-profile.children.cycles-pp.activate_task
0.96 ± 8% -0.4 0.58 ± 17% perf-profile.children.cycles-pp.tcp_clean_rtx_queue
0.99 ± 5% -0.4 0.64 ± 9% perf-profile.children.cycles-pp.ttwu_do_activate
0.78 ± 5% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.enqueue_entity
1.06 ± 5% -0.3 0.73 ± 6% perf-profile.children.cycles-pp.sock_def_wakeup
0.39 ± 10% -0.3 0.11 ± 7% perf-profile.children.cycles-pp.skb_release_all
1.40 ± 5% -0.3 1.12 ± 3% perf-profile.children.cycles-pp.tcp_conn_request
0.53 ± 9% -0.3 0.27 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.36 ± 10% -0.3 0.10 ± 10% perf-profile.children.cycles-pp.skb_release_head_state
0.35 ± 12% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.skb_release_data
0.48 ± 13% -0.2 0.24 ± 13% perf-profile.children.cycles-pp.__skb_datagram_iter
0.29 ± 10% -0.2 0.06 ± 63% perf-profile.children.cycles-pp.dst_release
1.59 ± 9% -0.2 1.35 ± 4% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 11% -0.2 0.11 ± 17% perf-profile.children.cycles-pp.__switch_to
0.48 ± 12% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.27 ± 9% -0.2 0.04 ± 58% perf-profile.children.cycles-pp.copy_user_generic_unrolled
0.55 ± 9% -0.2 0.32 ± 11% perf-profile.children.cycles-pp.set_next_entity
0.69 ± 4% -0.2 0.47 ± 6% perf-profile.children.cycles-pp.native_write_msr
0.29 ± 20% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.ipv4_mtu
0.27 ± 6% -0.2 0.06 ± 14% perf-profile.children.cycles-pp.call_rcu
0.45 ± 11% -0.2 0.24 ± 7% perf-profile.children.cycles-pp.sched_clock
1.13 ± 4% -0.2 0.93 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_bh
0.51 ± 7% -0.2 0.30 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair
0.71 ± 3% -0.2 0.52 ± 3% perf-profile.children.cycles-pp.sk_reset_timer
0.50 ± 5% -0.2 0.31 ± 6% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.29 ± 5% -0.2 0.10 ± 23% perf-profile.children.cycles-pp.__slab_free
0.90 ± 4% -0.2 0.71 ± 6% perf-profile.children.cycles-pp.dequeue_task_fair
0.24 ± 11% -0.2 0.06 ± 26% perf-profile.children.cycles-pp.ipv4_dst_check
0.67 ± 13% -0.2 0.50 ± 2% perf-profile.children.cycles-pp.__next_timer_interrupt
0.22 ± 6% -0.2 0.05 ± 58% perf-profile.children.cycles-pp.__ksize
0.40 ± 14% -0.2 0.23 ± 8% perf-profile.children.cycles-pp.native_sched_clock
0.19 ± 15% -0.2 0.03 ±102% perf-profile.children.cycles-pp.nf_hook_slow
0.20 ± 18% -0.2 0.04 ± 60% perf-profile.children.cycles-pp.sock_rfree
0.22 ± 3% -0.2 0.07 ± 19% perf-profile.children.cycles-pp.sockfd_lookup_light
0.23 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.__switch_to_asm
0.52 ± 5% -0.1 0.38 ± 2% perf-profile.children.cycles-pp.lapic_next_deadline
0.21 ± 9% -0.1 0.07 ± 16% perf-profile.children.cycles-pp.__check_object_size
0.47 ± 15% -0.1 0.33 ± 7% perf-profile.children.cycles-pp._find_next_bit
0.17 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp._cond_resched
0.25 ± 20% -0.1 0.12 ± 23% perf-profile.children.cycles-pp.__x64_sys_close
0.27 ± 18% -0.1 0.14 ± 38% perf-profile.children.cycles-pp.update_ts_time_stats
0.50 ± 13% -0.1 0.36 ± 13% perf-profile.children.cycles-pp.tcp_create_openreq_child
0.17 ± 9% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.validate_xmit_skb
0.48 ± 9% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.update_rq_clock
0.18 ± 16% -0.1 0.05 ± 58% perf-profile.children.cycles-pp.__might_sleep
0.21 ± 14% -0.1 0.08 ± 59% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
0.23 ± 10% -0.1 0.10 ± 19% perf-profile.children.cycles-pp.rcu_eqs_exit
0.25 ± 11% -0.1 0.13 ± 22% perf-profile.children.cycles-pp.nr_iowait_cpu
0.19 ± 11% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.__fget_light
0.21 ± 15% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.run_ksoftirqd
0.20 ± 11% -0.1 0.08 ± 23% perf-profile.children.cycles-pp.select_idle_sibling
0.20 ± 24% -0.1 0.08 ± 74% perf-profile.children.cycles-pp.wake_up_klogd_work_func
0.15 ± 10% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.update_cfs_group
0.21 ± 23% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.36 ± 6% -0.1 0.25 ± 16% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.23 ± 19% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.smpboot_thread_fn
0.44 ± 7% -0.1 0.34 ± 4% perf-profile.children.cycles-pp.tcp_send_fin
1.51 ± 5% -0.1 1.41 ± 5% perf-profile.children.cycles-pp.wait_woken
0.14 ± 19% -0.1 0.03 ±100% perf-profile.children.cycles-pp.simple_copy_to_iter
0.16 ± 29% -0.1 0.06 ± 66% perf-profile.children.cycles-pp.rb_next
0.36 ± 12% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.inet_csk_destroy_sock
0.27 ± 13% -0.1 0.17 ± 16% perf-profile.children.cycles-pp.__sk_dst_check
0.76 ± 7% -0.1 0.67 ± 6% perf-profile.children.cycles-pp.dequeue_entity
0.15 ± 12% -0.1 0.06 ± 58% perf-profile.children.cycles-pp.sk_forced_mem_schedule
0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.inet_reqsk_alloc
0.27 ± 6% -0.1 0.18 ± 15% perf-profile.children.cycles-pp.memcpy_erms
0.37 ± 11% -0.1 0.28 ± 15% perf-profile.children.cycles-pp.inet_csk_clone_lock
0.14 ± 18% -0.1 0.05 ± 64% perf-profile.children.cycles-pp.inet_sk_rebuild_header
0.42 ± 8% -0.1 0.34 ± 16% perf-profile.children.cycles-pp.native_irq_return_iret
0.18 ± 10% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.rcu_eqs_enter
0.11 ± 24% -0.1 0.04 ±107% perf-profile.children.cycles-pp.native_apic_mem_write
0.15 ± 25% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.tcp_current_mss
0.19 ± 10% -0.1 0.12 ± 17% perf-profile.children.cycles-pp._copy_to_iter
0.11 ± 14% -0.1 0.04 ± 60% perf-profile.children.cycles-pp.reweight_entity
0.16 ± 23% -0.1 0.09 ± 37% perf-profile.children.cycles-pp.menu_reflect
0.12 ± 8% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.sock_recvmsg
0.12 ± 29% -0.1 0.05 ± 67% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.12 ± 19% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.tcp_release_cb
0.10 ± 23% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.run_local_timers
0.25 ± 12% -0.1 0.19 ± 14% perf-profile.children.cycles-pp.__tcp_send_ack
0.09 ± 25% -0.1 0.03 ±100% perf-profile.children.cycles-pp.rb_insert_color
0.26 ± 5% -0.1 0.20 ± 10% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.12 ± 3% -0.1 0.06 ± 62% perf-profile.children.cycles-pp.security_socket_recvmsg
0.12 ± 28% -0.1 0.07 ± 31% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.tcp_add_backlog
0.12 ± 13% -0.1 0.07 ± 17% perf-profile.children.cycles-pp.security_sk_alloc
0.11 ± 13% -0.1 0.06 ± 20% perf-profile.children.cycles-pp.apparmor_sk_alloc_security
0.19 ± 13% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.tcp_make_synack
0.07 ± 11% -0.0 0.03 ±102% perf-profile.children.cycles-pp.security_socket_getsockopt
0.20 ± 6% -0.0 0.16 ± 10% perf-profile.children.cycles-pp.sk_prot_alloc
0.10 ± 27% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.tcp_get_metrics
0.11 ± 17% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__update_load_avg_se
0.11 ± 14% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.pick_next_entity
0.11 ± 9% -0.0 0.08 ± 23% perf-profile.children.cycles-pp.inode_init_always
0.15 ± 12% -0.0 0.12 ± 15% perf-profile.children.cycles-pp.remove_wait_queue
0.07 ± 16% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.read
0.06 ± 14% +0.0 0.09 ± 17% perf-profile.children.cycles-pp.forkshell
0.07 ± 13% +0.0 0.10 ± 10% perf-profile.children.cycles-pp.security_sk_free
0.06 +0.0 0.10 ± 15% perf-profile.children.cycles-pp.inet_csk_reqsk_queue_drop
0.14 ± 7% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.evict
0.11 ± 6% +0.0 0.15 ± 12% perf-profile.children.cycles-pp.sock_alloc_inode
0.06 ± 66% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.add_wait_queue
0.03 ±100% +0.1 0.08 ± 16% perf-profile.children.cycles-pp.tick_sched_do_timer
0.08 ± 17% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.try_to_del_timer_sync
0.04 ± 58% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.down_write
0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_group_exit
0.03 ±102% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.do_exit
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.alloc_set_pte
0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.bpf_sk_storage_free
0.09 ± 38% +0.1 0.15 ± 19% perf-profile.children.cycles-pp.execve
0.01 ±173% +0.1 0.07 ± 10% perf-profile.children.cycles-pp.update_rt_rq_load_avg
0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.sock_owned_by_me
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.clear_inode
0.01 ±173% +0.1 0.07 ± 22% perf-profile.children.cycles-pp.__sk_free
0.00 +0.1 0.06 ± 17% perf-profile.children.cycles-pp.__fd_install
0.18 ± 15% +0.1 0.25 ± 12% perf-profile.children.cycles-pp.del_timer
0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.__libc_start_main
0.03 ±102% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.main
0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.update_dl_rq_load_avg
0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.rcu_read_lock_bh_held
0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.lockdep_init_map
0.00 +0.1 0.07 ± 25% perf-profile.children.cycles-pp.netlink_has_listeners
0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.ksys_read
0.06 ± 22% +0.1 0.12 ± 8% perf-profile.children.cycles-pp.vfs_read
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.register_lock_class
0.00 +0.1 0.07 ± 21% perf-profile.children.cycles-pp.inet_twsk_hashdance
0.32 ± 9% +0.1 0.39 ± 5% perf-profile.children.cycles-pp.alloc_empty_file
0.01 ±173% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.cmd_stat
0.00 +0.1 0.07 ± 15% perf-profile.children.cycles-pp.inet_csk_init_xmit_timers
0.32 ± 6% +0.1 0.39 ± 9% perf-profile.children.cycles-pp.inet_create
0.31 ± 11% +0.1 0.38 ± 6% perf-profile.children.cycles-pp.__alloc_file
0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.do_filp_open
0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.path_openat
0.01 ±173% +0.1 0.09 ± 7% perf-profile.children.cycles-pp.__note_gp_changes
0.00 +0.1 0.08 ± 20% perf-profile.children.cycles-pp.trace_hardirqs_off_caller
0.33 ± 6% +0.1 0.41 ± 9% perf-profile.children.cycles-pp.tcp_done
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.inet_sk_state_store
0.14 ± 7% +0.1 0.22 ± 13% perf-profile.children.cycles-pp.sk_alloc
0.06 ± 13% +0.1 0.14 ± 32% perf-profile.children.cycles-pp.rcu_gp_kthread
0.03 ±100% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.tcp_update_metrics
0.04 ± 59% +0.1 0.12 ± 27% perf-profile.children.cycles-pp._copy_from_user
0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.inet_put_port
0.16 ± 17% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__x64_sys_setsockopt
0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.detach_if_pending
0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.cgroup_sk_alloc
0.01 ±173% +0.1 0.10 ± 15% perf-profile.children.cycles-pp.enqueue_timer
0.21 ± 8% +0.1 0.30 ± 7% perf-profile.children.cycles-pp.__inet_bind
0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_open
0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.do_sys_openat2
0.15 ± 14% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.__sys_setsockopt
0.03 ±100% +0.1 0.12 ± 11% perf-profile.children.cycles-pp.filemap_map_pages
0.01 ±173% +0.1 0.10 ± 4% perf-profile.children.cycles-pp.__update_idle_core
0.00 +0.1 0.09 ± 27% perf-profile.children.cycles-pp.ret_from_intr
0.17 ± 10% +0.1 0.26 ± 9% perf-profile.children.cycles-pp.newidle_balance
0.00 +0.1 0.10 ± 11% perf-profile.children.cycles-pp.init_timer_key
0.03 ±100% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.pick_next_task_idle
0.00 +0.1 0.10 ± 21% perf-profile.children.cycles-pp.do_raw_spin_trylock
0.00 +0.1 0.10 ± 18% perf-profile.children.cycles-pp._raw_write_lock_bh
0.01 ±173% +0.1 0.12 ± 3% perf-profile.children.cycles-pp.set_next_task_idle
0.00 +0.1 0.11 ± 20% perf-profile.children.cycles-pp.restore_regs_and_return_to_kernel
0.15 ± 10% +0.1 0.26 ± 16% perf-profile.children.cycles-pp.inet_csk_route_req
0.09 ± 8% +0.1 0.21 ± 5% perf-profile.children.cycles-pp.sock_setsockopt
0.24 ± 10% +0.1 0.36 ± 6% perf-profile.children.cycles-pp.__kmalloc_reserve
0.22 ± 8% +0.1 0.33 ± 9% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.00 +0.1 0.12 ± 32% perf-profile.children.cycles-pp.__local_bh_enable
2.08 ± 2% +0.1 2.21 ± 3% perf-profile.children.cycles-pp.sk_wait_data
0.39 ± 7% +0.1 0.51 ± 9% perf-profile.children.cycles-pp.tcp_mstamp_refresh
0.09 ± 20% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.tcp_set_state
0.09 ± 26% +0.1 0.22 ± 11% perf-profile.children.cycles-pp.del_timer_sync
0.03 ±100% +0.1 0.15 ± 14% perf-profile.children.cycles-pp.cpuacct_charge
0.07 ± 62% +0.1 0.20 ± 6% perf-profile.children.cycles-pp.__handle_mm_fault
0.06 ± 61% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.handle_pte_fault
0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.14 ± 7% perf-profile.children.cycles-pp.trace_softirqs_on
0.00 +0.1 0.14 ± 12% perf-profile.children.cycles-pp.lock_unpin_lock
0.34 ± 19% +0.1 0.48 ± 10% perf-profile.children.cycles-pp.enqueue_to_backlog
0.09 ± 8% +0.1 0.24 ± 11% perf-profile.children.cycles-pp.dput
0.20 ± 13% +0.1 0.35 ± 7% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.00 +0.1 0.15 ± 29% perf-profile.children.cycles-pp.trace_softirqs_off
0.00 +0.2 0.15 ± 12% perf-profile.children.cycles-pp.lockdep_rtnl_is_held
0.00 +0.2 0.15 ± 14% perf-profile.children.cycles-pp.trace_hardirqs_on_thunk
0.23 ± 12% +0.2 0.39 ± 11% perf-profile.children.cycles-pp.__sk_destruct
0.07 ± 64% +0.2 0.22 ± 4% perf-profile.children.cycles-pp.handle_mm_fault
0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.__raw_spin_lock_init
0.00 +0.2 0.16 ± 15% perf-profile.children.cycles-pp._raw_spin_unlock_bh
0.00 +0.2 0.16 ± 13% perf-profile.children.cycles-pp.lock_pin_lock
0.06 ± 11% +0.2 0.22 ± 14% perf-profile.children.cycles-pp.trigger_load_balance
0.10 ± 28% +0.2 0.26 ± 4% perf-profile.children.cycles-pp.do_page_fault
0.22 ± 21% +0.2 0.39 ± 15% perf-profile.children.cycles-pp.fib_table_lookup
0.11 ± 31% +0.2 0.28 ± 6% perf-profile.children.cycles-pp.page_fault
0.16 ± 7% +0.2 0.33 ± 9% perf-profile.children.cycles-pp.sk_filter_trim_cap
0.16 ± 49% +0.2 0.34 ± 29% perf-profile.children.cycles-pp.enqueue_hrtimer
0.00 +0.2 0.18 ± 14% perf-profile.children.cycles-pp._raw_spin_lock_nested
0.19 ± 26% +0.2 0.38 ± 25% perf-profile.children.cycles-pp.finish_task_switch
0.09 ± 21% +0.2 0.29 ± 9% perf-profile.children.cycles-pp.___perf_sw_event
0.19 ± 14% +0.2 0.40 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.00 +0.2 0.21 ± 31% perf-profile.children.cycles-pp.trace_hardirqs_off_thunk
0.00 +0.2 0.21 ± 11% perf-profile.children.cycles-pp.tracer_hardirqs_on
0.11 ± 15% +0.2 0.33 ± 12% perf-profile.children.cycles-pp.raise_softirq
0.18 ± 4% +0.2 0.40 ± 8% perf-profile.children.cycles-pp.kfree
0.31 +0.2 0.53 ± 14% perf-profile.children.cycles-pp.ktime_get_with_offset
1.40 ± 5% +0.2 1.63 ± 2% perf-profile.children.cycles-pp.dev_hard_start_xmit
0.39 ± 9% +0.3 0.65 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc
0.22 ± 37% +0.3 0.49 ± 9% perf-profile.children.cycles-pp.run_rebalance_domains
0.00 +0.3 0.27 ± 12% perf-profile.children.cycles-pp.tracer_hardirqs_off
0.18 ± 26% +0.3 0.46 ± 8% perf-profile.children.cycles-pp.update_blocked_averages
0.27 ± 8% +0.3 0.54 ± 3% perf-profile.children.cycles-pp.update_curr
0.00 +0.3 0.28 ± 6% perf-profile.children.cycles-pp.debug_check_no_locks_freed
0.00 +0.3 0.29 ± 14% perf-profile.children.cycles-pp.do_raw_spin_unlock
0.00 +0.3 0.32 ± 10% perf-profile.children.cycles-pp.reacquire_held_locks
0.39 ± 10% +0.4 0.74 ± 13% perf-profile.children.cycles-pp.kmem_cache_free
0.31 ± 7% +0.4 0.67 ± 5% perf-profile.children.cycles-pp.ip_route_output_flow
0.39 ± 12% +0.4 0.75 ± 7% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu
0.48 ± 51% +0.4 0.85 ± 18% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.00 +0.4 0.38 ± 13% perf-profile.children.cycles-pp.__local_bh_disable_ip
0.62 ± 11% +0.4 1.01 ± 4% perf-profile.children.cycles-pp.netif_rx_internal
0.00 +0.4 0.40 ± 10% perf-profile.children.cycles-pp.trace_hardirqs_on
2.03 ± 6% +0.4 2.44 ± 4% perf-profile.children.cycles-pp.__dev_queue_xmit
0.33 ± 28% +0.4 0.75 ± 40% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.00 +0.4 0.42 ± 7% perf-profile.children.cycles-pp.fs_reclaim_acquire
0.20 ± 6% +0.4 0.63 ± 6% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.00 +0.4 0.43 ± 21% perf-profile.children.cycles-pp._raw_spin_unlock_irq
0.43 ± 11% +0.4 0.87 ± 6% perf-profile.children.cycles-pp.ip_route_output_key_hash
0.47 ± 15% +0.4 0.91 ± 20% perf-profile.children.cycles-pp.scheduler_tick
2.42 +0.4 2.87 ± 7% perf-profile.children.cycles-pp.tcp_data_queue
0.13 ± 20% +0.4 0.58 ± 10% perf-profile.children.cycles-pp.__might_fault
0.13 ± 18% +0.5 0.58 ± 8% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.28 ± 65% +0.5 0.74 ± 5% perf-profile.children.cycles-pp.run_timer_softirq
0.00 +0.6 0.58 ± 5% perf-profile.children.cycles-pp.rcu_is_watching
0.64 ± 10% +0.6 1.23 ± 4% perf-profile.children.cycles-pp.netif_rx
0.70 ± 34% +0.7 1.36 ± 34% perf-profile.children.cycles-pp.irq_enter
0.92 ± 11% +0.7 1.60 ± 4% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.49 ± 10% +0.8 1.25 ± 3% perf-profile.children.cycles-pp.___might_sleep
0.00 +0.8 0.79 ± 5% perf-profile.children.cycles-pp.trace_hardirqs_off
0.00 +0.8 0.82 ± 5% perf-profile.children.cycles-pp.find_held_lock
0.22 ± 8% +0.9 1.08 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq
1.29 ± 16% +0.9 2.23 ± 25% perf-profile.children.cycles-pp.update_process_times
1.35 ± 16% +0.9 2.29 ± 26% perf-profile.children.cycles-pp.tick_sched_handle
0.26 ± 31% +0.9 1.20 ± 13% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.00 +1.0 1.03 ± 2% perf-profile.children.cycles-pp.mark_held_locks
0.00 +1.0 1.04 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock
0.70 ± 14% +1.1 1.76 ± 4% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.49 ± 11% +1.1 1.58 ± 30% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +1.1 1.15 ± 11% perf-profile.children.cycles-pp.lockdep_hardirqs_off
2.52 ± 10% +1.2 3.70 ± 2% perf-profile.children.cycles-pp.rcu_core
0.00 +1.2 1.19 ± 3% perf-profile.children.cycles-pp.match_held_lock
1.65 ± 14% +1.3 2.90 ± 23% perf-profile.children.cycles-pp.tick_sched_timer
0.00 +1.4 1.37 ± 11% perf-profile.children.cycles-pp.rcu_lockdep_current_cpu_online
3.23 ± 13% +1.6 4.83 ± 11% perf-profile.children.cycles-pp.menu_select
0.00 +1.7 1.68 ± 3% perf-profile.children.cycles-pp.rcu_read_lock_held
2.81 ± 9% +1.7 4.56 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +1.8 1.76 ± 4% perf-profile.children.cycles-pp.lockdep_hardirqs_on
0.00 +1.8 1.79 ± 12% perf-profile.children.cycles-pp.debug_lockdep_rcu_enabled
1.40 ± 24% +1.8 3.21 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event
0.00 +2.2 2.19 perf-profile.children.cycles-pp.mark_lock
1.68 ± 20% +2.2 3.88 ± 13% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +2.5 2.46 ± 9% perf-profile.children.cycles-pp.__lock_text_start
0.00 +2.7 2.65 ± 4% perf-profile.children.cycles-pp.rcu_read_lock_held_common
4.25 ± 6% +2.9 7.11 ± 2% perf-profile.children.cycles-pp.irq_exit
0.00 +3.0 2.97 ± 4% perf-profile.children.cycles-pp.do_raw_spin_lock
2.18 ± 33% +3.4 5.57 ± 4% perf-profile.children.cycles-pp.ktime_get
2.75 ± 16% +3.9 6.65 ± 25% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.00 +4.7 4.71 ± 3% perf-profile.children.cycles-pp.lock_release
0.00 +4.7 4.74 perf-profile.children.cycles-pp.rcu_read_lock_sched_held
0.00 +6.2 6.18 ± 2% perf-profile.children.cycles-pp.lock_is_held_type
4.58 ± 8% +6.6 11.14 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt
49.50 ± 3% +7.4 56.87 perf-profile.children.cycles-pp.cpuidle_enter
49.48 ± 3% +7.4 56.85 perf-profile.children.cycles-pp.cpuidle_enter_state
56.39 ± 2% +8.1 64.48 perf-profile.children.cycles-pp.do_idle
56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.secondary_startup_64
56.36 ± 2% +8.1 64.47 perf-profile.children.cycles-pp.cpu_startup_entry
54.68 ± 3% +8.7 63.34 perf-profile.children.cycles-pp.start_secondary
10.58 ± 4% +10.5 21.06 ± 9% perf-profile.children.cycles-pp.apic_timer_interrupt
9.66 ± 4% +10.6 20.30 ± 9% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
0.00 +10.8 10.77 ± 3% perf-profile.children.cycles-pp.__lock_acquire
0.00 +14.2 14.15 ± 2% perf-profile.children.cycles-pp.lock_acquire
2.65 ± 2% -2.3 0.32 ± 8% perf-profile.self.cycles-pp.poll_idle
2.35 ± 8% -1.8 0.58 ± 6% perf-profile.self.cycles-pp.do_syscall_64
1.62 ± 6% -1.5 0.12 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.47 ± 6% -1.1 0.32 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64
1.45 ± 7% -1.1 0.35 ± 10% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.81 ± 9% -0.6 0.17 ± 16% perf-profile.self.cycles-pp.__inet_lookup_established
1.14 ± 5% -0.6 0.56 ± 13% perf-profile.self.cycles-pp.read_tsc
1.26 ± 15% -0.6 0.70 ± 21% perf-profile.self.cycles-pp.menu_select
0.81 ± 8% -0.5 0.32 ± 4% perf-profile.self.cycles-pp.__sched_text_start
0.56 ± 10% -0.4 0.17 ± 15% perf-profile.self.cycles-pp.tcp_recvmsg
0.63 ± 4% -0.4 0.25 ± 14% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.50 ± 10% -0.4 0.14 ± 19% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.46 ± 10% -0.4 0.11 ± 7% perf-profile.self.cycles-pp.tcp_ack
0.42 ± 12% -0.3 0.11 ± 12% perf-profile.self.cycles-pp.__dev_queue_xmit
0.47 ± 6% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.tcp_v4_rcv
0.50 ± 14% -0.3 0.22 ± 7% perf-profile.self.cycles-pp.do_idle
0.57 ± 7% -0.3 0.29 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.48 ± 12% -0.3 0.21 ± 4% perf-profile.self.cycles-pp.___might_sleep
0.35 ± 12% -0.3 0.09 ± 18% perf-profile.self.cycles-pp.skb_release_data
0.36 ± 5% -0.3 0.10 ± 11% perf-profile.self.cycles-pp.__netif_receive_skb_core
0.38 ± 11% -0.3 0.12 ± 9% perf-profile.self.cycles-pp.set_next_entity
0.29 ± 17% -0.2 0.04 ± 60% perf-profile.self.cycles-pp.sock_def_readable
0.31 ± 12% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.26 ± 11% -0.2 0.03 ±100% perf-profile.self.cycles-pp.copy_user_generic_unrolled
0.29 ± 12% -0.2 0.06 ± 64% perf-profile.self.cycles-pp.dst_release
0.35 ± 10% -0.2 0.12 ± 5% perf-profile.self.cycles-pp.ip_finish_output2
0.69 ± 4% -0.2 0.47 ± 6% perf-profile.self.cycles-pp.native_write_msr
0.32 ± 9% -0.2 0.10 ± 19% perf-profile.self.cycles-pp.__switch_to
0.35 ± 14% -0.2 0.14 ± 21% perf-profile.self.cycles-pp.update_rq_clock
0.29 ± 8% -0.2 0.09 ± 12% perf-profile.self.cycles-pp.__ip_queue_xmit
0.27 ± 20% -0.2 0.07 ± 10% perf-profile.self.cycles-pp.ipv4_mtu
1.03 ± 7% -0.2 0.83 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state
0.24 ± 8% -0.2 0.05 ± 60% perf-profile.self.cycles-pp.tcp_sendmsg_locked
0.28 ± 5% -0.2 0.10 ± 23% perf-profile.self.cycles-pp.__slab_free
0.27 ± 7% -0.2 0.09 ± 21% perf-profile.self.cycles-pp.select_task_rq_fair
0.22 ± 6% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.__ksize
0.20 ± 20% -0.2 0.03 ±105% perf-profile.self.cycles-pp.net_rx_action
0.39 ± 15% -0.2 0.22 ± 9% perf-profile.self.cycles-pp.native_sched_clock
0.21 ± 8% -0.2 0.05 ± 58% perf-profile.self.cycles-pp.aa_sk_perm
0.24 ± 9% -0.2 0.07 ± 15% perf-profile.self.cycles-pp.loopback_xmit
0.22 ± 12% -0.2 0.06 ± 16% perf-profile.self.cycles-pp.enqueue_entity
0.25 ± 10% -0.2 0.09 ± 17% perf-profile.self.cycles-pp.process_backlog
0.22 ± 21% -0.2 0.06 ± 20% perf-profile.self.cycles-pp.fib_table_lookup
0.19 ± 16% -0.2 0.04 ±107% perf-profile.self.cycles-pp.ipv4_dst_check
0.23 ± 14% -0.2 0.08 ± 21% perf-profile.self.cycles-pp.tcp_rcv_established
0.19 ± 23% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.enqueue_to_backlog
0.23 ± 11% -0.1 0.08 ± 23% perf-profile.self.cycles-pp.__switch_to_asm
0.19 ± 16% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.sock_rfree
0.35 ± 14% -0.1 0.20 ± 17% perf-profile.self.cycles-pp.note_gp_changes
0.46 ± 14% -0.1 0.32 ± 6% perf-profile.self.cycles-pp._find_next_bit
0.21 ± 18% -0.1 0.08 ± 27% perf-profile.self.cycles-pp.kmem_cache_alloc
0.21 ± 23% -0.1 0.07 ± 17% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.21 ± 14% -0.1 0.08 ± 59% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit
0.19 ± 11% -0.1 0.07 ± 13% perf-profile.self.cycles-pp.__fget_light
0.16 ± 16% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.__might_sleep
0.24 ± 12% -0.1 0.13 ± 22% perf-profile.self.cycles-pp.nr_iowait_cpu
0.16 ± 2% -0.1 0.05 ± 60% perf-profile.self.cycles-pp.tcp_rcv_state_process
0.14 ± 24% -0.1 0.03 ±100% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.15 ± 10% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.update_cfs_group
0.17 ± 8% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.tcp_write_xmit
0.14 ± 13% -0.1 0.04 ±100% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.19 ± 18% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.17 ± 11% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.mod_timer
0.17 ± 17% -0.1 0.08 ± 20% perf-profile.self.cycles-pp.ip_local_deliver
0.14 ± 15% -0.1 0.05 ± 58% perf-profile.self.cycles-pp.sk_forced_mem_schedule
0.27 ± 6% -0.1 0.18 ± 13% perf-profile.self.cycles-pp.memcpy_erms
0.18 ± 8% -0.1 0.09 ± 17% perf-profile.self.cycles-pp.ip_output
0.14 ± 27% -0.1 0.06 ± 66% perf-profile.self.cycles-pp.rb_next
0.18 ± 10% -0.1 0.10 ± 15% perf-profile.self.cycles-pp.rcu_eqs_enter
0.42 ± 8% -0.1 0.34 ± 16% perf-profile.self.cycles-pp.native_irq_return_iret
0.30 ± 14% -0.1 0.22 ± 11% perf-profile.self.cycles-pp.__softirqentry_text_start
0.14 ± 8% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.11 ± 14% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.reweight_entity
0.18 ± 6% -0.1 0.11 ± 21% perf-profile.self.cycles-pp.kfree
0.12 ± 20% -0.1 0.05 ± 67% perf-profile.self.cycles-pp.menu_reflect
0.12 ± 12% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.dequeue_entity
0.12 ± 13% -0.1 0.07 ± 25% perf-profile.self.cycles-pp.newidle_balance
0.08 ± 23% -0.1 0.03 ±100% perf-profile.self.cycles-pp.run_local_timers
0.08 ± 19% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rb_insert_color
0.24 ± 12% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.__next_timer_interrupt
0.12 ± 18% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.resched_curr
0.11 ± 14% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__update_load_avg_se
0.12 ± 24% -0.0 0.07 ± 10% perf-profile.self.cycles-pp.update_curr
0.10 ± 14% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu
0.09 ± 15% -0.0 0.04 ± 59% perf-profile.self.cycles-pp.rcu_eqs_exit
0.07 ± 22% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.dev_hard_start_xmit
0.11 ± 4% -0.0 0.08 ± 22% perf-profile.self.cycles-pp.finish_task_switch
0.10 ± 15% -0.0 0.07 ± 21% perf-profile.self.cycles-pp.inet_csk_accept
0.08 ± 8% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.release_sock
0.08 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.rcu_idle_exit
0.05 ± 62% +0.0 0.08 ± 15% perf-profile.self.cycles-pp.__wake_up_common
0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.register_lock_class
0.15 ± 27% +0.1 0.22 ± 15% perf-profile.self.cycles-pp.rcu_core
0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.reacquire_held_locks
0.03 ±100% +0.1 0.12 ± 18% perf-profile.self.cycles-pp.smp_apic_timer_interrupt
0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp._raw_spin_unlock
0.00 +0.1 0.10 ± 25% perf-profile.self.cycles-pp.__lock_text_start
0.00 +0.1 0.10 ± 21% perf-profile.self.cycles-pp.do_raw_spin_trylock
0.00 +0.1 0.11 ± 10% perf-profile.self.cycles-pp.lock_pin_lock
0.00 +0.1 0.12 ± 14% perf-profile.self.cycles-pp.lock_unpin_lock
0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.trace_softirqs_on
0.00 +0.1 0.14 ± 31% perf-profile.self.cycles-pp.trace_softirqs_off
0.00 +0.2 0.18 ± 11% perf-profile.self.cycles-pp.rcu_read_lock_held
0.00 +0.2 0.18 ± 17% perf-profile.self.cycles-pp.tracer_hardirqs_on
0.00 +0.2 0.21 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_on
0.00 +0.2 0.24 ± 10% perf-profile.self.cycles-pp.tracer_hardirqs_off
0.00 +0.3 0.28 ± 13% perf-profile.self.cycles-pp.do_raw_spin_unlock
0.00 +0.3 0.28 ± 6% perf-profile.self.cycles-pp.debug_check_no_locks_freed
0.00 +0.3 0.28 ± 3% perf-profile.self.cycles-pp.__local_bh_disable_ip
0.00 +0.4 0.39 ± 14% perf-profile.self.cycles-pp.rcu_read_lock_sched_held
0.00 +0.4 0.43 ± 7% perf-profile.self.cycles-pp.trace_hardirqs_off
0.00 +0.5 0.48 ± 6% perf-profile.self.cycles-pp.rcu_is_watching
0.00 +0.5 0.51 ± 9% perf-profile.self.cycles-pp.find_held_lock
0.00 +0.5 0.53 ± 3% perf-profile.self.cycles-pp.mark_held_locks
0.00 +0.7 0.68 ± 8% perf-profile.self.cycles-pp.rcu_read_lock_held_common
0.00 +0.8 0.75 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_on
0.00 +0.9 0.88 ± 6% perf-profile.self.cycles-pp.match_held_lock
0.00 +1.1 1.13 ± 11% perf-profile.self.cycles-pp.lockdep_hardirqs_off
0.00 +1.3 1.30 ± 5% perf-profile.self.cycles-pp.do_raw_spin_lock
0.00 +1.3 1.31 ± 11% perf-profile.self.cycles-pp.rcu_lockdep_current_cpu_online
0.00 +1.5 1.55 ± 12% perf-profile.self.cycles-pp.debug_lockdep_rcu_enabled
0.00 +2.1 2.07 perf-profile.self.cycles-pp.mark_lock
0.00 +3.3 3.29 ± 2% perf-profile.self.cycles-pp.lock_release
0.00 +3.3 3.30 ± 2% perf-profile.self.cycles-pp.lock_acquire
0.00 +5.4 5.41 perf-profile.self.cycles-pp.lock_is_held_type
0.00 +9.1 9.07 ± 4% perf-profile.self.cycles-pp.__lock_acquire
netperf.Throughput_tps
35000 +-------------------------------------------------------------------+
| +.+..+.+..+..+.+ +.+..+.+..+..+ +.+..+..+.+..+..+.+..+.+..|
30000 |-+: : : : : |
| : : : : : |
25000 |-: : : : : |
| : : : : : |
20000 |-: : : : : |
| :O O O O O O: O :O O : : |
15000 |:+ : : : : |
|: : : : : |
10000 |:+ : : O O:O: O O O O O |
|: : : O O :: |
5000 |-+ : : |
| : : |
0 +-------------------------------------------------------------------+
netperf.Throughput_total_tps
35000 +-------------------------------------------------------------------+
| +.+..+.+..+..+.+ +.+..+.+..+..+ +.+..+..+.+..+..+.+..+.+..|
30000 |-+: : : : : |
| : : : : : |
25000 |-: : : : : |
| : : : : : |
20000 |-: : : : : |
| :O O O O O O: O :O O : : |
15000 |:+ : : : : |
|: : : : : |
10000 |:+ : : O O:O: O O O O O |
|: : : O O :: |
5000 |-+ : : |
| : : |
0 +-------------------------------------------------------------------+
netperf.workload
1e+07 +-------------------------------------------------------------------+
9e+06 |-+: : : +.+..+..+ + +.+..+..+.+..+.+..|
| : : : : : |
8e+06 |-: : : : : |
7e+06 |-: : : : : |
| : : : : : |
6e+06 |-: : : : : |
5e+06 |-:O O O O O O :O :O O : : |
4e+06 |:+ : : : : |
|: : : : : |
3e+06 |:+ : : O O O O:O: O O O O O |
2e+06 |:+ : : :: |
| : : |
1e+06 |-+ : : |
0 +-------------------------------------------------------------------+
netperf.time.user_time
40 +----------------------------------------------------------------------+
| |
35 |-+ + .+.. .+.. |
30 |-++.+..+..+..+.+..+ .+..+.. +: +.+..+. + +..+..+.+..|
| : : +.+. + : : |
25 |-: : : : : |
| : : : : : |
20 |-: O O O : :O O : : |
| :O O O: O : : : |
15 |:+ : : : : |
10 |:+ : : O O O O :O: O O O |
|: : : : : O O |
5 |:+ : : :: |
| : : |
0 +----------------------------------------------------------------------+
netperf.time.voluntary_context_switches
2e+07 +-----------------------------------------------------------------+
1.8e+07 |-+ .+.+..+..+ +.+..+.+..+.+..+.+..+.+..|
| +.+..+.+.. .+..+ +.+. : : |
1.6e+07 |-+: + : : : : |
1.4e+07 |-+: : : : : |
| : : : : : |
1.2e+07 |-: : : : : |
1e+07 |-:O O O O O O:O :O O : : |
8e+06 |-: : : : : |
|: : : : : |
6e+06 |:+ : : O O O O:O: O O O O O |
4e+06 |:+ :: :: |
|: :: : |
2e+06 |-+ : : |
0 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
8 months, 2 weeks
[RDMA/core] 7b491b3fb2: BUG:kernel_hang_in_test_stage
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: 7b491b3fb2f4f5749432aba03a6ee31731446ea6 ("[PATCH 2/4] RDMA/core: Introduce shared CQ pool API")
url: https://github.com/0day-ci/linux/commits/Yamin-Friedman/Introducing-RDMA-...
base: https://git.kernel.org/cgit/linux/kernel/git/rdma/rdma.git for-next
in testcase: rcuperf
with following parameters:
runtime: 300s
perf_type: rcu
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------+------------+------------+
| | 3b3f80b960 | 7b491b3fb2 |
+-------------------------------+------------+------------+
| boot_successes | 4 | 1 |
| boot_failures | 0 | 5 |
| BUG:kernel_hang_in_test_stage | 0 | 5 |
+-------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 14.746983] random: systemd: uninitialized urandom read (16 bytes read)
[ 14.758869] watchdog: watchdog0: watchdog did not stop!
[ 454.101407] random: fast init done
[ 867.382064] random: crng init done
BUG: kernel hang in test stage
To reproduce:
# build kernel
cd linux
cp config-5.7.0-rc1-00152-g7b491b3fb2f4f .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
8 months, 2 weeks
[KVM] 6b6a864bd7: kernel-selftests.kvm.vmx_tsc_adjust_test.fail
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: 6b6a864bd7d7eb12a82ec6da1c2dd97a43f79449 ("[PATCH v2] KVM: nVMX: Tweak handling of failure code for nested VM-Enter failure")
url: https://github.com/0day-ci/linux/commits/Sean-Christopherson/KVM-nVMX-Twe...
base: https://git.kernel.org/cgit/virt/kvm/kvm.git linux-next
in testcase: kernel-selftests
with following parameters:
group: kselftests-kvm
ucode: 0x500002c
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
# selftests: kvm: vmx_tsc_adjust_test
# ==== Test Assertion Failure ====
# x86_64/vmx_tsc_adjust_test.c:153: false
# pid=12157 tid=12157 - Interrupted system call
# 1 0x000000000040116a: main at vmx_tsc_adjust_test.c:153
# 2 0x00007fafd54bce0a: ?? ??:0
# 3 0x00000000004011e9: _start at ??:?
# Failed guest assert: (vmreadz(VM_EXIT_REASON) == (0x80000000 | 33))
# IA32_TSC_ADJUST is -4294972240 (-1 * TSC_ADJUST_VALUE + -4944).
not ok 14 selftests: kvm: vmx_tsc_adjust_test # exit=254
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
8 months, 2 weeks
[selftests] 5c39e08fc4: kernel-selftests.netfilter.nft_nat.sh.fail
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: 5c39e08fc417b91d24d243bc6d1a305e1c38fb9f ("selftests: netfilter: add simple masq/redirect test cases")
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-4.19.y
in testcase: kernel-selftests
with following parameters:
group: kselftests-02
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
selftests: netfilter: nft_nat.sh
========================================
/dev/stdin:18:6-16: Error: Could not process rule: Operation not supported
map nsincounter {
^^^^^^^^^^^
/dev/stdin:18:6-16: Error: Could not process rule: No such file or directory
map nsincounter {
^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:52:3-40: Error: Could not process rule: No such file or directory
counter name ip saddr map @nsincounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:3-87: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:57:3-41: Error: Could not process rule: No such file or directory
counter name ip daddr map @nsoutcounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:3-88: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:18:6-16: Error: Could not process rule: Operation not supported
map nsincounter {
^^^^^^^^^^^
/dev/stdin:18:6-16: Error: Could not process rule: No such file or directory
map nsincounter {
^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:52:3-40: Error: Could not process rule: No such file or directory
counter name ip saddr map @nsincounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:3-87: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:57:3-41: Error: Could not process rule: No such file or directory
counter name ip daddr map @nsoutcounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:3-88: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:18:6-16: Error: Could not process rule: Operation not supported
map nsincounter {
^^^^^^^^^^^
/dev/stdin:18:6-16: Error: Could not process rule: No such file or directory
map nsincounter {
^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:26:6-17: Error: Could not process rule: No such file or directory
map nsincounter6 {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:34:6-17: Error: Could not process rule: No such file or directory
map nsoutcounter {
^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:42:6-18: Error: Could not process rule: No such file or directory
map nsoutcounter6 {
^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:15-46: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:52:3-40: Error: Could not process rule: No such file or directory
counter name ip saddr map @nsincounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:53:3-87: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 saddr map @nsincounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:57:3-41: Error: Could not process rule: No such file or directory
counter name ip daddr map @nsoutcounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/dev/stdin:58:3-88: Error: Could not process rule: No such file or directory
icmpv6 type { "echo-request", "echo-reply" } counter name ip6 daddr map @nsoutcounter6
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
ERROR: ns1 counter in ns0 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
ERROR: ns0in6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
ERROR: ns0out6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
ERROR: ns0in6 counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
ERROR: ns0out6 counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns2 counter in ns0 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2
^^^^^^
/dev/stdin:2:8-13: Error: Could not process rule: No such file or directory
chain output {
^^^^^^
/dev/stdin:4:3-38: Error: Could not process rule: No such file or directory
ip daddr 10.0.1.99 dnat to 10.0.2.99
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns2in counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns2out counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: Could not process rule: No such file or directory
flush chain ip nat output
^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns2in counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns2out counter in ns0 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns0 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns2in counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns2out counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
/dev/stdin:2:8-13: Error: Could not process rule: No such file or directory
chain output {
^^^^^^
/dev/stdin:4:3-41: Error: Could not process rule: No such file or directory
ip6 daddr dead:1::99 dnat to dead:2::99
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SKIP: Could not add add ip6 dnat hook
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
/dev/stdin:2:8-18: Error: Could not process rule: No such file or directory
chain postrouting {
^^^^^^^^^^^
/dev/stdin:4:3-27: Error: Could not process rule: No such file or directory
meta oif veth0 masquerade
^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns1in counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns1out counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
Error: Could not process rule: No such file or directory
flush chain ip nat postrouting
^^^
ERROR: Could not flush nat postrouting
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
/dev/stdin:2:8-18: Error: Could not process rule: No such file or directory
chain postrouting {
^^^^^^^^^^^
/dev/stdin:4:3-27: Error: Could not process rule: No such file or directory
meta oif veth0 masquerade
^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
Error: Could not process rule: No such file or directory
flush chain ip6 nat postrouting
^^^
ERROR: Could not flush ip6 nat postrouting
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns2in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
ERROR: ns1in counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns2out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
ERROR: ns1out counter in ns2 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns1out
^^^^^^
/dev/stdin:2:8-17: Error: Could not process rule: No such file or directory
chain prerouting {
^^^^^^^^^^
/dev/stdin:4:3-80: Error: Could not process rule: No such file or directory
meta iif veth1 ip protocol icmp ip saddr 10.0.2.99 ip daddr 10.0.1.99 redirect
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: No such file or directory
list counter inet filter ns2in
^^^^^^
ERROR: ns0in counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0in
^^^^^^
Error: No such file or directory
list counter inet filter ns2out
^^^^^^
ERROR: ns0out counter in ns1 has unexpected value (expected packets 1 bytes 84)
Error: No such file or directory
list counter inet filter ns0out
^^^^^^
Error: Could not process rule: No such file or directory
delete table ip nat
^^^
ERROR: Could not delete nat table
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns2in6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
ERROR: ns1in6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns2out6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
ERROR: ns1out6 counter in ns2 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns1out6
^^^^^^
/dev/stdin:2:8-17: Error: Could not process rule: No such file or directory
chain prerouting {
^^^^^^^^^^
/dev/stdin:4:3-87: Error: Could not process rule: No such file or directory
meta iif veth1 meta l4proto icmpv6 ip6 saddr dead:2::99 ip6 daddr dead:1::99 redirect
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 0 bytes 0)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: No such file or directory
list counter inet filter ns2in6
^^^^^^
ERROR: ns0in6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0in6
^^^^^^
Error: No such file or directory
list counter inet filter ns2out6
^^^^^^
ERROR: ns0out6 counter in ns1 has unexpected value (expected packets 1 bytes 104)
Error: No such file or directory
list counter inet filter ns0out6
^^^^^^
Error: Could not process rule: No such file or directory
delete table ip6 nat
^^^
ERROR: Could not delete ip6 nat table
not ok 1..2 selftests: netfilter: nft_nat.sh [FAIL]
To reproduce:
# build kernel
cd linux
cp config-4.19.28-00103-g5c39e08fc417b .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
8 months, 2 weeks
[pipe] f2af7d90e2: xfstests.btrfs.052.fail
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-7):
commit: f2af7d90e25ce1cf03318bf3290fcfc1e762d8f9 ("pipe: read/write_iter() handler should check for IOCB_NOWAIT")
url: https://github.com/0day-ci/linux/commits/UPDATE-20200501-052027/Jens-Axbo...
in testcase: xfstests
with following parameters:
disk: 6HDD
fs: btrfs
test: btrfs-group00
test-description: xfstests is a regression test suite for xfs and other files ystems.
test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
2020-05-06 21:03:14 export TEST_DIR=/fs/vda
2020-05-06 21:03:14 export TEST_DEV=/dev/vda
2020-05-06 21:03:14 export FSTYP=btrfs
2020-05-06 21:03:14 export SCRATCH_MNT=/fs/scratch
2020-05-06 21:03:14 mkdir /fs/scratch -p
2020-05-06 21:03:15 export SCRATCH_DEV_POOL="/dev/vdb /dev/vdc /dev/vdd /dev/vde /dev/vdf"
2020-05-06 21:03:15 sed "s:^:btrfs/:" //lkp/benchmarks/xfstests/tests/btrfs-group00
2020-05-06 21:03:15 ./check btrfs/001 btrfs/004 btrfs/007 btrfs/010 btrfs/013 btrfs/016 btrfs/019 btrfs/022 btrfs/025 btrfs/028 btrfs/031 btrfs/034 btrfs/037 btrfs/040 btrfs/043 btrfs/046 btrfs/049 btrfs/052 btrfs/055 btrfs/058 btrfs/061 btrfs/064 btrfs/067 btrfs/071 btrfs/074 btrfs/077 btrfs/080 btrfs/083 btrfs/086 btrfs/089 btrfs/092 btrfs/095 btrfs/098 btrfs/101 btrfs/104 btrfs/107 btrfs/110 btrfs/113 btrfs/116 btrfs/119 btrfs/122 btrfs/125 btrfs/128 btrfs/131 btrfs/134 btrfs/137 btrfs/140 btrfs/143 btrfs/146 btrfs/149 btrfs/152 btrfs/155 btrfs/158 btrfs/161 btrfs/164 btrfs/167 btrfs/170 btrfs/173 btrfs/176 btrfs/179 btrfs/182 btrfs/185 btrfs/188 btrfs/191 btrfs/194 btrfs/197 btrfs/200 btrfs/203 btrfs/206 btrfs/209
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 vm-snb-30 5.7.0-rc3-00044-gf2af7d90e25ce #1 SMP Wed May 6 04:20:50 CST 2020
MKFS_OPTIONS -- /dev/vdb
MOUNT_OPTIONS -- /dev/vdb /fs/scratch
btrfs/001 3s
btrfs/004 73s
btrfs/007 1s
btrfs/010 129s
btrfs/013 11s
btrfs/016 2s
btrfs/019 0s
btrfs/022 5s
btrfs/025 1s
btrfs/028 31s
btrfs/031 0s
btrfs/034 6s
btrfs/037 1s
btrfs/040 1s
btrfs/043 1s
btrfs/046 4s
btrfs/049 6s
btrfs/052 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/052.out.bad)
--- tests/btrfs/052.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/052.out.bad 2020-05-06 21:07:55.779485703 +0800
@@ -1,643 +1,139 @@
QA output created by 052
Testing with a cow file (default)
-Blocks modified: [0 - 1]
-Blocks modified: [2 - 3]
-Blocks modified: [4 - 5]
-Blocks modified: [6 - 7]
-Blocks modified: [8 - 23]
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/052.out /lkp/benchmarks/xfstests/results//btrfs/052.out.bad' to see the entire diff)
btrfs/055 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/055.out.bad)
--- tests/btrfs/055.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/055.out.bad 2020-05-06 21:07:56.994485703 +0800
@@ -1,337 +1,59 @@
QA output created by 055
Testing without the NO_HOLES feature
-Blocks modified: [0 - 1]
-Blocks modified: [2 - 3]
-Blocks modified: [6 - 7]
-Blocks modified: [8 - 9]
-Blocks modified: [0 - 23]
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/055.out /lkp/benchmarks/xfstests/results//btrfs/055.out.bad' to see the entire diff)
btrfs/058 3s
btrfs/061 97s
btrfs/064 88s
btrfs/067 51s
btrfs/071 22s
btrfs/074 45s
btrfs/077 0s
btrfs/080 43s
btrfs/083 1s
btrfs/086 1s
btrfs/089 1s
btrfs/092 1s
btrfs/095 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/095.out.bad)
--- tests/btrfs/095.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/095.out.bad 2020-05-06 21:13:51.276485703 +0800
@@ -1,35 +1,9 @@
QA output created by 095
-Blocks modified: [135 - 164]
-Blocks modified: [768 - 792]
+awk: line 19: function strtonum never defined
+awk: line 19: function strtonum never defined
+awk: line 19: function strtonum never defined
+awk: line 19: function strtonum never defined
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/095.out /lkp/benchmarks/xfstests/results//btrfs/095.out.bad' to see the entire diff)
btrfs/098 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/098.out.bad)
--- tests/btrfs/098.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/098.out.bad 2020-05-06 21:13:52.311485703 +0800
@@ -1,22 +1,7 @@
QA output created by 098
-Blocks modified: [200 - 224]
+awk: line 19: function strtonum never defined
+awk: line 19: function strtonum never defined
File contents before power failure:
-0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
-*
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/098.out /lkp/benchmarks/xfstests/results//btrfs/098.out.bad' to see the entire diff)
btrfs/101 17s
btrfs/104 1s
btrfs/107 1s
btrfs/110 1s
btrfs/113 1s
btrfs/116 [not run] FITRIM not supported on /fs/scratch
btrfs/119 1s
btrfs/122 6s
btrfs/125 19s
btrfs/128 2s
btrfs/131 2s
btrfs/134 1s
btrfs/137 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/137.out.bad)
--- tests/btrfs/137.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/137.out.bad 2020-05-06 21:14:46.418485703 +0800
@@ -17,22 +17,15 @@
d3dc847171f9081bd75d7a2d3b53d322 SCRATCH_MNT/snap2/bar
File snap1/foo fiemap results in the original filesystem:
-0: [0..7]: data
File snap1/bar fiemap results in the original filesystem:
-0: [0..7]: data
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/137.out /lkp/benchmarks/xfstests/results//btrfs/137.out.bad' to see the entire diff)
btrfs/140 1s
btrfs/143 2s
btrfs/146 2s
btrfs/149 2s
btrfs/152 1s
btrfs/155 1s
btrfs/158 1s
btrfs/161 0s
btrfs/164 2s
btrfs/167 1s
btrfs/170 1s
btrfs/173 0s
btrfs/176 2s
btrfs/179 133s
btrfs/182 3s
btrfs/185 0s
btrfs/188 1s
btrfs/191 1s
btrfs/194 20s
btrfs/197 - output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/197.out.bad)
--- tests/btrfs/197.out 2020-04-09 10:45:28.000000000 +0800
+++ /lkp/benchmarks/xfstests/results//btrfs/197.out.bad 2020-05-06 21:17:44.450485703 +0800
@@ -3,23 +3,19 @@
Label: none uuid: <UUID>
Total devices <NUM> FS bytes used <SIZE>
devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
- *** Some devices missing
raid5
Label: none uuid: <UUID>
...
(Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/197.out /lkp/benchmarks/xfstests/results//btrfs/197.out.bad' to see the entire diff)
btrfs/200 1s
btrfs/203 1s
btrfs/206 [not run] This test requires a valid $LOGWRITES_DEV
btrfs/209 0s
Ran: btrfs/001 btrfs/004 btrfs/007 btrfs/010 btrfs/013 btrfs/016 btrfs/019 btrfs/022 btrfs/025 btrfs/028 btrfs/031 btrfs/034 btrfs/037 btrfs/040 btrfs/043 btrfs/046 btrfs/049 btrfs/052 btrfs/055 btrfs/058 btrfs/061 btrfs/064 btrfs/067 btrfs/071 btrfs/074 btrfs/077 btrfs/080 btrfs/083 btrfs/086 btrfs/089 btrfs/092 btrfs/095 btrfs/098 btrfs/101 btrfs/104 btrfs/107 btrfs/110 btrfs/113 btrfs/116 btrfs/119 btrfs/122 btrfs/125 btrfs/128 btrfs/131 btrfs/134 btrfs/137 btrfs/140 btrfs/143 btrfs/146 btrfs/149 btrfs/152 btrfs/155 btrfs/158 btrfs/161 btrfs/164 btrfs/167 btrfs/170 btrfs/173 btrfs/176 btrfs/179 btrfs/182 btrfs/185 btrfs/188 btrfs/191 btrfs/194 btrfs/197 btrfs/200 btrfs/203 btrfs/206 btrfs/209
Not run: btrfs/116 btrfs/206
Failures: btrfs/052 btrfs/055 btrfs/095 btrfs/098 btrfs/137 btrfs/197
Failed 6 of 70 tests
To reproduce:
# build kernel
cd linux
cp config-5.7.0-rc3-00044-gf2af7d90e25ce .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
8 months, 2 weeks
9ef0b22bca ("fs: call file_{start,end}_write from .."): [ 45.496622] WARNING: possible recursive locking detected
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://git.infradead.org/users/hch/misc.git cleanup-kernel-rw-helpers
commit 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1
Author: Christoph Hellwig <hch(a)lst.de>
AuthorDate: Thu May 7 19:27:06 2020 +0200
Commit: Christoph Hellwig <hch(a)lst.de>
CommitDate: Fri May 8 11:11:54 2020 +0200
fs: call file_{start,end}_write from __kernel_write
We always need to take a reference on the file system we are writing
to.
Signed-off-by: Christoph Hellwig <hch(a)lst.de>
79dede78c0 Merge branch 'for-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
9ef0b22bca fs: call file_{start,end}_write from __kernel_write
c2c302133b fs: don't change the address limit for ->read_iter in __kernel_read
+-------------------------------------------------+------------+------------+------------+
| | 79dede78c0 | 9ef0b22bca | c2c302133b |
+-------------------------------------------------+------------+------------+------------+
| boot_successes | 151 | 39 | 0 |
| boot_failures | 2 | 13 | 1 |
| INFO:rcu_sched_self-detected_stall_on_CPU | 1 | | |
| RIP:iov_iter_copy_from_user_atomic | 1 | | |
| BUG:soft_lockup-CPU##stuck_for#s![trinity-c2:#] | 2 | | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 2 | | |
| RIP:smp_call_function_single | 1 | | |
| WARNING:possible_recursive_locking_detected | 0 | 13 | 1 |
| BUG:unable_to_handle_page_fault_for_address | 0 | 1 | |
| Oops:#[##] | 0 | 1 | |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 1 | |
+-------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory
[ 45.219908] audit: type=1326 audit(1589183752.519:33): auid=4294967295 uid=65534 gid=65534 ses=4294967295 subj=kernel pid=3333 comm="trinity-c1" exe="/bin/trinity" sig=9 arch=c000003e syscall=8 compat=0 ip=0x4587a7 code=0x0
modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory
[ 45.495619]
[ 45.495976] ============================================
[ 45.496622] WARNING: possible recursive locking detected
[ 45.497266] 5.7.0-rc4-00209-g9ef0b22bca82d #1 Not tainted
[ 45.497882] --------------------------------------------
[ 45.498489] trinity-c7/3230 is trying to acquire lock:
[ 45.499081] ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: __kernel_write+0x12f/0x180
[ 45.500032]
[ 45.500032] but task is already holding lock:
[ 45.500791] ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: do_sendfile+0x370/0x3c0
[ 45.501761]
[ 45.501761] other info that might help us debug this:
[ 45.502615] Possible unsafe locking scenario:
[ 45.502615]
[ 45.503383] CPU0
[ 45.503756] ----
[ 45.504128] lock(sb_writers#6);
[ 45.504568] lock(sb_writers#6);
[ 45.505008]
[ 45.505008] *** DEADLOCK ***
[ 45.505008]
[ 45.505896] May be due to missing lock nesting notation
[ 45.505896]
[ 45.506738] 1 lock held by trinity-c7/3230:
[ 45.507259] #0: ffff888220b40448 (sb_writers#6){.+.+}-{0:0}, at: do_sendfile+0x370/0x3c0
[ 45.508242]
[ 45.508242] stack backtrace:
[ 45.508903] CPU: 1 PID: 3230 Comm: trinity-c7 Not tainted 5.7.0-rc4-00209-g9ef0b22bca82d #1
[ 45.509922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 45.510919] Call Trace:
[ 45.511292] dump_stack+0x8f/0xcb
[ 45.511735] validate_chain+0x1228/0x1240
[ 45.512235] __lock_acquire+0x86f/0xbc0
[ 45.512720] lock_acquire+0xac/0x390
[ 45.513188] ? __kernel_write+0x12f/0x180
[ 45.513699] __sb_start_write+0x137/0x220
[ 45.514198] ? __kernel_write+0x12f/0x180
[ 45.514699] ? do_splice_direct+0xd0/0xd0
[ 45.515202] __kernel_write+0x12f/0x180
[ 45.515693] ? do_splice_direct+0xd0/0xd0
[ 45.516192] write_pipe_buf+0x76/0xa0
[ 45.516664] __splice_from_pipe+0xc3/0x1c0
[ 45.517171] ? do_splice_direct+0xd0/0xd0
[ 45.521759] splice_from_pipe+0x5d/0x90
[ 45.522250] default_file_splice_write+0x15/0x20
[ 45.522810] direct_splice_actor+0x35/0x40
[ 45.523342] splice_direct_to_actor+0x110/0x250
[ 45.523907] ? generic_pipe_buf_nosteal+0x10/0x10
[ 45.524485] do_splice_direct+0xa0/0xd0
[ 45.524976] do_sendfile+0x1d5/0x3c0
[ 45.525451] __x64_sys_sendfile64+0x5c/0xd0
[ 45.525968] do_syscall_64+0x60/0x280
[ 45.526449] entry_SYSCALL_64_after_hwframe+0x49/0xb3
[ 45.527042] RIP: 0033:0x457729
[ 45.527468] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00
[ 45.529442] RSP: 002b:00007fffa97ea1f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
[ 45.530378] RAX: ffffffffffffffda RBX: 0000000000000028 RCX: 0000000000457729
[ 45.531178] RDX: 00007fcc78f87000 RSI: 00000000000000ba RDI: 00000000000000ba
[ 45.531942] RBP: 00007fffa97ea2a0 R08: 0000000000100000 R09: 0000000000000048
[ 45.532703] R10: 0000000000001000 R11: 0000000000000246 R12: 0000000000000002
[ 45.533470] R13: 00007fcc7988b058 R14: 000000000277b830 R15: 00007fcc7988b000
modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory
[child2:3389] read (0) returned ENOSYS, marking as inactive.
modprobe: FATAL: Could not load /lib/modules/5.7.0-rc4-00209-g9ef0b22bca82d/modules.dep: No such file or directory
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 18c86474c709bd4e3317a8d0dddb160bdae94996 0e698dfa282211e414076f9dc7e83c1c288314fd --
git bisect bad f06471c67ef6929f7e054de7651a47fdfa00c8e8 # 00:05 B 7 3 0 0 Merge 'linux-review/Yonghong-Song/bpf-implement-bpf-iterator-for-kernel-data/20200510-033248' into devel-hourly-2020051021
git bisect good 459e15afbe338077afbba837212c1c1a7527b202 # 00:26 G 26 0 1 1 Merge 'linux-review/Steven-Rostedt/tools-bootconfig-Fix-apply_xbc-to-return-zero-on-success/20200509-070610' into devel-hourly-2020051021
git bisect good df47a754d7e6c065c132eb060a6824c1e99276d9 # 00:44 G 26 0 0 0 Merge 'linux-review/Dan-Murphy/net-ethernet-ti-cpts-Fix-linker-issue-when-TI_CPTS-is-defined/20200508-230627' into devel-hourly-2020051021
git bisect good cfbd4971d03fefcc627e8aeacc4d305964114535 # 01:05 G 24 0 0 0 Merge 'linux-review/Mike-Marciniszyn/IB-hfi1-Fix-another-case-where-pq-is-left-on-waitlist/20200505-070157' into devel-hourly-2020051021
git bisect good da8cd247cb72b5e588884a4b745b32f32740c95c # 01:33 G 24 0 0 0 Merge 'linux-review/Charan-Teja-Reddy/dma-buf-fix-use-after-free-in-dmabuffs_dname/20200509-032857' into devel-hourly-2020051021
git bisect bad d6bae61b2c781390851cf4182ea2fdc74ec03b5f # 01:49 B 4 2 0 0 Merge 'linux-review/Enric-Balletbo-i-Serra/Convert-mtk-dpi-to-drm_bridge-API/20200505-072853' into devel-hourly-2020051021
git bisect good 18f312ea74b4185bc135eccc8fc8a161c0402ceb # 02:31 G 25 0 0 0 Merge 'linux-review/Thierry-Reding/clk-tegra-Fix-initial-rate-for-pll_a-on-Tegra124/20200506-024253' into devel-hourly-2020051021
git bisect bad 9ed636bcf097595b4946aa746ce3b541becb0e71 # 03:02 B 16 1 0 0 Merge 'linux-review/Christophe-JAILLET/clk-clk-xgene-Fix-a-typo-in-Kconfig/20200505-034755' into devel-hourly-2020051021
git bisect good 365c8030c03fd5109d1191fdf9aab067663b8c54 # 03:32 G 51 0 1 1 Merge 'linux-review/Yong-Zhi/ASoC-max98373-reorder-max98373_reset-in-resume/20200502-074911' into devel-hourly-2020051021
git bisect bad 9fd97e3bc948c40e14fb27d9ac3fc788b7cae193 # 03:47 B 2 1 0 0 Merge 'hch-misc/cleanup-kernel-rw-helpers' into devel-hourly-2020051021
git bisect bad dd0b01a50f3de2d8e71b4865f50bbe49810e061a # 04:12 B 8 1 0 0 fs: don't change the address limit for ->write_iter in __kernel_write
git bisect bad bcdb099498e6c632d1363b6a1bf83898bdd9015d # 04:38 B 4 1 0 0 fs: remove the call_{read,write}_iter functions
git bisect bad 31a9c7ee10bc2bef9f623a11f77bf7e2d72aba5c # 05:00 B 5 1 0 0 fs: check FMODE_WRITE in __kernel_write
git bisect bad 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1 # 05:38 B 7 1 0 0 fs: call file_{start,end}_write from __kernel_write
# first bad commit: [9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1] fs: call file_{start,end}_write from __kernel_write
git bisect good 79dede78c0573618e3137d3d8cbf78c84e25fabd # 06:34 G 150 0 3 3 Merge branch 'for-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
# extra tests with debug options
git bisect bad 9ef0b22bca82dcdc3fb570f177ca1d6f4d0973b1 # 06:55 B 0 1 17 0 fs: call file_{start,end}_write from __kernel_write
# extra tests on head commit of hch-misc/cleanup-kernel-rw-helpers
git bisect bad c2c302133bed00699eb52d5516f45d79c4f4ee85 # 07:35 B 0 1 17 0 fs: don't change the address limit for ->read_iter in __kernel_read
# bad: [c2c302133bed00699eb52d5516f45d79c4f4ee85] fs: don't change the address limit for ->read_iter in __kernel_read
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/lkp@lists.01.org
8 months, 2 weeks