[mm] [confidence: ] e49f335ded: BUG:sleeping_function_called_from_invalid_context_at_mm/util.c
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: e49f335dedc18a2ee0ac7791134b282abd9c857f ("mm: check for sleepable context in kvfree")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/jlayton/linux.git mm-5.4
in testcase: xfstests
with following parameters:
disk: 4HDD
fs: f2fs
test: generic-quick3
test-description: xfstests is a regression test suite for xfs and other files ystems.
test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+----------------------------------------------------------------+------------+------------+
| | 7b5cf701ea | e49f335ded |
+----------------------------------------------------------------+------------+------------+
| boot_successes | 67 | 0 |
| boot_failures | 96 | 100 |
| WARNING:at_arch/x86/kernel/rtc.c:#mach_get_cmos_time | 4 | |
| RIP:mach_get_cmos_time | 4 | |
| BUG:kernel_reboot-without-warning_in_test_stage | 55 | |
| INFO:rcu_sched_self-detected_stall_on_CPU | 1 | |
| RIP:__memcpy | 1 | |
| BUG:kernel_hang_in_boot_stage | 2 | |
| BUG:kernel_NULL_pointer_dereference,address | 35 | |
| Oops:#[##] | 35 | |
| RIP:dma_direct_max_mapping_size | 35 | |
| Kernel_panic-not_syncing:Fatal_exception | 35 | |
| BUG:sleeping_function_called_from_invalid_context_at_mm/util.c | 0 | 100 |
+----------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 24.662886] BUG: sleeping function called from invalid context at mm/util.c:485
[ 24.665429] in_atomic(): 1, irqs_disabled(): 1, pid: 1466, name: mount
[ 24.667323] CPU: 1 PID: 1466 Comm: mount Not tainted 5.3.0-rc1-00057-ge49f335dedc18a #1
[ 24.669343] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 24.671642] Call Trace:
[ 24.673336] dump_stack+0x5c/0x7b
[ 24.674753] ___might_sleep+0xf1/0x110
[ 24.676063] kvfree+0x61/0x80
[ 24.677374] f2fs_build_node_manager+0x5f7/0x720 [f2fs]
[ 24.678973] f2fs_fill_super+0xeb8/0x1e30 [f2fs]
[ 24.680544] ? f2fs_commit_super+0x190/0x190 [f2fs]
[ 24.682097] mount_bdev+0x173/0x1b0
[ 24.683321] legacy_get_tree+0x27/0x40
[ 24.684820] vfs_get_tree+0x25/0xf0
[ 24.686029] do_mount+0x691/0x9c0
[ 24.687412] ksys_mount+0x80/0xd0
[ 24.688667] __x64_sys_mount+0x21/0x30
[ 24.689889] do_syscall_64+0x5b/0x1f0
[ 24.691325] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 24.692824] RIP: 0033:0x7f9b2f22c48a
[ 24.694212] Code: 48 8b 0d 11 fa 2a 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d de f9 2a 00 f7 d8 64 89 01 48
[ 24.698836] RSP: 002b:00007ffd44d022b8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
[ 24.700896] RAX: ffffffffffffffda RBX: 000055d6197fd080 RCX: 00007f9b2f22c48a
[ 24.702940] RDX: 000055d6197fd260 RSI: 000055d6197fd2e0 RDI: 000055d6197fd2c0
[ 24.704928] RBP: 0000000000000000 R08: 000055d6197fd280 R09: 000000000000002f
[ 24.706866] R10: 00000000c0ed0001 R11: 0000000000000246 R12: 000055d6197fd2c0
[ 24.708988] R13: 000055d6197fd260 R14: 00007ffd44d02438 R15: 000055d6197fd2c0
[ 24.719635] F2FS-fs (vdd): Need to recover fsync data, but write access unavailable
[ 24.867539] F2FS-fs (vdd): recover fsync data on readonly fs
[ 24.873295] F2FS-fs (vdd): Mounted with checkpoint version = 23c9d0ea
[ 24.991834] F2FS-fs (vdd): recover fsync data on readonly fs
[ 24.999918] F2FS-fs (vdd): Mounted with checkpoint version = 23c9d0ea
[ 25.125365] run fstests generic/051 at 2019-07-24 16:42:56
[ 25.261578] result_service=inn:/result, RESULT_MNT=/inn/result, RESULT_ROOT=/inn/result/xfstests/4HDD-f2fs-generic-quick3/vm-snb-2G/debian-x86_64-2018-04-03.cgz/x86_64-rhel-7.6/gcc-7/e49f335dedc18a2ee0ac7791134b282abd9c857f/3
[ 25.261581]
[ 25.270070] mount.nfs: try 1 time... mount.nfs -o vers=3 inn:/result /inn/result
[ 25.270074]
[ 25.279356] run-job /lkp/jobs/scheduled/vm-snb-2G-3631a88de49b/xfstests-4HDD-f2fs-generic-quick3-debian-x86_64-2018-04-03.cgz-e49f3-20190724-6050-17o56r9-1.yaml
[ 25.279358]
[ 25.347888] F2FS-fs (vda): Found nat_bits in checkpoint
[ 25.512446] F2FS-fs (vda): Mounted with checkpoint version = 58eca5b1
[ 26.041459] F2FS-fs (vdd): Found nat_bits in checkpoint
[ 26.208391] F2FS-fs (vdd): Mounted with checkpoint version = 7f40f917
[ 26.749255] F2FS-fs (vdd): Found nat_bits in checkpoint
[ 26.907409] F2FS-fs (vdd): Mounted with checkpoint version = 2d85b69f
[ 27.917438] /usr/bin/wget -q --timeout=1800 --tries=1 --local-encoding=UTF-8 http://inn:80/~lkp/cgi-bin/lkp-jobfile-append-var?job_file=/lkp/jobs/sche... -O /dev/null
[ 27.917441]
[ 27.925520] target ucode:
[ 27.925522]
[ 27.932165] 2019-07-24 16:42:45 dmsetup remove_all
[ 27.932168]
[ 27.940098] 2019-07-24 16:42:45 wipefs -a --force /dev/vda
[ 27.940102]
[ 27.944768] 2019-07-24 16:42:45 wipefs -a --force /dev/vdb
[ 27.944771]
[ 27.949325] 2019-07-24 16:42:45 wipefs -a --force /dev/vdc
[ 27.949327]
[ 27.954154] 2019-07-24 16:42:45 wipefs -a --force /dev/vdd
[ 27.954156]
[ 27.959379] 2019-07-24 16:42:45 mkfs -t f2fs /dev/vdc
[ 27.959381]
[ 27.963492] 2019-07-24 16:42:45 mkfs -t f2fs /dev/vda
[ 27.963494]
[ 27.967545] 2019-07-24 16:42:45 mkfs -t f2fs /dev/vdd
[ 27.967547]
[ 27.971238] 2019-07-24 16:42:45 mkfs -t f2fs /dev/vdb
[ 27.971240]
[ 27.973770]
[ 27.977245] F2FS-tools: mkfs.f2fs Ver: 1.12.0 (2019-05-20)
[ 27.977247]
[ 27.979883]
[ 27.982161] Info: Disable heap-based policy
[ 27.982162]
[ 27.985739] Info: Debug level = 0
[ 27.985741]
[ 27.988669] Info: Trim is enabled
[ 27.988672]
[ 27.991478] Info: Segments per section = 1
[ 27.991480]
[ 27.994821] Info: Sections per zone = 1
[ 27.994823]
[ 27.997439] Info: sector size = 512
[ 27.997441]
[ 28.000862] Info: total sectors = 536870912 (262144 MB)
[ 28.000864]
[ 28.004563] Info: zone aligned segment0 blkaddr: 512
[ 28.004565]
[ 28.007679] Info: format version with
[ 28.007681]
[ 28.013886] "Linux version 5.3.0-rc1-00057-ge49f335dedc18a (kbuild@108c9eb08fcc) (gcc version 7.4.0 (Debian 7.4.0-10)) #1 SMP Wed Jul 24 03:51:28 CST 2019"
[ 28.013888]
[ 28.023110] Info: [/dev/vda] Discarding device
[ 28.023112]
[ 28.027786] Info: This device doesn't support BLKSECDISCARD
[ 28.027789]
[ 28.037600] Info: This device doesn't support BLKDISCARD
[ 28.037603]
[ 28.042607] Info: Overprovision ratio = 0.390%
[ 28.042609]
[ 28.046632] Info: Overprovision segments = 1027 (GC reserved = 520)
[ 28.046634]
To reproduce:
# build kernel
cd linux
cp config-5.3.0-rc1-00057-ge49f335dedc18a .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
3 years
- 1
- 0
14a92f82b2 ("v4l2-dev/ioctl: require non-zero device_caps, .."): WARNING: CPU: 0 PID: 1 at drivers/media/v4l2-core/v4l2-dev.c:863 __video_register_device
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://github.com/0day-ci/linux/commits/Hans-Verkuil/v4l2-dev-ioctl-requ...
commit 14a92f82b2770a43568b79262408cfd257c62262
Author: Hans Verkuil <hverkuil-cisco(a)xs4all.nl>
AuthorDate: Tue Jul 23 10:21:25 2019 +0200
Commit: 0day robot <lkp(a)intel.com>
CommitDate: Wed Jul 24 00:45:31 2019 +0800
v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results
Now that all V4L2 drivers set device_caps in struct video_device, we can add
a check for this to ensure all future drivers fill this in.
Also verify that when the querycap ioctl is called the driver didn't mess
with the device_caps value and that capabilities is a superset of device_caps.
Signed-off-by: Hans Verkuil <hverkuil-cisco(a)xs4all.nl>
d3bfed33e2 media: hantro: Add support for MPEG-2 decoding on RK3328
14a92f82b2 v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results
+------------------------------------------------------------------------+------------+------------+
| | d3bfed33e2 | 14a92f82b2 |
+------------------------------------------------------------------------+------------+------------+
| boot_successes | 32 | 0 |
| boot_failures | 0 | 11 |
| WARNING:at_drivers/media/v4l2-core/v4l2-dev.c:#__video_register_device | 0 | 11 |
| RIP:__video_register_device | 0 | 11 |
+------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 1.985919] vimc vimc.0: bound vimc-capture.5.auto (ops vimc_cap_comp_ops)
[ 1.987715] vimc vimc.0: bound vimc-sensor.6.auto (ops vimc_sen_comp_ops)
[ 1.988958] vimc vimc.0: bound vimc-scaler.7.auto (ops vimc_sca_comp_ops)
[ 1.990355] vimc vimc.0: bound vimc-capture.8.auto (ops vimc_cap_comp_ops)
[ 1.991799] ------------[ cut here ]------------
[ 1.992688] WARNING: CPU: 0 PID: 1 at drivers/media/v4l2-core/v4l2-dev.c:863 __video_register_device+0x62c/0x6d0
[ 1.993647] Modules linked in:
[ 1.993647] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.3.0-rc1-00031-g14a92f8 #1
[ 1.993647] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 1.993647] RIP: 0010:__video_register_device+0x62c/0x6d0
[ 1.993647] Code: c7 c0 78 ae 82 e8 e4 d7 4f 00 41 c7 86 10 06 00 00 ff ff ff ff 89 e8 e9 fa fd ff ff 48 c7 c7 f8 6a 70 82 31 c0 e8 9d d6 8c ff <0f> 0b b8 ea ff ff ff e9 e0 fd ff ff 48 c7 c6 60 03 3f 82 48 c7 c7
[ 1.993647] RSP: 0000:ffffc9000000fcb8 EFLAGS: 00010296
[ 1.993647] RAX: 0000000000000024 RBX: ffff88801c330030 RCX: 0000000000000000
[ 1.993647] RDX: 0000000000000002 RSI: ffffffff82a3b6f8 RDI: ffffffff82a3b6e0
[ 1.993647] RBP: ffffffff82af89e0 R08: 0000000000000000 R09: 0000000000000000
[ 1.993647] R10: 0000000000000010 R11: 0000000000000000 R12: ffff88801bdde8f0
[ 1.993647] R13: ffffffff82af8210 R14: ffff88801bdde8f0 R15: ffff88801c31f470
[ 1.993647] FS: 0000000000000000(0000) GS:ffff88801ec00000(0000) knlGS:0000000000000000
[ 1.993647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.993647] CR2: 0000000000000000 CR3: 0000000002a10001 CR4: 00000000001606b0
[ 1.993647] Call Trace:
[ 1.993647] ? v4l2_device_register_subdev_nodes+0xde/0x180
[ 1.993647] ? vimc_comp_bind+0xf1/0x160
[ 1.993647] ? try_to_bring_up_master+0x11a/0x170
[ 1.993647] ? __component_add+0x99/0x180
[ 1.993647] ? platform_drv_probe+0x3f/0x90
[ 1.993647] ? really_probe+0x1c0/0x2d0
[ 1.993647] ? driver_probe_device+0x4c/0xe0
[ 1.993647] ? device_driver_attach+0x4a/0x50
[ 1.993647] ? __driver_attach+0x6b/0xb0
[ 1.993647] ? device_driver_attach+0x50/0x50
[ 1.993647] ? bus_for_each_dev+0x50/0x90
[ 1.993647] ? bus_add_driver+0x14a/0x1f0
[ 1.993647] ? driver_register+0x56/0xe0
[ 1.993647] ? vimc_sca_pdrv_init+0xe/0xe
[ 1.993647] ? do_one_initcall+0x39/0x1a0
[ 1.993647] ? kernel_init_freeable+0x121/0x1a7
[ 1.993647] ? rest_init+0x140/0x140
[ 1.993647] ? kernel_init+0x5/0x100
[ 1.993647] ? ret_from_fork+0x1f/0x30
[ 1.993647] ---[ end trace 1c657743bf245402 ]---
[ 2.028924] vimc vimc.0: vimc subdev nodes registration failed (err=-22)
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start b45f49d006d9572940661923de7c8559e4a5421e 5f9e832c137075045d15cd6899ab0505cfb2ca4b --
git bisect bad 6f614e8580aac6cb99166a2d2b80ff9299c4b613 # 11:39 B 0 2 18 0 Merge 'linux-review/Chris-Wilson/drm-i915-gtt-Correct-unshifted-from-for-gen8_ppgtt_alloc-errors/20190720-055102' into devel-hourly-2019072417
git bisect bad eb69444e9e4fb48599c6b7f538cb60c843e99f4f # 12:01 B 0 2 18 0 Merge 'linux-review/Tony-Lindgren/ARM-dts-Fix-incomplete-dts-data-for-am3-and-am4-mmc/20190723-193425' into devel-hourly-2019072417
git bisect good c8be6dc793573586354268e92dbb093f2034dd5e # 12:26 G 10 0 0 0 Merge 'linux-review/Daniel-Vetter/drm-vblank-Document-and-fix-vblank-count-barrier-semantics/20190724-032504' into devel-hourly-2019072417
git bisect good b17e596e2535857ecf92edf4882b3bfc6762d55e # 12:47 G 10 0 0 0 Merge 'linux-review/Fabio-Estevam/drm-bridge-Improve-the-help-text-for-DRM_ANALOGIX_ANX78XX/20190723-095912' into devel-hourly-2019072417
git bisect bad 5d088703de354dfc4d59d3c593d2f92ba53653b5 # 13:08 B 0 1 17 0 Merge 'linux-review/Benjamin-Poirier/qlge-Move-drivers-net-ethernet-qlogic-qlge-to-drivers-staging-qlge/20190723-203935' into devel-hourly-2019072417
git bisect good ee4149b35250a0c0fc46827935a0b3b8ded50151 # 13:33 G 10 0 0 0 Merge 'linux-review/Ezequiel-Garcia/ARM-dts-am335x-cm-t335-Remove-regulator-boot-off-property/20190722-110601' into devel-hourly-2019072417
git bisect good f48186c6b4fb90ec46bc56ab18e46043adfe9159 # 13:51 G 11 0 1 1 Merge 'linux-review/Chunming-Zhou/drm-syncobj-extend-syncobj-query-ability-v2/20190724-083352' into devel-hourly-2019072417
git bisect good 87992524d9bbeb26031b5576d80a8e2606be53b0 # 14:15 G 11 0 0 0 Merge 'linux-review/Paul-Cercueil/power-supply-ingenic-battery-Don-t-change-scale-if-there-s-only-one/20190724-013356' into devel-hourly-2019072417
git bisect bad b380291286490e3590994f676d9d110dddd3b22f # 14:44 B 0 2 18 0 Merge 'linux-review/Hans-Verkuil/v4l2-dev-ioctl-require-non-zero-device_caps-verify-sane-querycap-results/20190724-004530' into devel-hourly-2019072417
git bisect bad 14a92f82b2770a43568b79262408cfd257c62262 # 15:16 B 0 6 21 0 v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results
# first bad commit: [14a92f82b2770a43568b79262408cfd257c62262] v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results
git bisect good d3bfed33e2d80ce5a25549c2f935c91775b2b3f3 # 15:36 G 30 0 1 1 media: hantro: Add support for MPEG-2 decoding on RK3328
# extra tests on HEAD of linux-devel/devel-hourly-2019072417
git bisect bad b45f49d006d9572940661923de7c8559e4a5421e # 15:36 B 0 343 361 0 0day head guard for 'devel-hourly-2019072417'
# extra tests on tree/branch linux-review/Hans-Verkuil/v4l2-dev-ioctl-require-non-zero-device_caps-verify-sane-querycap-results/20190724-004530
git bisect bad 14a92f82b2770a43568b79262408cfd257c62262 # 15:40 B 0 11 26 0 v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results
# extra tests with first bad commit reverted
git bisect good 7f73de8b32d191c6e227ed18c910474e5da75067 # 15:58 G 10 0 2 2 Revert "v4l2-dev/ioctl: require non-zero device_caps, verify sane querycap results"
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
3 years
- 1
- 0
5015a300a5 ("lib: introduce test_meminit module"): BUG: sleeping function called from invalid context at mm/page_alloc.c:4664
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
commit 5015a300a522c8fb542dc993140e4c360cf4cf5f
Author: Alexander Potapenko <glider(a)google.com>
AuthorDate: Tue Jul 16 16:27:27 2019 -0700
Commit: Linus Torvalds <torvalds(a)linux-foundation.org>
CommitDate: Tue Jul 16 19:23:22 2019 -0700
lib: introduce test_meminit module
Add tests for heap and pagealloc initialization. These can be used to
check init_on_alloc and init_on_free implementations as well as other
approaches to initialization.
Expected test output in the case the kernel provides heap initialization
(e.g. when running with either init_on_alloc=1 or init_on_free=1):
test_meminit: all 10 tests in test_pages passed
test_meminit: all 40 tests in test_kvmalloc passed
test_meminit: all 60 tests in test_kmemcache passed
test_meminit: all 10 tests in test_rcu_persistent passed
test_meminit: all 120 tests passed!
Link: http://lkml.kernel.org/r/20190529123812.43089-4-glider@google.com
Signed-off-by: Alexander Potapenko <glider(a)google.com>
Acked-by: Kees Cook <keescook(a)chromium.org>
Cc: Christoph Lameter <cl(a)linux.com>
Cc: Nick Desaulniers <ndesaulniers(a)google.com>
Cc: Kostya Serebryany <kcc(a)google.com>
Cc: Dmitry Vyukov <dvyukov(a)google.com>
Cc: Sandeep Patil <sspatil(a)android.com>
Cc: Laura Abbott <labbott(a)redhat.com>
Cc: Jann Horn <jannh(a)google.com>
Cc: Marco Elver <elver(a)google.com>
Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org>
8e060c21ae lib/test_overflow.c: avoid tainting the kernel and fix wrap size
5015a300a5 lib: introduce test_meminit module
bed38c3e2d Merge tag 'powerpc-5.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
9e6dfe8045 Add linux-next specific files for 20190724
+----------------------------------------------------------------------+------------+------------+------------+---------------+
| | 8e060c21ae | 5015a300a5 | bed38c3e2d | next-20190724 |
+----------------------------------------------------------------------+------------+------------+------------+---------------+
| boot_successes | 190 | 2 | 0 | 0 |
| boot_failures | 1 | 64 | 66 | 3 |
| kernel_BUG_at_security/keys/keyring.c | 1 | | | |
| invalid_opcode:#[##] | 1 | | | |
| RIP:__key_link_begin | 1 | | | |
| Kernel_panic-not_syncing:Fatal_exception | 1 | 56 | | |
| BUG:sleeping_function_called_from_invalid_context_at_mm/page_alloc.c | 0 | 43 | 66 | 3 |
| BUG:unable_to_handle_page_fault_for_address | 0 | 56 | | |
| Oops:#[##] | 0 | 56 | | |
| RIP:slob_page_alloc | 0 | 42 | | |
| RIP:fill_with_garbage_skip | 0 | 14 | | |
+----------------------------------------------------------------------+------------+------------+------------+---------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 8.991602] test_stackinit: packed_none FAIL (uninit bytes: 32)
[ 8.992253] test_stackinit: user FAIL (uninit bytes: 32)
[ 8.992842] test_stackinit: failures: 23
[ 9.006247] test_meminit: all 10 tests in test_pages passed
[ 9.014607] test_meminit: test_kvmalloc failed 10 out of 40 times
[ 9.017906] BUG: sleeping function called from invalid context at mm/page_alloc.c:4664
[ 9.019075] in_atomic(): 0, irqs_disabled(): 0, pid: 1, name: swapper
[ 9.019793] 1 lock held by swapper/1:
[ 9.020207] #0: (____ptrval____) (rcu_read_lock){....}, at: test_meminit_init+0x64b/0xc9f
[ 9.021156] Preemption disabled at:
[ 9.021161] [<ffffffff812b1e1a>] slob_alloc+0x62/0x2eb
[ 9.022305] CPU: 0 PID: 1 Comm: swapper Tainted: G T 5.2.0-10846-g5015a30 #1
[ 9.023218] Call Trace:
[ 9.023505] dump_stack+0x2e/0x3e
[ 9.023883] ___might_sleep+0x28c/0x2ad
[ 9.024406] __might_sleep+0xa1/0xaf
[ 9.024815] __alloc_pages_nodemask+0x138/0xf74
[ 9.025318] ? kvm_clock_read+0x2b/0x62
[ 9.025745] ? kvm_sched_clock_read+0x10/0x20
[ 9.026351] ? sched_clock_cpu+0xe1/0x11f
[ 9.026800] slob_new_pages+0x14/0x4b
[ 9.027286] slob_alloc+0x19f/0x2eb
[ 9.027746] __kmalloc+0x69/0xc4
[ 9.028110] test_meminit_init+0x6a4/0xc9f
[ 9.028567] ? test_meminit_init+0x64b/0xc9f
[ 9.029043] ? fill_with_garbage+0x43/0x43
[ 9.029501] do_one_initcall+0x9a/0x274
[ 9.029980] kernel_init_freeable+0x268/0x3e4
[ 9.030543] ? rest_init+0x1ac/0x1ac
[ 9.030962] kernel_init+0x10/0x1b1
[ 9.031369] ret_from_fork+0x35/0x40
[ 9.032240] BUG: unable to handle page fault for address: ffff88801b033ac4
[ 9.033000] #PF: supervisor read access in kernel mode
[ 9.033566] #PF: error_code(0x0000) - not-present page
[ 9.034276] PGD 5c00067 P4D 5c00067 PUD 5c01067 PMD 1f928067 PTE 800fffffe4fcc060
[ 9.034280] Oops: 0000 [#1] PREEMPT DEBUG_PAGEALLOC PTI
[ 9.034280] CPU: 0 PID: 1 Comm: swapper Tainted: G W T 5.2.0-10846-g5015a30 #1
[ 9.034280] RIP: 0010:slob_page_alloc+0x31/0x262
[ 9.034280] Code: c9 48 89 e5 41 57 41 56 41 55 49 89 cd 41 54 53 c6 01 00 4c 63 f2 48 8b 4f 20 49 d1 ea 49 89 fc 4d 89 f3 45 89 d7 31 db 31 d2 <66> 44 8b 01 66 45 85 c0 7e 09 48 ff 05 28 1f 82 03 eb 06 41 b8 01
[ 9.034280] RSP: 0000:ffff88801e843d68 EFLAGS: 00010006
[ 9.034280] RAX: ffff88801b033ac4 RBX: ffff88801b02b230 RCX: ffff88801b033ac4
[ 9.034280] RDX: ffff88801b02b230 RSI: 000000000000010c RDI: ffff88801b02b230
[ 9.034280] RBP: ffff88801e843d90 R08: 0000000000000101 R09: 0000000000000000
[ 9.034280] R10: 000000000000010c R11: 0000000000000008 R12: ffffea00006c0ac0
[ 9.034280] R13: ffff88801e843dbf R14: 0000000000000008 R15: 000000000000010c
[ 9.034280] FS: 0000000000000000(0000) GS:ffffffff8387a000(0000) knlGS:0000000000000000
[ 9.034280] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9.034280] CR2: ffff88801b033ac4 CR3: 0000000003824001 CR4: 00000000003606f0
[ 9.034280] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9.034280] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 9.034280] Call Trace:
[ 9.034280] slob_alloc+0xb2/0x2eb
[ 9.034280] kmem_cache_alloc+0x56/0xf2
[ 9.034280] test_meminit_init+0x5bf/0xc9f
[ 9.034280] ? fill_with_garbage+0x43/0x43
[ 9.034280] do_one_initcall+0x9a/0x274
[ 9.034280] kernel_init_freeable+0x268/0x3e4
[ 9.034280] ? rest_init+0x1ac/0x1ac
[ 9.034280] kernel_init+0x10/0x1b1
[ 9.034280] ret_from_fork+0x35/0x40
[ 9.034280] Modules linked in:
[ 9.034280] CR2: ffff88801b033ac4
[ 9.034280] ---[ end trace a392338ad698f6fd ]---
[ 9.034280] RIP: 0010:slob_page_alloc+0x31/0x262
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 5f9e832c137075045d15cd6899ab0505cfb2ca4b v5.2 --
git bisect good 106f1466e7e7057ec6f4dc9516c13ea8cb9dffa0 # 22:47 G 10 0 0 0 Merge tag 'kconfig-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
git bisect bad fdcec00405fae0befdd7bbcbe738b7325e5746fb # 23:05 B 0 3 18 0 Merge tag 'rproc-v5.3' of git://github.com/andersson/remoteproc
git bisect good fa6e951a2a440babd7a7310d0f4713e618061767 # 23:41 G 10 0 1 1 Merge tag 'ecryptfs-5.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tyhicks/ecryptfs
git bisect good c309b6f24222246c18a8b65d3950e6e755440865 # 00:06 G 10 0 0 0 Merge tag 'docs/v5.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media
git bisect bad fa121bb3fed6313b1f0af23952301e06cf6d32ed # 00:22 B 0 1 16 0 Merge tag 'mips_5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux
git bisect good 0a8ad0ffa4d80a544f6cbff703bf6394339afcdf # 00:48 G 10 0 1 1 Merge tag 'for-linus-5.3-ofs1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux
git bisect bad 415bfd9cdb175cf870fb173ae9d3958862de2c97 # 00:56 B 0 2 17 0 Merge tag 'for-linus-20190617' of git://git.sourceforge.jp/gitroot/uclinux-h8/linux
git bisect bad 57a8ec387e1441ea5e1232bc0749fb99a8cba7e7 # 01:13 B 0 1 16 0 Merge branch 'akpm' (patches from Andrew)
git bisect bad 6e51f8aa76b67d0a6eb168fd41a81e8478ae07a9 # 01:47 B 0 1 16 0 coda: potential buffer overflow in coda_psdev_write()
git bisect good fe6ba88b251aa76a94be2cb441d2e6b7c623b989 # 02:14 G 10 0 0 0 arch: replace _BITUL() in kernel-space headers with BIT()
git bisect bad b4658cdd8cab49c978334dc5db9070d0d881e3dd # 02:33 B 3 1 3 3 lib/string_helpers: fix some kerneldoc warnings
git bisect good 33d6e0ff68af74be0c846c8e042e84a9a1a0561e # 04:15 G 61 0 1 1 lib/test_string.c: avoid masking memset16/32/64 failures
git bisect bad 5015a300a522c8fb542dc993140e4c360cf4cf5f # 04:29 B 1 1 1 1 lib: introduce test_meminit module
git bisect good 8e060c21ae2c265a2b596e9e7f9f97ec274151a4 # 05:35 G 61 0 4 4 lib/test_overflow.c: avoid tainting the kernel and fix wrap size
# first bad commit: [5015a300a522c8fb542dc993140e4c360cf4cf5f] lib: introduce test_meminit module
git bisect good 8e060c21ae2c265a2b596e9e7f9f97ec274151a4 # 06:27 G 180 0 5 9 lib/test_overflow.c: avoid tainting the kernel and fix wrap size
# extra tests on HEAD of internal-eywa/master
git bisect bad 5a380be2937f374c3207f002f9d823a382e9e9ae # 06:27 B 0 13 31 0 Intel Next: Add release files for v5.3-rc1 2019-07-23
# extra tests on tree/branch linus/master
git bisect bad bed38c3e2dca01b358a62b5e73b46e875742fd75 # 06:40 B 0 2 17 0 Merge tag 'powerpc-5.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
# extra tests on tree/branch linux-next/master
git bisect bad 9e6dfe8045f85f9b5aade47e4192482927e2791a # 06:51 B 0 2 17 0 Add linux-next specific files for 20190724
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
3 years
- 1
- 0
a0335bd901 ("mm: assert that mmap_sem is held over .."): WARNING: CPU: 0 PID: 152 at include/linux/mmu_notifier.h:296 unmap_vmas
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://git.infradead.org/users/hch/misc.git mmu-notifier-cleanup
commit a0335bd9017dd91a934284f569c028e4a10bce45
Author: Christoph Hellwig <hch(a)lst.de>
AuthorDate: Tue Jul 23 10:23:09 2019 +0200
Commit: Christoph Hellwig <hch(a)lst.de>
CommitDate: Tue Jul 23 10:26:52 2019 +0200
mm: assert that mmap_sem is held over invalidate_range_{start,end}
The notifiers rely on the fact that mmap_sem is held, so add lockdep
annotations to assert that this is actually true.
Signed-off-by: Christoph Hellwig <hch(a)lst.de>
8bc15c4606 mm: merge struct mmu_notifier_mm into struct mm_struct
a0335bd901 mm: assert that mmap_sem is held over invalidate_range_{start,end}
fd2a4ea66b mm: remove the hmm_mirror abstraction
+-------------------------------------------------------------+------------+------------+------------+
| | 8bc15c4606 | a0335bd901 | fd2a4ea66b |
+-------------------------------------------------------------+------------+------------+------------+
| boot_successes | 21 | 0 | 0 |
| boot_failures | 5 | 11 | 11 |
| Mem-Info | 5 | 0 | 1 |
| invoked_oom-killer:gfp_mask=0x | 5 | | |
| RIP:__put_user_4 | 1 | | |
| RIP:copy_user_generic_unrolled | 1 | | |
| WARNING:at_include/linux/mmu_notifier.h:#unmap_vmas | 0 | 11 | 11 |
| RIP:unmap_vmas | 0 | 11 | 11 |
| WARNING:at_include/linux/mmu_notifier.h:#try_to_unmap_one | 0 | 10 | 9 |
| RIP:try_to_unmap_one | 0 | 10 | 9 |
| WARNING:at_include/linux/mmu_notifier.h:#__oom_reap_task_mm | 0 | 0 | 1 |
| RIP:__oom_reap_task_mm | 0 | 0 | 1 |
+-------------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 2.963118] Unpacking initramfs...
[ 8.236666] Freeing initrd memory: 29884K
[ 8.241268] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x29b38900952, max_idle_ns: 440795323692 ns
[ 8.243454] mce: Machine check injector initialized
[ 8.256012] ------------[ cut here ]------------
[ 8.256935] WARNING: CPU: 0 PID: 152 at include/linux/mmu_notifier.h:296 unmap_vmas+0x265/0x360
[ 8.258699] Modules linked in:
[ 8.258728] CPU: 0 PID: 152 Comm: modprobe Not tainted 5.3.0-rc1-00010-ga0335bd9017dd9 #1
[ 8.258728] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 8.258728] RIP: 0010:unmap_vmas+0x265/0x360
[ 8.258728] Code: d1 d2 e8 ff 48 c7 c7 f8 bc 72 83 31 c0 48 83 05 80 1c ba 03 01 48 83 05 80 1c ba 03 01 e8 3f 14 da ff 48 83 05 7b 1c ba 03 01 <0f> 0b 48 83 05 79 1c ba 03 01 be 01 00 00 00 e9 15 fe ff ff 48 8b
[ 8.258728] RSP: 0018:ffff88800fe4fcc8 EFLAGS: 00010202
[ 8.258728] RAX: 0000000000000024 RBX: ffff88801180c438 RCX: ffffffff8133155b
[ 8.258728] RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffffffff8407efa0
[ 8.258728] RBP: ffff88800fe4fd18 R08: fffffbfff0808dce R09: fffffbfff0808dce
[ 8.258728] R10: 000000000193b077 R11: fffffbfff0808dcd R12: ffffffffffffffff
[ 8.258728] R13: 0000000000000000 R14: ffff88800fe4fd28 R15: ffff888000309480
[ 8.258728] FS: 0000000000000000(0000) GS:ffffffff83a8b000(0000) knlGS:0000000000000000
[ 8.258728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8.258728] CR2: 00007f48799e5000 CR3: 00000000139a5000 CR4: 00000000000006f0
[ 8.258728] Call Trace:
[ 8.258728] exit_mmap+0x17e/0x400
[ 8.258728] ? mutex_unlock+0x22/0x30
[ 8.258728] ? uprobe_clear_state+0x63/0x1e0
[ 8.258728] mmput+0xeb/0x2a0
[ 8.258728] do_exit+0x8d8/0x1760
[ 8.258728] do_group_exit+0xa4/0x1f0
[ 8.258728] __x64_sys_exit_group+0x34/0x40
[ 8.258728] do_syscall_64+0x105/0xe60
[ 8.258728] ? trace_hardirqs_off_thunk+0x1a/0x20
[ 8.258728] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 8.258728] RIP: 0033:0x7f48794c7408
[ 8.258728] Code: 00 00 be 3c 00 00 00 eb 19 66 0f 1f 84 00 00 00 00 00 48 89 d7 89 f0 0f 05 48 3d 00 f0 ff ff 77 19 f4 48 89 d7 44 89 c0 0f 05 <48> 3d 00 f0 ff ff 76 e0 f7 d8 64 41 89 01 eb d8 f7 d8 64 41 89 01
[ 8.258728] RSP: 002b:00007ffd9ef81278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[ 8.258728] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f48794c7408
[ 8.258728] RDX: 0000000000000001 RSI: 000000000000003c RDI: 0000000000000001
[ 8.258728] RBP: 00007f48797bb820 R08: 00000000000000e7 R09: ffffffffffffffa0
[ 8.258728] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f48797bb820
[ 8.258728] R13: 0000000000000001 R14: 00007ffd9ef81faa R15: 00007ffd9ef81450
[ 8.258728] irq event stamp: 2378
[ 8.258728] hardirqs last enabled at (2377): [<ffffffff81241657>] console_unlock+0x757/0xd00
[ 8.258728] hardirqs last disabled at (2378): [<ffffffff8100490a>] trace_hardirqs_off_thunk+0x1a/0x20
[ 8.258728] softirqs last enabled at (2300): [<ffffffff827d24c3>] unix_sock_destructor+0x1c3/0x330
[ 8.258728] softirqs last disabled at (2298): [<ffffffff827d24c3>] unix_sock_destructor+0x1c3/0x330
[ 8.258728] ---[ end trace f5979ded5fa7d06f ]---
[ 8.317833] ------------[ cut here ]------------
[ 8.317833] ------------[ cut here ]------------
[ 8.319590] WARNING: CPU: 0 PID: 152 at include/linux/mmu_notifier.h:317 unmap_vmas+0x2e0/0x360
[ 8.322155] Modules linked in:
[ 8.322939] CPU: 0 PID: 152 Comm: modprobe Tainted: G W 5.3.0-rc1-00010-ga0335bd9017dd9 #1
[ 8.322981] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 8.322981] RIP: 0010:unmap_vmas+0x2e0/0x360
[ 8.322981] Code: 56 d2 e8 ff 48 c7 c7 f8 bc 72 83 31 c0 48 83 05 75 1c ba 03 01 48 83 05 75 1c ba 03 01 e8 c4 13 da ff 48 83 05 70 1c ba 03 01 <0f> 0b 48 83 05 6e 1c ba 03 01 be 01 00 00 00 e9 a0 fe ff ff 48 83
[ 8.322981] RSP: 0018:ffff88800fe4fcc8 EFLAGS: 00010202
[ 8.322981] RAX: 0000000000000024 RBX: 0000000000000000 RCX: ffffffff8133155b
[ 8.322981] RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffffffff8407efa0
[ 8.322981] RBP: ffff88800fe4fd18 R08: fffffbfff0808dce R09: fffffbfff0808dce
[ 8.322981] R10: 000000000193fde1 R11: fffffbfff0808dcd R12: ffffffffffffffff
[ 8.322981] R13: 0000000000000000 R14: ffff88800fe4fd28 R15: ffff888000309400
[ 8.322981] FS: 00007f48799e2700(0000) GS:ffffffff83a8b000(0000) knlGS:0000000000000000
[ 8.322981] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8.322981] CR2: 00007f48799e5000 CR3: 00000000139a5000 CR4: 00000000000006f0
[ 8.322981] Call Trace:
[ 8.322981] exit_mmap+0x17e/0x400
[ 8.322981] mmput+0xeb/0x2a0
[ 8.322981] do_exit+0x8d8/0x1760
[ 8.322981] do_group_exit+0xa4/0x1f0
[ 8.322981] __x64_sys_exit_group+0x34/0x40
[ 8.322981] do_syscall_64+0x105/0xe60
[ 8.322981] ? trace_hardirqs_off_thunk+0x1a/0x20
[ 8.322981] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 8.322981] RIP: 0033:0x7f48794c7408
[ 8.322981] Code: Bad RIP value.
[ 8.322981] RSP: 002b:00007ffd9ef81278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[ 8.322981] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f48794c7408
[ 8.322981] RDX: 0000000000000001 RSI: 000000000000003c RDI: 0000000000000001
[ 8.322981] RBP: 00007f48797bb820 R08: 00000000000000e7 R09: ffffffffffffffa0
[ 8.322981] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f48797bb820
[ 8.322981] R13: 0000000000000001 R14: 00007ffd9ef81faa R15: 00007ffd9ef81450
[ 8.322981] irq event stamp: 2530
[ 8.322981] hardirqs last enabled at (2529): [<ffffffff82a25bae>] _raw_spin_unlock_irq+0x4e/0x140
[ 8.322981] hardirqs last disabled at (2530): [<ffffffff8100490a>] trace_hardirqs_off_thunk+0x1a/0x20
[ 8.322981] softirqs last enabled at (2524): [<ffffffff82e00969>] __do_softirq+0x969/0xa91
[ 8.322981] softirqs last disabled at (2465): [<ffffffff81193c8b>] irq_exit+0x13b/0x210
[ 8.322981] ---[ end trace f5979ded5fa7d070 ]---
[ 8.380468] camellia-x86_64: performance on this CPU would be suboptimal: disabling camellia-x86_64.
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start e5a938a3c4b7e70489d04ab8a251f4a1203dd89a 5f9e832c137075045d15cd6899ab0505cfb2ca4b --
git bisect bad 50db48bb68d3056f3fbf5748e8e6f1e2f8e01a3f # 06:43 B 0 2 18 0 Merge 'linux-review/Hariprasad-Kelam/staging-rtl8723bs-core-Remove-Unneeded-variable-ret/20190718-091135' into devel-hourly-2019072317
git bisect good 7f335e9d1db882f26e68f3627a3588383ac6fe1c # 13:17 G 10 0 2 2 Merge 'linux-review/Waiman-Long/mm-slab-Move-memcg_cache_params-structure-to-mm-slab-h/20190721-155027' into devel-hourly-2019072317
git bisect bad 4d6bd7eb292ebf5a2073ae91f976b6f58e0164ff # 14:06 B 0 4 20 0 Merge 'linux-review/Gwendal-Grignou/iio-cros_ec-Remove-replacing-error-code-with-EIO/20190720-015313' into devel-hourly-2019072317
git bisect good 802a35c35c086201f4621cef3eaffff4aabd4acf # 14:18 G 10 0 3 3 Merge 'xiang-linux/erofs-pollingmode' into devel-hourly-2019072317
git bisect good 5b2678045193393791cb3b0111bd452a24b359ef # 14:46 G 10 0 5 5 Merge 'linux-review/Konstantin-Khlebnikov/cgroup-writeback-use-online-cgroup-when-switching-from-dying-bdi_writebacks/20190720-123648' into devel-hourly-2019072317
git bisect bad f413a08ff11a7ab2988299664cbd3e92e6c8e711 # 14:58 B 0 2 17 0 Merge 'linux-review/Arnaldo-Carvalho-de-Melo/libbpf-Fix-endianness-macro-usage-for-some-compilers/20190720-091903' into devel-hourly-2019072317
git bisect bad ae5bd092b0fb846a35eb8e44532ceb6548a2db99 # 15:27 B 0 1 17 0 Merge 'linux-review/Jan-Pieter-van-Woerkom/Re-MyGica-T230-dvb-t2-data-corruption-since-commit-5fa8815/20190720-111430' into devel-hourly-2019072317
git bisect good 720d8ece19321700dfe8e509bc500ddf0020f848 # 19:15 G 10 0 2 2 Merge 'linux-review/Rafael-J-Wysocki/cpuidle-teo-Allow-tick-to-be-stopped-if-PM-QoS-is-used/20190720-114003' into devel-hourly-2019072317
git bisect bad 8239629a94ded66b25aea52026afeb4292d40445 # 20:32 B 0 2 18 0 Merge 'hch-misc/mmu-notifier-cleanup' into devel-hourly-2019072317
git bisect bad a8f1cb5758b9fe0acf4f59da6bd9775a15e451b4 # 21:07 B 0 1 17 0 nouveau: pass struct nouveau_svmm to nouveau_range_fault
git bisect good 8bc15c4606e324e2b2ad73eb15a8ec017fd59dc7 # 21:40 G 10 0 7 7 mm: merge struct mmu_notifier_mm into struct mm_struct
git bisect bad bad02b300045dad0961b225e6079eb9568f9e077 # 22:03 B 0 2 18 0 amdkfd: reorder create_process
git bisect bad a0335bd9017dd91a934284f569c028e4a10bce45 # 23:06 B 0 2 18 0 mm: assert that mmap_sem is held over invalidate_range_{start,end}
# first bad commit: [a0335bd9017dd91a934284f569c028e4a10bce45] mm: assert that mmap_sem is held over invalidate_range_{start,end}
git bisect good 8bc15c4606e324e2b2ad73eb15a8ec017fd59dc7 # 23:17 G 31 0 10 17 mm: merge struct mmu_notifier_mm into struct mm_struct
# extra tests on HEAD of linux-devel/devel-hourly-2019072317
git bisect bad e5a938a3c4b7e70489d04ab8a251f4a1203dd89a # 23:17 B 0 343 361 0 0day head guard for 'devel-hourly-2019072317'
# extra tests on tree/branch hch-misc/mmu-notifier-cleanup
git bisect bad fd2a4ea66b2e93fbe75a8c51a686bf44be3a7f3c # 23:38 B 0 1 16 0 mm: remove the hmm_mirror abstraction
# extra tests with first bad commit reverted
git bisect good 6486db285e2389fc163687a8c9fe7bdf2b4b4dcf # 00:03 G 11 0 4 4 Revert "mm: assert that mmap_sem is held over invalidate_range_{start,end}"
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
3 years
- 1
- 0
a0d14b8909 ("x86/mm, tracing: Fix CR2 corruption"): Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
commit a0d14b8909de55139b8702fe0c7e80b69763dcfb
Author: Peter Zijlstra <peterz(a)infradead.org>
AuthorDate: Thu Jul 11 13:40:59 2019 +0200
Commit: Thomas Gleixner <tglx(a)linutronix.de>
CommitDate: Wed Jul 17 23:17:38 2019 +0200
x86/mm, tracing: Fix CR2 corruption
Despite the current efforts to read CR2 before tracing happens there still
exist a number of possible holes:
idtentry page_fault do_page_fault has_error_code=1
call error_entry
TRACE_IRQS_OFF
call trace_hardirqs_off*
#PF // modifies CR2
CALL_enter_from_user_mode
__context_tracking_exit()
trace_user_exit(0)
#PF // modifies CR2
call do_page_fault
address = read_cr2(); /* whoopsie */
And similar for i386.
Fix it by pulling the CR2 read into the entry code, before any of that
stuff gets a chance to run and ruin things.
Reported-by: He Zhe <zhe.he(a)windriver.com>
Reported-by: Eiichi Tsukata <devel(a)etsukata.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz(a)infradead.org>
Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de>
Reviewed-by: Andy Lutomirski <luto(a)kernel.org>
Cc: bp(a)alien8.de
Cc: rostedt(a)goodmis.org
Cc: torvalds(a)linux-foundation.org
Cc: hpa(a)zytor.com
Cc: dave.hansen(a)linux.intel.com
Cc: jgross(a)suse.com
Cc: joel(a)joelfernandes.org
Link: https://lkml.kernel.org/r/20190711114336.116812491@infradead.org
Debugged-by: Steven Rostedt <rostedt(a)goodmis.org>
4234653e88 x86/entry/64: Update comments and sanity tests for create_gap
a0d14b8909 x86/mm, tracing: Fix CR2 corruption
ad5e427e0f Merge branch 'parisc-5.3-3' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
9e6dfe8045 Add linux-next specific files for 20190724
+------------------------------------------------+------------+------------+------------+---------------+
| | 4234653e88 | a0d14b8909 | ad5e427e0f | next-20190724 |
+------------------------------------------------+------------+------------+------------+---------------+
| boot_successes | 31 | 0 | 0 | 0 |
| boot_failures | 0 | 11 | 6 | 8 |
| Kernel_panic-not_syncing:No_working_init_found | 0 | 11 | 6 | 8 |
+------------------------------------------------+------------+------------+------------+---------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 15.632639] Starting init: /sbin/init exists but couldn't execute it (error -14)
[ 15.634040] Run /etc/init as init process
[ 15.634890] Run /bin/init as init process
[ 15.635402] Run /bin/sh as init process
[ 15.636008] Starting init: /bin/sh exists but couldn't execute it (error -14)
[ 15.636789] Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
[ 15.638408] CPU: 0 PID: 1 Comm: init Tainted: G T 5.2.0-08454-ga0d14b8 #1
[ 15.639263] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 15.640145] Call Trace:
[ 15.640427] ? dump_stack+0x8f/0xd5
[ 15.640812] ? rest_init+0x190/0x250
[ 15.641206] ? panic+0xd7/0x35b
[ 15.641624] ? rest_init+0x250/0x250
[ 15.642070] ? kernel_init+0x17b/0x190
[ 15.642479] ? ret_from_fork+0x33/0x40
[ 15.643608] Kernel Offset: disabled
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 5f9e832c137075045d15cd6899ab0505cfb2ca4b 22051d9c4a57d3b4a8b5a7407efc80c71c7bfb16 --
git bisect good a84d2d2906f983fb80f5dcc3e8e7c3ad70aa9f0d # 08:14 G 10 0 0 0 Merge tag 'csky-for-linus-5.3-rc1' of git://github.com/c-sky/csky-linux
git bisect good 70e6e1b971e46f5c1c2d72217ba62401a2edc22b # 13:28 G 10 0 0 0 Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad f1a3b43cc1f50c6ee5ba582f2025db3dea891208 # 13:54 B 0 2 18 0 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input
git bisect good 46f5c0cc3af0ecb76224a91d2997d74e35ff7821 # 14:27 G 10 0 1 1 Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad c6dd78fcb8eefa15dd861889e0f59d301cb5230c # 14:48 B 0 3 19 0 Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 55aedddb6149ab71bec9f050846855113977b033 # 15:10 G 11 0 0 0 x86/paravirt: Make read_cr2() CALLEE_SAVE
git bisect bad cd6697b8b8751b65abd7859af55cf06f36b8e716 # 15:29 B 0 2 18 0 x86/boot/efi: Remove unused variables
git bisect good 2fd37912cfb019228bf246215938e6f7619516a2 # 15:57 G 11 0 0 0 x86/entry/64: Simplify idtentry a little
git bisect bad a0d14b8909de55139b8702fe0c7e80b69763dcfb # 16:06 B 0 1 16 0 x86/mm, tracing: Fix CR2 corruption
git bisect good 4234653e882740cbf6625eeee294e388b3176583 # 16:36 G 11 0 0 0 x86/entry/64: Update comments and sanity tests for create_gap
# first bad commit: [a0d14b8909de55139b8702fe0c7e80b69763dcfb] x86/mm, tracing: Fix CR2 corruption
git bisect good 4234653e882740cbf6625eeee294e388b3176583 # 16:43 G 30 0 0 0 x86/entry/64: Update comments and sanity tests for create_gap
# extra tests on HEAD of linus/master
git bisect bad 7b5cf701ea9c395c792e2a7e3b7caf4c68b87721 # 17:06 B 0 1 17 0 Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# extra tests on tree/branch linus/master
git bisect bad ad5e427e0f6b702e52c11d1f7b2b7be3bac7de82 # 20:09 B 0 7 23 0 Merge branch 'parisc-5.3-3' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
# extra tests on tree/branch linux-next/master
git bisect bad 9e6dfe8045f85f9b5aade47e4192482927e2791a # 20:30 B 0 6 23 0 Add linux-next specific files for 20190724
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
3 years
- 2
- 1
7a2452a9b0 ("mm: assert that mmap_sem is held over .."): WARNING: CPU: 1 PID: 150 at include/linux/mmu_notifier.h:296 unmap_vmas
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://git.infradead.org/users/hch/misc.git mmu-notifier-cleanup
commit 7a2452a9b098b402be81bef0b49953f3376e35dc
Author: Christoph Hellwig <hch(a)lst.de>
AuthorDate: Tue Jul 23 10:23:09 2019 +0200
Commit: Christoph Hellwig <hch(a)lst.de>
CommitDate: Tue Jul 23 12:34:53 2019 +0200
mm: assert that mmap_sem is held over invalidate_range_{start,end}
The notifiers rely on the fact that mmap_sem is held, so add lockdep
annotations to assert that this is actually true.
Signed-off-by: Christoph Hellwig <hch(a)lst.de>
58f936694c mm: merge struct mmu_notifier_mm into struct mm_struct
7a2452a9b0 mm: assert that mmap_sem is held over invalidate_range_{start,end}
fd2a4ea66b mm: remove the hmm_mirror abstraction
+-----------------------------------------------------------------+------------+------------+------------+
| | 58f936694c | 7a2452a9b0 | fd2a4ea66b |
+-----------------------------------------------------------------+------------+------------+------------+
| boot_successes | 31 | 0 | 0 |
| boot_failures | 4 | 13 | 13 |
| BUG:soft_lockup-CPU##stuck_for#s | 2 | 1 | |
| RIP:__asan_load1 | 1 | | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 2 | 1 | |
| RIP:__asan_load4 | 1 | | |
| invoked_oom-killer:gfp_mask=0x | 2 | | |
| Mem-Info | 2 | | |
| WARNING:at_include/linux/mmu_notifier.h:#unmap_vmas | 0 | 12 | 13 |
| RIP:unmap_vmas | 0 | 12 | 13 |
| WARNING:at_include/linux/mmu_notifier.h:#__unmap_hugepage_range | 0 | 12 | 13 |
| RIP:__unmap_hugepage_range | 0 | 12 | 13 |
| WARNING:at_include/linux/mmu_notifier.h:#try_to_unmap_one | 0 | 6 | 6 |
| RIP:try_to_unmap_one | 0 | 6 | 6 |
| RIP:__asan_load8 | 0 | 1 | |
+-----------------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 23.120145] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 23.121141] Run /init as init process
[ 23.141368] random: init: uninitialized urandom read (12 bytes read)
[ 23.149842] init: Failed to spawn kernel-tests main process: unable to execute: No such file or directory
[ 23.156208] ------------[ cut here ]------------
[ 23.157217] WARNING: CPU: 1 PID: 150 at include/linux/mmu_notifier.h:296 unmap_vmas+0x18d/0x260
[ 23.159194] Modules linked in:
[ 23.159696] CPU: 1 PID: 150 Comm: init Not tainted 5.3.0-rc1-00010-g7a2452a #1
[ 23.160848] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 23.162148] RIP: 0010:unmap_vmas+0x18d/0x260
[ 23.162850] Code: 00 37 02 01 85 c0 0f 85 94 00 00 00 48 c7 c7 10 df c7 af 31 c0 48 83 05 10 00 37 02 01 e8 72 fc ec ff 48 83 05 0b 00 37 02 01 <0f> 0b 48 83 05 09 00 37 02 01 4c 8b 7d b8 e9 ca fe ff ff 48 8b 45
[ 23.165765] RSP: 0018:ffff88801eebfcf8 EFLAGS: 00010202
[ 23.166739] RAX: 0000000000000024 RBX: ffff888012cd8300 RCX: ffffffffae5af16b
[ 23.168127] RDX: 1ffffffff5fdb464 RSI: dffffc0000000000 RDI: 0000000000000246
[ 23.169289] RBP: ffff88801eebfd48 R08: fffffbfff5fdb465 R09: fffffbfff5fdb465
[ 23.170413] R10: 0000000002c55508 R11: fffffbfff5fdb464 R12: ffffffffffffffff
[ 23.171592] R13: 0000000000000000 R14: ffff88801eebfd58 R15: ffff888000158600
[ 23.173001] FS: 0000000000000000(0000) GS:ffff888018f00000(0000) knlGS:0000000000000000
[ 23.174304] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 23.175264] CR2: 00007f4dae70b1b8 CR3: 00000000125e0000 CR4: 00000000000006e0
[ 23.176403] Call Trace:
[ 23.176807] exit_mmap+0x100/0x2f0
[ 23.177342] ? __might_sleep+0x7b/0x120
[ 23.177926] mmput+0xea/0x2e0
[ 23.178393] do_exit+0x6f9/0x1bd0
[ 23.178954] ? find_held_lock+0xd9/0x100
[ 23.179594] ? __context_tracking_exit+0x9c/0xa0
[ 23.180319] do_group_exit+0x95/0x1c0
[ 23.180878] __x64_sys_exit_group+0x2f/0x30
[ 23.181488] do_syscall_64+0xc6/0x670
[ 23.182030] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 23.182781] RIP: 0033:0x7f4dae205408
[ 23.183448] Code: 00 00 be 3c 00 00 00 eb 19 66 0f 1f 84 00 00 00 00 00 48 89 d7 89 f0 0f 05 48 3d 00 f0 ff ff 77 19 f4 48 89 d7 44 89 c0 0f 05 <48> 3d 00 f0 ff ff 76 e0 f7 d8 64 41 89 01 eb d8 f7 d8 64 41 89 01
[ 23.186213] RSP: 002b:00007fff62896c58 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[ 23.187467] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007f4dae205408
[ 23.188614] RDX: 00000000000000ff RSI: 000000000000003c RDI: 00000000000000ff
[ 23.189794] RBP: 00007f4dae4f9820 R08: 00000000000000e7 R09: ffffffffffffffa0
[ 23.190869] R10: 00007f4dae500fa8 R11: 0000000000000246 R12: 00007f4dae4f9820
[ 23.192035] R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
[ 23.193197] ---[ end trace 71cd3a11be9963a5 ]---
[ 23.200209] ------------[ cut here ]------------
[ 23.200209] ------------[ cut here ]------------
[ 23.201412] WARNING: CPU: 1 PID: 151 at include/linux/mmu_notifier.h:296 unmap_vmas+0x18d/0x260
[ 23.203921] Modules linked in:
[ 23.204675] CPU: 1 PID: 151 Comm: init Tainted: G W 5.3.0-rc1-00010-g7a2452a #1
[ 23.206691] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 23.208637] RIP: 0010:unmap_vmas+0x18d/0x260
[ 23.209655] Code: 00 37 02 01 85 c0 0f 85 94 00 00 00 48 c7 c7 10 df c7 af 31 c0 48 83 05 10 00 37 02 01 e8 72 fc ec ff 48 83 05 0b 00 37 02 01 <0f> 0b 48 83 05 09 00 37 02 01 4c 8b 7d b8 e9 ca fe ff ff 48 8b 45
[ 23.214736] RSP: 0018:ffff88801ef07b50 EFLAGS: 00010202
[ 23.216293] RAX: 0000000000000024 RBX: ffff888012daae00 RCX: ffffffffae5af16b
[ 23.218383] RDX: 1ffffffff5fdb464 RSI: dffffc0000000000 RDI: 0000000000000246
[ 23.220467] RBP: ffff88801ef07ba0 R08: fffffbfff5fdb465 R09: fffffbfff5fdb465
[ 23.222577] R10: 0000000002c66395 R11: fffffbfff5fdb464 R12: ffffffffffffffff
[ 23.224681] R13: 0000000000000000 R14: ffff88801ef07bb0 R15: ffff888000158040
[ 23.226787] FS: 0000000000000000(0000) GS:ffff888018f00000(0000) knlGS:0000000000000000
[ 23.229162] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 23.230862] CR2: 00005565193f9de9 CR3: 000000001ef5a000 CR4: 00000000000006e0
[ 23.232986] Call Trace:
[ 23.233761] exit_mmap+0x100/0x2f0
[ 23.234805] ? __might_sleep+0x7b/0x120
[ 23.235986] mmput+0xea/0x2e0
[ 23.236897] flush_old_exec+0x375/0x580
[ 23.238068] load_elf_binary+0x720/0x1ac1
[ 23.239294] ? match_held_lock+0x24/0x360
[ 23.240585] ? find_held_lock+0xd9/0x100
[ 23.241823] search_binary_handler+0xf8/0x3a0
[ 23.243147] __do_execve_file+0xcf3/0x10c0
[ 23.244594] do_execve+0x28/0x40
[ 23.245586] __x64_sys_execve+0x56/0x70
[ 23.246707] do_syscall_64+0xc6/0x670
[ 23.247793] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 23.248936] RIP: 0033:0x7f4dae205427
[ 23.249759] Code: Bad RIP value.
[ 23.250504] RSP: 002b:00007fff62896be8 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[ 23.252310] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4dae205427
[ 23.254394] RDX: 000055651b2a7a30 RSI: 000055651b2a86b0 RDI: 00007fff62896c28
[ 23.256886] RBP: 00007fff62896cd0 R08: 0000000000000009 R09: 0000000000000000
[ 23.258568] R10: 0000000000000008 R11: 0000000000000246 R12: 000055651b2a7a30
[ 23.259851] R13: 000055651b2a880d R14: 000055651b2a8811 R15: 00007fff62896c28
[ 23.260977] ---[ end trace 71cd3a11be9963a6 ]---
[ 23.262868] ------------[ cut here ]------------
[ 23.262868] ------------[ cut here ]------------
[ 23.263984] WARNING: CPU: 1 PID: 150 at include/linux/mmu_notifier.h:317 unmap_vmas+0x1e7/0x260
[ 23.265630] Modules linked in:
[ 23.266132] CPU: 1 PID: 150 Comm: init Tainted: G W 5.3.0-rc1-00010-g7a2452a #1
[ 23.267521] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 23.268816] RIP: 0010:unmap_vmas+0x1e7/0x260
[ 23.269504] Code: 48 83 05 1b 00 37 02 01 85 c0 75 2d 48 c7 c7 10 df c7 af 31 c0 48 83 05 16 00 37 02 01 e8 18 fc ec ff 48 83 05 11 00 37 02 01 <0f> 0b 48 83 05 0f 00 37 02 01 e9 19 ff ff ff 48 83 05 ea ff 36 02
[ 23.272522] RSP: 0018:ffff88801eebfcf8 EFLAGS: 00010202
[ 23.273345] RAX: 0000000000000024 RBX: 0000000000000000 RCX: ffffffffae5af16b
[ 23.274463] RDX: 1ffffffff5fdb464 RSI: dffffc0000000000 RDI: 0000000000000246
[ 23.275583] RBP: ffff88801eebfd48 R08: fffffbfff5fdb465 R09: fffffbfff5fdb465
[ 23.276561] R10: 0000000002c6abf8 R11: fffffbfff5fdb464 R12: ffffffffffffffff
[ 23.277543] R13: 0000000000000000 R14: ffff88801eebfd58 R15: ffff888000158600
[ 23.278525] FS: 00007f4daef8c700(0000) GS:ffff888018f00000(0000) knlGS:0000000000000000
[ 23.279752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 23.280547] CR2: 00007f4dae2053fd CR3: 000000001ef5a000 CR4: 00000000000006e0
[ 23.281553] Call Trace:
[ 23.281922] exit_mmap+0x100/0x2f0
[ 23.282420] mmput+0xea/0x2e0
[ 23.282849] do_exit+0x6f9/0x1bd0
[ 23.283426] ? find_held_lock+0xd9/0x100
[ 23.283983] ? __context_tracking_exit+0x9c/0xa0
[ 23.284630] do_group_exit+0x95/0x1c0
[ 23.285151] __x64_sys_exit_group+0x2f/0x30
[ 23.285740] do_syscall_64+0xc6/0x670
[ 23.286264] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 23.287063] RIP: 0033:0x7f4dae205408
[ 23.287888] Code: Bad RIP value.
[ 23.288632] RSP: 002b:00007fff62896c58 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[ 23.290305] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007f4dae205408
[ 23.291968] RDX: 00000000000000ff RSI: 000000000000003c RDI: 00000000000000ff
[ 23.293643] RBP: 00007f4dae4f9820 R08: 00000000000000e7 R09: ffffffffffffffa0
[ 23.295355] R10: 00007f4dae500fa8 R11: 0000000000000246 R12: 00007f4dae4f9820
[ 23.297018] R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000000
[ 23.298701] ---[ end trace 71cd3a11be9963a7 ]---
[ 23.301781] ------------[ cut here ]------------
[ 23.301781] ------------[ cut here ]------------
[ 23.303298] WARNING: CPU: 1 PID: 151 at include/linux/mmu_notifier.h:317 unmap_vmas+0x1e7/0x260
[ 23.306417] Modules linked in:
[ 23.307416] CPU: 1 PID: 151 Comm: init Tainted: G W 5.3.0-rc1-00010-g7a2452a #1
[ 23.309931] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 23.312604] RIP: 0010:unmap_vmas+0x1e7/0x260
[ 23.314069] Code: 48 83 05 1b 00 37 02 01 85 c0 75 2d 48 c7 c7 10 df c7 af 31 c0 48 83 05 16 00 37 02 01 e8 18 fc ec ff 48 83 05 11 00 37 02 01 <0f> 0b 48 83 05 0f 00 37 02 01 e9 19 ff ff ff 48 83 05 ea ff 36 02
[ 23.320445] RSP: 0018:ffff88801ef07b50 EFLAGS: 00010202
[ 23.322254] RAX: 0000000000000024 RBX: 0000000000000000 RCX: ffffffffae5af16b
[ 23.324586] RDX: 1ffffffff5fdb464 RSI: dffffc0000000000 RDI: 0000000000000246
[ 23.327152] RBP: ffff88801ef07ba0 R08: fffffbfff5fdb465 R09: fffffbfff5fdb465
[ 23.329643] R10: 0000000002c70424 R11: fffffbfff5fdb464 R12: ffffffffffffffff
[ 23.332287] R13: 0000000000000000 R14: ffff88801ef07bb0 R15: ffff888000158040
[ 23.334851] FS: 00007f4daef8c700(0000) GS:ffff888018f00000(0000) knlGS:0000000000000000
[ 23.337830] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 23.339949] CR2: 00007f4dae2053de CR3: 000000001ef5a000 CR4: 00000000000006e0
[ 23.342667] Call Trace:
[ 23.343651] exit_mmap+0x100/0x2f0
[ 23.344960] mmput+0xea/0x2e0
[ 23.346099] flush_old_exec+0x375/0x580
[ 23.347560] load_elf_binary+0x720/0x1ac1
[ 23.348221] ? match_held_lock+0x24/0x360
[ 23.348865] ? find_held_lock+0xd9/0x100
[ 23.349511] search_binary_handler+0xf8/0x3a0
[ 23.350218] __do_execve_file+0xcf3/0x10c0
[ 23.351005] do_execve+0x28/0x40
[ 23.351533] __x64_sys_execve+0x56/0x70
[ 23.352191] do_syscall_64+0xc6/0x670
[ 23.352792] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 23.353595] RIP: 0033:0x7f4dae205427
[ 23.354176] Code: Bad RIP value.
[ 23.354754] RSP: 002b:00007fff62896be8 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[ 23.356467] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4dae205427
[ 23.358049] RDX: 000055651b2a7a30 RSI: 000055651b2a86b0 RDI: 00007fff62896c28
[ 23.359466] RBP: 00007fff62896cd0 R08: 0000000000000009 R09: 0000000000000000
[ 23.360576] R10: 0000000000000008 R11: 0000000000000246 R12: 000055651b2a7a30
[ 23.361683] R13: 000055651b2a880d R14: 000055651b2a8811 R15: 00007fff62896c28
[ 23.362833] ---[ end trace 71cd3a11be9963a8 ]---
[ 23.364976] init (150) used greatest stack depth: 29472 bytes left
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 9753d961df5e44d13629e2ab95a25c0b9e412040 5f9e832c137075045d15cd6899ab0505cfb2ca4b --
git bisect bad 797829a5e97cb5bc7298aeb5ac203f3472b662ed # 12:19 B 2 1 2 2 Merge 'linux-review/Wei-Yongjun/RDMA-siw-fix-error-return-code-in-siw_init_module/20190718-205449' into devel-hourly-2019072323
git bisect good ff446f0a684d90c251a580d9dc354388fcca4500 # 13:39 G 13 0 6 6 Merge 'linux-review/Eric-Auger/dma-mapping-Protect-dma_addressing_limited-against-NULL-dma_mask/20190723-060727' into devel-hourly-2019072323
git bisect good a322704b127411fc05b51d0d4adbb7793bb3af36 # 14:01 G 12 0 3 3 Merge 'linux-review/Maxime-Chevallier/net-mvpp2-Don-t-check-for-3-consecutive-Idle-frames-for-10G-links/20190720-082727' into devel-hourly-2019072323
git bisect bad f10cb51d7c82c62dca224c646872d0d708108766 # 14:20 B 1 2 1 1 Merge 'linux-review/Peter-Zijlstra/stacktrace-Force-USER_DS-for-stack_trace_save_user/20190719-150828' into devel-hourly-2019072323
git bisect bad 1dc21bdc4480ef140cf80bd5523612990b67dcab # 14:48 B 0 2 18 0 Merge 'linux-review/UPDATE-20190721-044728/Yoshiaki-Okamoto/USB-serial-option-Add-support-for-ZTE-MF871A/20190712-053808' into devel-hourly-2019072323
git bisect bad 12b2224057cf9bbc6138de5165f5d8da5a8e9755 # 15:09 B 0 3 19 0 Merge 'hch-misc/mmu-notifier-cleanup' into devel-hourly-2019072323
git bisect good 73df828ed5111467daefc7a882af0fd714820eb4 # 15:21 G 12 0 5 5 Merge 'linux-review/Chuhong-Yuan/ax88179_178a-Merge-memcpy-le32_to_cpus-to-get_unaligned_le32/20190720-070926' into devel-hourly-2019072323
git bisect good 9f79f1a79dab2f8a1096d25b3f03b34e415038da # 15:40 G 12 0 5 5 Merge 'linux-review/Marek-Szyprowski/phy-exynos5-usbdrd-Calibrating-makes-sense-only-for-USB2-0-PHY/20190720-064551' into devel-hourly-2019072323
git bisect good f8eef02736194483854d7f7eed529e8f23c8fe29 # 16:00 G 12 0 2 2 Merge 'linux-review/Marek-Szyprowski/phy-core-document-calibrate-method/20190720-062046' into devel-hourly-2019072323
git bisect bad c4495f22b9f390611e7752f6b03c2eca57afaab9 # 16:21 B 0 2 18 0 nouveau: pass struct nouveau_svmm to nouveau_range_fault
git bisect good 58f936694c5fd6a4c50217e65ee2ee8b88438b66 # 17:59 G 13 0 3 3 mm: merge struct mmu_notifier_mm into struct mm_struct
git bisect bad 2f80e0c686f69d6071f1dd92f0faae36101a3871 # 18:19 B 0 3 19 0 amdkfd: reorder create_process
git bisect bad 7a2452a9b098b402be81bef0b49953f3376e35dc # 18:32 B 0 1 17 1 mm: assert that mmap_sem is held over invalidate_range_{start,end}
# first bad commit: [7a2452a9b098b402be81bef0b49953f3376e35dc] mm: assert that mmap_sem is held over invalidate_range_{start,end}
git bisect good 58f936694c5fd6a4c50217e65ee2ee8b88438b66 # 18:40 G 37 0 7 10 mm: merge struct mmu_notifier_mm into struct mm_struct
# extra tests on HEAD of linux-devel/devel-hourly-2019072323
git bisect bad 9753d961df5e44d13629e2ab95a25c0b9e412040 # 18:41 B 0 304 361 39 0day head guard for 'devel-hourly-2019072323'
# extra tests on tree/branch hch-misc/mmu-notifier-cleanup
git bisect bad fd2a4ea66b2e93fbe75a8c51a686bf44be3a7f3c # 19:02 B 0 1 17 0 mm: remove the hmm_mirror abstraction
# extra tests with first bad commit reverted
git bisect good b4f1849202ffa17d500232caeebcdea2b6f8ff0e # 19:39 G 12 0 4 4 Revert "mm: assert that mmap_sem is held over invalidate_range_{start,end}"
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
3 years
- 1
- 0
Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression
by Rong Chen
On 6/26/19 11:17 AM, Josef Bacik wrote:
> On Wed, Jun 26, 2019 at 10:39:36AM +0800, Rong Chen wrote:
>> On 6/25/19 10:22 PM, Josef Bacik wrote:
>>> On Fri, Jun 21, 2019 at 08:48:03AM +0800, Huang, Ying wrote:
>>>> "Huang, Ying" <ying.huang(a)intel.com> writes:
>>>>
>>>>> "Huang, Ying" <ying.huang(a)intel.com> writes:
>>>>>
>>>>>> Hi, Josef,
>>>>>>
>>>>>> kernel test robot <rong.a.chen(a)intel.com> writes:
>>>>>>
>>>>>>> Greeting,
>>>>>>>
>>>>>>> FYI, we noticed a -11.7% regression of aim7.jobs-per-min due to commit:
>>>>>>>
>>>>>>>
>>>>>>> commit: c8eaeac7b734347c3afba7008b7af62f37b9c140 ("btrfs: reserve
>>>>>>> delalloc metadata differently")
>>>>>>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>>>>>>
>>>>>>> in testcase: aim7
>>>>>>> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
>>>>>>> with following parameters:
>>>>>>>
>>>>>>> disk: 4BRD_12G
>>>>>>> md: RAID0
>>>>>>> fs: btrfs
>>>>>>> test: disk_rr
>>>>>>> load: 1500
>>>>>>> cpufreq_governor: performance
>>>>>>>
>>>>>>> test-description: AIM7 is a traditional UNIX system level benchmark
>>>>>>> suite which is used to test and measure the performance of multiuser
>>>>>>> system.
>>>>>>> test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
>>>>>> Here's another regression, do you have time to take a look at this?
>>>>> Ping
>>>> Ping again ...
>>>>
>>> Finally got time to look at this but I can't get the reproducer to work
>>>
>>> root@destiny ~/lkp-tests# bin/lkp run ~/job-aim.yaml
>>> Traceback (most recent call last):
>>> 11: from /root/lkp-tests/bin/run-local:18:in `<main>'
>>> 10: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 9: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 8: from /root/lkp-tests/lib/yaml.rb:5:in `<top (required)>'
>>> 7: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 6: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 5: from /root/lkp-tests/lib/common.rb:9:in `<top (required)>'
>>> 4: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 3: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> 2: from /root/lkp-tests/lib/array_ext.rb:3:in `<top (required)>'
>>> 1: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require'
>>> /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- active_support/core_ext/enumerable (LoadError)
>> Hi Josef,
>>
>> I tried the latest lkp-tests, and didn't have the problem. Could you please
>> update the lkp-tests repo and run "lkp install" again?
>>
> I updated it this morning, and I just updated it now, my tree is on
>
> 2c5b1a95b08dbe81bba64419c482a877a3b424ac
>
> lkp install says everything is installed except
>
> No match for argument: libipc-run-perl
I've just fixed it. could you add "libipc-run-perl: perl-IPC-Run" to the
end of distro/adaptation/fedora?
Thanks,
Rong Chen
>
> and it still doesn't run properly. Thanks,
>
> Josef
3 years
- 2
- 2
[gpio] f69e00bd21: unixbench.score -24.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -24.2% regression of unixbench.score due to commit:
commit: f69e00bd21aa6a1961c521b6eb199137fcb8a76a ("gpio: mmio: Support two direction registers")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: unixbench
on test machine: 16 threads Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz with 8G memory
with following parameters:
runtime: 300s
nr_task: 30%
test: context1
cpufreq_governor: performance
ucode: 0x7000017
test-description: UnixBench is the original BYTE UNIX benchmark suite aims to test performance of Unix-like system.
test-url: https://github.com/kdlucas/byte-unixbench
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/30%/debian-x86_64-2019-05-14.cgz/300s/lkp-bdw-de1/context1/unixbench/0x7000017
commit:
v5.1-rc1
f69e00bd21 ("gpio: mmio: Support two direction registers")
v5.1-rc1 f69e00bd21aa6a1961c521b6eb1
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
1650 -24.2% 1250 unixbench.score
777.63 +10.8% 861.58 unixbench.time.system_time
67.38 -7.1% 62.58 ± 2% unixbench.time.user_time
1.971e+08 -24.4% 1.489e+08 unixbench.time.voluntary_context_switches
2.832e+08 ± 15% -30.1% 1.981e+08 unixbench.workload
34131 ± 7% -14.5% 29165 ± 7% vmstat.system.in
1396 ± 8% -28.6% 997.50 ± 17% slabinfo.kmalloc-rcl-96.active_objs
1396 ± 8% -28.6% 997.50 ± 17% slabinfo.kmalloc-rcl-96.num_objs
60056 +1.3% 60833 proc-vmstat.nr_anon_pages
4932 +2.0% 5033 proc-vmstat.nr_kernel_stack
10728 +4.0% 11162 proc-vmstat.nr_slab_unreclaimable
3161239 ± 11% +868.3% 30611146 ± 6% turbostat.C1
0.22 ± 34% +2.7 2.94 ± 55% turbostat.C1%
2838830 ± 15% -69.9% 855014 ± 86% turbostat.C1E
4.32 ± 32% -3.3 1.02 ± 88% turbostat.C1E%
14719752 ± 29% +1194.8% 1.906e+08 ± 51% cpuidle.C1.time
3161973 ± 11% +868.2% 30613793 ± 6% cpuidle.C1.usage
2839063 ± 15% -69.9% 855260 ± 86% cpuidle.C1E.usage
3.872e+08 +27.3% 4.929e+08 cpuidle.POLL.time
3.908e+08 -31.2% 2.69e+08 cpuidle.POLL.usage
305522 ± 16% +36.5% 416925 ± 38% sched_debug.cfs_rq:/.load.max
128895 ± 11% +58.5% 204250 ± 10% sched_debug.cfs_rq:/.min_vruntime.min
1527 ± 3% -27.2% 1112 ± 20% sched_debug.cpu.curr->pid.avg
307138 ± 15% +35.7% 416925 ± 38% sched_debug.cpu.load.max
0.00 ± 24% +40.7% 0.00 ± 11% sched_debug.cpu.next_balance.stddev
2396 ± 10% +36.7% 3276 ± 12% sched_debug.cpu.nr_load_updates.stddev
23815922 ± 10% -24.7% 17932434 ± 9% sched_debug.cpu.nr_switches.avg
36235327 ± 11% -30.4% 25204089 ± 9% sched_debug.cpu.nr_switches.max
6473488 ± 19% -36.2% 4126891 ± 12% sched_debug.cpu.nr_switches.stddev
70277 ± 85% -87.9% 8479 ± 10% interrupts.CPU0.RES:Rescheduling_interrupts
191623 ± 61% -93.0% 13479 ± 6% interrupts.CPU1.RES:Rescheduling_interrupts
91876 ± 41% -82.5% 16122 ± 22% interrupts.CPU10.RES:Rescheduling_interrupts
153310 ± 70% -91.9% 12413 ± 21% interrupts.CPU11.RES:Rescheduling_interrupts
134997 ± 30% -87.5% 16852 ± 28% interrupts.CPU12.RES:Rescheduling_interrupts
83408 ± 34% -84.6% 12866 ± 17% interrupts.CPU13.RES:Rescheduling_interrupts
104409 ± 49% -85.1% 15507 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts
113840 ± 70% -84.6% 17548 ± 13% interrupts.CPU15.RES:Rescheduling_interrupts
102300 ± 25% -88.3% 11974 ± 12% interrupts.CPU2.RES:Rescheduling_interrupts
132576 ± 47% -85.4% 19311 ± 20% interrupts.CPU3.RES:Rescheduling_interrupts
171698 ± 28% -92.1% 13577 ± 6% interrupts.CPU4.RES:Rescheduling_interrupts
98087 ± 47% -86.5% 13248 ± 25% interrupts.CPU5.RES:Rescheduling_interrupts
209072 ± 61% -93.7% 13263 ± 19% interrupts.CPU6.RES:Rescheduling_interrupts
130968 ± 46% -90.3% 12730 ± 15% interrupts.CPU7.RES:Rescheduling_interrupts
71714 ± 57% -75.3% 17682 ± 18% interrupts.CPU8.RES:Rescheduling_interrupts
120473 ± 51% -86.1% 16698 ± 21% interrupts.CPU9.RES:Rescheduling_interrupts
1980634 ± 18% -88.3% 231754 ± 10% interrupts.RES:Rescheduling_interrupts
197143 ± 11% +1024.8% 2217418 ± 7% perf-stat.i.dTLB-load-misses
83.30 ± 4% -17.7 65.61 ± 3% perf-stat.i.iTLB-load-miss-rate%
2186220 ± 13% +79.4% 3922852 ± 8% perf-stat.i.iTLB-load-misses
294678 ± 11% +661.5% 2243879 ± 7% perf-stat.i.iTLB-loads
4232 ± 8% -38.2% 2615 ± 5% perf-stat.i.instructions-per-iTLB-miss
20.10 -10.2% 18.06 perf-stat.overall.MPKI
1.67 -0.3 1.40 perf-stat.overall.branch-miss-rate%
1.35 -6.5% 1.27 perf-stat.overall.cpi
67.38 +4.1% 70.15 perf-stat.overall.cycles-between-cache-misses
0.01 ± 28% +0.1 0.07 perf-stat.overall.dTLB-load-miss-rate%
88.08 -24.5 63.60 perf-stat.overall.iTLB-load-miss-rate%
4925 ± 2% -40.7% 2922 perf-stat.overall.instructions-per-iTLB-miss
0.74 +6.9% 0.79 perf-stat.overall.ipc
16297 ± 12% +46.6% 23896 perf-stat.overall.path-length
196702 ± 11% +1024.5% 2211891 ± 7% perf-stat.ps.dTLB-load-misses
2180784 ± 13% +79.4% 3913017 ± 8% perf-stat.ps.iTLB-load-misses
293953 ± 11% +661.4% 2238259 ± 7% perf-stat.ps.iTLB-loads
4.523e+12 +4.6% 4.732e+12 perf-stat.total.instructions
13.64 -2.9 10.73 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.27 -2.8 10.48 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
13.14 ± 2% -2.1 11.01 ± 9% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.73 ± 2% -2.1 10.65 ± 9% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.24 ± 2% -2.1 7.18 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
7.53 ± 2% -1.8 5.68 ± 3% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
7.32 ± 2% -1.8 5.54 ± 3% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.pipe_wait.pipe_read.new_sync_read
5.89 ± 5% -1.0 4.86 ± 2% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
5.67 ± 5% -1.0 4.71 ± 2% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary
4.04 ± 6% -0.8 3.25 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.pipe_wait.pipe_read
3.39 ± 8% -0.7 2.65 ± 7% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.pipe_wait
3.74 ± 5% -0.5 3.19 ± 2% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.44 ± 3% -0.5 0.93 ± 11% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
0.66 ± 6% -0.4 0.28 ±100% perf-profile.calltrace.cycles-pp.__switch_to
2.37 ± 5% -0.4 2.01 perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
0.73 ± 3% -0.3 0.43 ± 59% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.ttwu_do_activate
1.85 ± 5% -0.3 1.55 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
1.81 ± 5% -0.3 1.53 perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
1.63 ± 5% -0.3 1.37 perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
0.94 ± 5% -0.2 0.70 perf-profile.calltrace.cycles-pp.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.21 ± 5% -0.2 0.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
2.30 ± 5% -0.2 2.09 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry
1.03 ± 7% -0.2 0.83 ± 8% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry
0.96 ± 7% -0.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.__next_timer_interrupt.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select
1.66 ± 5% -0.2 1.47 ± 9% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
0.92 ± 3% -0.2 0.75 ± 8% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.64 ± 2% -0.1 1.50 perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle
0.76 -0.1 0.62 ± 11% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule
0.72 ± 4% -0.1 0.60 ± 9% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +0.8 0.78 ± 16% perf-profile.calltrace.cycles-pp.__orc_find.unwind_next_frame.__save_stack_trace.save_stack_trace_tsk.__account_scheduler_latency
14.57 +2.0 16.58 perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
42.75 ± 3% +2.5 45.26 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
26.66 ± 2% +3.2 29.88 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.88 ± 2% +3.3 30.19 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
0.00 +3.8 3.77 ± 5% perf-profile.calltrace.cycles-pp.unwind_next_frame.__save_stack_trace.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity
13.58 ± 2% +5.2 18.73 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
13.72 ± 2% +5.2 18.90 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.62 ± 2% +5.3 17.89 ± 9% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.91 ± 2% +5.3 18.23 ± 9% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +5.7 5.73 ± 6% perf-profile.calltrace.cycles-pp.__save_stack_trace.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
10.73 +5.8 16.52 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
10.35 +5.8 16.16 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
9.74 ± 2% +5.9 15.69 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
9.53 ± 2% +6.0 15.54 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
0.00 +6.5 6.45 ± 5% perf-profile.calltrace.cycles-pp.save_stack_trace_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.ttwu_do_activate
4.48 ± 2% +6.9 11.43 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.38 ± 2% +7.0 11.35 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
3.66 ± 7% +7.2 10.82 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.00 +8.1 8.10 ± 2% perf-profile.calltrace.cycles-pp.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up
15.71 ± 2% -3.3 12.37 perf-profile.children.cycles-pp.ksys_read
15.22 ± 2% -3.2 11.98 perf-profile.children.cycles-pp.vfs_read
13.65 -2.9 10.74 perf-profile.children.cycles-pp.new_sync_read
13.29 -2.8 10.50 perf-profile.children.cycles-pp.pipe_read
13.29 ± 3% -2.6 10.65 perf-profile.children.cycles-pp.__sched_text_start
9.30 ± 2% -2.1 7.22 perf-profile.children.cycles-pp.pipe_wait
7.54 ± 2% -1.7 5.81 perf-profile.children.cycles-pp.schedule
6.05 ± 4% -1.0 5.04 ± 2% perf-profile.children.cycles-pp.schedule_idle
3.04 -1.0 2.05 ± 4% perf-profile.children.cycles-pp.update_load_avg
4.16 ± 2% -0.8 3.33 perf-profile.children.cycles-pp.dequeue_task_fair
3.73 ± 3% -0.8 2.98 perf-profile.children.cycles-pp.dequeue_entity
1.73 ± 5% -0.6 1.16 ± 4% perf-profile.children.cycles-pp.update_curr
3.88 ± 4% -0.6 3.32 perf-profile.children.cycles-pp.menu_select
2.27 ± 4% -0.5 1.76 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64
2.31 ± 2% -0.4 1.92 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
1.42 ± 5% -0.4 1.03 perf-profile.children.cycles-pp.mutex_unlock
2.46 ± 4% -0.4 2.08 perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
3.12 ± 4% -0.4 2.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair
1.27 ± 3% -0.3 0.93 ± 5% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.44 ± 3% -0.3 1.11 ± 2% perf-profile.children.cycles-pp.select_task_rq_fair
1.37 -0.3 1.06 perf-profile.children.cycles-pp.update_rq_clock
1.67 ± 4% -0.3 1.36 ± 6% perf-profile.children.cycles-pp.security_file_permission
1.88 ± 3% -0.3 1.59 perf-profile.children.cycles-pp.tick_nohz_next_event
1.70 ± 4% -0.3 1.41 perf-profile.children.cycles-pp.get_next_timer_interrupt
0.84 ± 6% -0.3 0.57 ± 6% perf-profile.children.cycles-pp.select_idle_sibling
1.09 ± 5% -0.2 0.84 ± 5% perf-profile.children.cycles-pp.__next_timer_interrupt
0.59 ± 7% -0.2 0.35 ± 8% perf-profile.children.cycles-pp.__update_load_avg_se
0.97 ± 5% -0.2 0.74 ± 3% perf-profile.children.cycles-pp.tick_nohz_idle_exit
1.37 ± 6% -0.2 1.15 ± 9% perf-profile.children.cycles-pp.mutex_lock
0.86 ± 5% -0.2 0.64 perf-profile.children.cycles-pp.__switch_to
1.07 ± 6% -0.2 0.86 ± 8% perf-profile.children.cycles-pp.switch_mm_irqs_off
1.01 ± 5% -0.2 0.81 ± 3% perf-profile.children.cycles-pp.update_cfs_group
0.37 ± 6% -0.2 0.18 ± 4% perf-profile.children.cycles-pp.__fsnotify_parent
0.50 ± 11% -0.2 0.31 ± 6% perf-profile.children.cycles-pp.cpus_share_cache
0.24 ± 9% -0.2 0.06 ± 7% perf-profile.children.cycles-pp.clear_buddies
0.76 ± 3% -0.2 0.58 ± 4% perf-profile.children.cycles-pp.copy_user_generic_unrolled
1.00 ± 3% -0.2 0.84 ± 3% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.67 ± 3% -0.2 0.52 ± 4% perf-profile.children.cycles-pp.copyout
0.94 ± 3% -0.2 0.79 ± 2% perf-profile.children.cycles-pp.check_preempt_curr
0.65 ± 3% -0.1 0.51 ± 4% perf-profile.children.cycles-pp.find_next_bit
0.68 ± 4% -0.1 0.53 ± 7% perf-profile.children.cycles-pp.file_has_perm
1.71 ± 2% -0.1 1.57 ± 2% perf-profile.children.cycles-pp.set_next_entity
0.84 ± 5% -0.1 0.71 ± 7% perf-profile.children.cycles-pp.selinux_file_permission
0.50 ± 2% -0.1 0.37 ± 7% perf-profile.children.cycles-pp.reweight_entity
0.98 -0.1 0.85 ± 4% perf-profile.children.cycles-pp.sched_clock
0.62 ± 3% -0.1 0.49 ± 6% perf-profile.children.cycles-pp.finish_task_switch
0.81 ± 3% -0.1 0.68 ± 7% perf-profile.children.cycles-pp.__switch_to_asm
0.51 ± 6% -0.1 0.38 ± 7% perf-profile.children.cycles-pp.nr_iowait_cpu
0.51 ± 9% -0.1 0.38 ± 7% perf-profile.children.cycles-pp.update_ts_time_stats
0.73 ± 3% -0.1 0.60 ± 4% perf-profile.children.cycles-pp.resched_curr
0.44 ± 5% -0.1 0.31 ± 7% perf-profile.children.cycles-pp.___might_sleep
0.48 ± 6% -0.1 0.36 ± 4% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.83 ± 4% -0.1 0.70 ± 3% perf-profile.children.cycles-pp.ktime_get
0.93 ± 2% -0.1 0.81 ± 3% perf-profile.children.cycles-pp.native_sched_clock
1.06 -0.1 0.94 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.30 ± 3% -0.1 0.18 ± 14% perf-profile.children.cycles-pp.rb_erase_cached
0.84 ± 8% -0.1 0.72 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
0.46 ± 2% -0.1 0.34 ± 3% perf-profile.children.cycles-pp.avc_has_perm
0.54 ± 4% -0.1 0.42 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter
0.66 ± 6% -0.1 0.55 ± 3% perf-profile.children.cycles-pp.read_tsc
0.40 ± 10% -0.1 0.29 ± 9% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.55 ± 6% -0.1 0.44 ± 7% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.28 ± 8% -0.1 0.17 ± 6% perf-profile.children.cycles-pp.__calc_delta
0.39 ± 11% -0.1 0.29 ± 13% perf-profile.children.cycles-pp.___perf_sw_event
0.45 ± 9% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.pick_next_entity
0.36 ± 8% -0.1 0.27 ± 10% perf-profile.children.cycles-pp.rcu_idle_exit
1.08 ± 2% -0.1 0.99 ± 7% perf-profile.children.cycles-pp.copy_page_to_iter
0.22 ± 5% -0.1 0.13 ± 12% perf-profile.children.cycles-pp.current_time
0.36 ± 3% -0.1 0.28 ± 4% perf-profile.children.cycles-pp.prepare_to_wait
0.28 ± 6% -0.1 0.20 ± 9% perf-profile.children.cycles-pp.touch_atime
0.32 -0.1 0.23 ± 16% perf-profile.children.cycles-pp.available_idle_cpu
0.22 ± 8% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.atime_needs_update
0.37 ± 8% -0.1 0.29 ± 5% perf-profile.children.cycles-pp.fsnotify
0.17 ± 9% -0.1 0.10 ± 10% perf-profile.children.cycles-pp.__x64_sys_write
0.28 ± 5% -0.1 0.22 ± 8% perf-profile.children.cycles-pp.__might_sleep
0.26 ± 5% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.menu_reflect
0.47 ± 7% -0.1 0.41 ± 3% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.27 ± 4% -0.1 0.21 ± 7% perf-profile.children.cycles-pp.__inode_security_revalidate
0.23 ± 8% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.put_prev_task_fair
0.22 ± 8% -0.1 0.17 ± 8% perf-profile.children.cycles-pp._cond_resched
0.18 ± 18% -0.1 0.13 ± 11% perf-profile.children.cycles-pp.file_update_time
0.18 ± 12% -0.0 0.13 ± 8% perf-profile.children.cycles-pp.__x64_sys_read
0.14 ± 3% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.cpuacct_charge
0.17 ± 7% -0.0 0.12 ± 8% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.14 ± 23% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.sched_ttwu_pending
0.15 ± 12% -0.0 0.11 ± 10% perf-profile.children.cycles-pp.put_prev_entity
0.18 ± 15% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.account_entity_dequeue
0.10 ± 15% -0.0 0.07 ± 15% perf-profile.children.cycles-pp.__sb_start_write
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.get_stack_info
0.00 +0.1 0.07 ± 19% perf-profile.children.cycles-pp.ttwu_stat
0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.00 +0.1 0.08 ± 11% perf-profile.children.cycles-pp.in_lock_functions
0.07 ± 14% +0.1 0.15 ± 4% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.11 ± 9% +0.1 0.20 ± 14% perf-profile.children.cycles-pp.__list_del_entry_valid
0.00 +0.1 0.09 ± 16% perf-profile.children.cycles-pp.put_task_stack
0.18 ± 25% +0.1 0.28 ± 14% perf-profile.children.cycles-pp.wait_for_xmitr
0.18 ± 23% +0.1 0.28 ± 14% perf-profile.children.cycles-pp.uart_console_write
0.17 ± 24% +0.1 0.27 ± 13% perf-profile.children.cycles-pp.serial8250_console_putchar
0.18 ± 23% +0.1 0.29 ± 15% perf-profile.children.cycles-pp.serial8250_console_write
0.19 ± 24% +0.1 0.31 ± 14% perf-profile.children.cycles-pp.console_unlock
0.22 ± 10% +0.1 0.34 ± 12% perf-profile.children.cycles-pp.__might_fault
0.00 +0.1 0.15 ± 7% perf-profile.children.cycles-pp.task_of
0.00 +0.2 0.22 ± 13% perf-profile.children.cycles-pp.in_sched_functions
0.00 +0.2 0.23 ± 12% perf-profile.children.cycles-pp.stack_access_ok
0.00 +0.3 0.30 ± 8% perf-profile.children.cycles-pp.kernel_text_address
1.00 +0.4 1.39 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +0.4 0.41 ± 4% perf-profile.children.cycles-pp.__kernel_text_address
0.00 +0.5 0.53 ± 3% perf-profile.children.cycles-pp.orc_find
0.00 +0.5 0.54 ± 4% perf-profile.children.cycles-pp.unwind_get_return_address
0.00 +0.6 0.57 ± 5% perf-profile.children.cycles-pp.__unwind_start
0.00 +0.6 0.60 ± 8% perf-profile.children.cycles-pp.save_stack_address
0.00 +0.9 0.93 ± 8% perf-profile.children.cycles-pp.__orc_find
31.89 ± 2% +1.7 33.56 perf-profile.children.cycles-pp.do_syscall_64
32.16 ± 2% +1.7 33.91 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
46.24 ± 3% +2.0 48.27 perf-profile.children.cycles-pp.cpuidle_enter_state
14.96 ± 2% +2.2 17.13 perf-profile.children.cycles-pp.poll_idle
0.00 +4.3 4.27 ± 3% perf-profile.children.cycles-pp.unwind_next_frame
15.08 ± 2% +4.9 20.02 ± 2% perf-profile.children.cycles-pp.vfs_write
15.43 ± 2% +5.0 20.40 ± 2% perf-profile.children.cycles-pp.ksys_write
13.60 ± 2% +5.1 18.74 ± 2% perf-profile.children.cycles-pp.pipe_write
13.73 ± 2% +5.2 18.92 ± 2% perf-profile.children.cycles-pp.new_sync_write
11.07 +5.8 16.82 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock
10.39 +5.8 16.21 ± 2% perf-profile.children.cycles-pp.__wake_up_common
0.00 +5.9 5.90 ± 3% perf-profile.children.cycles-pp.__save_stack_trace
9.75 ± 2% +6.0 15.71 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
9.56 ± 2% +6.0 15.56 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
0.00 +6.6 6.58 ± 2% perf-profile.children.cycles-pp.save_stack_trace_tsk
4.48 ± 2% +7.0 11.43 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate
4.40 ± 3% +7.0 11.36 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
3.81 ± 3% +7.1 10.86 ± 2% perf-profile.children.cycles-pp.enqueue_entity
0.00 +8.1 8.11 ± 2% perf-profile.children.cycles-pp.__account_scheduler_latency
2.93 ± 6% -0.8 2.10 ± 2% perf-profile.self.cycles-pp.__sched_text_start
2.22 ± 4% -0.5 1.74 perf-profile.self.cycles-pp.entry_SYSCALL_64
1.20 ± 2% -0.4 0.78 ± 2% perf-profile.self.cycles-pp.update_load_avg
2.31 ± 2% -0.4 1.91 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
1.41 ± 5% -0.4 1.02 perf-profile.self.cycles-pp.mutex_unlock
1.21 ± 6% -0.4 0.83 ± 4% perf-profile.self.cycles-pp.update_curr
1.37 ± 7% -0.4 1.00 ± 5% perf-profile.self.cycles-pp.do_idle
1.27 ± 3% -0.3 0.93 ± 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.99 ± 2% -0.2 0.76 ± 2% perf-profile.self.cycles-pp.update_rq_clock
0.56 ± 7% -0.2 0.34 ± 10% perf-profile.self.cycles-pp.__update_load_avg_se
0.81 ± 5% -0.2 0.60 ± 2% perf-profile.self.cycles-pp.__switch_to
0.36 ± 5% -0.2 0.16 ± 2% perf-profile.self.cycles-pp.__fsnotify_parent
1.06 ± 6% -0.2 0.86 ± 8% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.87 ± 9% -0.2 0.67 ± 6% perf-profile.self.cycles-pp.pick_next_task_fair
0.50 ± 11% -0.2 0.31 ± 6% perf-profile.self.cycles-pp.cpus_share_cache
1.15 ± 4% -0.2 0.96 ± 3% perf-profile.self.cycles-pp.menu_select
0.99 ± 6% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.update_cfs_group
0.74 ± 3% -0.2 0.56 ± 4% perf-profile.self.cycles-pp.copy_user_generic_unrolled
0.65 ± 6% -0.2 0.47 ± 8% perf-profile.self.cycles-pp.dequeue_entity
0.94 ± 3% -0.2 0.78 ± 6% perf-profile.self.cycles-pp.pipe_read
1.01 ± 7% -0.2 0.86 ± 10% perf-profile.self.cycles-pp.mutex_lock
0.59 ± 3% -0.1 0.45 ± 3% perf-profile.self.cycles-pp.pipe_wait
0.53 ± 6% -0.1 0.39 ± 7% perf-profile.self.cycles-pp.__next_timer_interrupt
0.50 ± 3% -0.1 0.36 ± 7% perf-profile.self.cycles-pp.reweight_entity
0.81 ± 3% -0.1 0.68 ± 7% perf-profile.self.cycles-pp.__switch_to_asm
0.73 ± 3% -0.1 0.60 ± 4% perf-profile.self.cycles-pp.resched_curr
0.43 ± 5% -0.1 0.30 ± 8% perf-profile.self.cycles-pp.___might_sleep
0.75 ± 7% -0.1 0.63 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
0.63 ± 4% -0.1 0.50 ± 4% perf-profile.self.cycles-pp.__wake_up_common
0.50 ± 6% -0.1 0.38 ± 8% perf-profile.self.cycles-pp.nr_iowait_cpu
0.45 ± 3% -0.1 0.34 ± 3% perf-profile.self.cycles-pp.avc_has_perm
0.26 ± 5% -0.1 0.14 ± 16% perf-profile.self.cycles-pp.rb_erase_cached
0.60 ± 8% -0.1 0.49 ± 10% perf-profile.self.cycles-pp.enqueue_task_fair
0.89 ± 2% -0.1 0.78 ± 3% perf-profile.self.cycles-pp.native_sched_clock
0.46 ± 5% -0.1 0.35 ± 4% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.48 ± 4% -0.1 0.37 ± 6% perf-profile.self.cycles-pp.finish_task_switch
0.64 ± 6% -0.1 0.54 ± 3% perf-profile.self.cycles-pp.read_tsc
0.56 ± 5% -0.1 0.45 ± 3% perf-profile.self.cycles-pp.find_next_bit
0.36 ± 11% -0.1 0.26 ± 6% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.43 ± 9% -0.1 0.33 ± 14% perf-profile.self.cycles-pp.pick_next_entity
0.27 ± 9% -0.1 0.17 ± 6% perf-profile.self.cycles-pp.__calc_delta
0.84 ± 2% -0.1 0.75 ± 2% perf-profile.self.cycles-pp.try_to_wake_up
0.77 ± 2% -0.1 0.68 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.31 ± 2% -0.1 0.22 ± 16% perf-profile.self.cycles-pp.available_idle_cpu
0.36 ± 6% -0.1 0.27 ± 9% perf-profile.self.cycles-pp.pipe_write
0.32 ± 7% -0.1 0.23 ± 4% perf-profile.self.cycles-pp.vfs_read
0.40 ± 10% -0.1 0.32 ± 8% perf-profile.self.cycles-pp.dequeue_task_fair
0.35 ± 7% -0.1 0.28 ± 7% perf-profile.self.cycles-pp.fsnotify
0.30 ± 7% -0.1 0.22 ± 11% perf-profile.self.cycles-pp.new_sync_read
0.15 ± 7% -0.1 0.09 ± 16% perf-profile.self.cycles-pp.__x64_sys_write
0.25 ± 7% -0.1 0.20 ± 9% perf-profile.self.cycles-pp.ksys_read
0.20 ± 9% -0.1 0.15 ± 7% perf-profile.self.cycles-pp.rcu_idle_exit
0.12 ± 7% -0.1 0.07 ± 23% perf-profile.self.cycles-pp.current_time
0.16 ± 7% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.schedule_idle
0.13 ± 9% -0.0 0.09 ± 19% perf-profile.self.cycles-pp.schedule
0.24 ± 6% -0.0 0.20 ± 10% perf-profile.self.cycles-pp.__might_sleep
0.26 ± 6% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.cpuidle_enter_state
0.14 ± 3% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.cpuacct_charge
0.14 ± 23% -0.0 0.10 ± 9% perf-profile.self.cycles-pp.sched_ttwu_pending
0.22 ± 9% -0.0 0.18 ± 12% perf-profile.self.cycles-pp.call_cpuidle
0.17 ± 16% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.account_entity_dequeue
0.15 ± 5% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.16 ± 5% -0.0 0.13 ± 11% perf-profile.self.cycles-pp.copy_page_to_iter
0.12 ± 9% -0.0 0.08 ± 21% perf-profile.self.cycles-pp.atime_needs_update
0.20 ± 9% -0.0 0.16 ± 6% perf-profile.self.cycles-pp.prepare_to_wait
0.10 ± 9% -0.0 0.07 ± 16% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.11 ± 7% -0.0 0.08 ± 15% perf-profile.self.cycles-pp._cond_resched
0.10 ± 15% -0.0 0.07 ± 15% perf-profile.self.cycles-pp.__sb_start_write
0.09 ± 11% -0.0 0.07 ± 17% perf-profile.self.cycles-pp.__inode_security_revalidate
0.12 ± 11% +0.0 0.16 ± 9% perf-profile.self.cycles-pp.new_sync_write
0.00 +0.1 0.06 ± 22% perf-profile.self.cycles-pp.ttwu_stat
0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.in_lock_functions
0.28 ± 12% +0.1 0.35 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.07 ± 14% +0.1 0.15 ± 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.11 ± 6% +0.1 0.19 ± 11% perf-profile.self.cycles-pp.ksys_write
0.00 +0.1 0.09 ± 16% perf-profile.self.cycles-pp.put_task_stack
0.10 ± 14% +0.1 0.19 ± 16% perf-profile.self.cycles-pp.__list_del_entry_valid
0.14 ± 18% +0.1 0.24 ± 11% perf-profile.self.cycles-pp.vfs_write
0.00 +0.1 0.10 ± 18% perf-profile.self.cycles-pp.hrtimer_get_next_event
0.33 ± 15% +0.1 0.44 ± 4% perf-profile.self.cycles-pp.do_syscall_64
0.00 +0.1 0.11 ± 19% perf-profile.self.cycles-pp.__kernel_text_address
0.00 +0.1 0.14 ± 19% perf-profile.self.cycles-pp.in_sched_functions
0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.unwind_get_return_address
0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.kernel_text_address
0.00 +0.2 0.18 ± 12% perf-profile.self.cycles-pp.__unwind_start
0.07 ± 17% +0.2 0.25 ± 16% perf-profile.self.cycles-pp.__might_fault
0.00 +0.2 0.21 ± 11% perf-profile.self.cycles-pp.stack_access_ok
0.47 ± 9% +0.2 0.72 ± 4% perf-profile.self.cycles-pp.enqueue_entity
0.97 +0.3 1.27 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.3 0.33 ± 5% perf-profile.self.cycles-pp.__save_stack_trace
0.00 +0.4 0.36 ± 9% perf-profile.self.cycles-pp.save_stack_address
0.00 +0.5 0.49 ± 2% perf-profile.self.cycles-pp.orc_find
0.00 +0.7 0.67 ± 3% perf-profile.self.cycles-pp.save_stack_trace_tsk
0.00 +0.8 0.77 ± 4% perf-profile.self.cycles-pp.__account_scheduler_latency
0.00 +0.9 0.91 ± 8% perf-profile.self.cycles-pp.__orc_find
14.65 ± 2% +2.1 16.74 perf-profile.self.cycles-pp.poll_idle
0.00 +2.6 2.57 ± 2% perf-profile.self.cycles-pp.unwind_next_frame
unixbench.time.voluntary_context_switches
2e+08 +-+---------------------------------------------------------------+
1.8e+08 +-+.+ : : : |
| : : : : |
1.6e+08 +-+ : : : : O |
1.4e+08 +-+ : : O :O O O O O O O O O O O O O O O O O O O
O O O:O O: O : : |
1.2e+08 +-+ : : : : |
1e+08 +-+ : : :: |
8e+07 +-+ : : :: |
| : : :: |
6e+07 +-+ : : :: |
4e+07 +-+ : : : |
| : : : |
2e+07 +-+ : : : |
0 +-+-------O---O--O-------O---O-----------------O-O--------O-------+
unixbench.score
1800 +-+------------------------------------------------------------------+
| +.+.+.+ + |
1600 +-+.+ : : : |
1400 +-+ : : : : |
| : : : : O O O O O O O O O O O
1200 O-O O:O O: O O: :O O O O O O O O O |
1000 +-+ : : : : |
| : : : : |
800 +-+ : : : : |
600 +-+ : : : : |
| : : : : |
400 +-+ : : : |
200 +-+ : : : |
| : : : |
0 +-+-------O---O---O-------O---O------------------O-O---------O-------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
3 years
- 3
- 2
[x86/modules] f2c65fb322: will-it-scale.per_process_ops -2.9% regression
by kernel test robot
Greeting,
FYI, we noticed a -2.9% regression of will-it-scale.per_process_ops due to commit:
commit: f2c65fb3221adc6b73b0549fc7ba892022db9797 ("x86/modules: Avoid breaking W^X while loading modules")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory
with following parameters:
nr_task: 100%
mode: process
test: poll1
cpufreq_governor: performance
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-2019-05-14.cgz/lkp-csl-2ap1/poll1/will-it-scale
commit:
7298e24f90 ("x86/kprobes: Set instruction page as executable")
f2c65fb322 ("x86/modules: Avoid breaking W^X while loading modules")
7298e24f904224fa f2c65fb3221adc6b73b0549fc7b
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 50% 2:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
5017865 -2.9% 4874578 will-it-scale.per_process_ops
9.634e+08 -2.9% 9.359e+08 will-it-scale.workload
33.31 -1.0% 32.99 boot-time.dhcp
10541056 ± 7% +12.1% 11820032 meminfo.DirectMap2M
652.58 ± 7% +10.3% 719.62 ± 8% sched_debug.cfs_rq:/.util_avg.min
19388 ± 14% -17.5% 16003 ± 6% softirqs.CPU134.RCU
1140906 -2.7% 1109655 ± 2% proc-vmstat.pgalloc_normal
1069002 ± 2% -4.7% 1018671 ± 3% proc-vmstat.pgfree
2124 ± 11% -21.8% 1662 ± 4% slabinfo.pool_workqueue.active_objs
2220 ± 11% -21.6% 1742 ± 5% slabinfo.pool_workqueue.num_objs
8708 ± 3% +11.6% 9714 ± 6% numa-vmstat.node0.nr_kernel_stack
6771 ± 7% -12.5% 5924 ± 2% numa-vmstat.node3.nr_kernel_stack
5180 ± 19% -31.7% 3540 ± 13% numa-vmstat.node3.nr_slab_reclaimable
8710 ± 3% +11.5% 9714 ± 6% numa-meminfo.node0.KernelStack
20724 ± 19% -31.7% 14162 ± 13% numa-meminfo.node3.KReclaimable
6773 ± 7% -12.5% 5923 ± 2% numa-meminfo.node3.KernelStack
20724 ± 19% -31.7% 14162 ± 13% numa-meminfo.node3.SReclaimable
82391 ± 9% -12.4% 72177 ± 8% numa-meminfo.node3.Slab
171.00 ±103% -87.6% 21.25 ± 70% interrupts.CPU110.RES:Rescheduling_interrupts
8457 -37.6% 5278 ± 34% interrupts.CPU145.NMI:Non-maskable_interrupts
8457 -37.6% 5278 ± 34% interrupts.CPU145.PMI:Performance_monitoring_interrupts
8453 -37.5% 5285 ± 34% interrupts.CPU146.NMI:Non-maskable_interrupts
8453 -37.5% 5285 ± 34% interrupts.CPU146.PMI:Performance_monitoring_interrupts
8458 -37.5% 5284 ± 34% interrupts.CPU147.NMI:Non-maskable_interrupts
8458 -37.5% 5284 ± 34% interrupts.CPU147.PMI:Performance_monitoring_interrupts
8456 -37.6% 5279 ± 34% interrupts.CPU149.NMI:Non-maskable_interrupts
8456 -37.6% 5279 ± 34% interrupts.CPU149.PMI:Performance_monitoring_interrupts
8449 -37.5% 5278 ± 34% interrupts.CPU151.NMI:Non-maskable_interrupts
8449 -37.5% 5278 ± 34% interrupts.CPU151.PMI:Performance_monitoring_interrupts
14.75 ± 41% +2237.3% 344.75 ± 93% interrupts.CPU154.RES:Rescheduling_interrupts
108.75 ±112% -85.7% 15.50 ± 41% interrupts.CPU168.RES:Rescheduling_interrupts
168.00 ± 19% -41.8% 97.75 ± 40% interrupts.CPU18.RES:Rescheduling_interrupts
1882 ± 87% -94.9% 95.50 ± 50% interrupts.CPU19.RES:Rescheduling_interrupts
218.75 ± 88% +196.9% 649.50 ± 69% interrupts.CPU22.RES:Rescheduling_interrupts
8448 -37.4% 5291 ± 34% interrupts.CPU31.NMI:Non-maskable_interrupts
8448 -37.4% 5291 ± 34% interrupts.CPU31.PMI:Performance_monitoring_interrupts
79.50 ± 23% +537.4% 506.75 ± 75% interrupts.CPU33.RES:Rescheduling_interrupts
8436 -37.5% 5274 ± 34% interrupts.CPU34.NMI:Non-maskable_interrupts
8436 -37.5% 5274 ± 34% interrupts.CPU34.PMI:Performance_monitoring_interrupts
195.50 ±127% +423.1% 1022 ± 47% interrupts.CPU36.RES:Rescheduling_interrupts
43.00 ± 46% +3592.4% 1587 ±150% interrupts.CPU37.RES:Rescheduling_interrupts
82.00 ± 28% +103.4% 166.75 ± 52% interrupts.CPU41.RES:Rescheduling_interrupts
102.75 ± 29% -53.3% 48.00 ± 45% interrupts.CPU50.RES:Rescheduling_interrupts
474.00 ± 30% -73.0% 127.75 ± 75% interrupts.CPU68.RES:Rescheduling_interrupts
8042 ± 4% +6.5% 8565 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts
0.02 -6.0% 0.02 ± 5% perf-stat.i.MPKI
1.111e+11 -2.8% 1.079e+11 perf-stat.i.branch-instructions
2.454e+08 -3.4% 2.371e+08 perf-stat.i.branch-misses
726252 ± 3% -7.8% 669916 ± 3% perf-stat.i.cache-misses
13163065 -8.5% 12044270 ± 5% perf-stat.i.cache-references
1.02 +3.0% 1.05 perf-stat.i.cpi
835228 +7.0% 893984 ± 4% perf-stat.i.cycles-between-cache-misses
1.419e+11 -2.8% 1.38e+11 perf-stat.i.dTLB-loads
0.00 -0.0 0.00 ± 3% perf-stat.i.dTLB-store-miss-rate%
13827 -8.8% 12606 ± 3% perf-stat.i.dTLB-store-misses
1.023e+11 -2.8% 9.937e+10 perf-stat.i.dTLB-stores
2.909e+08 -12.0% 2.56e+08 ± 2% perf-stat.i.iTLB-load-misses
5.448e+11 -2.8% 5.293e+11 perf-stat.i.instructions
1876 +10.4% 2071 ± 2% perf-stat.i.instructions-per-iTLB-miss
0.98 -2.9% 0.95 perf-stat.i.ipc
3297 -1.7% 3241 perf-stat.i.minor-faults
31602 ± 11% -21.5% 24805 ± 15% perf-stat.i.node-loads
3297 -1.7% 3241 perf-stat.i.page-faults
0.02 -5.8% 0.02 ± 5% perf-stat.overall.MPKI
1.02 +3.0% 1.05 perf-stat.overall.cpi
766162 ± 3% +8.5% 831132 ± 3% perf-stat.overall.cycles-between-cache-misses
0.00 -0.0 0.00 ± 3% perf-stat.overall.dTLB-store-miss-rate%
1872 +10.4% 2068 ± 2% perf-stat.overall.instructions-per-iTLB-miss
0.98 -2.9% 0.95 perf-stat.overall.ipc
85.08 +3.2 88.25 ± 2% perf-stat.overall.node-load-miss-rate%
1.107e+11 -2.8% 1.075e+11 perf-stat.ps.branch-instructions
2.446e+08 -3.4% 2.363e+08 perf-stat.ps.branch-misses
723856 ± 3% -7.8% 667682 ± 3% perf-stat.ps.cache-misses
13118781 -8.5% 12003520 ± 5% perf-stat.ps.cache-references
1.414e+11 -2.8% 1.375e+11 perf-stat.ps.dTLB-loads
13781 -8.8% 12564 ± 3% perf-stat.ps.dTLB-store-misses
1.019e+11 -2.8% 9.903e+10 perf-stat.ps.dTLB-stores
2.899e+08 -12.0% 2.551e+08 ± 2% perf-stat.ps.iTLB-load-misses
5.429e+11 -2.8% 5.275e+11 perf-stat.ps.instructions
3286 -1.7% 3230 perf-stat.ps.minor-faults
31502 ± 11% -21.5% 24724 ± 15% perf-stat.ps.node-loads
3286 -1.7% 3230 perf-stat.ps.page-faults
1.623e+14 -3.3% 1.57e+14 perf-stat.total.instructions
25.77 -0.7 25.07 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.poll
3.34 -0.3 3.08 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_from_user.do_sys_poll.__x64_sys_poll.do_syscall_64
18.82 -0.2 18.60 perf-profile.calltrace.cycles-pp._copy_from_user.do_sys_poll.__x64_sys_poll.do_syscall_64.entry_SYSCALL_64_after_hwframe
98.24 -0.2 98.08 perf-profile.calltrace.cycles-pp.poll
4.33 -0.2 4.17 perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_sys_poll.__x64_sys_poll.do_syscall_64
4.96 -0.1 4.86 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.poll
3.32 -0.1 3.21 perf-profile.calltrace.cycles-pp.___might_sleep.__might_fault._copy_from_user.do_sys_poll.__x64_sys_poll
1.63 +0.2 1.79 perf-profile.calltrace.cycles-pp.testcase
6.54 +0.2 6.72 perf-profile.calltrace.cycles-pp.__might_fault._copy_from_user.do_sys_poll.__x64_sys_poll.do_syscall_64
1.85 +0.2 2.05 perf-profile.calltrace.cycles-pp.__might_sleep.__might_fault._copy_from_user.do_sys_poll.__x64_sys_poll
1.41 +0.3 1.74 ± 2% perf-profile.calltrace.cycles-pp.check_stack_object.__check_object_size.do_sys_poll.__x64_sys_poll.do_syscall_64
3.44 +0.5 3.91 perf-profile.calltrace.cycles-pp.__check_object_size.do_sys_poll.__x64_sys_poll.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.12 +0.6 1.68 perf-profile.calltrace.cycles-pp.poll_freewait.do_sys_poll.__x64_sys_poll.do_syscall_64.entry_SYSCALL_64_after_hwframe
64.34 +0.7 65.03 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.poll
60.51 +0.7 61.25 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.poll
49.87 +0.8 50.68 perf-profile.calltrace.cycles-pp.do_sys_poll.__x64_sys_poll.do_syscall_64.entry_SYSCALL_64_after_hwframe.poll
56.87 +1.0 57.84 perf-profile.calltrace.cycles-pp.__x64_sys_poll.do_syscall_64.entry_SYSCALL_64_after_hwframe.poll
17.21 -0.4 16.78 perf-profile.children.cycles-pp.entry_SYSCALL_64
13.72 -0.4 13.32 perf-profile.children.cycles-pp.syscall_return_via_sysret
3.67 -0.3 3.38 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
5.37 -0.3 5.10 perf-profile.children.cycles-pp.copy_user_generic_unrolled
18.32 -0.1 18.20 perf-profile.children.cycles-pp._copy_from_user
3.53 -0.1 3.42 perf-profile.children.cycles-pp.___might_sleep
98.40 -0.1 98.31 perf-profile.children.cycles-pp.poll
1.82 +0.1 1.96 perf-profile.children.cycles-pp.testcase
6.96 +0.2 7.15 perf-profile.children.cycles-pp.__might_fault
1.96 +0.2 2.17 perf-profile.children.cycles-pp.__might_sleep
1.55 +0.3 1.82 perf-profile.children.cycles-pp.check_stack_object
3.79 +0.6 4.40 perf-profile.children.cycles-pp.__check_object_size
1.13 +0.6 1.76 perf-profile.children.cycles-pp.poll_freewait
64.46 +0.7 65.13 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
60.89 +0.7 61.62 perf-profile.children.cycles-pp.do_syscall_64
56.65 +0.9 57.52 perf-profile.children.cycles-pp.__x64_sys_poll
51.27 +1.1 52.38 perf-profile.children.cycles-pp.do_sys_poll
13.70 -0.4 13.30 perf-profile.self.cycles-pp.syscall_return_via_sysret
4.78 -0.3 4.45 perf-profile.self.cycles-pp.__x64_sys_poll
3.40 -0.3 3.15 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
5.01 -0.2 4.78 perf-profile.self.cycles-pp.copy_user_generic_unrolled
11.45 -0.2 11.25 perf-profile.self.cycles-pp.poll
8.55 -0.2 8.39 perf-profile.self.cycles-pp.entry_SYSCALL_64
3.96 -0.1 3.85 perf-profile.self.cycles-pp.do_syscall_64
3.40 -0.1 3.28 perf-profile.self.cycles-pp.___might_sleep
3.53 -0.1 3.47 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.73 -0.0 0.71 perf-profile.self.cycles-pp.poll_select_set_timeout
2.78 +0.1 2.84 perf-profile.self.cycles-pp._copy_from_user
1.58 +0.1 1.66 perf-profile.self.cycles-pp.__might_fault
1.56 +0.1 1.65 perf-profile.self.cycles-pp.testcase
1.41 +0.2 1.57 perf-profile.self.cycles-pp.check_stack_object
1.73 +0.2 1.94 perf-profile.self.cycles-pp.__might_sleep
2.30 +0.3 2.61 perf-profile.self.cycles-pp.__check_object_size
23.36 +0.3 23.70 perf-profile.self.cycles-pp.do_sys_poll
0.92 +0.6 1.53 perf-profile.self.cycles-pp.poll_freewait
will-it-scale.per_process_ops
6e+06 +-+-----------------------------------------------------------------+
| |
5e+06 +-++...+ +..+...+..+...+..+..+...+..+...+..+..+...+..+...+..|
O O O O O O O O O O O O O O O O O O O
| : : |
4e+06 +-+ : : |
| : : |
3e+06 +-+ : : |
| : : |
2e+06 +-+ : : |
| : : |
| : : |
1e+06 +-+ : : |
| : : |
0 +-+-------O---O-----------------------------------------------------+
will-it-scale.workload
1e+09 +-+-----------------------------------------------------------------+
9e+08 O-+O...O O..O. O O O..O..O...O..O...O..O..O...O. O O..O
| : : |
8e+08 +-+ : : |
7e+08 +-+ : : |
| : : |
6e+08 +-+ : : |
5e+08 +-+ : : |
4e+08 +-+ : : |
| : : |
3e+08 +-+ : : |
2e+08 +-+ : : |
| : : |
1e+08 +-+ : : |
0 +-+-------O---O-----------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
3 years
- 3
- 2
[sched/fair] b3e5e9a1dd: hackbench.throughput 38.7% improvement
by kernel test robot
Greeting,
FYI, we noticed a 38.7% improvement of hackbench.throughput due to commit:
commit: b3e5e9a1ddb6f4122985b1164dd9186be788e138 ("sched/fair: use load instead of runnable load")
https://git.linaro.org/people/vincent.guittot/kernel.git sched-rework-load_balance
in testcase: hackbench
on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:
nr_threads: 100%
mode: threads
ipc: pipe
cpufreq_governor: performance
ucode: 0xb8
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc...
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput 47.5% improvement |
| test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | mode=process |
| | nr_threads=50% |
| | ucode=0xb000036 |
+------------------+----------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/50%/debian-x86_64-2019-05-14.cgz/lkp-bdw-ep3/hackbench/0xb000036
commit:
e1f4f2a13d ("sched/fair: rework load_balance")
b3e5e9a1dd ("sched/fair: use load instead of runnable load")
e1f4f2a13d354801 b3e5e9a1ddb6f4122985b1164dd
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
1:4 -25% :4 dmesg.WARNING:at_ip__mutex_lock/0x
%stddev %change %stddev
\ | \
125832 +38.7% 174555 hackbench.throughput
5.356e+09 ± 2% -66.8% 1.78e+09 hackbench.time.involuntary_context_switches
413240 ± 9% +47.7% 610252 ± 6% hackbench.time.minor_page_faults
8755 -1.8% 8601 hackbench.time.percent_of_cpu_this_job_got
47704 -9.3% 43288 hackbench.time.system_time
9815 +30.0% 12755 hackbench.time.user_time
6.691e+09 -40.2% 3.999e+09 hackbench.time.voluntary_context_switches
8.448e+08 +37.5% 1.162e+09 hackbench.workload
0.61 ± 7% +1.5 2.12 mpstat.cpu.all.idle%
0.00 ±149% +0.0 0.01 ± 28% mpstat.cpu.all.iowait%
16.98 +5.3 22.29 mpstat.cpu.all.usr%
10090882 ± 22% +535.7% 64146280 numa-numastat.node0.local_node
10113609 ± 22% +534.4% 64159127 numa-numastat.node0.numa_hit
9203066 ± 16% +605.6% 64936050 numa-numastat.node1.local_node
9208860 ± 16% +605.3% 64951721 numa-numastat.node1.numa_hit
81.75 -8.3% 75.00 vmstat.cpu.sy
16.50 ± 3% +33.3% 22.00 vmstat.cpu.us
18272710 -50.1% 9115472 vmstat.system.cs
1117858 ± 3% +11.3% 1244045 vmstat.system.in
83213 ± 15% -29.9% 58328 ± 3% numa-meminfo.node0.SUnreclaim
118525 ± 10% -20.1% 94712 ± 4% numa-meminfo.node0.Slab
18693 ± 69% +78.5% 33372 ± 3% numa-meminfo.node1.KernelStack
71505 ± 18% +37.3% 98162 numa-meminfo.node1.SUnreclaim
108548 ± 11% +24.3% 134900 ± 3% numa-meminfo.node1.Slab
20803 ± 15% -29.9% 14581 ± 3% numa-vmstat.node0.nr_slab_unreclaimable
5305690 ± 21% +509.5% 32340772 numa-vmstat.node0.numa_hit
5283141 ± 21% +510.0% 32227947 numa-vmstat.node0.numa_local
18690 ± 69% +78.8% 33415 ± 3% numa-vmstat.node1.nr_kernel_stack
17875 ± 18% +37.3% 24546 numa-vmstat.node1.nr_slab_unreclaimable
5050885 ± 7% +551.8% 32921576 numa-vmstat.node1.numa_hit
4896930 ± 7% +571.0% 32857640 numa-vmstat.node1.numa_local
55893771 ± 16% +373.8% 2.648e+08 cpuidle.C1.time
4166692 ± 15% +1006.3% 46095081 cpuidle.C1.usage
28618199 ± 13% +186.2% 81912840 ± 4% cpuidle.C1E.time
843928 ± 10% +1466.9% 13223566 ± 6% cpuidle.C1E.usage
80274720 ± 54% +122.9% 1.789e+08 ± 20% cpuidle.C3.time
406597 ± 22% +4522.4% 18794736 ± 3% cpuidle.C3.usage
2710248 ± 6% +6688.1% 1.84e+08 cpuidle.POLL.time
1979212 ± 6% +6201.8% 1.247e+08 cpuidle.POLL.usage
4164203 ± 15% +1006.9% 46091802 turbostat.C1
0.10 ± 16% +0.4 0.46 turbostat.C1%
840432 ± 10% +1472.7% 13217629 ± 6% turbostat.C1E
0.05 ± 9% +0.1 0.15 ± 3% turbostat.C1E%
405971 ± 22% +4529.4% 18793954 ± 3% turbostat.C3
0.14 ± 55% +0.2 0.31 ± 20% turbostat.C3%
0.33 ± 5% +189.2% 0.94 ± 3% turbostat.CPU%c1
7.54e+08 ± 2% +11.1% 8.376e+08 turbostat.IRQ
280.53 -3.4% 271.09 turbostat.PkgWatt
9.41 ± 9% -20.2% 7.51 turbostat.RAMWatt
1627 ± 11% +24.2% 2021 slabinfo.UNIX.active_objs
1627 ± 11% +24.2% 2021 slabinfo.UNIX.num_objs
571.00 ± 9% +28.0% 731.00 ± 3% slabinfo.kmem_cache_node.active_objs
624.00 ± 8% +25.6% 784.00 ± 3% slabinfo.kmem_cache_node.num_objs
11297 +22.3% 13813 slabinfo.pid.active_objs
11297 +22.3% 13813 slabinfo.pid.num_objs
1113 ± 5% +17.0% 1302 ± 2% slabinfo.proc_dir_entry.active_objs
1113 ± 5% +17.0% 1302 ± 2% slabinfo.proc_dir_entry.num_objs
3078 ± 9% +21.1% 3728 ± 2% slabinfo.sock_inode_cache.active_objs
3078 ± 9% +21.1% 3728 ± 2% slabinfo.sock_inode_cache.num_objs
12793 +18.2% 15122 slabinfo.task_delay_info.active_objs
12793 +18.2% 15122 slabinfo.task_delay_info.num_objs
4602 +2.0% 4697 proc-vmstat.nr_inactive_anon
18088 +1.1% 18279 proc-vmstat.nr_slab_reclaimable
38692 +1.1% 39134 proc-vmstat.nr_slab_unreclaimable
4602 +2.0% 4697 proc-vmstat.nr_zone_inactive_anon
339762 ± 11% +48.4% 504341 ± 7% proc-vmstat.numa_hint_faults
308740 ± 11% +51.2% 466758 ± 7% proc-vmstat.numa_hint_faults_local
19318659 ± 8% +568.3% 1.291e+08 proc-vmstat.numa_hit
19290129 ± 8% +569.2% 1.291e+08 proc-vmstat.numa_local
22872 ± 16% +47.7% 33774 ± 16% proc-vmstat.numa_pages_migrated
376287 ± 11% +44.7% 544498 ± 5% proc-vmstat.numa_pte_updates
19443245 ± 8% +564.8% 1.293e+08 proc-vmstat.pgalloc_normal
1816628 ± 2% +18.3% 2148484 proc-vmstat.pgfault
19392441 ± 8% +566.3% 1.292e+08 proc-vmstat.pgfree
22872 ± 16% +47.7% 33774 ± 16% proc-vmstat.pgmigrate_success
16842 ±173% +845.4% 159225 ± 13% sched_debug.cfs_rq:/.MIN_vruntime.avg
504557 ±173% +1875.1% 9965626 ± 16% sched_debug.cfs_rq:/.MIN_vruntime.max
89660 ±173% +1248.2% 1208788 ± 12% sched_debug.cfs_rq:/.MIN_vruntime.stddev
2220 ± 41% -88.6% 252.02 ± 86% sched_debug.cfs_rq:/.load.min
3.62 ± 21% -36.0% 2.32 ± 27% sched_debug.cfs_rq:/.load_avg.min
16842 ±173% +845.4% 159225 ± 13% sched_debug.cfs_rq:/.max_vruntime.avg
504557 ±173% +1875.1% 9965626 ± 16% sched_debug.cfs_rq:/.max_vruntime.max
89660 ±173% +1248.2% 1208788 ± 12% sched_debug.cfs_rq:/.max_vruntime.stddev
497595 ± 11% -55.6% 221090 ± 15% sched_debug.cfs_rq:/.min_vruntime.stddev
0.12 ± 9% +38.9% 0.16 ± 7% sched_debug.cfs_rq:/.nr_running.stddev
1.63 ± 35% -88.9% 0.18 ± 49% sched_debug.cfs_rq:/.runnable_load_avg.min
2223 ± 28% -94.3% 126.52 ± 31% sched_debug.cfs_rq:/.runnable_weight.min
-176398 +304.6% -713751 sched_debug.cfs_rq:/.spread0.avg
936274 ± 30% -83.5% 154606 ±112% sched_debug.cfs_rq:/.spread0.max
497215 ± 11% -55.4% 221511 ± 15% sched_debug.cfs_rq:/.spread0.stddev
697.94 ± 4% -26.6% 512.30 ± 11% sched_debug.cfs_rq:/.util_avg.min
125.38 ± 9% +46.8% 184.09 ± 6% sched_debug.cfs_rq:/.util_avg.stddev
1059 ± 2% +20.8% 1280 sched_debug.cfs_rq:/.util_est_enqueued.max
207.94 ± 9% -97.8% 4.61 ± 64% sched_debug.cfs_rq:/.util_est_enqueued.min
138.66 ± 5% +102.6% 280.97 sched_debug.cfs_rq:/.util_est_enqueued.stddev
273955 ± 15% -64.4% 97578 ± 26% sched_debug.cpu.avg_idle.avg
938931 ± 2% -72.7% 256656 ± 18% sched_debug.cpu.avg_idle.max
1563 ± 4% +491.3% 9243 ± 91% sched_debug.cpu.avg_idle.min
291839 ± 7% -80.5% 56945 ± 29% sched_debug.cpu.avg_idle.stddev
96.76 ± 12% -64.2% 34.66 ± 23% sched_debug.cpu.clock.stddev
96.76 ± 12% -64.2% 34.66 ± 23% sched_debug.cpu.clock_task.stddev
13996 ± 4% +25.1% 17507 ± 4% sched_debug.cpu.curr->pid.avg
15668 ± 3% +22.7% 19229 sched_debug.cpu.curr->pid.max
11936 ± 10% -59.1% 4877 ± 46% sched_debug.cpu.curr->pid.min
693.40 ± 13% +209.9% 2148 ± 16% sched_debug.cpu.curr->pid.stddev
0.00 ± 9% -45.8% 0.00 ± 9% sched_debug.cpu.next_balance.stddev
4.04 ± 9% -91.0% 0.36 ± 46% sched_debug.cpu.nr_running.min
2.18 ± 3% +113.6% 4.66 ± 10% sched_debug.cpu.nr_running.stddev
64531739 ± 4% -51.5% 31316434 sched_debug.cpu.nr_switches.avg
72352082 ± 4% -55.1% 32474123 sched_debug.cpu.nr_switches.max
58295154 ± 3% -48.1% 30272582 sched_debug.cpu.nr_switches.min
2780772 ± 10% -82.8% 476944 ± 11% sched_debug.cpu.nr_switches.stddev
0.78 ± 22% +78.6% 1.39 ± 16% sched_debug.cpu.nr_uninterruptible.avg
-959.51 +20.8% -1159 sched_debug.cpu.nr_uninterruptible.min
397.15 ± 6% +14.0% 452.95 ± 4% sched_debug.cpu.nr_uninterruptible.stddev
12.49 +4.7% 13.07 perf-stat.i.MPKI
2.492e+10 -18.8% 2.023e+10 perf-stat.i.branch-instructions
4.07e+08 -18.1% 3.333e+08 perf-stat.i.branch-misses
3.49 ± 33% -2.3 1.15 ± 9% perf-stat.i.cache-miss-rate%
52727638 ± 33% -72.2% 14681234 ± 9% perf-stat.i.cache-misses
1.543e+09 -14.0% 1.327e+09 perf-stat.i.cache-references
18354197 -50.1% 9152810 perf-stat.i.context-switches
1.97 +20.8% 2.38 perf-stat.i.cpi
5220493 -76.8% 1213124 perf-stat.i.cpu-migrations
10992 ± 51% +92.9% 21201 ± 7% perf-stat.i.cycles-between-cache-misses
0.08 +0.1 0.14 perf-stat.i.dTLB-load-miss-rate%
29464252 +51.7% 44702666 perf-stat.i.dTLB-load-misses
3.695e+10 -15.9% 3.108e+10 perf-stat.i.dTLB-loads
0.11 +0.1 0.18 perf-stat.i.dTLB-store-miss-rate%
25874600 +39.4% 36059386 perf-stat.i.dTLB-store-misses
2.345e+10 -14.2% 2.012e+10 perf-stat.i.dTLB-stores
94.72 -8.9 85.84 perf-stat.i.iTLB-load-miss-rate%
29159908 ± 14% +37.6% 40133279 ± 6% perf-stat.i.iTLB-load-misses
1559944 ± 6% +312.6% 6436938 ± 6% perf-stat.i.iTLB-loads
1.238e+11 -17.9% 1.016e+11 perf-stat.i.instructions
4517 ± 4% -42.4% 2603 ± 4% perf-stat.i.instructions-per-iTLB-miss
0.51 -17.4% 0.42 perf-stat.i.ipc
2704 ± 2% +19.3% 3226 perf-stat.i.minor-faults
27293891 ± 32% -77.4% 6169710 ± 10% perf-stat.i.node-load-misses
62551 ± 32% -48.1% 32461 ± 8% perf-stat.i.node-loads
11040548 ± 33% -71.9% 3106135 ± 10% perf-stat.i.node-store-misses
5967407 ± 30% -73.3% 1595763 ± 8% perf-stat.i.node-stores
2704 ± 2% +19.3% 3226 perf-stat.i.page-faults
12.46 +4.8% 13.06 perf-stat.overall.MPKI
3.43 ± 33% -2.3 1.11 ± 9% perf-stat.overall.cache-miss-rate%
1.97 +21.1% 2.38 perf-stat.overall.cpi
5286 ± 39% +214.3% 16616 ± 9% perf-stat.overall.cycles-between-cache-misses
0.08 +0.1 0.14 perf-stat.overall.dTLB-load-miss-rate%
0.11 +0.1 0.18 perf-stat.overall.dTLB-store-miss-rate%
94.83 -8.7 86.12 perf-stat.overall.iTLB-load-miss-rate%
4319 ± 12% -41.2% 2541 ± 6% perf-stat.overall.instructions-per-iTLB-miss
0.51 -17.4% 0.42 perf-stat.overall.ipc
96209 -40.8% 56931 perf-stat.overall.path-length
2.488e+10 -18.8% 2.02e+10 perf-stat.ps.branch-instructions
4.064e+08 -18.1% 3.328e+08 perf-stat.ps.branch-misses
52646103 ± 33% -72.2% 14660030 ± 9% perf-stat.ps.cache-misses
1.54e+09 -14.0% 1.325e+09 perf-stat.ps.cache-references
18325407 -50.1% 9138511 perf-stat.ps.context-switches
5212308 -76.8% 1211237 perf-stat.ps.cpu-migrations
29418755 +51.7% 44633091 perf-stat.ps.dTLB-load-misses
3.689e+10 -15.9% 3.103e+10 perf-stat.ps.dTLB-loads
25834588 +39.4% 36003348 perf-stat.ps.dTLB-store-misses
2.341e+10 -14.2% 2.009e+10 perf-stat.ps.dTLB-stores
29114771 ± 14% +37.6% 40070432 ± 6% perf-stat.ps.iTLB-load-misses
1557519 ± 6% +312.6% 6426824 ± 6% perf-stat.ps.iTLB-loads
1.236e+11 -17.9% 1.015e+11 perf-stat.ps.instructions
2700 ± 2% +19.3% 3221 perf-stat.ps.minor-faults
27251553 ± 32% -77.4% 6160705 ± 10% perf-stat.ps.node-load-misses
62463 ± 32% -48.1% 32419 ± 8% perf-stat.ps.node-loads
11023464 ± 33% -71.9% 3101650 ± 10% perf-stat.ps.node-store-misses
5958217 ± 30% -73.3% 1593414 ± 8% perf-stat.ps.node-stores
2700 ± 2% +19.3% 3221 perf-stat.ps.page-faults
8.128e+13 -18.6% 6.613e+13 perf-stat.total.instructions
490.50 ± 12% +98.8% 975.25 ± 51% interrupts.37:PCI-MSI.3145734-edge.eth0-TxRx-5
16474141 ± 10% +39.7% 23012664 ± 7% interrupts.CAL:Function_call_interrupts
185660 ± 10% +41.1% 262033 ± 7% interrupts.CPU0.CAL:Function_call_interrupts
189509 ± 10% +41.9% 268861 ± 7% interrupts.CPU0.TLB:TLB_shootdowns
187182 ± 9% +41.1% 264126 ± 7% interrupts.CPU1.CAL:Function_call_interrupts
190168 ± 10% +41.7% 269487 ± 7% interrupts.CPU1.TLB:TLB_shootdowns
189291 ± 10% +39.0% 263058 ± 8% interrupts.CPU10.CAL:Function_call_interrupts
190089 ± 10% +41.7% 269340 ± 7% interrupts.CPU10.TLB:TLB_shootdowns
188189 ± 11% +39.1% 261746 ± 7% interrupts.CPU11.CAL:Function_call_interrupts
189646 ± 10% +42.1% 269507 ± 7% interrupts.CPU11.TLB:TLB_shootdowns
188222 ± 10% +40.0% 263590 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
189537 ± 10% +42.0% 269122 ± 7% interrupts.CPU12.TLB:TLB_shootdowns
187003 ± 10% +40.2% 262217 ± 8% interrupts.CPU13.CAL:Function_call_interrupts
189617 ± 10% +41.9% 269011 ± 7% interrupts.CPU13.TLB:TLB_shootdowns
186764 ± 10% +40.5% 262329 ± 8% interrupts.CPU14.CAL:Function_call_interrupts
189502 ± 10% +42.1% 269234 ± 7% interrupts.CPU14.TLB:TLB_shootdowns
187729 ± 10% +39.8% 262381 ± 7% interrupts.CPU15.CAL:Function_call_interrupts
189737 ± 10% +41.9% 269168 ± 7% interrupts.CPU15.TLB:TLB_shootdowns
490.50 ± 12% +98.8% 975.25 ± 51% interrupts.CPU16.37:PCI-MSI.3145734-edge.eth0-TxRx-5
186559 ± 10% +41.2% 263402 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
189809 ± 10% +41.8% 269101 ± 7% interrupts.CPU16.TLB:TLB_shootdowns
187061 ± 10% +41.2% 264113 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
189580 ± 10% +42.1% 269333 ± 7% interrupts.CPU17.TLB:TLB_shootdowns
187767 ± 10% +39.0% 261069 ± 7% interrupts.CPU18.CAL:Function_call_interrupts
189733 ± 10% +41.7% 268865 ± 7% interrupts.CPU18.TLB:TLB_shootdowns
187204 ± 10% +40.6% 263151 ± 7% interrupts.CPU19.CAL:Function_call_interrupts
189550 ± 10% +42.1% 269322 ± 7% interrupts.CPU19.TLB:TLB_shootdowns
188996 ± 10% +40.0% 264674 ± 8% interrupts.CPU2.CAL:Function_call_interrupts
189754 ± 10% +41.8% 269142 ± 8% interrupts.CPU2.TLB:TLB_shootdowns
187467 ± 10% +39.4% 261241 ± 6% interrupts.CPU20.CAL:Function_call_interrupts
189670 ± 10% +42.3% 269883 ± 7% interrupts.CPU20.TLB:TLB_shootdowns
188393 ± 11% +38.6% 261152 ± 6% interrupts.CPU21.CAL:Function_call_interrupts
189965 ± 10% +41.8% 269459 ± 7% interrupts.CPU21.TLB:TLB_shootdowns
186052 ± 10% +41.8% 263742 ± 7% interrupts.CPU22.CAL:Function_call_interrupts
6934924 ± 5% +11.3% 7718927 interrupts.CPU22.RES:Rescheduling_interrupts
189793 ± 10% +42.0% 269562 ± 7% interrupts.CPU22.TLB:TLB_shootdowns
188346 ± 9% +39.2% 262116 ± 7% interrupts.CPU23.CAL:Function_call_interrupts
6546095 ± 8% +16.3% 7613369 interrupts.CPU23.RES:Rescheduling_interrupts
190046 ± 10% +41.7% 269303 ± 7% interrupts.CPU23.TLB:TLB_shootdowns
188184 ± 10% +39.6% 262678 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
189902 ± 10% +41.9% 269396 ± 7% interrupts.CPU24.TLB:TLB_shootdowns
186200 ± 10% +40.4% 261509 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
6497143 ± 8% +18.9% 7722112 interrupts.CPU25.RES:Rescheduling_interrupts
189538 ± 10% +42.1% 269402 ± 7% interrupts.CPU25.TLB:TLB_shootdowns
186595 ± 10% +40.2% 261684 ± 7% interrupts.CPU26.CAL:Function_call_interrupts
6795717 ± 4% +14.0% 7746610 interrupts.CPU26.RES:Rescheduling_interrupts
189654 ± 10% +41.9% 269067 ± 7% interrupts.CPU26.TLB:TLB_shootdowns
187848 ± 10% +39.6% 262212 ± 7% interrupts.CPU27.CAL:Function_call_interrupts
189783 ± 10% +41.6% 268664 ± 7% interrupts.CPU27.TLB:TLB_shootdowns
187513 ± 9% +39.8% 262167 ± 7% interrupts.CPU28.CAL:Function_call_interrupts
6183467 ± 5% +23.8% 7654743 interrupts.CPU28.RES:Rescheduling_interrupts
189924 ± 10% +41.8% 269315 ± 7% interrupts.CPU28.TLB:TLB_shootdowns
188776 ± 10% +37.7% 259964 ± 8% interrupts.CPU29.CAL:Function_call_interrupts
189756 ± 10% +41.7% 268928 ± 7% interrupts.CPU29.TLB:TLB_shootdowns
188607 ± 10% +39.7% 263417 ± 7% interrupts.CPU3.CAL:Function_call_interrupts
6801914 ± 6% +13.3% 7708260 interrupts.CPU3.RES:Rescheduling_interrupts
189645 ± 10% +42.2% 269719 ± 7% interrupts.CPU3.TLB:TLB_shootdowns
187006 ± 9% +39.3% 260548 ± 7% interrupts.CPU30.CAL:Function_call_interrupts
6669372 +16.4% 7765155 interrupts.CPU30.RES:Rescheduling_interrupts
189851 ± 10% +41.4% 268418 ± 7% interrupts.CPU30.TLB:TLB_shootdowns
186701 ± 10% +39.2% 259893 ± 7% interrupts.CPU31.CAL:Function_call_interrupts
190056 ± 10% +41.3% 268560 ± 7% interrupts.CPU31.TLB:TLB_shootdowns
187241 ± 9% +39.5% 261162 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
6856900 ± 6% +13.8% 7804889 interrupts.CPU32.RES:Rescheduling_interrupts
189966 ± 10% +41.6% 268900 ± 7% interrupts.CPU32.TLB:TLB_shootdowns
188189 ± 10% +38.8% 261228 ± 7% interrupts.CPU33.CAL:Function_call_interrupts
6814724 ± 4% +14.2% 7779751 interrupts.CPU33.RES:Rescheduling_interrupts
190123 ± 10% +41.2% 268463 ± 8% interrupts.CPU33.TLB:TLB_shootdowns
188667 ± 10% +38.1% 260474 ± 8% interrupts.CPU34.CAL:Function_call_interrupts
6774727 ± 4% +14.1% 7732284 interrupts.CPU34.RES:Rescheduling_interrupts
190013 ± 10% +41.4% 268584 ± 7% interrupts.CPU34.TLB:TLB_shootdowns
186808 ± 10% +40.4% 262270 ± 7% interrupts.CPU35.CAL:Function_call_interrupts
6986395 ± 4% +11.5% 7792318 interrupts.CPU35.RES:Rescheduling_interrupts
189882 ± 10% +41.8% 269189 ± 7% interrupts.CPU35.TLB:TLB_shootdowns
187111 ± 9% +40.0% 262039 ± 7% interrupts.CPU36.CAL:Function_call_interrupts
6567116 ± 8% +16.7% 7663157 interrupts.CPU36.RES:Rescheduling_interrupts
189684 ± 10% +42.0% 269347 ± 7% interrupts.CPU36.TLB:TLB_shootdowns
186575 ± 9% +40.0% 261162 ± 7% interrupts.CPU37.CAL:Function_call_interrupts
190189 ± 10% +41.3% 268767 ± 7% interrupts.CPU37.TLB:TLB_shootdowns
187904 ± 10% +39.4% 261988 ± 8% interrupts.CPU38.CAL:Function_call_interrupts
190222 ± 10% +41.5% 269225 ± 7% interrupts.CPU38.TLB:TLB_shootdowns
186902 ± 9% +39.6% 260903 ± 8% interrupts.CPU39.CAL:Function_call_interrupts
6855278 ± 3% +12.6% 7717686 interrupts.CPU39.RES:Rescheduling_interrupts
190296 ± 10% +41.6% 269450 ± 7% interrupts.CPU39.TLB:TLB_shootdowns
187756 ± 10% +38.8% 260650 ± 6% interrupts.CPU4.CAL:Function_call_interrupts
189547 ± 10% +42.0% 269068 ± 7% interrupts.CPU4.TLB:TLB_shootdowns
188116 ± 10% +39.1% 261702 ± 8% interrupts.CPU40.CAL:Function_call_interrupts
189905 ± 10% +41.4% 268536 ± 7% interrupts.CPU40.TLB:TLB_shootdowns
186869 ± 9% +40.3% 262105 ± 7% interrupts.CPU41.CAL:Function_call_interrupts
6907024 ± 5% +12.4% 7761868 interrupts.CPU41.RES:Rescheduling_interrupts
189929 ± 10% +41.9% 269585 ± 7% interrupts.CPU41.TLB:TLB_shootdowns
187984 ± 10% +38.7% 260814 ± 7% interrupts.CPU42.CAL:Function_call_interrupts
6536550 ± 5% +16.3% 7602630 interrupts.CPU42.RES:Rescheduling_interrupts
190333 ± 10% +41.5% 269246 ± 7% interrupts.CPU42.TLB:TLB_shootdowns
186377 ± 10% +39.7% 260366 ± 8% interrupts.CPU43.CAL:Function_call_interrupts
189831 ± 10% +41.8% 269237 ± 8% interrupts.CPU43.TLB:TLB_shootdowns
185932 ± 11% +40.7% 261695 ± 7% interrupts.CPU44.CAL:Function_call_interrupts
189445 ± 10% +42.4% 269745 ± 7% interrupts.CPU44.TLB:TLB_shootdowns
186413 ± 10% +39.8% 260697 ± 8% interrupts.CPU45.CAL:Function_call_interrupts
6637836 ± 6% +14.4% 7592686 interrupts.CPU45.RES:Rescheduling_interrupts
189543 ± 10% +42.4% 269972 ± 7% interrupts.CPU45.TLB:TLB_shootdowns
187316 ± 10% +40.3% 262840 ± 7% interrupts.CPU46.CAL:Function_call_interrupts
6729052 ± 2% +12.8% 7587106 interrupts.CPU46.RES:Rescheduling_interrupts
189577 ± 10% +42.1% 269444 ± 7% interrupts.CPU46.TLB:TLB_shootdowns
187894 ± 11% +39.6% 262328 ± 7% interrupts.CPU47.CAL:Function_call_interrupts
6760980 ± 3% +13.7% 7683855 interrupts.CPU47.RES:Rescheduling_interrupts
189659 ± 10% +42.0% 269290 ± 7% interrupts.CPU47.TLB:TLB_shootdowns
186440 ± 10% +40.2% 261462 ± 7% interrupts.CPU48.CAL:Function_call_interrupts
189687 ± 10% +41.9% 269250 ± 7% interrupts.CPU48.TLB:TLB_shootdowns
187784 ± 10% +39.3% 261603 ± 7% interrupts.CPU49.CAL:Function_call_interrupts
189657 ± 10% +41.9% 269071 ± 7% interrupts.CPU49.TLB:TLB_shootdowns
187793 ± 10% +39.9% 262799 ± 7% interrupts.CPU5.CAL:Function_call_interrupts
189926 ± 10% +41.7% 269054 ± 7% interrupts.CPU5.TLB:TLB_shootdowns
187306 ± 10% +40.0% 262163 ± 7% interrupts.CPU50.CAL:Function_call_interrupts
189718 ± 10% +42.0% 269348 ± 8% interrupts.CPU50.TLB:TLB_shootdowns
187409 ± 10% +39.9% 262099 ± 7% interrupts.CPU51.CAL:Function_call_interrupts
189503 ± 10% +42.3% 269747 ± 7% interrupts.CPU51.TLB:TLB_shootdowns
186596 ± 10% +39.3% 259957 ± 8% interrupts.CPU52.CAL:Function_call_interrupts
189968 ± 10% +42.0% 269738 ± 7% interrupts.CPU52.TLB:TLB_shootdowns
188014 ± 11% +38.5% 260416 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
189876 ± 10% +41.9% 269512 ± 7% interrupts.CPU53.TLB:TLB_shootdowns
187841 ± 10% +38.6% 260276 ± 7% interrupts.CPU54.CAL:Function_call_interrupts
189553 ± 10% +42.4% 270000 ± 7% interrupts.CPU54.TLB:TLB_shootdowns
187783 ± 10% +38.9% 260744 ± 7% interrupts.CPU55.CAL:Function_call_interrupts
189892 ± 10% +41.9% 269509 ± 7% interrupts.CPU55.TLB:TLB_shootdowns
188178 ± 10% +39.1% 261808 ± 7% interrupts.CPU56.CAL:Function_call_interrupts
189633 ± 10% +42.2% 269747 ± 7% interrupts.CPU56.TLB:TLB_shootdowns
187735 ± 10% +38.5% 260067 ± 7% interrupts.CPU57.CAL:Function_call_interrupts
189727 ± 10% +41.9% 269257 ± 7% interrupts.CPU57.TLB:TLB_shootdowns
185453 ± 10% +41.2% 261927 ± 8% interrupts.CPU58.CAL:Function_call_interrupts
189558 ± 10% +42.2% 269481 ± 7% interrupts.CPU58.TLB:TLB_shootdowns
186606 ± 10% +40.4% 261905 ± 8% interrupts.CPU59.CAL:Function_call_interrupts
189562 ± 10% +42.4% 269902 ± 7% interrupts.CPU59.TLB:TLB_shootdowns
188335 ± 10% +39.9% 263480 ± 7% interrupts.CPU6.CAL:Function_call_interrupts
189762 ± 10% +42.1% 269682 ± 7% interrupts.CPU6.TLB:TLB_shootdowns
186930 ± 10% +40.5% 262544 ± 7% interrupts.CPU60.CAL:Function_call_interrupts
189553 ± 10% +42.1% 269407 ± 7% interrupts.CPU60.TLB:TLB_shootdowns
187926 ± 10% +39.6% 262285 ± 7% interrupts.CPU61.CAL:Function_call_interrupts
6730913 ± 3% +13.7% 7651297 interrupts.CPU61.RES:Rescheduling_interrupts
189613 ± 10% +42.2% 269688 ± 7% interrupts.CPU61.TLB:TLB_shootdowns
186893 ± 10% +39.3% 260310 ± 7% interrupts.CPU62.CAL:Function_call_interrupts
189639 ± 10% +42.3% 269775 ± 7% interrupts.CPU62.TLB:TLB_shootdowns
188020 ± 10% +38.5% 260347 ± 7% interrupts.CPU63.CAL:Function_call_interrupts
189913 ± 10% +41.8% 269272 ± 7% interrupts.CPU63.TLB:TLB_shootdowns
186999 ± 10% +40.5% 262720 ± 8% interrupts.CPU64.CAL:Function_call_interrupts
189860 ± 10% +42.2% 270040 ± 7% interrupts.CPU64.TLB:TLB_shootdowns
186551 ± 11% +39.8% 260855 ± 7% interrupts.CPU65.CAL:Function_call_interrupts
189603 ± 10% +42.0% 269303 ± 7% interrupts.CPU65.TLB:TLB_shootdowns
187181 ± 10% +40.0% 262007 ± 6% interrupts.CPU66.CAL:Function_call_interrupts
6774119 ± 4% +13.4% 7683579 interrupts.CPU66.RES:Rescheduling_interrupts
190052 ± 10% +42.0% 269866 ± 7% interrupts.CPU66.TLB:TLB_shootdowns
184308 ± 9% +41.3% 260494 ± 6% interrupts.CPU67.CAL:Function_call_interrupts
6536979 ± 5% +15.7% 7563567 interrupts.CPU67.RES:Rescheduling_interrupts
189912 ± 10% +41.8% 269351 ± 7% interrupts.CPU67.TLB:TLB_shootdowns
186698 ± 10% +39.8% 260911 ± 8% interrupts.CPU68.CAL:Function_call_interrupts
190025 ± 10% +41.7% 269261 ± 7% interrupts.CPU68.TLB:TLB_shootdowns
186568 ± 10% +39.5% 260248 ± 7% interrupts.CPU69.CAL:Function_call_interrupts
190301 ± 10% +41.5% 269195 ± 7% interrupts.CPU69.TLB:TLB_shootdowns
188066 ± 10% +39.8% 262970 ± 7% interrupts.CPU7.CAL:Function_call_interrupts
6686727 ± 3% +13.8% 7609250 interrupts.CPU7.RES:Rescheduling_interrupts
189753 ± 10% +41.8% 269087 ± 7% interrupts.CPU7.TLB:TLB_shootdowns
185801 ± 9% +40.7% 261431 ± 7% interrupts.CPU70.CAL:Function_call_interrupts
190198 ± 10% +41.5% 269187 ± 8% interrupts.CPU70.TLB:TLB_shootdowns
187946 ± 9% +39.2% 261535 ± 7% interrupts.CPU71.CAL:Function_call_interrupts
6847010 ± 6% +12.3% 7692468 interrupts.CPU71.RES:Rescheduling_interrupts
190179 ± 10% +41.7% 269573 ± 7% interrupts.CPU71.TLB:TLB_shootdowns
187066 ± 10% +39.9% 261612 ± 7% interrupts.CPU72.CAL:Function_call_interrupts
6175738 ± 6% +23.3% 7614317 interrupts.CPU72.RES:Rescheduling_interrupts
190339 ± 10% +41.6% 269549 ± 7% interrupts.CPU72.TLB:TLB_shootdowns
187352 ± 10% +38.9% 260308 ± 7% interrupts.CPU73.CAL:Function_call_interrupts
189955 ± 10% +41.8% 269345 ± 7% interrupts.CPU73.TLB:TLB_shootdowns
186529 ± 9% +39.9% 260973 ± 7% interrupts.CPU74.CAL:Function_call_interrupts
6572114 ± 3% +17.6% 7729332 interrupts.CPU74.RES:Rescheduling_interrupts
190217 ± 10% +41.6% 269435 ± 7% interrupts.CPU74.TLB:TLB_shootdowns
186985 ± 10% +39.7% 261161 ± 8% interrupts.CPU75.CAL:Function_call_interrupts
190217 ± 10% +41.6% 269429 ± 7% interrupts.CPU75.TLB:TLB_shootdowns
187445 ± 10% +38.8% 260221 ± 7% interrupts.CPU76.CAL:Function_call_interrupts
190196 ± 10% +41.3% 268838 ± 7% interrupts.CPU76.TLB:TLB_shootdowns
186706 ± 10% +39.3% 260027 ± 7% interrupts.CPU77.CAL:Function_call_interrupts
6956600 ± 4% +11.2% 7738923 interrupts.CPU77.RES:Rescheduling_interrupts
190364 ± 10% +41.2% 268756 ± 7% interrupts.CPU77.TLB:TLB_shootdowns
187454 ± 10% +39.6% 261639 ± 8% interrupts.CPU78.CAL:Function_call_interrupts
6827966 ± 6% +12.3% 7666413 interrupts.CPU78.RES:Rescheduling_interrupts
190019 ± 10% +42.0% 269837 ± 7% interrupts.CPU78.TLB:TLB_shootdowns
185795 ± 9% +40.9% 261817 ± 7% interrupts.CPU79.CAL:Function_call_interrupts
7025236 ± 2% +9.8% 7715577 interrupts.CPU79.RES:Rescheduling_interrupts
190119 ± 10% +41.7% 269424 ± 7% interrupts.CPU79.TLB:TLB_shootdowns
186232 ± 10% +40.5% 261659 ± 8% interrupts.CPU8.CAL:Function_call_interrupts
189688 ± 10% +41.9% 269162 ± 8% interrupts.CPU8.TLB:TLB_shootdowns
186850 ± 10% +39.2% 260150 ± 6% interrupts.CPU80.CAL:Function_call_interrupts
6628028 ± 5% +15.7% 7669046 interrupts.CPU80.RES:Rescheduling_interrupts
189943 ± 10% +41.9% 269603 ± 7% interrupts.CPU80.TLB:TLB_shootdowns
185980 ± 9% +39.3% 259080 ± 7% interrupts.CPU81.CAL:Function_call_interrupts
6545523 ± 6% +15.6% 7567357 interrupts.CPU81.RES:Rescheduling_interrupts
190140 ± 10% +42.0% 269964 ± 7% interrupts.CPU81.TLB:TLB_shootdowns
185975 ± 10% +39.4% 259324 ± 7% interrupts.CPU82.CAL:Function_call_interrupts
189806 ± 10% +41.9% 269365 ± 7% interrupts.CPU82.TLB:TLB_shootdowns
186694 ± 11% +39.7% 260791 ± 7% interrupts.CPU83.CAL:Function_call_interrupts
6880565 ± 2% +11.8% 7694808 interrupts.CPU83.RES:Rescheduling_interrupts
190196 ± 10% +41.6% 269226 ± 7% interrupts.CPU83.TLB:TLB_shootdowns
186842 ± 10% +38.8% 259332 ± 6% interrupts.CPU84.CAL:Function_call_interrupts
189948 ± 10% +41.9% 269512 ± 7% interrupts.CPU84.TLB:TLB_shootdowns
186489 ± 9% +39.2% 259519 ± 8% interrupts.CPU85.CAL:Function_call_interrupts
189929 ± 10% +41.7% 269130 ± 7% interrupts.CPU85.TLB:TLB_shootdowns
186281 ± 10% +39.9% 260515 ± 7% interrupts.CPU86.CAL:Function_call_interrupts
6685741 ± 6% +14.5% 7657812 interrupts.CPU86.RES:Rescheduling_interrupts
190291 ± 10% +41.6% 269519 ± 7% interrupts.CPU86.TLB:TLB_shootdowns
185945 ± 10% +39.3% 259039 ± 8% interrupts.CPU87.CAL:Function_call_interrupts
6930771 ± 4% +10.2% 7635153 interrupts.CPU87.RES:Rescheduling_interrupts
189743 ± 10% +42.0% 269524 ± 7% interrupts.CPU87.TLB:TLB_shootdowns
188961 ± 10% +38.4% 261469 ± 7% interrupts.CPU9.CAL:Function_call_interrupts
189992 ± 10% +41.6% 269025 ± 7% interrupts.CPU9.TLB:TLB_shootdowns
6.037e+08 ± 3% +11.8% 6.748e+08 interrupts.RES:Rescheduling_interrupts
16707718 ± 10% +41.9% 23700193 ± 7% interrupts.TLB:TLB_shootdowns
101283 -19.8% 81191 softirqs.CPU0.RCU
11853 ± 8% +314.6% 49137 softirqs.CPU0.SCHED
238901 ± 2% -12.7% 208676 softirqs.CPU0.TIMER
102701 -19.2% 82944 ± 2% softirqs.CPU1.RCU
9387 ± 6% +348.8% 42131 softirqs.CPU1.SCHED
238227 ± 3% -12.9% 207451 softirqs.CPU1.TIMER
102858 ± 2% -19.3% 82965 softirqs.CPU10.RCU
8879 ± 2% +383.6% 42934 ± 2% softirqs.CPU10.SCHED
233517 ± 2% -12.3% 204784 softirqs.CPU10.TIMER
102524 ± 3% -19.2% 82888 ± 3% softirqs.CPU11.RCU
9026 ± 3% +376.8% 43035 softirqs.CPU11.SCHED
232053 ± 2% -11.2% 205963 softirqs.CPU11.TIMER
103308 ± 3% -20.5% 82126 softirqs.CPU12.RCU
8931 ± 5% +388.9% 43663 softirqs.CPU12.SCHED
230603 ± 2% -11.1% 204928 softirqs.CPU12.TIMER
102998 ± 2% -20.3% 82061 softirqs.CPU13.RCU
8907 +383.0% 43018 softirqs.CPU13.SCHED
230896 ± 2% -11.3% 204812 softirqs.CPU13.TIMER
105301 ± 2% -21.2% 83011 softirqs.CPU14.RCU
8714 ± 2% +387.5% 42487 softirqs.CPU14.SCHED
235967 ± 2% -12.2% 207100 softirqs.CPU14.TIMER
108551 ± 3% -20.2% 86614 softirqs.CPU15.RCU
8675 ± 2% +395.8% 43013 softirqs.CPU15.SCHED
237481 ± 2% -12.5% 207789 softirqs.CPU15.TIMER
109959 ± 2% -21.5% 86340 softirqs.CPU16.RCU
8770 ± 2% +391.7% 43121 softirqs.CPU16.SCHED
234140 -11.9% 206176 softirqs.CPU16.TIMER
108751 -21.0% 85866 softirqs.CPU17.RCU
8876 ± 2% +388.6% 43370 softirqs.CPU17.SCHED
232850 ± 2% -12.0% 205019 softirqs.CPU17.TIMER
109252 ± 4% -21.0% 86256 softirqs.CPU18.RCU
8729 ± 2% +398.7% 43536 softirqs.CPU18.SCHED
233877 -12.1% 205521 softirqs.CPU18.TIMER
110200 ± 3% -21.4% 86583 softirqs.CPU19.RCU
8977 ± 3% +382.8% 43344 ± 2% softirqs.CPU19.SCHED
235458 ± 2% -12.4% 206304 softirqs.CPU19.TIMER
104633 -21.1% 82530 softirqs.CPU2.RCU
9169 ± 3% +360.4% 42214 softirqs.CPU2.SCHED
237928 ± 2% -12.5% 208278 softirqs.CPU2.TIMER
109731 -21.3% 86317 softirqs.CPU20.RCU
8981 ± 4% +369.1% 42130 softirqs.CPU20.SCHED
236141 -12.7% 206254 softirqs.CPU20.TIMER
107025 ± 3% -20.7% 84924 softirqs.CPU21.RCU
8727 ± 4% +388.0% 42587 softirqs.CPU21.SCHED
241761 ± 4% -16.3% 202378 softirqs.CPU21.TIMER
112121 ± 2% -22.1% 87383 softirqs.CPU22.RCU
9562 ± 4% +354.0% 43407 ± 2% softirqs.CPU22.SCHED
240291 ± 2% -13.6% 207532 ± 2% softirqs.CPU22.TIMER
111418 ± 3% -21.5% 87420 softirqs.CPU23.RCU
8689 +387.7% 42381 ± 2% softirqs.CPU23.SCHED
237522 -12.8% 207107 softirqs.CPU23.TIMER
110078 ± 2% -21.0% 86947 softirqs.CPU24.RCU
8646 ± 4% +400.2% 43246 softirqs.CPU24.SCHED
235643 -12.8% 205575 softirqs.CPU24.TIMER
109680 ± 2% -20.8% 86906 softirqs.CPU25.RCU
8654 ± 2% +401.4% 43390 softirqs.CPU25.SCHED
232501 -12.2% 204209 softirqs.CPU25.TIMER
110697 ± 3% -21.0% 87462 softirqs.CPU26.RCU
8634 ± 4% +408.2% 43878 softirqs.CPU26.SCHED
262016 ± 16% -21.8% 204957 softirqs.CPU26.TIMER
109232 ± 3% -20.5% 86870 softirqs.CPU27.RCU
8607 +407.1% 43645 softirqs.CPU27.SCHED
234570 ± 2% -13.0% 204176 softirqs.CPU27.TIMER
110108 ± 3% -17.8% 90530 ± 2% softirqs.CPU28.RCU
8257 ± 2% +419.7% 42915 softirqs.CPU28.SCHED
236394 ± 2% -11.7% 208692 softirqs.CPU28.TIMER
111788 ± 2% -22.0% 87170 softirqs.CPU29.RCU
8830 ± 2% +384.8% 42810 softirqs.CPU29.SCHED
235673 -12.7% 205771 softirqs.CPU29.TIMER
101667 ± 3% -19.2% 82143 softirqs.CPU3.RCU
8693 ± 2% +393.9% 42936 softirqs.CPU3.SCHED
235633 -12.6% 206009 softirqs.CPU3.TIMER
105973 ± 2% -20.3% 84499 softirqs.CPU30.RCU
8472 ± 3% +418.5% 43930 softirqs.CPU30.SCHED
237032 -13.5% 205051 softirqs.CPU30.TIMER
104288 ± 5% -18.4% 85067 softirqs.CPU31.RCU
8679 ± 3% +400.4% 43435 softirqs.CPU31.SCHED
239331 ± 3% -14.5% 204718 softirqs.CPU31.TIMER
107453 -21.1% 84770 softirqs.CPU32.RCU
8841 +400.4% 44243 softirqs.CPU32.SCHED
233118 -12.3% 204541 softirqs.CPU32.TIMER
106953 -20.6% 84917 softirqs.CPU33.RCU
8652 +406.8% 43855 softirqs.CPU33.SCHED
232875 -12.2% 204540 softirqs.CPU33.TIMER
107322 ± 2% -21.1% 84716 softirqs.CPU34.RCU
8742 +399.8% 43696 softirqs.CPU34.SCHED
236213 ± 2% -13.1% 205201 softirqs.CPU34.TIMER
105872 ± 2% -20.4% 84252 softirqs.CPU35.RCU
8813 ± 3% +403.5% 44375 softirqs.CPU35.SCHED
233189 -12.5% 204126 softirqs.CPU35.TIMER
107494 ± 2% -20.9% 85066 softirqs.CPU36.RCU
8580 +403.5% 43204 softirqs.CPU36.SCHED
238265 -13.3% 206537 softirqs.CPU36.TIMER
108862 ± 2% -22.2% 84697 ± 3% softirqs.CPU37.RCU
8500 +402.9% 42744 softirqs.CPU37.SCHED
239677 -14.3% 205419 softirqs.CPU37.TIMER
110045 ± 2% -22.5% 85319 softirqs.CPU38.RCU
8827 ± 2% +392.4% 43470 softirqs.CPU38.SCHED
236783 ± 2% -13.0% 206044 softirqs.CPU38.TIMER
105613 ± 3% -19.8% 84693 softirqs.CPU39.RCU
8656 ± 2% +401.4% 43407 softirqs.CPU39.SCHED
236319 ± 2% -12.9% 205765 softirqs.CPU39.TIMER
101364 ± 2% -18.8% 82281 softirqs.CPU4.RCU
8651 +392.6% 42621 softirqs.CPU4.SCHED
236082 -12.8% 205856 softirqs.CPU4.TIMER
105108 ± 2% -19.8% 84325 softirqs.CPU40.RCU
8609 ± 4% +413.6% 44218 softirqs.CPU40.SCHED
236557 -13.6% 204434 softirqs.CPU40.TIMER
106578 ± 2% -20.9% 84297 softirqs.CPU41.RCU
8908 ± 4% +396.0% 44186 softirqs.CPU41.SCHED
232468 -12.2% 204217 softirqs.CPU41.TIMER
109643 ± 2% -21.9% 85597 softirqs.CPU42.RCU
8562 +402.0% 42985 softirqs.CPU42.SCHED
238955 ± 2% -13.4% 206872 softirqs.CPU42.TIMER
106460 -20.0% 85165 softirqs.CPU43.RCU
8637 ± 3% +397.7% 42983 softirqs.CPU43.SCHED
243180 ± 5% -15.5% 205490 softirqs.CPU43.TIMER
89578 ± 2% -17.6% 73847 softirqs.CPU44.RCU
8345 +412.3% 42750 softirqs.CPU44.SCHED
236264 -13.1% 205205 softirqs.CPU44.TIMER
108752 ± 2% -23.1% 83663 ± 2% softirqs.CPU45.RCU
8731 ± 4% +374.7% 41452 softirqs.CPU45.SCHED
237148 ± 2% -13.3% 205556 softirqs.CPU45.TIMER
105428 ± 3% -20.5% 83804 softirqs.CPU46.RCU
8732 ± 2% +379.2% 41843 softirqs.CPU46.SCHED
236162 ± 2% -12.6% 206490 softirqs.CPU46.TIMER
105152 ± 2% -21.1% 82948 softirqs.CPU47.RCU
8772 ± 3% +386.3% 42659 softirqs.CPU47.SCHED
232707 ± 2% -12.1% 204504 softirqs.CPU47.TIMER
103331 ± 2% -19.6% 83068 softirqs.CPU48.RCU
8620 +392.1% 42418 softirqs.CPU48.SCHED
234790 -12.8% 204849 softirqs.CPU48.TIMER
103185 ± 2% -19.5% 83047 softirqs.CPU49.RCU
8661 +394.1% 42796 ± 2% softirqs.CPU49.SCHED
233237 -12.5% 203988 softirqs.CPU49.TIMER
102218 ± 2% -19.7% 82034 softirqs.CPU5.RCU
8986 ± 5% +373.2% 42524 softirqs.CPU5.SCHED
233852 -12.4% 204890 softirqs.CPU5.TIMER
104885 -19.6% 84308 softirqs.CPU50.RCU
8803 +387.5% 42920 softirqs.CPU50.SCHED
234873 ± 2% -12.6% 205388 softirqs.CPU50.TIMER
104340 -19.7% 83785 ± 2% softirqs.CPU51.RCU
8813 ± 3% +372.0% 41599 softirqs.CPU51.SCHED
236754 ± 3% -12.6% 206851 softirqs.CPU51.TIMER
104306 -20.0% 83490 ± 2% softirqs.CPU52.RCU
8842 ± 3% +378.1% 42277 softirqs.CPU52.SCHED
232847 -12.0% 204960 softirqs.CPU52.TIMER
104830 -20.7% 83116 softirqs.CPU53.RCU
9089 ± 3% +366.4% 42389 softirqs.CPU53.SCHED
234750 -12.7% 205002 softirqs.CPU53.TIMER
103807 -19.3% 83767 softirqs.CPU54.RCU
8783 ± 3% +386.6% 42742 softirqs.CPU54.SCHED
235163 ± 3% -12.8% 205154 softirqs.CPU54.TIMER
103753 -20.0% 83047 softirqs.CPU55.RCU
9000 ± 4% +378.0% 43022 softirqs.CPU55.SCHED
232577 ± 2% -11.9% 204799 softirqs.CPU55.TIMER
104260 -19.4% 84062 ± 2% softirqs.CPU56.RCU
8965 ± 3% +378.4% 42886 softirqs.CPU56.SCHED
232060 ± 2% -11.7% 204982 softirqs.CPU56.TIMER
104369 -20.5% 83025 softirqs.CPU57.RCU
8768 +388.0% 42792 ± 2% softirqs.CPU57.SCHED
230747 -11.5% 204212 softirqs.CPU57.TIMER
108289 ± 2% -23.0% 83359 softirqs.CPU58.RCU
8868 ± 2% +380.7% 42634 softirqs.CPU58.SCHED
233974 -11.8% 206408 softirqs.CPU58.TIMER
108097 -22.4% 83875 softirqs.CPU59.RCU
8890 ± 3% +371.6% 41932 softirqs.CPU59.SCHED
235591 ± 2% -11.9% 207635 softirqs.CPU59.TIMER
103756 -20.3% 82730 softirqs.CPU6.RCU
8939 ± 2% +372.1% 42209 softirqs.CPU6.SCHED
236830 ± 2% -12.3% 207760 softirqs.CPU6.TIMER
107021 -21.1% 84461 softirqs.CPU60.RCU
8835 ± 3% +379.4% 42356 softirqs.CPU60.SCHED
234843 ± 2% -12.3% 206043 softirqs.CPU60.TIMER
105769 -20.7% 83906 softirqs.CPU61.RCU
8870 ± 2% +386.3% 43132 softirqs.CPU61.SCHED
231876 -11.9% 204222 softirqs.CPU61.TIMER
106095 -20.1% 84812 ± 2% softirqs.CPU62.RCU
8719 ± 2% +388.6% 42606 softirqs.CPU62.SCHED
234917 ± 2% -12.6% 205432 softirqs.CPU62.TIMER
107799 ± 2% -21.7% 84444 softirqs.CPU63.RCU
8870 ± 4% +386.7% 43173 ± 2% softirqs.CPU63.SCHED
232768 ± 2% -11.8% 205407 softirqs.CPU63.TIMER
107300 -21.4% 84335 softirqs.CPU64.RCU
8825 ± 3% +376.7% 42067 softirqs.CPU64.SCHED
234775 -12.3% 205914 softirqs.CPU64.TIMER
107228 -21.8% 83885 softirqs.CPU65.RCU
8639 ± 4% +393.6% 42643 softirqs.CPU65.SCHED
112813 -23.1% 86759 softirqs.CPU66.RCU
8669 +398.0% 43174 ± 2% softirqs.CPU66.SCHED
235000 ± 2% -12.4% 205817 softirqs.CPU66.TIMER
111265 -22.2% 86600 softirqs.CPU67.RCU
8447 ± 3% +400.2% 42249 ± 2% softirqs.CPU67.SCHED
238026 ± 2% -13.1% 206797 softirqs.CPU67.TIMER
110871 ± 2% -22.1% 86382 softirqs.CPU68.RCU
8580 ± 4% +399.7% 42871 softirqs.CPU68.SCHED
235353 -12.9% 204959 softirqs.CPU68.TIMER
109959 -21.4% 86395 softirqs.CPU69.RCU
8518 ± 3% +403.9% 42926 softirqs.CPU69.SCHED
233661 -12.5% 204461 softirqs.CPU69.TIMER
103670 -20.5% 82428 softirqs.CPU7.RCU
8910 ± 4% +371.6% 42024 softirqs.CPU7.SCHED
236061 ± 2% -12.4% 206743 softirqs.CPU7.TIMER
108070 ± 7% -20.2% 86259 softirqs.CPU70.RCU
8660 ± 5% +407.5% 43952 softirqs.CPU70.SCHED
239057 ± 2% -14.4% 204609 softirqs.CPU70.TIMER
110088 ± 2% -21.8% 86043 softirqs.CPU71.RCU
8803 ± 3% +388.6% 43010 ± 2% softirqs.CPU71.SCHED
233164 ± 2% -12.3% 204421 softirqs.CPU71.TIMER
109902 -21.2% 86647 softirqs.CPU72.RCU
8283 ± 3% +419.3% 43013 softirqs.CPU72.SCHED
236967 ± 2% -13.1% 205885 softirqs.CPU72.TIMER
111977 -22.8% 86472 softirqs.CPU73.RCU
8586 ± 4% +392.4% 42279 softirqs.CPU73.SCHED
234492 ± 2% -12.2% 205778 softirqs.CPU73.TIMER
109259 ± 2% -21.4% 85928 softirqs.CPU74.RCU
8347 ± 4% +422.1% 43585 softirqs.CPU74.SCHED
235505 -13.2% 204324 softirqs.CPU74.TIMER
94051 ± 3% -16.2% 78791 ± 2% softirqs.CPU75.RCU
8556 ± 3% +406.7% 43350 softirqs.CPU75.SCHED
263324 ± 19% -22.3% 204516 softirqs.CPU75.TIMER
96034 -18.8% 77960 softirqs.CPU76.RCU
8949 +390.1% 43861 softirqs.CPU76.SCHED
232727 -12.6% 203417 softirqs.CPU76.TIMER
95967 ± 2% -18.8% 77899 softirqs.CPU77.RCU
8642 ± 2% +406.8% 43803 softirqs.CPU77.SCHED
233207 -12.8% 203274 softirqs.CPU77.TIMER
96771 -19.4% 78031 softirqs.CPU78.RCU
8734 ± 2% +395.3% 43258 softirqs.CPU78.SCHED
235185 ± 2% -12.9% 204839 softirqs.CPU78.TIMER
94875 -17.5% 78246 softirqs.CPU79.RCU
8614 +409.8% 43915 softirqs.CPU79.SCHED
233477 -12.5% 204302 softirqs.CPU79.TIMER
103589 -20.5% 82382 softirqs.CPU8.RCU
8780 ± 2% +387.3% 42786 softirqs.CPU8.SCHED
234263 ± 2% -12.2% 205741 softirqs.CPU8.TIMER
97077 -18.5% 79076 softirqs.CPU80.RCU
8548 ± 2% +403.9% 43078 softirqs.CPU80.SCHED
234473 -12.6% 204821 softirqs.CPU80.TIMER
97954 ± 3% -19.4% 78925 ± 2% softirqs.CPU81.RCU
8635 ± 3% +398.5% 43051 softirqs.CPU81.SCHED
98169 -19.6% 78888 softirqs.CPU82.RCU
8561 +404.2% 43166 softirqs.CPU82.SCHED
237971 ± 2% -13.6% 205563 softirqs.CPU82.TIMER
95597 ± 2% -18.2% 78205 softirqs.CPU83.RCU
8628 ± 3% +402.3% 43338 softirqs.CPU83.SCHED
234210 -12.7% 204559 softirqs.CPU83.TIMER
96012 -18.2% 78507 ± 2% softirqs.CPU84.RCU
8671 ± 3% +403.9% 43694 ± 2% softirqs.CPU84.SCHED
232502 -12.5% 203438 softirqs.CPU84.TIMER
96003 -18.2% 78522 softirqs.CPU85.RCU
8903 ± 2% +382.6% 42965 softirqs.CPU85.SCHED
231986 -12.1% 203991 softirqs.CPU85.TIMER
98431 ± 2% -19.4% 79343 softirqs.CPU86.RCU
8540 ± 3% +403.8% 43020 softirqs.CPU86.SCHED
236887 -13.4% 205139 softirqs.CPU86.TIMER
95059 ± 3% -17.2% 78680 softirqs.CPU87.RCU
8611 ± 3% +394.7% 42602 softirqs.CPU87.SCHED
103978 -21.0% 82187 softirqs.CPU9.RCU
9033 ± 3% +372.4% 42672 softirqs.CPU9.SCHED
232215 ± 2% -11.6% 205224 softirqs.CPU9.TIMER
9245938 -20.4% 7357534 softirqs.RCU
772928 +390.2% 3789198 softirqs.SCHED
20816171 -12.5% 18217025 softirqs.TIMER
27.49 ± 4% -11.8 15.73 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
26.91 ± 5% -11.7 15.23 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
26.64 ± 5% -11.6 15.04 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
29.16 ± 4% -10.2 18.93 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
11.05 ± 16% -7.8 3.29 perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
8.96 ± 19% -6.7 2.27 perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
16.66 -5.9 10.80 ± 2% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
14.40 ± 2% -4.9 9.52 ± 3% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
14.05 ± 2% -4.7 9.35 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
5.54 ± 23% -4.5 1.03 perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
6.26 ± 2% -4.3 1.98 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.38 ± 3% -4.2 31.18 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
6.04 ± 2% -4.1 1.90 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.90 ± 3% -4.0 31.88 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.83 ± 2% -4.0 1.84 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
22.76 ± 2% -3.8 18.96 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
23.45 ± 2% -3.8 19.69 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
88.47 -3.6 84.84 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
87.65 -3.6 84.06 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
38.51 ± 2% -2.7 35.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.97 ± 2% -2.6 24.38 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.99 ± 4% -2.6 4.42 ± 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
8.67 ± 3% -2.4 6.26 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
8.58 ± 3% -2.4 6.21 ± 3% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
28.49 ± 2% -2.3 26.22 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.24 ± 3% -2.2 6.08 ± 3% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
2.82 ± 2% -1.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
40.45 ± 2% -1.4 39.09 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.01 ± 3% -1.2 0.79 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.79 ± 2% -1.0 1.83 ± 3% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
2.31 ± 2% -1.0 1.35 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
3.86 -0.9 2.93 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.70 -0.9 0.80 perf-profile.calltrace.cycles-pp.__switch_to_asm
1.06 ± 3% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.__switch_to
1.34 -0.7 0.62 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr
1.63 ± 2% -0.6 1.04 ± 3% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.34 ± 2% -0.5 0.82 ± 4% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.27 ± 2% -0.5 0.76 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.34 ± 17% -0.5 0.87 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
1.31 -0.5 0.85 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.90 ± 2% -0.4 0.54 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.95 ± 3% -0.3 0.60 ± 2% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.08 ± 2% -0.3 0.73 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.93 ± 2% -0.3 0.59 ± 4% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
1.17 ± 2% -0.3 0.83 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.83 ± 2% -0.2 0.60 ± 3% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.81 ± 2% -0.2 0.61 ± 4% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.54 ± 3% +0.1 0.64 ± 2% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read
0.61 ± 2% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write
0.81 +0.2 1.01 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.84 +0.2 1.05 ± 3% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.54 ± 2% +0.2 0.77 perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.65 ± 5% +0.2 0.89 perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read
0.58 ± 4% +0.3 0.90 ± 4% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.63 ± 5% +0.3 0.95 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.71 ± 4% +0.4 1.10 ± 2% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
0.42 ± 57% +0.4 0.84 ± 3% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
0.98 ± 3% +0.5 1.48 perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.00 +0.5 0.52 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +0.6 0.55 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.avc_has_perm.file_has_perm.security_file_permission.vfs_read.ksys_read
0.99 ± 4% +0.6 1.57 perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
0.00 +0.6 0.60 perf-profile.calltrace.cycles-pp.atime_needs_update.touch_atime.pipe_read.new_sync_read.vfs_read
0.13 ±173% +0.6 0.73 ± 4% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_read.do_syscall_64
0.00 +0.6 0.61 ± 3% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.80 ± 5% +0.6 1.42 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.10 ± 2% +0.6 1.73 perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
1.01 ± 2% +0.6 1.66 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.89 ± 5% +0.7 1.55 perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
1.26 ± 2% +0.7 1.94 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.29 ± 2% +0.7 1.98 perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.7 0.72 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait
0.00 +0.7 0.73 ± 3% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_write.new_sync_write
0.00 +0.8 0.79 ± 2% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_write.new_sync_write.vfs_write
1.43 ± 5% +0.8 2.24 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +0.8 0.81 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 +0.8 0.83 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.9 0.91 perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.68 ± 8% +0.9 2.63 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.67 ± 4% +0.9 2.62 ± 2% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.92 ± 3% +1.0 2.94 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.80 ± 3% +1.0 2.82 perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 +1.1 1.08 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
0.00 +1.1 1.08 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.46 ± 2% +1.1 2.58 ± 2% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
1.08 ± 15% +1.2 2.25 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.4 1.45 ± 2% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_write.new_sync_write.vfs_write.ksys_write
0.59 ± 3% +1.5 2.08 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
0.83 ± 2% +1.6 2.47 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
3.01 ± 3% +1.7 4.69 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
3.12 ± 3% +1.8 4.88 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
0.00 +2.0 2.02 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +2.0 2.03 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +2.0 2.03 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
0.00 +2.1 2.06 perf-profile.calltrace.cycles-pp.secondary_startup_64
27.56 ± 4% -11.5 16.04 ± 2% perf-profile.children.cycles-pp.__wake_up_common
26.93 ± 5% -11.4 15.51 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
26.95 ± 4% -11.3 15.67 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
29.65 ± 4% -10.1 19.51 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock
20.51 -8.1 12.44 ± 2% perf-profile.children.cycles-pp.schedule
11.07 ± 16% -7.7 3.37 perf-profile.children.cycles-pp.select_task_rq_fair
20.28 -7.1 13.19 ± 2% perf-profile.children.cycles-pp.__schedule
9.11 ± 19% -6.7 2.37 perf-profile.children.cycles-pp.select_idle_sibling
5.59 ± 23% -4.5 1.08 perf-profile.children.cycles-pp.available_idle_cpu
16.74 ± 2% -4.5 12.28 ± 2% perf-profile.children.cycles-pp.pipe_wait
35.43 ± 3% -4.2 31.27 perf-profile.children.cycles-pp.pipe_write
6.30 ± 2% -4.1 2.16 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
35.93 ± 3% -4.0 31.92 perf-profile.children.cycles-pp.new_sync_write
22.80 ± 2% -3.8 19.05 perf-profile.children.cycles-pp.pipe_read
23.48 ± 2% -3.8 19.73 perf-profile.children.cycles-pp.new_sync_read
88.58 -3.6 84.99 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
87.77 -3.5 84.23 perf-profile.children.cycles-pp.do_syscall_64
38.55 ± 2% -2.6 35.91 perf-profile.children.cycles-pp.vfs_write
27.02 ± 2% -2.6 24.43 perf-profile.children.cycles-pp.vfs_read
5.40 -2.5 2.89 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
8.81 ± 3% -2.3 6.50 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate
28.52 ± 2% -2.3 26.25 perf-profile.children.cycles-pp.ksys_read
8.72 ± 3% -2.3 6.46 ± 2% perf-profile.children.cycles-pp.activate_task
7.03 ± 4% -2.2 4.79 ± 3% perf-profile.children.cycles-pp.dequeue_task_fair
8.40 ± 3% -2.1 6.34 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
4.36 -1.6 2.75 ± 2% perf-profile.children.cycles-pp.update_load_avg
40.48 ± 2% -1.3 39.14 perf-profile.children.cycles-pp.ksys_write
1.65 ± 4% -1.2 0.43 ± 6% perf-profile.children.cycles-pp.set_task_cpu
2.01 ± 3% -1.2 0.80 ± 2% perf-profile.children.cycles-pp.switch_fpu_return
1.42 ± 21% -1.1 0.28 ± 2% perf-profile.children.cycles-pp.cpumask_next_wrap
3.41 -1.0 2.38 ± 2% perf-profile.children.cycles-pp.update_curr
1.85 ± 2% -1.0 0.85 perf-profile.children.cycles-pp.__switch_to_asm
1.79 ± 2% -0.9 0.86 ± 3% perf-profile.children.cycles-pp.___perf_sw_event
2.67 -0.9 1.77 ± 2% perf-profile.children.cycles-pp.reweight_entity
3.96 -0.9 3.07 ± 2% perf-profile.children.cycles-pp.enqueue_entity
1.85 ± 4% -0.9 1.00 perf-profile.children.cycles-pp.update_rq_clock
1.60 ± 2% -0.8 0.77 ± 3% perf-profile.children.cycles-pp.__switch_to
2.88 ± 2% -0.8 2.06 ± 2% perf-profile.children.cycles-pp.dequeue_entity
1.05 ± 6% -0.8 0.26 ± 4% perf-profile.children.cycles-pp.migrate_task_rq_fair
1.45 -0.8 0.69 ± 3% perf-profile.children.cycles-pp.native_write_msr
1.22 -0.7 0.48 ± 3% perf-profile.children.cycles-pp.pick_next_entity
1.14 -0.6 0.50 ± 2% perf-profile.children.cycles-pp.put_prev_entity
1.50 -0.5 0.96 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.77 ± 14% -0.5 0.29 ± 2% perf-profile.children.cycles-pp.find_next_bit
1.43 -0.4 1.00 ± 2% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.38 -0.4 0.97 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
1.64 -0.4 1.24 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se
0.94 -0.4 0.57 perf-profile.children.cycles-pp.check_preempt_wakeup
0.85 -0.4 0.48 ± 2% perf-profile.children.cycles-pp.__calc_delta
1.00 ± 2% -0.4 0.63 ± 2% perf-profile.children.cycles-pp.finish_task_switch
1.10 ± 2% -0.3 0.77 perf-profile.children.cycles-pp.check_preempt_curr
1.19 ± 2% -0.3 0.86 perf-profile.children.cycles-pp.ttwu_do_wakeup
0.48 ± 10% -0.3 0.18 perf-profile.children.cycles-pp.cpus_share_cache
0.47 ± 2% -0.3 0.19 ± 3% perf-profile.children.cycles-pp.clear_buddies
0.70 -0.3 0.43 perf-profile.children.cycles-pp.sched_clock_cpu
0.64 -0.3 0.38 ± 2% perf-profile.children.cycles-pp.sched_clock
0.50 ± 7% -0.3 0.25 ± 5% perf-profile.children.cycles-pp.cpuacct_charge
0.31 ± 10% -0.3 0.06 perf-profile.children.cycles-pp.attach_entity_load_avg
0.38 ± 4% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.34 ± 3% -0.2 0.10 ± 7% perf-profile.children.cycles-pp.remove_entity_load_avg
0.57 -0.2 0.34 ± 2% perf-profile.children.cycles-pp.native_sched_clock
0.87 ± 2% -0.2 0.64 perf-profile.children.cycles-pp.__enqueue_entity
0.27 ± 3% -0.2 0.09 ± 4% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.41 ± 2% -0.1 0.27 perf-profile.children.cycles-pp.anon_pipe_buf_release
0.48 -0.1 0.34 ± 2% perf-profile.children.cycles-pp.account_entity_enqueue
0.16 ± 7% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.perf_exclude_event
0.41 ± 2% -0.1 0.30 ± 4% perf-profile.children.cycles-pp.rb_erase_cached
0.45 ± 2% -0.1 0.34 ± 3% perf-profile.children.cycles-pp.update_min_vruntime
0.14 ± 10% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.native_load_tls
0.56 ± 3% -0.1 0.48 ± 2% perf-profile.children.cycles-pp.account_entity_dequeue
0.26 -0.1 0.18 ± 2% perf-profile.children.cycles-pp.cpumask_next
0.10 ± 4% -0.1 0.03 ±100% perf-profile.children.cycles-pp.rcu_note_context_switch
0.22 ± 4% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.resched_curr
0.22 ± 5% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.set_next_buddy
0.12 ± 6% -0.1 0.06 perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.15 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.__list_add_valid
0.14 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.generic_update_time
0.16 ± 4% -0.0 0.14 perf-profile.children.cycles-pp.rb_insert_color_cached
0.10 ± 5% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.__mark_inode_dirty
0.06 ± 7% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.default_send_IPI_single_phys
0.06 ± 6% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.kill_fasync
0.18 ± 4% +0.0 0.20 ± 4% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.01 ±173% +0.0 0.05 ± 9% perf-profile.children.cycles-pp.ktime_get
0.11 ± 4% +0.0 0.16 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.01 ±173% +0.0 0.06 perf-profile.children.cycles-pp.interrupt_entry
0.03 ±100% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.bpf_fd_pass
0.00 +0.1 0.05 perf-profile.children.cycles-pp.fput
0.09 ± 9% +0.1 0.14 ± 6% perf-profile.children.cycles-pp.iov_iter_init
0.18 ± 4% +0.1 0.24 ± 2% perf-profile.children.cycles-pp.rw_verify_area
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.__next_timer_interrupt
0.35 ± 4% +0.1 0.41 ± 3% perf-profile.children.cycles-pp.hrtimer_interrupt
0.11 ± 4% +0.1 0.17 ± 2% perf-profile.children.cycles-pp.__sb_end_write
0.41 ± 4% +0.1 0.47 ± 3% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
0.18 ± 8% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.tick_sched_timer
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.schedule_preempt_disabled
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.45 ± 3% +0.1 0.52 ± 3% perf-profile.children.cycles-pp.apic_timer_interrupt
0.15 ± 7% +0.1 0.23 ± 4% perf-profile.children.cycles-pp.update_process_times
0.16 ± 9% +0.1 0.23 ± 4% perf-profile.children.cycles-pp.tick_sched_handle
0.11 ± 7% +0.1 0.18 ± 4% perf-profile.children.cycles-pp.scheduler_tick
0.09 ± 7% +0.1 0.17 ± 4% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.24 ± 6% +0.1 0.32 ± 5% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.09 ± 7% +0.1 0.17 ± 4% perf-profile.children.cycles-pp.task_tick_fair
0.23 ± 3% +0.1 0.31 ± 2% perf-profile.children.cycles-pp.__x64_sys_write
0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.33 ± 9% +0.1 0.41 ± 3% perf-profile.children.cycles-pp.wake_up_q
0.11 ± 4% +0.1 0.20 ± 2% perf-profile.children.cycles-pp.timespec64_trunc
0.19 ± 4% +0.1 0.29 ± 2% perf-profile.children.cycles-pp.__x64_sys_read
0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.tick_nohz_next_event
0.22 ± 5% +0.1 0.32 perf-profile.children.cycles-pp.rcu_all_qs
0.14 ± 9% +0.1 0.25 perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.15 ± 7% +0.1 0.27 perf-profile.children.cycles-pp.inode_has_perm
0.65 ± 10% +0.1 0.78 ± 2% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.00 +0.1 0.13 ± 3% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.1 0.15 ± 3% perf-profile.children.cycles-pp.osq_lock
0.74 +0.2 0.90 perf-profile.children.cycles-pp.__might_sleep
0.57 ± 4% +0.2 0.77 perf-profile.children.cycles-pp.__fsnotify_parent
0.62 ± 2% +0.2 0.82 ± 2% perf-profile.children.cycles-pp.file_update_time
0.07 ± 20% +0.2 0.27 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.05 ± 59% +0.2 0.25 ± 6% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.47 ± 5% +0.2 0.68 perf-profile.children.cycles-pp.__might_fault
0.48 ± 2% +0.2 0.70 ± 2% perf-profile.children.cycles-pp.prepare_to_wait
0.37 ± 7% +0.2 0.61 perf-profile.children.cycles-pp.atime_needs_update
0.66 ± 5% +0.2 0.91 perf-profile.children.cycles-pp.touch_atime
0.00 +0.3 0.26 ± 3% perf-profile.children.cycles-pp.menu_select
0.46 ± 4% +0.3 0.75 ± 2% perf-profile.children.cycles-pp.current_time
0.09 ± 13% +0.3 0.40 perf-profile.children.cycles-pp.preempt_schedule_common
0.73 ± 5% +0.4 1.08 ± 5% perf-profile.children.cycles-pp.avc_has_perm
0.01 ±173% +0.4 0.37 ± 3% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.82 ± 2% +0.4 1.19 ± 2% perf-profile.children.cycles-pp.fsnotify
0.72 ± 4% +0.4 1.12 ± 2% perf-profile.children.cycles-pp.copyin
0.85 ± 3% +0.4 1.25 perf-profile.children.cycles-pp.___might_sleep
0.00 +0.4 0.44 ± 2% perf-profile.children.cycles-pp.poll_idle
0.83 ± 4% +0.5 1.30 perf-profile.children.cycles-pp.fput_many
0.57 ± 5% +0.5 1.07 perf-profile.children.cycles-pp.__inode_security_revalidate
0.52 ± 5% +0.5 1.04 perf-profile.children.cycles-pp._cond_resched
0.00 +0.5 0.53 perf-profile.children.cycles-pp.schedule_idle
2.41 +0.5 2.95 perf-profile.children.cycles-pp.mutex_lock
0.00 +0.6 0.62 ± 2% perf-profile.children.cycles-pp.intel_idle
1.16 ± 5% +0.6 1.80 ± 3% perf-profile.children.cycles-pp.file_has_perm
0.90 ± 5% +0.7 1.58 perf-profile.children.cycles-pp.copyout
0.34 ± 8% +0.8 1.09 ± 3% perf-profile.children.cycles-pp.__mutex_lock
1.47 ± 4% +0.8 2.29 perf-profile.children.cycles-pp.copy_page_from_iter
2.09 +0.9 2.98 ± 2% perf-profile.children.cycles-pp.__fget_light
1.60 ± 2% +0.9 2.50 ± 2% perf-profile.children.cycles-pp.__fget
2.15 +0.9 3.08 ± 2% perf-profile.children.cycles-pp.__fdget_pos
1.53 ± 5% +1.0 2.56 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.83 ± 3% +1.0 2.88 perf-profile.children.cycles-pp.copy_page_to_iter
0.00 +1.1 1.09 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state
2.00 ± 4% +1.1 3.10 perf-profile.children.cycles-pp.selinux_file_permission
0.00 +1.1 1.10 ± 2% perf-profile.children.cycles-pp.cpuidle_enter
2.15 ± 2% +1.2 3.34 ± 2% perf-profile.children.cycles-pp.mutex_unlock
2.95 ± 7% +1.3 4.25 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
2.26 +1.5 3.75 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
3.12 ± 3% +1.8 4.88 perf-profile.children.cycles-pp.entry_SYSCALL_64
3.46 ± 4% +1.8 5.30 perf-profile.children.cycles-pp.syscall_return_via_sysret
3.62 ± 4% +2.0 5.60 perf-profile.children.cycles-pp.security_file_permission
0.00 +2.0 2.03 perf-profile.children.cycles-pp.start_secondary
0.00 +2.1 2.06 perf-profile.children.cycles-pp.secondary_startup_64
0.00 +2.1 2.06 perf-profile.children.cycles-pp.cpu_startup_entry
0.00 +2.1 2.06 perf-profile.children.cycles-pp.do_idle
2.30 ± 10% +3.7 5.96 ± 3% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
5.56 ± 23% -4.5 1.07 perf-profile.self.cycles-pp.available_idle_cpu
2.00 ± 3% -1.2 0.79 ± 2% perf-profile.self.cycles-pp.switch_fpu_return
1.82 ± 2% -1.0 0.83 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
2.59 ± 4% -1.0 1.60 ± 3% perf-profile.self.cycles-pp.__schedule
1.65 ± 2% -0.8 0.80 ± 3% perf-profile.self.cycles-pp.___perf_sw_event
1.52 ± 2% -0.8 0.73 ± 3% perf-profile.self.cycles-pp.__switch_to
1.44 -0.8 0.68 ± 3% perf-profile.self.cycles-pp.native_write_msr
1.29 ± 11% -0.7 0.58 perf-profile.self.cycles-pp.select_idle_sibling
1.54 -0.7 0.84 ± 2% perf-profile.self.cycles-pp.select_task_rq_fair
0.84 ± 22% -0.7 0.17 ± 3% perf-profile.self.cycles-pp.cpumask_next_wrap
1.39 ± 5% -0.6 0.76 ± 2% perf-profile.self.cycles-pp.update_rq_clock
1.82 ± 2% -0.6 1.25 ± 3% perf-profile.self.cycles-pp.update_load_avg
1.24 ± 2% -0.5 0.74 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.60 ± 8% -0.5 0.14 ± 3% perf-profile.self.cycles-pp.migrate_task_rq_fair
1.17 -0.4 0.74 ± 2% perf-profile.self.cycles-pp.pick_next_task_fair
1.05 -0.4 0.62 perf-profile.self.cycles-pp.reweight_entity
0.59 ± 16% -0.4 0.17 ± 9% perf-profile.self.cycles-pp.set_task_cpu
1.34 -0.4 0.92 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.66 ± 14% -0.4 0.26 ± 3% perf-profile.self.cycles-pp.find_next_bit
0.86 ± 2% -0.4 0.48 ± 4% perf-profile.self.cycles-pp.dequeue_task_fair
1.60 -0.4 1.21 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
0.85 -0.4 0.48 ± 2% perf-profile.self.cycles-pp.__calc_delta
1.62 -0.3 1.28 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.57 ± 2% -0.3 1.25 ± 3% perf-profile.self.cycles-pp.update_curr
0.48 ± 10% -0.3 0.18 ± 2% perf-profile.self.cycles-pp.cpus_share_cache
1.21 ± 2% -0.3 0.91 ± 2% perf-profile.self.cycles-pp.enqueue_task_fair
0.52 -0.3 0.24 ± 5% perf-profile.self.cycles-pp.pick_next_entity
1.02 -0.3 0.75 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.50 ± 7% -0.3 0.25 ± 5% perf-profile.self.cycles-pp.cpuacct_charge
0.43 -0.3 0.18 ± 2% perf-profile.self.cycles-pp.schedule
0.41 ± 2% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.clear_buddies
0.30 ± 11% -0.2 0.06 perf-profile.self.cycles-pp.attach_entity_load_avg
0.37 ± 5% -0.2 0.14 ± 5% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.55 -0.2 0.32 perf-profile.self.cycles-pp.native_sched_clock
0.86 ± 2% -0.2 0.63 ± 2% perf-profile.self.cycles-pp.__enqueue_entity
0.31 ± 8% -0.2 0.12 ± 3% perf-profile.self.cycles-pp.activate_task
0.76 -0.2 0.58 perf-profile.self.cycles-pp.vfs_read
0.43 -0.2 0.25 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup
0.24 ± 2% -0.2 0.09 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.22 -0.2 0.07 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.70 ± 3% -0.1 0.56 ± 3% perf-profile.self.cycles-pp.try_to_wake_up
0.42 -0.1 0.28 ± 3% perf-profile.self.cycles-pp.account_entity_enqueue
0.38 -0.1 0.26 perf-profile.self.cycles-pp.anon_pipe_buf_release
0.39 ± 2% -0.1 0.27 ± 4% perf-profile.self.cycles-pp.dequeue_entity
0.27 ± 3% -0.1 0.15 ± 3% perf-profile.self.cycles-pp.set_next_entity
0.20 ± 3% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.put_prev_entity
0.30 -0.1 0.19 ± 4% perf-profile.self.cycles-pp.rb_erase_cached
0.43 -0.1 0.33 ± 3% perf-profile.self.cycles-pp.update_min_vruntime
0.62 ± 2% -0.1 0.53 ± 3% perf-profile.self.cycles-pp.__wake_up_common
0.46 ± 2% -0.1 0.38 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue
0.27 ± 4% -0.1 0.19 ± 3% perf-profile.self.cycles-pp.pipe_wait
0.13 ± 10% -0.1 0.06 ± 9% perf-profile.self.cycles-pp.native_load_tls
0.47 ± 3% -0.1 0.40 ± 2% perf-profile.self.cycles-pp.finish_task_switch
0.28 ± 2% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.prepare_to_wait
0.22 ± 3% -0.1 0.15 ± 4% perf-profile.self.cycles-pp.resched_curr
0.15 ± 3% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.autoremove_wake_function
0.20 ± 6% -0.1 0.15 ± 3% perf-profile.self.cycles-pp.set_next_buddy
0.42 ± 3% -0.0 0.38 ± 2% perf-profile.self.cycles-pp.enqueue_entity
0.14 ± 3% -0.0 0.10 ± 5% perf-profile.self.cycles-pp.__list_add_valid
0.16 ± 4% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.rb_insert_color_cached
0.10 ± 5% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__mark_inode_dirty
0.07 ± 7% -0.0 0.05 perf-profile.self.cycles-pp.sched_clock_cpu
0.13 ± 5% +0.0 0.15 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.05 +0.0 0.07 ± 10% perf-profile.self.cycles-pp.kill_fasync
0.23 ± 2% +0.0 0.25 perf-profile.self.cycles-pp.__wake_up_common_lock
0.14 ± 11% +0.0 0.16 ± 2% perf-profile.self.cycles-pp.__mutex_unlock_slowpath
0.26 ± 4% +0.0 0.29 perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ± 6% +0.0 0.21 ± 3% perf-profile.self.cycles-pp.touch_atime
0.17 ± 4% +0.0 0.20 ± 3% perf-profile.self.cycles-pp.finish_wait
0.07 ± 6% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__fdget_pos
0.10 ± 5% +0.0 0.14 ± 6% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.08 ± 8% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.iov_iter_init
0.01 ±173% +0.1 0.06 ± 6% perf-profile.self.cycles-pp.reschedule_interrupt
0.18 ± 4% +0.1 0.23 ± 3% perf-profile.self.cycles-pp.rw_verify_area
0.11 ± 4% +0.1 0.16 ± 2% perf-profile.self.cycles-pp.__sb_end_write
0.01 ±173% +0.1 0.07 ± 6% perf-profile.self.cycles-pp.copyout
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.bpf_fd_pass
0.00 +0.1 0.06 perf-profile.self.cycles-pp.interrupt_entry
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.22 ± 11% +0.1 0.28 perf-profile.self.cycles-pp.atime_needs_update
0.00 +0.1 0.07 ± 10% perf-profile.self.cycles-pp.scheduler_ipi
0.20 ± 2% +0.1 0.27 ± 2% perf-profile.self.cycles-pp.__x64_sys_write
0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.copyin
0.07 ± 15% +0.1 0.15 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.11 ± 4% +0.1 0.18 ± 2% perf-profile.self.cycles-pp.timespec64_trunc
0.16 ± 5% +0.1 0.24 ± 2% perf-profile.self.cycles-pp.__x64_sys_read
0.17 ± 4% +0.1 0.26 perf-profile.self.cycles-pp.rcu_all_qs
0.47 ± 3% +0.1 0.56 ± 2% perf-profile.self.cycles-pp.copy_page_to_iter
0.21 ± 3% +0.1 0.30 perf-profile.self.cycles-pp.__inode_security_revalidate
0.12 ± 10% +0.1 0.21 ± 3% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.14 ± 8% +0.1 0.23 perf-profile.self.cycles-pp.inode_has_perm
0.22 ± 3% +0.1 0.32 ± 2% perf-profile.self.cycles-pp.file_update_time
0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.menu_select
0.26 ± 4% +0.1 0.36 perf-profile.self.cycles-pp.ksys_read
0.20 ± 6% +0.1 0.31 perf-profile.self.cycles-pp._cond_resched
0.18 ± 4% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.ksys_write
0.27 ± 3% +0.1 0.40 ± 2% perf-profile.self.cycles-pp.current_time
0.44 +0.1 0.57 ± 2% perf-profile.self.cycles-pp.new_sync_write
0.67 +0.1 0.80 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.osq_lock
0.33 ± 4% +0.2 0.48 ± 3% perf-profile.self.cycles-pp.security_file_permission
0.25 ± 4% +0.2 0.41 ± 3% perf-profile.self.cycles-pp.file_has_perm
0.53 ± 4% +0.2 0.71 perf-profile.self.cycles-pp.__fsnotify_parent
0.30 ± 6% +0.2 0.49 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter
0.17 ± 8% +0.2 0.38 ± 4% perf-profile.self.cycles-pp.__mutex_lock
0.29 ± 5% +0.2 0.50 ± 2% perf-profile.self.cycles-pp.vfs_write
1.22 +0.2 1.44 perf-profile.self.cycles-pp.pipe_read
0.72 ± 5% +0.3 1.07 ± 5% perf-profile.self.cycles-pp.avc_has_perm
0.01 ±173% +0.4 0.37 ± 4% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.79 ± 3% +0.4 1.15 ± 2% perf-profile.self.cycles-pp.fsnotify
0.83 ± 3% +0.4 1.21 perf-profile.self.cycles-pp.___might_sleep
1.52 +0.4 1.93 perf-profile.self.cycles-pp.mutex_lock
0.00 +0.4 0.42 ± 2% perf-profile.self.cycles-pp.poll_idle
0.82 ± 4% +0.5 1.28 perf-profile.self.cycles-pp.fput_many
1.41 ± 3% +0.6 1.99 perf-profile.self.cycles-pp.selinux_file_permission
0.00 +0.6 0.60 ± 3% perf-profile.self.cycles-pp.intel_idle
1.58 ± 2% +0.9 2.46 ± 2% perf-profile.self.cycles-pp.__fget
0.77 ± 5% +1.0 1.73 ± 2% perf-profile.self.cycles-pp.pipe_write
1.51 ± 5% +1.0 2.52 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
2.10 +1.2 3.27 perf-profile.self.cycles-pp.mutex_unlock
3.12 ± 3% +1.8 4.88 perf-profile.self.cycles-pp.entry_SYSCALL_64
3.45 ± 4% +1.8 5.29 perf-profile.self.cycles-pp.syscall_return_via_sysret
2.29 ± 10% +3.7 5.95 ± 3% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
9.89 ± 3% +5.3 15.16 perf-profile.self.cycles-pp.do_syscall_64
hackbench.throughput
80000 +-+-----------------------------------------------------------------+
| OO O OO O O O |
70000 +-+ O O O O |
60000 O-OO OO OO OO OO O OO O |
| |
50000 +-+ + ++.++.++.+.++.++.+ .++.+.+ +.++.+ .+.++.++.++.++.++.|
| : : : +.++ +.++.+ + |
40000 +-+ : : |
| : :: : |
30000 +-+ :: : |
20000 +-+:: :: |
| :: :: |
10000 +-+: : |
| : : |
0 +-+-----------------------------------------------------------------+
hackbench.workload
4.5e+08 +-+--------------------OO-OO-OO-OO-OO-OO--------------------------+
O OO OO OO OO OO OO OO |
4e+08 +-+ |
3.5e+08 +-+ |
|.+ + ++.++.++.++.++.++.++.++.++.++.++.++.++.++.++.++.++.++.++.+|
3e+08 +-+ : : |
2.5e+08 +-+ : : |
| : :: : |
2e+08 +-+ :: : |
1.5e+08 +-+:: :: |
| :: :: |
1e+08 +-+:: :: |
5e+07 +-+: : |
| : : |
0 +-+---------------------------------------------------------------+
hackbench.time.minor_page_faults
140000 +-+----------------------------------------------------------------+
| OO OO OO O OO OO O |
120000 O-OO OO OO OO OO OO OO |
| |
100000 +-+ + .++. .++.++. |
|.+ + :+.++ ++ ++.++.+.++.++.++.++.++.++.++.++.++.++.++.|
80000 +-+ : : |
| : : : |
60000 +-+ :: : |
| : :: : |
40000 +-+:: :: |
| :: :: |
20000 +-+: : |
| : : |
0 +-+----------------------------------------------------------------+
hackbench.time.voluntary_context_switches
1.8e+09 +-+---------------------------------------------------------------+
|.+ + ++.++ ++.++.++.++.++.++.++.++.++.++.++.++.++.++. +.++.+|
1.6e+09 +-+ : : + |
1.4e+09 +-+ : : |
| : OO : |
1.2e+09 O-OO:: :OO OO OO OO OO |
1e+09 +-+ :: : OO OO OO OO OO OO |
| : :: : |
8e+08 +-+:: :: |
6e+08 +-+:: :: |
| :: :: |
4e+08 +-+: : |
2e+08 +-+: : |
| : : |
0 +-+---------------------------------------------------------------+
hackbench.time.involuntary_context_switches
1.2e+09 +-+---------------------------------------------------------------+
| ++.+ |
1e+09 +-+ +. + +.++.++. |
|.+ + + ++ ++.++.++.++.++.++.++.++.++.++.++.++.++.+|
| : : : |
8e+08 +-+ : : |
| : :: : |
6e+08 +-+ :: : |
| : :: : |
4e+08 O-OO:OO:OO OO OO OO OO |
| :: :: OO OO OO OO OO OO |
| :: :: |
2e+08 +-+: : |
| : : |
0 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-bdw-ep3: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 64G memory
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-05-14.cgz/lkp-bdw-ep3/hackbench/0xb000036
commit:
e1f4f2a13d ("sched/fair: rework load_balance")
b3e5e9a1dd ("sched/fair: use load instead of runnable load")
e1f4f2a13d354801 b3e5e9a1ddb6f4122985b1164dd
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at_ip__mutex_lock/0x
%stddev %change %stddev
\ | \
132922 +47.5% 195997 hackbench.throughput
3.462e+09 ± 3% -71.7% 9.797e+08 ± 2% hackbench.time.involuntary_context_switches
609760 +49.9% 914228 hackbench.time.minor_page_faults
8755 -2.1% 8569 hackbench.time.percent_of_cpu_this_job_got
43405 -8.7% 39643 hackbench.time.system_time
11071 +32.2% 14638 hackbench.time.user_time
5.071e+09 ± 2% -45.5% 2.766e+09 hackbench.time.voluntary_context_switches
8.448e+08 +50.0% 1.267e+09 hackbench.workload
4311 -10.3% 3868 meminfo.max_used_kB
0.58 ± 4% +2.0 2.57 mpstat.cpu.all.idle%
79.99 -8.6 71.34 mpstat.cpu.all.sys%
19.43 +6.7 26.08 mpstat.cpu.all.usr%
23807783 ± 10% +294.6% 93938477 numa-numastat.node0.local_node
23824874 ± 10% +294.3% 93947059 numa-numastat.node0.numa_hit
24871518 ± 12% +284.2% 95568095 numa-numastat.node1.local_node
24882938 ± 12% +284.2% 95588026 numa-numastat.node1.numa_hit
79.25 -10.4% 71.00 vmstat.cpu.sy
19.00 +35.5% 25.75 vmstat.cpu.us
13671862 -54.9% 6163817 vmstat.system.cs
1132433 -25.4% 844680 vmstat.system.in
12297899 ± 11% +285.4% 47391168 numa-vmstat.node0.numa_hit
12280639 ± 11% +284.7% 47249112 numa-vmstat.node0.numa_local
17260 ± 39% +723.1% 142059 ± 5% numa-vmstat.node0.numa_other
12480510 ± 10% +286.4% 48227554 numa-vmstat.node1.numa_hit
12321040 ± 10% +291.1% 48192582 numa-vmstat.node1.numa_local
159471 ± 4% -78.1% 34976 ± 23% numa-vmstat.node1.numa_other
30814582 ± 7% +1314.6% 4.359e+08 cpuidle.C1.time
2851342 ± 6% +2242.2% 66783036 cpuidle.C1.usage
22074192 ± 6% +337.7% 96608185 ± 3% cpuidle.C1E.time
805640 ± 4% +1587.8% 13597309 ± 3% cpuidle.C1E.usage
98777804 ± 33% +178.4% 2.75e+08 ± 20% cpuidle.C3.time
654069 ± 8% +2682.4% 18198520 ± 2% cpuidle.C3.usage
6235274 ± 5% +2258.8% 1.471e+08 cpuidle.POLL.time
4859491 ± 5% +1813.8% 93003193 cpuidle.POLL.usage
2777 -1.2% 2745 turbostat.Avg_MHz
2848374 ± 6% +2244.5% 66780102 turbostat.C1
0.06 ± 9% +0.7 0.78 ± 2% turbostat.C1%
800541 ± 4% +1598.1% 13594009 ± 3% turbostat.C1E
0.04 ± 11% +0.1 0.18 ± 2% turbostat.C1E%
653019 ± 8% +2686.8% 18198365 ± 2% turbostat.C3
0.18 ± 33% +0.3 0.49 ± 20% turbostat.C3%
0.31 ± 6% +356.6% 1.39 ± 3% turbostat.CPU%c1
7.074e+08 -24.1% 5.372e+08 turbostat.IRQ
274.76 -2.6% 267.75 turbostat.PkgWatt
7.85 ± 2% -7.1% 7.29 turbostat.RAMWatt
4594 +3.1% 4734 proc-vmstat.nr_inactive_anon
17744 -2.4% 17325 proc-vmstat.nr_page_table_pages
19236 +5.4% 20277 proc-vmstat.nr_slab_reclaimable
52787 +6.8% 56367 proc-vmstat.nr_slab_unreclaimable
4594 +3.1% 4734 proc-vmstat.nr_zone_inactive_anon
72442 ± 7% +56.1% 113118 ± 3% proc-vmstat.numa_hint_faults
48721419 ± 4% +289.0% 1.895e+08 proc-vmstat.numa_hit
48692899 ± 4% +289.2% 1.895e+08 proc-vmstat.numa_local
63533 ± 4% +54.9% 98441 proc-vmstat.numa_pages_migrated
95473 ± 17% +55.9% 148850 ± 12% proc-vmstat.numa_pte_updates
48873737 ± 4% +288.3% 1.898e+08 proc-vmstat.pgalloc_normal
1947075 +25.5% 2444066 proc-vmstat.pgfault
48802461 ± 4% +288.7% 1.897e+08 proc-vmstat.pgfree
63533 ± 4% +54.9% 98441 proc-vmstat.pgmigrate_success
15051 ± 4% +20.4% 18124 ± 3% slabinfo.cred_jar.active_objs
15061 ± 4% +20.3% 18125 ± 3% slabinfo.cred_jar.num_objs
1067 ± 10% +16.3% 1241 slabinfo.dmaengine-unmap-16.active_objs
1067 ± 10% +16.3% 1241 slabinfo.dmaengine-unmap-16.num_objs
11745 +27.0% 14914 slabinfo.files_cache.active_objs
11745 +27.0% 14915 slabinfo.files_cache.num_objs
8234 +19.6% 9847 slabinfo.kmalloc-192.active_objs
8243 +19.5% 9847 slabinfo.kmalloc-192.num_objs
8181 +17.9% 9642 slabinfo.kmalloc-1k.active_objs
8244 +17.8% 9712 slabinfo.kmalloc-1k.num_objs
53554 ± 2% +13.4% 60711 ± 2% slabinfo.kmalloc-32.active_objs
53597 ± 2% +13.3% 60711 ± 2% slabinfo.kmalloc-32.num_objs
31526 +11.3% 35094 slabinfo.lsm_file_cache.active_objs
31526 +11.3% 35094 slabinfo.lsm_file_cache.num_objs
9804 +15.8% 11355 slabinfo.mm_struct.active_objs
9828 +15.9% 11387 slabinfo.mm_struct.num_objs
10485 ± 5% +33.3% 13979 ± 3% slabinfo.pid.active_objs
10495 ± 5% +33.2% 13980 ± 3% slabinfo.pid.num_objs
18158 ± 3% +23.3% 22387 ± 3% slabinfo.proc_inode_cache.active_objs
20494 +16.2% 23823 ± 3% slabinfo.proc_inode_cache.num_objs
5679 ± 2% +22.1% 6936 slabinfo.sighand_cache.active_objs
5703 ± 2% +22.2% 6971 slabinfo.sighand_cache.num_objs
7645 ± 2% +24.4% 9509 ± 2% slabinfo.signal_cache.active_objs
7674 ± 2% +24.4% 9550 ± 2% slabinfo.signal_cache.num_objs
12814 +21.6% 15575 ± 3% slabinfo.task_delay_info.active_objs
12814 +21.5% 15575 ± 3% slabinfo.task_delay_info.num_objs
148.00 ± 9% +63.0% 241.25 ± 14% slabinfo.xfrm_state.active_objs
148.00 ± 9% +63.0% 241.25 ± 14% slabinfo.xfrm_state.num_objs
6863 ±173% +2366.9% 169317 ± 25% sched_debug.cfs_rq:/.MIN_vruntime.avg
603991 ±173% +1996.6% 12663108 ± 21% sched_debug.cfs_rq:/.MIN_vruntime.max
64018 ±173% +2146.3% 1438068 ± 22% sched_debug.cfs_rq:/.MIN_vruntime.stddev
12083 +10.2% 13310 ± 2% sched_debug.cfs_rq:/.load.avg
26622 ± 2% +179.3% 74358 ± 61% sched_debug.cfs_rq:/.load.max
2009 ± 22% -68.5% 632.89 ±128% sched_debug.cfs_rq:/.load.min
4071 +138.0% 9692 ± 48% sched_debug.cfs_rq:/.load.stddev
3.66 ± 24% -38.5% 2.25 ± 27% sched_debug.cfs_rq:/.load_avg.min
6863 ±173% +2366.9% 169317 ± 25% sched_debug.cfs_rq:/.max_vruntime.avg
603991 ±173% +1996.6% 12663108 ± 21% sched_debug.cfs_rq:/.max_vruntime.max
64018 ±173% +2146.3% 1438068 ± 22% sched_debug.cfs_rq:/.max_vruntime.stddev
526495 ± 29% -57.2% 225383 ± 5% sched_debug.cfs_rq:/.min_vruntime.stddev
0.39 ± 19% -64.7% 0.14 ±110% sched_debug.cfs_rq:/.nr_running.min
0.10 ± 8% +52.6% 0.15 ± 19% sched_debug.cfs_rq:/.nr_running.stddev
27.57 ± 4% +171.7% 74.91 ± 59% sched_debug.cfs_rq:/.runnable_load_avg.max
1.59 ± 9% -97.1% 0.05 ±100% sched_debug.cfs_rq:/.runnable_load_avg.min
4.42 +155.4% 11.30 ± 40% sched_debug.cfs_rq:/.runnable_load_avg.stddev
26239 +180.2% 73533 ± 63% sched_debug.cfs_rq:/.runnable_weight.max
1305 ± 30% -87.4% 164.70 ± 73% sched_debug.cfs_rq:/.runnable_weight.min
4389 +154.8% 11183 ± 42% sched_debug.cfs_rq:/.runnable_weight.stddev
-29090 +2848.4% -857705 sched_debug.cfs_rq:/.spread0.avg
1062762 ± 34% -84.5% 164770 ± 84% sched_debug.cfs_rq:/.spread0.max
525591 ± 29% -57.0% 226145 ± 5% sched_debug.cfs_rq:/.spread0.stddev
653.68 ± 3% -23.2% 502.09 ± 3% sched_debug.cfs_rq:/.util_avg.min
152.93 +23.3% 188.61 sched_debug.cfs_rq:/.util_avg.stddev
1118 ± 2% +15.6% 1292 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max
99.70 ± 20% -93.4% 6.55 ± 38% sched_debug.cfs_rq:/.util_est_enqueued.min
179.06 +67.5% 299.85 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
179186 ± 17% -60.0% 71685 sched_debug.cpu.avg_idle.avg
614169 ± 12% -76.9% 142123 ± 21% sched_debug.cpu.avg_idle.max
173113 ± 16% -85.3% 25388 ± 20% sched_debug.cpu.avg_idle.stddev
89.03 ± 11% -65.5% 30.69 ± 9% sched_debug.cpu.clock.stddev
89.03 ± 11% -65.5% 30.69 ± 9% sched_debug.cpu.clock_task.stddev
14262 +27.6% 18197 sched_debug.cpu.curr->pid.avg
15856 +26.4% 20047 sched_debug.cpu.curr->pid.max
12727 ± 5% -82.5% 2228 ± 25% sched_debug.cpu.curr->pid.min
602.64 ± 9% +347.1% 2694 ± 6% sched_debug.cpu.curr->pid.stddev
0.00 ± 8% -41.1% 0.00 ± 2% sched_debug.cpu.next_balance.stddev
21.57 ± 3% +8.1% 23.32 ± 3% sched_debug.cpu.nr_running.max
2.09 ± 6% -87.0% 0.27 ± 23% sched_debug.cpu.nr_running.min
3.31 +65.9% 5.48 ± 2% sched_debug.cpu.nr_running.stddev
47293241 -55.3% 21140164 sched_debug.cpu.nr_switches.avg
50196057 ± 2% -56.1% 22025217 sched_debug.cpu.nr_switches.max
44426882 -54.5% 20212219 sched_debug.cpu.nr_switches.min
1294461 ± 23% -72.5% 356239 ± 10% sched_debug.cpu.nr_switches.stddev
858.20 ± 7% +37.5% 1179 ± 15% sched_debug.cpu.nr_uninterruptible.max
-815.16 +51.7% -1236 sched_debug.cpu.nr_uninterruptible.min
351.13 ± 7% +43.1% 502.39 ± 5% sched_debug.cpu.nr_uninterruptible.stddev
4.26 +12.5% 4.79 sched_debug.rt_rq:/.rt_runtime.stddev
14.34 -19.8% 11.51 perf-stat.i.MPKI
2.126e+10 -12.6% 1.858e+10 perf-stat.i.branch-instructions
1.71 -0.1 1.59 perf-stat.i.branch-miss-rate%
3.627e+08 -18.3% 2.962e+08 perf-stat.i.branch-misses
1.45 ± 17% -0.5 0.99 ± 2% perf-stat.i.cache-miss-rate%
21535993 ± 16% -51.5% 10455605 ± 2% perf-stat.i.cache-misses
1.535e+09 -28.7% 1.095e+09 perf-stat.i.cache-references
13724709 -54.9% 6185496 perf-stat.i.context-switches
2.28 +11.1% 2.53 perf-stat.i.cpi
2.433e+11 -1.1% 2.407e+11 perf-stat.i.cpu-cycles
3059887 ± 3% -76.4% 723208 perf-stat.i.cpu-migrations
25727 ± 14% +20.4% 30981 perf-stat.i.cycles-between-cache-misses
1.59 ± 4% -0.7 0.87 ± 3% perf-stat.i.dTLB-load-miss-rate%
5.179e+08 ± 4% -50.1% 2.582e+08 ± 3% perf-stat.i.dTLB-load-misses
3.205e+10 -8.6% 2.929e+10 perf-stat.i.dTLB-loads
0.34 -0.0 0.31 ± 2% perf-stat.i.dTLB-store-miss-rate%
69600464 -14.1% 59786152 ± 2% perf-stat.i.dTLB-store-misses
2.041e+10 -6.4% 1.91e+10 perf-stat.i.dTLB-stores
55.17 +12.0 67.19 perf-stat.i.iTLB-load-miss-rate%
1.245e+08 -31.5% 85371293 perf-stat.i.iTLB-load-misses
1.015e+08 -58.5% 42096825 ± 2% perf-stat.i.iTLB-loads
1.071e+11 -11.1% 9.521e+10 perf-stat.i.instructions
866.41 ± 2% +29.7% 1124 perf-stat.i.instructions-per-iTLB-miss
0.44 -10.1% 0.40 perf-stat.i.ipc
3055 +23.6% 3775 perf-stat.i.minor-faults
11324561 ± 17% -64.2% 4052419 ± 3% perf-stat.i.node-load-misses
34252 ± 14% +58.1% 54150 ± 2% perf-stat.i.node-loads
60.36 ± 2% +5.5 65.83 perf-stat.i.node-store-miss-rate%
4481606 ± 16% -56.4% 1951887 ± 3% perf-stat.i.node-store-misses
2878709 ± 16% -65.7% 987963 ± 4% perf-stat.i.node-stores
3055 +23.6% 3775 perf-stat.i.page-faults
14.34 -19.8% 11.50 perf-stat.overall.MPKI
1.71 -0.1 1.59 perf-stat.overall.branch-miss-rate%
1.40 ± 16% -0.4 0.96 ± 2% perf-stat.overall.cache-miss-rate%
2.27 +11.3% 2.53 perf-stat.overall.cpi
11631 ± 17% +98.1% 23038 ± 2% perf-stat.overall.cycles-between-cache-misses
1.59 ± 4% -0.7 0.87 ± 3% perf-stat.overall.dTLB-load-miss-rate%
0.34 -0.0 0.31 ± 2% perf-stat.overall.dTLB-store-miss-rate%
55.10 +11.9 66.97 perf-stat.overall.iTLB-load-miss-rate%
860.33 ± 2% +29.7% 1115 perf-stat.overall.instructions-per-iTLB-miss
0.44 -10.1% 0.40 perf-stat.overall.ipc
60.90 +5.5 66.40 perf-stat.overall.node-store-miss-rate%
78780 ± 2% -39.7% 47532 perf-stat.overall.path-length
2.122e+10 -12.6% 1.855e+10 perf-stat.ps.branch-instructions
3.621e+08 -18.3% 2.957e+08 perf-stat.ps.branch-misses
21505632 ± 16% -51.4% 10441242 ± 2% perf-stat.ps.cache-misses
1.533e+09 -28.7% 1.093e+09 perf-stat.ps.cache-references
13701970 -54.9% 6175672 perf-stat.ps.context-switches
2.429e+11 -1.1% 2.403e+11 perf-stat.ps.cpu-cycles
3054809 ± 3% -76.4% 722072 perf-stat.ps.cpu-migrations
5.17e+08 ± 4% -50.1% 2.578e+08 ± 3% perf-stat.ps.dTLB-load-misses
3.2e+10 -8.6% 2.925e+10 perf-stat.ps.dTLB-loads
69485670 -14.1% 59690301 ± 2% perf-stat.ps.dTLB-store-misses
2.038e+10 -6.4% 1.907e+10 perf-stat.ps.dTLB-stores
1.243e+08 -31.4% 85234805 perf-stat.ps.iTLB-load-misses
1.013e+08 -58.5% 42030210 ± 2% perf-stat.ps.iTLB-loads
1.069e+11 -11.1% 9.506e+10 perf-stat.ps.instructions
3052 +23.5% 3770 perf-stat.ps.minor-faults
11308305 ± 17% -64.2% 4046881 ± 3% perf-stat.ps.node-load-misses
34216 ± 14% +58.1% 54085 ± 2% perf-stat.ps.node-loads
4475214 ± 16% -56.4% 1949277 ± 3% perf-stat.ps.node-store-misses
2874628 ± 16% -65.7% 986618 ± 4% perf-stat.ps.node-stores
3052 +23.5% 3770 perf-stat.ps.page-faults
6.655e+13 ± 2% -9.5% 6.023e+13 perf-stat.total.instructions
365.00 ± 4% +111.0% 770.00 ± 72% interrupts.34:PCI-MSI.3145731-edge.eth0-TxRx-2
368.25 ± 3% +202.4% 1113 ± 21% interrupts.38:PCI-MSI.3145735-edge.eth0-TxRx-6
6943750 ± 2% -29.1% 4923373 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts
6702232 -27.8% 4836817 ± 2% interrupts.CPU1.RES:Rescheduling_interrupts
6855495 -28.9% 4873551 ± 2% interrupts.CPU10.RES:Rescheduling_interrupts
6934808 ± 2% -29.7% 4873699 ± 2% interrupts.CPU11.RES:Rescheduling_interrupts
6816693 ± 3% -28.8% 4850530 ± 2% interrupts.CPU12.RES:Rescheduling_interrupts
365.00 ± 4% +111.0% 770.00 ± 72% interrupts.CPU13.34:PCI-MSI.3145731-edge.eth0-TxRx-2
6819399 -27.9% 4914674 interrupts.CPU13.RES:Rescheduling_interrupts
6691456 ± 2% -28.0% 4818210 ± 2% interrupts.CPU14.RES:Rescheduling_interrupts
6653702 -28.0% 4793836 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts
6635361 ± 3% -28.0% 4774200 ± 2% interrupts.CPU16.RES:Rescheduling_interrupts
368.25 ± 3% +202.4% 1113 ± 21% interrupts.CPU17.38:PCI-MSI.3145735-edge.eth0-TxRx-6
6712978 ± 2% -28.0% 4835779 ± 2% interrupts.CPU17.RES:Rescheduling_interrupts
6799911 ± 2% -28.4% 4867787 ± 2% interrupts.CPU18.RES:Rescheduling_interrupts
6699582 ± 2% -27.5% 4854616 interrupts.CPU19.RES:Rescheduling_interrupts
6740388 -28.0% 4851379 ± 2% interrupts.CPU2.RES:Rescheduling_interrupts
6650982 -27.6% 4816379 interrupts.CPU20.RES:Rescheduling_interrupts
6680083 -28.3% 4789122 ± 2% interrupts.CPU21.RES:Rescheduling_interrupts
6864478 -30.9% 4740013 interrupts.CPU22.RES:Rescheduling_interrupts
6740867 ± 2% -28.9% 4792089 interrupts.CPU23.RES:Rescheduling_interrupts
6896995 ± 2% -30.6% 4785343 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts
6865754 ± 2% -29.1% 4868069 interrupts.CPU25.RES:Rescheduling_interrupts
6936961 ± 3% -30.0% 4855447 interrupts.CPU26.RES:Rescheduling_interrupts
6855283 -29.2% 4855403 interrupts.CPU27.RES:Rescheduling_interrupts
6729254 ± 3% -28.4% 4816468 ± 2% interrupts.CPU28.RES:Rescheduling_interrupts
6785228 ± 4% -29.4% 4789382 interrupts.CPU29.RES:Rescheduling_interrupts
6741511 ± 2% -27.7% 4874484 ± 2% interrupts.CPU3.RES:Rescheduling_interrupts
6855117 -29.2% 4855395 interrupts.CPU30.RES:Rescheduling_interrupts
6914858 ± 2% -30.1% 4834204 ± 2% interrupts.CPU31.RES:Rescheduling_interrupts
6960073 -30.2% 4856967 interrupts.CPU32.RES:Rescheduling_interrupts
6836780 ± 3% -29.6% 4813056 ± 2% interrupts.CPU33.RES:Rescheduling_interrupts
6721770 ± 2% -28.1% 4834958 interrupts.CPU34.RES:Rescheduling_interrupts
6975150 -30.1% 4876972 interrupts.CPU35.RES:Rescheduling_interrupts
6702112 ± 2% -28.5% 4794082 interrupts.CPU36.RES:Rescheduling_interrupts
6673649 ± 2% -28.8% 4752937 interrupts.CPU37.RES:Rescheduling_interrupts
6725653 ± 2% -29.3% 4753467 interrupts.CPU38.RES:Rescheduling_interrupts
6724426 ± 3% -28.0% 4843223 ± 2% interrupts.CPU39.RES:Rescheduling_interrupts
6723671 -27.8% 4852800 interrupts.CPU4.RES:Rescheduling_interrupts
6817451 ± 2% -28.9% 4845853 ± 2% interrupts.CPU40.RES:Rescheduling_interrupts
6804144 ± 3% -28.1% 4891997 interrupts.CPU41.RES:Rescheduling_interrupts
6632274 ± 2% -27.9% 4782346 interrupts.CPU42.RES:Rescheduling_interrupts
6586976 ± 3% -27.7% 4762757 interrupts.CPU43.RES:Rescheduling_interrupts
6926563 -30.6% 4803685 interrupts.CPU44.RES:Rescheduling_interrupts
6714525 ± 2% -29.2% 4755428 ± 2% interrupts.CPU45.RES:Rescheduling_interrupts
6665899 ± 3% -28.6% 4760837 ± 2% interrupts.CPU46.RES:Rescheduling_interrupts
6755978 ± 2% -28.1% 4857579 interrupts.CPU47.RES:Rescheduling_interrupts
6836794 ± 3% -29.0% 4850743 interrupts.CPU48.RES:Rescheduling_interrupts
6924153 ± 2% -30.5% 4813870 ± 2% interrupts.CPU49.RES:Rescheduling_interrupts
6774757 ± 2% -28.1% 4874202 ± 2% interrupts.CPU5.RES:Rescheduling_interrupts
6702070 -28.0% 4822681 interrupts.CPU50.RES:Rescheduling_interrupts
6671394 ± 2% -28.4% 4778999 ± 2% interrupts.CPU51.RES:Rescheduling_interrupts
6826885 ± 3% -30.0% 4781231 ± 2% interrupts.CPU52.RES:Rescheduling_interrupts
6668360 ± 2% -27.1% 4862470 interrupts.CPU53.RES:Rescheduling_interrupts
6858566 -29.8% 4816316 interrupts.CPU54.RES:Rescheduling_interrupts
6759358 ± 3% -28.3% 4848673 interrupts.CPU55.RES:Rescheduling_interrupts
6752670 ± 2% -27.8% 4874015 interrupts.CPU56.RES:Rescheduling_interrupts
6841827 ± 2% -28.7% 4875468 ± 2% interrupts.CPU57.RES:Rescheduling_interrupts
6736995 -29.0% 4785853 interrupts.CPU58.RES:Rescheduling_interrupts
6547489 ± 2% -26.7% 4798514 interrupts.CPU59.RES:Rescheduling_interrupts
6736541 ± 2% -28.4% 4822184 interrupts.CPU6.RES:Rescheduling_interrupts
6594592 ± 2% -27.9% 4755955 ± 2% interrupts.CPU60.RES:Rescheduling_interrupts
6727460 ± 2% -28.1% 4836439 interrupts.CPU61.RES:Rescheduling_interrupts
6766302 ± 2% -28.5% 4841025 interrupts.CPU62.RES:Rescheduling_interrupts
6604374 ± 4% -27.8% 4767606 interrupts.CPU63.RES:Rescheduling_interrupts
6592012 -27.5% 4781455 ± 2% interrupts.CPU64.RES:Rescheduling_interrupts
6725764 ± 2% -28.8% 4785414 ± 2% interrupts.CPU65.RES:Rescheduling_interrupts
6639222 -29.4% 4686246 ± 2% interrupts.CPU66.RES:Rescheduling_interrupts
6631806 ± 3% -28.5% 4739015 interrupts.CPU67.RES:Rescheduling_interrupts
6868378 ± 2% -30.6% 4767131 interrupts.CPU68.RES:Rescheduling_interrupts
6799940 ± 2% -29.1% 4822426 interrupts.CPU69.RES:Rescheduling_interrupts
6749325 ± 3% -29.3% 4771694 interrupts.CPU7.RES:Rescheduling_interrupts
6911649 ± 3% -30.6% 4794836 interrupts.CPU70.RES:Rescheduling_interrupts
6835631 -29.2% 4842020 ± 2% interrupts.CPU71.RES:Rescheduling_interrupts
6700674 -28.2% 4807931 ± 2% interrupts.CPU72.RES:Rescheduling_interrupts
6761278 ± 3% -29.9% 4742785 ± 2% interrupts.CPU73.RES:Rescheduling_interrupts
6947730 -30.1% 4855238 interrupts.CPU74.RES:Rescheduling_interrupts
6909693 -30.3% 4818676 interrupts.CPU75.RES:Rescheduling_interrupts
6917690 ± 3% -30.4% 4815754 interrupts.CPU76.RES:Rescheduling_interrupts
6831728 ± 3% -29.4% 4822997 interrupts.CPU77.RES:Rescheduling_interrupts
6788225 ± 3% -28.8% 4834983 ± 2% interrupts.CPU78.RES:Rescheduling_interrupts
6932726 ± 3% -30.0% 4854798 interrupts.CPU79.RES:Rescheduling_interrupts
6803021 -29.2% 4817096 interrupts.CPU8.RES:Rescheduling_interrupts
6752792 ± 2% -29.4% 4769949 interrupts.CPU80.RES:Rescheduling_interrupts
6616018 -28.4% 4735095 interrupts.CPU81.RES:Rescheduling_interrupts
6729256 ± 3% -30.0% 4707562 interrupts.CPU82.RES:Rescheduling_interrupts
6764958 ± 3% -28.6% 4826819 interrupts.CPU83.RES:Rescheduling_interrupts
6846946 ± 2% -30.2% 4781840 ± 2% interrupts.CPU84.RES:Rescheduling_interrupts
6830674 ± 2% -29.2% 4836187 interrupts.CPU85.RES:Rescheduling_interrupts
6792707 -29.9% 4762613 interrupts.CPU86.RES:Rescheduling_interrupts
6712127 -29.4% 4741523 ± 2% interrupts.CPU87.RES:Rescheduling_interrupts
6761001 ± 2% -27.7% 4887497 interrupts.CPU9.RES:Rescheduling_interrupts
5.96e+08 -28.9% 4.238e+08 interrupts.RES:Rescheduling_interrupts
92962 -18.7% 75542 ± 3% softirqs.CPU0.RCU
13931 ± 2% +260.5% 50217 ± 2% softirqs.CPU0.SCHED
96508 -20.0% 77185 ± 3% softirqs.CPU1.RCU
10554 ± 3% +317.1% 44024 softirqs.CPU1.SCHED
95409 -20.0% 76299 ± 4% softirqs.CPU10.RCU
10819 ± 3% +307.0% 44033 softirqs.CPU10.SCHED
94891 -20.1% 75807 ± 3% softirqs.CPU11.RCU
10527 ± 5% +320.2% 44235 softirqs.CPU11.SCHED
95242 ± 2% -19.7% 76447 ± 4% softirqs.CPU12.RCU
10484 ± 3% +328.4% 44917 softirqs.CPU12.SCHED
94166 ± 2% -18.9% 76353 ± 4% softirqs.CPU13.RCU
10486 ± 3% +323.1% 44372 softirqs.CPU13.SCHED
96327 ± 2% -21.0% 76077 ± 2% softirqs.CPU14.RCU
10212 ± 5% +325.8% 43486 softirqs.CPU14.SCHED
102082 -20.9% 80746 ± 2% softirqs.CPU15.RCU
10185 ± 2% +325.9% 43378 softirqs.CPU15.SCHED
102036 -21.0% 80631 ± 2% softirqs.CPU16.RCU
10366 ± 2% +320.8% 43619 softirqs.CPU16.SCHED
100603 -20.0% 80489 ± 2% softirqs.CPU17.RCU
10605 ± 3% +316.8% 44209 softirqs.CPU17.SCHED
101509 -20.5% 80658 ± 2% softirqs.CPU18.RCU
10575 ± 2% +314.3% 43814 softirqs.CPU18.SCHED
102886 -20.6% 81724 ± 2% softirqs.CPU19.RCU
10671 ± 3% +308.6% 43606 softirqs.CPU19.SCHED
95821 -18.8% 77782 ± 4% softirqs.CPU2.RCU
10851 ± 5% +304.1% 43855 softirqs.CPU2.SCHED
102059 -20.8% 80828 ± 2% softirqs.CPU20.RCU
10142 ± 2% +321.8% 42774 softirqs.CPU20.SCHED
99755 ± 5% -18.8% 80988 ± 2% softirqs.CPU21.RCU
10541 ± 4% +307.9% 43003 softirqs.CPU21.SCHED
221231 -9.3% 200704 softirqs.CPU21.TIMER
105312 -23.2% 80910 ± 2% softirqs.CPU22.RCU
11663 ± 4% +275.8% 43830 ± 2% softirqs.CPU22.SCHED
227611 ± 3% -10.0% 204892 softirqs.CPU22.TIMER
104002 -21.6% 81565 ± 2% softirqs.CPU23.RCU
10621 ± 6% +317.2% 44317 softirqs.CPU23.SCHED
103130 -20.2% 82248 softirqs.CPU24.RCU
10719 ± 4% +306.2% 43542 softirqs.CPU24.SCHED
102405 -21.2% 80674 softirqs.CPU25.RCU
10932 ± 5% +304.1% 44179 softirqs.CPU25.SCHED
102786 -21.2% 80983 ± 2% softirqs.CPU26.RCU
10806 ± 4% +308.8% 44172 softirqs.CPU26.SCHED
101863 -19.5% 81978 ± 3% softirqs.CPU27.RCU
10505 ± 2% +319.8% 44098 softirqs.CPU27.SCHED
103190 -17.4% 85190 ± 3% softirqs.CPU28.RCU
10659 ± 2% +312.1% 43923 softirqs.CPU28.SCHED
103332 -21.7% 80923 ± 2% softirqs.CPU29.RCU
10662 ± 3% +312.3% 43957 softirqs.CPU29.SCHED
95324 -19.9% 76371 ± 3% softirqs.CPU3.RCU
10645 ± 6% +311.2% 43776 softirqs.CPU3.SCHED
98433 -20.0% 78701 ± 2% softirqs.CPU30.RCU
10774 +312.4% 44435 softirqs.CPU30.SCHED
97890 ± 2% -19.2% 79084 softirqs.CPU31.RCU
10707 ± 3% +312.9% 44211 softirqs.CPU31.SCHED
97710 ± 2% -19.4% 78744 ± 2% softirqs.CPU32.RCU
10583 ± 5% +317.4% 44171 softirqs.CPU32.SCHED
98458 -20.3% 78451 ± 2% softirqs.CPU33.RCU
10629 ± 5% +319.1% 44551 softirqs.CPU33.SCHED
98545 ± 2% -20.0% 78830 ± 3% softirqs.CPU34.RCU
10586 ± 3% +319.3% 44386 softirqs.CPU34.SCHED
244797 ± 18% -17.6% 201621 softirqs.CPU34.TIMER
98005 -20.4% 78036 ± 2% softirqs.CPU35.RCU
10835 ± 5% +310.1% 44439 softirqs.CPU35.SCHED
100403 -20.8% 79504 ± 2% softirqs.CPU36.RCU
10980 ± 4% +296.5% 43539 softirqs.CPU36.SCHED
101241 -20.8% 80179 ± 3% softirqs.CPU37.RCU
10528 ± 5% +312.8% 43459 softirqs.CPU37.SCHED
222768 ± 2% -8.1% 204624 softirqs.CPU37.TIMER
100650 -21.2% 79265 ± 2% softirqs.CPU38.RCU
10642 ± 4% +312.5% 43895 softirqs.CPU38.SCHED
223043 ± 3% -9.2% 202469 softirqs.CPU38.TIMER
98707 -20.5% 78451 ± 3% softirqs.CPU39.RCU
10600 ± 4% +317.5% 44257 softirqs.CPU39.SCHED
220545 ± 2% -9.1% 200578 softirqs.CPU39.TIMER
95521 -19.8% 76615 ± 3% softirqs.CPU4.RCU
10762 ± 2% +306.9% 43793 softirqs.CPU4.SCHED
98348 ± 2% -20.5% 78188 ± 3% softirqs.CPU40.RCU
10846 ± 5% +311.5% 44631 softirqs.CPU40.SCHED
218178 ± 2% -8.4% 199745 softirqs.CPU40.TIMER
98402 -20.2% 78520 ± 2% softirqs.CPU41.RCU
10492 ± 7% +323.2% 44405 softirqs.CPU41.SCHED
101129 -21.5% 79397 ± 3% softirqs.CPU42.RCU
10606 ± 2% +313.4% 43847 softirqs.CPU42.SCHED
223525 ± 2% -9.3% 202745 softirqs.CPU42.TIMER
98305 ± 3% -19.4% 79193 ± 2% softirqs.CPU43.RCU
10352 ± 2% +323.2% 43810 softirqs.CPU43.SCHED
225884 ± 3% -10.4% 202477 softirqs.CPU43.TIMER
84026 -16.8% 69869 ± 3% softirqs.CPU44.RCU
9995 ± 3% +337.3% 43715 softirqs.CPU44.SCHED
101636 -23.6% 77644 ± 3% softirqs.CPU45.RCU
10579 ± 2% +302.8% 42615 softirqs.CPU45.SCHED
97996 -20.9% 77481 ± 3% softirqs.CPU46.RCU
10566 ± 3% +308.8% 43190 softirqs.CPU46.SCHED
96184 -20.0% 76985 ± 3% softirqs.CPU47.RCU
10556 ± 5% +312.1% 43498 softirqs.CPU47.SCHED
97247 -20.3% 77530 ± 2% softirqs.CPU48.RCU
10820 ± 4% +302.2% 43515 softirqs.CPU48.SCHED
96289 ± 2% -19.9% 77095 ± 3% softirqs.CPU49.RCU
10536 ± 6% +314.4% 43659 softirqs.CPU49.SCHED
95298 -19.7% 76514 ± 3% softirqs.CPU5.RCU
10205 ± 6% +334.1% 44296 softirqs.CPU5.SCHED
98217 ± 2% -20.9% 77674 ± 3% softirqs.CPU50.RCU
10416 ± 4% +314.3% 43158 softirqs.CPU50.SCHED
97946 -20.8% 77584 ± 3% softirqs.CPU51.RCU
10672 ± 4% +302.8% 42989 softirqs.CPU51.SCHED
97584 -20.4% 77645 ± 3% softirqs.CPU52.RCU
10584 ± 2% +308.1% 43201 softirqs.CPU52.SCHED
96102 ± 2% -19.6% 77313 ± 2% softirqs.CPU53.RCU
10298 ± 2% +323.3% 43597 softirqs.CPU53.SCHED
96900 ± 2% -21.0% 76543 ± 4% softirqs.CPU54.RCU
10627 ± 3% +311.0% 43682 softirqs.CPU54.SCHED
95916 ± 2% -19.6% 77106 ± 2% softirqs.CPU55.RCU
10426 ± 5% +322.0% 44002 softirqs.CPU55.SCHED
96069 ± 2% -19.8% 77051 ± 2% softirqs.CPU56.RCU
10378 ± 3% +320.9% 43684 softirqs.CPU56.SCHED
95003 -18.9% 77092 ± 2% softirqs.CPU57.RCU
10468 ± 4% +314.7% 43411 softirqs.CPU57.SCHED
96701 -19.7% 77680 ± 2% softirqs.CPU58.RCU
10006 +330.3% 43055 softirqs.CPU58.SCHED
97043 -20.1% 77567 ± 2% softirqs.CPU59.RCU
10225 +319.5% 42894 softirqs.CPU59.SCHED
97522 ± 2% -20.2% 77835 ± 4% softirqs.CPU6.RCU
10659 +307.1% 43394 softirqs.CPU6.SCHED
100347 -21.0% 79285 ± 2% softirqs.CPU60.RCU
10254 ± 2% +320.0% 43070 softirqs.CPU60.SCHED
99248 -20.7% 78704 ± 2% softirqs.CPU61.RCU
10704 ± 4% +305.7% 43432 softirqs.CPU61.SCHED
99634 -20.3% 79398 softirqs.CPU62.RCU
10568 ± 2% +311.4% 43479 softirqs.CPU62.SCHED
100503 -21.1% 79274 ± 2% softirqs.CPU63.RCU
10491 ± 4% +312.2% 43244 softirqs.CPU63.SCHED
100247 -21.1% 79123 ± 2% softirqs.CPU64.RCU
10364 ± 2% +313.2% 42821 softirqs.CPU64.SCHED
100138 ± 2% -21.2% 78957 ± 2% softirqs.CPU65.RCU
10461 ± 2% +310.3% 42926 softirqs.CPU65.SCHED
103527 ± 2% -22.0% 80741 ± 2% softirqs.CPU66.RCU
10762 ± 4% +302.0% 43263 softirqs.CPU66.SCHED
104070 ± 2% -22.2% 80967 ± 2% softirqs.CPU67.RCU
10828 ± 7% +299.2% 43228 softirqs.CPU67.SCHED
223091 ± 3% -9.2% 202637 softirqs.CPU67.TIMER
102873 -21.7% 80565 ± 2% softirqs.CPU68.RCU
10901 ± 3% +295.9% 43159 softirqs.CPU68.SCHED
102451 ± 2% -21.8% 80100 ± 2% softirqs.CPU69.RCU
10740 ± 4% +309.7% 43998 softirqs.CPU69.SCHED
95715 -19.4% 77139 ± 3% softirqs.CPU7.RCU
10318 ± 3% +323.5% 43701 softirqs.CPU7.SCHED
102978 -22.7% 79644 softirqs.CPU70.RCU
10823 ± 4% +304.6% 43794 softirqs.CPU70.SCHED
102079 -20.7% 80937 softirqs.CPU71.RCU
10637 +316.9% 44350 softirqs.CPU71.SCHED
103658 -21.6% 81281 ± 2% softirqs.CPU72.RCU
10635 +307.6% 43349 softirqs.CPU72.SCHED
102899 ± 2% -21.6% 80688 softirqs.CPU73.RCU
10735 ± 4% +307.2% 43712 softirqs.CPU73.SCHED
99856 ± 3% -19.6% 80248 softirqs.CPU74.RCU
10964 +304.4% 44344 softirqs.CPU74.SCHED
90192 -18.8% 73261 ± 2% softirqs.CPU75.RCU
10598 ± 5% +316.7% 44163 softirqs.CPU75.SCHED
89840 -18.8% 72908 ± 2% softirqs.CPU76.RCU
10630 ± 3% +312.1% 43809 softirqs.CPU76.SCHED
89640 -18.0% 73485 ± 2% softirqs.CPU77.RCU
10839 +309.8% 44423 softirqs.CPU77.SCHED
87179 ± 5% -15.5% 73665 ± 2% softirqs.CPU78.RCU
10674 ± 3% +312.1% 43993 softirqs.CPU78.SCHED
220341 ± 3% -9.2% 200067 softirqs.CPU78.TIMER
89307 -17.7% 73537 ± 2% softirqs.CPU79.RCU
10950 ± 3% +303.6% 44197 softirqs.CPU79.SCHED
95608 -19.3% 77194 ± 3% softirqs.CPU8.RCU
10484 ± 2% +316.3% 43648 softirqs.CPU8.SCHED
90965 -18.5% 74151 ± 2% softirqs.CPU80.RCU
10708 ± 5% +303.3% 43186 softirqs.CPU80.SCHED
91430 -18.9% 74154 ± 2% softirqs.CPU81.RCU
10737 ± 3% +302.8% 43254 softirqs.CPU81.SCHED
92220 -19.8% 73956 ± 2% softirqs.CPU82.RCU
10521 ± 4% +310.0% 43141 softirqs.CPU82.SCHED
221147 ± 3% -8.9% 201404 softirqs.CPU82.TIMER
89022 -17.6% 73337 ± 2% softirqs.CPU83.RCU
10382 ± 4% +323.3% 43952 softirqs.CPU83.SCHED
89261 -17.5% 73609 ± 2% softirqs.CPU84.RCU
10602 ± 6% +313.8% 43876 softirqs.CPU84.SCHED
89312 -17.9% 73296 ± 2% softirqs.CPU85.RCU
10686 ± 5% +311.7% 43999 softirqs.CPU85.SCHED
90674 -18.2% 74195 ± 2% softirqs.CPU86.RCU
10395 ± 2% +317.6% 43411 softirqs.CPU86.SCHED
221731 ± 2% -9.1% 201509 softirqs.CPU86.TIMER
89796 ± 2% -17.6% 74023 ± 2% softirqs.CPU87.RCU
10273 ± 4% +320.6% 43212 softirqs.CPU87.SCHED
248569 ± 18% -18.5% 202480 softirqs.CPU87.TIMER
94895 -19.2% 76663 ± 3% softirqs.CPU9.RCU
10420 ± 3% +321.5% 43925 softirqs.CPU9.SCHED
8588648 -20.1% 6862303 ± 2% softirqs.RCU
935243 +312.2% 3854839 softirqs.SCHED
87.82 -72.0 15.78 ± 20% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
87.04 -71.4 15.65 ± 20% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
36.18 ± 2% -28.7 7.47 ± 17% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.00 ± 2% -27.8 7.22 ± 17% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.95 -25.9 5.02 ± 23% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
29.88 -25.0 4.85 ± 23% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
22.20 ± 7% -8.3 13.87 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
21.57 ± 8% -8.2 13.38 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
21.34 ± 8% -8.1 13.21 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
24.03 ± 6% -7.1 16.89 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
17.73 -6.8 10.96 ± 3% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
6.54 ± 8% -6.4 0.16 ±173% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.39 ± 8% -6.2 0.16 ±173% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.77 -6.0 9.77 ± 3% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
25.28 -6.0 19.33 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
25.94 -5.9 20.03 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.47 -5.8 9.62 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
6.25 ± 8% -4.4 1.88 ± 11% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.70 ± 30% -3.8 2.87 ± 9% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
5.10 ± 38% -3.4 1.72 ± 15% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.01 ± 4% -3.2 0.79 ± 21% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
3.65 ± 5% -2.9 0.73 ± 20% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
6.75 -2.8 3.99 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
8.00 ± 2% -2.7 5.29 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
7.94 ± 2% -2.7 5.26 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
7.71 ± 3% -2.6 5.14 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
2.90 ± 48% -2.2 0.68 ± 3% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
31.48 ± 3% -1.7 29.82 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
32.03 ± 3% -1.5 30.49 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.07 ± 9% -1.5 0.54 ± 6% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
1.93 ± 7% -1.4 0.55 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
3.50 -1.3 2.16 ± 11% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
2.43 -1.3 1.15 ± 6% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read
2.71 ± 2% -1.2 1.53 ± 12% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.66 ± 8% -1.1 0.58 ± 2% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
1.99 -1.1 0.92 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
1.59 ± 9% -1.0 0.64 ± 2% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.10 -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.pipe_wait
1.07 -0.7 0.39 ± 57% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.08 ± 2% -0.7 0.40 ± 57% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.36 -0.7 0.69 ± 3% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.00 -0.6 0.42 ± 57% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.12 ± 2% -0.5 0.57 ± 4% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.14 ± 5% -0.5 0.59 ± 4% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.49 ± 4% -0.2 1.27 ± 4% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
0.74 ± 2% -0.2 0.53 ± 2% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.59 ± 4% -0.2 1.40 ± 4% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
0.60 ± 4% +0.1 0.71 ± 5% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
0.65 ± 2% +0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
1.12 ± 6% +0.2 1.31 ± 5% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.40 ± 57% +0.3 0.65 ± 5% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read
1.89 ± 7% +0.3 2.19 ± 6% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.85 ± 6% +0.3 2.15 ± 7% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
2.41 ± 4% +0.4 2.80 ± 6% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.6 0.56 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.00 +0.6 0.57 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.00 +0.6 0.57 ± 3% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.54 ± 6% +0.6 2.14 ± 7% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +1.0 1.00 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.1 1.07 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_write.new_sync_write
0.00 +1.1 1.09 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_write.new_sync_write.vfs_write
0.80 ± 7% +1.2 1.96 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
1.06 ± 8% +1.3 2.37 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
0.00 +1.5 1.50 ± 17% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.00 +1.8 1.77 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.80 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +1.8 1.83 ± 5% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_write.new_sync_write.vfs_write.ksys_write
0.00 +1.9 1.94 ± 14% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
0.00 +1.9 1.95 ± 13% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +2.0 2.05 ± 4% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__GI___libc_write
0.00 +2.1 2.07 ± 5% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__GI___libc_read
0.00 +2.4 2.44 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_write
0.00 +2.5 2.50 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_read
0.00 +3.0 2.96 ± 10% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +3.0 2.97 ± 10% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +3.0 2.97 ± 10% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
0.00 +3.0 3.02 ± 10% perf-profile.calltrace.cycles-pp.secondary_startup_64
0.00 +20.0 20.00 ± 4% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.00 +20.9 20.90 ± 4% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.00 +27.1 27.11 ± 4% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +28.4 28.38 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.00 +28.4 28.39 ± 4% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +28.7 28.72 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.00 +34.9 34.95 ± 4% perf-profile.calltrace.cycles-pp.__GI___libc_read
0.00 +37.3 37.34 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +37.6 37.63 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.00 +43.0 43.03 ± 4% perf-profile.calltrace.cycles-pp.__GI___libc_write
22.34 ± 2% -8.9 13.44 ± 2% perf-profile.children.cycles-pp.schedule
22.38 -8.1 14.25 ± 2% perf-profile.children.cycles-pp.__schedule
22.27 ± 7% -8.1 14.18 ± 2% perf-profile.children.cycles-pp.__wake_up_common
21.76 ± 7% -8.0 13.80 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
21.60 ± 8% -8.0 13.65 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
24.44 ± 6% -7.0 17.43 perf-profile.children.cycles-pp.__wake_up_common_lock
25.34 -5.9 19.41 perf-profile.children.cycles-pp.pipe_read
25.97 -5.9 20.07 perf-profile.children.cycles-pp.new_sync_read
87.94 -5.6 82.29 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
87.17 -5.6 81.54 perf-profile.children.cycles-pp.do_syscall_64
29.93 -5.0 24.89 perf-profile.children.cycles-pp.vfs_read
30.98 -5.0 25.95 perf-profile.children.cycles-pp.ksys_read
17.86 -4.9 12.94 ± 2% perf-profile.children.cycles-pp.pipe_wait
6.63 ± 7% -4.1 2.50 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
6.73 ± 30% -3.6 3.09 ± 3% perf-profile.children.cycles-pp.select_task_rq_fair
5.18 ± 37% -3.0 2.14 ± 3% perf-profile.children.cycles-pp.select_idle_sibling
8.08 ± 2% -2.5 5.54 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate
8.02 ± 2% -2.5 5.50 ± 2% perf-profile.children.cycles-pp.activate_task
7.82 ± 2% -2.4 5.39 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
6.81 -2.4 4.44 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
4.64 ± 3% -2.0 2.60 ± 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
2.94 ± 48% -2.0 0.98 ± 4% perf-profile.children.cycles-pp.available_idle_cpu
4.20 ± 2% -1.7 2.47 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
31.54 ± 3% -1.6 29.90 perf-profile.children.cycles-pp.pipe_write
32.06 ± 3% -1.5 30.52 perf-profile.children.cycles-pp.new_sync_write
3.35 ± 8% -1.5 1.88 ± 4% perf-profile.children.cycles-pp.update_cfs_group
3.47 -1.2 2.30 perf-profile.children.cycles-pp.update_load_avg
2.33 ± 3% -1.1 1.25 ± 3% perf-profile.children.cycles-pp.load_new_mm_cr3
3.35 ± 2% -1.0 2.33 ± 2% perf-profile.children.cycles-pp.update_curr
3.56 -0.9 2.65 ± 2% perf-profile.children.cycles-pp.enqueue_entity
2.82 ± 2% -0.8 2.07 perf-profile.children.cycles-pp.dequeue_entity
1.47 ± 5% -0.7 0.72 ± 3% perf-profile.children.cycles-pp.switch_fpu_return
2.25 -0.7 1.51 perf-profile.children.cycles-pp.reweight_entity
1.13 ± 8% -0.7 0.41 ± 3% perf-profile.children.cycles-pp.set_task_cpu
1.57 ± 8% -0.7 0.88 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
1.31 ± 6% -0.6 0.70 ± 4% perf-profile.children.cycles-pp.__switch_to_asm
1.29 ± 4% -0.6 0.72 ± 2% perf-profile.children.cycles-pp.__switch_to
1.38 ± 10% -0.5 0.89 ± 4% perf-profile.children.cycles-pp.update_rq_clock
0.74 ± 44% -0.5 0.25 ± 2% perf-profile.children.cycles-pp.cpumask_next_wrap
1.02 ± 3% -0.5 0.56 ± 4% perf-profile.children.cycles-pp.native_write_msr
0.85 ± 6% -0.4 0.40 ± 2% perf-profile.children.cycles-pp.pick_next_entity
0.83 ± 3% -0.4 0.40 ± 4% perf-profile.children.cycles-pp.put_prev_entity
1.28 -0.4 0.87 perf-profile.children.cycles-pp.set_next_entity
0.60 ± 12% -0.4 0.25 ± 3% perf-profile.children.cycles-pp.migrate_task_rq_fair
1.19 -0.3 0.84 ± 2% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.33 -0.3 0.98 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se
1.19 -0.3 0.86 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.85 ± 3% -0.3 0.53 ± 4% perf-profile.children.cycles-pp.__enqueue_entity
0.78 ± 2% -0.3 0.47 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup
1.01 -0.3 0.72 ± 3% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.93 -0.3 0.64 ± 3% perf-profile.children.cycles-pp.check_preempt_curr
0.64 ± 2% -0.3 0.37 ± 9% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.67 ± 2% -0.2 0.42 perf-profile.children.cycles-pp.__calc_delta
0.94 ± 13% -0.2 0.70 ± 2% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.74 ± 6% -0.2 0.54 ± 6% perf-profile.children.cycles-pp.finish_task_switch
0.47 ± 32% -0.2 0.28 ± 2% perf-profile.children.cycles-pp.find_next_bit
0.34 ± 10% -0.2 0.17 ± 5% perf-profile.children.cycles-pp.cpus_share_cache
0.33 ± 5% -0.2 0.15 ± 2% perf-profile.children.cycles-pp.clear_buddies
0.46 ± 3% -0.2 0.29 ± 3% perf-profile.children.cycles-pp.rb_erase_cached
0.56 ± 4% -0.2 0.40 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.51 ± 3% -0.2 0.35 ± 3% perf-profile.children.cycles-pp.sched_clock
0.38 ± 11% -0.1 0.23 ± 2% perf-profile.children.cycles-pp.cpuacct_charge
0.46 ± 3% -0.1 0.32 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.28 ± 7% -0.1 0.14 ± 3% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.45 ± 2% -0.1 0.32 ± 5% perf-profile.children.cycles-pp.update_min_vruntime
0.55 ± 4% -0.1 0.43 perf-profile.children.cycles-pp.account_entity_dequeue
0.22 ± 11% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.remove_entity_load_avg
0.17 ± 17% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg
0.40 -0.1 0.30 ± 2% perf-profile.children.cycles-pp.account_entity_enqueue
0.18 ± 8% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.20 ± 7% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.native_load_tls
0.15 ± 5% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.rcu_note_context_switch
0.21 ± 19% -0.1 0.13 ± 5% perf-profile.children.cycles-pp.wake_q_add
0.19 ± 4% -0.1 0.12 ± 6% perf-profile.children.cycles-pp.set_next_buddy
0.09 -0.1 0.03 ±100% perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.10 ± 5% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.perf_exclude_event
0.17 ± 2% -0.0 0.12 ± 4% perf-profile.children.cycles-pp.resched_curr
0.15 ± 4% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.rb_insert_color_cached
0.06 ± 6% -0.0 0.03 ±100% perf-profile.children.cycles-pp.clockevents_program_event
0.22 ± 6% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.cpumask_next
0.15 ± 5% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.rb_next
0.12 ± 5% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.__list_add_valid
0.08 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.default_send_IPI_single_phys
0.06 ± 15% +0.0 0.08 ± 26% perf-profile.children.cycles-pp.__vfs_read
0.06 ± 6% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.kill_fasync
0.12 ± 8% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.iov_iter_init
0.04 ± 57% +0.0 0.07 ± 17% perf-profile.children.cycles-pp.irq_exit
0.13 ± 5% +0.0 0.17 ± 13% perf-profile.children.cycles-pp.scheduler_tick
0.19 ± 2% +0.0 0.24 ± 11% perf-profile.children.cycles-pp.tick_sched_timer
0.11 ± 6% +0.0 0.15 ± 15% perf-profile.children.cycles-pp.task_tick_fair
0.26 ± 2% +0.0 0.31 ± 8% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.17 ± 2% +0.0 0.22 ± 10% perf-profile.children.cycles-pp.tick_sched_handle
0.17 ± 2% +0.0 0.22 ± 10% perf-profile.children.cycles-pp.update_process_times
0.11 ± 11% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.11 ± 9% +0.1 0.17 ± 3% perf-profile.children.cycles-pp.__sb_end_write
0.23 ± 7% +0.1 0.28 ± 2% perf-profile.children.cycles-pp.rw_verify_area
0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.get_page_from_freelist
0.01 ±173% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.bpf_fd_pass
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.__next_timer_interrupt
0.00 +0.1 0.06 ± 22% perf-profile.children.cycles-pp.__softirqentry_text_start
0.47 ± 3% +0.1 0.53 ± 8% perf-profile.children.cycles-pp.apic_timer_interrupt
0.14 ± 5% +0.1 0.21 ± 2% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.01 ±173% +0.1 0.09 ± 5% perf-profile.children.cycles-pp.schedule_preempt_disabled
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.14 ± 7% +0.1 0.22 ± 3% perf-profile.children.cycles-pp.timespec64_trunc
0.17 ± 6% +0.1 0.26 perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.89 ± 4% +0.1 0.98 ± 3% perf-profile.children.cycles-pp.touch_atime
0.55 ± 5% +0.1 0.65 ± 2% perf-profile.children.cycles-pp.atime_needs_update
0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.tick_nohz_next_event
0.23 ± 6% +0.1 0.33 ± 2% perf-profile.children.cycles-pp.rcu_all_qs
0.27 ± 18% +0.1 0.37 ± 2% perf-profile.children.cycles-pp.preempt_schedule_common
0.13 ± 17% +0.1 0.24 ± 4% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.14 ± 15% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.16 ± 7% +0.1 0.28 ± 2% perf-profile.children.cycles-pp.inode_has_perm
0.00 +0.1 0.13 ± 5% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.60 ± 4% +0.1 1.73 perf-profile.children.cycles-pp.copyout
0.66 ± 5% +0.1 0.79 perf-profile.children.cycles-pp.file_update_time
0.63 ± 7% +0.1 0.77 perf-profile.children.cycles-pp.__fsnotify_parent
1.67 ± 2% +0.1 1.82 ± 2% perf-profile.children.cycles-pp.__fget_light
0.50 ± 5% +0.2 0.66 ± 2% perf-profile.children.cycles-pp.__might_fault
1.76 ± 2% +0.2 1.94 perf-profile.children.cycles-pp.__fdget_pos
0.53 ± 7% +0.2 0.73 ± 2% perf-profile.children.cycles-pp.current_time
0.66 ± 5% +0.2 0.88 ± 2% perf-profile.children.cycles-pp.__might_sleep
1.01 ± 5% +0.2 1.25 perf-profile.children.cycles-pp.copyin
0.00 +0.2 0.24 ± 4% perf-profile.children.cycles-pp.menu_select
0.85 ± 4% +0.2 1.10 ± 2% perf-profile.children.cycles-pp.fsnotify
0.84 ± 3% +0.3 1.10 ± 2% perf-profile.children.cycles-pp.avc_has_perm
0.74 ± 11% +0.3 1.03 perf-profile.children.cycles-pp._cond_resched
0.91 ± 6% +0.3 1.22 perf-profile.children.cycles-pp.___might_sleep
2.52 ± 4% +0.3 2.83 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.42 ± 2% +0.3 0.74 ± 3% perf-profile.children.cycles-pp.prepare_to_wait
0.00 +0.3 0.35 ± 33% perf-profile.children.cycles-pp.osq_lock
0.73 ± 8% +0.4 1.10 perf-profile.children.cycles-pp.__inode_security_revalidate
0.09 ± 20% +0.4 0.46 ± 6% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.00 +0.4 0.38 ± 3% perf-profile.children.cycles-pp.poll_idle
2.63 ± 4% +0.4 3.06 perf-profile.children.cycles-pp.copy_page_to_iter
1.28 ± 3% +0.5 1.82 perf-profile.children.cycles-pp.file_has_perm
1.88 ± 6% +0.6 2.44 perf-profile.children.cycles-pp.copy_page_from_iter
2.20 ± 3% +0.6 2.75 perf-profile.children.cycles-pp.mutex_lock
2.63 ± 6% +0.6 3.19 perf-profile.children.cycles-pp.selinux_file_permission
0.00 +0.6 0.58 ± 2% perf-profile.children.cycles-pp.schedule_idle
0.53 ± 12% +0.9 1.46 ± 12% perf-profile.children.cycles-pp.__mutex_lock
2.10 ± 3% +1.0 3.12 perf-profile.children.cycles-pp.mutex_unlock
4.33 ± 6% +1.3 5.61 perf-profile.children.cycles-pp.security_file_permission
4.12 ± 5% +1.3 5.46 perf-profile.children.cycles-pp.syscall_return_via_sysret
2.27 ± 3% +1.4 3.65 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +1.5 1.52 ± 18% perf-profile.children.cycles-pp.intel_idle
4.01 ± 4% +1.7 5.73 perf-profile.children.cycles-pp.entry_SYSCALL_64
3.64 ± 12% +1.8 5.42 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.00 +2.0 1.96 ± 14% perf-profile.children.cycles-pp.cpuidle_enter_state
0.00 +2.0 1.97 ± 14% perf-profile.children.cycles-pp.cpuidle_enter
0.03 ±105% +2.9 2.97 ± 10% perf-profile.children.cycles-pp.start_secondary
0.03 ±105% +3.0 3.01 ± 10% perf-profile.children.cycles-pp.do_idle
0.03 ±105% +3.0 3.02 ± 10% perf-profile.children.cycles-pp.secondary_startup_64
0.03 ±105% +3.0 3.02 ± 10% perf-profile.children.cycles-pp.cpu_startup_entry
0.00 +35.4 35.40 ± 4% perf-profile.children.cycles-pp.__GI___libc_read
0.00 +43.4 43.41 ± 4% perf-profile.children.cycles-pp.__GI___libc_write
2.93 ± 48% -2.0 0.97 ± 4% perf-profile.self.cycles-pp.available_idle_cpu
3.33 ± 8% -1.5 1.86 ± 4% perf-profile.self.cycles-pp.update_cfs_group
2.32 ± 3% -1.1 1.25 ± 3% perf-profile.self.cycles-pp.load_new_mm_cr3
2.32 ± 3% -1.0 1.35 ± 4% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.46 ± 5% -0.7 0.72 ± 4% perf-profile.self.cycles-pp.switch_fpu_return
2.27 -0.7 1.56 perf-profile.self.cycles-pp.__schedule
1.45 ± 8% -0.6 0.82 ± 4% perf-profile.self.cycles-pp.___perf_sw_event
1.27 ± 6% -0.6 0.66 ± 4% perf-profile.self.cycles-pp.__switch_to_asm
1.23 ± 4% -0.5 0.69 ± 2% perf-profile.self.cycles-pp.__switch_to
1.61 ± 3% -0.5 1.08 ± 2% perf-profile.self.cycles-pp.update_load_avg
1.78 ± 3% -0.5 1.30 ± 2% perf-profile.self.cycles-pp.update_curr
1.02 ± 4% -0.5 0.56 ± 4% perf-profile.self.cycles-pp.native_write_msr
1.24 ± 2% -0.4 0.79 perf-profile.self.cycles-pp.select_task_rq_fair
1.02 ± 12% -0.4 0.65 ± 4% perf-profile.self.cycles-pp.update_rq_clock
0.53 ± 6% -0.4 0.16 ± 7% perf-profile.self.cycles-pp.set_task_cpu
0.85 ± 22% -0.3 0.51 ± 2% perf-profile.self.cycles-pp.select_idle_sibling
0.95 ± 2% -0.3 0.62 perf-profile.self.cycles-pp._raw_spin_lock
1.29 -0.3 0.96 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
1.11 -0.3 0.78 ± 2% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.84 ± 3% -0.3 0.52 ± 3% perf-profile.self.cycles-pp.__enqueue_entity
0.92 ± 3% -0.3 0.62 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.44 ± 44% -0.3 0.15 ± 2% perf-profile.self.cycles-pp.cpumask_next_wrap
0.82 -0.3 0.53 ± 2% perf-profile.self.cycles-pp.reweight_entity
0.62 ± 2% -0.3 0.36 ± 9% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.68 -0.3 0.42 perf-profile.self.cycles-pp.dequeue_task_fair
0.67 -0.2 0.42 perf-profile.self.cycles-pp.__calc_delta
0.98 -0.2 0.75 ± 2% perf-profile.self.cycles-pp.enqueue_task_fair
1.41 -0.2 1.20 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.89 -0.2 0.69 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.33 ± 13% -0.2 0.13 ± 3% perf-profile.self.cycles-pp.migrate_task_rq_fair
0.39 ± 6% -0.2 0.21 ± 2% perf-profile.self.cycles-pp.pick_next_entity
0.34 ± 11% -0.2 0.16 ± 2% perf-profile.self.cycles-pp.cpus_share_cache
0.33 ± 4% -0.2 0.16 ± 7% perf-profile.self.cycles-pp.schedule
0.42 ± 31% -0.2 0.25 ± 2% perf-profile.self.cycles-pp.find_next_bit
0.45 ± 3% -0.2 0.29 ± 2% perf-profile.self.cycles-pp.pipe_wait
0.29 ± 5% -0.2 0.14 ± 3% perf-profile.self.cycles-pp.clear_buddies
0.67 -0.1 0.53 perf-profile.self.cycles-pp.__wake_up_common
0.38 ± 10% -0.1 0.23 ± 2% perf-profile.self.cycles-pp.cpuacct_charge
0.27 ± 9% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.67 -0.1 0.54 ± 2% perf-profile.self.cycles-pp.try_to_wake_up
0.35 -0.1 0.22 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup
0.32 ± 3% -0.1 0.19 ± 3% perf-profile.self.cycles-pp.rb_erase_cached
0.43 ± 3% -0.1 0.31 ± 3% perf-profile.self.cycles-pp.native_sched_clock
0.43 ± 2% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.update_min_vruntime
0.44 ± 5% -0.1 0.34 perf-profile.self.cycles-pp.account_entity_dequeue
0.16 ± 18% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.attach_entity_load_avg
0.15 ± 8% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.20 ± 10% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.activate_task
0.16 ± 9% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.34 -0.1 0.26 perf-profile.self.cycles-pp.account_entity_enqueue
0.34 ± 2% -0.1 0.26 ± 3% perf-profile.self.cycles-pp.dequeue_entity
0.19 ± 6% -0.1 0.11 ± 6% perf-profile.self.cycles-pp.native_load_tls
0.12 ± 7% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.rcu_note_context_switch
0.40 ± 2% -0.1 0.32 ± 2% perf-profile.self.cycles-pp.enqueue_entity
0.20 ± 17% -0.1 0.12 ± 8% perf-profile.self.cycles-pp.wake_q_add
0.64 -0.1 0.56 perf-profile.self.cycles-pp.vfs_read
0.14 ± 5% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.put_prev_entity
0.36 ± 3% -0.1 0.30 ± 2% perf-profile.self.cycles-pp.finish_task_switch
0.25 ± 2% -0.1 0.19 ± 2% perf-profile.self.cycles-pp.prepare_to_wait
0.20 ± 3% -0.1 0.14 perf-profile.self.cycles-pp.set_next_entity
0.20 ± 20% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.__mutex_unlock_slowpath
0.18 ± 3% -0.1 0.12 ± 5% perf-profile.self.cycles-pp.set_next_buddy
0.13 ± 5% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function
0.17 ± 4% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.resched_curr
0.08 ± 14% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.wake_up_q
0.15 ± 4% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.rb_insert_color_cached
0.14 ± 6% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.rb_next
0.30 ± 2% -0.0 0.27 perf-profile.self.cycles-pp.__x64_sys_write
0.11 ± 4% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.__list_add_valid
0.22 ± 5% +0.0 0.23 ± 2% perf-profile.self.cycles-pp.__wake_up_common_lock
0.05 ± 8% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.copyout
0.05 ± 8% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.kill_fasync
0.12 ± 9% +0.0 0.14 ± 3% perf-profile.self.cycles-pp.iov_iter_init
0.10 ± 4% +0.0 0.13 ± 3% perf-profile.self.cycles-pp.__fdget_pos
0.04 ± 58% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.copyin
0.10 ± 13% +0.0 0.14 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.11 ± 6% +0.0 0.16 ± 2% perf-profile.self.cycles-pp.__sb_end_write
0.00 +0.1 0.05 perf-profile.self.cycles-pp.do_idle
0.00 +0.1 0.05 perf-profile.self.cycles-pp.bpf_fd_pass
0.22 ± 6% +0.1 0.27 ± 3% perf-profile.self.cycles-pp.rw_verify_area
0.23 ± 8% +0.1 0.29 ± 2% perf-profile.self.cycles-pp.file_update_time
0.00 +0.1 0.06 perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.12 ± 8% +0.1 0.19 ± 2% perf-profile.self.cycles-pp.timespec64_trunc
0.12 ± 6% +0.1 0.19 ± 4% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.15 ± 5% +0.1 0.22 perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.46 ± 3% +0.1 0.54 ± 2% perf-profile.self.cycles-pp.new_sync_write
0.29 ± 6% +0.1 0.38 ± 2% perf-profile.self.cycles-pp.current_time
0.18 ± 9% +0.1 0.27 ± 4% perf-profile.self.cycles-pp.rcu_all_qs
0.51 ± 3% +0.1 0.59 ± 3% perf-profile.self.cycles-pp.copy_page_to_iter
0.21 ± 6% +0.1 0.30 perf-profile.self.cycles-pp.__inode_security_revalidate
0.21 ± 6% +0.1 0.29 perf-profile.self.cycles-pp.ksys_write
0.00 +0.1 0.09 ± 7% perf-profile.self.cycles-pp.menu_select
0.26 ± 7% +0.1 0.35 ± 3% perf-profile.self.cycles-pp.ksys_read
0.23 ± 5% +0.1 0.32 ± 3% perf-profile.self.cycles-pp._cond_resched
0.14 ± 5% +0.1 0.24 ± 4% perf-profile.self.cycles-pp.inode_has_perm
0.28 ± 12% +0.1 0.39 ± 4% perf-profile.self.cycles-pp.security_file_permission
0.59 ± 6% +0.1 0.72 ± 2% perf-profile.self.cycles-pp.__fsnotify_parent
1.63 ± 2% +0.1 1.77 ± 2% perf-profile.self.cycles-pp.__fget_light
0.26 ± 7% +0.2 0.41 ± 2% perf-profile.self.cycles-pp.file_has_perm
0.26 ± 8% +0.2 0.43 ± 6% perf-profile.self.cycles-pp.__mutex_lock
0.33 ± 5% +0.2 0.51 ± 2% perf-profile.self.cycles-pp.vfs_write
0.35 ± 6% +0.2 0.53 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter
1.88 ± 5% +0.2 2.07 ± 2% perf-profile.self.cycles-pp.selinux_file_permission
0.59 ± 6% +0.2 0.79 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.83 ± 4% +0.2 1.07 ± 2% perf-profile.self.cycles-pp.fsnotify
0.83 ± 2% +0.3 1.08 ± 2% perf-profile.self.cycles-pp.avc_has_perm
2.49 ± 4% +0.3 2.79 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.89 ± 5% +0.3 1.19 perf-profile.self.cycles-pp.___might_sleep
0.00 +0.3 0.35 ± 33% perf-profile.self.cycles-pp.osq_lock
0.00 +0.4 0.36 ± 4% perf-profile.self.cycles-pp.poll_idle
0.09 ± 20% +0.4 0.46 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner
1.43 ± 2% +0.4 1.81 perf-profile.self.cycles-pp.mutex_lock
0.00 +0.5 0.53 ± 6% perf-profile.self.cycles-pp.__GI___libc_write
0.94 ± 6% +0.7 1.65 ± 2% perf-profile.self.cycles-pp.pipe_write
0.00 +0.9 0.92 ± 5% perf-profile.self.cycles-pp.__GI___libc_read
2.05 ± 3% +1.0 3.06 perf-profile.self.cycles-pp.mutex_unlock
4.01 ± 4% +1.2 5.16 perf-profile.self.cycles-pp.entry_SYSCALL_64
4.12 ± 5% +1.3 5.45 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.00 +1.5 1.51 ± 18% perf-profile.self.cycles-pp.intel_idle
3.63 ± 12% +1.8 5.41 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
11.07 ± 5% +4.5 15.59 perf-profile.self.cycles-pp.do_syscall_64
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
3 years, 1 month
- 1
- 0