May 2019 - LKP - Ml01.01.Org

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 7ac2b00f8fd7070459aab5a9addee36eefc13b3f ("mm/swap: Access struct pagevec remotely") https://git.kernel.org/cgit/linux/kernel/git/bigeasy/staging.git mm_swap in testcase: ltp with following parameters: disk: 1HDD test: mm-01 test-description: The LTP testsuite contains a collection of tools for testing the Linux kernel and related features. test-url: http://linux-test-project.github.io/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): <<<test_start>>> tag=ksm03_1 stime=1557012455 cmdline="ksm03 -u 128" contacts="" analysis=exit <<<test_output>>> tst_test.c:1096: INFO: Timeout per run is 0h 05m 00s mem.c:411: INFO: wait for all children to stop. mem.c:377: INFO: child 2 stops. mem.c:377: INFO: child 1 stops. mem.c:377: INFO: child 0 stops. mem.c:484: INFO: KSM merging... mem.c:423: INFO: resume all children. mem.c:411: INFO: wait for all children to stop. mem.c:333: INFO: child 2 continues... mem.c:337: INFO: child 2 allocates 128 MB filled with 'a' mem.c:333: INFO: child 1 continues... mem.c:337: INFO: child 1 allocates 128 MB filled with 'a' mem.c:333: INFO: child 0 continues... mem.c:337: INFO: child 0 allocates 128 MB filled with 'c' mem.c:389: INFO: child 1 stops. mem.c:389: INFO: child 0 stops. mem.c:389: INFO: child 2 stops. ksm_helper.c:48: INFO: ksm daemon takes 2s to run two full scans mem.c:261: INFO: check! mem.c:254: PASS: run is 1. mem.c:254: PASS: pages_shared is 2. mem.c:254: PASS: pages_sharing is 98302. mem.c:254: PASS: pages_volatile is 0. mem.c:254: PASS: pages_unshared is 0. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:423: INFO: resume all children. mem.c:411: INFO: wait for all children to stop. mem.c:295: INFO: child 0 verifies memory content. mem.c:295: INFO: child 2 verifies memory content. mem.c:295: INFO: child 1 verifies memory content. mem.c:333: INFO: child 0 continues... mem.c:337: INFO: child 0 allocates 128 MB filled with 'c' mem.c:333: INFO: child 1 continues... mem.c:337: INFO: child 1 allocates 128 MB filled with 'b' mem.c:333: INFO: child 2 continues... mem.c:337: INFO: child 2 allocates 128 MB filled with 'a' mem.c:389: INFO: child 0 stops. mem.c:389: INFO: child 1 stops. mem.c:389: INFO: child 2 stops. ksm_helper.c:48: INFO: ksm daemon takes 1s to run two full scans mem.c:261: INFO: check! mem.c:254: PASS: run is 1. mem.c:254: PASS: pages_shared is 3. mem.c:252: FAIL: pages_sharing is not 98301 but 98293. mem.c:252: FAIL: pages_volatile is not 0 but 8. mem.c:254: PASS: pages_unshared is 0. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:423: INFO: resume all children. mem.c:411: INFO: wait for all children to stop. mem.c:295: INFO: child 0 verifies memory content. mem.c:295: INFO: child 2 verifies memory content. mem.c:295: INFO: child 1 verifies memory content. mem.c:333: INFO: child 0 continues... mem.c:333: INFO: child 2 continues... mem.c:337: INFO: child 0 allocates 128 MB filled with 'd' mem.c:337: INFO: child 2 allocates 128 MB filled with 'd' mem.c:333: INFO: child 1 continues... mem.c:337: INFO: child 1 allocates 128 MB filled with 'd' mem.c:389: INFO: child 2 stops. mem.c:389: INFO: child 0 stops. mem.c:389: INFO: child 1 stops. ksm_helper.c:48: INFO: ksm daemon takes 1s to run two full scans mem.c:261: INFO: check! mem.c:254: PASS: run is 1. mem.c:254: PASS: pages_shared is 1. mem.c:252: FAIL: pages_sharing is not 98303 but 98300. mem.c:252: FAIL: pages_volatile is not 0 but 3. mem.c:254: PASS: pages_unshared is 0. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:423: INFO: resume all children. mem.c:411: INFO: wait for all children to stop. mem.c:295: INFO: child 0 verifies memory content. mem.c:295: INFO: child 2 verifies memory content. mem.c:295: INFO: child 1 verifies memory content. mem.c:333: INFO: child 2 continues... mem.c:337: INFO: child 2 allocates 128 MB filled with 'd' mem.c:333: INFO: child 0 continues... mem.c:337: INFO: child 0 allocates 128 MB filled with 'd' mem.c:333: INFO: child 1 continues... mem.c:342: INFO: child 1 allocates 128 MB filled with 'd' except one page with 'e' mem.c:389: INFO: child 1 stops. mem.c:389: INFO: child 0 stops. mem.c:389: INFO: child 2 stops. ksm_helper.c:48: INFO: ksm daemon takes 1s to run two full scans mem.c:261: INFO: check! mem.c:254: PASS: run is 1. mem.c:254: PASS: pages_shared is 1. mem.c:254: PASS: pages_sharing is 98302. mem.c:254: PASS: pages_volatile is 0. mem.c:254: PASS: pages_unshared is 1. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:507: INFO: KSM unmerging... mem.c:423: INFO: resume all children. mem.c:261: INFO: check! mem.c:254: PASS: run is 2. mem.c:254: PASS: pages_shared is 0. mem.c:254: PASS: pages_sharing is 0. mem.c:254: PASS: pages_volatile is 0. mem.c:254: PASS: pages_unshared is 0. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:513: INFO: stop KSM. mem.c:261: INFO: check! mem.c:254: PASS: run is 0. mem.c:254: PASS: pages_shared is 0. mem.c:254: PASS: pages_sharing is 0. mem.c:254: PASS: pages_volatile is 0. mem.c:254: PASS: pages_unshared is 0. mem.c:254: PASS: sleep_millisecs is 0. mem.c:254: PASS: pages_to_scan is 98304. mem.c:295: INFO: child 0 verifies memory content. mem.c:295: INFO: child 1 verifies memory content. mem.c:295: INFO: child 1 verifies memory content. mem.c:295: INFO: child 2 verifies memory content. mem.c:404: INFO: child 0 finished. mem.c:404: INFO: child 1 finished. mem.c:404: INFO: child 2 finished. Summary: passed 38 failed 4 skipped 0 warnings 0 <<<execution_status>>> initiation_status="ok" duration=8 termination_type=exited termination_id=1 corefile=no cutime=180 cstime=172 <<<test_end>>> If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> To reproduce: # build kernel cd linux cp config-5.1.0-rc6-00003-g7ac2b00 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 5 months

1
0
0 / 0

[ext4] 345c0dbf3a: xfstests.ext4.303.fail

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 345c0dbf3a30872d9b204db96b5857cd00808cae ("ext4: protect journal inode's blocks using block_validity") https://git.kernel.org/cgit/linux/kernel/git/tytso/ext4.git dev in testcase: xfstests with following parameters: disk: 4HDD fs: ext4 test: ext4-run test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> 2019-04-26 12:48:55 export TEST_DIR=/fs/vda 2019-04-26 12:48:55 export TEST_DEV=/dev/vda 2019-04-26 12:48:55 export FSTYP=ext4 2019-04-26 12:48:55 export SCRATCH_MNT=/fs/scratch 2019-04-26 12:48:55 mkdir /fs/scratch -p 2019-04-26 12:48:55 export SCRATCH_DEV=/dev/vdd 2019-04-26 12:48:55 sed "s:^:ext4/:" /lkp/lkp/src/pack/xfstests-addon/tests/ext4-run | grep -F -f merged_ignored_files 2019-04-26 12:48:55 sed "s:^:ext4/:" /lkp/lkp/src/pack/xfstests-addon/tests/ext4-run | grep -v -F -f merged_ignored_files 2019-04-26 12:48:55 ./check ext4/001 ext4/002 ext4/003 ext4/004 ext4/005 ext4/010 ext4/011 ext4/012 ext4/013 ext4/014 ext4/015 ext4/016 ext4/017 ext4/018 ext4/019 ext4/020 ext4/021 ext4/022 ext4/023 ext4/024 ext4/025 ext4/026 ext4/027 ext4/028 ext4/032 ext4/033 ext4/034 ext4/271 ext4/301 ext4/302 ext4/303 ext4/305 ext4/306 ext4/307 ext4/308 FSTYP -- ext4 PLATFORM -- Linux/x86_64 vm-snb-2G-405 5.1.0-rc3-00007-g345c0dbf MKFS_OPTIONS -- /dev/vdd MOUNT_OPTIONS -- -o acl,user_xattr /dev/vdd /fs/scratch ext4/001 - output mismatch (see /lkp/benchmarks/xfstests/results//ext4/001.out.bad) --- tests/ext4/001.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/001.out.bad 2019-04-26 12:49:25.516490198 +0800 @@ -1,79 +1,45 @@ QA output created by 001 1. into a hole -0: [0..127]: hole -1: [128..383]: unwritten -2: [384..639]: hole +0: [0..639]: hole 1aca77e2188f52a62674fe8a873bdaba ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/001.out /lkp/benchmarks/xfstests/results//ext4/001.out.bad' to see the entire diff) ext4/002 - output mismatch (see /lkp/benchmarks/xfstests/results//ext4/002.out.bad) --- tests/ext4/002.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/002.out.bad 2019-04-26 12:49:43.719490198 +0800 @@ -1,7 +1,13 @@ QA output created by 002 -Blocks modified: [0 - 0] -Blocks modified: [0 - 0] -Blocks modified: [0 - 9] -Blocks modified: [0 - 9] -Blocks modified: [64 - 64] -Blocks modified: [64 - 64] ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/002.out /lkp/benchmarks/xfstests/results//ext4/002.out.bad' to see the entire diff) ext4/003 8s ext4/004 12s ext4/005 14s ext4/010 55s ext4/011 56s ext4/012 39s ext4/013 47s ext4/014 52s ext4/015 44s ext4/016 42s ext4/017 42s ext4/018 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//ext4/018.out.bad) --- tests/ext4/018.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/018.out.bad 2019-04-26 12:57:00.169490198 +0800 @@ -4,9 +4,8 @@ + make some files + check fs + corrupt image +./tests/ext4/018: line 59: 9287 Directory ACL: 0: syntax error in expression (error token is "Directory ACL: 0") + mount image + modify attrs -+ repair fs ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/018.out /lkp/benchmarks/xfstests/results//ext4/018.out.bad' to see the entire diff) ext4/019 21s ext4/020 19s ext4/021 40s ext4/022 24s ext4/023 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//ext4/023.out.bad) --- tests/ext4/023.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/023.out.bad 2019-04-26 13:08:07.375490198 +0800 @@ -1,3 +1,2 @@ QA output created by 023 Format and populate -Mount ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/023.out /lkp/benchmarks/xfstests/results//ext4/023.out.bad' to see the entire diff) ext4/024 27s ext4/025 21s ext4/026 [not run] mkfs.ext4 doesn't support ea_inode feature ext4/027 17s ext4/028 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//ext4/028.out.bad) --- tests/ext4/028.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/028.out.bad 2019-04-26 13:09:34.968490198 +0800 @@ -1,3 +1,2 @@ QA output created by 028 Format and mount -Compare fsmap ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/028.out /lkp/benchmarks/xfstests/results//ext4/028.out.bad' to see the entire diff) ext4/032 83s ext4/033 52s ext4/034 15s ext4/271 1s ext4/301 55s ext4/302 _check_generic_filesystem: filesystem on /dev/vdd is inconsistent (see /lkp/benchmarks/xfstests/results//ext4/302.full for details) ext4/303 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//ext4/303.out.bad) --- tests/ext4/303.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/303.out.bad 2019-04-26 13:14:26.046490198 +0800 @@ -2,3 +2,5 @@ Start defragment activity +failed: '/usr/bin/fio /tmp/8398.fio' +(see /lkp/benchmarks/xfstests/results//ext4/303.full for details) ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/303.out /lkp/benchmarks/xfstests/results//ext4/303.out.bad' to see the entire diff) ext4/305 19s ext4/306 11s ext4/307 - output mismatch (see /lkp/benchmarks/xfstests/results//ext4/307.out.bad) --- tests/ext4/307.out 2019-04-25 09:04:55.000000000 +0800 +++ /lkp/benchmarks/xfstests/results//ext4/307.out.bad 2019-04-26 13:15:02.522490198 +0800 @@ -1,6 +1,7 @@ QA output created by 307 Run fsstress +./tests/ext4/307: line 34: gawk: command not found Allocate donor file Perform compacting Check data ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/ext4/307.out /lkp/benchmarks/xfstests/results//ext4/307.out.bad' to see the entire diff) ext4/308 13s Ran: ext4/001 ext4/002 ext4/003 ext4/004 ext4/005 ext4/010 ext4/011 ext4/012 ext4/013 ext4/014 ext4/015 ext4/016 ext4/017 ext4/018 ext4/019 ext4/020 ext4/021 ext4/022 ext4/023 ext4/024 ext4/025 ext4/026 ext4/027 ext4/028 ext4/032 ext4/033 ext4/034 ext4/271 ext4/301 ext4/302 ext4/303 ext4/305 ext4/306 ext4/307 ext4/308 Not run: ext4/026 Failures: ext4/001 ext4/002 ext4/018 ext4/023 ext4/028 ext4/302 ext4/303 ext4/307 Failed 8 of 35 tests To reproduce: # build kernel cd linux cp config-5.1.0-rc3-00007-g345c0dbf .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 5 months

3
2
0 / 0

[x86/alternatives] 4fc19708b1: kernel_BUG_at_arch/x86/mm/init.c

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 4fc19708b165c1c152fa1f12f6600e66184b7786 ("x86/alternatives: Initialize temporary mm for patching") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------+------------+------------+ | | 13585fa066 | 4fc19708b1 | +------------------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 4 | | kernel_BUG_at_arch/x86/mm/init.c | 0 | 4 | | invalid_opcode:#[##] | 0 | 4 | | EIP:poking_init | 0 | 4 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 4 | +------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 5.118979] kernel BUG at arch/x86/mm/init.c:716! [ 5.119018] invalid opcode: 0000 [#1] SMP [ 5.122979] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.1.0-rc7-00022-g4fc1970 #1 [ 5.122979] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 [ 5.122979] EIP: poking_init+0x21/0xb8 [ 5.122979] Code: db 8d 65 f4 5b 5e 5f 5d c3 55 89 e5 53 52 8b 1d f8 2e 5e db 31 eb 81 f3 e3 02 89 18 e8 43 a2 1d ff a3 0c a1 56 db 85 c0 75 02 <0f> 0b d1 c3 c7 05 08 a1 56 db 00 00 00 40 b8 28 1b 48 db e8 82 a9 [ 5.122979] EAX: 00000000 EBX: 239c2c8f ECX: da865bad EDX: 00000007 [ 5.122979] ESI: 000003ca EDI: 00000000 EBP: db5bff78 ESP: db5bff70 [ 5.122979] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210246 [ 5.122979] CR0: 80050033 CR2: ffffffff CR3: 1b766000 CR4: 000006b0 [ 5.122979] Call Trace: [ 5.122979] start_kernel+0x447/0x47b [ 5.122979] i386_start_kernel+0xd6/0xee [ 5.122979] startup_32_smp+0x15f/0x170 [ 5.122979] Modules linked in: [ 5.123041] random: get_random_bytes called from print_oops_end_marker+0x4f/0x60 with crng_init=0 [ 5.127013] ---[ end trace 790e10dcf838c933 ]--- To reproduce: # build kernel cd linux cp config-5.1.0-rc7-00022-g4fc1970 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year, 5 months

2
1
0 / 0

[mm/mmap] 8c5c906fa5: Kernel_panic-not_syncing:No_working_init_found

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 8c5c906fa51596651e5d11e968797deda86e7fea ("mm/mmap: Use current_syscall_task_size() in get_unmapped_area()") https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git x86/fixes in testcase: rcutorture with following parameters: runtime: 300s test: cpuhotplug torture_type: srcu test-description: rcutorture is rcutorture kernel module load/unload test. test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------------+------------+------------+ | | c0e0591f33 | 8c5c906fa5 | +------------------------------------------------+------------+------------+ | boot_successes | 30 | 0 | | boot_failures | 0 | 28 | | Kernel_panic-not_syncing:No_working_init_found | 0 | 28 | +------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 12.143700] Run /etc/init as init process [ 12.145113] Starting init: /etc/init exists but couldn't execute it (error -13) [ 12.147601] Run /bin/init as init process [ 12.149019] Run /bin/sh as init process [ 12.150400] Starting init: /bin/sh exists but couldn't execute it (error -14) [ 12.152630] Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. [ 12.157043] CPU: 1 PID: 1 Comm: init Not tainted 5.0.0-rc7-00028-g8c5c906f #1 [ 12.159256] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 [ 12.161958] Call Trace: [ 12.164218] dump_stack+0x5c/0x7b [ 12.165439] ? rest_init+0x80/0xd0 [ 12.166687] panic+0xfe/0x2b2 [ 12.167809] ? rest_init+0xd0/0xd0 [ 12.169057] kernel_init+0xf5/0x110 [ 12.170317] ret_from_fork+0x35/0x40 [ 12.171661] Kernel Offset: 0xd000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) Elapsed time: 20 qemu-img create -f qcow2 disk-vm-snb-8G-403-0 256G To reproduce: # build kernel cd linux cp config-5.0.0-rc7-00028-g8c5c906f .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 5 months

1
0
0 / 0

[srcu] 2ce62cbf3e: kobject(#):tried_to_init_an_initialized_object, something_is_seriously_wrong

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 2ce62cbf3ebf647d3ef3640969b81e8b0b9466ad ("srcu: Allocate per-CPU data for DEFINE_SRCU() in modules") https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev.2019.04.28a in testcase: locktorture with following parameters: runtime: 300s test: cpuhotplug test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors. test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-----------------------------------------------------------------------------+------------+------------+ | | dccea1a8c4 | 2ce62cbf3e | +-----------------------------------------------------------------------------+------------+------------+ | boot_successes | 6 | 0 | | boot_failures | 4 | 10 | | BUG:kernel_reboot-without-warning_in_test_stage | 4 | | | kobject(#):tried_to_init_an_initialized_object,something_is_seriously_wrong | 0 | 10 | | general_protection_fault:#[##] | 0 | 1 | | RIP:string | 0 | 1 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 1 | | BUG:kernel_hang_in_test_stage | 0 | 1 | +-----------------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 23.381931] [TTM] Finalizing DMA pool allocator LKP: HOSTNAME vm-snb-2G-162, MAC 52:54:00:12:34:56, kernel 5.1.0-rc1-00102-g2ce62cb 1, serial console /dev/ttyS0 [ 23.407428] [TTM] Zone kernel: Used memory at exit: 0 kiB [ 23.441178] [drm] Found bochs VGA, ID 0xb0c0. [ 23.442080] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebf0000. [ 23.482582] kobject ((____ptrval____)): tried to init an initialized object, something is seriously wrong. [ 23.484087] CPU: 0 PID: 348 Comm: systemd-udevd Not tainted 5.1.0-rc1-00102-g2ce62cb #1 [ 23.485247] Call Trace: [ 23.485633] dump_stack+0x6c/0x8f [ 23.486173] kobject_init+0x45/0xa5 [ 23.486748] kobject_init_and_add+0x3d/0xa6 [ 23.487541] ? __module_address+0x8b/0xac [ 23.488122] ? is_module_address+0x10/0x1f [ 23.488729] ? static_obj+0x32/0x4a [ 23.489236] ? 0xffffffffa00ef000 [ 23.489781] ttm_mem_global_init+0xb6/0x319 [ttm] [ 23.490525] ? vprintk_func+0xd1/0xda [ 23.491139] ? drm_dbg+0x7e/0x8b [drm] [ 23.491714] ? printk+0x53/0x5b [ 23.492176] ttm_bo_device_init+0x68/0x2c3 [ttm] [ 23.492913] bochs_mm_init+0x40/0xa8 [bochs_drm] [ 23.493761] bochs_pci_probe+0x117/0x17f [bochs_drm] [ 23.494837] local_pci_probe+0x47/0x9d [ 23.495594] pci_device_probe+0x17f/0x1cd [ 23.496443] ? pci_device_remove+0xcf/0xcf [ 23.497255] really_probe+0x255/0x584 [ 23.498033] ? pm_runtime_put_noidle+0x1c/0x1f [ 23.498958] driver_probe_device+0x128/0x176 [ 23.499834] device_driver_attach+0x4e/0x6a [ 23.500737] __driver_attach+0x170/0x182 [ 23.501538] ? device_driver_attach+0x6a/0x6a [ 23.502398] bus_for_each_dev+0x9d/0xc5 [ 23.503143] driver_attach+0x22/0x25 [ 23.503848] bus_add_driver+0x157/0x240 [ 23.504622] driver_register+0xe4/0x131 [ 23.505361] __pci_register_driver+0x78/0x7f [ 23.506189] bochs_init+0x62/0x1000 [bochs_drm] [ 23.507075] ? 0xffffffffa012e000 [ 23.507921] do_one_initcall+0x7f/0x17f [ 23.508785] ? cache_alloc_debugcheck_after+0x4b/0x148 [ 23.509793] ? do_init_module+0x21/0x3b1 [ 23.510569] ? kmem_cache_alloc+0x9b/0xbb [ 23.511393] do_init_module+0x5a/0x3b1 [ 23.512133] load_module+0x11c6/0x1786 [ 23.512878] ? vfs_read+0xf3/0x102 [ 23.519413] ? allow_write_access+0x27/0x2a [ 23.520276] ? kernel_read_file+0x17f/0x191 [ 23.521214] __do_sys_finit_module+0xb5/0xc5 [ 23.522138] ? __do_sys_finit_module+0xb5/0xc5 [ 23.523121] __x64_sys_finit_module+0x15/0x17 [ 23.524236] do_syscall_64+0x1a0/0x1bd [ 23.525042] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 23.526078] RIP: 0033:0x7f1b0a25a229 [ 23.526845] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3f 4c 2b 00 f7 d8 64 89 01 48 [ 23.530673] RSP: 002b:00007ffd8ca3e9f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 23.532210] RAX: ffffffffffffffda RBX: 0000559388c5d5a0 RCX: 00007f1b0a25a229 [ 23.533655] RDX: 0000000000000000 RSI: 00007f1b0ab73265 RDI: 0000000000000014 [ 23.535344] RBP: 00007f1b0ab73265 R08: 0000000000000000 R09: 00007ffd8ca3ef70 [ 23.536799] R10: 0000000000000014 R11: 0000000000000246 R12: 0000000000000000 [ 23.538252] R13: 0000559388c5cb30 R14: 0000000000020000 R15: 000055938884ecbc [ 23.539976] [TTM] Zone kernel: Available graphics memory: 1007378 kiB [ 23.541308] [TTM] Zone kernel: Available graphics memory: 1007378 kiB [ 23.542643] [TTM] Initializing pool allocator [ 23.543570] [TTM] Initializing DMA pool allocator [ 23.544623] kobject ((____ptrval____)): tried to init an initialized object, something is seriously wrong. [ 23.546549] CPU: 0 PID: 348 Comm: systemd-udevd Not tainted 5.1.0-rc1-00102-g2ce62cb #1 [ 23.548159] Call Trace: [ 23.548694] dump_stack+0x6c/0x8f [ 23.549389] kobject_init+0x45/0xa5 [ 23.550098] kobject_init_and_add+0x3d/0xa6 [ 23.550961] ? alloc_pages_current+0xcb/0xdb [ 23.551860] ttm_bo_device_init+0x156/0x2c3 [ttm] [ 23.552850] bochs_mm_init+0x40/0xa8 [bochs_drm] [ 23.553817] bochs_pci_probe+0x117/0x17f [bochs_drm] [ 23.554945] local_pci_probe+0x47/0x9d [ 23.555560] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 23.566712] pci_device_probe+0x17f/0x1cd [ 23.566721] ? pci_device_remove+0xcf/0xcf [ 23.566732] really_probe+0x255/0x584 [ 23.576323] ? pm_runtime_put_noidle+0x1c/0x1f [ 23.577326] driver_probe_device+0x128/0x176 [ 23.578213] device_driver_attach+0x4e/0x6a [ 23.581556] __driver_attach+0x170/0x182 [ 23.582294] ? device_driver_attach+0x6a/0x6a [ 23.583115] bus_for_each_dev+0x9d/0xc5 [ 23.583852] driver_attach+0x22/0x25 [ 23.584553] bus_add_driver+0x157/0x240 [ 23.585292] driver_register+0xe4/0x131 [ 23.586166] __pci_register_driver+0x78/0x7f [ 23.586987] bochs_init+0x62/0x1000 [bochs_drm] [ 23.587839] ? 0xffffffffa012e000 [ 23.588481] do_one_initcall+0x7f/0x17f [ 23.589209] ? cache_alloc_debugcheck_after+0x4b/0x148 [ 23.590167] ? do_init_module+0x21/0x3b1 [ 23.590913] ? kmem_cache_alloc+0x9b/0xbb [ 23.591676] do_init_module+0x5a/0x3b1 [ 23.592395] load_module+0x11c6/0x1786 [ 23.593109] ? vfs_read+0xf3/0x102 [ 23.593740] ? allow_write_access+0x27/0x2a [ 23.594511] ? kernel_read_file+0x17f/0x191 [ 23.595269] __do_sys_finit_module+0xb5/0xc5 [ 23.596054] ? __do_sys_finit_module+0xb5/0xc5 [ 23.596865] __x64_sys_finit_module+0x15/0x17 [ 23.597656] do_syscall_64+0x1a0/0x1bd [ 23.598345] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 23.599297] RIP: 0033:0x7f1b0a25a229 [ 23.599976] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3f 4c 2b 00 f7 d8 64 89 01 48 [ 23.603530] RSP: 002b:00007ffd8ca3e9f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 23.604925] RAX: ffffffffffffffda RBX: 0000559388c5d5a0 RCX: 00007f1b0a25a229 [ 23.606369] RDX: 0000000000000000 RSI: 00007f1b0ab73265 RDI: 0000000000000014 [ 23.607728] RBP: 00007f1b0ab73265 R08: 0000000000000000 R09: 00007ffd8ca3ef70 [ 23.609066] R10: 0000000000000014 R11: 0000000000000246 R12: 0000000000000000 [ 23.612060] R13: 0000559388c5cb30 R14: 0000000000020000 R15: 000055938884ecbc [ 23.625106] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:02.0 on minor 0 [ 23.830835] Probing IDE interface ide1... [ 23.949174] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input6 To reproduce: # build kernel cd linux cp config-5.1.0-rc1-00102-g2ce62cb .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email Thanks, lkp

1 year, 5 months

1
0
0 / 0

[xfrm] bb9cd077e2: BUG:kernel_reboot-without-warning_in_boot_stage

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: bb9cd077e216b886438c5698e1cd75f762ecd3c9 ("xfrm: remove unneeded export_symbols") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------+------------+------------+ | | c53ac41e37 | bb9cd077e2 | +--------------------------------------------------+------------+------------+ | boot_successes | 15 | 0 | | boot_failures | 5 | 22 | | BUG:kernel_hang_in_boot_stage | 3 | | | IP-Config:Auto-configuration_of_network_failed | 1 | | | WARNING:at_net/sched/sch_generic.c:#dev_watchdog | 1 | | | RIP:dev_watchdog | 1 | | | BUG:kernel_reboot-without-warning_in_boot_stage | 0 | 22 | +--------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 0.043126] RAMDISK: [mem 0x7b8d4000-0x7ffcffff] [ 0.044017] ACPI: Early table checksum verification disabled [ 0.045128] ACPI: RSDP 0x00000000000F6840 000014 (v00 BOCHS ) [ 0.046233] ACPI: RSDT 0x000000007FFE15C9 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001) [ 0.047603] ACPI: FACP 0x000000007FFE149D 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001) BUG: kernel reboot-without-warning in boot stage Elapsed time: 10 qemu-img create -f qcow2 disk-vm-snb-2G-160-0 256G qemu-img create -f qcow2 disk-vm-snb-2G-160-1 256G To reproduce: # build kernel cd linux cp config-5.0.0-11713-gbb9cd07 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year, 5 months

1
0
0 / 0

[fsnotify] 5f02a87763: will-it-scale.per_thread_ops -5.1% regression

by kernel test robot

Greeting, FYI, we noticed a -5.1% regression of will-it-scale.per_thread_ops due to commit: commit: 5f02a877638472e83cb5e335f9eec27052b1c7c2 ("fsnotify: annotate directory entry modification events") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory with following parameters: nr_task: 16 mode: thread test: unlink1 cpufreq_governor: performance ucode: 0x20 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/thread/16/debian-x86_64-2018-04-03.cgz/lkp-ivb-d02/unlink1/will-it-scale/0x20 commit: v5.0-rc4 5f02a87763 ("fsnotify: annotate directory entry modification events") v5.0-rc4 5f02a877638472e83cb5e335f9e ---------------- --------------------------- %stddev %change %stddev \ | \ 19323 -5.1% 18346 will-it-scale.per_thread_ops 191165 ± 2% +4.1% 198910 will-it-scale.time.involuntary_context_switches 762321 ± 15% +61.3% 1229571 ± 7% will-it-scale.time.voluntary_context_switches 309177 -5.1% 293549 will-it-scale.workload 432464 ± 27% +102.6% 876020 ± 11% cpuidle.POLL.usage 283.25 ± 28% +66.6% 472.00 ± 21% slabinfo.skbuff_head_cache.active_objs 352.25 ± 21% +41.9% 500.00 ± 14% slabinfo.skbuff_head_cache.num_objs 1303292 -15.7% 1099147 vmstat.memory.cache 10069 ± 7% +31.0% 13187 ± 4% vmstat.system.cs 1261118 -16.1% 1057781 meminfo.Cached 1617123 -12.6% 1412934 meminfo.Memused 1244897 -16.4% 1041195 meminfo.Unevictable 5403 -12.5% 4730 meminfo.max_used_kB 227.62 ± 15% -22.8% 175.76 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.avg 488355 ± 5% +23.7% 603940 ± 2% sched_debug.cpu.nr_switches.avg 510161 ± 5% +23.8% 631404 ± 3% sched_debug.cpu.nr_switches.max 468309 ± 5% +23.6% 578813 ± 3% sched_debug.cpu.nr_switches.min 157186 +3.2% 162275 proc-vmstat.nr_dirty_background_threshold 314758 +3.2% 324948 proc-vmstat.nr_dirty_threshold 315280 -16.1% 264445 proc-vmstat.nr_file_pages 1607301 +3.2% 1658312 proc-vmstat.nr_free_pages 10531 -1.8% 10338 proc-vmstat.nr_slab_reclaimable 311224 -16.4% 260298 proc-vmstat.nr_unevictable 311224 -16.4% 260298 proc-vmstat.nr_zone_unevictable 8845997 -6.8% 8245187 proc-vmstat.numa_hit 8845997 -6.8% 8245187 proc-vmstat.numa_local 16702645 -6.5% 15612083 proc-vmstat.pgalloc_normal 16691594 -6.5% 15601846 proc-vmstat.pgfree 1993 ± 13% -20.9% 1576 ± 10% interrupts.24:PCI-MSI.1572864-edge.eth0 28179 ± 5% +13.3% 31936 ± 4% interrupts.CPU0.RES:Rescheduling_interrupts 4167 ± 55% -99.7% 14.00 ±145% interrupts.CPU1.NMI:Non-maskable_interrupts 4167 ± 55% -99.7% 14.00 ±145% interrupts.CPU1.PMI:Performance_monitoring_interrupts 28494 ± 5% +11.4% 31739 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts 1993 ± 13% -20.9% 1576 ± 10% interrupts.CPU2.24:PCI-MSI.1572864-edge.eth0 28158 ± 6% +13.9% 32060 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts 1947 ± 2% +10.2% 2145 ± 5% interrupts.CPU3.CAL:Function_call_interrupts 2677 ±100% +205.9% 8191 ± 32% interrupts.CPU3.NMI:Non-maskable_interrupts 2677 ±100% +205.9% 8191 ± 32% interrupts.CPU3.PMI:Performance_monitoring_interrupts 28455 ± 5% +14.3% 32526 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts 113288 ± 5% +13.2% 128262 ± 4% interrupts.RES:Rescheduling_interrupts 1.321e+09 -3.7% 1.272e+09 perf-stat.i.branch-instructions 15326992 -5.4% 14505572 perf-stat.i.branch-misses 4546846 ± 3% -6.5% 4251911 perf-stat.i.cache-misses 10143 ± 7% +31.2% 13307 ± 4% perf-stat.i.context-switches 1.62 +3.1% 1.67 perf-stat.i.cpi 308.48 +4.4% 322.20 ± 2% perf-stat.i.cpu-migrations 2321 ± 3% +5.8% 2456 perf-stat.i.cycles-between-cache-misses 1.943e+09 -3.8% 1.869e+09 perf-stat.i.dTLB-loads 1.084e+09 -3.6% 1.045e+09 perf-stat.i.dTLB-stores 933404 +7.0% 998689 perf-stat.i.iTLB-load-misses 38106 ± 4% -7.0% 35421 ± 2% perf-stat.i.iTLB-loads 6.52e+09 -3.9% 6.263e+09 perf-stat.i.instructions 7049 ± 2% -10.5% 6311 perf-stat.i.instructions-per-iTLB-miss 0.62 -3.1% 0.60 perf-stat.i.ipc 5.15 ± 2% +4.1% 5.36 perf-stat.overall.MPKI 1.61 +3.1% 1.66 perf-stat.overall.cpi 2313 ± 3% +5.8% 2448 perf-stat.overall.cycles-between-cache-misses 6986 ± 2% -10.2% 6272 perf-stat.overall.instructions-per-iTLB-miss 0.62 -3.0% 0.60 perf-stat.overall.ipc 1.318e+09 -3.8% 1.268e+09 perf-stat.ps.branch-instructions 15293474 -5.5% 14458274 perf-stat.ps.branch-misses 4536767 ± 3% -6.6% 4237911 perf-stat.ps.cache-misses 10119 ± 7% +31.1% 13264 ± 4% perf-stat.ps.context-switches 307.80 +4.3% 321.14 ± 2% perf-stat.ps.cpu-migrations 1.939e+09 -3.9% 1.863e+09 perf-stat.ps.dTLB-loads 1.082e+09 -3.7% 1.042e+09 perf-stat.ps.dTLB-stores 931359 +6.9% 995399 perf-stat.ps.iTLB-load-misses 38023 ± 4% -7.1% 35306 ± 2% perf-stat.ps.iTLB-loads 6.505e+09 -4.0% 6.242e+09 perf-stat.ps.instructions 1.973e+12 -4.0% 1.894e+12 perf-stat.total.instructions 2.94 ± 11% -0.5 2.46 ± 8% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.do_unlinkat 14.02 -0.4 13.65 perf-profile.calltrace.cycles-pp.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open.do_sys_open 13.98 -0.4 13.61 perf-profile.calltrace.cycles-pp.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat.do_filp_open 14.29 -0.3 13.95 perf-profile.calltrace.cycles-pp.down_write.path_openat.do_filp_open.do_sys_open.do_syscall_64 2.77 ± 5% -0.3 2.50 ± 4% perf-profile.calltrace.cycles-pp.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 1.50 ± 3% -0.3 1.24 ± 6% perf-profile.calltrace.cycles-pp.dentry_kill.dput.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.64 -0.2 8.39 perf-profile.calltrace.cycles-pp.rwsem_spin_on_owner.rwsem_down_write_failed.call_rwsem_down_write_failed.down_write.path_openat 0.62 ± 2% +0.1 0.71 ± 5% perf-profile.calltrace.cycles-pp.__alloc_fd.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_open 0.67 ± 7% +0.1 0.77 ± 7% perf-profile.calltrace.cycles-pp.selinux_inode_init_security.security_inode_init_security.shmem_mknod.path_openat.do_filp_open 1.21 ± 3% +0.1 1.31 ± 4% perf-profile.calltrace.cycles-pp.call_rwsem_wake.up_write.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.77 ± 10% +0.1 0.88 ± 9% perf-profile.calltrace.cycles-pp.selinux_inode_permission.security_inode_permission.link_path_walk.path_parentat.filename_parentat 0.88 ± 3% +0.2 1.05 ± 5% perf-profile.calltrace.cycles-pp.security_inode_init_security.shmem_mknod.path_openat.do_filp_open.do_sys_open 3.61 +0.2 3.81 ± 3% perf-profile.calltrace.cycles-pp.path_parentat.filename_parentat.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.86 ± 3% +0.6 5.44 ± 4% perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 1.19 ± 3% +0.9 2.09 ± 3% perf-profile.calltrace.cycles-pp.d_delete.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.54 ± 5% -0.6 6.93 ± 5% perf-profile.children.cycles-pp.osq_lock 1.51 ± 2% -0.3 1.25 ± 5% perf-profile.children.cycles-pp.dentry_kill 0.95 ± 11% -0.1 0.80 ± 7% perf-profile.children.cycles-pp.__follow_mount_rcu 0.71 ± 4% -0.1 0.60 ± 8% perf-profile.children.cycles-pp.iput 0.20 ± 7% -0.1 0.14 ± 6% perf-profile.children.cycles-pp.prandom_u32 0.16 ± 8% -0.0 0.11 ± 15% perf-profile.children.cycles-pp.shmem_create 0.17 ± 29% -0.0 0.13 ± 6% perf-profile.children.cycles-pp.irq_work_run_list 0.12 ± 9% -0.0 0.07 ± 14% perf-profile.children.cycles-pp.selinux_d_instantiate 0.12 ± 12% -0.0 0.08 ± 19% perf-profile.children.cycles-pp.mpol_shared_policy_init 0.15 ± 12% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.irq_work_interrupt 0.15 ± 12% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.smp_irq_work_interrupt 0.15 ± 13% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.irq_work_run 0.15 ± 13% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.printk 0.15 ± 13% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.vprintk_emit 0.09 ± 27% -0.0 0.06 ± 28% perf-profile.children.cycles-pp.module_put 0.08 ± 34% +0.0 0.12 ± 14% perf-profile.children.cycles-pp.process_measurement 0.01 ±173% +0.1 0.06 ± 13% perf-profile.children.cycles-pp.dequeue_entity 0.07 ± 63% +0.1 0.12 ± 7% perf-profile.children.cycles-pp.locks_remove_file 0.15 ± 15% +0.1 0.21 ± 12% perf-profile.children.cycles-pp.map_id_up 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.18 ± 14% +0.1 0.25 ± 11% perf-profile.children.cycles-pp.___d_drop 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.unlink@plt 0.20 ± 13% +0.1 0.29 ± 10% perf-profile.children.cycles-pp.__d_drop 0.16 ± 13% +0.1 0.26 ± 8% perf-profile.children.cycles-pp.memcpy_erms 0.68 ± 6% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.selinux_inode_init_security 0.63 ± 3% +0.1 0.73 ± 4% perf-profile.children.cycles-pp.__alloc_fd 0.07 ± 58% +0.1 0.17 ± 22% perf-profile.children.cycles-pp.expand_files 0.90 ± 2% +0.2 1.07 ± 6% perf-profile.children.cycles-pp.security_inode_init_security 3.64 +0.2 3.82 ± 3% perf-profile.children.cycles-pp.path_parentat 0.00 +0.3 0.28 ± 10% perf-profile.children.cycles-pp.lockref_get_not_zero 0.00 +0.3 0.28 ± 7% perf-profile.children.cycles-pp.take_dentry_name_snapshot 0.00 +0.3 0.31 ± 12% perf-profile.children.cycles-pp.dget_parent 4.89 ± 3% +0.6 5.46 ± 4% perf-profile.children.cycles-pp.vfs_unlink 1.19 ± 3% +0.9 2.12 ± 3% perf-profile.children.cycles-pp.d_delete 7.36 ± 5% -0.6 6.77 ± 5% perf-profile.self.cycles-pp.osq_lock 0.54 ± 13% -0.1 0.42 ± 11% perf-profile.self.cycles-pp.__follow_mount_rcu 0.44 ± 7% -0.1 0.36 ± 8% perf-profile.self.cycles-pp.__fput 0.12 ± 27% -0.1 0.04 ±107% perf-profile.self.cycles-pp.security_inode_free 0.36 ± 8% -0.1 0.28 ± 8% perf-profile.self.cycles-pp.may_link 0.90 ± 3% -0.1 0.82 ± 6% perf-profile.self.cycles-pp.link_path_walk 0.52 ± 7% -0.1 0.45 ± 3% perf-profile.self.cycles-pp._cond_resched 0.29 ± 10% -0.1 0.22 ± 17% perf-profile.self.cycles-pp.iput 0.17 ± 6% -0.1 0.11 ± 19% perf-profile.self.cycles-pp.shmem_unlink 0.15 ± 7% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.shmem_create 0.16 ± 15% -0.1 0.10 ± 34% perf-profile.self.cycles-pp.dentry_unlink_inode 0.15 ± 13% -0.0 0.11 ± 28% perf-profile.self.cycles-pp.__x64_sys_unlink 0.31 ± 3% -0.0 0.27 ± 10% perf-profile.self.cycles-pp.lockref_put_or_lock 0.09 ± 8% -0.0 0.06 ± 59% perf-profile.self.cycles-pp.security_file_open 0.11 ± 10% -0.0 0.07 ± 11% perf-profile.self.cycles-pp.mpol_shared_policy_init 0.09 ± 14% -0.0 0.06 ± 16% perf-profile.self.cycles-pp.selinux_d_instantiate 0.05 ± 8% +0.0 0.08 ± 8% perf-profile.self.cycles-pp.security_task_getsecid 0.17 ± 5% +0.0 0.20 ± 11% perf-profile.self.cycles-pp.shmem_free_inode 0.09 ± 23% +0.0 0.13 ± 6% perf-profile.self.cycles-pp.always_delete_dentry 0.25 ± 5% +0.0 0.29 ± 5% perf-profile.self.cycles-pp.__alloc_fd 0.15 ± 13% +0.0 0.20 ± 8% perf-profile.self.cycles-pp.simple_lookup 0.07 ± 63% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.locks_remove_file 0.04 ± 58% +0.1 0.09 ± 20% perf-profile.self.cycles-pp.shmem_truncate_range 0.11 ± 34% +0.1 0.16 ± 17% perf-profile.self.cycles-pp.d_delete 0.06 ± 61% +0.1 0.12 ± 21% perf-profile.self.cycles-pp.get_cached_acl 0.07 ± 26% +0.1 0.13 ± 17% perf-profile.self.cycles-pp.security_transition_sid 0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.unlink@plt 0.17 ± 14% +0.1 0.24 ± 10% perf-profile.self.cycles-pp.___d_drop 0.28 ± 10% +0.1 0.35 ± 12% perf-profile.self.cycles-pp.do_unlinkat 0.20 ± 10% +0.1 0.28 ± 6% perf-profile.self.cycles-pp.security_inode_init_security 0.07 ± 59% +0.1 0.14 ± 23% perf-profile.self.cycles-pp.expand_files 0.19 ± 6% +0.1 0.28 ± 17% perf-profile.self.cycles-pp.lookup_fast 0.52 ± 10% +0.1 0.63 ± 5% perf-profile.self.cycles-pp.dput 0.14 ± 13% +0.1 0.25 ± 8% perf-profile.self.cycles-pp.memcpy_erms 0.38 ± 9% +0.1 0.49 ± 10% perf-profile.self.cycles-pp.__virt_addr_valid 0.00 +0.3 0.26 ± 11% perf-profile.self.cycles-pp.lockref_get_not_zero 1.29 ± 2% +0.3 1.56 ± 3% perf-profile.self.cycles-pp.selinux_inode_permission will-it-scale.per_thread_ops 19600 +-+-----------------------------------------------------------------+ | +.. .+.. .. + | 19400 +-+ .+..+. .. .+. + +..+ + .+. .+.. | 19200 +-++ + + + : + .+. +..+..+ +.+ | | +.. : +.+. | 19000 +-+ : | | + | 18800 +-+ | | O O O O | 18600 +-+ O O O O O O O O O O | 18400 O-+ O O O | | O O O 18200 +-+ O O O O | | O O | 18000 +-+-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 5 months

1
0
0 / 0

[loop] 56a85fd837: fxmark.hdd_btrfs_MWUM_1_bufferedio.works/sec 224.0% improvement

by kernel test robot

Greeting, FYI, we noticed a 224.0% improvement of fxmark.hdd_btrfs_MWUM_1_bufferedio.works/sec due to commit: commit: 56a85fd8376ef32458efb6ea97a820754e12f6bb ("loop: properly observe rotational flag of underlying device") https://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git for-5.2/block in testcase: fxmark on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory with following parameters: disk: 1HDD media: hdd test: MWUM fstype: btrfs directio: bufferedio cpufreq_governor: performance ucode: 0x42d In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | fxmark: fxmark.hdd_btrfs_MWUM_1_directio.works/sec 236.5% improvement | | test machine | 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory | | test parameters | cpufreq_governor=performance | | | directio=directio | | | disk=1HDD | | | fstype=btrfs | | | media=hdd | | | test=MWUM | | | ucode=0x42d | +------------------+-----------------------------------------------------------------------+ | testcase: change | fxmark: fxmark.hdd_btrfs_MWUL_1_directio.works/sec -33.9% regression | | test machine | 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory | | test parameters | cpufreq_governor=performance | | | directio=directio | | | disk=1HDD | | | fstype=btrfs | | | media=hdd | | | test=MWUL | | | ucode=0x42d | +------------------+-----------------------------------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/bufferedio/1HDD/btrfs/x86_64-rhel-7.6/hdd/debian-x86_64-2018-04-03.cgz/ivb44/MWUM/fxmark/0x42d commit: 4438cf50e7 ("doc, block, bfq: add information on bfq execution time") 56a85fd837 ("loop: properly observe rotational flag of underlying device") 4438cf50e7b315ff 56a85fd8376ef32458efb6ea97a ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 0:4 2% 0:4 perf-profile.children.cycles-pp.error_entry 0:4 3% 0:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 17.58 ± 4% -47.0% 9.31 ± 52% fxmark.hdd_btrfs_MWUM_12_bufferedio.user_util 10.58 ± 3% -24.7% 7.97 ± 20% fxmark.hdd_btrfs_MWUM_16_bufferedio.iowait_util 20.54 ± 4% -8.0% 18.89 fxmark.hdd_btrfs_MWUM_1_bufferedio.iowait_sec 86.59 -17.3% 71.58 fxmark.hdd_btrfs_MWUM_1_bufferedio.iowait_util 24.03 ± 3% +10.9% 26.66 fxmark.hdd_btrfs_MWUM_1_bufferedio.real_sec 23.97 ± 3% +11.0% 26.60 fxmark.hdd_btrfs_MWUM_1_bufferedio.secs 0.10 ± 4% +39.0% 0.14 ± 9% fxmark.hdd_btrfs_MWUM_1_bufferedio.softirq_sec 0.43 ± 6% +24.6% 0.54 ± 7% fxmark.hdd_btrfs_MWUM_1_bufferedio.softirq_util 2.83 ± 3% +148.2% 7.01 fxmark.hdd_btrfs_MWUM_1_bufferedio.sys_sec 11.92 ± 2% +122.9% 26.57 fxmark.hdd_btrfs_MWUM_1_bufferedio.sys_util 0.25 ± 12% +36.6% 0.35 ± 3% fxmark.hdd_btrfs_MWUM_1_bufferedio.user_sec 1.06 ± 8% +23.2% 1.31 ± 3% fxmark.hdd_btrfs_MWUM_1_bufferedio.user_util 47985 +260.0% 172754 fxmark.hdd_btrfs_MWUM_1_bufferedio.works 2004 ± 4% +224.0% 6495 fxmark.hdd_btrfs_MWUM_1_bufferedio.works/sec 0.29 ± 9% -23.8% 0.22 ± 6% fxmark.hdd_btrfs_MWUM_2_bufferedio.softirq_util 2.17 ± 30% -41.1% 1.28 ± 5% fxmark.hdd_btrfs_MWUM_2_bufferedio.sys_sec 4.02 ± 41% -46.2% 2.16 ± 5% fxmark.hdd_btrfs_MWUM_2_bufferedio.sys_util 15122 ±104% -83.7% 2468 ± 15% fxmark.hdd_btrfs_MWUM_2_bufferedio.works 599.22 ±115% -86.3% 82.18 ± 15% fxmark.hdd_btrfs_MWUM_2_bufferedio.works/sec 2747 ±166% -97.2% 76.08 ± 25% fxmark.hdd_btrfs_MWUM_4_bufferedio.works/sec 107.14 ± 13% +451.3% 590.70 ± 2% fxmark.time.elapsed_time 107.14 ± 13% +451.3% 590.70 ± 2% fxmark.time.elapsed_time.max 1092168 ± 6% +443.2% 5932256 ± 3% fxmark.time.file_system_outputs 116421 ± 2% +314.6% 482732 fxmark.time.involuntary_context_switches 10.50 ± 10% +145.2% 25.75 ± 19% fxmark.time.percent_of_cpu_this_job_got 6.12 ± 2% +178.6% 17.05 fxmark.time.system_time 5.29 ± 21% +2488.7% 137.01 ± 20% fxmark.time.user_time 61250 ± 39% +161.7% 160299 ± 4% fxmark.time.voluntary_context_switches 29.38 ± 8% -12.2 17.16 ± 4% mpstat.cpu.all.idle% 7.55 ± 13% -2.2 5.32 ± 3% mpstat.cpu.all.sys% 6.79 ± 26% +16.8 23.60 ± 5% mpstat.cpu.all.usr% 524443 ± 7% +221.0% 1683330 ± 4% numa-numastat.node0.local_node 533969 ± 6% +216.0% 1687419 ± 4% numa-numastat.node0.numa_hit 92268 ± 47% +167.2% 246564 ± 23% numa-numastat.node1.local_node 98645 ± 44% +161.9% 258346 ± 21% numa-numastat.node1.numa_hit 27.75 ± 9% -40.5% 16.50 ± 6% vmstat.cpu.id 6.25 ± 28% +272.0% 23.25 ± 5% vmstat.cpu.us 178.50 ± 27% -78.3% 38.75 ± 4% vmstat.io.bi 36059 ± 6% -14.2% 30956 ± 2% vmstat.io.bo 1328891 +24.6% 1655470 vmstat.memory.cache 11112 ± 12% -20.5% 8834 ± 3% vmstat.system.cs 28.80 ± 9% -40.5% 17.14 ± 5% iostat.cpu.idle 7.14 ± 13% -24.5% 5.39 ± 2% iostat.cpu.system 6.46 ± 27% +263.6% 23.49 ± 5% iostat.cpu.user 1.37 ± 3% -10.7% 1.22 iostat.sda.avgqu-sz 346.97 ± 6% -26.2% 255.90 ± 3% iostat.sda.avgrq-sz 9.84 ± 2% +11.0% 10.93 iostat.sda.util 121.14 ± 2% +23.7% 149.85 iostat.sda.w/s 19961 ± 5% -11.8% 17610 iostat.sda.wkB/s 88.86 ± 4% +9.6% 97.40 iostat.sda.wrqm/s 20279326 ± 35% +650.7% 1.522e+08 ± 38% cpuidle.C1.time 299578 ± 16% +525.9% 1875124 ± 22% cpuidle.C1.usage 96298070 ± 27% +363.9% 4.467e+08 ± 32% cpuidle.C1E.time 324745 ± 29% +279.6% 1232581 ± 35% cpuidle.C1E.usage 1.005e+08 ± 37% +211.6% 3.131e+08 ± 8% cpuidle.C3.time 282616 ± 35% +139.7% 677380 ± 52% cpuidle.C3.usage 2.981e+08 ± 13% +210.5% 9.256e+08 ± 8% cpuidle.C6.time 363597 ± 7% +191.1% 1058366 ± 11% cpuidle.C6.usage 48812 ± 35% +1775.0% 915237 ±131% cpuidle.POLL.time 76720 ± 42% +418.0% 397431 ± 28% cpuidle.POLL.usage 322430 +44.9% 467345 meminfo.Active 72883 ± 5% +194.7% 214811 meminfo.Active(file) 101543 ± 10% +95.2% 198169 meminfo.AnonHugePages 1236230 +14.6% 1416469 meminfo.Cached 778.50 ± 7% +40.3% 1092 ± 5% meminfo.Dirty 55357 ± 2% +77.4% 98223 meminfo.Inactive 16246 +38.9% 22567 ± 4% meminfo.Inactive(anon) 39110 ± 3% +93.4% 75655 ± 2% meminfo.Inactive(file) 92300 +158.6% 238718 meminfo.KReclaimable 1942348 +16.1% 2254116 meminfo.Memused 227.75 ±130% +641.4% 1688 ± 24% meminfo.Mlocked 92300 +158.6% 238718 meminfo.SReclaimable 89450 +45.9% 130499 meminfo.SUnreclaim 16467 +43.6% 23644 ± 4% meminfo.Shmem 181752 +103.1% 369219 meminfo.Slab 1233 ± 7% -64.2% 441.75 ± 10% meminfo.Writeback 18996 ± 14% -77.5% 4271 ± 2% meminfo.max_used_kB 191433 ± 26% +73.9% 332877 ± 15% numa-meminfo.node0.Active 72301 ± 5% +179.2% 201850 ± 3% numa-meminfo.node0.Active(file) 755.50 ± 10% +35.3% 1022 ± 6% numa-meminfo.node0.Dirty 676145 ± 4% +29.0% 872381 numa-meminfo.node0.FilePages 47747 ± 9% +89.1% 90308 ± 5% numa-meminfo.node0.Inactive 11387 ± 49% +81.2% 20635 ± 5% numa-meminfo.node0.Inactive(anon) 36359 ± 7% +91.6% 69672 ± 6% numa-meminfo.node0.Inactive(file) 68533 +200.0% 205610 numa-meminfo.node0.KReclaimable 1072189 ± 4% +33.0% 1425930 ± 4% numa-meminfo.node0.MemUsed 131.50 ±130% +530.8% 829.50 ± 24% numa-meminfo.node0.Mlocked 68533 +200.0% 205610 numa-meminfo.node0.SReclaimable 55745 ± 3% +70.3% 94941 ± 3% numa-meminfo.node0.SUnreclaim 11600 ± 48% +86.9% 21680 ± 5% numa-meminfo.node0.Shmem 124278 ± 2% +141.8% 300552 numa-meminfo.node0.Slab 867.50 ± 51% -68.1% 276.50 ± 25% numa-meminfo.node0.Writeback 511.50 ± 47% +2449.8% 13042 ± 43% numa-meminfo.node1.Active(file) 23701 ± 10% +39.7% 33120 ± 9% numa-meminfo.node1.KReclaimable 13731 ± 24% -21.5% 10779 numa-meminfo.node1.Mapped 95.75 ±131% +797.1% 859.00 ± 33% numa-meminfo.node1.Mlocked 23701 ± 10% +39.7% 33120 ± 9% numa-meminfo.node1.SReclaimable 18083 ± 5% +179.1% 50464 ± 3% numa-vmstat.node0.nr_active_file 307479 ± 11% +519.9% 1906179 ± 14% numa-vmstat.node0.nr_dirtied 198.25 ± 7% +29.0% 255.75 ± 7% numa-vmstat.node0.nr_dirty 169038 ± 4% +29.0% 218101 numa-vmstat.node0.nr_file_pages 2846 ± 49% +81.2% 5159 ± 5% numa-vmstat.node0.nr_inactive_anon 9083 ± 7% +91.8% 17421 ± 6% numa-vmstat.node0.nr_inactive_file 32.75 ±131% +531.3% 206.75 ± 24% numa-vmstat.node0.nr_mlock 2900 ± 48% +86.9% 5420 ± 5% numa-vmstat.node0.nr_shmem 17128 +200.1% 51402 numa-vmstat.node0.nr_slab_reclaimable 13938 ± 3% +70.3% 23736 ± 3% numa-vmstat.node0.nr_slab_unreclaimable 185.75 ± 47% -65.0% 65.00 ± 27% numa-vmstat.node0.nr_writeback 306114 ± 11% +521.7% 1903090 ± 14% numa-vmstat.node0.nr_written 18083 ± 5% +179.1% 50464 ± 3% numa-vmstat.node0.nr_zone_active_file 2846 ± 49% +81.2% 5159 ± 5% numa-vmstat.node0.nr_zone_inactive_anon 9083 ± 7% +91.8% 17421 ± 6% numa-vmstat.node0.nr_zone_inactive_file 707100 ± 8% +87.1% 1322828 ± 6% numa-vmstat.node0.numa_hit 697656 ± 8% +89.0% 1318732 ± 6% numa-vmstat.node0.numa_local 128.00 ± 47% +2446.5% 3259 ± 43% numa-vmstat.node1.nr_active_file 39945 ± 69% +1064.1% 465000 ± 35% numa-vmstat.node1.nr_dirtied 3432 ± 24% -21.5% 2694 numa-vmstat.node1.nr_mapped 23.75 ±131% +803.2% 214.50 ± 33% numa-vmstat.node1.nr_mlock 5924 ± 10% +39.8% 8280 ± 9% numa-vmstat.node1.nr_slab_reclaimable 39837 ± 69% +1065.8% 464426 ± 35% numa-vmstat.node1.nr_written 128.00 ± 47% +2446.5% 3259 ± 43% numa-vmstat.node1.nr_zone_active_file 62408 +1.2% 63137 proc-vmstat.nr_active_anon 18216 ± 4% +194.8% 53704 proc-vmstat.nr_active_file 714319 ± 10% +500.0% 4285627 ± 3% proc-vmstat.nr_dirtied 205.75 ± 6% +32.9% 273.50 ± 4% proc-vmstat.nr_dirty 308956 +14.6% 354129 proc-vmstat.nr_file_pages 4062 +38.9% 5643 ± 4% proc-vmstat.nr_inactive_anon 9675 ± 4% +95.6% 18921 ± 2% proc-vmstat.nr_inactive_file 11885 -1.1% 11758 proc-vmstat.nr_kernel_stack 57.00 ±130% +639.9% 421.75 ± 24% proc-vmstat.nr_mlock 1122 -3.6% 1082 proc-vmstat.nr_page_table_pages 4119 +43.5% 5912 ± 4% proc-vmstat.nr_shmem 23069 +158.9% 59734 proc-vmstat.nr_slab_reclaimable 22375 +45.9% 32639 proc-vmstat.nr_slab_unreclaimable 252.75 ± 11% -58.3% 105.50 ± 9% proc-vmstat.nr_writeback 707337 ± 9% +502.7% 4262826 ± 3% proc-vmstat.nr_written 62408 +1.2% 63137 proc-vmstat.nr_zone_active_anon 18216 ± 4% +194.8% 53704 proc-vmstat.nr_zone_active_file 4062 +38.9% 5643 ± 4% proc-vmstat.nr_zone_inactive_anon 9675 ± 4% +95.6% 18921 ± 2% proc-vmstat.nr_zone_inactive_file 447.25 ± 8% -16.8% 372.00 ± 4% proc-vmstat.nr_zone_write_pending 662642 ± 5% +198.2% 1975999 proc-vmstat.numa_hit 646735 ± 5% +203.1% 1960116 proc-vmstat.numa_local 80763 ± 6% +217.7% 256554 proc-vmstat.pgactivate 814052 ± 5% +201.0% 2450399 proc-vmstat.pgalloc_normal 1823 ± 7% +121.4% 4038 ± 5% proc-vmstat.pgdeactivate 431214 ± 8% +250.6% 1511771 ± 2% proc-vmstat.pgfault 773651 ± 6% +207.9% 2382081 ± 2% proc-vmstat.pgfree 19211 ± 10% +21.0% 23238 ± 4% proc-vmstat.pgpgin 3859882 ± 10% +375.5% 18352553 ± 3% proc-vmstat.pgpgout 29797 +1.1% 30135 proc-vmstat.pgrotated 471257 +272.1% 1753661 proc-vmstat.slabs_scanned 78816 +138.1% 187665 slabinfo.Acpi-Parse.active_objs 1100 +134.5% 2579 slabinfo.Acpi-Parse.active_slabs 80343 +134.4% 188306 slabinfo.Acpi-Parse.num_objs 1100 +134.5% 2579 slabinfo.Acpi-Parse.num_slabs 1007 ± 6% +193.5% 2957 ± 2% slabinfo.Acpi-State.active_objs 1325 ± 5% +146.9% 3272 ± 2% slabinfo.Acpi-State.num_objs 495.50 ± 2% +22.5% 607.00 ± 5% slabinfo.avc_xperms_data.active_objs 204.75 ± 4% +161.1% 534.50 ± 6% slabinfo.blkdev_ioc.active_objs 496.50 ± 2% +63.4% 811.50 ± 5% slabinfo.blkdev_ioc.num_objs 36217 ± 3% +301.5% 145411 slabinfo.btrfs_delayed_node.active_objs 703.50 ± 3% +298.1% 2800 slabinfo.btrfs_delayed_node.active_slabs 36612 ± 3% +297.8% 145644 slabinfo.btrfs_delayed_node.num_objs 703.50 ± 3% +298.1% 2800 slabinfo.btrfs_delayed_node.num_slabs 1382 ± 5% +214.6% 4350 slabinfo.btrfs_extent_buffer.active_objs 1722 ± 5% +170.7% 4662 slabinfo.btrfs_extent_buffer.num_objs 5553 ± 3% +112.5% 11801 slabinfo.btrfs_extent_map.active_objs 108.25 ± 2% +166.5% 288.50 slabinfo.btrfs_extent_map.active_slabs 6082 ± 2% +166.3% 16199 slabinfo.btrfs_extent_map.num_objs 108.25 ± 2% +166.5% 288.50 slabinfo.btrfs_extent_map.num_slabs 35872 ± 3% +305.2% 145366 slabinfo.btrfs_inode.active_objs 1291 ± 3% +302.5% 5196 slabinfo.btrfs_inode.active_slabs 36151 ± 3% +302.5% 145512 slabinfo.btrfs_inode.num_objs 1291 ± 3% +302.5% 5196 slabinfo.btrfs_inode.num_slabs 377.50 ± 16% +135.5% 889.00 ± 14% slabinfo.btrfs_ordered_extent.active_objs 572.00 ± 19% +90.6% 1090 ± 11% slabinfo.btrfs_ordered_extent.num_objs 1355 ± 11% +144.6% 3316 ± 5% slabinfo.btrfs_path.active_objs 1537 ± 10% +127.5% 3496 ± 5% slabinfo.btrfs_path.num_objs 81315 +133.1% 189530 slabinfo.dentry.active_objs 1959 +130.7% 4520 slabinfo.dentry.active_slabs 82321 +130.6% 189873 slabinfo.dentry.num_objs 1959 +130.7% 4520 slabinfo.dentry.num_slabs 1818 +80.5% 3282 ± 4% slabinfo.kmalloc-128.active_objs 4048 +16.7% 4723 ± 3% slabinfo.kmalloc-128.num_objs 1522 ± 8% -20.2% 1215 slabinfo.kmalloc-192.active_objs 289.00 -43.5% 163.25 ± 2% slabinfo.kmalloc-8k.active_objs 334.75 -39.2% 203.50 ± 2% slabinfo.kmalloc-8k.num_objs 630.75 ± 16% +37.9% 869.50 ± 12% slabinfo.proc_inode_cache.active_objs 13320 +20.5% 16048 slabinfo.radix_tree_node.active_objs 14608 +13.3% 16544 slabinfo.radix_tree_node.num_objs 12170 ± 15% +542.4% 78185 ± 6% softirqs.BLOCK 3690 ± 5% +635.8% 27156 ± 6% softirqs.CPU0.BLOCK 27309 ± 10% +322.2% 115303 ± 5% softirqs.CPU0.RCU 14409 ± 8% +126.3% 32613 ± 11% softirqs.CPU0.SCHED 2616 ± 2% +686.6% 20580 ± 2% softirqs.CPU0.TASKLET 56869 ± 11% +426.4% 299346 ± 3% softirqs.CPU0.TIMER 6362 ± 19% +421.6% 33187 ± 6% softirqs.CPU1.BLOCK 14732 ± 17% +332.9% 63775 ± 3% softirqs.CPU1.RCU 8987 ± 10% +204.7% 27380 ± 10% softirqs.CPU1.SCHED 7344 ± 20% +485.8% 43021 ± 5% softirqs.CPU1.TASKLET 35918 ± 13% +326.5% 153174 ± 9% softirqs.CPU1.TIMER 3058 ± 22% +388.5% 14938 ± 12% softirqs.CPU16.RCU 9104 ± 11% +309.3% 37263 ± 17% softirqs.CPU16.TIMER 3235 ± 11% +425.9% 17011 ± 8% softirqs.CPU17.RCU 9182 ± 7% +300.7% 36795 ± 15% softirqs.CPU17.TIMER 3656 ± 43% +382.4% 17640 ± 7% softirqs.CPU18.RCU 9113 ± 9% +300.5% 36505 ± 17% softirqs.CPU18.TIMER 3133 ± 9% +437.0% 16828 ± 10% softirqs.CPU19.RCU 8885 ± 6% +308.0% 36249 ± 16% softirqs.CPU19.TIMER 9057 ± 18% +382.8% 43727 ± 7% softirqs.CPU2.RCU 3663 ± 24% +212.0% 11428 ± 13% softirqs.CPU2.SCHED 25233 ± 35% +294.5% 99549 ± 10% softirqs.CPU2.TIMER 2690 ± 43% +257.3% 9612 ± 17% softirqs.CPU20.RCU 7410 ± 2% +183.8% 21032 ± 19% softirqs.CPU20.TIMER 2654 ± 38% +277.0% 10008 ± 10% softirqs.CPU21.RCU 7516 ± 5% +233.3% 25051 ± 27% softirqs.CPU21.TIMER 2433 ± 40% +334.3% 10568 ± 31% softirqs.CPU22.RCU 9428 ± 39% +120.4% 20784 ± 20% softirqs.CPU22.TIMER 7385 ± 7% +210.0% 22895 ± 21% softirqs.CPU23.TIMER 10963 ± 17% +312.7% 45240 ± 6% softirqs.CPU3.RCU 4502 ± 19% +125.6% 10157 ± 10% softirqs.CPU3.SCHED 25273 ± 17% +303.7% 102035 ± 6% softirqs.CPU3.TIMER 3945 ± 7% +562.4% 26134 ± 2% softirqs.CPU4.RCU 11596 ± 5% +433.3% 61840 ± 10% softirqs.CPU4.TIMER 4012 ± 7% +552.0% 26160 ± 7% softirqs.CPU5.RCU 11690 ± 4% +424.2% 61275 ± 17% softirqs.CPU5.TIMER 4329 ± 18% +557.8% 28479 ± 4% softirqs.CPU6.RCU 11685 ± 3% +417.8% 60510 ± 14% softirqs.CPU6.TIMER 4890 ± 8% +412.6% 25065 ± 12% softirqs.CPU7.RCU 12680 ± 13% +400.3% 63441 ± 15% softirqs.CPU7.TIMER 153116 ± 10% +247.1% 531466 ± 4% softirqs.RCU 110587 ± 4% +76.4% 195064 ± 4% softirqs.SCHED 10691 ± 14% +501.7% 64332 ± 2% softirqs.TASKLET 465137 ± 5% +189.7% 1347412 ± 8% softirqs.TIMER 0.12 ± 22% -80.1% 0.02 ± 60% perf-stat.i.MPKI 32264538 ± 34% -84.8% 4908413 ± 33% perf-stat.i.branch-instructions 0.14 ± 20% -0.1 0.02 ± 6% perf-stat.i.branch-miss-rate% 2347695 ± 38% -84.9% 354908 ± 34% perf-stat.i.branch-misses 0.55 ± 20% -0.4 0.10 ± 3% perf-stat.i.cache-miss-rate% 218451 ± 26% -83.1% 36882 ± 6% perf-stat.i.cache-misses 765568 ± 23% -84.2% 120694 ± 2% perf-stat.i.cache-references 11274 ± 10% -21.7% 8829 ± 3% perf-stat.i.context-switches 0.03 ± 13% -80.8% 0.01 ± 34% perf-stat.i.cpi 1.904e+08 ± 29% -84.4% 29787431 ± 16% perf-stat.i.cpu-cycles 267.06 ± 2% -17.0% 221.78 ± 4% perf-stat.i.cpu-migrations 16.67 ± 19% -83.5% 2.74 ± 21% perf-stat.i.cycles-between-cache-misses 0.00 ± 17% -0.0 0.00 ± 40% perf-stat.i.dTLB-load-miss-rate% 58282 ± 12% -81.8% 10594 ± 12% perf-stat.i.dTLB-load-misses 30020546 ± 33% -84.6% 4617083 ± 30% perf-stat.i.dTLB-loads 0.00 ± 3% -0.0 0.00 ± 21% perf-stat.i.dTLB-store-miss-rate% 8607 ± 12% -83.2% 1448 ± 4% perf-stat.i.dTLB-store-misses 15481321 ± 25% -84.7% 2370348 ± 20% perf-stat.i.dTLB-stores 1.62 ± 16% -1.3 0.28 ± 2% perf-stat.i.iTLB-load-miss-rate% 17137 ± 13% -83.3% 2865 ± 4% perf-stat.i.iTLB-load-misses 3563 ± 6% -82.2% 634.96 ± 5% perf-stat.i.iTLB-loads 1.53e+08 ± 34% -84.8% 23283520 ± 33% perf-stat.i.instructions 164.81 ± 38% -83.5% 27.27 ± 33% perf-stat.i.instructions-per-iTLB-miss 0.01 ± 22% -82.9% 0.00 ± 23% perf-stat.i.ipc 3737 ± 5% -34.5% 2449 perf-stat.i.minor-faults 0.39 ± 10% -0.3 0.07 ± 25% perf-stat.i.node-load-miss-rate% 7270 ± 9% -80.5% 1414 ± 24% perf-stat.i.node-load-misses 43041 ± 28% -83.3% 7194 ± 20% perf-stat.i.node-loads 0.28 ± 28% -0.2 0.06 ± 40% perf-stat.i.node-store-miss-rate% 46796 ± 52% -81.0% 8897 ± 30% perf-stat.i.node-stores 3738 ± 5% -34.5% 2449 perf-stat.i.page-faults 32033779 ± 34% -84.6% 4918392 ± 33% perf-stat.ps.branch-instructions 2331066 ± 38% -84.7% 355655 ± 34% perf-stat.ps.branch-misses 217143 ± 26% -83.0% 36975 ± 6% perf-stat.ps.cache-misses 760933 ± 23% -84.1% 120998 ± 2% perf-stat.ps.cache-references 11164 ± 10% -21.0% 8816 ± 3% perf-stat.ps.context-switches 1.892e+08 ± 28% -84.2% 29855538 ± 16% perf-stat.ps.cpu-cycles 264.50 ± 2% -16.3% 221.45 ± 4% perf-stat.ps.cpu-migrations 57898 ± 12% -81.7% 10617 ± 12% perf-stat.ps.dTLB-load-misses 29806077 ± 32% -84.5% 4626290 ± 30% perf-stat.ps.dTLB-loads 8557 ± 11% -83.0% 1452 ± 4% perf-stat.ps.dTLB-store-misses 15376824 ± 25% -84.6% 2375476 ± 20% perf-stat.ps.dTLB-stores 17027 ± 13% -83.1% 2871 ± 4% perf-stat.ps.iTLB-load-misses 3538 ± 6% -82.0% 635.92 ± 5% perf-stat.ps.iTLB-loads 1.519e+08 ± 34% -84.6% 23330678 ± 33% perf-stat.ps.instructions 3702 ± 5% -34.0% 2445 perf-stat.ps.minor-faults 7222 ± 9% -80.4% 1417 ± 24% perf-stat.ps.node-load-misses 42707 ± 28% -83.1% 7201 ± 20% perf-stat.ps.node-loads 46491 ± 52% -80.8% 8914 ± 30% perf-stat.ps.node-stores 3703 ± 5% -34.0% 2445 perf-stat.ps.page-faults 90.25 ± 14% +487.3% 530.00 ± 20% interrupts.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 114.00 ± 46% +168.2% 305.75 ± 2% interrupts.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 58.00 ± 17% +764.7% 501.50 ± 39% interrupts.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 150.75 ± 80% +267.7% 554.25 ± 40% interrupts.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 79.75 ± 54% +392.2% 392.50 ± 38% interrupts.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 58.00 ± 16% +563.4% 384.75 ± 21% interrupts.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 14555 ± 60% +774.3% 127259 ± 2% interrupts.75:IR-PCI-MSI.2097152-edge.isci-msix 83416 ± 14% +46.7% 122365 ± 12% interrupts.CAL:Function_call_interrupts 33.75 ± 33% +462.2% 189.75 ± 17% interrupts.CPU0.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 83.00 ± 62% +207.2% 255.00 ± 34% interrupts.CPU0.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 16.75 ± 25% +1120.9% 204.50 ± 52% interrupts.CPU0.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 81.75 ± 81% +244.3% 281.50 ± 39% interrupts.CPU0.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 24.00 ± 55% +508.3% 146.00 ± 53% interrupts.CPU0.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 15.25 ± 5% +713.1% 124.00 ± 21% interrupts.CPU0.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 3862 ± 57% +963.2% 41061 ± 2% interrupts.CPU0.75:IR-PCI-MSI.2097152-edge.isci-msix 3046 ± 6% +185.7% 8703 ± 32% interrupts.CPU0.CAL:Function_call_interrupts 215872 ± 13% +444.1% 1174553 ± 4% interrupts.CPU0.LOC:Local_timer_interrupts 13610 ± 36% +177.6% 37781 ± 5% interrupts.CPU0.RES:Rescheduling_interrupts 133.50 ± 20% +186.7% 382.75 ± 18% interrupts.CPU0.TLB:TLB_shootdowns 39.25 ± 18% +581.5% 267.50 ± 24% interrupts.CPU1.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 70.50 ± 20% +74.5% 123.00 ± 4% interrupts.CPU1.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 47.50 ± 40% +394.2% 234.75 ± 23% interrupts.CPU1.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 18.75 ± 6% +348.0% 84.00 ± 18% interrupts.CPU1.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 10570 ± 63% +713.9% 86036 ± 5% interrupts.CPU1.75:IR-PCI-MSI.2097152-edge.isci-msix 2771 ± 12% +151.5% 6971 ± 22% interrupts.CPU1.CAL:Function_call_interrupts 158997 ± 16% +401.8% 797825 ± 6% interrupts.CPU1.LOC:Local_timer_interrupts 1.75 ±173% +3.5e+05% 6043 ± 36% interrupts.CPU1.NMI:Non-maskable_interrupts 1.75 ±173% +3.5e+05% 6043 ± 36% interrupts.CPU1.PMI:Performance_monitoring_interrupts 12388 ± 57% +157.6% 31915 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts 86.00 ± 18% +162.2% 225.50 ± 20% interrupts.CPU1.TLB:TLB_shootdowns 2.25 ± 96% +4522.2% 104.00 ±162% interrupts.CPU15.RES:Rescheduling_interrupts 2108 ± 17% +176.0% 5817 ± 45% interrupts.CPU16.CAL:Function_call_interrupts 23021 ± 7% +736.9% 192658 ± 13% interrupts.CPU16.LOC:Local_timer_interrupts 507.00 ± 78% +390.3% 2486 ± 40% interrupts.CPU16.RES:Rescheduling_interrupts 2179 ± 15% +70.4% 3714 ± 4% interrupts.CPU17.CAL:Function_call_interrupts 23398 ± 7% +721.0% 192090 ± 13% interrupts.CPU17.LOC:Local_timer_interrupts 680.00 ± 89% +136.8% 1610 ± 28% interrupts.CPU17.RES:Rescheduling_interrupts 2453 ± 15% +85.4% 4548 ± 25% interrupts.CPU18.CAL:Function_call_interrupts 24008 ± 6% +703.4% 192897 ± 13% interrupts.CPU18.LOC:Local_timer_interrupts 2090 ± 11% +120.6% 4610 ± 46% interrupts.CPU19.CAL:Function_call_interrupts 23873 ± 7% +706.9% 192644 ± 13% interrupts.CPU19.LOC:Local_timer_interrupts 577.75 ± 91% +226.9% 1888 ± 30% interrupts.CPU19.RES:Rescheduling_interrupts 19.50 ± 53% +306.4% 79.25 ± 21% interrupts.CPU2.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 22.00 ± 34% +678.4% 171.25 ± 22% interrupts.CPU2.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 2630 ± 5% +117.0% 5707 ± 22% interrupts.CPU2.CAL:Function_call_interrupts 88898 ± 29% +479.2% 514888 ± 10% interrupts.CPU2.LOC:Local_timer_interrupts 0.00 +4e+105% 4025 ± 49% interrupts.CPU2.NMI:Non-maskable_interrupts 0.00 +4e+105% 4025 ± 49% interrupts.CPU2.PMI:Performance_monitoring_interrupts 5082 ± 75% +146.1% 12508 ± 12% interrupts.CPU2.RES:Rescheduling_interrupts 37.50 ± 51% +149.3% 93.50 ± 44% interrupts.CPU2.TLB:TLB_shootdowns 1983 ± 13% +37.3% 2722 ± 13% interrupts.CPU20.CAL:Function_call_interrupts 15478 ± 5% +548.7% 100401 ± 24% interrupts.CPU20.LOC:Local_timer_interrupts 15466 ± 4% +550.7% 100638 ± 24% interrupts.CPU21.LOC:Local_timer_interrupts 30.75 ± 25% +861.8% 295.75 ± 32% interrupts.CPU21.RES:Rescheduling_interrupts 1859 ± 16% +45.2% 2700 ± 22% interrupts.CPU22.CAL:Function_call_interrupts 15198 ± 3% +561.7% 100566 ± 24% interrupts.CPU22.LOC:Local_timer_interrupts 1906 ± 15% +129.3% 4372 ± 48% interrupts.CPU23.CAL:Function_call_interrupts 15510 ± 4% +547.1% 100373 ± 24% interrupts.CPU23.LOC:Local_timer_interrupts 227.00 ± 90% +863.1% 2186 ± 78% interrupts.CPU23.RES:Rescheduling_interrupts 46.25 ± 97% +224.9% 150.25 ± 10% interrupts.CPU3.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 2824 ± 19% +117.8% 6150 ± 26% interrupts.CPU3.CAL:Function_call_interrupts 88752 ± 29% +479.3% 514160 ± 10% interrupts.CPU3.LOC:Local_timer_interrupts 4973 ± 30% +107.9% 10339 ± 16% interrupts.CPU3.RES:Rescheduling_interrupts 2271 ± 10% +115.2% 4887 ± 23% interrupts.CPU4.CAL:Function_call_interrupts 33054 ± 9% +852.6% 314879 ± 10% interrupts.CPU4.LOC:Local_timer_interrupts 1762 ± 36% +122.6% 3922 ± 36% interrupts.CPU4.RES:Rescheduling_interrupts 2366 ± 7% +164.8% 6265 ± 36% interrupts.CPU5.CAL:Function_call_interrupts 33087 ± 10% +851.6% 314849 ± 10% interrupts.CPU5.LOC:Local_timer_interrupts 1270 ± 41% +181.5% 3575 ± 40% interrupts.CPU5.RES:Rescheduling_interrupts 2543 ± 10% +115.9% 5493 ± 19% interrupts.CPU6.CAL:Function_call_interrupts 33202 ± 9% +847.9% 314737 ± 10% interrupts.CPU6.LOC:Local_timer_interrupts 1275 ± 70% +188.7% 3683 ± 31% interrupts.CPU6.RES:Rescheduling_interrupts 32.75 ±110% +425.2% 172.00 ± 45% interrupts.CPU7.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 2281 ± 13% +148.9% 5679 ± 24% interrupts.CPU7.CAL:Function_call_interrupts 33380 ± 9% +844.3% 315206 ± 10% interrupts.CPU7.LOC:Local_timer_interrupts 578.75 ± 34% +526.6% 3626 ± 40% interrupts.CPU7.RES:Rescheduling_interrupts 1047530 ± 12% +438.4% 5640297 ± 9% interrupts.LOC:Local_timer_interrupts 863.25 ± 59% +4898.7% 43151 ± 24% interrupts.NMI:Non-maskable_interrupts 863.25 ± 59% +4898.7% 43151 ± 24% interrupts.PMI:Performance_monitoring_interrupts 45861 ± 29% +162.8% 120545 ± 2% interrupts.RES:Rescheduling_interrupts 343.25 ± 14% +180.6% 963.25 ± 17% interrupts.TLB:TLB_shootdowns 959.64 ±100% -79.6% 195.46 ±173% sched_debug.cfs_rq:/.MIN_vruntime.stddev 17007 ± 62% +1860.2% 333391 ± 98% sched_debug.cfs_rq:/.load.avg 0.00 +1.1e+107% 111405 ± 24% sched_debug.cfs_rq:/.load.min 188.18 ± 17% +990.0% 2051 ± 83% sched_debug.cfs_rq:/.load_avg.avg 1296 ± 27% +175.7% 3574 ± 54% sched_debug.cfs_rq:/.load_avg.max 4.75 ± 47% +31555.4% 1503 ± 96% sched_debug.cfs_rq:/.load_avg.min 959.64 ±100% -79.6% 195.46 ±173% sched_debug.cfs_rq:/.max_vruntime.stddev 11846 ± 6% +7399.1% 888367 ± 11% sched_debug.cfs_rq:/.min_vruntime.avg 33194 ± 13% +2749.7% 945932 ± 10% sched_debug.cfs_rq:/.min_vruntime.max 6672 ± 8% +11820.6% 795381 ± 14% sched_debug.cfs_rq:/.min_vruntime.min 4324 ± 8% +1296.5% 60389 ± 27% sched_debug.cfs_rq:/.min_vruntime.stddev 0.29 ± 13% +148.6% 0.73 ± 5% sched_debug.cfs_rq:/.nr_running.avg 0.00 +5.2e+101% 0.52 ± 13% sched_debug.cfs_rq:/.nr_running.min 0.45 ± 4% -51.6% 0.22 ± 16% sched_debug.cfs_rq:/.nr_running.stddev 121.51 ± 22% -93.8% 7.58 ± 34% sched_debug.cfs_rq:/.removed.load_avg.avg 1024 -88.0% 123.37 ± 38% sched_debug.cfs_rq:/.removed.load_avg.max 326.63 ± 10% -91.3% 28.46 ± 31% sched_debug.cfs_rq:/.removed.load_avg.stddev 5605 ± 22% -93.8% 349.00 ± 34% sched_debug.cfs_rq:/.removed.runnable_sum.avg 47413 -88.0% 5684 ± 37% sched_debug.cfs_rq:/.removed.runnable_sum.max 15063 ± 10% -91.3% 1308 ± 31% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 39.99 ± 20% -92.6% 2.96 ± 44% sched_debug.cfs_rq:/.removed.util_avg.avg 431.75 ± 13% -87.9% 52.25 ± 34% sched_debug.cfs_rq:/.removed.util_avg.max 111.67 ± 9% -90.1% 11.03 ± 41% sched_debug.cfs_rq:/.removed.util_avg.stddev 9.48 ± 97% +1659.9% 166.91 ± 57% sched_debug.cfs_rq:/.runnable_load_avg.avg 0.00 +8.7e+103% 86.54 ± 25% sched_debug.cfs_rq:/.runnable_load_avg.min 15572 ± 68% +1898.0% 311140 ±103% sched_debug.cfs_rq:/.runnable_weight.avg 0.00 +9.5e+106% 94876 ± 22% sched_debug.cfs_rq:/.runnable_weight.min -270.52 +25569.1% -69440 sched_debug.cfs_rq:/.spread0.min 4326 ± 8% +1295.8% 60390 ± 27% sched_debug.cfs_rq:/.spread0.stddev 1641 ± 26% -51.0% 803.39 ± 5% sched_debug.cfs_rq:/.util_avg.max 102.00 ± 92% +202.1% 308.14 ± 23% sched_debug.cfs_rq:/.util_avg.min 289.84 ± 12% -41.1% 170.78 ± 21% sched_debug.cfs_rq:/.util_avg.stddev 443.50 ± 7% -72.5% 121.90 ± 64% sched_debug.cfs_rq:/.util_est_enqueued.max 0.00 +3.4e+103% 34.41 ±116% sched_debug.cfs_rq:/.util_est_enqueued.min 121.85 ± 13% -78.4% 26.33 ± 53% sched_debug.cfs_rq:/.util_est_enqueued.stddev 540117 ± 3% +46.0% 788667 ± 3% sched_debug.cpu.avg_idle.avg 15537 ±127% +3862.8% 615726 ± 7% sched_debug.cpu.avg_idle.min 246385 ± 5% -54.1% 112968 ± 26% sched_debug.cpu.avg_idle.stddev 32423 +882.0% 318397 ± 5% sched_debug.cpu.clock.avg 32425 +881.9% 318397 ± 5% sched_debug.cpu.clock.max 32420 +882.1% 318396 ± 5% sched_debug.cpu.clock.min 1.27 ± 5% -73.8% 0.33 ± 16% sched_debug.cpu.clock.stddev 32423 +882.0% 318397 ± 5% sched_debug.cpu.clock_task.avg 32425 +881.9% 318397 ± 5% sched_debug.cpu.clock_task.max 32420 +882.1% 318396 ± 5% sched_debug.cpu.clock_task.min 1.27 ± 5% -73.8% 0.33 ± 16% sched_debug.cpu.clock_task.stddev 10.12 ± 94% +1554.2% 167.40 ± 61% sched_debug.cpu.cpu_load[0].avg 0.00 +8.9e+103% 89.29 ± 33% sched_debug.cpu.cpu_load[0].min 8.52 ± 61% +1877.0% 168.35 ± 44% sched_debug.cpu.cpu_load[1].avg 0.00 +9.2e+103% 91.70 ± 27% sched_debug.cpu.cpu_load[1].min 21.44 ±129% +743.7% 180.86 ±113% sched_debug.cpu.cpu_load[1].stddev 8.85 ± 36% +1590.0% 149.64 ± 44% sched_debug.cpu.cpu_load[2].avg 83.25 ±117% +630.1% 607.79 ±107% sched_debug.cpu.cpu_load[2].max 0.00 +8.2e+103% 82.36 ± 40% sched_debug.cpu.cpu_load[2].min 13.58 ± 99% +1053.0% 156.60 ±114% sched_debug.cpu.cpu_load[2].stddev 9.90 ± 23% +1155.0% 124.26 ± 48% sched_debug.cpu.cpu_load[3].avg 76.50 ± 42% +541.2% 490.54 ±111% sched_debug.cpu.cpu_load[3].max 0.00 +6.4e+103% 63.96 ± 42% sched_debug.cpu.cpu_load[3].min 12.92 ± 44% +896.9% 128.83 ±116% sched_debug.cpu.cpu_load[3].stddev 11.53 ± 14% +750.7% 98.10 ± 49% sched_debug.cpu.cpu_load[4].avg 0.00 +4.7e+103% 47.34 ± 37% sched_debug.cpu.cpu_load[4].min 420.46 ± 17% +1601.5% 7153 ± 4% sched_debug.cpu.curr->pid.avg 1627 +467.6% 9236 ± 4% sched_debug.cpu.curr->pid.max 0.00 +5.7e+105% 5712 ± 8% sched_debug.cpu.curr->pid.min 668.78 ± 7% +133.1% 1558 ± 7% sched_debug.cpu.curr->pid.stddev 16656 ± 66% +1924.8% 337260 ± 98% sched_debug.cpu.load.avg 0.00 +1.2e+107% 120213 ± 27% sched_debug.cpu.load.min 7312 ± 2% +3882.6% 291227 ± 5% sched_debug.cpu.nr_load_updates.avg 14652 ± 2% +1900.1% 293066 ± 4% sched_debug.cpu.nr_load_updates.max 5756 ± 2% +4946.5% 290501 ± 5% sched_debug.cpu.nr_load_updates.min 1428 ± 2% -48.9% 730.36 ± 50% sched_debug.cpu.nr_load_updates.stddev 0.29 ± 16% +182.4% 0.82 ± 9% sched_debug.cpu.nr_running.avg 0.00 +6.4e+101% 0.64 ± 14% sched_debug.cpu.nr_running.min 0.45 ± 5% -53.5% 0.21 ± 9% sched_debug.cpu.nr_running.stddev 1375 +43827.9% 604207 ± 3% sched_debug.cpu.nr_switches.avg 13207 ± 12% +4702.9% 634333 ± 3% sched_debug.cpu.nr_switches.max 310.50 ± 7% +1.9e+05% 581402 ± 4% sched_debug.cpu.nr_switches.min 2126 ± 9% +906.4% 21402 ± 24% sched_debug.cpu.nr_switches.stddev 10.25 ± 27% +1350.4% 148.66 ± 40% sched_debug.cpu.nr_uninterruptible.max 3.94 ± 11% +2183.1% 89.93 ± 57% sched_debug.cpu.nr_uninterruptible.stddev 32420 +882.1% 318396 ± 5% sched_debug.cpu_clk 29544 +968.0% 315520 ± 5% sched_debug.ktime 32903 +869.1% 318879 ± 5% sched_debug.sched_clk 24.00 -38.6% 14.73 ± 2% sched_debug.sysctl_sched.sysctl_sched_latency 3.00 -38.6% 1.84 ± 2% sched_debug.sysctl_sched.sysctl_sched_min_granularity 4.00 -38.6% 2.45 ± 2% sched_debug.sysctl_sched.sysctl_sched_wakeup_granularity 17.69 ± 73% -17.7 0.00 perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.33 ± 71% -9.3 0.00 perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.87 ± 71% -8.9 0.00 perf-profile.calltrace.cycles-pp.btrfs_unlink.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.61 ± 69% -6.6 0.00 perf-profile.calltrace.cycles-pp.btrfs_unlink_inode.btrfs_unlink.vfs_unlink.do_unlinkat.do_syscall_64 6.61 ± 69% -6.6 0.00 perf-profile.calltrace.cycles-pp.__btrfs_unlink_inode.btrfs_unlink_inode.btrfs_unlink.vfs_unlink.do_unlinkat 6.50 ± 76% -6.5 0.00 perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.97 ± 72% -6.0 0.00 perf-profile.calltrace.cycles-pp.btrfs_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.96 ± 66% -5.0 0.00 perf-profile.calltrace.cycles-pp.btrfs_lookup_dir_item.__btrfs_unlink_inode.btrfs_unlink_inode.btrfs_unlink.vfs_unlink 4.70 ± 67% -4.7 0.00 perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_dir_item.__btrfs_unlink_inode.btrfs_unlink_inode.btrfs_unlink 0.00 +0.7 0.71 ± 18% perf-profile.calltrace.cycles-pp.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer 0.00 +0.8 0.76 ± 8% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow 0.00 +0.8 0.84 ± 24% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +1.0 1.01 ± 11% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item 0.00 +1.0 1.05 ± 10% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary 0.00 +1.1 1.05 ± 10% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +1.1 1.09 ± 20% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.00 +1.3 1.28 ± 40% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.00 +1.5 1.49 ± 50% perf-profile.calltrace.cycles-pp.btrfs_lookup_dentry.btrfs_lookup.path_openat.do_filp_open.do_sys_open 0.00 +1.5 1.50 ± 50% perf-profile.calltrace.cycles-pp.btrfs_lookup.path_openat.do_filp_open.do_sys_open.do_syscall_64 0.00 +1.5 1.53 ± 30% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.00 +1.9 1.92 ± 67% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create.path_openat 0.00 +2.0 1.96 ± 22% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.20 ±173% +2.0 2.17 ± 59% perf-profile.calltrace.cycles-pp.__btrfs_update_delayed_inode.__btrfs_run_delayed_items.flush_space.btrfs_async_reclaim_metadata_space.process_one_work 0.00 +2.1 2.12 ± 66% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create.path_openat.do_filp_open 0.00 +2.1 2.13 ± 66% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.__btrfs_run_delayed_items.flush_space.btrfs_async_reclaim_metadata_space.process_one_work 0.35 ±173% +2.3 2.67 ± 24% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.00 +2.6 2.55 ± 25% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 1.03 ±103% +2.9 3.92 ± 32% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 1.03 ±103% +3.1 4.13 ± 33% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 0.78 ± 88% +3.4 4.14 ± 27% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 1.25 ±100% +3.5 4.73 ± 30% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +3.7 3.70 ± 28% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +4.3 4.30 ± 61% perf-profile.calltrace.cycles-pp.btrfs_new_inode.btrfs_create.path_openat.do_filp_open.do_sys_open 0.00 +4.4 4.39 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link 0.00 +4.6 4.60 ± 29% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +5.3 5.28 ± 20% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +5.4 5.42 ± 19% perf-profile.calltrace.cycles-pp.apic_timer_interrupt 0.00 +6.0 5.96 ± 15% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create 0.00 +6.0 6.02 ± 14% perf-profile.calltrace.cycles-pp.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat 1.15 ± 75% +6.1 7.28 ± 22% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 0.00 +6.8 6.84 ± 17% perf-profile.calltrace.cycles-pp.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat.do_filp_open 0.00 +7.2 7.19 ± 18% perf-profile.calltrace.cycles-pp.btrfs_add_link.btrfs_create.path_openat.do_filp_open.do_sys_open 0.00 +7.4 7.36 ± 22% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.00 +15.6 15.58 ± 37% perf-profile.calltrace.cycles-pp.btrfs_create.path_openat.do_filp_open.do_sys_open.do_syscall_64 0.00 +17.5 17.47 ± 23% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.00 +19.6 19.60 ± 40% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +26.5 26.45 ± 23% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 6.59 ± 65% +27.7 34.28 ± 18% perf-profile.calltrace.cycles-pp.secondary_startup_64 0.00 +31.6 31.64 ± 19% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +31.7 31.70 ± 19% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +31.7 31.70 ± 19% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 17.69 ± 73% -17.6 0.10 ±173% perf-profile.children.cycles-pp.do_unlinkat 9.33 ± 71% -9.3 0.04 ±173% perf-profile.children.cycles-pp.vfs_unlink 8.87 ± 71% -8.8 0.04 ±173% perf-profile.children.cycles-pp.btrfs_unlink 6.61 ± 69% -6.6 0.03 ±173% perf-profile.children.cycles-pp.__btrfs_unlink_inode 6.61 ± 69% -6.6 0.03 ±173% perf-profile.children.cycles-pp.btrfs_unlink_inode 0.02 ±173% +0.1 0.10 ± 31% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.08 ± 26% perf-profile.children.cycles-pp.__alloc_file 0.00 +0.1 0.09 ± 32% perf-profile.children.cycles-pp.resched_curr 0.00 +0.1 0.10 ± 35% perf-profile.children.cycles-pp.alloc_empty_file 0.03 ±173% +0.1 0.12 ± 45% perf-profile.children.cycles-pp.reweight_entity 0.00 +0.1 0.10 ± 35% perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.00 +0.1 0.10 ± 40% perf-profile.children.cycles-pp.hrtimer_forward 0.00 +0.1 0.10 ± 36% perf-profile.children.cycles-pp.find_inode 0.04 ±100% +0.1 0.14 ± 31% perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event 0.00 +0.1 0.10 ± 43% perf-profile.children.cycles-pp.cpuidle_not_available 0.04 ±100% +0.1 0.15 ± 32% perf-profile.children.cycles-pp.cpu_load_update 0.02 ±173% +0.1 0.12 ± 29% perf-profile.children.cycles-pp.rcu_irq_enter 0.00 +0.1 0.11 ± 31% perf-profile.children.cycles-pp.note_gp_changes 0.00 +0.1 0.11 ± 51% perf-profile.children.cycles-pp.btrfs_alloc_delayed_item 0.02 ±173% +0.1 0.14 ± 33% perf-profile.children.cycles-pp.run_local_timers 0.04 ±100% +0.1 0.16 ± 30% perf-profile.children.cycles-pp.rb_insert_color 0.00 +0.1 0.12 ± 28% perf-profile.children.cycles-pp.__hrtimer_get_next_event 0.00 +0.1 0.12 ± 67% perf-profile.children.cycles-pp.hrtimer_active 0.00 +0.1 0.12 ± 53% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.05 ±100% +0.1 0.17 ± 18% perf-profile.children.cycles-pp.rcu_idle_exit 0.11 ± 62% +0.1 0.23 ± 16% perf-profile.children.cycles-pp.__switch_to_asm 0.02 ±173% +0.1 0.15 ± 18% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.02 ±173% +0.1 0.15 ± 50% perf-profile.children.cycles-pp.perf_event_task_tick 0.02 ±173% +0.1 0.15 ± 35% perf-profile.children.cycles-pp.nr_iowait_cpu 0.02 ±173% +0.1 0.16 ± 61% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.02 ±173% +0.1 0.16 ± 22% perf-profile.children.cycles-pp.cpuacct_charge 0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.get_cpu_device 0.02 ±173% +0.1 0.16 ± 42% perf-profile.children.cycles-pp.__x64_sys_close 0.00 +0.1 0.14 ± 51% perf-profile.children.cycles-pp.btrfs_find_free_objectid 0.00 +0.1 0.14 ± 39% perf-profile.children.cycles-pp.complete_walk 0.00 +0.2 0.16 ± 30% perf-profile.children.cycles-pp.update_blocked_averages 0.10 ± 80% +0.2 0.26 ± 36% perf-profile.children.cycles-pp.__pagevec_release 0.00 +0.2 0.16 ± 53% perf-profile.children.cycles-pp.rcu_eqs_enter 0.00 +0.2 0.16 ± 54% perf-profile.children.cycles-pp.btrfs_find_free_ino 0.04 ±173% +0.2 0.20 ± 42% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.2 0.17 ± 61% perf-profile.children.cycles-pp.__mutex_lock 0.00 +0.2 0.18 ± 25% perf-profile.children.cycles-pp.inode_insert5 0.02 ±173% +0.2 0.20 ± 9% perf-profile.children.cycles-pp.btrfs_update_root_times 0.02 ±173% +0.2 0.21 ± 38% perf-profile.children.cycles-pp.update_ts_time_stats 0.00 +0.2 0.19 ± 25% perf-profile.children.cycles-pp.pm_qos_request 0.02 ±173% +0.2 0.21 ± 15% perf-profile.children.cycles-pp.timerqueue_add 0.00 +0.2 0.19 ± 28% perf-profile.children.cycles-pp.insert_inode_locked4 0.02 ±173% +0.2 0.22 ± 45% perf-profile.children.cycles-pp.interrupt_entry 0.09 ±112% +0.2 0.29 ± 9% perf-profile.children.cycles-pp.__hrtimer_next_event_base 0.02 ±173% +0.2 0.24 ± 68% perf-profile.children.cycles-pp.__push_leaf_left 0.03 ±173% +0.2 0.24 ± 27% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.02 ±173% +0.2 0.25 ± 31% perf-profile.children.cycles-pp.enqueue_hrtimer 0.00 +0.2 0.23 ± 10% perf-profile.children.cycles-pp.timerqueue_del 0.00 +0.2 0.23 ± 43% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.2 0.24 ± 52% perf-profile.children.cycles-pp.d_instantiate_new 0.00 +0.3 0.25 ± 63% perf-profile.children.cycles-pp.update_sd_lb_stats 0.03 ±173% +0.3 0.29 ± 45% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.3 0.26 ± 60% perf-profile.children.cycles-pp.find_busiest_group 0.07 ±113% +0.3 0.33 ± 33% perf-profile.children.cycles-pp.__next_timer_interrupt 0.00 +0.3 0.27 ± 20% perf-profile.children.cycles-pp.__remove_hrtimer 0.06 ±173% +0.3 0.33 ± 29% perf-profile.children.cycles-pp.update_rq_clock 0.07 ±113% +0.3 0.34 ± 43% perf-profile.children.cycles-pp.__fput 0.11 ±173% +0.3 0.40 ± 66% perf-profile.children.cycles-pp.__list_del_entry_valid 0.10 ± 80% +0.3 0.39 ± 29% perf-profile.children.cycles-pp.push_leaf_left 0.09 ± 64% +0.3 0.40 ± 54% perf-profile.children.cycles-pp.run_timer_softirq 0.19 ± 79% +0.3 0.52 ± 61% perf-profile.children.cycles-pp.btrfs_leaf_free_space 0.00 +0.3 0.34 ± 56% perf-profile.children.cycles-pp.load_balance 0.11 ± 83% +0.3 0.45 ± 31% perf-profile.children.cycles-pp.update_curr 0.19 ± 86% +0.4 0.56 ± 33% perf-profile.children.cycles-pp.dequeue_entity 0.11 ± 83% +0.4 0.48 ± 65% perf-profile.children.cycles-pp.btrfs_get_or_create_delayed_node 0.03 ±173% +0.4 0.40 ± 44% perf-profile.children.cycles-pp.do_dentry_open 0.05 ±100% +0.4 0.43 ± 35% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.07 ±113% +0.4 0.45 ± 35% perf-profile.children.cycles-pp.task_work_run 0.02 ±173% +0.4 0.41 ± 84% perf-profile.children.cycles-pp.btrfs_delayed_inode_release_metadata 0.26 ± 73% +0.4 0.68 ± 23% perf-profile.children.cycles-pp.dequeue_task_fair 0.00 +0.4 0.42 ± 87% perf-profile.children.cycles-pp.queued_write_lock_slowpath 0.00 +0.4 0.42 ± 14% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.05 ±100% +0.4 0.49 ± 30% perf-profile.children.cycles-pp.tick_irq_enter 0.02 ±173% +0.4 0.46 ± 57% perf-profile.children.cycles-pp.__push_leaf_right 0.11 ± 77% +0.5 0.56 ± 53% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.20 ± 71% +0.5 0.67 ± 25% perf-profile.children.cycles-pp.native_irq_return_iret 0.38 ± 65% +0.5 0.87 ± 13% perf-profile.children.cycles-pp.push_leaf_right 0.08 ±100% +0.5 0.58 ± 29% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.00 +0.5 0.51 ± 50% perf-profile.children.cycles-pp.inode_tree_add 0.00 +0.5 0.52 ± 81% perf-profile.children.cycles-pp.alloc_inode 0.17 ± 72% +0.5 0.70 ± 46% perf-profile.children.cycles-pp.exit_to_usermode_loop 0.27 ± 71% +0.5 0.81 ± 27% perf-profile.children.cycles-pp.native_write_msr 0.00 +0.6 0.56 ± 50% perf-profile.children.cycles-pp.btrfs_insert_delayed_dir_index 0.00 +0.6 0.58 ± 82% perf-profile.children.cycles-pp.new_inode_pseudo 0.09 ± 69% +0.6 0.68 ± 37% perf-profile.children.cycles-pp.lapic_next_deadline 0.07 ± 59% +0.6 0.67 ± 21% perf-profile.children.cycles-pp.irq_enter 0.24 ± 95% +0.6 0.88 ± 61% perf-profile.children.cycles-pp.btrfs_delayed_update_inode 0.02 ±173% +0.6 0.67 ± 43% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.00 +0.7 0.66 ± 70% perf-profile.children.cycles-pp.new_inode 0.20 ± 60% +0.7 0.88 ± 40% perf-profile.children.cycles-pp.unlock_up 0.00 +0.7 0.70 ± 42% perf-profile.children.cycles-pp.prepare_to_wait_event 0.10 ±127% +0.7 0.81 ± 46% perf-profile.children.cycles-pp.read_tsc 0.11 ± 62% +0.7 0.83 ± 27% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.02 ±173% +0.8 0.78 ± 17% perf-profile.children.cycles-pp.task_tick_fair 0.29 ±135% +0.8 1.08 ± 79% perf-profile.children.cycles-pp.__btrfs_release_delayed_node 0.26 ± 89% +0.8 1.06 ± 56% perf-profile.children.cycles-pp.btrfs_update_inode 0.12 ± 97% +0.8 0.93 ± 56% perf-profile.children.cycles-pp.wait_reserve_ticket 0.13 ± 67% +0.8 0.96 ± 28% perf-profile.children.cycles-pp.tick_nohz_next_event 0.13 ± 70% +0.9 1.05 ± 33% perf-profile.children.cycles-pp.clockevents_program_event 0.16 ± 59% +1.0 1.16 ± 24% perf-profile.children.cycles-pp.ktime_get 0.21 ± 83% +1.0 1.22 ± 24% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.43 ± 97% +1.1 1.50 ± 27% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.05 ±102% +1.1 1.16 ± 11% perf-profile.children.cycles-pp.schedule_idle 0.24 ± 81% +1.3 1.55 ± 18% perf-profile.children.cycles-pp.autoremove_wake_function 0.32 ± 89% +1.3 1.64 ± 17% perf-profile.children.cycles-pp.__wake_up_common 0.09 ± 64% +1.4 1.46 ± 25% perf-profile.children.cycles-pp.split_leaf 0.15 ±100% +1.4 1.58 ± 22% perf-profile.children.cycles-pp.scheduler_tick 0.00 +1.6 1.57 ± 53% perf-profile.children.cycles-pp.btrfs_lookup_dentry 0.00 +1.6 1.58 ± 54% perf-profile.children.cycles-pp.btrfs_lookup 0.23 ±109% +1.7 1.89 ± 47% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.00 +1.7 1.73 ± 43% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.37 ± 74% +1.8 2.15 ± 49% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 2.21 ± 71% +1.9 4.13 ± 23% perf-profile.children.cycles-pp._raw_spin_lock 1.77 ± 58% +2.0 3.76 ± 29% perf-profile.children.cycles-pp.setup_items_for_insert 0.47 ± 74% +2.4 2.88 ± 21% perf-profile.children.cycles-pp.menu_select 1.29 ± 93% +2.8 4.12 ± 29% perf-profile.children.cycles-pp.update_process_times 1.31 ± 91% +3.0 4.29 ± 30% perf-profile.children.cycles-pp.tick_sched_handle 1.58 ± 84% +3.3 4.92 ± 26% perf-profile.children.cycles-pp.tick_sched_timer 0.00 +4.3 4.33 ± 61% perf-profile.children.cycles-pp.btrfs_new_inode 1.81 ± 81% +4.8 6.62 ± 23% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.00 +6.0 6.04 ± 14% perf-profile.children.cycles-pp.insert_with_overflow 0.00 +6.9 6.86 ± 18% perf-profile.children.cycles-pp.btrfs_insert_dir_item 2.08 ± 78% +7.0 9.07 ± 25% perf-profile.children.cycles-pp.hrtimer_interrupt 0.00 +7.2 7.21 ± 18% perf-profile.children.cycles-pp.btrfs_add_link 1.62 ± 61% +9.9 11.49 ± 20% perf-profile.children.cycles-pp.btrfs_insert_empty_items 2.70 ± 73% +10.9 13.55 ± 14% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 3.32 ± 70% +11.0 14.31 ± 13% perf-profile.children.cycles-pp.apic_timer_interrupt 0.00 +15.6 15.63 ± 37% perf-profile.children.cycles-pp.btrfs_create 1.85 ± 60% +16.9 18.78 ± 21% perf-profile.children.cycles-pp.intel_idle 0.31 ± 87% +19.6 19.91 ± 39% perf-profile.children.cycles-pp.path_openat 0.31 ± 87% +19.7 19.97 ± 39% perf-profile.children.cycles-pp.do_filp_open 0.31 ±101% +20.0 20.34 ± 40% perf-profile.children.cycles-pp.do_sys_open 5.93 ± 68% +22.9 28.78 ± 20% perf-profile.children.cycles-pp.cpuidle_enter_state 6.59 ± 65% +27.7 34.27 ± 18% perf-profile.children.cycles-pp.do_idle 6.59 ± 65% +27.7 34.28 ± 18% perf-profile.children.cycles-pp.secondary_startup_64 6.59 ± 65% +27.7 34.28 ± 18% perf-profile.children.cycles-pp.cpu_startup_entry 0.00 +31.7 31.70 ± 19% perf-profile.children.cycles-pp.start_secondary 0.04 ±100% +0.1 0.12 ± 21% perf-profile.self.cycles-pp.sched_clock_idle_wakeup_event 0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.rcu_irq_enter 0.00 +0.1 0.09 ± 32% perf-profile.self.cycles-pp.resched_curr 0.00 +0.1 0.09 ± 46% perf-profile.self.cycles-pp.cpuidle_not_available 0.02 ±173% +0.1 0.12 ± 24% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.00 +0.1 0.10 ± 40% perf-profile.self.cycles-pp.hrtimer_forward 0.00 +0.1 0.10 ± 36% perf-profile.self.cycles-pp.find_inode 0.04 ±100% +0.1 0.15 ± 32% perf-profile.self.cycles-pp.cpu_load_update 0.04 ±100% +0.1 0.15 ± 26% perf-profile.self.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 28% perf-profile.self.cycles-pp.enqueue_entity 0.09 ± 64% +0.1 0.20 ± 18% perf-profile.self.cycles-pp.__softirqentry_text_start 0.02 ±173% +0.1 0.14 ± 33% perf-profile.self.cycles-pp.timerqueue_add 0.00 +0.1 0.12 ± 53% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.02 ±173% +0.1 0.14 ± 16% perf-profile.self.cycles-pp.__next_timer_interrupt 0.11 ± 62% +0.1 0.23 ± 16% perf-profile.self.cycles-pp.__switch_to_asm 0.02 ±173% +0.1 0.14 ± 47% perf-profile.self.cycles-pp.irq_exit 0.00 +0.1 0.13 ± 23% perf-profile.self.cycles-pp.task_tick_fair 0.00 +0.1 0.13 ± 54% perf-profile.self.cycles-pp.clockevents_program_event 0.02 ±173% +0.1 0.15 ± 50% perf-profile.self.cycles-pp.perf_event_task_tick 0.02 ±173% +0.1 0.15 ± 27% perf-profile.self.cycles-pp.smp_apic_timer_interrupt 0.02 ±173% +0.1 0.15 ± 35% perf-profile.self.cycles-pp.nr_iowait_cpu 0.02 ±173% +0.1 0.16 ± 22% perf-profile.self.cycles-pp.cpuacct_charge 0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.get_cpu_device 0.00 +0.1 0.15 ± 41% perf-profile.self.cycles-pp.lapic_next_deadline 0.00 +0.2 0.16 ± 53% perf-profile.self.cycles-pp.rcu_eqs_enter 0.05 ±102% +0.2 0.21 ± 27% perf-profile.self.cycles-pp.tick_nohz_next_event 0.05 ±102% +0.2 0.21 ± 56% perf-profile.self.cycles-pp.run_timer_softirq 0.00 +0.2 0.17 ± 36% perf-profile.self.cycles-pp.finish_task_switch 0.03 ±173% +0.2 0.20 ± 42% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.00 +0.2 0.18 ± 34% perf-profile.self.cycles-pp.tick_sched_timer 0.09 ±124% +0.2 0.26 ± 62% perf-profile.self.cycles-pp.btrfs_tree_read_lock 0.04 ±100% +0.2 0.22 ± 32% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.2 0.19 ± 25% perf-profile.self.cycles-pp.pm_qos_request 0.02 ±173% +0.2 0.21 ± 41% perf-profile.self.cycles-pp.try_to_wake_up 0.02 ±173% +0.2 0.22 ± 45% perf-profile.self.cycles-pp.interrupt_entry 0.07 ±103% +0.2 0.27 ± 20% perf-profile.self.cycles-pp.__hrtimer_next_event_base 0.00 +0.2 0.20 ± 54% perf-profile.self.cycles-pp.__hrtimer_run_queues 0.11 ± 77% +0.2 0.31 ± 19% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.00 +0.3 0.27 ± 32% perf-profile.self.cycles-pp.update_rq_clock 0.04 ±100% +0.3 0.34 ± 39% perf-profile.self.cycles-pp.do_idle 0.28 ± 67% +0.3 0.58 ± 15% perf-profile.self.cycles-pp.__sched_text_start 0.08 ±173% +0.3 0.40 ± 66% perf-profile.self.cycles-pp.__list_del_entry_valid 0.04 ±173% +0.4 0.42 ± 61% perf-profile.self.cycles-pp.__btrfs_release_delayed_node 0.05 ±100% +0.4 0.43 ± 35% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.10 ± 80% +0.4 0.49 ± 25% perf-profile.self.cycles-pp.ktime_get 0.00 +0.4 0.39 ± 59% perf-profile.self.cycles-pp.inode_tree_add 0.20 ± 71% +0.5 0.67 ± 25% perf-profile.self.cycles-pp.native_irq_return_iret 0.02 ±173% +0.5 0.56 ± 41% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.27 ± 71% +0.5 0.81 ± 27% perf-profile.self.cycles-pp.native_write_msr 0.24 ± 63% +0.6 0.84 ± 27% perf-profile.self.cycles-pp.cpuidle_enter_state 0.10 ±127% +0.7 0.76 ± 46% perf-profile.self.cycles-pp.read_tsc 0.24 ± 70% +0.9 1.16 ± 27% perf-profile.self.cycles-pp.menu_select 0.00 +1.7 1.72 ± 44% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 1.85 ± 60% +16.9 18.76 ± 21% perf-profile.self.cycles-pp.intel_idle fxmark.hdd_btrfs_MWUM_1_bufferedio.works 180000 +-+----------------------------------------------------------------+ O O O O O O O O O O O O O O O O O O O O O O 160000 +-+ | 140000 +-+ | | | 120000 +-+ | 100000 +-+ | | | 80000 +-+ | 60000 +-+ | |..+..+.+..+..+..+ + + + +..+..+.+..+..+..+..+.+ | 40000 +-+ : :: : : : : : | 20000 +-+ : : : : : : : : | | : : : : : : :: | 0 +-+O----------O----------O-----------------------------------------+ fxmark.hdd_btrfs_MWUM_1_bufferedio.works_sec 7000 O-+----------------------------------------------O-----O-------------+ | O O O O O O O O O O O O O O O O O 6000 +-+ O O | | | 5000 +-+ | | | 4000 +-+ | | | 3000 +-+ | | | 2000 +-++..+..+..+.+..+ + + + +..+..+..+..+..+..+.+..+ | | : : : : : : : : | 1000 +-+ : : : : : : : : | | : : : : : : :: | 0 +-+O----------O-----------O------------------------------------------+ fxmark.hdd_btrfs_MWUM_1_bufferedio.sys_sec 8 +-+------------------------------O-----O--------------------------------+ O O O O O O O O O O O | 7 +-+ O O O O O O O O O 6 +-+ | | | 5 +-+ | | | 4 +-+ | | | 3 +-++..+..+..+..+..+ + + + +..+..+..+..+..+..+..+..+ | 2 +-+ : : : : : : : : | | : : : : : : : : | 1 +-+ : : : : : : : : | | : : : : : : : : | 0 +-+O-----------O-----------O--------------------------------------------+ fxmark.hdd_btrfs_MWUM_1_bufferedio.sys_util 30 +-+--------------------------------------------------------------------+ O O O O O O O O O O O O O O | 25 +-+ O O O O O O O O | | | | 20 +-+ | | | 15 +-+ | | .+..+..+..+..+ + + +..+.. .+.. .+..+.. | 10 +-++. : :: + : : +. +. +..+ | | : : : : : : : : | | : : : : : : : : | 5 +-+ : : : : : : : : | | :: :: : : :: | 0 +-+O-----------O-----------O-------------------------------------------+ fxmark.hdd_btrfs_MWUM_1_bufferedio.iowait_util 90 +-+--------------------------------------------------------------------+ | +..+..+..+..+ + + + +..+. +. +..+. + | 80 +-+ : : : : : | 70 O-+ O O O O O O O O O O O O O O O O O O O O O | : : : : : :: : | 60 +-+ : : : : : : : : | 50 +-+ : : : : : : : : | | : : : : : : : : | 40 +-+ : : : : : : : : | 30 +-+ : : : : : : : : | | : : : : : : : : | 20 +-+ : : : : : : :: | 10 +-+ : : : : | | : : : : | 0 +-+O-----------O-----------O-------------------------------------------+ fxmark.time.system_time 20 +-+---------O-----------------O----------------------------------------+ 18 O-+ O O O O O O O O O O O O O | | O O O O O O 16 +-+ | 14 +-+ | | | 12 +-+ | 10 +-+ | 8 +-+ | | +..+..+.. .+..+ + + .+..+.. | 6 +-+ +. : : : : : + +..+..+. +..+..+..+ | 4 +-+ : : : : : : : : | | : : : : : : : : | 2 +-+ : : : : : : :: | 0 +-+O-----------O-----------O-------------------------------------------+ fxmark.time.elapsed_time 800 +-+-------------------------------------------------------------------+ | O O O | 700 O-+ O O O O O O O | 600 +-+ O O O O O O O O O | | O O 500 +-+ | | | 400 +-+ | | | 300 +-+ | 200 +-+ | | | 100 +-++..+..+..+..+..+ +.. +.. +.. +..+..+..+..+. .+..+..+ | |. + .. .. .. .. +. | 0 +-+O-----------O----------O-------------------------------------------+ fxmark.time.elapsed_time.max 800 +-+-------------------------------------------------------------------+ | O O O | 700 O-+ O O O O O O O | 600 +-+ O O O O O O O O O | | O O 500 +-+ | | | 400 +-+ | | | 300 +-+ | 200 +-+ | | | 100 +-++..+..+..+..+..+ +.. +.. +.. +..+..+..+..+. .+..+..+ | |. + .. .. .. .. +. | 0 +-+O-----------O----------O-------------------------------------------+ fxmark.time.involuntary_context_switches 600000 +-+----------------------------------------------------------------+ | | 500000 O-+ O O O O O O | | O O O O O O O O O O O O O O O | | 400000 +-+ | | | 300000 +-+ | | | 200000 +-+ | | | |..+..+.+..+..+..+ + + + +..+..+.+..+..+..+..+.+ | 100000 +-+ + : + + + + : + | | + : + + + + : + | 0 +-+O----------O----------O-----------------------------------------+ fxmark.time.file_system_outputs 8e+06 +-+-----------------------------------------------------------------+ | O | 7e+06 O-+ O O O | 6e+06 +-+ O O O O O O O O O O O O | | O O O O O 5e+06 +-+ | | | 4e+06 +-+ | | | 3e+06 +-+ | 2e+06 +-+ | | | 1e+06 +-++..+..+.+..+..+.. +.. +.. +.. +..+.+..+..+..+..+..+.+ | | .. + .. .. | 0 +-+O----------O-----------O-----------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** ivb44: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory ========================================================================================= compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/directio/1HDD/btrfs/x86_64-rhel-7.6/hdd/debian-x86_64-2018-04-03.cgz/ivb44/MWUM/fxmark/0x42d commit: 4438cf50e7 ("doc, block, bfq: add information on bfq execution time") 56a85fd837 ("loop: properly observe rotational flag of underlying device") 4438cf50e7b315ff 56a85fd8376ef32458efb6ea97a ---------------- --------------------------- %stddev %change %stddev \ | \ 3371 ±165% -96.6% 113.96 ± 20% fxmark.hdd_btrfs_MWUM_12_directio.works/sec 20.68 -12.1% 18.18 ± 2% fxmark.hdd_btrfs_MWUM_1_directio.iowait_sec 87.19 -18.4% 71.15 fxmark.hdd_btrfs_MWUM_1_directio.iowait_util 0.10 ± 14% +33.3% 0.14 ± 5% fxmark.hdd_btrfs_MWUM_1_directio.softirq_sec 2.71 ± 3% +154.4% 6.90 fxmark.hdd_btrfs_MWUM_1_directio.sys_sec 11.43 ± 2% +136.3% 27.02 fxmark.hdd_btrfs_MWUM_1_directio.sys_util 0.22 ± 10% +46.8% 0.33 ± 3% fxmark.hdd_btrfs_MWUM_1_directio.user_sec 0.94 ± 9% +36.5% 1.28 ± 4% fxmark.hdd_btrfs_MWUM_1_directio.user_util 47985 +259.9% 172720 fxmark.hdd_btrfs_MWUM_1_directio.works 1999 +236.5% 6728 ± 2% fxmark.hdd_btrfs_MWUM_1_directio.works/sec 1.94 ± 10% -32.8% 1.30 fxmark.hdd_btrfs_MWUM_2_directio.sys_sec 3.28 ± 9% -32.8% 2.20 fxmark.hdd_btrfs_MWUM_2_directio.sys_util 8087 ± 25% -69.5% 2469 ± 6% fxmark.hdd_btrfs_MWUM_2_directio.works 269.28 ± 25% -69.6% 81.99 ± 6% fxmark.hdd_btrfs_MWUM_2_directio.works/sec 22.08 ± 89% +142.2% 53.47 fxmark.hdd_btrfs_MWUM_4_directio.iowait_sec 12.83 ± 88% +134.5% 30.10 fxmark.hdd_btrfs_MWUM_4_directio.real_sec 3.21 ± 85% -58.4% 1.34 ± 3% fxmark.hdd_btrfs_MWUM_4_directio.sys_util 8.24 ±166% -97.5% 0.21 fxmark.hdd_btrfs_MWUM_4_directio.user_util 3255 ±167% -97.9% 69.79 ± 2% fxmark.hdd_btrfs_MWUM_4_directio.works/sec 103.91 ± 14% +459.5% 581.40 ± 2% fxmark.time.elapsed_time 103.91 ± 14% +459.5% 581.40 ± 2% fxmark.time.elapsed_time.max 1090720 ± 5% +467.3% 6187885 fxmark.time.file_system_outputs 115485 ± 5% +314.7% 478935 ± 2% fxmark.time.involuntary_context_switches 13.50 ± 28% +48.1% 20.00 ± 10% fxmark.time.percent_of_cpu_this_job_got 6.04 +184.6% 17.20 fxmark.time.system_time 8.28 ± 54% +1141.9% 102.86 ± 15% fxmark.time.user_time 47465 ± 9% +246.3% 164378 ± 4% fxmark.time.voluntary_context_switches 27.53 ± 13% -7.7 19.85 ± 5% mpstat.cpu.all.idle% 7.44 ± 11% -2.1 5.37 mpstat.cpu.all.sys% 9.91 ± 23% +11.0 20.93 ± 4% mpstat.cpu.all.usr% 520961 ± 14% +241.0% 1776251 numa-numastat.node0.local_node 532808 ± 14% +234.4% 1781585 ± 2% numa-numastat.node0.numa_hit 97321 ± 37% +57.3% 153082 ± 17% numa-numastat.node1.numa_hit 23.75 ± 13% -20.0% 19.00 ± 4% vmstat.cpu.id 9.75 ± 22% +108.5% 20.33 ± 4% vmstat.cpu.us 171.25 ± 29% -75.5% 42.00 ± 12% vmstat.io.bi 35370 ± 5% -9.1% 32142 ± 2% vmstat.io.bo 1323519 +25.1% 1655202 vmstat.memory.cache 10548 ± 9% -12.0% 9282 ± 2% vmstat.system.cs 27.25 ± 13% -27.2% 19.84 ± 5% iostat.cpu.idle 6.96 ± 11% -21.3% 5.48 iostat.cpu.system 9.53 ± 23% +118.5% 20.83 ± 4% iostat.cpu.user 1.40 ± 6% -10.4% 1.25 iostat.sda.avgqu-sz 341.40 ± 6% -22.7% 263.81 iostat.sda.avgrq-sz 9.62 ± 2% +14.3% 11.00 iostat.sda.util 118.55 ± 3% +27.3% 150.89 iostat.sda.w/s 19735 ± 4% -8.0% 18156 iostat.sda.wkB/s 85.26 ± 2% +16.7% 99.46 iostat.sda.wrqm/s 21547184 ± 47% +444.1% 1.172e+08 ± 67% cpuidle.C1.time 315822 ± 33% +399.1% 1576235 ± 36% cpuidle.C1.usage 1.001e+08 ± 37% +512.4% 6.129e+08 ± 2% cpuidle.C1E.time 265278 ± 39% +476.8% 1530170 cpuidle.C1E.usage 38041474 ± 54% +344.6% 1.691e+08 ± 47% cpuidle.C3.time 108012 ± 48% +367.2% 504612 ± 44% cpuidle.C3.usage 4.1e+08 ± 19% +146.6% 1.011e+09 ± 8% cpuidle.C6.time 410345 ± 30% +291.0% 1604416 ± 19% cpuidle.C6.usage 44723 ± 26% +400.1% 223649 ± 3% cpuidle.POLL.time 58595 ± 18% +525.8% 366715 ± 4% cpuidle.POLL.usage 320784 +44.9% 464888 meminfo.Active 70618 ± 4% +200.6% 212264 meminfo.Active(file) 102470 ± 8% +92.9% 197646 meminfo.AnonHugePages 1232869 +14.8% 1415574 meminfo.Cached 778.75 ± 9% +41.3% 1100 ± 2% meminfo.Dirty 54344 +77.7% 96586 meminfo.Inactive 16242 +34.6% 21862 ± 7% meminfo.Inactive(anon) 38101 ± 2% +96.1% 74722 meminfo.Inactive(file) 91380 +160.8% 238298 meminfo.KReclaimable 1937696 +16.2% 2252355 meminfo.Memused 234.00 ±173% +457.7% 1305 ± 33% meminfo.Mlocked 91380 +160.8% 238298 meminfo.SReclaimable 89314 +45.1% 129592 meminfo.SUnreclaim 16473 +38.9% 22883 ± 6% meminfo.Shmem 180695 +103.6% 367892 meminfo.Slab 1150 ± 17% -56.9% 496.00 ± 6% meminfo.Writeback 19295 ± 14% -77.6% 4328 ± 2% meminfo.max_used_kB 175441 ± 29% +68.6% 295809 ± 10% numa-meminfo.node0.Active 69633 ± 5% +195.6% 205857 numa-meminfo.node0.Active(file) 819.25 ± 7% +27.7% 1046 ± 4% numa-meminfo.node0.Dirty 679771 ± 3% +26.1% 857241 ± 3% numa-meminfo.node0.FilePages 49955 ± 5% +76.5% 88160 ± 4% numa-meminfo.node0.Inactive 35584 ± 7% +106.7% 73557 numa-meminfo.node0.Inactive(file) 67037 ± 2% +207.5% 206156 ± 2% numa-meminfo.node0.KReclaimable 1059620 ± 4% +29.2% 1368715 ± 4% numa-meminfo.node0.MemUsed 135.25 ±173% +460.7% 758.33 ± 33% numa-meminfo.node0.Mlocked 67037 ± 2% +207.5% 206156 ± 2% numa-meminfo.node0.SReclaimable 55564 ± 3% +68.0% 93333 ± 2% numa-meminfo.node0.SUnreclaim 122602 ± 2% +144.3% 299490 ± 2% numa-meminfo.node0.Slab 985.00 ± 84% +588.7% 6783 ± 35% numa-meminfo.node1.Active(file) 52798 ± 63% +158.8% 136624 ± 17% numa-meminfo.node1.AnonHugePages 1871 ± 15% +288.0% 7261 ± 69% numa-meminfo.node1.Inactive(anon) 24434 +31.6% 32159 ± 9% numa-meminfo.node1.KReclaimable 24434 +31.6% 32159 ± 9% numa-meminfo.node1.SReclaimable 1885 ± 15% +286.4% 7286 ± 69% numa-meminfo.node1.Shmem 58206 ± 3% +17.6% 68427 ± 6% numa-meminfo.node1.Slab 17437 ± 5% +195.0% 51435 numa-vmstat.node0.nr_active_file 309678 ± 17% +598.5% 2163146 ± 5% numa-vmstat.node0.nr_dirtied 203.75 ± 12% +34.8% 274.67 ± 2% numa-vmstat.node0.nr_dirty 169986 ± 3% +26.1% 214269 ± 3% numa-vmstat.node0.nr_file_pages 8910 ± 7% +106.2% 18377 numa-vmstat.node0.nr_inactive_file 33.25 ±173% +469.4% 189.33 ± 33% numa-vmstat.node0.nr_mlock 16776 ± 2% +207.1% 51520 ± 2% numa-vmstat.node0.nr_slab_reclaimable 13892 ± 3% +67.9% 23328 ± 2% numa-vmstat.node0.nr_slab_unreclaimable 308469 ± 17% +600.1% 2159645 ± 5% numa-vmstat.node0.nr_written 17437 ± 5% +195.0% 51435 numa-vmstat.node0.nr_zone_active_file 8910 ± 7% +106.2% 18377 numa-vmstat.node0.nr_zone_inactive_file 736094 ± 8% +83.4% 1350345 ± 7% numa-vmstat.node0.numa_hit 724374 ± 8% +85.7% 1345015 ± 7% numa-vmstat.node0.numa_local 245.25 ± 85% +591.4% 1695 ± 35% numa-vmstat.node1.nr_active_file 31300 ± 69% +728.5% 259314 ± 51% numa-vmstat.node1.nr_dirtied 467.75 ± 15% +288.0% 1815 ± 69% numa-vmstat.node1.nr_inactive_anon 24.25 ±173% +460.8% 136.00 ± 33% numa-vmstat.node1.nr_mlock 471.00 ± 15% +286.6% 1821 ± 69% numa-vmstat.node1.nr_shmem 6109 +31.6% 8039 ± 9% numa-vmstat.node1.nr_slab_reclaimable 31188 ± 69% +730.6% 259044 ± 51% numa-vmstat.node1.nr_written 245.25 ± 85% +591.4% 1695 ± 35% numa-vmstat.node1.nr_zone_active_file 467.75 ± 15% +288.0% 1815 ± 69% numa-vmstat.node1.nr_zone_inactive_anon 17678 ± 4% +200.4% 53103 proc-vmstat.nr_active_file 695682 ± 9% +529.4% 4378932 proc-vmstat.nr_dirtied 209.00 ± 8% +30.9% 273.67 ± 5% proc-vmstat.nr_dirty 308255 +14.8% 353955 proc-vmstat.nr_file_pages 4060 +34.6% 5466 ± 7% proc-vmstat.nr_inactive_anon 9528 ± 2% +96.2% 18697 proc-vmstat.nr_inactive_file 11934 -1.5% 11752 proc-vmstat.nr_kernel_stack 7469 -1.1% 7386 proc-vmstat.nr_mapped 58.75 ±173% +454.3% 325.67 ± 33% proc-vmstat.nr_mlock 1110 -2.5% 1082 proc-vmstat.nr_page_table_pages 4116 +39.0% 5721 ± 6% proc-vmstat.nr_shmem 22890 +160.3% 59580 proc-vmstat.nr_slab_reclaimable 22321 +45.1% 32396 proc-vmstat.nr_slab_unreclaimable 309.50 ± 10% -64.1% 111.00 ± 8% proc-vmstat.nr_writeback 689573 ± 9% +531.7% 4356002 proc-vmstat.nr_written 17678 ± 4% +200.4% 53103 proc-vmstat.nr_zone_active_file 4060 +34.6% 5466 ± 7% proc-vmstat.nr_zone_inactive_anon 9528 ± 2% +96.2% 18697 proc-vmstat.nr_zone_inactive_file 510.50 ± 6% -25.6% 380.00 ± 3% proc-vmstat.nr_zone_write_pending 1222 ± 53% -97.5% 30.67 ±105% proc-vmstat.numa_hint_faults_local 658998 ± 6% +197.8% 1962342 proc-vmstat.numa_hit 643072 ± 6% +202.7% 1946433 proc-vmstat.numa_local 12977 ± 40% -54.8% 5864 ± 69% proc-vmstat.numa_pages_migrated 81778 ± 8% +210.7% 254110 proc-vmstat.pgactivate 818853 ± 6% +197.9% 2439647 proc-vmstat.pgalloc_normal 1848 ± 12% +115.4% 3983 ± 3% proc-vmstat.pgdeactivate 429974 ± 9% +246.4% 1489231 ± 2% proc-vmstat.pgfault 798429 ± 7% +204.6% 2431959 proc-vmstat.pgfree 12977 ± 40% -54.8% 5864 ± 69% proc-vmstat.pgmigrate_success 19082 ± 12% +31.3% 25060 ± 8% proc-vmstat.pgpgin 3756945 ± 10% +397.8% 18702952 proc-vmstat.pgpgout 471743 +270.9% 1749602 proc-vmstat.slabs_scanned 12230 ± 12% +513.7% 75057 ± 8% softirqs.BLOCK 3697 ± 3% +588.2% 25444 ± 2% softirqs.CPU0.BLOCK 27049 ± 6% +322.0% 114156 ± 2% softirqs.CPU0.RCU 13868 ± 8% +156.3% 35549 ± 3% softirqs.CPU0.SCHED 2637 +661.4% 20081 softirqs.CPU0.TASKLET 57544 ± 13% +406.7% 291604 ± 2% softirqs.CPU0.TIMER 6178 ± 22% +491.3% 36529 ± 4% softirqs.CPU1.BLOCK 13555 ± 16% +363.5% 62832 ± 2% softirqs.CPU1.RCU 8817 ± 9% +208.4% 27190 ± 11% softirqs.CPU1.SCHED 6914 ± 22% +518.8% 42781 ± 3% softirqs.CPU1.TASKLET 31444 ± 27% +373.4% 148863 ± 14% softirqs.CPU1.TIMER 5257 ± 16% +221.2% 16886 ± 8% softirqs.CPU16.RCU 8750 ± 14% +290.3% 34157 ± 15% softirqs.CPU16.TIMER 3820 ± 24% +326.3% 16287 ± 5% softirqs.CPU17.RCU 9592 ± 29% +253.1% 33867 ± 16% softirqs.CPU17.TIMER 4403 ± 30% +260.0% 15851 ± 5% softirqs.CPU18.RCU 9074 ± 13% +275.4% 34066 ± 16% softirqs.CPU18.TIMER 4065 ± 11% +286.3% 15702 ± 12% softirqs.CPU19.RCU 8643 ± 11% +288.6% 33591 ± 17% softirqs.CPU19.TIMER 8226 ± 21% +445.4% 44863 ± 6% softirqs.CPU2.RCU 3151 ± 25% +237.0% 10622 ± 7% softirqs.CPU2.SCHED 17684 ± 41% +455.2% 98184 ± 18% softirqs.CPU2.TIMER 7360 ± 8% +156.3% 18864 ± 7% softirqs.CPU20.TIMER 2335 ± 18% +283.0% 8943 ± 20% softirqs.CPU21.RCU 7855 ± 15% +178.4% 21867 ± 18% softirqs.CPU21.TIMER 2587 ± 24% +276.7% 9747 ± 20% softirqs.CPU22.RCU 7339 ± 14% +236.0% 24664 ± 35% softirqs.CPU22.TIMER 2880 ± 8% +245.6% 9954 ± 32% softirqs.CPU23.RCU 7477 ± 11% +161.4% 19549 ± 6% softirqs.CPU23.TIMER 8954 ± 20% +370.9% 42162 ± 5% softirqs.CPU3.RCU 3510 ± 24% +250.9% 12316 ± 7% softirqs.CPU3.SCHED 18856 ± 40% +432.9% 100493 ± 12% softirqs.CPU3.TIMER 5110 ± 6% +416.5% 26393 ± 7% softirqs.CPU4.RCU 12345 ± 20% +391.1% 60631 ± 14% softirqs.CPU4.TIMER 5957 ± 23% +325.4% 25343 ± 5% softirqs.CPU5.RCU 11487 ± 13% +443.1% 62393 ± 8% softirqs.CPU5.TIMER 4832 ± 15% +455.2% 26831 ± 12% softirqs.CPU6.RCU 11332 ± 16% +454.5% 62839 ± 18% softirqs.CPU6.TIMER 5257 ± 17% +407.4% 26674 ± 3% softirqs.CPU7.RCU 12022 ± 17% +438.2% 64705 ± 17% softirqs.CPU7.TIMER 165149 +217.1% 523772 ± 3% softirqs.RCU 101949 ± 14% +98.9% 202828 softirqs.SCHED 10281 ± 14% +518.5% 63586 ± 2% softirqs.TASKLET 429927 ± 13% +208.1% 1324678 ± 9% softirqs.TIMER 78066 +140.2% 187478 slabinfo.Acpi-Parse.active_objs 1093 +135.6% 2576 slabinfo.Acpi-Parse.active_slabs 79862 +135.5% 188106 slabinfo.Acpi-Parse.num_objs 1093 +135.6% 2576 slabinfo.Acpi-Parse.num_slabs 1126 ± 7% +160.5% 2934 slabinfo.Acpi-State.active_objs 1440 ± 5% +128.4% 3290 slabinfo.Acpi-State.num_objs 230.25 ± 10% +118.0% 502.00 ± 6% slabinfo.blkdev_ioc.active_objs 535.50 ± 6% +48.3% 794.33 ± 2% slabinfo.blkdev_ioc.num_objs 35428 ± 2% +308.2% 144624 slabinfo.btrfs_delayed_node.active_objs 688.00 ± 2% +304.9% 2785 slabinfo.btrfs_delayed_node.active_slabs 35813 ± 2% +304.5% 144873 slabinfo.btrfs_delayed_node.num_objs 688.00 ± 2% +304.9% 2785 slabinfo.btrfs_delayed_node.num_slabs 1354 ± 4% +219.1% 4320 slabinfo.btrfs_extent_buffer.active_objs 1659 ± 3% +178.4% 4618 slabinfo.btrfs_extent_buffer.num_objs 5328 ± 3% +121.1% 11783 ± 2% slabinfo.btrfs_extent_map.active_objs 104.00 ± 3% +174.4% 285.33 slabinfo.btrfs_extent_map.active_slabs 5851 ± 3% +173.6% 16010 slabinfo.btrfs_extent_map.num_objs 104.00 ± 3% +174.4% 285.33 slabinfo.btrfs_extent_map.num_slabs 35092 ± 2% +312.1% 144613 slabinfo.btrfs_inode.active_objs 1261 ± 2% +309.7% 5169 slabinfo.btrfs_inode.active_slabs 35342 ± 2% +309.6% 144765 slabinfo.btrfs_inode.num_objs 1261 ± 2% +309.7% 5169 slabinfo.btrfs_inode.num_slabs 361.75 ± 17% +144.9% 886.00 ± 5% slabinfo.btrfs_ordered_extent.active_objs 514.25 ± 7% +107.9% 1069 ± 8% slabinfo.btrfs_ordered_extent.num_objs 1376 ± 7% +142.0% 3331 slabinfo.btrfs_path.active_objs 1577 ± 6% +123.4% 3522 slabinfo.btrfs_path.num_objs 80653 +134.7% 189273 slabinfo.dentry.active_objs 1945 +131.9% 4513 slabinfo.dentry.active_slabs 81731 +131.9% 189574 slabinfo.dentry.num_objs 1945 +131.9% 4513 slabinfo.dentry.num_slabs 1886 +88.2% 3550 ± 14% slabinfo.kmalloc-128.active_objs 4081 +21.9% 4975 ± 7% slabinfo.kmalloc-128.num_objs 1517 ± 5% -17.8% 1247 ± 2% slabinfo.kmalloc-192.active_objs 8374 ± 3% -9.6% 7568 slabinfo.kmalloc-8.active_objs 287.25 ± 2% -42.8% 164.33 slabinfo.kmalloc-8k.active_objs 331.50 ± 2% -39.5% 200.67 slabinfo.kmalloc-8k.num_objs 3826 -11.5% 3385 slabinfo.lsm_file_cache.active_objs 638.50 ± 4% -19.8% 512.33 ± 2% slabinfo.pde_opener.active_objs 1000 ± 3% -10.4% 896.33 slabinfo.pid.active_objs 1720 ± 5% +11.4% 1916 ± 6% slabinfo.pool_workqueue.num_objs 13389 +19.4% 15989 slabinfo.radix_tree_node.active_objs 14706 +12.0% 16473 slabinfo.radix_tree_node.num_objs 5262 ± 3% -11.7% 4649 slabinfo.vm_area_struct.active_objs 6173 ± 3% -9.2% 5606 ± 2% slabinfo.vm_area_struct.num_objs 605.00 ± 6% +67.7% 1014 ± 5% slabinfo.xfrm_dst_cache.active_objs 953.75 ± 4% +41.4% 1348 ± 4% slabinfo.xfrm_dst_cache.num_objs 0.20 ± 81% -88.1% 0.02 ± 17% perf-stat.i.MPKI 28930661 ± 28% -83.6% 4746705 ± 13% perf-stat.i.branch-instructions 0.15 ± 15% -0.1 0.03 perf-stat.i.branch-miss-rate% 2105071 ± 27% -83.3% 352162 ± 12% perf-stat.i.branch-misses 0.64 ± 16% -0.5 0.11 ± 5% perf-stat.i.cache-miss-rate% 248420 ± 14% -81.5% 45906 ± 8% perf-stat.i.cache-misses 746487 ± 9% -80.3% 146828 ± 8% perf-stat.i.cache-references 10753 ± 9% -13.8% 9266 ± 2% perf-stat.i.context-switches 0.04 ± 56% -86.1% 0.00 ± 6% perf-stat.i.cpi 1.745e+08 ± 16% -82.0% 31326038 ± 8% perf-stat.i.cpu-cycles 248.01 ± 3% -10.1% 222.96 ± 6% perf-stat.i.cpu-migrations 14.01 ± 28% -82.8% 2.41 ± 13% perf-stat.i.cycles-between-cache-misses 0.01 ± 67% -0.0 0.00 ± 3% perf-stat.i.dTLB-load-miss-rate% 63485 ± 19% -82.7% 11007 ± 10% perf-stat.i.dTLB-load-misses 27498834 ± 25% -83.7% 4474433 ± 13% perf-stat.i.dTLB-loads 0.00 ± 31% -0.0 0.00 ± 9% perf-stat.i.dTLB-store-miss-rate% 9473 ± 5% -79.2% 1972 ± 9% perf-stat.i.dTLB-store-misses 14624912 ± 16% -82.5% 2559744 ± 6% perf-stat.i.dTLB-stores 1.66 ± 15% -1.4 0.29 perf-stat.i.iTLB-load-miss-rate% 16736 ± 21% -82.8% 2885 ± 10% perf-stat.i.iTLB-load-misses 3205 ± 16% -81.6% 591.28 ± 16% perf-stat.i.iTLB-loads 1.378e+08 ± 27% -83.6% 22584373 ± 13% perf-stat.i.instructions 157.98 ± 38% -82.7% 27.37 ± 5% perf-stat.i.instructions-per-iTLB-miss 0.01 ± 21% -82.3% 0.00 ± 10% perf-stat.i.ipc 3728 ± 5% -34.3% 2450 perf-stat.i.minor-faults 10359 ± 6% -7.1% 9624 ± 3% perf-stat.i.msec 0.45 ± 45% -0.4 0.08 ± 3% perf-stat.i.node-load-miss-rate% 10995 ± 44% -84.7% 1679 ± 4% perf-stat.i.node-load-misses 48917 ± 17% -84.8% 7451 ± 7% perf-stat.i.node-loads 0.31 ± 41% -0.2 0.07 ± 8% perf-stat.i.node-store-miss-rate% 8224 ± 55% -79.8% 1661 ± 49% perf-stat.i.node-store-misses 57973 ± 31% -86.0% 8139 ± 33% perf-stat.i.node-stores 3728 ± 5% -34.3% 2450 perf-stat.i.page-faults 28746595 ± 28% -83.5% 4756457 ± 13% perf-stat.ps.branch-instructions 2092001 ± 27% -83.1% 352907 ± 12% perf-stat.ps.branch-misses 247060 ± 13% -81.4% 45998 ± 8% perf-stat.ps.cache-misses 742450 ± 9% -80.2% 147137 ± 8% perf-stat.ps.cache-references 10647 ± 9% -13.1% 9251 ± 2% perf-stat.ps.context-switches 1.735e+08 ± 15% -81.9% 31393197 ± 8% perf-stat.ps.cpu-cycles 245.60 ± 2% -9.4% 222.60 ± 6% perf-stat.ps.cpu-migrations 63084 ± 18% -82.5% 11028 ± 10% perf-stat.ps.dTLB-load-misses 27323655 ± 25% -83.6% 4483293 ± 13% perf-stat.ps.dTLB-loads 9423 ± 5% -79.0% 1976 ± 9% perf-stat.ps.dTLB-store-misses 14535653 ± 15% -82.4% 2564990 ± 6% perf-stat.ps.dTLB-stores 16632 ± 21% -82.6% 2891 ± 10% perf-stat.ps.iTLB-load-misses 3184 ± 15% -81.4% 592.02 ± 16% perf-stat.ps.iTLB-loads 1.369e+08 ± 27% -83.5% 22630330 ± 13% perf-stat.ps.instructions 3694 ± 5% -33.8% 2446 perf-stat.ps.minor-faults 10277 ± 6% -6.5% 9611 ± 3% perf-stat.ps.msec 10916 ± 44% -84.6% 1680 ± 4% perf-stat.ps.node-load-misses 48575 ± 17% -84.7% 7453 ± 7% perf-stat.ps.node-loads 8173 ± 55% -79.6% 1664 ± 49% perf-stat.ps.node-store-misses 57618 ± 31% -85.9% 8147 ± 33% perf-stat.ps.node-stores 3694 ± 5% -33.8% 2446 perf-stat.ps.page-faults 172.25 ± 91% +218.3% 548.33 ± 24% interrupts.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 102.25 ± 38% +351.5% 461.67 ± 29% interrupts.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 57.50 ± 10% +427.0% 303.00 ± 2% interrupts.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 202.50 ±107% +199.6% 606.67 interrupts.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 71.00 ± 42% +355.9% 323.67 ± 7% interrupts.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 75.50 ± 28% +515.0% 464.33 ± 47% interrupts.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 94.00 ± 35% +355.3% 428.00 ± 24% interrupts.CPU0.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 111.75 ±103% +204.5% 340.33 ± 2% interrupts.CPU0.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 20.75 ± 35% +377.1% 99.00 ± 2% interrupts.CPU0.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 21.50 ± 20% +583.7% 147.00 ± 52% interrupts.CPU0.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 2810 ± 9% +83.1% 5146 ± 9% interrupts.CPU0.CAL:Function_call_interrupts 189007 ± 27% +518.5% 1169077 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts 680.75 ±128% +545.9% 4397 ± 56% interrupts.CPU0.NMI:Non-maskable_interrupts 680.75 ±128% +545.9% 4397 ± 56% interrupts.CPU0.PMI:Performance_monitoring_interrupts 10187 ± 10% +260.7% 36743 ± 3% interrupts.CPU0.RES:Rescheduling_interrupts 123.25 ± 38% +234.3% 412.00 ± 22% interrupts.CPU0.TLB:TLB_shootdowns 51.50 ± 35% +348.5% 231.00 ± 26% interrupts.CPU1.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 38.75 ± 15% +434.2% 207.00 ± 4% interrupts.CPU1.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 37.00 ± 16% +247.7% 128.67 ± 6% interrupts.CPU1.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 42.25 ± 31% +427.8% 223.00 ± 9% interrupts.CPU1.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 2948 ± 8% +73.3% 5110 ± 5% interrupts.CPU1.CAL:Function_call_interrupts 136336 ± 27% +490.1% 804511 ± 3% interrupts.CPU1.LOC:Local_timer_interrupts 10477 ± 8% +201.5% 31595 ± 4% interrupts.CPU1.RES:Rescheduling_interrupts 131.25 ± 13% +49.6% 196.33 ± 9% interrupts.CPU1.TLB:TLB_shootdowns 2226 ± 15% +110.4% 4685 ± 27% interrupts.CPU16.CAL:Function_call_interrupts 27285 ± 20% +580.5% 185667 ± 12% interrupts.CPU16.LOC:Local_timer_interrupts 27339 ± 19% +581.6% 186357 ± 12% interrupts.CPU17.LOC:Local_timer_interrupts 2180 ± 14% +146.0% 5365 ± 25% interrupts.CPU18.CAL:Function_call_interrupts 27681 ± 18% +574.6% 186736 ± 12% interrupts.CPU18.LOC:Local_timer_interrupts 530.25 ± 39% +474.1% 3044 ± 54% interrupts.CPU18.RES:Rescheduling_interrupts 2163 ± 19% +157.8% 5576 ± 43% interrupts.CPU19.CAL:Function_call_interrupts 27638 ± 17% +576.2% 186887 ± 12% interrupts.CPU19.LOC:Local_timer_interrupts 22.50 ± 37% +891.1% 223.00 ± 53% interrupts.CPU2.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 2509 ± 15% +103.5% 5106 ± 24% interrupts.CPU2.CAL:Function_call_interrupts 68246 ± 44% +663.7% 521166 ± 3% interrupts.CPU2.LOC:Local_timer_interrupts 3283 ± 16% +202.1% 9920 ± 6% interrupts.CPU2.RES:Rescheduling_interrupts 1985 ± 27% +77.2% 3517 ± 22% interrupts.CPU20.CAL:Function_call_interrupts 20176 ± 25% +338.7% 88514 ± 4% interrupts.CPU20.LOC:Local_timer_interrupts 158.75 ± 85% +423.5% 831.00 ± 81% interrupts.CPU20.RES:Rescheduling_interrupts 17714 ± 21% +403.0% 89101 ± 4% interrupts.CPU21.LOC:Local_timer_interrupts 19989 ± 26% +345.8% 89114 ± 4% interrupts.CPU22.LOC:Local_timer_interrupts 1917 ± 16% +53.3% 2938 ± 14% interrupts.CPU23.CAL:Function_call_interrupts 20259 ± 25% +337.8% 88694 ± 4% interrupts.CPU23.LOC:Local_timer_interrupts 65.50 ± 47% +935.6% 678.33 ± 42% interrupts.CPU23.RES:Rescheduling_interrupts 42.25 ±106% +213.2% 132.33 ± 3% interrupts.CPU3.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 2459 ± 5% +65.8% 4079 ± 11% interrupts.CPU3.CAL:Function_call_interrupts 67886 ± 44% +668.3% 521548 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts 3210 ± 11% +258.6% 11513 ± 6% interrupts.CPU3.RES:Rescheduling_interrupts 30.75 ± 83% +291.3% 120.33 ± 14% interrupts.CPU3.TLB:TLB_shootdowns 2426 ± 13% +83.4% 4448 ± 36% interrupts.CPU4.CAL:Function_call_interrupts 37124 ± 15% +761.3% 319758 ± 8% interrupts.CPU4.LOC:Local_timer_interrupts 2399 ± 13% +49.5% 3586 ± 15% interrupts.CPU5.CAL:Function_call_interrupts 36954 ± 15% +766.0% 320024 ± 8% interrupts.CPU5.LOC:Local_timer_interrupts 1160 ± 20% +270.2% 4294 ± 35% interrupts.CPU5.RES:Rescheduling_interrupts 2312 ± 13% +88.3% 4354 ± 36% interrupts.CPU6.CAL:Function_call_interrupts 36695 ± 15% +771.4% 319752 ± 8% interrupts.CPU6.LOC:Local_timer_interrupts 26.75 ± 93% +454.5% 148.33 ± 20% interrupts.CPU7.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 2264 ± 14% +44.9% 3280 ± 4% interrupts.CPU7.CAL:Function_call_interrupts 37045 ± 16% +763.5% 319871 ± 8% interrupts.CPU7.LOC:Local_timer_interrupts 988469 ± 19% +467.0% 5604547 ± 4% interrupts.LOC:Local_timer_interrupts 682.50 ±128% +4928.1% 34316 ± 64% interrupts.NMI:Non-maskable_interrupts 682.50 ±128% +4928.1% 34316 ± 64% interrupts.PMI:Performance_monitoring_interrupts 42962 ± 15% +174.9% 118108 ± 3% interrupts.RES:Rescheduling_interrupts 363.75 ± 25% +174.7% 999.33 ± 8% interrupts.TLB:TLB_shootdowns 25990 ±122% +2646.0% 713704 ± 54% sched_debug.cfs_rq:/.load.avg 64235 ± 51% +9796.7% 6357197 ± 67% sched_debug.cfs_rq:/.load.max 12139 ±173% +1078.2% 143022 ± 13% sched_debug.cfs_rq:/.load.min 17932 ± 52% +9596.0% 1738721 ± 67% sched_debug.cfs_rq:/.load.stddev 38895 ±121% +1749.5% 719393 ± 29% sched_debug.cfs_rq:/.min_vruntime.avg 55160 ± 81% +1328.0% 787707 ± 27% sched_debug.cfs_rq:/.min_vruntime.max 34031 ±132% +1714.5% 617519 ± 30% sched_debug.cfs_rq:/.min_vruntime.min 4690 ± 35% +1321.4% 66668 ± 12% sched_debug.cfs_rq:/.min_vruntime.stddev 0.38 ± 34% +91.5% 0.72 ± 2% sched_debug.cfs_rq:/.nr_running.avg 0.08 ±173% +540.0% 0.53 ± 17% sched_debug.cfs_rq:/.nr_running.min 0.42 ± 16% -47.0% 0.22 ± 16% sched_debug.cfs_rq:/.nr_running.stddev 51.26 ± 71% -76.5% 12.07 ±129% sched_debug.cfs_rq:/.removed.load_avg.avg 853.33 ± 34% -88.0% 102.43 ± 81% sched_debug.cfs_rq:/.removed.load_avg.max 192.81 ± 48% -84.9% 29.03 ±107% sched_debug.cfs_rq:/.removed.load_avg.stddev 2358 ± 71% -76.5% 554.12 ±128% sched_debug.cfs_rq:/.removed.runnable_sum.avg 39372 ± 34% -88.0% 4728 ± 81% sched_debug.cfs_rq:/.removed.runnable_sum.max 8865 ± 48% -84.9% 1335 ±107% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 19.30 ± 70% -72.7% 5.27 ±133% sched_debug.cfs_rq:/.removed.util_avg.avg 334.25 ± 42% -88.3% 39.03 ± 96% sched_debug.cfs_rq:/.removed.util_avg.max 73.16 ± 55% -84.2% 11.55 ±118% sched_debug.cfs_rq:/.removed.util_avg.stddev 20.06 ±135% +715.8% 163.62 ± 23% sched_debug.cfs_rq:/.runnable_load_avg.avg 36.92 ± 91% +1230.2% 491.07 ± 42% sched_debug.cfs_rq:/.runnable_load_avg.max 11.00 ±173% +729.4% 91.23 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.min 10.98 ± 68% +1078.5% 129.38 ± 48% sched_debug.cfs_rq:/.runnable_load_avg.stddev 20325 ±130% +3215.0% 673797 ± 55% sched_debug.cfs_rq:/.runnable_weight.avg 36337 ± 89% +17255.7% 6306603 ± 67% sched_debug.cfs_rq:/.runnable_weight.max 10942 ±173% +920.9% 111713 ± 12% sched_debug.cfs_rq:/.runnable_weight.min 11513 ± 61% +14930.8% 1730643 ± 67% sched_debug.cfs_rq:/.runnable_weight.stddev -1228 +6393.3% -79797 sched_debug.cfs_rq:/.spread0.min 4690 ± 35% +1321.3% 66669 ± 12% sched_debug.cfs_rq:/.spread0.stddev 613.69 ± 7% -18.4% 501.06 ± 6% sched_debug.cfs_rq:/.util_avg.avg 1466 ± 22% -47.2% 773.77 ± 8% sched_debug.cfs_rq:/.util_avg.max 390.67 ± 34% -71.6% 110.77 ± 16% sched_debug.cfs_rq:/.util_est_enqueued.max 0.08 ±173% +12340.0% 10.37 ±134% sched_debug.cfs_rq:/.util_est_enqueued.min 110.31 ± 36% -73.9% 28.77 ± 15% sched_debug.cfs_rq:/.util_est_enqueued.stddev 525263 ± 10% +35.1% 709585 ± 9% sched_debug.cpu.avg_idle.avg 113812 ±169% +390.1% 557751 ± 17% sched_debug.cpu.avg_idle.min 47864 ± 54% +540.1% 306389 sched_debug.cpu.clock.avg 47866 ± 54% +540.1% 306389 sched_debug.cpu.clock.max 47862 ± 54% +540.1% 306388 sched_debug.cpu.clock.min 1.05 ± 31% -70.7% 0.31 ± 5% sched_debug.cpu.clock.stddev 47864 ± 54% +540.1% 306389 sched_debug.cpu.clock_task.avg 47866 ± 54% +540.1% 306389 sched_debug.cpu.clock_task.max 47862 ± 54% +540.1% 306388 sched_debug.cpu.clock_task.min 1.05 ± 31% -70.7% 0.31 ± 5% sched_debug.cpu.clock_task.stddev 20.50 ±132% +659.5% 155.68 ± 25% sched_debug.cpu.cpu_load[0].avg 44.08 ± 72% +744.9% 372.47 ± 56% sched_debug.cpu.cpu_load[0].max 11.00 ±173% +732.7% 91.60 ± 6% sched_debug.cpu.cpu_load[0].min 11.62 ± 64% +747.7% 98.48 ± 63% sched_debug.cpu.cpu_load[0].stddev 21.23 ±118% +457.1% 118.31 ± 26% sched_debug.cpu.cpu_load[1].avg 66.50 ± 70% +300.2% 266.13 ± 47% sched_debug.cpu.cpu_load[1].max 11.00 ±173% +533.9% 69.73 ± 13% sched_debug.cpu.cpu_load[1].min 12.94 ± 54% +421.5% 67.47 ± 51% sched_debug.cpu.cpu_load[1].stddev 259.42 ± 13% -50.6% 128.27 ± 31% sched_debug.cpu.cpu_load[4].max 957.37 ± 94% +594.8% 6651 sched_debug.cpu.curr->pid.avg 2129 ± 39% +315.8% 8853 sched_debug.cpu.curr->pid.max 446.83 ±173% +1051.5% 5145 ± 2% sched_debug.cpu.curr->pid.min 708.90 ± 11% +128.0% 1616 ± 4% sched_debug.cpu.curr->pid.stddev 25516 ±125% +2655.5% 703092 ± 57% sched_debug.cpu.load.avg 64235 ± 51% +9748.4% 6326187 ± 68% sched_debug.cpu.load.max 12139 ±173% +1040.9% 138498 ± 18% sched_debug.cpu.load.min 17351 ± 56% +9861.2% 1728442 ± 68% sched_debug.cpu.load.stddev 21460 ±120% +1188.5% 276515 sched_debug.cpu.nr_load_updates.avg 26877 ± 91% +939.1% 279268 sched_debug.cpu.nr_load_updates.max 20126 ±130% +1268.8% 275499 sched_debug.cpu.nr_load_updates.min 0.37 ± 36% +92.6% 0.71 ± 4% sched_debug.cpu.nr_running.avg 0.08 ±173% +500.0% 0.50 ± 16% sched_debug.cpu.nr_running.min 0.42 ± 16% -42.6% 0.24 ± 15% sched_debug.cpu.nr_running.stddev 43921 ±167% +1277.1% 604829 ± 5% sched_debug.cpu.nr_switches.avg 54096 ±130% +1084.5% 640782 ± 5% sched_debug.cpu.nr_switches.max 42673 ±171% +1252.3% 577085 ± 5% sched_debug.cpu.nr_switches.min 2063 ± 10% +1163.0% 26068 ± 20% sched_debug.cpu.nr_switches.stddev -1.94 -5537.1% 105.53 ± 47% sched_debug.cpu.nr_uninterruptible.avg 15.08 ± 45% +1351.0% 218.87 ± 35% sched_debug.cpu.nr_uninterruptible.max 11.51 ±106% +687.1% 90.62 ± 38% sched_debug.cpu.nr_uninterruptible.stddev 47862 ± 54% +540.1% 306388 sched_debug.cpu_clk 44986 ± 58% +574.6% 303491 sched_debug.ktime 48925 ± 52% +527.3% 306925 sched_debug.sched_clk 12.36 ±122% -12.4 0.00 perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.08 ±103% -6.1 0.00 perf-profile.calltrace.cycles-pp.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.62 ±101% -5.6 0.00 perf-profile.calltrace.cycles-pp.btrfs_unlink.vfs_unlink.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.7 0.71 ± 26% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +1.1 1.14 ± 72% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create.path_openat.do_filp_open 0.18 ±173% +1.3 1.48 ± 60% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +1.4 1.36 ± 18% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.15 ±173% +1.5 1.66 ± 53% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.31 ±101% +1.8 2.16 ± 51% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.00 +1.9 1.91 ± 57% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.45 ±114% +2.3 2.76 ± 54% perf-profile.calltrace.cycles-pp.__btrfs_drop_extents.insert_reserved_file_extent.btrfs_finish_ordered_io.normal_work_helper.process_one_work 0.00 +2.4 2.42 ± 58% perf-profile.calltrace.cycles-pp.btrfs_new_inode.btrfs_create.path_openat.do_filp_open.do_sys_open 0.54 ±122% +2.5 3.02 ± 56% perf-profile.calltrace.cycles-pp.insert_reserved_file_extent.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread 0.00 +2.8 2.77 ± 27% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link 0.27 ±173% +2.8 3.09 ± 45% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 0.27 ±173% +2.9 3.18 ± 42% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 0.58 ±100% +3.0 3.54 ± 41% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.93 ±118% +3.1 4.05 ± 60% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +3.4 3.39 ± 26% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create 0.00 +3.4 3.42 ± 26% perf-profile.calltrace.cycles-pp.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat 0.49 ±100% +3.5 3.95 ± 38% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +3.9 3.89 ± 54% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +4.1 4.08 ± 31% perf-profile.calltrace.cycles-pp.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat.do_filp_open 0.00 +4.2 4.23 ± 31% perf-profile.calltrace.cycles-pp.btrfs_add_link.btrfs_create.path_openat.do_filp_open.do_sys_open 0.79 ±101% +4.3 5.06 ± 38% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 1.25 ±101% +7.5 8.72 ± 39% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 0.00 +8.0 8.02 ± 34% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.00 +8.7 8.71 ± 42% perf-profile.calltrace.cycles-pp.btrfs_create.path_openat.do_filp_open.do_sys_open.do_syscall_64 0.00 +11.1 11.08 ± 45% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +21.0 20.98 ± 46% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.00 +31.1 31.14 ± 46% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +38.0 37.98 ± 45% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +38.1 38.11 ± 45% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +38.1 38.11 ± 45% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 12.36 ±122% -12.4 0.00 perf-profile.children.cycles-pp.do_unlinkat 6.08 ±103% -6.1 0.00 perf-profile.children.cycles-pp.vfs_unlink 5.62 ±101% -5.6 0.00 perf-profile.children.cycles-pp.btrfs_unlink 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.do_faccessat 0.01 ±173% +0.1 0.08 ± 31% perf-profile.children.cycles-pp.block_rsv_use_bytes 0.00 +0.1 0.09 ± 39% perf-profile.children.cycles-pp.may_open 0.01 ±173% +0.1 0.10 ± 22% perf-profile.children.cycles-pp.run_local_timers 0.00 +0.1 0.09 ± 22% perf-profile.children.cycles-pp.trigger_load_balance 0.01 ±173% +0.1 0.11 ± 68% perf-profile.children.cycles-pp.__x64_sys_close 0.00 +0.1 0.10 ± 31% perf-profile.children.cycles-pp.resched_curr 0.00 +0.1 0.10 ± 52% perf-profile.children.cycles-pp.__d_alloc 0.01 ±173% +0.1 0.12 ± 29% perf-profile.children.cycles-pp.tick_check_broadcast_expired 0.01 ±173% +0.1 0.12 ± 42% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.01 ±173% +0.1 0.12 ± 23% perf-profile.children.cycles-pp.security_inode_alloc 0.00 +0.1 0.11 ± 26% perf-profile.children.cycles-pp.call_cpuidle 0.00 +0.1 0.11 ± 29% perf-profile.children.cycles-pp.tick_nohz_tick_stopped 0.01 ±173% +0.1 0.14 ± 39% perf-profile.children.cycles-pp.note_gp_changes 0.00 +0.1 0.13 ± 68% perf-profile.children.cycles-pp.__push_leaf_left 0.01 ±173% +0.1 0.15 ± 66% perf-profile.children.cycles-pp.cpu_load_update 0.01 ±173% +0.1 0.15 ± 30% perf-profile.children.cycles-pp.d_alloc 0.01 ±173% +0.1 0.15 ± 35% perf-profile.children.cycles-pp.alloc_set_pte 0.01 ±173% +0.1 0.15 ± 55% perf-profile.children.cycles-pp.get_cpu_device 0.00 +0.1 0.14 ± 43% perf-profile.children.cycles-pp.btrfs_set_token_64 0.01 ±173% +0.1 0.16 ± 44% perf-profile.children.cycles-pp.inode_init_always 0.00 +0.1 0.15 ± 33% perf-profile.children.cycles-pp.ksoftirqd_running 0.07 ±107% +0.1 0.22 ± 26% perf-profile.children.cycles-pp.intel_pmu_disable_all 0.00 +0.2 0.15 ± 5% perf-profile.children.cycles-pp.run_rebalance_domains 0.01 ±173% +0.2 0.16 ± 49% perf-profile.children.cycles-pp.perf_event_task_tick 0.06 ±128% +0.2 0.21 ± 41% perf-profile.children.cycles-pp.select_task_rq_fair 0.01 ±173% +0.2 0.17 ± 17% perf-profile.children.cycles-pp.hrtimer_forward 0.00 +0.2 0.16 ± 41% perf-profile.children.cycles-pp.tick_program_event 0.01 ±173% +0.2 0.17 ± 38% perf-profile.children.cycles-pp.wp_page_copy 0.00 +0.2 0.17 ± 22% perf-profile.children.cycles-pp.idle_cpu 0.00 +0.2 0.17 ± 22% perf-profile.children.cycles-pp.cpu_load_update_active 0.00 +0.2 0.17 ± 45% perf-profile.children.cycles-pp.update_cfs_group 0.01 ±173% +0.2 0.18 ± 13% perf-profile.children.cycles-pp.update_blocked_averages 0.01 ±173% +0.2 0.18 ± 82% perf-profile.children.cycles-pp.do_dentry_open 0.03 ±173% +0.2 0.20 ± 30% perf-profile.children.cycles-pp.push_leaf_left 0.01 ±173% +0.2 0.19 ± 43% perf-profile.children.cycles-pp.d_alloc_parallel 0.05 ±173% +0.2 0.24 ± 50% perf-profile.children.cycles-pp.rb_next 0.03 ±173% +0.2 0.21 ± 62% perf-profile.children.cycles-pp.rcu_idle_exit 0.00 +0.2 0.19 ± 47% perf-profile.children.cycles-pp.__hrtimer_get_next_event 0.00 +0.2 0.20 ± 48% perf-profile.children.cycles-pp.rcu_eqs_enter 0.01 ±173% +0.2 0.21 ± 72% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.00 +0.2 0.20 ± 52% perf-profile.children.cycles-pp.pm_qos_read_value 0.03 ±173% +0.2 0.23 ± 50% perf-profile.children.cycles-pp.do_wp_page 0.01 ±173% +0.2 0.24 ± 67% perf-profile.children.cycles-pp.__push_leaf_right 0.10 ±146% +0.2 0.33 ± 58% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.00 +0.2 0.24 ± 26% perf-profile.children.cycles-pp.btrfs_alloc_inode 0.00 +0.2 0.24 ± 54% perf-profile.children.cycles-pp.inode_tree_add 0.05 ±173% +0.2 0.29 ± 57% perf-profile.children.cycles-pp.clear_state_bit 0.06 ±173% +0.3 0.31 ± 47% perf-profile.children.cycles-pp.timerqueue_del 0.07 ±173% +0.3 0.33 ± 49% perf-profile.children.cycles-pp.__remove_hrtimer 0.01 ±173% +0.3 0.27 ± 52% perf-profile.children.cycles-pp.poll_idle 0.04 ±173% +0.3 0.30 ± 51% perf-profile.children.cycles-pp.find_busiest_group 0.03 ±173% +0.3 0.30 ± 31% perf-profile.children.cycles-pp.interrupt_entry 0.01 ±173% +0.3 0.29 ± 73% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.10 ±100% +0.3 0.38 ± 19% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.01 ±173% +0.3 0.30 ± 51% perf-profile.children.cycles-pp.update_sd_lb_stats 0.00 +0.3 0.29 ± 36% perf-profile.children.cycles-pp.new_slab 0.04 ±173% +0.3 0.34 ± 64% perf-profile.children.cycles-pp.__hrtimer_next_event_base 0.07 ±173% +0.3 0.38 ± 48% perf-profile.children.cycles-pp.btrfs_get_or_create_delayed_node 0.13 ±115% +0.3 0.44 ± 33% perf-profile.children.cycles-pp.update_rq_clock 0.07 ±107% +0.3 0.39 ± 25% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.01 ±173% +0.3 0.33 ± 13% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.3 0.33 ± 13% perf-profile.children.cycles-pp.menu_reflect 0.01 ±173% +0.3 0.36 ± 55% perf-profile.children.cycles-pp.timerqueue_add 0.21 ±127% +0.4 0.57 ± 45% perf-profile.children.cycles-pp.btrfs_tree_lock 0.00 +0.4 0.37 ± 38% perf-profile.children.cycles-pp.___slab_alloc 0.00 +0.4 0.37 ± 38% perf-profile.children.cycles-pp.__slab_alloc 0.08 ±100% +0.4 0.46 ± 16% perf-profile.children.cycles-pp.run_timer_softirq 0.18 ±122% +0.4 0.56 ± 17% perf-profile.children.cycles-pp.push_leaf_right 0.01 ±173% +0.4 0.40 ± 32% perf-profile.children.cycles-pp.alloc_inode 0.01 ±173% +0.4 0.42 ± 36% perf-profile.children.cycles-pp.new_inode_pseudo 0.07 ±107% +0.4 0.49 ± 28% perf-profile.children.cycles-pp.native_sched_clock 0.01 ±173% +0.4 0.44 ± 67% perf-profile.children.cycles-pp.enqueue_hrtimer 0.03 ±173% +0.4 0.45 ± 77% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.07 ±107% +0.4 0.50 ± 26% perf-profile.children.cycles-pp.sched_clock 0.01 ±173% +0.4 0.45 ± 41% perf-profile.children.cycles-pp.new_inode 0.00 +0.4 0.45 ± 54% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.13 ±115% +0.5 0.59 ± 29% perf-profile.children.cycles-pp.lapic_next_deadline 0.01 ±173% +0.5 0.49 ± 72% perf-profile.children.cycles-pp.wait_reserve_ticket 0.00 +0.5 0.48 ± 66% perf-profile.children.cycles-pp.prepare_to_wait_event 0.00 +0.5 0.49 ± 56% perf-profile.children.cycles-pp.btrfs_insert_delayed_dir_index 0.09 ±173% +0.5 0.58 ± 68% perf-profile.children.cycles-pp.irq_enter 0.00 +0.5 0.50 ± 38% perf-profile.children.cycles-pp.find_next_bit 0.17 ±119% +0.5 0.67 ± 36% perf-profile.children.cycles-pp.btrfs_delayed_update_inode 0.07 ±107% +0.5 0.57 ± 18% perf-profile.children.cycles-pp.sched_clock_cpu 0.04 ±173% +0.5 0.54 ± 64% perf-profile.children.cycles-pp.rebalance_domains 0.04 ±173% +0.5 0.57 ± 61% perf-profile.children.cycles-pp.load_balance 0.24 ±133% +0.5 0.77 ± 33% perf-profile.children.cycles-pp.btrfs_update_inode 0.11 ±128% +0.6 0.67 ± 22% perf-profile.children.cycles-pp.native_irq_return_iret 0.03 ±173% +0.6 0.63 ± 66% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.17 ±100% +0.6 0.79 ± 12% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.14 ±107% +0.6 0.76 ± 17% perf-profile.children.cycles-pp.native_write_msr 0.00 +0.7 0.69 ± 71% perf-profile.children.cycles-pp.__next_timer_interrupt 0.10 ±173% +0.7 0.83 ± 30% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.21 ±135% +0.8 0.98 ± 17% perf-profile.children.cycles-pp.clockevents_program_event 0.03 ±173% +0.8 0.80 ± 27% perf-profile.children.cycles-pp.schedule_idle 0.16 ±139% +0.8 0.95 ± 7% perf-profile.children.cycles-pp.ktime_get 0.04 ±173% +0.8 0.84 ± 20% perf-profile.children.cycles-pp.split_leaf 0.16 ±139% +0.9 1.03 ± 30% perf-profile.children.cycles-pp.__wake_up_common_lock 0.00 +0.9 0.87 ± 56% perf-profile.children.cycles-pp.btrfs_lookup 0.00 +0.9 0.87 ± 56% perf-profile.children.cycles-pp.btrfs_lookup_dentry 0.14 ±173% +0.9 1.07 ± 18% perf-profile.children.cycles-pp.autoremove_wake_function 0.06 ±128% +0.9 1.01 ±115% perf-profile.children.cycles-pp.btrfs_run_delalloc_range 0.06 ±128% +0.9 1.01 ±115% perf-profile.children.cycles-pp.cow_file_range 0.13 ±115% +1.0 1.09 ± 61% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.15 ±154% +1.0 1.13 ± 15% perf-profile.children.cycles-pp.__wake_up_common 0.19 ±100% +1.0 1.19 ± 13% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.26 ±109% +1.3 1.53 ± 51% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 0.12 ±173% +1.3 1.40 ± 51% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.23 ±127% +1.5 1.70 ± 61% perf-profile.children.cycles-pp.tick_nohz_next_event 0.04 ±173% +1.6 1.63 ± 13% perf-profile.children.cycles-pp.scheduler_tick 0.82 ±134% +1.9 2.72 ± 37% perf-profile.children.cycles-pp.setup_items_for_insert 0.24 ±122% +2.0 2.20 ± 62% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +2.5 2.45 ± 59% perf-profile.children.cycles-pp.btrfs_new_inode 0.54 ±122% +2.5 3.02 ± 56% perf-profile.children.cycles-pp.insert_reserved_file_extent 0.41 ±149% +3.1 3.48 ± 38% perf-profile.children.cycles-pp.update_process_times 1.00 ±102% +3.1 4.12 ± 26% perf-profile.children.cycles-pp.__softirqentry_text_start 0.93 ±118% +3.1 4.05 ± 60% perf-profile.children.cycles-pp.normal_work_helper 0.42 ±149% +3.1 3.57 ± 35% perf-profile.children.cycles-pp.tick_sched_handle 0.00 +3.4 3.45 ± 27% perf-profile.children.cycles-pp.insert_with_overflow 0.69 ±111% +3.5 4.15 ± 32% perf-profile.children.cycles-pp.tick_sched_timer 0.77 ±106% +3.7 4.51 ± 18% perf-profile.children.cycles-pp.irq_exit 0.00 +4.1 4.11 ± 32% perf-profile.children.cycles-pp.btrfs_insert_dir_item 0.38 ±102% +4.2 4.61 ± 54% perf-profile.children.cycles-pp.menu_select 0.00 +4.3 4.26 ± 32% perf-profile.children.cycles-pp.btrfs_add_link 0.97 ±112% +5.2 6.13 ± 21% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.85 ±106% +5.9 6.77 ± 33% perf-profile.children.cycles-pp.btrfs_insert_empty_items 1.24 ±104% +7.3 8.55 ± 21% perf-profile.children.cycles-pp.hrtimer_interrupt 0.00 +8.8 8.78 ± 42% perf-profile.children.cycles-pp.btrfs_create 0.24 ±101% +11.2 11.40 ± 46% perf-profile.children.cycles-pp.path_openat 0.24 ±101% +11.2 11.42 ± 46% perf-profile.children.cycles-pp.do_filp_open 0.30 ±103% +11.3 11.62 ± 46% perf-profile.children.cycles-pp.do_sys_open 1.78 ±100% +12.0 13.76 ± 15% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 2.22 ±100% +12.4 14.62 ± 14% perf-profile.children.cycles-pp.apic_timer_interrupt 0.00 +38.1 38.11 ± 45% perf-profile.children.cycles-pp.start_secondary 0.00 +0.1 0.07 ± 28% perf-profile.self.cycles-pp.sched_clock_idle_wakeup_event 0.00 +0.1 0.08 ± 36% perf-profile.self.cycles-pp.irq_enter 0.00 +0.1 0.08 ± 48% perf-profile.self.cycles-pp.___slab_alloc 0.00 +0.1 0.09 ± 27% perf-profile.self.cycles-pp.trigger_load_balance 0.01 ±173% +0.1 0.10 ± 22% perf-profile.self.cycles-pp.tick_check_broadcast_expired 0.00 +0.1 0.10 ± 31% perf-profile.self.cycles-pp.resched_curr 0.00 +0.1 0.11 ± 26% perf-profile.self.cycles-pp.call_cpuidle 0.00 +0.1 0.13 ± 33% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.00 +0.1 0.13 ± 31% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length 0.01 ±173% +0.1 0.14 ± 62% perf-profile.self.cycles-pp.get_cpu_device 0.00 +0.1 0.13 ± 43% perf-profile.self.cycles-pp.clockevents_program_event 0.00 +0.1 0.13 ± 36% perf-profile.self.cycles-pp.__hrtimer_get_next_event 0.07 ±107% +0.1 0.20 ± 20% perf-profile.self.cycles-pp._raw_read_lock 0.01 ±173% +0.1 0.15 ± 66% perf-profile.self.cycles-pp.cpu_load_update 0.00 +0.1 0.14 ± 37% perf-profile.self.cycles-pp.filemap_map_pages 0.00 +0.1 0.15 ± 33% perf-profile.self.cycles-pp.ksoftirqd_running 0.01 ±173% +0.2 0.16 ± 49% perf-profile.self.cycles-pp.perf_event_task_tick 0.00 +0.2 0.15 ± 55% perf-profile.self.cycles-pp.scheduler_tick 0.00 +0.2 0.16 ± 24% perf-profile.self.cycles-pp.new_slab 0.01 ±173% +0.2 0.17 ± 17% perf-profile.self.cycles-pp.hrtimer_forward 0.03 ±173% +0.2 0.18 ± 44% perf-profile.self.cycles-pp.try_to_wake_up 0.00 +0.2 0.16 ± 41% perf-profile.self.cycles-pp.tick_program_event 0.00 +0.2 0.16 ± 23% perf-profile.self.cycles-pp.cpu_load_update_active 0.00 +0.2 0.16 ± 44% perf-profile.self.cycles-pp.update_cfs_group 0.00 +0.2 0.17 ± 22% perf-profile.self.cycles-pp.idle_cpu 0.03 ±173% +0.2 0.19 ± 62% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.01 ±173% +0.2 0.18 ± 37% perf-profile.self.cycles-pp.update_sd_lb_stats 0.07 ±107% +0.2 0.25 ± 12% perf-profile.self.cycles-pp.update_rq_clock 0.05 ±173% +0.2 0.24 ± 50% perf-profile.self.cycles-pp.rb_next 0.00 +0.2 0.19 ± 55% perf-profile.self.cycles-pp.rcu_eqs_enter 0.00 +0.2 0.20 ± 50% perf-profile.self.cycles-pp.pm_qos_read_value 0.00 +0.2 0.21 ± 46% perf-profile.self.cycles-pp.smp_apic_timer_interrupt 0.01 ±173% +0.2 0.23 ± 41% perf-profile.self.cycles-pp.__hrtimer_run_queues 0.10 ±146% +0.2 0.32 ± 59% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.00 +0.2 0.23 ± 48% perf-profile.self.cycles-pp.inode_tree_add 0.05 ±173% +0.2 0.29 ± 57% perf-profile.self.cycles-pp.__btrfs_release_delayed_node 0.07 ±107% +0.2 0.31 ± 67% perf-profile.self.cycles-pp.setup_items_for_insert 0.01 ±173% +0.3 0.26 ± 48% perf-profile.self.cycles-pp.timerqueue_add 0.01 ±173% +0.3 0.27 ± 52% perf-profile.self.cycles-pp.poll_idle 0.03 ±173% +0.3 0.30 ± 31% perf-profile.self.cycles-pp.interrupt_entry 0.10 ±100% +0.3 0.37 ± 20% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.00 +0.3 0.28 ± 12% perf-profile.self.cycles-pp.menu_reflect 0.01 ±173% +0.3 0.29 ± 51% perf-profile.self.cycles-pp.hrtimer_interrupt 0.03 ±173% +0.3 0.31 ± 62% perf-profile.self.cycles-pp.__hrtimer_next_event_base 0.01 ±173% +0.3 0.30 ± 16% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.07 ±107% +0.3 0.37 ± 19% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.03 ±173% +0.3 0.33 ± 37% perf-profile.self.cycles-pp.run_timer_softirq 0.00 +0.3 0.31 ± 72% perf-profile.self.cycles-pp.__next_timer_interrupt 0.06 ±128% +0.3 0.39 ± 50% perf-profile.self.cycles-pp.tick_nohz_next_event 0.00 +0.4 0.37 ± 63% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.03 ±173% +0.4 0.43 ± 33% perf-profile.self.cycles-pp.native_sched_clock 0.01 ±173% +0.4 0.44 ± 38% perf-profile.self.cycles-pp.ktime_get 0.00 +0.5 0.46 ± 44% perf-profile.self.cycles-pp.find_next_bit 0.10 ±100% +0.5 0.64 ± 19% perf-profile.self.cycles-pp.__sched_text_start 0.17 ±100% +0.5 0.72 ± 27% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.11 ±128% +0.5 0.66 ± 23% perf-profile.self.cycles-pp.native_irq_return_iret 0.14 ±107% +0.6 0.76 ± 17% perf-profile.self.cycles-pp.native_write_msr 0.10 ±173% +0.7 0.82 ± 31% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.12 ±105% +1.6 1.68 ± 40% perf-profile.self.cycles-pp.menu_select *************************************************************************************************** ivb44: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory ========================================================================================= compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/directio/1HDD/btrfs/x86_64-rhel-7.6/hdd/debian-x86_64-2018-04-03.cgz/ivb44/MWUL/fxmark/0x42d commit: 4438cf50e7 ("doc, block, bfq: add information on bfq execution time") 56a85fd837 ("loop: properly observe rotational flag of underlying device") 4438cf50e7b315ff 56a85fd8376ef32458efb6ea97a ---------------- --------------------------- %stddev %change %stddev \ | \ 193.22 ± 5% +52.0% 293.74 fxmark.hdd_btrfs_MWUL_12_directio.idle_sec 33.12 ± 5% +67.1% 55.34 fxmark.hdd_btrfs_MWUL_12_directio.iowait_sec 14.29 +10.1% 15.73 fxmark.hdd_btrfs_MWUL_12_directio.iowait_util 19.83 ± 4% +52.2% 30.17 fxmark.hdd_btrfs_MWUL_12_directio.real_sec 19.82 ± 4% +52.0% 30.12 fxmark.hdd_btrfs_MWUL_12_directio.secs 4.96 -53.2% 2.32 ± 10% fxmark.hdd_btrfs_MWUL_12_directio.sys_sec 2.15 ± 4% -69.3% 0.66 ± 10% fxmark.hdd_btrfs_MWUL_12_directio.sys_util 0.13 ± 5% -38.0% 0.08 ± 2% fxmark.hdd_btrfs_MWUL_12_directio.user_util 41694 -90.3% 4050 ± 12% fxmark.hdd_btrfs_MWUL_12_directio.works 2109 ± 4% -93.6% 134.50 ± 12% fxmark.hdd_btrfs_MWUL_12_directio.works/sec 272.51 ± 10% +50.6% 410.51 fxmark.hdd_btrfs_MWUL_16_directio.idle_sec 33.40 ± 12% +66.8% 55.72 fxmark.hdd_btrfs_MWUL_16_directio.iowait_sec 10.70 +11.0% 11.88 fxmark.hdd_btrfs_MWUL_16_directio.iowait_util 19.98 ± 10% +51.0% 30.17 fxmark.hdd_btrfs_MWUL_16_directio.real_sec 19.92 ± 10% +51.6% 30.20 fxmark.hdd_btrfs_MWUL_16_directio.secs 5.28 ± 4% -55.1% 2.37 ± 6% fxmark.hdd_btrfs_MWUL_16_directio.sys_sec 1.71 ± 8% -70.4% 0.51 ± 6% fxmark.hdd_btrfs_MWUL_16_directio.sys_util 0.11 ± 9% -37.0% 0.07 ± 5% fxmark.hdd_btrfs_MWUL_16_directio.user_util 41487 -91.0% 3729 ± 12% fxmark.hdd_btrfs_MWUL_16_directio.works 2106 ± 10% -94.1% 123.51 ± 12% fxmark.hdd_btrfs_MWUL_16_directio.works/sec 2.92 ± 3% +562.3% 19.34 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.iowait_sec 4.47 ± 2% +507.0% 27.16 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.real_sec 4.41 ± 2% +514.1% 27.10 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.secs 0.02 ± 19% +488.9% 0.13 ± 16% fxmark.hdd_btrfs_MWUL_1_directio.softirq_sec 1.38 ± 2% +411.2% 7.07 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.sys_sec 31.18 ± 2% -15.7% 26.29 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.sys_util 0.11 ± 6% +218.2% 0.35 ± 4% fxmark.hdd_btrfs_MWUL_1_directio.user_sec 2.48 ± 7% -47.6% 1.30 ± 3% fxmark.hdd_btrfs_MWUL_1_directio.user_util 40914 +306.1% 166158 fxmark.hdd_btrfs_MWUL_1_directio.works 9278 ± 2% -33.9% 6136 ± 2% fxmark.hdd_btrfs_MWUL_1_directio.works/sec 4.26 ± 16% +64.6% 7.02 ± 38% fxmark.hdd_btrfs_MWUL_2_directio.idle_sec 21.71 ± 15% +132.2% 50.41 ± 5% fxmark.hdd_btrfs_MWUL_2_directio.iowait_sec 74.52 ± 4% +14.4% 85.28 ± 5% fxmark.hdd_btrfs_MWUL_2_directio.iowait_util 14.81 ± 11% +103.3% 30.11 fxmark.hdd_btrfs_MWUL_2_directio.real_sec 14.74 ± 11% +103.8% 30.05 fxmark.hdd_btrfs_MWUL_2_directio.secs 0.33 ± 13% -28.2% 0.24 ± 18% fxmark.hdd_btrfs_MWUL_2_directio.softirq_util 2.79 ± 3% -51.2% 1.36 ± 2% fxmark.hdd_btrfs_MWUL_2_directio.sys_sec 9.69 ± 8% -76.3% 2.30 ± 2% fxmark.hdd_btrfs_MWUL_2_directio.sys_util 0.65 -52.3% 0.31 ± 6% fxmark.hdd_btrfs_MWUL_2_directio.user_util 41050 -92.1% 3249 ± 6% fxmark.hdd_btrfs_MWUL_2_directio.works 2821 ± 10% -96.2% 108.14 ± 6% fxmark.hdd_btrfs_MWUL_2_directio.works/sec 29.77 ± 5% +114.6% 63.89 fxmark.hdd_btrfs_MWUL_4_directio.idle_sec 22.13 ± 6% +133.3% 51.63 fxmark.hdd_btrfs_MWUL_4_directio.iowait_sec 14.37 ± 6% +110.4% 30.22 fxmark.hdd_btrfs_MWUL_4_directio.real_sec 14.30 ± 6% +110.9% 30.16 fxmark.hdd_btrfs_MWUL_4_directio.secs 0.10 ± 14% +42.9% 0.15 ± 16% fxmark.hdd_btrfs_MWUL_4_directio.softirq_sec 0.19 ± 9% -32.1% 0.13 ± 17% fxmark.hdd_btrfs_MWUL_4_directio.softirq_util 3.59 ± 3% -38.4% 2.21 ± 10% fxmark.hdd_btrfs_MWUL_4_directio.sys_sec 6.45 ± 3% -71.0% 1.87 ± 9% fxmark.hdd_btrfs_MWUL_4_directio.sys_util 0.21 ± 10% +15.3% 0.24 ± 4% fxmark.hdd_btrfs_MWUL_4_directio.user_sec 0.38 ± 7% -45.5% 0.21 ± 4% fxmark.hdd_btrfs_MWUL_4_directio.user_util 41386 -82.2% 7348 ± 15% fxmark.hdd_btrfs_MWUL_4_directio.works 2906 ± 6% -91.6% 243.53 ± 14% fxmark.hdd_btrfs_MWUL_4_directio.works/sec 110.14 ± 10% +61.5% 177.85 fxmark.hdd_btrfs_MWUL_8_directio.idle_sec 31.07 ± 10% +74.5% 54.21 fxmark.hdd_btrfs_MWUL_8_directio.iowait_sec 18.81 ± 10% +60.1% 30.10 fxmark.hdd_btrfs_MWUL_8_directio.real_sec 18.78 ± 10% +60.0% 30.05 fxmark.hdd_btrfs_MWUL_8_directio.secs 0.10 ± 11% -32.5% 0.07 ± 16% fxmark.hdd_btrfs_MWUL_8_directio.softirq_util 4.58 ± 5% -51.4% 2.22 ± 8% fxmark.hdd_btrfs_MWUL_8_directio.sys_sec 3.15 ± 5% -69.9% 0.95 ± 8% fxmark.hdd_btrfs_MWUL_8_directio.sys_util 0.34 ± 11% -20.0% 0.27 ± 4% fxmark.hdd_btrfs_MWUL_8_directio.user_sec 0.23 ± 13% -50.6% 0.12 ± 4% fxmark.hdd_btrfs_MWUL_8_directio.user_util 41677 -89.7% 4290 ± 9% fxmark.hdd_btrfs_MWUL_8_directio.works 2246 ± 11% -93.6% 142.77 ± 9% fxmark.hdd_btrfs_MWUL_8_directio.works/sec 209.59 ± 2% +407.7% 1064 fxmark.time.elapsed_time 209.59 ± 2% +407.7% 1064 fxmark.time.elapsed_time.max 14186 ± 2% -20.7% 11254 ± 4% fxmark.time.file_system_inputs 1402120 ± 2% +363.3% 6495744 fxmark.time.file_system_outputs 108821 ± 4% +381.2% 523616 ± 5% fxmark.time.involuntary_context_switches 7.25 ± 17% -69.0% 2.25 ± 19% fxmark.time.percent_of_cpu_this_job_got 7.89 +141.6% 19.06 fxmark.time.system_time 167121 ± 2% +15.3% 192617 ± 2% fxmark.time.voluntary_context_switches 35.66 ± 3% +9.8 45.51 mpstat.cpu.all.iowait% 5.05 -1.5 3.53 ± 2% mpstat.cpu.all.sys% 4.43 ± 15% -3.5 0.98 ± 16% mpstat.cpu.all.usr% 694220 ± 4% +282.7% 2656546 numa-numastat.node0.local_node 703679 ± 4% +279.0% 2666820 numa-numastat.node0.numa_hit 159588 ± 11% +155.5% 407780 ± 11% numa-numastat.node1.local_node 166029 ± 10% +149.0% 413398 ± 12% numa-numastat.node1.numa_hit 52.50 ± 2% -7.1% 48.75 vmstat.cpu.id 35.25 ± 3% +28.4% 45.25 vmstat.cpu.wa 99.75 ± 13% -75.9% 24.00 ± 12% vmstat.io.bi 44840 -43.0% 25581 vmstat.io.bo 1338807 +26.4% 1692708 vmstat.memory.cache 18901 -59.9% 7576 vmstat.system.cs 17520 ± 2% -22.9% 13503 ± 10% vmstat.system.in 62746964 ± 12% +374.6% 2.978e+08 ± 46% cpuidle.C1.time 1425690 ± 2% +173.1% 3893177 ± 39% cpuidle.C1.usage 2.554e+08 ± 21% +484.7% 1.494e+09 ± 9% cpuidle.C1E.time 940670 ± 50% +394.7% 4653803 ± 42% cpuidle.C1E.usage 1.083e+09 ± 15% +332.0% 4.679e+09 ± 9% cpuidle.C6.time 1591508 ± 31% +249.6% 5564109 ± 2% cpuidle.C6.usage 218610 ± 2% +303.8% 882839 ± 73% cpuidle.POLL.time 314892 ± 2% +135.6% 741934 ± 16% cpuidle.POLL.usage 54.36 ± 2% -8.4% 49.79 iostat.cpu.idle 35.27 ± 3% +28.8% 45.42 iostat.cpu.iowait 4.68 -23.5% 3.59 iostat.cpu.system 4.29 ± 16% -77.7% 0.96 ± 16% iostat.cpu.user 1.38 -18.3% 1.13 iostat.sda.avgqu-sz 389.97 -47.9% 203.35 iostat.sda.avgrq-sz 10.65 +6.2% 11.31 iostat.sda.util 132.07 +19.4% 157.69 iostat.sda.w/s 24369 -38.4% 15019 iostat.sda.wkB/s 91.11 +17.1% 106.68 iostat.sda.wrqm/s 196784 ± 20% +94.6% 382883 ± 9% numa-meminfo.node0.Active 80877 ± 5% +184.1% 229800 ± 3% numa-meminfo.node0.Active(file) 657695 ± 2% +33.4% 877093 ± 3% numa-meminfo.node0.FilePages 43803 ± 11% +142.4% 106184 ± 4% numa-meminfo.node0.Inactive 32692 ± 8% +188.5% 94307 ± 5% numa-meminfo.node0.Inactive(file) 58506 ± 2% +204.2% 177961 numa-meminfo.node0.KReclaimable 1016173 ± 5% +38.7% 1409076 ± 2% numa-meminfo.node0.MemUsed 58506 ± 2% +204.2% 177961 numa-meminfo.node0.SReclaimable 54423 ± 4% +58.7% 86372 ± 5% numa-meminfo.node0.SUnreclaim 112930 ± 2% +134.1% 264334 ± 2% numa-meminfo.node0.Slab 9361 ± 55% +70.2% 15936 ± 25% numa-meminfo.node1.Inactive 32598 ± 4% +40.9% 45932 ± 3% numa-meminfo.node1.KReclaimable 32598 ± 4% +40.9% 45932 ± 3% numa-meminfo.node1.SReclaimable 70803 ± 3% +23.3% 87282 ± 6% numa-meminfo.node1.Slab 338162 +47.6% 499163 meminfo.Active 87276 +180.0% 244379 meminfo.Active(file) 158421 +31.4% 208195 meminfo.AnonHugePages 1247393 +17.8% 1469212 meminfo.Cached 53141 +129.7% 122069 meminfo.Inactive 17515 +34.6% 23581 meminfo.Inactive(anon) 35625 +176.5% 98488 meminfo.Inactive(file) 91080 +145.8% 223850 meminfo.KReclaimable 1925145 +18.7% 2285482 meminfo.Memused 697.25 ±100% +140.0% 1673 ± 13% meminfo.Mlocked 91080 +145.8% 223850 meminfo.SReclaimable 92595 +37.9% 127720 meminfo.SUnreclaim 17846 +39.9% 24961 meminfo.Shmem 183676 +91.4% 351571 meminfo.Slab 9537 ± 3% -74.7% 2414 meminfo.max_used_kB 20184 ± 5% +184.6% 57444 ± 3% numa-vmstat.node0.nr_active_file 754305 ± 13% +254.8% 2676564 ± 6% numa-vmstat.node0.nr_dirtied 179.50 ± 8% +26.0% 226.25 ± 3% numa-vmstat.node0.nr_dirty 164374 ± 2% +33.4% 219262 ± 3% numa-vmstat.node0.nr_file_pages 8159 ± 8% +188.9% 23570 ± 5% numa-vmstat.node0.nr_inactive_file 88.00 ±102% +153.7% 223.25 numa-vmstat.node0.nr_mlock 14626 ± 2% +204.2% 44487 numa-vmstat.node0.nr_slab_reclaimable 13605 ± 4% +58.7% 21593 ± 5% numa-vmstat.node0.nr_slab_unreclaimable 146.25 ± 20% -69.6% 44.50 ± 24% numa-vmstat.node0.nr_writeback 745730 ± 13% +258.4% 2672434 ± 6% numa-vmstat.node0.nr_written 20184 ± 5% +184.6% 57444 ± 3% numa-vmstat.node0.nr_zone_active_file 8159 ± 8% +188.9% 23570 ± 5% numa-vmstat.node0.nr_zone_inactive_file 310.50 ± 8% -23.5% 237.50 ± 6% numa-vmstat.node0.nr_zone_write_pending 658998 ± 4% +145.9% 1620694 ± 3% numa-vmstat.node0.numa_hit 649527 ± 5% +147.9% 1610414 ± 3% numa-vmstat.node0.numa_local 144908 ± 60% +212.9% 453395 ± 42% numa-vmstat.node1.nr_dirtied 8149 ± 4% +40.9% 11482 ± 3% numa-vmstat.node1.nr_slab_reclaimable 142126 ± 61% +218.6% 452773 ± 43% numa-vmstat.node1.nr_written 574963 ± 5% +30.1% 747952 ± 9% numa-vmstat.node1.numa_hit 413376 ± 8% +42.2% 587779 ± 11% numa-vmstat.node1.numa_local 62731 +1.5% 63696 proc-vmstat.nr_active_anon 21848 +179.7% 61105 proc-vmstat.nr_active_file 62693 +1.2% 63433 proc-vmstat.nr_anon_pages 76.75 +31.9% 101.25 proc-vmstat.nr_anon_transparent_hugepages 1767315 ± 3% +253.1% 6241266 proc-vmstat.nr_dirtied 189.50 ± 2% +18.5% 224.50 ± 2% proc-vmstat.nr_dirty 311881 +17.8% 367315 proc-vmstat.nr_file_pages 4379 +34.6% 5895 proc-vmstat.nr_inactive_anon 8891 +176.9% 24624 proc-vmstat.nr_inactive_file 12107 -3.3% 11705 proc-vmstat.nr_kernel_stack 174.75 ±100% +139.3% 418.25 ± 13% proc-vmstat.nr_mlock 1122 -2.4% 1095 proc-vmstat.nr_page_table_pages 4461 +39.9% 6241 proc-vmstat.nr_shmem 22819 +145.3% 55981 proc-vmstat.nr_slab_reclaimable 23153 +37.9% 31928 proc-vmstat.nr_slab_unreclaimable 174.75 ± 14% -67.1% 57.50 ± 8% proc-vmstat.nr_writeback 1739838 ± 3% +257.2% 6215147 proc-vmstat.nr_written 62731 +1.5% 63696 proc-vmstat.nr_zone_active_anon 21848 +179.7% 61105 proc-vmstat.nr_zone_active_file 4379 +34.6% 5895 proc-vmstat.nr_zone_inactive_anon 8891 +176.9% 24624 proc-vmstat.nr_zone_inactive_file 345.25 ± 7% -27.7% 249.75 ± 2% proc-vmstat.nr_zone_write_pending 3707 ± 82% -66.0% 1262 ±139% proc-vmstat.numa_hint_faults_local 897890 +246.4% 3110289 proc-vmstat.numa_hit 881981 +250.8% 3094390 proc-vmstat.numa_local 142288 +142.1% 344443 ± 2% proc-vmstat.pgactivate 1074500 +235.2% 3601375 proc-vmstat.pgalloc_normal 3013 ± 2% +100.8% 6052 ± 2% proc-vmstat.pgdeactivate 658693 ± 2% +288.6% 2559777 proc-vmstat.pgfault 1069950 +235.7% 3591932 proc-vmstat.pgfree 21175 ± 11% +25.8% 26646 ± 7% proc-vmstat.pgpgin 9508094 ± 3% +187.4% 27321590 proc-vmstat.pgpgout 30091 +1.9% 30666 proc-vmstat.pgrotated 439624 +285.0% 1692566 proc-vmstat.slabs_scanned 446.93 ± 24% +55.0% 692.87 ± 10% sched_debug.cfs_rq:/.load_avg.avg 149.94 ± 70% +139.6% 359.32 ± 21% sched_debug.cfs_rq:/.load_avg.min 58835 ± 13% +27.5% 75024 ± 10% sched_debug.cfs_rq:/.min_vruntime.avg 41480 ± 26% +56.1% 64749 ± 14% sched_debug.cfs_rq:/.min_vruntime.min 0.39 ± 16% -22.9% 0.30 ± 3% sched_debug.cfs_rq:/.nr_running.stddev 869.56 ± 10% -27.8% 627.99 ± 5% sched_debug.cfs_rq:/.util_avg.max 69.46 ±117% -91.1% 6.17 ± 26% sched_debug.cfs_rq:/.util_est_enqueued.avg 255.56 ± 26% -76.6% 59.80 ± 38% sched_debug.cfs_rq:/.util_est_enqueued.max 57.09 ± 20% -73.4% 15.16 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.stddev 123588 +346.0% 551256 ± 2% sched_debug.cpu.clock.avg 123589 +346.0% 551257 ± 2% sched_debug.cpu.clock.max 123587 +346.0% 551256 ± 2% sched_debug.cpu.clock.min 0.61 ± 11% -47.4% 0.32 ± 11% sched_debug.cpu.clock.stddev 123588 +346.0% 551256 ± 2% sched_debug.cpu.clock_task.avg 123589 +346.0% 551257 ± 2% sched_debug.cpu.clock_task.max 123587 +346.0% 551256 ± 2% sched_debug.cpu.clock_task.min 0.61 ± 11% -47.4% 0.32 ± 11% sched_debug.cpu.clock_task.stddev 135.94 ± 19% -45.5% 74.09 ± 35% sched_debug.cpu.cpu_load[4].max 1861 ± 22% +331.6% 8035 ± 8% sched_debug.cpu.curr->pid.avg 4221 +251.0% 14814 ± 2% sched_debug.cpu.curr->pid.max 844.69 ±100% +502.1% 5085 ± 12% sched_debug.cpu.curr->pid.min 1327 ± 30% +195.7% 3924 ± 5% sched_debug.cpu.curr->pid.stddev 96381 +443.8% 524082 ± 2% sched_debug.cpu.nr_load_updates.avg 99314 +430.8% 527202 ± 2% sched_debug.cpu.nr_load_updates.max 95092 +449.7% 522696 ± 2% sched_debug.cpu.nr_load_updates.min 0.39 ± 16% -22.2% 0.30 ± 3% sched_debug.cpu.nr_running.stddev 239136 ± 13% +185.0% 681612 ± 3% sched_debug.cpu.nr_switches.avg 255369 ± 11% +179.1% 712761 ± 3% sched_debug.cpu.nr_switches.max 225112 ± 15% +189.1% 650706 ± 4% sched_debug.cpu.nr_switches.min 10213 ± 26% +132.5% 23748 ± 29% sched_debug.cpu.nr_switches.stddev -20.55 -959.0% 176.51 ± 27% sched_debug.cpu.nr_uninterruptible.avg 58.31 ± 23% +751.7% 496.62 ± 8% sched_debug.cpu.nr_uninterruptible.max -146.56 +106.4% -302.54 sched_debug.cpu.nr_uninterruptible.min 75.90 ± 44% +341.5% 335.12 ± 10% sched_debug.cpu.nr_uninterruptible.stddev 123587 +346.0% 551256 ± 2% sched_debug.cpu_clk 120708 +354.3% 548379 ± 2% sched_debug.ktime 124070 +344.7% 551740 ± 2% sched_debug.sched_clk 78299 +124.6% 175827 slabinfo.Acpi-Parse.active_objs 1085 +122.4% 2413 slabinfo.Acpi-Parse.active_slabs 79233 +122.4% 176218 slabinfo.Acpi-Parse.num_objs 1085 +122.4% 2413 slabinfo.Acpi-Parse.num_slabs 3271 ± 4% +32.7% 4339 ± 3% slabinfo.Acpi-State.active_objs 3610 ± 4% +27.9% 4619 ± 3% slabinfo.Acpi-State.num_objs 9125 -11.5% 8073 ± 6% slabinfo.anon_vma_chain.num_objs 1000 ± 2% -11.3% 886.75 slabinfo.avc_xperms_data.active_objs 403.50 +81.2% 731.25 ± 4% slabinfo.blkdev_ioc.active_objs 656.50 ± 3% +52.8% 1003 ± 4% slabinfo.blkdev_ioc.num_objs 35676 +276.2% 134224 slabinfo.btrfs_delayed_node.active_objs 690.25 +274.4% 2584 slabinfo.btrfs_delayed_node.active_slabs 35920 +274.2% 134423 slabinfo.btrfs_delayed_node.num_objs 690.25 +274.4% 2584 slabinfo.btrfs_delayed_node.num_slabs 1912 ± 4% +127.6% 4352 slabinfo.btrfs_extent_buffer.active_objs 2176 ± 3% +116.0% 4700 slabinfo.btrfs_extent_buffer.num_objs 7889 ± 2% +39.5% 11008 slabinfo.btrfs_extent_map.active_objs 155.00 ± 2% +76.5% 273.50 slabinfo.btrfs_extent_map.active_slabs 8701 ± 2% +76.4% 15347 slabinfo.btrfs_extent_map.num_objs 155.00 ± 2% +76.5% 273.50 slabinfo.btrfs_extent_map.num_slabs 34867 +284.7% 134127 slabinfo.btrfs_inode.active_objs 1251 +283.1% 4794 slabinfo.btrfs_inode.active_slabs 35056 +283.0% 134254 slabinfo.btrfs_inode.num_objs 1251 +283.1% 4794 slabinfo.btrfs_inode.num_slabs 2732 +23.4% 3371 ± 4% slabinfo.btrfs_path.active_objs 2893 +22.6% 3546 ± 4% slabinfo.btrfs_path.num_objs 79973 +121.5% 177127 slabinfo.dentry.active_objs 1920 +119.9% 4223 slabinfo.dentry.active_slabs 80675 +119.9% 177383 slabinfo.dentry.num_objs 1920 +119.9% 4223 slabinfo.dentry.num_slabs 1401 ± 5% -11.0% 1247 ± 5% slabinfo.eventpoll_pwq.num_objs 2269 ± 3% +74.7% 3965 ± 3% slabinfo.kmalloc-128.active_objs 4080 +24.4% 5074 slabinfo.kmalloc-128.num_objs 2679 ± 7% -44.5% 1486 ± 3% slabinfo.kmalloc-192.active_objs 3481 ± 5% -21.3% 2739 slabinfo.kmalloc-192.num_objs 898.50 ± 7% -12.0% 790.75 ± 3% slabinfo.kmalloc-rcl-64.active_objs 4372 -11.3% 3880 slabinfo.lsm_file_cache.active_objs 91.75 ± 29% +115.8% 198.00 ± 7% slabinfo.nfs_commit_data.active_objs 187.00 ± 28% +76.9% 330.75 ± 5% slabinfo.nfs_commit_data.num_objs 84.00 ± 9% +103.6% 171.00 ± 11% slabinfo.nfs_read_data.active_objs 151.50 ± 8% +121.3% 335.25 ± 13% slabinfo.nfs_read_data.num_objs 908.75 ± 2% -19.8% 729.00 slabinfo.pde_opener.active_objs 1145 -10.9% 1020 slabinfo.pid.active_objs 13795 +17.7% 16232 slabinfo.radix_tree_node.active_objs 14780 +14.7% 16948 slabinfo.radix_tree_node.num_objs 892.00 ± 9% -22.6% 690.75 ± 10% slabinfo.skbuff_head_cache.active_objs 1529 ± 7% -11.8% 1348 ± 6% slabinfo.skbuff_head_cache.num_objs 6474 ± 4% -12.3% 5678 ± 3% slabinfo.vm_area_struct.active_objs 7214 ± 5% -12.2% 6335 ± 3% slabinfo.vm_area_struct.num_objs 1669 ± 7% -13.3% 1448 ± 6% slabinfo.xfrm_dst_cache.active_objs 33612 ± 5% +341.3% 148332 softirqs.BLOCK 3556 +682.7% 27837 softirqs.CPU0.BLOCK 39960 ± 5% +318.1% 167062 softirqs.CPU0.RCU 18883 ± 2% +243.1% 64797 ± 2% softirqs.CPU0.SCHED 2563 +785.9% 22706 softirqs.CPU0.TASKLET 104220 ± 9% +403.9% 525161 ± 4% softirqs.CPU0.TIMER 14807 ± 5% +455.0% 82179 ± 2% softirqs.CPU1.BLOCK 30579 ± 7% +326.8% 130524 ± 10% softirqs.CPU1.RCU 14199 ± 3% +330.5% 61134 ± 3% softirqs.CPU1.SCHED 17373 ± 4% +462.2% 97667 softirqs.CPU1.TASKLET 74093 ± 12% +308.1% 302342 ± 17% softirqs.CPU1.TIMER 12767 ± 12% +335.4% 55586 ± 22% softirqs.CPU16.RCU 6385 ± 3% +250.6% 22390 ± 3% softirqs.CPU16.SCHED 35098 ± 12% +236.1% 117949 ± 18% softirqs.CPU16.TIMER 13243 ± 8% +326.8% 56520 ± 18% softirqs.CPU17.RCU 6331 ± 2% +250.4% 22186 ± 2% softirqs.CPU17.SCHED 34984 ± 11% +238.0% 118241 ± 18% softirqs.CPU17.TIMER 13151 ± 8% +313.0% 54310 ± 12% softirqs.CPU18.RCU 6560 ± 3% +239.5% 22271 ± 2% softirqs.CPU18.SCHED 34836 ± 10% +235.0% 116695 ± 18% softirqs.CPU18.TIMER 13919 ± 14% +286.7% 53831 ± 15% softirqs.CPU19.RCU 6518 ± 3% +244.8% 22476 ± 2% softirqs.CPU19.SCHED 34164 ± 10% +230.8% 113004 ± 14% softirqs.CPU19.TIMER 23237 ± 5% +371.3% 109508 ± 13% softirqs.CPU2.RCU 8367 ± 3% +313.9% 34630 softirqs.CPU2.SCHED 60014 ± 15% +312.2% 247368 ± 18% softirqs.CPU2.TIMER 8210 ± 22% +263.8% 29871 ± 19% softirqs.CPU20.RCU 19520 ± 10% +197.9% 58150 ± 17% softirqs.CPU20.TIMER 7227 ± 15% +292.1% 28340 ± 17% softirqs.CPU21.RCU 19563 ± 13% +176.8% 54148 ± 9% softirqs.CPU21.TIMER 6738 ± 8% +312.1% 27771 ± 16% softirqs.CPU22.RCU 19856 ± 12% +193.2% 58227 ± 17% softirqs.CPU22.TIMER 7068 ± 12% +295.8% 27976 ± 21% softirqs.CPU23.RCU 20532 ± 13% +206.9% 63019 ± 26% softirqs.CPU23.TIMER 24162 ± 12% +376.9% 115228 ± 13% softirqs.CPU3.RCU 8685 ± 4% +304.9% 35167 softirqs.CPU3.SCHED 70700 ± 16% +252.2% 249000 ± 17% softirqs.CPU3.TIMER 18076 ± 8% +360.9% 83322 ± 12% softirqs.CPU4.RCU 6638 ± 3% +259.3% 23854 ± 2% softirqs.CPU4.SCHED 52818 ± 16% +254.0% 186954 ± 22% softirqs.CPU4.TIMER 19156 ± 6% +349.5% 86102 ± 13% softirqs.CPU5.RCU 6655 ± 4% +258.2% 23839 ± 2% softirqs.CPU5.SCHED 53180 ± 15% +253.1% 187754 ± 22% softirqs.CPU5.TIMER 18758 ± 4% +376.4% 89360 ± 15% softirqs.CPU6.RCU 6639 ± 5% +258.1% 23772 softirqs.CPU6.SCHED 52940 ± 16% +256.6% 188770 ± 23% softirqs.CPU6.TIMER 18941 ± 2% +327.8% 81024 ± 6% softirqs.CPU7.RCU 6768 ± 3% +250.0% 23685 ± 2% softirqs.CPU7.SCHED 54451 ± 16% +245.9% 188338 ± 23% softirqs.CPU7.TIMER 324173 ± 6% +286.1% 1251558 ± 11% softirqs.RCU 171016 +174.8% 469998 softirqs.SCHED 20655 ± 4% +486.3% 121103 softirqs.TASKLET 954209 ± 9% +212.1% 2978155 ± 13% softirqs.TIMER 0.10 ± 44% -87.6% 0.01 ± 26% perf-stat.i.MPKI 11917649 ± 28% -79.1% 2486349 ± 19% perf-stat.i.branch-instructions 0.07 ± 8% -0.1 0.01 ± 3% perf-stat.i.branch-miss-rate% 889209 ± 27% -80.1% 176655 ± 21% perf-stat.i.branch-misses 0.30 ± 5% -0.2 0.06 ± 3% perf-stat.i.cache-miss-rate% 124863 ± 8% -83.0% 21240 ± 4% perf-stat.i.cache-misses 394540 ± 11% -82.9% 67377 ± 3% perf-stat.i.cache-references 18966 ± 2% -60.0% 7583 perf-stat.i.context-switches 0.02 ± 31% -84.8% 0.00 ± 16% perf-stat.i.cpi 81719357 ± 15% -80.9% 15619893 ± 12% perf-stat.i.cpu-cycles 296.41 -56.4% 129.34 ± 2% perf-stat.i.cpu-migrations 6.07 ± 12% -77.5% 1.37 ± 10% perf-stat.i.cycles-between-cache-misses 0.00 ± 32% -0.0 0.00 ± 25% perf-stat.i.dTLB-load-miss-rate% 26515 ± 18% -78.6% 5678 ± 19% perf-stat.i.dTLB-load-misses 11211406 ± 28% -78.8% 2378574 ± 19% perf-stat.i.dTLB-loads 0.00 ± 17% -0.0 0.00 ± 14% perf-stat.i.dTLB-store-miss-rate% 4617 ± 17% -81.9% 836.25 ± 6% perf-stat.i.dTLB-store-misses 6514374 ± 21% -80.4% 1276707 ± 13% perf-stat.i.dTLB-stores 0.81 ± 3% -0.7 0.15 ± 2% perf-stat.i.iTLB-load-miss-rate% 7806 ± 14% -78.8% 1657 ± 11% perf-stat.i.iTLB-load-misses 1566 ± 22% -75.4% 384.63 ± 5% perf-stat.i.iTLB-loads 56576304 ± 28% -79.1% 11809110 ± 19% perf-stat.i.instructions 63.87 ± 18% -79.4% 13.18 ± 17% perf-stat.i.instructions-per-iTLB-miss 0.01 ± 17% -77.3% 0.00 ± 8% perf-stat.i.ipc 2912 -20.4% 2317 perf-stat.i.minor-faults 16709 ± 2% -16.6% 13928 perf-stat.i.msec 0.24 ± 7% -0.2 0.04 ± 16% perf-stat.i.node-load-miss-rate% 6615 ± 76% -85.3% 969.37 ± 38% perf-stat.i.node-load-misses 21880 ± 19% -79.9% 4403 ± 16% perf-stat.i.node-loads 0.18 ± 26% -0.2 0.03 ± 14% perf-stat.i.node-store-miss-rate% 6679 ± 17% -85.1% 997.76 ± 12% perf-stat.i.node-store-misses 31549 ± 6% -81.1% 5950 ± 14% perf-stat.i.node-stores 2912 -20.4% 2317 perf-stat.i.page-faults 7.48 ± 4% -0.4 7.09 ± 2% perf-stat.overall.branch-miss-rate% 11892777 ± 28% -79.0% 2493017 ± 19% perf-stat.ps.branch-instructions 887326 ± 27% -80.0% 177145 ± 21% perf-stat.ps.branch-misses 124722 ± 8% -82.9% 21307 ± 4% perf-stat.ps.cache-misses 394052 ± 11% -82.8% 67593 ± 3% perf-stat.ps.cache-references 18874 ± 2% -59.9% 7576 perf-stat.ps.context-switches 81586204 ± 15% -80.8% 15666785 ± 12% perf-stat.ps.cpu-cycles 294.95 -56.2% 129.22 ± 2% perf-stat.ps.cpu-migrations 26472 ± 18% -78.5% 5693 ± 19% perf-stat.ps.dTLB-load-misses 11187646 ± 28% -78.7% 2384826 ± 19% perf-stat.ps.dTLB-loads 4613 ± 17% -81.8% 838.99 ± 5% perf-stat.ps.dTLB-store-misses 6503546 ± 21% -80.3% 1280306 ± 13% perf-stat.ps.dTLB-stores 7794 ± 14% -78.7% 1661 ± 11% perf-stat.ps.iTLB-load-misses 1562 ± 22% -75.3% 385.47 ± 5% perf-stat.ps.iTLB-loads 56457333 ± 28% -79.0% 11840667 ± 19% perf-stat.ps.instructions 2899 -20.2% 2315 perf-stat.ps.minor-faults 16645 ± 2% -16.4% 13919 perf-stat.ps.msec 6599 ± 76% -85.3% 971.51 ± 38% perf-stat.ps.node-load-misses 21822 ± 19% -79.8% 4410 ± 16% perf-stat.ps.node-loads 6669 ± 17% -85.0% 1000 ± 12% perf-stat.ps.node-store-misses 31502 ± 6% -81.1% 5966 ± 14% perf-stat.ps.node-stores 2899 -20.2% 2315 perf-stat.ps.page-faults 244.50 ± 60% +308.3% 998.25 ± 27% interrupts.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 234.00 ± 68% +242.7% 802.00 ± 32% interrupts.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 219.25 ± 56% +214.8% 690.25 ± 33% interrupts.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 165.00 ± 31% +233.3% 550.00 ± 2% interrupts.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 133.00 ± 30% +464.8% 751.25 ± 30% interrupts.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 197.50 ± 73% +181.9% 556.75 ± 2% interrupts.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 119.25 ± 10% +365.0% 554.50 ± 3% interrupts.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 162.75 ± 29% +430.0% 862.50 ± 36% interrupts.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 29671 ± 57% +711.6% 240799 interrupts.75:IR-PCI-MSI.2097152-edge.isci-msix 108741 ± 2% +69.5% 184290 ± 4% interrupts.CAL:Function_call_interrupts 183.00 ± 84% +312.7% 755.25 ± 29% interrupts.CPU0.36:IR-PCI-MSI.2621442-edge.eth0-TxRx-1 27.25 ± 38% +280.7% 103.75 ± 3% interrupts.CPU0.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 19.00 ± 39% +668.4% 146.00 ± 42% interrupts.CPU0.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 75.50 ± 90% +165.9% 200.75 ± 4% interrupts.CPU0.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 18.00 ± 26% +480.6% 104.50 ± 3% interrupts.CPU0.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 29.50 ± 44% +501.7% 177.50 ± 49% interrupts.CPU0.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 3760 ± 57% +1105.1% 45313 interrupts.CPU0.75:IR-PCI-MSI.2097152-edge.isci-msix 3356 ± 10% +178.2% 9336 ± 16% interrupts.CPU0.CAL:Function_call_interrupts 424176 ± 2% +384.8% 2056303 ± 7% interrupts.CPU0.LOC:Local_timer_interrupts 32371 ± 12% +34.8% 43643 ± 8% interrupts.CPU0.RES:Rescheduling_interrupts 176.25 ± 31% +234.9% 590.25 ± 38% interrupts.CPU0.TLB:TLB_shootdowns 35.75 ± 58% +772.0% 311.75 ± 8% interrupts.CPU1.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 130.00 ± 26% +306.0% 527.75 ± 26% interrupts.CPU1.37:IR-PCI-MSI.2621443-edge.eth0-TxRx-2 22.50 ± 26% +325.6% 95.75 ± 4% interrupts.CPU1.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 41.25 ± 26% +437.6% 221.75 ± 18% interrupts.CPU1.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 98.75 ± 8% +353.4% 447.75 ± 3% interrupts.CPU1.41:IR-PCI-MSI.2621447-edge.eth0-TxRx-6 24.00 ± 26% +427.1% 126.50 ± 16% interrupts.CPU1.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 25787 ± 58% +657.4% 195323 interrupts.CPU1.75:IR-PCI-MSI.2097152-edge.isci-msix 3547 ± 15% +141.9% 8581 ± 14% interrupts.CPU1.CAL:Function_call_interrupts 366025 ± 3% +356.0% 1669062 ± 6% interrupts.CPU1.LOC:Local_timer_interrupts 28035 ± 18% +58.6% 44474 ± 14% interrupts.CPU1.RES:Rescheduling_interrupts 92.25 ± 18% +208.4% 284.50 ± 16% interrupts.CPU1.TLB:TLB_shootdowns 2.75 ± 30% +3809.1% 107.50 ±161% interrupts.CPU12.RES:Rescheduling_interrupts 6458 ± 7% +104.2% 13190 ± 8% interrupts.CPU16.CAL:Function_call_interrupts 161518 ± 5% +270.6% 598601 ± 17% interrupts.CPU16.LOC:Local_timer_interrupts 6882 ± 9% +91.8% 13202 ± 7% interrupts.CPU17.CAL:Function_call_interrupts 162111 ± 5% +269.3% 598629 ± 17% interrupts.CPU17.LOC:Local_timer_interrupts 6071 ± 9% +115.1% 13057 ± 12% interrupts.CPU18.CAL:Function_call_interrupts 162778 ± 5% +267.4% 598039 ± 17% interrupts.CPU18.LOC:Local_timer_interrupts 8133 ± 7% +61.6% 13146 ± 4% interrupts.CPU19.CAL:Function_call_interrupts 162807 ± 5% +276.2% 612554 ± 13% interrupts.CPU19.LOC:Local_timer_interrupts 97.00 ± 22% +454.1% 537.50 ± 35% interrupts.CPU2.42:IR-PCI-MSI.2621448-edge.eth0-TxRx-7 3447 ± 13% +152.5% 8704 ± 18% interrupts.CPU2.CAL:Function_call_interrupts 299158 ± 4% +329.7% 1285348 ± 9% interrupts.CPU2.LOC:Local_timer_interrupts 5497 ± 12% +39.9% 7691 ± 5% interrupts.CPU2.RES:Rescheduling_interrupts 40.25 ± 19% +106.8% 83.25 ± 39% interrupts.CPU2.TLB:TLB_shootdowns 4339 ± 16% +44.2% 6256 ± 5% interrupts.CPU20.CAL:Function_call_interrupts 82165 ± 11% +251.6% 288905 ± 16% interrupts.CPU20.LOC:Local_timer_interrupts 3425 ± 14% +118.5% 7485 ± 5% interrupts.CPU21.CAL:Function_call_interrupts 82124 ± 12% +266.4% 300890 ± 8% interrupts.CPU21.LOC:Local_timer_interrupts 3889 ± 15% +80.2% 7009 ± 12% interrupts.CPU22.CAL:Function_call_interrupts 82060 ± 12% +252.2% 289035 ± 16% interrupts.CPU22.LOC:Local_timer_interrupts 3156 ± 10% +117.7% 6870 ± 14% interrupts.CPU23.CAL:Function_call_interrupts 82305 ± 12% +264.2% 299736 ± 9% interrupts.CPU23.LOC:Local_timer_interrupts 102.00 ± 24% +241.7% 348.50 ± 2% interrupts.CPU3.38:IR-PCI-MSI.2621444-edge.eth0-TxRx-3 3307 ± 2% +138.3% 7882 ± 13% interrupts.CPU3.CAL:Function_call_interrupts 299207 ± 4% +327.1% 1277903 ± 10% interrupts.CPU3.LOC:Local_timer_interrupts 4325 ± 6% +41.9% 6138 ± 10% interrupts.CPU3.RES:Rescheduling_interrupts 67.00 ± 26% +455.2% 372.00 ± 33% interrupts.CPU4.39:IR-PCI-MSI.2621445-edge.eth0-TxRx-4 2950 ± 4% +145.3% 7239 ± 20% interrupts.CPU4.CAL:Function_call_interrupts 234082 ± 4% +300.8% 938197 ± 13% interrupts.CPU4.LOC:Local_timer_interrupts 2753 ± 11% +38.7% 3818 ± 14% interrupts.CPU4.RES:Rescheduling_interrupts 3.75 ± 54% +1540.0% 61.50 ±143% interrupts.CPU40.RES:Rescheduling_interrupts 3093 ± 5% +137.8% 7354 ± 17% interrupts.CPU5.CAL:Function_call_interrupts 233637 ± 4% +300.4% 935496 ± 14% interrupts.CPU5.LOC:Local_timer_interrupts 2616 ± 9% +39.5% 3649 ± 18% interrupts.CPU5.RES:Rescheduling_interrupts 3110 ± 5% +122.7% 6927 ± 11% interrupts.CPU6.CAL:Function_call_interrupts 233360 ± 4% +300.6% 934844 ± 14% interrupts.CPU6.LOC:Local_timer_interrupts 624.00 ± 17% -53.3% 291.50 ± 35% interrupts.CPU6.NMI:Non-maskable_interrupts 624.00 ± 17% -53.3% 291.50 ± 35% interrupts.CPU6.PMI:Performance_monitoring_interrupts 65.00 ± 57% +596.9% 453.00 ± 26% interrupts.CPU7.35:IR-PCI-MSI.2621441-edge.eth0-TxRx-0 75.75 ± 38% +249.8% 265.00 ± 2% interrupts.CPU7.40:IR-PCI-MSI.2621446-edge.eth0-TxRx-5 2918 +129.8% 6706 ± 18% interrupts.CPU7.CAL:Function_call_interrupts 233720 ± 4% +305.4% 947415 ± 11% interrupts.CPU7.LOC:Local_timer_interrupts 3522671 ± 4% +293.1% 13846285 ± 11% interrupts.LOC:Local_timer_interrupts 94.00 +18.1% 111.00 interrupts.MCP:Machine_check_polls 89232 ± 4% +39.8% 124725 ± 6% interrupts.RES:Rescheduling_interrupts 495.75 ± 13% +164.4% 1310 ± 36% interrupts.TLB:TLB_shootdowns 15.33 ± 17% -9.6 5.77 ± 14% perf-profile.calltrace.cycles-pp.ret_from_fork 15.33 ± 17% -9.6 5.77 ± 14% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 9.42 ± 68% -9.4 0.00 perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.50 ± 55% -9.1 1.43 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 10.49 ± 55% -9.1 1.42 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.36 ± 22% -6.4 3.93 ± 16% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 10.12 ± 21% -6.3 3.82 ± 17% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 4.03 ± 56% -2.9 1.15 ± 11% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork 4.35 ± 25% -2.8 1.53 ± 15% perf-profile.calltrace.cycles-pp.kthread_worker_fn.kthread.ret_from_fork 4.30 ± 25% -2.8 1.51 ± 15% perf-profile.calltrace.cycles-pp.loop_queue_work.kthread_worker_fn.kthread.ret_from_fork 3.93 ± 54% -2.8 1.14 ± 11% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread 3.61 ± 28% -2.6 1.03 ± 8% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread 3.62 ± 28% -2.6 1.03 ± 8% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork 2.64 ± 35% -2.1 0.50 ± 59% perf-profile.calltrace.cycles-pp.__btrfs_drop_extents.insert_reserved_file_extent.btrfs_finish_ordered_io.normal_work_helper.process_one_work 2.82 ± 34% -2.1 0.70 ± 9% perf-profile.calltrace.cycles-pp.insert_reserved_file_extent.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread 6.26 ± 12% -2.0 4.21 ± 36% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64 6.25 ± 12% -2.0 4.21 ± 36% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64 6.22 ± 12% -2.0 4.20 ± 36% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 2.28 ± 37% -2.0 0.31 ±100% perf-profile.calltrace.cycles-pp.lo_write_bvec.loop_queue_work.kthread_worker_fn.kthread.ret_from_fork 2.24 ± 37% -1.9 0.31 ±100% perf-profile.calltrace.cycles-pp.do_iter_write.lo_write_bvec.loop_queue_work.kthread_worker_fn.kthread 2.19 ± 39% -1.9 0.30 ±100% perf-profile.calltrace.cycles-pp.do_iter_readv_writev.do_iter_write.lo_write_bvec.loop_queue_work.kthread_worker_fn 2.19 ± 39% -1.9 0.30 ±100% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.do_iter_readv_writev.do_iter_write.lo_write_bvec.loop_queue_work 2.09 ± 38% -1.8 0.28 ±100% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.do_iter_readv_writev.do_iter_write.lo_write_bvec 1.98 ± 20% -1.0 0.97 ± 11% perf-profile.calltrace.cycles-pp.btrfs_sync_file.loop_queue_work.kthread_worker_fn.kthread.ret_from_fork 1.01 ± 23% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread 1.01 ± 23% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.calltrace.cycles-pp.fb_flashcursor.process_one_work.worker_thread.kthread.ret_from_fork 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.calltrace.cycles-pp.bit_cursor.fb_flashcursor.process_one_work.worker_thread.kthread 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.calltrace.cycles-pp.soft_cursor.bit_cursor.fb_flashcursor.process_one_work.worker_thread 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.calltrace.cycles-pp.mga_dirty_update.soft_cursor.bit_cursor.fb_flashcursor.process_one_work 2.12 ± 22% -0.7 1.42 ± 34% perf-profile.calltrace.cycles-pp.memcpy_toio.mga_dirty_update.soft_cursor.bit_cursor.fb_flashcursor 0.79 ± 37% -0.5 0.27 ±100% perf-profile.calltrace.cycles-pp.irq_exit.do_IRQ.ret_from_intr.cpuidle_enter_state.do_idle 0.13 ±173% +0.5 0.66 ± 7% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt 0.14 ±173% +0.5 0.69 ± 10% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +0.6 0.61 ± 8% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.53 ± 62% +0.6 1.16 ± 17% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.00 +0.7 0.67 ± 19% perf-profile.calltrace.cycles-pp.run_timer_softirq.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.15 ±173% +0.7 0.83 ± 8% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 0.15 ±173% +0.9 1.10 ± 7% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.16 ±173% +1.0 1.12 ± 6% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.92 ± 12% +1.3 2.21 ± 22% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.48 ±105% +1.6 2.12 ± 23% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 1.89 ± 34% +2.1 3.99 ± 21% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 2.10 ± 30% +2.4 4.52 ± 22% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 2.53 ± 31% +3.3 5.81 ± 26% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 4.15 ± 27% +4.3 8.43 ± 16% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 5.91 ± 29% +6.4 12.27 ± 14% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 10.84 ± 28% +9.0 19.87 ± 12% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 10.61 ± 32% +10.2 20.81 ± 10% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 34.32 ± 28% +15.4 49.73 ± 7% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 49.61 ± 29% +25.3 74.93 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 64.73 ± 24% +25.5 90.24 perf-profile.calltrace.cycles-pp.secondary_startup_64 58.35 ± 27% +27.5 85.86 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 58.47 ± 27% +27.6 86.03 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 58.47 ± 27% +27.6 86.03 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 16.37 ± 80% -14.0 2.37 ± 10% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 16.34 ± 80% -14.0 2.36 ± 9% perf-profile.children.cycles-pp.do_syscall_64 15.33 ± 17% -9.6 5.77 ± 14% perf-profile.children.cycles-pp.kthread 15.34 ± 17% -9.6 5.79 ± 14% perf-profile.children.cycles-pp.ret_from_fork 9.43 ± 68% -9.4 0.00 perf-profile.children.cycles-pp.do_unlinkat 10.36 ± 22% -6.4 3.93 ± 16% perf-profile.children.cycles-pp.worker_thread 10.12 ± 21% -6.3 3.82 ± 17% perf-profile.children.cycles-pp.process_one_work 7.51 ± 42% -6.0 1.49 ± 8% perf-profile.children.cycles-pp.btrfs_search_slot 4.13 ± 53% -3.0 1.14 ± 11% perf-profile.children.cycles-pp.flush_space 3.18 ±122% -2.9 0.29 ± 34% perf-profile.children.cycles-pp.down_write 4.03 ± 56% -2.9 1.15 ± 11% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space 3.13 ±122% -2.9 0.28 ± 31% perf-profile.children.cycles-pp.call_rwsem_down_write_failed 3.13 ±122% -2.9 0.28 ± 31% perf-profile.children.cycles-pp.rwsem_down_write_failed 4.35 ± 25% -2.8 1.53 ± 15% perf-profile.children.cycles-pp.kthread_worker_fn 4.30 ± 25% -2.8 1.51 ± 15% perf-profile.children.cycles-pp.loop_queue_work 3.62 ± 28% -2.6 1.03 ± 8% perf-profile.children.cycles-pp.normal_work_helper 3.61 ± 28% -2.6 1.03 ± 8% perf-profile.children.cycles-pp.btrfs_finish_ordered_io 2.92 ± 34% -2.2 0.76 ± 13% perf-profile.children.cycles-pp.__btrfs_drop_extents 2.37 ±118% -2.1 0.25 ± 31% perf-profile.children.cycles-pp.rwsem_spin_on_owner 2.82 ± 34% -2.1 0.70 ± 9% perf-profile.children.cycles-pp.insert_reserved_file_extent 2.26 ±102% -2.1 0.18 ± 14% perf-profile.children.cycles-pp.__btrfs_run_delayed_items 6.26 ± 12% -2.0 4.21 ± 36% perf-profile.children.cycles-pp.start_kernel 2.28 ± 37% -1.8 0.51 ± 22% perf-profile.children.cycles-pp.lo_write_bvec 2.24 ± 37% -1.7 0.50 ± 24% perf-profile.children.cycles-pp.do_iter_write 2.19 ± 39% -1.7 0.48 ± 26% perf-profile.children.cycles-pp.do_iter_readv_writev 2.19 ± 39% -1.7 0.48 ± 26% perf-profile.children.cycles-pp.btrfs_file_write_iter 2.10 ± 38% -1.6 0.46 ± 24% perf-profile.children.cycles-pp.btrfs_buffered_write 1.82 ± 28% -1.6 0.20 ± 21% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 1.97 ± 44% -1.6 0.37 ± 18% perf-profile.children.cycles-pp.btrfs_lookup_file_extent 1.64 ± 45% -1.6 0.06 ± 63% perf-profile.children.cycles-pp.btrfs_lookup_dir_item 2.22 ± 54% -1.5 0.68 ± 16% perf-profile.children.cycles-pp.__sched_text_start 1.70 ± 21% -1.5 0.17 ± 20% perf-profile.children.cycles-pp.btrfs_tree_read_lock 1.85 ± 52% -1.5 0.38 ± 30% perf-profile.children.cycles-pp.try_to_wake_up 1.61 ± 54% -1.4 0.21 ± 15% perf-profile.children.cycles-pp.btrfs_del_items 1.22 ± 85% -1.1 0.09 ± 21% perf-profile.children.cycles-pp.__btrfs_update_delayed_inode 1.81 ± 50% -1.1 0.72 ± 19% perf-profile.children.cycles-pp._raw_spin_lock 1.98 ± 20% -1.0 0.97 ± 11% perf-profile.children.cycles-pp.btrfs_sync_file 1.09 ± 60% -1.0 0.12 ± 54% perf-profile.children.cycles-pp.__wake_up_common 1.15 ± 38% -0.9 0.20 ± 63% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.05 ± 60% -0.9 0.12 ± 48% perf-profile.children.cycles-pp.autoremove_wake_function 1.48 ± 27% -0.9 0.59 ± 4% perf-profile.children.cycles-pp.do_writepages 0.99 ± 67% -0.9 0.10 ± 45% perf-profile.children.cycles-pp.__wake_up_common_lock 1.48 ± 28% -0.9 0.61 ± 3% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 1.23 ± 58% -0.8 0.41 ± 16% perf-profile.children.cycles-pp.schedule 0.84 ± 58% -0.8 0.03 ±105% perf-profile.children.cycles-pp.reserve_metadata_bytes 0.83 ± 88% -0.8 0.06 ± 58% perf-profile.children.cycles-pp.btrfs_release_path 1.06 ± 42% -0.8 0.28 ± 22% perf-profile.children.cycles-pp.setup_items_for_insert 1.01 ± 47% -0.8 0.26 ± 21% perf-profile.children.cycles-pp.schedule_idle 1.47 ± 29% -0.7 0.73 ± 10% perf-profile.children.cycles-pp.__btrfs_cow_block 0.87 ± 22% -0.7 0.13 ± 25% perf-profile.children.cycles-pp.prepare_to_wait_event 1.47 ± 29% -0.7 0.73 ± 10% perf-profile.children.cycles-pp.btrfs_cow_block 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.children.cycles-pp.fb_flashcursor 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.children.cycles-pp.bit_cursor 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.children.cycles-pp.soft_cursor 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.children.cycles-pp.mga_dirty_update 2.21 ± 23% -0.7 1.48 ± 33% perf-profile.children.cycles-pp.memcpy_toio 0.91 ± 77% -0.7 0.18 ± 20% perf-profile.children.cycles-pp.read_block_for_search 0.90 ± 50% -0.7 0.18 ± 24% perf-profile.children.cycles-pp.btrfs_get_token_32 0.82 ± 45% -0.7 0.11 ± 42% perf-profile.children.cycles-pp.btrfs_tree_lock 1.21 ± 7% -0.7 0.51 ± 17% perf-profile.children.cycles-pp.btrfs_run_delayed_refs 1.21 ± 7% -0.7 0.51 ± 17% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs 0.77 ± 50% -0.7 0.10 ± 37% perf-profile.children.cycles-pp.btrfs_lock_root_node 0.73 ± 74% -0.7 0.07 ± 23% perf-profile.children.cycles-pp.start_transaction 0.94 ± 34% -0.6 0.30 ± 21% perf-profile.children.cycles-pp.start_ordered_ops 0.91 ± 37% -0.6 0.28 ± 25% perf-profile.children.cycles-pp.extent_writepages 0.77 ± 51% -0.6 0.14 ± 53% perf-profile.children.cycles-pp.generic_bin_search 0.71 ± 74% -0.6 0.08 ± 23% perf-profile.children.cycles-pp.btrfs_lookup_inode 0.91 ± 37% -0.6 0.28 ± 23% perf-profile.children.cycles-pp.btrfs_fdatawrite_range 0.85 ± 33% -0.6 0.27 ± 23% perf-profile.children.cycles-pp.extent_write_cache_pages 0.69 ± 44% -0.6 0.12 ± 12% perf-profile.children.cycles-pp.memmove 0.74 ± 53% -0.6 0.18 ± 30% perf-profile.children.cycles-pp.ttwu_do_activate 0.86 ± 38% -0.5 0.34 ± 53% perf-profile.children.cycles-pp.blk_done_softirq 1.03 ± 30% -0.5 0.52 ± 15% perf-profile.children.cycles-pp.btrfs_commit_transaction 0.75 ± 33% -0.5 0.25 ± 29% perf-profile.children.cycles-pp.__extent_writepage 0.77 ± 40% -0.5 0.28 ± 47% perf-profile.children.cycles-pp.blk_update_request 0.60 ± 84% -0.5 0.12 ± 16% perf-profile.children.cycles-pp.find_extent_buffer 0.62 ± 41% -0.5 0.14 ± 28% perf-profile.children.cycles-pp.btrfs_copy_from_user 0.70 ± 39% -0.5 0.22 ± 51% perf-profile.children.cycles-pp.btrfs_end_bio 0.54 ± 39% -0.5 0.06 ± 20% perf-profile.children.cycles-pp.memmove_extent_buffer 0.60 ± 44% -0.5 0.13 ± 27% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 0.52 ±124% -0.5 0.05 ± 64% perf-profile.children.cycles-pp.write 0.58 ± 43% -0.5 0.12 ± 28% perf-profile.children.cycles-pp.memcpy_from_page 1.22 ± 21% -0.5 0.77 ± 17% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.58 ± 61% -0.4 0.14 ± 23% perf-profile.children.cycles-pp.btrfs_set_token_32 0.53 ± 38% -0.4 0.11 ± 74% perf-profile.children.cycles-pp.btrfs_dirty_pages 0.65 ± 45% -0.4 0.23 ± 64% perf-profile.children.cycles-pp.scsi_io_completion 0.54 ± 56% -0.4 0.13 ± 42% perf-profile.children.cycles-pp.kmem_cache_alloc 0.64 ± 43% -0.4 0.23 ± 64% perf-profile.children.cycles-pp.scsi_end_request 0.65 ± 52% -0.4 0.26 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair 0.45 ± 53% -0.4 0.09 ±106% perf-profile.children.cycles-pp.end_bio_extent_writepage 0.39 ± 75% -0.4 0.03 ±100% perf-profile.children.cycles-pp.btrfs_block_rsv_add 0.65 ± 30% -0.3 0.35 ± 21% perf-profile.children.cycles-pp.copy_page 0.37 ± 28% -0.3 0.08 ± 17% perf-profile.children.cycles-pp.__clear_extent_bit 0.33 ± 42% -0.3 0.05 ±102% perf-profile.children.cycles-pp.smpboot_thread_fn 0.59 ± 19% -0.3 0.32 ± 24% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction 0.32 ± 40% -0.3 0.05 ± 62% perf-profile.children.cycles-pp.map_private_extent_buffer 0.37 ± 20% -0.3 0.11 ± 28% perf-profile.children.cycles-pp.__extent_writepage_io 0.36 ± 31% -0.3 0.10 ± 42% perf-profile.children.cycles-pp.poll_idle 0.34 ± 46% -0.3 0.08 ± 45% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty 0.40 ± 18% -0.3 0.14 ± 36% perf-profile.children.cycles-pp.__libc_fork 0.57 ± 22% -0.3 0.31 ± 19% perf-profile.children.cycles-pp.btree_write_cache_pages 0.53 ± 16% -0.2 0.28 ± 17% perf-profile.children.cycles-pp.__btrfs_free_extent 0.41 ± 30% -0.2 0.16 ± 28% perf-profile.children.cycles-pp.__set_extent_bit 0.33 ± 32% -0.2 0.09 ± 60% perf-profile.children.cycles-pp.kmem_cache_free 0.32 ± 53% -0.2 0.08 ± 33% perf-profile.children.cycles-pp.memcpy_extent_buffer 0.32 ± 67% -0.2 0.09 ± 16% perf-profile.children.cycles-pp.link_path_walk 0.33 ± 37% -0.2 0.11 ± 28% perf-profile.children.cycles-pp.submit_extent_page 0.30 ± 28% -0.2 0.08 ± 28% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.45 ± 26% -0.2 0.24 ± 34% perf-profile.children.cycles-pp.btrfs_csum_file_blocks 0.50 ± 8% -0.2 0.28 ± 24% perf-profile.children.cycles-pp.btrfs_log_changed_extents 0.28 ± 27% -0.2 0.08 ± 24% perf-profile.children.cycles-pp.get_page_from_freelist 0.26 ± 43% -0.2 0.06 ± 71% perf-profile.children.cycles-pp.set_extent_buffer_dirty 0.23 ± 46% -0.2 0.03 ±100% perf-profile.children.cycles-pp.___might_sleep 0.23 ± 71% -0.2 0.03 ±100% perf-profile.children.cycles-pp.__radix_tree_lookup 0.58 ± 11% -0.2 0.38 ± 20% perf-profile.children.cycles-pp.btrfs_log_dentry_safe 0.58 ± 11% -0.2 0.38 ± 20% perf-profile.children.cycles-pp.btrfs_log_inode_parent 0.58 ± 12% -0.2 0.38 ± 19% perf-profile.children.cycles-pp.btrfs_log_inode 0.31 ± 46% -0.2 0.12 ± 29% perf-profile.children.cycles-pp.add_pending_csums 0.36 ± 21% -0.2 0.17 ± 21% perf-profile.children.cycles-pp.write_one_eb 0.21 ± 50% -0.2 0.03 ±100% perf-profile.children.cycles-pp.__list_del_entry_valid 0.21 ± 20% -0.2 0.03 ±100% perf-profile.children.cycles-pp.clear_state_bit 0.24 ± 38% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.select_task_rq_fair 0.31 ± 34% -0.2 0.14 ± 18% perf-profile.children.cycles-pp.pagecache_get_page 0.25 ± 66% -0.2 0.08 ± 70% perf-profile.children.cycles-pp.set_next_entity 0.25 ± 19% -0.2 0.08 ± 76% perf-profile.children.cycles-pp.__filemap_fdatawait_range 0.25 ± 19% -0.2 0.08 ± 76% perf-profile.children.cycles-pp.filemap_fdatawait_range 0.20 ± 36% -0.2 0.03 ±105% perf-profile.children.cycles-pp.btrfs_unlock_up_safe 0.34 ± 24% -0.2 0.18 ± 22% perf-profile.children.cycles-pp.read 0.20 ± 14% -0.2 0.04 ±102% perf-profile.children.cycles-pp.run_ksoftirqd 0.19 ± 46% -0.2 0.03 ±100% perf-profile.children.cycles-pp.btrfs_get_token_64 0.21 ± 35% -0.2 0.06 ± 14% perf-profile.children.cycles-pp.update_curr 0.24 ± 27% -0.1 0.10 ± 33% perf-profile.children.cycles-pp.lock_extent_bits 0.20 ± 43% -0.1 0.06 ± 15% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need 0.19 ± 32% -0.1 0.05 ± 61% perf-profile.children.cycles-pp.clear_extent_bit 0.18 ± 23% -0.1 0.05 ±114% perf-profile.children.cycles-pp.blk_mq_do_dispatch_sched 0.16 ± 31% -0.1 0.04 ±107% perf-profile.children.cycles-pp._cond_resched 0.14 ± 69% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__switch_to 0.18 ± 32% -0.1 0.07 ± 70% perf-profile.children.cycles-pp.blk_flush_plug_list 0.18 ± 34% -0.1 0.07 ± 70% perf-profile.children.cycles-pp.blk_mq_flush_plug_list 0.15 ± 16% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.__lookup_extent_mapping 0.21 ± 14% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.end_bio_extent_buffer_writepage 0.14 ± 63% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.do_anonymous_page 0.14 ± 35% -0.1 0.04 ±113% perf-profile.children.cycles-pp.btrfs_writepage_endio_finish_ordered 0.14 ± 35% -0.1 0.04 ±113% perf-profile.children.cycles-pp.end_extent_writepage 0.18 ± 23% -0.1 0.08 ± 59% perf-profile.children.cycles-pp.seq_read 0.22 ± 35% -0.1 0.14 ± 24% perf-profile.children.cycles-pp.blk_mq_run_hw_queue 0.16 ± 34% -0.1 0.08 ± 22% perf-profile.children.cycles-pp.find_get_entry 0.22 ± 25% -0.1 0.14 ± 32% perf-profile.children.cycles-pp.__blk_mq_run_hw_queue 0.22 ± 25% -0.1 0.14 ± 32% perf-profile.children.cycles-pp.blk_mq_sched_dispatch_requests 0.13 ± 31% -0.1 0.05 ±114% perf-profile.children.cycles-pp.btrfs_wait_extents 0.17 ± 17% -0.1 0.10 ± 30% perf-profile.children.cycles-pp.proc_reg_read 0.20 ± 28% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.__blk_mq_delay_run_hw_queue 0.10 ± 38% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__etree_search 0.03 ±100% +0.1 0.10 ± 9% perf-profile.children.cycles-pp.call_timer_fn 0.01 ±173% +0.1 0.09 ± 26% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.08 ± 26% perf-profile.children.cycles-pp.hrtimer_run_queues 0.03 ±100% +0.1 0.12 ± 33% perf-profile.children.cycles-pp.tick_program_event 0.04 ±173% +0.1 0.13 ± 36% perf-profile.children.cycles-pp.rcu_segcblist_ready_cbs 0.04 ±115% +0.1 0.14 ± 34% perf-profile.children.cycles-pp.profile_tick 0.15 ± 8% +0.1 0.25 ± 34% perf-profile.children.cycles-pp.rcu_eqs_exit 0.00 +0.1 0.10 ± 27% perf-profile.children.cycles-pp.call_function_single_interrupt 0.08 ± 68% +0.1 0.21 ± 32% perf-profile.children.cycles-pp.calc_global_load_tick 0.23 ± 5% +0.1 0.37 ± 26% perf-profile.children.cycles-pp.rcu_idle_exit 0.09 ± 33% +0.1 0.23 ± 12% perf-profile.children.cycles-pp.run_local_timers 0.35 ± 8% +0.1 0.49 ± 10% perf-profile.children.cycles-pp.native_sched_clock 0.36 ± 7% +0.1 0.51 ± 11% perf-profile.children.cycles-pp.sched_clock 0.14 ± 34% +0.2 0.29 ± 13% perf-profile.children.cycles-pp.intel_pmu_disable_all 0.48 ± 10% +0.2 0.64 ± 9% perf-profile.children.cycles-pp.sched_clock_cpu 0.17 ± 19% +0.2 0.33 ± 33% perf-profile.children.cycles-pp.cpu_load_update 0.18 ± 15% +0.2 0.38 ± 29% perf-profile.children.cycles-pp.perf_event_task_tick 0.06 ±104% +0.2 0.25 ± 22% perf-profile.children.cycles-pp.ksoftirqd_running 0.09 ± 50% +0.2 0.28 ± 32% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.11 ± 59% +0.2 0.32 ± 36% perf-profile.children.cycles-pp.account_process_tick 0.23 ± 35% +0.2 0.44 ± 19% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.54 ± 32% +0.2 0.77 ± 9% perf-profile.children.cycles-pp.native_write_msr 0.16 ± 24% +0.2 0.40 ± 33% perf-profile.children.cycles-pp.run_posix_cpu_timers 0.47 ± 34% +0.3 0.74 ± 12% perf-profile.children.cycles-pp.lapic_next_deadline 0.31 ± 43% +0.3 0.59 ± 8% perf-profile.children.cycles-pp.find_busiest_group 0.27 ± 48% +0.3 0.56 ± 10% perf-profile.children.cycles-pp.update_sd_lb_stats 0.13 ± 60% +0.3 0.43 ± 51% perf-profile.children.cycles-pp.tick_sched_do_timer 0.22 ± 36% +0.3 0.53 ± 22% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.29 ± 33% +0.3 0.64 ± 9% perf-profile.children.cycles-pp.run_rebalance_domains 0.39 ± 22% +0.4 0.76 ± 17% perf-profile.children.cycles-pp.run_timer_softirq 0.38 ± 32% +0.4 0.80 ± 8% perf-profile.children.cycles-pp.load_balance 0.24 ± 54% +0.5 0.73 ± 30% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.36 ± 44% +0.5 0.90 ± 12% perf-profile.children.cycles-pp.rebalance_domains 0.67 ± 31% +0.6 1.23 ± 17% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 1.23 ± 18% +1.1 2.38 ± 24% perf-profile.children.cycles-pp.scheduler_tick 0.85 ± 44% +1.4 2.20 ± 22% perf-profile.children.cycles-pp.clockevents_program_event 0.93 ± 32% +1.5 2.41 ± 18% perf-profile.children.cycles-pp.ktime_get 2.21 ± 28% +2.1 4.27 ± 22% perf-profile.children.cycles-pp.update_process_times 2.42 ± 24% +2.3 4.76 ± 23% perf-profile.children.cycles-pp.tick_sched_handle 2.95 ± 25% +3.1 6.03 ± 24% perf-profile.children.cycles-pp.tick_sched_timer 5.00 ± 27% +3.8 8.76 ± 15% perf-profile.children.cycles-pp.__hrtimer_run_queues 6.79 ± 30% +5.7 12.53 ± 13% perf-profile.children.cycles-pp.hrtimer_interrupt 11.63 ± 29% +8.3 19.95 ± 12% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 12.37 ± 27% +8.8 21.16 ± 11% perf-profile.children.cycles-pp.apic_timer_interrupt 37.70 ± 26% +14.5 52.19 ± 6% perf-profile.children.cycles-pp.intel_idle 54.85 ± 25% +24.2 79.06 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state 64.73 ± 24% +25.5 90.24 perf-profile.children.cycles-pp.secondary_startup_64 64.73 ± 24% +25.5 90.24 perf-profile.children.cycles-pp.cpu_startup_entry 64.71 ± 24% +25.6 90.27 perf-profile.children.cycles-pp.do_idle 58.47 ± 27% +27.6 86.03 ± 2% perf-profile.children.cycles-pp.start_secondary 2.37 ±118% -2.1 0.25 ± 31% perf-profile.self.cycles-pp.rwsem_spin_on_owner 1.15 ± 38% -0.9 0.20 ± 63% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.19 ± 23% -0.7 1.48 ± 33% perf-profile.self.cycles-pp.memcpy_toio 0.78 ± 49% -0.6 0.14 ± 25% perf-profile.self.cycles-pp.btrfs_get_token_32 0.67 ± 47% -0.6 0.11 ± 9% perf-profile.self.cycles-pp.memmove 0.58 ± 43% -0.5 0.12 ± 28% perf-profile.self.cycles-pp.memcpy_from_page 0.54 ± 46% -0.4 0.12 ± 48% perf-profile.self.cycles-pp.__sched_text_start 0.55 ± 63% -0.4 0.12 ± 16% perf-profile.self.cycles-pp.btrfs_set_token_32 0.37 ± 61% -0.3 0.07 ± 62% perf-profile.self.cycles-pp.generic_bin_search 0.62 ± 34% -0.3 0.35 ± 21% perf-profile.self.cycles-pp.copy_page 0.34 ± 28% -0.3 0.07 ± 66% perf-profile.self.cycles-pp.poll_idle 0.30 ± 39% -0.3 0.05 ± 62% perf-profile.self.cycles-pp.map_private_extent_buffer 0.23 ± 71% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__radix_tree_lookup 0.27 ± 64% -0.2 0.07 ± 68% perf-profile.self.cycles-pp.kmem_cache_alloc 0.23 ± 46% -0.2 0.03 ±100% perf-profile.self.cycles-pp.___might_sleep 0.20 ± 51% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__list_del_entry_valid 0.22 ± 43% -0.2 0.07 ± 72% perf-profile.self.cycles-pp.try_to_wake_up 0.18 ± 40% -0.1 0.05 ± 59% perf-profile.self.cycles-pp.kmem_cache_free 0.14 ± 67% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__switch_to 0.14 ± 20% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__lookup_extent_mapping 0.09 ± 24% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__etree_search 0.08 ± 17% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.find_get_entry 0.00 +0.1 0.06 ± 20% perf-profile.self.cycles-pp.__intel_pmu_disable_all 0.02 ±173% +0.1 0.09 ± 25% perf-profile.self.cycles-pp.rebalance_domains 0.01 ±173% +0.1 0.09 ± 17% perf-profile.self.cycles-pp.apic_timer_interrupt 0.03 ±100% +0.1 0.11 ± 36% perf-profile.self.cycles-pp.tick_nohz_idle_got_tick 0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.hrtimer_run_queues 0.03 ±100% +0.1 0.11 ± 11% perf-profile.self.cycles-pp.update_process_times 0.01 ±173% +0.1 0.10 ± 36% perf-profile.self.cycles-pp.tick_irq_enter 0.00 +0.1 0.09 ± 17% perf-profile.self.cycles-pp.intel_pmu_disable_all 0.03 ±100% +0.1 0.12 ± 33% perf-profile.self.cycles-pp.tick_program_event 0.04 ±115% +0.1 0.14 ± 34% perf-profile.self.cycles-pp.profile_tick 0.08 ± 59% +0.1 0.17 ± 20% perf-profile.self.cycles-pp.rcu_eqs_exit 0.03 ±173% +0.1 0.13 ± 38% perf-profile.self.cycles-pp.load_balance 0.04 ±106% +0.1 0.15 ± 23% perf-profile.self.cycles-pp.run_local_timers 0.08 ± 68% +0.1 0.20 ± 33% perf-profile.self.cycles-pp.calc_global_load_tick 0.01 ±173% +0.1 0.14 ± 39% perf-profile.self.cycles-pp.tick_sched_do_timer 0.16 ± 41% +0.1 0.29 ± 19% perf-profile.self.cycles-pp.hrtimer_interrupt 0.16 ± 42% +0.2 0.32 ± 28% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.30 ± 4% +0.2 0.46 ± 11% perf-profile.self.cycles-pp.native_sched_clock 0.17 ± 19% +0.2 0.33 ± 33% perf-profile.self.cycles-pp.cpu_load_update 0.21 ± 57% +0.2 0.37 ± 20% perf-profile.self.cycles-pp.update_sd_lb_stats 0.03 ±100% +0.2 0.20 ± 21% perf-profile.self.cycles-pp.rcu_core 0.18 ± 15% +0.2 0.37 ± 29% perf-profile.self.cycles-pp.perf_event_task_tick 0.06 ±104% +0.2 0.25 ± 22% perf-profile.self.cycles-pp.ksoftirqd_running 0.11 ± 59% +0.2 0.32 ± 36% perf-profile.self.cycles-pp.account_process_tick 0.06 ± 73% +0.2 0.28 ± 31% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.54 ± 32% +0.2 0.77 ± 9% perf-profile.self.cycles-pp.native_write_msr 0.16 ± 24% +0.2 0.40 ± 33% perf-profile.self.cycles-pp.run_posix_cpu_timers 0.30 ± 23% +0.3 0.58 ± 19% perf-profile.self.cycles-pp.run_timer_softirq 0.12 ± 21% +0.4 0.48 ± 44% perf-profile.self.cycles-pp.tick_sched_timer 0.20 ± 35% +0.4 0.60 ± 27% perf-profile.self.cycles-pp.rcu_sched_clock_irq 1.10 ± 25% +0.9 2.00 ± 5% perf-profile.self.cycles-pp.cpuidle_enter_state 0.43 ± 40% +1.2 1.61 ± 29% perf-profile.self.cycles-pp.ktime_get 37.57 ± 26% +14.5 52.03 ± 6% perf-profile.self.cycles-pp.intel_idle Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 5 months

1
0
0 / 0

[RFC PATCH] mm/workingset ] dea0795270: BUG:kernel_hang_in_boot_stage

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: dea079527088cffccd014c815ecaf5b0f0506c59 ("[[repost]RFC PATCH] mm/workingset : judge file page activity via timestamp") url: https://github.com/0day-ci/linux/commits/Zhaoyang-Huang/RFC-PATCH-mm-work... in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------+------------+------------+ | | 9520b5324b | dea0795270 | +-------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 4 | | BUG:kernel_hang_in_boot_stage | 0 | 4 | +-------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 0.067096] ACPI: Early table checksum verification disabled [ 0.068955] ACPI: RSDP 0x00000000000F6840 000014 (v00 BOCHS ) [ 0.071039] ACPI: RSDT 0x000000007FFE15C9 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001) [ 0.073473] ACPI: FACP 0x000000007FFE149D 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001) [ 0.075983] ACPI: DSDT 0x000000007FFE0040 00145D (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001) BUG: kernel hang in boot stage Elapsed time: 140 qemu-img create -f qcow2 disk-vm-snb-2G-406-0 256G qemu-img create -f qcow2 disk-vm-snb-2G-406-1 256G To reproduce: # build kernel cd linux cp config-5.1.0-rc6-00158-gdea07952 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year, 5 months

1
0
0 / 0

2020

2019

2018

2017

2016

2015

2014

2013

LKP May 2019