[block] 6861428921: vm-scalability.median 1003.2% improvement
by kernel test robot
Greeting,
FYI, we noticed a 1003.2% improvement of vm-scalability.median due to commit:
commit: 6861428921b51113520cd47897be6c2774e4fc58 ("block: always define BIO_MAX_PAGES as 256")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: vm-scalability
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:
runtime: 300
thp_enabled: always
thp_defrag: always
nr_task: 32
nr_ssd: 1
test: swap-w-seq
cpufreq_governor: performance
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_ssd/nr_task/rootfs/runtime/tbox_group/test/testcase/thp_defrag/thp_enabled:
gcc-7/performance/x86_64-rhel-7.6/1/32/debian-x86_64-2018-04-03.cgz/300/lkp-skl-2sp7/swap-w-seq/vm-scalability/always/always
commit:
07173c3ec2 ("block: enable multipage bvecs")
6861428921 ("block: always define BIO_MAX_PAGES as 256")
07173c3ec276cbb1 6861428921b51113520cd47897b
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.Out_of_memory:Kill_process
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
2:4 -50% :4 dmesg.WARNING:stack_recursion
%stddev %change %stddev
\ | \
0.12 ± 32% -65.5% 0.04 ± 4% vm-scalability.free_time
89576 ± 28% +1003.2% 988196 ± 5% vm-scalability.median
51.89 ± 13% -78.0% 11.43 ± 3% vm-scalability.stddev
8556830 ± 8% +310.9% 35161381 ± 5% vm-scalability.throughput
141.53 ± 20% -88.7% 15.95 ± 5% vm-scalability.time.elapsed_time
141.53 ± 20% -88.7% 15.95 ± 5% vm-scalability.time.elapsed_time.max
2774 ± 15% +45.1% 4024 ± 13% vm-scalability.time.involuntary_context_switches
204787 ± 3% +43.0% 292906 ± 13% vm-scalability.time.minor_page_faults
167.75 ± 20% +1210.9% 2199 vm-scalability.time.percent_of_cpu_this_job_got
106.53 ± 7% +110.2% 223.98 ± 7% vm-scalability.time.system_time
121.58 +4.4% 126.97 vm-scalability.time.user_time
38699 ± 21% -97.4% 994.00 ± 2% vm-scalability.time.voluntary_context_switches
29.00 +4.0% 30.15 ± 3% boot-time.boot
30.57 ± 2% -29.7 0.87 mpstat.cpu.all.iowait%
0.10 ±115% -0.1 0.00 ± 49% mpstat.cpu.all.soft%
1.12 ± 14% +17.6 18.68 ± 2% mpstat.cpu.all.sys%
1.29 ± 25% +9.6 10.89 ± 3% mpstat.cpu.all.usr%
363832 ± 11% -33.5% 242033 ± 3% numa-numastat.node0.local_node
372060 ± 11% -34.5% 243702 ± 3% numa-numastat.node0.numa_hit
323271 ± 6% -36.1% 206635 ± 9% numa-numastat.node1.local_node
329122 ± 7% -33.4% 219052 ± 8% numa-numastat.node1.numa_hit
34179 ± 7% -12.1% 30028 ± 8% numa-meminfo.node1.KReclaimable
34179 ± 7% -12.1% 30028 ± 8% numa-meminfo.node1.SReclaimable
56266 ± 3% -12.8% 49084 ± 6% numa-meminfo.node1.SUnreclaim
1880 ± 38% +329.5% 8076 ± 52% numa-meminfo.node1.Shmem
90446 ± 5% -12.5% 79113 ± 6% numa-meminfo.node1.Slab
726448 ± 7% -11.7% 641589 ± 10% numa-vmstat.node0.numa_hit
718153 ± 8% -10.9% 639936 ± 10% numa-vmstat.node0.numa_local
470.75 ± 39% +328.8% 2018 ± 52% numa-vmstat.node1.nr_shmem
8544 ± 7% -12.1% 7507 ± 8% numa-vmstat.node1.nr_slab_reclaimable
14066 ± 3% -12.8% 12270 ± 6% numa-vmstat.node1.nr_slab_unreclaimable
1.455e+08 ±149% -99.6% 591042 ± 8% cpuidle.C1.time
1256451 ±158% -99.4% 8079 cpuidle.C1.usage
5.623e+09 ± 20% -87.0% 7.331e+08 ± 5% cpuidle.C1E.time
14043488 ± 11% -88.8% 1575926 ± 5% cpuidle.C1E.usage
241125 ± 36% -94.8% 12566 ± 11% cpuidle.POLL.time
36871 ± 20% -87.0% 4780 ± 26% cpuidle.POLL.usage
112.25 ± 10% +724.9% 926.00 turbostat.Avg_MHz
5.44 ± 3% +24.8 30.25 turbostat.Busy%
2070 ± 8% +48.3% 3070 turbostat.Bzy_MHz
1253728 ±158% -99.7% 4008 ± 2% turbostat.C1
1.18 ±144% -1.1 0.04 ± 12% turbostat.C1%
14041415 ± 11% -88.8% 1579538 ± 5% turbostat.C1E
89.11 ± 6% -23.0% 68.62 turbostat.CPU%c1
47.50 ± 4% +10.2% 52.33 ± 3% turbostat.CoreTmp
20865755 ± 20% -88.0% 2504935 ± 5% turbostat.IRQ
1.40 ± 58% +10.3 11.68 ± 32% turbostat.PKG_%
110.81 ± 5% +99.5% 221.02 turbostat.PkgWatt
33.30 ± 4% +139.1% 79.64 ± 2% turbostat.RAMWatt
67.00 +7.5% 72.00 vmstat.cpu.id
0.50 ±100% +1766.7% 9.33 ± 5% vmstat.cpu.us
29.75 ± 2% -100.0% 0.00 vmstat.cpu.wa
153.50 ± 48% +316.1% 638.67 ± 24% vmstat.io.bi
394547 ± 3% -97.3% 10790 ± 4% vmstat.io.bo
43779329 ± 9% +40.0% 61286364 vmstat.memory.free
23332648 ± 25% -92.8% 1677588 ± 3% vmstat.memory.swpd
22.50 ± 3% -100.0% 0.00 vmstat.procs.b
1.25 ± 34% +1473.3% 19.67 ± 2% vmstat.procs.r
137.50 ± 54% +280.6% 523.33 ± 29% vmstat.swap.si
394532 ± 3% -97.3% 10682 ± 4% vmstat.swap.so
2924 ± 2% -8.3% 2682 ± 2% vmstat.system.cs
145508 -8.6% 132947 vmstat.system.in
84326843 ± 5% -16.3% 70555519 meminfo.Active
84326614 ± 5% -16.3% 70555392 meminfo.Active(anon)
85966552 ± 5% -19.8% 68973123 meminfo.AnonHugePages
86219683 ± 5% -18.3% 70470647 meminfo.AnonPages
2.369e+08 ± 2% -18.5% 1.931e+08 meminfo.Committed_AS
306860 ± 9% -12.5% 268446 ± 7% meminfo.DirectMap4k
2420185 ± 3% -32.0% 1645128 ± 2% meminfo.Inactive
2418347 ± 3% -32.0% 1644489 ± 2% meminfo.Inactive(anon)
42221972 ± 11% +34.7% 56854854 meminfo.MemAvailable
42804230 ± 11% +34.2% 57438579 meminfo.MemFree
88905904 ± 5% -16.5% 74271557 meminfo.Memused
218034 ± 4% -34.5% 142816 meminfo.PageTables
18404 ± 26% +44.3% 26562 ± 12% meminfo.Shmem
540318 ± 9% +201.2% 1627241 ± 3% meminfo.SwapCached
851206 ± 28% +690.9% 6732372 ± 3% meminfo.max_used_kB
527.00 ± 12% -20.9% 417.00 ± 3% slabinfo.biovec-128.active_objs
527.00 ± 12% -20.9% 417.00 ± 3% slabinfo.biovec-128.num_objs
344.25 ± 21% -72.1% 96.00 ± 3% slabinfo.biovec-max.active_objs
346.75 ± 20% -72.3% 96.00 ± 3% slabinfo.biovec-max.num_objs
1943 ± 3% -20.0% 1555 slabinfo.blkdev_ioc.active_objs
1943 ± 3% -20.0% 1555 slabinfo.blkdev_ioc.num_objs
112.00 +96.1% 219.67 ± 9% slabinfo.btrfs_extent_map.active_objs
112.00 +96.1% 219.67 ± 9% slabinfo.btrfs_extent_map.num_objs
17394 ± 4% -7.1% 16157 ± 4% slabinfo.filp.active_objs
17608 ± 4% -7.2% 16333 ± 4% slabinfo.filp.num_objs
8561 ± 5% -15.3% 7252 ± 2% slabinfo.kmalloc-96.active_objs
8570 ± 5% -14.7% 7311 ± 2% slabinfo.kmalloc-96.num_objs
1248 ± 3% -18.2% 1021 ± 15% slabinfo.task_group.active_objs
1248 ± 3% -18.2% 1021 ± 15% slabinfo.task_group.num_objs
141.50 ± 43% -47.5% 74.33 ± 81% slabinfo.xfs_btree_cur.active_objs
141.50 ± 43% -47.5% 74.33 ± 81% slabinfo.xfs_btree_cur.num_objs
67.36 +7.8% 72.58 iostat.cpu.idle
30.11 ± 2% -97.4% 0.79 iostat.cpu.iowait
1.23 ± 8% +1268.7% 16.84 ± 3% iostat.cpu.system
1.30 ± 24% +651.8% 9.79 ± 2% iostat.cpu.user
56.50 ± 2% -97.0% 1.68 ± 5% iostat.sdc.avgqu-sz
1857 -97.4% 47.62 iostat.sdc.avgrq-sz
134.57 -97.4% 3.56 ± 5% iostat.sdc.await
166.35 ± 12% -70.6% 48.83 ± 3% iostat.sdc.await.max
24.14 ± 53% +299.8% 96.54 ± 28% iostat.sdc.r/s
74.44 ± 25% -99.5% 0.37 ± 21% iostat.sdc.r_await
125.94 ± 7% -96.2% 4.73 ± 39% iostat.sdc.r_await.max
138.16 ± 53% +293.3% 543.33 ± 28% iostat.sdc.rkB/s
9.66 ± 57% +258.4% 34.64 ± 33% iostat.sdc.rrqm/s
39.63 ± 3% -90.8% 3.63 ± 13% iostat.sdc.util
385.01 ± 3% -92.8% 27.62 ± 9% iostat.sdc.w/s
136.30 -70.9% 39.63 ± 14% iostat.sdc.w_await
166.66 ± 12% +238.0% 563.33 ± 13% iostat.sdc.w_await.max
391580 ± 3% -97.3% 10682 ± 4% iostat.sdc.wkB/s
342.04 ± 33% +672.7% 2643 ± 4% iostat.sdc.wrqm/s
19.90 ± 30% -19.9 0.00 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
17.41 ± 33% -17.4 0.00 perf-profile.calltrace.cycles-pp.page_fault
17.41 ± 33% -17.4 0.00 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault
17.40 ± 33% -17.4 0.00 perf-profile.calltrace.cycles-pp.__do_page_fault.do_page_fault.page_fault
17.39 ± 33% -17.4 0.00 perf-profile.calltrace.cycles-pp.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
17.38 ± 33% -17.4 0.00 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault.page_fault
17.31 ± 34% -17.3 0.00 perf-profile.calltrace.cycles-pp.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
16.88 ± 30% -16.9 0.00 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
14.20 ± 49% -14.2 0.00 perf-profile.calltrace.cycles-pp.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
13.67 ± 50% -13.7 0.00 perf-profile.calltrace.cycles-pp.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
79.58 ± 9% -12.3 67.25 ± 4% perf-profile.calltrace.cycles-pp.secondary_startup_64
78.64 ± 8% -11.4 67.25 ± 4% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
78.64 ± 8% -11.4 67.25 ± 4% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
78.59 ± 8% -11.3 67.25 ± 4% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
10.66 ± 30% -10.7 0.00 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
10.32 ± 50% -10.3 0.00 perf-profile.calltrace.cycles-pp.clear_page_erms.clear_subpage.clear_huge_page.do_huge_pmd_anonymous_page.__handle_mm_fault
6.14 ± 34% -6.1 0.00 perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
5.50 ± 32% -5.5 0.00 perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
4.78 ± 28% -4.8 0.00 perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
19069 ± 39% -59.4% 7750 ± 90% proc-vmstat.compact_free_scanned
9577 ± 42% -67.8% 3082 ± 97% proc-vmstat.compact_isolated
21104747 ± 5% -16.4% 17643515 proc-vmstat.nr_active_anon
21583000 ± 5% -18.4% 17603447 proc-vmstat.nr_anon_pages
42043 ± 5% -19.9% 33672 proc-vmstat.nr_anon_transparent_hugepages
1050704 ± 11% +34.9% 1417852 proc-vmstat.nr_dirty_background_threshold
2103983 ± 11% +34.9% 2839208 proc-vmstat.nr_dirty_threshold
445307 ± 3% +61.9% 720882 proc-vmstat.nr_file_pages
10676807 ± 11% +34.4% 14354359 proc-vmstat.nr_free_pages
605531 ± 3% -32.1% 411181 ± 3% proc-vmstat.nr_inactive_anon
11350 -77.0% 2611 ± 8% proc-vmstat.nr_isolated_anon
13240 -2.8% 12870 proc-vmstat.nr_kernel_stack
54623 ± 4% -34.6% 35718 proc-vmstat.nr_page_table_pages
4602 ± 26% +44.3% 6640 ± 12% proc-vmstat.nr_shmem
30951 -5.3% 29303 proc-vmstat.nr_slab_unreclaimable
9.50 ±173% +777.2% 83.33 ± 50% proc-vmstat.nr_vmscan_immediate_reclaim
58616 ± 14% +4992.7% 2985134 ± 10% proc-vmstat.nr_vmscan_write
79569 ± 17% -36.3% 50713 ± 2% proc-vmstat.nr_written
21104747 ± 5% -16.4% 17643515 proc-vmstat.nr_zone_active_anon
605531 ± 3% -32.1% 411155 ± 3% proc-vmstat.nr_zone_inactive_anon
21148 ± 26% +333.6% 91702 ± 24% proc-vmstat.numa_foreign
726653 ± 8% -33.1% 486398 ± 3% proc-vmstat.numa_hit
85245 ± 2% +14.8% 97846 ± 4% proc-vmstat.numa_huge_pte_updates
712567 ± 8% -33.7% 472313 ± 3% proc-vmstat.numa_local
21148 ± 26% +333.6% 91702 ± 24% proc-vmstat.numa_miss
35234 ± 15% +200.2% 105786 ± 21% proc-vmstat.numa_other
43663389 ± 2% +14.9% 50170277 ± 4% proc-vmstat.numa_pte_updates
15711286 ± 22% -91.1% 1393556 ± 6% proc-vmstat.pgdeactivate
590550 ± 12% -39.2% 359010 ± 10% proc-vmstat.pgfault
77729133 +2.1% 79393516 proc-vmstat.pgfree
4505 ± 41% -54.1% 2067 ± 71% proc-vmstat.pgmigrate_success
56874030 ± 22% -99.6% 204725 ± 2% proc-vmstat.pgpgout
96029 ± 51% -55.6% 42642 ± 4% proc-vmstat.pgrefill
78643 ± 17% -36.6% 49878 ± 2% proc-vmstat.pgrotated
197998 ± 28% +3237.5% 6608253 ± 8% proc-vmstat.pgscan_direct
29.75 ±149% +70918.5% 21128 ± 78% proc-vmstat.pgscan_kswapd
79484 ± 17% -35.8% 51047 proc-vmstat.pgsteal_direct
14217995 ± 22% -99.6% 50668 ± 2% proc-vmstat.pswpout
150640 +2.4% 154272 proc-vmstat.thp_deferred_split_page
27686 ± 22% -90.6% 2603 ± 5% proc-vmstat.thp_split_pmd
27668 ± 22% -100.0% 0.00 proc-vmstat.thp_swpout
2.878e+09 ± 25% +815.4% 2.635e+10 ± 3% perf-stat.i.branch-instructions
0.78 ± 33% -0.6 0.22 ± 23% perf-stat.i.branch-miss-rate%
7946383 ± 24% +245.0% 27415662 ± 5% perf-stat.i.branch-misses
18.20 ± 24% +31.4 49.57 perf-stat.i.cache-miss-rate%
16435347 ± 14% +2031.0% 3.502e+08 ± 2% perf-stat.i.cache-misses
63696811 ± 13% +1023.7% 7.158e+08 perf-stat.i.cache-references
2851 -18.5% 2324 ± 3% perf-stat.i.context-switches
1.21 ± 6% -30.8% 0.84 ± 4% perf-stat.i.cpi
7.395e+09 ± 11% +874.9% 7.21e+10 perf-stat.i.cpu-cycles
7.16 ± 14% +212.9% 22.39 ± 6% perf-stat.i.cpu-migrations
747.18 ± 18% -68.3% 237.10 perf-stat.i.cycles-between-cache-misses
0.07 ± 58% -0.1 0.00 ± 5% perf-stat.i.dTLB-load-miss-rate%
2.049e+09 ± 23% +831.5% 1.909e+10 ± 2% perf-stat.i.dTLB-loads
0.02 ± 49% -0.0 0.00 ± 15% perf-stat.i.dTLB-store-miss-rate%
70657 ± 38% +123.4% 157882 ± 14% perf-stat.i.dTLB-store-misses
7.366e+08 ± 22% +664.9% 5.634e+09 ± 5% perf-stat.i.dTLB-stores
9.412e+09 ± 24% +843.3% 8.878e+10 ± 3% perf-stat.i.instructions
192493 ± 70% +657.2% 1457546 ± 14% perf-stat.i.instructions-per-iTLB-miss
1.05 ± 13% +25.3% 1.31 ± 3% perf-stat.i.ipc
8.82 ± 45% +539.0% 56.34 ± 51% perf-stat.i.major-faults
4060 ± 11% +442.7% 22033 ± 16% perf-stat.i.minor-faults
277510 ± 20% +2857.2% 8206676 ± 9% perf-stat.i.node-load-misses
1262778 ± 10% +4905.1% 63203290 perf-stat.i.node-loads
8.83 ± 97% -5.3 3.58 ± 11% perf-stat.i.node-store-miss-rate%
42108 ± 20% +2480.3% 1086527 ± 8% perf-stat.i.node-store-misses
3464808 ± 27% +931.8% 35749595 ± 5% perf-stat.i.node-stores
4068 ± 11% +442.9% 22089 ± 16% perf-stat.i.page-faults
0.29 ± 37% -0.2 0.10 perf-stat.overall.branch-miss-rate%
26.56 ± 15% +22.4 48.92 perf-stat.overall.cache-miss-rate%
446.06 ± 2% -53.8% 206.15 ± 3% perf-stat.overall.cycles-between-cache-misses
0.03 ± 61% -0.0 0.00 ± 2% perf-stat.overall.dTLB-load-miss-rate%
0.01 ± 46% -0.0 0.00 ± 9% perf-stat.overall.dTLB-store-miss-rate%
157221 ±113% +566.3% 1047514 ± 11% perf-stat.overall.instructions-per-iTLB-miss
18.23 ± 26% -6.7 11.49 ± 9% perf-stat.overall.node-load-miss-rate%
1.19 ± 8% +1.8 2.97 ± 13% perf-stat.overall.node-store-miss-rate%
2.918e+09 ± 22% +745.4% 2.467e+10 ± 3% perf-stat.ps.branch-instructions
7929699 ± 24% +225.2% 25790635 ± 4% perf-stat.ps.branch-misses
16720398 ± 10% +1860.8% 3.278e+08 ± 2% perf-stat.ps.cache-misses
64027917 ± 13% +946.5% 6.701e+08 perf-stat.ps.cache-references
2832 -22.9% 2183 ± 3% perf-stat.ps.context-switches
7.452e+09 ± 9% +806.2% 6.754e+10 perf-stat.ps.cpu-cycles
7.24 ± 12% +190.7% 21.05 ± 5% perf-stat.ps.cpu-migrations
2.077e+09 ± 20% +760.4% 1.787e+10 ± 2% perf-stat.ps.dTLB-loads
70147 ± 38% +111.1% 148113 ± 13% perf-stat.ps.dTLB-store-misses
7.454e+08 ± 19% +607.6% 5.275e+09 ± 5% perf-stat.ps.dTLB-stores
9.534e+09 ± 21% +771.8% 8.312e+10 ± 3% perf-stat.ps.instructions
8.95 ± 42% +493.8% 53.12 ± 52% perf-stat.ps.major-faults
4079 ± 9% +406.4% 20658 ± 15% perf-stat.ps.minor-faults
142918 -5.6% 134916 perf-stat.ps.msec
275536 ± 20% +2688.1% 7682267 ± 10% perf-stat.ps.node-load-misses
1256768 ± 10% +4605.7% 59140271 perf-stat.ps.node-loads
42074 ± 18% +2317.7% 1017230 ± 8% perf-stat.ps.node-store-misses
3533362 ± 23% +847.4% 33473753 ± 5% perf-stat.ps.node-stores
4088 ± 9% +406.6% 20711 ± 15% perf-stat.ps.page-faults
12346 ± 34% -81.8% 2243 ± 22% sched_debug.cfs_rq:/.load.avg
453397 ± 26% -95.1% 22090 ± 24% sched_debug.cfs_rq:/.load.max
66975 ± 31% -91.6% 5640 ± 11% sched_debug.cfs_rq:/.load.stddev
42.00 ± 3% +124.3% 94.19 ± 21% sched_debug.cfs_rq:/.load_avg.avg
751.00 ± 16% +37.8% 1034 sched_debug.cfs_rq:/.load_avg.max
140.48 ± 11% +87.7% 263.71 ± 13% sched_debug.cfs_rq:/.load_avg.stddev
46652 ± 9% -59.0% 19148 ± 9% sched_debug.cfs_rq:/.min_vruntime.avg
92238 ± 11% -67.0% 30398 ± 11% sched_debug.cfs_rq:/.min_vruntime.max
29251 ± 7% -88.0% 3517 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev
0.06 ± 9% +123.1% 0.13 ± 27% sched_debug.cfs_rq:/.nr_running.avg
0.22 ± 3% +49.8% 0.34 ± 12% sched_debug.cfs_rq:/.nr_running.stddev
15.30 ± 13% +268.0% 56.31 ± 35% sched_debug.cfs_rq:/.removed.load_avg.avg
384.00 ± 19% +166.7% 1024 sched_debug.cfs_rq:/.removed.load_avg.max
74.20 ± 9% +207.6% 228.21 ± 16% sched_debug.cfs_rq:/.removed.load_avg.stddev
707.25 ± 13% +267.5% 2599 ± 35% sched_debug.cfs_rq:/.removed.runnable_sum.avg
17831 ± 19% +167.1% 47625 sched_debug.cfs_rq:/.removed.runnable_sum.max
3429 ± 9% +207.4% 10541 ± 16% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
5.99 ± 8% +261.3% 21.65 ± 28% sched_debug.cfs_rq:/.removed.util_avg.avg
192.71 ± 19% +163.1% 507.00 sched_debug.cfs_rq:/.removed.util_avg.max
30.20 ± 14% +201.6% 91.09 ± 10% sched_debug.cfs_rq:/.removed.util_avg.stddev
5.55 ± 22% -68.0% 1.78 ± 19% sched_debug.cfs_rq:/.runnable_load_avg.avg
341.04 ± 26% -95.3% 16.00 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.max
41.28 ± 25% -88.7% 4.65 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.stddev
12298 ± 34% -82.9% 2098 ± 16% sched_debug.cfs_rq:/.runnable_weight.avg
449033 ± 27% -96.0% 18087 ± 2% sched_debug.cfs_rq:/.runnable_weight.max
66673 ± 31% -92.0% 5347 ± 5% sched_debug.cfs_rq:/.runnable_weight.stddev
40040 ± 40% -58.6% 16557 ± 44% sched_debug.cfs_rq:/.spread0.max
29251 ± 7% -88.0% 3519 ± 8% sched_debug.cfs_rq:/.spread0.stddev
173.85 ± 17% +145.2% 426.28 ± 2% sched_debug.cfs_rq:/.util_avg.avg
858.50 ± 14% +50.7% 1293 ± 6% sched_debug.cfs_rq:/.util_avg.max
153.07 ± 12% +78.4% 273.07 ± 6% sched_debug.cfs_rq:/.util_avg.stddev
10.79 ± 21% +156.9% 27.72 ± 12% sched_debug.cfs_rq:/.util_est_enqueued.avg
236.58 ± 15% +115.7% 510.33 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.max
43.34 ± 11% +141.3% 104.56 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev
829892 ± 3% -25.9% 615172 sched_debug.cpu.avg_idle.avg
228918 ± 73% -99.4% 1421 ± 21% sched_debug.cpu.avg_idle.min
153019 ± 14% +57.6% 241163 ± 8% sched_debug.cpu.avg_idle.stddev
87991 ± 15% -60.9% 34364 ± 3% sched_debug.cpu.clock.avg
88116 ± 15% -61.0% 34368 ± 3% sched_debug.cpu.clock.max
87240 ± 15% -60.6% 34360 ± 3% sched_debug.cpu.clock.min
280.54 ± 26% -99.2% 2.30 ± 15% sched_debug.cpu.clock.stddev
87991 ± 15% -60.9% 34364 ± 3% sched_debug.cpu.clock_task.avg
88116 ± 15% -61.0% 34368 ± 3% sched_debug.cpu.clock_task.max
87240 ± 15% -60.6% 34360 ± 3% sched_debug.cpu.clock_task.min
280.54 ± 26% -99.2% 2.30 ± 15% sched_debug.cpu.clock_task.stddev
5.62 ± 23% -62.9% 2.09 ± 15% sched_debug.cpu.cpu_load[0].avg
341.38 ± 26% -95.3% 16.00 ± 5% sched_debug.cpu.cpu_load[0].max
41.31 ± 25% -88.2% 4.89 ± 6% sched_debug.cpu.cpu_load[0].stddev
5.77 ± 24% -54.1% 2.65 ± 21% sched_debug.cpu.cpu_load[1].avg
334.12 ± 26% -93.5% 21.67 ± 33% sched_debug.cpu.cpu_load[1].max
40.30 ± 26% -88.1% 4.82 ± 16% sched_debug.cpu.cpu_load[1].stddev
5.46 ± 32% -33.0% 3.66 ± 25% sched_debug.cpu.cpu_load[2].avg
299.12 ± 38% -82.7% 51.67 ± 40% sched_debug.cpu.cpu_load[2].max
35.71 ± 37% -79.4% 7.36 ± 36% sched_debug.cpu.cpu_load[2].stddev
259.21 ± 55% -70.9% 75.33 ± 14% sched_debug.cpu.cpu_load[3].max
31.38 ± 53% -62.7% 11.70 ± 18% sched_debug.cpu.cpu_load[3].stddev
120.21 ± 9% +100.4% 240.89 ± 27% sched_debug.cpu.curr->pid.avg
3203 ± 11% -46.0% 1729 sched_debug.cpu.curr->pid.max
12408 ± 34% -80.1% 2471 ± 32% sched_debug.cpu.load.avg
454839 ± 27% -95.1% 22090 ± 24% sched_debug.cpu.load.max
67115 ± 31% -91.2% 5938 ± 18% sched_debug.cpu.load.stddev
0.00 ± 25% -89.5% 0.00 ± 11% sched_debug.cpu.next_balance.stddev
62558 ± 21% -86.5% 8451 sched_debug.cpu.nr_load_updates.avg
69133 ± 20% -77.1% 15845 ± 8% sched_debug.cpu.nr_load_updates.max
59990 ± 21% -89.0% 6608 ± 18% sched_debug.cpu.nr_load_updates.min
0.06 ± 8% +137.0% 0.15 ± 26% sched_debug.cpu.nr_running.avg
0.23 ± 2% +54.4% 0.35 ± 11% sched_debug.cpu.nr_running.stddev
3321 ± 17% -66.4% 1114 ± 11% sched_debug.cpu.nr_switches.avg
27971 ± 15% -74.1% 7248 ± 17% sched_debug.cpu.nr_switches.max
542.54 ± 12% -38.8% 332.00 ± 15% sched_debug.cpu.nr_switches.min
3735 ± 9% -63.0% 1383 ± 8% sched_debug.cpu.nr_switches.stddev
0.23 ± 9% -85.9% 0.03 ± 20% sched_debug.cpu.nr_uninterruptible.avg
87202 ± 15% -60.6% 34360 ± 3% sched_debug.cpu_clk
84395 ± 16% -62.6% 31575 ± 3% sched_debug.ktime
87780 ± 15% -60.2% 34931 ± 3% sched_debug.sched_clk
30382 ± 22% -93.9% 1860 ± 15% softirqs.BLOCK
33695 ± 24% -93.1% 2330 ± 17% softirqs.CPU0.RCU
21554 ± 18% -64.6% 7624 ± 10% softirqs.CPU0.SCHED
66621 ± 24% -79.3% 13785 ± 2% softirqs.CPU0.TIMER
34498 ± 19% -89.9% 3496 ± 70% softirqs.CPU1.RCU
19179 ± 18% -80.0% 3833 ± 34% softirqs.CPU1.SCHED
65051 ± 25% -78.2% 14171 ± 38% softirqs.CPU1.TIMER
38090 ± 27% -92.6% 2823 ± 15% softirqs.CPU10.RCU
19372 ± 20% -84.9% 2927 ± 12% softirqs.CPU10.SCHED
66363 ± 25% -81.7% 12126 ± 7% softirqs.CPU10.TIMER
35164 ± 26% -91.6% 2945 ± 33% softirqs.CPU11.RCU
19482 ± 22% -83.0% 3312 ± 6% softirqs.CPU11.SCHED
65570 ± 25% -81.5% 12108 ± 7% softirqs.CPU11.TIMER
36102 ± 24% -94.2% 2099 ± 5% softirqs.CPU12.RCU
27007 ± 59% -88.6% 3090 softirqs.CPU12.SCHED
66236 ± 25% -81.8% 12072 ± 5% softirqs.CPU12.TIMER
37852 ± 27% -91.1% 3385 ± 52% softirqs.CPU13.RCU
19566 ± 20% -81.9% 3535 ± 19% softirqs.CPU13.SCHED
66208 ± 25% -79.1% 13818 ± 19% softirqs.CPU13.TIMER
33605 ± 24% -92.8% 2420 ± 28% softirqs.CPU14.RCU
19345 ± 20% -84.0% 3092 ± 17% softirqs.CPU14.SCHED
64337 ± 23% -81.6% 11829 ± 3% softirqs.CPU14.TIMER
26910 ± 24% -92.5% 2012 ± 20% softirqs.CPU15.RCU
19239 ± 20% -85.5% 2789 ± 13% softirqs.CPU15.SCHED
65447 ± 25% -82.3% 11615 ± 4% softirqs.CPU15.TIMER
26523 ± 23% -91.1% 2359 ± 22% softirqs.CPU16.RCU
19218 ± 19% -84.1% 3051 ± 9% softirqs.CPU16.SCHED
66088 ± 26% -81.1% 12511 ± 8% softirqs.CPU16.TIMER
26962 ± 24% -87.9% 3262 ± 69% softirqs.CPU17.RCU
19703 ± 20% -83.5% 3245 ± 5% softirqs.CPU17.SCHED
66543 ± 25% -81.7% 12153 ± 6% softirqs.CPU17.TIMER
33228 ± 23% -92.2% 2597 ± 26% softirqs.CPU18.RCU
19627 ± 23% -81.4% 3652 ± 14% softirqs.CPU18.SCHED
51119 ± 21% -75.6% 12487 ± 9% softirqs.CPU18.TIMER
33693 ± 23% -91.4% 2894 ± 15% softirqs.CPU19.RCU
19758 ± 20% -85.7% 2822 ± 20% softirqs.CPU19.SCHED
57958 ± 38% -74.0% 15055 ± 24% softirqs.CPU19.TIMER
36652 ± 27% -90.2% 3587 ± 51% softirqs.CPU2.RCU
19522 ± 19% -82.7% 3369 ± 9% softirqs.CPU2.SCHED
67481 ± 22% -82.5% 11824 ± 6% softirqs.CPU2.TIMER
33741 ± 26% -94.1% 1987 ± 30% softirqs.CPU20.RCU
19347 ± 20% -83.8% 3130 ± 16% softirqs.CPU20.SCHED
49615 ± 19% -77.1% 11379 ± 4% softirqs.CPU20.TIMER
33377 ± 22% -93.9% 2023 ± 18% softirqs.CPU21.RCU
19061 ± 20% -83.5% 3154 ± 19% softirqs.CPU21.SCHED
48446 ± 20% -76.3% 11478 ± 6% softirqs.CPU21.TIMER
32776 ± 24% -92.0% 2620 ± 55% softirqs.CPU22.RCU
19233 ± 19% -86.1% 2675 ± 26% softirqs.CPU22.SCHED
49106 ± 22% -73.8% 12851 ± 10% softirqs.CPU22.TIMER
31660 ± 22% -94.1% 1883 ± 28% softirqs.CPU23.RCU
18764 ± 20% -82.0% 3386 ± 12% softirqs.CPU23.SCHED
46906 ± 19% -75.3% 11594 ± 7% softirqs.CPU23.TIMER
32115 ± 24% -94.7% 1688 ± 30% softirqs.CPU24.RCU
19048 ± 19% -83.6% 3130 ± 15% softirqs.CPU24.SCHED
49073 ± 19% -76.5% 11533 ± 7% softirqs.CPU24.TIMER
31473 ± 23% -93.2% 2152 ± 3% softirqs.CPU25.RCU
19257 ± 19% -83.0% 3280 ± 13% softirqs.CPU25.SCHED
51553 ± 22% -77.5% 11596 ± 4% softirqs.CPU25.TIMER
33810 ± 26% -91.7% 2813 ± 48% softirqs.CPU26.RCU
19343 ± 18% -83.2% 3244 ± 14% softirqs.CPU26.SCHED
49490 ± 21% -76.0% 11864 ± 5% softirqs.CPU26.TIMER
32552 ± 24% -89.6% 3373 ± 69% softirqs.CPU27.RCU
18875 ± 19% -82.8% 3254 ± 4% softirqs.CPU27.SCHED
48454 ± 19% -76.0% 11626 ± 5% softirqs.CPU27.TIMER
30030 ± 46% -93.2% 2047 ± 25% softirqs.CPU28.RCU
19167 ± 17% -83.5% 3172 ± 6% softirqs.CPU28.SCHED
47942 ± 17% -75.9% 11566 ± 3% softirqs.CPU28.TIMER
33103 ± 20% -87.8% 4025 ± 39% softirqs.CPU29.RCU
19403 ± 16% -84.3% 3047 ± 5% softirqs.CPU29.SCHED
54323 ± 7% -73.7% 14297 ± 29% softirqs.CPU29.TIMER
36877 ± 29% -78.1% 8089 ± 11% softirqs.CPU3.RCU
19493 ± 19% -81.8% 3550 ± 14% softirqs.CPU3.SCHED
65990 ± 25% -75.8% 15969 ± 8% softirqs.CPU3.TIMER
28611 ± 23% -94.3% 1630 ± 11% softirqs.CPU30.RCU
19185 ± 19% -85.7% 2740 ± 23% softirqs.CPU30.SCHED
49156 ± 21% -76.1% 11737 ± 5% softirqs.CPU30.TIMER
27996 ± 22% -92.3% 2145 ± 19% softirqs.CPU31.RCU
19271 ± 18% -82.2% 3434 ± 6% softirqs.CPU31.SCHED
50908 ± 22% -76.5% 11985 ± 2% softirqs.CPU31.TIMER
28302 ± 24% -93.8% 1742 ± 30% softirqs.CPU32.RCU
18860 ± 20% -83.0% 3198 ± 3% softirqs.CPU32.SCHED
46783 ± 20% -74.9% 11725 ± 2% softirqs.CPU32.TIMER
30319 ± 23% -93.4% 2012 ± 9% softirqs.CPU33.RCU
19493 ± 20% -83.7% 3180 ± 4% softirqs.CPU33.SCHED
66157 ± 32% -82.1% 11831 ± 3% softirqs.CPU33.TIMER
28390 ± 23% -91.6% 2376 ± 24% softirqs.CPU34.RCU
19349 ± 19% -82.7% 3348 ± 8% softirqs.CPU34.SCHED
48588 ± 21% -75.7% 11819 softirqs.CPU34.TIMER
28752 ± 22% -93.6% 1836 ± 6% softirqs.CPU35.RCU
19247 ± 18% -83.5% 3173 ± 8% softirqs.CPU35.SCHED
49822 ± 19% -76.0% 11964 softirqs.CPU35.TIMER
32018 ± 22% -92.8% 2313 ± 11% softirqs.CPU36.RCU
19323 ± 20% -87.5% 2413 ± 12% softirqs.CPU36.SCHED
65438 ± 25% -82.1% 11711 ± 6% softirqs.CPU36.TIMER
34935 ± 22% -90.9% 3191 ± 6% softirqs.CPU37.RCU
19599 ± 21% -79.3% 4055 ± 18% softirqs.CPU37.SCHED
66573 ± 27% -80.5% 12976 softirqs.CPU37.TIMER
33636 ± 25% -91.7% 2798 ± 33% softirqs.CPU38.RCU
19747 ± 22% -82.2% 3506 ± 19% softirqs.CPU38.SCHED
66313 ± 26% -81.5% 12280 ± 8% softirqs.CPU38.TIMER
33655 ± 26% -90.7% 3140 ± 58% softirqs.CPU39.RCU
18984 ± 21% -83.0% 3233 ± 14% softirqs.CPU39.SCHED
63569 ± 26% -77.1% 14569 ± 24% softirqs.CPU39.TIMER
35962 ± 25% -89.2% 3900 ± 52% softirqs.CPU4.RCU
19414 ± 20% -85.4% 2828 ± 30% softirqs.CPU4.SCHED
65642 ± 25% -80.4% 12836 ± 2% softirqs.CPU4.TIMER
35225 ± 24% -94.2% 2049 ± 5% softirqs.CPU40.RCU
19411 ± 21% -83.0% 3295 ± 11% softirqs.CPU40.SCHED
65189 ± 25% -81.5% 12076 ± 4% softirqs.CPU40.TIMER
28976 ± 38% -92.8% 2090 ± 22% softirqs.CPU41.RCU
18919 ± 21% -83.2% 3179 ± 16% softirqs.CPU41.SCHED
64523 ± 26% -81.5% 11968 ± 4% softirqs.CPU41.TIMER
30996 ± 22% -94.2% 1803 ± 24% softirqs.CPU42.RCU
19314 ± 20% -83.6% 3163 ± 18% softirqs.CPU42.SCHED
67652 ± 29% -82.9% 11580 ± 4% softirqs.CPU42.TIMER
31537 ± 21% -93.6% 2024 ± 11% softirqs.CPU43.RCU
18801 ± 19% -83.9% 3034 ± 24% softirqs.CPU43.SCHED
62622 ± 22% -80.9% 11973 ± 5% softirqs.CPU43.TIMER
34114 ± 25% -93.6% 2172 ± 11% softirqs.CPU44.RCU
19184 ± 20% -84.6% 2960 ± 23% softirqs.CPU44.SCHED
65380 ± 25% -82.1% 11726 ± 7% softirqs.CPU44.TIMER
31132 ± 21% -93.3% 2077 ± 26% softirqs.CPU45.RCU
21106 ± 26% -83.3% 3517 ± 8% softirqs.CPU45.SCHED
65507 ± 26% -81.7% 11999 softirqs.CPU45.TIMER
34797 ± 23% -91.9% 2819 ± 23% softirqs.CPU46.RCU
19317 ± 21% -82.5% 3378 ± 19% softirqs.CPU46.SCHED
65578 ± 26% -81.8% 11941 ± 6% softirqs.CPU46.TIMER
25883 ± 41% -90.4% 2497 ± 23% softirqs.CPU47.RCU
19536 ± 20% -83.8% 3160 ± 23% softirqs.CPU47.SCHED
65082 ± 25% -81.9% 11767 ± 3% softirqs.CPU47.TIMER
35469 ± 24% -93.2% 2416 ± 19% softirqs.CPU48.RCU
19539 ± 19% -83.9% 3141 ± 22% softirqs.CPU48.SCHED
65712 ± 26% -81.7% 12001 ± 4% softirqs.CPU48.TIMER
36383 ± 21% -95.5% 1625 ± 19% softirqs.CPU49.RCU
19286 ± 21% -79.1% 4038 ± 35% softirqs.CPU49.SCHED
66192 ± 23% -82.0% 11883 ± 3% softirqs.CPU49.TIMER
36443 ± 26% -93.8% 2263 ± 16% softirqs.CPU5.RCU
19520 ± 20% -83.8% 3157 ± 7% softirqs.CPU5.SCHED
65193 ± 25% -81.4% 12096 ± 6% softirqs.CPU5.TIMER
32092 ± 23% -94.5% 1754 ± 12% softirqs.CPU50.RCU
20294 ± 23% -85.7% 2892 ± 24% softirqs.CPU50.SCHED
64333 ± 24% -82.2% 11432 ± 2% softirqs.CPU50.TIMER
31428 ± 19% -93.3% 2106 ± 25% softirqs.CPU51.RCU
21209 ± 24% -81.9% 3844 ± 34% softirqs.CPU51.SCHED
65230 ± 25% -81.8% 11888 ± 5% softirqs.CPU51.TIMER
33852 ± 23% -93.6% 2171 ± 21% softirqs.CPU52.RCU
19506 ± 20% -84.6% 3002 ± 21% softirqs.CPU52.SCHED
65431 ± 26% -81.5% 12125 ± 7% softirqs.CPU52.TIMER
35488 ± 24% -91.1% 3145 ± 64% softirqs.CPU53.RCU
19538 ± 21% -85.4% 2855 ± 13% softirqs.CPU53.SCHED
65716 ± 26% -82.3% 11613 ± 4% softirqs.CPU53.TIMER
28481 ± 22% -93.6% 1831 ± 28% softirqs.CPU54.RCU
19105 ± 18% -83.8% 3093 ± 6% softirqs.CPU54.SCHED
48345 ± 18% -75.9% 11629 ± 4% softirqs.CPU54.TIMER
27730 ± 20% -95.9% 1144 ± 26% softirqs.CPU55.RCU
18638 ± 17% -77.1% 4262 ± 32% softirqs.CPU55.SCHED
45647 ± 16% -74.3% 11712 ± 3% softirqs.CPU55.TIMER
28531 ± 23% -94.3% 1624 ± 29% softirqs.CPU56.RCU
22300 ± 15% -86.0% 3125 ± 5% softirqs.CPU56.SCHED
46258 ± 17% -74.5% 11775 ± 5% softirqs.CPU56.TIMER
28703 ± 21% -94.7% 1508 ± 2% softirqs.CPU57.RCU
18009 ± 23% -83.9% 2905 ± 8% softirqs.CPU57.SCHED
45957 ± 18% -75.0% 11491 ± 3% softirqs.CPU57.TIMER
27503 ± 21% -94.5% 1514 ± 22% softirqs.CPU58.RCU
18444 ± 20% -81.6% 3399 ± 14% softirqs.CPU58.SCHED
47391 ± 20% -75.2% 11773 ± 4% softirqs.CPU58.TIMER
28356 ± 21% -94.1% 1681 ± 14% softirqs.CPU59.RCU
18172 ± 18% -84.2% 2871 ± 15% softirqs.CPU59.SCHED
47058 ± 18% -75.6% 11472 ± 6% softirqs.CPU59.TIMER
32712 ± 21% -92.3% 2509 ± 11% softirqs.CPU6.RCU
19308 ± 21% -82.8% 3317 ± 9% softirqs.CPU6.SCHED
66288 ± 25% -81.6% 12223 ± 6% softirqs.CPU6.TIMER
32168 ± 24% -95.3% 1501 ± 2% softirqs.CPU60.RCU
18584 ± 18% -85.2% 2746 ± 20% softirqs.CPU60.SCHED
47481 ± 18% -75.7% 11525 ± 6% softirqs.CPU60.TIMER
30453 ± 22% -90.2% 2989 ± 56% softirqs.CPU61.RCU
18686 ± 18% -85.8% 2655 ± 22% softirqs.CPU61.SCHED
50042 ± 22% -76.6% 11691 ± 5% softirqs.CPU61.TIMER
30580 ± 24% -94.0% 1846 ± 7% softirqs.CPU62.RCU
18292 ± 18% -86.4% 2492 ± 27% softirqs.CPU62.SCHED
46824 ± 19% -75.2% 11627 ± 2% softirqs.CPU62.TIMER
31793 ± 19% -95.6% 1402 ± 8% softirqs.CPU63.RCU
18089 ± 17% -83.9% 2907 ± 14% softirqs.CPU63.SCHED
47219 ± 17% -75.6% 11510 ± 4% softirqs.CPU63.TIMER
32333 ± 29% -90.1% 3201 ± 56% softirqs.CPU64.RCU
18330 ± 17% -85.6% 2643 ± 24% softirqs.CPU64.SCHED
45631 ± 17% -73.4% 12148 ± 5% softirqs.CPU64.TIMER
30285 ± 29% -95.9% 1248 ± 18% softirqs.CPU65.RCU
18423 ± 17% -83.8% 2979 ± 5% softirqs.CPU65.SCHED
49334 ± 21% -77.3% 11222 ± 2% softirqs.CPU65.TIMER
31483 ± 26% -94.7% 1679 ± 32% softirqs.CPU66.RCU
18350 ± 19% -83.9% 2953 ± 26% softirqs.CPU66.SCHED
46858 ± 18% -75.1% 11658 ± 3% softirqs.CPU66.TIMER
30388 ± 22% -95.6% 1338 ± 5% softirqs.CPU67.RCU
18446 ± 19% -85.5% 2668 ± 19% softirqs.CPU67.SCHED
49426 ± 22% -76.7% 11495 ± 4% softirqs.CPU67.TIMER
30256 ± 26% -95.1% 1476 ± 26% softirqs.CPU68.RCU
18127 ± 19% -86.1% 2522 ± 19% softirqs.CPU68.SCHED
47197 ± 17% -75.4% 11612 ± 7% softirqs.CPU68.TIMER
27610 ± 27% -93.2% 1864 ± 19% softirqs.CPU69.RCU
18502 ± 19% -86.5% 2491 ± 18% softirqs.CPU69.SCHED
47136 ± 21% -75.3% 11665 ± 6% softirqs.CPU69.TIMER
33062 ± 21% -92.9% 2357 ± 26% softirqs.CPU7.RCU
19330 ± 20% -84.0% 3088 ± 10% softirqs.CPU7.SCHED
63992 ± 23% -81.3% 11959 ± 5% softirqs.CPU7.TIMER
30725 ± 24% -90.5% 2919 ± 62% softirqs.CPU70.RCU
18410 ± 19% -86.0% 2572 ± 16% softirqs.CPU70.SCHED
45716 ± 18% -74.5% 11635 ± 4% softirqs.CPU70.TIMER
31738 ± 27% -96.1% 1223 ± 19% softirqs.CPU71.RCU
18528 ± 19% -87.1% 2393 ± 16% softirqs.CPU71.SCHED
46690 ± 18% -75.2% 11575 ± 3% softirqs.CPU71.TIMER
39101 ± 30% -90.3% 3805 ± 39% softirqs.CPU8.RCU
19596 ± 20% -76.3% 4640 ± 18% softirqs.CPU8.SCHED
66597 ± 25% -76.5% 15634 ± 26% softirqs.CPU8.TIMER
33648 ± 29% -93.3% 2253 softirqs.CPU9.RCU
19448 ± 21% -81.3% 3637 ± 13% softirqs.CPU9.SCHED
66104 ± 25% -81.5% 12215 ± 5% softirqs.CPU9.TIMER
2308563 ± 23% -92.6% 171957 ± 12% softirqs.RCU
1393678 ± 20% -83.4% 231487 ± 3% softirqs.SCHED
4127446 ± 22% -78.8% 876092 ± 2% softirqs.TIMER
143.50 ± 40% -84.4% 22.33 ± 75% interrupts.69:PCI-MSI.12589058-edge.eth3-TxRx-1
192.00 ± 61% -94.8% 10.00 ± 16% interrupts.70:PCI-MSI.12589059-edge.eth3-TxRx-2
105.25 ± 20% -73.1% 28.33 ± 94% interrupts.71:PCI-MSI.12589060-edge.eth3-TxRx-3
264.00 ±106% -94.4% 14.67 ± 17% interrupts.72:PCI-MSI.12589061-edge.eth3-TxRx-4
246.25 ± 51% -89.6% 25.67 ± 70% interrupts.73:PCI-MSI.12589062-edge.eth3-TxRx-5
135.75 ± 30% -82.6% 23.67 ± 78% interrupts.74:PCI-MSI.12589063-edge.eth3-TxRx-6
194309 ± 24% -69.8% 58683 ± 11% interrupts.CAL:Function_call_interrupts
1981 ± 20% -65.9% 675.00 ± 9% interrupts.CPU0.CAL:Function_call_interrupts
284946 ± 20% -88.1% 33839 ± 5% interrupts.CPU0.LOC:Local_timer_interrupts
552.00 ± 99% -100.0% 0.00 interrupts.CPU0.NMI:Non-maskable_interrupts
552.00 ± 99% -100.0% 0.00 interrupts.CPU0.PMI:Performance_monitoring_interrupts
1901 ± 22% -63.6% 691.33 ± 15% interrupts.CPU1.CAL:Function_call_interrupts
284845 ± 20% -88.2% 33562 ± 6% interrupts.CPU1.LOC:Local_timer_interrupts
499.25 ±109% -100.0% 0.00 interrupts.CPU1.NMI:Non-maskable_interrupts
499.25 ±109% -100.0% 0.00 interrupts.CPU1.PMI:Performance_monitoring_interrupts
1849 ± 19% -54.7% 837.33 ± 27% interrupts.CPU10.CAL:Function_call_interrupts
284556 ± 20% -88.1% 33988 ± 5% interrupts.CPU10.LOC:Local_timer_interrupts
151.50 ± 35% -100.0% 0.00 interrupts.CPU10.NMI:Non-maskable_interrupts
151.50 ± 35% -100.0% 0.00 interrupts.CPU10.PMI:Performance_monitoring_interrupts
1659 ± 14% -60.4% 657.33 ± 3% interrupts.CPU11.CAL:Function_call_interrupts
284538 ± 20% -88.1% 33824 ± 5% interrupts.CPU11.LOC:Local_timer_interrupts
170.00 ± 43% -100.0% 0.00 interrupts.CPU11.NMI:Non-maskable_interrupts
170.00 ± 43% -100.0% 0.00 interrupts.CPU11.PMI:Performance_monitoring_interrupts
1763 ± 6% -64.4% 628.33 ± 6% interrupts.CPU12.CAL:Function_call_interrupts
284719 ± 20% -88.2% 33574 ± 5% interrupts.CPU12.LOC:Local_timer_interrupts
200.25 ± 34% -100.0% 0.00 interrupts.CPU12.NMI:Non-maskable_interrupts
200.25 ± 34% -100.0% 0.00 interrupts.CPU12.PMI:Performance_monitoring_interrupts
1737 ± 9% -55.7% 769.33 ± 29% interrupts.CPU13.CAL:Function_call_interrupts
284593 ± 20% -88.2% 33557 ± 6% interrupts.CPU13.LOC:Local_timer_interrupts
283.00 ± 59% -100.0% 0.00 interrupts.CPU13.NMI:Non-maskable_interrupts
283.00 ± 59% -100.0% 0.00 interrupts.CPU13.PMI:Performance_monitoring_interrupts
1671 ± 13% -64.1% 599.67 ± 6% interrupts.CPU14.CAL:Function_call_interrupts
284472 ± 20% -88.2% 33428 ± 6% interrupts.CPU14.LOC:Local_timer_interrupts
230.50 ± 44% -99.9% 0.33 ±141% interrupts.CPU14.NMI:Non-maskable_interrupts
230.50 ± 44% -99.9% 0.33 ±141% interrupts.CPU14.PMI:Performance_monitoring_interrupts
1800 ± 6% -51.6% 872.00 ± 28% interrupts.CPU15.CAL:Function_call_interrupts
284375 ± 20% -88.3% 33355 ± 6% interrupts.CPU15.LOC:Local_timer_interrupts
151.75 ± 38% -100.0% 0.00 interrupts.CPU15.NMI:Non-maskable_interrupts
151.75 ± 38% -100.0% 0.00 interrupts.CPU15.PMI:Performance_monitoring_interrupts
5.50 ± 73% +5803.0% 324.67 ±118% interrupts.CPU15.RES:Rescheduling_interrupts
1645 ± 16% -52.8% 776.00 ± 34% interrupts.CPU16.CAL:Function_call_interrupts
284481 ± 20% -88.2% 33582 ± 4% interrupts.CPU16.LOC:Local_timer_interrupts
188.25 ± 31% -100.0% 0.00 interrupts.CPU16.NMI:Non-maskable_interrupts
188.25 ± 31% -100.0% 0.00 interrupts.CPU16.PMI:Performance_monitoring_interrupts
1648 ± 15% -63.8% 597.33 ± 9% interrupts.CPU17.CAL:Function_call_interrupts
284886 ± 20% -88.3% 33251 ± 5% interrupts.CPU17.LOC:Local_timer_interrupts
568.25 ±120% -100.0% 0.00 interrupts.CPU17.NMI:Non-maskable_interrupts
568.25 ±120% -100.0% 0.00 interrupts.CPU17.PMI:Performance_monitoring_interrupts
3484 ± 36% -74.2% 899.00 ± 36% interrupts.CPU18.CAL:Function_call_interrupts
284608 ± 20% -88.2% 33724 ± 5% interrupts.CPU18.LOC:Local_timer_interrupts
611.75 ±114% -100.0% 0.00 interrupts.CPU18.NMI:Non-maskable_interrupts
611.75 ±114% -100.0% 0.00 interrupts.CPU18.PMI:Performance_monitoring_interrupts
3470 ± 26% -77.8% 771.00 ± 16% interrupts.CPU19.CAL:Function_call_interrupts
284822 ± 20% -88.2% 33555 ± 4% interrupts.CPU19.LOC:Local_timer_interrupts
741.00 ±125% -100.0% 0.00 interrupts.CPU19.NMI:Non-maskable_interrupts
741.00 ±125% -100.0% 0.00 interrupts.CPU19.PMI:Performance_monitoring_interrupts
143.50 ± 40% -84.4% 22.33 ± 75% interrupts.CPU2.69:PCI-MSI.12589058-edge.eth3-TxRx-1
1872 ± 21% -47.5% 983.00 ± 20% interrupts.CPU2.CAL:Function_call_interrupts
284688 ± 20% -88.3% 33217 ± 6% interrupts.CPU2.LOC:Local_timer_interrupts
152.00 ± 32% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
152.00 ± 32% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
3306 ± 35% -73.9% 864.00 ± 43% interrupts.CPU20.CAL:Function_call_interrupts
284777 ± 20% -88.2% 33517 ± 4% interrupts.CPU20.LOC:Local_timer_interrupts
943.25 ±137% -100.0% 0.00 interrupts.CPU20.NMI:Non-maskable_interrupts
943.25 ±137% -100.0% 0.00 interrupts.CPU20.PMI:Performance_monitoring_interrupts
3531 ± 42% -76.9% 816.00 ± 33% interrupts.CPU21.CAL:Function_call_interrupts
284825 ± 20% -88.3% 33392 ± 4% interrupts.CPU21.LOC:Local_timer_interrupts
170.00 ± 27% -100.0% 0.00 interrupts.CPU21.NMI:Non-maskable_interrupts
170.00 ± 27% -100.0% 0.00 interrupts.CPU21.PMI:Performance_monitoring_interrupts
3357 ± 25% -73.2% 898.00 ± 44% interrupts.CPU22.CAL:Function_call_interrupts
284958 ± 20% -88.2% 33698 ± 4% interrupts.CPU22.LOC:Local_timer_interrupts
172.50 ± 34% -100.0% 0.00 interrupts.CPU22.NMI:Non-maskable_interrupts
172.50 ± 34% -100.0% 0.00 interrupts.CPU22.PMI:Performance_monitoring_interrupts
3454 ± 33% -77.6% 774.33 ± 48% interrupts.CPU23.CAL:Function_call_interrupts
284646 ± 20% -88.2% 33619 ± 4% interrupts.CPU23.LOC:Local_timer_interrupts
205.00 ± 40% -100.0% 0.00 interrupts.CPU23.NMI:Non-maskable_interrupts
205.00 ± 40% -100.0% 0.00 interrupts.CPU23.PMI:Performance_monitoring_interrupts
3063 ± 31% -71.0% 887.67 ± 41% interrupts.CPU24.CAL:Function_call_interrupts
284624 ± 20% -88.3% 33398 ± 4% interrupts.CPU24.LOC:Local_timer_interrupts
299.50 ± 55% -99.8% 0.67 ±141% interrupts.CPU24.NMI:Non-maskable_interrupts
299.50 ± 55% -99.8% 0.67 ±141% interrupts.CPU24.PMI:Performance_monitoring_interrupts
2687 ± 18% -68.2% 854.33 ± 42% interrupts.CPU25.CAL:Function_call_interrupts
284821 ± 20% -88.3% 33398 ± 4% interrupts.CPU25.LOC:Local_timer_interrupts
147.50 ± 20% -99.5% 0.67 ± 70% interrupts.CPU25.NMI:Non-maskable_interrupts
147.50 ± 20% -99.5% 0.67 ± 70% interrupts.CPU25.PMI:Performance_monitoring_interrupts
284698 ± 20% -88.2% 33657 ± 6% interrupts.CPU26.LOC:Local_timer_interrupts
207.75 ± 22% -100.0% 0.00 interrupts.CPU26.NMI:Non-maskable_interrupts
207.75 ± 22% -100.0% 0.00 interrupts.CPU26.PMI:Performance_monitoring_interrupts
3079 ± 29% -79.4% 633.33 ± 12% interrupts.CPU27.CAL:Function_call_interrupts
284642 ± 20% -88.3% 33407 ± 6% interrupts.CPU27.LOC:Local_timer_interrupts
244.25 ± 42% -100.0% 0.00 interrupts.CPU27.NMI:Non-maskable_interrupts
244.25 ± 42% -100.0% 0.00 interrupts.CPU27.PMI:Performance_monitoring_interrupts
295.50 ± 48% -82.5% 51.67 ±103% interrupts.CPU27.TLB:TLB_shootdowns
2523 ± 33% -73.5% 668.67 ± 5% interrupts.CPU28.CAL:Function_call_interrupts
284601 ± 20% -88.2% 33635 ± 4% interrupts.CPU28.LOC:Local_timer_interrupts
196.00 ± 34% -100.0% 0.00 interrupts.CPU28.NMI:Non-maskable_interrupts
196.00 ± 34% -100.0% 0.00 interrupts.CPU28.PMI:Performance_monitoring_interrupts
2827 ± 34% -74.5% 721.67 ± 12% interrupts.CPU29.CAL:Function_call_interrupts
284946 ± 20% -88.4% 33151 ± 5% interrupts.CPU29.LOC:Local_timer_interrupts
201.50 ± 30% -100.0% 0.00 interrupts.CPU29.NMI:Non-maskable_interrupts
201.50 ± 30% -100.0% 0.00 interrupts.CPU29.PMI:Performance_monitoring_interrupts
21.00 ± 22% +647.6% 157.00 ±113% interrupts.CPU29.RES:Rescheduling_interrupts
192.00 ± 61% -94.8% 10.00 ± 16% interrupts.CPU3.70:PCI-MSI.12589059-edge.eth3-TxRx-2
1839 ± 24% -59.2% 751.00 ± 24% interrupts.CPU3.CAL:Function_call_interrupts
284621 ± 20% -88.2% 33619 ± 6% interrupts.CPU3.LOC:Local_timer_interrupts
405.00 ±110% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
405.00 ±110% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
3440 ± 36% -78.1% 753.67 ± 8% interrupts.CPU30.CAL:Function_call_interrupts
284824 ± 20% -88.2% 33596 ± 5% interrupts.CPU30.LOC:Local_timer_interrupts
273.75 ± 34% -100.0% 0.00 interrupts.CPU30.NMI:Non-maskable_interrupts
273.75 ± 34% -100.0% 0.00 interrupts.CPU30.PMI:Performance_monitoring_interrupts
3467 ± 40% -82.6% 603.33 ± 9% interrupts.CPU31.CAL:Function_call_interrupts
284780 ± 20% -88.3% 33370 ± 4% interrupts.CPU31.LOC:Local_timer_interrupts
215.00 ± 38% -100.0% 0.00 interrupts.CPU31.NMI:Non-maskable_interrupts
215.00 ± 38% -100.0% 0.00 interrupts.CPU31.PMI:Performance_monitoring_interrupts
3950 ± 39% -82.4% 696.33 ± 9% interrupts.CPU32.CAL:Function_call_interrupts
284694 ± 20% -88.3% 33367 ± 4% interrupts.CPU32.LOC:Local_timer_interrupts
305.75 ± 47% -100.0% 0.00 interrupts.CPU32.NMI:Non-maskable_interrupts
305.75 ± 47% -100.0% 0.00 interrupts.CPU32.PMI:Performance_monitoring_interrupts
7621 ±127% -91.8% 622.33 ± 9% interrupts.CPU33.CAL:Function_call_interrupts
284927 ± 20% -88.3% 33343 ± 4% interrupts.CPU33.LOC:Local_timer_interrupts
287.25 ± 47% -100.0% 0.00 interrupts.CPU33.NMI:Non-maskable_interrupts
287.25 ± 47% -100.0% 0.00 interrupts.CPU33.PMI:Performance_monitoring_interrupts
6481 ±166% -98.7% 82.33 ± 76% interrupts.CPU33.TLB:TLB_shootdowns
3102 ± 49% -83.7% 506.00 ± 46% interrupts.CPU34.CAL:Function_call_interrupts
284717 ± 20% -88.2% 33564 ± 4% interrupts.CPU34.LOC:Local_timer_interrupts
228.50 ± 38% -100.0% 0.00 interrupts.CPU34.NMI:Non-maskable_interrupts
228.50 ± 38% -100.0% 0.00 interrupts.CPU34.PMI:Performance_monitoring_interrupts
7056 ± 99% -90.7% 653.33 ± 11% interrupts.CPU35.CAL:Function_call_interrupts
284425 ± 20% -88.2% 33439 ± 4% interrupts.CPU35.LOC:Local_timer_interrupts
250.00 ± 51% -100.0% 0.00 interrupts.CPU35.NMI:Non-maskable_interrupts
250.00 ± 51% -100.0% 0.00 interrupts.CPU35.PMI:Performance_monitoring_interrupts
1880 ± 13% -63.5% 685.67 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
284607 ± 20% -88.3% 33236 ± 5% interrupts.CPU36.LOC:Local_timer_interrupts
317.50 ± 48% -100.0% 0.00 interrupts.CPU36.NMI:Non-maskable_interrupts
317.50 ± 48% -100.0% 0.00 interrupts.CPU36.PMI:Performance_monitoring_interrupts
1843 ± 7% -65.9% 628.33 ± 15% interrupts.CPU37.CAL:Function_call_interrupts
284781 ± 20% -88.3% 33420 ± 5% interrupts.CPU37.LOC:Local_timer_interrupts
710.25 ±118% -100.0% 0.00 interrupts.CPU37.NMI:Non-maskable_interrupts
710.25 ±118% -100.0% 0.00 interrupts.CPU37.PMI:Performance_monitoring_interrupts
1719 ± 12% -63.7% 624.33 ± 14% interrupts.CPU38.CAL:Function_call_interrupts
284571 ± 20% -88.3% 33395 ± 6% interrupts.CPU38.LOC:Local_timer_interrupts
186.00 ± 29% -100.0% 0.00 interrupts.CPU38.NMI:Non-maskable_interrupts
186.00 ± 29% -100.0% 0.00 interrupts.CPU38.PMI:Performance_monitoring_interrupts
1810 ± 10% -65.4% 626.00 ± 8% interrupts.CPU39.CAL:Function_call_interrupts
284611 ± 20% -88.3% 33268 ± 6% interrupts.CPU39.LOC:Local_timer_interrupts
546.25 ±117% -99.9% 0.67 ± 70% interrupts.CPU39.NMI:Non-maskable_interrupts
546.25 ±117% -99.9% 0.67 ± 70% interrupts.CPU39.PMI:Performance_monitoring_interrupts
105.25 ± 20% -73.1% 28.33 ± 94% interrupts.CPU4.71:PCI-MSI.12589060-edge.eth3-TxRx-3
1886 ± 20% -51.6% 914.00 ± 28% interrupts.CPU4.CAL:Function_call_interrupts
284385 ± 20% -88.1% 33728 ± 5% interrupts.CPU4.LOC:Local_timer_interrupts
542.50 ±117% -100.0% 0.00 interrupts.CPU4.NMI:Non-maskable_interrupts
542.50 ±117% -100.0% 0.00 interrupts.CPU4.PMI:Performance_monitoring_interrupts
1793 ± 7% -65.7% 614.33 ± 12% interrupts.CPU40.CAL:Function_call_interrupts
284529 ± 20% -88.3% 33390 ± 6% interrupts.CPU40.LOC:Local_timer_interrupts
797.00 ±125% -100.0% 0.00 interrupts.CPU40.NMI:Non-maskable_interrupts
797.00 ±125% -100.0% 0.00 interrupts.CPU40.PMI:Performance_monitoring_interrupts
416.50 ±163% -97.8% 9.33 ±111% interrupts.CPU40.RES:Rescheduling_interrupts
1847 ± 15% -63.7% 669.67 ± 22% interrupts.CPU41.CAL:Function_call_interrupts
284478 ± 20% -88.3% 33379 ± 6% interrupts.CPU41.LOC:Local_timer_interrupts
949.25 ± 78% -99.9% 0.67 ± 70% interrupts.CPU41.NMI:Non-maskable_interrupts
949.25 ± 78% -99.9% 0.67 ± 70% interrupts.CPU41.PMI:Performance_monitoring_interrupts
1773 ± 10% -65.0% 621.00 ± 13% interrupts.CPU42.CAL:Function_call_interrupts
284344 ± 20% -88.2% 33447 ± 6% interrupts.CPU42.LOC:Local_timer_interrupts
260.75 ± 30% -100.0% 0.00 interrupts.CPU42.NMI:Non-maskable_interrupts
260.75 ± 30% -100.0% 0.00 interrupts.CPU42.PMI:Performance_monitoring_interrupts
1871 ± 17% -63.7% 678.67 ± 25% interrupts.CPU43.CAL:Function_call_interrupts
284513 ± 20% -88.2% 33432 ± 6% interrupts.CPU43.LOC:Local_timer_interrupts
751.50 ±135% -100.0% 0.33 ±141% interrupts.CPU43.NMI:Non-maskable_interrupts
751.50 ±135% -100.0% 0.33 ±141% interrupts.CPU43.PMI:Performance_monitoring_interrupts
1777 ± 13% -62.5% 667.00 ± 22% interrupts.CPU44.CAL:Function_call_interrupts
284355 ± 20% -88.3% 33284 ± 4% interrupts.CPU44.LOC:Local_timer_interrupts
312.00 ± 47% -100.0% 0.00 interrupts.CPU44.NMI:Non-maskable_interrupts
312.00 ± 47% -100.0% 0.00 interrupts.CPU44.PMI:Performance_monitoring_interrupts
1786 ± 9% -68.7% 559.33 interrupts.CPU45.CAL:Function_call_interrupts
284581 ± 20% -88.2% 33582 ± 5% interrupts.CPU45.LOC:Local_timer_interrupts
314.25 ± 59% -100.0% 0.00 interrupts.CPU45.NMI:Non-maskable_interrupts
314.25 ± 59% -100.0% 0.00 interrupts.CPU45.PMI:Performance_monitoring_interrupts
4338 ±171% -99.9% 4.00 ±124% interrupts.CPU45.RES:Rescheduling_interrupts
1764 ± 8% -59.4% 715.67 ± 27% interrupts.CPU46.CAL:Function_call_interrupts
284541 ± 20% -88.2% 33684 ± 5% interrupts.CPU46.LOC:Local_timer_interrupts
195.75 ± 41% -99.7% 0.67 ±141% interrupts.CPU46.NMI:Non-maskable_interrupts
195.75 ± 41% -99.7% 0.67 ±141% interrupts.CPU46.PMI:Performance_monitoring_interrupts
1664 ± 16% -58.6% 689.33 ± 19% interrupts.CPU47.CAL:Function_call_interrupts
284290 ± 20% -88.1% 33829 ± 5% interrupts.CPU47.LOC:Local_timer_interrupts
180.75 ± 30% -98.7% 2.33 ±141% interrupts.CPU47.NMI:Non-maskable_interrupts
180.75 ± 30% -98.7% 2.33 ±141% interrupts.CPU47.PMI:Performance_monitoring_interrupts
1661 ± 16% -58.9% 683.00 ± 25% interrupts.CPU48.CAL:Function_call_interrupts
284400 ± 20% -88.2% 33626 ± 6% interrupts.CPU48.LOC:Local_timer_interrupts
196.00 ± 30% -100.0% 0.00 interrupts.CPU48.NMI:Non-maskable_interrupts
196.00 ± 30% -100.0% 0.00 interrupts.CPU48.PMI:Performance_monitoring_interrupts
1729 ± 9% -64.4% 615.33 ± 12% interrupts.CPU49.CAL:Function_call_interrupts
284324 ± 20% -88.1% 33754 ± 5% interrupts.CPU49.LOC:Local_timer_interrupts
218.50 ± 15% -100.0% 0.00 interrupts.CPU49.NMI:Non-maskable_interrupts
218.50 ± 15% -100.0% 0.00 interrupts.CPU49.PMI:Performance_monitoring_interrupts
264.00 ±106% -94.4% 14.67 ± 17% interrupts.CPU5.72:PCI-MSI.12589061-edge.eth3-TxRx-4
1872 ± 23% -60.8% 734.00 ± 29% interrupts.CPU5.CAL:Function_call_interrupts
284343 ± 20% -88.3% 33274 ± 5% interrupts.CPU5.LOC:Local_timer_interrupts
677.75 ± 72% -100.0% 0.00 interrupts.CPU5.NMI:Non-maskable_interrupts
677.75 ± 72% -100.0% 0.00 interrupts.CPU5.PMI:Performance_monitoring_interrupts
1825 ± 6% -61.4% 704.00 ± 28% interrupts.CPU50.CAL:Function_call_interrupts
284502 ± 20% -88.2% 33464 ± 6% interrupts.CPU50.LOC:Local_timer_interrupts
230.75 ± 18% -99.9% 0.33 ±141% interrupts.CPU50.NMI:Non-maskable_interrupts
230.75 ± 18% -99.9% 0.33 ±141% interrupts.CPU50.PMI:Performance_monitoring_interrupts
1692 ± 18% -59.6% 683.33 ± 24% interrupts.CPU51.CAL:Function_call_interrupts
284635 ± 20% -88.2% 33671 ± 5% interrupts.CPU51.LOC:Local_timer_interrupts
171.25 ± 47% -100.0% 0.00 interrupts.CPU51.NMI:Non-maskable_interrupts
171.25 ± 47% -100.0% 0.00 interrupts.CPU51.PMI:Performance_monitoring_interrupts
1715 ± 10% -59.1% 701.33 ± 25% interrupts.CPU52.CAL:Function_call_interrupts
284601 ± 20% -88.3% 33394 ± 6% interrupts.CPU52.LOC:Local_timer_interrupts
198.50 ± 16% -100.0% 0.00 interrupts.CPU52.NMI:Non-maskable_interrupts
198.50 ± 16% -100.0% 0.00 interrupts.CPU52.PMI:Performance_monitoring_interrupts
1809 ± 6% -63.9% 652.33 ± 14% interrupts.CPU53.CAL:Function_call_interrupts
284563 ± 20% -88.2% 33441 ± 6% interrupts.CPU53.LOC:Local_timer_interrupts
482.25 ± 98% -100.0% 0.00 interrupts.CPU53.NMI:Non-maskable_interrupts
482.25 ± 98% -100.0% 0.00 interrupts.CPU53.PMI:Performance_monitoring_interrupts
3422 ± 43% -81.1% 646.00 ± 7% interrupts.CPU54.CAL:Function_call_interrupts
284884 ± 20% -88.3% 33384 ± 4% interrupts.CPU54.LOC:Local_timer_interrupts
307.75 ± 57% -99.9% 0.33 ±141% interrupts.CPU54.NMI:Non-maskable_interrupts
307.75 ± 57% -99.9% 0.33 ±141% interrupts.CPU54.PMI:Performance_monitoring_interrupts
3743 ± 40% -83.7% 608.33 ± 14% interrupts.CPU55.CAL:Function_call_interrupts
284545 ± 20% -88.3% 33392 ± 4% interrupts.CPU55.LOC:Local_timer_interrupts
396.25 ± 56% -99.9% 0.33 ±141% interrupts.CPU55.NMI:Non-maskable_interrupts
396.25 ± 56% -99.9% 0.33 ±141% interrupts.CPU55.PMI:Performance_monitoring_interrupts
18.75 ± 63% +10557.8% 1998 ±139% interrupts.CPU55.RES:Rescheduling_interrupts
2990 ± 40% -77.7% 667.33 ± 11% interrupts.CPU56.CAL:Function_call_interrupts
284602 ± 20% -88.3% 33389 ± 4% interrupts.CPU56.LOC:Local_timer_interrupts
468.75 ± 72% -99.9% 0.67 ±141% interrupts.CPU56.NMI:Non-maskable_interrupts
468.75 ± 72% -99.9% 0.67 ±141% interrupts.CPU56.PMI:Performance_monitoring_interrupts
3768 ± 41% -82.1% 676.00 ± 11% interrupts.CPU57.CAL:Function_call_interrupts
284795 ± 20% -88.3% 33329 ± 4% interrupts.CPU57.LOC:Local_timer_interrupts
221.50 ± 44% -99.8% 0.33 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts
221.50 ± 44% -99.8% 0.33 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts
2742 ± 33% -76.8% 636.67 ± 9% interrupts.CPU58.CAL:Function_call_interrupts
284604 ± 20% -88.2% 33549 ± 4% interrupts.CPU58.LOC:Local_timer_interrupts
226.00 ± 57% -100.0% 0.00 interrupts.CPU58.NMI:Non-maskable_interrupts
226.00 ± 57% -100.0% 0.00 interrupts.CPU58.PMI:Performance_monitoring_interrupts
2936 ± 35% -74.8% 738.67 ± 19% interrupts.CPU59.CAL:Function_call_interrupts
284808 ± 20% -88.2% 33465 ± 4% interrupts.CPU59.LOC:Local_timer_interrupts
281.00 ± 60% -100.0% 0.00 interrupts.CPU59.NMI:Non-maskable_interrupts
281.00 ± 60% -100.0% 0.00 interrupts.CPU59.PMI:Performance_monitoring_interrupts
246.25 ± 51% -89.6% 25.67 ± 70% interrupts.CPU6.73:PCI-MSI.12589062-edge.eth3-TxRx-5
1803 ± 10% -60.4% 714.33 ± 21% interrupts.CPU6.CAL:Function_call_interrupts
284294 ± 20% -88.3% 33257 ± 5% interrupts.CPU6.LOC:Local_timer_interrupts
283.50 ± 57% -100.0% 0.00 interrupts.CPU6.NMI:Non-maskable_interrupts
283.50 ± 57% -100.0% 0.00 interrupts.CPU6.PMI:Performance_monitoring_interrupts
3451 ± 45% -77.7% 770.33 ± 24% interrupts.CPU60.CAL:Function_call_interrupts
284660 ± 20% -88.3% 33377 ± 4% interrupts.CPU60.LOC:Local_timer_interrupts
218.50 ± 46% -99.8% 0.33 ±141% interrupts.CPU60.NMI:Non-maskable_interrupts
218.50 ± 46% -99.8% 0.33 ±141% interrupts.CPU60.PMI:Performance_monitoring_interrupts
2634 ± 36% -70.9% 767.33 ± 20% interrupts.CPU61.CAL:Function_call_interrupts
284804 ± 20% -88.3% 33392 ± 4% interrupts.CPU61.LOC:Local_timer_interrupts
177.50 ± 48% -99.8% 0.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts
177.50 ± 48% -99.8% 0.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts
2755 ± 45% -72.1% 768.67 ± 19% interrupts.CPU62.CAL:Function_call_interrupts
284635 ± 20% -88.3% 33372 ± 4% interrupts.CPU62.LOC:Local_timer_interrupts
208.50 ± 56% -100.0% 0.00 interrupts.CPU62.NMI:Non-maskable_interrupts
208.50 ± 56% -100.0% 0.00 interrupts.CPU62.PMI:Performance_monitoring_interrupts
3960 ± 43% -82.3% 702.67 ± 15% interrupts.CPU63.CAL:Function_call_interrupts
284765 ± 20% -88.3% 33420 ± 4% interrupts.CPU63.LOC:Local_timer_interrupts
288.00 ± 65% -100.0% 0.00 interrupts.CPU63.NMI:Non-maskable_interrupts
288.00 ± 65% -100.0% 0.00 interrupts.CPU63.PMI:Performance_monitoring_interrupts
4003 ± 41% -82.2% 713.33 ± 18% interrupts.CPU64.CAL:Function_call_interrupts
284828 ± 20% -88.3% 33462 ± 5% interrupts.CPU64.LOC:Local_timer_interrupts
190.25 ± 50% -100.0% 0.00 interrupts.CPU64.NMI:Non-maskable_interrupts
190.25 ± 50% -100.0% 0.00 interrupts.CPU64.PMI:Performance_monitoring_interrupts
284682 ± 20% -88.2% 33648 ± 5% interrupts.CPU65.LOC:Local_timer_interrupts
200.50 ± 65% -100.0% 0.00 interrupts.CPU65.NMI:Non-maskable_interrupts
200.50 ± 65% -100.0% 0.00 interrupts.CPU65.PMI:Performance_monitoring_interrupts
3508 ± 37% -78.8% 744.67 ± 17% interrupts.CPU66.CAL:Function_call_interrupts
284752 ± 20% -88.2% 33619 ± 5% interrupts.CPU66.LOC:Local_timer_interrupts
232.75 ± 50% -100.0% 0.00 interrupts.CPU66.NMI:Non-maskable_interrupts
232.75 ± 50% -100.0% 0.00 interrupts.CPU66.PMI:Performance_monitoring_interrupts
3081 ± 40% -75.3% 760.00 ± 20% interrupts.CPU67.CAL:Function_call_interrupts
284736 ± 20% -88.3% 33398 ± 4% interrupts.CPU67.LOC:Local_timer_interrupts
259.50 ± 53% -100.0% 0.00 interrupts.CPU67.NMI:Non-maskable_interrupts
259.50 ± 53% -100.0% 0.00 interrupts.CPU67.PMI:Performance_monitoring_interrupts
3458 ± 37% -77.6% 773.67 ± 20% interrupts.CPU68.CAL:Function_call_interrupts
284880 ± 20% -88.3% 33363 ± 4% interrupts.CPU68.LOC:Local_timer_interrupts
262.25 ± 55% -100.0% 0.00 interrupts.CPU68.NMI:Non-maskable_interrupts
262.25 ± 55% -100.0% 0.00 interrupts.CPU68.PMI:Performance_monitoring_interrupts
3247 ± 36% -76.5% 764.33 ± 16% interrupts.CPU69.CAL:Function_call_interrupts
284772 ± 20% -88.4% 33084 ± 6% interrupts.CPU69.LOC:Local_timer_interrupts
259.25 ± 45% -99.9% 0.33 ±141% interrupts.CPU69.NMI:Non-maskable_interrupts
259.25 ± 45% -99.9% 0.33 ±141% interrupts.CPU69.PMI:Performance_monitoring_interrupts
135.75 ± 30% -82.6% 23.67 ± 78% interrupts.CPU7.74:PCI-MSI.12589063-edge.eth3-TxRx-6
1784 ± 25% -65.2% 621.00 ± 9% interrupts.CPU7.CAL:Function_call_interrupts
284388 ± 20% -88.3% 33183 ± 5% interrupts.CPU7.LOC:Local_timer_interrupts
213.75 ± 68% -100.0% 0.00 interrupts.CPU7.NMI:Non-maskable_interrupts
213.75 ± 68% -100.0% 0.00 interrupts.CPU7.PMI:Performance_monitoring_interrupts
3907 ± 41% -80.3% 769.33 ± 19% interrupts.CPU70.CAL:Function_call_interrupts
284422 ± 20% -88.3% 33403 ± 4% interrupts.CPU70.LOC:Local_timer_interrupts
240.25 ± 47% -99.9% 0.33 ±141% interrupts.CPU70.NMI:Non-maskable_interrupts
240.25 ± 47% -99.9% 0.33 ±141% interrupts.CPU70.PMI:Performance_monitoring_interrupts
6753 ± 35% -87.2% 867.33 ± 5% interrupts.CPU71.CAL:Function_call_interrupts
284231 ± 20% -88.3% 33367 ± 4% interrupts.CPU71.LOC:Local_timer_interrupts
303.25 ± 58% -99.9% 0.33 ±141% interrupts.CPU71.NMI:Non-maskable_interrupts
303.25 ± 58% -99.9% 0.33 ±141% interrupts.CPU71.PMI:Performance_monitoring_interrupts
3778 ±154% -91.5% 319.33 ± 13% interrupts.CPU71.TLB:TLB_shootdowns
1741 ± 7% -66.4% 585.33 ± 5% interrupts.CPU8.CAL:Function_call_interrupts
284450 ± 20% -88.1% 33716 ± 4% interrupts.CPU8.LOC:Local_timer_interrupts
242.00 ± 46% -100.0% 0.00 interrupts.CPU8.NMI:Non-maskable_interrupts
242.00 ± 46% -100.0% 0.00 interrupts.CPU8.PMI:Performance_monitoring_interrupts
1827 ± 7% -61.9% 696.67 ± 7% interrupts.CPU9.CAL:Function_call_interrupts
284698 ± 20% -88.1% 33873 ± 5% interrupts.CPU9.LOC:Local_timer_interrupts
242.50 ± 48% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts
242.50 ± 48% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts
20493273 ± 20% -88.2% 2410789 ± 5% interrupts.LOC:Local_timer_interrupts
23214 ± 24% -100.0% 10.00 ± 37% interrupts.NMI:Non-maskable_interrupts
23214 ± 24% -100.0% 10.00 ± 37% interrupts.PMI:Performance_monitoring_interrupts
42829 ± 45% -66.4% 14402 ± 11% interrupts.RES:Rescheduling_interrupts
vm-scalability.throughput
4e+07 +-+---------------------------------------------------------------+
| O O O O |
3.5e+07 O-+ O O O O O O O O O O O |
3e+07 +-+ |
| |
2.5e+07 +-+ |
| |
2e+07 +-+ + + |
| +.. :: :: |
1.5e+07 +-+ : : : : : |
1e+07 +-+ .+ .+..O : +. : : : O .+.O |
| +. + .+ .+..+ + : + +..+.+. +..+.+..+.|
5e+06 +-+ : +. +.+ + : |
| : +: |
0 +-O--O-O--O----O-O---------O--------------------------------------+
vm-scalability.time.system_time
250 +-+-----------------------------------------------------------O-------+
O O O O O O O O O O |
| O O O O |
200 +-+ O |
| |
| |
150 +-+ +.. +.. |
| .. +.+..+.. : .+. |
100 +-+ +.+ + + : +. +..+.+..|
| : + + +. .+ +..+..+.+..+ |
| : + +..+ : : |
50 +-+ : : : |
| : : : |
| : :: |
0 +-+O-O--O-O-----O-O--O-------O-------------------O----------O---------+
vm-scalability.time.percent_of_cpu_this_job_got
2500 +-+------------------------------------------------------------------+
| O O O |
O O O O O O O O O O O O O |
2000 +-+ |
| |
| |
1500 +-+ |
| |
1000 +-+ |
| |
| |
500 +-+ +. + +.. |
| .+.. .+. .. +.. + + .. .+.. |
| .+ +.+. +..+..+.+ +. + + + +.+..+.+..+.+..|
0 +-+O-O--O-O----O--O-O-------O--------------------O---------O---------+
vm-scalability.median
1.2e+06 +-+---------------------------------------------------------------+
| |
1e+06 +-+ O O O O O O O |
O O O O O O O O |
| O |
800000 +-+ |
| |
600000 +-+ |
| |
400000 +-+ +.. |
| : +.. + |
| : +. : + + |
200000 +-+ .+. .+.. .+ +.. : + +..+. .+. .|
| .+. +..+ O.+.+. O : +. +..+.+..+ |
0 +-O--O-O--O----O-O----------------------------O---------O---------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 10 months
[static_assert] d94e96122a: bpf_jit_disasm.c: undefined reference to `assert'
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: d94e96122a4778c029ef08ffc0a8b315a48a30f5 ("[PATCH] static_assert: move before people start using it")
url: https://github.com/0day-ci/linux/commits/Alexey-Dobriyan/static_assert-mo...
in testcase: bpf_offload
with following parameters:
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
/usr/src/perf_selftests-x86_64-rhel-7.6-d94e96122a4778c029ef08ffc0a8b315a48a30f5/tools/bpf/bpf_jit_disasm.c: In function ‘get_exec_path’:
/usr/src/perf_selftests-x86_64-rhel-7.6-d94e96122a4778c029ef08ffc0a8b315a48a30f5/tools/bpf/bpf_jit_disasm.c:44:2: warning: implicit declaration of function ‘assert’ [-Wimplicit-function-declaration]
assert(path);
^~~~~~
bpf_jit_disasm.o: In function `get_asm_insns':
bpf_jit_disasm.c:(.text+0x78): undefined reference to `assert'
bpf_jit_disasm.c:(.text+0xb6): undefined reference to `assert'
bpf_jit_disasm.c:(.text+0xcc): undefined reference to `assert'
bpf_jit_disasm.c:(.text+0x134): undefined reference to `assert'
bpf_jit_disasm.o: In function `main':
bpf_jit_disasm.c:(.text.startup+0x1b9): undefined reference to `assert'
bpf_jit_disasm.o:bpf_jit_disasm.c:(.text.startup+0x1e7): more undefined references to `assert' follow
collect2: error: ld returned 1 exit status
make: *** [bpf_jit_disasm] Error 1
failed to prepare_for_bpf
/lkp/lkp/src/lib/debug.sh:7: die
/lkp/lkp/src/tests/bpf_offload:35: main
:0:
To reproduce:
# build kernel
cd linux
cp config-5.0.0-10154-gd94e961 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months
c1d8e1d13e [ 11.190345] BUG: unable to handle kernel NULL pointer dereference at 00000094
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git mount-api-viro
commit c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18
Author: David Howells <dhowells(a)redhat.com>
AuthorDate: Mon Jan 21 14:04:22 2019 +0000
Commit: David Howells <dhowells(a)redhat.com>
CommitDate: Thu Mar 21 09:47:14 2019 +0000
vfs: Convert debugfs to fs_context
Signed-off-by: David Howells <dhowells(a)redhat.com>
cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
cc: "Rafael J. Wysocki" <rafael(a)kernel.org>
661e6721f4 vfs: Add a single-or-reconfig keying to vfs_get_super()
c1d8e1d13e vfs: Convert debugfs to fs_context
914ae609fc vfs: Kill off mount_single()
+------------------------------------------+------------+------------+------------+
| | 661e6721f4 | c1d8e1d13e | 914ae609fc |
+------------------------------------------+------------+------------+------------+
| boot_successes | 73 | 0 | 2 |
| boot_failures | 0 | 13 | 13 |
| BUG:unable_to_handle_kernel | 0 | 13 | 13 |
| Oops:#[##] | 0 | 13 | 13 |
| EIP:reconfigure_super | 0 | 13 | 13 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 12 | 13 |
+------------------------------------------+------------+------------+------------+
[ 11.020260] random: init: uninitialized urandom read (12 bytes read)
Kernel tests: Boot OK!
[ 11.089703] init: plymouth main process (134) killed by ABRT signal
[ 11.126682] random: trinity: uninitialized urandom read (4 bytes read)
[ 11.176076] random: mountall: uninitialized urandom read (12 bytes read)
[ 11.190345] BUG: unable to handle kernel NULL pointer dereference at 00000094
[ 11.191350] #PF error: [normal kernel read fault]
[ 11.192033] *pde = 00000000
[ 11.192446] Oops: 0000 [#1]
[ 11.192887] CPU: 0 PID: 154 Comm: mount Not tainted 5.1.0-rc1-00054-gc1d8e1d #1
[ 11.193881] EIP: reconfigure_super+0x21/0x1f0
[ 11.194470] Code: d8 5b 5e 5f 5d c3 8d 76 00 55 89 e5 83 ec 24 89 5d f4 89 75 f8 89 7d fc e8 d0 f0 e7 ff 8b 50 14 f7 40 38 ae ff 7f fd 8b 78 34 <8b> b2 94 00 00 00 0f 85 6b 01 00 00 8b 96 b8 01 00 00 85 d2 74 11
[ 11.197049] EAX: 5f22c8c8 EBX: 00000000 ECX: 51234c8a EDX: 00000000
[ 11.197929] ESI: 5f22c8c8 EDI: 00000000 EBP: 5f225ea8 ESP: 5f225e84
[ 11.198808] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 11.199747] CR0: 80050033 CR2: 00000094 CR3: 1f29e000 CR4: 00000690
[ 11.200610] Call Trace:
[ 11.200988] ? sget_fc+0x227/0x240
[ 11.201448] ? get_anon_bdev+0x50/0x50
[ 11.201997] ? vfs_get_super+0x9d/0x100
[ 11.202536] ? debugfs_get_tree+0x17/0x20
[ 11.203117] ? vfs_get_tree+0x5a/0x120
[ 11.203650] ? vfs_parse_fs_string+0xa0/0xa0
[ 11.204257] ? do_mount+0x97b/0xce0
[ 11.204766] ? __copy_user_ll+0xd/0x10
[ 11.205289] ? copy_mount_options+0xe/0x250
[ 11.205889] ? ksys_mount+0x96/0xc0
[ 11.206361] ? sys_mount+0x26/0x30
[ 11.206866] ? do_fast_syscall_32+0x146/0x2c0
[ 11.207472] ? entry_SYSENTER_32+0x7e/0xe3
[ 11.208068] CR2: 0000000000000094
[ 11.208555] ---[ end trace 9351b91c2625bf52 ]---
[ 11.209246] EIP: reconfigure_super+0x21/0x1f0
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 914ae609fc3e9362513e6581e3dfe01eccfbba4e 9e98c678c2d6ae3a17cb2de55d17f69dddaa231b --
git bisect good 67731f56ba4992064d66efcbfd902cad69bd8786 # 22:02 G 11 0 0 0 vfs: Convert ibmasmfs to fs_context
git bisect good 6a11b3d644efaa8ae8157f972836ea13b1a74160 # 22:13 G 11 0 0 0 mtd: Provide fs_context-aware mount_mtd() replacement
git bisect good 9e38124e829636542a7392b03144a80540723e37 # 22:26 G 10 0 0 0 vfs: Convert functionfs to fs_context
git bisect bad 8779c90793189ce2fbc8999449abcfbed34e3cc2 # 22:32 B 0 1 17 2 vfs: Convert pstore to fs_context
git bisect bad c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18 # 22:48 B 0 2 16 0 vfs: Convert debugfs to fs_context
git bisect good 661e6721f46b54ea15c0a2b521d3b8e894944c4c # 23:01 G 11 0 0 1 vfs: Add a single-or-reconfig keying to vfs_get_super()
# first bad commit: [c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18] vfs: Convert debugfs to fs_context
git bisect good 661e6721f46b54ea15c0a2b521d3b8e894944c4c # 23:03 G 30 0 0 1 vfs: Add a single-or-reconfig keying to vfs_get_super()
# extra tests with debug options
# extra tests on HEAD of dhowells-fs/mount-api-viro
git bisect bad 914ae609fc3e9362513e6581e3dfe01eccfbba4e # 23:19 B 0 13 31 0 vfs: Kill off mount_single()
# extra tests on tree/branch dhowells-fs/mount-api-viro
git bisect bad 914ae609fc3e9362513e6581e3dfe01eccfbba4e # 23:20 B 0 13 31 0 vfs: Kill off mount_single()
# extra tests with first bad commit reverted
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 10 months
c1d8e1d13e ("vfs: Convert debugfs to fs_context"): BUG: unable to handle kernel NULL pointer dereference at 00000094
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git mount-api-viro
commit c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18
Author: David Howells <dhowells(a)redhat.com>
AuthorDate: Mon Jan 21 14:04:22 2019 +0000
Commit: David Howells <dhowells(a)redhat.com>
CommitDate: Thu Mar 21 09:47:14 2019 +0000
vfs: Convert debugfs to fs_context
Signed-off-by: David Howells <dhowells(a)redhat.com>
cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
cc: "Rafael J. Wysocki" <rafael(a)kernel.org>
661e6721f4 vfs: Add a single-or-reconfig keying to vfs_get_super()
c1d8e1d13e vfs: Convert debugfs to fs_context
914ae609fc vfs: Kill off mount_single()
+------------------------------------------+------------+------------+------------+
| | 661e6721f4 | c1d8e1d13e | 914ae609fc |
+------------------------------------------+------------+------------+------------+
| boot_successes | 73 | 0 | 2 |
| boot_failures | 0 | 13 | 13 |
| BUG:unable_to_handle_kernel | 0 | 13 | 13 |
| Oops:#[##] | 0 | 13 | 13 |
| EIP:reconfigure_super | 0 | 13 | 13 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 12 | 13 |
+------------------------------------------+------------+------------+------------+
[ 11.020260] random: init: uninitialized urandom read (12 bytes read)
Kernel tests: Boot OK!
[ 11.089703] init: plymouth main process (134) killed by ABRT signal
[ 11.126682] random: trinity: uninitialized urandom read (4 bytes read)
[ 11.176076] random: mountall: uninitialized urandom read (12 bytes read)
[ 11.190345] BUG: unable to handle kernel NULL pointer dereference at 00000094
[ 11.191350] #PF error: [normal kernel read fault]
[ 11.192033] *pde = 00000000
[ 11.192446] Oops: 0000 [#1]
[ 11.192887] CPU: 0 PID: 154 Comm: mount Not tainted 5.1.0-rc1-00054-gc1d8e1d #1
[ 11.193881] EIP: reconfigure_super+0x21/0x1f0
[ 11.194470] Code: d8 5b 5e 5f 5d c3 8d 76 00 55 89 e5 83 ec 24 89 5d f4 89 75 f8 89 7d fc e8 d0 f0 e7 ff 8b 50 14 f7 40 38 ae ff 7f fd 8b 78 34 <8b> b2 94 00 00 00 0f 85 6b 01 00 00 8b 96 b8 01 00 00 85 d2 74 11
[ 11.197049] EAX: 5f22c8c8 EBX: 00000000 ECX: 51234c8a EDX: 00000000
[ 11.197929] ESI: 5f22c8c8 EDI: 00000000 EBP: 5f225ea8 ESP: 5f225e84
[ 11.198808] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 11.199747] CR0: 80050033 CR2: 00000094 CR3: 1f29e000 CR4: 00000690
[ 11.200610] Call Trace:
[ 11.200988] ? sget_fc+0x227/0x240
[ 11.201448] ? get_anon_bdev+0x50/0x50
[ 11.201997] ? vfs_get_super+0x9d/0x100
[ 11.202536] ? debugfs_get_tree+0x17/0x20
[ 11.203117] ? vfs_get_tree+0x5a/0x120
[ 11.203650] ? vfs_parse_fs_string+0xa0/0xa0
[ 11.204257] ? do_mount+0x97b/0xce0
[ 11.204766] ? __copy_user_ll+0xd/0x10
[ 11.205289] ? copy_mount_options+0xe/0x250
[ 11.205889] ? ksys_mount+0x96/0xc0
[ 11.206361] ? sys_mount+0x26/0x30
[ 11.206866] ? do_fast_syscall_32+0x146/0x2c0
[ 11.207472] ? entry_SYSENTER_32+0x7e/0xe3
[ 11.208068] CR2: 0000000000000094
[ 11.208555] ---[ end trace 9351b91c2625bf52 ]---
[ 11.209246] EIP: reconfigure_super+0x21/0x1f0
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 914ae609fc3e9362513e6581e3dfe01eccfbba4e 9e98c678c2d6ae3a17cb2de55d17f69dddaa231b --
git bisect good 67731f56ba4992064d66efcbfd902cad69bd8786 # 22:02 G 11 0 0 0 vfs: Convert ibmasmfs to fs_context
git bisect good 6a11b3d644efaa8ae8157f972836ea13b1a74160 # 22:13 G 11 0 0 0 mtd: Provide fs_context-aware mount_mtd() replacement
git bisect good 9e38124e829636542a7392b03144a80540723e37 # 22:26 G 10 0 0 0 vfs: Convert functionfs to fs_context
git bisect bad 8779c90793189ce2fbc8999449abcfbed34e3cc2 # 22:32 B 0 1 17 2 vfs: Convert pstore to fs_context
git bisect bad c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18 # 22:48 B 0 2 16 0 vfs: Convert debugfs to fs_context
git bisect good 661e6721f46b54ea15c0a2b521d3b8e894944c4c # 23:01 G 11 0 0 1 vfs: Add a single-or-reconfig keying to vfs_get_super()
# first bad commit: [c1d8e1d13e86fe9f2aa55aa126cf1ab88b9c2a18] vfs: Convert debugfs to fs_context
git bisect good 661e6721f46b54ea15c0a2b521d3b8e894944c4c # 23:03 G 30 0 0 1 vfs: Add a single-or-reconfig keying to vfs_get_super()
# extra tests with debug options
# extra tests on HEAD of dhowells-fs/mount-api-viro
git bisect bad 914ae609fc3e9362513e6581e3dfe01eccfbba4e # 23:19 B 0 13 31 0 vfs: Kill off mount_single()
# extra tests on tree/branch dhowells-fs/mount-api-viro
git bisect bad 914ae609fc3e9362513e6581e3dfe01eccfbba4e # 23:20 B 0 13 31 0 vfs: Kill off mount_single()
# extra tests with first bad commit reverted
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 10 months
[rcu] f836ea2ec9: BUG:unable_to_handle_kernel
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: f836ea2ec954a20af25861e68075ad743be046f4 ("rcu: Forbid DEFINE{,_STATIC}_SRCU() from modules")
https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev.2019.03.18a
in testcase: rcutorture
with following parameters:
runtime: 300s
test: cpuhotplug
torture_type: srcu
test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------+------------+------------+
| | f28943f8f0 | f836ea2ec9 |
+-------------------------------------------------+------------+------------+
| boot_successes | 1 | 1 |
| boot_failures | 3 | 7 |
| BUG:kernel_reboot-without-warning_in_test_stage | 3 | 3 |
| BUG:unable_to_handle_kernel | 0 | 4 |
| Oops:#[##] | 0 | 4 |
| EIP:rcu_torture_init | 0 | 4 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 4 |
+-------------------------------------------------+------------+------------+
[ 9.610129] BUG: unable to handle kernel NULL pointer dereference at 00000054
[ 9.611320] #PF error: [normal kernel read fault]
[ 9.611823] *pdpt = 000000002a082001 *pde = 0000000000000000
[ 9.612440] Oops: 0000 [#1] PTI
[ 9.612781] CPU: 0 PID: 592 Comm: modprobe Not tainted 5.1.0-rc1-00052-gf836ea2 #2
[ 9.613583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 9.614469] EIP: rcu_torture_init+0x75e/0x1000 [rcutorture]
[ 9.615079] Code: 5b e8 46 4b 1f 00 eb 58 50 31 db 68 90 5d e1 f7 e8 70 bf 28 c9 68 07 61 e1 f7 e8 66 bf 28 c9 83 c4 0c 8b 04 9d 8c 62 e1 f7 43 <ff> 70 54 68 94 60 e1 f7 e8 4e bf 28 c9 83 fb 06 58 5a 75 e4 68 d8
[ 9.617052] EAX: 00000000 EBX: 00000003 ECX: 00000006 EDX: f7e16142
[ 9.617719] ESI: fffffff0 EDI: ea0d11b0 EBP: 00000001 ESP: e9bf5de8
[ 9.618392] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 EFLAGS: 00010207
[ 9.619118] CR0: 80050033 CR2: 00000054 CR3: 2a0ae000 CR4: 000406b0
[ 9.619784] Call Trace:
[ 9.620060] ? 0xf7e1d000
[ 9.620349] ? do_one_initcall+0x52/0x250
[ 9.620784] ? do_init_module+0x19/0x1bb
[ 9.621213] ? rcu_read_lock_sched_held+0x38/0x60
[ 9.621719] ? kmem_cache_alloc_trace+0x105/0x270
[ 9.622230] ? slow_virt_to_phys+0x27/0x120
[ 9.622682] ? do_init_module+0x47/0x1bb
[ 9.623111] ? load_module+0x2030/0x2430
[ 9.623537] ? restore_all_kernel+0xf/0x74
[ 9.623980] ? sys_init_module+0xf7/0x140
[ 9.624420] ? kvm_sched_clock_read+0x13/0x30
[ 9.624891] ? do_fast_syscall_32+0xa1/0x2d0
[ 9.625355] ? entry_SYSENTER_32+0x69/0xbc
[ 9.625797] Modules linked in: rcutorture(+) torture parport_pc serio_raw parport qemu_fw_cfg
[ 9.626706] CR2: 0000000000000054
[ 9.627081] ---[ end trace c93ea0cabc92d6b1 ]---
To reproduce:
# build kernel
cd linux
cp config-5.1.0-rc1-00052-gf836ea2 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months
[x86] f8eb0235f6: kmsg.pcengines_apuv2:failed_to_detect_apu_board_via_dmi
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: f8eb0235f65989fc5521c40c78d1261e7f25cdbe ("x86: pcengines apuv2 gpio/leds/keys platform driver")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: rcutorture
with following parameters:
runtime: 300s
test: default
torture_type: srcud
test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
[ 64.151440 ] acerhdf: Acer Aspire One Fan driver, v.0.7.0
[ 64.168193 ] acerhdf: unknown (unsupported) BIOS version QEMU/Standard PC (i440FX + PIIX, 1996)/1.10.2-1, please report, aborting!
[ 64.213363 ] pcengines_apuv2: failed to detect apu board via dmi
[ 64.234398 ] IDT PCI-E Non-Transparent Bridge Driver 2.0
[ 64.253221 ] Intel(R) PCI-E Non-Transparent Bridge Driver 2.0
[ 64.276914 ] intel_rapl: driver does not support CPU family 6 model 42
[ 64.306656 ] parport0: cannot grant exclusive access for device MTS64
[ 64.329350 ] snd_mts64: probe of snd_mts64.0 failed with error -5
[ 64.352451 ] parport0: cannot grant exclusive access for device portman
[ 64.373677 ] snd_portman2x4: probe of snd_portman2x4.0 failed with error -5
[ 64.410229 ] _warn_unseeded_randomness: 1 callbacks suppressed
[ 64.410264 ] random: get_random_u64 called from __kmem_cache_create+0x2f/0x3e2 with crng_init=0
[ 64.455796 ] random: get_random_u64 called from cache_random_seq_create+0x55/0xf3 with crng_init=0
[ 64.485905 ] random: get_random_u64 called from __kmem_cache_create+0x2f/0x3e2 with crng_init=0
[ 64.518465 ] NET: Registered protocol family 10
[ 64.551480 ] Segment Routing with IPv6
[ 64.563597 ] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 64.587255 ] 9pnet: Installing 9P2000 support
To reproduce:
# build kernel
cd linux
cp config-5.0.0-rc6-00105-gf8eb023 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months
[hugetlbfs] 2284cf59cb: BUG:KASAN:global-out-of-bounds_in_f
by kernel test robot
FYI, we noticed the following commit (built with gcc-5):
commit: 2284cf59cbcec2f17e50139e2db6d6d761521cd3 ("hugetlbfs: Convert to fs_context")
https://git.kernel.org/cgit/linux/kernel/git/viro/vfs.git R48
in testcase: rcutorture
with following parameters:
runtime: 300s
test: cpuhotplug
torture_type: rcu
test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-----------------------------------------------------------------------------+------------+------------+
| | 0ecab105a8 | 2284cf59cb |
+-----------------------------------------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 51 | 4 |
| kobject(#):tried_to_init_an_initialized_object,something_is_seriously_wrong | 42 | |
| BUG:KASAN:double-free_or_invalid-free_in_k | 43 | |
| BUG:KASAN:use-after-free_in_t | 13 | |
| BUG:KASAN:user-memory-access_in_s | 1 | |
| BUG:unable_to_handle_kernel | 17 | |
| Oops:#[##] | 17 | |
| RIP:string | 2 | |
| Kernel_panic-not_syncing:Fatal_exception | 18 | |
| BUG:KASAN:slab-out-of-bounds_in_t | 3 | |
| general_protection_fault:#[##] | 1 | |
| WARNING:at_kernel/locking/lockdep.c:#lock_downgrade | 2 | |
| RIP:lock_downgrade | 2 | |
| BUG:kernel_in_stage | 3 | |
| BUG:KASAN:user-memory-access_in_t | 19 | |
| RIP:ttm_mem_global_init[ttm] | 16 | |
| BUG:kernel_hang_in_boot-around-mounting-root_stage | 5 | |
| BUG:KASAN:global-out-of-bounds_in_f | 0 | 4 |
+-----------------------------------------------------------------------------+------------+------------+
[ 5.777052] BUG: KASAN: global-out-of-bounds in fs_validate_description+0xeb/0x3c0
[ 5.778184] Read of size 8 at addr ffffffffafade9b0 by task swapper/1
[ 5.778184]
[ 5.778184] CPU: 0 PID: 1 Comm: swapper Not tainted 5.0.0-rc2-00037-g2284cf5 #2
[ 5.778184] Call Trace:
[ 5.778184] print_address_description+0x1dd/0x290
[ 5.778184] ? fs_validate_description+0xeb/0x3c0
[ 5.778184] ? fs_validate_description+0xeb/0x3c0
[ 5.778184] kasan_report+0x134/0x1a4
[ 5.778184] ? f_dupfd+0xa0/0xf0
[ 5.778184] ? fs_validate_description+0xeb/0x3c0
[ 5.778184] fs_validate_description+0xeb/0x3c0
[ 5.778184] ? kmem_cache_create_usercopy+0xa2/0x2f0
[ 5.778184] register_filesystem+0x23/0xc0
[ 5.778184] init_hugetlbfs_fs+0xc3/0x286
[ 5.778184] ? init_ramfs_fs+0x7c/0x7c
[ 5.778184] do_one_initcall+0xb3/0x300
[ 5.778184] ? initcall_blacklisted+0x120/0x120
[ 5.778184] ? check_flags+0x1d0/0x270
[ 5.778184] ? __lock_is_held+0x37/0xd0
[ 5.778184] kernel_init_freeable+0x418/0x66c
[ 5.778184] ? rest_init+0x140/0x140
[ 5.778184] kernel_init+0xf/0x120
[ 5.778184] ? _raw_spin_unlock_irq+0x29/0x40
[ 5.778184] ? rest_init+0x140/0x140
[ 5.778184] ret_from_fork+0x24/0x30
[ 5.778184]
[ 5.778184] The buggy address belongs to the variable:
[ 5.778184] hugetlb_param_specs+0x70/0xc0
[ 5.778184]
[ 5.778184] Memory state around the buggy address:
[ 5.778184] ffffffffafade880: 00 01 fa fa fa fa fa fa 05 fa fa fa fa fa fa fa
[ 5.778184] ffffffffafade900: 04 fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
[ 5.778184] >ffffffffafade980: 00 00 00 00 00 00 fa fa fa fa fa fa 00 00 00 00
[ 5.778184] ^
[ 5.778184] ffffffffafadea00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 5.778184] ffffffffafadea80: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00
[ 5.778184] ==================================================================
[ 5.778184] Disabling lock debugging due to kernel taint
[ 5.827396] pnp: PnP ACPI init
[ 5.830044] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
[ 5.833615] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
[ 5.837002] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
[ 5.839483] pnp 00:03: [dma 2]
[ 5.840964] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
[ 5.845533] pnp 00:04: Plug and Play ACPI device, IDs PNP0400 (active)
[ 5.849958] pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
[ 5.854484] pnp 00:06: Plug and Play ACPI device, IDs PNP0501 (active)
[ 5.870601] pnp: PnP ACPI: found 7 devices
[ 5.882050] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 5.884525] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 5.886139] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 5.887790] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 5.889583] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 5.894067] NET: Registered protocol family 2
[ 5.895541] random: get_random_u32 called from neigh_hash_alloc+0xa4/0xf0 with crng_init=0
[ 5.900301] random: get_random_u32 called from bucket_table_alloc+0xbe/0x210 with crng_init=0
[ 5.903284] random: get_random_u32 called from rt_genid_init+0x78/0xc0 with crng_init=0
[ 5.905616] tcp_listen_portaddr_hash hash table entries: 4096 (order: 6, 360448 bytes)
[ 5.908791] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[ 5.913051] TCP bind hash table entries: 32768 (order: 9, 2621440 bytes)
[ 5.923551] TCP: Hash tables configured (established 65536 bind 32768)
[ 5.931298] UDP hash table entries: 4096 (order: 7, 786432 bytes)
[ 5.935818] UDP-Lite hash table entries: 4096 (order: 7, 786432 bytes)
[ 5.940087] NET: Registered protocol family 1
[ 5.943920] RPC: Registered named UNIX socket transport module.
[ 5.945542] RPC: Registered udp transport module.
[ 5.946777] RPC: Registered tcp transport module.
[ 5.948034] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 5.949740] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 5.951332] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 5.952971] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 5.954821] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 5.957382] PCI: CLS 0 bytes, default 64
[ 5.959720] Unpacking initramfs...
Elapsed time: 10
qemu-img create -f qcow2 disk-vm-snb-8G-423-0 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-1 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-2 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-3 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-4 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-5 256G
qemu-img create -f qcow2 disk-vm-snb-8G-423-6 256G
kvm=(
qemu-system-x86_64
-enable-kvm
-cpu SandyBridge
-kernel $kernel
-initrd initrd-vm-snb-8G-423
-m 8192
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0,hostfwd=tcp::26422-:22
-boot order=nc
-no-reboot
-watchdog i6300esb
-watchdog-action debug
-rtc base=localtime
To reproduce:
# build kernel
cd linux
cp config-5.0.0-rc2-00037-g2284cf5 .config
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months
[lib] 50ceaa95ea: BUG:KASAN:stack-out-of-bounds_in_l
by kernel test robot
FYI, we noticed the following commit (built with gcc-5):
commit: 50ceaa95ea09703722b30b4afa617c972071cd7f ("lib: Introduce test_stackinit module")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: locktorture
with following parameters:
runtime: 300s
test: cpuhotplug
test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors.
test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------+------------+------------+
| | 81a56f6dcd | 50ceaa95ea |
+------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 4 |
| BUG:KASAN:stack-out-of-bounds_in_l | 0 | 4 |
+------------------------------------+------------+------------+
[ 46.692222] BUG: KASAN: stack-out-of-bounds in leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] Write of size 32 at addr ffff888063d87b60 by task swapper/1
[ 46.692716]
[ 46.692716] CPU: 0 PID: 1 Comm: swapper Not tainted 5.0.0-rc3-00002-g50ceaa9 #2
[ 46.692716] Call Trace:
[ 46.692716] dump_stack+0x19/0x1b
[ 46.692716] print_address_description+0x6b/0x22c
[ 46.692716] ? leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] ? leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] kasan_report+0x18a/0x1bc
[ 46.692716] ? leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] ? test_switch_2_none+0x17c/0x25d
[ 46.692716] check_memory_region+0x69/0x16d
[ 46.692716] ? leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] memset+0x26/0x3b
[ 46.692716] leaf_big_hole_none+0xf9/0x1c9
[ 46.692716] ? test_small_hole_none+0x269/0x269
[ 46.692716] ? console_unlock+0x15c/0x570
[ 46.692716] ? lock_downgrade+0x2ce/0x2ce
[ 46.692716] ? vprintk_emit+0x1ff/0x211
[ 46.692716] ? test_switch_2_none+0x25d/0x25d
[ 46.692716] test_big_hole_zero+0xd1/0x295
[ 46.692716] ? memset+0x34/0x3b
[ 46.692716] ? test_big_hole_zero+0xd1/0x295
[ 46.692716] ? test_big_hole_static_partial+0x292/0x292
[ 46.692716] ? kmsg_dump_rewind+0x56/0x56
[ 46.692716] ? initcall_blacklisted+0x74/0x140
[ 46.692716] ? test_switch_2_none+0x25d/0x25d
[ 46.692716] test_stackinit_init+0x4d/0x1b1
[ 46.692716] ? test_switch_2_none+0x25d/0x25d
[ 46.692716] ? test_stackinit_init+0x4d/0x1b1
[ 46.692716] do_one_initcall+0xf8/0x2be
[ 46.692716] ? initcall_blacklisted+0x140/0x140
[ 46.692716] ? __asan_load1+0x6c/0x6c
[ 46.692716] ? kernel_init_freeable+0x245/0x528
[ 46.692716] kernel_init_freeable+0x303/0x528
[ 46.692716] ? rest_init+0x127/0x127
[ 46.692716] kernel_init+0xc/0x100
[ 46.692716] ? rest_init+0x127/0x127
[ 46.692716] ret_from_fork+0x24/0x30
[ 46.692716]
[ 46.692716] The buggy address belongs to the page:
[ 46.692716] page:ffffea00018f61c0 count:0 mapcount:0 mapping:0000000000000000 index:0x0
[ 46.692716] flags: 0x0()
[ 46.692716] raw: 0000000000000000 ffffea00018f61c8 ffffea00018f61c8 0000000000000000
[ 46.692716] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000
[ 46.692716] page dumped because: kasan: bad access detected
[ 46.692716]
[ 46.692716] Memory state around the buggy address:
[ 46.692716] ffff888063d87a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 46.692716] ffff888063d87a80: f1 f1 f1 f1 f1 f1 f1 f1 00 00 00 00 00 00 00 00
[ 46.692716] >ffff888063d87b00: 00 00 00 00 00 00 00 00 f2 f2 f2 f2 f2 f2 f2 f2
[ 46.692716] ^
[ 46.692716] ffff888063d87b80: f3 f3 f3 f3 f3 f3 f3 f3 00 00 00 00 00 00 00 00
[ 46.692716] ffff888063d87c00: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 f1 f1 f1 f1
[ 46.692716] ==================================================================
[ 46.692716] Disabling lock debugging due to kernel taint
[ 46.762914] test_stackinit: big_hole_zero ok
[ 46.764142] test_stackinit: trailing_hole_zero ok
[ 46.765372] test_stackinit: packed_zero ok
[ 46.766441] test_stackinit: small_hole_dynamic_partial ok
[ 46.767819] test_stackinit: big_hole_dynamic_partial ok
[ 46.769254] test_stackinit: trailing_hole_dynamic_partial ok
[ 46.770796] test_stackinit: packed_dynamic_partial ok
[ 46.772130] test_stackinit: small_hole_static_partial ok
[ 46.773565] test_stackinit: big_hole_static_partial ok
[ 46.774901] test_stackinit: trailing_hole_static_partial ok
[ 46.776372] test_stackinit: packed_static_partial ok
[ 46.777662] test_stackinit: small_hole_static_all ok
[ 46.778953] test_stackinit: big_hole_static_all ok
[ 46.780219] test_stackinit: trailing_hole_static_all ok
[ 46.781581] test_stackinit: packed_static_all ok
[ 46.782782] test_stackinit: small_hole_dynamic_all ok
[ 46.784136] test_stackinit: big_hole_dynamic_all ok
[ 46.785468] test_stackinit: trailing_hole_dynamic_all ok
[ 46.786848] test_stackinit: packed_dynamic_all ok
[ 46.788101] test_stackinit: small_hole_runtime_partial ok
[ 46.789507] test_stackinit: big_hole_runtime_partial ok
[ 46.790852] test_stackinit: trailing_hole_runtime_partial ok
[ 46.792362] test_stackinit: packed_runtime_partial ok
[ 46.793655] test_stackinit: small_hole_runtime_all ok
[ 46.794913] test_stackinit: big_hole_runtime_all ok
[ 46.796221] test_stackinit: trailing_hole_runtime_all ok
[ 46.797696] test_stackinit: packed_runtime_all ok
[ 46.798912] test_stackinit: u8_none ok
[ 46.799893] test_stackinit: u16_none ok
[ 46.800933] test_stackinit: u32_none ok
[ 46.801896] test_stackinit: u64_none ok
[ 46.802905] test_stackinit: char_array_none ok
[ 46.804083] test_stackinit: switch_1_none ok
[ 46.805192] test_stackinit: switch_2_none ok
[ 46.806349] test_stackinit: small_hole_none ok
[ 46.807512] test_stackinit: big_hole_none ok
[ 46.808689] test_stackinit: trailing_hole_none ok
[ 46.809858] test_stackinit: packed_none ok
[ 46.810923] test_stackinit: user ok
[ 46.811844] test_stackinit: all tests passed!
[ 46.868870] random: get_random_u64 called from copy_process+0x5cd/0x3f82 with crng_init=0
[ 46.874707] intel_idle: Please enable MWAIT in BIOS SETUP
[ 47.248232] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 47.968055] serial8250: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
To reproduce:
# build kernel
cd linux
cp config-5.0.0-rc3-00002-g50ceaa9 .config
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-5 CC=gcc-5 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
lkp
1 year, 10 months
1a92702b56 ("slob: Respect list_head abstraction layer"): kernel BUG at lib/list_debug.c:28!
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://github.com/tcharding/linux slab-lru
commit 1a92702b56442beb103172415a9930e8d8ae9eea
Author: Tobin C. Harding <tobin(a)kernel.org>
AuthorDate: Thu Mar 14 10:38:59 2019 +1100
Commit: Tobin C. Harding <tobin(a)kernel.org>
CommitDate: Mon Mar 18 10:19:44 2019 +1100
slob: Respect list_head abstraction layer
Currently we reach inside the list_head. This is a violation of the
layer of abstraction provided by the list_head. It makes the code
fragile. More importantly it makes the code wicked hard to understand.
The code logic is based on the page in which an allocation was made, we
want to modify the slob_list we are working on to have this page at the
front. We already have a function to check if an entry is at the front
of the list. Recently a function was added to list.h to do the list
rotation. We can use these two functions to reduce line count, reduce
code fragility, and reduce cognitive load required to read the code.
Use list_head functions to interact with lists thereby maintaining the
abstraction provided by the list_head structure.
Signed-off-by: Tobin C. Harding <tobin(a)kernel.org>
17c11cee46 list: Add function list_rotate_to_front()
1a92702b56 slob: Respect list_head abstraction layer
88e00f38f1 mm: Remove stale comment from page struct
+-------------------------------------------------------+------------+------------+------------+
| | 17c11cee46 | 1a92702b56 | 88e00f38f1 |
+-------------------------------------------------------+------------+------------+------------+
| boot_successes | 115 | 29 | 0 |
| boot_failures | 0 | 9 | 2 |
| kernel_BUG_at_lib/list_debug.c | 0 | 9 | 2 |
| invalid_opcode:#[##] | 0 | 9 | 2 |
| RIP:__list_add_valid | 0 | 9 | 2 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 9 | 2 |
| WARNING:possible_circular_locking_dependency_detected | 0 | 3 | |
+-------------------------------------------------------+------------+------------+------------+
[ 22.850195] PCI Interrupt Link [LNKC] enabled at IRQ 11
[ 24.387212] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56
[ 24.399757] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection
[ 28.526977] list_add corruption. prev->next should be next (ffffffff82cfc990), but was ffff88801f31dd90. (prev=ffff88801f347f50).
[ 28.527552] ------------[ cut here ]------------
[ 28.527552] kernel BUG at lib/list_debug.c:28!
[ 28.527552] invalid opcode: 0000 [#1] DEBUG_PAGEALLOC
[ 28.527552] CPU: 0 PID: 1 Comm: swapper Not tainted 5.1.0-rc1-00002-g1a92702 #3
[ 28.527552] RIP: 0010:__list_add_valid+0x6f/0xd5
[ 28.527552] Code: d9 02 49 8b 10 48 39 c2 74 37 4c 89 c1 48 89 c6 48 c7 c7 d7 5f 82 82 48 ff 05 42 67 d9 02 e8 49 ad b0 ff 48 ff 05 3e 67 d9 02 <0f> 0b 48 ff 05 3d 67 d9 02 48 ff 05 3e 67 d9 02 48 ff 05 3f 67 d9
[ 28.527552] RSP: 0000:ffffc9000000b848 EFLAGS: 00010002
[ 28.527552] RAX: 0000000000000075 RBX: 0000000000000050 RCX: 0000000001304118
[ 28.527552] RDX: 0000000000000002 RSI: 0000000000000000 RDI: ffffffff81126964
[ 28.527552] RBP: ffffc9000000b848 R08: 0000000000000001 R09: 000000000000000c
[ 28.527552] R10: 0000000000000028 R11: ffffc9000000b6c7 R12: ffff88801bc13fb0
[ 28.527552] R13: ffffffff82cfc990 R14: ffffffff82cfc990 R15: ffff88801f347f50
[ 28.527552] FS: 0000000000000000(0000) GS:ffffffff82a43000(0000) knlGS:0000000000000000
[ 28.527552] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 28.527552] CR2: 0000000000000000 CR3: 0000000002a1f001 CR4: 00000000000206b0
[ 28.527552] Call Trace:
[ 28.527552] slob_alloc+0x119/0x320
[ 28.527552] slob_alloc_node+0x5c/0x356
[ 28.527552] kmem_cache_alloc+0x10/0x19
[ 28.527552] acpi_ut_create_generic_state+0x3d/0x56
[ 28.527552] acpi_ut_create_update_state+0x18/0x3c
[ 28.527552] acpi_ut_create_update_state_and_push+0x28/0x5d
[ 28.527552] acpi_ut_update_object_reference+0x12e/0x2b0
[ 28.527552] acpi_ut_remove_reference+0x4c/0x57
[ 28.527552] acpi_ds_clear_implicit_return+0x37/0x4d
[ 28.527552] acpi_ds_do_implicit_return+0x61/0x96
[ 28.527552] acpi_ds_is_result_used+0x55/0x1a8
[ 28.527552] acpi_ds_delete_result_if_not_used+0x56/0x9b
[ 28.527552] acpi_ds_exec_end_op+0x5d4/0x618
[ 28.527552] acpi_ps_parse_loop+0x777/0x834
[ 28.527552] acpi_ps_parse_aml+0xda/0x490
[ 28.527552] acpi_ps_execute_method+0x214/0x26d
[ 28.527552] acpi_ns_evaluate+0x27a/0x39d
[ 28.527552] acpi_ut_evaluate_object+0x82/0x225
[ 28.527552] acpi_rs_get_prt_method_data+0x28/0x68
[ 28.527552] acpi_get_irq_routing_table+0x37/0x42
[ 28.527552] acpi_pci_irq_find_prt_entry+0x93/0x22e
[ 28.527552] ? _raw_spin_unlock_irqrestore+0x75/0x8e
[ 28.527552] acpi_pci_irq_lookup+0x2a/0x20e
[ 28.527552] acpi_pci_irq_disable+0x7c/0xf0
[ 28.527552] pcibios_disable_device+0x20/0x2a
[ 28.527552] do_pci_disable_device+0x5b/0x66
[ 28.527552] pci_disable_device+0xce/0xee
[ 28.527552] e1000_remove+0x128/0x136
[ 28.527552] pci_device_remove+0x54/0xeb
[ 28.527552] really_probe+0x240/0x4b7
[ 28.527552] driver_probe_device+0x104/0x15b
[ 28.527552] device_driver_attach+0x5b/0x80
[ 28.527552] __driver_attach+0x104/0x114
[ 28.527552] ? device_driver_attach+0x80/0x80
[ 28.527552] bus_for_each_dev+0x9b/0xc5
[ 28.527552] driver_attach+0x20/0x29
[ 28.527552] bus_add_driver+0x107/0x29c
[ 28.527552] ? loopback_net_init+0xc6/0xc6
[ 28.527552] driver_register+0xfe/0x15f
[ 28.527552] ? loopback_net_init+0xc6/0xc6
[ 28.527552] __pci_register_driver+0x74/0x7e
[ 28.527552] e1000_init_module+0x5c/0xb2
[ 28.527552] do_one_initcall+0x141/0x32c
[ 28.527552] kernel_init_freeable+0x3eb/0x4e5
[ 28.527552] ? rest_init+0x1c3/0x1c3
[ 28.527552] kernel_init+0x10/0x199
[ 28.527552] ret_from_fork+0x24/0x30
[ 28.527552] Modules linked in:
[ 28.527552] ---[ end trace 8cf1e75eb01c64f0 ]---
[ 28.527552] RIP: 0010:__list_add_valid+0x6f/0xd5
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 7a049d2b8343f75d1e017d049d765e24154a19e4 9e98c678c2d6ae3a17cb2de55d17f69dddaa231b --
git bisect bad 0bc32e6f805f2cdedf5a582815d9ee54f97961df # 17:17 B 0 1 16 0 Merge 'ak/perf/intel-json-updates-20190314-1' into devel-hourly-2019031915
git bisect bad fc05f86e9d9750c2ffb9fec26cce8eb014baf763 # 17:32 B 0 1 16 0 Merge 'sunxi/sunxi/fixes-for-5.1' into devel-hourly-2019031915
git bisect bad ab613583149eca0cd573216ffe07e4d03ccb142a # 17:45 B 0 4 19 0 Merge 'linux-review/Dongli-Zhang/loop-access-lo_backing_file-only-when-the-loop-device-is-Lo_bound/20190319-090237' into devel-hourly-2019031915
git bisect bad f774cf2125ad4df6c73a0a27c046492d2cfedb96 # 18:04 B 2 4 0 0 Merge 'linux-review/Tobias-Klauser/doc-fix-link-to-MSG_ZEROCOPY-patchset/20190319-044733' into devel-hourly-2019031915
git bisect good 4019977d0d1dbf3a0d7922b02a89af52774c36b8 # 18:25 G 38 0 1 1 Merge 'linux-review/Jakub-Wilk/coredump-remove-outdated-comment/20190319-111221' into devel-hourly-2019031915
git bisect bad 9450c9f25bc561f65b5ae083b07efdb2d7b0b302 # 18:40 B 0 3 18 0 Merge 'linux-review/Stanley-Chu/scsi-ufs-Fix-regulator-operations-and-remove-name-fixed-regulator-device-tree-property/20190318-132501' into devel-hourly-2019031915
git bisect bad c54d70ad8916d2783602581b082aba1605737f7e # 18:55 B 3 2 0 0 Merge 'linux-review/Gustavo-A-R-Silva/usb-usb251xb-Remove-unnecessary-comparison-of-unsigned-integer-with-0/20190319-021832' into devel-hourly-2019031915
git bisect bad be19a10da83a34c9e7e3fd13736d1433f237805a # 19:13 B 0 4 19 0 Merge 'tcharding/slab-lru' into devel-hourly-2019031915
git bisect good f9d26cadd67349d2c3abfbb70bf5a4f7ec65e810 # 19:31 G 38 0 2 2 Merge 'linux-review/Souptick-Joarder/include-linux-hugetlb-h-Convert-to-use-vm_fault_t/20190319-081101' into devel-hourly-2019031915
git bisect bad 50519825f04c37455f754545a79aa5ac7b7912c7 # 19:46 B 0 1 16 0 slub: Add comments to endif pre-processor macros
git bisect bad 1a92702b56442beb103172415a9930e8d8ae9eea # 19:58 B 2 1 2 2 slob: Respect list_head abstraction layer
git bisect good 17c11cee46ec01db4ad978333402433e6ec67f4d # 20:19 G 39 0 0 0 list: Add function list_rotate_to_front()
# first bad commit: [1a92702b56442beb103172415a9930e8d8ae9eea] slob: Respect list_head abstraction layer
git bisect good 17c11cee46ec01db4ad978333402433e6ec67f4d # 20:27 G 115 0 4 4 list: Add function list_rotate_to_front()
# extra tests with debug options
git bisect bad 1a92702b56442beb103172415a9930e8d8ae9eea # 20:41 B 0 3 18 0 slob: Respect list_head abstraction layer
# extra tests on HEAD of linux-devel/devel-hourly-2019031915
git bisect bad 7a049d2b8343f75d1e017d049d765e24154a19e4 # 20:41 B 11 2 0 0 0day head guard for 'devel-hourly-2019031915'
# extra tests on tree/branch tcharding/slab-lru
git bisect bad 88e00f38f182be4e0ec8261dfff5d3ddde9e6191 # 21:03 B 0 2 17 0 mm: Remove stale comment from page struct
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 10 months
[net] 63c8902805: BUG:unable_to_handle_kernel
by kernel test robot
FYI, we noticed the following commit (built with gcc-5):
commit: 63c89028058f5920d4b5a9d38452fa4623469583 ("net: Make softirq vector masking finegrained on release_sock()")
https://git.kernel.org/cgit/linux/kernel/git/frederic/linux-dynticks.git softirq/soft-interruptible-v2
in testcase: trinity
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 2G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-----------------------------------------------------------------------------+------------+------------+
| | fa0c945eed | 63c8902805 |
+-----------------------------------------------------------------------------+------------+------------+
| boot_successes | 411 | 401 |
| boot_failures | 0 | 15 |
| BUG:unable_to_handle_kernel | 0 | 13 |
| Oops:#[##] | 0 | 13 |
| WARNING:stack_recursion | 0 | 14 |
| Kernel_panic-not_syncing:Fatal_exception_in_interrupt | 0 | 15 |
| general_protection_fault:f8bc[##] | 0 | 1 |
| invalid_opcode:#[##] | 0 | 1 |
| BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/mutex.c | 0 | 1 |
| EIP:__lock_acquire | 0 | 1 |
+-----------------------------------------------------------------------------+------------+------------+
[ 884.807502] BUG: unable to handle kernel NULL pointer dereference at 00000000
[ 884.817943] #PF error: [normal kernel read fault]
[ 884.825687] *pde = 00000000
[ 884.830940] Oops: 0000 [#1] PREEMPT SMP
[ 884.837809] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.0.0-rc2-00263-g63c8902 #1
[ 884.851286] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 884.865377] EIP: (null)
[ 884.870469] Code: Bad RIP value.
[ 884.876526] EAX: 0009ab9a EBX: 00000020 ECX: b7be4be0 EDX: 00000001
[ 884.888228] ESI: 00000001 EDI: 781eff98 EBP: b7be4420 ESP: 781eff60
[ 884.899832] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010002
[ 884.910585] CR0: 80050033 CR2: ffffffd6 CR3: 3d06d000 CR4: 00000690
[ 884.920249] Call Trace:
[ 884.924235] <SOFTIRQ>
[ 884.928212] ? smp_reschedule_interrupt+0x1e/0x20
[ 884.935929] ? run_rebalance_domains+0xa1/0xc0
[ 884.943804] ? __do_softirq+0x1a9/0x1f2
[ 884.950614] ? __irqentry_text_end+0x6/0x6
[ 884.957636] ? do_softirq_own_stack+0x28/0x40
[ 884.964383] </SOFTIRQ>
[ 884.966640] sh: can't kill pid 352: No such process
[ 884.968310] WARNING: stack recursion on stack type 3
[ 884.968335] CR2: 0000000000000000
[ 884.968341] ---[ end trace c9ff70ea2dfb9f25 ]---
To reproduce:
# build kernel
cd linux
cp config-5.0.0-rc2-00263-g63c8902 .config
make HOSTCC=gcc-5 CC=gcc-5 ARCH=i386 olddefconfig
make HOSTCC=gcc-5 CC=gcc-5 ARCH=i386 prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=i386 modules_prepare
make HOSTCC=gcc-5 CC=gcc-5 ARCH=i386 SHELL=/bin/bash
make HOSTCC=gcc-5 CC=gcc-5 ARCH=i386 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months