[sched/fair] aaaee291c0: stress-ng.lockf.ops_per_sec -14.0% regression
by kernel test robot
Greeting,
FYI, we noticed a -14.0% regression of stress-ng.lockf.ops_per_sec due to commit:
commit: aaaee291c0203f87a96cfae4b5c859000f090177 ("sched/fair: rework load_balance")
https://git.linaro.org/people/vincent.guittot/kernel.git sched/pelt
in testcase: stress-ng
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:
nr_threads: 100%
disk: 1HDD
testtime: 1s
class: os
cpufreq_governor: performance
ucode: 0x500002b
fs: ext4
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode:
os/gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/100%/debian-x86_64-2019-09-23.cgz/lkp-csl-2sp5/stress-ng/1s/0x500002b
commit:
0a1e496375 ("sched/fair: remove meaningless imbalance calculation")
aaaee291c0 ("sched/fair: rework load_balance")
0a1e4963754eb1a8 aaaee291c0203f87a96cfae4b5c
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
3:16 -19% :6 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:16 6% 1:6 dmesg.WARNING:at_ip__slab_free/0x
1:16 -6% :6 dmesg.WARNING:at_ip_perf_event_mmap_output/0x
%stddev %change %stddev
\ | \
4006873 ± 4% -14.0% 3444927 ± 8% stress-ng.lockf.ops
4005413 ± 4% -14.0% 3443418 ± 8% stress-ng.lockf.ops_per_sec
2636101 +25.7% 3314011 ± 9% stress-ng.mq.ops
2633930 +25.7% 3311441 ± 9% stress-ng.mq.ops_per_sec
6540459 -35.5% 4220947 ± 6% stress-ng.schedpolicy.ops
6541634 -35.5% 4222168 ± 6% stress-ng.schedpolicy.ops_per_sec
1078148 ± 3% -13.5% 932584 ± 3% stress-ng.seal.ops
1078274 ± 3% -13.4% 933393 ± 4% stress-ng.seal.ops_per_sec
1152876 ± 5% +24.1% 1430686 ± 8% stress-ng.sigrt.ops
1140868 ± 5% +24.3% 1418333 ± 8% stress-ng.sigrt.ops_per_sec
1089582 ± 5% +27.1% 1385151 ± 5% stress-ng.sigsuspend.ops
1088357 ± 5% +27.1% 1383482 ± 5% stress-ng.sigsuspend.ops_per_sec
509750 ± 3% +23.3% 628384 ± 18% stress-ng.tee.ops
509397 ± 3% +23.1% 626947 ± 18% stress-ng.tee.ops_per_sec
12207 -2.3% 11931 stress-ng.time.system_time
1271 -3.3% 1229 ± 2% stress-ng.time.user_time
40786346 +9.3% 44576202 ± 2% stress-ng.time.voluntary_context_switches
4529 ± 6% +13.2% 5128 ± 6% perf-stat.ps.major-faults
12.95 ± 88% -5.1 7.84 ±141% perf-profile.calltrace.cycles-pp.__x64_sys_move_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
12.95 ± 88% -5.1 7.84 ±141% perf-profile.calltrace.cycles-pp.kernel_move_pages.__x64_sys_move_pages.do_syscall_64.entry_SYSCALL_64_after_hwframe.syscall
1336614 ± 17% -44.1% 747052 ± 24% sched_debug.cfs_rq:/.min_vruntime.stddev
1336729 ± 17% -44.1% 747326 ± 24% sched_debug.cfs_rq:/.spread0.stddev
2.62 ±208% -100.0% 0.00 iostat.sdb.avgqu-sz.max
48.48 ±208% -100.0% 0.00 iostat.sdb.await.max
2.11 ±221% -100.0% 0.00 iostat.sdb.r_await.max
0.27 ±208% -100.0% 0.00 iostat.sdb.svctm.max
54.73 ±208% -100.0% 0.00 iostat.sdb.w_await.max
158.94 ± 97% -100.0% 0.00 interrupts.101:PCI-MSI.31981634-edge.i40e-eth0-TxRx-65
156.38 ± 92% -100.0% 0.00 interrupts.102:PCI-MSI.31981635-edge.i40e-eth0-TxRx-66
133.69 ± 79% -100.0% 0.00 interrupts.103:PCI-MSI.31981636-edge.i40e-eth0-TxRx-67
150.81 ± 92% -100.0% 0.00 interrupts.104:PCI-MSI.31981637-edge.i40e-eth0-TxRx-68
154.69 ± 71% -100.0% 0.00 interrupts.105:PCI-MSI.31981638-edge.i40e-eth0-TxRx-69
147.56 ± 81% -100.0% 0.00 interrupts.107:PCI-MSI.31981640-edge.i40e-eth0-TxRx-71
111.25 ± 98% -100.0% 0.00 interrupts.111:PCI-MSI.31981644-edge.i40e-eth0-TxRx-75
14180 ± 10% -100.0% 0.00 interrupts.316:PCI-MSI.376832-edge.ahci[0000:00:17.0]
173.25 ± 7% -100.0% 0.00 interrupts.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
196.75 ± 84% -100.0% 0.00 interrupts.36:PCI-MSI.31981569-edge.i40e-eth0-TxRx-0
148.44 ± 80% -100.0% 0.00 interrupts.37:PCI-MSI.31981570-edge.i40e-eth0-TxRx-1
155.00 ± 68% -100.0% 0.00 interrupts.38:PCI-MSI.31981571-edge.i40e-eth0-TxRx-2
148.06 ± 89% -100.0% 0.00 interrupts.39:PCI-MSI.31981572-edge.i40e-eth0-TxRx-3
174.06 ± 85% -100.0% 0.00 interrupts.40:PCI-MSI.31981573-edge.i40e-eth0-TxRx-4
167.31 ± 88% -100.0% 0.00 interrupts.41:PCI-MSI.31981574-edge.i40e-eth0-TxRx-5
186.12 ± 83% -100.0% 0.00 interrupts.42:PCI-MSI.31981575-edge.i40e-eth0-TxRx-6
163.12 ± 82% -100.0% 0.00 interrupts.43:PCI-MSI.31981576-edge.i40e-eth0-TxRx-7
159.50 ± 83% -100.0% 0.00 interrupts.44:PCI-MSI.31981577-edge.i40e-eth0-TxRx-8
191.31 ± 82% -100.0% 0.00 interrupts.45:PCI-MSI.31981578-edge.i40e-eth0-TxRx-9
207.56 ±146% -100.0% 0.00 interrupts.46:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
131.69 ± 80% -100.0% 0.00 interrupts.48:PCI-MSI.31981581-edge.i40e-eth0-TxRx-12
142.06 ± 76% -100.0% 0.00 interrupts.49:PCI-MSI.31981582-edge.i40e-eth0-TxRx-13
179.50 ± 71% -100.0% 0.00 interrupts.50:PCI-MSI.31981583-edge.i40e-eth0-TxRx-14
229.25 ± 75% -100.0% 0.00 interrupts.51:PCI-MSI.31981584-edge.i40e-eth0-TxRx-15
154.12 ± 78% -100.0% 0.00 interrupts.52:PCI-MSI.31981585-edge.i40e-eth0-TxRx-16
157.19 ± 77% -100.0% 0.00 interrupts.57:PCI-MSI.31981590-edge.i40e-eth0-TxRx-21
179.69 ±111% -100.0% 0.00 interrupts.59:PCI-MSI.31981592-edge.i40e-eth0-TxRx-23
129.88 ±104% -100.0% 0.00 interrupts.65:PCI-MSI.31981598-edge.i40e-eth0-TxRx-29
94.88 ±104% -100.0% 0.00 interrupts.66:PCI-MSI.31981599-edge.i40e-eth0-TxRx-30
147.50 ± 80% -100.0% 0.00 interrupts.84:PCI-MSI.31981617-edge.i40e-eth0-TxRx-48
151.25 ± 77% -100.0% 0.00 interrupts.88:PCI-MSI.31981621-edge.i40e-eth0-TxRx-52
173.75 ± 75% -100.0% 0.00 interrupts.89:PCI-MSI.31981622-edge.i40e-eth0-TxRx-53
173.62 ± 64% -100.0% 0.00 interrupts.90:PCI-MSI.31981623-edge.i40e-eth0-TxRx-54
146.25 ± 92% -100.0% 0.00 interrupts.91:PCI-MSI.31981624-edge.i40e-eth0-TxRx-55
169.75 ± 93% -100.0% 0.00 interrupts.92:PCI-MSI.31981625-edge.i40e-eth0-TxRx-56
137.88 ± 84% -100.0% 0.00 interrupts.94:PCI-MSI.31981627-edge.i40e-eth0-TxRx-58
186.00 ± 87% -100.0% 0.00 interrupts.96:PCI-MSI.31981629-edge.i40e-eth0-TxRx-60
150.88 ± 78% -100.0% 0.00 interrupts.97:PCI-MSI.31981630-edge.i40e-eth0-TxRx-61
171.81 ± 91% -100.0% 0.00 interrupts.98:PCI-MSI.31981631-edge.i40e-eth0-TxRx-62
196.12 ± 84% -100.0% 0.00 interrupts.CPU0.36:PCI-MSI.31981569-edge.i40e-eth0-TxRx-0
147.75 ± 80% -100.0% 0.00 interrupts.CPU1.37:PCI-MSI.31981570-edge.i40e-eth0-TxRx-1
207.00 ±146% -100.0% 0.00 interrupts.CPU10.46:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
131.19 ± 80% -100.0% 0.00 interrupts.CPU12.48:PCI-MSI.31981581-edge.i40e-eth0-TxRx-12
141.50 ± 76% -100.0% 0.00 interrupts.CPU13.49:PCI-MSI.31981582-edge.i40e-eth0-TxRx-13
178.88 ± 71% -100.0% 0.00 interrupts.CPU14.50:PCI-MSI.31981583-edge.i40e-eth0-TxRx-14
228.81 ± 76% -100.0% 0.00 interrupts.CPU15.51:PCI-MSI.31981584-edge.i40e-eth0-TxRx-15
153.31 ± 79% -100.0% 0.00 interrupts.CPU16.52:PCI-MSI.31981585-edge.i40e-eth0-TxRx-16
154.62 ± 68% -100.0% 0.00 interrupts.CPU2.38:PCI-MSI.31981571-edge.i40e-eth0-TxRx-2
66354 ± 49% +100.0% 132694 ± 26% interrupts.CPU2.RES:Rescheduling_interrupts
178.94 ±111% -100.0% 0.00 interrupts.CPU23.59:PCI-MSI.31981592-edge.i40e-eth0-TxRx-23
146.81 ± 80% -100.0% 0.00 interrupts.CPU48.84:PCI-MSI.31981617-edge.i40e-eth0-TxRx-48
166.69 ± 89% -100.0% 0.00 interrupts.CPU5.41:PCI-MSI.31981574-edge.i40e-eth0-TxRx-5
150.56 ± 78% -100.0% 0.00 interrupts.CPU52.88:PCI-MSI.31981621-edge.i40e-eth0-TxRx-52
173.19 ± 75% -100.0% 0.00 interrupts.CPU53.89:PCI-MSI.31981622-edge.i40e-eth0-TxRx-53
173.00 ± 65% -100.0% 0.00 interrupts.CPU54.90:PCI-MSI.31981623-edge.i40e-eth0-TxRx-54
145.50 ± 92% -100.0% 0.00 interrupts.CPU55.91:PCI-MSI.31981624-edge.i40e-eth0-TxRx-55
169.06 ± 93% -100.0% 0.00 interrupts.CPU56.92:PCI-MSI.31981625-edge.i40e-eth0-TxRx-56
103603 ± 43% -65.8% 35433 ± 41% interrupts.CPU56.RES:Rescheduling_interrupts
137.25 ± 85% -100.0% 0.00 interrupts.CPU58.94:PCI-MSI.31981627-edge.i40e-eth0-TxRx-58
185.56 ± 84% -100.0% 0.00 interrupts.CPU6.42:PCI-MSI.31981575-edge.i40e-eth0-TxRx-6
150.25 ± 78% -100.0% 0.00 interrupts.CPU61.97:PCI-MSI.31981630-edge.i40e-eth0-TxRx-61
171.06 ± 92% -100.0% 0.00 interrupts.CPU62.98:PCI-MSI.31981631-edge.i40e-eth0-TxRx-62
158.12 ± 97% -100.0% 0.00 interrupts.CPU65.101:PCI-MSI.31981634-edge.i40e-eth0-TxRx-65
155.94 ± 92% -100.0% 0.00 interrupts.CPU66.102:PCI-MSI.31981635-edge.i40e-eth0-TxRx-66
149.88 ± 92% -100.0% 0.00 interrupts.CPU68.104:PCI-MSI.31981637-edge.i40e-eth0-TxRx-68
153.94 ± 71% -100.0% 0.00 interrupts.CPU69.105:PCI-MSI.31981638-edge.i40e-eth0-TxRx-69
173.25 ± 7% -100.0% 0.00 interrupts.CPU7.35:PCI-MSI.31981568-edge.i40e-0000:3d:00.0:misc
162.62 ± 82% -100.0% 0.00 interrupts.CPU7.43:PCI-MSI.31981576-edge.i40e-eth0-TxRx-7
146.81 ± 82% -100.0% 0.00 interrupts.CPU71.107:PCI-MSI.31981640-edge.i40e-eth0-TxRx-71
158.88 ± 83% -100.0% 0.00 interrupts.CPU8.44:PCI-MSI.31981577-edge.i40e-eth0-TxRx-8
190.69 ± 82% -100.0% 0.00 interrupts.CPU9.45:PCI-MSI.31981578-edge.i40e-eth0-TxRx-9
stress-ng.schedpolicy.ops
7e+06 +-+-----------------------------------------------------------------+
|+++++++ ++++++++.+++++++++++++++++++++++++++++++++.++++++++++++++++|
6e+06 +-+ : : |
| : : |
5e+06 +-+ : : |
| : : O O O O O |
4e+06 +-+ : : OOOOOO OO OOOOOOOOOOOO O O O OO |
O :O: O OOO |
3e+06 +-+ : |
| : |
2e+06 +-+ : |
| : |
1e+06 +-+ : |
| : |
0 +OOOOOOO-OO---------------------------------------------------------+
stress-ng.schedpolicy.ops_per_sec
7e+06 +-+-----------------------------------------------------------------+
|+++++++ ++++++++.+++++++++++++++++++++++++++++++++.++++++++++++++++|
6e+06 +-+ : : |
| : : |
5e+06 +-+ : : |
| : : O O O O O |
4e+06 +-+ : : OOOOOO OO OOOOOOOOOOOO O O O OO |
O :O: O OOO |
3e+06 +-+ : |
| : |
2e+06 +-+ : |
| : |
1e+06 +-+ : |
| : |
0 +OOOOOOO-OO---------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 3 months
- 1
- 0
[scsi] cc2f854c79: suspend_stress.fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: cc2f854c79d5b766282507100c9482062838461a ("[RFC PATCH V4 2/2] scsi: core: don't limit per-LUN queue depth for SSD")
url: https://github.com/0day-ci/linux/commits/Ming-Lei/scsi-avoid-atomic-opera...
base: https://git.kernel.org/cgit/linux/kernel/git/jejb/scsi.git for-next
in testcase: suspend_stress
with following parameters:
mode: freeze
iterations: 10
on test machine: 4 threads Skylake with 8G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
test started
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
1 year, 3 months
- 1
- 0
[mm, meminit] ba8514481c: BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/mutex.c
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: ba8514481ccdb50a60f5e26a62bedcdc3c1ca61c ("mm, meminit: recalculate pcpu batch and high limits after init completes")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-----------------------------------------------------------------------------+------------+------------+
| | 8434a0af25 | ba8514481c |
+-----------------------------------------------------------------------------+------------+------------+
| boot_successes | 12 | 0 |
| boot_failures | 0 | 22 |
| BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/mutex.c | 0 | 22 |
+-----------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 0.713593] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:281
[ 0.713593] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 19, name: pgdatinit0
[ 0.713593] CPU: 1 PID: 19 Comm: pgdatinit0 Not tainted 5.4.0-rc3-00176-gba8514481ccdb #1
[ 0.713593] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 0.713593] Call Trace:
[ 0.713593] dump_stack+0x5c/0x7b
[ 0.713593] ___might_sleep+0x102/0x120
[ 0.713593] mutex_lock+0x1c/0x40
[ 0.713593] zone_pcp_update+0x1d/0x59
[ 0.713593] deferred_init_memmap+0x143/0x1b3
[ 0.713593] ? alloc_large_system_hash+0x26a/0x26a
[ 0.713593] kthread+0x11e/0x140
[ 0.713593] ? kthread_park+0x90/0x90
[ 0.713593] ret_from_fork+0x35/0x40
[ 0.767610] node 0 initialised, 1181418 pages in 55ms
[ 0.769875] devtmpfs: initialized
[ 0.770704] x86/mm: Memory block size: 128MB
[ 0.773892] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.774604] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.776243] pinctrl core: initialized pinctrl subsystem
[ 0.776743] PM: RTC time: 00:49:11, date: 2019-10-23
[ 0.777812] NET: Registered protocol family 16
[ 0.778829] audit: initializing netlink subsys (disabled)
[ 0.779625] audit: type=2000 audit(1571762951.739:1): state=initialized audit_enabled=0 res=1
[ 0.783611] cpuidle: using governor menu
[ 0.784814] ACPI: bus type PCI registered
[ 0.785598] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.786748] PCI: Using configuration type 1 for base access
[ 0.789646] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.792690] ACPI: Added _OSI(Module Device)
[ 0.793602] ACPI: Added _OSI(Processor Device)
[ 0.794597] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.795625] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.796626] ACPI: Added _OSI(Linux-Dell-Video)
[ 0.797613] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 0.798597] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 0.800205] ACPI: 1 ACPI AML tables successfully acquired and loaded
[ 0.801818] ACPI: Interpreter enabled
[ 0.802609] ACPI: (supports S0 S3 S4 S5)
[ 0.803598] ACPI: Using IOAPIC for interrupt routing
[ 0.804612] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.805709] ACPI: Enabled 3 GPEs in block 00 to 0F
[ 0.808855] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.809601] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3]
[ 0.810606] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.811839] acpiphp: Slot [3] registered
[ 0.812622] acpiphp: Slot [4] registered
[ 0.813636] acpiphp: Slot [5] registered
[ 0.814619] acpiphp: Slot [6] registered
[ 0.815617] acpiphp: Slot [7] registered
[ 0.816616] acpiphp: Slot [8] registered
[ 0.817617] acpiphp: Slot [9] registered
[ 0.818608] acpiphp: Slot [10] registered
[ 0.819627] acpiphp: Slot [11] registered
[ 0.820622] acpiphp: Slot [12] registered
[ 0.821617] acpiphp: Slot [13] registered
[ 0.822616] acpiphp: Slot [14] registered
[ 0.823616] acpiphp: Slot [15] registered
[ 0.824619] acpiphp: Slot [16] registered
[ 0.825616] acpiphp: Slot [17] registered
[ 0.826616] acpiphp: Slot [18] registered
[ 0.827617] acpiphp: Slot [19] registered
[ 0.828619] acpiphp: Slot [20] registered
[ 0.829616] acpiphp: Slot [21] registered
[ 0.830617] acpiphp: Slot [22] registered
[ 0.831631] acpiphp: Slot [23] registered
[ 0.832618] acpiphp: Slot [24] registered
[ 0.833615] acpiphp: Slot [25] registered
[ 0.834614] acpiphp: Slot [26] registered
[ 0.835619] acpiphp: Slot [27] registered
[ 0.836619] acpiphp: Slot [28] registered
[ 0.837617] acpiphp: Slot [29] registered
[ 0.838616] acpiphp: Slot [30] registered
[ 0.839618] acpiphp: Slot [31] registered
[ 0.840611] PCI host bridge to bus 0000:00
[ 0.841598] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.842597] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.843597] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.844597] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 0.845597] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.846645] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 0.848149] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 0.849338] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 0.853169] pci 0000:00:01.1: reg 0x20: [io 0xc080-0xc08f]
[ 0.855628] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.856598] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.857597] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.858597] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.859819] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 0.861184] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.861609] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.862890] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000
[ 0.864599] pci 0000:00:02.0: reg 0x10: [mem 0xfd000000-0xfdffffff pref]
[ 0.867598] pci 0000:00:02.0: reg 0x18: [mem 0xfebf0000-0xfebf0fff]
[ 0.874599] pci 0000:00:02.0: reg 0x30: [mem 0xfebe0000-0xfebeffff pref]
[ 0.875844] pci 0000:00:03.0: [8086:100e] type 00 class 0x020000
[ 0.877598] pci 0000:00:03.0: reg 0x10: [mem 0xfebc0000-0xfebdffff]
[ 0.879598] pci 0000:00:03.0: reg 0x14: [io 0xc000-0xc03f]
[ 0.885599] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
[ 0.886901] pci 0000:00:04.0: [1af4:1004] type 00 class 0x010000
[ 0.890447] pci 0000:00:04.0: reg 0x10: [io 0xc040-0xc07f]
To reproduce:
# build kernel
cd linux
cp config-5.4.0-rc3-00176-gba8514481ccdb .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
1 year, 3 months
- 1
- 0
[cgroup, blkcg] cea875453c: WARNING:at_mm/page-writeback.c:#clear_page_dirty_for_io
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: cea875453cd9f75abeceb2c42ce00fc34d3be7e8 ("cgroup, blkcg: prevent dirty inodes to pin dying memory cgroups")
https://github.com/rgushchin/linux.git cgwb.4
in testcase: xfstests
with following parameters:
disk: 4HDD
fs: btrfs
test: generic-group26
test-description: xfstests is a regression test suite for xfs and other files ystems.
test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------------------------------------------+----------+------------+
| | v5.4-rc4 | cea875453c |
+-------------------------------------------------------------------------------------+----------+------------+
| boot_successes | 2831 | 0 |
| boot_failures | 347 | 7 |
| WARNING:at_drivers/net/wireless/mac80211_hwsim.c:#mac80211_hwsim_tx[mac80211_hwsim] | 1 | |
| RIP:mac80211_hwsim_tx[mac80211_hwsim] | 1 | |
| WARNING:at_kernel/trace/trace_hwlat.c:#start_kthread | 8 | |
| RIP:start_kthread | 8 | |
| BUG:kernel_NULL_pointer_dereference,address | 6 | |
| WARNING:at_fs/iomap/direct-io.c:#iomap_dio_actor | 6 | |
| RIP:iomap_dio_actor | 6 | |
| BUG:kernel_hang_in_boot_stage | 16 | |
| Assertion_failed | 9 | |
| WARNING:at_fs/xfs/xfs_message.c:#assfail[xfs] | 9 | |
| RIP:assfail[xfs] | 9 | |
| BUG:soft_lockup-CPU##stuck_for#s | 1 | |
| RIP:smp_call_function_single | 1 | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 1 | |
| BUG:kernel_hang_in_test_stage | 3 | |
| WARNING:at_net/sched/sch_generic.c:#dev_watchdog | 1 | |
| RIP:dev_watchdog | 1 | |
| RIP:native_safe_halt | 5 | |
| WARNING:at_net/mac80211/tx.c:#__ieee80211_csa_update_counter[mac80211] | 1 | |
| RIP:__ieee80211_csa_update_counter[mac80211] | 1 | |
| RIP:_raw_spin_unlock_irqrestore | 1 | |
| BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/rwsem.c | 288 | |
| general_protection_fault:#[##] | 4 | |
| RIP:kmem_cache_alloc_trace | 7 | |
| Kernel_panic-not_syncing:Fatal_exception | 49 | |
| BUG:unable_to_handle_page_fault_for_address | 16 | |
| Oops:#[##] | 16 | |
| RIP:__kmalloc | 28 | |
| kernel_BUG_at_mm/slub.c | 16 | |
| invalid_opcode:#[##] | 19 | |
| RIP:kfree | 16 | |
| stack_segment:#[##] | 17 | |
| IP-Config:Auto-configuration_of_network_failed | 7 | |
| Kernel_panic-not_syncing:Fatal_exception_in_interrupt | 6 | |
| RIP:clear_page_rep | 1 | |
| RIP:account_kernel_stack | 1 | |
| kernel_BUG_at_mm/usercopy.c | 3 | |
| RIP:usercopy_abort | 3 | |
| RIP:mod_zone_page_state | 1 | |
| WARNING:at_mm/page-writeback.c:#clear_page_dirty_for_io | 0 | 7 |
| RIP:clear_page_dirty_for_io | 0 | 7 |
+-------------------------------------------------------------------------------------+----------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 100.489677] WARNING: CPU: 0 PID: 714 at mm/page-writeback.c:2706 clear_page_dirty_for_io+0x1cf/0x240
[ 100.492297] Modules linked in: btrfs xor zstd_decompress zstd_compress raid6_pq libcrc32c dm_mod intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel bochs_drm drm_vram_helper ttm sr_mod drm_kms_helper aesni_intel cdrom syscopyarea crypto_simd sg ata_generic ppdev snd_pcm cryptd pata_acpi glue_helper sysfillrect snd_timer snd soundcore joydev pcspkr sysimgblt serio_raw fb_sys_fops drm i2c_piix4 ata_piix floppy parport_pc parport ip_tables
[ 100.502758] CPU: 0 PID: 714 Comm: mount Not tainted 5.4.0-rc4-00001-gcea875453cd9f #1
[ 100.504678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 100.506746] RIP: 0010:clear_page_dirty_for_io+0x1cf/0x240
[ 100.508076] Code: ff ff 49 8d 7d 70 48 c7 c6 ff ff ff ff 0f bd 05 2b 19 63 01 8d 14 c5 08 00 00 00 e8 4b f4 27 00 b8 01 00 00 00 e9 20 ff ff ff <0f> 0b 31 c0 e9 17 ff ff ff 48 8b 45 30 48 8d 78 08 e8 6b 70 89 00
[ 100.512627] RSP: 0018:ffff9f608040f8d8 EFLAGS: 00010246
[ 100.513894] RAX: 0000000000000000 RBX: ffffdffa47637b00 RCX: 0000000000000000
[ 100.515657] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffdffa47637b00
[ 100.517411] RBP: ffff8d3459d106b0 R08: 0000000000000000 R09: 0000000000000000
[ 100.519188] R10: 00002005b89958bf R11: 0000000000000000 R12: 0000000000000000
[ 100.520961] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8d3457ea5168
[ 100.522759] FS: 00007f2b4b37de40(0000) GS:ffff8d34bfc00000(0000) knlGS:0000000000000000
[ 100.524793] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 100.526277] CR2: 00007f39f6b33000 CR3: 00000001d1ab8000 CR4: 00000000000406f0
[ 100.528047] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 100.529704] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 100.532772] Call Trace:
[ 100.534718] ? _cond_resched+0x19/0x30
[ 100.536950] io_ctl_prepare_pages+0x139/0x190 [btrfs]
[ 100.539395] __btrfs_write_out_cache+0xad/0x4c0 [btrfs]
[ 100.541818] btrfs_write_out_cache+0x83/0x100 [btrfs]
[ 100.544433] btrfs_start_dirty_block_groups+0x231/0x540 [btrfs]
[ 100.547264] btrfs_commit_transaction+0x119/0xa40 [btrfs]
[ 100.549882] ? start_transaction+0x9b/0x470 [btrfs]
[ 100.552413] btrfs_create_uuid_tree+0x4d/0x150 [btrfs]
[ 100.554954] open_ctree+0x1b3d/0x2080 [btrfs]
[ 100.557411] btrfs_mount_root+0x612/0x760 [btrfs]
[ 100.559888] ? fs_parse+0x4d/0x3d0
[ 100.562037] ? selinux_fs_context_parse_param+0x33/0x80
[ 100.564673] ? legacy_get_tree+0x27/0x40
[ 100.566959] ? btrfs_decode_error+0x20/0x20 [btrfs]
[ 100.569444] legacy_get_tree+0x27/0x40
[ 100.571745] vfs_get_tree+0x25/0xc0
[ 100.573877] fc_mount+0xe/0x30
[ 100.575962] vfs_kern_mount+0x6f/0x80
[ 100.578278] btrfs_mount+0x179/0x910 [btrfs]
[ 100.580626] ? filename_lookup+0xf2/0x180
[ 100.582912] ? fs_lookup_key+0x30/0x50
[ 100.585259] ? fs_parse+0x4d/0x3d0
[ 100.587455] ? cred_has_capability+0x85/0x130
[ 100.589841] ? legacy_get_tree+0x27/0x40
[ 100.592175] legacy_get_tree+0x27/0x40
[ 100.594455] vfs_get_tree+0x25/0xc0
[ 100.596492] do_mount+0x715/0x9a0
[ 100.598487] ? memdup_user+0x4b/0x70
[ 100.600546] ksys_mount+0x80/0xd0
[ 100.602439] __x64_sys_mount+0x21/0x30
[ 100.604404] do_syscall_64+0x5b/0x1d0
[ 100.606303] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 100.608657] RIP: 0033:0x7f2b4aa3a48a
[ 100.610559] Code: 48 8b 0d 11 fa 2a 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d de f9 2a 00 f7 d8 64 89 01 48
[ 100.617153] RSP: 002b:00007ffcf0a4ddd8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5
[ 100.620017] RAX: ffffffffffffffda RBX: 00005617f53ef5c0 RCX: 00007f2b4aa3a48a
[ 100.622779] RDX: 00005617f53ef7a0 RSI: 00005617f53ef7e0 RDI: 00005617f53ef7c0
[ 100.625618] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000020
[ 100.628464] R10: 00000000c0ed0000 R11: 0000000000000206 R12: 00005617f53ef7c0
[ 100.631331] R13: 00005617f53ef7a0 R14: 0000000000000000 R15: 00000000ffffffff
[ 100.634157] ---[ end trace 39ba6cab008599a6 ]---
To reproduce:
# build kernel
cd linux
cp config-5.4.0-rc4-00001-gcea875453cd9f .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
lkp
1 year, 3 months
- 1
- 0
[perf/core] 7fa343b7fd: vm-scalability.median -1.5% regression
by kernel test robot
Greeting,
FYI, we noticed a -1.5% regression of vm-scalability.median due to commit:
commit: 7fa343b7fdc4f351de4e3f28d5c285937dd1f42f ("perf/core: Fix corner case in perf_rotate_context()")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: vm-scalability
on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
with following parameters:
runtime: 300s
size: 512G
test: anon-w-rand
cpufreq_governor: performance
ucode: 0x16
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2019-09-23.cgz/300s/512G/lkp-hsw-4ex1/anon-w-rand/vm-scalability/0x16
commit:
d44248a413 ("perf/core: Rework memory accounting in perf_mmap()")
7fa343b7fd ("perf/core: Fix corner case in perf_rotate_context()")
d44248a41337731a 7fa343b7fdc4f351de4e3f28d5c
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
0:4 -5% 0:4 perf-profile.children.cycles-pp.error_entry
0:4 -4% 0:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
35614 -1.5% 35070 vm-scalability.median
4909819 -1.6% 4832961 vm-scalability.throughput
133541 +5.7% 141118 ± 2% vm-scalability.time.involuntary_context_switches
2731 +3.3% 2821 vm-scalability.time.system_time
35579 +1.4% 36086 vm-scalability.time.user_time
3708205 ±100% -86.9% 487323 ±157% cpuidle.C1E.usage
3698401 ±101% -87.1% 478037 ±161% turbostat.C1E
4973 ± 22% +35.4% 6734 ± 15% numa-vmstat.node2.nr_slab_reclaimable
68267 ± 4% -5.5% 64522 ± 5% numa-vmstat.node3.nr_unevictable
68267 ± 4% -5.5% 64522 ± 5% numa-vmstat.node3.nr_zone_unevictable
19895 ± 22% +35.4% 26937 ± 15% numa-meminfo.node2.KReclaimable
19895 ± 22% +35.4% 26937 ± 15% numa-meminfo.node2.SReclaimable
69426 ± 11% -13.6% 59976 ± 10% numa-meminfo.node3.Slab
273071 ± 4% -5.5% 258090 ± 5% numa-meminfo.node3.Unevictable
196.00 -11.2% 174.00 ± 9% proc-vmstat.nr_isolated_anon
7144 +2.6% 7327 proc-vmstat.nr_mapped
64412 +1.8% 65599 proc-vmstat.nr_shmem
51657 +1.0% 52191 proc-vmstat.nr_slab_unreclaimable
5764 ± 7% +39.9% 8061 ± 8% proc-vmstat.pgactivate
2685 ± 19% +32.9% 3569 sched_debug.cfs_rq:/.runnable_weight.min
-1316030 +24.8% -1642376 sched_debug.cfs_rq:/.spread0.min
631.04 ± 6% -10.1% 567.17 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.max
17178 ± 10% +19.8% 20576 ± 15% sched_debug.cpu.nr_switches.max
6029 ± 3% +22.4% 7379 ± 15% sched_debug.cpu.ttwu_count.max
5111 ± 5% +31.8% 6739 ± 19% sched_debug.cpu.ttwu_local.max
90333 ± 10% +34.4% 121364 ± 11% syscalls.sys_close.max
14504 ± 2% +10.9% 16083 ± 3% syscalls.sys_mmap.med
1.535e+08 ± 11% +5.1e+08 6.604e+08 ± 79% syscalls.sys_mmap.noise.100%
3.79e+08 ± 4% +5.1e+08 8.894e+08 ± 57% syscalls.sys_mmap.noise.2%
3.469e+08 ± 4% +5.1e+08 8.548e+08 ± 60% syscalls.sys_mmap.noise.25%
3.776e+08 ± 4% +5.1e+08 8.875e+08 ± 58% syscalls.sys_mmap.noise.5%
2.896e+08 ± 5% +5.1e+08 7.978e+08 ± 65% syscalls.sys_mmap.noise.50%
198724 ± 36% +42.5% 283239 ± 20% syscalls.sys_write.max
5.145e+08 +0.9% 5.193e+08 perf-stat.i.cache-references
78.93 +5.2 84.10 perf-stat.i.iTLB-load-miss-rate%
394771 ± 9% +29.5% 511163 ± 4% perf-stat.i.iTLB-load-misses
327477 ± 8% -26.8% 239622 ± 5% perf-stat.i.iTLB-loads
5408504 ± 9% +13.6% 6141622 ± 6% perf-stat.i.node-store-misses
54.74 ± 7% +13.5 68.20 ± 2% perf-stat.overall.iTLB-load-miss-rate%
109266 ± 10% -23.4% 83711 ± 4% perf-stat.overall.instructions-per-iTLB-miss
1.17 ± 10% +0.2 1.33 ± 6% perf-stat.overall.node-store-miss-rate%
5.126e+08 +0.9% 5.172e+08 perf-stat.ps.cache-references
392782 ± 9% +29.4% 508415 ± 3% perf-stat.ps.iTLB-load-misses
323887 ± 8% -26.8% 237034 ± 5% perf-stat.ps.iTLB-loads
5327487 ± 9% +13.6% 6051553 ± 6% perf-stat.ps.node-store-misses
134501 ± 5% +11.1% 149465 ± 2% softirqs.CPU109.TIMER
135152 ± 5% +10.7% 149653 ± 2% softirqs.CPU113.TIMER
131528 ± 4% +13.0% 148634 ± 3% softirqs.CPU114.TIMER
26149 ± 8% +17.2% 30651 softirqs.CPU16.RCU
137403 ± 5% +8.8% 149539 ± 2% softirqs.CPU16.TIMER
124932 ± 6% +8.1% 135007 ± 5% softirqs.CPU20.TIMER
133590 ± 5% +13.3% 151311 ± 2% softirqs.CPU24.TIMER
135781 ± 5% +12.3% 152519 softirqs.CPU37.TIMER
125112 ± 3% +18.4% 148121 ± 2% softirqs.CPU40.TIMER
135323 ± 4% +11.8% 151302 softirqs.CPU41.TIMER
134035 ± 4% +13.7% 152358 softirqs.CPU42.TIMER
131180 ± 3% +10.4% 144805 ± 3% softirqs.CPU43.TIMER
142460 ± 7% -10.8% 127131 ± 4% softirqs.CPU62.TIMER
141232 ± 8% -11.3% 125314 ± 4% softirqs.CPU68.TIMER
31741 ± 4% +16.2% 36894 ± 6% softirqs.CPU72.RCU
137740 ± 7% +16.7% 160714 ± 5% softirqs.CPU90.TIMER
32909 ± 5% -12.7% 28718 ± 6% softirqs.CPU94.RCU
10850 ± 3% +11.3% 12080 ± 3% softirqs.CPU94.SCHED
131437 ± 5% +13.3% 148956 ± 3% softirqs.CPU96.TIMER
133760 ± 4% +9.1% 145885 ± 4% softirqs.CPU97.TIMER
262.00 ± 48% -33.7% 173.75 interrupts.50:PCI-MSI.1572868-edge.eth0-TxRx-4
465045 +4.6% 486327 interrupts.CAL:Function_call_interrupts
2969 ± 23% -42.4% 1709 interrupts.CPU107.NMI:Non-maskable_interrupts
2969 ± 23% -42.4% 1709 interrupts.CPU107.PMI:Performance_monitoring_interrupts
424.00 ±173% +414.2% 2180 ± 29% interrupts.CPU11.NMI:Non-maskable_interrupts
424.00 ±173% +414.2% 2180 ± 29% interrupts.CPU11.PMI:Performance_monitoring_interrupts
260.75 ± 88% -80.1% 52.00 ± 57% interrupts.CPU115.RES:Rescheduling_interrupts
476.00 ±145% +514.4% 2924 ± 23% interrupts.CPU120.NMI:Non-maskable_interrupts
476.00 ±145% +514.4% 2924 ± 23% interrupts.CPU120.PMI:Performance_monitoring_interrupts
179.00 ± 37% +143.6% 436.00 ± 46% interrupts.CPU13.RES:Rescheduling_interrupts
3366 -39.5% 2037 ± 33% interrupts.CPU131.NMI:Non-maskable_interrupts
3366 -39.5% 2037 ± 33% interrupts.CPU131.PMI:Performance_monitoring_interrupts
80.00 ±104% +2714.4% 2251 ± 30% interrupts.CPU137.NMI:Non-maskable_interrupts
80.00 ±104% +2714.4% 2251 ± 30% interrupts.CPU137.PMI:Performance_monitoring_interrupts
654.00 ± 97% +288.8% 2542 ± 32% interrupts.CPU138.NMI:Non-maskable_interrupts
654.00 ± 97% +288.8% 2542 ± 32% interrupts.CPU138.PMI:Performance_monitoring_interrupts
234.75 ± 60% +616.6% 1682 ± 49% interrupts.CPU140.RES:Rescheduling_interrupts
70.75 ± 47% +301.8% 284.25 ± 49% interrupts.CPU142.RES:Rescheduling_interrupts
2989 ± 24% -36.2% 1908 ± 21% interrupts.CPU16.NMI:Non-maskable_interrupts
2989 ± 24% -36.2% 1908 ± 21% interrupts.CPU16.PMI:Performance_monitoring_interrupts
836.25 ± 99% +214.6% 2631 ± 27% interrupts.CPU19.NMI:Non-maskable_interrupts
836.25 ± 99% +214.6% 2631 ± 27% interrupts.CPU19.PMI:Performance_monitoring_interrupts
3038 ± 4% +12.2% 3409 interrupts.CPU23.CAL:Function_call_interrupts
1250 ±110% +132.8% 2910 ± 25% interrupts.CPU24.NMI:Non-maskable_interrupts
1250 ±110% +132.8% 2910 ± 25% interrupts.CPU24.PMI:Performance_monitoring_interrupts
1696 ± 69% +95.6% 3316 ± 2% interrupts.CPU3.NMI:Non-maskable_interrupts
1696 ± 69% +95.6% 3316 ± 2% interrupts.CPU3.PMI:Performance_monitoring_interrupts
341.75 ± 79% +399.2% 1706 ± 73% interrupts.CPU37.RES:Rescheduling_interrupts
667.00 ± 82% -76.0% 159.75 ± 61% interrupts.CPU39.RES:Rescheduling_interrupts
262.00 ± 48% -33.7% 173.75 interrupts.CPU4.50:PCI-MSI.1572868-edge.eth0-TxRx-4
407.50 ±172% +414.8% 2098 ± 29% interrupts.CPU42.NMI:Non-maskable_interrupts
407.50 ±172% +414.8% 2098 ± 29% interrupts.CPU42.PMI:Performance_monitoring_interrupts
639.75 ±102% +267.6% 2351 ± 29% interrupts.CPU43.NMI:Non-maskable_interrupts
639.75 ±102% +267.6% 2351 ± 29% interrupts.CPU43.PMI:Performance_monitoring_interrupts
171.25 ± 12% +162.8% 450.00 ± 64% interrupts.CPU44.RES:Rescheduling_interrupts
846.25 ± 99% +202.2% 2557 ± 29% interrupts.CPU45.NMI:Non-maskable_interrupts
846.25 ± 99% +202.2% 2557 ± 29% interrupts.CPU45.PMI:Performance_monitoring_interrupts
1087 ± 64% +166.0% 2891 ± 24% interrupts.CPU53.NMI:Non-maskable_interrupts
1087 ± 64% +166.0% 2891 ± 24% interrupts.CPU53.PMI:Performance_monitoring_interrupts
94.75 ± 53% +402.9% 476.50 ± 54% interrupts.CPU68.RES:Rescheduling_interrupts
946.50 ± 81% +161.1% 2471 ± 31% interrupts.CPU7.NMI:Non-maskable_interrupts
946.50 ± 81% +161.1% 2471 ± 31% interrupts.CPU7.PMI:Performance_monitoring_interrupts
590.75 ±162% +395.4% 2926 ± 25% interrupts.CPU88.NMI:Non-maskable_interrupts
590.75 ±162% +395.4% 2926 ± 25% interrupts.CPU88.PMI:Performance_monitoring_interrupts
858.75 ± 96% +233.1% 2860 ± 23% interrupts.CPU91.NMI:Non-maskable_interrupts
858.75 ± 96% +233.1% 2860 ± 23% interrupts.CPU91.PMI:Performance_monitoring_interrupts
99.00 ± 99% +344.7% 440.25 ± 76% interrupts.CPU94.RES:Rescheduling_interrupts
252147 ± 9% +39.3% 351310 interrupts.NMI:Non-maskable_interrupts
252147 ± 9% +39.3% 351310 interrupts.PMI:Performance_monitoring_interrupts
741.75 ± 10% -16.3% 621.00 ± 14% interrupts.TLB:TLB_shootdowns
50.17 ± 4% -4.3 45.90 ± 2% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
28.40 ± 13% -3.7 24.72 ± 4% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
28.57 ± 13% -3.6 24.93 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
38.27 ± 8% -3.6 34.66 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
61.59 ± 4% -3.5 58.09 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt
64.08 ± 4% -3.3 60.74 perf-profile.calltrace.cycles-pp.apic_timer_interrupt
3.09 ± 35% -1.2 1.86 ± 5% perf-profile.calltrace.cycles-pp.run_local_timers.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
3.43 ± 14% -0.7 2.75 ± 5% perf-profile.calltrace.cycles-pp.account_user_time.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
2.42 ± 16% -0.6 1.79 ± 16% perf-profile.calltrace.cycles-pp.hrtimer_active.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle
1.08 ± 22% -0.6 0.53 ± 58% perf-profile.calltrace.cycles-pp.memcpy_erms.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread
1.24 ± 21% -0.5 0.76 ± 16% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
1.23 ± 20% -0.5 0.75 ± 15% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
1.17 ± 20% -0.5 0.71 ± 16% perf-profile.calltrace.cycles-pp.drm_fb_helper_dirty_work.process_one_work.worker_thread.kthread.ret_from_fork
0.83 ± 8% -0.3 0.49 ± 58% perf-profile.calltrace.cycles-pp.read_tsc.ktime_get_update_offsets_now.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
1.48 ± 12% +0.3 1.79 ± 7% perf-profile.calltrace.cycles-pp.fpregs_assert_state_consistent.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
2.09 ± 5% +0.4 2.47 ± 12% perf-profile.calltrace.cycles-pp.native_write_msr.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt
2.68 ± 5% +0.5 3.13 ± 10% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
19.59 ± 3% +1.8 21.36 ± 6% perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode
15.36 ± 4% +2.0 17.40 ± 6% perf-profile.calltrace.cycles-pp.prepare_exit_to_usermode.swapgs_restore_regs_and_return_to_usermode
50.31 ± 4% -4.3 46.04 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt
28.46 ± 13% -3.7 24.78 ± 4% perf-profile.children.cycles-pp.update_process_times
28.60 ± 13% -3.6 24.96 ± 4% perf-profile.children.cycles-pp.tick_sched_handle
38.36 ± 8% -3.6 34.78 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
61.73 ± 4% -3.5 58.27 perf-profile.children.cycles-pp.smp_apic_timer_interrupt
64.31 ± 4% -3.4 60.93 perf-profile.children.cycles-pp.apic_timer_interrupt
3.09 ± 34% -1.2 1.88 ± 5% perf-profile.children.cycles-pp.run_local_timers
3.43 ± 14% -0.7 2.75 ± 5% perf-profile.children.cycles-pp.account_user_time
2.42 ± 16% -0.6 1.81 ± 17% perf-profile.children.cycles-pp.hrtimer_active
1.99 ± 14% -0.6 1.38 ± 9% perf-profile.children.cycles-pp.irq_work_run
1.99 ± 14% -0.6 1.38 ± 9% perf-profile.children.cycles-pp.printk
1.99 ± 14% -0.6 1.38 ± 9% perf-profile.children.cycles-pp.vprintk_emit
2.04 ± 14% -0.6 1.46 ± 7% perf-profile.children.cycles-pp.irq_work_run_list
1.95 ± 13% -0.6 1.38 ± 9% perf-profile.children.cycles-pp.irq_work_interrupt
1.95 ± 13% -0.6 1.38 ± 9% perf-profile.children.cycles-pp.smp_irq_work_interrupt
1.24 ± 21% -0.5 0.76 ± 16% perf-profile.children.cycles-pp.worker_thread
1.23 ± 20% -0.5 0.75 ± 15% perf-profile.children.cycles-pp.process_one_work
1.17 ± 20% -0.5 0.71 ± 16% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
1.15 ± 17% -0.4 0.70 ± 16% perf-profile.children.cycles-pp.memcpy_erms
0.28 ± 9% -0.1 0.18 ± 23% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.23 ± 27% -0.1 0.15 ± 27% perf-profile.children.cycles-pp.run_rebalance_domains
0.04 ± 59% +0.0 0.09 ± 17% perf-profile.children.cycles-pp.new_sync_read
0.04 ± 57% +0.1 0.09 ± 30% perf-profile.children.cycles-pp.force_qs_rnp
0.03 ±100% +0.1 0.10 ± 48% perf-profile.children.cycles-pp.__do_munmap
0.01 ±173% +0.1 0.09 ± 26% perf-profile.children.cycles-pp.select_idle_sibling
0.10 ± 31% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.45 ± 24% +0.1 0.60 ± 14% perf-profile.children.cycles-pp.tick_program_event
0.24 ± 26% +0.1 0.39 ± 27% perf-profile.children.cycles-pp.load_elf_binary
0.24 ± 26% +0.2 0.40 ± 26% perf-profile.children.cycles-pp.search_binary_handler
0.41 ± 26% +0.2 0.61 ± 16% perf-profile.children.cycles-pp.__do_execve_file
0.07 ± 58% +0.2 0.26 ± 74% perf-profile.children.cycles-pp.rcu_gp_kthread
0.42 ± 28% +0.2 0.61 ± 15% perf-profile.children.cycles-pp.__x64_sys_execve
0.41 ± 29% +0.2 0.61 ± 18% perf-profile.children.cycles-pp.execve
0.23 ± 40% +0.2 0.45 ± 21% perf-profile.children.cycles-pp.read
1.51 ± 11% +0.3 1.81 ± 6% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.14 ± 37% +0.3 0.45 ± 45% perf-profile.children.cycles-pp.schedule
0.14 ± 37% +0.3 0.46 ± 43% perf-profile.children.cycles-pp.__schedule
2.71 ± 5% +0.4 3.15 ± 10% perf-profile.children.cycles-pp.lapic_next_deadline
19.62 ± 3% +1.8 21.43 ± 6% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
15.54 ± 4% +2.0 17.59 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode
3.08 ± 35% -1.2 1.85 ± 6% perf-profile.self.cycles-pp.run_local_timers
2.40 ± 17% -0.6 1.78 ± 18% perf-profile.self.cycles-pp.hrtimer_active
2.48 ± 10% -0.6 1.87 ± 4% perf-profile.self.cycles-pp.account_user_time
0.33 ± 16% -0.1 0.21 ± 30% perf-profile.self.cycles-pp.delay_tsc
0.28 ± 9% -0.1 0.18 ± 23% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.03 ±100% +0.1 0.08 ± 36% perf-profile.self.cycles-pp.clear_page_erms
1.19 ± 9% +0.1 1.32 ± 6% perf-profile.self.cycles-pp.run_timer_softirq
vm-scalability.throughput
5.2e+06 +-+--------------------------------------------------------------+
| + + |
5.15e+06 +-+ : : |
|: : : : : |
5.1e+06 +-+ : : : : |
5.05e+06 +-+ : : : : |
| : : : : : |
5e+06 +-+ : : : : |
| : : : : : |
4.95e+06 +-+: : : : : |
4.9e+06 +-++..+..+..+..+..+..+..+ +..+.+..+ +..+..+..+..+..+..+..|
| |
4.85e+06 +-+O O O
O O O O O O O O O O O O O O O O O O O |
4.8e+06 +-+-----------------------------------------------O--------------+
vm-scalability.median
36000 +-+-----------------------------------------------------------------+
| + + |
35800 +-+ + : .. : |
| : +.. .+..+... + : +.. .+ : +... .+.. |
| : .. +. +.. .+ : .. +. : .. +. +..+.. |
35600 +-++ +. + + +..|
| |
35400 +-+ |
| |
35200 +-+ O O
| O O O O O |
| O O O O O O O |
35000 +-+ O O O O O |
O O O |
34800 +-+--------------------------------------------------O--------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Oliver Sang
1 year, 3 months
- 1
- 0
Re: [PATCH] seccomp: fix SECCOMP_USER_NOTIF_FLAG_CONTINUE test
by Kees Cook
On Mon, Oct 21, 2019 at 07:50:13AM -0600, Tycho Andersen wrote:
> On Mon, Oct 21, 2019 at 11:10:55AM +0200, Christian Brauner wrote:
> > The ifndef for SECCOMP_USER_NOTIF_FLAG_CONTINUE was placed under the
> > ifndef for the SECCOMP_FILTER_FLAG_NEW_LISTENER feature. This will not
> > work on systems that do support SECCOMP_FILTER_FLAG_NEW_LISTENER but do not
> > support SECCOMP_USER_NOTIF_FLAG_CONTINUE. So move the latter ifndef out of
> > the former ifndef's scope.
> >
> > 2019-10-20 11:14:01 make run_tests -C seccomp
> > make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-0eebfed2954f152259cae0ad57b91d3ea92968e8/tools/testing/selftests/seccomp'
> > gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
> > seccomp_bpf.c: In function ‘user_notification_continue’:
> > seccomp_bpf.c:3562:15: error: ‘SECCOMP_USER_NOTIF_FLAG_CONTINUE’ undeclared (first use in this function)
> > resp.flags = SECCOMP_USER_NOTIF_FLAG_CONTINUE;
> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > seccomp_bpf.c:3562:15: note: each undeclared identifier is reported only once for each function it appears in
> > Makefile:12: recipe for target 'seccomp_bpf' failed
> > make: *** [seccomp_bpf] Error 1
> > make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-0eebfed2954f152259cae0ad57b91d3ea92968e8/tools/testing/selftests/seccomp'
> >
> > Reported-by: kernel test robot <rong.a.chen(a)intel.com>
> > Fixes: 0eebfed2954f ("seccomp: test SECCOMP_USER_NOTIF_FLAG_CONTINUE")
> > Cc: linux-kselftest(a)vger.kernel.org
> > Signed-off-by: Christian Brauner <christian.brauner(a)ubuntu.com>
>
> Reviewed-by: Tycho Andersen <tycho(a)tycho.ws>
Thanks! Applied to my for-next/seccomp tree.
--
Kees Cook
1 year, 3 months
- 1
- 0
28875945ba ("rcu: Add support for consolidated-RCU reader .."): WARNING: suspicious RCU usage
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
commit 28875945ba98d1b47a8a706812b6494d165bb0a0
Author: Joel Fernandes (Google) <joel(a)joelfernandes.org>
AuthorDate: Tue Jul 16 18:12:22 2019 -0400
Commit: Paul E. McKenney <paulmck(a)linux.ibm.com>
CommitDate: Fri Aug 9 11:00:35 2019 -0700
rcu: Add support for consolidated-RCU reader checking
This commit adds RCU-reader checks to list_for_each_entry_rcu() and
hlist_for_each_entry_rcu(). These checks are optional, and are indicated
by a lockdep expression passed to a new optional argument to these two
macros. If this optional lockdep expression is omitted, these two macros
act as before, checking for an RCU read-side critical section.
Signed-off-by: Joel Fernandes (Google) <joel(a)joelfernandes.org>
[ paulmck: Update to eliminate return within macro and update comment. ]
Signed-off-by: Paul E. McKenney <paulmck(a)linux.ibm.com>
9147089bee rcu: Remove redundant debug_locks check in rcu_read_lock_sched_held()
28875945ba rcu: Add support for consolidated-RCU reader checking
7d194c2100 Linux 5.4-rc4
c4b9850b36 Add linux-next specific files for 20191018
+--------------------------------------------------------------------------+------------+------------+----------+---------------+
| | 9147089bee | 28875945ba | v5.4-rc4 | next-20191018 |
+--------------------------------------------------------------------------+------------+------------+----------+---------------+
| boot_successes | 30 | 0 | 0 | 0 |
| boot_failures | 0 | 11 | 22 | 15 |
| WARNING:suspicious_RCU_usage | 0 | 11 | 22 | 15 |
| kernel/workqueue.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | 22 | 15 |
| kernel/events/core.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | 22 | 15 |
| drivers/base/power/runtime.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | | |
| drivers/acpi/osl.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | | |
| arch/x86/pci/mmconfig-shared.c:#RCU-list_traversed_in_non-reader_section | 0 | 11 | | |
| kernel/module.c:#RCU-list_traversed_in_non-reader_section | 0 | 10 | 22 | 15 |
| kernel/kprobes.c:#RCU-list_traversed_in_non-reader_section | 0 | 1 | 2 | 1 |
| net/ipv4/fib_trie.c:#RCU-list_traversed_in_non-reader_section | 0 | 2 | 5 | 5 |
+--------------------------------------------------------------------------+------------+------------+----------+---------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 4.186589] Spectre V2 : Spectre mitigation: kernel not compiled with retpoline; no mitigation available!
[ 4.186592] Speculative Store Bypass: Vulnerable
[ 4.188777] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[ 4.194116]
[ 4.194465] =============================
[ 4.195246] WARNING: suspicious RCU usage
[ 4.195903] 5.3.0-rc2-00003-g28875945ba98d #1 Tainted: G T
[ 4.197186] -----------------------------
[ 4.197956] kernel/workqueue.c:4285 RCU-list traversed in non-reader section!!
[ 4.199899]
[ 4.199899] other info that might help us debug this:
[ 4.199899]
[ 4.201401]
[ 4.201401] rcu_scheduler_active = 2, debug_locks = 1
[ 4.202629] 2 locks held by swapper/1:
[ 4.203366] #0: (____ptrval____) (wq_pool_mutex){+.+.}, at: alloc_workqueue+0x3be/0x7e0
[ 4.203899] #1: (____ptrval____) (&wq->mutex){+.+.}, at: alloc_workqueue+0x3c8/0x7e0
[ 4.205383]
[ 4.205383] stack backtrace:
[ 4.206235] CPU: 0 PID: 1 Comm: swapper Tainted: G T 5.3.0-rc2-00003-g28875945ba98d #1
[ 4.207900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 4.209478] Call Trace:
[ 4.209975] dump_stack+0x19/0x25
[ 4.211908] lockdep_rcu_suspicious+0xd6/0xe0
[ 4.212761] alloc_workqueue+0x763/0x7e0
[ 4.213576] init_mm_internals+0x2c/0xd7
[ 4.214341] kernel_init_freeable+0xa7/0x387
[ 4.215188] ? rest_init+0x160/0x160
[ 4.215913] kernel_init+0x13/0x130
[ 4.216596] ? rest_init+0x160/0x160
[ 4.217289] ? rest_init+0x160/0x160
[ 4.217984] ret_from_fork+0x24/0x30
[ 4.218880] Performance Events: unsupported p6 CPU model 60 no PMU driver, software events only.
[ 4.220448]
[ 4.220778] =============================
[ 4.221546] WARNING: suspicious RCU usage
[ 4.222334] 5.3.0-rc2-00003-g28875945ba98d #1 Tainted: G T
[ 4.223894] -----------------------------
[ 4.224664] kernel/events/core.c:10151 RCU-list traversed in non-reader section!!
[ 4.226405]
[ 4.226405] other info that might help us debug this:
[ 4.226405]
[ 4.227894]
[ 4.227894] rcu_scheduler_active = 2, debug_locks = 1
[ 4.229120] 1 lock held by swapper/1:
[ 4.229821] #0: (____ptrval____) (&pmus_srcu){....}, at: perf_event_alloc+0x9e5/0x15e0
[ 4.231893]
[ 4.231893] stack backtrace:
[ 4.232728] CPU: 0 PID: 1 Comm: swapper Tainted: G T 5.3.0-rc2-00003-g28875945ba98d #1
[ 4.234452] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 4.235893] Call Trace:
[ 4.236386] dump_stack+0x19/0x25
[ 4.237031] lockdep_rcu_suspicious+0xd6/0xe0
[ 4.237877] perf_event_alloc+0x158d/0x15e0
[ 4.238723] perf_event_create_kernel_counter+0x21/0x1f0
[ 4.239916] hardlockup_detector_event_create+0x44/0x80
[ 4.240921] hardlockup_detector_perf_init+0x28/0x8e
[ 4.241866] watchdog_nmi_probe+0x1c/0x40
[ 4.242655] lockup_detector_init+0x40/0xad
[ 4.243894] kernel_init_freeable+0x102/0x387
[ 4.244470] ? rest_init+0x160/0x160
[ 4.244928] kernel_init+0x13/0x130
[ 4.245369] ? rest_init+0x160/0x160
[ 4.245818] ? rest_init+0x160/0x160
[ 4.246271] ret_from_fork+0x24/0x30
[ 4.246806] NMI watchdog: Perf NMI watchdog permanently disabled
[ 4.247677] TSC deadline timer enabled
[ 4.247876] devtmpfs: initialized
[ 4.248179] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 4.249411] futex hash table entries: 256 (order: 2, 24576 bytes, linear)
[ 4.250702] pinctrl core: initialized pinctrl subsystem
[ 4.252363]
[ 4.252583] =============================
[ 4.253082] WARNING: suspicious RCU usage
[ 4.253580] 5.3.0-rc2-00003-g28875945ba98d #1 Tainted: G T
[ 4.254437] -----------------------------
[ 4.254965] drivers/base/power/runtime.c:1665 RCU-list traversed in non-reader section!!
[ 4.255894]
[ 4.255894] other info that might help us debug this:
[ 4.255894]
[ 4.256878]
[ 4.256878] rcu_scheduler_active = 2, debug_locks = 1
[ 4.257717] 2 locks held by swapper/1:
[ 4.258225] #0: (____ptrval____) (&dev->mutex){....}, at: __device_driver_lock+0x59/0x60
[ 4.259300] #1: (____ptrval____) (device_links_srcu){....}, at: device_links_read_lock+0x5/0x80
[ 4.259892]
[ 4.259892] stack backtrace:
[ 4.260440] CPU: 0 PID: 1 Comm: swapper Tainted: G T 5.3.0-rc2-00003-g28875945ba98d #1
[ 4.261583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 4.262731] Call Trace:
[ 4.263053] dump_stack+0x19/0x25
[ 4.263477] lockdep_rcu_suspicious+0xd6/0xe0
[ 4.263905] pm_runtime_get_suppliers+0xdd/0xf0
[ 4.264482] driver_probe_device+0x3c/0x170
[ 4.265015] device_driver_attach+0x7e/0x90
[ 4.265574] __driver_attach+0xa2/0x100
[ 4.266096] ? device_driver_attach+0x90/0x90
[ 4.266647] bus_for_each_dev+0x9b/0xf0
[ 4.267144] driver_attach+0x2b/0x30
[ 4.267615] bus_add_driver+0x23b/0x310
[ 4.267906] driver_register+0xd3/0x1c0
[ 4.268399] __platform_driver_register+0x6c/0x80
[ 4.268993] regulator_dummy_init+0xb2/0x106
[ 4.269566] regulator_init+0xbb/0xf0
[ 4.270043] ? regulator_late_cleanup+0x35c/0x35c
[ 4.270637] do_one_initcall+0x79/0x37a
[ 4.271128] ? rcu_read_lock_sched_held+0x6d/0x90
[ 4.271724] kernel_init_freeable+0x258/0x387
[ 4.271899] ? rest_init+0x160/0x160
[ 4.272361] kernel_init+0x13/0x130
[ 4.272827] ? rest_init+0x160/0x160
[ 4.273302] ? rest_init+0x160/0x160
[ 4.273755] ret_from_fork+0x24/0x30
[ 4.275194]
[ 4.275421] =============================
[ 4.275893] WARNING: suspicious RCU usage
[ 4.276399] 5.3.0-rc2-00003-g28875945ba98d #1 Tainted: G T
[ 4.277371] -----------------------------
[ 4.277874] drivers/base/power/runtime.c:1686 RCU-list traversed in non-reader section!!
[ 4.279109]
[ 4.279109] other info that might help us debug this:
[ 4.279109]
[ 4.279893]
[ 4.279893] rcu_scheduler_active = 2, debug_locks = 1
[ 4.280724] 2 locks held by swapper/1:
[ 4.281212] #0: (____ptrval____) (&dev->mutex){....}, at: __device_driver_lock+0x59/0x60
[ 4.282209] #1: (____ptrval____) (device_links_srcu){....}, at: device_links_read_lock+0x5/0x80
[ 4.283282]
[ 4.283282] stack backtrace:
[ 4.283826] CPU: 0 PID: 1 Comm: swapper Tainted: G T 5.3.0-rc2-00003-g28875945ba98d #1
[ 4.283892] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 4.284917] Call Trace:
[ 4.285238] dump_stack+0x19/0x25
[ 4.285697] lockdep_rcu_suspicious+0xd6/0xe0
[ 4.286325] pm_runtime_put_suppliers+0xe7/0xf0
[ 4.286894] driver_probe_device+0xb2/0x170
[ 4.287469] device_driver_attach+0x7e/0x90
[ 4.287900] __driver_attach+0xa2/0x100
[ 4.288384] ? device_driver_attach+0x90/0x90
[ 4.288898] bus_for_each_dev+0x9b/0xf0
[ 4.289391] driver_attach+0x2b/0x30
[ 4.289845] bus_add_driver+0x23b/0x310
[ 4.290335] driver_register+0xd3/0x1c0
[ 4.290821] __platform_driver_register+0x6c/0x80
[ 4.291440] regulator_dummy_init+0xb2/0x106
[ 4.291898] regulator_init+0xbb/0xf0
[ 4.292362] ? regulator_late_cleanup+0x35c/0x35c
[ 4.292947] do_one_initcall+0x79/0x37a
[ 4.293435] ? rcu_read_lock_sched_held+0x6d/0x90
[ 4.294029] kernel_init_freeable+0x258/0x387
[ 4.294574] ? rest_init+0x160/0x160
[ 4.295054] kernel_init+0x13/0x130
[ 4.295534] ? rest_init+0x160/0x160
[ 4.295894] ? rest_init+0x160/0x160
[ 4.296347] ret_from_fork+0x24/0x30
[ 4.297742] NET: Registered protocol family 16
[ 4.301875] cpuidle: using governor ladder
[ 4.302487] cpuidle: using governor menu
[ 4.303635] ACPI: bus type PCI registered
[ 4.304016] PCI: Using configuration type 1 for base access
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 4f5cafb5cb8471e54afdc9054d973535614f7675 v5.3 --
git bisect bad 3c6a6910a81eae3566bb5fef6ea0f624382595e6 # 14:27 B 0 22 38 0 Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
git bisect bad 1f7d290a7275edb270dbee13212c37cb59940221 # 14:28 B 0 22 38 0 Merge tag 'driver-core-5.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core
git bisect bad e0d60a1e68a3fbf42cdf3546004e00230d9048ba # 14:28 B 0 22 38 0 Merge branch 'x86-entry-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good d0a16fe934383ecdb605ab9312d700fb9099f75e # 14:28 G 21 0 0 1 Merge branch 'parisc-5.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
git bisect bad 98c82b4b8be60b05bc96aa4ab664ca0b0e39001f # 14:28 B 0 22 38 0 Merge branch 'core-stacktrace-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good cef7298262e9af841fb70d8673af45caf55300a1 # 14:28 G 21 0 0 1 Merge tag 'armsoc-dt' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good d75a43c645c26ab58118bd35405666a12971350d # 14:28 G 22 0 0 0 Merge branch 'core-objtool-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 94d18ee9340e00ee3455bb45661484093e3b2674 # 14:28 B 0 22 38 0 Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad ce5215c1342c6c89b3c3c45fea82cddf0b013787 # 14:28 B 0 22 38 0 rcu/nocb: Use separate flag to indicate offloaded ->cblist
git bisect good ba31ebfa7b749906e0befcc1e0c0db5e7463d55e # 14:28 G 22 0 0 0 MAINTAINERS: Update e-mail address for Andrea Parri
git bisect bad 31da067023dd0e35c5ec5556f0be7a31e5588277 # 14:28 B 0 22 38 0 Merge branches 'consolidate.2019.08.01b', 'fixes.2019.08.12a', 'lists.2019.08.13a' and 'torture.2019.08.01b' into HEAD
git bisect bad bee6f87166e9c6b8d81a7570995bd637e8da485a # 14:28 B 0 22 38 0 acpi: Use built-in RCU list checking for acpi_ioremaps list
git bisect bad fbab8d6735e2643365040bd9e1057addc0d9b4cf # 14:28 B 0 11 27 0 rcu/sync: Remove custom check for RCU readers
git bisect good 9147089bee3a6b504821dd8462e2be229e6dbfae # 14:29 G 22 0 0 0 rcu: Remove redundant debug_locks check in rcu_read_lock_sched_held()
git bisect bad 28875945ba98d1b47a8a706812b6494d165bb0a0 # 14:29 B 0 11 27 0 rcu: Add support for consolidated-RCU reader checking
# first bad commit: [28875945ba98d1b47a8a706812b6494d165bb0a0] rcu: Add support for consolidated-RCU reader checking
git bisect good 9147089bee3a6b504821dd8462e2be229e6dbfae # 14:32 G 30 0 0 0 rcu: Remove redundant debug_locks check in rcu_read_lock_sched_held()
# extra tests on HEAD of linux-devel/devel-hourly-2019101909
git bisect bad 2fcbef48534d2d6e9f20773312e7d6eec2f91617 # 14:33 B 0 34 54 1 0day head guard for 'devel-hourly-2019101909'
# extra tests on tree/branch linus/master
git bisect bad 7d194c2100ad2a6dded545887d02754948ca5241 # 17:17 B 0 1 28 11 Linux 5.4-rc4
# extra tests with first bad commit reverted
# extra tests on tree/branch linux-next/master
git bisect bad c4b9850b3676869ac0def5885d781d17f64b3a86 # 20:14 B 0 3 30 11 Add linux-next specific files for 20191018
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation
1 year, 3 months
- 1
- 0
[net] 635f03c839: WARNING:at_net/core/datagram.c:#__skb_datagram_iter
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 635f03c839fb2c207dc2e2cea65d382a0798abea ("[PATCH] net: core: datagram: tidy up copy functions a bit")
url: https://github.com/0day-ci/linux/commits/Vito-Caputo/net-core-datagram-ti...
in testcase: packetdrill
with following parameters:
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-----------------------------------------------------+------------+------------+
| | 524900a212 | 635f03c839 |
+-----------------------------------------------------+------------+------------+
| boot_successes | 15 | 8 |
| boot_failures | 0 | 20 |
| WARNING:at_net/core/datagram.c:#__skb_datagram_iter | 0 | 20 |
| RIP:__skb_datagram_iter | 0 | 20 |
+-----------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 24.142627] WARNING: CPU: 0 PID: 5761 at net/core/datagram.c:434 __skb_datagram_iter+0x1bb/0x280
[ 24.143997] Modules linked in: tun intel_rapl_msr intel_rapl_common sr_mod cdrom crct10dif_pclmul sg crc32_pclmul ata_generic crc32c_intel pata_acpi ghash_clmulni_intel bochs_drm drm_vram_helper ttm drm_kms_helper ppdev syscopyarea sysfillrect sysimgblt fb_sys_fops drm snd_pcm aesni_intel crypto_simd snd_timer cryptd snd glue_helper soundcore joydev pcspkr serio_raw ata_piix libata i2c_piix4 floppy parport_pc parport ip_tables
[ 24.149092] CPU: 0 PID: 5761 Comm: packetdrill Not tainted 5.4.0-rc1-00465-g635f03c839fb2 #1
[ 24.150229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 24.151339] RIP: 0010:__skb_datagram_iter+0x1bb/0x280
[ 24.152045] Code: f2 fe ff ff 48 8b 44 24 28 44 8b 44 24 20 4c 8b 4c 24 18 44 8b 5c 24 14 8b 90 bc 00 00 00 48 8b 80 c0 00 00 00 e9 13 ff ff ff <0f> 0b e9 43 ff ff ff 4c 89 e1 4c 8b 69 08 4d 85 ed 74 6f 44 0f b6
[ 24.154518] RSP: 0018:ffffac078290faf8 EFLAGS: 00010297
[ 24.155226] RAX: ffff96ed8afac800 RBX: 0000000000000000 RCX: ffff96ed8afacac0
[ 24.156162] RDX: 00000000000002c0 RSI: 0000000000000000 RDI: ffff96ed06da6d00
[ 24.157119] RBP: 0000000000000000 R08: 0000000000000410 R09: 0000000000000030
[ 24.158036] R10: ffffac078290fc50 R11: 0000000000000000 R12: ffff96ed8afacaf0
[ 24.158979] R13: 0000000000000034 R14: 0000000000000000 R15: 0000000000000034
[ 24.159919] FS: 0000000001e39880(0000) GS:ffff96edffc00000(0000) knlGS:0000000000000000
[ 24.160991] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 24.161750] CR2: 000000000188d1e0 CR3: 00000001df2de000 CR4: 00000000000406f0
[ 24.162699] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 24.163646] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 24.164589] Call Trace:
[ 24.164928] ? skb_copy_datagram_iter+0xa0/0xa0
[ 24.165580] skb_copy_datagram_iter+0x3b/0xa0
[ 24.166183] ipv6_recv_error+0xa1/0x3d0
[ 24.166718] ? ip_rcv_finish_core+0x10c/0x390
[ 24.167434] ? ip_rcv_finish+0x62/0x90
[ 24.167966] ? ip_rcv+0xd1/0xe0
[ 24.168383] ? tcp_recvmsg+0x791/0xb90
[ 24.168875] ? ip6_datagram_recv_specific_ctl+0x580/0x580
[ 24.169602] tcp_recvmsg+0x791/0xb90
[ 24.170078] ? rw_copy_check_uvector+0x5b/0x150
[ 24.170705] ? import_iovec+0x36/0xf0
[ 24.171214] inet6_recvmsg+0x5d/0x100
[ 24.171703] ___sys_recvmsg+0xf5/0x240
[ 24.172226] ? __might_fault+0x2b/0x30
[ 24.172748] ? __might_fault+0x2b/0x30
[ 24.173271] ? sock_getsockopt+0x3d4/0xb90
[ 24.173842] ? __sys_recvmsg+0x5b/0xa0
[ 24.174332] __sys_recvmsg+0x5b/0xa0
[ 24.174828] do_syscall_64+0x5b/0x1d0
[ 24.175901] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 24.177116] RIP: 0033:0x42db60
[ 24.178108] Code: 40 00 41 54 55 41 89 d4 53 48 89 f5 89 fb 48 83 ec 10 e8 03 f7 ff ff 48 63 fb 41 89 c0 49 63 d4 48 89 ee b8 2f 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 1b 44 89 c7 48 89 44 24 08 e8 3b f7 ff ff 48
[ 24.181690] RSP: 002b:00007ffd8ff5dff0 EFLAGS: 00000293 ORIG_RAX: 000000000000002f
[ 24.183219] RAX: ffffffffffffffda RBX: 0000000000000009 RCX: 000000000042db60
[ 24.184724] RDX: 0000000000002000 RSI: 0000000001e40d90 RDI: 0000000000000009
[ 24.186296] RBP: 0000000001e40d90 R08: 0000000000000000 R09: 00000000000007e0
[ 24.187797] R10: 0000000000000040 R11: 0000000000000293 R12: 0000000000002000
[ 24.189236] R13: 0000000000434bc0 R14: 0000000000000000 R15: 0000000000000000
[ 24.190659] ---[ end trace 2cde539dc387f671 ]---
To reproduce:
# build kernel
cd linux
cp config-5.4.0-rc1-00465-g635f03c839fb2 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 3 months
- 1
- 0
[sched/fair] 407039b23c: hackbench.throughput 23.9% improvement
by kernel test robot
Greeting,
FYI, we noticed a 23.9% improvement of hackbench.throughput due to commit:
commit: 407039b23cdec256b143bd4ae59db28e003217d3 ("sched/fair: use load instead of runnable load in wakeup path")
https://git.linaro.org/people/vincent.guittot/kernel.git sched/pelt
in testcase: hackbench
on test machine: 72 threads Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz with 192G memory
with following parameters:
nr_threads: 100%
mode: threads
ipc: pipe
ucode: 0x200005e
cpufreq_governor: performance
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc...
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2019-05-14.cgz/lkp-skl-2sp8/hackbench/0x200005e
commit:
da882bf9d0 ("sched/fair: use utilization to select misfit task")
407039b23c ("sched/fair: use load instead of runnable load in wakeup path")
da882bf9d0c36b3d 407039b23cdec256b143bd4ae59
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:4 75% 3:4 dmesg.WARNING:at_ip__fsnotify_parent/0x
%stddev %change %stddev
\ | \
114012 +23.9% 141286 ± 2% hackbench.throughput
667.63 -8.1% 613.65 ± 2% hackbench.time.elapsed_time
667.63 -8.1% 613.65 ± 2% hackbench.time.elapsed_time.max
4.334e+09 -29.3% 3.065e+09 ± 2% hackbench.time.involuntary_context_switches
40794 -10.8% 36393 hackbench.time.system_time
7049 +6.2% 7487 ± 3% hackbench.time.user_time
6.222e+09 -16.4% 5.203e+09 hackbench.time.voluntary_context_switches
7.776e+08 +13.9% 8.856e+08 ± 4% hackbench.workload
17.56 -1.7% 17.26 boot-time.dhcp
1735 ± 19% -27.7% 1255 ± 30% meminfo.Mlocked
0.61 ± 9% +0.1 0.74 ± 6% mpstat.cpu.all.idle%
14.66 +2.3 16.95 mpstat.cpu.all.usr%
1443 ± 16% +28.9% 1861 ± 6% slabinfo.UNIX.active_objs
1443 ± 16% +28.9% 1861 ± 6% slabinfo.UNIX.num_objs
777.75 ± 7% +14.8% 892.50 ± 9% slabinfo.mnt_cache.active_objs
777.75 ± 7% +14.8% 892.50 ± 9% slabinfo.mnt_cache.num_objs
84.00 -2.7% 81.75 vmstat.cpu.sy
14.00 +16.1% 16.25 ± 2% vmstat.cpu.us
15749870 -14.6% 13442537 ± 2% vmstat.system.cs
944907 -7.4% 875163 vmstat.system.in
27978345 ± 26% +68.5% 47153512 ± 8% cpuidle.C1.time
1888044 ± 12% +166.8% 5036504 ± 13% cpuidle.C1.usage
588012 ± 25% +45.4% 855195 ± 5% cpuidle.C1E.usage
1422630 +984.4% 15426349 ± 20% cpuidle.POLL.time
639993 +1508.1% 10291511 ± 18% cpuidle.POLL.usage
11514622 ± 12% +115.6% 24830286 ± 12% numa-numastat.node0.local_node
11522779 ± 12% +115.5% 24830342 ± 12% numa-numastat.node0.numa_hit
9409492 ± 6% +155.5% 24038395 ± 7% numa-numastat.node1.local_node
9424658 ± 6% +155.3% 24061675 ± 7% numa-numastat.node1.numa_hit
15168 ± 33% +53.5% 23286 numa-numastat.node1.other_node
1885855 ± 12% +167.0% 5035021 ± 13% turbostat.C1
0.06 ± 25% +0.0 0.11 ± 8% turbostat.C1%
585468 ± 25% +46.1% 855097 ± 5% turbostat.C1E
0.94 ± 7% +11.5% 1.04 ± 5% turbostat.CPU%c1
6.527e+08 ± 2% -14.4% 5.586e+08 ± 2% turbostat.IRQ
240.50 -13.9% 207.00 ± 8% numa-vmstat.node0.nr_inactive_file
240.50 -13.9% 207.00 ± 8% numa-vmstat.node0.nr_zone_inactive_file
6720375 ± 8% +104.2% 13725722 ± 10% numa-vmstat.node0.numa_hit
6712292 ± 8% +103.0% 13625140 ± 10% numa-vmstat.node0.numa_local
5182459 ± 6% +142.9% 12589826 ± 8% numa-vmstat.node1.numa_hit
5006447 ± 7% +149.8% 12505557 ± 8% numa-vmstat.node1.numa_local
83506 -2.0% 81840 proc-vmstat.nr_active_anon
4679 -1.4% 4615 proc-vmstat.nr_inactive_anon
53999 -1.6% 53160 proc-vmstat.nr_kernel_stack
433.25 ± 19% -27.5% 314.00 ± 30% proc-vmstat.nr_mlock
83506 -2.0% 81840 proc-vmstat.nr_zone_active_anon
4679 -1.4% 4615 proc-vmstat.nr_zone_inactive_anon
20973308 ± 4% +133.2% 48920238 ± 9% proc-vmstat.numa_hit
20949973 ± 4% +133.4% 48896885 ± 9% proc-vmstat.numa_local
17735 ± 5% -11.2% 15741 ± 10% proc-vmstat.pgactivate
21126486 ± 4% +132.3% 49080287 ± 9% proc-vmstat.pgalloc_normal
21083692 ± 4% +132.6% 49040769 ± 9% proc-vmstat.pgfree
27609 +95.3% 53918 ± 74% sched_debug.cfs_rq:/.load.max
4398 ± 2% +72.6% 7590 ± 62% sched_debug.cfs_rq:/.load.stddev
4.57 ± 8% +23.6% 5.65 ± 6% sched_debug.cfs_rq:/.load_avg.min
897673 ± 17% -46.7% 478313 ± 30% sched_debug.cfs_rq:/.min_vruntime.stddev
0.10 ± 13% +25.5% 0.12 ± 9% sched_debug.cfs_rq:/.nr_running.stddev
27.02 ± 2% +10.5% 29.86 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max
4.46 ± 2% +25.9% 5.61 ± 8% sched_debug.cfs_rq:/.runnable_load_avg.stddev
26876 ± 2% +96.6% 52840 ± 77% sched_debug.cfs_rq:/.runnable_weight.max
2408 ± 6% -58.1% 1010 ± 53% sched_debug.cfs_rq:/.runnable_weight.min
4468 +84.9% 8262 ± 59% sched_debug.cfs_rq:/.runnable_weight.stddev
1182419 ± 55% -89.8% 120916 ±135% sched_debug.cfs_rq:/.spread0.max
905007 ± 18% -46.5% 484043 ± 30% sched_debug.cfs_rq:/.spread0.stddev
231.61 ± 20% -64.2% 82.88 ± 12% sched_debug.cfs_rq:/.util_est_enqueued.min
365584 ± 9% -56.1% 160454 ± 26% sched_debug.cpu.avg_idle.avg
994525 -15.2% 842885 ± 8% sched_debug.cpu.avg_idle.max
1573 ± 7% +214.9% 4955 ± 52% sched_debug.cpu.avg_idle.min
348975 -35.5% 225191 ± 17% sched_debug.cpu.avg_idle.stddev
18665 +9.5% 20446 ± 6% sched_debug.cpu.curr->pid.avg
5.93 ± 19% -70.6% 1.74 ± 22% sched_debug.cpu.nr_running.min
68417151 -18.7% 55647643 ± 4% sched_debug.cpu.nr_switches.avg
77069929 -18.9% 62500934 ± 5% sched_debug.cpu.nr_switches.max
60412728 -16.2% 50607521 ± 4% sched_debug.cpu.nr_switches.min
3591588 ± 8% -34.6% 2349475 ± 12% sched_debug.cpu.nr_switches.stddev
1211 ± 5% -22.0% 944.93 ± 13% sched_debug.cpu.nr_uninterruptible.max
503.79 ± 3% -26.4% 370.80 ± 8% sched_debug.cpu.nr_uninterruptible.stddev
2.241e+10 -4.6% 2.138e+10 perf-stat.i.branch-instructions
1.55 +0.1 1.61 perf-stat.i.branch-miss-rate%
3.478e+08 -1.2% 3.436e+08 perf-stat.i.branch-misses
1.537e+09 -7.0% 1.429e+09 ± 2% perf-stat.i.cache-references
15818417 -14.6% 13506286 ± 2% perf-stat.i.context-switches
1.86 +4.4% 1.95 perf-stat.i.cpi
1456746 -32.0% 990274 ± 3% perf-stat.i.cpu-migrations
0.08 +0.0 0.10 ± 2% perf-stat.i.dTLB-load-miss-rate%
24331603 +25.5% 30526995 ± 2% perf-stat.i.dTLB-load-misses
3.248e+10 -2.3% 3.172e+10 perf-stat.i.dTLB-loads
0.00 ± 2% +0.0 0.00 ± 5% perf-stat.i.dTLB-store-miss-rate%
45106 +29.2% 58287 ± 5% perf-stat.i.dTLB-store-misses
1.94e+10 -1.3% 1.914e+10 perf-stat.i.dTLB-stores
26580536 ± 4% +29.9% 34517327 ± 8% perf-stat.i.iTLB-load-misses
1.116e+11 -3.9% 1.072e+11 perf-stat.i.instructions
4352 ± 2% -24.7% 3279 ± 5% perf-stat.i.instructions-per-iTLB-miss
0.54 -4.0% 0.52 perf-stat.i.ipc
91.43 ± 2% -6.7 84.71 perf-stat.i.node-load-miss-rate%
10689551 ± 18% -44.9% 5892014 ± 10% perf-stat.i.node-load-misses
81.96 ± 5% -19.2 62.75 perf-stat.i.node-store-miss-rate%
143823 ± 3% +154.7% 366290 ± 10% perf-stat.i.node-stores
13.78 -3.3% 13.33 ± 2% perf-stat.overall.MPKI
1.55 +0.1 1.61 perf-stat.overall.branch-miss-rate%
1.86 +4.2% 1.94 perf-stat.overall.cpi
0.07 +0.0 0.10 ± 2% perf-stat.overall.dTLB-load-miss-rate%
0.00 +0.0 0.00 ± 5% perf-stat.overall.dTLB-store-miss-rate%
4206 ± 4% -25.6% 3130 ± 8% perf-stat.overall.instructions-per-iTLB-miss
0.54 -4.0% 0.52 perf-stat.overall.ipc
94.54 -4.3 90.21 perf-stat.overall.node-load-miss-rate%
92.36 -14.6 77.73 perf-stat.overall.node-store-miss-rate%
95693 -22.4% 74303 ± 2% perf-stat.overall.path-length
2.238e+10 -4.6% 2.135e+10 perf-stat.ps.branch-instructions
3.473e+08 -1.2% 3.431e+08 perf-stat.ps.branch-misses
1.535e+09 -7.0% 1.427e+09 ± 2% perf-stat.ps.cache-references
15793664 -14.6% 13484141 ± 2% perf-stat.ps.context-switches
1454382 -32.0% 988375 ± 3% perf-stat.ps.cpu-migrations
24296291 +25.4% 30477596 ± 2% perf-stat.ps.dTLB-load-misses
3.243e+10 -2.4% 3.166e+10 perf-stat.ps.dTLB-loads
45042 +29.2% 58187 ± 5% perf-stat.ps.dTLB-store-misses
1.937e+10 -1.3% 1.911e+10 perf-stat.ps.dTLB-stores
26541760 ± 4% +29.8% 34460575 ± 8% perf-stat.ps.iTLB-load-misses
1.114e+11 -3.9% 1.071e+11 perf-stat.ps.instructions
10676935 ± 18% -44.9% 5882750 ± 10% perf-stat.ps.node-load-misses
143699 ± 3% +154.5% 365741 ± 10% perf-stat.ps.node-stores
7.441e+13 -11.7% 6.574e+13 ± 2% perf-stat.total.instructions
65.50 ±167% -100.0% 0.00 interrupts.47:PCI-MSI.31981578-edge.i40e-eth0-TxRx-9
72.25 ±156% -100.0% 0.00 interrupts.48:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
7806980 ± 4% -19.6% 6275769 ± 3% interrupts.CPU0.RES:Rescheduling_interrupts
7268631 ± 3% -15.9% 6109728 ± 3% interrupts.CPU1.RES:Rescheduling_interrupts
72.00 ±157% -100.0% 0.00 interrupts.CPU10.48:PCI-MSI.31981579-edge.i40e-eth0-TxRx-10
7175384 ± 3% -16.6% 5985464 ± 3% interrupts.CPU10.RES:Rescheduling_interrupts
7400764 ± 2% -16.7% 6166126 ± 3% interrupts.CPU11.RES:Rescheduling_interrupts
7572617 ± 5% -19.8% 6076352 ± 2% interrupts.CPU12.RES:Rescheduling_interrupts
7191776 ± 7% -17.5% 5932172 ± 2% interrupts.CPU13.RES:Rescheduling_interrupts
7373359 ± 4% -20.2% 5882027 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts
7381568 ± 6% -16.0% 6197541 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts
7639218 ± 8% -21.9% 5964422 ± 3% interrupts.CPU17.RES:Rescheduling_interrupts
7533710 ± 7% -20.5% 5987810 ± 5% interrupts.CPU18.RES:Rescheduling_interrupts
7291016 ± 4% -15.6% 6156667 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts
7120718 ± 4% -15.6% 6008287 interrupts.CPU2.RES:Rescheduling_interrupts
7422332 ± 3% -17.4% 6127408 ± 4% interrupts.CPU20.RES:Rescheduling_interrupts
7150652 ± 2% -15.9% 6015990 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts
7230450 ± 5% -16.4% 6041688 ± 3% interrupts.CPU22.RES:Rescheduling_interrupts
6837953 ± 3% -14.4% 5856054 ± 4% interrupts.CPU23.RES:Rescheduling_interrupts
7124369 ± 5% -16.5% 5947143 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts
7230806 ± 5% -17.9% 5935865 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts
7141240 ± 5% -17.7% 5876174 ± 2% interrupts.CPU26.RES:Rescheduling_interrupts
7239441 ± 5% -18.7% 5882885 ± 3% interrupts.CPU27.RES:Rescheduling_interrupts
7182666 ± 5% -17.3% 5943524 ± 3% interrupts.CPU28.RES:Rescheduling_interrupts
7212177 ± 3% -15.5% 6097241 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts
7180462 ± 4% -14.2% 6158684 ± 4% interrupts.CPU3.RES:Rescheduling_interrupts
7464279 ± 2% -19.2% 6032605 ± 2% interrupts.CPU30.RES:Rescheduling_interrupts
7134644 ± 3% -14.3% 6114818 ± 4% interrupts.CPU31.RES:Rescheduling_interrupts
7143780 ± 5% -15.0% 6075104 ± 4% interrupts.CPU32.RES:Rescheduling_interrupts
7330322 ± 5% -17.0% 6080915 ± 3% interrupts.CPU33.RES:Rescheduling_interrupts
6990218 ± 2% -14.9% 5951943 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts
7305951 ± 6% -19.0% 5920590 ± 2% interrupts.CPU35.RES:Rescheduling_interrupts
7223751 ± 3% -13.9% 6218642 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts
7347459 ± 8% -15.4% 6219391 ± 3% interrupts.CPU37.RES:Rescheduling_interrupts
7233333 ± 5% -13.4% 6261402 interrupts.CPU38.RES:Rescheduling_interrupts
7124562 ± 6% -17.0% 5915899 interrupts.CPU39.RES:Rescheduling_interrupts
7427571 ± 6% -16.6% 6192853 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts
7138497 ± 3% -19.0% 5785716 ± 6% interrupts.CPU40.RES:Rescheduling_interrupts
6986910 ± 4% -15.9% 5874131 ± 3% interrupts.CPU41.RES:Rescheduling_interrupts
6889593 ± 6% -12.5% 6030024 ± 5% interrupts.CPU42.RES:Rescheduling_interrupts
7152310 ± 5% -16.4% 5982558 interrupts.CPU43.RES:Rescheduling_interrupts
6998127 ± 4% -14.8% 5959666 interrupts.CPU44.RES:Rescheduling_interrupts
7175940 ± 3% -17.9% 5891069 ± 2% interrupts.CPU45.RES:Rescheduling_interrupts
7312735 -17.1% 6060717 ± 4% interrupts.CPU46.RES:Rescheduling_interrupts
7242447 -18.2% 5921623 ± 4% interrupts.CPU47.RES:Rescheduling_interrupts
7068463 ± 2% -13.1% 6141615 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts
7226681 ± 4% -15.9% 6080869 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts
7166167 ± 6% -14.8% 6106877 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts
7189170 ± 5% -16.3% 6014966 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts
7378516 ± 4% -19.8% 5920008 ± 3% interrupts.CPU51.RES:Rescheduling_interrupts
6873851 -13.2% 5968061 ± 2% interrupts.CPU52.RES:Rescheduling_interrupts
6975656 ± 2% -13.0% 6068180 interrupts.CPU53.RES:Rescheduling_interrupts
7226321 ± 4% -16.8% 6009737 interrupts.CPU54.RES:Rescheduling_interrupts
6995846 ± 3% -17.1% 5796713 ± 4% interrupts.CPU55.RES:Rescheduling_interrupts
7128765 ± 3% -16.1% 5978763 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts
7031743 ± 4% -14.3% 6027204 ± 2% interrupts.CPU57.RES:Rescheduling_interrupts
6900127 ± 4% -16.3% 5776381 ± 4% interrupts.CPU58.RES:Rescheduling_interrupts
7027071 ± 4% -15.7% 5924908 ± 2% interrupts.CPU59.RES:Rescheduling_interrupts
7280971 ± 2% -17.3% 6019406 ± 2% interrupts.CPU6.RES:Rescheduling_interrupts
7222288 ± 6% -19.5% 5814176 ± 6% interrupts.CPU60.RES:Rescheduling_interrupts
7318858 ± 2% -19.8% 5871336 ± 3% interrupts.CPU61.RES:Rescheduling_interrupts
7264267 ± 3% -21.4% 5707772 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts
6663069 ± 8% -13.1% 5786970 ± 4% interrupts.CPU63.RES:Rescheduling_interrupts
7098943 ± 4% -17.4% 5865146 interrupts.CPU64.RES:Rescheduling_interrupts
7050249 ± 4% -14.3% 6044359 ± 4% interrupts.CPU65.RES:Rescheduling_interrupts
7010296 ± 4% -16.9% 5829012 ± 2% interrupts.CPU66.RES:Rescheduling_interrupts
7196834 ± 6% -21.1% 5680005 ± 3% interrupts.CPU67.RES:Rescheduling_interrupts
6883721 ± 4% -15.4% 5820244 ± 2% interrupts.CPU68.RES:Rescheduling_interrupts
7149798 ± 4% -17.2% 5922219 interrupts.CPU69.RES:Rescheduling_interrupts
7363364 ± 3% -14.7% 6284183 ± 3% interrupts.CPU7.RES:Rescheduling_interrupts
7053810 ± 6% -16.3% 5903649 ± 2% interrupts.CPU70.RES:Rescheduling_interrupts
7159730 ± 6% -20.5% 5689354 ± 3% interrupts.CPU71.RES:Rescheduling_interrupts
7373157 ± 7% -16.7% 6143162 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts
65.00 ±169% -100.0% 0.00 interrupts.CPU9.47:PCI-MSI.31981578-edge.i40e-eth0-TxRx-9
7197343 ± 5% -15.4% 6090747 ± 2% interrupts.CPU9.RES:Rescheduling_interrupts
144.00 -50.0% 72.00 interrupts.MCP:Machine_check_polls
5.178e+08 ± 2% -16.6% 4.317e+08 interrupts.RES:Rescheduling_interrupts
10749 ± 4% +42.0% 15268 ± 7% softirqs.CPU0.SCHED
233917 -10.8% 208578 ± 2% softirqs.CPU0.TIMER
6608 ± 5% +46.6% 9691 ± 13% softirqs.CPU1.SCHED
6370 +54.6% 9851 ± 17% softirqs.CPU10.SCHED
229114 -11.1% 203776 ± 2% softirqs.CPU10.TIMER
6383 +54.8% 9879 ± 9% softirqs.CPU11.SCHED
227070 -10.4% 203437 ± 2% softirqs.CPU11.TIMER
6306 ± 4% +53.1% 9657 ± 12% softirqs.CPU12.SCHED
226162 -10.0% 203644 softirqs.CPU12.TIMER
6403 ± 2% +56.3% 10008 ± 13% softirqs.CPU13.SCHED
6456 ± 2% +52.6% 9854 ± 11% softirqs.CPU14.SCHED
226254 -10.2% 203202 softirqs.CPU14.TIMER
6355 ± 3% +55.1% 9859 ± 15% softirqs.CPU15.SCHED
227173 -11.1% 201985 softirqs.CPU15.TIMER
6416 ± 4% +57.6% 10109 ± 13% softirqs.CPU16.SCHED
6539 ± 6% +48.4% 9707 ± 11% softirqs.CPU17.SCHED
228122 -10.9% 203281 softirqs.CPU17.TIMER
6623 ± 6% +54.0% 10202 ± 6% softirqs.CPU18.SCHED
234088 ± 2% -11.5% 207127 ± 2% softirqs.CPU18.TIMER
6213 ± 5% +54.3% 9589 ± 4% softirqs.CPU19.SCHED
230788 -10.5% 206657 ± 2% softirqs.CPU19.TIMER
6489 ± 2% +52.8% 9914 ± 18% softirqs.CPU2.SCHED
6042 +60.0% 9667 ± 10% softirqs.CPU20.SCHED
229315 -10.2% 205850 ± 2% softirqs.CPU20.TIMER
5922 +60.3% 9496 ± 11% softirqs.CPU21.SCHED
231477 -11.0% 205970 softirqs.CPU21.TIMER
231597 -10.1% 208254 ± 2% softirqs.CPU23.TIMER
257444 ± 19% -20.0% 206031 ± 2% softirqs.CPU24.TIMER
6125 ± 3% +61.0% 9859 ± 8% softirqs.CPU25.SCHED
228069 -10.3% 204583 ± 2% softirqs.CPU25.TIMER
6035 ± 3% +64.2% 9909 ± 6% softirqs.CPU26.SCHED
231693 -11.3% 205594 ± 2% softirqs.CPU26.TIMER
228443 -9.4% 206949 ± 2% softirqs.CPU27.TIMER
5838 ± 4% +69.9% 9919 ± 8% softirqs.CPU28.SCHED
232152 -12.1% 204108 ± 2% softirqs.CPU28.TIMER
6136 ± 6% +59.1% 9760 ± 9% softirqs.CPU29.SCHED
230680 -10.9% 205524 ± 2% softirqs.CPU29.TIMER
6525 ± 4% +56.3% 10197 ± 8% softirqs.CPU3.SCHED
226200 ± 2% -10.1% 203279 softirqs.CPU3.TIMER
6070 ± 3% +64.6% 9993 ± 3% softirqs.CPU30.SCHED
257922 ± 18% -20.7% 204497 ± 2% softirqs.CPU30.TIMER
5903 ± 3% +68.4% 9938 ± 6% softirqs.CPU32.SCHED
230453 -11.0% 205013 softirqs.CPU32.TIMER
5874 ± 3% +71.2% 10058 ± 4% softirqs.CPU33.SCHED
229517 ± 2% -11.0% 204363 ± 2% softirqs.CPU33.TIMER
5881 ± 3% +64.9% 9696 ± 8% softirqs.CPU34.SCHED
230280 -10.6% 205806 ± 2% softirqs.CPU34.TIMER
5775 ± 2% +68.2% 9717 ± 6% softirqs.CPU35.SCHED
234613 -11.9% 206775 ± 2% softirqs.CPU35.TIMER
5995 ± 2% +65.3% 9910 ± 13% softirqs.CPU36.SCHED
227213 -11.0% 202304 softirqs.CPU36.TIMER
6384 ± 9% +49.1% 9522 ± 14% softirqs.CPU37.SCHED
6500 ± 3% +49.5% 9719 ± 15% softirqs.CPU38.SCHED
227877 -10.9% 202978 softirqs.CPU38.TIMER
6077 ± 3% +60.9% 9775 ± 9% softirqs.CPU39.SCHED
6170 ± 4% +60.0% 9875 ± 7% softirqs.CPU4.SCHED
227811 -10.8% 203198 ± 2% softirqs.CPU4.TIMER
6235 ± 6% +50.2% 9366 ± 6% softirqs.CPU40.SCHED
226345 ± 2% -9.8% 204086 ± 2% softirqs.CPU40.TIMER
6233 ± 3% +49.4% 9313 ± 11% softirqs.CPU41.SCHED
228102 -11.0% 203120 ± 2% softirqs.CPU41.TIMER
6276 ± 3% +57.3% 9873 ± 7% softirqs.CPU42.SCHED
226276 -11.0% 201397 ± 2% softirqs.CPU42.TIMER
6271 ± 4% +55.3% 9738 ± 13% softirqs.CPU43.SCHED
228063 -10.8% 203488 softirqs.CPU43.TIMER
6358 ± 2% +51.6% 9640 ± 12% softirqs.CPU44.SCHED
227909 -11.3% 202179 softirqs.CPU44.TIMER
6513 ± 3% +45.3% 9463 ± 9% softirqs.CPU45.SCHED
6296 ± 5% +56.7% 9865 ± 13% softirqs.CPU46.SCHED
226915 -10.0% 204226 ± 3% softirqs.CPU46.TIMER
6239 ± 2% +55.3% 9686 ± 10% softirqs.CPU47.SCHED
226751 -10.4% 203217 softirqs.CPU47.TIMER
6250 ± 4% +52.1% 9509 ± 10% softirqs.CPU48.SCHED
227646 ± 2% -10.9% 202782 ± 2% softirqs.CPU48.TIMER
6259 ± 3% +53.7% 9621 ± 16% softirqs.CPU49.SCHED
225874 -10.7% 201635 ± 2% softirqs.CPU49.TIMER
6471 ± 2% +43.0% 9252 ± 7% softirqs.CPU5.SCHED
228266 -11.0% 203254 softirqs.CPU5.TIMER
6365 ± 3% +54.4% 9825 ± 11% softirqs.CPU50.SCHED
225664 -10.4% 202220 ± 2% softirqs.CPU50.TIMER
6358 ± 3% +52.8% 9715 ± 20% softirqs.CPU51.SCHED
6328 ± 4% +54.2% 9759 ± 17% softirqs.CPU52.SCHED
227204 ± 2% -10.7% 202821 ± 3% softirqs.CPU52.TIMER
6255 ± 5% +59.1% 9953 ± 12% softirqs.CPU53.SCHED
227928 -11.6% 201591 ± 2% softirqs.CPU53.TIMER
5719 +69.5% 9696 ± 5% softirqs.CPU54.SCHED
232078 ± 2% -11.9% 204496 ± 2% softirqs.CPU54.TIMER
229947 -9.8% 207435 ± 2% softirqs.CPU55.TIMER
5979 ± 3% +55.5% 9297 ± 8% softirqs.CPU56.SCHED
229803 -10.6% 205435 ± 2% softirqs.CPU56.TIMER
5836 +65.3% 9648 ± 9% softirqs.CPU57.SCHED
232083 -11.8% 204599 ± 2% softirqs.CPU57.TIMER
229431 -10.7% 204949 ± 2% softirqs.CPU58.TIMER
229271 ± 2% -8.0% 211000 ± 3% softirqs.CPU59.TIMER
6656 ± 5% +47.9% 9842 ± 11% softirqs.CPU6.SCHED
225460 ± 2% -10.2% 202548 ± 2% softirqs.CPU6.TIMER
258922 ± 19% -20.1% 206949 ± 2% softirqs.CPU60.TIMER
6117 ± 5% +64.3% 10051 ± 10% softirqs.CPU61.SCHED
227981 -10.2% 204635 softirqs.CPU61.TIMER
5891 ± 4% +60.0% 9425 ± 6% softirqs.CPU62.SCHED
231145 -11.2% 205334 ± 2% softirqs.CPU62.TIMER
5973 ± 2% +62.8% 9727 ± 10% softirqs.CPU64.SCHED
227648 -10.5% 203857 ± 2% softirqs.CPU64.TIMER
5862 ± 2% +60.3% 9396 ± 9% softirqs.CPU65.SCHED
229580 -10.4% 205657 ± 2% softirqs.CPU65.TIMER
5983 ± 4% +62.9% 9744 ± 4% softirqs.CPU66.SCHED
230219 -10.8% 205407 ± 2% softirqs.CPU66.TIMER
233054 -10.0% 209701 ± 2% softirqs.CPU67.TIMER
6119 ± 3% +58.8% 9719 ± 4% softirqs.CPU69.SCHED
230472 -11.5% 204066 ± 2% softirqs.CPU69.TIMER
6304 ± 3% +57.0% 9895 ± 15% softirqs.CPU7.SCHED
226471 -11.1% 201406 softirqs.CPU7.TIMER
5882 ± 2% +58.8% 9339 ± 8% softirqs.CPU70.SCHED
229396 -10.1% 206325 softirqs.CPU70.TIMER
5967 +57.4% 9395 ± 6% softirqs.CPU71.SCHED
233509 -11.6% 206398 softirqs.CPU71.TIMER
6328 ± 4% +55.0% 9812 ± 12% softirqs.CPU8.SCHED
6359 ± 5% +56.4% 9943 ± 9% softirqs.CPU9.SCHED
226717 -10.6% 202694 ± 2% softirqs.CPU9.TIMER
448156 +56.0% 699181 ± 7% softirqs.SCHED
16585735 -10.5% 14839361 softirqs.TIMER
33.87 -4.1 29.72 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
32.17 ± 2% -4.0 28.14 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
31.35 ± 2% -4.0 27.34 ± 7% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
31.71 ± 2% -4.0 27.70 ± 7% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
14.21 ± 6% -3.2 11.02 ± 18% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
12.40 ± 8% -2.9 9.49 ± 21% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
38.73 -2.4 36.33 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
39.15 -2.3 36.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
41.41 -1.7 39.70 ± 3% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.83 ± 3% -1.6 4.28 ± 8% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.61 ± 3% -1.5 4.12 ± 8% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.40 ± 3% -1.5 3.95 ± 9% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.46 ± 9% -1.4 4.08 ± 21% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
87.55 -1.3 86.24 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
88.21 -1.3 86.94 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
18.37 -1.3 17.11 ± 5% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
15.96 -1.2 14.81 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
16.30 -1.1 15.18 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
7.82 ± 2% -0.8 7.02 ± 7% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
9.77 -0.8 8.99 ± 6% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
9.72 -0.8 8.95 ± 6% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
9.45 -0.7 8.75 ± 6% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
2.57 ± 2% -0.6 1.98 ± 9% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
2.50 ± 2% -0.3 2.15 ± 5% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.70 ± 2% -0.3 0.43 ± 57% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
1.66 ± 2% -0.3 1.40 ± 3% perf-profile.calltrace.cycles-pp.__switch_to_asm
0.97 ± 4% -0.2 0.73 ± 11% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.66 ± 8% -0.2 0.42 ± 57% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait
1.32 ± 3% -0.2 1.09 ± 4% perf-profile.calltrace.cycles-pp.native_write_msr
0.64 ± 8% -0.2 0.42 ± 57% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
1.41 ± 4% -0.2 1.19 ± 9% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.71 ± 2% -0.2 1.52 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.39 ± 2% -0.2 1.23 ± 3% perf-profile.calltrace.cycles-pp.__switch_to
1.08 ± 3% -0.1 0.95 ± 9% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.14 ± 2% -0.1 1.04 ± 7% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.69 ± 6% -0.1 0.59 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.77 ± 4% -0.1 0.67 ± 7% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
0.76 -0.1 0.69 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.63 ± 5% -0.1 0.57 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.75 ± 7% +0.1 0.84 ± 8% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
0.90 ± 6% +0.1 1.01 ± 5% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
1.05 ± 4% +0.1 1.17 ± 5% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.07 ± 4% +0.1 1.19 ± 5% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.62 ± 6% +0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.54 ± 3% +0.1 0.68 ± 6% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read
0.53 ± 2% +0.1 0.67 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.72 +0.2 0.87 ± 2% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.73 ± 2% +0.2 0.89 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.60 ± 2% +0.2 0.77 ± 5% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
0.91 ± 3% +0.2 1.08 ± 7% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.89 ± 2% +0.2 1.09 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.74 +0.2 0.96 ± 6% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
0.92 +0.2 1.14 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
0.83 +0.2 1.06 ± 6% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
0.40 ± 57% +0.3 0.67 ± 6% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.39 ± 57% +0.3 0.67 ± 8% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
1.20 +0.3 1.52 ± 6% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
1.60 ± 2% +0.4 1.95 ± 7% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
1.53 +0.4 1.93 ± 7% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.69 +0.4 2.13 ± 7% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.47 ± 58% +0.5 0.95 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait
0.13 ±173% +0.5 0.65 ± 4% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_read.do_syscall_64
0.00 +0.6 0.56 ± 7% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 +0.6 0.57 ± 9% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_read.new_sync_read.vfs_read.ksys_read
2.54 ± 10% +0.7 3.23 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
3.10 +0.8 3.91 ± 7% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
1.55 ± 11% +0.8 2.37 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
3.27 +0.9 4.20 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
34.34 -4.0 30.33 ± 6% perf-profile.children.cycles-pp.__wake_up_common_lock
32.24 ± 2% -3.9 28.36 ± 6% perf-profile.children.cycles-pp.__wake_up_common
31.73 ± 2% -3.8 27.88 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function
31.52 ± 2% -3.8 27.68 ± 6% perf-profile.children.cycles-pp.try_to_wake_up
14.23 ± 6% -3.1 11.10 ± 18% perf-profile.children.cycles-pp.select_task_rq_fair
12.47 ± 8% -2.9 9.60 ± 21% perf-profile.children.cycles-pp.select_idle_sibling
38.77 -2.4 36.38 ± 4% perf-profile.children.cycles-pp.pipe_write
39.18 -2.3 36.89 ± 3% perf-profile.children.cycles-pp.new_sync_write
21.97 -2.1 19.83 ± 4% perf-profile.children.cycles-pp.schedule
21.65 -2.0 19.62 ± 4% perf-profile.children.cycles-pp.__schedule
41.44 -1.7 39.74 ± 3% perf-profile.children.cycles-pp.vfs_write
5.86 ± 3% -1.6 4.30 ± 8% perf-profile.children.cycles-pp.exit_to_usermode_loop
5.49 ± 9% -1.4 4.13 ± 21% perf-profile.children.cycles-pp.available_idle_cpu
87.64 -1.3 86.34 perf-profile.children.cycles-pp.do_syscall_64
88.27 -1.3 87.00 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
9.85 -0.8 9.08 ± 5% perf-profile.children.cycles-pp.ttwu_do_activate
9.80 -0.8 9.03 ± 5% perf-profile.children.cycles-pp.activate_task
9.54 -0.7 8.86 ± 5% perf-profile.children.cycles-pp.enqueue_task_fair
7.96 ± 2% -0.6 7.32 ± 5% perf-profile.children.cycles-pp.dequeue_task_fair
0.96 ± 31% -0.6 0.33 ± 24% perf-profile.children.cycles-pp.set_task_cpu
5.10 -0.5 4.59 ± 5% perf-profile.children.cycles-pp.pick_next_task_fair
3.57 ± 3% -0.5 3.08 ± 8% perf-profile.children.cycles-pp.update_load_avg
0.69 ± 25% -0.4 0.25 ± 22% perf-profile.children.cycles-pp.migrate_task_rq_fair
2.50 ± 2% -0.3 2.16 ± 5% perf-profile.children.cycles-pp.switch_fpu_return
2.12 ± 3% -0.3 1.83 ± 4% perf-profile.children.cycles-pp.__switch_to
1.47 -0.3 1.17 ± 6% perf-profile.children.cycles-pp.___perf_sw_event
1.70 ± 2% -0.3 1.43 ± 3% perf-profile.children.cycles-pp.__switch_to_asm
3.56 ± 3% -0.3 3.29 ± 4% perf-profile.children.cycles-pp.reweight_entity
1.42 ± 3% -0.2 1.19 ± 3% perf-profile.children.cycles-pp.native_write_msr
1.12 ± 3% -0.2 0.89 ± 9% perf-profile.children.cycles-pp.put_prev_entity
0.76 ± 10% -0.2 0.57 ± 6% perf-profile.children.cycles-pp.finish_task_switch
1.20 ± 2% -0.2 1.04 ± 5% perf-profile.children.cycles-pp.pick_next_entity
0.23 ± 21% -0.2 0.08 ± 24% perf-profile.children.cycles-pp.remove_entity_load_avg
0.49 ± 11% -0.1 0.37 ± 4% perf-profile.children.cycles-pp.cpus_share_cache
1.34 ± 2% -0.1 1.22 ± 2% perf-profile.children.cycles-pp.__lock_text_start
1.20 ± 2% -0.1 1.09 ± 7% perf-profile.children.cycles-pp.check_preempt_wakeup
0.35 ± 14% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.12 ± 27% -0.1 0.03 ±100% perf-profile.children.cycles-pp.attach_entity_load_avg
0.19 ± 16% -0.1 0.11 ± 13% perf-profile.children.cycles-pp.get_xsave_addr
0.34 ± 6% -0.1 0.28 ± 7% perf-profile.children.cycles-pp.wakeup_preempt_entity
1.35 -0.1 1.29 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.51 ± 2% -0.1 0.45 ± 7% perf-profile.children.cycles-pp.clear_buddies
0.26 ± 3% -0.1 0.20 ± 12% perf-profile.children.cycles-pp.resched_curr
0.11 ± 13% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.perf_exclude_event
0.51 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.account_entity_enqueue
0.13 ± 3% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.rb_next
0.09 ± 4% +0.0 0.11 ± 6% perf-profile.children.cycles-pp.__mark_inode_dirty
0.12 ± 3% +0.0 0.14 ± 5% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.14 ± 6% +0.0 0.16 ± 2% perf-profile.children.cycles-pp.generic_update_time
0.09 +0.0 0.11 ± 7% perf-profile.children.cycles-pp.rw_verify_area
0.08 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.wake_q_add
0.06 ± 6% +0.0 0.09 ± 16% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.21 ± 3% +0.0 0.24 ± 4% perf-profile.children.cycles-pp.rb_insert_color
0.15 ± 7% +0.0 0.18 ± 8% perf-profile.children.cycles-pp.timespec64_trunc
0.10 ± 4% +0.0 0.14 ± 6% perf-profile.children.cycles-pp.iov_iter_init
0.26 ± 3% +0.0 0.30 ± 3% perf-profile.children.cycles-pp.finish_wait
0.36 ± 3% +0.0 0.40 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid
0.17 ± 4% +0.0 0.21 ± 9% perf-profile.children.cycles-pp.rcu_all_qs
0.16 ± 5% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.wake_up_q
0.11 ± 3% +0.0 0.15 ± 7% perf-profile.children.cycles-pp.__sb_end_write
0.22 ± 3% +0.0 0.27 ± 7% perf-profile.children.cycles-pp.__x64_sys_write
0.22 ± 3% +0.0 0.27 ± 5% perf-profile.children.cycles-pp.__x64_sys_read
0.00 +0.1 0.05 perf-profile.children.cycles-pp.bpf_fd_pass
0.42 +0.1 0.47 ± 3% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.15 ± 5% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.inode_has_perm
0.13 +0.1 0.19 ± 11% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.18 ± 11% +0.1 0.26 ± 8% perf-profile.children.cycles-pp.__mutex_lock
0.07 ± 7% +0.1 0.14 ± 12% perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.08 ± 23% perf-profile.children.cycles-pp.start_secondary
0.41 ± 3% +0.1 0.49 ± 8% perf-profile.children.cycles-pp.__might_fault
0.00 +0.1 0.08 ± 24% perf-profile.children.cycles-pp.do_idle
0.00 +0.1 0.08 ± 24% perf-profile.children.cycles-pp.secondary_startup_64
0.00 +0.1 0.08 ± 24% perf-profile.children.cycles-pp.cpu_startup_entry
0.37 ± 5% +0.1 0.46 ± 9% perf-profile.children.cycles-pp.current_time
0.34 ± 4% +0.1 0.44 ± 8% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.31 ± 6% +0.1 0.41 ± 10% perf-profile.children.cycles-pp.atime_needs_update
0.52 ± 2% +0.1 0.64 ± 4% perf-profile.children.cycles-pp.__might_sleep
0.55 ± 4% +0.1 0.69 ± 6% perf-profile.children.cycles-pp.touch_atime
0.39 ± 5% +0.1 0.54 ± 8% perf-profile.children.cycles-pp._cond_resched
0.61 ± 2% +0.1 0.76 ± 6% perf-profile.children.cycles-pp.fsnotify
0.48 ± 5% +0.2 0.64 ± 8% perf-profile.children.cycles-pp.__fsnotify_parent
2.10 ± 2% +0.2 2.26 ± 4% perf-profile.children.cycles-pp.mutex_lock
0.62 ± 3% +0.2 0.78 ± 5% perf-profile.children.cycles-pp.copyin
0.44 ± 3% +0.2 0.63 ± 7% perf-profile.children.cycles-pp.__inode_security_revalidate
0.63 ± 10% +0.2 0.82 ± 7% perf-profile.children.cycles-pp.avc_has_perm
0.79 +0.2 1.01 ± 7% perf-profile.children.cycles-pp.___might_sleep
0.50 ± 2% +0.2 0.73 ± 11% perf-profile.children.cycles-pp.prepare_to_wait
0.84 +0.2 1.07 ± 6% perf-profile.children.cycles-pp.copyout
1.42 ± 3% +0.3 1.69 ± 5% perf-profile.children.cycles-pp.__fget
1.79 ± 2% +0.3 2.07 ± 4% perf-profile.children.cycles-pp.__fget_light
1.84 ± 2% +0.3 2.12 ± 4% perf-profile.children.cycles-pp.__fdget_pos
1.06 ± 5% +0.3 1.38 ± 7% perf-profile.children.cycles-pp.file_has_perm
1.22 +0.3 1.55 ± 6% perf-profile.children.cycles-pp.copy_page_from_iter
1.62 ± 2% +0.4 1.98 ± 6% perf-profile.children.cycles-pp.copy_page_to_iter
1.38 +0.4 1.75 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.82 +0.4 2.26 ± 6% perf-profile.children.cycles-pp.selinux_file_permission
4.92 ± 8% +0.7 5.61 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
3.25 +0.8 4.09 ± 7% perf-profile.children.cycles-pp.security_file_permission
3.57 +0.9 4.49 ± 7% perf-profile.children.cycles-pp.syscall_return_via_sysret
3.27 +0.9 4.20 ± 8% perf-profile.children.cycles-pp.entry_SYSCALL_64
3.15 ± 7% +1.3 4.43 ± 2% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
5.45 ± 9% -1.4 4.10 ± 21% perf-profile.self.cycles-pp.available_idle_cpu
5.10 ± 9% -1.2 3.87 ± 24% perf-profile.self.cycles-pp.select_idle_sibling
2.44 ± 4% -0.5 1.94 ± 8% perf-profile.self.cycles-pp._raw_spin_lock
2.62 -0.4 2.25 ± 3% perf-profile.self.cycles-pp.__schedule
2.49 ± 2% -0.3 2.15 ± 5% perf-profile.self.cycles-pp.switch_fpu_return
1.75 ± 5% -0.3 1.45 ± 10% perf-profile.self.cycles-pp.update_load_avg
1.36 -0.3 1.08 ± 7% perf-profile.self.cycles-pp.___perf_sw_event
1.69 ± 2% -0.3 1.42 ± 3% perf-profile.self.cycles-pp.__switch_to_asm
0.41 ± 28% -0.3 0.15 ± 22% perf-profile.self.cycles-pp.migrate_task_rq_fair
1.42 ± 3% -0.2 1.18 ± 4% perf-profile.self.cycles-pp.native_write_msr
1.36 ± 2% -0.2 1.15 ± 5% perf-profile.self.cycles-pp.enqueue_task_fair
1.86 -0.2 1.66 ± 3% perf-profile.self.cycles-pp.__switch_to
0.27 ± 45% -0.2 0.08 ± 29% perf-profile.self.cycles-pp.set_task_cpu
1.40 ± 6% -0.2 1.24 perf-profile.self.cycles-pp.select_task_rq_fair
0.48 ± 11% -0.1 0.36 ± 4% perf-profile.self.cycles-pp.cpus_share_cache
1.53 ± 5% -0.1 1.42 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.34 ± 13% -0.1 0.24 ± 8% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.12 ± 25% -0.1 0.03 ±100% perf-profile.self.cycles-pp.attach_entity_load_avg
1.01 -0.1 0.93 ± 3% perf-profile.self.cycles-pp.dequeue_task_fair
0.18 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.get_xsave_addr
0.25 ± 9% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.activate_task
0.54 ± 3% -0.1 0.46 ± 4% perf-profile.self.cycles-pp.pick_next_entity
0.41 ± 6% -0.1 0.34 ± 5% perf-profile.self.cycles-pp.finish_task_switch
0.30 ± 7% -0.1 0.25 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.25 ± 2% -0.1 0.20 ± 12% perf-profile.self.cycles-pp.resched_curr
0.21 ± 5% -0.1 0.16 ± 4% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.46 ± 2% -0.0 0.41 ± 8% perf-profile.self.cycles-pp.clear_buddies
0.45 -0.0 0.41 ± 3% perf-profile.self.cycles-pp.account_entity_enqueue
0.33 ± 4% -0.0 0.30 ± 2% perf-profile.self.cycles-pp.set_next_entity
0.81 -0.0 0.78 perf-profile.self.cycles-pp.__calc_delta
0.21 ± 6% -0.0 0.18 ± 4% perf-profile.self.cycles-pp.put_prev_entity
0.97 -0.0 0.94 ± 2% perf-profile.self.cycles-pp.__lock_text_start
0.09 ± 4% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__mark_inode_dirty
0.10 ± 4% +0.0 0.12 ± 5% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.12 +0.0 0.14 ± 6% perf-profile.self.cycles-pp.rb_next
0.09 +0.0 0.11 ± 6% perf-profile.self.cycles-pp.rw_verify_area
0.07 ± 12% +0.0 0.09 ± 9% perf-profile.self.cycles-pp.__mutex_unlock_slowpath
0.08 ± 5% +0.0 0.10 ± 10% perf-profile.self.cycles-pp.wake_q_add
0.21 ± 3% +0.0 0.24 ± 4% perf-profile.self.cycles-pp.rb_insert_color
0.10 ± 10% +0.0 0.13 ± 9% perf-profile.self.cycles-pp.__mutex_lock
0.10 ± 5% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.iov_iter_init
0.24 ± 5% +0.0 0.27 ± 4% perf-profile.self.cycles-pp.check_preempt_curr
0.23 ± 4% +0.0 0.26 ± 5% perf-profile.self.cycles-pp.finish_wait
0.15 ± 4% +0.0 0.18 ± 5% perf-profile.self.cycles-pp._cond_resched
0.62 +0.0 0.66 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.12 ± 3% +0.0 0.16 ± 9% perf-profile.self.cycles-pp.rcu_all_qs
0.11 ± 3% +0.0 0.15 ± 7% perf-profile.self.cycles-pp.__sb_end_write
0.21 ± 3% +0.0 0.25 ± 8% perf-profile.self.cycles-pp.__x64_sys_write
0.21 ± 2% +0.0 0.24 ± 6% perf-profile.self.cycles-pp.__x64_sys_read
0.35 ± 4% +0.0 0.39 ± 4% perf-profile.self.cycles-pp.__list_del_entry_valid
0.16 ± 2% +0.0 0.20 ± 7% perf-profile.self.cycles-pp.__inode_security_revalidate
0.38 ± 2% +0.0 0.42 ± 4% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.15 ± 5% +0.0 0.20 ± 9% perf-profile.self.cycles-pp.current_time
0.17 ± 5% +0.0 0.22 ± 10% perf-profile.self.cycles-pp.ksys_write
0.13 ± 6% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.inode_has_perm
0.23 ± 3% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.ksys_read
0.12 +0.1 0.18 ± 11% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.24 ± 6% +0.1 0.30 ± 8% perf-profile.self.cycles-pp.security_file_permission
0.23 +0.1 0.30 ± 8% perf-profile.self.cycles-pp.copy_page_from_iter
0.28 ± 2% +0.1 0.35 ± 8% perf-profile.self.cycles-pp.vfs_write
0.26 ± 4% +0.1 0.33 ± 6% perf-profile.self.cycles-pp.file_has_perm
0.34 ± 3% +0.1 0.43 ± 8% perf-profile.self.cycles-pp.new_sync_write
0.45 +0.1 0.56 ± 4% perf-profile.self.cycles-pp.__might_sleep
0.45 ± 5% +0.1 0.59 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent
0.59 ± 2% +0.1 0.74 ± 6% perf-profile.self.cycles-pp.fsnotify
1.11 +0.2 1.28 ± 6% perf-profile.self.cycles-pp.pipe_read
0.62 ± 10% +0.2 0.82 ± 7% perf-profile.self.cycles-pp.avc_has_perm
0.65 ± 2% +0.2 0.88 ± 9% perf-profile.self.cycles-pp.pipe_write
0.76 +0.2 0.99 ± 7% perf-profile.self.cycles-pp.___might_sleep
1.35 ± 2% +0.2 1.58 ± 6% perf-profile.self.cycles-pp.selinux_file_permission
1.39 ± 4% +0.3 1.66 ± 5% perf-profile.self.cycles-pp.__fget
1.35 +0.4 1.73 ± 6% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
3.56 +0.9 4.47 ± 7% perf-profile.self.cycles-pp.syscall_return_via_sysret
3.27 +0.9 4.20 ± 8% perf-profile.self.cycles-pp.entry_SYSCALL_64
3.13 ± 7% +1.3 4.42 ± 2% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
6.94 +1.4 8.30 ± 4% perf-profile.self.cycles-pp.do_syscall_64
hackbench.throughput
160000 +-+----------------------------------------------------------------+
155000 O-O OO O |
| O O O O O O O |
150000 +-+ O OO O OO O |
145000 +-+ O O |
| O O |
140000 +-+ O O |
135000 +-+ |
130000 +-+ |
| |
125000 +-+ +.++.+.+ .++.++.+ |
120000 +-+ + +.++ +.+ |
|.++.++. +.+ .+ + : +. .+ |
115000 +-+ + + +.++.++.++ +.+ ++ +.++.++.|
110000 +-+----------------------------------------------------------------+
hackbench.time.involuntary_context_switches
4.6e+09 +-+---------------------------------------------------------------+
4.4e+09 +-+ .+ .+ +.++.+ .+ +.++. |
| +.+ .+ .++.++.++.++.++ + +.+ + +.+ ++. +.++.++.++.+|
4.2e+09 +-+ + + + |
4e+09 +-+ |
| |
3.8e+09 +-+ |
3.6e+09 +-+ |
3.4e+09 +-+ |
| |
3.2e+09 +-+ O O |
3e+09 +-+ O O OO |
| O O O |
2.8e+09 +-OO OO O O OO O O O O |
2.6e+09 O-+------------O--------O-----O-----------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 3 months
- 1
- 0
[xfs] 11bfe2874e: Assertion_failed
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 11bfe2874e3e739889e2b54728988d925760e1a8 ("xfs: convert if () { XFS_ERROR_REPORT to if (XFS_CORRUPT_ON...")
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git report-corruption
in testcase: xfstests
with following parameters:
disk: 4HDD
fs: xfs
test: xfs-group19
test-description: xfstests is a regression test suite for xfs and other files ystems.
test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+--------------------------------------------------+------------+------------+
| | 1f50dc47e7 | 11bfe2874e |
+--------------------------------------------------+------------+------------+
| boot_successes | 1 | 2 |
| boot_failures | 1 | 114 |
| WARNING:at_kernel/workqueue.c:#destroy_workqueue | 1 | |
| RIP:destroy_workqueue | 1 | |
| Assertion_failed | 0 | 114 |
| kernel_BUG_at_fs/xfs/xfs_message.c | 0 | 114 |
| invalid_opcode:#[##] | 0 | 114 |
| RIP:assfail[xfs] | 0 | 114 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 114 |
+--------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 168.208859] 00000070: 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00 00 ................
[ 168.215101] XFS (vdd): Unmounting Filesystem
[ 168.307301] XFS (vdd): Mounting V5 Filesystem
[ 168.315637] XFS (vdd): Ending clean mount
[ 168.319544] Mounted xfs file system at /fs/scratch supports timestamps until 2038 (0x7fffffff)
[ 168.330806] XFS (vdd): Assertion failed: rval != 0 && args->dp->i_d.di_size != args->geo->blksize, file: fs/xfs/libxfs/xfs_dir2.c, line: 607
[ 168.334135] ------------[ cut here ]------------
[ 168.335371] kernel BUG at fs/xfs/xfs_message.c:102!
[ 168.336887] invalid opcode: 0000 [#1] SMP PTI
[ 168.338284] CPU: 1 PID: 15479 Comm: ls Not tainted 5.4.0-rc3-00151-g11bfe2874e3e7 #1
[ 168.340897] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 168.343560] RIP: 0010:assfail+0x26/0x30 [xfs]
[ 168.345188] Code: c3 0f 1f 00 66 66 66 66 90 41 89 c8 48 89 d1 48 89 f2 48 c7 c6 b0 49 4f c0 e8 66 f9 ff ff 80 3d 53 9f 0f 00 00 75 03 0f 0b c3 <0f> 0b 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 63 f6 6a 01 49 89
[ 168.350891] RSP: 0018:ffffb55e00cf3d88 EFLAGS: 00010202
[ 168.352809] RAX: 0000000000000000 RBX: ffffb55e00cf3dc0 RCX: 0000000000000000
[ 168.355102] RDX: 00000000ffffffc0 RSI: 000000000000000a RDI: ffffffffc04e6058
[ 168.357526] RBP: ffffb55e00cf3dbc R08: 0000000000000000 R09: 0000000000000000
[ 168.359858] R10: ffffb55e00cf3e50 R11: f000000000000000 R12: ffffb55e00cf3ec0
[ 168.362047] R13: 0000000000000004 R14: ffff9743993ce000 R15: 0000000000000001
[ 168.363670] FS: 00007fe99cbbef40(0000) GS:ffff9743ffd00000(0000) knlGS:0000000000000000
[ 168.365390] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 168.366843] CR2: 000055906ceead68 CR3: 00000001922ac000 CR4: 00000000000406e0
[ 168.368475] Call Trace:
[ 168.369528] xfs_dir2_isblock+0x96/0xb0 [xfs]
[ 168.370878] xfs_readdir+0xe1/0x1d0 [xfs]
[ 168.372077] iterate_dir+0x8d/0x1a0
[ 168.373722] __x64_sys_getdents+0xd1/0x180
[ 168.375008] ? verify_dirent_name+0x40/0x40
[ 168.376220] ? do_syscall_64+0x5b/0x1d0
[ 168.377390] ? __ia32_sys_getdents+0x170/0x170
[ 168.378741] do_syscall_64+0x5b/0x1d0
[ 168.379924] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 168.381239] RIP: 0033:0x7fe99c491f2b
[ 168.382413] Code: fc ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 57 41 56 48 63 ff 41 55 41 54 b8 4e 00 00 00 55 53 48 89 f3 48 83 ec 08 0f 05 <48> 3d 00 f0 ff ff 77 55 48 8d 2c 06 49 89 c4 48 39 ee 73 34 90 44
[ 168.386330] RSP: 002b:00007fffffc54000 EFLAGS: 00000206 ORIG_RAX: 000000000000004e
[ 168.388091] RAX: ffffffffffffffda RBX: 000055906cee2d60 RCX: 00007fe99c491f2b
[ 168.389769] RDX: 0000000000008000 RSI: 000055906cee2d60 RDI: 0000000000000004
[ 168.391488] RBP: 000055906cee2d60 R08: 0000000000000003 R09: 0000000000008040
[ 168.393128] R10: 000000000000008d R11: 0000000000000206 R12: fffffffffffffeb0
[ 168.394837] R13: 0000000000000000 R14: 000055906cee2d10 R15: 000055906b1e6595
[ 168.396525] Modules linked in: dm_delay xfs libcrc32c dm_mod intel_rapl_msr intel_rapl_common crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sr_mod cdrom sg ppdev ata_generic pata_acpi snd_pcm aesni_intel snd_timer crypto_simd bochs_drm snd drm_vram_helper ttm cryptd glue_helper drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops soundcore pcspkr drm joydev i2c_piix4 serio_raw ata_piix parport_pc parport floppy ip_tables
[ 168.404943] ---[ end trace 9e1dbc26c49a4256 ]---
[ 168.406392] RIP: 0010:assfail+0x26/0x30 [xfs]
[ 168.407758] Code: c3 0f 1f 00 66 66 66 66 90 41 89 c8 48 89 d1 48 89 f2 48 c7 c6 b0 49 4f c0 e8 66 f9 ff ff 80 3d 53 9f 0f 00 00 75 03 0f 0b c3 <0f> 0b 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 63 f6 6a 01 49 89
[ 168.413519] RSP: 0018:ffffb55e00cf3d88 EFLAGS: 00010202
[ 168.415481] RAX: 0000000000000000 RBX: ffffb55e00cf3dc0 RCX: 0000000000000000
[ 168.417927] RDX: 00000000ffffffc0 RSI: 000000000000000a RDI: ffffffffc04e6058
[ 168.420242] RBP: ffffb55e00cf3dbc R08: 0000000000000000 R09: 0000000000000000
[ 168.422544] R10: ffffb55e00cf3e50 R11: f000000000000000 R12: ffffb55e00cf3ec0
[ 168.424974] R13: 0000000000000004 R14: ffff9743993ce000 R15: 0000000000000001
[ 168.427337] FS: 00007fe99cbbef40(0000) GS:ffff9743ffd00000(0000) knlGS:0000000000000000
[ 168.430125] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 168.432159] CR2: 000055906ceead68 CR3: 00000001922ac000 CR4: 00000000000406e0
[ 168.434730] Kernel panic - not syncing: Fatal exception
[ 168.437456] Kernel Offset: 0x2200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Elapsed time: 170
qemu-img create -f qcow2 disk-vm-snb-49316eae392f-0 256G
To reproduce:
# build kernel
cd linux
cp config-5.4.0-rc3-00151-g11bfe2874e3e7 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 3 months
- 1
- 0