June 2014 - LKP - Ml01.01.Org

[clockevents] WARNING: CPU: 0 PID: 0 at kernel/time/tick-oneshot.c:32 tick_stop_event()

by Jet Chen

Hi Viresh, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.linaro.org/people/vireshk/linux tick/ONESHOT-STOPPED commit d5a085970b0b88b8c82303d2c45f09c9427fd02a Author: Viresh Kumar <viresh.kumar(a)linaro.org> AuthorDate: Tue Jun 10 13:52:25 2014 +0530 Commit: Viresh Kumar <viresh.kumar(a)linaro.org> CommitDate: Thu Jun 19 10:35:12 2014 +0530 clockevents: x86: add support for ONESHOT_STOPPED mode Clockevent devices can be stopped from core (hrtimers/tick) when they aren't used anymore. This will mostly happen on a idle or tickless cpu. This is done to avoid getting spurious interrupts when core doesn't want to handle them anymore. For stopping clockevent devices, core requests drivers to switch to a special optional mode: CLOCK_EVT_MODE_ONESHOT_STOPPED. At this time platforms can also save some power by disabling these devices. This patch adds support for handling CLOCK_EVT_MODE_ONESHOT_STOPPED mode for x86 drivers. Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> +-----------------------------------------------------------------+------------+------------+ | | f4f5df6f14 | d5a085970b | +-----------------------------------------------------------------+------------+------------+ | boot_successes | 120 | 9 | | boot_failures | 0 | 31 | | WARNING:CPU:PID:at_kernel/time/tick-oneshot.c:tick_stop_event() | 0 | 31 | | backtrace:cpu_startup_entry | 0 | 19 | +-----------------------------------------------------------------+------------+------------+ [ 0.247455] WARNING: CPU: 0 PID: 0 at kernel/time/tick-oneshot.c:32 tick_stop_event+0x43/0x50() [ 0.247455] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-rc1-00016-gd5a0859 #1 [ 0.247455] 0000000000000009 ffff880012203eb0 ffffffff81658a72 0000000000000000 [ 0.247455] ffff880012203ee8 ffffffff8108c128 ffff88001220ca80 7fffffffffffffff [ 0.247455] 000000000e82ddd2 7fffffffffffffff ffff8800123cd560 ffff880012203ef8 [ 0.247455] Call Trace: [ 0.247455] <IRQ> [<ffffffff81658a72>] dump_stack+0x4d/0x66 [ 0.247455] [<ffffffff8108c128>] warn_slowpath_common+0x78/0xa0 [ 0.247455] [<ffffffff8108c23a>] warn_slowpath_null+0x1a/0x20 [ 0.247455] [<ffffffff81101983>] tick_stop_event+0x43/0x50 [ 0.247455] [<ffffffff81102275>] __tick_nohz_idle_enter+0x515/0x580 [ 0.247455] [<ffffffff81102649>] ? tick_nohz_irq_exit+0x29/0x30 [ 0.247455] [<ffffffff81102649>] tick_nohz_irq_exit+0x29/0x30 [ 0.247455] [<ffffffff810913c5>] irq_exit+0xb5/0xd0 [ 0.247455] [<ffffffff81028095>] smp_call_function_single_interrupt+0x35/0x40 [ 0.247455] [<ffffffff81662892>] call_function_single_interrupt+0x72/0x80 [ 0.247455] <EOI> [<ffffffff81032b96>] ? native_safe_halt+0x6/0x10 [ 0.247455] [<ffffffff8100c964>] default_idle+0x24/0xe0 [ 0.247455] [<ffffffff8100d3cf>] arch_cpu_idle+0xf/0x20 [ 0.247455] [<ffffffff810d4f16>] cpu_startup_entry+0x386/0x4f0 [ 0.247455] [<ffffffff81652da6>] rest_init+0xc6/0xd0 [ 0.247455] [<ffffffff81652ce5>] ? rest_init+0x5/0xd0 [ 0.247455] [<ffffffff81f23f96>] start_kernel+0x41e/0x429 [ 0.247455] [<ffffffff81f23981>] ? set_init_arg+0x53/0x53 [ 0.247455] [<ffffffff81f23120>] ? early_idt_handlers+0x120/0x120 [ 0.247455] [<ffffffff81f23549>] x86_64_start_reservations+0x2a/0x2c [ 0.247455] [<ffffffff81f23679>] x86_64_start_kernel+0x12e/0x13d git bisect start 1ded43568e7bee60c6548e5fb3b4bd6d1d0783b1 a497c3ba1d97fc69c1e78e7b96435ba8c2cb42ee -- git bisect good 5f069b44108f255ab9a881ff366c793bc1cda5e9 # 14:27 21+ 0 Merge 'ipvs-next/master' into devel-hourly-2014062313 git bisect good 70eab242a357abfe990c30656a396f1b0a0ba465 # 14:33 21+ 0 Merge 'cifs/for-next' into devel-hourly-2014062313 git bisect good 64550da9a486e3766b85928f442756f29c39437a # 14:40 21+ 0 Merge 'rcu/rcu_cond_resched.2014.06.20c' into devel-hourly-2014062313 git bisect bad bf774a68a67dd0c0f938258f56346763d619bdbf # 14:44 0- 6 Merge 'vireshk/tick/lowres-go-tickless' into devel-hourly-2014062313 git bisect bad b1efa076fdd9532ba3fdf0bddb475ed968f33c13 # 14:50 2- 1 Merge 'vireshk/tick/ONESHOT-STOPPED' into devel-hourly-2014062313 git bisect good f935474b892829a400f31ec076385b792034a32e # 15:02 40+ 0 clockevents: Introduce CLOCK_EVT_MODE_ONESHOT_STOPPED mode git bisect good f4f5df6f142f6797c9aae44d8b570843544dcca9 # 15:12 40+ 0 clockevents: mips: add support for ONESHOT_STOPPED mode git bisect bad 2d74fb379fa8c9d388fcce15e84772713ac7331d # 15:16 5- 8 clockevents: misc: add support for ONESHOT_STOPPED mode git bisect bad 6e9ed902730ca7f5cde9fcf47c6b1ca2a143babf # 15:33 1- 14 clockevents: drivers: add support for ONESHOT_STOPPED mode git bisect bad d5a085970b0b88b8c82303d2c45f09c9427fd02a # 15:43 1- 2 clockevents: x86: add support for ONESHOT_STOPPED mode # first bad commit: [d5a085970b0b88b8c82303d2c45f09c9427fd02a] clockevents: x86: add support for ONESHOT_STOPPED mode git bisect good f4f5df6f142f6797c9aae44d8b570843544dcca9 # 15:46 120+ 0 clockevents: mips: add support for ONESHOT_STOPPED mode git bisect bad 1ded43568e7bee60c6548e5fb3b4bd6d1d0783b1 # 15:46 0- 11 0day head guard for 'devel-hourly-2014062313' git bisect good a497c3ba1d97fc69c1e78e7b96435ba8c2cb42ee # 15:49 120+ 0 Linux 3.16-rc2 git bisect good 58ae500a03a6bf68eee323c342431bfdd3f460b6 # 15:56 120+ 0 Add linux-next specific files for 20140623 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-i386.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/blob/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

3
2
0 / 0

[Device/core] BUG: unable to handle kernel NULL pointer dereference at (null)

by Jet Chen

Hi Tianyu, 0day kernel testing robot got the below dmesg and the first bad commit is git://bee.sh.intel.com/git/tianyu/usb.git dep_support commit 6ce060a8738b36c60ad4b2567d393132b2620121 Author: Lan Tianyu <tianyu.lan(a)intel.com> AuthorDate: Thu Apr 10 13:18:43 2014 +0800 Commit: Lan Tianyu <tianyu.lan(a)intel.com> CommitDate: Wed Jun 18 10:40:29 2014 +0800 Device/core: Add platform_dev_attach/detach_notify() callback For ACPI platform, ACPI table provides device probe dependency information and ACPI subsystem needs to check the driver probing status of dependent devices. This patch is to add two callbacks to notify platform when device is attached and detached from driver. Signed-off-by: Lan Tianyu <tianyu.lan(a)intel.com> +------------------------------------------------------------+------------+------------+ | | 96e8c1a7c3 | 6ce060a873 | +------------------------------------------------------------+------------+------------+ | boot_successes | 60 | 0 | | boot_failures | 0 | 20 | | BUG:unable_to_handle_kernel_NULL_pointer_dereference | 0 | 20 | | Oops | 0 | 20 | | WARNING:CPU:PID:at_kernel/smp.c:smp_call_function_single() | 0 | 20 | | Kernel_panic-not_syncing:Attempted_to_kill_init_exitcode= | 0 | 20 | | backtrace:__platform_driver_register | 0 | 20 | | backtrace:regulator_dummy_init | 0 | 20 | | backtrace:regulator_init | 0 | 20 | | backtrace:kernel_init_freeable | 0 | 20 | +------------------------------------------------------------+------------+------------+ [ 0.180030] prefetch64-sse: 137.000 MB/sec [ 0.220055] generic_sse: 132.000 MB/sec [ 0.220691] xor: using function: prefetch64-sse (137.000 MB/sec) [ 0.222565] BUG: unable to handle kernel NULL pointer dereference at (null) [ 0.223565] IP: [< (null)>] (null) [ 0.224000] PGD 0 [ 0.224000] Oops: 0010 [#1] PREEMPT SMP DEBUG_PAGEALLOC [ 0.224000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.16.0-rc1-00011-g6ce060a #1 [ 0.224000] task: ffff8800128c8000 ti: ffff8800128c4000 task.ti: ffff8800128c4000 [ 0.224000] RIP: 0010:[<0000000000000000>] [< (null)>] (null) [ 0.224000] RSP: 0000:ffff8800128c7d68 EFLAGS: 00010246 [ 0.224000] RAX: ffff8800128abbe8 RBX: ffff8800128ab7c0 RCX: 000000000000008f [ 0.224000] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff8800128ab7c0 [ 0.224000] RBP: ffff8800128c7d90 R08: 0000000000000002 R09: 000000000011e08f [ 0.224000] R10: 0000000000000000 R11: 00000000001d2f90 R12: ffffffff827fda98 [ 0.224000] R13: ffffffff827fda98 R14: 0000000000000000 R15: 0000000000000000 [ 0.224000] FS: 0000000000000000(0000) GS:ffff880013600000(0000) knlGS:0000000000000000 [ 0.224000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.224000] CR2: 0000000000000000 CR3: 000000000260d000 CR4: 00000000000006f0 [ 0.224000] Stack: [ 0.224000] ffffffff816ee1a7 ffff8800128ab7c0 ffffffff827fda98 ffff8800128ab820 [ 0.224000] 0000000000000000 ffff8800128c7db8 ffffffff816ee4c6 0000000000000000 [ 0.224000] ffffffff827fda98 ffffffff816ee431 ffff8800128c7df0 ffffffff816ec084 [ 0.224000] Call Trace: [ 0.224000] [<ffffffff816ee1a7>] ? driver_probe_device+0x8c/0x316 [ 0.224000] [<ffffffff816ee4c6>] __driver_attach+0x95/0xd3 [ 0.224000] [<ffffffff816ee431>] ? driver_probe_device+0x316/0x316 [ 0.224000] [<ffffffff816ec084>] bus_for_each_dev+0xbe/0xc9 [ 0.224000] [<ffffffff816ee65b>] driver_attach+0x20/0x29 [ 0.224000] [<ffffffff816ecc5d>] bus_add_driver+0x112/0x2ab [ 0.224000] [<ffffffff816eefe3>] driver_register+0xec/0x152 [ 0.224000] [<ffffffff816f096f>] __platform_driver_register+0x61/0x6a [ 0.224000] [<ffffffff82c81ca7>] regulator_dummy_init+0xa0/0xde [ 0.224000] [<ffffffff82c81a2f>] regulator_init+0x90/0x9c [ 0.224000] [<ffffffff82c8199f>] ? virtio_balloon_driver_init+0x20/0x20 [ 0.224000] [<ffffffff82c486e9>] do_one_initcall+0x171/0x248 [ 0.224000] [<ffffffff810e2f00>] ? parameq+0x6/0x2d [ 0.224000] [<ffffffff810e31bd>] ? parse_args+0x296/0x3e8 [ 0.224000] [<ffffffff82c48aa9>] kernel_init_freeable+0x2e9/0x3e8 [ 0.224000] [<ffffffff82c47911>] ? do_early_param+0xd7/0xd7 [ 0.224000] [<ffffffff81caf6a1>] ? rest_init+0x115/0x115 [ 0.224000] [<ffffffff81caf6b1>] kernel_init+0x10/0x175 [ 0.224000] [<ffffffff81cd533c>] ret_from_fork+0x7c/0xb0 [ 0.224000] [<ffffffff81caf6a1>] ? rest_init+0x115/0x115 [ 0.224000] Code: Bad RIP value. [ 0.224000] RIP [< (null)>] (null) [ 0.224000] RSP <ffff8800128c7d68> [ 0.224000] CR2: 0000000000000000 [ 0.224000] ---[ end trace d3295e197e7ac925 ]--- [ 0.224000] ------------[ cut here ]------------ git bisect start aafca41bf8b2c093befa74e04c59580a4401b8e9 a497c3ba1d97fc69c1e78e7b96435ba8c2cb42ee -- git bisect good 5bb41e68bb216e78a850319a64b549c0ceaeddf8 # 15:15 20+ 0 Merge 'ipvs-next/master' into devel-hourly-2014062314 git bisect good 8c804d4bb30cfef4a4af0f561d6ef31e84014ad5 # 15:23 20+ 0 Merge 'asoc/topic/intel' into devel-hourly-2014062314 git bisect good b3b5fba3e162ab85c6b32b5425074cce21fbdfbc # 15:28 20+ 0 Merge 'block/for-linus' into devel-hourly-2014062314 git bisect bad 07e8581d455a832a85e7917a9601aa1aa4b30965 # 15:29 0- 20 Merge 'tianyu/dep_support' into devel-hourly-2014062314 git bisect good 4033d5f4a151180ad86e6b0b45760f872b07f3e3 # 15:34 20+ 0 Merge 'net-next/master' into devel-hourly-2014062314 git bisect good ca2bb6349593d783b7524e550d468d3ba259d62b # 15:40 20+ 0 I2C/ACPI: Add i2c ACPI operation region support git bisect bad f93d3e49ea1609fe7c7828a94b16619ff68c4f27 # 15:43 0- 20 ACPI: Resolve battery probe issue on the T100TA git bisect bad 6ce060a8738b36c60ad4b2567d393132b2620121 # 15:46 0- 20 Device/core: Add platform_dev_attach/detach_notify() callback git bisect good 96e8c1a7c36ba7e14644aba544283ff4dfa2bff8 # 15:50 20+ 0 I2C/ACPI: Clean up I2C ACPI code and Add CONFIG_I2C_ACPI config # first bad commit: [6ce060a8738b36c60ad4b2567d393132b2620121] Device/core: Add platform_dev_attach/detach_notify() callback git bisect good 96e8c1a7c36ba7e14644aba544283ff4dfa2bff8 # 15:52 60+ 0 I2C/ACPI: Clean up I2C ACPI code and Add CONFIG_I2C_ACPI config git bisect bad aafca41bf8b2c093befa74e04c59580a4401b8e9 # 15:52 0- 13 0day head guard for 'devel-hourly-2014062314' git bisect good a497c3ba1d97fc69c1e78e7b96435ba8c2cb42ee # 15:56 60+ 0 Linux 3.16-rc2 git bisect good 58ae500a03a6bf68eee323c342431bfdd3f460b6 # 16:03 60+ 0 Add linux-next specific files for 20140623 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

2
1
0 / 0

[tracing] 939c7a4f04f: -46.4% cpuidle.C3-IVT.time

by Jet Chen

Hi Yoshihiro, FYI, we noticed the below changes on commit 939c7a4f04fcd2162109744e8bf88194948a6e65 ("tracing: Introduce saved_cmdlines_size file") test case: brickland3/aim7/3000-brk_test beba4bb096201ce 939c7a4f04fcd2162109744e8 --------------- ------------------------- 352643 ~40% -46.4% 189136 ~15% TOTAL cpuidle.C3-IVT.time 750 ~ 2% -13.3% 650 ~ 2% TOTAL vmstat.procs.b 362016 ~ 3% -6.0% 340296 ~ 3% TOTAL proc-vmstat.pgmigrate_success 362016 ~ 3% -6.0% 340296 ~ 3% TOTAL proc-vmstat.numa_pages_migrated 1642328 ~ 3% +13.3% 1860721 ~ 2% TOTAL softirqs.RCU 2278829 ~ 2% -14.0% 1959647 ~ 2% TOTAL time.voluntary_context_switches 207 ~ 0% -10.4% 185 ~ 0% TOTAL time.user_time ~ 0% +8.5% ~ 0% TOTAL turbostat.RAM_W 4418246 ~ 3% -6.4% 4136447 ~ 1% TOTAL time.minor_page_faults ~ 0% +3.4% ~ 0% TOTAL turbostat.Cor_W ~ 0% +3.0% ~ 0% TOTAL turbostat.Pkg_W 11354889 ~ 0% +2.0% 11579897 ~ 0% TOTAL time.involuntary_context_switches Legend: ~XX% - stddev percent [+-]XX% - change percent time.voluntary_context_switches 2.4e+06 ++---------------------------------------------------*------------+ *.. *.. .. .. | 2.3e+06 ++ .*.. *.. .. * | | *..*. . .*.. .. * *.. + *.. *..* | *. * . .*..*.. + .. | 2.2e+06 ++ *..*. * * | | | 2.1e+06 ++ | | | 2e+06 O+ O O O | | O O O O | | O O O O O | 1.9e+06 ++ O O O O | | O O O | 1.8e+06 ++----------------------------------------------------------------+ turbostat.Pkg_W ++--------------------------------------------------------------------+ O+ O O O O O | | O O O O O O O O O O O O | ++ O O | ++ | ++ | ++ | | | ++ | ++ | ++ *.. .*.. | *+.*... .. *...*. *...*..*..*...*..*.. ..*.. *...*..*..*...*..* | * *. .. | ++ * | ++--------------------------------------------------------------------+ turbostat.Cor_W ++--------------------------------------------------------------------+ O+ O O O O O O O O O | | O O O O O O O O O O | ++ | ++ | | | ++ | ++ | ++ | | | ++ *.. | *+.*... .. *...*..*..*...*..*..*...*..*.. ..*.. *... .*..*...*..* | * *. .. *. | ++ * | ++--------------------------------------------------------------------+ turbostat.RAM_W ++---------------------------------------------------------------------+ | O O O O O O O O | O+ O O O O O O O O O O | | O | ++ | | | ++ | | | ++ | | * | ++ .. + *.. | | .*..* + *.. *.. + ..*..*.. .*... | ++ .. + .. .. . + *. *...*. *..* *..*...*..* * *...* * | ++---------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Jet

7 years, 5 months

4
4
0 / 0

[net] b58537a1f56: +89.2% netperf.Throughput_Mbps

by Jet Chen

Hi Daniel, FYI, we noticed the below changes on commit b58537a1f5629bdc98a8b9dc2051ce0e952f6b4b ("net: sctp: fix permissions for rto_alpha and rto_beta knobs") e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 950 ~ 1% +93.7% 1841 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 750 ~ 1% +83.4% 1375 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1700 ~ 1% +89.2% 3217 ~ 0% TOTAL netperf.Throughput_Mbps e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 7841 ~ 0% +50.4% 11792 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 7603 ~ 0% +37.6% 10463 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 15445 ~ 0% +44.1% 22256 ~ 0% TOTAL netperf.Throughput_tps e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 5.93 ~ 1% -99.0% 0.06 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 6.53 ~ 1% -99.1% 0.06 ~12% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 12.46 ~ 1% -99.1% 0.12 ~ 6% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_cmp_addr_exact.sctp_assoc_lookup_paddr.sctp_endpoint_lookup_assoc.sctp_sendmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 6.35 ~ 0% -98.3% 0.11 ~ 3% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.95 ~ 5% -100.0% 0.00 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 2.26 ~ 6% -100.0% 0.00 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 6.68 ~ 0% -98.5% 0.10 ~10% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 18.23 ~ 1% -98.9% 0.21 ~ 7% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_chunk_iif.sctp_ulpevent_make_rcvmsg.sctp_ulpq_tail_data.sctp_cmd_interpreter e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3.07 ~ 0% -99.0% 0.03 ~12% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 4.84 ~ 0% -100.0% 0.00 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 3.92 ~ 5% -100.0% 0.00 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 3.17 ~ 0% -99.0% 0.03 ~12% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 15.01 ~ 1% -99.6% 0.06 ~12% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_cmp_addr_exact.sctp_assoc_lookup_paddr.sctp_assoc_is_match.__sctp_lookup_association e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.92 ~ 1% -97.9% 0.04 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 3.78 ~ 5% -99.3% 0.03 ~18% lkp-nex04/netperf/300s-200%-SCTP_RR 2.91 ~ 5% -99.0% 0.03 ~14% lkp-nex05/netperf/300s-200%-SCTP_RR 2.00 ~ 1% -98.2% 0.04 ~13% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 10.61 ~ 3% -98.8% 0.13 ~10% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_eat_data.sctp_sf_eat_data_6_2.sctp_do_sm.sctp_assoc_bh_rcv e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3.36 ~ 0% -97.1% 0.10 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 6.16 ~ 0% -99.3% 0.04 ~23% lkp-nex04/netperf/300s-200%-SCTP_RR 4.91 ~ 5% -98.9% 0.05 ~22% lkp-nex05/netperf/300s-200%-SCTP_RR 3.47 ~ 1% -97.4% 0.09 ~ 7% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 17.90 ~ 2% -98.4% 0.28 ~11% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_rcv.ip_local_deliver_finish.ip_local_deliver.ip_rcv_finish e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 6.94 ~ 0% -97.3% 0.18 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 7.54 ~ 0% -97.6% 0.18 ~10% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 14.48 ~ 0% -97.5% 0.36 ~ 7% TOTAL perf-profile.cpu-cycles.sctp_get_af_specific.sctp_sockaddr_af.sctp_sendmsg.inet_sendmsg.sock_sendmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 6207 ~ 0% +273.8% 23202 ~36% lkp-nex05/netperf/300s-200%-SCTP_RR 6207 ~ 0% +273.8% 23202 ~36% TOTAL numa-vmstat.node3.nr_active_file e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 24831 ~ 0% +273.8% 92810 ~36% lkp-nex05/netperf/300s-200%-SCTP_RR 24831 ~ 0% +273.8% 92810 ~36% TOTAL numa-meminfo.node3.Active(file) e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 27053 ~ 3% +237.7% 91368 ~10% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 606 ~26% -68.2% 192 ~11% lkp-nex05/netperf/300s-200%-SCTP_RR 83164 ~14% +322.9% 351725 ~26% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 110824 ~12% +300.0% 443286 ~23% TOTAL cpuidle.C1E-NHM.usage e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 16056 ~ 8% +278.8% 60825 ~ 6% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 56085 ~13% +177.5% 155640 ~12% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 72141 ~12% +200.1% 216465 ~10% TOTAL cpuidle.C1-NHM.usage e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 2.96 ~ 0% +28.7% 3.81 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 4.55 ~ 2% -86.3% 0.62 ~ 7% lkp-nex04/netperf/300s-200%-SCTP_RR 3.73 ~ 4% -88.3% 0.44 ~ 2% lkp-nex05/netperf/300s-200%-SCTP_RR 3.04 ~ 0% +65.8% 5.04 ~ 1% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 14.29 ~ 2% -30.6% 9.91 ~ 2% TOTAL perf-profile.cpu-cycles.lock_timer_base.isra.34.mod_timer.sctp_transport_reset_timers.sctp_outq_flush.sctp_outq_uncork e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 20838 ~ 3% -42.8% 11916 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 62995 ~10% -78.7% 13392 ~10% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 83834 ~ 8% -69.8% 25309 ~ 6% TOTAL cpuidle.C3-NHM.usage e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 2.21 ~ 1% +22.3% 2.70 ~ 6% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.57 ~ 4% -86.5% 0.21 ~11% lkp-nex04/netperf/300s-200%-SCTP_RR 1.29 ~ 6% -89.2% 0.14 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 2.30 ~ 1% +64.5% 3.79 ~ 2% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 7.38 ~ 2% -7.3% 6.84 ~ 4% TOTAL perf-profile.cpu-cycles._raw_spin_lock_irqsave.mod_timer.sctp_transport_reset_timers.sctp_outq_flush.sctp_outq_uncork e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 6.74 ~ 0% +17.9% 7.94 ~ 5% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 5.93 ~ 0% -79.4% 1.22 ~ 4% lkp-nex04/netperf/300s-200%-SCTP_RR 4.78 ~ 6% -80.4% 0.94 ~ 2% lkp-nex05/netperf/300s-200%-SCTP_RR 6.84 ~ 0% +55.5% 10.63 ~ 1% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 24.28 ~ 1% -14.6% 20.73 ~ 3% TOTAL perf-profile.cpu-cycles.sctp_transport_timeout.sctp_transport_reset_timers.sctp_outq_flush.sctp_outq_uncork.sctp_cmd_interpreter e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.86 ~ 4% +172.3% 2.34 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.09 ~ 9% +207.7% 3.34 ~ 5% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.94 ~ 7% +192.1% 5.68 ~ 5% TOTAL perf-profile.cpu-cycles._raw_spin_lock.free_one_page.__free_pages_ok.__free_pages.__free_kmem_pages e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 43260 ~ 2% +158.2% 111705 ~28% lkp-nex05/netperf/300s-200%-SCTP_RR 43260 ~ 2% +158.2% 111705 ~28% TOTAL numa-meminfo.node3.Active e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.439e+08 ~ 1% +94.1% 2.793e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.434e+08 ~ 2% +81.3% 2.599e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.872e+08 ~ 1% +87.7% 5.392e+08 ~ 0% TOTAL numa-numastat.node3.local_node e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.439e+08 ~ 1% +94.1% 2.793e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.434e+08 ~ 2% +81.3% 2.599e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.873e+08 ~ 1% +87.7% 5.392e+08 ~ 0% TOTAL numa-numastat.node3.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 52800 ~ 7% -53.7% 24464 ~ 3% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 52800 ~ 7% -53.7% 24464 ~ 3% TOTAL cpuidle.C6-NHM.usage e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.18 ~ 4% +93.1% 2.28 ~ 5% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.38 ~ 6% +141.2% 3.32 ~ 6% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.56 ~ 5% +119.0% 5.61 ~ 5% TOTAL perf-profile.cpu-cycles._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.alloc_kmem_pages_node.kmalloc_large_node e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 33854097 ~ 2% +111.8% 71718667 ~ 1% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 56631220 ~ 2% +99.7% 1.131e+08 ~ 1% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 90485318 ~ 2% +104.2% 1.848e+08 ~ 1% TOTAL proc-vmstat.pgalloc_dma32 e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71085188 ~ 1% +93.5% 1.375e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 165875 ~31% +68.2% 279079 ~33% lkp-nex05/netperf/300s-200%-SCTP_RR 70852917 ~ 2% +80.5% 1.279e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.421e+08 ~ 1% +87.0% 2.657e+08 ~ 0% TOTAL numa-vmstat.node3.numa_local e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 568 ~16% +42.3% 809 ~29% lkp-nex04/netperf/300s-200%-SCTP_RR 568 ~16% +42.3% 809 ~29% TOTAL numa-meminfo.node1.Mlocked e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.452e+08 ~ 1% +93.1% 2.804e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.426e+08 ~ 1% +85.0% 2.638e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.878e+08 ~ 1% +89.1% 5.442e+08 ~ 0% TOTAL numa-numastat.node0.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.452e+08 ~ 1% +93.1% 2.804e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.426e+08 ~ 1% +85.0% 2.638e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.878e+08 ~ 1% +89.1% 5.442e+08 ~ 0% TOTAL numa-numastat.node0.local_node e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 5351090 ~19% +66.0% 8883481 ~14% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 7254502 ~10% +103.9% 14794044 ~16% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 12605592 ~14% +87.8% 23677525 ~15% TOTAL cpuidle.C1-NHM.time e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.446e+08 ~ 1% +93.2% 2.794e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.414e+08 ~ 0% +84.4% 2.608e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.86e+08 ~ 1% +88.9% 5.402e+08 ~ 0% TOTAL numa-numastat.node1.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.446e+08 ~ 1% +93.2% 2.794e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.414e+08 ~ 0% +84.4% 2.608e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.86e+08 ~ 1% +88.9% 5.402e+08 ~ 0% TOTAL numa-numastat.node1.local_node e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3.043e+09 ~ 1% +93.6% 5.892e+09 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 3.001e+09 ~ 1% +83.4% 5.504e+09 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 6.044e+09 ~ 1% +88.5% 1.14e+10 ~ 0% TOTAL proc-vmstat.pgfree e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 5.777e+08 ~ 1% +93.5% 1.118e+09 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 5.698e+08 ~ 1% +83.3% 1.044e+09 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.147e+09 ~ 1% +88.4% 2.162e+09 ~ 0% TOTAL proc-vmstat.numa_local e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 5.777e+08 ~ 1% +93.5% 1.118e+09 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 5.698e+08 ~ 1% +83.3% 1.044e+09 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.148e+09 ~ 1% +88.4% 2.162e+09 ~ 0% TOTAL proc-vmstat.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71917522 ~ 1% +92.1% 1.382e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 70494566 ~ 0% +84.2% 1.298e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.424e+08 ~ 1% +88.2% 2.68e+08 ~ 0% TOTAL numa-vmstat.node0.numa_local e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71921559 ~ 1% +92.1% 1.382e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 70498641 ~ 0% +84.2% 1.298e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.424e+08 ~ 1% +88.2% 2.68e+08 ~ 0% TOTAL numa-vmstat.node0.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3.009e+09 ~ 1% +93.4% 5.82e+09 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.945e+09 ~ 1% +83.1% 5.391e+09 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 5.954e+09 ~ 1% +88.3% 1.121e+10 ~ 0% TOTAL proc-vmstat.pgalloc_normal e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71389128 ~ 1% +92.7% 1.376e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 69896469 ~ 0% +83.3% 1.281e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.413e+08 ~ 1% +88.1% 2.657e+08 ~ 0% TOTAL numa-vmstat.node1.numa_local e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71445543 ~ 1% +92.7% 1.376e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 69943098 ~ 0% +83.2% 1.282e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.414e+08 ~ 1% +88.0% 2.658e+08 ~ 0% TOTAL numa-vmstat.node1.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71144941 ~ 1% +93.4% 1.376e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 244384 ~21% +45.4% 355361 ~26% lkp-nex05/netperf/300s-200%-SCTP_RR 70900937 ~ 2% +80.4% 1.279e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.423e+08 ~ 1% +86.8% 2.659e+08 ~ 0% TOTAL numa-vmstat.node3.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.44e+08 ~ 1% +93.7% 2.789e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.424e+08 ~ 3% +82.4% 2.597e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.864e+08 ~ 2% +88.1% 5.386e+08 ~ 0% TOTAL numa-numastat.node2.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.44e+08 ~ 1% +93.7% 2.789e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.424e+08 ~ 3% +82.4% 2.597e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.864e+08 ~ 2% +88.1% 5.386e+08 ~ 0% TOTAL numa-numastat.node2.local_node e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71218513 ~ 1% +92.8% 1.373e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 70566629 ~ 3% +81.1% 1.278e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.418e+08 ~ 2% +87.0% 2.652e+08 ~ 0% TOTAL numa-vmstat.node2.numa_local e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 71277689 ~ 1% +92.8% 1.374e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 70612117 ~ 3% +81.1% 1.279e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.419e+08 ~ 2% +86.9% 2.653e+08 ~ 0% TOTAL numa-vmstat.node2.numa_hit e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.049e+08 ~ 1% +93.7% 2.031e+08 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 3.011e+08 ~ 0% +50.4% 4.528e+08 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 2.92e+08 ~ 0% +37.6% 4.018e+08 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 1.034e+08 ~ 1% +83.4% 1.897e+08 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 8.014e+08 ~ 0% +55.7% 1.247e+09 ~ 0% TOTAL softirqs.NET_RX e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 144 ~16% +40.9% 203 ~29% lkp-nex04/netperf/300s-200%-SCTP_RR 234 ~15% -16.4% 196 ~14% lkp-nex05/netperf/300s-200%-SCTP_RR 379 ~15% +5.5% 400 ~21% TOTAL numa-vmstat.node1.nr_unevictable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 578 ~16% +41.0% 815 ~29% lkp-nex04/netperf/300s-200%-SCTP_RR 938 ~15% -16.4% 784 ~14% lkp-nex05/netperf/300s-200%-SCTP_RR 1516 ~15% +5.5% 1600 ~21% TOTAL numa-meminfo.node1.Unevictable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 4.99 ~ 0% +58.4% 7.90 ~ 1% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 4.43 ~ 1% +55.3% 6.87 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 9.41 ~ 1% +56.9% 14.77 ~ 1% TOTAL perf-profile.cpu-cycles.copy_user_generic_string.sctp_user_addto_chunk.sctp_datamsg_from_user.sctp_sendmsg.inet_sendmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 156328 ~ 1% +45.6% 227560 ~15% lkp-nex05/netperf/300s-200%-SCTP_RR 156328 ~ 1% +45.6% 227560 ~15% TOTAL numa-meminfo.node3.FilePages e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 39081 ~ 1% +45.6% 56889 ~15% lkp-nex05/netperf/300s-200%-SCTP_RR 39081 ~ 1% +45.6% 56889 ~15% TOTAL numa-vmstat.node3.nr_file_pages e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.09 ~12% +42.2% 1.55 ~ 2% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.05 ~16% +43.8% 1.51 ~ 6% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.14 ~14% +43.0% 3.06 ~ 4% TOTAL perf-profile.cpu-cycles.get_page_from_freelist.__alloc_pages_nodemask.alloc_kmem_pages_node.kmalloc_large_node.__kmalloc_node_track_caller e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 11745167 ~15% +66.1% 19514308 ~22% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 11745167 ~15% +66.1% 19514308 ~22% TOTAL cpuidle.C1E-NHM.time e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3.95 ~ 1% +47.3% 5.81 ~ 1% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 3.44 ~ 2% +47.2% 5.06 ~ 1% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 7.39 ~ 1% +47.3% 10.88 ~ 1% TOTAL perf-profile.cpu-cycles.memcpy.sctp_outq_flush.sctp_outq_uncork.sctp_cmd_interpreter.sctp_do_sm e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 2.14 ~ 1% +44.7% 3.09 ~ 1% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.84 ~ 3% +42.7% 2.63 ~ 2% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 3.98 ~ 2% +43.7% 5.72 ~ 1% TOTAL perf-profile.cpu-cycles.memcpy.sctp_packet_transmit_chunk.sctp_outq_flush.sctp_outq_uncork.sctp_cmd_interpreter e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 5.27 ~ 0% +44.6% 7.62 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 4.71 ~ 1% +39.0% 6.55 ~ 1% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 9.98 ~ 1% +42.0% 14.17 ~ 1% TOTAL perf-profile.cpu-cycles.copy_user_generic_string.skb_copy_datagram_iovec.sctp_recvmsg.sock_common_recvmsg.sock_recvmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.92 ~ 3% +44.9% 1.34 ~ 3% lkp-nex04/netperf/300s-200%-SCTP_RR 0.94 ~ 3% +34.3% 1.26 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 1.86 ~ 3% +39.5% 2.60 ~ 3% TOTAL perf-profile.cpu-cycles.sctp_recvmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.64 ~ 3% +40.8% 0.90 ~ 8% lkp-nex04/netperf/300s-200%-SCTP_RR 0.64 ~ 3% +40.8% 0.90 ~ 8% TOTAL perf-profile.cpu-cycles.enqueue_entity.enqueue_task_fair.enqueue_task.activate_task.ttwu_do_activate e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.10 ~ 2% +24.8% 1.37 ~ 3% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1.10 ~ 3% +55.9% 1.72 ~ 2% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 2.20 ~ 2% +40.4% 3.09 ~ 2% TOTAL perf-profile.cpu-cycles.sctp_outq_flush.sctp_outq_uncork.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.81 ~12% +26.8% 1.03 ~ 2% lkp-nex04/netperf/300s-200%-SCTP_RR 0.81 ~12% +26.8% 1.03 ~ 2% TOTAL perf-profile.cpu-cycles.sctp_outq_flush.sctp_outq_uncork.sctp_cmd_interpreter.sctp_do_sm.sctp_primitive_SEND e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.94 ~ 3% +40.4% 1.31 ~ 1% lkp-nex04/netperf/300s-200%-SCTP_RR 1.03 ~ 5% +35.5% 1.39 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 1.96 ~ 4% +37.8% 2.70 ~ 2% TOTAL perf-profile.cpu-cycles.__schedule.schedule.schedule_timeout.sctp_recvmsg.sock_common_recvmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.72 ~11% +33.1% 0.96 ~ 2% lkp-nex05/netperf/300s-200%-SCTP_RR 0.72 ~11% +33.1% 0.96 ~ 2% TOTAL perf-profile.cpu-cycles.idle_cpu.select_task_rq_fair.try_to_wake_up.default_wake_function.autoremove_wake_function e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1.14 ~ 7% +36.1% 1.55 ~ 7% lkp-nex04/netperf/300s-200%-SCTP_RR 1.21 ~ 6% +37.5% 1.66 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 2.34 ~ 7% +36.8% 3.20 ~ 6% TOTAL perf-profile.cpu-cycles.sctp_sendmsg.inet_sendmsg.sock_sendmsg.___sys_sendmsg.__sys_sendmsg e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.73 ~ 7% +35.4% 0.99 ~ 2% lkp-nex04/netperf/300s-200%-SCTP_RR 0.75 ~ 4% +32.3% 0.99 ~ 2% lkp-nex05/netperf/300s-200%-SCTP_RR 1.48 ~ 6% +33.8% 1.98 ~ 2% TOTAL perf-profile.cpu-cycles.select_task_rq_fair.try_to_wake_up.default_wake_function.autoremove_wake_function.__wake_up_common e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.76 ~ 9% +38.9% 1.06 ~11% lkp-nex04/netperf/300s-200%-SCTP_RR 0.85 ~ 4% +33.4% 1.13 ~ 5% lkp-nex05/netperf/300s-200%-SCTP_RR 1.61 ~ 6% +36.0% 2.19 ~ 8% TOTAL perf-profile.cpu-cycles.sctp_check_transmitted.sctp_outq_sack.sctp_cmd_interpreter.sctp_do_sm.sctp_assoc_bh_rcv e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1897 ~18% +27.9% 2427 ~11% lkp-nex04/netperf/300s-200%-SCTP_RR 1803 ~ 7% +16.4% 2099 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 3700 ~12% +22.3% 4527 ~ 8% TOTAL proc-vmstat.pgmigrate_success e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1897 ~18% +27.9% 2427 ~11% lkp-nex04/netperf/300s-200%-SCTP_RR 1803 ~ 7% +16.4% 2099 ~ 4% lkp-nex05/netperf/300s-200%-SCTP_RR 3700 ~12% +22.3% 4527 ~ 8% TOTAL proc-vmstat.numa_pages_migrated e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 0.63 ~ 9% -14.2% 0.54 ~14% lkp-nex04/netperf/300s-200%-SCTP_RR 0.63 ~ 9% -14.2% 0.54 ~14% TOTAL turbostat.%c1 e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 3097 ~ 3% +19.3% 3695 ~ 8% lkp-nex05/netperf/300s-200%-SCTP_RR 3097 ~ 3% +19.3% 3695 ~ 8% TOTAL numa-vmstat.node3.nr_slab_reclaimable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 12389 ~ 3% +19.3% 14780 ~ 8% lkp-nex05/netperf/300s-200%-SCTP_RR 12389 ~ 3% +19.3% 14780 ~ 8% TOTAL numa-meminfo.node3.SReclaimable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 10788 ~ 0% +19.0% 12843 ~ 9% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 10788 ~ 0% +19.0% 12843 ~ 9% TOTAL numa-meminfo.node0.SReclaimable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 2696 ~ 0% +19.0% 3210 ~ 9% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 2696 ~ 0% +19.0% 3210 ~ 9% TOTAL numa-vmstat.node0.nr_slab_reclaimable e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 634073 ~ 0% +12.3% 712155 ~ 5% lkp-nex05/netperf/300s-200%-SCTP_RR 634073 ~ 0% +12.3% 712155 ~ 5% TOTAL numa-meminfo.node3.MemUsed e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 727 ~ 4% +12.1% 816 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 727 ~ 4% +12.1% 816 ~ 4% TOTAL slabinfo.dnotify_mark.num_objs e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 727 ~ 4% +12.1% 816 ~ 4% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 727 ~ 4% +12.1% 816 ~ 4% TOTAL slabinfo.dnotify_mark.active_objs e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 121677 ~ 5% -6.7% 113473 ~ 5% lkp-nex04/netperf/300s-200%-SCTP_RR 121677 ~ 5% -6.7% 113473 ~ 5% TOTAL meminfo.DirectMap4k e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 1541914 ~ 3% -10.5% 1380345 ~ 2% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 1541914 ~ 3% -10.5% 1380345 ~ 2% TOTAL softirqs.RCU e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 37167 ~ 3% +10.3% 41005 ~ 2% lkp-nex05/netperf/300s-200%-SCTP_RR 37167 ~ 3% +10.3% 41005 ~ 2% TOTAL numa-meminfo.node3.Slab e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 93363 ~ 0% -9.2% 84779 ~ 1% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 93363 ~ 0% -9.2% 84779 ~ 1% TOTAL proc-vmstat.pgactivate e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 7017 ~ 3% +15.3% 8091 ~14% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 7017 ~ 3% +15.3% 8091 ~14% TOTAL numa-meminfo.node0.KernelStack e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 174505 ~ 1% +92.8% 336502 ~ 0% lkp-nex04/netperf/300s-200%-10K-SCTP_STREAM_MANY 1942254 ~ 1% +51.6% 2944825 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 1904295 ~ 0% +37.7% 2621695 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 173470 ~ 1% +82.5% 316560 ~ 0% lkp-wsx02/netperf/300s-200%-10K-SCTP_STREAM_MANY 4194525 ~ 0% +48.3% 6219584 ~ 0% TOTAL vmstat.system.cs e4f7ae930afafd4 b58537a1f5629bdc98a8b9dc2 --------------- ------------------------- 66725 ~ 0% +3.8% 69259 ~ 0% lkp-nex04/netperf/300s-200%-SCTP_RR 67344 ~ 0% +2.5% 69040 ~ 0% lkp-nex05/netperf/300s-200%-SCTP_RR 134069 ~ 0% +3.2% 138300 ~ 0% TOTAL vmstat.system.in Legend: ~XX% - stddev percent [+-]XX% - change percent netperf.Throughput_Mbps 1900 O+------O------------------------------------------------------------+ 1800 ++ O O O O O O O O O O O O O O O O O | | | 1700 ++ | 1600 ++ | | | 1500 ++ | 1400 ++ | 1300 ++ | | | 1200 ++ | 1100 ++ | | | 1000 *+. .*..*..*.*..*..*. .*..*.*..*..*.*..*. .*..*. .*..*.*..*..*. .* 900 ++-*-----------------*--------------------*-------*---------------*--+ vmstat.system.cs 360000 ++-----------------------------------------------------------------+ 340000 O+ O O O O | | O O O O O O O O O O O O O O | 320000 ++ | 300000 ++ | | | 280000 ++ | 260000 ++ | 240000 ++ | | | 220000 ++ | 200000 ++ | | | 180000 *+.*.*..*.*..*.*..*..*.*..*.*..*..*.*..*.*..*.*..*..*.*..*.*..*.*..* 160000 ++-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Jet

7 years, 5 months

3
4
0 / 0

[HZ] INFO: task kworker/0:1:16 blocked for more than 120 seconds.

by Jet Chen

Hi Maarten, 0day kernel testing robot got the below dmesg and the first bad commit is git://people.freedesktop.org/~mlankhorst/linux master commit 52774d9273d6b8aff39822ccfe10e5a0d7c3a7f8 Author: Maarten Lankhorst <m.b.lankhorst(a)gmail.com> AuthorDate: Thu Jan 16 16:24:04 2014 +0100 Commit: Maarten Lankhorst <maarten.lankhorst(a)canonical.com> CommitDate: Wed Jun 18 14:19:52 2014 +0200 There's some really badly broken software out there that is entirely dependant on HZ for its maximum performance. Raise the maximum HZ value to some higher and slightly unreasonable values up to some higher and completely obscene values. -ck +--------------------------------------------------+------------+------------+ | | c03339ffa3 | 52774d9273 | +--------------------------------------------------+------------+------------+ | boot_successes | 60 | 0 | | boot_failures | 0 | 20 | | INFO:task_blocked_for_more_than_seconds | 0 | 20 | | RIP:flat_send_IPI_mask | 0 | 20 | | Kernel_panic-not_syncing:hung_task:blocked_tasks | 0 | 20 | | backtrace:serio_handle_event | 0 | 20 | | backtrace:watchdog | 0 | 20 | +--------------------------------------------------+------------+------------+ [ 69.176969] torture thread lock_torture_writer parking due to system shutdown [ 120.553588] spin_lock-torture: Writes: Total: 2 Max/Min: 0/0 Fail: 0 [ 120.554375] torture thread lock_torture_stats parking due to system shutdown [ 240.031059] INFO: task kworker/0:1:16 blocked for more than 120 seconds. [ 240.031837] Not tainted 3.16.0-rc1-00045-g52774d9 #1 [ 240.032443] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 240.033280] kworker/0:1 D ffff880013612580 0 16 2 0x00000000 git bisect start 91283063c4f74fb202ac7809a8df3cc00f22203e 7171511eaec5bf23fb06078f59784a3a0626b38f -- git bisect good bc106bf69fdfba649ad98b3c7cfd2b63e84431a8 # 01:57 20+ 0 Merge 'dynticks/timers/missing-kick-v2' into devel-hourly-2014061900 git bisect bad 25f92e043e7e4ca029592e03a2d77a09f226f395 # 02:15 0- 1 Merge 'at91/abe/at91_3.16-rc1_CCF' into devel-hourly-2014061900 git bisect good 6c815b36fd13fd40f3545c9e6e33eb7d4b396d8d # 02:18 20+ 0 Merge 'efi/next' into devel-hourly-2014061900 git bisect good a235b89e0b1332b44baff6277b5dbc711df34b3d # 02:25 20+ 0 Merge 'kvm-ppc/kvm-ppc-queue' into devel-hourly-2014061900 git bisect bad 8b4870846e71960a223e729e3c18942edfe81f16 # 02:44 0- 4 Merge 'spi/for-next' into devel-hourly-2014061900 git bisect bad 2115cd10081ea65213b45d9970aeebe88dabcfb3 # 02:55 0- 5 Merge 'mlankhorst/master' into devel-hourly-2014061900 git bisect bad 52774d9273d6b8aff39822ccfe10e5a0d7c3a7f8 # 03:00 0- 4 There's some really badly broken software out there that is entirely git bisect good 0da2b800ec8dfad78c888b33be2ad2aea3411796 # 03:06 20+ 0 Merge branch 'for-nouveau' git bisect good c03339ffa3d9d2330c8c14b8d73e9a4da38101ea # 03:09 20+ 0 drm/nouveau: add guard pages to vm # first bad commit: [52774d9273d6b8aff39822ccfe10e5a0d7c3a7f8] There's some really badly broken software out there that is entirely git bisect good c03339ffa3d9d2330c8c14b8d73e9a4da38101ea # 03:12 60+ 0 drm/nouveau: add guard pages to vm git bisect bad 91283063c4f74fb202ac7809a8df3cc00f22203e # 03:12 0- 13 0day head guard for 'devel-hourly-2014061900' git bisect good e99cfa2d0634881b8a41d56c48b5956b9a3ba162 # 03:50 60+ 0 Merge tag 'dt-fixes-for-3.16' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux git bisect good 4ca1fbc122fcb09efa2bb1448bff8b71b46434c8 # 04:05 60+ 0 Add linux-next specific files for 20140618 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/blob/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

1
0
0 / 0

[x86_64, vsyscall] 21d4ab4881a: -11.1% will-it-scale.per_process_ops

by Jet Chen

Hi Andy, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git x86/vsyscall commit 21d4ab4881ad9b257bec75d04480105dad4336e1 ("x86_64,vsyscall: Move all of the gate_area code to vsyscall_64.c") test case: lkp-wsx01/will-it-scale/signal1 a7781f1035319a7 21d4ab4881ad9b257bec75d04 --------------- ------------------------- 259032 ~ 1% -11.1% 230288 ~ 0% TOTAL will-it-scale.per_process_ops 0.04 ~ 9% +4276.2% 1.84 ~ 1% TOTAL perf-profile.cpu-cycles.map_id_up.do_tkill.sys_tgkill.system_call_fastpath.raise 2.36 ~ 0% -63.8% 0.85 ~ 2% TOTAL perf-profile.cpu-cycles._atomic_dec_and_lock.free_uid.__sigqueue_free.__dequeue_signal.dequeue_signal 2.25 ~14% -55.2% 1.01 ~ 1% TOTAL perf-profile.cpu-cycles.raise 42.41 ~ 0% +34.5% 57.04 ~ 0% TOTAL perf-profile.cpu-cycles.__sigqueue_alloc.__send_signal.send_signal.do_send_sig_info.do_send_specific 40.70 ~ 0% -23.9% 30.96 ~ 0% TOTAL perf-profile.cpu-cycles.__sigqueue_free.part.11.__dequeue_signal.dequeue_signal.get_signal_to_deliver.do_signal 252 ~11% -18.8% 204 ~ 9% TOTAL numa-vmstat.node1.nr_page_table_pages 1012 ~11% -18.3% 827 ~ 9% TOTAL numa-meminfo.node1.PageTables 520 ~ 7% -17.1% 431 ~ 5% TOTAL cpuidle.C1-NHM.usage Legend: ~XX% - stddev percent [+-]XX% - change percent will-it-scale.per_process_ops 270000 ++-----------------------------------------------------------------+ | | 260000 *+.*..*.. .*.. .*.. .*..*... .*.. *.. .*.. *.. .*..* | *..*. *. *. *. .. *. .. *..*. | 250000 ++ * * | | | 240000 ++ | | | 230000 ++ O O O O O O O O O O O | O O O | 220000 ++ O O O | | O O O O | 210000 ++ O | | | 200000 ++-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Jet

7 years, 5 months

1
0
0 / 0

[bisect done] [ 0.013865] WARNING: CPU: 0 PID: 0 at fs/kernfs/dir.c:947 kernfs_activate()

by Jet Chen

Hi David, 0day kernel testing robot got the below dmesg and the first bad commit is git://people.freedesktop.org/~dvdhrm/linux fops commit 6a616be3329e4ae59ae33e8de7be9b62e9535baa Author: David Herrmann <dh.herrmann(a)gmail.com> AuthorDate: Tue Jun 17 22:02:20 2014 +0200 Commit: David Herrmann <dh.herrmann(a)gmail.com> CommitDate: Tue Jun 17 22:13:23 2014 +0200 kactive: extract active-ref implementation of kernfs This introduces a new reference-type "struct kactive". Unlike kref, this type manages "active references". That means, references can only be acquired if the object is active. At any time the object can be deactivated, causing any new attempt to acquire an active reference to fail. This patch extracts the kactive code from kernfs and turns it into an unlocked variant. kernfs is updated to use the new code. Signed-off-by: David Herrmann <dh.herrmann(a)gmail.com> +------------------------------------------------------+------------+------------+ | | 809f893c60 | 6a616be332 | +------------------------------------------------------+------------+------------+ | boot_successes | 60 | 0 | | boot_failures | 0 | 20 | | WARNING:CPU:PID:at_fs/kernfs/dir.c:kernfs_activate() | 0 | 20 | | backtrace:kernfs_create_root | 0 | 20 | | backtrace:sysfs_init | 0 | 20 | | backtrace:mnt_init | 0 | 20 | | backtrace:vfs_caches_init | 0 | 20 | | Out_of_memory:Kill_process | 0 | 1 | +------------------------------------------------------+------------+------------+ [ 0.011522] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.012007] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.013069] ------------[ cut here ]------------ [ 0.013865] WARNING: CPU: 0 PID: 0 at fs/kernfs/dir.c:947 kernfs_activate+0x90/0xd0() [ 0.016004] Modules linked in: [ 0.016765] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-rc1-00002-g6a616be #18 [ 0.018102] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 0.019012] 0000000000000000 ffffffff82003e18 ffffffff819c2b6e 0000000000000000 [ 0.020393] ffffffff82003e50 ffffffff810e0e90 ffffffff81249048 ffff88001167f000 [ 0.021966] ffff88001167f000 0000000000000002 0000000000000000 ffffffff82003e60 [ 0.023587] Call Trace: [ 0.024007] [<ffffffff819c2b6e>] dump_stack+0x4d/0x66 [ 0.024847] [<ffffffff810e0e90>] warn_slowpath_common+0x7f/0x98 [ 0.025744] [<ffffffff81249048>] ? kernfs_activate+0x90/0xd0 [ 0.026620] [<ffffffff810e0f57>] warn_slowpath_null+0x1a/0x1c [ 0.028005] [<ffffffff81249048>] kernfs_activate+0x90/0xd0 [ 0.028887] [<ffffffff812492f7>] kernfs_create_root+0xd8/0xe7 [ 0.029779] [<ffffffff82182212>] sysfs_init+0x13/0x51 [ 0.030593] [<ffffffff8217fa9d>] mnt_init+0x116/0x215 [ 0.031415] [<ffffffff8217f6cb>] vfs_caches_init+0x99/0x110 [ 0.032007] [<ffffffff82153eb1>] start_kernel+0x3d2/0x424 [ 0.032864] [<ffffffff821538ed>] ? set_init_arg+0x53/0x53 [ 0.033713] [<ffffffff82153120>] ? early_idt_handlers+0x120/0x120 [ 0.034706] [<ffffffff821534a2>] x86_64_start_reservations+0x2a/0x2c [ 0.036006] [<ffffffff821535df>] x86_64_start_kernel+0x13b/0x148 [ 0.036945] ---[ end trace a625764830fea92a ]--- [ 0.037981] Initializing cgroup subsys memory git bisect start 9cafa0eb8a6d3009f961c7b7fa14a2c5f149f9e2 7171511eaec5bf23fb06078f59784a3a0626b38f -- git bisect good 809f893c604209db19b38927ca4b927358eed321 # 04:43 20+ 0 fs: add f_op invokation guards git bisect bad 6a616be3329e4ae59ae33e8de7be9b62e9535baa # 04:44 0- 1 kactive: extract active-ref implementation of kernfs # first bad commit: [6a616be3329e4ae59ae33e8de7be9b62e9535baa] kactive: extract active-ref implementation of kernfs git bisect good 809f893c604209db19b38927ca4b927358eed321 # 04:46 60+ 0 fs: add f_op invokation guards git bisect bad 9cafa0eb8a6d3009f961c7b7fa14a2c5f149f9e2 # 04:46 0- 13 fs: add f_op->xy() active-protection git bisect good ebe06187bf2aec10d537ce4595e416035367d703 # 04:49 60+ 0 epoll: fix use-after-free in eventpoll_release_file git bisect good 5f295cdf5c5dbbb0c40f10f2ddae02ff46bbf773 # 04:52 60+ 0 Add linux-next specific files for 20140617 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

1
0
0 / 0

[idr] WARNING: at lib/idr.c:669 idr_find_slowpath()

by Jet Chen

Hi Tejun, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 0ffc2a9c8072969253a20821c2c733a2cbb4c7c7 Author: Tejun Heo <tj(a)kernel.org> AuthorDate: Wed Feb 27 17:05:08 2013 -0800 Commit: Linus Torvalds <torvalds(a)linux-foundation.org> CommitDate: Wed Feb 27 19:10:21 2013 -0800 idr: implement lookup hint While idr lookup isn't a particularly heavy operation, it still is too substantial to use in hot paths without worrying about the performance implications. With recent changes, each idr_layer covers 256 slots which should be enough to cover most use cases with single idr_layer making lookup hint very attractive. This patch adds idr->hint which points to the idr_layer which allocated an ID most recently and the fast path lookup becomes if (look up target's prefix matches that of the hinted layer) return hint->ary[ID's offset in the leaf layer]; which can be inlined. idr->hint is set to the leaf node on idr_fill_slot() and cleared from free_layer(). [andriy.shevchenko(a)linux.intel.com: always do slow path when hint is uninitialized] Signed-off-by: Tejun Heo <tj(a)kernel.org> Cc: Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> Cc: Sasha Levin <sasha.levin(a)oracle.com> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org> +--------------------------------------------------------------------------+------------+------------+ | | 54616283c2 | 0ffc2a9c80 | +--------------------------------------------------------------------------+------------+------------+ | boot_successes | 140 | 31 | | boot_failures | 320 | 84 | | BUG:soft_lockup-CPU_stuck_for_s | 128 | 40 | | EIP_is_at_stop_machine_cpu_stop | 128 | 40 | | Kernel_panic-not_syncing:softlockup:hung_tasks | 128 | 40 | | backtrace:smpboot_thread_fn | 128 | 40 | | BUG:unable_to_handle_kernel_paging_request | 67 | 18 | | Oops | 40 | 12 | | EIP_is_at_static_key_slow_inc | 35 | 10 | | Kernel_panic-not_syncing:Fatal_exception | 39 | 12 | | backtrace:perf_swevent_init | 37 | 10 | | backtrace:perf_init_event | 144 | 35 | | BUG:unable_to_handle_kernel_NULL_pointer_dereference | 2 | | | EIP_is_at_jump_label_update | 2 | | | backtrace:static_key_slow_inc | 2 | | | WARNING:at_lib/idr.c:idr_find() | 119 | | | EIP_is_at_atomic_dec_and_mutex_lock | 2 | 2 | | backtrace:idr_find | 111 | | | WARNING:at_kernel/jump_label.c:__static_key_slow_dec() | 3 | 1 | | BUG:kernel_test_crashed | 18 | 4 | | WARNING:at_kernel/lockdep.c:__bfs() | 1 | | | EIP_is_at__bfs | 1 | | | WARNING:at_arch/x86/kernel/hw_breakpoint.c:arch_install_hw_breakpoint() | 1 | | | BUG:sleeping_function_called_from_invalid_context_at_arch/x86/mm/fault.c | 1 | | | WARNING:at_lib/idr.c:idr_find_slowpath() | 0 | 27 | | backtrace:idr_find_slowpath | 0 | 26 | | WARNING:at_drivers/tty/tty_buffer.c:flush_to_ldisc() | 0 | 1 | | backtrace:flush_to_ldisc | 0 | 1 | +--------------------------------------------------------------------------+------------+------------+ Kernel tests: Boot OK! [ 17.603698] sock: process `trinity-main' is using obsolete setsockopt SO_BSDCOMPAT [ 17.798944] ------------[ cut here ]------------ [ 17.798972] WARNING: at lib/idr.c:669 idr_find_slowpath+0x28/0xca() [ 17.798973] Hardware name: Bochs [ 17.798976] Pid: 655, comm: trinity-main Not tainted 3.8.0-06482-g0ffc2a9 #68 [ 17.798976] Call Trace: [ 17.798989] [<c022ace6>] warn_slowpath_common+0x72/0x89 [ 17.798992] [<c051f001>] ? idr_find_slowpath+0x28/0xca [ 17.798994] [<c051f001>] ? idr_find_slowpath+0x28/0xca [ 17.798997] [<c022ad9b>] warn_slowpath_null+0x1d/0x1f [ 17.798999] [<c051f001>] idr_find_slowpath+0x28/0xca [ 17.799007] [<c027fab7>] perf_init_event+0x82/0x124 [ 17.799013] [<c024321f>] ? __mutex_init+0x3f/0x44 [ 17.799015] [<c027fda9>] perf_event_alloc+0x250/0x360 [ 17.799018] [<c028040e>] sys_perf_event_open+0x336/0x664 [ 17.799036] [<c0944280>] syscall_call+0x7/0xb [ 17.799042] [<c0940000>] ? SetAgcIf.isra.6+0x156/0x290 [ 17.799043] ---[ end trace 8e00e88be2b81021 ]--- [ 76.161086] Unregister pv shared memory for cpu 0 Attached dmesg for the parent commit, too, to help confirm whether it is a noise error. git bisect start 6dbe51c251a327e012439c4772097a13df43c5b8 v3.8 -- git bisect good d414c104e26fd3b597f855cc29473a8b1527fb4c # 02:41 35+ 35 Merge tag 'please-pull-vm_unwrapped' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux git bisect bad ee89f81252179dcbf6cd65bd48299f5e52292d88 # 02:46 10- 14 Merge branch 'for-3.9/core' of git://git.kernel.dk/linux-block git bisect good a6590b9f01dca2d45a38b8387f59812c44f68a2f # 03:05 40+ 40 Merge tag 'upstream-3.9-rc1' of git://git.infradead.org/linux-ubifs git bisect good 09884964335e85e897876d17783c2ad33cf8a2e0 # 03:13 40+ 40 mm: do not grow the stack vma just because of an overrun on preceding vma git bisect good ad60a9333035f2323840b71ab8ad07bbab728faf # 03:29 40+ 17 Merge branch 'misc' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild git bisect good e3c4877de8b9d93bd47b6ee88eb594b1c1e10da5 # 04:16 40+ 15 Merge branch 'x86-efi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip git bisect bad 2a7d2b96d5cba7568139d9ab157a0e97ab32440f # 04:29 1- 3 Merge branch 'akpm' (final batch from Andrew) git bisect good 6fa780095f058f0960f220754217701a53a9e403 # 05:44 40+ 19 IB/cxgb3: convert to idr_alloc() git bisect good 050a6b47d98e2bcea909c1129111e721668aaa2c # 06:06 40+ 36 idr: make idr_layer larger git bisect bad 459a86d83dce6641f66311d37fbfe1419c477a5a # 06:28 19- 21 mtd: mtd_nandecctest: use prandom_bytes instead of get_random_bytes() git bisect bad 06004e6eebdaf101a0f9b533fac77b5768d2c09c # 06:40 5- 6 block/partitions/mac.c: obey the state->limit constraint git bisect bad d941aeae42942c2ef61ed5e05cc1ad0428085704 # 06:52 0- 1 ipmi: add new kernel options to prevent automatic ipmi init git bisect bad 0ffc2a9c8072969253a20821c2c733a2cbb4c7c7 # 07:00 9- 10 idr: implement lookup hint git bisect good 54616283c2948812a44240858ced610e7cacbde1 # 07:06 115+ 73 idr: add idr_layer->prefix # first bad commit: [0ffc2a9c8072969253a20821c2c733a2cbb4c7c7] idr: implement lookup hint git bisect good 54616283c2948812a44240858ced610e7cacbde1 # 07:15 345+ 320 idr: add idr_layer->prefix git bisect good d67a0e110187abd560a1de63fa172894a52839d5 # 07:30 345+ 11 morphologies: Enable CONFIG_FHANDLE git bisect good 0e04c641b199435f3779454055f6a7de258ecdfc # 07:45 345+ 22 Merge tag 'dm-3.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm git bisect good b1cce620fd4b6864c92e7307be7839789f9c8be0 # 08:03 345+ 18 Add linux-next specific files for 20140612 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-i386.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/blob/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

2
1
0 / 0

[random] BUG: spinlock trylock failure on UP on CPU#0, swapper/1

by Jet Chen

Hi Greg, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7 Author: Greg Price <price(a)MIT.EDU> AuthorDate: Thu Dec 5 19:32:19 2013 -0500 Commit: Theodore Ts'o <tytso(a)mit.edu> CommitDate: Wed Mar 19 22:18:51 2014 -0400 random: simplify accounting code With this we handle "reserved" in just one place. As a bonus the code becomes less nested, and the "wakeup_write" flag variable becomes unnecessary. The variable "flags" was already unused. This code behaves identically to the previous version except in two pathological cases that don't occur. If the argument "nbytes" is already less than "min", then we didn't previously enforce "min". If r->limit is false while "reserved" is nonzero, then we previously applied "reserved" in checking whether we had enough bits, even though we don't apply it to actually limit how many we take. The callers of account() never exercise either of these cases. Before the previous commit, it was possible for "nbytes" to be less than "min" if userspace chose a pathological configuration, but no longer. Cc: Jiri Kosina <jkosina(a)suse.cz> Cc: "H. Peter Anvin" <hpa(a)zytor.com> Signed-off-by: Greg Price <price(a)mit.edu> Signed-off-by: "Theodore Ts'o" <tytso(a)mit.edu> +-------------------------------------------+------------+------------+ | | 8c2aa3390e | 0fb7a01af5 | +-------------------------------------------+------------+------------+ | boot_successes | 66 | 2 | | boot_failures | 0 | 20 | | BUG:spinlock_trylock_failure_on_UP_on_CPU | 0 | 20 | | backtrace:prandom_reseed | 0 | 11 | | backtrace:kernel_init_freeable | 0 | 11 | +-------------------------------------------+------------+------------+ [ 374.075477] raid6test: test_disks(62, 63): faila= 62(P) failb= 63(Q) OK [ 374.192518] raid6test: [ 374.312137] raid6test: complete (2429 tests, 0 failures) [ 374.434548] BUG: spinlock trylock failure on UP on CPU#0, swapper/1 [ 374.444114] lock: lock.13830+0x0/0x50, .magic: dead4ead, .owner: swapper/1, .owner_cpu: 0 [ 374.444114] CPU: 0 PID: 1 Comm: swapper Not tainted 3.14.0-rc2-00009-g0fb7a01 #1 [ 374.444114] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 374.444114] 0000000000000000 ffff88001386fc28 ffffffff81ec8af8 ffff88001386fc48 [ 374.444114] ffffffff81ec26d9 0000000000000000 ffffffff827db7a0 ffff88001386fc68 [ 374.444114] ffffffff81ec2704 ffffffff827db7a0 ffffffff823b74f3 ffff88001386fc80 [ 374.444114] Call Trace: [ 374.444114] [<ffffffff81ec8af8>] dump_stack+0x19/0x1b [ 374.444114] [<ffffffff81ec26d9>] spin_dump+0x85/0x8a [ 374.444114] [<ffffffff81ec2704>] spin_bug+0x26/0x28 [ 374.444114] [<ffffffff810ae690>] do_raw_spin_trylock+0x37/0x3c [ 374.444114] [<ffffffff81ed707b>] _raw_spin_lock_irqsave+0x4b/0x7a [ 374.444114] [<ffffffff814f5f69>] ? __prandom_reseed+0x1a/0xad [ 374.444114] [<ffffffff814f5f69>] __prandom_reseed+0x1a/0xad [ 374.444114] [<ffffffff814f6183>] prandom_reseed_late+0xe/0x10 [ 374.444114] [<ffffffff8162f31c>] credit_entropy_bits+0x116/0x21e [ 374.444114] [<ffffffff8162f8d9>] _xfer_secondary_pool+0x86/0x8f [ 374.444114] [<ffffffff810abe66>] ? __lock_acquire.isra.20+0x4cf/0x5cd [ 374.444114] [<ffffffff8162f963>] xfer_secondary_pool+0x5d/0x5f [ 374.444114] [<ffffffff8162fa0a>] extract_entropy+0xa5/0x16b [ 374.444114] [<ffffffff816301cc>] get_random_bytes+0x1b/0x1d [ 374.444114] [<ffffffff814f5f8f>] __prandom_reseed+0x40/0xad [ 374.444114] [<ffffffff810e7b32>] ? put_page+0x20/0x22 [ 374.444114] [<ffffffff8295f922>] ? radix_tree_init+0xce/0xce [ 374.444114] [<ffffffff8295f92e>] prandom_reseed+0xc/0x49 [ 374.444114] [<ffffffff8295f922>] ? radix_tree_init+0xce/0xce [ 374.444114] [<ffffffff8293ae11>] do_one_initcall+0x7d/0x108 [ 374.444114] [<ffffffff81098f9c>] ? parse_args+0x180/0x246 [ 374.444114] [<ffffffff8293af92>] kernel_init_freeable+0xf6/0x187 [ 374.444114] [<ffffffff8293a70e>] ? do_early_param+0x8a/0x8a [ 374.444114] [<ffffffff81ebe658>] ? rest_init+0xbc/0xbc [ 374.444114] [<ffffffff81ebe661>] kernel_init+0x9/0xd0 [ 374.444114] [<ffffffff81ed850a>] ret_from_fork+0x7a/0xb0 [ 374.444114] [<ffffffff81ebe658>] ? rest_init+0xbc/0xbc Elapsed time: 380 qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel /kernel/x86_64-randconfig-hsxa1-06150953/0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7/vmlinuz-3.14.0-rc2-00009-g0fb7a01 -append 'hung_task_panic=1 earlyprintk=ttyS0,115200 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=10 softlockup_panic=1 nmi_watchdog=panic prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw link=/kbuild-tests/run-queue/kvm/x86_64-randconfig-hsxa1-06150953/linux-devel:devel-hourly-2014061509:0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7:bisect-linux3/.vmlinuz-0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7-20140615135429-17-ivb41 branch=linux-devel/devel-hourly-2014061509 BOOT_IMAGE=/kernel/x86_64-randconfig-hsxa1-06150953/0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7/vmlinuz-3.14.0-rc2-00009-g0fb7a01 drbd.minor_count=8' -initrd /kernel-tests/initrd/yocto-minimal-x86_64.cgz -m 320 -smp 2 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -pidfile /dev/shm/kboot/pid-yocto-ivb41-47 -serial file:/dev/shm/kboot/serial-yocto-ivb41-47 -daemonize -display none -monitor null git bisect start v3.15 v3.14 -- git bisect bad 674366e90e5a2430e3ddef40ce51512d681944e4 # 10:49 0- 3 Merge tag 'pci-v3.15-fixes-1' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci git bisect bad 9f800363bb0ea459e15bef0928a72c88d374e489 # 11:28 2- 14 Merge tag 'fixes-non-critical-3.15' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc git bisect good 1aabc5990d205cdb0789a1c26143c53601e9bb07 # 11:38 21+ 0 x86: Fix dumpstack_64 to keep state of "used" variable in loop git bisect good d0cb5f71c5cde8e1ef6d03983641366800ceabdb # 11:51 21+ 0 Merge tag 'vfio-v3.15-rc1' of git://github.com/awilliam/linux-vfio git bisect good d15fee814d8d30bbb4859ef8fef7a1f96327635b # 12:07 21+ 0 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse git bisect bad d1d9cfc330e2622f4b4e0b82d3b41a34619614d4 # 12:16 1- 8 Merge tag 'random_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/random git bisect good 24e7ea3bea94fe05eae5019f5f12bcdc98fc5157 # 12:27 21+ 0 Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 git bisect good d15e03104eb9a4f8e244ab6ed3ca5a107e46db13 # 12:40 21+ 0 Merge tag 'xfs-for-linus-3.15-rc1' of git://oss.sgi.com/xfs/xfs git bisect good cda540ace6a194850e23c79955cc2e46fd91c19a # 13:15 21+ 0 mm: get_user_pages(write,force) refuse to COW in shared areas git bisect good 8c2aa3390ebb59cba4495a56557b70ad0575eef5 # 13:34 21+ 0 random: tighten bound on random_read_wakeup_thresh git bisect bad 46884442fc5bb81a896f7245bd850fde9b435509 # 13:44 0- 2 random: use the architectural HWRNG for the SHA's IV in extract_buf() git bisect bad 7d1b08c40c4f02c119476b29eca9bbc8d98d2a83 # 13:52 2- 9 random: entropy_bytes is actually bits git bisect bad 0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7 # 14:01 0- 5 random: simplify accounting code # first bad commit: [0fb7a01af5b0cbe5bf365891fc4d186f2caa23f7] random: simplify accounting code git bisect good 8c2aa3390ebb59cba4495a56557b70ad0575eef5 # 14:11 66+ 0 random: tighten bound on random_read_wakeup_thresh git bisect bad 1d08ab37636d6a169c67e879cba37675cd82e6ac # 14:11 0- 11 0day head guard for 'devel-hourly-2014061509' git bisect bad abf04af74a9f27a65172a43b43ccabcbc2bbdc39 # 14:27 3- 2 Merge tag 'scsi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi git bisect bad f9801c532e045c1ab89801d0597353c5e2a55671 # 14:41 5- 8 Add linux-next specific files for 20140613 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/blob/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Jet

7 years, 5 months

2
1
0 / 0

[CPU hotplug, smp] WARNING: CPU: 1 PID: 0 at kernel/smp.c:209 generic_smp_call_function_single_interrupt()

by Jet Chen

Hi Srivatsa, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master commit ab7a42783d939cdbe729c18ab32dbf0d25746ea2 Author: Srivatsa S. Bhat <srivatsa.bhat(a)linux.vnet.ibm.com> AuthorDate: Thu May 22 10:44:06 2014 +1000 Commit: Stephen Rothwell <sfr(a)canb.auug.org.au> CommitDate: Thu May 22 10:44:06 2014 +1000 CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline During CPU offline, in the stop-machine loop, we use 2 separate stages to disable interrupts, to ensure that the CPU going offline doesn't get any new IPIs from the other CPUs after it has gone offline. However, an IPI sent much earlier might arrive late on the target CPU (possibly _after_ the CPU has gone offline) due to hardware latencies, and due to this, the smp-call-function callbacks queued on the outgoing CPU might not get noticed (and hence not executed) at all. This is somewhat theoretical, but in any case, it makes sense to explicitly loop through the call_single_queue and flush any pending callbacks before the CPU goes completely offline. So, flush the queued smp-call-function callbacks in the MULTI_STOP_DISABLE_IRQ_ACTIVE stage, after disabling interrupts on the active CPU. This can be trivially achieved by invoking the generic_smp_call_function_single_interrupt() function itself (and since the outgoing CPU is still online at this point, we won't trigger the "IPI to offline CPU" warning in this function; so we are safe to call it here). This way, we would have handled all the queued callbacks before going offline, and also, no new IPIs can be sent by the other CPUs to the outgoing CPU at that point, because they will all be executing the stop-machine code with interrupts disabled. Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat(a)linux.vnet.ibm.com> Suggested-by: Frederic Weisbecker <fweisbec(a)gmail.com> Reviewed-by: Tejun Heo <tj(a)kernel.org> Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Oleg Nesterov <oleg(a)redhat.com> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> +------------------------------------------------------------------------------+------------+------------+ | | c80d40e1f2 | ab7a42783d | +------------------------------------------------------------------------------+------------+------------+ | boot_successes | 1194 | 261 | | boot_failures | 6 | 39 | | BUG:kernel_test_crashed | 6 | | | WARNING:CPU:PID:at_kernel/smp.c:generic_smp_call_function_single_interrupt() | 0 | 39 | | backtrace:stop_machine_from_inactive_cpu | 0 | 39 | | backtrace:mtrr_ap_init | 0 | 39 | | general_protection_fault | 0 | 0 | | RIP:__lock_acquire | 0 | 0 | | BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/rwsem.c | 0 | 0 | | INFO:lockdep_is_turned_off | 0 | 0 | | backtrace:do_mount | 0 | 0 | | backtrace:SyS_mount | 0 | 0 | +------------------------------------------------------------------------------+------------+------------+ [ 62.119017] masked ExtINT on CPU#1 [ 62.119017] numa_add_cpu cpu 1 node 0: mask now 0-1 [ 62.261606] ------------[ cut here ]------------ [ 62.261606] WARNING: CPU: 1 PID: 0 at kernel/smp.c:209 generic_smp_call_function_single_interrupt+0xc5/0x155() [ 62.261606] IPI on offline CPU 1 [ 62.261606] Modules linked in: [ 62.261606] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.15.0-rc5-01473-gab7a427 #4510 [ 62.261606] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 62.261606] 0000000000000000 ffff88001e7e1d28 ffffffff81b01705 ffff88001e7e1d70 [ 62.261606] ffff88001e7e1d60 ffffffff810754b3 ffffffff810e282c 0000000000000000 [ 62.261606] 0000000000000000 ffffffff82568b01 0000000000000082 ffff88001e7e1dc0 [ 62.261606] Call Trace: [ 62.261606] [<ffffffff81b01705>] dump_stack+0x4d/0x66 [ 62.261606] [<ffffffff810754b3>] warn_slowpath_common+0x7f/0x98 [ 62.261606] [<ffffffff810e282c>] ? generic_smp_call_function_single_interrupt+0xc5/0x155 [ 62.261606] [<ffffffff81075518>] warn_slowpath_fmt+0x4c/0x4e [ 62.261606] [<ffffffff810e282c>] generic_smp_call_function_single_interrupt+0xc5/0x155 [ 62.261606] [<ffffffff810f4792>] multi_cpu_stop+0xcf/0x136 [ 62.261606] [<ffffffff810f5003>] stop_machine_from_inactive_cpu+0xf7/0x132 [ 62.261606] [<ffffffff81056cfe>] ? mtrr_restore+0x45/0x45 [ 62.261606] [<ffffffff810573ba>] mtrr_ap_init+0x5a/0x5c [ 62.261606] [<ffffffff8104689e>] identify_secondary_cpu+0x19/0x1b [ 62.261606] [<ffffffff8105beda>] smp_store_cpu_info+0x3d/0x3f [ 62.261606] [<ffffffff8105c6cc>] start_secondary+0x16e/0x254 [ 62.261606] ---[ end trace 1752731e5a47c773 ]--- [ 62.312240] KVM setup async PF for cpu 1 git bisect start 27a4e439fe5cd92b70137ae237c7aa6888c07b5a 56b54984d49285d515dc6421c93648ccdc0a06bc -- git bisect bad 473496a03d9dbf8654f2fd789383d71df4dc7fde # 15:58 88- 3 Merge branch 'akpm-current/current' git bisect good b41b3b6c0ff37f89caf1dd5ec8f203f00d5c9080 # 16:23 300+ 0 mm: page_alloc: convert hot/cold parameter and immediate callers to bool git bisect good a0a7e7f4b2fbf31729ea5cd5d3d5cb7ee5c0b494 # 16:44 300+ 1 rtc: rtc-palmas: make of_device_id array const git bisect bad ab7a42783d939cdbe729c18ab32dbf0d25746ea2 # 16:49 45- 3 CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline git bisect good faa7c21459067de9686cdd62cfb47685bc5c60fc # 16:55 300+ 1 fs/hfsplus/wrapper.c: replace min/casting by min_t git bisect good 6a2ecc87dcbae7fa62ce1f156c4ac4a52d96b3fd # 17:01 300+ 1 signals: kill rm_from_queue(), change prepare_signal() to use for_each_thread() git bisect good ebe499d62a3a08e94fb1067f3486e00eca49edca # 17:09 300+ 1 signals: introduce kernel_sigaction() git bisect good ddb7fec09b32ec61e428e7e882b6821a10d3df3f # 17:43 300+ 1 smp-print-more-useful-debug-info-upon-receiving-ipi-on-an-offline-cpu-v5 git bisect good e840b4a6701321414618aa1fa623925368d66141 # 17:47 300+ 2 cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu-v3 git bisect good c80d40e1f29243cf7d932b8e35126bcae0e875b4 # 18:01 300+ 2 cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu-v5 # first bad commit: [ab7a42783d939cdbe729c18ab32dbf0d25746ea2] CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline git bisect good c80d40e1f29243cf7d932b8e35126bcae0e875b4 # 18:10 900+ 6 cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu-v5 git bisect bad 27a4e439fe5cd92b70137ae237c7aa6888c07b5a # 18:10 0- 7 Add linux-next specific files for 20140610 git bisect good 64b2d1fbbfda07765dae3f601862796a61b2c451 # 18:19 900+ 99 Merge tag 'for-f2fs-3.16' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs git bisect bad 27a4e439fe5cd92b70137ae237c7aa6888c07b5a # 18:19 0- 7 Add linux-next specific files for 20140610 Thanks, Jet

7 years, 5 months

2
1
0 / 0

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP June 2014