11:59 a.m.
Hi Peter,
FYI, here is another bisected warning to this commit.
Attached a reproduce script for your convenience.
https://github.com/0day-ci/linux
peter-enderborg-sonymobile-com/android-Collect-statistics-from-lowmemorykiller/20170215-004327
commit c3263ba15e872666094c18a1d882597bfe1ce1c1
Author: Peter Enderborg <peter.enderborg(a)sonymobile.com>
AuthorDate: Tue Feb 14 17:09:32 2017 +0100
Commit: 0day robot <fengguang.wu(a)intel.com>
CommitDate: Wed Feb 15 00:43:32 2017 +0800
mm: Remove RCU and tasklocks from lmk
Fundamental changes:
1 Does NOT take any RCU lock in shrinker functions.
2 It returns same result for scan and counts, so we dont need to do
shinker will know when it is pointless to call scan.
3 It does not lock any other process than the one that is
going to be killed.
Background.
The low memory killer scans for process that can be killed to free
memory. This can be cpu consuming when there is a high demand for
memory. This can be seen by analysing the kswapd0 task work.
The stats function added in earler patch adds a counter for waste work.
How it works.
This patch create a structure within the lowmemory killer that caches
the user spaces processes that it might kill. It is done with a
sorted rbtree so we can very easy find the candidate to be killed,
and knows its properies as memory usage and sorted by oom_score_adj
to look up the task with highest oom_score_adj. To be able to achive
this it uses oom_score_notify events.
This patch also as a other effect, we are now free to do other
lowmemorykiller configurations. Without the patch there is a need
for a tradeoff between freed memory and task and rcu locks. This
is no longer a concern for tuning lmk. This patch is not intended
to do any calculation changes other than we do use the cache for
calculate the count values and that makes kswapd0 to shrink other
areas.
Signed-off-by: Peter Enderborg <peter.enderborg(a)sonymobile.com>
+-------------------------------------------------+------------+------------+------------------+
| | 0174c40bf1 | c3263ba15e |
v4.10-rc8_021805 |
+-------------------------------------------------+------------+------------+------------------+
| boot_successes | 261 | 0 | 70
|
| boot_failures | 3 | 71 | 25
|
| BUG:kernel_hang_in_test_stage | 3 | 1 | 4
|
| inconsistent{IN-SOFTIRQ-W}->{SOFTIRQ-ON-W}usage | 0 | 6 | 10
|
| calltrace:lmk_oom_score_notifier | 0 | 70 |
|
| calltrace:smpboot_thread_fn | 0 | 3 |
|
| calltrace:__set_oom_adj | 0 | 2 |
|
| inconsistent{SOFTIRQ-ON-W}->{IN-SOFTIRQ-W}usage | 0 | 64 | 10
|
| calltrace:SyS_write | 0 | 63 |
|
| calltrace:rcu_process_callbacks | 0 | 64 | 10
|
| calltrace:SyS_munmap | 0 | 1 |
|
| calltrace:irq_exit | 0 | 1 | 4
|
| calltrace:SyS_mmap_pgoff | 0 | 1 |
|
| calltrace:copy_process | 0 | 2 | 3
|
| calltrace:n_tty_write | 0 | 1 |
|
| BUG:kernel_in_stage | 0 | 0 | 1
|
| calltrace:vprintk_emit | 0 | 0 | 2
|
| calltrace:univ8250_release_irq | 0 | 0 | 1
|
+-------------------------------------------------+------------+------------+------------------+
[ 9.180183]
[ 9.180610] =================================
[ 9.181713] [ INFO: inconsistent lock state ]
[ 9.182811] 4.10.0-rc7-00639-gc3263ba #2 Not tainted
[ 9.184067] ---------------------------------
[ 9.185260] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 9.186780] ksoftirqd/0/6 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 9.188173] (lmk_task_lock){+.?...}, at: [<ffffffff81441d54>]
lmk_oom_score_notifier+0x64/0x150
[ 9.192310] {SOFTIRQ-ON-W} state was registered at:
[ 9.193508]
[ 9.193515] [<ffffffff81095509>] __lock_acquire+0x5d9/0x1760
[ 9.197638]
[ 9.197644] [<ffffffff81096acf>] lock_acquire+0x8f/0xe0
[ 9.200487]
[ 9.200494] [<ffffffff8151084c>] _raw_spin_lock+0x3c/0x80
[ 9.203921]
[ 9.203930] [<ffffffff81441bf0>] lmk_task_watch+0x40/0x140
[ 9.207799]
[ 9.207807] [<ffffffff81441d12>] lmk_oom_score_notifier+0x22/0x150
[ 9.211497]
[ 9.211504] [<ffffffff8107dec4>] notifier_call_chain+0x34/0xa0
[ 9.216504]
[ 9.216508] [<ffffffff8107df97>] atomic_notifier_call_chain+0x67/0x100
[ 9.218637]
[ 9.218644] [<ffffffff810e3014>] oom_score_notify_update+0x24/0x40
[ 9.224726]
[ 9.224734] [<ffffffff811a7c3d>] __set_oom_adj+0x15d/0x4a0
[ 9.226560]
[ 9.226563] [<ffffffff811a803d>] oom_score_adj_write+0xbd/0xd0
[ 9.228466]
[ 9.228471] [<ffffffff81149d33>] __vfs_write+0x23/0x120
[ 9.231499]
[ 9.231503] [<ffffffff8114b8a4>] vfs_write+0xc4/0x1c0
[ 9.237215]
[ 9.237221] [<ffffffff8114ba84>] SyS_write+0x44/0xa0
[ 9.238895]
[ 9.238900] [<ffffffff815116fd>] entry_SYSCALL_64_fastpath+0x1f/0xc1
[ 9.241805] irq event stamp: 41246
[ 9.244701] hardirqs last enabled at (41246): [<ffffffff81112a4b>]
free_hot_cold_page+0x1cb/0x2e0
[ 9.247554] hardirqs last disabled at (41245): [<ffffffff811129ae>]
free_hot_cold_page+0x12e/0x2e0
[ 9.251752] softirqs last enabled at (41088): [<ffffffff81060c6a>]
__do_softirq+0x20a/0x270
[ 9.256594] softirqs last disabled at (41093): [<ffffffff81060cef>]
run_ksoftirqd+0x1f/0x70
[ 9.258397]
[ 9.258397] other info that might help us debug this:
[ 9.261476] Possible unsafe locking scenario:
[ 9.261476]
[ 9.264234] CPU0
[ 9.267364] ----
[ 9.267992] lock(lmk_task_lock);
[ 9.268899] <Interrupt>
[ 9.269565] lock(lmk_task_lock);
[ 9.272459]
[ 9.272459] *** DEADLOCK ***
[ 9.272459]
[ 9.273949] 2 locks held by ksoftirqd/0/6:
[ 9.276583] #0: (rcu_callback){......}, at: [<ffffffff810abfe9>]
rcu_process_callbacks+0x269/0x620
[ 9.280664] #1: (rcu_read_lock){......}, at: [<ffffffff8107df30>]
atomic_notifier_call_chain+0x0/0x100
[ 9.283754]
[ 9.283754] stack backtrace:
[ 9.287175] CPU: 0 PID: 6 Comm: ksoftirqd/0 Not tainted 4.10.0-rc7-00639-gc3263ba #2
[ 9.289626] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.9.3-20161025_171302-gandalf 04/01/2014
[ 9.294629] Call Trace:
[ 9.296338] dump_stack+0x19/0x26
[ 9.297172] print_usage_bug+0x1df/0x1f0
[ 9.300938] mark_lock+0x5be/0x650
[ 9.301788] ? print_shortest_lock_dependencies+0x1a0/0x1a0
[ 9.303269] __lock_acquire+0x517/0x1760
[ 9.307267] ? kvm_clock_read+0x22/0x40
[ 9.308326] ? kvm_sched_clock_read+0x9/0x20
[ 9.309521] ? kvm_clock_read+0x22/0x40
[ 9.312740] ? kvm_sched_clock_read+0x9/0x20
[ 9.313959] lock_acquire+0x8f/0xe0
[ 9.315280] ? lmk_oom_score_notifier+0x64/0x150
[ 9.318345] _raw_spin_lock+0x3c/0x80
[ 9.319269] ? lmk_oom_score_notifier+0x64/0x150
[ 9.322357] lmk_oom_score_notifier+0x64/0x150
[ 9.323474] notifier_call_chain+0x34/0xa0
[ 9.327487] ? will_become_orphaned_pgrp+0x90/0x90
[ 9.328755] atomic_notifier_call_chain+0x67/0x100
[ 9.330050] ? notifier_call_chain+0xa0/0xa0
[ 9.333014] oom_score_notify_free+0x21/0x40
[ 9.334183] ? kmem_cache_free+0x18/0x40
[ 9.335978] __put_task_struct+0x57/0x170
[ 9.341088] delayed_put_task_struct+0x29/0x30
[ 9.342256] rcu_process_callbacks+0x2d0/0x620
[ 9.343371] ? rcu_process_callbacks+0x269/0x620
[ 9.344565] __do_softirq+0x12a/0x270
[ 9.346733] ? smpboot_thread_fn+0x23/0x250
[ 9.348353] run_ksoftirqd+0x1f/0x70
[ 9.351651] smpboot_thread_fn+0x16a/0x250
[ 9.352724] kthread+0x102/0x140
[ 9.353541] ? sort_range+0x20/0x20
[ 9.356468] ? kthread_create_on_node+0x40/0x40
[ 9.357650] ret_from_fork+0x31/0x40
[ 9.419370] random: trinity: uninitialized urandom read (4 bytes read)
mountall: Event failed
[ 9.462308] random: mountall: uninitialized urandom read (12 bytes read)
[ 9.519336] init: Failed to create pty - disabling logging for job
[ 9.529057] init: Temporary process spawn error: No such file or directory
git bisect start 804340e99508771d2c3709a1605ca0d7aa7ef761
7089db84e356562f8ba737c29e472cc42d530dbc --
git bisect bad 0d2799238644a776f5bfcdb060e361ed56a36cd6 # 23:06 0- 16 Merge
'renesas/devel' into devel-hourly-2017021805
git bisect good 157f37653a4382f7eeddb3ea6d460af714fb50a8 # 23:18 65+ 1 Merge
'linux-review/Geert-Uytterhoeven/ARM-PM-Drop-useless-checks-for-PM_SUSPEND_STANDBY/20170218-025420'
into devel-hourly-2017021805
git bisect good 1d4f149c684d839094ccadab501860440200d98a # 23:46 63+ 0 Merge
'linux-review/Olav-Haugan/staging-speakup-coding-style-Simplify-comparisons-to-NULL/20170217-131253'
into devel-hourly-2017021805
git bisect bad cf14a82cebeec14fb05b5fa844f10e22964d658a # 23:58 0- 2 Merge
'linux-review/thor-thayer-linux-intel-com/mfd-altr-a10sr-Add-Arria10-SR-sysfs-attributes/20170215-022501'
into devel-hourly-2017021805
git bisect bad 644e51d644f6c446930b4a3f6dbc664dd27776c6 # 00:11 0- 6 Merge
'linux-review/Colin-King/usb-misc-usbtest-remove-redundant-check-on-retval-0/20170213-024141'
into devel-hourly-2017021805
git bisect good 54a136f5b3b9a36eb5c717644d3e97fd69a030b8 # 00:25 64+ 1 Merge
'linux-review/Bjorn-Andersson/arm64-dts-qcom-pm8994-Add-rtc-node/20170218-020240'
into devel-hourly-2017021805
git bisect good e79a5f53ec94ec08e7b7997150ef85d4b516c41b # 00:45 63+ 0 Merge
'linux-review/Paolo-Bonzini/KVM-change-gfn-hva-cache-to-use-per-VCPU-memslots/20170216-065425'
into devel-hourly-2017021805
git bisect good 9382124d4146803e057e7dbe8d92a73eabb0cf14 # 01:09 63+ 0 Merge
'vfio/next' into devel-hourly-2017021805
git bisect bad 108f1e75acb82a913bec494f26eec68d497098a3 # 01:38 0- 5 Merge
'linux-review/Alban/MIPS-Allow-compressed-images-to-be-loaded-at-any-address/20170214-053759'
into devel-hourly-2017021805
git bisect bad 11bc53d63ef26e59ce1c7c8ca88a4b7f959b878d # 01:54 0- 1 Merge
'linux-review/peter-enderborg-sonymobile-com/android-Collect-statistics-from-lowmemorykiller/20170215-004327'
into devel-hourly-2017021805
git bisect good 0174c40bf153def3ac7b287f000a885b15048a38 # 02:16 61+ 0 oom: Add
notification for oom_score_adj
git bisect bad c3263ba15e872666094c18a1d882597bfe1ce1c1 # 02:31 0- 15 mm:
Remove RCU and tasklocks from lmk
# first bad commit: [c3263ba15e872666094c18a1d882597bfe1ce1c1] mm: Remove RCU and
tasklocks from lmk
git bisect good 0174c40bf153def3ac7b287f000a885b15048a38 # 02:43 189+ 0 oom: Add
notification for oom_score_adj
# extra tests with CONFIG_DEBUG_INFO_REDUCED
git bisect bad c3263ba15e872666094c18a1d882597bfe1ce1c1 # 03:16 0- 7 mm:
Remove RCU and tasklocks from lmk
# extra tests on HEAD of linux-devel/devel-hourly-2017021805
git bisect bad 804340e99508771d2c3709a1605ca0d7aa7ef761 # 03:16 0- 23 0day
head guard for 'devel-hourly-2017021805'
# extra tests on tree/branch
linux-review/peter-enderborg-sonymobile-com/android-Collect-statistics-from-lowmemorykiller/20170215-004327
git bisect bad c3263ba15e872666094c18a1d882597bfe1ce1c1 # 03:23 0- 68 mm:
Remove RCU and tasklocks from lmk
# extra tests with first bad commit reverted
git bisect good 1f3fd50274732605285e1de35c03b552107df194 # 16:21 187+ 1 Revert
"mm: Remove RCU and tasklocks from lmk"
# extra tests on tree/branch linus/master
git bisect good 00ea1ceebe0d9f2dc1cc2b7bd575a00100c27869 # 05:14 183+ 4 ipv6:
release dst on error in ip6_dst_lookup_tail
# extra tests on tree/branch linux-next/master
git bisect good 4ce4a759a3e221b5265ebd03c2f8888b69a7cf3e # 17:24 183+ 17 Add
linux-next specific files for 20170217
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation