FYI, we noticed the following commit: https://github.com/jpoimboe/linux encode-rbp-on-entry commit d8c446013c8d16af4f9f0cf7409b23a3c98d7f94 ("fix stack unwinders") on test machine: 2 threads qemu-system-x86_64 -enable-kvm with 360M memory caused below changes: +-------------------------------------------------------------+------------+------------+ | | af692f5112 | d8c446013c | +-------------------------------------------------------------+------------+------------+ | boot_successes | 0 | 0 | | boot_failures | 42 | 42 | | RIP:note_page | 15 | | | Kernel_panic-not_syncing:softlockup:hung_tasks | 42 | | | backtrace:mark_rodata_ro | 39 | | | RIP:__asan_load8 | 19 | | | RIP:rb_insert_color | 1 | | | backtrace:insert | 1 | | | backtrace:rbtree_test_init | 3 | | | backtrace:kernel_init_freeable | 3 | | | RIP:__rb_erase_color | 1 | | | backtrace:erase_augmented | 2 | | | RIP:__asan_load4 | 2 | | | RIP:ptdump_walk_pgd_level_core | 4 | | | BUG:unable_to_handle_kernel | 0 | 42 | | Oops | 0 | 42 | | RIP:print_context_stack | 0 | 42 | | double_fault:#[##]DEBUG_PAGEALLOC_KASAN | 0 | 19 | | RIP:number | 0 | 19 | | RIP:show_stack_log_lvl | 0 | 19 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 19 | | BUG:KASAN:slab-out-of-bounds_in_vsnprintf_at_addr | 0 | 4 | | BUG:KASAN:slab-out-of-bounds_in_number_at_addr | 0 | 3 | | BUG:KASAN:stack-out-of-bounds_in__probe_kernel_read_at_addr | 0 | 1 | | backtrace:_do_fork | 0 | 3 | +-------------------------------------------------------------+------------+------------+ [ 2.714880] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 2.716548] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 2.717459] CPU: Intel QEMU Virtual CPU version 2.4.0 (family: 0x6, model: 0x6, stepping: 0x3) [ 2.810896] BUG: unable to handle kernel paging request at ffff880010988000 [ 2.811997] IP: [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 2.832807] PGD 58b7067 PUD 58b8067 PMD 14d7c067 PTE 8000000010988060 [ 2.833710] Oops: 0000 [#1] DEBUG_PAGEALLOC KASAN [ 2.834313] CPU: 0 PID: 2 Comm: swapper Not tainted 4.7.0-rc4-00347-gd8c4460 #1 [ 2.835268] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 2.836561] task: ffff880010978080 ti: ffff880010980000 task.ti: ffff880010980000 [ 2.837781] RIP: 0010:[<ffffffff81025f3b>] [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 2.839209] RSP: 0000:ffff880010987b40 EFLAGS: 00010006 [ 2.840049] RAX: 0000000000000000 RBX: ffff8800109c0000 RCX: ffffffff810e84ab [ 2.840973] RDX: dffffc0000000000 RSI: ffffffff810ea8c0 RDI: ffffffff8320c884 [ 2.841890] RBP: ffff880010987b88 R08: 0000000000000006 R09: 0000000000000000 [ 2.842798] R10: 0000000000000001 R11: 0000000000000000 R12: ffff880010980000 [ 2.862932] R13: ffff880010978080 R14: 0000000000000018 R15: ffff880010988000 [ 2.864038] FS: 0000000000000000(0000) GS:ffffffff82c5c000(0000) knlGS:0000000000000000 [ 2.865264] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2.866128] CR2: ffff880010988000 CR3: 0000000002c15000 CR4: 00000000000006b0 [ 2.867216] Stack: [ 2.883485] ffff880010978080 ffffffff83a64738 ffffffff8221a260 ffff880010987f48 [ 2.884706] ffffffff83a64738 ffffffff8221a260 ffffffff83a64738 ffff880010987bf8 [ 2.886056] ffff880010987bc8 ffff880010987bf8 ffffffff81025843 ffff880010987bc4 [ 2.887365] Call Trace: [ 2.887787] [<ffffffff81025843>] dump_trace+0x243/0x2b0 [ 2.888507] [<ffffffff81031755>] save_stack_trace+0x25/0x40 [ 2.889229] [<ffffffff81104536>] save_trace+0x76/0x130 [ 2.889905] [<ffffffff8110785a>] mark_lock+0x43a/0x810 [ 2.890581] [<ffffffff8110a2d9>] __lock_acquire+0x3d9/0x1f30 [ 2.891312] [<ffffffff8110ba19>] ? __lock_acquire+0x1b19/0x1f30 [ 2.895906] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 2.896958] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 2.897821] [<ffffffff81104d31>] ? check_chain_key+0x1a1/0x200 [ 2.898585] [<ffffffff8110c37a>] lock_acquire+0xba/0xf0 [ 2.899286] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 2.900044] [<ffffffff82077362>] _raw_spin_lock+0x32/0x50 [ 2.916743] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 2.917578] [<ffffffff81216183>] __set_task_comm+0x23/0x60 [ 2.918342] [<ffffffff810ea8ea>] kthreadd+0x2a/0x190 [ 2.919039] [<ffffffff8207804f>] ret_from_fork+0x1f/0x40 [ 2.919787] [<ffffffff810ea8c0>] ? kthread_stop+0x80/0x80 [ 2.925585] BUG: unable to handle kernel paging request at ffff880010988000 [ 2.926777] IP: [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 2.927817] PGD 58b7067 PUD 58b8067 PMD 14d7c067 PTE 8000000010988060 [ 2.935897] Oops: 0000 [#2] DEBUG_PAGEALLOC KASAN [ 2.940585] CPU: 0 PID: 2 Comm: swapper Not tainted 4.7.0-rc4-00347-gd8c4460 #1 [ 2.941592] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 2.942828] task: ffff880010978080 ti: ffff880010980000 task.ti: ffff880010980000 [ 2.943867] RIP: 0010:[<ffffffff81025f3b>] [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 2.953179] RSP: 0000:ffff8800109876d8 EFLAGS: 00010006 [ 2.953948] RAX: 0000000000000000 RBX: ffff8800109c0000 RCX: ffffffff810e84ab [ 2.954967] RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffffffff8320c884 [ 2.961105] RBP: ffff880010987720 R08: 0000000000000001 R09: 0000000000000000 [ 2.962107] R10: ffffffff83367f4b R11: 0000000000000003 R12: ffff880010980000 [ 2.963142] R13: ffff880010978080 R14: 0000000000000018 R15: ffff880010988000 [ 2.964087] FS: 0000000000000000(0000) GS:ffffffff82c5c000(0000) knlGS:0000000000000000 [ 2.973391] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2.974229] CR2: ffff880010988000 CR3: 0000000002c15000 CR4: 00000000000006b0 [ 2.984420] Stack: [ 2.984763] ffff880010978080 ffffffff829f2db5 ffffffff82213200 ffff880010987f48 [ 2.986014] ffffffff829f2db5 ffffffff82213200 ffffffff829f2db5 ffff880010987b88 [ 2.987056] ffff880010987b40 ffff880010987790 ffffffff81025843 ffff88001098775c [ 2.988133] Call Trace: [ 2.995465] [<ffffffff81025843>] dump_trace+0x243/0x2b0 [ 2.996233] [<ffffffff81026076>] show_trace_log_lvl+0x46/0x60 [ 2.996992] [<ffffffff8102594b>] show_stack_log_lvl+0x9b/0x140 [ 2.997755] [<ffffffff81025a5a>] show_regs+0x6a/0x170 [ 2.998422] [<ffffffff81026307>] __die+0x97/0xe0 [ 2.999027] [<ffffffff8104daf3>] no_context+0x4c3/0x540 [ 2.999720] [<ffffffff8104d630>] ? pgtable_bad+0xc0/0xc0 [ 3.008586] [<ffffffff8104dd83>] __bad_area_nosemaphore+0x213/0x240 [ 3.009493] [<ffffffff8104ddbf>] bad_area_nosemaphore+0xf/0x20 [ 3.010311] [<ffffffff8104e2b9>] __do_page_fault+0x1b9/0x4b0 [ 3.011111] [<ffffffff811082b8>] ? trace_hardirqs_off_caller+0x38/0x1a0 [ 3.012072] [<ffffffff8104e602>] do_page_fault+0x22/0x30 [ 3.020939] [<ffffffff81048f1d>] do_async_page_fault+0x2d/0x60 [ 3.021750] [<ffffffff82079075>] async_page_fault+0x25/0x30 [ 3.022528] [<ffffffff810e84ab>] ? __kernel_text_address+0x2b/0x70 [ 3.023326] [<ffffffff810ea8c0>] ? kthread_stop+0x80/0x80 [ 3.024095] [<ffffffff81025f3b>] ? print_context_stack+0x4b/0xd0 [ 3.033026] [<ffffffff81025843>] ? dump_trace+0x243/0x2b0 [ 3.033735] [<ffffffff81031755>] ? save_stack_trace+0x25/0x40 [ 3.034487] [<ffffffff81104536>] ? save_trace+0x76/0x130 [ 3.044373] [<ffffffff8110785a>] ? mark_lock+0x43a/0x810 [ 3.045213] [<ffffffff8110a2d9>] ? __lock_acquire+0x3d9/0x1f30 [ 3.046078] [<ffffffff8110ba19>] ? __lock_acquire+0x1b19/0x1f30 [ 3.046937] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 3.047853] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 3.052815] [<ffffffff81104d31>] ? check_chain_key+0x1a1/0x200 [ 3.053629] [<ffffffff8110c37a>] ? lock_acquire+0xba/0xf0 [ 3.054369] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.064231] [<ffffffff82077362>] ? _raw_spin_lock+0x32/0x50 [ 3.064964] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.065748] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.066495] [<ffffffff810ea8ea>] ? kthreadd+0x2a/0x190 [ 3.067164] [<ffffffff8207804f>] ? ret_from_fork+0x1f/0x40 [ 3.067895] [<ffffffff810ea8c0>] ? kthread_stop+0x80/0x80 [ 3.075779] BUG: unable to handle kernel paging request at ffff880010988000 [ 3.076762] IP: [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 3.077576] PGD 58b7067 PUD 58b8067 PMD 14d7c067 PTE 8000000010988060 [ 3.078479] Oops: 0000 [#3] DEBUG_PAGEALLOC KASAN [ 3.079081] CPU: 0 PID: 2 Comm: swapper Not tainted 4.7.0-rc4-00347-gd8c4460 #1 [ 3.080019] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 3.092423] task: ffff880010978080 ti: ffff880010980000 task.ti: ffff880010980000 [ 3.093646] RIP: 0010:[<ffffffff81025f3b>] [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 3.095075] RSP: 0000:ffff880010987268 EFLAGS: 00010006 [ 3.100867] RAX: 0000000000000000 RBX: ffff8800109c0000 RCX: ffffffff810e84ab [ 3.101773] RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffffffff8320c884 [ 3.102681] RBP: ffff8800109872b0 R08: 0000000000000001 R09: 0000000000000000 [ 3.103583] R10: ffffffff83367f4b R11: 0000000000000003 R12: ffff880010980000 [ 3.112628] R13: 7fff880010987a98 R14: 0000000000000018 R15: ffff880010988000 [ 3.113626] FS: 0000000000000000(0000) GS:ffffffff82c5c000(0000) knlGS:0000000000000000 [ 3.114654] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3.124544] CR2: ffff880010988000 CR3: 0000000002c15000 CR4: 00000000000006b0 [ 3.125607] Stack: [ 3.125908] 7fff880010987a98 ffffffff829f2db5 ffffffff82213200 ffff880010987a88 [ 3.127024] ffffffff829f2db5 ffffffff82213200 ffffffff829f2db5 ffff880010987720 [ 3.128156] ffff8800109876d8 ffff880010987320 ffffffff81025843 ffff8800109872ec [ 3.136364] Call Trace: [ 3.136697] [<ffffffff81025843>] dump_trace+0x243/0x2b0 [ 3.137389] [<ffffffff81026076>] show_trace_log_lvl+0x46/0x60 [ 3.138137] [<ffffffff8102594b>] show_stack_log_lvl+0x9b/0x140 [ 3.138901] [<ffffffff81025a5a>] show_regs+0x6a/0x170 [ 3.139567] [<ffffffff81026307>] __die+0x97/0xe0 [ 3.148309] [<ffffffff8104daf3>] no_context+0x4c3/0x540 [ 3.152942] [<ffffffff8110c3e6>] ? lock_release+0x36/0x4a0 [ 3.153793] [<ffffffff8110c3e6>] ? lock_release+0x36/0x4a0 [ 3.154613] [<ffffffff8104d630>] ? pgtable_bad+0xc0/0xc0 [ 3.156540] [<ffffffff81108296>] ? trace_hardirqs_off_caller+0x16/0x1a0 [ 3.157643] [<ffffffff8110f826>] ? do_raw_spin_lock+0x176/0x1a0 [ 3.158638] [<ffffffff81108296>] ? trace_hardirqs_off_caller+0x16/0x1a0 [ 3.159736] [<ffffffff8104dd83>] __bad_area_nosemaphore+0x213/0x240 [ 3.168701] [<ffffffff8111567a>] ? console_unlock+0x50a/0x5d0 [ 3.169457] [<ffffffff8104ddbf>] bad_area_nosemaphore+0xf/0x20 [ 3.170211] [<ffffffff8104e2b9>] __do_page_fault+0x1b9/0x4b0 [ 3.170954] [<ffffffff81108296>] ? trace_hardirqs_off_caller+0x16/0x1a0 [ 3.171824] [<ffffffff8104e602>] do_page_fault+0x22/0x30 [ 3.183866] [<ffffffff81048f1d>] do_async_page_fault+0x2d/0x60 [ 3.184664] [<ffffffff82079075>] async_page_fault+0x25/0x30 [ 3.185420] [<ffffffff810e84ab>] ? __kernel_text_address+0x2b/0x70 [ 3.186212] [<ffffffff81025f3b>] ? print_context_stack+0x4b/0xd0 [ 3.186996] [<ffffffff81025f46>] ? print_context_stack+0x56/0xd0 [ 3.187776] [<ffffffff81025843>] ? dump_trace+0x243/0x2b0 [ 3.192680] [<ffffffff81026076>] ? show_trace_log_lvl+0x46/0x60 [ 3.193662] [<ffffffff8102594b>] ? show_stack_log_lvl+0x9b/0x140 [ 3.194666] [<ffffffff81025a5a>] ? show_regs+0x6a/0x170 [ 3.197495] [<ffffffff81026307>] ? __die+0x97/0xe0 [ 3.198303] [<ffffffff8104daf3>] ? no_context+0x4c3/0x540 [ 3.199211] [<ffffffff8104d630>] ? pgtable_bad+0xc0/0xc0 [ 3.200073] [<ffffffff8104dd83>] ? __bad_area_nosemaphore+0x213/0x240 [ 3.204951] [<ffffffff8104ddbf>] ? bad_area_nosemaphore+0xf/0x20 [ 3.208889] [<ffffffff8104e2b9>] ? __do_page_fault+0x1b9/0x4b0 [ 3.209658] [<ffffffff811082b8>] ? trace_hardirqs_off_caller+0x38/0x1a0 [ 3.210523] [<ffffffff8104e602>] ? do_page_fault+0x22/0x30 [ 3.211234] [<ffffffff81048f1d>] ? do_async_page_fault+0x2d/0x60 [ 3.212137] [<ffffffff82079075>] ? async_page_fault+0x25/0x30 [ 3.215879] [<ffffffff810e84ab>] ? __kernel_text_address+0x2b/0x70 [ 3.216889] [<ffffffff810ea8c0>] ? kthread_stop+0x80/0x80 [ 3.217691] [<ffffffff81025f3b>] ? print_context_stack+0x4b/0xd0 [ 3.218588] [<ffffffff81025843>] ? dump_trace+0x243/0x2b0 [ 3.219395] [<ffffffff81031755>] ? save_stack_trace+0x25/0x40 [ 3.224363] [<ffffffff81104536>] ? save_trace+0x76/0x130 [ 3.228223] [<ffffffff8110785a>] ? mark_lock+0x43a/0x810 [ 3.228995] [<ffffffff8110a2d9>] ? __lock_acquire+0x3d9/0x1f30 [ 3.229810] [<ffffffff8110ba19>] ? __lock_acquire+0x1b19/0x1f30 [ 3.230587] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 3.231401] [<ffffffff81109f00>] ? debug_show_all_locks+0x1b0/0x1b0 [ 3.235238] [<ffffffff81104d31>] ? check_chain_key+0x1a1/0x200 [ 3.236095] [<ffffffff8110c37a>] ? lock_acquire+0xba/0xf0 [ 3.236948] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.237780] [<ffffffff82077362>] ? _raw_spin_lock+0x32/0x50 [ 3.238578] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.239402] [<ffffffff81216183>] ? __set_task_comm+0x23/0x60 [ 3.244312] [<ffffffff810ea8ea>] ? kthreadd+0x2a/0x190 [ 3.245078] [<ffffffff8207804f>] ? ret_from_fork+0x1f/0x40 [ 3.245930] [<ffffffff810ea8c0>] ? kthread_stop+0x80/0x80 [ 3.246707] BUG: unable to handle kernel paging request at ffff880010988000 [ 3.247728] IP: [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 3.252420] PGD 58b7067 PUD 58b8067 PMD 14d7c067 PTE 8000000010988060 [ 3.253534] Oops: 0000 [#4] DEBUG_PAGEALLOC KASAN [ 3.254311] CPU: 0 PID: 2 Comm: swapper Not tainted 4.7.0-rc4-00347-gd8c4460 #1 [ 3.260604] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 3.261845] task: ffff880010978080 ti: ffff880010980000 task.ti: ffff880010980000 [ 3.262910] RIP: 0010:[<ffffffff81025f3b>] [<ffffffff81025f3b>] print_context_stack+0x4b/0xd0 [ 3.264116] RSP: 0000:ffff880010986df8 EFLAGS: 00010006 [ 3.269061] RAX: 0000000000000000 RBX: ffff8800109c0000 RCX: ffffffff810e84ab [ 3.270086] RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffffffff8320c884 [ 3.271080] RBP: ffff880010986e40 R08: 0000000000000001 R09: 0000000000000000 [ 3.272150] R10: ffffffff83367f4b R11: 0000000000000003 R12: ffff880010980000 [ 3.276236] R13: 7fff880010987628 R14: 0000000000000018 R15: ffff880010988000 [ 3.277147] FS: 0000000000000000(0000) GS:ffffffff82c5c000(0000) knlGS:0000000000000000 [ 3.278170] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3.278911] CR2: ffff880010988000 CR3: 0000000002c15000 CR4: 00000000000006b0 [ 3.279820] Stack: [ 3.280132] 7fff880010987628 ffffffff829f2db5 ffffffff82213200 ffff880010987618 [ 3.288407] ffffffff829f2db5 ffffffff82213200 ffffffff829f2db5 ffff8800109872b0 [ 3.289427] ffff880010987268 ffff880010986eb0 ffffffff81025843 ffff880010986e7c [ 3.290447] Call Trace: [ 3.290778] [<ffffffff81025843>] dump_trace+0x243/0x2b0 [ 3.291467] [<ffffffff81026076>] show_trace_log_lvl+0x46/0x60 FYI, raw QEMU command line is: qemu-system-x86_64 -enable-kvm -kernel /pkg/linux/x86_64-randconfig-s4-06250514/gcc-6/d8c446013c8d16af4f9f0cf7409b23a3c98d7f94/vmlinuz-4.7.0-rc4-00347-gd8c4460 -append 'root=/dev/ram0 user=lkp job=/lkp/scheduled/vm-vp-quantal-x86_64-41/bisect_boot-1-quantal-core-x86_64.cgz-x86_64-randconfig-s4-06250514-d8c446013c8d16af4f9f0cf7409b23a3c98d7f94-20160625-37689-8g0ic3-0.yaml ARCH=x86_64 kconfig=x86_64-randconfig-s4-06250514 branch=linux-devel/devel-spot-201606250443 commit=d8c446013c8d16af4f9f0cf7409b23a3c98d7f94 BOOT_IMAGE=/pkg/linux/x86_64-randconfig-s4-06250514/gcc-6/d8c446013c8d16af4f9f0cf7409b23a3c98d7f94/vmlinuz-4.7.0-rc4-00347-gd8c4460 max_uptime=600 RESULT_ROOT=/result/boot/1/vm-vp-quantal-x86_64/quantal-core-x86_64.cgz/x86_64-randconfig-s4-06250514/gcc-6/d8c446013c8d16af4f9f0cf7409b23a3c98d7f94/0 LKP_SERVER=inn earlyprintk=ttyS0,115200 systemd.log_level=err debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal rw ip=::::vm-vp-quantal-x86_64-41::dhcp drbd.minor_count=8' -initrd /fs/sde1/initrd-vm-vp-quantal-x86_64-41 -m 360 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -pidfile /dev/shm/kboot/pid-vm-vp-quantal-x86_64-41 -serial file:/dev/shm/kboot/serial-vm-vp-quantal-x86_64-41 -daemonize -display none -monitor null Thanks, Xiaolong