Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git x86/vmap_stack commit 56c8928ebabd0c51b34056128ba686e38c867fe1 Author: Andy Lutomirski <luto(a)kernel.org&gt; AuthorDate: Sat Jun 25 15:44:16 2016 -0700 Commit: Andy Lutomirski <luto(a)kernel.org&gt; CommitDate: Sun Jun 26 21:33:03 2016 -0700 sched: Free the stack early if CONFIG_THREAD_INFO_IN_TASK We currently keep every task's stack around until the task_struct itself is freed. This means that we keep the stack allocation alive for longer than necessary and that, under load, we free stacks in big batches whenever RCU drops the last task reference. Neither of these is good for reuse of cache-hot memory, and freeing in batches prevents us from usefully caching small numbers of vmalloced stacks. On architectures that have thread_info on the stack, we can't easily change this, but on architectures that set THREAD_INFO_IN_TASK, we can free it as soon as the task is dead. Cc: Oleg Nesterov <oleg(a)redhat.com&gt; Cc: Peter Zijlstra <peterz(a)infradead.org&gt; Signed-off-by: Andy Lutomirski <luto(a)kernel.org&gt; +----------------------------------------------+------------+------------+------------+ | | 09b2041c78 | 56c8928eba | d4fec1ba0e | +----------------------------------------------+------------+------------+------------+ | boot_successes | 77 | 25 | 0 | | boot_failures | 0 | 11 | 21 | | WARNING:at_kernel/fork.c:#free_task | 0 | 2 | 4 | | backtrace:_do_fork | 0 | 2 | 4 | | backtrace:SyS_clone | 0 | 2 | 4 | | INFO:trying_to_register_non-static_key | 0 | 9 | 1 | | INFO:rcu_sched_detected_stalls_on_CPUs/tasks | 0 | 1 | | | BUG:spinlock_lockup_suspected_on_CPU | 0 | 1 | | | BUG:unable_to_handle_kernel | 0 | 9 | 15 | | Oops | 0 | 9 | 15 | | RIP:spin_dump | 0 | 1 | | | Kernel_panic-not_syncing:Fatal_exception | 0 | 9 | 16 | | backtrace:do_mount | 0 | 9 | 14 | | backtrace:SyS_mount | 0 | 9 | 14 | | backtrace:schedule_timeout | 0 | 1 | | | RIP:__list_add | 0 | 8 | 1 | | general_protection_fault:#[##]SMP | 0 | 0 | 1 | | RIP:__lock_acquire | 0 | 0 | 15 | | backtrace:async_run_entry_fn | 0 | 0 | 2 | | BUG:kernel_boot_hang | 0 | 0 | 1 | +----------------------------------------------+------------+------------+------------+ [ 42.374829] UDF-fs: warning (device nbd4): udf_fill_super: No partition found (1) [ 42.506727] EXT4-fs (nbd0): unable to read superblock [ 42.507544] EXT4-fs (nbd0): unable to read superblock [ 42.525683] INFO: trying to register non-static key. [ 42.525684] the code is fine but needs lockdep annotation. [ 42.525684] turning off the locking correctness validator. [ 42.525687] CPU: 1 PID: 737 Comm: mount Not tainted 4.7.0-rc4-00255-g56c8928 #1 [ 42.525688] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [ 42.525693] 0000000000000000 ffff88002900fa98 ffffffff81776247 0000000000000000 [ 42.525695] 0000000000000001 ffff88002900fb00 ffffffff8111ebaa 0000000000000000 [ 42.525698] 0000000000000002 000000002900fad8 ffffffff8112179a ffff8800321d8fc0 [ 42.525699] Call Trace: [ 42.525717] [<ffffffff81776247>] dump_stack+0x82/0xb8 [ 42.525720] [<ffffffff8111ebaa>] register_lock_class+0x142/0x2f1 [ 42.525723] [<ffffffff8112179a>] ? trace_hardirqs_on_caller+0x17c/0x198 [ 42.525725] [<ffffffff81121bc7>] __lock_acquire+0x15a/0xe6f [ 42.525728] [<ffffffff82ecbaab>] ? __wait_for_common+0x115/0x148 [ 42.525730] [<ffffffff82ecb095>] ? __schedule+0x623/0x75d [ 42.525733] [<ffffffff81122ca5>] lock_acquire+0x13d/0x1d6 [ 42.525734] [<ffffffff81122ca5>] ? lock_acquire+0x13d/0x1d6 [ 42.525736] [<ffffffff82ecbaab>] ? __wait_for_common+0x115/0x148 [ 42.525739] [<ffffffff82eced97>] _raw_spin_lock_irq+0x3f/0x75 [ 42.525740] [<ffffffff82ecbaab>] ? __wait_for_common+0x115/0x148 [ 42.525742] [<ffffffff82ecbaab>] __wait_for_common+0x115/0x148 [ 42.525744] [<ffffffff82ece3be>] ? usleep_range+0x4b/0x4b [ 42.525748] [<ffffffff81108749>] ? wake_up_q+0x4c/0x4c [ 42.525750] [<ffffffff82ecbb02>] wait_for_completion+0x24/0x26 [ 42.525752] [<ffffffff810ff577>] kthread_stop+0x107/0x2a2 [ 42.525756] [<ffffffff810f972e>] destroy_workqueue+0x14d/0x201 [ 42.525759] [<ffffffff814fbba8>] xfs_destroy_mount_workqueues+0x3d/0x64 [ 42.525761] [<ffffffff814fc687>] xfs_fs_fill_super+0x2b1/0x4cb [ 42.525765] [<ffffffff8121ca45>] mount_bdev+0x144/0x197 [ 42.525767] [<ffffffff814fc3d6>] ? xfs_test_remount_options+0x5d/0x5d [ 42.525769] [<ffffffff814fb11a>] xfs_fs_mount+0x15/0x17 [ 42.525771] [<ffffffff8121d477>] mount_fs+0x67/0x131 [ 42.525774] [<ffffffff81235179>] vfs_kern_mount+0x6b/0xdb [ 42.525776] [<ffffffff81237c67>] do_mount+0x8da/0x9e0 [ 42.525779] [<ffffffff811d4036>] ? strndup_user+0x3f/0x53 [ 42.525781] [<ffffffff81237f8d>] SyS_mount+0x77/0x9f [ 42.525783] [<ffffffff82ecf741>] entry_SYSCALL_64_fastpath+0x1f/0xc2 [ 42.525787] [<ffffffff8111e4d3>] ? trace_hardirqs_off_caller+0x3f/0xab [ 42.537023] EXT2-fs (nbd0): error: unable to read superblock [ 42.537272] FAT-fs (nbd0): utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive! [ 42.537465] FAT-fs (nbd0): unable to read boot sector git bisect start d4fec1ba0e20fa072c6e7fa7e1656e8f73410185 4c2e07c6a29e0129e975727b9f57eede813eea85 -- git bisect bad d1e789da54eb522386c5ee5f78e87bc41480cd91 # 14:37 0- 1 Merge 'linux-review/Pedro-Ignacio-Nieto/Staging-comedi-das16-c-Fixed-a-comment-coding-style-issue/20160627-091528' into devel-catchup-201606271239 git bisect bad bce9f5e150724fb18957ff4b7da8cae5f76ce0cf # 14:46 0- 6 Merge 'linux-review/Subhransu-S-Prusty/ASoC-hdac-Add-hdac-generic-driver/20160627-120229' into devel-catchup-201606271239 git bisect bad b6e2b25c291c3e0a9ea9c63a951910af8e4b6ed2 # 14:58 0- 25 Merge 'luto/x86/vmap_stack' into devel-catchup-201606271239 git bisect good d271138d53adcac75ea1f11fa809567808f2c1a1 # 15:13 24+ 0 0day base guard for 'devel-catchup-201606271239' git bisect good efad3585eef8334bc26a90f8debc16bfa1ed4d82 # 15:13 44+ 0 x86/mm: Improve stack-overflow #PF handling git bisect good 90f2d0658c9288e388be613b0fdec4c387c2b53c # 15:13 32+ 0 x86/entry: Get rid of pt_regs_to_thread_info() git bisect bad 56c8928ebabd0c51b34056128ba686e38c867fe1 # 15:32 0- 1 sched: Free the stack early if CONFIG_THREAD_INFO_IN_TASK git bisect good 854e4b1a56d4fd7937470afd9d1424413a7ca4b0 # 15:44 25+ 0 sched: Allow putting thread_info into task_struct git bisect good 09b2041c784974696573457e4de8bd0298307261 # 16:05 25+ 0 x86: Move thread_info into task_struct # first bad commit: [56c8928ebabd0c51b34056128ba686e38c867fe1] sched: Free the stack early if CONFIG_THREAD_INFO_IN_TASK git bisect good 09b2041c784974696573457e4de8bd0298307261 # 16:08 73+ 0 x86: Move thread_info into task_struct # extra tests with CONFIG_DEBUG_INFO_REDUCED git bisect bad 56c8928ebabd0c51b34056128ba686e38c867fe1 # 16:27 0- 11 sched: Free the stack early if CONFIG_THREAD_INFO_IN_TASK # extra tests on HEAD of linux-devel/devel-catchup-201606271239 git bisect bad d4fec1ba0e20fa072c6e7fa7e1656e8f73410185 # 16:28 0- 21 0day head guard for 'devel-catchup-201606271239' # extra tests on tree/branch luto/x86/vmap_stack git bisect bad 53dae9db922e4446150ca9619ea33003f53bb90e # 16:29 0- 62 scatterlist: Support virtually-mapped stacks in sg_init_stackbuf() # extra tests with first bad commit reverted git bisect good 23d66f5806e7219559b874608032e53939715139 # 16:58 69+ 0 Revert "sched: Free the stack early if CONFIG_THREAD_INFO_IN_TASK" # extra tests on tree/branch linus/master git bisect good 4c2e07c6a29e0129e975727b9f57eede813eea85 # 16:59 137+ 0 Linux 4.7-rc5 # extra tests on tree/branch linux-next/master git bisect bad 2cf991dfda8b36d2878c249bcdf492366ec24c19 # 17:28 0- 28 Add linux-next specific files for 20160624 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/lkp Intel Corporation