Greeting, FYI, we noticed the following commit (built with clang-14): commit: 1ecd8ea291a89955841d6beede2679677433eef9 ("Revert "crypto: api - Fix built-in testing dependency failures"") https://github.com/krzk/linux wip/dt-bindings-max77802 in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------------------+------------+------------+ | | 875fd02ecd | 1ecd8ea291 | +--------------------------------------------------------------+------------+------------+ | BUG:workqueue_lockup-pool | 179 | 169 | | INFO:rcu_preempt_detected_stalls_on_CPUs/tasks | 0 | 41 | +--------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com&gt; [ 123.368914][ C0] BUG: workqueue lockup - pool cpus=0 flags=0x4 nice=0 stuck for 59s! [ 123.370957][ C0] Showing busy workqueues and worker pools: [ 123.372271][ C0] workqueue events: flags=0x0 [ 123.373291][ C0] pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 [ 123.374927][ C0] pending: stop_one_cpu_nowait_workfn [ 123.376231][ C0] workqueue events_unbound: flags=0x2 [ 123.377440][ C0] pwq 2: cpus=0 flags=0x4 nice=0 active=1/512 refcnt=3 [ 123.379065][ C0] in-flight: 7:async_run_entry_fn [ 123.380256][ C0] workqueue events_power_efficient: flags=0x82 [ 123.381496][ C0] pwq 2: cpus=0 flags=0x4 nice=0 active=3/256 refcnt=5 [ 123.382913][ C0] pending: do_cache_clean, neigh_periodic_work, check_lifetime [ 123.384712][ C0] pool 2: cpus=0 flags=0x4 nice=0 hung=59s workers=3 idle: 82 76 [ 144.562054][ T1] test_free_pages: Testing with GFP_KERNEL | __GFP_COMP [ 165.811939][ C0] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 165.811939][ C0] (detected by 0, t=31502 jiffies, g=21557, q=6) [ 165.811939][ C0] rcu: All QSes seen, last rcu_preempt kthread activity 31502 (4294926781-4294895279), jiffies_till_next_fqs=2, root ->qsmask 0x0 [ 165.811939][ C0] rcu: rcu_preempt kthread starved for 31502 jiffies! g21557 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [ 165.811939][ C0] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 165.811939][ C0] rcu: RCU grace-period kthread stack dump: [ 165.811939][ C0] task:rcu_preempt state:R running task stack:14928 pid: 14 ppid: 2 flags:0x00004000 [ 165.811939][ C0] Call Trace: [ 165.811939][ C0] <TASK> [ 165.811939][ C0] __schedule (kernel/sched/core.c:4943 kernel/sched/core.c:6221) [ 165.811939][ C0] schedule (kernel/sched/core.c:6301) [ 165.811939][ C0] schedule_timeout (kernel/time/timer.c:1882) [ 165.811939][ C0] ? update_process_times (kernel/time/timer.c:1805) [ 165.811939][ C0] rcu_gp_fqs_loop (kernel/rcu/tree.c:1959) [ 165.811939][ C0] ? rcu_gp_init (kernel/rcu/tree.c:1849) [ 165.811939][ C0] ? preempt_count_sub (kernel/sched/core.c:5432) [ 165.811939][ C0] rcu_gp_kthread (kernel/rcu/tree.c:2135) [ 165.811939][ C0] kthread (kernel/kthread.c:328) [ 165.811939][ C0] ? rcu_report_qs_rsp (kernel/rcu/tree.c:2107) [ 165.811939][ C0] ? kthread_unuse_mm (kernel/kthread.c:272) [ 165.811939][ C0] ret_from_fork (??:?) [ 165.811939][ C0] </TASK> [ 165.811939][ C0] rcu: Stack dump where RCU GP kthread last ran: [ 165.811939][ C0] NMI backtrace for cpu 0 [ 165.811939][ C0] CPU: 0 PID: 1 Comm: swapper Not tainted 5.15.0-rc3-03890-g1ecd8ea291a8 #1 [ 165.811939][ C0] Call Trace: [ 165.811939][ C0] <IRQ> [ 165.811939][ C0] dump_stack_lvl (lib/dump_stack.c:108) [ 165.811939][ C0] dump_stack (lib/dump_stack.c:114) [ 165.811939][ C0] nmi_trigger_cpumask_backtrace (include/linux/instrumented.h:87 include/asm-generic/bitops/instrumented-atomic.h:41 include/linux/cpumask.h:327 lib/nmi_backtrace.c:108 lib/nmi_backtrace.c:62) [ 165.811939][ C0] arch_trigger_cpumask_backtrace (arch/x86/kernel/apic/hw_nmi.c:41) [ 165.811939][ C0] rcu_check_gp_kthread_starvation (kernel/rcu/tree_stall.h:485) [ 165.811939][ C0] rcu_sched_clock_irq (kernel/rcu/tree_stall.h:114 kernel/rcu/tree_stall.h:588 kernel/rcu/tree_stall.h:729 kernel/rcu/tree.c:3882 kernel/rcu/tree.c:2601) [ 165.811939][ C0] ? _raw_spin_unlock_irqrestore (arch/x86/include/asm/preempt.h:103 include/linux/spinlock_api_smp.h:161 kernel/locking/spinlock.c:194) [ 165.811939][ C0] update_process_times (arch/x86/include/asm/preempt.h:27 kernel/time/timer.c:1787) [ 165.811939][ C0] tick_nohz_handler (kernel/time/tick-sched.c:227 kernel/time/tick-sched.c:1320) [ 165.811939][ C0] __sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1104) [ 165.811939][ C0] sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1097) [ 165.811939][ C0] </IRQ> [ 165.811939][ C0] <TASK> [ 165.811939][ C0] asm_sysvec_apic_timer_interrupt (??:?) [ 165.811939][ C0] RIP: 0010:kcsan_setup_watchpoint (kernel/kcsan/core.c:338 kernel/kcsan/core.c:602) [ 165.811939][ C0] Code: 00 48 81 c5 00 08 00 00 80 3d 36 ab 53 03 00 75 15 48 8b 44 24 20 25 00 02 00 00 48 85 c0 74 06 e8 10 87 fb ff fb 48 8b 43 30 <48> 89 45 30 48 8b 43 28 48 89 45 28 48 8b 43 20 48 89 45 20 48 8b All code ======== 0: 00 48 81 add %cl,-0x7f(%rax) 3: c5 00 08 (bad) 6: 00 00 add %al,(%rax) 8: 80 3d 36 ab 53 03 00 cmpb $0x0,0x353ab36(%rip) # 0x353ab45 f: 75 15 jne 0x26 11: 48 8b 44 24 20 mov 0x20(%rsp),%rax 16: 25 00 02 00 00 and $0x200,%eax 1b: 48 85 c0 test %rax,%rax 1e: 74 06 je 0x26 20: e8 10 87 fb ff callq 0xfffffffffffb8735 25: fb sti 26: 48 8b 43 30 mov 0x30(%rbx),%rax 2a:* 48 89 45 30 mov %rax,0x30(%rbp) <-- trapping instruction 2e: 48 8b 43 28 mov 0x28(%rbx),%rax 32: 48 89 45 28 mov %rax,0x28(%rbp) 36: 48 8b 43 20 mov 0x20(%rbx),%rax 3a: 48 89 45 20 mov %rax,0x20(%rbp) 3e: 48 rex.W 3f: 8b .byte 0x8b Code starting with the faulting instruction =========================================== 0: 48 89 45 30 mov %rax,0x30(%rbp) 4: 48 8b 43 28 mov 0x28(%rbx),%rax 8: 48 89 45 28 mov %rax,0x28(%rbp) c: 48 8b 43 20 mov 0x20(%rbx),%rax 10: 48 89 45 20 mov %rax,0x20(%rbp) 14: 48 rex.W 15: 8b .byte 0x8b [ 165.811939][ C0] RSP: 0000:ffff8881001a7870 EFLAGS: 00000282 [ 165.811939][ C0] RAX: 00c4addf00c4add4 RBX: ffff888100262178 RCX: 0000000000000000 [ 165.811939][ C0] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffffff835cffd8 [ 165.811939][ C0] RBP: ffff888100260800 R08: 00018881001a79bb R09: 0000000000000000 [ 165.811939][ C0] R10: 0000000000000010 R11: 0000000000000010 R12: ffff8881001a79b8 [ 165.811939][ C0] R13: aaaaaaaaaaaaaaaa R14: ffff888100262150 R15: 0000000000000000 [ 165.811939][ C0] ? stack_trace_consume_entry (kernel/stacktrace.c:93) [ 165.811939][ C0] stack_trace_consume_entry (kernel/stacktrace.c:93) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] ? stack_trace_save (kernel/stacktrace.c:82) [ 165.811939][ C0] arch_stack_walk (arch/x86/kernel/stacktrace.c:27) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] stack_trace_save (kernel/stacktrace.c:122) [ 165.811939][ C0] save_stack (mm/page_owner.c:120) [ 165.811939][ C0] ? free_pcp_prepare (include/linux/page_owner.h:? mm/page_alloc.c:1338 mm/page_alloc.c:1411) [ 165.811939][ C0] ? free_unref_page (mm/page_alloc.c:3315 mm/page_alloc.c:3394) [ 165.811939][ C0] ? __put_page (mm/swap.c:131) [ 165.811939][ C0] ? test_free_pages (include/linux/mm.h:? include/linux/mm.h:1287 lib/test_free_pages.c:25) [ 165.811939][ C0] ? m_in (lib/test_free_pages.c:35) [ 165.811939][ C0] ? do_one_initcall (init/main.c:1303) [ 165.811939][ C0] ? do_initcall_level (init/main.c:1375) [ 165.811939][ C0] ? do_initcalls (init/main.c:1389) [ 165.811939][ C0] ? do_basic_setup (init/main.c:1412) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] __reset_page_owner (mm/page_owner.c:140) [ 165.811939][ C0] free_pcp_prepare (include/linux/page_owner.h:? mm/page_alloc.c:1338 mm/page_alloc.c:1411) [ 165.811939][ C0] free_unref_page (mm/page_alloc.c:3315 mm/page_alloc.c:3394) [ 165.811939][ C0] __put_page (mm/swap.c:131) [ 165.811939][ C0] test_free_pages (include/linux/mm.h:? include/linux/mm.h:1287 lib/test_free_pages.c:25) [ 165.811939][ C0] ? __leaf_switch_none (lib/test_free_pages.c:31) [ 165.811939][ C0] m_in (lib/test_free_pages.c:35) [ 165.811939][ C0] do_one_initcall (init/main.c:1303) [ 165.811939][ C0] ? rcu_read_lock_sched_held (kernel/rcu/update.c:125) [ 165.811939][ C0] do_initcall_level (init/main.c:1375) [ 165.811939][ C0] do_initcalls (init/main.c:1389) [ 165.811939][ C0] do_basic_setup (init/main.c:1412) [ 165.811939][ C0] kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] ? rest_init (init/main.c:1497) [ 165.811939][ C0] kernel_init (init/main.c:1507) [ 165.811939][ C0] ? rest_init (init/main.c:1497) [ 165.811939][ C0] ret_from_fork (??:?) [ 165.811939][ C0] </TASK> [ 175.608197][ T1] test_free_pages: Test completed [ 175.609667][ T1] glob: 64 self-tests passed, 0 failed [ 175.612731][ T1] gpio_winbond: chip ID at 2e is ffff [ 175.613875][ T1] gpio_winbond: not an our chip [ 175.614961][ T1] gpio_winbond: chip ID at 4e is ffff [ 175.616198][ T1] gpio_winbond: not an our chip [ 175.619125][ T1] IPMI message handler: version 39.2 [ 175.620532][ T1] ipmi device interface [ 175.621713][ T1] ipmi_si: IPMI System Interface driver [ 175.623807][ T1] ipmi_si: Unable to find any System Interface(s) [ 175.625144][ T1] ipmi_ssif: IPMI SSIF Interface driver [ 175.627821][ T1] ERST DBG: ERST support is disabled. [ 175.632004][ T101] clocksource_wdtest: --- Verify jiffies-like uncertainty margin. [ 175.633769][ T101] clocksource: wdtest-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns [ 175.807028][ T1] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 175.809360][ T1] serial 00:05: GPIO lookup for consumer rs485-term [ 175.810908][ T1] serial 00:05: using ACPI for GPIO lookup [ 175.812252][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpios [ 175.813692][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpio [ 175.815081][ T1] serial 00:05: using lookup tables for GPIO lookup [ 175.816663][ T1] serial 00:05: No GPIO consumer rs485-term found [ 175.855621][ T1] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 175.859039][ T1] serial 00:06: GPIO lookup for consumer rs485-term [ 175.860433][ T1] serial 00:06: using ACPI for GPIO lookup [ 175.861647][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpios [ 175.863108][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpio [ 175.864457][ T1] serial 00:06: using lookup tables for GPIO lookup [ 175.865837][ T1] serial 00:06: No GPIO consumer rs485-term found [ 175.900036][ T1] 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 175.907614][ T1] Non-volatile memory driver v1.3 [ 175.912094][ T1] SPI driver at25 has no spi_device_id for cypress,fm25 [ 175.918473][ T1] vcan: Virtual CAN interface driver [ 175.919701][ T1] CAN device driver interface [ 175.920953][ T1] e1000: Intel(R) PRO/1000 Network Driver [ 175.922231][ T1] e1000: Copyright (c) 1999-2006 Intel Corporation. To reproduce: # build kernel cd linux cp config-5.15.0-rc3-03890-g1ecd8ea291a8 .config make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang