From da958a4d24db4dfddd875298efb23733d05272d5 Mon Sep 17 00:00:00 2001 From: Dmitry Vyukov Date: Wed, 12 May 2021 10:27:53 +0200 Subject: pkg/report: don't consider handle_mm_fault as anchor frame We are getting some duplicate reports where the stall actually happens in some syscall, but the syscall triggers a page fault repeatedly. As the result we can attribute some of the stalls to the next frame after handle_mm_fault. So don't consider handle_mm_fault as an anchor frame. We need to be careful to not skip entry into page fault from user space, because that's a completly different case. In that case the stall is indeed in the page fault handler. It's also unclear if this is the right thing to do. If the stall actually happens in the fault handler and the root cause in the fault handler, then after this change we will produce duplicate stalls for every place in the kernel that can trigger a page fault (we will be skipping the handler itself and attribute it to the innocent caller). But so far we don't have such examples. --- pkg/report/testdata/linux/report/610 | 73 +++++++++++++++ pkg/report/testdata/linux/report/611 | 80 +++++++++++++++++ pkg/report/testdata/linux/report/612 | 167 +++++++++++++++++++++++++++++++++++ 3 files changed, 320 insertions(+) create mode 100644 pkg/report/testdata/linux/report/610 create mode 100644 pkg/report/testdata/linux/report/611 create mode 100644 pkg/report/testdata/linux/report/612 (limited to 'pkg/report/testdata/linux') diff --git a/pkg/report/testdata/linux/report/610 b/pkg/report/testdata/linux/report/610 new file mode 100644 index 000000000..575db2508 --- /dev/null +++ b/pkg/report/testdata/linux/report/610 @@ -0,0 +1,73 @@ +TITLE: INFO: rcu detected stall in sys_recvmmsg +ALT: stall in sys_recvmmsg +TYPE: HANG + +[ 444.457807][ C0] rcu: INFO: rcu_sched self-detected stall on CPU +[ 444.464365][ C0] rcu: 0-...!: (10500 ticks this GP) idle=5be/1/0x4000000000000000 softirq=5432/5432 fqs=4 +[ 444.474605][ C0] (t=10502 jiffies g=2601 q=18) +[ 444.479528][ C0] rcu: rcu_sched kthread starved for 10494 jiffies! g2601 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1 +[ 444.490606][ C0] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior. +[ 444.500377][ C0] rcu: RCU grace-period kthread stack dump: +[ 444.506244][ C0] task:rcu_sched state:I stack: 0 pid: 11 ppid: 2 flags:0x00004000 +[ 444.515423][ C0] Call Trace: +[ 444.518702][ C0] __schedule+0x988/0x26f0 +[ 444.533738][ C0] schedule+0xd7/0x280 +[ 444.537793][ C0] schedule_timeout+0x15e/0x260 +[ 444.558706][ C0] rcu_gp_kthread+0x8e8/0x1790 +[ 444.602481][ C0] kthread+0x39a/0x490 +[ 444.617067][ C0] ret_from_fork+0x1f/0x30 +[ 444.621500][ C0] NMI backtrace for cpu 0 +[ 444.625817][ C0] CPU: 0 PID: 5028 Comm: syz-executor030 Not tainted 5.10.0-syzkaller #0 +[ 444.634201][ C0] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 +[ 444.644231][ C0] Call Trace: +[ 444.647498][ C0] +[ 444.650334][ C0] dump_stack+0x111/0x171 +[ 444.660263][ C0] nmi_cpu_backtrace.cold+0x44/0xd8 +[ 444.670633][ C0] nmi_trigger_cpumask_backtrace+0x1e7/0x210 +[ 444.676594][ C0] arch_trigger_cpumask_backtrace+0x14/0x20 +[ 444.682469][ C0] rcu_dump_cpu_stacks+0x1e8/0x226 +[ 444.687586][ C0] rcu_sched_clock_irq.cold+0x48f/0x95b +[ 444.693122][ C0] update_process_times+0x141/0x1c0 +[ 444.698303][ C0] tick_sched_handle+0xa2/0x190 +[ 444.703137][ C0] tick_sched_timer+0x1d0/0x2a0 +[ 444.713164][ C0] __hrtimer_run_queues+0x204/0xec0 +[ 444.723718][ C0] hrtimer_interrupt+0x30d/0x900 +[ 444.728649][ C0] __sysvec_apic_timer_interrupt+0xfa/0x430 +[ 444.734525][ C0] asm_call_irq_on_stack+0xf/0x20 +[ 444.739525][ C0] +[ 444.742447][ C0] sysvec_apic_timer_interrupt+0x98/0xb0 +[ 444.748065][ C0] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 444.754030][ C0] RIP: 0010:count_memcg_event_mm.part.0+0x12e/0x2a0 +[ 444.760602][ C0] Code: e8 97 05 cf ff 4d 85 f6 0f 85 84 00 00 00 e8 49 0d cf ff ba 01 00 00 00 44 89 ee 4c 89 e7 e8 49 d3 13 00 e8 34 0d cf ff 53 9d 2d 0d cf ff e8 e8 f3 d6 02 31 ff 89 c3 89 c6 e8 3d 05 cf ff 85 +[ 444.780185][ C0] RSP: 0018:ffff88810f8e7700 EFLAGS: 00000293 +[ 444.786236][ C0] RAX: 0000000000000000 RBX: 0000000000000293 RCX: 1ffffffff0ea9136 +[ 444.794189][ C0] RDX: ffff888114eac180 RSI: ffffffff819c792c RDI: ffffffff819c79b5 +[ 444.802141][ C0] RBP: ffff88810f8e7720 R08: 0000000000000001 R09: ffffffff8753dd5f +[ 444.810093][ C0] R10: fffffbfff0ea7bab R11: 0000000000000000 R12: ffff888100304000 +[ 444.818048][ C0] R13: 0000000000000011 R14: 0000000000000200 R15: ffff888112f40948 +[ 444.837948][ C0] handle_mm_fault+0x145/0x4140 +[ 444.863608][ C0] do_user_addr_fault+0x49f/0xa00 +[ 444.868621][ C0] exc_page_fault+0xa3/0x180 +[ 444.873199][ C0] asm_exc_page_fault+0x1e/0x30 +[ 444.878038][ C0] RIP: 0010:copy_user_generic_unrolled+0x86/0xc0 +[ 444.884349][ C0] Code: 4c 8b 5e 38 4c 89 47 20 4c 89 4f 28 4c 89 57 30 4c 89 5f 38 48 8d 76 40 48 8d 7f 40 ff c9 75 b6 89 d1 83 e2 07 c1 e9 03 74 12 <4c> 8b 06 4c 89 07 48 8d 76 08 48 8d 7f 08 ff c9 75 ee 21 d2 74 10 +[ 444.903933][ C0] RSP: 0018:ffff88810f8e79d8 EFLAGS: 00050202 +[ 444.909983][ C0] RAX: 0000000000000001 RBX: 000000002000e780 RCX: 0000000000000007 +[ 444.917935][ C0] RDX: 0000000000000000 RSI: 000000002000e780 RDI: ffff88810f8e7a50 +[ 444.925886][ C0] RBP: ffff88810f8e7a08 R08: 0000000000000001 R09: ffff88810f8e7a87 +[ 444.933839][ C0] R10: ffffed1021f1cf50 R11: 0000000000000000 R12: 0000000000000038 +[ 444.941794][ C0] R13: 000000002000e7b8 R14: ffff88810f8e7a50 R15: 0000000000000000 +[ 444.954604][ C0] __copy_msghdr_from_user+0xa7/0x4e0 +[ 444.975077][ C0] ___sys_recvmsg+0xf5/0x230 +[ 445.015581][ C0] do_recvmmsg+0x277/0x730 +[ 445.039426][ C0] __x64_sys_recvmmsg+0x22c/0x280 +[ 445.049712][ C0] do_syscall_64+0x32/0x50 +[ 445.054113][ C0] entry_SYSCALL_64_after_hwframe+0x44/0xa9 +[ 445.059983][ C0] RIP: 0033:0x449069 +[ 445.063877][ C0] Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 a1 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 +[ 445.083461][ C0] RSP: 002b:00007f18d0a67318 EFLAGS: 00000246 ORIG_RAX: 000000000000012b +[ 445.091858][ C0] RAX: ffffffffffffffda RBX: 00000000004ce4c8 RCX: 0000000000449069 +[ 445.099812][ C0] RDX: 0400000000000249 RSI: 0000000020008880 RDI: 0000000000000004 +[ 445.107766][ C0] RBP: 00000000004ce4c0 R08: 0000000000000000 R09: 0000000000000000 +[ 445.115717][ C0] R10: 0000000044000102 R11: 0000000000000246 R12: 000000000049d484 +[ 445.123671][ C0] R13: 00007fff8247bf9f R14: 00007f18d0a67400 R15: 0000000000022000 diff --git a/pkg/report/testdata/linux/report/611 b/pkg/report/testdata/linux/report/611 new file mode 100644 index 000000000..63d87ed48 --- /dev/null +++ b/pkg/report/testdata/linux/report/611 @@ -0,0 +1,80 @@ +TITLE: INFO: rcu detected stall in sys_sendmmsg +ALT: stall in sys_sendmmsg +TYPE: HANG + +[ 180.633803][ C1] rcu: INFO: rcu_sched self-detected stall on CPU +[ 180.640352][ C1] rcu: 1-...!: (10499 ticks this GP) idle=72a/1/0x4000000000000000 softirq=10448/10448 fqs=3 +[ 180.650842][ C1] (t=10500 jiffies g=9717 q=30) +[ 180.655770][ C1] rcu: rcu_sched kthread starved for 10494 jiffies! g9717 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 +[ 180.666678][ C1] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior. +[ 180.676457][ C1] rcu: RCU grace-period kthread stack dump: +[ 180.682329][ C1] task:rcu_sched state:R running task on cpu 0 stack: 0 pid: 11 ppid: 2 flags:0x00004000 +[ 180.693859][ C1] Call Trace: +[ 180.697138][ C1] __schedule+0x988/0x26f0 +[ 180.712182][ C1] schedule+0xd7/0x280 +[ 180.716236][ C1] schedule_timeout+0x15e/0x260 +[ 180.737153][ C1] rcu_gp_kthread+0x8e8/0x1790 +[ 180.780919][ C1] kthread+0x39a/0x490 +[ 180.795618][ C1] ret_from_fork+0x1f/0x30 +[ 180.800050][ C1] NMI backtrace for cpu 1 +[ 180.804366][ C1] CPU: 1 PID: 7502 Comm: syz-executor.5 Not tainted 5.10.0-syzkaller #0 +[ 180.812670][ C1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 +[ 180.822789][ C1] Call Trace: +[ 180.826055][ C1] +[ 180.828894][ C1] dump_stack+0x111/0x171 +[ 180.838824][ C1] nmi_cpu_backtrace.cold+0x44/0xd8 +[ 180.849193][ C1] nmi_trigger_cpumask_backtrace+0x1e7/0x210 +[ 180.855160][ C1] arch_trigger_cpumask_backtrace+0x14/0x20 +[ 180.861038][ C1] rcu_dump_cpu_stacks+0x1e8/0x226 +[ 180.866136][ C1] rcu_sched_clock_irq.cold+0x48f/0x95b +[ 180.871685][ C1] update_process_times+0x141/0x1c0 +[ 180.876870][ C1] tick_sched_handle+0xa2/0x190 +[ 180.881707][ C1] tick_sched_timer+0x1d0/0x2a0 +[ 180.891729][ C1] __hrtimer_run_queues+0x204/0xec0 +[ 180.902285][ C1] hrtimer_interrupt+0x30d/0x900 +[ 180.907219][ C1] __sysvec_apic_timer_interrupt+0xfa/0x430 +[ 180.913094][ C1] asm_call_irq_on_stack+0xf/0x20 +[ 180.918094][ C1] +[ 180.921035][ C1] sysvec_apic_timer_interrupt+0x98/0xb0 +[ 180.926656][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 180.932623][ C1] RIP: 0010:lock_is_held_type+0xb0/0xe0 +[ 180.938158][ C1] Code: ff 74 11 41 0f b6 47 22 45 31 c0 83 e0 03 39 d0 41 0f 94 c0 b8 ff ff ff ff 65 0f c1 05 59 93 8e 7b 83 f8 01 75 23 ff 75 d0 9d <48> 83 c4 10 44 89 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 45 31 c0 eb +[ 180.957769][ C1] RSP: 0018:ffff88811570f3d0 EFLAGS: 00000202 +[ 180.963821][ C1] RAX: 0000000000000001 RBX: ffffffff86187aa0 RCX: 1ffffffff0d765f3 +[ 180.971777][ C1] RDX: 0000000000000000 RSI: ffffffff86187aa0 RDI: ffff8881118f8cb0 +[ 180.979731][ C1] RBP: ffff88811570f408 R08: 0000000000000000 R09: ffffffff86bb000f +[ 180.987690][ C1] R10: fffffbfff0d76001 R11: 0000000000000000 R12: ffff8881118f8300 +[ 180.995646][ C1] R13: ffff8881118f8cb0 R14: 0000000000000001 R15: ffff8881118f8cb0 +[ 181.003632][ C1] rcu_read_lock_sched_held+0x41/0x80 +[ 181.009007][ C1] lock_acquire+0x5c1/0x770 +[ 181.032369][ C1] _raw_spin_lock+0x2b/0x40 +[ 181.042299][ C1] __migration_entry_wait+0x24/0x3f0 +[ 181.047574][ C1] migration_entry_wait+0x114/0x190 +[ 181.052763][ C1] do_swap_page+0x1616/0x23b0 +[ 181.062451][ C1] handle_mm_fault+0x1eef/0x4140 +[ 181.076886][ C1] do_user_addr_fault+0x49f/0xa00 +[ 181.081899][ C1] exc_page_fault+0xa3/0x180 +[ 181.086472][ C1] asm_exc_page_fault+0x1e/0x30 +[ 181.091307][ C1] RIP: 0010:copy_user_generic_unrolled+0x86/0xc0 +[ 181.097620][ C1] Code: 4c 8b 5e 38 4c 89 47 20 4c 89 4f 28 4c 89 57 30 4c 89 5f 38 48 8d 76 40 48 8d 7f 40 ff c9 75 b6 89 d1 83 e2 07 c1 e9 03 74 12 <4c> 8b 06 4c 89 07 48 8d 76 08 48 8d 7f 08 ff c9 75 ee 21 d2 74 10 +[ 181.117215][ C1] RSP: 0018:ffff88811570f8f0 EFLAGS: 00050202 +[ 181.123268][ C1] RAX: 0000000000000001 RBX: 0000000020003300 RCX: 0000000000000007 +[ 181.131223][ C1] RDX: 0000000000000000 RSI: 0000000020003300 RDI: ffff88811570f968 +[ 181.139175][ C1] RBP: ffff88811570f920 R08: 0000000000000001 R09: ffff88811570f99f +[ 181.147128][ C1] R10: ffffed1022ae1f33 R11: 0000000000000000 R12: 0000000000000038 +[ 181.155083][ C1] R13: 0000000020003338 R14: ffff88811570f968 R15: 0000000000000000 +[ 181.167899][ C1] __copy_msghdr_from_user+0xa7/0x4e0 +[ 181.178367][ C1] sendmsg_copy_msghdr+0xb1/0x170 +[ 181.197885][ C1] ___sys_sendmsg+0xe8/0x190 +[ 181.243058][ C1] __sys_sendmmsg+0x1bf/0x4d0 +[ 181.280044][ C1] __x64_sys_sendmmsg+0x9d/0x100 +[ 181.290858][ C1] do_syscall_64+0x32/0x50 +[ 181.295261][ C1] entry_SYSCALL_64_after_hwframe+0x44/0xa9 +[ 181.301135][ C1] RIP: 0033:0x4665f9 +[ 181.305026][ C1] Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48 +[ 181.324613][ C1] RSP: 002b:00007ff913845188 EFLAGS: 00000246 ORIG_RAX: 0000000000000133 +[ 181.333009][ C1] RAX: ffffffffffffffda RBX: 000000000056bf60 RCX: 00000000004665f9 +[ 181.340963][ C1] RDX: 00000000000001a3 RSI: 0000000020002cc0 RDI: 0000000000000004 +[ 181.348916][ C1] RBP: 00000000004c0881 R08: 0000000000000000 R09: 0000000000000000 +[ 181.356871][ C1] R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056bf60 +[ 181.364825][ C1] R13: 00007ffea137e1ff R14: 00007ff913845300 R15: 0000000000022000 diff --git a/pkg/report/testdata/linux/report/612 b/pkg/report/testdata/linux/report/612 new file mode 100644 index 000000000..d90f80d0b --- /dev/null +++ b/pkg/report/testdata/linux/report/612 @@ -0,0 +1,167 @@ +TITLE: INFO: rcu detected stall in ext4_file_read_iter +ALT: stall in ext4_file_read_iter +TYPE: HANG + +[ 483.600289][ C1] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: +[ 483.607580][ C1] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P9/1:b..l +[ 483.615937][ C1] (detected by 1, t=10502 jiffies, g=49045, q=284) +[ 483.622522][ C1] task:syz-executor.0 state:R running task stack:24880 pid:17688 ppid: 13379 flags:0x00004000 +[ 483.633869][ C1] Call Trace: +[ 483.637152][ C1] __schedule+0x911/0x21b0 +[ 483.651670][ C1] preempt_schedule_irq+0x4e/0x90 +[ 483.656701][ C1] irqentry_exit+0x7a/0xa0 +[ 483.661116][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 483.667102][ C1] RIP: 0010:lock_acquire+0x1ef/0x740 +[ 483.672397][ C1] Code: 50 a8 7e 83 f8 01 0f 85 ee 03 00 00 9c 58 f6 c4 02 0f 85 d9 03 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00 00 00 00 fc ff df <48> 01 c3 48 c7 03 00 00 00 00 48 c7 43 08 00 00 00 00 48 8b 84 24 +[ 483.692003][ C1] RSP: 0018:ffffc90002c0f098 EFLAGS: 00000206 +[ 483.698066][ C1] RAX: dffffc0000000000 RBX: 1ffff92000581e15 RCX: 000000005caa8f7c +[ 483.706035][ C1] RDX: 1ffff1100236013d RSI: 0000000000000000 RDI: 0000000000000000 +[ 483.714002][ C1] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff8fa988a7 +[ 483.721969][ C1] R10: fffffbfff1f53114 R11: 0000000000000000 R12: 0000000000000002 +[ 483.729935][ C1] R13: ffffffff8bf74320 R14: 0000000000000000 R15: 0000000000000000 +[ 483.753445][ C1] find_get_pages_range+0x11b/0x890 +[ 483.773673][ C1] pagevec_lookup_range+0x32/0x70 +[ 483.778702][ C1] clean_bdev_aliases+0x19d/0x830 +[ 483.827100][ C1] __block_write_begin_int+0xa48/0x1770 +[ 483.854080][ C1] block_page_mkwrite+0x21f/0x310 +[ 483.859113][ C1] ext4_page_mkwrite+0xbdf/0x1a10 +[ 483.875868][ C1] do_page_mkwrite+0x1a7/0x530 +[ 483.880638][ C1] __handle_mm_fault+0x2baa/0x4f70 +[ 483.890713][ C1] handle_mm_fault+0x1bc/0x7e0 +[ 483.895484][ C1] do_user_addr_fault+0x483/0x1210 +[ 483.900608][ C1] exc_page_fault+0x9e/0x180 +[ 483.905459][ C1] asm_exc_page_fault+0x1e/0x30 +[ 483.910309][ C1] RIP: 0010:copy_user_generic_string+0x2c/0x40 +[ 483.916467][ C1] Code: cb 83 fa 08 72 27 89 f9 83 e1 07 74 15 83 e9 08 f7 d9 29 ca 8a 06 88 07 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 c1 e9 03 83 e2 07 48 a5 89 d1 f3 a4 31 c0 0f 01 ca c3 0f 1f 80 00 00 00 00 0f 01 +[ 483.936073][ C1] RSP: 0018:ffffc90002c0fa00 EFLAGS: 00050246 +[ 483.942141][ C1] RAX: 0000000000000001 RBX: 0000000000001000 RCX: 0000000000000030 +[ 483.950110][ C1] RDX: 0000000000000000 RSI: ffff888032d9ae80 RDI: 000000002018f000 +[ 483.958085][ C1] RBP: 000000002018e180 R08: 0000000000000000 R09: ffff888032d9afff +[ 483.966673][ C1] R10: ffffed10065b35ff R11: 0000000000000000 R12: ffff888032d9a000 +[ 483.974642][ C1] R13: 000000002018f180 R14: 0000000000000000 R15: 0000000000001000 +[ 483.982623][ C1] copyout.part.0+0xe4/0x110 +[ 483.987217][ C1] copy_page_to_iter+0x41b/0xee0 +[ 483.992161][ C1] filemap_read+0x61e/0xe40 +[ 484.007972][ C1] generic_file_read_iter+0x397/0x4f0 +[ 484.013358][ C1] ext4_file_read_iter+0x1d4/0x5d0 +[ 484.018477][ C1] new_sync_read+0x41e/0x6e0 +[ 484.036965][ C1] vfs_read+0x35c/0x570 +[ 484.041125][ C1] ksys_read+0x12d/0x250 +[ 484.055694][ C1] do_syscall_64+0x2d/0x70 +[ 484.060117][ C1] entry_SYSCALL_64_after_hwframe+0x44/0xae +[ 484.066015][ C1] RIP: 0033:0x466459 +[ 484.069903][ C1] RSP: 002b:00007faf4cb38188 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 +[ 484.078316][ C1] RAX: ffffffffffffffda RBX: 000000000056c008 RCX: 0000000000466459 +[ 484.086287][ C1] RDX: 00000000fffffe47 RSI: 0000000020000180 RDI: 0000000000000006 +[ 484.094254][ C1] RBP: 00000000004bf9fb R08: 0000000000000000 R09: 0000000000000000 +[ 484.102222][ C1] R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056c008 +[ 484.110189][ C1] R13: 00007ffc561d590f R14: 00007faf4cb38300 R15: 0000000000022000 +[ 484.118175][ C1] task:kworker/u4:0 state:R running task stack:25072 pid: 9 ppid: 2 flags:0x00004000 +[ 484.128951][ C1] Workqueue: bat_events batadv_iv_send_outstanding_bat_ogm_packet +[ 484.136761][ C1] Call Trace: +[ 484.140036][ C1] __schedule+0x911/0x21b0 +[ 484.154524][ C1] preempt_schedule_irq+0x4e/0x90 +[ 484.159552][ C1] irqentry_exit+0x7a/0xa0 +[ 484.163967][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 484.169947][ C1] RIP: 0010:lock_acquire+0x1ef/0x740 +[ 484.175235][ C1] Code: 50 a8 7e 83 f8 01 0f 85 ee 03 00 00 9c 58 f6 c4 02 0f 85 d9 03 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00 00 00 00 fc ff df <48> 01 c3 48 c7 03 00 00 00 00 48 c7 43 08 00 00 00 00 48 8b 84 24 +[ 484.194842][ C1] RSP: 0018:ffffc90000ce7aa8 EFLAGS: 00000206 +[ 484.200908][ C1] RAX: dffffc0000000000 RBX: 1ffff9200019cf57 RCX: 00000000f9e69e84 +[ 484.208875][ C1] RDX: 1ffff1100222f13d RSI: 0000000000000000 RDI: 0000000000000000 +[ 484.216844][ C1] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff8fa988a7 +[ 484.224813][ C1] R10: fffffbfff1f53114 R11: 0000000000000000 R12: 0000000000000002 +[ 484.232780][ C1] R13: ffffffff8bf74320 R14: 0000000000000000 R15: 0000000000000000 +[ 484.261152][ C1] batadv_iv_ogm_schedule_buff+0x5d0/0x1410 +[ 484.290571][ C1] batadv_iv_send_outstanding_bat_ogm_packet+0x675/0x920 +[ 484.297605][ C1] process_one_work+0x98d/0x1600 +[ 484.317896][ C1] worker_thread+0x64c/0x1120 +[ 484.327786][ C1] kthread+0x3b1/0x4a0 +[ 484.336972][ C1] ret_from_fork+0x1f/0x30 +[ 484.341402][ C1] rcu: rcu_preempt kthread starved for 10500 jiffies! g49045 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1 +[ 484.352595][ C1] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. +[ 484.362554][ C1] rcu: RCU grace-period kthread stack dump: +[ 484.368430][ C1] task:rcu_preempt state:R running task stack:28872 pid: 14 ppid: 2 flags:0x00004000 +[ 484.379200][ C1] Call Trace: +[ 484.382474][ C1] __schedule+0x911/0x21b0 +[ 484.397567][ C1] schedule+0xcf/0x270 +[ 484.401643][ C1] schedule_timeout+0x14a/0x250 +[ 484.428092][ C1] rcu_gp_kthread+0xd07/0x2250 +[ 484.468669][ C1] kthread+0x3b1/0x4a0 +[ 484.477858][ C1] ret_from_fork+0x1f/0x30 +[ 484.482286][ C1] rcu: Stack dump where RCU GP kthread last ran: +[ 484.488597][ C1] NMI backtrace for cpu 1 +[ 484.492912][ C1] CPU: 1 PID: 17698 Comm: syz-executor.3 Not tainted 5.12.0-rc6-syzkaller #0 +[ 484.501670][ C1] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 +[ 484.511718][ C1] Call Trace: +[ 484.514989][ C1] +[ 484.517830][ C1] dump_stack+0x141/0x1d7 +[ 484.522164][ C1] nmi_cpu_backtrace.cold+0x44/0xd7 +[ 484.532569][ C1] nmi_trigger_cpumask_backtrace+0x1b3/0x230 +[ 484.538556][ C1] rcu_check_gp_kthread_starvation+0x1c7/0x1e4 +[ 484.544710][ C1] rcu_sched_clock_irq.cold+0x9a3/0x11dd +[ 484.560418][ C1] update_process_times+0x16d/0x200 +[ 484.565619][ C1] tick_sched_handle+0x9b/0x180 +[ 484.570475][ C1] tick_sched_timer+0x1b0/0x2d0 +[ 484.580876][ C1] __hrtimer_run_queues+0x1c0/0xe40 +[ 484.598148][ C1] hrtimer_interrupt+0x330/0xa00 +[ 484.603101][ C1] __sysvec_apic_timer_interrupt+0x146/0x540 +[ 484.609095][ C1] sysvec_apic_timer_interrupt+0x40/0xc0 +[ 484.614736][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 484.620721][ C1] RIP: 0010:unwind_next_frame+0x34f/0x1ce0 +[ 484.626527][ C1] Code: e8 96 f5 ff ff 49 89 c0 4d 85 c0 0f 84 39 02 00 00 4d 8d 78 04 48 b8 00 00 00 00 00 fc ff df 4c 89 fa 48 c1 ea 03 0f b6 04 02 <4c> 89 fa 83 e2 07 38 d0 7f 08 84 c0 0f 85 1e 09 00 00 41 0f b6 40 +[ 484.646136][ C1] RSP: 0000:ffffc90000dc07b8 EFLAGS: 00000216 +[ 484.652200][ C1] RAX: 0000000000000000 RBX: 1ffff920001b80ff RCX: ffffffff8902d429 +[ 484.660167][ C1] RDX: 1ffffffff1da0a8b RSI: ffffffff8ed05456 RDI: ffffffff8e33ff84 +[ 484.668140][ C1] RBP: 0000000000000001 R08: ffffffff8ed05456 R09: ffffffff8ed05456 +[ 484.676108][ C1] R10: fffff520001b811d R11: 0000000000084087 R12: ffffc90000dc08d8 +[ 484.684082][ C1] R13: ffffc90000dc08c5 R14: ffffc90000dc0890 R15: ffffffff8ed0545a +[ 484.721433][ C1] arch_stack_walk+0x7d/0xe0 +[ 484.735470][ C1] stack_trace_save+0x8c/0xc0 +[ 484.745965][ C1] kasan_save_stack+0x1b/0x40 +[ 484.911315][ C1] kasan_set_track+0x1c/0x30 +[ 484.915906][ C1] kasan_set_free_info+0x20/0x30 +[ 484.920847][ C1] __kasan_slab_free+0xf5/0x130 +[ 484.925701][ C1] slab_free_freelist_hook+0x92/0x210 +[ 484.931079][ C1] kmem_cache_free+0x8a/0x740 +[ 484.945401][ C1] kfree_skbmem+0xef/0x1b0 +[ 484.949820][ C1] consume_skb+0xcf/0x160 +[ 484.954154][ C1] mac80211_hwsim_tx_frame+0x157/0x1e0 +[ 484.959617][ C1] mac80211_hwsim_beacon_tx+0x4ba/0x910 +[ 484.965172][ C1] __iterate_interfaces+0x1e5/0x520 +[ 484.987891][ C1] ieee80211_iterate_active_interfaces_atomic+0x8d/0x170 +[ 484.994935][ C1] mac80211_hwsim_beacon+0xd5/0x1a0 +[ 485.000143][ C1] __hrtimer_run_queues+0x609/0xe40 +[ 485.017405][ C1] hrtimer_run_softirq+0x17b/0x360 +[ 485.022518][ C1] __do_softirq+0x29b/0x9f6 +[ 485.027032][ C1] irq_exit_rcu+0x134/0x200 +[ 485.031537][ C1] sysvec_apic_timer_interrupt+0x93/0xc0 +[ 485.037168][ C1] +[ 485.040100][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 485.046084][ C1] RIP: 0010:preempt_schedule_irq+0x49/0x90 +[ 485.051893][ C1] Code: 55 53 65 48 8b 1c 25 00 f0 01 00 48 89 dd 48 c1 ed 03 48 01 c5 bf 01 00 00 00 e8 72 e2 49 f8 e8 3d 8c 75 f8 fb bf 01 00 00 00 c2 d0 ff ff 9c 58 fa f6 c4 02 75 27 bf 01 00 00 00 e8 b0 d0 49 +[ 485.071496][ C1] RSP: 0000:ffffc90002c5fe68 EFLAGS: 00000206 +[ 485.077566][ C1] RAX: 00000000005a35db RBX: ffff88801f359c40 RCX: 1ffffffff1b89bc9 +[ 485.085534][ C1] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001 +[ 485.093500][ C1] RBP: ffffed1003e6b388 R08: 0000000000000001 R09: 0000000000000001 +[ 485.101471][ C1] R10: ffffffff8179e058 R11: 0000000000000001 R12: 0000000000000000 +[ 485.109440][ C1] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 +[ 485.127658][ C1] irqentry_exit+0x7a/0xa0 +[ 485.132075][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 485.138060][ C1] RIP: 0010:exit_to_user_mode_prepare+0x10e/0x250 +[ 485.144477][ C1] Code: f6 c4 02 0f 85 0f 01 00 00 65 48 8b 04 25 00 f0 01 00 48 8b 18 f7 c3 0e 30 02 00 0f 84 2f ff ff ff e8 e6 99 17 00 fb f6 c3 08 <74> bf e8 1b 00 a2 07 f6 c7 10 74 ba 48 89 ef e8 2e d6 32 00 f7 c3 +[ 485.164080][ C1] RSP: 0000:ffffc90002c5ff30 EFLAGS: 00000202 +[ 485.170146][ C1] RAX: 000000000023ebc1 RBX: 0000000000000008 RCX: 1ffffffff1b89bc9 +[ 485.178117][ C1] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 +[ 485.186082][ C1] RBP: ffffc90002c5ff58 R08: 0000000000000001 R09: 0000000000000001 +[ 485.194051][ C1] R10: ffffffff8179e058 R11: 0000000000000001 R12: ffff88801f359c40 +[ 485.202021][ C1] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 +[ 485.221095][ C1] irqentry_exit_to_user_mode+0x5/0x40 +[ 485.226556][ C1] asm_sysvec_apic_timer_interrupt+0x12/0x20 +[ 485.232539][ C1] RIP: 0033:0x466459 +[ 485.236431][ C1] Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48 +[ 485.256034][ C1] RSP: 002b:00007f74636b8188 EFLAGS: 00000246 +[ 485.262099][ C1] RAX: 0000000000000000 RBX: 000000000056bf60 RCX: 0000000000466459 +[ 485.270065][ C1] RDX: 0000000020000000 RSI: 0000000000005452 RDI: 0000000000000003 +[ 485.278033][ C1] RBP: 00000000004bf9fb R08: 0000000000000000 R09: 0000000000000000 +[ 485.285999][ C1] R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056bf60 +[ 485.293968][ C1] R13: 00007ffc06f8e1df R14: 00007f74636b8300 R15: 0000000000022000 -- cgit mrf-deployment