Messages in this thread | | | From | Doug Anderson <> | Date | Mon, 27 Nov 2023 13:55:16 -0800 | Subject | Re: [RFC] softlockup: serialized softlockup's log |
| |
Hi,
On Mon, Nov 27, 2023 at 9:13 AM Petr Mladek <pmladek@suse.com> wrote: > > > I've tried several times, but unfortunately I haven't been able to > > reproduce the problem you mentioned. My concern is that if the lock > > is shared, there will be potential deadlock issues because hardlockup > > exploits nmi. > > Yes, one has to be very careful when using locks under NMI. > I am aware of three approaches: > > 1. It should be safe to use a lock which is used only in NMI. > The only problem might be ABBA deadlocks. > > But this is not usable to serialize softlockup and hardlockup. > > > 2. Using try_lock() in NMI and skipping the report when the lock > could not be taken. > > But we probably should not skip the detailed info printed by the > hardlockup detector just because another softlockup or > hardlockup report is running in parallel. > > > 3. Using a lock which might be nested on the same CPU. > > It does not prevent mixed output completely. The nested > context will print the messages in the middle of the outer > one on the same CPU. But it would serialize output against > other CPUs. > > There already is one used to serialize some backtraces, > see printk_cpu_sync_get_irqsave() in dump_stack_lvl() and > nmi_cpu_backtrace(). > > It should be safe when the locked code does not wait for > other CPUs. And when it is a leaf lock so that there > is no need to think about ABBA deadlocks against > other locks. > > IMHO, printk_cpu_sync_get_irqsave() probably should be > safe to serialize the softlockup and hardlockup > detectors. > > > Adding John into Cc. He is familiar with > printk_cpu_sync_get_irqsave(). He wanted to use it also > for synchronizing console drivers. Well, it will be done > another way in the end.
Thanks for all the detailed info! Using printk_cpu_sync_get_irqsave() looks pretty promising. I'll try to do some testing with this and then aim to send a patch later this week that builds upon Li Zhe's patch. If everything works out I'll probably remove the newly added spinlock and use printk_cpu_sync_get_irqsave() instead, but the spinlock should work fine in the short term in any case.
-Doug
| |