Messages in this thread | | | Date | Fri, 1 Oct 2021 14:08:55 +0200 | From | Sebastian Andrzej Siewior <> | Subject | Re: [PATCH 4/5] irq_work: Handle some irq_work in SOFTIRQ on PREEMPT_RT |
| |
On 2021-10-01 12:32:38 [+0200], Peter Zijlstra wrote: > On Thu, Sep 30, 2021 at 06:38:58PM +0200, Sebastian Andrzej Siewior wrote: > > On 2021-09-30 16:39:51 [+0200], Peter Zijlstra wrote: > > > > Runing them all at the same prio still sucks (much like the single > > > net-RX thing), but at least a kthread is somewhat controllable. > > > > I could replace the softirq processing with a per-CPU thread. This > > should work. But I would have to (still) delay the wake-up of the thread > > to the timer tick - or - we try the wake from the irqwork-self-IPI. > > That, just wake the thread from the hardirq.
"just". Let me do that and see how bad it gets ;)
> > I > > just don't know how many will arrive back-to-back. The RCU callback > > (rcu_preempt_deferred_qs_handler()) pops up a lot. By my naive guesswork > > I would say that the irqwork is not needed since preempt-enable > > somewhere should do needed scheduling. But then commit > > 0864f057b050b ("rcu: Use irq_work to get scheduler's attention in clean context") > > > > claims it is not enough. > > Oh gawd, that was something really nasty. I'm not sure that Changelog > captures all (at least I'm not sure I fully understand the problem again > reading it). > > But basically that thing wants to reschedule, but suffers the same > problem as: > > preempt_disable(); > > <TIF_NEED_RESCHED gets set> > > local_irq_disable(); > preempt_enable(); > // cannea schedule because IRQs are disabled > local_irq_enable(); > // lost a reschedule > > > Yes, that will _eventually_ reschedule, but violates the PREEMPT rules > because there is an unspecified amount of time until it does actually do > reschedule.
Yeah but buh. We could let local_irq_enable/restore() check that need-resched bit if the above is considered pretty and supported _or_ start to yell if it is not. A middle way would be to trigger that self-IPI in such a case. I mean everyone suffers from that lost reschedule and, if I'm not mistaken, you don't receive a remote wakeup because the remote CPU notices need-resched bit and assumes that it is about to be handled. So RCU isn't special here.
> So what RCU does there is basically trigger a self-IPI, which guarantees > that we reschedule after IRQs are finally enabled, which then triggers a > resched. > > I see no problem marking that particular irq_work as HARD tho, it really > doesn't do anything (other than tell RCU the GP is no longer blocked) > and triggering the return-from-interrupt path.
Hmm. Your Highness. I'm going back to my peasant village to build the thread you asked for. I will look into this. I see two of those irq-work things that is the scheduler thingy and this.
Thanks.
> There's also a fun comment in perf_lock_task_context() that possibly > predates the above RCU fix.
Sebastian
| |