Messages in this thread | | | Date | Tue, 12 Jan 2021 16:51:06 +0100 | From | Peter Zijlstra <> | Subject | Re: [PATCH -tip V3 0/8] workqueue: break affinity initiatively |
| |
On Tue, Jan 12, 2021 at 07:57:26AM -0700, Jens Axboe wrote: > On 1/11/21 12:21 PM, Valentin Schneider wrote: > > On 11/01/21 18:16, Peter Zijlstra wrote: > >> Sadly it appears like io_uring() uses kthread_create_on_cpu() without > >> then having any hotplug crud on, so that needs additinoal frobbing. > >> > > > > I noticed that as well sometime ago, and I believed then (still do) this > > usage is broken. I don't think usage of kthread_create_on_cpu() outside > > of smpboot makes sense, because without any hotplug step to park the > > thread, its affinity can end up being reset after its dedicated CPU gets > > offlined. > > > > I'm clueless about io_uring, but if it *actually* has a good reason to > > use some pcpu kthreads (it seems it doesn't have to be on all CPUs?), > > then it needs to register some hotplug step to park them / do something > > sensible on hotplug. > > For io_uring, it's purely used by the SQPOLL mode, which sets aside a > kernel thread for submissions so the application doesn't have to do > anything but write new SQE entries to submit. The thread then notices > these, and submits them. There's an option to affinitize that thread > to a single CPU, which often makes sense for setups like that. Think > of it like a strong hint.
OK, that matches what I could make of it earlier today.
> Things aren't going to break if this CPU goes away and we end up being > affinitized to some other CPU, though it is suboptimal. So I guess we > might need some notifiers to ensure that we reset the CPU back again > if it's gone offline+online again? I can take a look at that.
Indeed, that would make sense.
Thanks!
| |