Messages in this thread | | | Date | Tue, 03 May 2022 10:32:58 +0100 | From | Marc Zyngier <> | Subject | Re: irqdomain API: how to set affinity of parent irq of chained irqs? |
| |
On Mon, 02 May 2022 16:45:59 +0100, Marek Behún <kabel@kernel.org> wrote: > > On Mon, 02 May 2022 10:31:11 +0100 > Marc Zyngier <maz@kernel.org> wrote: > > > On Mon, 02 May 2022 09:21:37 +0100, > > Marek Behún <kabel@kernel.org> wrote: > > > > > > Dear Marc, Thomas, > > > > > > we have encountered the following problem that can hopefully be put > > > some light onto: What is the intended way to set affinity (and possibly > > > other irq attributes) of parent IRQ of chained IRQs, when using the > > > irqdomain API? > > > > Simples: you can't. What sense does it make to change the affinity of > > the parent interrupt, given that its fate is tied to *all* of the > > other interrupts that are muxed to it? > > Dear Marc, > > thank you for your answer. Still: > > What about when we want to set the same affinity for all the chained > interrupts? > > Example: on Armada 385 there are 4 PCIe controllers. Each controller > has one interrupt from which we trigger chained interrupts. We would > like to configure each controller to trigger interrupt (and thus all > chained interrupts in the domain) on different CPU core. > > Moreover we would really like to do this in runtime, through sysfs, > depending on for example whether there are cards plugged in the PCIe > ports. > > Maybe there should be some mechanism to allow to change affinity for > whole irqdomain, or something?
Should? Maybe. But not for an irqdomain (which really doesn't have anything to do with interrupt affinity).
What you may want is a new sysfs interface that would allow a parent interrupt affinity being changed, but also exposing to userspace all the interrupts this affects *at the same time*. something like:
/sys/kernel/irq/42/smp_affinity_list /sys/kernel/irq/42/muxed_irqs/ /sys/kernel/irq/42/muxed_irqs/56 -> ../../56 /sys/kernel/irq/42/muxed_irqs/57 -> ../../57
The main issues are that:
- we don't really track the muxing information in any of the data structures, so you can't just walk a short list and generate this information. You'd need to build the topology information at allocation time (or fish it out at runtime, but that's likely a pain).
- sysfs doesn't deal with affinities at all. procfs does, but adding more crap there is frowned upon.
- it *must* be a new interface. You can't repurpose the existing one, as something like irqbalance would be otherwise be massively confused by seeing interrupts moving around behind its back.
- conversely, you'll need to teach irqbalance how to deal with this new interface.
- this needs to be safe against CPU hotplug. It probably already is, but nobody ever tested it, given that userspace can't interact with these interrupts at the moment.
M.
-- Without deviation from the norm, progress is not possible.
| |