Messages in this thread | | | From | Thomas Gleixner <> | Subject | RE: IRQ affinity problem from virtio_blk | Date | Wed, 16 Nov 2022 14:27:11 +0100 |
| |
On Wed, Nov 16 2022 at 11:24, Angus Chen wrote: >> >> On Wed, Nov 16, 2022 at 12:24:24AM +0100, Thomas Gleixner wrote: >> > Any other information I need to provide,pls tell me. >> >> A sensible use case for 180+ virtio block devices in a single guest. >> > Our card can provide more than 512 virtio_blk devices . > one virtio_blk passthrough to one container,like docker.
I'm not sure whether that's sensible, but that's how your hardware is designed. You could have provided this information upfront instead of random memory dumps of the irq matrix internals.
> So we need so much devices. > In the first patch ,I del the IRQD_AFFINITY_MANAGED in virtio_blk .
There is no IRQD_AFFINITY_MANAGED in virtio_blk. That flag is internal to the interrupt core code and you can neither delete it nor fiddle with it from inside virtio_blk.
You can do that in your private kernel, but that's not an option for mainline as it will break existing setups and it's fundamentally wrong.
The block-mq code has assumptions about the semantics of managed interrupts. It happens to work for the single queue case because that always ends up with queue affinity == cpu_possible_mask.
For anything else which assigns the queues to partitions of the CPU space it definitely expects the semantics of managed interrupts.
> As you know, if we just use small queues number ,like 1or 2,we Still > occupy 80 vector ,that is kind of waste,and it is easy to eahausted > the Irq resource.
We know that by now. No point in repeating this over and over. Aside of that it's not that easy because this is the first time within 5 years that someone ran into this problem.
The real question is how to solve this proper without creating problems for other scenarios. That needs involvment of the blk-mq people.
Thanks,
tglx
| |