Messages in this thread | | | Subject | Re: [PATCH RFC 1/1] genirq: Make threaded handler use irq affinity for managed interrupt | From | John Garry <> | Date | Fri, 13 Dec 2019 12:08:54 +0000 |
| |
Hi Marc,
>> JFYI, we're still testing this and the patch itself seems to work as >> intended. >> >> Here's the kernel log if you just want to see how the interrupts are >> getting assigned: >> https://pastebin.com/hh3r810g > > It is a bit hard to make sense of this dump, specially on such a wide > machine (I want one!)
So do I :) That's the newer "D06CS" board.
without really knowing the topology of the system.
So it's 2x socket, each socket has 2x CPU dies, and each die has 6 clusters of 4 CPUs, which gives 96 in total.
> >> For me, I did get a performance boost for NVMe testing, but my >> colleague Xiang Chen saw a drop for our storage test of interest - >> that's the HiSi SAS controller. We're trying to make sense of it now. > > One of the difference is that with this patch, the initial affinity > is picked inside the NUMA node that matches the ITS.
Is that even for managed interrupts? We're testing the storage controller which uses managed interrupts. I should have made that clearer.
In your case, > that's either node 0 or 2. But it is unclear whether which CPUs these > map to. > > Given that I see interrupts mapped to CPUs 0-23 on one side, and 48-71 > on the other, it looks like half of your machine gets starved,
Seems that way.
So this is a mystery to me:
[ 23.584192] picked CPU62 IRQ147
147: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ITS-MSI 94404626 Edge hisi_sas_v3_hw cq
and
[ 25.896728] picked CPU62 IRQ183
183: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ITS-MSI 94437398 Edge hisi_sas_v3_hw cq
But mpstat reports for CPU62:
12:44:58 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 12:45:00 AM 62 6.54 0.00 42.99 0.00 6.54 12.15 0.00 0.00 6.54 25.23
I don't know what interrupts they are...
It's the "hisi_sas_v3_hw cq" interrupts which we're interested in.
and that > may be because no ITS targets the NUMA nodes they are part of.
So both storage controllers (which we're interested in for this test) are on socket #0, node #0.
It would > be interesting to see what happens if you manually set the affinity > of the interrupts outside of the NUMA node. >
Again, managed, so I don't think it's possible.
Thanks, John
| |