Messages in this thread | | | Date | Thu, 4 Aug 2022 08:22:24 -0600 | Subject | Re: [PATCH] PCI: hv: Only reuse existing IRTE allocation for Multi-MSI | From | Jeffrey Hugo <> |
| |
On 8/3/2022 8:51 PM, Dexuan Cui wrote: > Jeffrey's 4 recent patches added Multi-MSI support to the pci-hyperv driver. > Unluckily, one of the patches, i.e., b4b77778ecc5, causes a regression to a > fio test for the Azure VM SKU Standard L64s v2 (64 AMD vCPUs, 8 NVMe drives): > > when fio runs against all the 8 NVMe drives, it runs fine with a low io-depth > (e.g., 2 or 4); when fio runs with a high io-depth (e.g., 256), somehow > queue-29 of each NVMe drive suddenly no longer receives any interrupts, and > the NVMe core code has to abort the queue after a timeout of 30 seconds, and > then queue-29 starts to receive interrupts again for several seconds, and > later queue-29 no longer receives interrupts again, and this pattern repeats: > > [ 223.891249] nvme nvme2: I/O 320 QID 29 timeout, aborting > [ 223.896231] nvme nvme0: I/O 320 QID 29 timeout, aborting > [ 223.898340] nvme nvme4: I/O 832 QID 29 timeout, aborting > [ 259.471309] nvme nvme2: I/O 320 QID 29 timeout, aborting > [ 259.476493] nvme nvme0: I/O 321 QID 29 timeout, aborting > [ 259.482967] nvme nvme0: I/O 322 QID 29 timeout, aborting > > Some other symptoms are: the throughput of the NVMe drives drops due to > commit b4b77778ecc5. When the fio test is running, the kernel prints some > soft lock-up messages from time to time. > > Commit b4b77778ecc5 itself looks good, and at the moment it's unclear where > the issue is. While the issue is being investigated, restore the old behavior > in hv_compose_msi_msg(), i.e., don't reuse the existing IRTE allocation for > single-MSI and MSI-X. This is a stopgap for the above NVMe issue. > > Fixes: b4b77778ecc5 ("PCI: hv: Reuse existing IRTE allocation in compose_msi_msg()") > Signed-off-by: Dexuan Cui <decui@microsoft.com> > Cc: Jeffrey Hugo <quic_jhugo@quicinc.com> > Cc: Carl Vanderlip <quic_carlv@quicinc.com> > ---
I'm sorry a regression has been discovered. Right now, the issue doesn't make sense to me. I'd love to know what you find out.
This stopgap solution appears reasonable to me.
Reviewed-by: Jeffrey Hugo <quic_jhugo@quicinc.com>
| |