Messages in this thread | | | From | Stefan Hajnoczi <> | Date | Tue, 15 Aug 2023 11:18:42 -0400 | Subject | Re: LTP test related to virtio releasing and reassigning resource leads to guest hung |
| |
On Thu, 10 Aug 2023 at 22:27, longguang.yue <kvmluck@163.com> wrote: > > > 一) > Can you post the guest kernel messages (dmesg)? If the guest is hanging > then it may be easiest to configure a serial console so the kernel > messages are sent to the host where you can see them. > > Does the hang occur during the LTP code you linked or afterwards when > the PCI device is bound to a virtio driver? > > > > I used conosle, the hang occurred afterwards. dmesg shows that tpci test is finished without error. > LTP test case: https://github.com/linux-test-project/ltp/blob/522d7fba4afc84e07b252aa4cd91b241e81d6613/testcases/kernel/device-drivers/pci/tpci_kernel/ltp_tpci.c#L428 > kernel 5.10, qemu 6.2 > > different guest-configuration tests show different results. guest did not crash if hung-task-panic=0, in my case i enable hung-task-panic in order to trace. > > test case 1: > xml machine pc,virtio disk, virtio net —— guest's io hung, network broke down, though console is avilable but io operation hung. > > #ps -aux| grep D > root 7 0.0 0.0 0 0 ? D 14:37 0:00 [kworker/u16:0+flush-253:0] > root 483 0.0 0.0 0 0 ? D 14:37 0:00 [jbd2/vda3-8] > > test case 2: > xml machine q35,virtio/q35,scsi ——disk did not hung but network broke down. ping errors though everything looks ok and no crash and no kernel error > > > > 二) > I didn't see your original email so I missed the panic. I'd still like > to see the earlier kernel messages before the panic in order to > understand how the PCI device is bound. > > Is the vda device with hung I/O the same device that was accessed by > the LTP test earlier? I guess the LTP test runs against the device and > then the virtio driver binds to the device again afterwards? > > > the test is > ``` > // iterate all devices > …… > for (i = 0; i < 7; ++i) { // iterate current device's resources > if (r->flags & IORESOURCE_MEM && > r->flags & IORESOURCE_PREFETCH) { > pci_release_resource(dev, i); > ret = pci_assign_resource(dev, i); > prk_info("assign resource to '%d', ret '%d'", i, ret); > rc |= (ret < 0 && ret != -EBUSY) ? TFAIL : TPASS; > } > } > ``` > test does not do virtio device unbind and bind. > I only notice mem resource changed. see 'test-case 12' > > ——————————— > [ 88.905705] ltp_tpci: test-case 12 > [ 88.905706] ltp_tpci: assign resources > [ 88.905706] ltp_tpci: assign resource #0 > [ 88.905707] ltp_tpci: name = 0000:00:07.0, flags = 262401, start 0xc080, end 0xc0ff > [ 88.905707] ltp_tpci: assign resource #1 > [ 88.905708] ltp_tpci: name = 0000:00:07.0, flags = 262656, start 0xfebd4000, end 0xfebd4fff > [ 88.905709] ltp_tpci: assign resource #2 > [ 88.905709] ltp_tpci: name = 0000:00:07.0, flags = 0, start 0x0, end 0x0 > [ 88.905710] ltp_tpci: assign resource #3 > [ 88.905710] ltp_tpci: name = 0000:00:07.0, flags = 0, start 0x0, end 0x0 > [ 88.905711] ltp_tpci: assign resource #4 > [ 88.905711] ltp_tpci: name = 0000:00:07.0, flags = 1319436, start 0xfe00c000, end 0xfe00ffff > [ 88.905713] virtio-pci 0000:00:07.0: BAR 4: releasing [mem 0xfe00c000-0xfe00ffff 64bit pref] > [ 88.905715] virtio-pci 0000:00:07.0: BAR 4: assigned [mem 0x24000c000-0x24000ffff 64bit pref] > [ 88.906693] ltp_tpci: assign resource to '4', ret '0' > [ 88.906694] ltp_tpci: assign resource #5 > [ 88.906694] ltp_tpci: name = (null), flags = 0, start 0x0, end 0x0 > [ 88.906695] ltp_tpci: assign resource #6 > [ 88.906695] ltp_tpci: name = 0000:00:07.0, flags = 0, start 0x0, end 0x0 > > [ 88.906800] ltp_tpci: test-case 13
I don't know. Maybe the test case is leaving the device is a state that conflicts with the virtio drivers that are bound after testing finishes.
One approach is to trace the PCI BAR accesses after the test runs and compare against a trace when the tpci driver hasn't been loaded. That way you might be able to find out what is different.
Stefan
| |