Messages in this thread | | | Subject | Re: [PATCH] s390/vfio-ap: Clean up vfio_ap resources when KVM pointer invalidated | From | Tony Krowiak <> | Date | Fri, 11 Dec 2020 16:08:53 -0500 |
| |
On 12/7/20 7:40 PM, Halil Pasic wrote: > On Mon, 7 Dec 2020 14:05:55 -0500 > Tony Krowiak <akrowiak@linux.ibm.com> wrote: > >> >> On 12/2/20 6:41 PM, Tony Krowiak wrote: >>> The vfio_ap device driver registers a group notifier with VFIO when the >>> file descriptor for a VFIO mediated device for a KVM guest is opened to >>> receive notification that the KVM pointer is set (VFIO_GROUP_NOTIFY_SET_KVM >>> event). When the KVM pointer is set, the vfio_ap driver stashes the pointer >>> and calls the kvm_get_kvm() function to increment its reference counter. >>> When the notifier is called to make notification that the KVM pointer has >>> been set to NULL, the driver should clean up any resources associated with >>> the KVM pointer and decrement its reference counter. The current >>> implementation does not take care of this clean up. >>> >>> Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com> >>> --- >>> drivers/s390/crypto/vfio_ap_ops.c | 21 +++++++++++++-------- >>> 1 file changed, 13 insertions(+), 8 deletions(-) >>> >>> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c >>> index e0bde8518745..eeb9c9130756 100644 >>> --- a/drivers/s390/crypto/vfio_ap_ops.c >>> +++ b/drivers/s390/crypto/vfio_ap_ops.c >>> @@ -1083,6 +1083,17 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, >>> return NOTIFY_DONE; >>> } >>> >>> +static void vfio_ap_mdev_put_kvm(struct ap_matrix_mdev *matrix_mdev) >>> +{ >>> + if (matrix_mdev->kvm) { >>> + kvm_arch_crypto_clear_masks(matrix_mdev->kvm); >>> + matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; >>> + vfio_ap_mdev_reset_queues(matrix_mdev->mdev); >> This reset probably does not belong here since there is no >> reason to reset the queues in the group notifier (see below). > What about kvm_s390_gisc_unregister()? That needs a valid kvm > pointer, or? Or is it OK to not pair a kvm_s390_gisc_register() > with an kvm_s390_gisc_unregister()?
I probably should have been more specific about what I meant. I was thinking that the reset should not be dependent upon whether there is a KVM pointer or not since this function is also called from the release callback. On the other hand, the vfio_ap_mdev_reset_queues function calls the vfio_ap_irq_disable (AQIC) function after each queue is reset. The vfio_ap_irq_disable function also cleans up the AQIC resources which requires that the KVM point is valid, so if the vfio_ap_reset_queues function is not called with a valid KVM pointer, that could result in an exception.
The thing is, it is unnecessary to disable interrupts after resetting a queue because the reset disables interrupts, so I think I should include a patch for this fix that does the following:
1. Removes the disabling of interrupts subsequent to resetting a queue. 2. Includes the cleanup of AQIC resources when a queue is reset if a KVM pointer is present.
This will allow us to keep the reset in the function above as well as the other places from which reset is executed.
> > Regards, > Halil > >> The reset should be done in the release callback only regardless >> of whether the KVM pointer exists or not. >> >>> + kvm_put_kvm(matrix_mdev->kvm); >>> + matrix_mdev->kvm = NULL; >>> + } >>> +} >>> + >>> static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, >>> unsigned long action, void *data) >>> { >>> @@ -1095,7 +1106,7 @@ static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, >>> matrix_mdev = container_of(nb, struct ap_matrix_mdev, group_notifier); >>> >>> if (!data) { >>> - matrix_mdev->kvm = NULL; >>> + vfio_ap_mdev_put_kvm(matrix_mdev); >>> return NOTIFY_OK; >>> } >>> >>> @@ -1222,13 +1233,7 @@ static void vfio_ap_mdev_release(struct mdev_device *mdev) >>> struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); >>> >>> mutex_lock(&matrix_dev->lock); >>> - if (matrix_mdev->kvm) { >>> - kvm_arch_crypto_clear_masks(matrix_mdev->kvm); >>> - matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; >>> - vfio_ap_mdev_reset_queues(mdev); >> This release should be moved outside of the block and >> performed regardless of whether the KVM pointer exists or >> not. >> >>> - kvm_put_kvm(matrix_mdev->kvm); >>> - matrix_mdev->kvm = NULL; >>> - } >>> + vfio_ap_mdev_put_kvm(matrix_mdev); >>> mutex_unlock(&matrix_dev->lock); >>> >>> vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
| |