s390/pci: Serialize device addition and removal

JIRA: https://issues.redhat.com/browse/RHEL-94944

commit 774a1fa
Author: Niklas Schnelle <schnelle@linux.ibm.com>
Date:   Thu May 22 14:13:15 2025 +0200

    s390/pci: Serialize device addition and removal

    Prior changes ensured that when zpci_release_device() is called and it
    removed the zdev from the zpci_list this instance can not be found via
    the zpci_list anymore even while allowing re-add of reserved devices.
    This only accounts for the overall lifetime and zpci_list addition and
    removal, it does not yet prevent concurrent add of a new instance for
    the same underlying device. Such concurrent add would subsequently cause
    issues such as attempted re-use of the same IOMMU sysfs directory and is
    generally undesired.

    Introduce a new zpci_add_remove_lock mutex to serialize adding a new
    device with removal. Together this ensures that if a struct zpci_dev is
    not found in the zpci_list it was either already removed and torn down,
    or its removal and tear down is in progress with the
    zpci_add_remove_lock held.

    Cc: stable@vger.kernel.org
    Fixes: a46044a ("s390/pci: fix zpci_zdev_put() on reserve")
    Reviewed-by: Gerd Bayer <gbayer@linux.ibm.com>
    Tested-by: Gerd Bayer <gbayer@linux.ibm.com>
    Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
    Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

Signed-off-by: Mete Durlu <mdurlu@redhat.com>

Author: Mete Durlu

arch/s390/pci/pci.c

@@ -44,6 +44,7 @@
 /* list of all detected zpci devices */
 static LIST_HEAD(zpci_list);
 static DEFINE_SPINLOCK(zpci_list_lock);
+static DEFINE_MUTEX(zpci_add_remove_lock);
 
 static DECLARE_BITMAP(zpci_domain, ZPCI_DOMAIN_BITMAP_SIZE);
 static DEFINE_SPINLOCK(zpci_domain_lock);
@@ -73,7 +74,9 @@ void zpci_zdev_put(struct zpci_dev *zdev)
 {
 	if (!zdev)
 		return;
+	mutex_lock(&zpci_add_remove_lock);
 	kref_put_lock(&zdev->kref, zpci_release_device, &zpci_list_lock);
+	mutex_unlock(&zpci_add_remove_lock);
 }
 
 struct zpci_dev *get_zdev_by_fid(u32 fid)
@@ -838,6 +841,7 @@ int zpci_add_device(struct zpci_dev *zdev)
 {
 	int rc;
 
+	mutex_lock(&zpci_add_remove_lock);
 	zpci_dbg(1, "add fid:%x, fh:%x, c:%d\n", zdev->fid, zdev->fh, zdev->state);
 	rc = zpci_init_iommu(zdev);
 	if (rc)
@@ -851,12 +855,14 @@ int zpci_add_device(struct zpci_dev *zdev)
 	spin_lock(&zpci_list_lock);
 	list_add_tail(&zdev->entry, &zpci_list);
 	spin_unlock(&zpci_list_lock);
+	mutex_unlock(&zpci_add_remove_lock);
 	return 0;
 
 error_destroy_iommu:
 	zpci_destroy_iommu(zdev);
 error:
 	zpci_dbg(0, "add fid:%x, rc:%d\n", zdev->fid, rc);
+	mutex_unlock(&zpci_add_remove_lock);
 	return rc;
 }
 
@@ -947,6 +953,7 @@ void zpci_release_device(struct kref *kref)
 {
 	struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref);
 
+	lockdep_assert_held(&zpci_add_remove_lock);
 	WARN_ON(zdev->state != ZPCI_FN_STATE_RESERVED);
 	/*
 	 * We already hold zpci_list_lock thanks to kref_put_lock().