Messages in this thread | | | Date | Tue, 5 Sep 2023 09:32:19 +0800 | Subject | Re: [RFC PATCH v2 00/18] scsi: scsi_error: Introduce new error handle mechanism | From | "haowenchao (C)" <> |
| |
On 2023/9/1 17:41, Wenchao Hao wrote: > It's unbearable for systems with large scale scsi devices share HBAs to > block all devices' IOs when handle error commands, we need a new error > handle mechanism to address this issue. > > I consulted about this issue a year ago, the discuss link can be found in > refenence. Hannes replied about why we have to block the SCSI host > then perform error recovery kindly. I think it's unnecessary to block > SCSI host for all drivers and can try a small level recovery(LUN based for > example) first to avoid block the SCSI host. > > The new error handle mechanism introduced in this patchset has been > developed and tested with out self developed hardware since one year > ago, now we want this mechanism can be used by more drivers. > > Drivers can decide if using the new error handle mechanism and how to > handle error commands when scsi_device are scanned,the new mechanism > makes SCSI error handle more flexible.
Hi, Hannes and Mike, would you help take a look at these changes?
> > SCSI error recovery strategy after blocking host's IO is mainly > following steps: > > - LUN reset > - Target reset > - Bus reset > - Host reset > > Some drivers did not implement callbacks for host reset, it's unnecessary > to block host's IO for these drivers. For example, smartpqi only registered > device reset, if device reset failed, it's meaningless to fallback to target > reset, bus reset or host reset any more, because these steps would also > failed. > > Here are some drivers we concerned:(there are too many kinds of drivers > to figure out, so here I just list some drivers I am familiar with) > > +-------------+--------------+--------------+-----------+------------+ > | drivers | device_reset | target_reset | bus_reset | host_reset | > +-------------+--------------+--------------+-----------+------------+ > | mpt3sas | Y | Y | N | Y | > +-------------+--------------+--------------+-----------+------------+ > | smartpqi | Y | N | N | N | > +-------------+--------------+--------------+-----------+------------+ > | megaraidsas | N | Y | N | Y | > +-------------+--------------+--------------+-----------+------------+ > | virtioscsi | Y | N | N | N | > +-------------+--------------+--------------+-----------+------------+ > | iscsi_tcp | Y | Y | N | N | > +-------------+--------------+--------------+-----------+------------+ > | hisisas | Y | Y | N | N | > +-------------+--------------+--------------+-----------+------------+ > > For LUN based error handle, when scsi command is classified as error, > we would block the scsi device's IO and try to recover this scsi > device, if still can not recover all error commands, it might > fallback to target or host level recovery. > > It's same for target based error handle, but target based error handle > would block the scsi target's IO then try to recover the error commands > of this target. > > The first patch defines basic framework to support LUN/target based error > handle mechanism, three key operations are abstracted which are: > - add error command > - wake up error handle > - block IOs when error command is added and recoverying. > > Drivers can implement these three function callbacks and setup to SCSI > middle level; I also add a general LUN/target based error handle strategy > which can be called directly from drivers to implement LUN/tartget based > error handle. > > The changes of SCSI middle level's error handle are tested with scsi_debug > which support single LUN error injection, the scsi_debug patches can be > found in reference, following scenarios are tested. > > Scenario1: LUN based error handle is enabled: > +-----------+---------+-------------------------------------------------------+ > | lun reset | TUR | Desired result | > + --------- + ------- + ------------------------------------------------------+ > | success | success | retry or finish with EIO(may offline disk) | > + --------- + ------- + ------------------------------------------------------+ > | success | fail | fallback to host recovery, retry or finish with | > | | | EIO(may offline disk) | > + --------- + ------- + ------------------------------------------------------+ > | fail | NA | fallback to host recovery, retry or finish with | > | | | EIO(may offline disk) | > + --------- + ------- + ------------------------------------------------------+ > > Scenario2: target based error handle is enabled: > +-----------+---------+--------------+---------+------------------------------+ > | lun reset | TUR | target reset | TUR | Desired result | > +-----------+---------+--------------+---------+------------------------------+ > | success | success | NA | NA | retry or finish with | > | | | | | EIO(may offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | success | fail | success | success | retry or finish with | > | | | | | EIO(may offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | success | success | retry or finish with | > | | | | | EIO(may offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | success | fail | fallback to host recovery, | > | | | | | retry or finish with EIO(may | > | | | | | offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | fail | NA | fallback to host recovery, | > | | | | | retry or finish with EIO(may | > | | | | | offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > > Scenario3: both LUN and target based error handle are enabled: > +-----------+---------+--------------+---------+------------------------------+ > | lun reset | TUR | target reset | TUR | Desired result | > +-----------+---------+--------------+---------+------------------------------+ > | success | success | NA | NA | retry or finish with | > | | | | | EIO(may offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | success | fail | success | success | lun recovery fallback to | > | | | | | target recovery, retry or | > | | | | | finish with EIO(may offline | > | | | | | disk | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | success | success | lun recovery fallback to | > | | | | | target recovery, retry or | > | | | | | finish with EIO(may offline | > | | | | | disk | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | success | fail | lun recovery fallback to | > | | | | | target recovery, then fall | > | | | | | back to host recovery, retry | > | | | | | or fhinsi with EIO(may | > | | | | | offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > | fail | NA | fail | NA | lun recovery fallback to | > | | | | | target recovery, then fall | > | | | | | back to host recovery, retry | > | | | | | or fhinsi with EIO(may | > | | | | | offline disk) | > +-----------+---------+--------------+---------+------------------------------+ > > References: https://lore.kernel.org/linux-scsi/20230815122316.4129333-1-haowenchao2@huawei.com/ > References: https://lore.kernel.org/linux-scsi/71e09bb4-ff0a-23fe-38b4-fe6425670efa@huawei.com/ > > Wenchao Hao (19): > scsi: scsi_error: Define framework for LUN/target based error handle > scsi: scsi_error: Move complete variable eh_action from shost to sdevice > scsi: scsi_error: Check if to do reset in scsi_try_xxx_reset > scsi: scsi_error: Add helper scsi_eh_sdev_stu to do START_UNIT > scsi: scsi_error: Add helper scsi_eh_sdev_reset to do lun reset > scsi: scsi_error: Add flags to mark error handle steps has done > scsi: scsi_error: Add helper to handle scsi device's error command list > scsi: scsi_error: Add a general LUN based error handler > scsi: core: increase/decrease target_busy without check can_queue > scsi: scsi_error: Add helper to handle scsi target's error command list > scsi: scsi_error: Add a general target based error handler > scsi: scsi_debug: Add param to control LUN bassed error handler > scsi: scsi_debug: Add param to control target based error handle > scsi: mpt3sas: Add param to control LUN based error handle > scsi: mpt3sas: Add param to control target based error handle > scsi: smartpqi: Add param to control LUN based error handle > scsi: megaraid_sas: Add param to control target based error handle > scsi: virtio_scsi: Add param to control LUN based error handle > scsi: iscsi_tcp: Add param to control LUN based error handle > > drivers/scsi/iscsi_tcp.c | 20 + > drivers/scsi/megaraid/megaraid_sas_base.c | 20 + > drivers/scsi/mpt3sas/mpt3sas_scsih.c | 28 + > drivers/scsi/scsi_debug.c | 24 + > drivers/scsi/scsi_error.c | 756 ++++++++++++++++++++-- > drivers/scsi/scsi_lib.c | 23 +- > drivers/scsi/scsi_priv.h | 18 + > drivers/scsi/smartpqi/smartpqi_init.c | 14 + > drivers/scsi/virtio_scsi.c | 16 +- > include/scsi/scsi_device.h | 97 +++ > include/scsi/scsi_eh.h | 8 + > include/scsi/scsi_host.h | 2 - > 12 files changed, 963 insertions(+), 63 deletions(-) >
| |