lkml.org 
[lkml]   [2021]   [Mar]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: blktests: block/009 next-20210304 failure rate average of 1/448
On Tue, Mar 16, 2021 at 06:47:39PM +0000, Luis Chamberlain wrote:
> On Tue, Mar 16, 2021 at 05:46:45PM +0000, Luis Chamberlain wrote:
> > I've managed to reproduce blktests block/009 failures with kdevops [0]
> > on linux-next tag next-20210304 with a current failure rate average of
> > 1/448 (3 counted failures so far).
>
> Confirmed on next-20210316 with current failure rate at 1/1008

Just in case this was a scsi_debug issue instead (I am covering that
prospect on another bug just for scsi_debug korg#212337 [0]) I tried
a userspace solution based on what I have observed I still can reproduce
this block/009 failure. The failure rate is much lower though, I have it
now at 1/1705 but alas it is still failing.

[0] https://bugzilla.kernel.org/show_bug.cgi?id=212337

The patch below demonstrates the exra settle work for scsi_debug
attempted, and with it, this is still failing. So either the settle
work needs *more* effort, or this is a real issue.

diff --git a/common/scsi_debug b/common/scsi_debug
index b48cdc9..ecdbcc6 100644
--- a/common/scsi_debug
+++ b/common/scsi_debug
@@ -8,13 +8,42 @@ _have_scsi_debug() {
_have_modules scsi_debug
}

+# As per korg#212337 [0] we must do more work in userspace to settle
+# scsi_debug devices a bit more carefully.
+
+# [0] https://bugzilla.kernel.org/show_bug.cgi?id=212337
+_settle_scsi_debug_device() {
+ SCSI_DEBUG_MAX_WAIT=10
+ SCSI_DEBUG_COUNT_WAIT_LOOP=0
+ while true ; do
+ if [[ -b $1 ]]; then
+ SCSI_DEBUG_LSOF_COUNT=$(lsof $1 | wc -l)
+ if [[ $SCSI_DEBUG_LSOF_COUNT -ne 0 ]]; then
+ sleep 1;
+ else
+ break
+ fi
+ else
+ # Let device come up
+ sleep 1
+
+ let SCSI_DEBUG_COUNT_WAIT_LOOP=$SCSI_DEBUG_COUNT_WAIT_LOOP+1
+ if [[ $SCSI_DEBUG_COUNT_WAIT_LOOP -ge $SCSI_DEBUG_MAX_WAIT ]]; then
+ break
+ fi
+ fi
+ done
+}
+
_init_scsi_debug() {
if ! modprobe -r scsi_debug || ! modprobe scsi_debug "$@"; then
return 1
fi
-
udevadm settle

+ # Allow dependencies to load
+ sleep 1
+
local host_sysfs host target_sysfs target
SCSI_DEBUG_HOSTS=()
SCSI_DEBUG_TARGETS=()
@@ -43,6 +72,10 @@ _init_scsi_debug() {
return 1
fi

+ for i in $SCSI_DEBUG_DEVICES ; do
+ _settle_scsi_debug_device /dev/$i
+ done
+
return 0
}

\
 
 \ /
  Last update: 2021-03-18 20:34    [W:0.096 / U:1.024 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site