lkml.org 
[lkml]   [1999]   [Mar]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRe: disk head scheduling
Hi,

On Mon, 22 Mar 1999 21:39:23 +0100 (MET), Gerard Roudier
<groudier@club-internet.fr> said:

> Linus,

> I agree completely with all your proposals below. I just will try to
> detail what I understood and would be happy to have:

> The per major mapping function (kdev_t), as you suggest, seems indeed a
> excellent approach.

You want the driver to be able to define its own granularity. In
particular, you want to let it define multiple queues per major if
that happens to make sense to the driver (eg, one queue per controller
for ide or scsi). The existing ll_rw_block lets you do this.

There is another issue which has not been brought up yet, and which
I've suddenly become aware of because it is causing deadlocks in the
nbd code (primarily when using loopback, but also possible if you have
two hosts each nbd-exporting to the other). The problem is that we
not only have one request pool, we also have only one thread which
feeds dirty buffers to the request layer.

The result of these two limits combined is that bdflush can progress
no faster than the slowest currently-active writer device. As soon as
the request queue is full, we can only write as fast as requests
become free. That's just a performance problem, but it leads to a
deadlock too, any time satisfying a request causes a new write request
to come in from user space.

This is not the old loopback deadlock on the request queue. What
happens is that when you start generating a lot of dirty data against
the nbd device, you eventually trigger the wakeup_bdflush(1) call in
buffer.c and stall waiting for bdflush to complete. At this point,
_all_ writes will also stall at the same point, since only after
bdflush has started working will the too-many-dirty-buffers condition
be made untrue.

In particular, the nbd server blocks. There are enough free requests
to allow the underlying writes to succeed, but there is no worker
thread able to actually schedule those writes to disk, since bdflush
is busy stalled in get_request() satisfying the nbd_server requests.

Ultimately, I think the right way to fix the deadlock issue is for the
nbd server to perform its writes synchronously, and preferably from a
kernel server thread. As long as it doesn't rely on bdflush, there is
no deadlock. However, we still have bdflush as a bottleneck, and any
multi-queue system for device IOs will have to consider how to perform
per-device write throttling in a way which does not allow slow devices
to penalise faster ones.

--Stephen

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:50    [W:0.103 / U:0.140 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site