lkml.org 
[lkml]   [2015]   [Sep]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Unexpected slow block device write IO performance compared to uncached, unsynced direct IO using stock kernels
On Thursday 2015-06-18 09:35, Erik Cumps wrote:
| On Tue, Jun 16, 2015 at 3:56 PM, Erik Cumps <erik.cumps@esaturnus.com> wrote:
| > The context is a 16 GB 32-bit intel debian workstation, using an ext4
| > filesystem with journalling, on a lvm SATA3 SSD disk, with relatively
| > recent stock kernels from 3.2 onwards to 4.0, running some KVM virtual
| > machines. The host system (so not the virual machines) shows sporadic
| > extremely slow write performance (around 4 megabytes per second).
| > However, if we use the debian 3.2.0 kernel this problem does not
| > manifest itself.
[...]
| Actually, it is the *synchronous*, direct IO that matches the expected
| raw write performance of the device.
|
| The "regular IO" test is doing roughly this:
|
| echo 3 > /proc/sys/vm/drop_caches
| dd if=ramdisk_file of=test_file bs=1M count=100
| dd if=ramdisk_file of=test_file bs=1M count=100
| dd if=ramdisk_file of=test_file bs=1M count=100
|
| The direct IO test is doing roughly this:
|
| echo 3 > /proc/sys/vm/drop_caches
| dd if=ramdisk_file of=test_file oflag=sync,direct bs=1M count=100
| dd if=ramdisk_file of=test_file oflag=sync,direct bs=1M count=100
| dd if=ramdisk_file of=test_file oflag=sync,direct bs=1M count=100

I'm seeing this as well here on a number of new Dell Optiplex 7020
machines and one older Optiplex 780, all with 8GB RAM and running
Ubuntu 14.04 in 32-bit mode.

A simple dd command shows the problem:

$ dd bs=1M count=10 if=/dev/zero of=/tmp/ddtest
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 7.02392 s, 1.5 MB/s

$ dd bs=1M count=10 if=/dev/zero of=/tmp/ddtest oflag=sync,direct
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.535397 s, 19.6 MB/s

In my case, running:

echo 3 > /proc/sys/vm/drop_caches

will restore the normal speed for a limited time:

$ dd bs=1M count=10 if=/dev/zero of=/tmp/ddtest
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0123759 s, 847 MB/s

There is an old Ubuntu bug report describing the same issue:

https://bugs.launchpad.net/ubuntu/+source/linux-meta-lts-trusty/+bug/1333294

--
Dick


\
 
 \ /
  Last update: 2015-09-01 12:01    [W:0.472 / U:0.076 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site