Messages in this thread | | | Date | Sat, 05 Feb 2005 09:51:56 +1100 | From | Lincoln Dale <> | Subject | Re: Drive performance bottleneck |
| |
At 08:32 PM 4/02/2005, Andrew Morton wrote: >Something funny is happening here - it looks like there's plenty of CPU >capacity left over. [..] >Could you monitor the CPU load during the various tests? If the `dd' >workload isn't pegging the CPU then it could be that there's something >wrong with the I/O submission patterns.
as an educated guess, i'd say that the workload is running out of memory bandwidth ..
lets say the RAM is single-channel DDR400. peak bandwidth = 3.2Gb/s (400 x 10^6 x 64 bits / 1000000000). its fair to say that peak bandwidth is pretty rare thing to achieve with SDRAM given real-world access patterns -- lets take a conservative "it'll be 50% efficient" -- so DDR400 realistic peak = 1.6Gbps.
as far as memory-accesses go, a standard user-space read() from disk results in 4 memory-accesses (1. DMA from HBA to RAM, 2. read in copy_to_user(), 3. write in copy_to_user(), 4. userspace accessing that data). 1.6Gbps / 4 = 400MB/s -- or roughly what Ian was seeing.
sg_dd uses a window into a kernel DMA window. as such, two of the four memory acccesses are cut out (1. DMA from HBA to RAM, 2. userspace accessing data). 1.6Gbps / 2 = 800MB/s -- or roughly what Ian was seeing with sg_dd.
DIRECT_IO should achieve similar numbers to sg_dd, but perhaps not quite as efficient.
cheers,
lincoln. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |