lkml.org 
[lkml]   [2023]   [Aug]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: Commit 'sunrpc: Use sendmsg(MSG_SPLICE_PAGES) rather then sendpage' broke O_DIRECT over NFS
Date


> On Aug 17, 2023, at 11:57 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
>
>
>
>> On Aug 17, 2023, at 11:52 AM, Maxim Levitsky <mlevitsk@redhat.com> wrote:
>>
>> Hi!
>>
>> I just updated my developement systems to 6.5-rc6 (from 6.4) and now I can't start a VM
>> with a disk which is mounted over the NFS.
>>
>> The VM has two qcow2 files, one depends on another and qemu opens both.
>>
>> This is the command line of qemu:
>>
>> -drive if=none,id=os_image,file=./disk_s1.qcow2,aio=native,discard=unmap,cache=none
>>
>> The disk_s1.qcow2 depends on disk_s0.qcow2
>>
>> However this is what I get:
>>
>> qemu-system-x86_64: -drive if=none,id=os_image,file=./disk_s1.qcow2,aio=native,discard=unmap,cache=none: Could not open backing file: Could not open './QFI?': No such file or directory
>>
>> 'QFI?' is qcow2 file signature, which signals that there might be some nasty corruption happening.
>>
>> The program was supposed to read a field inside the disk_s1.qcow2 file which should read 'disk_s0.qcow2'
>> but instead it seems to read the first 4 bytes of the file.
>>
>>
>> Bisect leads to the above commit. Reverting it was not possible due to many changes.
>>
>> Both the client and the server were tested with the 6.5-rc6 kernel, but once rebooting the server into
>> the 6.4, the bug disappeared, thus I did a bisect on the server.
>>
>> When I tested a version before the offending commit on the server, the 6.5-rc6 client was able to work with it,
>> which increases the chances that the bug is in nfsd.
>>
>> Switching qemu to use write back paging also helps (aio=threads,discard=unmap,cache=writeback)
>> The client and the server (both 6.5-rc6) work with this configuration.
>>
>> Running the VM on the same machine (also 6.5-rc6) where the VM disk is located (thus avoiding NFS) works as well.
>>
>> I tested several VMs that I have, all are affected in the same way.
>>
>> I run somewhat outdated qemu, but running the latest qemu doesn't make a difference.
>>
>> I use nfs4.
>>
>> I can test patches and provide more info if needed.
>
> Linus just merged a possible fix for this issue. See:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ master

In particular:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c96e2a695e00bca5487824d84b85aab6aa2c1891


--
Chuck Lever


\
 
 \ /
  Last update: 2023-08-17 17:59    [W:0.039 / U:0.124 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site