Messages in this thread | | | From | Eric Van Hensbergen <> | Date | Mon, 27 Mar 2023 08:05:21 -0500 | Subject | Re: 9p caching with cache=loose and cache=fscache |
| |
Sorry, took a bit to unstack from day job, but while going through the patch queue I remembered I still had some questions to answer here.
On Fri, Mar 17, 2023 at 12:01 PM Luis Chamberlain <mcgrof@kernel.org> wrote: > > On Sun, Mar 12, 2023 at 01:22:34PM -0500, Eric Van Hensbergen wrote: > > I was looking at kdevops the other day - cool stuff. Was trying to > > figure out how we could do v9fs CI with it. > > Happy to help any questions you may have about it! > > > Both cache=loose and cache=fscache currently don't validate via host. > > What does this mean exactly? >
That's a good question - I guess the answer is "by design" they don't do anything special to check that the cache is up to date with the host. That being said, there are several places in the code where the cache will be invalidated (in some cases even if its up to date with the host, yes, those are likely bugs). Many of the invalidations that are present are overly conservative. The key missing one is v9fs_lookup_revalidate which currently fills missing inode data but should potentially also check to see if anything in the file changed and react accordingly. The other thing is that we aren't always using the caches when we should, if you look at the traffic even when everything should be perfectly cached in fscache we are getting data and metadata from the wire -- those are mostly bugs in the cache implementation that I'm trying to go through and fix now.
From the 9p perspective, we should be looking at qid.version values returned from the host and matching them to our internal notion of version. If the versions don't match we should be invalidating caches. qid.versions get returned on open, lookup, and most meta-data operations so there's lots of opportunities there. There are still some issues with this approach, namely that not all servers populate qid.version and right now the ones that do use a hashed timestamp (since qid.version is only 32-bits). This probably covers most bases, but its not ideal -- so currrently thinking through whether we do a further extension of the protocol or use some other mechanism. There's a possibility of using a full getattr every so often to back-up qid.version validation, but if we want truly tight coherence (not just open to close) then we need to open up some sort of back channel for invalidates from the server -- but I think the qid based invalidation probably gets us most of what we need so going to start with that.
> Right now a host with debian 6.0.0-6-amd64 certainly does not seem to push > out changes to 9p clients on VMs but Josef informs me that with 6.2-rc8 > he did see the changes propagate.
I did tighten up some of the invalidation in the last round of patches, however these are likely more on the overly conservative side versus doing the right thing -- however, its really not at the point where you can rely on it. If consistency is something you care about, I'd suggest cache=none until you can get cache=readahead.
> > Do none of the existing 9p cache modes not support open-to-close policies > at all? >
not specifically open-to-close, loose supports file and dir caching but without consistency, it might be tempting to try cache=mmap to see if it gets you closer, but my frame of reference is more the current patches versus the old code so not sure it would buy you anything.
> > Right now the cache mode used is cache=loose as that's the default, > what do you recommend for a typical kernel development environemnt? >
As I said, if you are interactively changing things I think you'd want to go for cache=none for now (as painful as it is). I have fixed what I hope to be my last bug with the new patch series so it should be going into linux-next today.
-eric
| |