Messages in this thread | | | From | Andy Lutomirski <> | Subject | Re: [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET_XCR0 to set XCR0 per-thread | Date | Tue, 7 Apr 2020 07:16:34 -0700 |
| |
> On Apr 7, 2020, at 7:07 AM, Dave Hansen <dave.hansen@intel.com> wrote: > > On 4/7/20 5:21 AM, Peter Zijlstra wrote: >> You had a fairly long changelog detailing what the patchd does; but I've >> failed to find a single word on _WHY_ we want to do any of that. > > The goal in these record/replay systems is to be able to recreate thee > exact same program state on two systems at two different times. To make > it reasonably fast, they try to minimize the number of snapshots they > have to take and avoid things like single stepping. > > So, there are some windows where they just let the CPU run and don't > bother with taking any snapshots of register state, for instance. Let's > say you read a word from shared memory, multiply it and shift it around > some registers, then stick it back in shared memory. Most of these > things will just a record the snapshot at the memory read and assume > that all the instructions in the middle execute deterministically. That > eliminates a ton of snapshots. > > But, what if an instruction in the middle isn't deterministic between > two machines. Let's say you record a trace on a a Broadwell system, > then try to replay it on a Skylake, and one of the non-snapshotted > instructions is xgetbv. Skylake added MPX, so xgetbv will return > different values. Your replay diverges from what was "recorded", and > life sucks. > > Same problem exists for CPUID, but that was hacked around in another set. > > I'm also trying to think of what kinds of things CPU companies add to > their architectures that would break this stuff. I can't recall ever > having a discussion with folks at Intel where we're designing a CPU > feature and we say, "Can't do that, it would break record/replay". I > suspect there are more of these landmines around and I bet that we're > building more of them into CPUs every day.
TSX!
I think rr should give the raw KVM API at least a try. It should be possible to fire up a vCPU in CPL3 in the correct state. No guest kernel required. I don’t know if there will be issues with the perf API, though.
If we actually do merge this XCR0 hack, I think the rule should be that it has no effect on kernel behavior. Signals, ptrace, etc reflect the normal XCR0, not the overridden value.
| |