Messages in this thread | | | Date | Wed, 2 Aug 2023 15:03:45 -0500 | Subject | Re: [Question] int3 instruction generates a #UD in SEV VM | From | Tom Lendacky <> |
| |
On 8/2/23 09:33, Tom Lendacky wrote: > On 8/2/23 09:25, Tom Lendacky wrote: >> On 8/2/23 09:01, Sean Christopherson wrote: >>> On Wed, Aug 02, 2023, Wu Zongyo wrote: >>>> On Mon, Jul 31, 2023 at 11:45:29PM +0800, wuzongyong wrote: >>>>> >>>>> On 2023/7/31 23:03, Tom Lendacky wrote: >>>>>> On 7/31/23 09:30, Sean Christopherson wrote: >>>>>>> On Sat, Jul 29, 2023, wuzongyong wrote: >>>>>>>> Hi, >>>>>>>> I am writing a firmware in Rust to support SEV based on project >>>>>>>> td-shim[1]. >>>>>>>> But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP) >>>>>>>> with the firmware, >>>>>>>> the linux kernel crashed because the int3 instruction in >>>>>>>> int3_selftest() cause a >>>>>>>> #UD. >>>>>>> >>>>>>> ... >>>>>>> >>>>>>>> BTW, if a create a normal VM without SEV by qemu & OVMF, the int3 >>>>>>>> instruction always generates a >>>>>>>> #BP. >>>>>>>> So I am confused now about the behaviour of int3 instruction, >>>>>>>> could anyone help to explain the behaviour? >>>>>>>> Any suggestion is appreciated! >>>>>>> >>>>>>> Have you tried my suggestions from the other thread[*]? >>>>> Firstly, I'm sorry for sending muliple mails with the same content. I >>>>> thought the mails I sent previously >>>>> didn't be sent successfully. >>>>> And let's talk the problem here. >>>>>>> >>>>>>> : > > I'm curious how this happend. I cannot find any condition >>>>>>> that would >>>>>>> : > > cause the int3 instruction generate a #UD according to >>>>>>> the AMD's spec. >>>>>>> : >>>>>>> : One possibility is that the value from memory that gets >>>>>>> executed diverges from the >>>>>>> : value that is read out be the #UD handler, e.g. due to >>>>>>> patching (doesn't seem to >>>>>>> : be the case in this test), stale cache/tlb entries, etc. >>>>>>> : >>>>>>> : > > BTW, it worked nomarlly with qemu and ovmf. >>>>>>> : > >>>>>>> : > Does this happen every time you boot the guest with your >>>>>>> firmware? What >>>>>>> : > processor are you running on? >>>>>>> : >>>>> Yes, every time. >>>>> The processor I used is EPYC 7T83. >>>>>>> : And have you ruled out KVM as the culprit? I.e. verified >>>>>>> that KVM is NOT injecting >>>>>>> : a #UD. That obviously shouldn't happen, but it should be >>>>>>> easy to check via KVM >>>>>>> : tracepoints. >>>>>> >>>>>> I have a feeling that KVM is injecting the #UD, but it will take >>>>>> instrumenting KVM to see which path the #UD is being injected from. >>>>>> >>>>>> Wu Zongyo, can you add some instrumentation to figure that out if >>>>>> the trace points towards KVM injecting the #UD? >>>>> Ok, I will try to do that. >>>> You're right. The #UD is injected by KVM. >>>> >>>> The path I found is: >>>> svm_vcpu_run >>>> svm_complete_interrupts >>>> kvm_requeue_exception // vector = 3 >>>> kvm_make_request >>>> >>>> vcpu_enter_guest >>>> kvm_check_and_inject_events >>>> svm_inject_exception >>>> svm_update_soft_interrupt_rip >>>> __svm_skip_emulated_instruction >>>> x86_emulate_instruction >>>> svm_can_emulate_instruction >>>> kvm_queue_exception(vcpu, UD_VECTOR) >>>> >>>> Does this mean a #PF intercept occur when the guest try to deliver a >>>> #BP through the IDT? But why? >>> >>> I doubt it's a #PF. A #NPF is much more likely, though it could be >>> something >>> else entirely, but I'm pretty sure that would require bugs in both the >>> host and >>> guest. >>> >>> What is the last exit recorded by trace_kvm_exit() before the #UD is >>> injected? >> >> I'm guessing it was a #NPF, too. Could it be related to the changes that >> went in around svm_update_soft_interrupt_rip()? >> >> 6ef88d6e36c2 ("KVM: SVM: Re-inject INT3/INTO instead of retrying the >> instruction") > > Sorry, that should have been: > > 7e5b5ef8dca3 ("KVM: SVM: Re-inject INTn instead of retrying the insn on > "failure"")
Doh! I was right the first time... sigh
6ef88d6e36c2 ("KVM: SVM: Re-inject INT3/INTO instead of retrying the instruction")
Thanks, Tom
> >> >> Before this the !nrips check would prevent the call into >> svm_skip_emulated_instruction(). But now, there is a call to: >> >> svm_update_soft_interrupt_rip() >> __svm_skip_emulated_instruction() >> kvm_emulate_instruction() >> x86_emulate_instruction() (passed a NULL insn pointer) >> kvm_can_emulate_insn() (passed a NULL insn pointer) >> svm_can_emulate_instruction() (passed NULL insn pointer) >> >> Because it is an SEV guest, it ends up in the "if (unlikely(!insn))" path >> and injects the #UD. >> >> Thanks, >> Tom >>
| |