lkml.org 
[lkml]   [2023]   [Aug]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [Question] int3 instruction generates a #UD in SEV VM
From
On 8/2/23 09:25, Tom Lendacky wrote:
> On 8/2/23 09:01, Sean Christopherson wrote:
>> On Wed, Aug 02, 2023, Wu Zongyo wrote:
>>> On Mon, Jul 31, 2023 at 11:45:29PM +0800, wuzongyong wrote:
>>>>
>>>> On 2023/7/31 23:03, Tom Lendacky wrote:
>>>>> On 7/31/23 09:30, Sean Christopherson wrote:
>>>>>> On Sat, Jul 29, 2023, wuzongyong wrote:
>>>>>>> Hi,
>>>>>>> I am writing a firmware in Rust to support SEV based on project
>>>>>>> td-shim[1].
>>>>>>> But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP)
>>>>>>> with the firmware,
>>>>>>> the linux kernel crashed because the int3 instruction in
>>>>>>> int3_selftest() cause a
>>>>>>> #UD.
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>> BTW, if a create a normal VM without SEV by qemu & OVMF, the int3
>>>>>>> instruction always generates a
>>>>>>> #BP.
>>>>>>> So I am confused now about the behaviour of int3 instruction, could
>>>>>>> anyone help to explain the behaviour?
>>>>>>> Any suggestion is appreciated!
>>>>>>
>>>>>> Have you tried my suggestions from the other thread[*]?
>>>> Firstly, I'm sorry for sending muliple mails with the same content. I
>>>> thought the mails I sent previously
>>>> didn't be sent successfully.
>>>> And let's talk the problem here.
>>>>>>
>>>>>>     : > > I'm curious how this happend. I cannot find any condition
>>>>>> that would
>>>>>>     : > > cause the int3 instruction generate a #UD according to the
>>>>>> AMD's spec.
>>>>>>     :
>>>>>>     : One possibility is that the value from memory that gets
>>>>>> executed diverges from the
>>>>>>     : value that is read out be the #UD handler, e.g. due to
>>>>>> patching (doesn't seem to
>>>>>>     : be the case in this test), stale cache/tlb entries, etc.
>>>>>>     :
>>>>>>     : > > BTW, it worked nomarlly with qemu and ovmf.
>>>>>>     : >
>>>>>>     : > Does this happen every time you boot the guest with your
>>>>>> firmware? What
>>>>>>     : > processor are you running on?
>>>>>>     :
>>>> Yes, every time.
>>>> The processor I used is EPYC 7T83.
>>>>>>     : And have you ruled out KVM as the culprit?  I.e. verified that
>>>>>> KVM is NOT injecting
>>>>>>     : a #UD.  That obviously shouldn't happen, but it should be easy
>>>>>> to check via KVM
>>>>>>     : tracepoints.
>>>>>
>>>>> I have a feeling that KVM is injecting the #UD, but it will take
>>>>> instrumenting KVM to see which path the #UD is being injected from.
>>>>>
>>>>> Wu Zongyo, can you add some instrumentation to figure that out if the
>>>>> trace points towards KVM injecting the #UD?
>>>> Ok, I will try to do that.
>>> You're right. The #UD is injected by KVM.
>>>
>>> The path I found is:
>>>      svm_vcpu_run
>>>          svm_complete_interrupts
>>>         kvm_requeue_exception // vector = 3
>>>             kvm_make_request
>>>
>>>      vcpu_enter_guest
>>>          kvm_check_and_inject_events
>>>         svm_inject_exception
>>>             svm_update_soft_interrupt_rip
>>>             __svm_skip_emulated_instruction
>>>                 x86_emulate_instruction
>>>                 svm_can_emulate_instruction
>>>                     kvm_queue_exception(vcpu, UD_VECTOR)
>>>
>>> Does this mean a #PF intercept occur when the guest try to deliver a
>>> #BP through the IDT? But why?
>>
>> I doubt it's a #PF.  A #NPF is much more likely, though it could be
>> something
>> else entirely, but I'm pretty sure that would require bugs in both the
>> host and
>> guest.
>>
>> What is the last exit recorded by trace_kvm_exit() before the #UD is
>> injected?
>
> I'm guessing it was a #NPF, too. Could it be related to the changes that
> went in around svm_update_soft_interrupt_rip()?
>
> 6ef88d6e36c2 ("KVM: SVM: Re-inject INT3/INTO instead of retrying the
> instruction")

Sorry, that should have been:

7e5b5ef8dca3 ("KVM: SVM: Re-inject INTn instead of retrying the insn on "failure"")

>
> Before this the !nrips check would prevent the call into
> svm_skip_emulated_instruction(). But now, there is a call to:
>
>   svm_update_soft_interrupt_rip()
>     __svm_skip_emulated_instruction()
>       kvm_emulate_instruction()
>         x86_emulate_instruction() (passed a NULL insn pointer)
>           kvm_can_emulate_insn() (passed a NULL insn pointer)
>             svm_can_emulate_instruction() (passed NULL insn pointer)
>
> Because it is an SEV guest, it ends up in the "if (unlikely(!insn))" path
> and injects the #UD.
>
> Thanks,
> Tom
>

\
 
 \ /
  Last update: 2023-08-02 16:34    [W:0.077 / U:2.088 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site