lkml.org 
[lkml]   [2023]   [Aug]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [Question] int3 instruction generates a #UD in SEV VM
From
On 8/2/23 09:01, Sean Christopherson wrote:
> On Wed, Aug 02, 2023, Wu Zongyo wrote:
>> On Mon, Jul 31, 2023 at 11:45:29PM +0800, wuzongyong wrote:
>>>
>>> On 2023/7/31 23:03, Tom Lendacky wrote:
>>>> On 7/31/23 09:30, Sean Christopherson wrote:
>>>>> On Sat, Jul 29, 2023, wuzongyong wrote:
>>>>>> Hi,
>>>>>> I am writing a firmware in Rust to support SEV based on project td-shim[1].
>>>>>> But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP) with the firmware,
>>>>>> the linux kernel crashed because the int3 instruction in int3_selftest() cause a
>>>>>> #UD.
>>>>>
>>>>> ...
>>>>>
>>>>>> BTW, if a create a normal VM without SEV by qemu & OVMF, the int3 instruction always generates a
>>>>>> #BP.
>>>>>> So I am confused now about the behaviour of int3 instruction, could anyone help to explain the behaviour?
>>>>>> Any suggestion is appreciated!
>>>>>
>>>>> Have you tried my suggestions from the other thread[*]?
>>> Firstly, I'm sorry for sending muliple mails with the same content. I thought the mails I sent previously
>>> didn't be sent successfully.
>>> And let's talk the problem here.
>>>>>
>>>>> : > > I'm curious how this happend. I cannot find any condition that would
>>>>> : > > cause the int3 instruction generate a #UD according to the AMD's spec.
>>>>> :
>>>>> : One possibility is that the value from memory that gets executed diverges from the
>>>>> : value that is read out be the #UD handler, e.g. due to patching (doesn't seem to
>>>>> : be the case in this test), stale cache/tlb entries, etc.
>>>>> :
>>>>> : > > BTW, it worked nomarlly with qemu and ovmf.
>>>>> : >
>>>>> : > Does this happen every time you boot the guest with your firmware? What
>>>>> : > processor are you running on?
>>>>> :
>>> Yes, every time.
>>> The processor I used is EPYC 7T83.
>>>>> : And have you ruled out KVM as the culprit? I.e. verified that KVM is NOT injecting
>>>>> : a #UD. That obviously shouldn't happen, but it should be easy to check via KVM
>>>>> : tracepoints.
>>>>
>>>> I have a feeling that KVM is injecting the #UD, but it will take instrumenting KVM to see which path the #UD is being injected from.
>>>>
>>>> Wu Zongyo, can you add some instrumentation to figure that out if the trace points towards KVM injecting the #UD?
>>> Ok, I will try to do that.
>> You're right. The #UD is injected by KVM.
>>
>> The path I found is:
>> svm_vcpu_run
>> svm_complete_interrupts
>> kvm_requeue_exception // vector = 3
>> kvm_make_request
>>
>> vcpu_enter_guest
>> kvm_check_and_inject_events
>> svm_inject_exception
>> svm_update_soft_interrupt_rip
>> __svm_skip_emulated_instruction
>> x86_emulate_instruction
>> svm_can_emulate_instruction
>> kvm_queue_exception(vcpu, UD_VECTOR)
>>
>> Does this mean a #PF intercept occur when the guest try to deliver a
>> #BP through the IDT? But why?
>
> I doubt it's a #PF. A #NPF is much more likely, though it could be something
> else entirely, but I'm pretty sure that would require bugs in both the host and
> guest.
>
> What is the last exit recorded by trace_kvm_exit() before the #UD is injected?

I'm guessing it was a #NPF, too. Could it be related to the changes that
went in around svm_update_soft_interrupt_rip()?

6ef88d6e36c2 ("KVM: SVM: Re-inject INT3/INTO instead of retrying the instruction")

Before this the !nrips check would prevent the call into
svm_skip_emulated_instruction(). But now, there is a call to:

svm_update_soft_interrupt_rip()
__svm_skip_emulated_instruction()
kvm_emulate_instruction()
x86_emulate_instruction() (passed a NULL insn pointer)
kvm_can_emulate_insn() (passed a NULL insn pointer)
svm_can_emulate_instruction() (passed NULL insn pointer)

Because it is an SEV guest, it ends up in the "if (unlikely(!insn))" path
and injects the #UD.

Thanks,
Tom

\
 
 \ /
  Last update: 2023-08-02 16:26    [W:0.059 / U:1.240 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site