Messages in this thread Patches in this message | | | Subject | Re: [PATCH v3 00/17] KVM: x86/pmu: Add support to enable Guest PEBS via DS | From | "Xu, Like" <> | Date | Tue, 26 Jan 2021 15:08:09 +0800 |
| |
On 2021/1/25 22:47, Liuxiangdong (Aven, Cloud Infrastructure Service Product Dept.) wrote: > Thanks for replying, > > On 2021/1/25 10:41, Like Xu wrote: >> + kvm@vger.kernel.org >> >> Hi Liuxiangdong, >> >> On 2021/1/22 18:02, Liuxiangdong (Aven, Cloud Infrastructure Service >> Product Dept.) wrote: >>> Hi Like, >>> >>> Some questions about >>> https://lore.kernel.org/kvm/20210104131542.495413-1-like.xu@linux.intel.com/ >>> <https://lore.kernel.org/kvm/20210104131542.495413-1-like.xu@linux.intel.com/> >>> >> >> Thanks for trying the PEBS feature in the guest, >> and I assume you have correctly applied the QEMU patches for guest PEBS. >> > Is there any other patch that needs to be apply? I use qemu 5.2.0. > (download from github on January 14th)
Two qemu patches are attached against qemu tree (commit 31ee895047bdcf7387e3570cbd2a473c6f744b08) and then run the guest with "-cpu,pebs=true".
Note, this two patch are just for test and not finalized for qemu upstream.
> >>> 1)Test in IceLake >> >> In the [PATCH v3 10/17] KVM: x86/pmu: Expose CPUIDs feature bits PDCM, >> DS, DTES64, we only support Ice Lake with the following x86_model(s): >> >> #define INTEL_FAM6_ICELAKE_X 0x6A >> #define INTEL_FAM6_ICELAKE_D 0x6C >> >> you can check the eax output of "cpuid -l 1 -1 -r", >> for example "0x000606a4" meets this requirement. > It's INTEL_FAM6_ICELAKE_X
Yes, it's the target hardware.
> cpuid -l 1 -1 -r > > CPU: > 0x00000001 0x00: eax=0x000606a6 ebx=0xb4800800 ecx=0x7ffefbf7 > edx=0xbfebfbff > >>> >>> HOST: >>> >>> CPU family: 6 >>> >>> Model: 106 >>> >>> Model name: Intel(R) Xeon(R) Platinum 8378A CPU $@ $@ >>> >>> microcode: sig=0x606a6, pf=0x1, revision=0xd000122 >> >> As long as you get the latest BIOS from the provider, >> you may check 'cat /proc/cpuinfo | grep code | uniq' with the latest one. > OK. I'll do it later. >> >>> >>> Guest: linux kernel 5.11.0-rc2 >> >> I assume it's the "upstream tag v5.11-rc2" which is fine. > Yes. >> >>> >>> We can find pebs/intel_pt flag in guest cpuinfo, but there still exists >>> error when we use perf >> >> Just a note, intel_pt and pebs are two features and we can write >> pebs records to intel_pt buffer with extra hardware support. >> (by default, pebs records are written to the pebs buffer) >> >> You may check the output of "dmesg | grep PEBS" in the guest >> to see if the guest PEBS cpuinfo is exposed and use "perf record >> –e cycles:pp" to see if PEBS feature actually works in the guest. > > I apply only pebs patch set to linux kernel 5.11.0-rc2, test perf in > guest and dump stack when return -EOPNOTSUPP
Yes, you may apply the qemu patches and try it again.
> > (1) > # perf record -e instructions:pp > Error: > instructions:pp: PMU Hardware doesn't support > sampling/overflow-interrupts. Try 'perf stat' > > [ 117.793266] Call Trace: > [ 117.793270] dump_stack+0x57/0x6a > [ 117.793275] intel_pmu_setup_lbr_filter+0x137/0x190 > [ 117.793280] intel_pmu_hw_config+0x18b/0x320 > [ 117.793288] hsw_hw_config+0xe/0xa0 > [ 117.793290] x86_pmu_event_init+0x8e/0x210 > [ 117.793293] perf_try_init_event+0x40/0x130 > [ 117.793297] perf_event_alloc.part.22+0x611/0xde0 > [ 117.793299] ? alloc_fd+0xba/0x180 > [ 117.793302] __do_sys_perf_event_open+0x1bd/0xd90 > [ 117.793305] do_syscall_64+0x33/0x40 > [ 117.793308] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > Do we need lbr when we use pebs?
No, lbr ane pebs are two features and we enable it separately.
> > I tried to apply lbr patch > set(https://lore.kernel.org/kvm/911adb63-ba05-ea93-c038-1c09cff15eda@intel.com/) > to kernel and qemu, but there is still other problem. > Error: > The sys_perf_event_open() syscall returned with 22 (Invalid argument) for > event > ...
We don't need that patch for PEBS feature.
> > (2) > # perf record -e instructions:ppp > Error: > instructions:ppp: PMU Hardware doesn't support > sampling/overflow-interrupts. Try 'perf stat' > > [ 115.188498] Call Trace: > [ 115.188503] dump_stack+0x57/0x6a > [ 115.188509] x86_pmu_hw_config+0x1eb/0x220 > [ 115.188515] intel_pmu_hw_config+0x13/0x320 > [ 115.188519] hsw_hw_config+0xe/0xa0 > [ 115.188521] x86_pmu_event_init+0x8e/0x210 > [ 115.188524] perf_try_init_event+0x40/0x130 > [ 115.188528] perf_event_alloc.part.22+0x611/0xde0 > [ 115.188530] ? alloc_fd+0xba/0x180 > [ 115.188534] __do_sys_perf_event_open+0x1bd/0xd90 > [ 115.188538] do_syscall_64+0x33/0x40 > [ 115.188541] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > This is beacuse x86_pmu.intel_cap.pebs_format is always 0 in > x86_pmu_max_precise(). > > We rdmsr MSR_IA32_PERF_CAPABILITIES(0x00000345) from HOST, it's f4c5. > From guest, it's 2000 > >>> >>> # perf record –e cycles:pp >>> >>> Error: >>> >>> cycles:pp: PMU Hardware doesn’t support sampling/overflow-interrupts. >>> Try ‘perf stat’ >>> >>> Could you give some advice? >> >> If you have more specific comments or any concerns, just let me know. >> >>> >>> 2)Test in Skylake >>> >>> HOST: >>> >>> CPU family: 6 >>> >>> Model: 85 >>> >>> Model name: Intel(R) Xeon(R) Gold 6146 CPU @ >>> >>> 3.20GHz >>> >>> microcode : 0x2000064 >>> >>> Guest: linux 4.18 >>> >>> we cannot find intel_pt flag in guest cpuinfo because >>> cpu_has_vmx_intel_pt() return false. >> >> You may check vmx_pebs_supported(). > It's true. >> >>> >>> SECONDARY_EXEC_PT_USE_GPA/VM_EXIT_CLEAR_IA32_RTIT_CTL/VM_ENTRY_LOAD_IA32_RTIT_CTL >>> are both disable. >>> >>> Is it because microcode is not supported? >>> >>> And, isthere a new macrocode which can support these bits? How can we >>> get this? >> >> Currently, this patch set doesn't support guest PEBS on the Skylake >> platforms, and if we choose to support it, we will let you know. >> > And now, we want to use pebs in skylake. If we develop based on pebs > patch set, do you have any suggestions?
- At least you need to pin guest memory such as "-overcommit mem-lock=true" for qemu - You may rewrite the patches 13 - 17 for Skylake specific because the records format is different with Ice Lake.
> I think microcode requirements need to be satisfied. Can we use > https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files ?
You may try it at your risk and again, this patch set doesn't support guest PEBS on the Skylake platforms currently.
> >> --- >> thx,likexu >> >>> >>> Thanks, >>> >>> Liuxiangdong >>> >> > Thanks. Liuxiangdong >
From 24a04b800d24e3b493e5094f88649402923147a2 Mon Sep 17 00:00:00 2001 From: Like Xu <like.xu@linux.intel.com> Date: Fri, 4 Sep 2020 10:19:27 +0800 Subject: [PATCH 1/2] target/i386: Expose PEBS capabilities in the FEAT_PERF_CAPABILITIES
The IA32_PERF_CAPABILITIES MSR provides enumeration of a variety of PEBS feature interfaces:
- PEBSTrap[6]: Trap/Fault-like indicator of PEBS recording assist; - PEBSArchRegs[7]: Indicator of PEBS assist save architectural registers; - PEBS_FMT[bits 11:8]: Specifies the encoding of the layout of PEBS records; - PEBS_BASELINE [bit 14]: If set, the following is true: (1) Extended PEBS is supported. All counters support the PEBS facility, and all events can generate PEBS records when PEBS is enabled. (2) Adaptive PEBS is supported. The PEBS_DATA_CFG MSR and adaptive record enable bits are supported.
Signed-off-by: Like Xu <like.xu@linux.intel.com> --- target/i386/cpu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 72a79e6019..14262c7bf7 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -1136,9 +1136,9 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { .type = MSR_FEATURE_WORD, .feat_names = { NULL, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, - NULL, "full-width-write", NULL, NULL, + NULL, NULL, "pebs-trap", "pebs-arch-reg", + "pebs-fmt-0", "pebs-fmt-1", "pebs-fmt-2", "pebs-fmt-3", + NULL, "full-width-write", "pebs-baseline", NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, -- 2.29.2 From be5246694aaf2132396ee0b907e679f5c9ccd089 Mon Sep 17 00:00:00 2001 From: Like Xu <like.xu@linux.intel.com> Date: Fri, 4 Sep 2020 10:42:28 +0800 Subject: [PATCH 2/2] target/i386: add -cpu,pebs=true support to enable guest PEBS
The PEBS feature would be enabled on the guest if: - the KVM is enabled and the PMU is enabled and, - the msr-based-feature IA32_PERF_CAPABILITIES is supporterd and, - the supported returned value for PEBS from this msr is not zero.
The PEBS feature would be disabled on the guest if: - the msr-based-feature IA32_PERF_CAPABILITIES is unsupporterd OR, - qemu set the IA32_PERF_CAPABILITIES msr feature without pebs_fmt values OR, - the requested guest vcpu model doesn't support PDCM.
Signed-off-by: Like Xu <like.xu@linux.intel.com> --- hw/i386/pc.c | 1 + target/i386/cpu.c | 20 ++++++++++++++++++++ target/i386/cpu.h | 7 +++++++ target/i386/kvm/kvm.c | 10 ++++++++++ 4 files changed, 38 insertions(+)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 5458f61d10..8e9c1b7545 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -330,6 +330,7 @@ GlobalProperty pc_compat_1_5[] = { { "Nehalem-" TYPE_X86_CPU, "min-level", "2" }, { "virtio-net-pci", "any_layout", "off" }, { TYPE_X86_CPU, "pmu", "on" }, + { TYPE_X86_CPU, "pebs", "on" }, { "i440FX-pcihost", "short_root_bus", "0" }, { "q35-pcihost", "short_root_bus", "0" }, }; diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 14262c7bf7..9dffc85542 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -4228,6 +4228,12 @@ static bool lmce_supported(void) return !!(mce_cap & MCG_LMCE_P); } +static inline bool lbr_supported(void) +{ + return kvm_enabled() && (kvm_arch_get_supported_msr_feature(kvm_state, + MSR_IA32_PERF_CAPABILITIES) & PERF_CAP_PEBS_FORMAT); +} + #define CPUID_MODEL_ID_SZ 48 /** @@ -4332,6 +4338,9 @@ static void max_x86_cpu_initfn(Object *obj) } object_property_set_bool(OBJECT(cpu), "pmu", true, &error_abort); + if (lbr_supported()) { + object_property_set_bool(OBJECT(cpu), "pebs", true, &error_abort); + } } static const TypeInfo max_x86_cpu_type_info = { @@ -5545,6 +5554,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, } if (!cpu->enable_pmu) { *ecx &= ~CPUID_EXT_PDCM; + if (cpu->enable_pebs) { + warn_report("PEBS is unsupported since guest PMU is disabled."); + exit(1); + } } break; case 2: @@ -6610,6 +6623,12 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp) } } + if (!cpu->max_features && cpu->enable_pebs && + !(env->features[FEAT_1_ECX] & CPUID_EXT_PDCM)) { + warn_report("requested vcpu model doesn't support PDCM for PEBS."); + exit(1); + } + if (cpu->ucode_rev == 0) { /* The default is the same as KVM's. */ if (IS_AMD_CPU(env)) { @@ -7192,6 +7211,7 @@ static Property x86_cpu_properties[] = { #endif DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID), DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false), + DEFINE_PROP_BOOL("pebs", X86CPU, enable_pebs, false), DEFINE_PROP_UINT32("hv-spinlocks", X86CPU, hyperv_spinlock_attempts, HYPERV_SPINLOCK_NEVER_NOTIFY), diff --git a/target/i386/cpu.h b/target/i386/cpu.h index d23a5b340a..eac8d8c68e 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -354,6 +354,12 @@ typedef enum X86Seg { #define ARCH_CAP_TSX_CTRL_MSR (1<<7) #define MSR_IA32_PERF_CAPABILITIES 0x345 +#define PERF_CAP_PEBS_TRAP BIT_ULL(6) +#define PERF_CAP_ARCH_REG BIT_ULL(7) +#define PERF_CAP_PEBS_FORMAT 0xf00 +#define PERF_CAP_PEBS_BASELINE BIT_ULL(14) +#define PERF_CAP_PEBS_MASK (PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG | \ + PERF_CAP_PEBS_FORMAT | PERF_CAP_PEBS_BASELINE) #define MSR_IA32_TSX_CTRL 0x122 #define MSR_IA32_TSCDEADLINE 0x6e0 @@ -1708,6 +1714,7 @@ struct X86CPU { * capabilities) directly to the guest. */ bool enable_pmu; + bool enable_pebs; /* LMCE support can be enabled/disabled via cpu option 'lmce=on/off'. It is * disabled by default to avoid breaking migration between QEMU with diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 6dc1ee052d..8fe1d2feea 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -2705,6 +2705,13 @@ static void kvm_msr_entry_add_perf(X86CPU *cpu, FeatureWordArray f) MSR_IA32_PERF_CAPABILITIES); if (kvm_perf_cap) { + if (!cpu->enable_pebs) { + kvm_perf_cap &= ~PERF_CAP_PEBS_MASK; + } + if (!(kvm_perf_cap & PERF_CAP_PEBS_MASK) && cpu->enable_pebs) { + warn_report("MSR_IA32_PERF_CAPABILITIES reported by KVM does not support PEBS."); + exit(1); + } kvm_msr_entry_add(cpu, MSR_IA32_PERF_CAPABILITIES, kvm_perf_cap & f[FEAT_PERF_CAPABILITIES]); } @@ -2744,6 +2751,9 @@ static void kvm_init_msrs(X86CPU *cpu) if (has_msr_perf_capabs && cpu->enable_pmu) { kvm_msr_entry_add_perf(cpu, env->features); + } else if (!has_msr_perf_capabs && cpu->enable_pebs) { + warn_report("KVM doesn't support MSR_IA32_PERF_CAPABILITIES for PEBS."); + exit(1); } if (has_msr_ucode_rev) { -- 2.29.2
| |