Messages in this thread | ![/](/images/icornerl.gif) | | Date | Thu, 8 Jun 2023 18:59:43 -0600 | Subject | kvm/x86: multichase benchmark | From | Yu Zhao <> |
| |
TLDR ==== Multichase in 64 microVMs achieved 6% more total samples (in ~4 hours) after this patchset [1].
Hardware ======== HOST $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 43 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Vendor ID: AuthenticAMD Model name: AMD Ryzen Threadripper PRO 3995WX 64-Cores CPU family: 23 Model: 49 Thread(s) per core: 2 Core(s) per socket: 64 Socket(s): 1 Stepping: 0 Frequency boost: disabled CPU max MHz: 4308.3979 CPU min MHz: 2200.0000 BogoMIPS: 5390.20 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ... Virtualization features: Virtualization: AMD-V Caches (sum of all): L1d: 2 MiB (64 instances) L1i: 2 MiB (64 instances) L2: 32 MiB (64 instances) L3: 256 MiB (16 instances) NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-127 Vulnerabilities: Itlb multihit: Not affected L1tf: Not affected Mds: Not affected Meltdown: Not affected Mmio stale data: Not affected Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected Srbds: Not affected Tsx async abort: Not affected
HOST $ numactl -H available: 1 nodes (0) node 0 cpus: 0-127 node 0 size: 257542 MB node 0 free: 224855 MB node distances: node 0 0: 10
HOST $ cat /sys/class/nvme/nvme0/model INTEL SSDPF21Q800GB
HOST $ cat /sys/class/nvme/nvme0/numa_node 0
Software ======== HOST $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"
HOST $ uname -a Linux x86 6.4.0-rc5+ #1 SMP PREEMPT_DYNAMIC Wed Jun 7 22:17:47 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
HOST $ cat /proc/swaps Filename Type Size Used Priority /dev/nvme0n1p2 partition 466838356 0 -2
HOST $ cat /sys/kernel/mm/lru_gen/enabled 0x000f
HOST $ cat /sys/kernel/mm/transparent_hugepage/enabled always madvise [never]
HOST $ cat /sys/kernel/mm/transparent_hugepage/defrag always defer defer+madvise madvise [never]
Procedure ========= HOST $ git clone https://github.com/google/multichase
HOST $ <Build multichase> HOST $ <Unpack /boot/initrd.img into ./initrd/>
HOST $ cp multichase/multichase ./initrd/bin/ HOST $ sed -i \ "/^maybe_break top$/i multichase -t 2 -m 4g -n 28800; poweroff" \ ./initrd/init
HOST $ <Pack ./initrd/ into ./initrd.img>
HOST $ cat run_microvms.sh memcgs=64
run() { path=/sys/fs/cgroup/memcg$1
mkdir $path echo $BASHPID >$path/cgroup.procs
qemu-system-x86_64 -M microvm,accel=kvm -cpu host -smp 2 -m 6g \ -nographic -kernel /boot/vmlinuz -initrd ./initrd.img \ -append "console=ttyS0 loglevel=0" }
for ((memcg = 0; memcg < $memcgs; memcg++)); do run $memcg & done
wait
Results ======= Before [1] After Change ---------------------------------------------- Total samples 6824 7237 +6%
Notes ===== [1] "mm: rmap: Don't flush TLB after checking PTE young for page reference" was included so that the comparison is apples to Apples. https://lore.kernel.org/r/20220706112041.3831-1-21cnbao@gmail.com/
| ![\](/images/icornerr.gif) |