[lkml]   [2020]   [Nov]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Patch in this message
Subject[PATCH] misc/vmw_vmci: bail out earlier on too big queue allocation
From: Norbert Slusarek <>
Date: Sun, 22 Nov 2020 19:16:41 +0100
Subject: [PATCH] misc/vmw_vmci: bail out earlier on too big queue allocation

For the allocation of a queue pair in qp_host_alloc_queue() an arbitrary value can be
passed for produce_size, which can lead to miscalculation of memory we'd like to allocate
in one take. A warning is triggered at __alloc_pages_nodemask() in mm/page_alloc.c:4737
which aborts the false allocation, but it results in a VMware machine freezing
for an infinite time.

Signed-off-by: Norbert Slusarek <>


To provide an accurate explanation of the problem, I'll describe my observations and
include a PoC you can run yourself.
The value for produce_size (0x7ffe7001) wasn't chosen randomly, it's the least value
which can trigger the warning. With this value involved, calculations are made in
qp_host_alloc_queue() before calling kzalloc() with queue_size + queue_page_size.
The calculation of queue_size involves taking the size of *queue->kernel_if, which on 5.10-rc4 has the size of 72 bytes and on 5.4.79 it's 168 bytes.
While on 5.10-rc4 the size argument for kzalloc() won't surpass
the maximum value of 4096*1024 for an individual allocation
(for produce_size = 0x7ffe7001 -> kzalloc(4194216)), it will be greater on the stable 5.4.79 kernel
(for produce_size = 0x7ffe7001 -> kzalloc(4194312)).
This will ultimately lead to a warning on the stable 5.4 kernel, but not on the upstream kernel,
so ideally my patch would be backported to stable kernels.
Eventhough the warning in __alloc_pages_nodemask() already aborts the oversized allocation of memory, VMware will hang for an infinite time, hence I wanted to provide this simple patch. We shouldn't rely on the page allocator to abort it anyways, it's better to keep it clean and check for
a too large allocation before calling kzalloc().
When I ran the PoC in QEMU and on a host machine, I didn't experience any freezes at all, but
the warning gets triggered.

PoC (run on 5.4 stable kernel and VMCI driver loaded for /dev/vmci):

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <endian.h>
#include <stdint.h>
#include <string.h>
#include <sys/syscall.h>
#include <sys/types.h>

#define IOCTL_VMCI_VERSION2 1959

struct vmci_handle {
unsigned int context;
unsigned int resource;

struct vmci_qp_alloc_info {
struct vmci_handle handle;
unsigned int peer;
unsigned int flags;
unsigned long produce_size;
unsigned long consume_size;
unsigned long ppn_va;
unsigned long num_ppns;
int result;
unsigned int version;

struct vmci_init_blk {
int cid;
int flags;

int main(void)
int fd, flag;

fd = syscall(__NR_openat, -100, "/dev/vmci", O_RDWR, 0);

syscall(__NR_ioctl, fd, IOCTL_VMCI_VERSION2, &flag);

struct vmci_init_blk cxt = {
syscall(__NR_ioctl, fd, IOCTL_VMCI_INIT_CONTEXT, &cxt);

struct vmci_qp_alloc_info qp = {
.handle.context = VMADDR_CID_LOCAL,
.handle.resource = 0,
.peer = 0,
.flags = 0,
.produce_size = 0x7ffe7001,
.consume_size = 0,
.ppn_va = 0,
.num_ppns = 0,
.result = -1,
.version = 0
syscall(__NR_ioctl, fd, IOCTL_VMCI_QUEUEPAIR_ALLOC, &qp);

return 0;

drivers/misc/vmw_vmci/vmci_queue_pair.c | 5 +++++
1 file changed, 5 insertions(+)

diff --git a/drivers/misc/vmw_vmci/vmci_queue_pair.c b/drivers/misc/vmw_vmci/vmci_queue_pair.c
index c49065887e8f..997ff32475b2 100644
--- a/drivers/misc/vmw_vmci/vmci_queue_pair.c
+++ b/drivers/misc/vmw_vmci/vmci_queue_pair.c
@@ -526,6 +526,7 @@ static struct vmci_queue *qp_host_alloc_queue(u64 size)
struct vmci_queue *queue;
size_t queue_page_size;
u64 num_pages;
+ unsigned int order;
const size_t queue_size = sizeof(*queue) + sizeof(*(queue->kernel_if));

if (size > SIZE_MAX - PAGE_SIZE)
@@ -537,6 +538,10 @@ static struct vmci_queue *qp_host_alloc_queue(u64 size)

queue_page_size = num_pages * sizeof(*queue->kernel_if->;

+ order = get_order(queue_size + queue_page_size);
+ if (order >= MAX_ORDER)
+ return NULL;
queue = kzalloc(queue_size + queue_page_size, GFP_KERNEL);
if (queue) {
queue->q_header = NULL;
 \ /
  Last update: 2020-11-22 18:45    [W:0.027 / U:7.288 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site