lkml.org 
[lkml]   [2019]   [Oct]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subjectmlockall(MCL_CURRENT) blocking infinitely
Date
Hi guys,

I've got an issue with `mlockall(MCL_CURRENT)` after upgrading Ubuntu
19.04 to 19.10 - i.e. kernel version change from 5.0.x to 5.3.x.

The following simple program hangs forever with one CPU running at 100%
(kernel):

#include <stdio.h>
#include <sys/mman.h>
int main(char** argv) {
  printf("Before mlockall(MCL_CURRENT)\n");
  // works in 5.0
  // hangs forever w/ 5.1 and newer
  mlockall(MCL_CURRENT);
  printf("After mlockall(MCL_CURRENT)\n");
}

All kernel versions since 5.1 (tried 5.1.0, 5.1.21, 5.2.21, 5.3.0-19,
5.3.7, 5.4-rc4) show the same symptom (hanging in mlockall(MCL_CURRENT)
with 100% kernel-CPU). 5.0 kernel versions (5.0.21) are fine.

First, I thought, that it's something generic, so I tried the above
program in a fresh install of Ubuntu eoan (5.3.x) in a VM in virtualbox,
but it works fine there. So I suspect, that it has to do with something
that's specific to my machine.

My first suspicion was that some library "hijacks" mlockall(), but
calling the test program with `LD_DEBUG=all` shows that glibc gets
called directly:
     12248:    symbol=mlockall;  lookup in file=./test [0]
     12248:    symbol=mlockall;  lookup in
file=/lib/x86_64-linux-gnu/libc.so.6 [0]
     12248:    binding file ./test [0] to
/lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `mlockall' [GLIBC_2.2.5]
An `strace` doesn't show anything meaningful (beside that mlockall's
been called but never returns). dmesg and syslog don't show anything
obvious (to me) as well.

Some information about the machine:
- Intel(R) Core(TM) i7-6900K, Intel X99 chipset
- NVMe 1.1b
- 64GB RAM (4x 16GB)

I've also reverted all changes for sysctl and ld.conf and checked for
other suspicious software without any luck.

I also tried a bunch of variations of the above program, but only
`mlockall(MCL_CURRENT)` or `mlockall(MCL_FUTURE | MCL_CURRENT)` hang.

A `git diff v5.0..v5.1 mm/` doesn't show anything obvious (to me).

It seems, there's no debug/trace information that would help to find out
what exactly it's doing.

I'm kinda lost at the moment.


PS: Variations of the above test program:

#include <stdio.h>
#include <sys/mman.h>
char foo[65536];
int main(char** argv) {
  printf("Before mlock()\n");
  int e = mlock(foo, 8192); // works in 5.0, 5.1, 5.2, 5.3, 5.4
  printf("After mlock()=%d\n", e);
}


#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
int main(char** argv) {
  printf("Before mlockall(MCL_FUTURE)\n");
  int e = mlockall(MCL_FUTURE); // works in 5.0, 5.1, 5.2, 5.3, 5.4
  printf("After mlockall(MCL_FUTURE) = %d\n", e);
  void* mem = malloc(1024 * 1024 * 1024);
  printf("After malloc()\n");
  mem = malloc(1024 * 1024 * 1024);
  printf("After malloc()\n");
  mem = malloc(1024 * 1024 * 1024);
  printf("After malloc()\n");
  // works in 5.0, 5.1, 5.2, 5.3, 5.4
}


#include <stdio.h>
#include <sys/mman.h>
int main(char** argv) {
  printf("Before munlockall()\n");
  int e = munlockall(); // works in 5.0, 5.1, 5.2, 5.3, 5.4
  printf("After munlockall() = %d\n", e);
}


#include <stdio.h>
#include <sys/mman.h>
int main(char** argv) {
  printf("Before mlockall(MCL_CURRENT|MCL_FUTURE)\n");
  // works in 5.0
  // hangs forever w/ 5.1 and newer
  int e = mlockall(MCL_CURRENT|MCL_FUTURE);
  printf("After mlockall(MCL_CURRENT|MCL_FUTURE) = %d\n", e);
}

PPS: Kernel version images installed from
https://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D

--
Robert Stupp
@snazy

\
 
 \ /
  Last update: 2019-10-24 10:00    [W:0.071 / U:0.096 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site