Messages in this thread Patch in this message | | | From | Peter Xu <> | Subject | [PATCH 1/7] mm/userfaultfd: Make uffd read() wait event exclusive | Date | Tue, 5 Sep 2023 17:42:29 -0400 |
| |
From: Andrea Arcangeli <aarcange@redhat.com>
When a new message is generated for an userfaultfd, instead of waking up all the readers, we can wake up only one exclusive reader to process the event. Waking up >1 readers for 1 message will be a waste of resource, where the rest readers will see nothing again and re-queue.
This should make userfaultfd read() O(1) on wakeups.
Note that queuing on head is intended (rather than tail) to make sure the readers are waked up in LIFO fashion; fairness doesn't matter much here, but caching does.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com> [peterx: modified subjects / commit message] Signed-off-by: Peter Xu <peterx@redhat.com> --- fs/userfaultfd.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 56eaae9dac1a..f7fda7d0c994 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1061,7 +1061,11 @@ static ssize_t userfaultfd_ctx_read(struct userfaultfd_ctx *ctx, int no_wait, /* always take the fd_wqh lock before the fault_pending_wqh lock */ spin_lock_irq(&ctx->fd_wqh.lock); - __add_wait_queue(&ctx->fd_wqh, &wait); + /* + * Only wake up one exclusive reader each time there's an event. + * Paired with wake_up_poll() when e.g. a new page fault msg generated. + */ + __add_wait_queue_exclusive(&ctx->fd_wqh, &wait); for (;;) { set_current_state(TASK_INTERRUPTIBLE); spin_lock(&ctx->fault_pending_wqh.lock); -- 2.41.0
| |