lkml.org 
[lkml]   [2021]   [Dec]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH 5.10 15/99] audit: improve robustness of the audit queue handling
    Date
    From: Paul Moore <paul@paul-moore.com>

    commit f4b3ee3c85551d2d343a3ba159304066523f730f upstream.

    If the audit daemon were ever to get stuck in a stopped state the
    kernel's kauditd_thread() could get blocked attempting to send audit
    records to the userspace audit daemon. With the kernel thread
    blocked it is possible that the audit queue could grow unbounded as
    certain audit record generating events must be exempt from the queue
    limits else the system enter a deadlock state.

    This patch resolves this problem by lowering the kernel thread's
    socket sending timeout from MAX_SCHEDULE_TIMEOUT to HZ/10 and tweaks
    the kauditd_send_queue() function to better manage the various audit
    queues when connection problems occur between the kernel and the
    audit daemon. With this patch, the backlog may temporarily grow
    beyond the defined limits when the audit daemon is stopped and the
    system is under heavy audit pressure, but kauditd_thread() will
    continue to make progress and drain the queues as it would for other
    connection problems. For example, with the audit daemon put into a
    stopped state and the system configured to audit every syscall it
    was still possible to shutdown the system without a kernel panic,
    deadlock, etc.; granted, the system was slow to shutdown but that is
    to be expected given the extreme pressure of recording every syscall.

    The timeout value of HZ/10 was chosen primarily through
    experimentation and this developer's "gut feeling". There is likely
    no one perfect value, but as this scenario is limited in scope (root
    privileges would be needed to send SIGSTOP to the audit daemon), it
    is likely not worth exposing this as a tunable at present. This can
    always be done at a later date if it proves necessary.

    Cc: stable@vger.kernel.org
    Fixes: 5b52330bbfe63 ("audit: fix auditd/kernel connection state tracking")
    Reported-by: Gaosheng Cui <cuigaosheng1@huawei.com>
    Tested-by: Gaosheng Cui <cuigaosheng1@huawei.com>
    Reviewed-by: Richard Guy Briggs <rgb@redhat.com>
    Signed-off-by: Paul Moore <paul@paul-moore.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    ---
    kernel/audit.c | 21 ++++++++++-----------
    1 file changed, 10 insertions(+), 11 deletions(-)

    --- a/kernel/audit.c
    +++ b/kernel/audit.c
    @@ -718,7 +718,7 @@ static int kauditd_send_queue(struct soc
    {
    int rc = 0;
    struct sk_buff *skb;
    - static unsigned int failed = 0;
    + unsigned int failed = 0;

    /* NOTE: kauditd_thread takes care of all our locking, we just use
    * the netlink info passed to us (e.g. sk and portid) */
    @@ -735,32 +735,30 @@ static int kauditd_send_queue(struct soc
    continue;
    }

    +retry:
    /* grab an extra skb reference in case of error */
    skb_get(skb);
    rc = netlink_unicast(sk, skb, portid, 0);
    if (rc < 0) {
    - /* fatal failure for our queue flush attempt? */
    + /* send failed - try a few times unless fatal error */
    if (++failed >= retry_limit ||
    rc == -ECONNREFUSED || rc == -EPERM) {
    - /* yes - error processing for the queue */
    sk = NULL;
    if (err_hook)
    (*err_hook)(skb);
    - if (!skb_hook)
    - goto out;
    - /* keep processing with the skb_hook */
    + if (rc == -EAGAIN)
    + rc = 0;
    + /* continue to drain the queue */
    continue;
    } else
    - /* no - requeue to preserve ordering */
    - skb_queue_head(queue, skb);
    + goto retry;
    } else {
    - /* it worked - drop the extra reference and continue */
    + /* skb sent - drop the extra reference and continue */
    consume_skb(skb);
    failed = 0;
    }
    }

    -out:
    return (rc >= 0 ? 0 : rc);
    }

    @@ -1609,7 +1607,8 @@ static int __net_init audit_net_init(str
    audit_panic("cannot initialize netlink socket in namespace");
    return -ENOMEM;
    }
    - aunet->sk->sk_sndtimeo = MAX_SCHEDULE_TIMEOUT;
    + /* limit the timeout in case auditd is blocked/stopped */
    + aunet->sk->sk_sndtimeo = HZ / 10;

    return 0;
    }

    \
     
     \ /
      Last update: 2021-12-20 15:51    [W:4.048 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site