lkml.org 
[lkml]   [2017]   [Mar]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subject[RFC][PATCH 0/4] printk: introduce printing kernel thread
Date
Hello,

RFC

This patch set adds a printk() kernel thread which lets us to
print kernel messages to the console from a non-atomic/schedule-able
context, avoiding different sort of lockups, stalls, etc.


The patch set is in [RFC] stage - it may contain ugly and dumb ideas.
The reason I decided to take a step back is because we've been using
printk kthread in development/production for ~10 months now and there
are some observations from my side, which I want to list/outline.

1) We had a `bool' flag to temporarily disable printk offloading (in suspend,
hibernate, etc.) to address a bug reported by Viresh Kumar [1]. Turned out
that our in-house kernel tweaks/ad hocs in PM code introduce nested sections
that disable/enable printk offloading and thus `bool' flag can re-enable
printk-kthread too early. (it wasn't me who added those PM 'tweaks' :)).

Would there be any objections if we switch to `int' flag and allow nested
printk-kthread enable/disable sections?


2) Just letting you know,
printk kthread changes the behavior of printk in one _corner case_.
The corner case is quite interesting and actually consists of two corner
cases. Suppose on SMP system there is only one CPU that printk()-s a lot,
the rest of CPUs don't lock console_sem and don't printk(). Previously
that printing CPU had been throttling itself (*) because of console drivers
call for every printk():

CPU0

printk("a")
console_unlock()
call_console_drivers("a")

...

printk("z")
console_unlock()
call_console_drivers("z")

* Given that no other CPU locks the console_sem.

With printk kthread the case turns into this one:

CPU0 CPU1

printk("a")
wake_up printk_kthread
... printk_kthread
printk("k") console_unlock()
... call_console_drivers("a")
printk("z") call_console_drivers("b")
call_console_drivers("c")
...


The second 'corner case' part here is that CPU0 may be much faster
than printing CPU, which may result in dropped printk messages.

This all is absolutely possible even with out the printk-kthread.
A single console_lock() call from CPUx will result in exactly the
same condition. So it's not necessarily a regression. But there may
be some scenarios in the kernel that may suffer from this change.
From the top of my head -- sysrq backtrace dump, and, probably, OOM
print out and backtrace dump.

sysrq is potentially even trickier. can we always wake_up() kernel
thread from sysrq? there probably might be cases when we can't rely
on the scheduler.

So may be we would want to export disable/enable printk-kthread offloading
symbols and let people use them in appropriate places.

What do you think?


3) A minor one,
'async printk' is, apparently, a confusing name. people assume that the
opposite to 'asynchronous printk' is 'synchronous printk'. which is not
the case. printk may accidentally be synchronous but in general there is
no way to make it always synchronous on SMP system for all CPUs (well,
except for wrapping every printk() in console_lock()/console_unlock(),
probably), because it's up to the CPU that holds the console_sem to print
kernel messages from console_unlock() at some point. so in some sense it's
asynchronous by design. printk-kthread targets different aims anyway.
but still I would probably prefer to avoid this confusion.



[1] this is from the days when printk-kthread had RT prio
lkml.kernel.org/r/20160714221251.GE3057@ubuntu



= Patches 0001/0002 just introduce the printk kthread, it's not eanbled yet.

= Patch 0003 is incomplete and a bit ugly. I need to think more. It exposes
some printk internals and in general I'm not really happy with it. PM parts,
probably, can be kept within the printk.c. For example, disable printk-kthread
in suspend_console() and enable it in resume_console(). unlike sysrq though.

= Patch 0004 enables printk-kthread.


Sergey Senozhatsky (4):
printk: introduce printing kernel thread
printk: offload printing from wake_up_klogd_work_func()
kernel, power: disable printk_kthread in unsafe places
printk: enable printk offloading

include/linux/console.h | 3 ++
kernel/kexec_core.c | 4 ++
kernel/power/hibernate.c | 8 +++
kernel/power/suspend.c | 4 ++
kernel/printk/printk.c | 133 ++++++++++++++++++++++++++++++++++++++++++++---
5 files changed, 144 insertions(+), 8 deletions(-)

--
2.12.0

\
 
 \ /
  Last update: 2017-03-06 13:53    [W:0.217 / U:0.824 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site