Messages in this thread Patch in this message | | | Date | Thu, 4 Jun 2020 16:56:34 +0200 | Subject | [PATCH v2 1/2] kcov, objtool: Make runtime functions noinstr-compatible | From | Marco Elver <> |
| |
While we lack a compiler attribute to add to noinstr that would disable KCOV, make the KCOV runtime functions return if the caller is in a noinstr section. We then whitelist __sanitizer_cov_*() functions in objtool. __sanitizer_cov_*() cannot safely become safe noinstr functions as-is, as they may fault due to accesses to vmalloc's memory.
Declare write_comp_data() as __always_inline to ensure it is inlined, and reduce stack usage and remove one extra call from the fast-path.
In future, our compilers may provide an attribute to implement __no_sanitize_coverage, which can then be added to noinstr, and the checks added in this patch can be guarded by an #ifdef checking if the compiler has such an attribute or not.
Signed-off-by: Marco Elver <elver@google.com> --- Apply after: https://lkml.kernel.org/r/20200604102241.466509982@infradead.org
v2: * Rewrite based on Peter's and Andrey's feedback -- v1 worked because we got lucky. Let's not rely on luck, as it will be difficult to ensure the same conditions remain true in future.
v1: https://lkml.kernel.org/r/20200604095057.259452-1-elver@google.com
Note: There are a set of KCOV patches from Andrey in -next: https://lkml.kernel.org/r/cover.1585233617.git.andreyknvl@google.com -- Git cleanly merges this patch with those patches, and no merge conflict is expected. --- kernel/kcov.c | 19 +++++++++++++++++-- tools/objtool/check.c | 7 +++++++ 2 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/kernel/kcov.c b/kernel/kcov.c index 8accc9722a81..3329a0fdb868 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -24,6 +24,7 @@ #include <linux/refcount.h> #include <linux/log2.h> #include <asm/setup.h> +#include <asm/sections.h> #define kcov_debug(fmt, ...) pr_debug("%s: " fmt, __func__, ##__VA_ARGS__) @@ -172,6 +173,12 @@ static notrace unsigned long canonicalize_ip(unsigned long ip) return ip; } +static __always_inline bool in_noinstr_section(unsigned long ip) +{ + return (unsigned long)__noinstr_text_start <= ip && + ip < (unsigned long)__noinstr_text_end; +} + /* * Entry point from instrumented code. * This is called once per basic-block/edge. @@ -180,13 +187,18 @@ void notrace __sanitizer_cov_trace_pc(void) { struct task_struct *t; unsigned long *area; - unsigned long ip = canonicalize_ip(_RET_IP_); + unsigned long ip; unsigned long pos; + if (unlikely(in_noinstr_section(_RET_IP_))) + return; + t = current; if (!check_kcov_mode(KCOV_MODE_TRACE_PC, t)) return; + ip = canonicalize_ip(_RET_IP_); + area = t->kcov_area; /* The first 64-bit word is the number of subsequent PCs. */ pos = READ_ONCE(area[0]) + 1; @@ -198,12 +210,15 @@ void notrace __sanitizer_cov_trace_pc(void) EXPORT_SYMBOL(__sanitizer_cov_trace_pc); #ifdef CONFIG_KCOV_ENABLE_COMPARISONS -static void notrace write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip) +static __always_inline void write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip) { struct task_struct *t; u64 *area; u64 count, start_index, end_pos, max_pos; + if (unlikely(in_noinstr_section(ip))) + return; + t = current; if (!check_kcov_mode(KCOV_MODE_TRACE_CMP, t)) return; diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 3e214f879ada..cb208959f560 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2213,6 +2213,13 @@ static inline bool noinstr_call_dest(struct symbol *func) if (!strncmp(func->name, "__ubsan_handle_", 15)) return true; + /* + * The __sanitizer_cov_*() calls include a check if the caller is in the + * noinstr section, and simply return if that is the case. + */ + if (!strncmp(func->name, "__sanitizer_cov_", 16)) + return true; + return false; } -- 2.27.0.rc2.251.g90737beb825-goog
| |