lkml.org 
[lkml]   [2019]   [Aug]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: [PATCH] x86/mm: Do not split_large_page() for set_kernel_text_rw()
Date
Cc: Steven Rostedt and Suresh Siddha

Hi Peter,

> On Aug 23, 2019, at 2:36 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Aug 22, 2019 at 10:23:35PM -0700, Song Liu wrote:
>> As 4k pages check was removed from cpa [1], set_kernel_text_rw() leads to
>> split_large_page() for all kernel text pages. This means a single kprobe
>> will put all kernel text in 4k pages:
>>
>> root@ ~# grep ffff81000000- /sys/kernel/debug/page_tables/kernel
>> 0xffffffff81000000-0xffffffff82400000 20M ro PSE x pmd
>>
>> root@ ~# echo ONE_KPROBE >> /sys/kernel/debug/tracing/kprobe_events
>> root@ ~# echo 1 > /sys/kernel/debug/tracing/events/kprobes/enable
>>
>> root@ ~# grep ffff81000000- /sys/kernel/debug/page_tables/kernel
>> 0xffffffff81000000-0xffffffff82400000 20M ro x pte
>>
>> To fix this issue, introduce CPA_FLIP_TEXT_RW to bypass "Text RO" check
>> in static_protections().
>>
>> Two helper functions set_text_rw() and set_text_ro() are added to flip
>> _PAGE_RW bit for kernel text.
>>
>> [1] commit 585948f4f695 ("x86/mm/cpa: Avoid the 4k pages check completely")
>
> ARGH; so this is because ftrace flips the whole kernel range to RW and
> back for giggles? I'm thinking _that_ is a bug, it's a clear W^X
> violation.

Thanks for your comments. Yes, it is related to ftrace, as we have
CONFIG_KPROBES_ON_FTRACE. However, after digging around, I am not sure
what is the expected behavior.

Kernel text region has two mappings to it. For x86_64 and four-level
page table, there are:

1. kernel identity mapping, from 0xffff888000100000;
2. kernel text mapping, from 0xffffffff81000000,

Per comments in arch/x86/mm/init_64.c:set_kernel_text_rw():

/*
* Make the kernel identity mapping for text RW. Kernel text
* mapping will always be RO. Refer to the comment in
* static_protections() in pageattr.c
*/
set_memory_rw(start, (end - start) >> PAGE_SHIFT);

kprobe (with CONFIG_KPROBES_ON_FTRACE) should work on kernel identity
mapping.

However, my experiment shows that kprobe actually operates on the
kernel text mapping (0xffffffff81000000-). It is the same w/ and w/o
CONFIG_KPROBES_ON_FTRACE. Therefore, I am not sure whether the comment
is out-dated (10-year old), or the kprobe is doing something wrong.


More information about the issue we are looking at.

We found with 5.2 kernel (no CONFIG_PAGE_TABLE_ISOLATION, w/
CONFIG_KPROBES_ON_FTRACE), a single kprobe will split _all_ PMDs in
kernel text mapping into pte-mapped pages. This increases iTLB
miss rate from about 300 per million instructions to about 700 per
million instructions (for the application I test with).

Per bisect, we found this behavior happens after commit 585948f4f695
("x86/mm/cpa: Avoid the 4k pages check completely"). That's why I
proposed this PATCH to fix/workaround this issue. However, per
Peter's comment and my study of the code, this doesn't seem the
real problem or the only here.

I also tested that the PMD split issue doesn't happen w/o
CONFIG_KPROBES_ON_FTRACE.


In summary, I have the following questions:

1. Which mapping should kprobe work on? Kernel identity mapping or
kernel text mapping?
2. FTRACE causes split of PMD mapped kernel text. How should we fix
this?

Thanks,
Song



\
 
 \ /
  Last update: 2019-08-26 06:45    [W:1.418 / U:0.136 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site