Messages in this thread Patch in this message | | | From | Bin Yang <> | Subject | [PATCH] x86/mm: fix cpu stuck issue in __change_page_attr_set_clr | Date | Thu, 28 Jun 2018 10:05:40 +0000 |
| |
This issue can be easily triggered by free_initmem() functuion on x86_64 cpu.
When changing page attr, __change_page_attr_set_clr will call __change_page_attr for every 4K page. And try_preserve_large_page will be called to check whether it needs to split the large page.
If cpu supports "pdpe1gb", kernel will try to use 1G large page. In worst case, it needs to check every 4K page in 1G range in try_preserve_large_page function. If __change_page_attr_set_clr needs to change lots of 4K pages, cpu will be stuck for long time.
This patch try to cache the last address which had been checked just now. If the next address is in same big page, the cache will be used without full range check.
Signed-off-by: Bin Yang <bin.yang@intel.com> --- arch/x86/mm/pageattr.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 3bded76e..b9241ac 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -552,16 +552,20 @@ static int try_preserve_large_page(pte_t *kpte, unsigned long address, struct cpa_data *cpa) { + static unsigned long address_cache; + static unsigned long do_split_cache = 1; unsigned long nextpage_addr, numpages, pmask, psize, addr, pfn, old_pfn; pte_t new_pte, old_pte, *tmp; pgprot_t old_prot, new_prot, req_prot; int i, do_split = 1; enum pg_level level; - if (cpa->force_split) + spin_lock(&pgd_lock); + if (cpa->force_split) { + do_split_cache = 1; return 1; + } - spin_lock(&pgd_lock); /* * Check for races, another CPU might have split this page * up already: @@ -627,13 +631,25 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, new_prot = static_protections(req_prot, address, pfn); + addr = address & pmask; + pfn = old_pfn; + /* + * If an address in same range had been checked just now, re-use the + * cache value without full range check. In the worst case, it needs to + * check every 4K page in 1G range, which causes cpu stuck for long + * time. + */ + if (!do_split_cache && + address_cache >= addr && address_cache < nextpage_addr && + pgprot_val(new_prot) == pgprot_val(old_prot)) { + do_split = do_split_cache; + goto out_unlock; + } /* * We need to check the full range, whether * static_protection() requires a different pgprot for one of * the pages in the range we try to preserve: */ - addr = address & pmask; - pfn = old_pfn; for (i = 0; i < (psize >> PAGE_SHIFT); i++, addr += PAGE_SIZE, pfn++) { pgprot_t chk_prot = static_protections(req_prot, addr, pfn); @@ -670,6 +686,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, } out_unlock: + address_cache = address; + do_split_cache = do_split; spin_unlock(&pgd_lock); return do_split; -- 2.7.4
| |