lkml.org 
[lkml]   [2021]   [Apr]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
SubjectRe: [PATCH v8 4/8] mm,memory_hotplug: Allocate memmap from the added memory range
Date
On 16.04.21 12:21, Oscar Salvador wrote:
> Physical memory hotadd has to allocate a memmap (struct page array) for
> the newly added memory section. Currently, alloc_pages_node() is used
> for those allocations.
>
> This has some disadvantages:
> a) an existing memory is consumed for that purpose
> (eg: ~2MB per 128MB memory section on x86_64)
> b) if the whole node is movable then we have off-node struct pages
> which has performance drawbacks.
> c) It might be there are no PMD_ALIGNED chunks so memmap array gets
> populated with base pages.
>
> This can be improved when CONFIG_SPARSEMEM_VMEMMAP is enabled.
>
> Vmemap page tables can map arbitrary memory.
> That means that we can simply use the beginning of each memory section and
> map struct pages there.
> struct pages which back the allocated space then just need to be treated
> carefully.
>
> Implementation wise we will reuse vmem_altmap infrastructure to override
> the default allocator used by __populate_section_memmap.
> Part of the implementation also relies on memory_block structure gaining
> a new field which specifies the number of vmemmap_pages at the beginning.
> This patch also introduces the following functions:
>
> - mhp_init_memmap_on_memory:
> Initializes vmemmap pages by calling move_pfn_range_to_zone(),
> calls kasan_add_zero_shadow(), and onlines as many sections
> as vmemmap pages fully span.
> - mhp_deinit_memmap_on_memory:
> Undoes what mhp_init_memmap_on_memory.
>
> The new function memory_block_online() calls mhp_init_memmap_on_memory() before
> doing the actual online_pages(). Should online_pages() fail, we clean up
> by calling mhp_deinit_memmap_on_memory().
> Adjusting of present_pages is done at the end once we know that online_pages()
> succedeed.
>
> On offline, memory_block_offline() needs to unaccount vmemmap pages from
> present_pages() before calling offline_pages().
> This is necessary because offline_pages() tears down some structures based
> on the fact whether the node or the zone become empty.
> If offline_pages() fails, we account back vmemmap pages.
> If it succeeds, we call mhp_deinit_memmap_on_memory().
>
> Hot-remove:
>
> We need to be careful when removing memory, as adding and
> removing memory needs to be done with the same granularity.
> To check that this assumption is not violated, we check the
> memory range we want to remove and if a) any memory block has
> vmemmap pages and b) the range spans more than a single memory
> block, we scream out loud and refuse to proceed.
>
> If all is good and the range was using memmap on memory (aka vmemmap pages),
> we construct an altmap structure so free_hugepage_table does the right
> thing and calls vmem_altmap_free instead of free_pagetable.
>
> Signed-off-by: Oscar Salvador <osalvador@suse.de>
> ---
> drivers/base/memory.c | 75 ++++++++++++++++--
> include/linux/memory.h | 8 +-
> include/linux/memory_hotplug.h | 17 +++-
> include/linux/memremap.h | 2 +-
> include/linux/mmzone.h | 7 +-
> mm/Kconfig | 5 ++
> mm/memory_hotplug.c | 171 ++++++++++++++++++++++++++++++++++++++---
> mm/sparse.c | 2 -
> 8 files changed, 265 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index f209925a5d4e..179857d53982 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -173,16 +173,76 @@ static int memory_block_online(struct memory_block *mem)
> {
> unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
> unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
> + unsigned long nr_vmemmap_pages = mem->nr_vmemmap_pages;
> + struct zone *zone;
> + int ret;
> +
> + zone = mhp_get_target_zone(start_pfn, nr_pages, mem->nid,
> + mem->online_type);
> +
> + /*
> + * Although vmemmap pages have a different lifecycle than the pages
> + * they describe (they remain until the memory is unplugged), doing
> + * its initialization and accounting at hot-{online,offline} stage

s/its/their/

s/hot-{online,offline}/memory onlining/offlining stage/

> + * simplifies things a lot
> + */
> + if (nr_vmemmap_pages) {
> + ret = mhp_init_memmap_on_memory(start_pfn, nr_vmemmap_pages, zone);
> + if (ret)
> + return ret;
> + }
> +
> + ret = online_pages(start_pfn + nr_vmemmap_pages,
> + nr_pages - nr_vmemmap_pages, zone);
> + if (ret) {
> + if (nr_vmemmap_pages)
> + mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
> + return ret;
> + }
> +
> + /*
> + * Account once onlining succeeded. If the page was unpopulated, it is

s/page/zone/

> + * now already properly populated.
> + */
> + if (nr_vmemmap_pages)
> + adjust_present_page_count(zone, nr_vmemmap_pages);
>
> - return online_pages(start_pfn, nr_pages, mem->online_type, mem->nid);
> + return ret;
> }
>
> static int memory_block_offline(struct memory_block *mem)
> {
> unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
> unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
> + unsigned long nr_vmemmap_pages = mem->nr_vmemmap_pages;
> + struct zone *zone;
> + int ret;
> +
> + zone = page_zone(pfn_to_page(start_pfn));
>
> - return offline_pages(start_pfn, nr_pages);
> + /*
> + * Unaccount before offlining, such that unpopulated zone and kthreads
> + * can properly be torn down in offline_pages().
> + */
> + if (nr_vmemmap_pages)
> + adjust_present_page_count(zone, -nr_vmemmap_pages);
> +
> + ret = offline_pages(start_pfn + nr_vmemmap_pages,
> + nr_pages - nr_vmemmap_pages);
> + if (ret) {
> + /* offline_pages() failed. Account back. */
> + if (nr_vmemmap_pages)
> + adjust_present_page_count(zone, nr_vmemmap_pages);
> + return ret;
> + }
> +
> + /*
> + * Re-adjust present pages if offline_pages() fails.
> + */

That comment is stale. I'd just drop it.

> + if (nr_vmemmap_pages)
> + mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
> +
> + return ret;
> }

[...]

> -static void adjust_present_page_count(struct zone *zone, long nr_pages)
> +/*
> + * This function should only be called by memory_block_{online,offline},
> + * and {online,offline}_pages.
> + */
> +void adjust_present_page_count(struct zone *zone, long nr_pages)
> {
> unsigned long flags;
>
> @@ -839,12 +850,64 @@ static void adjust_present_page_count(struct zone *zone, long nr_pages)
> pgdat_resize_unlock(zone->zone_pgdat, &flags);
> }
>
> -int __ref online_pages(unsigned long pfn, unsigned long nr_pages,
> - int online_type, int nid)
> +struct zone *mhp_get_target_zone(unsigned long pfn, unsigned long nr_pages,
> + int nid, int online_type)
> +{
> + return zone_for_pfn_range(online_type, nid, pfn, nr_pages);
> +}
> +

Oh, you can just use zone_for_pfn_range() directly for now. No need for
mhp_get_target_zone(). Sorry for not realizing this.

> +int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> + struct zone *zone)
> +{
> + unsigned long end_pfn = pfn + nr_pages;
> + int ret;
> +
> + /*
> + * Initialize vmemmap pages with the corresponding node, zone links set.
> + */
> + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE);
> +
> + ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
> + if (ret) {
> + remove_pfn_range_from_zone(zone, pfn, nr_pages);
> + return ret;
> + }

IIRC, we have to add the zero shadow first, before touching the memory.
This is also what mm/memremap.c does.

In mhp_deinit_memmap_on_memory(), you already remove in the proper
(reversed) order :)

> +
> +int __ref online_pages(unsigned long pfn, unsigned long nr_pages, struct zone *zone)
> {
> unsigned long flags;
> - struct zone *zone;
> int need_zonelists_rebuild = 0;
> + int nid;
> int ret;
> struct memory_notify arg;
>
> @@ -860,8 +923,9 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages,
>
> mem_hotplug_begin();
>
> + nid = zone_to_nid(zone);

I'd do that right above

const int nid = zone_to_nid(zone);

[...]

--
Thanks,

David / dhildenb

\
 
 \ /
  Last update: 2021-04-16 12:34    [W:0.061 / U:0.324 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site