Messages in this thread | | | Date | Thu, 23 Aug 2012 09:50:43 -0500 | From | Jacob Shin <> | Subject | Re: [PATCH 3/4] x86: Only direct map addresses that are marked as E820_RAM |
| |
On Wed, Aug 22, 2012 at 04:30:49PM -0700, H. Peter Anvin wrote: > On 08/14/2012 03:39 PM, Jacob Shin wrote: > > Currently direct mappings are created for [ 0 to max_low_pfn<<PAGE_SHIFT ) > > and [ 4GB to max_pfn<<PAGE_SHIFT ), which may include regions that are not > > backed by actual DRAM. This is fine for holes under 4GB which are covered > > by fixed and variable range MTRRs to be UC. However, we run into trouble > > on higher memory addresses which cannot be covered by MTRRs. > > > > Our system with 1TB of RAM has an e820 that looks like this: > > > > BIOS-e820: [mem 0x0000000000000000-0x00000000000983ff] usable > > BIOS-e820: [mem 0x0000000000098400-0x000000000009ffff] reserved > > BIOS-e820: [mem 0x00000000000d0000-0x00000000000fffff] reserved > > BIOS-e820: [mem 0x0000000000100000-0x00000000c7ebffff] usable > > BIOS-e820: [mem 0x00000000c7ec0000-0x00000000c7ed7fff] ACPI data > > BIOS-e820: [mem 0x00000000c7ed8000-0x00000000c7ed9fff] ACPI NVS > > BIOS-e820: [mem 0x00000000c7eda000-0x00000000c7ffffff] reserved > > BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved > > BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved > > BIOS-e820: [mem 0x00000000fff00000-0x00000000ffffffff] reserved > > BIOS-e820: [mem 0x0000000100000000-0x000000e037ffffff] usable > > BIOS-e820: [mem 0x000000e038000000-0x000000fcffffffff] reserved > > BIOS-e820: [mem 0x0000010000000000-0x0000011ffeffffff] usable > > > > and so direct mappings are created for huge memory hole between > > 0x000000e038000000 to 0x0000010000000000. Even though the kernel never > > generates memory accesses in that region, since the page tables mark > > them incorrectly as being WB, our (AMD) processor ends up causing a MCE > > while doing some memory bookkeeping/optimizations around that area. > > > > This patch iterates through e820 and only direct maps ranges that are > > marked as E820_RAM, and keeps track of those pfn ranges. Depending on > > the alignment of E820 ranges, this may possibly result in using smaller > > size (i.e. 4K instead of 2M or 1G) page tables. > > > > Signed-off-by: Jacob Shin <jacob.shin@amd.com> > > I have one concern with this, which is that it leaves in place mapping > below the initial max_pfn_mapped. Although that neatly resolves the > legacy area (0-1 MiB) issues, it really isn't right above the 1 MiB > point. Any way I could get you to seek out and unmap any such ranges? > We have already seen some Dell machines which put memory holes in low > RAM, and perhaps there are still some machines out there with an I/O > hole at 15 MiB.
So I believe in V2 of the patchset this was done, however, Dave Young from redhat reported that it broke their KVM guest with a user supplied memory map that looked like this:
>> [ 0.000000] e820: user-defined physical RAM map: >> [ 0.000000] user: [mem 0x0000000000010000-0x000000000009dbff] usable >> [ 0.000000] user: [mem 0x0000000024000000-0x0000000033f6bfff] usable
And looking into that scenario, the early boot code seems to allocates space for fixmap right under initial max_pfn_mapped, which is no longer direct mapped with my patch, and that seems to cause problems for later APIC code that initializes APIC base address into the fixmap area.
So I guess to address your concern, we need to go back to V2 and try to resolve the fixmap problem with user supplied memory map that reserves memory below initial max_pfn_mapped ?
> > -hpa > >
| |