Messages in this thread | | | Date | Sat, 25 Oct 2008 02:04:55 -0700 | From | "Yinghai Lu" <> | Subject | Re: [Bug] pci allocation resources problems on x86_64 |
| |
On Sat, Oct 25, 2008 at 1:45 AM, Yinghai Lu <yinghai@kernel.org> wrote: > On Fri, Oct 24, 2008 at 1:10 PM, <mathieu.taillefumier@free.fr> wrote: >> Hi, >> >>> Ultimately we need to do better at grabbing space for PCI allocations on x86. >>> I was hoping we'd have some patches in 2.6.28 that would help here, but they >>> weren't ready in time. Can you file a kernel.org bug for this problem with >>> the files attached? I'll try to find time to put together some >>> improvements... >> >> yes sure. I will do that this weekend. I already grab some more information >> about this problem by reading docs. The problem seems related to iommu setup >> since it is not activated when the mem < 3G but activated when the mem>3G. I do >> not understand however how the kernel can report more that 4g of memory when >> there are only 4g installed (So I think that the bios is buggy too). I also >> remarked that the size of the /proc/mem file is not what it is intended to be. >> Another supprizing thing is these lines of the dmesg-not-working file >> >> pci 0000:0a:00.0: BAR 0: got res [0x140000000-0x1400007ff] bus [0x140000000-0x14 >> 00007ff] flags 0x20200 >> pci 0000:0a:00.0: BAR 0: moved to bus [0x140000000-0x1400007ff] flags 0x20200 >> >> it seems that the drivers setup some resources that way behind the available >> memory. > > looks like that kernel do the right thing... > > pci 0000:00:1f.3: BAR 0: got res [0xc2000000-0xc20000ff] bus > [0xc2000000-0xc20000ff] flags 0x20200 > pci 0000:00:1f.3: BAR 0: moved to bus [0xc2000000-0xc20000ff] flags 0x20200 > pci 0000:00:1c.0: PCI bridge, secondary bus 0000:02 > pci 0000:00:1c.0: IO window: 0x2000-0x2fff > pci 0000:00:1c.0: MEM window: 0xf6000000-0xf7ffffff > pci 0000:00:1c.0: PREFETCH window: 0x000000f0000000-0x000000f1ffffff > pci 0000:00:1c.1: PCI bridge, secondary bus 0000:06 > pci 0000:00:1c.1: IO window: 0x3000-0x3fff > pci 0000:00:1c.1: MEM window: 0xf8000000-0xf9ffffff > pci 0000:00:1c.1: PREFETCH window: 0x000000f2000000-0x000000f3ffffff > pci 0000:07:00.0: BAR 6: got res [0xf4000000-0xf401ffff] bus > [0xf4000000-0xf401ffff] flags 0x27200 > pci 0000:00:1c.2: PCI bridge, secondary bus 0000:07 > pci 0000:00:1c.2: IO window: 0x4000-0x4fff > pci 0000:00:1c.2: MEM window: 0xfa000000-0xfbffffff > pci 0000:00:1c.2: PREFETCH window: 0x000000f4000000-0x000000f5ffffff > pci 0000:00:1c.3: PCI bridge, secondary bus 0000:08 > pci 0000:00:1c.3: IO window: 0x5000-0x5fff > pci 0000:00:1c.3: MEM window: 0xc8000000-0xc9ffffff > pci 0000:00:1c.3: PREFETCH window: 0x000000cc000000-0x000000cdffffff > pci 0000:09:04.0: CardBus bridge, secondary bus 0000:0a > pci 0000:09:04.0: IO window: 0x006000-0x0060ff > pci 0000:09:04.0: IO window: 0x006400-0x0064ff > pci 0000:09:04.0: PREFETCH window: 0xc4000000-0xc7ffffff > pci 0000:09:04.0: MEM window: 0x140000000-0x143ffffff > pci 0000:00:1e.0: PCI bridge, secondary bus 0000:09 > pci 0000:00:1e.0: IO window: 0x6000-0x6fff > pci 0000:00:1e.0: MEM window: 0xfc200000-0xfc2fffff > pci 0000:00:1e.0: PREFETCH window: 0x000000c4000000-0x000000c7ffffff > ... > > bus: 00 index 0 io port: [0, ffff] > bus: 00 index 1 mmio: [0, ffffffffffffffff] > bus: 02 index 0 io port: [2000, 2fff] > bus: 02 index 1 mmio: [f6000000, f7ffffff] > bus: 02 index 2 mmio: [f0000000, f1ffffff] > bus: 02 index 3 mmio: [0, 0] > bus: 06 index 0 io port: [3000, 3fff] > bus: 06 index 1 mmio: [f8000000, f9ffffff] > bus: 06 index 2 mmio: [f2000000, f3ffffff] > bus: 06 index 3 mmio: [0, 0] > bus: 07 index 0 io port: [4000, 4fff] > bus: 07 index 1 mmio: [fa000000, fbffffff] > bus: 07 index 2 mmio: [f4000000, f5ffffff] > bus: 07 index 3 mmio: [0, 0] > bus: 08 index 0 io port: [5000, 5fff] > bus: 08 index 1 mmio: [c8000000, c9ffffff] > bus: 08 index 2 mmio: [cc000000, cdffffff] > bus: 08 index 3 mmio: [0, 0] > bus: 09 index 0 io port: [6000, 6fff] > bus: 09 index 1 mmio: [fc200000, fc2fffff] > bus: 09 index 2 mmio: [c4000000, c7ffffff] > bus: 09 index 3 io port: [0, ffff] > bus: 09 index 4 mmio: [0, ffffffffffffffff] > bus: 0a index 0 io port: [6000, 60ff] > bus: 0a index 1 io port: [6400, 64ff] > bus: 0a index 2 mmio: [c4000000, c7ffffff] > bus: 0a index 3 mmio: [140000000, 143ffffff] > > because card bus need 0x4000000 > > resource before that already allocated to 0xce000000 > > and it can not use 0xd000000, because > pci 0000:00:02.0: found [8086/2a02] class 000300 header type 00 > PCI: 0000:00:02.0 reg 10 64bit mmio: [fc000000, fc0fffff] > PCI: 0000:00:02.0 reg 18 32bit mmio: [d0000000, dfffffff] > PCI: 0000:00:02.0 reg 20 io port: [1800, 1807] > > so it has to start from 0x140000000 after the RAM... > > and that range it can not use that... > > only solution could be > > Allocating PCI resources starting at c2000000 (gap: c0000000:20000000) > > about pci_mem_start from 0xc1000000 instead of 0xc200000 > > __init void e820_setup_gap(void) > { > unsigned long gapstart, gapsize, round; > int found; > > gapstart = 0x10000000; > gapsize = 0x400000; > found = e820_search_gap(&gapstart, &gapsize, 0, MAX_GAP_END); > > #ifdef CONFIG_X86_64 > if (!found) { > gapstart = (max_pfn << PAGE_SHIFT) + 1024*1024; > printk(KERN_ERR "PCI: Warning: Cannot find a gap in the 32bit " > "address range\n" > KERN_ERR "PCI: Unassigned devices with 32bit resource " > "registers may break!\n"); > } > #endif > > /* > * See how much we want to round up: start off with > * rounding to the next 1MB area. > */ > round = 0x100000; > while ((gapsize >> 4) > round) > round += round; > /* Fun with two's complement */ > pci_mem_start = (gapstart + round) & -round; > > printk(KERN_INFO > "Allocating PCI resources starting at %lx (gap: %lx:%lx)\n", > pci_mem_start, gapstart, gapsize); > } > > > please check if you can change memhole size in BIOS... if not, we can > have more patch for it....make pci_mem_start more compact... >
it is more worse
pci 0000:00:1c.3: scanning behind bridge, config 080800, pass 0 PCI: Scanning bus 0000:08 PCI: Fixups for bus 0000:08 PCI: bridge 0000:00:1c.3 io port: [5000, 5fff] PCI: bridge 0000:00:1c.3 32bit mmio: [c8000000, c9ffffff] PCI: bridge 0000:00:1c.3 64bit mmio pref: [cc000000, cdffffff] PCI: Bus scan for 0000:08 returning with max=08
that c800000 and cc00000 is allocated by BIOS...
may need to set pci_mem_start to 0xc0000000. and let 0x1f.3 to use 0xca00000
YH
| |