Messages in this thread | | | Date | Tue, 25 Aug 2009 23:10:09 -0700 | Subject | Re: how to get a DMA channel near a process? | From | Dan Williams <> |
| |
On Tue, Aug 25, 2009 at 6:05 AM, Brice Goglin<Brice.Goglin@inria.fr> wrote: > Hello, > > I am playing with DMA engine on a Nehalem box with two X58 chipsets > (Supermicro X8DAH). My understanding is that there are 8 DMA channels on > each chipset, so 8 channels near each processor. Unfortunately, my BIOS > and 2.6.31-rc still wrongly reports the physical location of my devices > (it claims all PCI devices are near the first processor) but I worked > around the problem manually. > > The offloaded copy performance changes a lot depending on whether the > process memory is allocated near the DMA device. So first I would like > to know if DMA channels are allocated near the requesting > processor/process. Then I guess it's possible to read the cpu mask near > a given chan by following chan->dev.device up to the pci device, right? > But is there any way to request a DMA channel near a specific socket or > NUMA node or cpu mask?
If we can determine the topology of which chipset is closer to which cpu package then we can modify the dmaengine channel_table to return "local" channels via dma_find_channel(). The current allocation policy is blind per-cpu round robin such that cpu-0 gets channel-0, cpu-1 gets channel-1... As you pointed out the challenge will be finding a reliable source for the topology information.
-- Dan
| |