Messages in this thread | | | Date | Mon, 4 Nov 2013 12:23:06 -0800 | Subject | Re: Solving M produces N consumers scalability problem | From | Anatol Pomozov <> |
| |
Hi
Thanks for your reply
On Mon, Nov 4, 2013 at 10:53 AM, Andi Kleen <andi@firstfloor.org> wrote: > Anatol Pomozov <anatol.pomozov@gmail.com> writes: >> >> One idea is not to use the spin_lock. It is the 'fair spin_lock' that >> has scalability problems >> http://pdos.csail.mit.edu/papers/linux:lock.pdf Maybe lockless >> datastructures can help here? > > The standard spin lock is already improved. > But better locks just give you a small advantage, they don't > solve the real scaling problem.
Do you know in what version the improvement happened? I use kernel 3.3 and I can backport the changes to my custom kernel to make the situation better.
>> Another idea is avoid global datasctructures but I have a few >> questions here. Let's say we want to use per-CPU lists. But the >> problem is that producers/consumers are not distributed across all >> CPUs. Some CPU might have too many producers, some other might not >> have consumers at all. So we need some kind of migration from hot CPU >> to the cold one. What is the best way to achieve it? Are there any >> examples how to do this? Any other ideas? > > per cpu is the standard approach, but usually overkill. Also > requires complex code to drain etc. > > Some older patches also use per node, but that works very poorly > these days (nodes are far too big) > > One way I like is to simply use a global (allocated) array of queues, > sized by total number of possible cpus (but significantly smaller) and > use the cpu number as a hash into the array.
This solution pretty-much equivalent to per-CPU data structures. And in this case there also will be "hot" nodes and "cold" nodes. So my question remains - what is the best way to implement data migration between nodes, is there a standard solution for this? examples?
| |