Messages in this thread | | | Date | Wed, 08 Jan 2014 22:15:54 +0800 | From | Alex Shi <> | Subject | Re: [PATCH 4/4] sched: bias to target cpu load to reduce task moving |
| |
On 01/07/2014 08:59 PM, Peter Zijlstra wrote: > On Tue, Jan 07, 2014 at 12:55:18PM +0000, Morten Rasmussen wrote: >> My understanding is that should_we_balance() decides which cpu is >> eligible for doing the load balancing for a given domain (and the >> domains above). That is, only one cpu in a group is allowed to load >> balance between the local group and other groups. That cpu would >> therefore be reponsible for pulling enough load that the groups are >> balanced even if it means temporarily overloading itself. The other cpus >> in the group will take care of load balancing the extra load within the >> local group later.
Thanks for both of you comments and explanations! :)
I know this patch's change is arguable and my attempt doesn't tune well. But I believe I am in a correct way. :) let me explain a bit for this patch again.
First cpu_load includes the history load info, so repeatedly decay and use the history load is kind of non-sense. and the old source/target_load randomly select history load or current load just according to max/min, it also owe a well explanation. Second, we consider the bias in source/target_load already. but still use imbalance_pct as last check in idlest/busiest group finding. It is also a kind of redundant job. If we can consider the source/target bias, we'd better not use imbalance_pct again. And last, imbalance pct overused with quickly core number increasing cpu. Like in find_busiset_group: Assume a 2 groups domain, each group has 8 cores cpus. The target group will bias 8 * (imbalance_pct -100) = 8 * (125 - 100) = 200. Since each of cpu bias .25 times load, for 8 cpus, totally bias 2 times average cpu load between groups. That is a too much. But if there only 2 cores in cpu group(common case when the code introduced). the bias is just 2 * 25 / 100 = 0.5 times average cpu load.
Now this patchset remove the cpu_load array avoid repeated history decay; reorganize the imbalance_pct usage to avoid redundant balance bias. and reduce the bias value between cpu groups -- maybe it isn't tune well. :)
> > Correct. > >> I may have missed something, but I don't understand the reason for the >> performance improvements that you are reporting. I see better numbers >> for a few benchmarks, but I still don't understand why the code makes >> sense after the cleanup. If we don't understand why it works, we cannot >> be sure that it doesn't harm other benchmarks. There is always a chance >> that we miss something but, IMHO, not having any idea to begin with >> increases the chances for problems later significantly. So why not get >> to the bottom of the problem of cleaning up cpu_load? >> >> Have you done more extensive benchmarking? Have you seen any regressions >> in other benchmarks? > > I only remember hackbench numbers and that generally fares well with a > more aggressive balancer since it has no actual work to speak of the > migration penalty is very low and because there's a metric ton of tasks > the aggressive leveling makes for more coherent 'throughput'.
I just tested hackbench on arm. and with more testing times plus rebase to .13-rc6, the variation increased, then the benefit become unclear. anyway still no regression find on both perf-stat cpu-migration times and real execute time.
On 0day performance testing should tested kbuild, hackbench, aim7, dbench, tbench, sysbench, netperf etc. etc. No regression found.
The 0day performance testing also catch a cpu migration reduced on kvm guest. https://lkml.org/lkml/2013/12/21/135
and another benchmark get benefit on the old patchset: like the testing results show on 0day performance testing:
https://lkml.org/lkml/2013/12/4/102
Hi Alex,
We obsevered 150% performance gain with vm-scalability/300s-mmap-pread-seq testcase with this patch applied. Here is a list of changes we got so far:
testbox : brickland testcase: vm-scalability/300s-mmap-pread-seq
f1b6442c7dd12802e622 d70495ef86f397816d73 (parent commit) (this commit) ------------------------ ------------------------ 26393249.80 +150.9% 66223933.60 vm-scalability.throughput
225.12 -49.9% 112.75 time.elapsed_time 36333.40 -90.7% 3392.20 vmstat.system.cs 2.40 +375.0% 11.40 vmstat.cpu.id 3770081.60 -97.7% 87673.40 time.major_page_faults 3975276.20 -97.0% 117409.60 time.voluntary_context_switches 3.05 +301.7% 12.24 iostat.cpu.idle 21118.41 -70.3% 6277.19 time.system_time 18.40 +130.4% 42.40 vmstat.cpu.us 77.00 -41.3% 45.20 vmstat.cpu.sy 47459.60 -31.3% 32592.20 vmstat.system.in 82435.40 -12.1% 72443.60 time.involuntary_context_switches 5128.13 +14.0% 5848.30 time.user_time 11656.20 -7.8% 10745.60 time.percent_of_cpu_this_job_got 1069997484.80 +0.3% 1073679919.00 time.minor_page_faults
Btw, the latest patchset include more clean up. git@github.com:alexshi/power-scheduling.git noload Guess fengguang's 0day performance is doing test on it.
-- Thanks Alex
| |