Messages in this thread | | | Date | Wed, 10 Jun 2015 10:58:26 +0100 | From | Mel Gorman <> | Subject | Re: [PATCH 2/4] mm: Send one IPI per CPU to TLB flush all entries after unmapping pages |
| |
On Wed, Jun 10, 2015 at 10:26:40AM +0200, Ingo Molnar wrote: > > * Mel Gorman <mgorman@suse.de> wrote: > > > On a 4-socket machine the results were > > > > 4.1.0-rc6 4.1.0-rc6 > > batchdirty-v6 batchunmap-v6 > > Ops lru-file-mmap-read-elapsed 121.27 ( 0.00%) 118.79 ( 2.05%) > > > > 4.1.0-rc6 4.1.0-rc6 > > batchdirty-v6 batchunmap-v6 > > User 620.84 608.48 > > System 4245.35 4152.89 > > Elapsed 122.65 120.15 > > > > In this case the workload completed faster and there was less CPU overhead > > but as it's a NUMA machine there are a lot of factors at play. It's easier > > to quantify on a single socket machine; > > > > 4.1.0-rc6 4.1.0-rc6 > > batchdirty-v6 batchunmap-v6 > > Ops lru-file-mmap-read-elapsed 20.35 ( 0.00%) 21.52 ( -5.75%) > > > > 4.1.0-rc6 4.1.0-rc6 > > batchdirty-v6r5batchunmap-v6r5 > > User 58.02 60.70 > > System 77.57 81.92 > > Elapsed 22.14 23.16 > > > > That shows the workload takes 5.75% longer to complete with a similar > > increase in the system CPU usage. > > Btw., do you have any stddev noise numbers? >
4.1.0-rc6 4.1.0-rc6 4.1.0-rc6 4.1.0-rc6 vanilla flushfull-v6r5 batchdirty-v6r5 batchunmap-v6r5 Ops lru-file-mmap-read-elapsed 25.43 ( 0.00%) 20.59 ( 19.03%) 20.35 ( 19.98%) 21.52 ( 15.38%) Ops lru-file-mmap-read-time_stddv 0.32 ( 0.00%) 0.32 ( -1.30%) 0.39 (-23.00%) 0.45 (-40.91%)
flushfull -- patch 2 batchdirty -- patch 3 batchunmap -- patch 4
So the impact of tracking the PFNs is outside the noise and there is definite direct cost to it. This was expected for both the PFN tracking and the individual flushes.
> The batching speedup is brutal enough to not need any noise estimations, it's a > clear winner. >
Agreed.
> But this PFN tracking patch is more difficult to judge as the numbers are pretty > close to each other. >
It's definitely measurable, no doubt about it and there never was. The concerns were always the refill costs due to flushing potentially active TLB entries unnecessarily. From https://lkml.org/lkml/2014/7/31/825, this is potentially high where it says that a 512 DTLB refill takes 22,000 cycles which is higher than the individual flushes. However, this is an estimate and it'll always be a case of "it depends". It's been asserted that the refill costs are really low so lets just go with that, drop patch 4 and wait and see who complains.
-- Mel Gorman SUSE Labs
| |