Messages in this thread | | | Date | Thu, 13 Feb 2020 22:08:34 -0800 | From | Josh Triplett <> | Subject | Re: Applying pipe fix this merge window? |
| |
On Wed, Feb 12, 2020 at 12:03:26PM -0800, Linus Torvalds wrote: > And I realized that I find it surprising that it makes your build > times noticeably better. > > Yes, I have that silly example program to show the issue in the commit > message, and yes, the exclusive directed write->read wakeups should > most definitely improve by that commit. > > But the make jobserver code ends up using "poll()/pselect()" and > non-blocking reads, because of how it handles the child death signals. > > Which means that none of the nice exclusive directed write->read > wakeups should even trigger in the first place, because the readers > never block, and he poll/pselect code doesn't use exclusive wakeups > (because it can't - it doesn't actually consume the data). > > So I was looking at it, and going "it should actually not help GNU > jobserver at all" in the fixed jobserver case.
I dug into this a little further yesterday and today:
- With hindsight, I realized that the performance improvements I observed for GNU make didn't measure the pipe fix in isolation; they measured 5.4 versus ~5.5-rc4 plus the pipe fix, which would include all the other pipe work in 5.5 and potentially other optimizations. *That* showed substantial performance improvements in GNU make, on the order of a couple of seconds in a 30-60 second kernel build. ("5.5-rc4 plus pipe fix" is what I hammered on for a month on various systems.)
- Measuring the pipe fix patch in isolation (0bf999f9c5e74c7ecf9dafb527146601e5c848b9, with and without the pipe fix reverted, with nothing else changed), GNU make performance indeed doesn't show any difference.
- Other things that use the GNU make jobserver (with pipe fds in blocking mode) benefit much more heavily, in wall-clock time and in total CPU time. I saw jobs that involved just a minute or two of wall-clock time, where the total CPU time went down by *minutes*.
Hope that helps, Josh Triplett
| |