Messages in this thread | | | Date | Thu, 6 Aug 2020 15:15:47 +0200 | From | peterz@infradea ... | Subject | Re: [PATCH 1/2] sched/topology: Allow archs to override cpu_smt_mask |
| |
On Thu, Aug 06, 2020 at 10:25:12PM +1000, Michael Ellerman wrote: > peterz@infradead.org writes: > > On Thu, Aug 06, 2020 at 03:32:25PM +1000, Michael Ellerman wrote: > > > >> That brings with it a bunch of problems, such as existing software that > >> has been developed/configured for Power8 and expects to see SMT8. > >> > >> We also allow LPARs to be live migrated from Power8 to Power9 (and back), so > >> maintaining the illusion of SMT8 is considered a requirement to make that work. > > > > So how does that work if the kernel booted on P9 and demuxed the SMT8 > > into 2xSMT4? If you migrate that state onto a P8 with actual SMT8 you're > > toast again. > > The SMT mask would be inaccurate on the P8, rather than the current case > where it's inaccurate on the P9. > > Which would be our preference, because the backward migration case is > not common AIUI. > > Or am I missing a reason we'd be even more toast than that?
Well, the scheduler might do a wee bit funny. We just had a patch that increase load-balancing opportunities between SMT siblings because they all share L1 anyway.
But yeah, nothing terminal.
> Under PowerVM the kernel does know it's being migrated, so we could > actually update the mask, but I'm not sure if that's really feasible.
As long as you get a notification, rebuilding the sched domains isn't terribly hard to do, there's more code that does that.
> >> Yeah I agree the naming is confusing. > >> > >> Let's call them "SMT4 cores" and "SMT8 cores"? > > > > Works for me, thanks! > > > >> The problem is we are already lying to userspace, because firmware lies to us. > >> > >> ie. the firmware on these systems shows us an SMT8 core, and so current kernels > >> show SMT8 to userspace. I don't think we can realistically change that fact now, > >> as these systems are already out in the field. > >> > >> What this patch tries to do is undo some of the mess, and at least give the > >> scheduler the right information. > > > > What a mess... I think it depends on what you do with that P9 to P8 > > migration case. Does it make sense to have a "p8_compat" boot arg for > > the case where you want LPAR migration back onto P8 systems -- in which > > case it simply takes the firmware's word as gospel and doesn't untangle > > things, because it can actually land on a P8. > > We already get told by firmware that we're running in "p8 compat" mode, > because we have to pretend to userspace that it's running on a P8. So we > could use that as a signal to leave things alone. > > But my understanding is most LPARs don't get migrated back and forth, > they'll start life on a P8 and only get migrated to a P9 once when the > customer gets a P9. They might then run for a long time (months to > years) on the P9 in P8 compat mode, not because they ever want to > migrate back to a real P8, but because the software in the LPAR is still > expecting to be on a P8. > > I'm not a real expert on all the Enterprisey stuff though, so someone > else might be able to give us a better picture. > > But the point of mentioning the migration stuff was mainly just to > explain why we feel we need to present SMT8 to userspace even on P9.
OK, fair enough. The patch wasn't particularly onerous, I was just wondering why etc..
The case of starting on a P8 and being migrated to a P9 makes sense to me; in that case you'd like to rebuild your sched domains, but can't go about changing user visible topolofy information.
I suppose:
Acked-by; Peter Zijlstra (Intel) <peterz@infradead.org>
An updated Changelog that recaps some of this discussion might also be nice.
| |