Messages in this thread | | | Date | Wed, 8 Jun 2022 11:55:14 -0400 | From | Johannes Weiner <> | Subject | Re: [PATCH v5 1/9] mm/demotion: Add support for explicit memory tiers |
| |
Hello,
On Wed, Jun 08, 2022 at 10:11:31AM -0400, Johannes Weiner wrote: > On Fri, Jun 03, 2022 at 07:12:29PM +0530, Aneesh Kumar K.V wrote: > > @@ -0,0 +1,20 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +#ifndef _LINUX_MEMORY_TIERS_H > > +#define _LINUX_MEMORY_TIERS_H > > + > > +#ifdef CONFIG_TIERED_MEMORY > > + > > +#define MEMORY_TIER_HBM_GPU 0 > > +#define MEMORY_TIER_DRAM 1 > > +#define MEMORY_TIER_PMEM 2 > > + > > +#define MEMORY_RANK_HBM_GPU 300 > > +#define MEMORY_RANK_DRAM 200 > > +#define MEMORY_RANK_PMEM 100 > > + > > +#define DEFAULT_MEMORY_TIER MEMORY_TIER_DRAM > > +#define MAX_MEMORY_TIERS 3 > > I understand the names are somewhat arbitrary, and the tier ID space > can be expanded down the line by bumping MAX_MEMORY_TIERS. > > But starting out with a packed ID space can get quite awkward for > users when new tiers - especially intermediate tiers - show up in > existing configurations. I mentioned in the other email that DRAM != > DRAM, so new tiers seem inevitable already. > > It could make sense to start with a bigger address space and spread > out the list of kernel default tiers a bit within it: > > MEMORY_TIER_GPU 0 > MEMORY_TIER_DRAM 10 > MEMORY_TIER_PMEM 20
Forgive me if I'm asking a question that has been answered. I went back to earlier threads and couldn't work it out - maybe there were some off-list discussions? Anyway...
Why is there a distinction between tier ID and rank? I undestand that rank was added because tier IDs were too few. But if rank determines ordering, what is the use of a separate tier ID? IOW, why not make the tier ID space wider and have the kernel pick a few spread out defaults based on known hardware, with plenty of headroom to be future proof.
$ ls tiers 100 # DEFAULT_TIER $ cat tiers/100/nodelist 0-1 # conventional numa nodes
<pmem is onlined>
$ grep . tiers/*/nodelist tiers/100/nodelist:0-1 # conventional numa tiers/200/nodelist:2 # pmem
$ grep . nodes/*/tier nodes/0/tier:100 nodes/1/tier:100 nodes/2/tier:200
<unknown device is online as node 3, defaults to 100>
$ grep . tiers/*/nodelist tiers/100/nodelist:0-1,3 tiers/200/nodelist:2
$ echo 300 >nodes/3/tier $ grep . tiers/*/nodelist tiers/100/nodelist:0-1 tiers/200/nodelist:2 tiers/300/nodelist:3
$ echo 200 >nodes/3/tier $ grep . tiers/*/nodelist tiers/100/nodelist:0-1 tiers/200/nodelist:2-3
etc.
| |