Messages in this thread | | | Subject | Re: [PATCH v5] Documentation, dt, numa: Add note to empty NUMA node | From | Gavin Shan <> | Date | Sun, 11 Jul 2021 10:59:58 +1000 |
| |
Hi Rob,
On 7/2/21 10:02 AM, Gavin Shan wrote: > On 7/2/21 3:25 AM, Rob Herring wrote: >> On Mon, Jun 28, 2021 at 05:34:11PM +0800, Gavin Shan wrote: >>> The empty memory nodes, where no memory resides in, are allowed. >>> For these empty memory nodes, the 'len' of 'reg' property is zero. >>> The NUMA node IDs are still valid and parsed, but memory may be >>> added to them through hotplug afterwards. I finds difficulty to >>> get where it's properly documented. >> >> This is already in use? If so, what platform(s)? >> > > It's not used yet, but will be used by QEMU once this patch is merged. > In QEMU, ARM64 could have multiple empty memory nodes. The corresponding > NUMA ID and distance map are still valid because memory may be added into > these empty memory nodes in future. > > For the QEMU case, the names of empty memory nodes are the biggest concern. > According to device-tree specification, the name follows the format of > 'memory@unit-address' and the 'unit-address' is equivalent to 'base-address'. > However, the 'base-address' should be invalid one. In current QEMU implementation, > the valid 'base-address' and 'unit-address' are provided to these empty > memory nodes. Another issue in QEMU is trying to populate two empty memory > nodes, which have same names. This leads to failure of device-tree population > because of the duplicated memory node names, blocking VM from booting. > >>> So lets add a section for empty memory nodes in NUMA binding >>> document. Also, the 'unit-address', equivalent to 'base-address' >>> in the 'reg' property of these empty memory nodes is suggested to >>> be the summation of highest memory address plus the NUMA node ID. >> >> What purpose does this serve? The kernel won't do anything with it other >> than validate the numa-node-id range. >> > > As mentioned above, the point is to have dummy, invalid and non-overlapped > 'base-address' and 'unit-address' for these empty memory nodes, to avoid > duplicated memory node names in devcie-tree. > >>> >>> Signed-off-by: Gavin Shan <gshan@redhat.com> >>> --- >>> v5: Separate section for empty memory node >>> --- >>> Documentation/devicetree/bindings/numa.txt | 61 +++++++++++++++++++++- >>> 1 file changed, 60 insertions(+), 1 deletion(-) >>> >>> diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt >>> index 21b35053ca5a..230c734af948 100644 >>> --- a/Documentation/devicetree/bindings/numa.txt >>> +++ b/Documentation/devicetree/bindings/numa.txt >>> @@ -103,7 +103,66 @@ Example: >>> }; >>> ============================================================================== >>> -4 - Example dts >>> +4 - Empty memory nodes >>> +============================================================================== >>> + >>> +Empty memory nodes, which no memory resides in, are allowed. The 'length' >>> +field of the 'reg' property is zero, but the 'base-address' is a dummy >>> +address and invalid. The 'base-address' could be the summation of highest >>> +memory address plus the NUMA node ID. However, the NUMA node IDs and >>> +distance maps are still valid and memory may be added into them through >>> +hotplug afterwards. >>> + >>> +Example: >>> + >>> + memory@0 { >>> + device_type = "memory"; >>> + reg = <0x0 0x0 0x0 0x80000000>; >>> + numa-node-id = <0>; >>> + }; >>> + >>> + memory@0x80000000 { >> >> unit-address should not have '0x'. >> > > Ok. Lets fix it in v6 after it's agreed to add the section into the > NUMA binding document. Actually, the '0x' is copied from the existing > example in same document. After this patch is finalized, I will post > separate patch to fix all wrong formats in same document as well. >
I posted v6 just now, where '0x' in 'unit-address' is dropped. After this is merged, I will post followup patch to remove '0x' in 'unit-address' for existing examples.
Thanks, Gavin
>>> + device_type = "memory"; >>> + reg = <0x0 0x80000000 0x0 0x80000000>; >>> + numa-node-id = <1>; >>> + }; >>> + >>> + /* Empty memory node */ >>> + memory@0x100000002 { >>> + device_type = "memory"; >>> + reg = <0x1 0x2 0x0 0x0>; >>> + numa-node-id = <2>; >>> + }; >>> + >>> + /* Empty memory node */ >>> + memory@0x100000003 { >>> + device_type = "memory"; >>> + reg = <0x1 0x3 0x0 0x0>; >>> + numa-node-id = <3>; >>> + }; >>> + >>> + distance-map { >>> + compatible = "numa-distance-map-v1"; >>> + distance-matrix = <0 0 10>, >>> + <0 1 20>, >>> + <0 2 40>, >>> + <0 3 20>, >>> + <1 0 20>, >>> + <1 1 10>, >>> + <1 2 20>, >>> + <1 3 40>, >>> + <2 0 40>, >>> + <2 1 20>, >>> + <2 2 10>, >>> + <2 3 20>, >>> + <3 0 20>, >>> + <3 1 40>, >>> + <3 2 20>, >>> + <3 3 10>; >>> + }; >>> + >>> +============================================================================== >>> +5 - Example dts >>> ============================================================================== >>> Dual socket system consists of 2 boards connected through ccn bus and >>> --
| |