Messages in this thread | | | From | Xuewen Yan <> | Date | Tue, 19 Dec 2023 14:22:45 +0800 | Subject | Re: [PATCH v5 23/23] Documentation: EM: Update with runtime modification design |
| |
Hi Lukasz,
On Wed, Nov 29, 2023 at 7:11 PM Lukasz Luba <lukasz.luba@arm.com> wrote: > > Add a new section 'Design' which covers the information about Energy > Model. It contains the design decisions, describes models and how they > reflect the reality. Remove description of the default EM. Change the > other section IDs. Add documentation bit for the new feature which > allows to modify the EM in runtime. > > Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> > --- > Documentation/power/energy-model.rst | 206 +++++++++++++++++++++++++-- > 1 file changed, 196 insertions(+), 10 deletions(-) > > diff --git a/Documentation/power/energy-model.rst b/Documentation/power/energy-model.rst > index 13225965c9a4..1f8cf36914b1 100644 > --- a/Documentation/power/energy-model.rst > +++ b/Documentation/power/energy-model.rst > @@ -72,16 +72,48 @@ required to have the same micro-architecture. CPUs in different performance > domains can have different micro-architectures. > > > -2. Core APIs > +2. Design > +----------------- > + > +2.1 Runtime modifiable EM > +^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +To better reflect power variation due to static power (leakage) the EM > +supports runtime modifications of the power values. The mechanism relies on > +RCU to free the modifiable EM perf_state table memory. Its user, the task > +scheduler, also uses RCU to access this memory. The EM framework provides > +API for allocating/freeing the new memory for the modifiable EM table. > +The old memory is freed automatically using RCU callback mechanism when there > +are no owners anymore for the given EM runtime table instance. This is tracked > +using kref mechanism. The device driver which provided the new EM at runtime, > +should call EM API to free it safely when it's no longer needed. The EM > +framework will handle the clean-up when it's possible. > + > +The kernel code which want to modify the EM values is protected from concurrent > +access using a mutex. Therefore, the device driver code must run in sleeping > +context when it tries to modify the EM. > + > +With the runtime modifiable EM we switch from a 'single and during the entire > +runtime static EM' (system property) design to a 'single EM which can be > +changed during runtime according e.g. to the workload' (system and workload > +property) design. > + > +It is possible also to modify the CPU performance values for each EM's > +performance state. Thus, the full power and performance profile (which > +is an exponential curve) can be changed according e.g. to the workload > +or system property. > + > + > +3. Core APIs > ------------ > > -2.1 Config options > +3.1 Config options > ^^^^^^^^^^^^^^^^^^ > > CONFIG_ENERGY_MODEL must be enabled to use the EM framework. > > > -2.2 Registration of performance domains > +3.2 Registration of performance domains > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Registration of 'advanced' EM > @@ -110,8 +142,8 @@ The last argument 'microwatts' is important to set with correct value. Kernel > subsystems which use EM might rely on this flag to check if all EM devices use > the same scale. If there are different scales, these subsystems might decide > to return warning/error, stop working or panic. > -See Section 3. for an example of driver implementing this > -callback, or Section 2.4 for further documentation on this API > +See Section 4. for an example of driver implementing this > +callback, or Section 3.4 for further documentation on this API > > Registration of EM using DT > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > @@ -156,7 +188,7 @@ The EM which is registered using this method might not reflect correctly the > physics of a real device, e.g. when static power (leakage) is important. > > > -2.3 Accessing performance domains > +3.3 Accessing performance domains > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > There are two API functions which provide the access to the energy model: > @@ -175,10 +207,83 @@ CPUfreq governor is in use in case of CPU device. Currently this calculation is > not provided for other type of devices. > > More details about the above APIs can be found in ``<linux/energy_model.h>`` > -or in Section 2.4 > +or in Section 3.5 > + > + > +3.4 Runtime modifications > +^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Drivers willing to update the EM at runtime should use the following dedicated > +function to allocate a new instance of the modified EM. The API is listed > +below:: > + > + struct em_perf_table __rcu *em_allocate_table(struct em_perf_domain *pd); > + > +This allows to allocate a structure which contains the new EM table with > +also RCU and kref needed by the EM framework. The 'struct em_perf_table' > +contains array 'struct em_perf_state state[]' which is a list of performance > +states in ascending order. That list must be populated by the device driver > +which wants to update the EM. The list of frequencies can be taken from > +existing EM (created during boot). The content in the 'struct em_perf_state' > +must be populated by the driver as well. > + > +This is the API which does the EM update, using RCU pointers swap:: > + > + int em_dev_update_perf_domain(struct device *dev, > + struct em_perf_table __rcu *new_table); > + > +Drivers must provide a pointer to the allocated and initialized new EM > +'struct em_perf_table'. That new EM will be safely used inside the EM framework > +and will be visible to other sub-systems in the kernel (thermal, powercap). > +The main design goal for this API is to be fast and avoid extra calculations > +or memory allocations at runtime. When pre-computed EMs are available in the > +device driver, than it should be possible to simply re-use them with low > +performance overhead. > + > +In order to free the EM, provided earlier by the driver (e.g. when the module > +is unloaded), there is a need to call the API:: > + > + void em_free_table(struct em_perf_table __rcu *table); > + > +It will allow the EM framework to safely remove the memory, when there is > +no other sub-system using it, e.g. EAS. > + > +To use the power values in other sub-systems (like thermal, powercap) there is > +a need to call API which protects the reader and provide consistency of the EM > +table data:: > > + struct em_perf_state *em_get_table(struct em_perf_domain *pd); > > -2.4 Description details of this API > +It returns the 'struct em_perf_state' pointer which is an array of performance > +states in ascending order. > + > +When the EM table is not needed anymore there is a need to call dedicated API:: > + > + void em_put_table(void); > + > +In this way the EM safely uses the RCU read section and protects the users. > +It also allows the EM framework to manage the memory and free it. > + > +There is dedicated API for device drivers to calculate em_perf_state::cost > +values:: > + > + int em_dev_compute_costs(struct device *dev, struct em_perf_state *table, > + int nr_states); > + > +These 'cost' values from EM are used in EAS. The new EM table should be passed > +together with the number of entries and device pointer. When the computation > +of the cost values is done properly the return value from the function is 0. > +The function takes care for right setting of inefficiency for each performance > +state as well. It updates em_perf_state::flags accordingly. > +Then such prepared new EM can be passed to the em_dev_update_perf_domain() > +function, which will allow to use it. > + > +More details about the above APIs can be found in ``<linux/energy_model.h>`` > +or in Section 4.2 with an example code showing simple implementation of the > +updating mechanism in a device driver. > + > + > +3.5 Description details of this API > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > .. kernel-doc:: include/linux/energy_model.h > :internal: > @@ -187,8 +292,11 @@ or in Section 2.4 > :export: > > > -3. Example driver > ------------------ > +4. Examples > +----------- > + > +4.1 Example driver with EM registration > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > The CPUFreq framework supports dedicated callback for registering > the EM for a given CPU(s) 'policy' object: cpufreq_driver::register_em(). > @@ -242,3 +350,81 @@ EM framework:: > 39 static struct cpufreq_driver foo_cpufreq_driver = { > 40 .register_em = foo_cpufreq_register_em, > 41 }; > + > + > +4.2 Example driver with EM modification > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +This section provides a simple example of a thermal driver modifying the EM. > +The driver implements a foo_thermal_em_update() function. The driver is woken > +up periodically to check the temperature and modify the EM data:: > + > + -> drivers/soc/example/example_em_mod.c > + > + 01 static void foo_get_new_em(struct device *dev)
Because now some drivers use the dev_pm_opp_of_register_em() to register energy model, and maybe we can add a new function to update the energy model using "EM_SET_ACTIVE_POWER_CB(em_cb, cb)" instead of letting users set power again?
Thanks!
> + 02 { > + 03 struct em_perf_table __rcu *runtime_table; > + 04 struct em_perf_state *table, *new_table; > + 05 struct em_perf_domain *pd; > + 06 unsigned long freq; > + 07 int i, ret; > + 08 > + 09 pd = em_pd_get(dev); > + 10 if (!pd) > + 11 return; > + 12 > + 13 runtime_table = em_allocate_table(pd); > + 14 if (!runtime_table) > + 15 return; > + 16 > + 17 new_table = runtime_table->state; > + 18 > + 19 table = em_get_table(pd); > + 20 for (i = 0; i < pd->nr_perf_states; i++) { > + 21 freq = table[i].frequency; > + 22 foo_get_power_perf_values(dev, freq, &new_table[i]); > + 23 } > + 24 em_put_table(); > + 25 > + 26 /* Calculate 'cost' values for EAS */ > + 27 ret = em_dev_compute_costs(dev, table, pd->nr_perf_states); > + 28 if (ret) { > + 29 dev_warn(dev, "EM: compute costs failed %d\n", ret); > + 30 em_free_table(runtime_table); > + 31 return; > + 32 } > + 33 > + 34 ret = em_dev_update_perf_domain(dev, runtime_table); > + 35 if (ret) { > + 36 dev_warn(dev, "EM: update failed %d\n", ret); > + 37 em_free_table(runtime_table); > + 38 return; > + 39 } > + 40 > + 41 ctx->runtime_table = runtime_table; > + 42 } > + 43 > + 44 /* > + 45 * Function called periodically to check the temperature and > + 46 * update the EM if needed > + 47 */ > + 48 static void foo_thermal_em_update(struct foo_context *ctx) > + 49 { > + 50 struct device *dev = ctx->dev; > + 51 int cpu; > + 52 > + 53 ctx->temperature = foo_get_temp(dev, ctx); > + 54 if (ctx->temperature < FOO_EM_UPDATE_TEMP_THRESHOLD) > + 55 return; > + 56 > + 57 foo_get_new_em(dev); > + 58 } > + 59 > + 60 static void foo_exit(void) > + 61 { > + 62 struct foo_context *ctx = glob_ctx; > + 63 > + 64 em_free_table(ctx->runtime_table); > + 65 } > + 66 > + 67 module_exit(foo_exit); > -- > 2.25.1 >
| |