lkml.org 
[lkml]   [2022]   [Feb]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH V2] remoteproc: support self recovery after rproc crash
From
Hi Peng,

On 2/14/22 19:41, Arnaud POULIQUEN wrote:
> Hi Peng,
>
> On 1/26/22 09:51, Peng Fan (OSS) wrote:
>> From: Peng Fan <peng.fan@nxp.com>
>>
>> Current logic only support main processor to stop/start the remote
>> processor after rproc crash. However to SoC, such as i.MX8QM/QXP, the
>> remote processor could do self recovery after crash and trigger watchdog
>> reboot. It does not need main processor to load image, stop/start M4
>> core.
>
>
> On stm32mp1 platform the remote processor watchdog generates an early interrupt
> that could be used to detach and reattach before the reset of the remote processor.
> I need to test race condition,but I suppose that this should works if the resource
> table is not reinitialized by the remote processor firmware.
>
> Another option for the stm32mp1 is that remoteproc manages the reset of the
> remote processor.
> For instance this allows to save a core-dump before manually resetting the remote
> processor.
> But looks like this use case can be handled later, as mentioned below.
>
>>
>> This patch add a new flag to indicate whether the SoC has self recovery
>> capability. And introduce two functions: rproc_self_recovery,
>> rproc_assisted_recovery for the two cases. Assisted recovery is as
>> before, let main processor to help recovery, while self recovery is
>> recover itself withou help. To self recovery, we only do detach and
>> attach.
>
>
>>
>> Signed-off-by: Peng Fan <peng.fan@nxp.com>
>> ---
>>
>> V2:
>> Nothing change in V2.
>> Only move this patch out from
>> https://patchwork.kernel.org/project/linux-remoteproc/list/?series=604364
>>
>> drivers/remoteproc/remoteproc_core.c | 66 ++++++++++++++++++++--------
>> include/linux/remoteproc.h | 2 +
>> 2 files changed, 49 insertions(+), 19 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
>> index 69f51acf235e..4bd5544dab8f 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -1887,6 +1887,49 @@ static int __rproc_detach(struct rproc *rproc)
>> return 0;
>> }
>>
>> +static int rproc_self_recovery(struct rproc *rproc)
>> +{
>> + int ret;
>> +
>> + mutex_unlock(&rproc->lock);
>> + ret = rproc_detach(rproc);
>> + mutex_lock(&rproc->lock);
>> + if (ret)
>> + return ret;
>
> Here we would want to perform a core dump and manually reset the
> co-processor.
> I suppose that a new rproc ops could be called here in a next step.
>
>> +
>> + if (atomic_inc_return(&rproc->power) > 1)
>> + return 0;
>
> Do you identify a use case that needs to test rproc->power to
> skip the attach?
> If yes could you add a comment to describe it?
>
>> + return rproc_attach(rproc);
>> +}
>> +
>> +static int rproc_assisted_recovery(struct rproc *rproc)
>> +{
>> + const struct firmware *firmware_p;
>> + struct device *dev = &rproc->dev;
>> + int ret;
>> +
>> + ret = rproc_stop(rproc, true);
>> + if (ret)
>> + return ret;
>> +
>> + /* generate coredump */
>> + rproc->ops->coredump(rproc);
>> +
>> + /* load firmware */
>> + ret = request_firmware(&firmware_p, rproc->firmware, dev);
>> + if (ret < 0) {
>> + dev_err(dev, "request_firmware failed: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + /* boot the remote processor up again */
>> + ret = rproc_start(rproc, firmware_p);
>> +
>> + release_firmware(firmware_p);
>> +
>> + return ret;
>> +}
>> +
>> /**
>> * rproc_trigger_recovery() - recover a remoteproc
>> * @rproc: the remote processor
>> @@ -1901,7 +1944,6 @@ static int __rproc_detach(struct rproc *rproc)
>> */
>> int rproc_trigger_recovery(struct rproc *rproc)
>> {
>> - const struct firmware *firmware_p;
>> struct device *dev = &rproc->dev;
>> int ret;
>>
>> @@ -1915,24 +1957,10 @@ int rproc_trigger_recovery(struct rproc *rproc)
>>
>> dev_err(dev, "recovering %s\n", rproc->name);
>>
>> - ret = rproc_stop(rproc, true);
>> - if (ret)
>> - goto unlock_mutex;
>> -
>> - /* generate coredump */
>> - rproc->ops->coredump(rproc);
>> -
>> - /* load firmware */
>> - ret = request_firmware(&firmware_p, rproc->firmware, dev);
>> - if (ret < 0) {
>> - dev_err(dev, "request_firmware failed: %d\n", ret);
>> - goto unlock_mutex;
>> - }
>> -
>> - /* boot the remote processor up again */
>> - ret = rproc_start(rproc, firmware_p);
>> -
>> - release_firmware(firmware_p);
>> + if (rproc->self_recovery)
>> + ret = rproc_self_recovery(rproc);
>
> If some platforms have to manually reset the remote processor (without
> reloading the firmware) the name could not be relevant...
>
> Following comments are only suggestions that needs to be commented by maintainers
>
> What about rproc_attach_recovery ?
>
>> + else
>> + ret = rproc_assisted_recovery(rproc);
>
> and rproc_firmware_recovery ?
>
>
>>
>> unlock_mutex:
>> mutex_unlock(&rproc->lock);
>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>> index e0600e1e5c17..b32ef46f8aa4 100644
>> --- a/include/linux/remoteproc.h
>> +++ b/include/linux/remoteproc.h
>> @@ -529,6 +529,7 @@ struct rproc_dump_segment {
>> * @elf_machine: firmware ELF machine
>> * @cdev: character device of the rproc
>> * @cdev_put_on_release: flag to indicate if remoteproc should be shutdown on @char_dev release
>> + * @self_recovery: flag to indicate if remoteproc support self recovery
>> */
>> struct rproc {
>> struct list_head node;
>> @@ -568,6 +569,7 @@ struct rproc {
>> u16 elf_machine;
>> struct cdev cdev;
>> bool cdev_put_on_release;
>> + bool self_recovery;
>
> This bool seems needed because we have lost the previous state before crash.
> I wonder if a new rproc->state such as RPROC_REBOOT could avoid this boolean.
>
>
> I will try to test you patch on stm32mp1 next week

I performed few tests on the stm32mp1 with your patch.
Thanks to the resetting of the resource tables on detachment, this works quite well.

Regards,
Arnaud

>
> Regards,
> Arnaud
>
>> };
>>
>> /**

\
 
 \ /
  Last update: 2022-02-24 15:09    [W:0.122 / U:0.432 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site