lkml.org 
[lkml]   [2022]   [Nov]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: PM-runtime: supplier looses track of consumer during probe
From
Hi linux-pm/linux-scsi,

Gentle reminder!

Can you please provide your suggestions on below race?

Thanks, Tushar Nimkar

On 10/14/2022 4:20 PM, Tushar Nimkar wrote:
> Hi linux-pm/linux-scsi,
>
> We have included fix [1] but continuing to observe supplier loosing
> track of consumer.
>
> Below is trace snippet with additional logging added.
> Here consumer is 0:0:0:0 and supplier is 0:0:0:49488. In Last three
> lines consumer resume is completed but supplier is put down.
>
>    kworker/u16:0-7     0.880014: rpm_idle:             0:0:0:0 flags-4
> cnt-0  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:0-7     0.880017: bprint:
> pm_runtime_mark_last_busy.46700: :#205 dev_name:0:0:0:0
> ktime_get_mono_fast_ns():852365364
>    kworker/u16:0-7     0.880019: rpm_suspend:          0:0:0:0 flags-8
> cnt-0  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:0-7     0.880022: bprint: pm_runtime_put_noidle.44083:
> pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:5
> decremented usage count
>    kworker/u16:0-7     0.880023: bprint: pm_runtime_put_noidle.44083:
> pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:4
> decremented usage count
>    kworker/u16:2-142   0.880024: rpm_resume:           0:0:0:0 flags-4
> cnt-1  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:0-7     0.880025: bprint: __rpm_put_suppliers:
> __rpm_put_suppliers: #348 consumer:0:0:0:0 supplier:0:0:0:49488
> usage_count:4
>    kworker/u16:0-7     0.880061: rpm_idle:             0:0:0:49488
> flags-1 cnt-4  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:0-7     0.880062: rpm_return_int:
> rpm_idle+0x16c:0:0:0:49488 ret=-11
>    kworker/u16:2-142   0.880062: bprint: __pm_runtime_resume:
> __pm_runtime_resume: #1147 dev_name:0:0:0:49488 dev usage_count:5
> incremented usage count
>    kworker/u16:2-142   0.880063: rpm_resume:           0:0:0:49488
> flags-4 cnt-5  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:2-142   0.880063: rpm_return_int:
> rpm_resume+0x690:0:0:0:49488 ret=1
>    kworker/u16:0-7     0.880063: rpm_return_int:
> rpm_suspend+0x68:0:0:0:0 ret=0
>    kworker/u16:2-142   0.880063: bprint: rpm_get_suppliers:
> rpm_get_suppliers: #300 consumer:0:0:0:0 supplier:0:0:0:49488 usage_count:5
>    kworker/u16:0-7     0.880065: bprint: pm_runtime_put_noidle.44083:
> pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:4
> decremented usage count
>    kworker/u16:2-142   0.880065: bprint:
> pm_runtime_mark_last_busy.44088: :#205 dev_name:0:0:0:0
> ktime_get_mono_fast_ns():852413749
>    kworker/u16:2-142   0.880065: rpm_idle:             0:0:0:0 flags-1
> cnt-1  dep-0  auto-1 p-0 irq-0 child-0
>    kworker/u16:2-142   0.880065: rpm_return_int: rpm_idle+0x16c:0:0:0:0
> ret=-11
>    kworker/u16:0-7     0.880066: bprint: __rpm_put_suppliers:
> __rpm_put_suppliers: #348 consumer:0:0:0:0 supplier:0:0:0:49488
> usage_count:4
>    kworker/u16:0-7     0.880067: rpm_return_int: rpm_idle+0x16c:0:0:0:0
> ret=-16
>    kworker/u16:2-142   0.880067: rpm_return_int:
> rpm_resume+0x690:0:0:0:0 ret=0
>
> Upon looking into this further the race looks to be in below two
> processes running in parallel and process-1 is putting down supplier at
> [C] because process-2 is setting runtime_status as resuming at [D].
>
> Also as per runtime PM documentation
> In order to use autosuspend, subsystems or drivers must call
> pm_runtime_use_autosuspend(), and thereafter they should use the various
> `*_autosuspend()` helper functions...
>
> It was also observed that *_autosuspend() API at point [A] was invoked
> without first invoking pm_runtime_use_autosuspend() which return
> expiration as zero at point [B] and proceeds ahead for immediate runtime
> suspend of device which seems lead to this race condition.
>
> Process -1
> ufshcd_async_scan context (process 1)
> scsi_autopm_put_device() //0:0:0:0
> pm_runtime_put_sync()
> __pm_runtime_idle()
> rpm_idle() -- RPM_GET_PUT(4)
>     __rpm_callback
>         scsi_runtime_idle()
>             pm_runtime_mark_last_busy()
>             pm_runtime_autosuspend()  --[A]
>                 rpm_suspend() -- RPM_AUTO(8)
>                     pm_runtime_autosuspend_expiration()
> use_autosuspend    is false return 0   --- [B]
>                         __update_runtime_status to RPM_SUSPENDING
>                     __rpm_callback()
>                         __rpm_put_suppliers(dev, false)
>                     __update_runtime_status to RPM_SUSPENDED
>                 rpm_suspend_suppliers()
>                     rpm_idle() for supplier -- RPM_ASYNC(1) return
> (-EAGAIN) [ Other consumer active for supplier]
>                 rpm_suspend() – END with return=0
>         scsi_runtime_idle() END return (-EBUSY) always.
>      /* Do that if resume fails too.*/
>     (dev->power.runtime_status == RPM_RESUMING && retval)))  return -EBUSY
>         __rpm_put_suppliers(dev, false)  -- [C]
> rpm_idle() END return -EBUSY
>
> Process -2
> sd_probe context (Process 2)
> scsi_autopm_get_device() //0:0:0:0
> __pm_runtime_resume(RPM_GET_PUT)
> rpm_resume() -- RPM_GET_PUT(4)
>     __update_runtime_status to RPM_RESUMING --[D]
>     __rpm_callback()
>         rpm_get_suppliers()
>             __pm_runtime_resume() - RPM_GET_PUT(4) – supplier
>                 rpm_resume() for supplier.
>     __update_runtime_status to RPM_ACTIVE
>     pm_runtime_mark_last_busy ()
> rpm_resume() END return 0
>
> Can you please provide your suggestions on addressing above race condition?
>
> This is also reported at [2].
>
> [1]: https://lore.kernel.org/lkml/4748074.GXAFRqVoOG@kreacher/T/
> [2]: https://lkml.org/lkml/2022/10/12/259
>
> Thanks,
> Tushar Nimkar

\
 
 \ /
  Last update: 2022-11-04 10:23    [W:0.090 / U:0.536 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site