Messages in this thread | | | Date | Fri, 4 Nov 2022 14:49:59 +0530 | Subject | Re: PM-runtime: supplier looses track of consumer during probe | From | Tushar Nimkar <> |
| |
Hi linux-pm/linux-scsi,
Gentle reminder!
Can you please provide your suggestions on below race?
Thanks, Tushar Nimkar
On 10/14/2022 4:20 PM, Tushar Nimkar wrote: > Hi linux-pm/linux-scsi, > > We have included fix [1] but continuing to observe supplier loosing > track of consumer. > > Below is trace snippet with additional logging added. > Here consumer is 0:0:0:0 and supplier is 0:0:0:49488. In Last three > lines consumer resume is completed but supplier is put down. > > kworker/u16:0-7 0.880014: rpm_idle: 0:0:0:0 flags-4 > cnt-0 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:0-7 0.880017: bprint: > pm_runtime_mark_last_busy.46700: :#205 dev_name:0:0:0:0 > ktime_get_mono_fast_ns():852365364 > kworker/u16:0-7 0.880019: rpm_suspend: 0:0:0:0 flags-8 > cnt-0 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:0-7 0.880022: bprint: pm_runtime_put_noidle.44083: > pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:5 > decremented usage count > kworker/u16:0-7 0.880023: bprint: pm_runtime_put_noidle.44083: > pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:4 > decremented usage count > kworker/u16:2-142 0.880024: rpm_resume: 0:0:0:0 flags-4 > cnt-1 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:0-7 0.880025: bprint: __rpm_put_suppliers: > __rpm_put_suppliers: #348 consumer:0:0:0:0 supplier:0:0:0:49488 > usage_count:4 > kworker/u16:0-7 0.880061: rpm_idle: 0:0:0:49488 > flags-1 cnt-4 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:0-7 0.880062: rpm_return_int: > rpm_idle+0x16c:0:0:0:49488 ret=-11 > kworker/u16:2-142 0.880062: bprint: __pm_runtime_resume: > __pm_runtime_resume: #1147 dev_name:0:0:0:49488 dev usage_count:5 > incremented usage count > kworker/u16:2-142 0.880063: rpm_resume: 0:0:0:49488 > flags-4 cnt-5 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:2-142 0.880063: rpm_return_int: > rpm_resume+0x690:0:0:0:49488 ret=1 > kworker/u16:0-7 0.880063: rpm_return_int: > rpm_suspend+0x68:0:0:0:0 ret=0 > kworker/u16:2-142 0.880063: bprint: rpm_get_suppliers: > rpm_get_suppliers: #300 consumer:0:0:0:0 supplier:0:0:0:49488 usage_count:5 > kworker/u16:0-7 0.880065: bprint: pm_runtime_put_noidle.44083: > pm_runtime_put_noidle: #112 dev_name:0:0:0:49488 dev usage_count:4 > decremented usage count > kworker/u16:2-142 0.880065: bprint: > pm_runtime_mark_last_busy.44088: :#205 dev_name:0:0:0:0 > ktime_get_mono_fast_ns():852413749 > kworker/u16:2-142 0.880065: rpm_idle: 0:0:0:0 flags-1 > cnt-1 dep-0 auto-1 p-0 irq-0 child-0 > kworker/u16:2-142 0.880065: rpm_return_int: rpm_idle+0x16c:0:0:0:0 > ret=-11 > kworker/u16:0-7 0.880066: bprint: __rpm_put_suppliers: > __rpm_put_suppliers: #348 consumer:0:0:0:0 supplier:0:0:0:49488 > usage_count:4 > kworker/u16:0-7 0.880067: rpm_return_int: rpm_idle+0x16c:0:0:0:0 > ret=-16 > kworker/u16:2-142 0.880067: rpm_return_int: > rpm_resume+0x690:0:0:0:0 ret=0 > > Upon looking into this further the race looks to be in below two > processes running in parallel and process-1 is putting down supplier at > [C] because process-2 is setting runtime_status as resuming at [D]. > > Also as per runtime PM documentation > In order to use autosuspend, subsystems or drivers must call > pm_runtime_use_autosuspend(), and thereafter they should use the various > `*_autosuspend()` helper functions... > > It was also observed that *_autosuspend() API at point [A] was invoked > without first invoking pm_runtime_use_autosuspend() which return > expiration as zero at point [B] and proceeds ahead for immediate runtime > suspend of device which seems lead to this race condition. > > Process -1 > ufshcd_async_scan context (process 1) > scsi_autopm_put_device() //0:0:0:0 > pm_runtime_put_sync() > __pm_runtime_idle() > rpm_idle() -- RPM_GET_PUT(4) > __rpm_callback > scsi_runtime_idle() > pm_runtime_mark_last_busy() > pm_runtime_autosuspend() --[A] > rpm_suspend() -- RPM_AUTO(8) > pm_runtime_autosuspend_expiration() > use_autosuspend is false return 0 --- [B] > __update_runtime_status to RPM_SUSPENDING > __rpm_callback() > __rpm_put_suppliers(dev, false) > __update_runtime_status to RPM_SUSPENDED > rpm_suspend_suppliers() > rpm_idle() for supplier -- RPM_ASYNC(1) return > (-EAGAIN) [ Other consumer active for supplier] > rpm_suspend() – END with return=0 > scsi_runtime_idle() END return (-EBUSY) always. > /* Do that if resume fails too.*/ > (dev->power.runtime_status == RPM_RESUMING && retval))) return -EBUSY > __rpm_put_suppliers(dev, false) -- [C] > rpm_idle() END return -EBUSY > > Process -2 > sd_probe context (Process 2) > scsi_autopm_get_device() //0:0:0:0 > __pm_runtime_resume(RPM_GET_PUT) > rpm_resume() -- RPM_GET_PUT(4) > __update_runtime_status to RPM_RESUMING --[D] > __rpm_callback() > rpm_get_suppliers() > __pm_runtime_resume() - RPM_GET_PUT(4) – supplier > rpm_resume() for supplier. > __update_runtime_status to RPM_ACTIVE > pm_runtime_mark_last_busy () > rpm_resume() END return 0 > > Can you please provide your suggestions on addressing above race condition? > > This is also reported at [2]. > > [1]: https://lore.kernel.org/lkml/4748074.GXAFRqVoOG@kreacher/T/ > [2]: https://lkml.org/lkml/2022/10/12/259 > > Thanks, > Tushar Nimkar
| |