Messages in this thread | | | Date | Wed, 8 Aug 2018 07:49:22 -0700 | From | "Paul E. McKenney" <> | Subject | Re: [PATCH v12 3/3] tracing: Centralize preemptirq tracepoints and unify their usage |
| |
On Wed, Aug 08, 2018 at 07:10:53AM -0700, Joel Fernandes wrote: > On Wed, Aug 8, 2018 at 6:00 AM, Paul E. McKenney > <paulmck@linux.vnet.ibm.com> wrote: > > On Tue, Aug 07, 2018 at 08:53:54PM -0700, Joel Fernandes wrote: > >> On Tue, Aug 7, 2018 at 8:44 PM, Joel Fernandes <joelaf@google.com> wrote: > >> > Hi Steve, > >> > > >> > On Tue, Aug 7, 2018 at 7:28 PM, Steven Rostedt <rostedt@goodmis.org> wrote: > >> [...] > >> >>> @@ -171,8 +174,7 @@ extern void syscall_unregfunc(void); > >> >>> } while ((++it_func_ptr)->func); \ > >> >>> } \ > >> >>> \ > >> >>> - if (rcuidle) \ > >> >>> - srcu_read_unlock_notrace(&tracepoint_srcu, idx);\ > >> >>> + srcu_read_unlock_notrace(ss, idx); \ > >> >> > >> >> Hmm, why do we have the two different srcu handles? > >> > > >> > Because if the memory operations happening on the normal SRCU handle > >> > (during srcu_read_lock) is interrupted by NMI, then the other handle > >> > (devoted to NMI) could be used instead and not bother the interrupted > >> > handle. Does that makes sense? > >> > > >> > When I talked to Paul few months ago about SRCU from NMI context, he > >> > mentioned the per-cpu memory operations during srcu_read_lock can be > >> > NMI interrupted, that's why we added that warning. > >> > >> So I looked more closely, __srcu_read_lock on 2 different handles may > >> still be doing a this_cpu_inc on the same location.. > >> (sp->sda->srcu_lock_count). :-( > >> > >> Paul any ideas on how to solve this? > > > > You lost me on this one. When you said "2 different handles", I assumed > > that you meant two different values of "sp", which would have two > > different addresses for &sp->sda->srcu_lock_count. What am I missing? > > Thanks a lot for the reply. > I thought "sda" is the same for different srcu_struct(s). May be it > was too late for me in the night, that's why I thought so? Which makes > no sense now that I think of it.
I know that feeling! ;-)
> In that case based on what you're saying, the patch I sent to using > different srcu_struct for NMI is still good I guess...
As long as you wait for both SRCU grace periods. Hmmm... Maybe that means that there is still a use for synchronize_rcu_mult():
void call_srcu_nmi(struct rcu_head *rhp, rcu_callback_t func) { call_srcu(&trace_srcu_struct_nmi, rhp, func); }
void call_srcu_nonmi(struct rcu_head *rhp, rcu_callback_t func) { call_srcu(&trace_srcu_struct_nonmi, rhp, func); }
...
/* Wait concurrently on the two grace periods. */ synchronize_rcu_mult(call_srcu_nmi, call_srcu_nonmi);
On the other hand, I bet that doing this is just fine in your use case:
synchronize_srcu(&trace_srcu_struct_nmi); synchronize_srcu(&trace_srcu_struct_nonmi);
But please note that synchronize_rcu_mult() is no more in my -rcu tree, so if you do want it please let me know (and please let me know why it is important).
> >> It does start to seem like a show stopper :-( > > > > I suppose that an srcu_read_lock_nmi() and srcu_read_unlock_nmi() could > > be added, which would do atomic ops on sp->sda->srcu_lock_count. Not sure > > whether this would be fast enough to be useful, but easy to provide: > > > > int __srcu_read_lock_nmi(struct srcu_struct *sp) /* UNTESTED. */ > > { > > int idx; > > > > idx = READ_ONCE(sp->srcu_idx) & 0x1; > > atomic_inc(&sp->sda->srcu_lock_count[idx]); > > smp_mb__after_atomic(); /* B */ /* Avoid leaking critical section. */ > > return idx; > > } > > > > void __srcu_read_unlock_nmi(struct srcu_struct *sp, int idx) > > { > > smp_mb__before_atomic(); /* C */ /* Avoid leaking critical section. */ > > atomic_inc(&sp->sda->srcu_unlock_count[idx]); > > } > > > > With appropriate adjustments to also allow Tiny RCU to also work. > > > > Note that you have to use _nmi() everywhere, not just in NMI handlers. > > In fact, the NMI handlers are the one place you -don't- need to use > > _nmi(), strangely enough. > > > > Might be worth a try -- smp_mb__{before,after}_atomic() is a no-op on > > some architectures, for example. > > Continuing Steve's question on regular interrupts, do we need to use > this atomic_inc API for regular interrupts as well?
If NMIs use one srcu_struct and non-NMI uses another, the current srcu_read_lock() and srcu_read_unlock() will work just fine. If any given srcu_struct needs both NMI and non-NMI readers, then we really do need __srcu_read_lock_nmi() and __srcu_read_unlock_nmi() for that srcu_struct.
Thanx, Paul
| |