Messages in this thread | | | From | Kalesh Singh <> | Date | Tue, 26 Oct 2021 18:31:21 -0700 | Subject | Re: [PATCH v4 6/8] tracing/histogram: Optimize division by a power of 2 |
| |
On Tue, Oct 26, 2021 at 6:15 PM Steven Rostedt <rostedt@goodmis.org> wrote: > > On Tue, 26 Oct 2021 18:09:22 -0700 > Kalesh Singh <kaleshsingh@google.com> wrote: > > > > delta = mult * div / 2^20 > > > > > > That is if mult is a power of two, then there would be no rounding > > > errors, and the delta is zero, making the max infinite: > > That should have been (as shown in the algorithm) > > delta = mult * div - 2 ^ 20 > > As mult is 2^20 / div; and the above should end up zero if there's no > rounding issues, as it would be: > > delta = (2^20 / div) * div - 2^20
Good catch. We're checking if we get back the exact value.
And IIUC max_div is an arbitrary value we decide on that's <= 2^shift? Is there a rule of thumb for choosing this?
Thanks, Kalesh > > -- Steve
| |