lkml.org 
[lkml]   [2021]   [Oct]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH v4 6/8] tracing/histogram: Optimize division by a power of 2
On Tue, 26 Oct 2021 18:09:22 -0700
Kalesh Singh <kaleshsingh@google.com> wrote:

> > delta = mult * div / 2^20
> >
> > That is if mult is a power of two, then there would be no rounding
> > errors, and the delta is zero, making the max infinite:

That should have been (as shown in the algorithm)

delta = mult * div - 2 ^ 20

As mult is 2^20 / div; and the above should end up zero if there's no
rounding issues, as it would be:

delta = (2^20 / div) * div - 2^20

-- Steve

\
 
 \ /
  Last update: 2021-10-27 03:16    [W:0.065 / U:1.820 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site