lkml.org 
[lkml]   [2019]   [Jan]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRe: fix int_sqrt() for very large numbers
On Sun, Jan 20, 2019 at 9:30 PM Crt Mori <cmo@melexis.com> wrote:
>
> I have just re-read the patch submit discussion and a sqrt of 64bit
> number can never be more than 32bit. That is why u32 return value is
> enough.

Right. And that's exactly why I thought it was so odd how the
mlx90632.c driver - which is the only user of int_sqrt64() - does a
nested one.

It would be sufficient to do

int_sqrt(int_sqrt64(..))

because the domain on the outer one isn't actually 64 bits, it's only 32 bits.

But it's just a slight confusion and doesn't really matter much.

The bug in int_sqrt64() is real, but as mentioned, the calculations
done to generate the argument for that one user do seem to all be
*signed*. If it overflows into the high bit of an u64 (which is where
the off-by-one bug would matter), then the types in that one user are
already wrong.

So it would probably be a good idea to clarify the single user too while at it.

Linus

\
 
 \ /
  Last update: 2019-01-20 11:01    [W:0.034 / U:0.532 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site