lkml.org 
[lkml]   [2021]   [Dec]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRe: [PATCH v11 08/15] iio: afe: rescale: fix accuracy for small fractional scales
On Wed, Dec 22, 2021 at 5:47 AM Liam Beguin <liambeguin@gmail.com> wrote:
>
> From: Liam Beguin <lvb@xiphos.com>
>
> The approximation caused by integer divisions can be costly on smaller
> scale values since the decimal part is significant compared to the
> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
> cases to maintain accuracy.

...

> - tmp = div_s64(tmp, 1000000000LL);
> +
> + tmp = div_s64_rem(tmp, 1000000000LL, &rem);
> *val = tmp;

> - return scale_type;
> +

It seems you may add this blank line to one of the previous patches, may you?

> + if (!rem)
> + return scale_type;

--
With Best Regards,
Andy Shevchenko

\
 
 \ /
  Last update: 2021-12-22 13:30    [W:0.641 / U:0.216 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site