lkml.org 
[lkml]   [2021]   [Aug]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[PATCH v8 09/14] iio: afe: rescale: fix accuracy for small fractional scales
Date
From: Liam Beguin <lvb@xiphos.com>

The approximation caused by integer divisions can be costly on smaller
scale values since the decimal part is significant compared to the
integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
cases to maintain accuracy.

Signed-off-by: Liam Beguin <lvb@xiphos.com>
---
drivers/iio/afe/iio-rescale.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
index c408c4057c08..7304306c9806 100644
--- a/drivers/iio/afe/iio-rescale.c
+++ b/drivers/iio/afe/iio-rescale.c
@@ -22,7 +22,7 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
int *val, int *val2)
{
s64 tmp;
- s32 rem;
+ s32 rem, rem2;
u32 mult;
u32 neg;

@@ -38,8 +38,31 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
tmp = (s64)*val * 1000000000LL;
tmp = div_s64(tmp, rescale->denominator);
tmp *= rescale->numerator;
- tmp = div_s64(tmp, 1000000000LL);
+
+ tmp = div_s64_rem(tmp, 1000000000LL, &rem);
*val = tmp;
+
+ /*
+ * For small values, the approximation can be costly,
+ * change scale type to maintain accuracy.
+ *
+ * 100 vs. 10000000 NANO caps the error to about 100 ppm.
+ */
+ if (scale_type == IIO_VAL_FRACTIONAL)
+ tmp = *val2;
+ else
+ tmp = 1 << *val2;
+
+ if (abs(rem) > 10000000 && abs(*val / tmp) < 100) {
+ *val = div_s64_rem(*val, tmp, &rem2);
+
+ *val2 = div_s64(rem, tmp);
+ if (rem2)
+ *val2 += div_s64(rem2 * 1000000000LL, tmp);
+
+ return IIO_VAL_INT_PLUS_NANO;
+ }
+
return scale_type;
case IIO_VAL_INT_PLUS_NANO:
case IIO_VAL_INT_PLUS_MICRO:
--
2.32.0.452.g940fe202adcb
\
 
 \ /
  Last update: 2021-08-20 21:19    [W:0.824 / U:0.068 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site