Char input to float changing the value by 1 digit

28 Nov 2016

Hey everyone,

I've run into a small problem and would like to ask the community for help to understand why it happens and the best solution. Info:

- send a distance to mbed: 123.450 mm - read the distance and convert to float using strtof - multiply the float by 1000 and set that to the return int

From time to time the value gets converted to a float like: 123.4999999 and the return int is then 123499 instead of 123450.

The problem is that this causes a drift in position over thousands of movement commands in the range of 0.020 - 0.040. I've rerun the same movement script many times and the error is always different. All of my movement commands are always multiples of 0.050 so I know exactly how far off it is but this value will change shortly to a value around 0.049 and then tracking it will be very difficult. I've also tried converting it to double using strtod and the same problem occurs.

Why does this occur? I've thought to manually remove the '.' character and then just convert directly to int but there must be an easier way?

Thanks Sebastian

30 Nov 2016

Sebastian,

It looks like you are running into float-to-integer conversion by truncation (the default) when you need rounding. You have to add that to your code yourself. After multiplying by 1000, just add 0.5 before conversion to an integer. If that is not clear, you need to post your relevant code here.

Fred

30 Nov 2016

Hi Fred,

The problem is even before the float to int conversion, it occurs when creating the char->float. Very rarely do I see this occur, but after the conversion the float will be 123.4999999 instead of 123.450. Let's take the normal case where it doesn't round down and my initial float number is 456.750, *1000 = 456750, add 0.5 = 456750.5 which will get rounded upto 456751 right?

// Retrieve the value for a given letter
float get_value( char letter, char *command)
{
    const char *cs = command;
    char *cn = NULL;
    char **ptr = nullptr;
    for (; *cs; cs++) {
        if( letter == *cs ) {
            cs++;
            float r = strtod(cs, &cn);
            if(ptr != nullptr) *ptr= cn;
            if (cn > cs)
            {   
                int distSteps = r*1000;
                return distSteps;
            }
        }
    }
    if(ptr != nullptr) *ptr= nullptr;
    return 0;
}

Thanks Sebastian

30 Nov 2016

Sebastian,

You asked " add 0.5 = 456750.5 which will get rounded upto 456751 right?". The answer is no. The floating point number is still truncated (from float 456750.5 to integer 456750). Float to integer assignment effectively just drops the digits after the decimal point.

Remember that the floating point values are composed of bits, not digits. Even when the reported value (in decimal) looks like what you expect there can be a small difference between the internal (binary) value and the reported value. Sometines that shows up as "0.99999...", sometimes it does not. You are correct that the small difference occurs at input, but that is just a fact of (binary) floating point values that must be accomodated.

The "add 0.5 before integer assignment" technique makes the subsequent truncation yeild the same (integer) result as rounding would have. Actually, for your purposes almost any small value between 0.000001 and 0.999999 might do as well as 0.5.

Fred

P.S. - Just to be explicit, the correct code is "int distSteps = r*1000 + 0.5;".

02 Dec 2016

Hi Fred,

Okay was unaware of many things you mentioned above. Will do the the r*1000 + 0.5 and see how things go. Thanks a lot for the help.

Thanks Sebastian