 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : a small oddity in cint() and hex$
Page 1 of 3   |
|||||
| Author | Message | ||||
| robert.rozee Guru  Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
have just spotted something odd in one of the basic test programs i use, which i've distilled below: > a=8454399 > ? hex$(a), hex$(int(a)) 810100 8100FF > b=a/1000.0 > ? b 8454.4 > b=b-8454 > ? b 0.399414 > b=b*1000 > ? b 399.414 > ? a 8.45440e+06 > ? cint(a) 8454400 > ? int(a) 8454399 > ? fix(a) 8454399 > ? hex$(a), hex$(fix(a)) 810100 8100FF > ? hex$(a), hex$(cint(a)) 810100 810100 > now apart from the fact that i'm storing a number way too large to be represented exactly in a 6-byte real, why are cint() and int() producing different results when the fractional part is only 0.414?! the behaviour seems to also occur with hex$(), which is where i first noticed it. cheers, rob :-) addendum: make that 4-byte real!, giving an upper usable limit of 0x800000  i should have read the manual more closely! the initial tripping point was the line of code i was using to assemble three reals (holding values between 0 and 255) into a single integer: CV%=(((Int(R) << 8) Or Int(G)) << 8) Or Int(B) ' ok ' CV%=(((Int(R) << 8) + Int(G)) << 8) + Int(B) ' ok ' CV%=(((Int(R)*256)+Int(G))*256)+Int(B) ' ok ' CV%=(((R*256)+G)*256)+B ' fails the last version is the one i was using, not realizing that the interpreter was using fp maths to combine the three values, and only converting to an integer when doing the assignment to CV% Edited 2020-05-09 22:13 by robert.rozee |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
Is this on a smaller 'mite / Maximite? I think you're using something with 32-bit (4-byte) reals and have something like 20 bits for the mantissa, perhaps 19+sign, but "a" won't fit. edit: I think I mean 23 bits or so (2^23 is 8388608). Anyway, what happens is that the real's exponent is increased and you lose some precision as the nearest value which can be represented in the limited number of bits is put into the mantissa. (This occurs sooner or later with all floating point numbers.) John Edited 2020-05-09 22:55 by JohnS |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
yep, a micromite: 32-bit reals containing 23-bit fraction, 8-bit exponent. 2^23 = 0x800000, or 8388608 decimal as the largest exact decimal what has thrown me a bit is the wikipedia page that says a 32-bit float can hold "all integers with 7 or fewer decimal digits". i'm just running a test to see if this is true... and it is! am up to 10270000. this is NOT using cint(), but converting to a string using str$(a, 0, 1) and comparing str$(i%)+".0": a =8300000 i%=8300000 Do If Str$(a,0,1)<>(Str$(i%)+".0") Then Print Str$(a,0,1), i%: End a=a+1 i%=i%+1 If (i% Mod 10000)=0 Then Print i% Loop so it does look like cint() could do better, but i'd not consider it an actual bug. cheers, rob :-) |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
hmm... I thought the PIC32 used IEEE 754, in which case 23 bits are stored explicitly but that means 24 actually. They tweak the exponent so that the leading bit of the mantissa is a 1. Then there's no reason to store it because it's always a 1. (OK, zero is special.) Now I'm worried as I think your number should work. Oh... I think the PIC32 does not use 754 but instead some software library. John |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
a further data point, the above program found an error at 16777216: 16777216.0 16777217 so last non-error value was 16777215 = 0xFFFFFF. the 16777... rang a bell, so i went back through old copies of the micromite manual searching for it. there it was, in the version 4.5D manual (pre-64bit integers) on page 31, "The range of integers (whole numbers) that can be manipulated without loss of accuracy is ±16777100". the line has been omitted from manuals for 4.6 onwards, probably because 64-bit integers became available. so it looks like it may be a regression, albeit a very minor one. i'm guessing that hex$() uses cint, while str$() does not. geoff: can you possibly comment? cheers, rob :-) addendum: for floats larger than 8388608 (0x800000) it looks like str$(x, 0, 0) returns an incorrect result, while str$(x, 0, 1) returns a correct result: Print Str$(a,0,1), Str$(a,0,0),, Str$(a) 8388600.0 8388600 8.38860e+06 8388601.0 8388601 8.38860e+06 8388602.0 8388602 8.38860e+06 8388603.0 8388603 8.38860e+06 8388604.0 8388604 8.38860e+06 8388605.0 8388605 8.38860e+06 8388606.0 8388606 8.38861e+06 8388607.0 8388607 8.38861e+06 8388608.0 8388608 8.38861e+06 8388609.0 8388610 8.38861e+06 8388610.0 8388610 8.38861e+06 8388611.0 8388612 8.38861e+06 8388612.0 8388612 8.38861e+06 8388613.0 8388614 8.38861e+06 8388614.0 8388614 8.38861e+06 8388615.0 8388616 8.38861e+06 8388616.0 8388616 8.38862e+06 Edited 2020-05-10 14:21 by robert.rozee |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
i think the problem may be in the function FloatToInt64 in the source file MMBasic.c: long long int FloatToInt64(MMFLOAT x) { if(x < (-(0x7fffffffffffffffLL) -1) - 0.5 || x > 0x7fffffffffffffffLL + 0.5) error("Number too large"); return (x >= 0 ? (long long int)(x + 0.5) : (long long int)(x - 0.5)) ; } (from v5.04.09 MMbasic source code) my suspicion is that the value returned should instead be: return (x >= 0 ? (long long int)(x) + 0.5 : (long long int)(x) - 0.5) ; ie, 0.5 should not be added/subtracted on to the value of x until after the type cast. actually, i change my mind. while i think the solution is something different to what is above, i am still thinking the original line is creating the problem. it could be that we need to check the magnitude of x and if it is above 0x7FFFFF then only add/subtract 0.25? nope, that doesn't work. i'm now thinking that for magnitudes of x over $7FFFFF we should simply not be adding/subtracting anything. the granularity up there is only half a bit anyway, which is where we are coming unstuck. try something like this to replace the single return line: if ((x < -0x7fffff) || (x > 0x7fffff)) return (long long int)(x) else return (x >= 0 ? (long long int)(x + 0.5) : (long long int)(x - 0.5)) ; cheers, rob :-) addendum: here is the test code: a =16777208 i%=16777208 ' comment out the next two lines to check how the high range behaves. ' the high range is expected to fail for values above 0xFFFFFFFF, ' though perhaps should throw an error when it does. a =8388600 i%=8388600 z%=i%+16 Do Print i%, Str$(a,0,1), Str$(a,0,0),, a a=a+1 i%=i%+1 Loop Until i%>z% Edited 2020-05-10 16:57 by robert.rozee |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 396 |
8388609 (signed) is too large an integer to fit exactly into the 24 bit range so MMBasic bumps it to 8388610. (At a guess..it is a while since I thought about this stuff) The page about single precision I think you mention, goes on ... Precision limitations on integer values Integers between 0 and 16777216 can be exactly represented (also applies for negative integers between -16777216 and 0) Integers between 2^24=16777216 and 2^25=33554432 round to a multiple of 2 (even number) .... For practical math a bit is used for the sign, ie the 16... million possible values in 24 bits are mapped to the signed numeric integer range with the lowest 8 or so million in the negative zone. Numbers outside the range start to be approximated/rounded with larger gaps. Hence the rounding skips past 8388609 to 8388610. ref A good reason to use strong typed variables !!  Edited 2020-05-10 18:37 by zeitfest |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
but if you look at the output: 8388613.0 8388614 8.38861e+06 8388614.0 8388614 8.38861e+06 8388615.0 8388616 8.38861e+06 8388616.0 8388616 8.38862e+06 8388615 and 8388616 resolve to different numbers using Str$(a,0,1). the trick, as john pointed out earlier on, is that the leading '1' of a (binary) float is not encoded, but instead assumed to exist. this is how 24-bits are encoded into a 23-bit wide space, facilitated by there being an exponent used to 'move' the decimal point (or, in this case, binary point). the sign is held in a separate bit, giving a total of 25 bits and a range of +/-0xFFFFFF.  from: https://en.wikipedia.org/wiki/Single-precision_floating-point_format cheers, rob :-) (edit: replaced 'enabled' with 'facilitated' to make the wording clearer) Edited 2020-05-10 20:33 by robert.rozee |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
my mistake, the granularity above $7FFFFF is 1 bit. i'm also thinking that the fault may in fact be in the pic32 C compiler - the observed behaviour is almost as if it is applying some sort of odd/even rounding for fractional values over $7FFFFF. because a type cast is being employed, the actual mechanism used by the compiler is obscured. either way, eliminating the 0.5 for values over +/-$7FFFFF feels right. cheers, rob :-) addendum: not a fault in the compiler, it is designed to do this: https://en.wikipedia.org/wiki/IEEE_754#Roundings_to_nearest it is applying the IEEE754 standard of rounding 'half-to-even', "if the number falls midway, it is rounded to the nearest value with an even least significant digit; this is the default for binary floating point". this rounding is happening within the addition/subtraction, before the type cast. MYSTERY SOLVED! Edited 2020-05-11 08:19 by robert.rozee |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 396 |
(begging forgiveness here as I am not giving this the concentration it deserves, I am fuzzy from flu/whatever etc) Is that still ... ? eg should 8388609 be skipped ? ed - it seems inconsistent with the statements about handling integers to 16 mill etc. Edited 2020-05-11 10:13 by zeitfest |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
the cut-off point is at 0x7FFFFF, but 0x800000 produces the 'correct' result as the 0.5 added on is rounded down. the same holds for 0x800002, 0x800004, etc. the function FloatToInt64() is used by many functions within MMbasic, but most of them expect 'small valued' parameters so there is not an issue. the statement from the 4.5D and earlier micromite manuals: dates from when MMbasic only supported 32-bit floats and NO integer type, hence the problematic routine FloatToInt64() would likely not have existed as such. someone would need to either check the old source code, or do a test with a 4.5D or earlier firmware loaded up on an MX150, to determine if the problem existed back then. i suspect it did not. 16777100 is slightly inaccurate, but probably seemed adequate at the time. incidentally, a similar problem likely also exists with the bigger micromites that support 64-bit floats. in their case the problem exists for floats between (i think!) 0x10000000000000 and 0x1FFFFFFFFFFFFF, as well as the corresponding negative range. has geoff or peter noticed this thread? i fear it may have been overlooked as i started out without having the full story worked out - initially i knew very little about binary floating point formats. cheers, rob :-) Edited 2020-05-11 12:33 by robert.rozee |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
can confirm the same behaviour is present in 4.5D. it is harder to expose, although does shows up in hex$(). cheers, rob :-) |
||||
Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3167 |
This has turned out to be an amazingly complex issue. Thanks for all the hard work, I will implement the fix in the next Micromite release and from there it will make its way into Peter's many releases. Rob did some amazing detective work digging through the source so now I know exactly what change to make. Thanks Rob. Geoff Geoff Graham - http://geoffg.net |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
Beaten me to it - finally found it's meant to be IEEE 754 as shown in Microchip info (So yes there should be a 24-bit mantissa and thus Rob's found a bug.) John Edited 2020-05-11 23:51 by JohnS |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 396 |
Using a standard 32 bit float : (not Basic) PROGRAM z C REAL x INTEGER*4 c x = 8388609.0 c = IFIX ( x ) PRINT x PRINT c END \ 8388609.000000 8388609 Edited 2020-05-12 09:39 by zeitfest |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
try using c = IFIX ( x + 0.5 ) here is an example written in pascal (lazarus/fpc), running on a 64-bit linux system: program project1; {$R *.res} var i,z:integer; // 64-bit integer a:single; // 32-bit float begin a:=8388600; i:=trunc(a); z:=i+16; repeat write(i:8, trunc(a+0.5):16); if i<>trunc(a+0.5) then writeln('error':8) else writeln; a:=a+1.0; i:=i+1 until i>z end. 8388600 8388600 8388601 8388601 8388602 8388602 8388603 8388603 8388604 8388604 8388605 8388605 8388606 8388606 8388607 8388607 8388608 8388608 8388609 8388610 error 8388610 8388610 8388611 8388612 error 8388612 8388612 8388613 8388614 error 8388614 8388614 8388615 8388616 error 8388616 8388616 cheers, rob :-) Edited 2020-05-12 17:07 by robert.rozee |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
I wrote a C program and it shows that a 32-bit float can hold all integers in the (inclusive) range -16777216 to 16777216 (without needing any adding or whatever of 0.5 or any other value) John Edited 2020-05-12 19:01 by JohnS |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 396 |
This is for my own peace of mind than anything else, but it is a good thing to check as suggested. It is on a pic32mx170b, using Microchip's C routines/libraries (which I think are excellent) underneath (ed ie the f77 interpreter uses the standard Microchip C/math libraries) Edited 2020-05-13 10:32 by zeitfest |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
You might like to try it for the full range I posted. John |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 396 |
( carefully avoiding any bugs I wrote  )PROGRAM z C REAL x INTEGER*4 c, i i = 1 x = 8388606.0 x = 16777214.0 DO WHILE ( i < 4 ) c = IFIX ( x + 0.5 ) PRINT x, " ", c x = x + 1.0 i = i + 1 END DO i = 1 x = - 16777214.0 DO WHILE ( i < 4 ) c = IFIX ( x - 0.5 ) PRINT x, " ", c x = x - 1.0 i = i + 1 END DO END \ 16777214.000000 16777214 16777215.000000 16777215 16777216.000000 16777216 -16777214.000000 -16777214 -16777215.000000 -16777215 -16777216.000000 -16777216 Edited 2020-05-14 18:14 by zeitfest |
||||
| Page 1 of 3 |
|||||
 Print this page |
