FILT glitches?
-
Why is it that when I attempt !set ctype x 268435455 (1111 1111 1111 1111 1111 1111 1111) it instead sets it to 536870912 (10 0000 0000 0000 0000 0000 0000 0000)
What's more, when I try to set it to 536870913, it just stays 536870912.
It's annoying, because it forces me to set large numbers manually without the console.
-
For me, 268435455 goes to 268435456. This is a rounding error which will be fixed in the next version. In the meantime, PROP tool should work.
If 268435455 goes to 536870912, then I have no idea what's wrong...
(Also, PROP and console accept hexadecimal, e.g. 0xFFFFFFF = 268435455. Although large hexadecimal numbers in the console suffer from the same problem as large decimal numbers in the current version.)
Edited 4 times by jacksonmj. Last: 1st Mar 2015 -
I may have gotten 268435456 and 536870912 mixed up.
Anyway, try setting any large number. I've found at around about 24+ bits, it's all over the shop. It's a good thing these problems don't seem to happen when I am doing bitwise operations and setting them with BRAY/DTEC ect. Just the console.
-
It's specific to parsing strings into integers in the console, so does not affect reactions or the PROP tool.
The problem occurs because the string is converted into an integer, then a float, then back to an integer. A single precision float has 23 significand bits (plus one implied bit at the start), so can store all integers up to 2^24 exactly. Storing anything larger may cause the number to be rounded to fit into the float.
Edited once by jacksonmj. Last: 1st Mar 2015
