So, I'm working on that msh format structure and I'm running into some incredibly large numbers.
Not large as in milllions and billions, but large as in 15, 20 and more places to the right of the decimal.
Does anyone here know (or even have a guess at) how precise XSI is with things like point locations?
I know the units are usually 3.something tall (approximatly). Just don't know how important that 10th or 11th decimal place is.
I Have the prog, just not installed and am hoping to save a few hours.
-edit- just realized this isn't really the proper place for this Question. :(
Feel free to move it to Mod Factory or Requests.
-edit
After a lot of trial and error with raw data conversion, I have determined that it is quite safe to use a precision of 6 decimal places.
The nature of floating point numbers (if I understand the IEEE spec correctly) is that the number should be reproducable with a minimum of 9 decimal places.
Of course, most systems are much more acurate than that, but knowing that minimum gives me a boundry to work with and keep my numbers at a sane length.
As a result, I will use 9 places in all my calculations, and only display however many are contextually appropriate and readable.
0.900000000 is just not right to display 90%.
0.90 is better, in that case.
For vertecies, normals, uv locations, rotation quaternions, excetera..., the 9 places should be more than precise enough.
edit by tirpider: tagged title