As mentioned on the start page, there are two types of the conversion precisions: The double-precision floating-point and high precision calculation.
Technical details: Double-precision floating-point format is a computer number format that requires 64 bits in computer memory in binary format:
1 sign (+ or -)
11 bits for exponent
52 bits for fraction (52 digits would be the precision if we would display result in binary format). In decimal format: 252 = 4.503599627370496×1015. 15-digit precision is reduced to avoid rounding problems (from binary to decimal). Without this "reduction" some problems may appear like 0.1+0.2 = 0.30000000000000004 in JS.
Base info: Or maybe it is easier to understand over these examples:
| decimal format | exponential format |
|---|---|
| 1234.5678901234 | =0.12345678901234x104 |
| 123456789.01234 | =0.12345678901234x109 |
| 0.00012345678901234 | =0.12345678901234x10-3 |
| 1234567890123400 | =0.12345678901234x1016 |
About floating-point, you can read more at mit.edu.
Base info: For conversions, where definition between source unit to base SI unit and base SI unit to destination unit is exactly defined, high precision, up to 800 decimal places can be chosen. This definition can be exact decimal number or fraction, for example metric foot is 1/3 m and not 0.33333333, this would be rounded number, no sense for high precision.
Technical details: High precision numbers are handled as textual value, converted into "big numbers" using bignumber.js, script by Michael Mclaughlin.
