Document Type | Technical Information
Category | Monitoring/Inspection
Applicable Product Versions | 7FS02, 7FS02PS
Document Number | TMOTI066
Overview
Testing cases where the integer value after truncating decimal operations differs from expectations
- Content: Test regarding differences in integer values after truncation that may occur when exceeding the decimal places
- Environment: Tibero 7.2.2, 7.2.3 and above
Method
Decimal Rounding Policy in Tibero
Supported decimal places and policies in Tibero
- Supported decimal places: Officially supports up to 38 decimal places for number type, but can be extended up to 40 places.
- Policy when exceeding decimal places: Rounding is performed at the exceeded decimal place.
Possible Cases
When the decimal part consists of repeating 9s and exceeds the decimal places
- Numbers in the form XXX.XXXX999999... can differ as XXX.XXXX or XXX.XXX(X+1) depending on whether rounding up or truncation is applied.
- Since exceeding decimal places with continuous 9s requires operations with very large numbers, it is unlikely to occur in real environments.
--Case where rounding causes issues
Original value: 148950.999999999999999999999999999999999999999(9)
Rounded: 148951.000000000000000000000000000000000000000
In this case, the integer part changes, causing an impact
--However, to create the above case by simple division:
999999999999999999999999999999
/1000000000000000000000000000000000000000
Requires such large-scale operations that it is unlikely to occur
Operations dividing and then multiplying by the same number
- When dividing a number results in more than 38 decimal places, and then multiplying by the same number, issues may arise.
- Ideally, the original value should be returned, but rounding during the process can cause differences due to rounding up or down.
- If rounding up occurs at the last decimal place of the division result, the value becomes slightly larger; if rounding down, slightly smaller.
- Multiplying back the divided number results in a value between (X-1).99999....9999 and X.000000.....1, so truncating can yield either X-1 or X.
This can also occur with non-repeating decimals; for example, truncating 461730 may result in a value smaller by 1.
Conclusion
- It is difficult to assume that dividing and then multiplying by the same number will always return the identical number.
- Decimal places up to 38 are a very small range, so generally there is no impact.
- However, depending on the operation, differences may occur that change the integer part when truncating.