Skip to main content

Search

Welcome to Tibero GTS!

When the Integer Value After Truncating a Decimal Operation Differs from Expectations

Document Type | Technical Information

Category | Monitoring/Inspection

Applicable Product Versions | 7FS02, 7FS02PS

Document Number | TMOTI066

 

Overview 

Testing cases where the integer value after truncating decimal operations differs from expectations

- Content: Test regarding differences in integer values after truncation that may occur when exceeding the decimal places 

- Environment: Tibero 7.2.2, 7.2.3 and above
  

 

Method

Decimal Rounding Policy in Tibero 

Supported decimal places and policies in Tibero 

  • Supported decimal places: Officially supports up to 38 decimal places for number type, but can be extended up to 40 places.
  • Policy when exceeding decimal places: Rounding is performed at the exceeded decimal place.

 

Possible Cases

When the decimal part consists of repeating 9s and exceeds the decimal places 

  • Numbers in the form XXX.XXXX999999... can differ as XXX.XXXX or XXX.XXX(X+1) depending on whether rounding up or truncation is applied.
  • Since exceeding decimal places with continuous 9s requires operations with very large numbers, it is unlikely to occur in real environments.
--Case where rounding causes issues
Original value: 148950.999999999999999999999999999999999999999(9) 
Rounded: 148951.000000000000000000000000000000000000000

In this case, the integer part changes, causing an impact

--However, to create the above case by simple division:
999999999999999999999999999999
/1000000000000000000000000000000000000000

Requires such large-scale operations that it is unlikely to occur


Operations dividing and then multiplying by the same number 

  • When dividing a number results in more than 38 decimal places, and then multiplying by the same number, issues may arise.
  • Ideally, the original value should be returned, but rounding during the process can cause differences due to rounding up or down.
  • If rounding up occurs at the last decimal place of the division result, the value becomes slightly larger; if rounding down, slightly smaller.
  • Multiplying back the divided number results in a value between (X-1).99999....9999 and X.000000.....1, so truncating can yield either X-1 or X.

This can also occur with non-repeating decimals; for example, truncating 461730 may result in a value smaller by 1.

 

Conclusion

  • It is difficult to assume that dividing and then multiplying by the same number will always return the identical number.
  • Decimal places up to 38 are a very small range, so generally there is no impact.
  • However, depending on the operation, differences may occur that change the integer part when truncating.