Sunday, September 7, 2008

Precision subtraction of floats

The normal subtraction of the double values in java result in a very long answer e.g. System.out.println(0.123 - 0.100); will result in 0.022999999999999993 as opposed to the expected 0.023. The Math.round function won't work here as it Returns the closest long/int to the argument.This precision problem can be fixed using the BigDecimal class.
BigDecimal first = new BigDecimal(0.123);
BigDecimal second = new BigDecimal(0.100);
System.out.println(first.subtract(second, new MathContext(2)));
The above program rounds off the answer to a precision of 2. i.e. the result will be 0.023.

No comments:

Post a Comment