# velocity-user mailing list archives

##### Site index · List index
Message view
Top
From Claude Brisson <cla...@renegat.net>
Subject Re: Decimal and rounding error - two problems
Date Wed, 03 Nov 2010 13:09:06 GMT
On 03/11/2010 12:00, David Parks wrote:
> I ran into a problem with decimal formats. Example:
>
> \$math.roundTo(10, 0.158343834599)
> Actual Result   : 0.15834383478471944
> Expected result : 0.1583438346
>
>
According to its souce code, the roundTo method is, for the time being,
limited to 9 digits.
> Aside from roundTo(...) producing incorrect numbers of decimal places (when
> 10 or more is entered as param #1) the rounding is blatantly incorrect (I
> expect this is due to Decimal rounding error).
>
> I wanted to fix this by writing my own methods to use BigDecimal, but here I
> ran into a problem. When I implemented:
>
> 	public Number myRoundTo(int decimalPlaces, String num){
> 		BigDecimal bg = new BigDecimal(num);
> 		return bg.setScale(decimalPlaces, RoundingMode.HALF_UP);
> 	}
>
> A call to \$mymath.myRoundTo(2, 1.234) is never called. This is because 1.234
> is converted to a Decimal and velocity finds no method with a signature of a
> (Integer, Decimal) (I can accept an Object, but then I still get a Decimal
> which will produce rounding error as documented in BigDecimal javadocs). My
> concerns are:
>
> 1) Why can't velocity convert a number parameter to a String or BigDecimal
> format if the method signature calls for that. This should not be complex
> using reflection. If someone will even point me to the right spot in the
> code I might be able to post a quick patch for it.
>
We are already using reflection to automatize several conversions - but
conversions between numbers and strings are not always done magically.
As you found out, quoting numbers is an easy way to convert them into
strings.
> 2) roundTo(...) should be fixed to avoid Decimal use, this unfortunately
> seems to depend on #1. In fact Decimals should be avoided altogether for
> view related cases, the performance gain over BigDecimal just isn't going to
> be notable in this context.
>
We can't make asumptions on the type of encountered numbers - neither
can we impose one subclass of Number or the other.
What we need to is to use BigDecimals when necessary and to fix the
MathTool to reach a consistent behaviour with BigDecimals.
> 3) Why convert a number - like the 2nd param in: \$math.roundTo(3, 1.234) -
> to a Decimal rather than a BigDecimal? Seems more logical to convert it to
> BigDecimal then pass it to the method as a Number.
>
>
I'm not convinced that we should use BigDecimals when unnecessary - what
would we gain?
> For now my solution is the inelegant:
>
> #mymath.myRoundTo(2, "1.234")
>
>
>

\$math.roundTo(2, 1.234) produces 1.23 as expected.

Claude

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org