velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Parks" <davidpark...@yahoo.com>
Subject Decimal and rounding error - two problems
Date Wed, 03 Nov 2010 11:00:00 GMT
I ran into a problem with decimal formats. Example:

$math.roundTo(10, 0.158343834599)
Actual Result   : 0.15834383478471944
Expected result : 0.1583438346

Aside from roundTo(...) producing incorrect numbers of decimal places (when
10 or more is entered as param #1) the rounding is blatantly incorrect (I
expect this is due to Decimal rounding error).

I wanted to fix this by writing my own methods to use BigDecimal, but here I
ran into a problem. When I implemented:

	public Number myRoundTo(int decimalPlaces, String num){
		BigDecimal bg = new BigDecimal(num);
		return bg.setScale(decimalPlaces, RoundingMode.HALF_UP);
	}

A call to $mymath.myRoundTo(2, 1.234) is never called. This is because 1.234
is converted to a Decimal and velocity finds no method with a signature of a
(Integer, Decimal) (I can accept an Object, but then I still get a Decimal
which will produce rounding error as documented in BigDecimal javadocs). My
concerns are:

1) Why can't velocity convert a number parameter to a String or BigDecimal
format if the method signature calls for that. This should not be complex
using reflection. If someone will even point me to the right spot in the
code I might be able to post a quick patch for it.

2) roundTo(...) should be fixed to avoid Decimal use, this unfortunately
seems to depend on #1. In fact Decimals should be avoided altogether for
view related cases, the performance gain over BigDecimal just isn't going to
be notable in this context. 

3) Why convert a number - like the 2nd param in: $math.roundTo(3, 1.234) -
to a Decimal rather than a BigDecimal? Seems more logical to convert it to
BigDecimal then pass it to the method as a Number.

For now my solution is the inelegant:

#mymath.myRoundTo(2, "1.234")




---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
For additional commands, e-mail: user-help@velocity.apache.org


Mime
View raw message