commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian S O'Neill <brone...@earthlink.net>
Subject Re: [TBM] Thread Benchmark example test results
Date Wed, 10 Dec 2003 06:37:19 GMT
The BenchTest has a few flaws. I wrote a few new tests and was shocked 
by the numbers. Running with two threads, each with half the iterations 
was over one hundred times slower than the single thread test. This 
makes no sense.

The main problem is the warmup thread. After you call join on the 
thread, it is useless. I would expect calling start again to throw an 
exception, but instead nothing seems to happen. Either way, thread 0 is 
always dead. This throws off the results for the single thread test, and 
the normalized results are way off too.

The second problem is including garbage collection times as part of the 
test. I don't recommend forcing a call to Runtime.gc(). If you're trying 
to determine how much time is spent collecting garbage, run with 
verbose:gc. If you want to see how many objects are allocated, run with 
a profiler.

When I commented out the warmup section and all the gc() related calls, 
the results were more like what I would expect. Doubling the threads and 
halving the iterations resulted in performance which was the same or 
only slightly worse. After these changes, here are my results:

JVM: Sun Microsystems Inc.  1.4.2
OS: Windows 2000  5.0  x86

Test Name: Baseline No Operation
Test Class: org.apache.commons.tbm.bench.BenchNOP

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	0 	1.00
2 	0 	0 	0 	?
4 	0 	0 	16 	?
8 	0 	0 	15 	?
16 	0 	0 	0 	?
32 	0 	0 	78 	?
64 	0 	0 	79 	?

Test Name: TimeZone.getDefault()
Test Class: org.apache.commons.tbm.bench.datetime.DefaultTimeZone

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	110 	1.00
2 	0 	0 	171 	1.55
4 	0 	0 	391 	3.55
8 	0 	0 	547 	4.97
16 	0 	0 	593 	5.39
32 	0 	0 	782 	7.11
64 	0 	0 	171 	1.55

Test Name: Construct SimpleDateFormat() and format new Date()
Test Class: org.apache.commons.tbm.bench.datetime.SimpleDateFormat

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	6188 	1.00
2 	0 	0 	6031 	0.97
4 	0 	0 	6391 	1.03
8 	0 	0 	8031 	1.30
16 	0 	0 	9578 	1.55
32 	0 	0 	9750 	1.58
64 	0 	0 	12125 	1.96

Test Name: Jakarta Commons-Lang FastDateFormat
Test Class: org.apache.commons.tbm.bench.datetime.CommonsLangFastDateFormat

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	1563 	1.00
2 	0 	0 	1531 	0.98
4 	0 	0 	1734 	1.11
8 	0 	0 	2516 	1.61
16 	0 	0 	2468 	1.58
32 	0 	0 	3110 	1.99
64 	0 	0 	3437 	2.20

Test Name: ThreadLocal SimpleDateFormat() and format new Date()
Test Class: 
org.apache.commons.tbm.bench.datetime.ThreadLocalSimpleDateFormat

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	719 	1.00
2 	0 	0 	719 	1.00
4 	0 	0 	718 	1.00
8 	0 	0 	735 	1.02
16 	0 	0 	750 	1.04
32 	0 	0 	781 	1.09
64 	0 	0 	813 	1.13

Test Name: java.sql.Timestamp.toString()
Test Class: org.apache.commons.tbm.bench.datetime.TimestampToString

Threads 	Memory Used 	GC Time (ms) 	Total Time (ms) 	Normalized
1 	0 	0 	3859 	1.00
2 	0 	0 	4344 	1.13
4 	0 	0 	7734 	2.00
8 	0 	0 	8281 	2.15
16 	0 	0 	8672 	2.25
32 	0 	0 	10906 	2.83
64 	0 	0 	12890 	3.34



Glenn Nielsen wrote:

> I have posted some example Thread Benchmark test results at:
>
> http://www.apache.org/~glenn/
>
>
> TBM still needs alot of work, there are only 6 benchmark tests so far.
> Three of them are for formatting a date, the results are interesting.
>
> Regards,
>
> Glenn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message