perl-asp mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joshua Chamas <jos...@chamas.com>
Subject Linux Hello World Benchmarks - 11/19/2001
Date Tue, 20 Nov 2001 01:05:53 GMT
Hey, [[ NUMBERS ARE BELOW ]]

It has been a while, but here's a new set of Hello World benchmarks!
What took me so long in getting these out is that the java web environments
that I had set up would keep crashing during the tests in ways that would
not only render their benchmarks meaningless, but also skewed the results
for other tests, say when a java thread would spin out of control!

You can download the latest benchmark suite to run on your system at:

  http://www.chamas.com/bench/hello.tar.gz

It seems to run on linux & solaris just fine, though the tests that
are run depend on what environments you have set up.  Please see the
distribution README on how to run the suite for yourself.

--Josh


METHODOLOGY: These latest numbers come from my running the pubbench.sh script:

  ]# cat pubbench.sh 
  #!/bin/bash

  ./bench.pl -version --ram --init-exec="perl javainit.pl -restart" -time=180 --store=results
  perl ./javainit.pl -stop

The javainit.pl will kill any of my existing java environments, and if the current
test needs java will start them up.  The test run for 180 seconds total in 60 second 
batches, and for each 60 second run the javainit.pl script will be executed.  

The --ram flag attempts to calculate the RAM used by each test on my linux system 
during each 60 second test with the results averaged across each, so for applications 
with memory leaks, the RAM used only shows 60 seconds worth of them.  The RAM
calculation looks at RAM used & SWAP used from the linux /proc/meminfo file... as
it looks at RAM before the httpd is fired up, but after the init-exec script
executes, the RAM does not take into account the java web server environments 
being started, just memory usage that occurs after they start being used.

DISCLAIMER: These numbers are just that, numbers.  Please don't let them
upset you.  If you have a positive contribution that you would like to make, 
you may download the source code and submit patches ( preferably as diff -u )
to the test suite.  This benchmark suite has evolved over years of benchmarking, 
and takes the point of view that web environments can only be well compared when 
run on the exact same system configuration... this is why the suite is done 
is such a way that you can run it on _your_ system giving you results relevant
to you!

NUMBERS: And without further adieu ... ( see NOTES below )

Test Name                           Test File Hits/sec  # of Hits Time(sec) secs/Hit  Bytes/Hit
Mem(KB)   
---------                           --------- --------- --------- --------- --------- ---------
--------- 
Apache::ASP v2.29 2000              h2000.asp  216.8     39042    180.09    0.004613  28998
    33632     
Apache::Registry v2.01 2000 mod_per h2000.reg  341.3     61442    180.04    0.002930  28179
    16081     
HTML::Embperl v1.3.0 2000           h2000.epl  113.2     20391    180.12    0.008833  28841
    20418     
HTML::Mason v1.03 2000              h2000.mas   84.0     15115    180.03    0.011911  28799
    112220    
HTML::Template v2.4 2000            h2000.htm   99.1     17848    180.03    0.010087  29152
    50292     
mod_caucho JSP 2000                 h2000.jsp   97.6     17617    180.54    0.010248  28965
    14057     
mod_jserv JSP 2000                  h2000.jsp  148.7     26773    180.04    0.006725  29408
    29613     
mod_php PHP 2000                    h2000.php  258.2     46477    180.02    0.003873  28866
    10212     
Template v2.04 Toolkit 2000         h2000.tt    53.3      9604    180.03    0.018745  28889
    55629     

Test Name                           Test File Hits/sec  # of Hits Time(sec) secs/Hit  Bytes/Hit
Mem(KB)   
---------                           --------- --------- --------- --------- --------- ---------
--------- 
Apache::ASP v2.29                   hello.asp  349.8     62985    180.05    0.002859  242
      29601     
Apache::Dispatch v0.09 handler      hello/wor  585.8    105443    180.01    0.001707  197
      9205      
Apache::ePerl                       hello.epe  347.1     62508    180.08    0.002881  218
      17477     
Apache::Registry v2.01 CGI Raw      hello.cgi  677.7    122003    180.03    0.001476  52 
      13061     
Apache::Registry v2.01 CGI.pm       hello.cgi  449.7     80967    180.05    0.002224  217
      20717     
Apache::SSI v2.16                   hello.sht  546.7     98426    180.03    0.001829  200
      13061     
CGI::SpeedyCGI CGI Raw              hello.scg  159.6     28722    180.00    0.006267  197
      9748      
CGI::SpeedyCGI CGI.pm               hello.scg  145.1     26117    180.01    0.006892  217
      10648     
HTML static                         hello.htm  879.2    150000    170.61    0.001137  312
      1982      
HTML::Embperl v1.3.0                hello.epl  463.6     83471    180.05    0.002157  221
      17333     
HTML::Mason v1.03                   hello.mas  368.5     66348    180.04    0.002714  198
      29009     
HTML::Template v2.4                 hello.htm  539.1     97056    180.03    0.001855  199
      14850     
mod_caucho JSP                      hello.jsp  147.6     26596    180.20    0.006775  231
      14442     
mod_cgi CGI Raw                     hello.cgi  156.7     28208    180.02    0.006382  197
      9506      
mod_cgi CGI.pm                      hello.cgi   11.1      2040    183.53    0.089966  217
      25893     
mod_include SSI                     hello.sht  241.8     43605    180.36    0.004136  199
      4297      
mod_jserv JSP                       hello.jsp   73.1     13209    180.81    0.013688  358
      10788     
mod_perl handler                    hello.ben  813.8    146490    180.00    0.001229  197
      5420      
mod_php PHP                         hello.php  687.5    123753    180.00    0.001455  226
      6008      
Template v2.04 Toolkit              hello.tt   469.2     84470    180.02    0.002131  199
      42472     

Test Name                           Test File Hits/sec  # of Hits Time(sec) secs/Hit  Bytes/Hit
Mem(KB)   
---------                           --------- --------- --------- --------- --------- ---------
--------- 
Apache::ASP v2.29 XSLT Hello        hxslt.xml  185.0     33292    180.00    0.005407  292
      39892     
AxKit v1.4 XSLT Hello               hxslt.xml  298.2     53678    180.03    0.003354  411
      30622     
Cocoon 1.8 mod_jserv XSLT Hello     hxslt.xml   56.1     10144    180.80    0.017823  507
      15784     

Test Name                           Test File Hits/sec  # of Hits Time(sec) secs/Hit  Bytes/Hit
Mem(KB)   
---------                           --------- --------- --------- --------- --------- ---------
--------- 
Apache::ASP v2.29 XSLT Big          hxsltbig.  138.6     24957    180.03    0.007214  21521
    50572     
AxKit v1.4 XSLT Big                 hxsltbig.  262.1     47206    180.08    0.003815  21859
    31390     
Cocoon 1.8 mod_jserv XSLT Big       hxsltbig.   79.2      9525    120.25    0.012625  21921
    29116     

Apache Server Header Tokens
---------------------------
(Unix)
Apache/1.3.20
AxKit/1.4
OpenSSL/0.9.6a
PHP/4.0.3pl1
Resin/1.2.1
mod_perl/1.25
mod_ssl/2.8.4
tomcat/1.0

PERL Version: 5.00503

JAVA Version: java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)
Classic VM (build 1.3.0, J2RE 1.3.0 IBM build cx130-20001124 (JIT enabled: jitc))

Operating System: Linux 2.2.14-9.0 (root@gate) (gcc egcs-2.91.66) #1 2CPU [gate.]

NOTES:

h2000.epl - Embperl 2.0 is much faster on this test, about the same speed as PHP,
but its not installable from CPAN which is why I don't run it.  When the release
stablizes ( from alpha/beta release ), I will run 2.0 for this test.

h2000.mas - HTML::Mason seems to have a memory leak on this test where it does not
seem to have it on the hello.mas.  If someone has a patch for the benchmark
to fix this memory leak which would be consistent with ordinary Mason programming
methods, please submit it.  If there is a real memory leak under Mason 
for this test, then perhaps this test will help the authors fix it?

h2000.jsp - mod_jserv beats mod_caucho on this test, but results flip flop for
hello.jsp.  I am not sure why this is.  The java environments tend to benchmark
unpredicatably, even for longer tests.  This may be a standard result though
and would not surprise me.

HTML Static - Static HTML should generally be much faster than any other test, but is
only slightly faster than mod_perl on my system.  This is likely because 
my box is a dual processor 450 PIII running Linux 2.2.14 ( Redhat 6.2 ),
and this version of linux does not deal efficiently with many processors.  
I imagine a benchmark on a 2.4.x linux kernel on this platform would show 
more of a difference.

Cocoon 1.8 mod_jserv XSLT Big - This test only ran for 120 seconds, instead
of 180 that all the other tests ran for.  This is because of the 3 60 seconds
test, the java process core dumped on one of them, so we only have 120 seconds
worth of data.  Also, why this test performs better than the Cocoon hxslt.xml,
I have no idea.

AxKit & Apache::ASP XSLT tests - These tests used XML::Sablotron, though they
could have used XML::LibXSLT.  For these tests it does not matter, because
the transformation ends up being cached to disk anyway.

Apache::ASP v2.29 XSLT Big - This test uses a lot of memory.  This is likely
due to Apache::ASP treating XSLT XML data & XSL templates as ASP scripts,
which involves first compiling them to perl bytecode, prior to execution
and final XSLT rendering of their output.  This will likely be fixed in
later releases by Apache::ASP detecting at runtime when files are 
static & don't have <% %> ASP code blocks, allow it to bypass the 
perl bytecode compilation process, which will help particularly for
static XSLT scenarios.

---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe@perl.apache.org
For additional commands, e-mail: asp-help@perl.apache.org


Mime
View raw message