velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "E.R. van Es" <e...@cdls.nl>
Subject Re: stress application
Date Thu, 27 Oct 2005 13:09:41 GMT
Hi,

  I gave the program 800Mb of memory, enabled caching and using a BufferedWriter, and the
program runs with no noticable slow-down at all.  Maybe I'll stress-test with twice the amount
of information for fun :)  Is it possible to do something like "memory leak detection" using
some kind of debugging on java programs?

Greets,
Eric

On Thu, 27 Oct 2005 09:26:00 +0200
"E.R. van Es" <eric@cdls.nl> wrote:

> The data is read from a plain text file and put in simple 'data' objects first.  I need
all data from the file to be available before processing the templates, so all data objects
are saved in an ArrayList, the getData() method search through the data objects ArrayList
for the key matching and returns only those data objects needed in a new ArrayList.
> 
> Greetings,
> Eric
> 
> On Thu, 27 Oct 2005 08:09:57 +0100
> Andrew Mason <andrew@assertis.co.uk> wrote:
> 
> > Just out of curiosity where is it gettng the data from ? database ? If so are 
> > you using a connection pool?
> > 
> > private 
> > 
> > On Thu October 27 2005 8:03 am, E.R. van Es wrote:
> > > Hi,
> > >
> > > I'm running a test now with the BufferedWriter, we'll see what happens. 
> > > About the getData() method, maybe this actually is the leak, it does this:
> > >
> > > *** CODE ***
> > > private ArrayList getData(String key) {
> > > 	ArrayList temp = new ArrayList();
> > > 	... fetch data here and fill temp ...
> > > 	return temp;
> > > }
> > > *** CODE END ***
> > >
> > > I'm not a complete java guru, but is there a new temp ArrayList allocated
> > > in memory every time I call this method?  won't previous created temp
> > > ArrayLists be collected by the garbage collector?
> > 
> > >
> > > Greetings,
> > > Eric
> > >
> > > On Wed, 26 Oct 2005 11:57:08 -0400
> > >
> > > James Kebinger <jkebinger@gmail.com> wrote:
> > > > I would just turn up the amount of memory java can use.
> > > > Does point to a possible memory leak somewhere though. What does
> > > > getData() do? Note you're also shadowing the data variable in
> > > > processData().
> > > >
> > > > On 10/26/05, E.R. van Es <eric@cdls.nl> wrote:
> > > > > Hi Andrew,
> > > > >
> > > > > does this really make a memory difference? The generated files
> > > > > themselves are not that big (up to 50kb each) and the writer is closed
> > > > > after each file... I didn't thought about it so I'll try of course
:)
> > > > >
> > > > > Greetings,
> > > > > Eric
> > > > >
> > > > > On Wed, 26 Oct 2005 16:12:28 +0100
> > > > >
> > > > > Andrew Mason <andrew@assertis.co.uk> wrote:
> > > > > > Have you thought about using a buffered writer ?
> > > > > >
> > > > > > On Wed October 26 2005 4:08 pm, E.R. van Es wrote:
> > > > > > > Hello Velocity List,
> > > > > > >
> > > > > > > i have to create some big monthly reports on CD. I use
a simple
> > > > > > > java program to generate all reports in HTML files using
velocity
> > > > >
> > > > > templates.
> > > > >
> > > > > > > All data is pre-loaded and then the reports are created
by
> > > > > > > iterating
> > > > >
> > > > > over
> > > > >
> > > > > > > the data. However, there are too many files to create (over
9600)
> > > > > > > and after file 7850 I get a java memory exception. I use
a scheme
> > > > > > > like
> > > > >
> > > > > this:
> > > > > > > *** JAVA CODE ***
> > > > > > > private Context context;
> > > > > > >
> > > > > > > private void processData() {
> > > > > > > ArrayList data = getDataKeys(); // this returns an array
with all
> > > > >
> > > > > "data
> > > > >
> > > > > > > keys" Iterator i = data.iterator();
> > > > > > > while (i.hasNext()) {
> > > > > > > String key = (String) i.next();
> > > > > > > ArrayList data = getData(key); // get all data objects
for this
> > > > > > > key. context.put("key", key);
> > > > > > > context.put("data", data);
> > > > > > > processTemplate("template.vm", key + ".html");
> > > > > > > }
> > > > > > > }
> > > > > > >
> > > > > > > private void processTemplate(String template, String outputFile)
{
> > > > > > > FileWriter outputFileWriter = new FileWriter(new File(outputFile));
> > > > > > >
> > > > > > > Template x = Velocity.getTemplate(template);
> > > > > > > x.merge(context, outputFileWriter);
> > > > > > >
> > > > > > > outputFileWriter.close();
> > > > > > > }
> > > > > > > *** JAVA CODE END ***
> > > > > > >
> > > > > > > *** TEMPLATE CODE ***
> > > > > > > <html>
> > > > > > > ... bla bla ...
> > > > > > > #foreach ( $el in $data )
> > > > > > > $data.getX()<br>
> > > > > > > $data.getY()<br>
> > > > > > > and so on and so forth...
> > > > > > > #end
> > > > > > > ... bla bla ...
> > > > > > > </html>
> > > > > > > *** TEMPLATE CODE END ***
> > > > > > >
> > > > > > > As you can see I put every data class in the context with
> > > > > > > identifier "data", my thought was to overwrite the previous
one and
> > > > > > > NOT get a
> > > > >
> > > > > memory
> > > > >
> > > > > > > problem, however I do get the memory problem. I can adjust
the java
> > > > >
> > > > > memory
> > > > >
> > > > > > > allocation to 1Gb but I was wondering if my approach just
isn't
> > > > >
> > > > > correct?
> > > > >
> > > > > > > I'm not sure about the processTemplate method, though it
workes
> > > > >
> > > > > perfectly
> > > > >
> > > > > > > for the first 7800 files :) By the way: I'm using velocity-1.4.
> > > > > > >
> > > > > > > Thanks in advance,
> > > > > > > Eric
> > > > > > >
> > > > > > > -------------------------------------------------------------------
> > > > > > >-- To unsubscribe, e-mail:
> > > > > > > velocity-user-unsubscribe@jakarta.apache.org For additional
> > > > > > > commands, e-mail: velocity-user-help@jakarta.apache.org
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > > > For additional commands, e-mail:
> > > > > > velocity-user-help@jakarta.apache.org
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message