velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Mason <and...@assertis.co.uk>
Subject Re: stress application
Date Wed, 26 Oct 2005 15:55:07 GMT
well you wouldn't think so, however as a 2nd year uni project we had to 
improve a logfile parser so that it could do X mb worth of logfiles. The main 
problem was it running out of memory. All I did was wrap the file writers in 
bufferedwriters and it allowed me to parse quite alot more logfiles. I guess 
its worth the try, you can just do

 new BufferedWriter(new FileWriter(outputFile));

so i guess its worth the 10 seconds it will take to implement just to see if 
it does have any effect.

Otherwise, make sure template caching is turned on..
erm...you may want to run the gc() every x amount of writes.

On Wed October 26 2005 4:17 pm, E.R. van Es wrote:
> Hi Andrew,
>
>   does this really make a memory difference?  The generated files
> themselves are not that big (up to 50kb each) and the writer is closed
> after each file... I didn't thought about it so I'll try of course :)
>
> Greetings,
> Eric
>
> On Wed, 26 Oct 2005 16:12:28 +0100
>
> Andrew Mason <andrew@assertis.co.uk> wrote:
> > Have you thought about using a buffered writer ?
> >
> > On Wed October 26 2005 4:08 pm, E.R. van Es wrote:
> > > Hello Velocity List,
> > >
> > >   i have to create some big monthly reports on CD.  I use a simple java
> > > program to generate all reports in HTML files using velocity templates.
> > > All data is pre-loaded and then the reports are created by iterating
> > > over the data.  However, there are too many files to create (over 9600)
> > > and after file 7850 I get a java memory exception.  I use a scheme like
> > > this:
> > >
> > > *** JAVA CODE ***
> > > private Context context;
> > >
> > > private void processData() {
> > > 	ArrayList data = getDataKeys();	// this returns an array with all
> > > "data keys" Iterator i = data.iterator();
> > > 	while (i.hasNext()) {
> > > 		String key = (String) i.next();
> > > 		ArrayList data = getData(key);	// get all data objects for this key.
> > > 		context.put("key", key);
> > > 		context.put("data", data);
> > > 		processTemplate("template.vm", key + ".html");
> > > 	}
> > > }
> > >
> > > private void processTemplate(String template, String outputFile) {
> > > 	FileWriter outputFileWriter = new FileWriter(new File(outputFile));
> > >
> > > 	Template x = Velocity.getTemplate(template);
> > > 	x.merge(context, outputFileWriter);
> > >
> > > 	outputFileWriter.close();
> > > }
> > > *** JAVA CODE END ***
> > >
> > > *** TEMPLATE CODE ***
> > > <html>
> > > 	... bla bla ...
> > > #foreach ( $el in $data )
> > > 	$data.getX()<br>
> > > 	$data.getY()<br>
> > > 	and so on and so forth...
> > > #end
> > > 	... bla bla ...
> > > </html>
> > > *** TEMPLATE CODE END ***
> > >
> > > As you can see I put every data class in the context with identifier
> > > "data", my thought was to overwrite the previous one and NOT get a
> > > memory problem, however I do get the memory problem.  I can adjust the
> > > java memory allocation to 1Gb but I was wondering if my approach just
> > > isn't correct? I'm not sure about the processTemplate method, though it
> > > workes perfectly for the first 7800 files :)  By the way: I'm using
> > > velocity-1.4.
> > >
> > > Thanks in advance,
> > > Eric
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message