velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "E.R. van Es" <e...@cdls.nl>
Subject Re: stress application
Date Thu, 27 Oct 2005 08:36:22 GMT
Hi,

 with the BufferedWriter the program gets further but really slowing down.  takes about 7
seconds to do 1 template now, against 0,15 seconds for the first ones...  However I haven't
got a memory exception yet I have to shutdown the program for now.  I'll do another test run
with more memory available, and template caching on (I don't know whether it's on now or not
- using the default).

Greetings,
Eric

On Wed, 26 Oct 2005 12:18:40 -0400
James Kebinger <jkebinger@gmail.com> wrote:

> The buffered writer will likely make things faster.
> Be careful about running gc() - it's usually just a band aid that will only
> alter how things break. Note its documented as just a request for garbage
> collection, and may or may not do anything depending on which vm you're
> running, what gc strategy has been configured, and how recently the gc has
> run.
> 
> On 10/26/05, Andrew Mason <andrew@assertis.co.uk> wrote:
> >
> > well you wouldn't think so, however as a 2nd year uni project we had to
> > improve a logfile parser so that it could do X mb worth of logfiles. The
> > main
> > problem was it running out of memory. All I did was wrap the file writers
> > in
> > bufferedwriters and it allowed me to parse quite alot more logfiles. I
> > guess
> > its worth the try, you can just do
> >
> > new BufferedWriter(new FileWriter(outputFile));
> >
> > so i guess its worth the 10 seconds it will take to implement just to see
> > if
> > it does have any effect.
> >
> > Otherwise, make sure template caching is turned on..
> > erm...you may want to run the gc() every x amount of writes.
> >
> > On Wed October 26 2005 4:17 pm, E.R. van Es wrote:
> > > Hi Andrew,
> > >
> > > does this really make a memory difference? The generated files
> > > themselves are not that big (up to 50kb each) and the writer is closed
> > > after each file... I didn't thought about it so I'll try of course :)
> > >
> > > Greetings,
> > > Eric
> > >
> > > On Wed, 26 Oct 2005 16:12:28 +0100
> > >
> > > Andrew Mason <andrew@assertis.co.uk> wrote:
> > > > Have you thought about using a buffered writer ?
> > > >
> > > > On Wed October 26 2005 4:08 pm, E.R. van Es wrote:
> > > > > Hello Velocity List,
> > > > >
> > > > > i have to create some big monthly reports on CD. I use a simple java
> > > > > program to generate all reports in HTML files using velocity
> > templates.
> > > > > All data is pre-loaded and then the reports are created by iterating
> > > > > over the data. However, there are too many files to create (over
> > 9600)
> > > > > and after file 7850 I get a java memory exception. I use a scheme
> > like
> > > > > this:
> > > > >
> > > > > *** JAVA CODE ***
> > > > > private Context context;
> > > > >
> > > > > private void processData() {
> > > > > ArrayList data = getDataKeys(); // this returns an array with all
> > > > > "data keys" Iterator i = data.iterator();
> > > > > while (i.hasNext()) {
> > > > > String key = (String) i.next();
> > > > > ArrayList data = getData(key); // get all data objects for this key.
> > > > > context.put("key", key);
> > > > > context.put("data", data);
> > > > > processTemplate("template.vm", key + ".html");
> > > > > }
> > > > > }
> > > > >
> > > > > private void processTemplate(String template, String outputFile)
{
> > > > > FileWriter outputFileWriter = new FileWriter(new File(outputFile));
> > > > >
> > > > > Template x = Velocity.getTemplate(template);
> > > > > x.merge(context, outputFileWriter);
> > > > >
> > > > > outputFileWriter.close();
> > > > > }
> > > > > *** JAVA CODE END ***
> > > > >
> > > > > *** TEMPLATE CODE ***
> > > > > <html>
> > > > > ... bla bla ...
> > > > > #foreach ( $el in $data )
> > > > > $data.getX()<br>
> > > > > $data.getY()<br>
> > > > > and so on and so forth...
> > > > > #end
> > > > > ... bla bla ...
> > > > > </html>
> > > > > *** TEMPLATE CODE END ***
> > > > >
> > > > > As you can see I put every data class in the context with identifier
> > > > > "data", my thought was to overwrite the previous one and NOT get
a
> > > > > memory problem, however I do get the memory problem. I can adjust
> > the
> > > > > java memory allocation to 1Gb but I was wondering if my approach
> > just
> > > > > isn't correct? I'm not sure about the processTemplate method, though
> > it
> > > > > workes perfectly for the first 7800 files :) By the way: I'm using
> > > > > velocity-1.4.
> > > > >
> > > > > Thanks in advance,
> > > > > Eric
> > > > >
> > > > >
> > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail:
> > velocity-user-help@jakarta.apache.org
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message