velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Kebinger <jkebin...@gmail.com>
Subject Re: stress application
Date Wed, 26 Oct 2005 16:18:40 GMT
The buffered writer will likely make things faster.
Be careful about running gc() - it's usually just a band aid that will only
alter how things break. Note its documented as just a request for garbage
collection, and may or may not do anything depending on which vm you're
running, what gc strategy has been configured, and how recently the gc has
run.

On 10/26/05, Andrew Mason <andrew@assertis.co.uk> wrote:
>
> well you wouldn't think so, however as a 2nd year uni project we had to
> improve a logfile parser so that it could do X mb worth of logfiles. The
> main
> problem was it running out of memory. All I did was wrap the file writers
> in
> bufferedwriters and it allowed me to parse quite alot more logfiles. I
> guess
> its worth the try, you can just do
>
> new BufferedWriter(new FileWriter(outputFile));
>
> so i guess its worth the 10 seconds it will take to implement just to see
> if
> it does have any effect.
>
> Otherwise, make sure template caching is turned on..
> erm...you may want to run the gc() every x amount of writes.
>
> On Wed October 26 2005 4:17 pm, E.R. van Es wrote:
> > Hi Andrew,
> >
> > does this really make a memory difference? The generated files
> > themselves are not that big (up to 50kb each) and the writer is closed
> > after each file... I didn't thought about it so I'll try of course :)
> >
> > Greetings,
> > Eric
> >
> > On Wed, 26 Oct 2005 16:12:28 +0100
> >
> > Andrew Mason <andrew@assertis.co.uk> wrote:
> > > Have you thought about using a buffered writer ?
> > >
> > > On Wed October 26 2005 4:08 pm, E.R. van Es wrote:
> > > > Hello Velocity List,
> > > >
> > > > i have to create some big monthly reports on CD. I use a simple java
> > > > program to generate all reports in HTML files using velocity
> templates.
> > > > All data is pre-loaded and then the reports are created by iterating
> > > > over the data. However, there are too many files to create (over
> 9600)
> > > > and after file 7850 I get a java memory exception. I use a scheme
> like
> > > > this:
> > > >
> > > > *** JAVA CODE ***
> > > > private Context context;
> > > >
> > > > private void processData() {
> > > > ArrayList data = getDataKeys(); // this returns an array with all
> > > > "data keys" Iterator i = data.iterator();
> > > > while (i.hasNext()) {
> > > > String key = (String) i.next();
> > > > ArrayList data = getData(key); // get all data objects for this key.
> > > > context.put("key", key);
> > > > context.put("data", data);
> > > > processTemplate("template.vm", key + ".html");
> > > > }
> > > > }
> > > >
> > > > private void processTemplate(String template, String outputFile) {
> > > > FileWriter outputFileWriter = new FileWriter(new File(outputFile));
> > > >
> > > > Template x = Velocity.getTemplate(template);
> > > > x.merge(context, outputFileWriter);
> > > >
> > > > outputFileWriter.close();
> > > > }
> > > > *** JAVA CODE END ***
> > > >
> > > > *** TEMPLATE CODE ***
> > > > <html>
> > > > ... bla bla ...
> > > > #foreach ( $el in $data )
> > > > $data.getX()<br>
> > > > $data.getY()<br>
> > > > and so on and so forth...
> > > > #end
> > > > ... bla bla ...
> > > > </html>
> > > > *** TEMPLATE CODE END ***
> > > >
> > > > As you can see I put every data class in the context with identifier
> > > > "data", my thought was to overwrite the previous one and NOT get a
> > > > memory problem, however I do get the memory problem. I can adjust
> the
> > > > java memory allocation to 1Gb but I was wondering if my approach
> just
> > > > isn't correct? I'm not sure about the processTemplate method, though
> it
> > > > workes perfectly for the first 7800 files :) By the way: I'm using
> > > > velocity-1.4.
> > > >
> > > > Thanks in advance,
> > > > Eric
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> velocity-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message