velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Mason <and...@assertis.co.uk>
Subject Re: stress application
Date Thu, 27 Oct 2005 13:14:23 GMT
It could just be that it actually DOES need more memory. Out of curiosity at 
what amount of memory does it stop being a problem ?


On Thu October 27 2005 2:09 pm, E.R. van Es wrote:
> Hi,
>
>   I gave the program 800Mb of memory, enabled caching and using a
> BufferedWriter, and the program runs with no noticable slow-down at all. 
> Maybe I'll stress-test with twice the amount of information for fun :)  Is
> it possible to do something like "memory leak detection" using some kind of
> debugging on java programs?
>
> Greets,
> Eric
>
> On Thu, 27 Oct 2005 09:26:00 +0200
>
> "E.R. van Es" <eric@cdls.nl> wrote:
> > The data is read from a plain text file and put in simple 'data' objects
> > first.  I need all data from the file to be available before processing
> > the templates, so all data objects are saved in an ArrayList, the
> > getData() method search through the data objects ArrayList for the key
> > matching and returns only those data objects needed in a new ArrayList.
> >
> > Greetings,
> > Eric
> >
> > On Thu, 27 Oct 2005 08:09:57 +0100
> >
> > Andrew Mason <andrew@assertis.co.uk> wrote:
> > > Just out of curiosity where is it gettng the data from ? database ? If
> > > so are you using a connection pool?
> > >
> > > private
> > >
> > > On Thu October 27 2005 8:03 am, E.R. van Es wrote:
> > > > Hi,
> > > >
> > > > I'm running a test now with the BufferedWriter, we'll see what
> > > > happens. About the getData() method, maybe this actually is the leak,
> > > > it does this:
> > > >
> > > > *** CODE ***
> > > > private ArrayList getData(String key) {
> > > > 	ArrayList temp = new ArrayList();
> > > > 	... fetch data here and fill temp ...
> > > > 	return temp;
> > > > }
> > > > *** CODE END ***
> > > >
> > > > I'm not a complete java guru, but is there a new temp ArrayList
> > > > allocated in memory every time I call this method?  won't previous
> > > > created temp ArrayLists be collected by the garbage collector?
> > > >
> > > >
> > > > Greetings,
> > > > Eric
> > > >
> > > > On Wed, 26 Oct 2005 11:57:08 -0400
> > > >
> > > > James Kebinger <jkebinger@gmail.com> wrote:
> > > > > I would just turn up the amount of memory java can use.
> > > > > Does point to a possible memory leak somewhere though. What does
> > > > > getData() do? Note you're also shadowing the data variable in
> > > > > processData().
> > > > >
> > > > > On 10/26/05, E.R. van Es <eric@cdls.nl> wrote:
> > > > > > Hi Andrew,
> > > > > >
> > > > > > does this really make a memory difference? The generated files
> > > > > > themselves are not that big (up to 50kb each) and the writer
is
> > > > > > closed after each file... I didn't thought about it so I'll
try
> > > > > > of course :)
> > > > > >
> > > > > > Greetings,
> > > > > > Eric
> > > > > >
> > > > > > On Wed, 26 Oct 2005 16:12:28 +0100
> > > > > >
> > > > > > Andrew Mason <andrew@assertis.co.uk> wrote:
> > > > > > > Have you thought about using a buffered writer ?
> > > > > > >
> > > > > > > On Wed October 26 2005 4:08 pm, E.R. van Es wrote:
> > > > > > > > Hello Velocity List,
> > > > > > > >
> > > > > > > > i have to create some big monthly reports on CD. I
use a
> > > > > > > > simple java program to generate all reports in HTML
files
> > > > > > > > using velocity
> > > > > >
> > > > > > templates.
> > > > > >
> > > > > > > > All data is pre-loaded and then the reports are created
by
> > > > > > > > iterating
> > > > > >
> > > > > > over
> > > > > >
> > > > > > > > the data. However, there are too many files to create
(over
> > > > > > > > 9600) and after file 7850 I get a java memory exception.
I
> > > > > > > > use a scheme like
> > > > > >
> > > > > > this:
> > > > > > > > *** JAVA CODE ***
> > > > > > > > private Context context;
> > > > > > > >
> > > > > > > > private void processData() {
> > > > > > > > ArrayList data = getDataKeys(); // this returns an
array with
> > > > > > > > all
> > > > > >
> > > > > > "data
> > > > > >
> > > > > > > > keys" Iterator i = data.iterator();
> > > > > > > > while (i.hasNext()) {
> > > > > > > > String key = (String) i.next();
> > > > > > > > ArrayList data = getData(key); // get all data objects
for
> > > > > > > > this key. context.put("key", key);
> > > > > > > > context.put("data", data);
> > > > > > > > processTemplate("template.vm", key + ".html");
> > > > > > > > }
> > > > > > > > }
> > > > > > > >
> > > > > > > > private void processTemplate(String template, String
> > > > > > > > outputFile) { FileWriter outputFileWriter = new
> > > > > > > > FileWriter(new File(outputFile));
> > > > > > > >
> > > > > > > > Template x = Velocity.getTemplate(template);
> > > > > > > > x.merge(context, outputFileWriter);
> > > > > > > >
> > > > > > > > outputFileWriter.close();
> > > > > > > > }
> > > > > > > > *** JAVA CODE END ***
> > > > > > > >
> > > > > > > > *** TEMPLATE CODE ***
> > > > > > > > <html>
> > > > > > > > ... bla bla ...
> > > > > > > > #foreach ( $el in $data )
> > > > > > > > $data.getX()<br>
> > > > > > > > $data.getY()<br>
> > > > > > > > and so on and so forth...
> > > > > > > > #end
> > > > > > > > ... bla bla ...
> > > > > > > > </html>
> > > > > > > > *** TEMPLATE CODE END ***
> > > > > > > >
> > > > > > > > As you can see I put every data class in the context
with
> > > > > > > > identifier "data", my thought was to overwrite the
previous
> > > > > > > > one and NOT get a
> > > > > >
> > > > > > memory
> > > > > >
> > > > > > > > problem, however I do get the memory problem. I can
adjust
> > > > > > > > the java
> > > > > >
> > > > > > memory
> > > > > >
> > > > > > > > allocation to 1Gb but I was wondering if my approach
just
> > > > > > > > isn't
> > > > > >
> > > > > > correct?
> > > > > >
> > > > > > > > I'm not sure about the processTemplate method, though
it
> > > > > > > > workes
> > > > > >
> > > > > > perfectly
> > > > > >
> > > > > > > > for the first 7800 files :) By the way: I'm using
> > > > > > > > velocity-1.4.
> > > > > > > >
> > > > > > > > Thanks in advance,
> > > > > > > > Eric
> > > > > > > >
> > > > > > > > -------------------------------------------------------------
> > > > > > > >------ -- To unsubscribe, e-mail:
> > > > > > > > velocity-user-unsubscribe@jakarta.apache.org For additional
> > > > > > > > commands, e-mail: velocity-user-help@jakarta.apache.org
> > > > > > >
> > > > > > > ---------------------------------------------------------------
> > > > > > >------ To unsubscribe, e-mail:
> > > > > > > velocity-user-unsubscribe@jakarta.apache.org For additional
> > > > > > > commands, e-mail:
> > > > > > > velocity-user-help@jakarta.apache.org
> > > > > >
> > > > > > -----------------------------------------------------------------
> > > > > >---- To unsubscribe, e-mail:
> > > > > > velocity-user-unsubscribe@jakarta.apache.org For additional
> > > > > > commands, e-mail: velocity-user-help@jakarta.apache.org
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> > > > velocity-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message