velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <lawre...@marchex.com>
Subject Re: OutOfMemoryException Processing Velocity Template
Date Fri, 06 May 2005 21:39:38 GMT
This is why I'm so confused.   I wouldn't expect it to use so much memory.  What I did to test
the memory usage was:

System.out.println("Free memory before merge: " + Runtime.getRuntime().freeMemory());
Velocity.mergeTemplate("/myfile.vm", "ISO-8859-1", context, w );
System.out.println("Free memory after merge: " + Runtime.getRuntime().freeMemory());

And then setting the JVM to -Xms256M and -Xmx256M.   When I use a String writer, the free
memory is reduced by ~200MB.   When I use an OutputStreamWriter, it is reduced ~13MB, a huge
difference.

BTW, I am using 1.4.2_05

-L
        
On Fri, May 06, 2005 at 02:22:01PM -0700, Will Glass-Husain wrote:
> Hi Lawrence,
> 
> I got interested, and just ran some similar code with JProfiler looking at 
> VM memory usage.  I created a list of 60,000 objects, each with 10 100char 
> columns.  Then I used StringWriter to merge as you did.
> 
> The VM used 3MB at the start, 6.5 MB after creating the list, and 25 after 
> merging.  (before I closed the writer).  Nowhere close to 200.  So perhaps 
> there's something else going on.  Although if the problem is one with 
> StringWriter, a lot may depend on the specific JVM you're using.  (I'm 
> using Sun 1.4.2_04-b05).
> 
> WILL
> 
> ----- Original Message ----- 
> From: <lawrence@marchex.com>
> To: "Velocity Users List" <velocity-user@jakarta.apache.org>
> Sent: Friday, May 06, 2005 2:08 PM
> Subject: Re: OutOfMemoryException Processing Velocity Template
> 
> 
> >Ok now I'm confused (suprise, suprise!).   I looked at the source for 
> >VelocityServlet, and mergeTemplate has:
> >
> >vw = (VelocityWriter) writerPool.get();
> >
> >if (vw == null)
> >{
> >   vw = new VelocityWriter(new OutputStreamWriter(output, encoding),4 * 
> >1024, true);
> >}
> >else
> >{
> >   vw.recycle(new OutputStreamWriter(output, encoding));
> >}
> >template.merge(context, vw);
> >
> >===
> >
> >So the VelocityServlet DOES use OutputStreamWriter already ... so then why 
> >does it still utilize so much memory?
> >
> >-L
> >
> >On Fri, May 06, 2005 at 02:02:07PM -0700, lawrence@marchex.com wrote:
> >>Alright, I changed the Writer to be:
> >>
> >>OutputStreamWriter w = new OutputStreamWriter(new 
> >>FileOutputStream("test.html"));
> >>
> >>And this fixes the memory problem.
> >>
> >>However, my application is running Velocity as a servlet, how should I 
> >>extend the VelocityServlet to use an OutputStreamWriter intead of a 
> >>StringWriter?  Should I post a different email thread about this?
> >>
> >>-L
> >>
> >>On Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
> >>> Well, Open a file, use a FileWriter.
> >>>
> >>> David
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: lawrence@marchex.com [mailto:lawrence@marchex.com]
> >>> Sent: Friday, May 06, 2005 1:48 PM
> >>> To: Velocity Users List
> >>> Subject: Re: OutOfMemoryException Processing Velocity Template
> >>>
> >>> I printed out the available memory before and after I call
> >>> "mergeTemplate":
> >>>
> >>> Free memory before merge: 217393352
> >>> Free memory after merge: 16271392
> >>>
> >>> so I would assume the StringWriter is approximately 200MB after the
> >>> merge.   What should I use/do instead?
> >>>
> >>> -Lawrence
> >>>
> >>> On Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
> >>> > Why do you use StringWriter? That is cached in the memory.
> >>> > How big is the StringWriter after the merge?
> >>> >
> >>> > David
> >>> >
> >>> >
> >>> > -----Original Message-----
> >>> > From: lawrence@marchex.com [mailto:lawrence@marchex.com]
> >>> > Sent: Friday, May 06, 2005 1:22 PM
> >>> > To: Velocity Users List
> >>> > Subject: Re: OutOfMemoryException Processing Velocity Template
> >>> >
> >>> > Alright, I ran this little piece of code:
> >>> >
> >>> > public static void main( String args[] ) throws Exception
> >>> >     {
> >>> >         /* first, we init the runtime engine.  Defaults are fine. */
> >>> >
> >>> >         Velocity.init();
> >>> >
> >>> >         /* lets make a Context and put data into it */
> >>> >
> >>> >         VelocityContext context = new VelocityContext();
> >>> >
> >>> >         List results = ReportBean.runLargeReport();
> >>> >
> >>> >         context.put("List", results);
> >>> >
> >>> >         /* lets render a template */
> >>> >
> >>> >         StringWriter w = new StringWriter();
> >>> >
> >>> >         Velocity.mergeTemplate("/myfile.vm", context, w );
> >>> >         System.out.println(" template : " + w );
> >>> >
> >>> >         /* lets make our own string to render */
> >>> >     }
> >>> >
> >>> > results is a List of 66000 objects and requires about 8megs of 
> >>> > memory.
> >>> > The "mergeTemplate" requires at least 230 MB of memory.   Any
> >>> > suggestions?
> >>> >
> >>> > -L
> >>> >
> >>> > On Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
> >>> > > Hi,
> >>> > >
> >>> > > I guess the writer Ilkka talked about is when using Velocity as
a
> >>> > > standalone template engine.
> >>> > >
> >>> > > Right now I don't have a good clue, could you run Velocity as
a
> >>> > > standalone template engine and render the page to see if there
is a
> >>> > > huge memory usage?
> >>> > >
> >>> > > Cheers,
> >>> > >
> >>> > > Jian
> >>> > >
> >>> > > On 5/6/05, lawrence@marchex.com <lawrence@marchex.com> wrote:
> >>> > > > Sorry, I'm not sure what you mean by what "kind of writer"
I'm
> >>> > using.   Where would I find out?   How would I change this (to maybe

> >>> > a
> >>> > StringBuffer writer?)
> >>> > > >
> >>> > > > Also, each column contains a String that is at most, 200
> >>> characters,
> >>> > most are closer to 10-20.   There are a total of 10 columns.
> >>> > > >
> >>> > > > Thanks.
> >>> > > >
> >>> > > > -Lawrence
> >>> > > >
> >>> > > > On Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
> >>> > > > > What kind of writer are you using and how long are the
items? 
> >>> > > > > 66
> >>> > 000 HTML
> >>> > > > > table rows with 5 numeric columns consumes about 32
MB when
> >>> > buffered into a
> >>> > > > > string writer. If the items are longer, it takes much
more
> >>> memory.
> >>> > > > >
> >>> > > > > -- Ilkka
> >>> > > > >
> >>> > > > >
> >>> > > > > lawrence@marchex.com wrote:
> >>> > > > > >I tried putting static HTML in the #foreach loop,
same 
> >>> > > > > >problem.
> >>> > Then I
> >>> > > > > >tried setting the large list as a "local" varible:
> >>> > > > > >
> >>> > > > > >#set ($localList = $List)
> >>> > > > > >#foreach($row in $localList)
> >>> > > > > >...
> >>> > > > > >
> >>> > > > > >but that didnt help either.   I don't see why velocity
should
> >>> use
> >>> > 250+ MB
> >>> > > > > >of memory iterating over a list ~8MB in size...
> >>> > > > > >-L
> >>> > > > > >
> >>> > > > > >On Thu, May 05, 2005 at 02:56:28PM -0700, jian chen
wrote:
> >>> > > > > >
> >>> > > > > >>Well, I don't have a good clue now.
> >>> > > > > >>
> >>> > > > > >>What about do the debugging step by step, i.e.,
use just
> >>> static
> >>> > html
> >>> > > > > >>in your #foreach loop? Will that blow up the
memory?
> >>> > > > > >>
> >>> > > > > >>Cheers,
> >>> > > > > >>
> >>> > > > > >>Jian
> >>> > > > > >>
> >>> > > > > >>
> >>> > > > > >>
> >>> > > > > >>On 5/5/05, lawrence@marchex.com <lawrence@marchex.com>
wrote:
> >>> > > > > >>
> >>> > > > > >>>Thanks for the suggestion.   I tried putting
that, along 
> >>> > > > > >>>with
> >>> > > > > >>>
> >>> > > > > >>>file.resource.loader.cache = true
> >>> > > > > >>>
> >>> > > > > >>>in my velocity.properties file.   Both did
not work.   Any
> >>> > other
> >>> > > > > >>>suggestions?
> >>> > > > > >>>
> >>> > > > > >>>Thanks!
> >>> > > > > >>>
> >>> > > > > >>>-L
> >>> > > > > >>>
> >>> > > > > >>>On Thu, May 05, 2005 at 02:11:28PM -0700,
jian chen wrote:
> >>> > > > > >>>
> >>> > > > > >>>>Hi,
> >>> > > > > >>>>
> >>> > > > > >>>>This maybe because you haven't turned
on template caching. 
> >>> > > > > >>>>I
> >>> > had
> >>> > > > > >>>>similar problem before, and even if
I only used
> >>> evaluate(...)
> >>> > method
> >>> > > > > >>>>and load the template using my own java
program, it 
> >>> > > > > >>>>happened
> >>> > when the
> >>> > > > > >>>>for loop is lot of iterations.
> >>> > > > > >>>>
> >>> > > > > >>>>The bottom line for my problem is I
need to turn on 
> >>> > > > > >>>>template
> >>> > caching,
> >>> > > > > >>>>so, velocity will not try to cache the
objects it parsed
> >>> > during each
> >>> > > > > >>>>time of the iteration.
> >>> > > > > >>>>
> >>> > > > > >>>>I did:
> >>> > > > > >>>>ri.setProperty(RuntimeConstants.FILE_RESOURCE_LOADER_CACHE,
> >>> > "true");
> >>> > > > > >>>>
> >>> > > > > >>>>where ri is a runtime instance.
> >>> > > > > >>>>
> >>> > > > > >>>>Will this fix your issue?
> >>> > > > > >>>>
> >>> > > > > >>>>Cheers,
> >>> > > > > >>>>
> >>> > > > > >>>>Jian
> >>> > > > > >>>>
> >>> > > > > >>>>
> >>> > > > > >>>>On 5/5/05, lawrence@marchex.com <lawrence@marchex.com>
> >>> wrote:
> >>> > > > > >>>>
> >>> > > > > >>>>>I am getting an OutOfMemoryException:
> >>> > > > > >>>>>
> >>> > > > > >>>>>org.apache.velocity.exception.MethodInvocationException:
> >>> > Invocation of
> >>> > > > > >>>>>method 'get' in  class
> >>> > org.apache.velocity.tools.struts.TilesTool
> >>> > > > > >>>>>threw exception class java.lang.OutOfMemoryError
: null
> >>> > > > > >>>>>
> >>> > > > > >>>>>when I am creating a large velocity
file.  The vm file
> >>> itself
> >>> > is
> >>> > > > > >>>>>fairly small, but this piece of
velocity is causing the
> >>> > exception:
> >>> > > > > >>>>>
> >>> > > > > >>>>>#foreach($row in $List)
> >>> > > > > >>>>><tr>
> >>> > > > > >>>>><td height="20" class="tbltxt">$!row.item1</td>
> >>> > > > > >>>>><td class="tbltxt">$!row.item2</td>
> >>> > > > > >>>>><td class="tbltxt">$!row.item3</td>
> >>> > > > > >>>>><td class="tbltxt">$!row.item4</td>
> >>> > > > > >>>>><td class="tbltxt">$!number.format('currency',
> >>> > $!row.item5)</td>
> >>> > > > > >>>>></tr>
> >>> > > > > >>>>>#end
> >>> > > > > >>>>>
> >>> > > > > >>>>>List is a list of relatively small
objects (5-6 Strings, a
> >>> > couple of
> >>> > > > > >>>>>BigDecimals), but the List itself
contains approximately
> >>> > 66,000
> >>> > > > > >>>>>entries.  Unfortunately, unless
I set the tomcat JVM to 
> >>> > > > > >>>>>use
> >>> a
> >>> > heap
> >>> > > > > >>>>>size of at least 256M, this velocity
template throws an
> >>> > > > > >>>>>OutOfMemoryException when it is
being processed.
> >>> > > > > >>>>>
> >>> > > > > >>>>>Any suggestions?
> >>> > > > > >>>>>
> >>> > > > > >>>>>Thanks.
> >>> > > > > >>>>>
> >>> > > > > >>>>>-L
> >>> > > > > >>>>>
> >>> > > > >
> >>> >
> >>> >>>>>-------------------------------------------------------------------
> >>> > --
> >>> > > > > >>>>>To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > >>>>>For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > > > >>>>>
> >>> > > > > >>>>>
> >>> > > > > >>>>
> >>> > > > >
> >>> >
> >>> >>>>--------------------------------------------------------------------
> >>> > -
> >>> > > > > >>>>To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > >>>>For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > > > >>>
> >>> > > > >
> >>> >
> >>> >>>---------------------------------------------------------------------
> >>> > > > > >>>To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > >>>For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > > > >>>
> >>> > > > > >>>
> >>> > > > > >>
> >>> > > > >
> >>> >
> >>> >>---------------------------------------------------------------------
> >>> > > > > >>To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > >>For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > >---------------------------------------------------------------------
> >>> > > > > >To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > >For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > > > >
> >>> > > > > >.
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > ---------------------------------------------------------------------
> >>> > > > > To unsubscribe, e-mail:
> >>> > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > > For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > >
> >>> > > >
> >>> > ---------------------------------------------------------------------
> >>> > > > To unsubscribe, e-mail:
> >>> velocity-user-unsubscribe@jakarta.apache.org
> >>> > > > For additional commands, e-mail:
> >>> > velocity-user-help@jakarta.apache.org
> >>> > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> ---------------------------------------------------------------------
> >>> > > To unsubscribe, e-mail: 
> >>> > > velocity-user-unsubscribe@jakarta.apache.org
> >>> > > For additional commands, e-mail:
> >>> velocity-user-help@jakarta.apache.org
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>> > For additional commands, e-mail: 
> >>> > velocity-user-help@jakarta.apache.org
> >>> >
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>> > For additional commands, e-mail: 
> >>> > velocity-user-help@jakarta.apache.org
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message