velocity-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <lawre...@marchex.com>
Subject Re: OutOfMemoryException Processing Velocity Template
Date Mon, 09 May 2005 16:36:50 GMT
The stack trace makes it seem like the problem still originates from the VelocityViewServlet,
and not the TilesTool:

org.apache.velocity.exception.MethodInvocationException: Invocation of method 'get' in  class
org.apache.velocity.tools.struts.TilesTool threw exception class java.lang.OutOfMemoryError
: null
	at org.apache.velocity.runtime.parser.node.ASTIdentifier.execute(ASTIdentifier.java:193)
	at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:175)
	at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:220)
	at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:230)
	at org.apache.velocity.Template.merge(Template.java:256)
	at tufte.controller.tools.servlet.TufteViewServlet.mergeTemplate(TufteViewServlet.java:106)
	at org.apache.velocity.tools.view.servlet.VelocityViewServlet.doRequest(Unknown Source)
	at org.apache.velocity.tools.view.servlet.VelocityViewServlet.doGet(Unknown Source)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:684)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:432)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:356)
	at org.apache.struts.action.RequestProcessor.doForward(RequestProcessor.java:1097)
	at org.apache.struts.tiles.TilesRequestProcessor.doForward(TilesRequestProcessor.java:302)
	at org.apache.struts.tiles.TilesRequestProcessor.processTilesDefinition(TilesRequestProcessor.java:278)
	at org.apache.struts.tiles.TilesRequestProcessor.processForwardConfig(TilesRequestProcessor.java:341)
	at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:272)
	at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1187)
	at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:433)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)
	at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:261)
	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:360)
	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:604)
	at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:562)
	at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:679)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
	at java.lang.Thread.run(Thread.java:534)

So I suppose the question is, how effecient is ServletOutputStream?  When I use an FileOutputStream,
very little memory is required.

-L

On Sat, May 07, 2005 at 11:43:03AM +0300, Ilkka Priha wrote:
> Maybe you are not referring to this memory eating template directly from 
> VelocityServlet, but through some tool, #include, etc., which uses a nested 
> StringWriter and not the servlet's OutputStreamWriter directly. According 
> to your first post, the original oom exception was thrown by the Velocity 
> Struts Tiles tool...
> 
> -- Ilkka
> 
> 
> lawrence@marchex.com wrote:
> >Ok now I'm confused (suprise, suprise!).   I looked at the source for 
> >VelocityServlet, and mergeTemplate has:
> >
> >vw = (VelocityWriter) writerPool.get();
> >
> >if (vw == null)
> >{
> >    vw = new VelocityWriter(new OutputStreamWriter(output, encoding),4 * 
> >    1024, true);
> >}
> >else
> >{
> >    vw.recycle(new OutputStreamWriter(output, encoding));
> >}
> >template.merge(context, vw);
> >
> >===
> >
> >So the VelocityServlet DOES use OutputStreamWriter already ... so then why 
> >does it still utilize so much memory?
> >
> >-L
> >
> >On Fri, May 06, 2005 at 02:02:07PM -0700, lawrence@marchex.com wrote:
> >
> >>Alright, I changed the Writer to be:
> >>
> >>OutputStreamWriter w = new OutputStreamWriter(new 
> >>FileOutputStream("test.html"));
> >>
> >>And this fixes the memory problem.
> >>
> >>However, my application is running Velocity as a servlet, how should I 
> >>extend the VelocityServlet to use an OutputStreamWriter intead of a 
> >>StringWriter?  Should I post a different email thread about this?
> >>
> >>-L
> >>
> >>On Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
> >>
> >>>Well, Open a file, use a FileWriter.
> >>>
> >>>David
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: lawrence@marchex.com [mailto:lawrence@marchex.com] 
> >>>Sent: Friday, May 06, 2005 1:48 PM
> >>>To: Velocity Users List
> >>>Subject: Re: OutOfMemoryException Processing Velocity Template
> >>>
> >>>I printed out the available memory before and after I call
> >>>"mergeTemplate":
> >>>
> >>>Free memory before merge: 217393352
> >>>Free memory after merge: 16271392
> >>>
> >>>so I would assume the StringWriter is approximately 200MB after the
> >>>merge.   What should I use/do instead?   
> >>>
> >>>-Lawrence
> >>>
> >>>On Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
> >>>
> >>>>Why do you use StringWriter? That is cached in the memory.
> >>>>How big is the StringWriter after the merge?
> >>>>
> >>>>David
> >>>>
> >>>>
> >>>>-----Original Message-----
> >>>>From: lawrence@marchex.com [mailto:lawrence@marchex.com] 
> >>>>Sent: Friday, May 06, 2005 1:22 PM
> >>>>To: Velocity Users List
> >>>>Subject: Re: OutOfMemoryException Processing Velocity Template
> >>>>
> >>>>Alright, I ran this little piece of code:
> >>>>
> >>>>public static void main( String args[] ) throws Exception
> >>>>   {
> >>>>       /* first, we init the runtime engine.  Defaults are fine. */
> >>>>
> >>>>       Velocity.init();
> >>>>
> >>>>       /* lets make a Context and put data into it */
> >>>>
> >>>>       VelocityContext context = new VelocityContext();
> >>>>
> >>>>       List results = ReportBean.runLargeReport();
> >>>>       
> >>>>       context.put("List", results);
> >>>>       
> >>>>       /* lets render a template */
> >>>>
> >>>>       StringWriter w = new StringWriter();
> >>>>
> >>>>       Velocity.mergeTemplate("/myfile.vm", context, w );
> >>>>       System.out.println(" template : " + w );
> >>>>
> >>>>       /* lets make our own string to render */
> >>>>   }
> >>>>
> >>>>results is a List of 66000 objects and requires about 8megs of memory.
> >>>>The "mergeTemplate" requires at least 230 MB of memory.   Any
> >>>>suggestions?
> >>>>
> >>>>-L
> >>>>
> >>>>On Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
> >>>>
> >>>>>Hi, 
> >>>>>
> >>>>>I guess the writer Ilkka talked about is when using Velocity as a
> >>>>>standalone template engine.
> >>>>>
> >>>>>Right now I don't have a good clue, could you run Velocity as a
> >>>>>standalone template engine and render the page to see if there is
a
> >>>>>huge memory usage?
> >>>>>
> >>>>>Cheers,
> >>>>>
> >>>>>Jian
> >>>>>
> >>>>>On 5/6/05, lawrence@marchex.com <lawrence@marchex.com> wrote:
> >>>>>
> >>>>>>Sorry, I'm not sure what you mean by what "kind of writer" I'm
> >>>>
> >>>>using.   Where would I find out?   How would I change this (to maybe
a
> >>>>StringBuffer writer?)
> >>>>
> >>>>>>Also, each column contains a String that is at most, 200
> >>>
> >>>characters,
> >>>
> >>>>most are closer to 10-20.   There are a total of 10 columns.
> >>>>
> >>>>>>Thanks.
> >>>>>>
> >>>>>>-Lawrence
> >>>>>>
> >>>>>>On Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
> >>>>>>
> >>>>>>>What kind of writer are you using and how long are the items?
66
> >>>>
> >>>>000 HTML
> >>>>
> >>>>>>>table rows with 5 numeric columns consumes about 32 MB when
> >>>>
> >>>>buffered into a
> >>>>
> >>>>>>>string writer. If the items are longer, it takes much more
> >>>
> >>>memory.
> >>>
> >>>>>>>-- Ilkka
> >>>>>>>
> >>>>>>>
> >>>>>>>lawrence@marchex.com wrote:
> >>>>>>>
> >>>>>>>>I tried putting static HTML in the #foreach loop, same
problem.
> >>>>
> >>>>Then I
> >>>>
> >>>>>>>>tried setting the large list as a "local" varible:
> >>>>>>>>
> >>>>>>>>#set ($localList = $List)
> >>>>>>>>#foreach($row in $localList)
> >>>>>>>>...
> >>>>>>>>
> >>>>>>>>but that didnt help either.   I don't see why velocity
should
> >>>
> >>>use
> >>>
> >>>>250+ MB
> >>>>
> >>>>>>>>of memory iterating over a list ~8MB in size...
> >>>>>>>>-L
> >>>>>>>>
> >>>>>>>>On Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>Well, I don't have a good clue now.
> >>>>>>>>>
> >>>>>>>>>What about do the debugging step by step, i.e., use
just
> >>>
> >>>static
> >>>
> >>>>html
> >>>>
> >>>>>>>>>in your #foreach loop? Will that blow up the memory?
> >>>>>>>>>
> >>>>>>>>>Cheers,
> >>>>>>>>>
> >>>>>>>>>Jian
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>On 5/5/05, lawrence@marchex.com <lawrence@marchex.com>
wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>Thanks for the suggestion.   I tried putting
that, along with
> >>>>>>>>>>
> >>>>>>>>>>file.resource.loader.cache = true
> >>>>>>>>>>
> >>>>>>>>>>in my velocity.properties file.   Both did not
work.   Any
> >>>>
> >>>>other
> >>>>
> >>>>>>>>>>suggestions?
> >>>>>>>>>>
> >>>>>>>>>>Thanks!
> >>>>>>>>>>
> >>>>>>>>>>-L
> >>>>>>>>>>
> >>>>>>>>>>On Thu, May 05, 2005 at 02:11:28PM -0700, jian
chen wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>Hi,
> >>>>>>>>>>>
> >>>>>>>>>>>This maybe because you haven't turned on
template caching. I
> >>>>
> >>>>had
> >>>>
> >>>>>>>>>>>similar problem before, and even if I only
used
> >>>
> >>>evaluate(...)
> >>>
> >>>>method
> >>>>
> >>>>>>>>>>>and load the template using my own java program,
it happened
> >>>>
> >>>>when the
> >>>>
> >>>>>>>>>>>for loop is lot of iterations.
> >>>>>>>>>>>
> >>>>>>>>>>>The bottom line for my problem is I need
to turn on template
> >>>>
> >>>>caching,
> >>>>
> >>>>>>>>>>>so, velocity will not try to cache the objects
it parsed
> >>>>
> >>>>during each
> >>>>
> >>>>>>>>>>>time of the iteration.
> >>>>>>>>>>>
> >>>>>>>>>>>I did:
> >>>>>>>>>>>ri.setProperty(RuntimeConstants.FILE_RESOURCE_LOADER_CACHE,
> >>>>
> >>>>"true");
> >>>>
> >>>>>>>>>>>where ri is a runtime instance.
> >>>>>>>>>>>
> >>>>>>>>>>>Will this fix your issue?
> >>>>>>>>>>>
> >>>>>>>>>>>Cheers,
> >>>>>>>>>>>
> >>>>>>>>>>>Jian
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>On 5/5/05, lawrence@marchex.com <lawrence@marchex.com>
> >>>
> >>>wrote:
> >>>
> >>>>>>>>>>>>I am getting an OutOfMemoryException:
> >>>>>>>>>>>>
> >>>>>>>>>>>>org.apache.velocity.exception.MethodInvocationException:
> >>>>
> >>>>Invocation of
> >>>>
> >>>>>>>>>>>>method 'get' in  class
> >>>>
> >>>>org.apache.velocity.tools.struts.TilesTool
> >>>>
> >>>>>>>>>>>>threw exception class java.lang.OutOfMemoryError
: null
> >>>>>>>>>>>>
> >>>>>>>>>>>>when I am creating a large velocity file.
 The vm file
> >>>
> >>>itself
> >>>
> >>>>is
> >>>>
> >>>>>>>>>>>>fairly small, but this piece of velocity
is causing the
> >>>>
> >>>>exception:
> >>>>
> >>>>>>>>>>>>#foreach($row in $List)
> >>>>>>>>>>>><tr>
> >>>>>>>>>>>><td height="20" class="tbltxt">$!row.item1</td>
> >>>>>>>>>>>><td class="tbltxt">$!row.item2</td>
> >>>>>>>>>>>><td class="tbltxt">$!row.item3</td>
> >>>>>>>>>>>><td class="tbltxt">$!row.item4</td>
> >>>>>>>>>>>><td class="tbltxt">$!number.format('currency',
> >>>>
> >>>>$!row.item5)</td>
> >>>>
> >>>>>>>>>>>></tr>
> >>>>>>>>>>>>#end
> >>>>>>>>>>>>
> >>>>>>>>>>>>List is a list of relatively small objects
(5-6 Strings, a
> >>>>
> >>>>couple of
> >>>>
> >>>>>>>>>>>>BigDecimals), but the List itself contains
approximately
> >>>>
> >>>>66,000
> >>>>
> >>>>>>>>>>>>entries.  Unfortunately, unless I set
the tomcat JVM to use
> >>>
> >>>a
> >>>
> >>>>heap
> >>>>
> >>>>>>>>>>>>size of at least 256M, this velocity
template throws an
> >>>>>>>>>>>>OutOfMemoryException when it is being
processed.
> >>>>>>>>>>>>
> >>>>>>>>>>>>Any suggestions?
> >>>>>>>>>>>>
> >>>>>>>>>>>>Thanks.
> >>>>>>>>>>>>
> >>>>>>>>>>>>-L
> >>>>>>>>>>>>
> >>>>>>>
> >>>>>>>>-------------------------------------------------------------------
> >>>>
> >>>>--
> >>>>
> >>>>>>>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>>>>>>>
> >>>>>>>--------------------------------------------------------------------
> >>>>
> >>>>-
> >>>>
> >>>>>>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>---------------------------------------------------------------------
> >>>>>>
> >>>>>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>>>>>
> >>>>>---------------------------------------------------------------------
> >>>>>
> >>>>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>>>
> >>>>>---------------------------------------------------------------------
> >>>>>
> >>>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>>>.
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>---------------------------------------------------------------------
> >>>>
> >>>>>>>To unsubscribe, e-mail:
> >>>>
> >>>>velocity-user-unsubscribe@jakarta.apache.org
> >>>>
> >>>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>
> >>>>---------------------------------------------------------------------
> >>>>
> >>>>>>To unsubscribe, e-mail:
> >>>
> >>>velocity-user-unsubscribe@jakarta.apache.org
> >>>
> >>>>>>For additional commands, e-mail:
> >>>>
> >>>>velocity-user-help@jakarta.apache.org
> >>>>
> >>>>>>
> >>>>>
> >>>---------------------------------------------------------------------
> >>>
> >>>>>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>>>>For additional commands, e-mail:
> >>>
> >>>velocity-user-help@jakarta.apache.org
> >>>
> >>>>---------------------------------------------------------------------
> >>>>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>>>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>>>
> >>>>
> >>>>---------------------------------------------------------------------
> >>>>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>>>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>>
> >>>---------------------------------------------------------------------
> >>>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>>
> >>>
> >>>---------------------------------------------------------------------
> >>>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >.
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org


Mime
View raw message