lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sangraal aiken" <sangr...@gmail.com>
Subject Re: Doc add limit
Date Fri, 28 Jul 2006 01:06:09 GMT
I'm running on Tomcat... and I've verified that the complete post is making
it through the SolrUpdate servlet and into the SolrCore object... thanks for
the info though.
--
So the code is hanging on this call in SolrCore.java

            writer.write("<result status=\"" + status + "\"></result>");

The thread dump:

"http-8080-Processor24" Id=32 in RUNNABLE (running in native) total cpu
time=40698.0440ms user time=38646.1680ms
     at java.net.SocketOutputStream.socketWrite0(Native Method)
     at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
     at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
     at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(
InternalOutputBuffer.java:746)
     at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:433)
     at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:348)
     at
org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite
(InternalOutputBuffer.java:769)
     at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(
ChunkedOutputFilter.java:125)
     at org.apache.coyote.http11.InternalOutputBuffer.doWrite(
InternalOutputBuffer.java:579)
     at org.apache.coyote.Response.doWrite(Response.java:559)
     at org.apache.catalina.connector.OutputBuffer.realWriteBytes(
OutputBuffer.java:361)
     at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:324)
     at org.apache.tomcat.util.buf.IntermediateOutputStream.write(
C2BConverter.java:235)
     at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java
:336)
     at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(
StreamEncoder.java:404)
     at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
     at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
     at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
     at org.apache.tomcat.util.buf.WriteConvertor.flush(C2BConverter.java
:184)
     at org.apache.tomcat.util.buf.C2BConverter.flushBuffer(
C2BConverter.java:127)
     at org.apache.catalina.connector.OutputBuffer.realWriteChars(
OutputBuffer.java:536)
     at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:439)
     at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java:370)
     at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java
:491)
     at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java
:161)
     at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java
:170)
     at org.apache.solr.core.SolrCore.update(SolrCore.java:695)
     at org.apache.solr.servlet.SolrUpdateServlet.doPost(
SolrUpdateServlet.java:52)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:252)
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:173)
     at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:213)
     at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:178)
     at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:126)
     at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:105)
     at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:107)
     at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:148)
     at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:869)
     at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)
     at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
PoolTcpEndpoint.java:527)
     at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
LeaderFollowerWorkerThread.java:80)
     at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:684)
     at java.lang.Thread.run(Thread.java:613)

On 7/27/06, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
>
> I haven't been following the thread, but....
> Not sure if you are using Tomcat or Jetty, but Jetty has a POST size limit
> (set somewhere in its configs) that may be the source of the problem.
>
> Otis
> P.S.
> Just occurred to me.
> Tomcat.  Jetty.  Tom & Jerry.  Jetty guys should have called their thing
> Jerry or Jerrymouse.
>
> ----- Original Message ----
> From: Mike Klaas <mike.klaas@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Thursday, July 27, 2006 6:33:16 PM
> Subject: Re: Doc add limit
>
> Hi Sangraal:
>
> Sorry--I tried not to imply that this might affect your issue.  You
> may have to crank up the solr logging to determine where it is
> freezing (and what might be happening).
>
> It is certainly worth investigating why this occurs, but I wonder
> about the advantages of using such huge batches.  Assuming a few
> hundred bytes per document, 6100 docs produces a POST over 1MB in
> size.
>
> -Mike
>
> On 7/27/06, sangraal aiken <sangraal@gmail.com> wrote:
> > Mike,
> >  I've been posting with the content type set like this:
> >       conn.setRequestProperty("Content-Type",
> "application/octet-stream");
> >
> > I tried your suggestion though, and unfortunately there was no change.
> >       conn.setRequestProperty("Content-Type", "text/xml;
> charset=utf-8");
> >
> > -Sangraal
> >
> >
> > On 7/27/06, Mike Klaas <mike.klaas@gmail.com> wrote:
> > >
> > > On 7/27/06, Yonik Seeley <yonik@apache.org> wrote:
> > >
> > > > class SolrConnection:
> > > >   def __init__(self, host='localhost:8983', solrBase='/solr'):
> > > >     self.host = host
> > > >     self.solrBase = solrBase
> > > >     #a connection to the server is not opened at this point.
> > > >     self.conn = httplib.HTTPConnection(self.host)
> > > >     #self.conn.set_debuglevel(1000000)
> > > >     self.postheaders = {"Connection":"close"}
> > > >
> > > >   def doUpdateXML(self, request):
> > > >     try:
> > > >       self.conn.request('POST', self.solrBase+'/update', request,
> > > > self.postheaders)
> > >
> > > Disgressive note: I'm not sure if it is necessary with tomcat, but in
> > > my experience driving solr with python using Jetty, it was necessary
> > > to specify the content-type when posting utf-8 data:
> > >
> > > self.postheaders.update({'Content-Type': 'text/xml; charset=utf-8'})
> > >
> > > -Mike
> > >
> >
> >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message