lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sangraal aiken" <sangr...@gmail.com>
Subject Re: Doc add limit
Date Fri, 28 Jul 2006 01:21:37 GMT
Commenting out the following line in SolrCore fixes my problem... but of
course I don't get the result status info... but this isn't a problem for me
really.

-Sangraal

writer.write("<result status=\"" + status + "\"></result>");
On 7/27/06, sangraal aiken <sangraal@gmail.com> wrote:
>
> I'm running on Tomcat... and I've verified that the complete post is
> making it through the SolrUpdate servlet and into the SolrCore object...
> thanks for the info though.
> --
> So the code is hanging on this call in SolrCore.java
>
>             writer.write("<result status=\"" + status + "\"></result>");
>
> The thread dump:
>
> "http-8080-Processor24" Id=32 in RUNNABLE (running in native) total cpu
> time= 40698.0440ms user time=38646.1680ms
>
>      at java.net.SocketOutputStream.socketWrite0(Native Method)
>      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java
> :92)
>      at java.net.SocketOutputStream.write (SocketOutputStream.java:136)
>      at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(
> InternalOutputBuffer.java:746)
>      at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
> :433)
>      at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:348)
>      at
> org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite
> (InternalOutputBuffer.java:769)
>      at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite (
> ChunkedOutputFilter.java:125)
>      at org.apache.coyote.http11.InternalOutputBuffer.doWrite(
> InternalOutputBuffer.java:579)
>      at org.apache.coyote.Response.doWrite(Response.java:559)
>      at org.apache.catalina.connector.OutputBuffer.realWriteBytes (
> OutputBuffer.java:361)
>      at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:324)
>      at org.apache.tomcat.util.buf.IntermediateOutputStream.write(
> C2BConverter.java:235)
>      at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes (StreamEncoder.java
> :336)
>      at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(
> StreamEncoder.java:404)
>      at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java
> :408)
>      at sun.nio.cs.StreamEncoder.flush (StreamEncoder.java:152)
>      at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
>      at org.apache.tomcat.util.buf.WriteConvertor.flush(C2BConverter.java
> :184)
>      at org.apache.tomcat.util.buf.C2BConverter.flushBuffer (
> C2BConverter.java:127)
>      at org.apache.catalina.connector.OutputBuffer.realWriteChars(
> OutputBuffer.java:536)
>      at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java
> :439)
>      at org.apache.tomcat.util.buf.CharChunk.append (CharChunk.java:370)
>      at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java
> :491)
>      at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java
> :161)
>      at org.apache.catalina.connector.CoyoteWriter.write (
> CoyoteWriter.java:170)
>      at org.apache.solr.core.SolrCore.update(SolrCore.java:695)
>      at org.apache.solr.servlet.SolrUpdateServlet.doPost(
> SolrUpdateServlet.java:52)
>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
>
>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java :252)
>      at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:173)
>      at org.apache.catalina.core.StandardWrapperValve.invoke(
> StandardWrapperValve.java:213)
>      at org.apache.catalina.core.StandardContextValve.invoke (
> StandardContextValve.java:178)
>      at org.apache.catalina.core.StandardHostValve.invoke(
> StandardHostValve.java:126)
>      at org.apache.catalina.valves.ErrorReportValve.invoke(
> ErrorReportValve.java:105)
>      at org.apache.catalina.core.StandardEngineValve.invoke(
> StandardEngineValve.java:107)
>      at org.apache.catalina.connector.CoyoteAdapter.service(
> CoyoteAdapter.java:148)
>      at org.apache.coyote.http11.Http11Processor.process (
> Http11Processor.java:869)
>      at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
> (Http11BaseProtocol.java:664)
>      at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
> PoolTcpEndpoint.java:527)
>      at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
> LeaderFollowerWorkerThread.java:80)
>      at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
> ThreadPool.java :684)
>
>      at java.lang.Thread.run(Thread.java:613)
>
> On 7/27/06, Otis Gospodnetic <otis_gospodnetic@yahoo.com > wrote:
> >
> > I haven't been following the thread, but....
> > Not sure if you are using Tomcat or Jetty, but Jetty has a POST size
> > limit (set somewhere in its configs) that may be the source of the problem.
> >
> > Otis
> > P.S.
> > Just occurred to me.
> > Tomcat.  Jetty.  Tom & Jerry.  Jetty guys should have called their thing
> > Jerry or Jerrymouse.
> >
> > ----- Original Message ----
> > From: Mike Klaas < mike.klaas@gmail.com>
> > To: solr-user@lucene.apache.org
> > Sent: Thursday, July 27, 2006 6:33:16 PM
> > Subject: Re: Doc add limit
> >
> > Hi Sangraal:
> >
> > Sorry--I tried not to imply that this might affect your issue.  You
> > may have to crank up the solr logging to determine where it is
> > freezing (and what might be happening).
> >
> > It is certainly worth investigating why this occurs, but I wonder
> > about the advantages of using such huge batches.  Assuming a few
> > hundred bytes per document, 6100 docs produces a POST over 1MB in
> > size.
> >
> > -Mike
> >
> > On 7/27/06, sangraal aiken <sangraal@gmail.com> wrote:
> > > Mike,
> > >  I've been posting with the content type set like this:
> > >       conn.setRequestProperty("Content-Type",
> > "application/octet-stream");
> > >
> > > I tried your suggestion though, and unfortunately there was no change.
> > >       conn.setRequestProperty ("Content-Type", "text/xml;
> > charset=utf-8");
> > >
> > > -Sangraal
> > >
> > >
> > > On 7/27/06, Mike Klaas <mike.klaas@gmail.com> wrote:
> > > >
> > > > On 7/27/06, Yonik Seeley <yonik@apache.org> wrote:
> > > >
> > > > > class SolrConnection:
> > > > >   def __init__(self, host='localhost:8983', solrBase='/solr'):
> > > > >     self.host = host
> > > > >     self.solrBase = solrBase
> > > > >     #a connection to the server is not opened at this point.
> > > > >     self.conn = httplib.HTTPConnection (self.host)
> > > > >     #self.conn.set_debuglevel(1000000)
> > > > >     self.postheaders = {"Connection":"close"}
> > > > >
> > > > >   def doUpdateXML(self, request):
> > > > >     try:
> > > > >       self.conn.request('POST', self.solrBase+'/update', request,
> > > > > self.postheaders)
> > > >
> > > > Disgressive note: I'm not sure if it is necessary with tomcat, but
> > in
> > > > my experience driving solr with python using Jetty, it was necessary
> > > > to specify the content-type when posting utf-8 data:
> > > >
> > > > self.postheaders.update({'Content-Type': 'text/xml; charset=utf-8'})
> >
> > > >
> > > > -Mike
> > > >
> > >
> > >
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message