lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sangraal aiken" <sangr...@gmail.com>
Subject Re: Doc add limit
Date Wed, 26 Jul 2006 17:31:37 GMT
Thanks for you help Yonik, I've responded to your questions below:

On 7/26/06, Yonik Seeley <yseeley@gmail.com> wrote:
>
> It's possible it's not hanging, but just takes a long time on a
> specific add.  This is because Lucene will occasionally merge
> segments.  When very large segments are merged, it can take a long
> time.


I've left it running (hung) for up to a half hour at a time and I've
verified that my cpu idles during the hang. I have witnessed much shorter
hangs on the ramp up to my 6,144 limit but they have been more like 2 - 10
seconds in length. Perhaps this is the Lucene merging you mentioned.

In the log file, add commands are followed by the number of
> milliseconds the operation took.  Next time Solr hangs, wait for a
> number of minutes until you see the operation logged and note how long
> it took.


Here are the last 5 log entries before the hang the last one is doc #6,144.
Also it looks like Tomcat is trying to redeploy the webapp those last tomcat
entries repeat indefinitely every 10 seconds or so. Perhaps this is a Tomcat
problem?

Jul 26, 2006 1:25:28 PM org.apache.solr.core.SolrCore update
INFO: add (id=110705) 0 36596
Jul 26, 2006 1:25:28 PM org.apache.solr.core.SolrCore update
INFO: add (id=110700) 0 36600
Jul 26, 2006 1:25:28 PM org.apache.solr.core.SolrCore update
INFO: add (id=110688) 0 36603
Jul 26, 2006 1:25:28 PM org.apache.solr.core.SolrCore update
INFO: add (id=110690) 0 36608
Jul 26, 2006 1:25:28 PM org.apache.solr.core.SolrCore update
INFO: add (id=110686) 0 36611
Jul 26, 2006 1:25:36 PM org.apache.catalina.startup.HostConfigcheckResources
FINE: Checking context[] redeploy resource /source/solr/apache-tomcat-5.5.17
/webapps/ROOT
Jul 26, 2006 1:25:36 PM org.apache.catalina.startup.HostConfigcheckResources
FINE: Checking context[] redeploy resource /source/solr/apache-tomcat-5.5.17
/webapps/ROOT/META-INF/context.xml
Jul 26, 2006 1:25:36 PM org.apache.catalina.startup.HostConfigcheckResources
FINE: Checking context[] reload resource /source/solr/apache-tomcat-5.5.17
/webapps/ROOT/WEB-INF/web.xml
Jul 26, 2006 1:25:36 PM org.apache.catalina.startup.HostConfigcheckResources
FINE: Checking context[] reload resource /source/solr/apache-tomcat-5.5.17
/webapps/ROOT/META-INF/context.xml
Jul 26, 2006 1:25:36 PM org.apache.catalina.startup.HostConfigcheckResources
FINE: Checking context[] reload resource /source/solr/apache-tomcat-5.5.17
/conf/context.xml

How many documents are in the index before you do a batch that causes
> a hang?  Does it happen on the first batch?  If so, you might be
> seeing some other bug.  What appserver are you using?  Do the admin
> pages respond when you see this hang?  If so, what does a stack trace
> look like?


I actually don't think I had the problem on the first batch, in fact my
first batch contained very close to 6,144 documents so perhaps there is a
relation there. Right now, I'm adding to an index with close to 90,000
documents in it.
I'm running Tomcat 5.5.17 and the admin pages respond just fine when it's
hung... I did a thread dump and this is the trace of my update:

"http-8080-Processor25" Id=33 in RUNNABLE (running in native) total cpu
time=6330.7360ms user time=5769.5920ms
     at java.net.SocketOutputStream.socketWrite0(Native Method)
     at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
     at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
     at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
     at java.io.PrintStream.write(PrintStream.java:412)
     at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java
:112)
     at sun.net.www.http.HttpClient.writeRequests(HttpClient.java:533)
     at sun.net.www.protocol.http.HttpURLConnection.writeRequests(
HttpURLConnection.java:410)
     at sun.net.www.protocol.http.HttpURLConnection.getInputStream(
HttpURLConnection.java:934)
     at com.gawker.solr.update.GanjaUpdate.doUpdate(GanjaUpdate.java:169)
     at com.gawker.solr.update.GanjaUpdate.update(GanjaUpdate.java:62)
     at org.apache.jsp.update_jsp._jspService(update_jsp.java:57)
     at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
     at org.apache.jasper.servlet.JspServletWrapper.service(
JspServletWrapper.java:332)
     at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java
:314)
     at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:252)
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:173)
     at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:213)
     at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:178)
     at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:126)
     at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:105)
     at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:107)
     at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:148)
     at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:869)
     at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)
     at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
PoolTcpEndpoint.java:527)
     at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
LeaderFollowerWorkerThread.java:80)
     at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:684)
     at java.lang.Thread.run(Thread.java:613)


-Yonik
>
>
> On 7/26/06, sangraal aiken <sangraal@gmail.com> wrote:
> > Hey there... I'm having an issue with large doc updates on my solr
> > installation. I'm adding in batches between 2-20,000 docs at a time and
> I've
> > noticed Solr seems to hang at 6,144 docs every time. Breaking the adds
> into
> > smaller batches works just fine, but I was wondering if anyone knew why
> this
> > would happen. I've tried doubling memory as well as tweaking various
> config
> > options but nothing seems to let me break the 6,144 barrier.
> >
> > This is the output from Solr admin. Any help would be greatly
> appreciated.
> >
> >
> > *name: * updateHandler  *class: *
> > org.apache.solr.update.DirectUpdateHandler2  *version: * 1.0
>   *description:
> > * Update handler that efficiently directly updates the on-disk main
> lucene
> > index  *stats: *commits : 0
> > optimizes : 0
> > docsPending : 6144
> > deletesPending : 6144
> > adds : 6144
> > deletesById : 0
> > deletesByQuery : 0
> > errors : 0
> > cumulative_adds : 6144
> > cumulative_deletesById : 0
> > cumulative_deletesByQuery : 0
> > cumulative_errors : 0
> > docsDeleted : 0
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message