lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oakley, Craig (NIH/NLM/NCBI) [C]" <craig.oak...@nih.gov>
Subject RE: Underlying file changed by an external force
Date Wed, 10 May 2017 17:56:07 GMT
> You need to look at all of your core.properties files and see if any of them point to
the same data directory.

All the core.properties files are each in their own directory with no overlap.

> Second: if you issue a "kill -9" you can leave write locks lingering.

We manage our Solr instances with supervisor, which can send a "kill -9" if "kill -6" does
not suffice; but the problem tends to manifest itself at some time other than startup

The Solr version is 5.4.1, in case that is relevant.


-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Thursday, May 04, 2017 3:20 PM
To: solr-user <solr-user@lucene.apache.org>
Subject: Re: Underlying file changed by an external force

You need to look at all of your core.properties files and see if any
of them point to the same data directory.

Second: if you issue a "kill -9" you can leave write locks lingering.

Best,
Erick

On Thu, May 4, 2017 at 11:00 AM, Oakley, Craig (NIH/NLM/NCBI) [C]
<craig.oakley@nih.gov> wrote:
> We have been having problems with different collections on different SolrCloud clusters,
all seeming to be related to the write.lock file with stack traces similar to the following.
Are there any suggestions what might be the cause and what might be the solution? Thanks
>
>
> org.apache.lucene.store.AlreadyClosedException: Underlying file changed by an external
force at 2017-04-13T20:43:08.630152Z, (lock=NativeFSLock(path=/data/solr/biosample/dba_test_shard1_replica1/data/index/write.lock,impl=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],ctime=2017-04-13T20:43:08.630152Z))
>
>        at org.apache.lucene.store.NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:179)
>
>        at org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:37)
>
>        at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:732)
>
>        at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:503)
>
>        at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:448)
>
>        at org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2099)
>
>        at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2041)
>
>        at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1083)
>
>        at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1125)
>
>        at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:131)
>
>        at org.apache.solr.update.DefaultSolrCoreState.changeWriter(DefaultSolrCoreState.java:183)
>
>        at org.apache.solr.update.DefaultSolrCoreState.newIndexWriter(DefaultSolrCoreState.java:207)
>
>        at org.apache.solr.core.SolrCore.reload(SolrCore.java:472)
>
>        at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:849)
>
>        at org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:768)
>
>        at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:230)
>
>        at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:184)
>
>        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>
>        at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
>
>        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
>
>        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
>
>        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
>
>        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>
>        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>
>        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>
>        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>
>        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>
>        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>
>        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>
>        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>
>        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>
>        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>
>        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>
>        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>
>        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>
>        at org.eclipse.jetty.server.Server.handle(Server.java:499)
>
>        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>
>        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>
>        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>
>        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>
>        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>
>        at java.lang.Thread.run(Thread.java:745)
>
Mime
View raw message