lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene Livis <eli...@basistech.com>
Subject Unable to RESTORE collections via Collections API
Date Wed, 08 Apr 2020 21:17:33 GMT
Hello,

I have been unsuccessfully trying to find a way to restore a collection
using the Collections API for the last several days. *I would
greatly appreciate any help as I am now stuck.* I am using Solr 8.2 in
cloud mode. To simplify things, I have only a single Solr node on my local
machine (though I have also tried multi-node setups). I am able to
successfully create collections, index documents, and run searches against
the index. General gist of what I am trying to do is to be able to backup
an existing collection, delete that collection, and then restore the
collection from backup when needed. Our use case requires creating
thousands of collections, which I believe is highly not recommended in Solr
Cloud (is that correct?), so this backup-delete-restore mechanism is our
way of reducing the number of collections in Solr at any given time. Just
FYI, in our use case it is ok for searches to take a couple of minutes to
produce results.

So far I have not been able to restore a single collection using the
collection RESTORE API. I am able to create backups by running the
following HTTP command in Solr admin console:

http://localhost:8983/solr/admin/collections?action=BACKUP&name=test1_backup2&collection=test1_20200407_170438_20200407_170441&location=C:\TEST\DELETE\BACKUPS


The backup appears to be successfully created. It contains a snapshot from
"shard1", which is the only shard in the cluster. It also contains the
"zk_backup" directory, which contains my Solr config and
"collection_state.json" file. And it contains a "backup.properties" file.
Everything looks good to me.

However, when I attempt to restore the collection using the following HTTP
command:

http://localhost:8983/solr/admin/collections?action=RESTORE&name=test1_backup2&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW


I get the same extremely vague error messages in Solr logs:


*RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing
SolrCore 'test1_backup_NEW_shard1_replica_n1': Unable to create core
[test1_backup_NEW_shard1_replica_n1] Caused by: nullERROR
(qtp1099855928-24) [c:test1_backup_NEW   ] o.a.s.s.HttpSolrCall
null:org.apache.solr.common.SolrException: ADDREPLICA failed to create
replica   *

Below is the full Solr log of the restore command:

2020-04-08 21:10:56.753 INFO  (qtp1099855928-24) [   ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :restore with params
name=test1_backup2&action=RESTORE&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW
and sendToOCPQueue=true
2020-04-08 21:10:56.783 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.RestoreCmd Using existing config
AutopsyConfig
2020-04-08 21:10:56.783 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.RestoreCmd Starting restore into
collection=test1_backup_NEW with backup_name=test1_backup2 at
location=file:///C:/TEST/DELETE/BACKUPS/
2020-04-08 21:10:56.784 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.CreateCollectionCmd Create collection
test1_backup_NEW
2020-04-08 21:10:56.900 WARN
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to
create a collection (test1_backup_NEW) without cores.
2020-04-08 21:10:56.908 INFO
 (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
  ] o.a.s.c.o.SliceMutator Update shard state invoked for collection:
test1_backup_NEW with message: {
  "shard1":"construction",
  "collection":"test1_backup_NEW",
  "operation":"updateshardstate"}
2020-04-08 21:10:56.908 INFO
 (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
  ] o.a.s.c.o.SliceMutator Update shard state shard1 to construction
2020-04-08 21:10:56.925 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.RestoreCmd Adding replica for
shard=shard1
collection=DocCollection(test1_backup_NEW//collections/test1_backup_NEW/state.json/0)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":1,
  "tlogReplicas":0}
2020-04-08 21:10:56.928 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.AddReplicaCmd Node Identified
localhost:8983_solr for creating new replica of shard shard1 for collection
test1_backup_NEW
2020-04-08 21:10:56.939 INFO
 (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica
command.
2020-04-08 21:10:56.945 INFO
 (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
  ] o.a.s.c.o.SliceMutator createReplica() {
  "operation":"addreplica",
  "collection":"test1_backup_NEW",
  "shard":"shard1",
  "core":"test1_backup_NEW_shard1_replica_n1",
  "state":"down",
  "base_url":"http://localhost:8983/solr",
  "node_name":"localhost:8983_solr",
  "type":"NRT"}
2020-04-08 21:10:57.348 INFO  (qtp1099855928-19) [
x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core
create command
qt=/admin/cores&coreNodeName=core_node2&collection.configName=AutopsyConfig&name=test1_backup_NEW_shard1_replica_n1&action=CREATE&collection=test1_backup_NEW&shard=shard1&wt=javabin&version=2&replicaType=NRT
2020-04-08 21:10:57.373 WARN  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.SolrConfig Couldn't add files from C:\total\crap\dir\ignored
filtered by null to classpath: C:\total\crap\dir\ignored
2020-04-08 21:10:57.374 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.SolrResourceLoader [test1_backup_NEW_shard1_replica_n1] Added 56
libs to classloader, from paths:
[/C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/clustering/lib,
/C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/extraction/lib,
/C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/langid/lib,
/C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/velocity/lib]
2020-04-08 21:10:57.395 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.2.0
2020-04-08 21:10:57.424 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.s.IndexSchema [test1_backup_NEW_shard1_replica_n1] Schema
name=Autopsy Keyword Search
2020-04-08 21:10:57.451 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.s.IndexSchema Loaded schema Autopsy Keyword Search/2.2 with uniqueid
field id
2020-04-08 21:10:57.473 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.CoreContainer Creating SolrCore
'test1_backup_NEW_shard1_replica_n1' using configuration from collection
test1_backup_NEW, trusted=true
2020-04-08 21:10:57.473 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr.core.test1_backup_NEW.shard1.replica_n1' (registry
'solr.core.test1_backup_NEW.shard1.replica_n1') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@214b199c
2020-04-08 21:10:57.473 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.SolrCore [[test1_backup_NEW_shard1_replica_n1] ] Opening new
SolrCore at
[C:\Bitnami\solr-8.2.0-2\apache-solr\server\solr\test1_backup_NEW_shard1_replica_n1],
dataDir=[C:\Bitnami\solr-8.2.0-2\apache-solr\server\solr\test1_backup_NEW_shard1_replica_n1\data\]
2020-04-08 21:10:57.491 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
2020-04-08 21:10:57.527 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.u.CommitTracker Hard AutoCommit: if 15000 uncommitted docs;
2020-04-08 21:10:57.527 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
2020-04-08 21:10:57.535 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
main]
2020-04-08 21:10:57.536 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with
znodeBase: /configs/AutopsyConfig
2020-04-08 21:10:57.537 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/AutopsyConfig
2020-04-08 21:10:57.537 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.h.c.SpellCheckComponent Initializing spell checkers
2020-04-08 21:10:57.537 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.s.DirectSolrSpellChecker init:
{name=default,field=name,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
2020-04-08 21:10:57.540 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
2020-04-08 21:10:57.541 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.c.SolrCore [test1_backup_NEW_shard1_replica_n1]  CLOSING SolrCore
org.apache.solr.core.SolrCore@3a4eb909
2020-04-08 21:10:57.541 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.QuerySenderListener
QuerySenderListener sending requests to
Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
2020-04-08 21:10:57.541 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.test1_backup_NEW.shard1.replica_n1, tag=3a4eb909
2020-04-08 21:10:57.542 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@54eb018f: rootName =
null, domain = solr.core.test1_backup_NEW.shard1.replica_n1, service url =
null, agent id = null] for registry
solr.core.test1_backup_NEW.shard1.replica_n1 /
com.codahale.metrics.MetricRegistry@55bc77fd
2020-04-08 21:10:57.542 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.S.Request
[test1_backup_NEW_shard1_replica_n1]  webapp=null path=null
params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
hits=0 status=0 QTime=1
2020-04-08 21:10:57.542 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.QuerySenderListener
QuerySenderListener done.
2020-04-08 21:10:57.542 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading
spell index for spellchecker: default
2020-04-08 21:10:57.542 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading
spell index for spellchecker: wordbreak
2020-04-08 21:10:57.542 INFO
 (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.SolrCore
[test1_backup_NEW_shard1_replica_n1] Registered new searcher
Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
2020-04-08 21:10:57.552 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.collection.test1_backup_NEW.shard1.leader, tag=3a4eb909
2020-04-08 21:10:57.560 ERROR (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
CREATEing SolrCore 'test1_backup_NEW_shard1_replica_n1': Unable to create
core [test1_backup_NEW_shard1_replica_n1] Caused by: null
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1195)
at
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:311)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
at
org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
at
org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Unable to create core
[test1_backup_NEW_shard1_replica_n1]
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1266)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1156)
... 50 more
Caused by: org.apache.solr.common.SolrException
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1055)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:882)
at
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
... 51 more
Caused by: java.lang.NullPointerException
at
org.apache.solr.core.SolrCore.bufferUpdatesIfConstructing(SolrCore.java:1097)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1029)
... 53 more

2020-04-08 21:10:57.561 INFO  (qtp1099855928-19) [c:test1_backup_NEW
s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=AutopsyConfig&name=test1_backup_NEW_shard1_replica_n1&action=CREATE&collection=test1_backup_NEW&shard=shard1&wt=javabin&version=2&replicaType=NRT}
status=400 QTime=213
2020-04-08 21:10:57.577 ERROR
(OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error
from shard: http://localhost:8983/solr =>
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
Error from server at null: Error CREATEing SolrCore
'test1_backup_NEW_shard1_replica_n1': Unable to create core
[test1_backup_NEW_shard1_replica_n1] Caused by: null
at
org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
Error from server at null: Error CREATEing SolrCore
'test1_backup_NEW_shard1_replica_n1': Unable to create core
[test1_backup_NEW_shard1_replica_n1] Caused by: null
at
org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
~[?:?]
at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:400)
~[?:?]
at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:746)
~[?:?]
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274)
~[?:?]
at
org.apache.solr.handler.component.HttpShardHandler.request(HttpShardHandler.java:238)
~[?:?]
at
org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:199)
~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_222-1-ojdkbuild]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_222-1-ojdkbuild]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_222-1-ojdkbuild]
at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
~[metrics-core-4.0.5.jar:4.0.5]
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
~[?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_222-1-ojdkbuild]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_222-1-ojdkbuild]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222-1-ojdkbuild]
2020-04-08 21:10:57.578 ERROR
(OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
[c:test1_backup_NEW   ] o.a.s.c.a.c.OverseerCollectionMessageHandler
Collection: test1_backup_NEW operation: restore
failed:org.apache.solr.common.SolrException: ADDREPLICA failed to create
replica
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler$ShardRequestTracker.processResponses(OverseerCollectionMessageHandler.java:1030)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler$ShardRequestTracker.processResponses(OverseerCollectionMessageHandler.java:1013)
at
org.apache.solr.cloud.api.collections.AddReplicaCmd.lambda$addReplica$1(AddReplicaCmd.java:177)
at
org.apache.solr.cloud.api.collections.AddReplicaCmd.addReplica(AddReplicaCmd.java:199)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.addReplica(OverseerCollectionMessageHandler.java:708)
at
org.apache.solr.cloud.api.collections.RestoreCmd.call(RestoreCmd.java:286)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:264)
at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:505)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
Error from server at null: Error CREATEing SolrCore
'test1_backup_NEW_shard1_replica_n1': Unable to create core
[test1_backup_NEW_shard1_replica_n1] Caused by: null
at
org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:400)
at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:746)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274)
at
org.apache.solr.handler.component.HttpShardHandler.request(HttpShardHandler.java:238)
at
org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:199)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
... 4 more

2020-04-08 21:10:57.604 INFO  (qtp1099855928-24) [c:test1_backup_NEW   ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
params={name=test1_backup2&action=RESTORE&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW}
status=500 QTime=850
2020-04-08 21:10:57.605 ERROR (qtp1099855928-24) [c:test1_backup_NEW   ]
o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA
failed to create replica
at
org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)
at
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:280)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:252)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
at java.lang.Thread.run(Thread.java:748)

2020-04-08 21:10:58.775 INFO
 (OverseerCollectionConfigSetProcessor-72057883336638464-localhost:8983_solr-n_0000000003)
[   ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000014 doesn't exist.  Requestor
may have disconnected from ZooKeeper
2020-04-08 21:11:16.690 INFO  (qtp1099855928-21) [   ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/metrics
params={wt=javabin&version=2&key=solr.core.test1_backup_NEW2.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:UPDATE./update.requests&key=solr.core.test1_backup_NEW2.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_backup_NEW2.shard1.replica_n1:UPDATE./update.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_backup_NEW.shard1.replica_n1:UPDATE./update.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:UPDATE./update.requests&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:QUERY./select.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_backup_NEW.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_backup_NEW.shard1.replica_n1:INDEX.sizeInBytes}
status=0 QTime=51
2020-04-08 21:11:16.731 INFO  (qtp1099855928-24) [   ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/metrics
params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used}
status=0 QTime=27

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message