lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-6138) Solr core load limit
Date Thu, 05 Jun 2014 04:48:02 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018470#comment-14018470
] 

Shawn Heisey commented on SOLR-6138:
------------------------------------

Solr itself does not have any hard limits on the number of cores that you can create, but
you are relying on other software than Solr itself.

In this case, I believe that you are running into a limitation in zookeeper, a dependency
for SolrCloud.  Zookeeper has a default max database size of 1MB.  Each new collection puts
data into zookeeper, and eventually you're going to run into this database size limit.

Search the following page for "jute.maxbuffer" to find out how to increase the maximum database
size in zookeeper:

http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html

I'm going to close this issue, because this is most likely not a problem in Solr.  If it does
turn out that there is a bug in Solr that is causing this problem, we can re-open the issue.

Please direct any followup to the Solr mailing list:

http://lucene.apache.org/solr/discussion.html


> Solr core load limit
> --------------------
>
>                 Key: SOLR-6138
>                 URL: https://issues.apache.org/jira/browse/SOLR-6138
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 4.6, 4.7.1
>         Environment: ubuntu 12.04
> memery 20G
>            Reporter: HuangTongwen
>              Labels: test
>   Original Estimate: 840h
>  Remaining Estimate: 840h
>
> We want to enrich our search ability by solr.We do an exercise for test that how many
cores in one machine solr cores can support.
> We find we can create more than 2000 cores without datas in one machine.But when we create
cores with data ,we just can create about 1000 cores,after more t han 1000 cores,we meet many
errors like following I will apend it .If you have meets the same or similar problem,please
tell me.
> I would be grateful if you could help me.
> Hear are some errors:
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:43:29 	WARN 	SolrResourceLoader 	Can't find (or read) directory to add to classloader:
/non/existent/dir/yields/warning (resolved as: /non/existent/dir/yields/warning).
> 09:46:15 	ERROR 	ShardLeaderElectionContext 	There was a problem trying to register as
the leader:org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
for /collections/ctest.test.3521/leaders/shard1
> 09:46:15 	WARN 	ElectionContext 	cancelElection did not find election node to remove
> 09:46:16 	WARN 	RecoveryStrategy 	Stopping recovery for zkNodeName=core_node1core=ctest.test.3521
> 09:46:17 	ERROR 	RecoveryStrategy 	Error while trying to recover. core=ctest.test.3521:org.apache.solr.common.SolrException:
No registered leader was found,‚Äč collection:ctest.test.3521 slice:shard1
> 09:46:17 	ERROR 	RecoveryStrategy 	Recovery failed - trying again... (0) core=ctest.test.3521
> 09:46:17 	ERROR 	RecoveryStrategy 	Recovery failed - interrupted. core=ctest.test.3521
> 09:46:17 	ERROR 	RecoveryStrategy 	Recovery failed - I give up. core=ctest.test.3521
> 09:46:18 	WARN 	RecoveryStrategy 	Stopping recovery for zkNodeName=core_node1core=ctest.test.3521
> 10:01:58 	ERROR 	SolrCore 	org.apache.solr.common.SolrException: Error handling 'status'
action
> 10:01:58 	ERROR 	SolrDispatchFilter 	null:org.apache.solr.common.SolrException: Error
handling 'status' action
> 10:15:59 	ERROR 	ZkController 	Error getting leader from zk
> 10:15:59 	ERROR 	ZkController 	Error registering SolrCore:org.apache.solr.common.SolrException:
Error getting leader from zk for shard shard1
> 10:16:18 	ERROR 	SolrCore 	org.apache.solr.common.SolrException: Error handling 'status'
action
> 10:16:18 	ERROR 	SolrDispatchFilter 	null:org.apache.solr.common.SolrException: Error
handling 'status' action



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message