lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-12568) Config option to throttle the number of recoveries happening in parallel
Date Sat, 21 Jul 2018 23:45:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551847#comment-16551847
] 

Erick Erickson commented on SOLR-12568:
---------------------------------------

Hmmm, seems like you'd also get replicas back faster if I/O throttling so this seems like
a good idea.

Let's say you've throttled I/O to 100MBPS and have 4 replicas recovering at once. And for
simplicity, let's also claim the indexes are all exactly the same size. They'd all come back
at once T seconds from now in parallel, whereas if they're only recovering one at a time you'd
get one back live every T/4 seconds.

> Config option to throttle the number of recoveries happening in parallel
> ------------------------------------------------------------------------
>
>                 Key: SOLR-12568
>                 URL: https://issues.apache.org/jira/browse/SOLR-12568
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: SolrCloud
>            Reporter: Tomás Fernández Löbbe
>            Priority: Major
>
> I've seen cases where a node that was out of service, once it reconnects, all the replicas
hosted in it will try to recover at the same time. In some cases this can cause the host to
run out of disk space (for reasons that can cause replication to duplicate the index size,
like a new leader, or simply segment churn). I'm mostly thinking in throttling the number
of parallel recoveries in a node, but maybe it's also useful to throttle per collection/cluster
for other reasons



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message