lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ishan Chattopadhyaya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-6184) Replication fetchLatestIndex always failed, that will occur the recovery error.
Date Sun, 02 Nov 2014 18:51:33 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14193955#comment-14193955
] 

Ishan Chattopadhyaya commented on SOLR-6184:
--------------------------------------------

[~raintung] Did you try increasing the commitReserveDuration parameter? Reserving a commit
point would ensure that the index files corresponding to the latest commit point being fetched
won't be deleted (due to, for example, lucene segment merges). 

Since it takes ~20 minutes to fetch the index, could you try setting this to ~20-25 minutes,
maybe?

> Replication fetchLatestIndex always failed, that will occur the recovery error.
> -------------------------------------------------------------------------------
>
>                 Key: SOLR-6184
>                 URL: https://issues.apache.org/jira/browse/SOLR-6184
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrCloud
>    Affects Versions: 4.6, 4.6.1
>         Environment: the index file size is more than 70G
>            Reporter: Raintung Li
>              Labels: difficulty-medium, impact-medium
>         Attachments: Solr-6184.txt
>
>
> Usually the copy full index 70G need 20 minutes at least, 100M read/write network or
disk r/w.  If in the 20 minutes happen one hard commit, that means the copy full index snap
pull will be failed, the temp folder will be removed because it is failed pull task. 
> In the production, update index will happen in every minute, redo pull task always failed
because index always change. 
> And also always redo the pull it will occur the network and disk usage keep the high
level.
> For my suggestion, the fetchLatestIndex can be do again in some frequency. Don't need
remove the tmp folder, and copy the largest index at first. Redo the fetchLatestIndex don't
download the same biggest file again, only will copy the commit index just now, at last the
task will be easy success.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message