lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-6460) Keep transaction logs around longer
Date Tue, 02 Sep 2014 00:47:20 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117807#comment-14117807
] 

Yonik Seeley commented on SOLR-6460:
------------------------------------

Some of the current tlog/peer-sync code won't work well with increased limits.
One simple example is peer-sync sends all the docs in a single message (non-streaming)...
obviously bad when the number of docs grows (guaranteed to result in an OOM exception).  This
is why this issue is separate from raising the peer-sync limit.
IIRC, another area of concern is that tlogs are kept open until they are deleted (and thus
we'll exhaust file descriptors).
Other code (like peersync code) needs to be audited to ensure that even if we keep gigabytes
of log files around, they won't all be read during a peersync request (i.e. peersync limit
needs to be decoupled from tlog lifetime).  Some of this issue is just ensuring that no bad
things happen when we drastically increase the tlog lifetime limit.

> Keep transaction logs around longer
> -----------------------------------
>
>                 Key: SOLR-6460
>                 URL: https://issues.apache.org/jira/browse/SOLR-6460
>             Project: Solr
>          Issue Type: Sub-task
>            Reporter: Yonik Seeley
>
> Transaction logs are currently deleted relatively quickly... but we need to keep them
around much longer to be used as a source for cross-datacenter recovery.  This will also be
useful in the future for enabling peer-sync to use more historical updates before falling
back to replication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message