cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default
Date Thu, 02 Jan 2014 18:59:51 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13860604#comment-13860604
] 

Jonathan Ellis commented on CASSANDRA-5351:
-------------------------------------------

Hmm, you should merge/rebase to latest trunk, or at least grab the logback settings to get
thread names in there.

But it looks like what I predicted, you have a compaction stomping on your to-be-repaired
sstable:

{noformat}
INFO  16:17:01 Compacting [SSTableReader(path='/Users/lyubentodorov/.ccm/5351_dec8/node2/data/system/local/system-local-jc-4-Data.db'),
SSTableReader(path='/Users/lyubentodorov/.ccm/5351_dec8/node2/data/system/local/system-local-jc-2-Data.db'),
SSTableReader(path='/Users/lyubentodorov/.ccm/5351_dec8/node2/data/system/local/system-local-jc-1-Data.db'),
SSTableReader(path='/Users/lyubentodorov/.ccm/5351_dec8/node2/data/system/local/system-local-jc-3-Data.db')]
...
DEBUG 16:17:01 Marking /Users/lyubentodorov/.ccm/5351_dec8/node2/data/system/local/system-local-jc-1-Data.db
compacted
{noformat}

Interestingly it looks like it almost catches this:
{noformat}
INFO  16:17:45 Skipping anticompaction for 1, required sstable was compacted and is no longer
available.
{noformat}

... but then it proceeds to repair/anticompact and hit the error anyway.

> Avoid repairing already-repaired data by default
> ------------------------------------------------
>
>                 Key: CASSANDRA-5351
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5351
>             Project: Cassandra
>          Issue Type: Task
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Lyuben Todorov
>              Labels: repair
>             Fix For: 2.1
>
>         Attachments: 5351_node1.log, 5351_node2.log, 5351_node3.log, 5351_nodetool.log
>
>
> Repair has always built its merkle tree from all the data in a columnfamily, which is
guaranteed to work but is inefficient.
> We can improve this by remembering which sstables have already been successfully repaired,
and only repairing sstables new since the last repair.  (This automatically makes CASSANDRA-3362
much less of a problem too.)
> The tricky part is, compaction will (if not taught otherwise) mix repaired data together
with non-repaired.  So we should segregate unrepaired sstables from the repaired ones.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message