cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5972) Reduce the amount of data to be transferred during repair
Date Tue, 03 Sep 2013 12:10:56 GMT


Sylvain Lebresne commented on CASSANDRA-5972:

Right. But that does assume a bunch of things that repair currently doesn't do like recomputing
hashes post streaming, exchanging merkle tree update and a lot more coordination (we pretty
much would need all nodes to have all merkle trees, while today repair uses a single node
to do the whole synchronization).

So please don't get me wrong: I'm in no way saying that repair cannot be made to transfer
less data (nor that your suggestion wouldn't allow that). I'm saying that the repair code
(the MerkleTree class in particular) are a lot uglier than then could be, making changes like
the one this ticket suggest (or CASSANDRA-3200 for that matter) harder to do that it sounds
on paper. And that to make your suggestion work in practice is a little more complicated that
what your initial description made it sound.

Now, I do abolutely believe that we should fix that excessive transfer at some point, but
I also think that once we invest the energy to refactor repair to fix that then CASSANDRA-3200
is imo a better solution since it minimizes the number of transfer done, while I don't think
the solution here does (at least not always).

> Reduce the amount of data to be transferred during repair
> ---------------------------------------------------------
>                 Key: CASSANDRA-5972
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jacek Lewandowski
>            Priority: Minor
> Currently, when a validator finds a token range different in n replicas, data streams
are initiated simultaneously between each possible pair of these n nodes, in both directions.
It yields n*(n-1) data stream in total. 
> It can be done in a sequence - Replica[1] -> R[2], R[2] -> R[3], ... , R[n-1] ->
R[n]. After this process, the data in R[n] are up to date. Then, we continue: R[n] -> R[1],
R[1] -> R[2], ... , R[n-2] -> R[n-1]. The active repair is done after 2*(n-1) data transfers
performed sequentially in 2*(n-1) steps.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message