cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Schuller (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-2559) Distinguish long and short running compactions
Date Tue, 26 Apr 2011 17:03:03 GMT


Peter Schuller commented on CASSANDRA-2559:

(And that in turn is useful when you have smaller CF:s with higher churn, lower GC grace,
that you want to repair more often than potentially significantly lager CF:s with less churn.)

> Distinguish long and short running compactions
> ----------------------------------------------
>                 Key: CASSANDRA-2559
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Sylvain Lebresne
>            Priority: Minor
>              Labels: compaction
> Unless you have SSD, multi-threaded compaction is mainly here to avoid accumulating lots
of newly flushed sstables while a long lasting compaction is running. But too many concurrent
compactions are bad for random IO. CASSANDRA-2558 will allow to limit the number of such concurrent
compactions, but choosing the right number there is not easy. If you pick too low a number,
you risk accumulating "young" sstables if 2 or 3 fairly long compaction runs at the same time.
On the other side, compacting multiple "small" sstables is likely to be less efficient (on
a spinning disk) than compacting them serially.
> It seems to me we could have the best of both world by distinguishing long and short
compactions. We could have 2 pools of thread, one for long compaction (whatever the exact
definition is) and one for short ones. With this, even with one thread in each pool you would
avoid most of the 'new sstable accumulation' problem while making sure you never run too many
concurrent compactions (note that in theory we could stratify further than "short" and "long",
but I'm not sure the benefits would out-weigh the added complexity).

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message