cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Bridges (JIRA)" <>
Subject [jira] Updated: (CASSANDRA-1187) make the number of compaction threads configurable
Date Sun, 01 Aug 2010 02:32:16 GMT


Sean Bridges updated CASSANDRA-1187:

    Attachment: CASSANDRA-1187-2.patch

Is this what you were thinking of?  

The patch adds a new ConcurrentCompactedRow which can read columns from multiple SSTables
in parallell.  I'm not sure how much parallelism this patch gives.  For the case where two
SSTables have no rows in common, there is no benefit.

Trying to read from multiple rows in parallell seems like it would get messy.

> make the number of compaction threads configurable
> --------------------------------------------------
>                 Key: CASSANDRA-1187
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Sean Bridges
>         Attachments: CASSANDRA-1187-2.patch, CASSANDRA-1187.patch
> On our test machines, compaction is the limiting factor when we are writing to Cassandra.
 It's easy to write to Cassandra faster than the single compaction thread can keep up, leading
to a large number of sstables.
> In one extreme example, we inserted a TB of data into a single cassandra node overnight,
and ended up with 100,000 sstables, which took another two days to finish compacting.
> If the number of compaction threads was configurable, we could tune cassandra to support
a higher write workload.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message