cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Terje Marthinussen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2558) Add "concurrent_compactions" configuration
Date Wed, 04 May 2011 09:52:03 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028692#comment-13028692
] 

Terje Marthinussen commented on CASSANDRA-2558:
-----------------------------------------------

You may be right. The tmp file could be memtable under flush. I will watch that a bit more.

A few additional concerns to the entire concurrent compaction (although they probably belong
in a different ticket):
If I read the code right, I think each thread looks for empty disk space by itself without
global coordination?

If so, a more nasty side effect of concurrent compaction is that the disk space check fails.

However, the threads should probably also continuously monitor available disk space and start
shutting themselves down in a controlled and coordinated fashion so all the compaction threads
don't write the disk full, but rather let some of them finish....



> Add "concurrent_compactions" configuration
> ------------------------------------------
>
>                 Key: CASSANDRA-2558
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2558
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.8 beta 1
>            Reporter: Sylvain Lebresne
>            Assignee: Sylvain Lebresne
>            Priority: Trivial
>             Fix For: 0.8.0 beta 2
>
>         Attachments: 0001-Make-compaction-thread-number-configurable-v2.patch, 0001-Make-compaction-thread-number-configurable.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> We should expose a way to configure the max number of thread to use when multi_threaded
compaction is turned on. So far, it uses nb_of_processors thread, which if you have many cores
may be unreasonably high (as far as random IO is concerned and thus independently of compaction
throttling)... at least unless you have SSD.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message