cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Boudreault (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
Date Thu, 20 Nov 2014 15:15:35 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219459#comment-14219459
] 

Alan Boudreault edited comment on CASSANDRA-7386 at 11/20/14 3:14 PM:
----------------------------------------------------------------------

[~snazy] In fact, my concern is not really the 2 full disks.... but more why can I fill all
my disks in 6 minutes without the patch and that with the patch, 7/9 of my disks are under
60% of usage after 15 minutes? I might be wrong since that stuff is new to me..... but is
there some *better* compaction/compression happening with your patch or was there something
wrong happening before? Thanks! 

Yes, will try with a *conservative* compaction throughput, like 16mb/s (default). 


was (Author: aboudreault):
[~snazy] In fact, my concern is not really the 2 full disks.... but more why can I fill all
my disks in 6 minutes without the patch and that with the patch, 7/9 of my disks are under
60% of usage after 15 minutes? I might be wrong since that stuff is new to me..... but is
there some *better* compaction/compression happening with your patch or was there something
wrong happening before? Thanks! 

Yes, will try with a *conservative* compaction throughput, like 20mb/s. 

> JBOD threshold to prevent unbalanced disk utilization
> -----------------------------------------------------
>
>                 Key: CASSANDRA-7386
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7386
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Chris Lohfink
>            Assignee: Robert Stupp
>            Priority: Minor
>             Fix For: 2.1.3
>
>         Attachments: 7386-2.0-v3.txt, 7386-2.0-v4.txt, 7386-2.0-v5.txt, 7386-2.1-v3.txt,
7386-2.1-v4.txt, 7386-2.1-v5.txt, 7386-v1.patch, 7386v2.diff, Mappe1.ods, mean-writevalue-7disks.png,
patch_2_1_branch_proto.diff, sstable-count-second-run.png, test1_no_patch.jpg, test1_with_patch.jpg,
test2_no_patch.jpg, test2_with_patch.jpg, test3_no_patch.jpg, test3_with_patch.jpg, test_regression_no_patch.jpg,
test_regression_with_patch.jpg
>
>
> Currently the pick the disks are picked first by number of current tasks, then by free
space.  This helps with performance but can lead to large differences in utilization in some
(unlikely but possible) scenarios.  Ive seen 55% to 10% and heard reports of 90% to 10% on
IRC.  With both LCS and STCS (although my suspicion is that STCS makes it worse since harder
to be balanced).
> I purpose the algorithm change a little to have some maximum range of utilization where
it will pick by free space over load (acknowledging it can be slower).  So if a disk A is
30% full and disk B is 5% full it will never pick A over B until it balances out.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message