cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "graham sanderson (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
Date Mon, 15 Sep 2014 23:35:34 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134695#comment-14134695
] 

graham sanderson edited comment on CASSANDRA-7546 at 9/15/14 11:35 PM:
-----------------------------------------------------------------------

# https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=log;h=f099e086f3f002789e24bd6c58e52b7553cd5381
is what was released according to the 2.1.0 tag in git vs what [~slebresne] said in the email
thread regarding no changes after c6a2c65a75adea9a62896269da98dd036c8e57f3 which was 2.1.0-tentative
# ok, I'll try offheap_objects instead (or as well)
# I'm still a bit confused about visit/revisit (which are in the 2.1.0 tagged release)...
I want to evenly spread the load across all my partitions (genernally using a new clustering
key every time, though I want to put a practical limit on the size of the partitions, so I
was hoping to let it wrap at 10M or so unique clustering key values)... so it ounds like i
want a visits=fixed(1) and revisits=not quite sure



was (Author: graham sanderson):
# https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=log;h=f099e086f3f002789e24bd6c58e52b7553cd5381
is what was released according to the 2.1.0 tag in git vs despite what [~slebresne] said in
the email thread regarding no changes after c6a2c65a75adea9a62896269da98dd036c8e57f3 which
was 2.1.0-tentative
# ok, I'll try offheap_objects instead (or as well)
# I'm still a bit confused about visit/revisit (which are in the 2.1.0 tagged release)...
I want to evenly spread the load across all my partitions (genernally using a new clustering
key every time, though I want to put a practical limit on the size of the partitions, so I
was hoping to let it wrap at 10M or so unique clustering key values)... so it ounds like i
want a visits=fixed(1) and revisits=not quite sure


> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7546
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: graham sanderson
>            Assignee: graham sanderson
>             Fix For: 2.1.1
>
>         Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 7546.20_4.txt, 7546.20_5.txt,
7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt,
hint_spikes.png, suggestion1.txt, suggestion1_21.txt, young_gen_gc.png
>
>
> In order to preserve atomicity, this code attempts to read, clone/update, then CAS the
state of the partition.
> Under heavy contention for updating a single partition this can cause some fairly staggering
memory growth (the more cores on your machine the worst it gets).
> Whilst many usage patterns don't do highly concurrent updates to the same partition,
hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory
allocation rates can be seen (especially when the updates being hinted are small updates to
different partitions which can happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation whilst not
slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message