cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hobbs (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6109) Consider coldness in STCS compaction
Date Fri, 25 Oct 2013 15:45:32 GMT


Tyler Hobbs commented on CASSANDRA-6109:

bq. When they make up more than X% do we stop discriminating or merge them only with other
cold sstables?

I was thinking we would stop discriminating.  The logic would basically be this:

total_reads = sum(sstable.reads_per_sec for sstable in sstables)
total_cold_reads = 0
cold_sstables = set()
for sstable in sorted(sstables, key=lambda sstable: sstable.reads_per_key_per_sec):
    if (sstable.reads_per_sec + total_cold_reads) / total_reads < configurable_threshold:
        total_cold_reads += sstable.reads_per_sec

getBuckets(sstable for sstable in sstables if sstable not in cold_sstables)

> Consider coldness in STCS compaction
> ------------------------------------
>                 Key: CASSANDRA-6109
>                 URL:
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Tyler Hobbs
>             Fix For: 2.0.2
>         Attachments: 6109-v1.patch, 6109-v2.patch
> I see two options:
> # Don't compact cold sstables at all
> # Compact cold sstables only if there is nothing more important to compact
> The latter is better if you have cold data that may become hot again...  but it's confusing
if you have a workload such that you can't keep up with *all* compaction, but you can keep
up with hot sstable.  (Compaction backlog stat becomes useless since we fall increasingly

This message was sent by Atlassian JIRA

View raw message