cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6916) Preemptive opening of compaction result
Date Tue, 22 Apr 2014 09:29:21 GMT


Benedict commented on CASSANDRA-6916:

[~enigmacurry]: 6916v3 should crash if you set the preheat_kernel_page_cache property to true
(just tested this locally). Setting the populate_io_cache_on_flush property of the CF would
probably work but simply have no effect. Are you sure you were running the correct branch?

bq. Furthermore, I hadn't realized when testing CASSANDRA-6746 that we could actually fare
well with the existing options like this

The problem is that we consider the default to be better at preventing dramatic page cache
churn during compaction, which this should continue to deliver but without the downsides.

The errors look plausible - but could we confirm we're running the correct (v3) branch given
it didn't crash with the preheat setting in the yaml?

> Preemptive opening of compaction result
> ---------------------------------------
>                 Key: CASSANDRA-6916
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>            Priority: Minor
>              Labels: performance
>             Fix For: 2.1
>         Attachments: 6916-stock2_1.mixed.cache_tweaks.tar.gz, 6916-stock2_1.mixed.logs.tar.gz,
6916v3-preempive-open-compact.logs.gz, 6916v3-preempive-open-compact.mixed.2.logs.tar.gz,
> Related to CASSANDRA-6812, but a little simpler: when compacting, we mess quite badly
with the page cache. One thing we can do to mitigate this problem is to use the sstable we're
writing before we've finished writing it, and to drop the regions from the old sstables from
the page cache as soon as the new sstables have them (even if they're only written to the
page cache). This should minimise any page cache churn, as the old sstables must be larger
than the new sstable, and since both will be in memory, dropping the old sstables is at least
as good as dropping the new.
> The approach is quite straight-forward. Every X MB written:
> # grab flushed length of index file;
> # grab second to last index summary record, after excluding those that point to positions
after the flushed length;
> # open index file, and check that our last record doesn't occur outside of the flushed
length of the data file (pretty unlikely)
> # Open the sstable with the calculated upper bound
> Some complications:
> # must keep running copy of compression metadata for reopening with
> # we need to be able to replace an sstable with itself but a different lower bound
> # we need to drop the old page cache only when readers have finished

This message was sent by Atlassian JIRA

View raw message