cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dominic Letz (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-8603) Cut tombstone memory footprint in half for cql deletes
Date Fri, 16 Jan 2015 01:08:35 GMT


Dominic Letz updated CASSANDRA-8603:
    Attachment: system.log

Sorry for the long delay here. I did testing before but your comment made me look deeper and
indeed something is wrong.

First off the attached patches do not catch the situation that they were intended to catch
- but they do catch something and that mislead me. So maybe there is a bug as you pointed

The test case I'm using is like this:

  WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
CREATE TABLE test.timeseries (name text, timestamp int, value text, PRIMARY KEY ((name), timestamp))
  WITH compaction = {'class': 'SizeTieredCompactionStrategy'} AND CLUSTERING ORDER BY (timestamp

INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 1, 'hello');
INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 2, 'world');
INSERT INTO test.timeseries (name, timestamp, value) VALUES('a', 5, 'hello world');
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 1;
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 2;
DELETE FROM test.timeseries WHERE name = 'a' AND timestamp = 5;

Plus adding this log line next to the initial patch content:
    public RangeTombstone(Composite start, Composite stop, DeletionTime delTime)
        super(start, stop.equals(start) ? start : stop, delTime);
        LoggerFactory.getLogger(RangeTombstone.class).error(String.format("Test: start.equals(stop)
%s (%s == %s) %s, ", start.equals(stop), new String(Hex.encodeHex(start.toByteBuffer().array())),
new String(Hex.encodeHex(stop.toByteBuffer().array())), start.toByteBuffer().compareTo(stop.toByteBuffer())));

Then I wiped the cassandra data directory and started from scratch, and once started ran the
said cql. The resulting system.log is attached

It seems that just during start-up there are a couple of hits into the new logic where the
tombstone .start and .end are actually exactly the same as shown by the debug output:
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,476 - Test: start.equals(stop)
true (000000 == 000000) 0, 
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,478 - Test: start.equals(stop)
false (0006746f6b656e73ff == 0006746f6b656e7301) -2, 
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,480 - Test: start.equals(stop)
true (000c626f6f74737472617070656400 == 000c626f6f74737472617070656400) 0, 
ERROR [CompactionExecutor:1] 2015-01-16 08:50:57,481 - Test: start.equals(stop)
true (000c636c75737465725f6e616d6500 == 000c636c75737465725f6e616d6500) 0, 

But then further down in the log when the actual test cql code is executed these messages
pop up:

ERROR [Thrift:1] 2015-01-16 08:51:14,089 - Test: start.equals(stop)
false (000400000001ff == 00040000000101) -2, 
ERROR [Thrift:1] 2015-01-16 08:51:14,092 - Test: start.equals(stop)
false (000400000002ff == 00040000000201) -2, 
ERROR [Thrift:1] 2015-01-16 08:51:14,094 - Test: start.equals(stop)
false (000400000005ff == 00040000000501) -2, 

Show that even though they keys match up until the provided sample value (1, 2, 5) there is
an additional byte that is creating a difference and thus the code is not catching. It seems
I wasn't deep enough into the code to actually make my proposed change successfully as I don't
know the meaning of that last byte - but the concept might still hold?

> Cut tombstone memory footprint in half for cql deletes
> ------------------------------------------------------
>                 Key: CASSANDRA-8603
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Dominic Letz
>              Labels: tombstone
>         Attachments: cassandra-2.0.11-8603.txt, cassandra-2.1-8603.txt, system.log
> As CQL does not yet support range deletes every delete from CQL results in a "Semi-RangeTombstone"
which actually has the same start and end values - but until today they are copies. Effectively
doubling the required heap memory to store the RangeTombstone.

This message was sent by Atlassian JIRA

View raw message