cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paulo Motta (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-10990) Support streaming of older version sstables in 3.0
Date Sat, 05 Mar 2016 02:51:40 GMT


Paulo Motta commented on CASSANDRA-10990:

Updated branch with reworked {{RewindableDataInputStreamPlus}}, now treating the spill file
as circular buffer with max size. Now required space for spill buffer file of legacy {{StreamDeserializer}}
is {{max(sstableSize, MAX_INT)}}, capping the max required space for streaming older version
sstables on ~2GB. I also updated documentation and tests. WDYT [~yukim]?

Besides that, I added a new [dtest|]
for testing repair of older version sstables on 3.0+, and it seems to work. Now we have the
following dtests:
* [bootstrap dtests|]
* [sstable loader dtests|]
* [repair dtests|]

For simplicity, I opted for writing the old streamed sstables in the new format without {{EncodingStats}}
(since there is no {{SerializationHeader}} available), what may make these sstables less optimized
in terms of storage space from what I understood. Do you think we should construct these stats
when receiving the sstable or not bother [~slebresne] ? Also, do you recall of any other edge
case we should watch here?


> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>                 Key: CASSANDRA-10990
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeremy Hanna
>            Assignee: Paulo Motta
> In 2.0 we introduced support for streaming older versioned sstables (CASSANDRA-5772).
 In 3.0, because of the rewrite of the storage layer, this became no longer supported.  So
currently, while 3.0 can read sstables in the 2.1/2.2 format, it cannot stream the older versioned
sstables.  We should do some work to make this still possible to be consistent with what CASSANDRA-5772

This message was sent by Atlassian JIRA

View raw message