cassandra-pr mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iamaleksey <...@git.apache.org>
Subject [GitHub] cassandra pull request #239: [CASSANDRA-14556] Optimize Streaming
Date Tue, 24 Jul 2018 18:58:58 GMT
Github user iamaleksey commented on a diff in the pull request:

    https://github.com/apache/cassandra/pull/239#discussion_r204874237
  
    --- Diff: src/java/org/apache/cassandra/db/streaming/CassandraOutgoingFile.java ---
    @@ -114,13 +155,51 @@ public void write(StreamSession session, DataOutputStreamPlus out,
int version)
             CassandraStreamHeader.serializer.serialize(header, out, version);
             out.flush();
     
    -        CassandraStreamWriter writer = header.compressionInfo == null ?
    -                                       new CassandraStreamWriter(sstable, header.sections,
session) :
    -                                       new CompressedCassandraStreamWriter(sstable, header.sections,
    -                                                                           header.compressionInfo,
session);
    +        IStreamWriter writer;
    +        if (shouldStreamFullSSTable())
    +        {
    +            writer = new CassandraBlockStreamWriter(sstable, session, components);
    +        }
    +        else
    +        {
    +            writer = (header.compressionInfo == null) ?
    +                     new CassandraStreamWriter(sstable, header.sections, session) :
    +                     new CompressedCassandraStreamWriter(sstable, header.sections,
    +                                                         header.compressionInfo, session);
    +        }
             writer.write(out);
         }
     
    +    @VisibleForTesting
    +    public boolean shouldStreamFullSSTable()
    +    {
    +        return isFullSSTableTransfersEnabled && isFullyContained;
    +    }
    +
    +    @VisibleForTesting
    +    public boolean fullyContainedIn(List<Range<Token>> normalizedRanges,
SSTableReader sstable)
    +    {
    +        if (normalizedRanges == null)
    +            return false;
    +
    +        RangeOwnHelper rangeOwnHelper = new RangeOwnHelper(normalizedRanges);
    +        try (KeyIterator iter = new KeyIterator(sstable.descriptor, sstable.metadata()))
    +        {
    +            while (iter.hasNext())
    +            {
    +                DecoratedKey key = iter.next();
    +                try
    +                {
    +                    rangeOwnHelper.check(key);
    +                } catch(RuntimeException e)
    --- End diff --
    
    @dineshjoshi Duplicating a reply here: I think it's fine to not block this PR on the improvements,
but disable zero copy for STCS for now, and commit that way. Once we optimise the containment
check - ideally before 4.0 - enable for STCS as well.
    
    Encoding information about the effective contained ranges is one way. Alternative way
would be to invert the ranges we are checking for, and look for presence of keys in the gaps
- but relying on summary and index to efficiently search rather than linearly scan. But again
- let's temporarily disable zero copy for STCS for now, to not delay the ticket beyond necessary.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Mime
View raw message