cassandra-pr mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dineshjoshi <...@git.apache.org>
Subject [GitHub] cassandra pull request #239: Optimize Streaming
Date Tue, 10 Jul 2018 19:18:49 GMT
Github user dineshjoshi commented on a diff in the pull request:

    https://github.com/apache/cassandra/pull/239#discussion_r201463376
  
    --- Diff: src/java/org/apache/cassandra/net/async/ByteBufDataOutputStreamPlus.java ---
    @@ -135,6 +140,53 @@ public ChannelFuture writeToChannel(ByteBuffer buffer) throws IOException
             return channelFuture;
         }
     
    +    /**
    +     * Writes all data in file channel to stream BUFFER_SIZE at a time.
    +     * Closes file channel when done
    +     *
    +     * @param f
    +     * @return number of bytes transferred
    +     * @throws IOException
    +     */
    +    public long writeToChannel(FileChannel f, StreamRateLimiter limiter) throws IOException
    +    {
    +        final long length = f.size();
    +        long bytesTransferred = 0;
    +
    +        try
    +        {
    +            while (bytesTransferred < length)
    +            {
    +                int toRead = (int) Math.min(bufferSize, length - bytesTransferred);
    +                NonClosingDefaultFileRegion fileRegion = new NonClosingDefaultFileRegion(f,
bytesTransferred, toRead);
    +
    +                if (!Uninterruptibles.tryAcquireUninterruptibly(channelRateLimiter, toRead,
5, TimeUnit.MINUTES))
    +                    throw new IOException(String.format("outbound channel was not writable.
Failed to acquire sufficient permits %d", toRead));
    +
    +                limiter.acquire(toRead);
    +
    +                bytesTransferred += toRead;
    +                final boolean shouldClose = (bytesTransferred == length); // this is
the last buffer, can safely close channel
    +
    +                channel.writeAndFlush(fileRegion).addListener(future -> {
    --- End diff --
    
    Yes, AFAICT, netty guarantees that the whole region will be transferred or an error generated.
We don't need to check the number of bytes transferred as we have handed off the file region
to the channel.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Mime
View raw message