drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhoukang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-6482) Fast clean unused direct memory when decompress
Date Fri, 08 Jun 2018 11:30:00 GMT
zhoukang created DRILL-6482:
-------------------------------

             Summary: Fast clean unused direct memory when decompress
                 Key: DRILL-6482
                 URL: https://issues.apache.org/jira/browse/DRILL-6482
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: zhoukang


When use *NonBlockedDecompressorStream* which call:

*SnappyDecompressor.setInput*

{code}

public synchronized void setInput(byte[] buffer, int off, int len) {
 SnappyUtil

public synchronized void setInput(byte[] buffer, int off, int len) {
 SnappyUtil.validateBuffer(buffer, off, len);

 if (inputBuffer.capacity() - inputBuffer.position() < len) {
 ByteBuffer newBuffer = ByteBuffer.allocateDirect(inputBuffer.position() + len);
 inputBuffer.rewind();
 newBuffer.put(inputBuffer);
 inputBuffer = newBuffer; 
 } else {
 inputBuffer.limit(inputBuffer.position() + len);
 }
 inputBuffer.put(buffer, off, len);
}

.validateBuffer(buffer, off, len);

 if (inputBuffer.capacity() - inputBuffer.position() < len) {
 ByteBuffer newBuffer = ByteBuffer.allocateDirect(inputBuffer.position() + len);
 inputBuffer.rewind();
 newBuffer.put(inputBuffer);
 inputBuffer = newBuffer; 
 } else {
 inputBuffer.limit(inputBuffer.position() + len);
 }
 inputBuffer.put(buffer, off, len);
}

{code}

If we do not get any full gc for old gen.we may failed by off-heap memory leak



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message