trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zellerh <...@git.apache.org>
Subject [GitHub] trafodion pull request #1626: [TRAFODION-3126] Refactored HDFS client implem...
Date Mon, 02 Jul 2018 23:47:45 GMT
Github user zellerh commented on a diff in the pull request:

    https://github.com/apache/trafodion/pull/1626#discussion_r199650506
  
    --- Diff: core/sql/src/main/java/org/trafodion/sql/HDFSClient.java ---
    @@ -142,21 +154,32 @@
           HDFSRead() 
           {
           }
    - 
    +    
           public Object call() throws IOException 
           {
              int bytesRead;
              int totalBytesRead = 0;
              if (compressed_) {
                 bufArray_ = new byte[ioByteArraySizeInKB_ * 1024];
    -         } else 
    -         if (! buf_.hasArray()) {
    -            try {
    -              fsdis_.seek(pos_);
    -            } catch (EOFException e) {
    -              isEOF_ = 1;
    -              return new Integer(totalBytesRead);
    -            } 
    +         } 
    +         else  {
    +            // alluxio doesn't support direct ByteBuffer reads
    +            // Hence, create a non-direct ByteBuffer, read into
    +            // byteArray backing up this ByteBuffer and 
    +            // then copy the data read to direct ByteBuffer for the 
    +            // native layer to process the data
    +            if ((! alluxioNotInstalled_) && fs_ instanceof alluxio.hadoop.FileSystem)
{
    +               savedBuf_ = buf_;
    +               buf_ = ByteBuffer.allocate(savedBuf_.capacity());
    --- End diff --
    
    Do we do this allocation every time we call the call() method? If so, could we avoid that
and do this only once? The same applies to line 163 above.


---

Mime
View raw message