drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [drill] vvysotskyi commented on a change in pull request #2184: DRILL-7874: Drill Fails to Read File Types on S3
Date Tue, 02 Mar 2021 20:35:19 GMT

vvysotskyi commented on a change in pull request #2184:
URL: https://github.com/apache/drill/pull/2184#discussion_r585890334



##########
File path: contrib/format-esri/src/main/java/org/apache/drill/exec/store/esri/ShpBatchReader.java
##########
@@ -125,7 +125,7 @@ public boolean next() {
 
   private void openFile(FileSchemaNegotiator negotiator) {
     try {
-      fileReaderShp = negotiator.fileSystem().open(split.getPath());
+      fileReaderShp = negotiator.fileSystem().openDecompressedInputStream(split.getPath());

Review comment:
       I have some doubts regarding the safety of the `openDecompressedInputStream()` method.
It contains the following line:
   ```
   byte[] bytes = IOUtils.toByteArray(compressedStream);
   ```
   that will read all the stream to the memory, but it may cause OOM.
   Why we can't use here another `openPossiblyCompressedStream()` method?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message