drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [drill] paul-rogers commented on a change in pull request #2026: DRILL-7330: Implement metadata usage for all format plugins
Date Sat, 14 Mar 2020 19:37:15 GMT
paul-rogers commented on a change in pull request #2026: DRILL-7330: Implement metadata usage
for all format plugins
URL: https://github.com/apache/drill/pull/2026#discussion_r392608447
 
 

 ##########
 File path: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/file/FileMetadataColumnDefn.java
 ##########
 @@ -53,7 +53,7 @@ public String toString() {
   public MajorType dataType() {
     return MajorType.newBuilder()
         .setMinorType(MinorType.VARCHAR)
-        .setMode(DataMode.REQUIRED)
+        .setMode(defn.isOptional() ? DataMode.OPTIONAL : DataMode.REQUIRED)
 
 Review comment:
   Does the `isOptional()` apply per-file or per-query? That is, if you've added a new implicit
column `foo`, will `foo` be set on some files but not others? Or, is it either available if
using metadata, but not available without metadata?
   
   If the value is per-file optional, then using an `OPTIONAL` mode makes sense. Though, it
is hard to see how we have file attributes available for some files but not others. If per-query
(query planned with metadata or without), then the column could be handled using the "missing
columns" mechanism: if the implicit value is not available, just treat it as missing and let
the "null column manager" fill it with nulls.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message