drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau" <jacques.dr...@gmail.com>
Subject Re: Review Request: Adding JSONRecordReader
Date Tue, 11 Jun 2013 18:38:44 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11587/#review21723
-----------------------------------------------------------

Ship it!


Ship It!

- Jacques Nadeau


On June 11, 2013, 6:18 a.m., Timothy Chen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11587/
> -----------------------------------------------------------
> 
> (Updated June 11, 2013, 6:18 a.m.)
> 
> 
> Review request for drill and Jacques Nadeau.
> 
> 
> Description
> -------
> 
> Added the JSONRecordReader based on the previous ScanJson work.
>  Does not support nested fields, maps or lists yet.
>  Currently it detects to move on to the next batch when any of the field batch cannot
hold another item for the current item being written. This also assumes the default batch
size can always hold at least one item from any field (which only is a problem for variable
length vectors).
> 
> 
> Diffs
> -----
> 
>   sandbox/prototype/common/src/main/java/org/apache/drill/common/logical/data/Scan.java
c8d396b 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/DeadBuf.java
dafb68c 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/AbstractFixedValueVector.java
b32f067 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/BaseValueVector.java
b001add 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/Fixed4.java
650029d 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableBit.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableFixed4.java
cc18538 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableValueVector.java
692ab87 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableVarLen4.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/TypeHelper.java
8e89c41 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen1.java
d87029d 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen2.java
ebd440a 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen4.java
b3cd712 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VariableVector.java
4247f14 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/InboundRpcMessage.java
be1ff6b 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/DiffSchema.java
016e097 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/Field.java
e19c099 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/IdGenerator.java
728e8e1 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/ListSchema.java
efdc8fd 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/NamedField.java
aa0d6aa 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/OrderedField.java
67fd2fa 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/SchemaRecorder.java
54a4e0e 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/JacksonHelper.java
0643710 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperator.java
e450ee9 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperatorIterator.java
bf4053e 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/ScanJson.java
a1c30e9 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/ProtobufSchemaTransformer.java
a81a9d9 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/SchemaTransformer.java
54d851f 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java
d5aaab2 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/BatchExceededException.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/JSONRecordReader.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java
67c84ed 
>   sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/VectorHolder.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/JSONRecordReaderTest.java
PRE-CREATION 
>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_1.json PRE-CREATION

>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_2.json PRE-CREATION

>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_3.json PRE-CREATION

>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_4.json PRE-CREATION

>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_5.json PRE-CREATION

>   sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_6.json PRE-CREATION

>   sandbox/prototype/exec/ref/src/test/resources/donuts.json 2d98b59 
>   sandbox/prototype/exec/ref/src/test/resources/simple_plan.json 0f38476 
>   sandbox/prototype/pom.xml 25f156d 
> 
> Diff: https://reviews.apache.org/r/11587/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message