drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@apache.org>
Subject Re: Review Request: Adding JSONRecordReader
Date Tue, 11 Jun 2013 18:40:51 GMT
Hey Timothy,

Can you file a JIRA for this and attach a patch?  Or is there one already?

I'll get it committed forthwith.

J


On Tue, Jun 11, 2013 at 11:38 AM, Jacques Nadeau <jacques.drill@gmail.com>wrote:

>    This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11587/
>
> Ship it!
>
> Ship It!
>
>
> - Jacques
>
> On June 11th, 2013, 6:18 a.m., Timothy Chen wrote:
>   Review request for drill and Jacques Nadeau.
> By Timothy Chen.
>
> *Updated June 11, 2013, 6:18 a.m.*
> Description
>
> Added the JSONRecordReader based on the previous ScanJson work.
>  Does not support nested fields, maps or lists yet.
>  Currently it detects to move on to the next batch when any of the field batch cannot
hold another item for the current item being written. This also assumes the default batch
size can always hold at least one item from any field (which only is a problem for variable
length vectors).
>
>   Diffs
>
>    - sandbox/prototype/common/src/main/java/org/apache/drill/common/logical/data/Scan.java
>    (c8d396b)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/DeadBuf.java
>    (dafb68c)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/AbstractFixedValueVector.java
>    (b32f067)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/BaseValueVector.java
>    (b001add)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/Fixed4.java
>    (650029d)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableBit.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableFixed4.java
>    (cc18538)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableValueVector.java
>    (692ab87)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/NullableVarLen4.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/TypeHelper.java
>    (8e89c41)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen1.java
>    (d87029d)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen2.java
>    (ebd440a)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VarLen4.java
>    (b3cd712)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/record/vector/VariableVector.java
>    (4247f14)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/InboundRpcMessage.java
>    (be1ff6b)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/DiffSchema.java
>    (016e097)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/Field.java
>    (e19c099)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/IdGenerator.java
>    (728e8e1)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/ListSchema.java
>    (efdc8fd)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/NamedField.java
>    (aa0d6aa)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/OrderedField.java
>    (67fd2fa)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/SchemaRecorder.java
>    (54a4e0e)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/JacksonHelper.java
>    (0643710)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperator.java
>    (e450ee9)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/PhysicalOperatorIterator.java
>    (bf4053e)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/json/jackson/ScanJson.java
>    (a1c30e9)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/ProtobufSchemaTransformer.java
>    (a81a9d9)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/schema/transform/SchemaTransformer.java
>    (54d851f)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java
>    (d5aaab2)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/BatchExceededException.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/JSONRecordReader.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java
>    (67c84ed)
>    - sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/VectorHolder.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/JSONRecordReaderTest.java
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_1.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_2.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_3.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_4.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_5.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/java-exec/src/test/resources/scan_json_test_6.json
>    (PRE-CREATION)
>    - sandbox/prototype/exec/ref/src/test/resources/donuts.json (2d98b59)
>    - sandbox/prototype/exec/ref/src/test/resources/simple_plan.json
>    (0f38476)
>    - sandbox/prototype/pom.xml (25f156d)
>
> View Diff <https://reviews.apache.org/r/11587/diff/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message