sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 35859: SQOOP-1493: Add ability to import/export true decimal in Avro instead of serializing it to String
Date Thu, 25 Jun 2015 18:13:40 GMT


> On June 25, 2015, 5 a.m., Qian Xu wrote:
> > ivy/libraries.properties, line 21
> > <https://reviews.apache.org/r/35859/diff/1/?file=991750#file991750line21>
> >
> >     I'm not sure, if a snapshot artifact is acceptable for sqoop 1.x release.

So, we'll have to change it when Avro 1.8.0 is released. But this feature exists in Avro master
only currently. I think it's ok since Sqoop1 is far from a release and Avro is close to a
release.


> On June 25, 2015, 5 a.m., Qian Xu wrote:
> > src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java, line 59
> > <https://reviews.apache.org/r/35859/diff/1/?file=991754#file991754line59>
> >
> >     In the head avro version, `DEFAULT_DEFLATE_LEVEL` becomes -1. If you use a magic
number 1, will it impact the avro internal handling?

I believe the value has been moved/changed around quite a bit: https://github.com/apache/avro/blob/branch-1.5/lang/java/mapred/src/main/java/org/apache/avro/mapred/AvroOutputFormat.java#L56.
Keeping it at 1 makes much more sense given the signature of the deflate method: https://github.com/apache/avro/blob/branch-1.7/lang/java/avro/src/main/java/org/apache/avro/file/CodecFactory.java#L48.
it expects a value between 1 and 9.


- Abraham


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35859/#review89314
-----------------------------------------------------------


On June 25, 2015, 6:13 p.m., Abraham Elmahrek wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35859/
> -----------------------------------------------------------
> 
> (Updated June 25, 2015, 6:13 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-1493
>     https://issues.apache.org/jira/browse/SQOOP-1493
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> commit 8583077a33956cd2a3abc05f73172210c31994a9
> Author: Abraham Elmahrek <abe@apache.org>
> Date:   Fri Jun 19 05:45:25 2015 +0300
> 
>     SQOOP-1493: Add ability to import/export true decimal in Avro instead of serializing
it to String
> 
> :100644 100644 2e3d884... 628491c... M  ivy/libraries.properties
> :100644 100644 ee3cf62... 5970981... M  src/java/org/apache/sqoop/avro/AvroUtil.java
> :100644 100644 d9569c5... 5650079... M  src/java/org/apache/sqoop/manager/ConnManager.java
> :100644 100644 20f056a... 76c3458... M  src/java/org/apache/sqoop/mapreduce/AvroExportMapper.java
> :100644 100644 aed1e72... a4ac46e... M  src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java
> :100644 100644 ab263c1... b60ee42... M  src/java/org/apache/sqoop/mapreduce/GenericRecordExportMapper
> :100644 100644 2576673... 8490582... M  src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java
> :100644 100644 1c6f7f4... aff8d08... M  src/java/org/apache/sqoop/orm/ClassWriter.java
> :100644 100644 663828c... 42e36ab... M  src/test/com/cloudera/sqoop/TestAvroExport.java
> :100644 100644 260e80a... 2f680c8... M  src/test/com/cloudera/sqoop/TestAvroImport.java
> 
> 
> Diffs
> -----
> 
>   ivy/libraries.properties 2e3d884 
>   src/java/org/apache/sqoop/avro/AvroUtil.java ee3cf62 
>   src/java/org/apache/sqoop/manager/ConnManager.java d9569c5 
>   src/java/org/apache/sqoop/mapreduce/AvroExportMapper.java 20f056a 
>   src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java aed1e72 
>   src/java/org/apache/sqoop/mapreduce/GenericRecordExportMapper.java ab263c1 
>   src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java 2576673 
>   src/java/org/apache/sqoop/orm/ClassWriter.java 1c6f7f4 
>   src/test/com/cloudera/sqoop/TestAvroExport.java 663828c 
>   src/test/com/cloudera/sqoop/TestAvroImport.java 260e80a 
> 
> Diff: https://reviews.apache.org/r/35859/diff/
> 
> 
> Testing
> -------
> 
> ant test && manual
> 
> 
> Thanks,
> 
> Abraham Elmahrek
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message