sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandish Kumar HN <sanysand...@gmail.com>
Subject Re: Review Request 61522: SQOOP-2907 : Export parquet files to RDBMS: don't require .metadata for parquet files
Date Wed, 04 Oct 2017 07:22:32 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61522/
-----------------------------------------------------------

(Updated Oct. 4, 2017, 7:22 a.m.)


Review request for Sqoop and Anna Szonyi.


Changes
-------

fiexed changes


Bugs: SQOOP-2907
    https://issues.apache.org/jira/browse/SQOOP-2907


Repository: sqoop-trunk


Description
-------

Kite currently requires .metadata.
Parquet files have their own metadata stored along data files.
It would be great for Export operation on parquet files to RDBMS not to require .metadata.
We have most of the files created by Spark and Hive, and they don't create .metadata, it only
Kite that does.
It makes sqoop export of parquet files usability very limited.


Diffs (updated)
-----

  src/java/org/apache/sqoop/avro/AvroUtil.java ee29f140 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 6f9afaf9 
  src/test/com/cloudera/sqoop/TestParquetExport.java 680fd73b 


Diff: https://reviews.apache.org/r/61522/diff/2/

Changes: https://reviews.apache.org/r/61522/diff/1-2/


Testing
-------

testSupportedParquetTypesForWithoutParquetMeta - done
testNullableFieldWithoutParquetMeta - done
testParquetRecordsNotSupportedWithoutParquetMeta -done
testMissingDatabaseFieldsWithoutParquetMeta - done
testMissingParquetFieldsWithoutParquetMeta - done


Thanks,

Sandish Kumar HN


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message