sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joey Echeverria" <j...@cloudera.com>
Subject Re: Review Request: SQOOP-319 The --hive-drop-import-delims option should accept a replacement string
Date Tue, 23 Aug 2011 23:01:05 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-08-23 23:01:05.651698)

Review request for Sqoop.


I added a hiveStringReplaceDelims() method and implemented hiveStringDropDelims() by calling
that method. I added validation to throw an error if both --hive-drop-import-delims and --hive-delims-replacement
are used. I also fixed the checkstyle issues that you found.

I added a test case for the validation code and also did manual testing of the feature.


I added a new option, --hive-delims-replacement, which lets you pass in a replacement string.
I did it with a new option to remain backwards compatible with the existing interface.

This addresses bug SQOOP-319.

Diffs (updated)

  src/docs/user/hive-args.txt 7e6b7a0 
  src/docs/user/hive.txt 059d7cb 
  src/java/com/cloudera/sqoop/SqoopOptions.java d760d39 
  src/java/com/cloudera/sqoop/lib/FieldFormatter.java 41536e1 
  src/java/com/cloudera/sqoop/orm/ClassWriter.java dd3994e 
  src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1 
  src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd 
  src/test/com/cloudera/sqoop/hive/TestHiveImport.java 35de2fd 
  testdata/hive/scripts/fieldWithNewlineReplacementImport.q PRE-CREATION 

Diff: https://reviews.apache.org/r/1598/diff


I added a unit test for the new option. I also tested the feature by hand. It works, but I
found a bug when doing --direct (at least with MySQL). It doesn't end up calling the hiveStringDropDelims()
function. Some other kind of escaping is going on. I'll file that as a separate JIRA.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message