sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park" <cheol...@cloudera.com>
Subject Re: Review Request: SQOOP-483 Allow target dir to be set to a different name than table name for hive import
Date Sun, 27 May 2012 00:19:46 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2012-05-27 00:19:46.441415)

Review request for Sqoop and Jarek Cecho.


Incorporate Jarcec's comments.


Currently, the table name (set by --table) has to be the same as the target dir (set by --target-dir)
for hive import. But it would be nice if the target dir could be set to a different name than
the table name.

The changes include:

1) Implement the following logic in TableDefWriter:
   - If target dir is specified, use it as the source.
   - Otherwise, use the name of input table as the source.

The original logic was less flexible: if input table is specified, use it as the source; otherwise,
use target dir as the source. However, this disallows target dir to be different from input
table name when importing the entire table. With the new logic, the user can freely specify
input table, target dir, and output table in any case.

2) Clean up output-path-related code in Hive import.

3) Add a unit test that verifies that TableDefWriter generates correct statements given input
table, output table, and target-dir are all different.

This addresses bug SQOOP-483.

Diffs (updated)

  /src/java/org/apache/sqoop/hive/HiveImport.java 1342838 
  /src/java/org/apache/sqoop/hive/TableDefWriter.java 1342838 
  /src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java 1342838 

Diff: https://reviews.apache.org/r/5248/diff


1) Ran ant test, ant test -Dthirdparty=true, and ant checkstyle.

2) Tested with Hive the following scenarios:
- sqoop hive-import ... --table FOO
- sqoop hive-import ... --table FOO --hive table FOO
- sqoop hive-import ... --table FOO --hive table BAR
- sqoop hive-import ... --table FOO --hive-table FOO --target-dir FOO
- sqoop hive-import ... --table FOO --hive-table FOO --target-dir BAR
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir FOO
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir BAR
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir BAZ



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message