sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szabolcs Vasas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3021) ClassWriter fails if a column name contains a backslash character
Date Thu, 06 Oct 2016 14:52:20 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Szabolcs Vasas updated SQOOP-3021:
----------------------------------
    Description: 
The following Sqoop command fails with a javac error:

{code}
sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select C1_INT,C4_VARCHAR20,
REGEXP_REPLACE(TRIM(C4_VARCHAR20),'\:','!') from T1_IMPORT WHERE \$CONDITIONS" --target-dir
regex_imp --delete-target-dir -m 1
{code}

The reason is that the REGEXP_REPLACE expression contains a backslash character which does
not get escaped in ClassWriter and an invalid string gets generated into the Java code.

SQOOP-2864 solved this problem for the double quote character we need to generalize that solution.

  was:
I've seen a user who created table with column names containing double quotes and while code
generation, we quite spectacularly failed:

{code}
16/03/02 12:14:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error: ')' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                            ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error: not a statement
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                  ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error: ';' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                    ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error: not a statement
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                           ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error: ';' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                                   ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error: ')' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                            ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error: not a statement
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                  ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error: ';' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                    ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error: not a statement
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                           ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error: ';' expected
    __sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
                                                   ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error: ')' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                  ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error: ';' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                        ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error: ';' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                                              ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:343: error: 'else' without
'if'
    else    if ("'single'".equals(__fieldName)) {
    ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error: ')' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                  ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error: ';' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                        ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error: ';' expected
    else    if (""tellmewhy"".equals(__fieldName)) {
                                              ^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:359: error: 'else' without
'if'
    else    if ("'single'".equals(__fieldName)) {
    ^
18 errors
16/03/02 12:14:14 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException:
Error returned by javac
  at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:217)
  at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:108)
  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
  at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
  at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
{code}

I've looked into it and the problem is that we've started preserving raw column names inside
the generated class, but we did not escape properly the case when the column name contain
double quotes.


> ClassWriter fails if a column name contains a backslash character
> -----------------------------------------------------------------
>
>                 Key: SQOOP-3021
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3021
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.6
>            Reporter: Szabolcs Vasas
>            Assignee: Szabolcs Vasas
>             Fix For: 1.4.7
>
>
> The following Sqoop command fails with a javac error:
> {code}
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select
C1_INT,C4_VARCHAR20, REGEXP_REPLACE(TRIM(C4_VARCHAR20),'\:','!') from T1_IMPORT WHERE \$CONDITIONS"
--target-dir regex_imp --delete-target-dir -m 1
> {code}
> The reason is that the REGEXP_REPLACE expression contains a backslash character which
does not get escaped in ClassWriter and an invalid string gets generated into the Java code.
> SQOOP-2864 solved this problem for the double quote character we need to generalize that
solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message