sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-330) Free form query import with column transformation failed without obvious error message
Date Sun, 04 Sep 2011 18:05:09 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jarek Jarcec Cecho updated SQOOP-330:
-------------------------------------

    Status: Patch Available  (was: Open)

Problem is that both getColumnName and getColumnLabel methods from class ResultSetMetaData
are returning empty string and not null value.

I tried to patch class SqlManager, method getColumnNamesForRawQuery to detect empty string
as column name and create universal name in format _RESULT_{int}. Doing that ended on not
clear error message about not existing column _RESULT_{int}.

This patch is adding check for empty column name and will rise an exception if it will be
detected.

> Free form query import with column transformation failed without obvious error message
> --------------------------------------------------------------------------------------
>
>                 Key: SQOOP-330
>                 URL: https://issues.apache.org/jira/browse/SQOOP-330
>             Project: Sqoop
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.4.0
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Jarek Jarcec Cecho
>         Attachments: SQOOP-330.patch
>
>
> Calling upstream version with free form query that contains column with transformation
will fail on MS SQL server (I did not try other RDBMS):
> [root@hddev01 sqoop]# sqoop import --driver com.microsoft.sqlserver.jdbc.SQLServerDriver
--connect jdbc:sqlserver://host\;databaseName=db --username jarcec --password password --as-textfile
--num-mappers 10 --target-dir /user/root/table SELECT convert(varchar(50), md5, 2) FROM table
where $CONDITIONS' --split-by id
> 11/09/04 13:16:41 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 11/09/04 13:16:41 INFO manager.SqlManager: Using default fetchSize of 1000
> 11/09/04 13:16:41 INFO tool.CodeGenTool: Beginning code generation
> 11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50),
md5, 2) FROM table where (1 = 0) 
> 11/09/04 13:16:43 INFO manager.SqlManager: Executing SQL statement: SELECT convert(varchar(50),
md5, 2) FROM table where (1 = 0) 
> 11/09/04 13:16:43 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
> 11/09/04 13:16:43 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/hadoop-core.jar
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:37: <identifier>
expected
>   private String ;
>                 ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:41: <identifier>
expected
>   public void set_(String ) {
>                          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: <identifier>
expected
>     this. = ;
>          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:42: illegal
start of expression
>     this. = ;
>             ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:44: <identifier>
expected
>   public QueryResult with_(String ) {
>                                  ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: <identifier>
expected
>     this. = ;
>          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:45: illegal
start of expression
>     this. = ;
>             ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier>
expected
>     equal = equal && (this. == null ? that. == null : this..equals(that.));
>                            ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier>
expected
>     equal = equal && (this. == null ? that. == null : this..equals(that.));
>                                            ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: malformed
floating point literal
>     equal = equal && (this. == null ? that. == null : this..equals(that.));
>                                                           ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: <identifier>
expected
>     equal = equal && (this. == null ? that. == null : this..equals(that.));
>                                                                         ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:57: ';' expected
>     equal = equal && (this. == null ? that. == null : this..equals(that.));
>                                                                          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:62: <identifier>
expected
>     this. = JdbcWritableBridge.readString(1, __dbResults);
>          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:72: illegal
start of expression
>     JdbcWritableBridge.writeString(, 1 + __off, 12, __dbStmt);
>                                    ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:77: <identifier>
expected
>         this. = null;
>              ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:79: <identifier>
expected
>     this. = Text.readString(__dataIn);
>          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:83: <identifier>
expected
>     if (null == this.) { 
>                      ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:87: illegal
start of expression
>     Text.writeString(__dataOut, );
>                                 ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal
start of expression
>     __sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
>                                                 ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:103: illegal
start of expression
>     __sb.append(FieldFormatter.escapeAndEnclose(==null?"null":, delimiters));
>                                                               ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:163: <identifier>
expected
>     if (__cur_str.equals("null")) { this. = null; } else {
>                                          ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:164: <identifier>
expected
>       this. = __cur_str;
>            ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:176: <identifier>
expected
>     __sqoop$field_map.put("", this.);
>                                    ^
> /tmp/sqoop-root/compile/462603a17b4956b3bd9151dab43f4227/QueryResult.java:182: <identifier>
expected
>       this. = (String) __fieldVal;
>            ^
> 24 errors
> 11/09/04 13:16:43 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: Error returned by javac
> 	at com.cloudera.sqoop.orm.CompilationManager.compile(CompilationManager.java:205)
> 	at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
> 	at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:367)
> 	at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:453)
> 	at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
> 	at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message