sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-359) Import fails with Unknown SQL datatype exception
Date Fri, 07 Oct 2011 21:02:32 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123196#comment-13123196
] 

jiraposter@reviews.apache.org commented on SQOOP-359:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2270/#review2442
-----------------------------------------------------------



/src/java/com/cloudera/sqoop/mapreduce/DataDrivenImportJob.java
<https://reviews.apache.org/r/2270/#comment5551>

    I could do that but then I will have to do a null check in DataDrivenDBInputFormat. Otherwise,
the following code will raise a NPE. Hence I kept it separate. If ou think that is better,
I can certainly change that. Please let me know.


- Arvind


On 2011-10-07 02:31:15, Arvind Prabhakar wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2270/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-10-07 02:31:15)
bq.  
bq.  
bq.  Review request for Sqoop and Bilung Lee.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Modified TestBoundaryQuery to introduce a test that reproduces this problem. Also added
a validation exception that is raised when the user uses a boundary query without specifying
a split-by column.
bq.  
bq.  
bq.  This addresses bug SQOOP-359.
bq.      https://issues.apache.org/jira/browse/SQOOP-359
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    /src/java/com/cloudera/sqoop/manager/SqlManager.java 1178856 
bq.    /src/java/com/cloudera/sqoop/mapreduce/DataDrivenImportJob.java 1178856 
bq.    /src/test/com/cloudera/sqoop/TestBoundaryQuery.java 1178856 
bq.  
bq.  Diff: https://reviews.apache.org/r/2270/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  All unit and thirdparty tests.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Arvind
bq.  
bq.


                
> Import fails with Unknown SQL datatype exception
> ------------------------------------------------
>
>                 Key: SQOOP-359
>                 URL: https://issues.apache.org/jira/browse/SQOOP-359
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/generic
>    Affects Versions: 1.3.0
>            Reporter: Arvind Prabhakar
>            Assignee: Arvind Prabhakar
>         Attachments: SQOOP-359-1.patch
>
>
> To reproduce this, run an import using a query with number of mappers set to 1 and no
boundary query specified. For example:
> {code}
> $ sqoop import --connect jdbc:mysql://localhost/testdb --username test --password ****
\
>     --query 'SELECT TDX.A, TDX.B FROM TDX WHERE $CONDITIONS' \
>     --target-dir /user/arvind/MYSQL/TDX1 -m 1
> {code}
> This import will fail as follows:
> {code}
> 11/10/06 15:37:59 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-arvind/compile/190f858175a9f99756e503727c931450/QueryResult.jar
> 11/10/06 15:37:59 INFO mapreduce.ImportJobBase: Beginning query import.
> 11/10/06 15:38:00 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(null),
MAX(null) FROM (SELECT TDX.A, TDX.B FROM TDX WHERE  (1 = 1) ) AS t1
> 11/10/06 15:38:00 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost/opt/site/cdh3u1/hadoop/data/tmp/mapred/staging/arvind/.staging/job_201110061528_0004
> 11/10/06 15:38:00 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: Unknown SQL data type: -3
> 	at com.cloudera.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:211)
> 	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944)
> 	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:961)
> 	at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
> 	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
> 	at org.apache.hadoop.mapreduce.Job.submit(Job.java:476)
> 	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)
> 	at com.cloudera.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:123)
> 	at com.cloudera.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:183)
> 	at com.cloudera.sqoop.manager.SqlManager.importQuery(SqlManager.java:450)
> 	at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:384)
> 	at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:455)
> 	at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
> 	at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
> {code}
> The problem seems to be the bounding value query that is using a {{null}} column name
for figuring out the datatype.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message