sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-352) Export of avro data imported from database table (using sqoop import) fails on year
Date Fri, 02 Mar 2012 22:50:54 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221329#comment-13221329
] 

jiraposter@reviews.apache.org commented on SQOOP-352:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3845/
-----------------------------------------------------------

(Updated 2012-03-02 22:48:24.704210)


Review request for Sqoop, Arvind Prabhakar, Tom White, and Jarek Cecho.


Changes
-------

Thanks for your comments, Arvind!

New patch is attached.


Summary
-------

MySQL YEAR is mapped to java.sql.Date when importing to HDFS. On the way back an exception
is thrown because the date and month fields are truncated (not used) - this is the MySQL behaviour.


This addresses bug SQOOP-352.
    https://issues.apache.org/jira/browse/SQOOP-352


Diffs (updated)
-----

  ./src/java/org/apache/sqoop/hive/TableDefWriter.java 1296490 
  ./src/java/org/apache/sqoop/manager/ConnManager.java 1296490 
  ./src/java/org/apache/sqoop/manager/MySQLManager.java 1296490 
  ./src/java/org/apache/sqoop/manager/OracleManager.java 1296490 
  ./src/java/org/apache/sqoop/manager/SqlManager.java 1296490 
  ./src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 1296490 
  ./src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java 1296490 
  ./src/java/org/apache/sqoop/orm/ClassWriter.java 1296490 
  ./src/test/com/cloudera/sqoop/manager/MySQLCompatTest.java 1296490 

Diff: https://reviews.apache.org/r/3845/diff


Testing
-------


Thanks,

Bilung


                
> Export of avro data imported from database table (using sqoop import) fails on year
> -----------------------------------------------------------------------------------
>
>                 Key: SQOOP-352
>                 URL: https://issues.apache.org/jira/browse/SQOOP-352
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>            Reporter: Tom White
>            Assignee: Bilung Lee
>
> Report from Wing Yew Poon:
> I use a MySQL database for testing sqoop. I created a table with a representative collection
of data types for the columns, excluding blob and text (and their different-sized variants).
> I run sqoop import --as-avrodatafile to import the table into hdfs.
> I create a second table with the same schema, and run sqoop export to export the avro
data to this second table. The export fails with
> {noformat}
> java.io.IOException: java.sql.SQLException: Data truncated for column 'yyyy' at row 1
> 	at com.cloudera.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:180)
> 	at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:540)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:649)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: java.sql.SQLException: Data truncated for column 'yyyy' at row 1
> 	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
> 	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
> 	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558)
> 	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490)
> 	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)
> 	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2109)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2648)
> 	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2077)
> 	at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1356)
> 	at com.cloudera.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:232)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message