sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "viru reddy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3158) Columns added to Mysql after initial sqoop import, export back to table with same schema fails
Date Fri, 14 Apr 2017 22:57:41 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969653#comment-15969653

viru reddy commented on SQOOP-3158:

I found a work around for this scenario.

After doing Incremental import as files, if we want to export the data back to MySQL then
do the following.

1) create a table test in hive using columns id , name and salary. Then load the data into
the table from the directory where we have both the initial and incremental data files.

Now when we query the table test

You will have 

1, raj,null
3, jill,2000

2) Then after this create a table mysql_export as select * from test
then you can export the data by providing mysql_export table location as target directory.

This worked for me, but It would be better if we have don't have to create two tables for
this kind of a scenarion

> Columns added to Mysql after initial sqoop import, export back to table with same schema
> -----------------------------------------------------------------------------------------------
>                 Key: SQOOP-3158
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3158
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: viru reddy
>            Assignee: Eric Lin
>              Labels: newbie
> I have table in MySQL with 2 columns until yesterday. The columns are id and name.
> 1,Raj
> 2,Jack
> I have imported this data into HDFS yesterday itself as a file. Today we added a new
column to the table in MySQL called salary. The table looks like below.
> 1,Raj
> 2,Jack
> 3,Jill,2000
> 4,Nick,3000
> Now I have done Incremental import on this table as a file.
> Part-m-00000 file contains
> 1,Raj
> 2,Jack
> Part-m-00001 file contains
> 3,Jill,2000
> 4,Nick,3000
> Now I created a new table in MySQL with same schema as Original MySQL table with columns
id name and salary.
> When I do sqoop export only last 2 rows are getting inserted to the new table in MySQL
 and the sqoop export fails
> How can I reflect all the rows to be inserted to the table.

This message was sent by Atlassian JIRA

View raw message