sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-482) import tool for mysql direct mode reports wrong record number
Date Fri, 04 May 2012 18:36:49 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268610#comment-13268610
] 

jiraposter@reviews.apache.org commented on SQOOP-482:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4988/#review7580
-----------------------------------------------------------

Ship it!


Nemon's fix seems to be right to me. The write() method should be called only once per record.
I also verified that his patch addresses the problem.

@Nemon
I am no commiter, so I cannot commit your patch. But you've got my vote! 

- Cheolsoo


On 2012-05-04 07:33:48, nemon lou wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4988/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-04 07:33:48)
bq.  
bq.  
bq.  Review request for Sqoop, Jarek Cecho and Cheolsoo Park.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  I simply change the way the MySQLDumpMapper writes contents.
bq.  May be there are other ways to fix it.Please let me know,thank you!
bq.  
bq.  
bq.  This addresses bug SQOOP-482.
bq.      https://issues.apache.org/jira/browse/SQOOP-482
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ./src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1332889 
bq.  
bq.  Diff: https://reviews.apache.org/r/4988/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  nemon
bq.  
bq.


                
> import tool for mysql direct mode reports wrong record number
> -------------------------------------------------------------
>
>                 Key: SQOOP-482
>                 URL: https://issues.apache.org/jira/browse/SQOOP-482
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.1-incubating
>         Environment: hadoop 1.0.0
>            Reporter: nemon lou
>            Priority: Minor
>             Fix For: 1.4.2-incubating
>
>         Attachments: directmodewrongnumber.patch
>
>
> When run the sqoop import (via mysqldump) task :
> sqoop import --connect jdbc:mysql://[ip]:3306/mysqlslap --table t1    --username root
-P  --hive-import --hive-overwrite   --hive-table tTable --verbose --direct -m 1
> The console shows this(part of the log):
>   12/05/02 16:21:16 INFO mapred.JobClient:     Map output records=6000594
>   12/05/02 16:21:16 INFO mapreduce.ImportJobBase: Retrieved 6000594 records.
> However,the real record number in mysql is 1000099.And the number of records in hive
is  1000099 ,too.
> So only the console log report the wrong number.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message