sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1037) Cannot import mysql table with into Hadoop
Date Tue, 21 May 2013 08:43:15 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662814#comment-13662814
] 

Jarek Jarcec Cecho commented on SQOOP-1037:
-------------------------------------------

As far as I know when all values in the split by columns are NULL, only one mapper will be
created. So I would assume that the import time should be comparable to the case when you're
importing with single mapper. However in Sqoop the bottleneck of the transmission is usually
on the database side rather than on the Hadoop side, so I guess that your db was under more
load when doing the first import?
                
> Cannot import mysql table with into Hadoop
> ------------------------------------------
>
>                 Key: SQOOP-1037
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1037
>             Project: Sqoop
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.4.3
>         Environment: bl1253{wenzhiw}/home/wenzhiw>cat /etc/redhat-release
> Scientific Linux SL release 5.3 (Boron)
> bl1253{wenzhiw}/home/wenzhiw>uname -a
> Linux bl1253.cn.alcatel-lucent.com 2.6.18-128.7.1.el5PAE #1 SMP Mon Aug 24 08:41:30 EDT
2009 i686 athlon i386 GNU/Linux
>            Reporter: wenzhi wang
>            Priority: Critical
>         Attachments: sqoop_import_error.log
>
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> Hi,
> I run Sqoop 1.4.3 on Apache Hadoop 1.0.4 and want to import table from mysql, but met
below errors. I also searched jira site for this issue, but didn't find any useful info, please
help to check. thanks!
> {code}
> bl1253{wenzhiw}/local/wenzhiw/sqoop2/sqoop-1.4.3.bin__hadoop-0.23/bin>sqoop import
--table IMSI --connect jdbc:mysql://135.252.158.250/hlr --username root -P
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> Enter password:
> 13/05/08 10:20:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> 13/05/08 10:20:53 INFO tool.CodeGenTool: Beginning code generation
> 13/05/08 10:20:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `IMSI`
AS t LIMIT 1
> 13/05/08 10:20:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `IMSI`
AS t LIMIT 1
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
> ...
> {code}
> Table definition:
> {code}
> mysql> desc IMSI;
> +-------------------------------+-------------+------+-----+---------+-------+
> | Field                         | Type        | Null | Key | Default | Extra |
> +-------------------------------+-------------+------+-----+---------+-------+
> | mSubIdentificationNumberId    | binary(6)   | NO   | PRI | NULL    |       |
> | mobileNetworkCodeId           | binary(3)   | NO   | PRI | NULL    |       |
> | mobileCountryCodeId           | binary(3)   | NO   | PRI | NULL    |       |
> | mainMSISDN                    | binary(12)  | YES  |     | NULL    |       |
> | alertMSISDN                   | binary(12)  | YES  |     | NULL    |       |
> | associatedMSISDN              | binary(88)  | YES  |     | NULL    |       |
> | gprsNotReachableReason        | tinyint(4)  | YES  |     | NULL    |       |
> | sgsnLastGlobalUpdateTime      | int(11)     | YES  |     | NULL    |       |
> | vlrIsdn                       | binary(10)  | YES  |     | NULL    |       |
> | mscIsdn                       | binary(10)  | YES  |     | NULL    |       |
> | supOfLocRegCount              | tinyint(4)  | YES  |     | NULL    |       |
> | locRegSubsPeriod              | smallint(6) | YES  |     | NULL    |       |
> | inducedCbocBsg                | binary(7)   | YES  |     | NULL    |       |
> | inducedCboiBsg                | binary(7)   | YES  |     | NULL    |       |
> | inducedCboehBsg               | binary(7)   | YES  |     | NULL    |       |
> | vlrLastGlobalUpdateTime       | int(11)     | YES  |     | NULL    |       |
> | currentMapScreeningProfile    | int(11)     | YES  |     | NULL    |       |
> | smscIsdn                      | binary(70)  | YES  |     | NULL    |       |
> | gsmNotReachableReason         | tinyint(4)  | YES  |     | NULL    |       |
> | timeStampWaitingSms           | binary(28)  | YES  |     | NULL    |       |
> | checkSSindicator              | tinyint(4)  | YES  |     | NULL    |       |
> | supOfAutReqCount              | tinyint(4)  | YES  |     | NULL    |       |
> | autReqSubsPeriod              | smallint(6) | YES  |     | NULL    |       |
> | underTestImsi                 | bit(1)      | YES  |     | NULL    |       |
> | cardStatus                    | bit(1)      | YES  |     | NULL    |       |
> | 
> +-------------------------------+-------------+------+-----+---------+-------+
> {code}
> Thanks,
> Wenzhi

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message