sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rajesh.bawal@orkash.com" <rajesh.ba...@orkash.com>
Subject Re: Error importing data from mysql to cassandra using sqoop
Date Tue, 07 Aug 2012 06:49:15 GMT
Hi Jarcec,

I am using sqoop utility provided by Datastax in their Enterprise 
version: Sqoop 1.4.1-dse-SNAPSHOT

I am using the following command to import :

./dse sqoop import --connect jdbc:mysql://192.168.1.102/npa_nxx_demo 
--username root --password *** --table npa_nxx --cassandra-keyspace 
pdc_crawler --cassandra-column-family npa_nxx_cf --cassandra-row-key 
npa_nxx_key --cassandra-thrift-host 192.168.0.203,192.168.0.224 
--cassandra-create-schema --verbose

This is the verbose output :
./dse sqoop import --connect jdbc:mysql://192.168.0.102/npa_nxx_demo 
--username root --password orka1 --table npa_nxx --cassandra-keyspace 
pdc_crawler --cassandra-column-family npa_nxx_cf --cassandra-row-key 
npa_nxx_key --cassandra-thrift-host 192.168.0.201,192.168.0.202 
--cassandra-create-schema --verbose
12/08/07 12:12:26 WARN tool.BaseSqoopTool: Setting your password on the 
command-line is insecure. Consider using -P instead.
12/08/07 12:12:26 INFO manager.MySQLManager: Preparing to use a MySQL 
streaming resultset.
12/08/07 12:12:26 INFO tool.CodeGenTool: Beginning code generation
12/08/07 12:12:27 INFO manager.SqlManager: Executing SQL statement: 
SELECT t.* FROM `npa_nxx` AS t LIMIT 1
12/08/07 12:12:27 INFO orm.CompilationManager: HADOOP_HOME is 
/home/Downloads/dse-2.1/resources/hadoop/bin/..
Note: 
/tmp/sqoop-root/compile/474195de4276c8abd8d19c14d4daeda5/npa_nxx.java 
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/08/07 12:12:28 INFO orm.CompilationManager: Writing jar file: 
/tmp/sqoop-root/compile/474195de4276c8abd8d19c14d4daeda5/npa_nxx.jar
12/08/07 12:12:28 WARN manager.MySQLManager: It looks like you are 
importing from mysql.
12/08/07 12:12:28 WARN manager.MySQLManager: This transfer can be 
faster! Use the --direct
12/08/07 12:12:28 WARN manager.MySQLManager: option to exercise a 
MySQL-specific fast path.
12/08/07 12:12:28 INFO manager.MySQLManager: Setting zero DATETIME 
behavior to convertToNull (mysql)
12/08/07 12:12:28 INFO mapreduce.ImportJobBase: Beginning import of npa_nxx
12/08/07 12:12:31 ERROR sqoop.Sqoop: Got exception running Sqoop: 
org.apache.cassandra.db.marshal.MarshalException: 97 is not recognized 
as a valid type
org.apache.cassandra.db.marshal.MarshalException: 97 is not recognized 
as a valid type
     at 
com.datastax.bdp.util.CompositeUtil.deserialize(CompositeUtil.java:93)
     at 
com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:585)
     at 
com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:563)
     at 
com.datastax.bdp.hadoop.cfs.CassandraFileSystem.getFileStatus(CassandraFileSystem.java:520)
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:768)
     at 
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:103)
     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:396)
     at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
     at 
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
     at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
     at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
     at 
org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:119)
     at 
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:179)
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:423)
     at 
org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:97)
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:380)
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
     at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)

Thanks a lot.
Rajesh Kumar

On Friday 03 August 2012 10:39 PM, Jarek Jarcec Cecho wrote:
> Hi Rajesh,
> could you please specify Sqoop version you're using, entire command line you've used
(please feel free to substitute sensitive information with asterisks) and entire log that
sqoop generated with --verbose flag?
>
> Jarcec
>
> On Fri, Aug 03, 2012 at 10:07:50AM +0530, rajesh.bawal@orkash.com wrote:
>> Hi Everyone,
>> I am trying to import data into cassandra column family from mysql
>> and i am getting the following error.
>>
>> ERROR sqoop.Sqoop: Got exception running Sqoop:
>> org.apache.cassandra.db.marshal.MarshalException: 97 is not
>> recognized as a valid type
>> org.apache.cassandra.db.marshal.MarshalException: 97 is not
>> recognized as a valid type
>>      at com.datastax.bdp.util.CompositeUtil.deserialize(CompositeUtil.java:93)
>>      at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:585)
>>      at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:563)
>>      at com.datastax.bdp.hadoop.cfs.CassandraFileSystem.getFileStatus(CassandraFileSystem.java:520)
>>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:768)
>>      at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:103)
>>      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
>>      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>>      at java.security.AccessController.doPrivileged(Native Method)
>>      at javax.security.auth.Subject.doAs(Subject.java:396)
>>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>>      at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>>      at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>>      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
>>      at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:119)
>>      at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:179)
>>      at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:423)
>>      at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:97)
>>      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:380)
>>      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:453)
>>      at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>      at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>      at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
>>
>> Could anyone help me solve this issue.
>>
>> Thanks
>> Rajesh Kumar



Mime
View raw message