sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chelikani narasimharao <narasimhar...@gmail.com>
Subject need help on merge tool in sqoop
Date Thu, 16 Feb 2012 14:54:15 GMT
Hi All,

    I am trying to merge the two datasets using the merge tool
functionality in sqoop.But i got struck up with the following error.

*java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
*

*---->below is my command executed to get the first data set specifying the
location to store the first dataset to dataset1 under narasimharao directory
*

sqoop import --connect jdbc:hpt4jdbc://
g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
com.hp.t4jdbc.HPT4Driver --username narasimharoa@hp.com -P --verbose
--split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and CNTCT_ID
in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
--target-dir /home/narasimharao/dataset1

successfully able to import the dataset1 and also the .class and .jar files
were written to the following location

*/tmp/sqoop-narasimharao/compile/08c0cc665bd3e89124987a7c297827a2*


*--->below is my command executed to get the second data set  after
updating the few records which exists in the dataset1 with the same
merge-key*


sqoop import --connect jdbc:hpt4jdbc://
g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
com.hp.t4jdbc.HPT4Driver --username narasimharoa.chelikani@hp.com -P
--verbose --split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and
CNTCT_ID in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
--target-dir /home/narasimharao/dataset2

successfully able to import the dataset1 and also the .class and .jar files
were written to the following location

*
/tmp/sqoop-narasimharao/compile/e26282f14a672e7722352fb424781e5a/CNTCT_D.jar
*


*--->Now the actually Sqoop merge command was given like below:*

*sqoop merge --new-data /home/narasimharao/dataset1 --onto
/home/narasimharao/dataset2 --target-dir /home/narasimharao/merged
--jar-file CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID*


Can you please help me which class name i need to mention over here under
the class-name attribute and what's wrong in this statement.

I tried the below options

1.copied the dataset1 corresponding class and jar files into my home
direcotry and tried like below...but getting the same error as below

sqoop merge --new-data /home/narasimharao/dataset2 --onto
/home/narasimharao/dataset1 --target-dir /home/narasimharao/merged
--jar-file ./CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID

Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.

12/02/16 13:56:08 INFO input.FileInputFormat: Total input paths to process
: 2
12/02/16 13:56:08 WARN snappy.LoadSnappy: Snappy native library is available
12/02/16 13:56:08 INFO util.NativeCodeLoader: Loaded the native-hadoop
library
12/02/16 13:56:08 INFO snappy.LoadSnappy: Snappy native library loaded
12/02/16 13:56:10 INFO mapred.JobClient: Running job: job_201202161202_0005
12/02/16 13:56:11 INFO mapred.JobClient:  map 0% reduce 0%
12/02/16 13:56:55 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000000_0, Status : FAILED
*java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
* at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
12/02/16 13:56:59 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000001_0, Status : FAILED
*java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
* at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
12/02/16 13:57:22 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
12/02/16 13:57:23 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000001_1, Status : FAILED
*java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
* at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
12/02/16 13:57:50 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000000_2, Status : FAILED
*java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
* at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
12/02/16 13:57:51 INFO mapred.JobClient: Task Id :
attempt_201202161202_0005_m_000001_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: CNTCT_D.class
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
 at
com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
 at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.ClassNotFoundException: CNTCT_D.class
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLo
attempt_201202161202_0005_m_000001_2: log4j:WARN No appenders could be
found for logger (org.apache.hadoop.mapred.Task).
attempt_201202161202_0005_m_000001_2: log4j:WARN Please initialize the
log4j system properly.
12/02/16 13:58:27 INFO mapred.JobClient: Job complete: job_201202161202_0005
12/02/16 13:58:28 INFO mapred.JobClient: Counters: 7
12/02/16 13:58:28 INFO mapred.JobClient:   Job Counters
12/02/16 13:58:28 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=249623
12/02/16 13:58:28 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
12/02/16 13:58:28 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
12/02/16 13:58:28 INFO mapred.JobClient:     Launched map tasks=8
12/02/16 13:58:28 INFO mapred.JobClient:     Data-local map tasks=8
12/02/16 13:58:28 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
12/02/16 13:58:28 INFO mapred.JobClient:     Failed map tasks=1
12/02/16 13:58:28 ERROR tool.MergeTool: MapReduce job failed!

2. given second import dataset2 corresponding classname..but the same error
it is also throwing

3.mentioned with full path for both .class and .jar file

sqoop merge --new-data /home/narasimharao/dataset2 --onto
/home/narasimharao/dataset1 --target-dir /home/narasimharao/merged
--jar-file
/tmp/sqoop-narasimharao/compile/08c0cc665bd3e89124987a7c297827a2/CNTCT_D.jar
--class-name
/tmp/sqoop-narasimharao/compile/08c0cc665bd3e89124987a7c297827a2/CNTCT_D.class
--merge-key CNTCT_ID
4.tried the above stmt3 with second dataset .class and .jar files
also..getting the same errors



Any Help Appreciated!!!!!!

Thanks,
Narasimharao

Mime
View raw message