sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-542) Error in importing data in Avro format
Date Tue, 04 Sep 2012 23:24:08 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448186#comment-13448186
] 

Cheolsoo Park commented on SQOOP-542:
-------------------------------------

Hi Giovanni,

This is because the avro jar file (avro-1.5.3.jar) is not present in classpath.

Given that you're building Sqoop from the source, I assume that you're running "ant clean
*jar* -Dhadoopversion=20" from the top directory where you build sqoop. If so, you need to
put dependency jars in the lib directory by yourself, or you will get ClassNotFound error.

Instead, you could run "ant clean *package* -Dhadoopversion=20". This command will create
the following directory for you:
{code}
<top directory>/build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20
{code}

And you can find dependency jars are downloaded for you:
{code}
cheolsoo@localhost:~/workspace/sqoop1/build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20 $ls -1 lib
ant-contrib-1.0b3.jar
ant-eclipse-1.0-jvm1.2.jar
avro-1.5.3.jar
avro-ipc-1.5.3.jar
avro-mapred-1.5.3.jar
commons-io-1.4.jar
hsqldb-1.8.0.10.jar
jackson-core-asl-1.7.3.jar
jackson-mapper-asl-1.7.3.jar
jopt-simple-3.2.jar
mysql-connector-java-5.1.18-bin.jar
paranamer-2.3.jar
snappy-java-1.0.3.2.jar
{code}

Using build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20 as the sqoop home directory, you will no
longer have this problem.

Lastly, if you don't want to deal with dependency jars at all, you can download the binaries
from:
http://www.gtlib.gatech.edu/pub/apache/sqoop/1.4.2/sqoop-1.4.2.bin__hadoop-0.20.tar.gz

When you untar the tarball, you will see the dependency jars are present in the lib directory
for you:

{code}
cheolsoo@localhost:~/workspace/sqoop-1.4.2.bin__hadoop-0.20 $ls -1 lib
ant-contrib-1.0b3.jar
ant-eclipse-1.0-jvm1.2.jar
avro-1.5.3.jar
avro-ipc-1.5.3.jar
avro-mapred-1.5.3.jar
commons-io-1.4.jar
hsqldb-1.8.0.10.jar
jackson-core-asl-1.7.3.jar
jackson-mapper-asl-1.7.3.jar
jopt-simple-3.2.jar
paranamer-2.3.jar
snappy-java-1.0.3.2.jar
{code}

                
> Error in importing data in Avro format
> --------------------------------------
>
>                 Key: SQOOP-542
>                 URL: https://issues.apache.org/jira/browse/SQOOP-542
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.1-incubating, 1.4.2
>         Environment: Mac Os X 10.7
> Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
> Hadoop: hadoop-0.20.2-cdh3u4
>            Reporter: Giovanni Ruggiero
>
> When performing an import as avro datafile, Sqoop fails with the following exception:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:247)
> 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
> 	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
> 	at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
> 	at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security
> attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable
to load realm info from SCDynamicStore
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message