sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinoth Chandar <vin...@uber.com>
Subject Re: Install a new connector jar into sqoop2
Date Wed, 12 Nov 2014 18:20:18 GMT
Thanks for the help!!..

>> If you cannot use **any** connector, change “common.loader” to load
correct jars will help.
Can you please elaborate what you mean by 'any' connector? As I mentioned,
I placed by connector jar in common.loader and I only got a
NoClassDefFoundError on org.apache.sqoop.connector.spi.SqoopConnector,
which is packaged into the WAR..


In other words, I think if my connector jar is part of the WAR, things will
work (effectively what I achieved by adding my connector also to the
pom.xml just like hdfs/jdbc connectors).. I am trying to understand whats
the right way to do this..

@Ravi , I tried the lib approach also (which is based off common.loader
again). did not help. Details below.

I have the sqoop2 branch checked out.
1) I first did the following to build out a binary package
    $ mvn clean package -DskipTests -Dbinary

2) Hopping into the install directory from there on
$ pwd
~/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop200
$ ls server/lib/*connector*jar
server/lib/my-connector-0.1-SNAPSHOT.jar # placed my connector jar in lib
$ jar tf ./server/target/sqoop.war | grep -i my # no match
$ jar tf ./server/webapps/sqoop.war | grep -i my # no match

When I start up the server, it cannot find the SqoopConnector class which
my jar references since I am extending SqoopConnector

Thanks
Vinoth


On Tue, Nov 11, 2014 at 7:09 PM, Xu, Qian A <qian.a.xu@intel.com> wrote:

>  Hi Vinoth,
>
>
>
> If you cannot use **any** connector, change “common.loader” to load
> correct jars will help.
>
> If only your connector meets NoClassDefFound issue, it means your jars are
> not packaged with sqoop.war.
>
>
>
> --Stanley
>
>
>
> *From:* Vinoth Chandar [mailto:vinoth@uber.com]
> *Sent:* Wednesday, November 12, 2014 9:33 AM
> *To:* user@sqoop.apache.org
> *Subject:* Install a new connector jar into sqoop2
>
>
>
> Hey guys,
>
>
>
> I am working on a connector which connects to a sharded mysql cluster as
> the source. I built out the connector jar with its
> sqoopconnector.properties.
>
>
>
> And I tried to make sqoop pick up the jar a couple of ways
>
>
>
> 1) Add the path to the jar in "common.loader" property in
> catalina.properties: This gives a NoClassDefFound on the
> SqoopConnector.class since it is probably in tomcat's system loader, which
> may be referenceable by external libs.
>
>
>
> 2) Randomly placing the connector jar in shared/server loader, which also
> did not work.
>
>
>
> Wound up adding the connector as a dependency in sqoop-server's pom and it
> worked. (sort of validates the theory in 1)
>
>
>
> I am probably a tomcat noob, so please let me know if there is a cleaner
> way to do this or supported documentation .. Or if this is an open issue,
> happy to work on a patch..
>
> Thanks
>
> Vinoth
>

Mime
View raw message