sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1821) External connector loading in Sqoop2
Date Tue, 06 Jan 2015 22:39:34 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266873#comment-14266873
] 

Veena Basavaraj edited comment on SQOOP-1821 at 1/6/15 10:39 PM:
-----------------------------------------------------------------

[~vinothchandar] got a chance to read the mulsesoft doc a bit more carefully today. The tomcats
commons loader is now been used and it does not work, was wondering if the shared.loader would
work.

So adding the connector to /some/path and adding this /some/path to the shared.loader property
in the 
'sqoop2/dist/src/main/server/conf/catalina.properties

>>>
The server loader should be left alone, but the shared loader still has many useful applications.
(Note: The shared loader will load its classes last during the start-up process, after the
Commons loader has finished loading its classes.)


update: my solutuon of using shared.loader might still not work, since the code below ends
up loading the sqoop jars way after the tomcat has finished its work....so I take it back:)

{code}
  // Using Boostrap class to boot the common.loader and other catalina specific
    // class loaders.
    Bootstrap bootstrap = new Bootstrap();
    bootstrap.init();

    // Now we need to add the sqoop webapp classes into the class loader. Sadly
    // we have to do a lot of things ourselves. The procedure is:
    // 1) Unpack Sqoop war file
    // 2) Build the ClassLoader using Tomcat's ClassLoaderFactory

    // Various paths to war file and locations inside the war file
    String webappPath = System.getProperty(PROPERTY_APPBASE_PATH, DEFAULT_APPBASE_PATH);
    String catalinaBase = Bootstrap.getCatalinaBase();
    String fullWebappPath = catalinaBase + File.separator + webappPath;
    String fullSqoopWarPath = fullWebappPath + File.separator + "sqoop.war";
    String fullSqoopClassesPath = fullWebappPath + File.separator + "sqoop" + File.separator
+ "WEB-INF" + File.separator + "classes";
    String fullSqoopLibPath = fullWebappPath + File.separator + "sqoop" + File.separator +
"WEB-INF" + File.separator + "lib";

    // Expand the war into the usual location, this operation is idempotent (nothing bad happens
if it's already expanded)
    Embedded embedded = new Embedded();
    Host host = embedded.createHost("Sqoop Tool Virtual Host", fullWebappPath);
    ExpandWar.expand(host, new URL("jar:file://" + fullSqoopWarPath + "!/"));

    // We have expanded war file, so we build the classloader from
    File [] unpacked = new File[1];   unpacked[0] = new File(fullSqoopClassesPath);
    File [] packed = new File[1];     packed[0] = new File(fullSqoopLibPath);
    ClassLoader loader = ClassLoaderFactory.createClassLoader(unpacked, packed, Thread.currentThread().getContextClassLoader());
    Thread.currentThread().setContextClassLoader(loader);


{code}



was (Author: vybs):
[~vinothchandar] got a chance to read the mulsesoft doc a bit more carefully today. The tomcats
commons loader is now been used and it does not work, was wondering if the shared.loader would
work.

So adding the connector to /some/path and adding this /some/path to the shared.loader property
in the 
'sqoop2/dist/src/main/server/conf/catalina.properties

>>>
The server loader should be left alone, but the shared loader still has many useful applications.
(Note: The shared loader will load its classes last during the start-up process, after the
Commons loader has finished loading its classes.)



> External connector loading in Sqoop2
> ------------------------------------
>
>                 Key: SQOOP-1821
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1821
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.99.5
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> Edit this wiki to begin with
> https://cwiki.apache.org/confluence/display/SQOOP/Setting+up+Sqoop+2#SettingupSqoop2-InstallinganewconnectortoSqoop2
> Also, add the same to the .rst of need be
> relevant code that loads all connectors 
> https://github.com/apache/sqoop/blob/sqoop2/core/src/main/java/org/apache/sqoop/connector/ConnectorManagerUtils.java#L44
> NOTE : External contributors had issues with getting a new sqoop connector working seamlessly
with the Sqoop2. They had to do some hacks to include this new jar into the class path. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message