sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anirudh <techie.anir...@gmail.com>
Subject Re: Error while importing from Teradata to HDFS
Date Mon, 23 Apr 2012 18:17:49 GMT
Dipesh,

It would be ideal if the jdbc driver is placed under Sqoop/lib as other
users too would be able to leverage the same. But given your situation, use
the -libjars option to point to the driver.

Hope this helps.

Thanks,
Anirudh

On Mon, Apr 23, 2012 at 11:01 AM, DIPESH KUMAR SINGH
<dipesh.tech@gmail.com>wrote:

> Hi Cheolsoo,
>
> Yeah, I know. I was looking for an alternative way by which we could tell
> Sqoop to load jars from a location different than Sqoop/lib. I was looking
> for such an alternative because currently i don't have
> permissions to add jars in Sqoop/lib, though i have initiated the process.
>
> Thanks & Regards,
>
> Dipesh
>
>
>
> On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <cheolsoo@cloudera.com>wrote:
>
>> Hi Dipesh,
>>
>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDriver
>>>
>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDrive
>>
>>
>> Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it
>> is to put *terajdbc4.jar* and *tdgssconfig.jar* in */usr/lib/sqoop/lib*.
>>
>> Thanks,
>> Cheolsoo
>>
>> On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <
>> dipesh.tech@gmail.com> wrote:
>>
>>> Hi Cheolsoo,
>>>
>>> As you suggested, I tried after removing "TeraDriver" line from
>>> connectors file but that didn't worked
>>> too. Please help..!!
>>>
>>> Here is the stack trace:
>>>
>>>>
>>>> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>
>>> Enter password:
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing
>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>> through URL:
>>>> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
>>>> java.net.URLClassLoader@27b15692
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>> JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar
>>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
>>>> java.net.FactoryURLClassLoader@19e3cd51
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>> com.cloudera.sqoop.manager.DefaultManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with
>>>> scheme: jdbc:teradata:
>>>>
>>> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of
>>>> 1000
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>>>> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>>>>
>>> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>>>>
>>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDriver
>>>>
>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDriver
>>>>
>>>  at
>>>> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>>>>
>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>>>>
>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>>>>
>>> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>>>>
>>> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>>>
>>> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>>>>
>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>>>>
>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>
>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>
>>>
>>>>
>>> Thanks,
>>>
>>> Dipesh
>>>
>>>
>>>
>>> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <cheolsoo@cloudera.com>wrote:
>>>
>>>> Hi Dispesh,
>>>>
>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>> terajdbc4.jar> along with
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>> sqoop-td-connector-1.0.5.jar>
>>>>> in connectors file inside managers.d directory.
>>>>
>>>>
>>>> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
>>>> terajdbc4.jar>" from your connector file.
>>>>
>>>> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
>>>> to sqoop-td-connector-1.0.5.jar>".
>>>>
>>>> Thanks,
>>>> Cheolsoo
>>>>
>>>>
>>>> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
>>>> dipesh.tech@gmail.com> wrote:
>>>>
>>>>> I am trying to import the data from Teradata to hdfs referring
>>>>> Cloudera Connector guide. I added the property
>>>>>
>>>>> (The Hadoop environment i am working in is a shared Hadoop
>>>>> installation, So users are not allowed to
>>>>> add any jars in lib directory or make changes in Sqoop-site.xml of
>>>>> central installation.
>>>>> However, users can use .profile i.e. user's configuration files to
>>>>> specify CLASSPATH )
>>>>>
>>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>> terajdbc4.jar> along with
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>> sqoop-td-connector-1.0.5.jar>
>>>>> in connectors file inside managers.d directory. In my CLASSPATH, i
>>>>> have added path to
>>>>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
>>>>> jars.
>>>>>
>>>>>
>>>>> With these I suppose i am able to connect to teradata using the
>>>>> teradata manager factory using the teradriver as well
>>>>> but, still i am facing this when i issue the sqoop import command as
>>>>> below
>>>>>
>>>>> sqoop import
>>>>> -libjars <path to tdgssconfig.jar>
>>>>> --driver com.teradata.jdbc.TeraDriver
>>>>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>>>>> --username hadoop --P
>>>>> --table SQOOP_TEST
>>>>> --num-mappers 3 --verbose
>>>>>
>>>>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on
>>>>> the command-line is insecure. Consider using -P instead.
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>>> java.net.URLClassLoader@6a8c436b
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>>> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>> jar /Sqoop/sqoop-td-connector-1.0.5.jar:
>>>>> java.net.FactoryURLClassLoader@45db05b2
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>>> /Sqoop/conf/managers.d/connectors
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>> class: com.teradata.jdbc.TeraDriver
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>>> java.net.FactoryURLClassLoader@45db05b2
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>> com.teradata.jdbc.TeraDriver
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>>> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>> jar /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified
by
>>>>> /Sqoop/conf/managers.d/connectors
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating
>>>>> database manager: java.lang.ClassCastException:
>>>>> com.teradata.jdbc.TeraDriver cannot be cast to
>>>>> com.cloudera.sqoop.manager.ManagerFactory
>>>>> at
>>>>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>>>>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>>>>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>>>>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>>>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>>>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>>
>>>>>
>>>>> Please help.
>>>>>
>>>>> Thanks & Regards,
>>>>> --
>>>>> Dipesh Kr. Singh
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dipesh Kr. Singh
>>>
>>>
>>>
>>>
>>>
>>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>
>

Mime
View raw message