sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Does sqoop2 support Teradata?
Date Fri, 14 Mar 2014 17:17:29 GMT
Hi John,
Sqoop2 is not polluting the DistributedCache like Sqoop 1 with putting entire classpath to
the MapReduce job. Instead we are propagating only jars that are required. Sadly this mechanism
is not working correctly in Teradata case as the JDBC driver consist of two jars that are
not referencing each other - the dependency is something that user have to know, not something
that jar file exposes. As a result Sqoop will propagate only one of them to the mapreduce
job and that will in turn fail. This will be fixed later when we introduce special Teradata
connector. For now, you could merge both Teradata JDBC driver jars together and put the resulting
jar into /var/log/sqoop2/.

Jarcec

On Fri, Mar 14, 2014 at 11:56:41AM +0800, John Ho wrote:
> Hi Sqoop Team,
> 
> Does sqoop2 support importing data from Teradata?
> 
> I'm using CDH4.5 and not sure where to put Teradata's JDBC driver files.
> 
> I upload tdgssconfig.jar and terajdbc4.jar to /var/lib/sqoop2 and add
> "-classpath
> /var/lib/sqoop2/tdgssconfig.jar" to Java Configuration Options for
> TaskTracker for TaskTracker but it does not work as expected.
> 
> I also upload the jar files to
> /opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/
> but it does not work.
> 
> There's no issues if I use sqoop 1.
> 
> 
> Thanks and regards,
> John Ho
> 
> ===
> Task Log (CDH4.5.0 and Teradata Express 14.10 on SLES11):
> 
> org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs
> during extractor run at
> org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:101) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:268) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by:
> org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable
> to execute the SQL statement at
> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:59)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:50)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.ja
> org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs
> during extractor run at
> org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:101) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:268) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by:
> org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable
> to execute the SQL statement at
> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:59)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:50)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.ja
> org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs
> during extractor run at
> org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:101) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:268) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by:
> org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable
> to execute the SQL statement at
> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:59)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:50)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.ja
> org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs
> during extractor run at
> org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:101) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at
> org.apache.hadoop.mapred.Child$4.run(Child.java:268) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by:
> org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable
> to execute the SQL statement at
> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:59)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:50)
> at
> org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.ja

Mime
View raw message