sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop hangs when import from Teradata
Date Wed, 18 Jun 2014 21:01:45 GMT
Hey there,

Glad every thing is working... and sorry it took so long. Did you add the
--verbose flag to the beginning of the import statement? Or did you need to
wait for the job to finish?

-Abe


On Wed, Jun 18, 2014 at 1:56 PM, Benyi Wang <bewang.tech@gmail.com> wrote:

> It turns out that I have to provide "--split-by actionId". It took 53mins
> to print out the exception in my system.
>
> 14/06/18 12:04:33 INFO teradata.TeradataManagerFactory: Loaded connector
> factory for 'Cloudera Connector Powered by Teradata' on version 1.2c5
> 14/06/18 12:04:33 INFO manager.SqlManager: Using default fetchSize of 1000
> 14/06/18 12:04:33 INFO options.ExtraOptions: Parsing extra arguments
> 14/06/18 12:04:33 INFO options.ExtraOptions: Loaded extra option for batch
> size 1000
> 14/06/18 12:04:33 INFO options.OptionsCompatibility: Checking options
> compatibility
> 14/06/18 12:04:35 INFO tool.CodeGenTool: Beginning code generation
> 14/06/18 12:04:35 INFO teradata.TeradataManager: Converting table import
> to query: SELECT * FROM "action"
> 14/06/18 12:04:36 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
> Note:
> /tmp/sqoop-bwang/compile/405654670de634debca58668abd0d1f4/action.java uses
> or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 14/06/18 12:04:40 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-bwang/compile/405654670de634debca58668abd0d1f4/action.jar
> 14/06/18 12:04:40 INFO teradata.TeradataManager: Beginning Teradata table
> based import
> 14/06/18 12:57:25 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Table action does not have primary key. Please
> use --split-by argument if applicable
> java.lang.RuntimeException: Table action does not have primary key. Please
> use --split-by argument if applicable
>         at
> com.cloudera.connector.teradata.TeradataManager.getPrimaryKey(TeradataManager.java:175)
>         at
> org.apache.sqoop.manager.SqlManager.getSplitColumn(SqlManager.java:546)
>         at
> com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273)
>         at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
>
>
>
> On Tue, Jun 17, 2014 at 6:09 PM, Benyi Wang <bewang.tech@gmail.com> wrote:
>
>> I have trouble import Teradata table into HDFS:
>>
>>    1. Installed CDH using cloudera manager on CentOS 6.
>>    2. Installed Sqoop-Teradata-Connector 1.2.5c5 using parcel.
>>    3. Install Sqoop Client service, and deploy the configuration file.
>>    4. Downloaded teradata jdbc driver v15.0.0.15 and v13.10.00.36, and
>>    extracted the jars in ~/apps/terajdbc.
>>    5. export
>>    HADOOP_CLASSPATH=~/apps/terajdbc/terajdbc4.jar:~/apps/terajdbc/tdgssconfig.jar
>>    6. sqoop import --connect jdbc:teradata://teraserver/DATABASE=MYDB
>>    --username bewang -P --table action --num-mappers 1, then it just hangs
>>    after creating jar. The table 'action' is pretty small, only 161 rows.
>>
>> 14/06/17 18:05:08 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-cloudera/compile/6c32f78ff095211b76e44a53cb3cdaef/action.jar
>> 14/06/17 18:05:08 DEBUG orm.CompilationManager: Scanning for .class files
>> in directory: /tmp/sqoop-cloudera/compile/6c32f78ff095211b76e44a53cb3cdaef
>> 14/06/17 18:05:08 DEBUG orm.CompilationManager: Got classfile:
>> /tmp/sqoop-cloudera/compile/6c32f78ff095211b76e44a53cb3cdaef/action.class
>> -> action.class
>> 14/06/17 18:05:08 DEBUG orm.CompilationManager: Finished writing jar file
>> /tmp/sqoop-cloudera/compile/6c32f78ff095211b76e44a53cb3cdaef/action.jar
>> 14/06/17 18:05:08 INFO teradata.TeradataManager: Beginning Teradata table
>> based import
>>
>>    1. I could successfully run "sqoop eval" to dump the table without
>>    problem.
>>
>> What's wrong with my method?
>>
>
>

Mime
View raw message