sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkat Ranganathan" <n....@live.com>
Subject Re: Review Request 39643: Review for SQOOP-2620:Sqoop fails with Incremental Imports and Upserts in Teradata using Teradata JDBC Driver
Date Sun, 01 Nov 2015 03:34:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39643/#review104649
-----------------------------------------------------------


All new files need to have apache license.   Please make sure you run through ant checkstyle
target and resolve any new violations


src/test/org/apache/sqoop/mapreduce/TestTeradataUpsertOutputFormat.java (line 30)
<https://reviews.apache.org/r/39643/#comment162897>

    Not sure why are we using jdbc.hsqldb here


- Venkat Ranganathan


On Oct. 29, 2015, 2:25 a.m., kopal niranjan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39643/
> -----------------------------------------------------------
> 
> (Updated Oct. 29, 2015, 2:25 a.m.)
> 
> 
> Review request for Sqoop and Venkat Ranganathan.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> When sqoop incremental import is run over Teradata DB,
> sqoop import --driver com.teradata.jdbc.TeraDriver --connect jdbc:teradata://10.126.62.46/DATABASE=SampleDb,TMODE=ANSI,LOGMECH=LDAP
--username abc -P --table test1 --incremental lastmodified --check-column DAT --last-value
'2014-10-22 07:00:00' --target-dir /user/$USER/hive_data_dynapart/test1dir
> then the following error occurs:
> 15/09/24 03:57:10 ERROR manager.SqlManager: SQL exception accessing current timestamp:
com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.10.00.26] [Error
3706] [SQLState 42000] Syntax error: expected something between '(' and ')'.
> com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.10.00.26]
[Error 3706] [SQLState 42000] Syntax error: expected something between '(' and ')'.
> at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:307)
> at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:109)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:314)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:202)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:123)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementController.run(StatementController.java:114)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:384)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:326)
> at com.teradata.jdbc.jdbc_4.TDStatement.doNonPrepExecuteQuery(TDStatement.java:314)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeQuery(TDStatement.java:1091)
> at org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp(SqlManager.java:987)
> at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:328)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> 15/09/24 03:57:10 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: Could not get current time from database
> at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:330)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> -----------------------------------------------------------------------------------------------------------------------------------------
> 
> 
> Sqoop export fails when we perform upserts from HDFS to Teradta
> 
> sqoop export --driver com.teradata.jdbc.TeraDriver --connect jdbc:teradata://10.127.54.34/DATABASE=user_db,TMODE=ANSI,LOGMECH=LDAP
--username abc -P --table upsert_tbl --export-dir /user/root/exportRec/ --update-key id --update-mode
allowinsert
> 
> Above command fails with the following error:
> 
> 15/10/16 02:51:19 ERROR tool.ExportTool: Error during export: Mixed update/insert is
not supported against the target database yet
> 
> 
> Diffs
> -----
> 
>   src/docs/user/connectors.txt c5ce4d6 
>   src/java/org/apache/sqoop/manager/TeradataJdbcManager.java PRE-CREATION 
>   src/java/org/apache/sqoop/mapreduce/TeradataUpsertOutputFormat.java PRE-CREATION 
>   src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java b734e05 
>   src/java/org/apache/sqoop/mapreduce/db/TeradataDataDrivenDBInputFormat.java PRE-CREATION

>   src/java/org/apache/sqoop/mapreduce/db/TeradataDataDrivenDBRecordReader.java PRE-CREATION

>   src/test/com/cloudera/sqoop/manager/TeradataJdbcManagerIncrementalImportTest.java PRE-CREATION

>   src/test/com/cloudera/sqoop/manager/TeradataJdbcManagerUpsertTest.java PRE-CREATION

>   src/test/org/apache/sqoop/mapreduce/TestTeradataUpsertOutputFormat.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/39643/diff/
> 
> 
> Testing
> -------
> 
> yes
> 
> 
> Thanks,
> 
> kopal niranjan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message