sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin BONNET (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3271) DirectNetezzaManager Fails for checkTable method for row validation
Date Thu, 18 Jan 2018 20:48:00 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benjamin BONNET updated SQOOP-3271:
-----------------------------------
    Description: 
While using --direct method which invokes DirectNetezzaManager, checkTable method tries to
validate 1 row using this query
{code:java}
  private static final String QUERY_CHECK_DICTIONARY_FOR_TABLE =
      "SELECT 1 FROM _V_TABLE WHERE OWNER= ? "
      + " AND TABLENAME = ? ";
{code}
For validity, the check introduced for the query
{code:java}
 if (!rs.next()) {code}
is already at the first row and when assessed for next ResultSet, gets into the exception
clause.

Here is an example of the error:
{code:java}
[sqoop@hdp261 sqoopjar]$ sqoop export --connect jdbc:netezza://10.10.20.14:5480/Test --table
MYTEST --username admin --password password --hcatalog-database default --hcatalog-table mysource
--input-fields-terminated-by "," --input-null-string "\\\\N" --input-null-non-string "\\\\N"
--direct --batch
Warning: /usr/hdp/2.6.1.0-129/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/hdp/2.6.1.0-129/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Listening for transport dt_socket at address: 14444
17/12/22 20:36:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.1.0-129
17/12/22 20:36:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
17/12/22 20:36:35 WARN tool.BaseSqoopTool: Input field/record delimiter options are not used
in HCatalog jobs unless the format is text.   It is better to use --hive-import in those cases.
 For text formats
17/12/22 20:36:35 INFO manager.SqlManager: Using default fetchSize of 1000
17/12/22 20:36:35 INFO tool.CodeGenTool: Beginning code generation
17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
17/12/22 20:36:45 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.1.0-129/hadoop-mapreduce
Note: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.java uses or overrides
a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/12/22 20:36:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.jar
17/12/22 20:36:48 ERROR manager.DirectNetezzaManager: MYTEST is not a valid Netezza table.
 Please make sure that you have connected to the Netezza DB and the table name is right. 
 The current values are
	  connection string : jdbc:netezza://10.10.20.14:5480/Test
	  table owner : admin
	  table name : MYTEST
17/12/22 20:36:48 ERROR tool.ExportTool: Encountered IOException running export job: java.io.IOException:
MYTEST is not a valid Netezza table.  Please make sure that you have connected to the Netezza
DB and the table name is right.   The current values are
	  connection string : jdbc:netezza://10.10.20.14:5480/Test
	  table owner : admin
	  table name : MYTEST

{code}

  was:
While using --direct method which invokes DirectNetezzaManager, checkTable method tries to
validate 1 row using this query

{code}
  private static final String QUERY_CHECK_DICTIONARY_FOR_TABLE =
      "SELECT 1 FROM _V_TABLE WHERE OWNER= ? "
      + " AND TABLENAME = ? ";
{code}

For validity, the check introduced for the query {code} if (!rs.next()) {code} is already
at the first row and when assessed for next ResultSet, gets into the exception clause. 

Here is an example of the error:

{code}
[sqoop@hdp261 sqoopjar]$ sqoop export --connect jdbc:netezza://10.10.20.14:5480/Test --table
MYTEST --username admin --password password --hcatalog-database default --hcatalog-table mysource
--input-fields-terminated-by "," --input-null-string "\\\\N" --input-null-non-string "\\\\N"
--direct --batch
Warning: /usr/hdp/2.6.1.0-129/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/hdp/2.6.1.0-129/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Listening for transport dt_socket at address: 14444
17/12/22 20:36:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.1.0-129
17/12/22 20:36:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
17/12/22 20:36:35 WARN tool.BaseSqoopTool: Input field/record delimiter options are not used
in HCatalog jobs unless the format is text.   It is better to use --hive-import in those cases.
 For text formats
17/12/22 20:36:35 INFO manager.SqlManager: Using default fetchSize of 1000
17/12/22 20:36:35 INFO tool.CodeGenTool: Beginning code generation
17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
17/12/22 20:36:45 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.1.0-129/hadoop-mapreduce
Note: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.java uses or overrides
a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/12/22 20:36:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.jar
17/12/22 20:36:48 ERROR manager.DirectNetezzaManager: MYTEST is not a valid Netezza table.
 Please make sure that you have connected to the Netezza DB and the table name is right. 
 The current values are
	  connection string : jdbc:netezza://10.10.20.14:5480/Test
	  table owner : admin
	  table name : MYTEST
17/12/22 20:36:48 ERROR tool.ExportTool: Encountered IOException running export job: java.io.IOException:
MYTEST is not a valid Netezza table.  Please make sure that you have connected to the Netezza
DB and the table name is right.   The current values are
	  connection string : jdbc:netezza://10.10.20.14:5480/Test
	  table owner : admin
	  table name : MYTEST

{code}



> DirectNetezzaManager Fails for checkTable method for row validation
> -------------------------------------------------------------------
>
>                 Key: SQOOP-3271
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3271
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors
>    Affects Versions: 1.4.6
>            Reporter: Shyam Rai
>            Priority: Major
>
> While using --direct method which invokes DirectNetezzaManager, checkTable method tries
to validate 1 row using this query
> {code:java}
>   private static final String QUERY_CHECK_DICTIONARY_FOR_TABLE =
>       "SELECT 1 FROM _V_TABLE WHERE OWNER= ? "
>       + " AND TABLENAME = ? ";
> {code}
> For validity, the check introduced for the query
> {code:java}
>  if (!rs.next()) {code}
> is already at the first row and when assessed for next ResultSet, gets into the exception
clause.
> Here is an example of the error:
> {code:java}
> [sqoop@hdp261 sqoopjar]$ sqoop export --connect jdbc:netezza://10.10.20.14:5480/Test
--table MYTEST --username admin --password password --hcatalog-database default --hcatalog-table
mysource --input-fields-terminated-by "," --input-null-string "\\\\N" --input-null-non-string
"\\\\N" --direct --batch
> Warning: /usr/hdp/2.6.1.0-129/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: /usr/hdp/2.6.1.0-129/accumulo does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> Listening for transport dt_socket at address: 14444
> 17/12/22 20:36:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.1.0-129
> 17/12/22 20:36:35 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 17/12/22 20:36:35 WARN tool.BaseSqoopTool: Input field/record delimiter options are not
used in HCatalog jobs unless the format is text.   It is better to use --hive-import in those
cases.  For text formats
> 17/12/22 20:36:35 INFO manager.SqlManager: Using default fetchSize of 1000
> 17/12/22 20:36:35 INFO tool.CodeGenTool: Beginning code generation
> 17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
> 17/12/22 20:36:45 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "MYTEST"
AS t WHERE 1=0
> 17/12/22 20:36:45 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.1.0-129/hadoop-mapreduce
> Note: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.java uses or overrides
a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 17/12/22 20:36:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-sqoop/compile/a82bdc2ed69a4cc79c5ca06fa06c18d8/MYTEST.jar
> 17/12/22 20:36:48 ERROR manager.DirectNetezzaManager: MYTEST is not a valid Netezza table.
 Please make sure that you have connected to the Netezza DB and the table name is right. 
 The current values are
> 	  connection string : jdbc:netezza://10.10.20.14:5480/Test
> 	  table owner : admin
> 	  table name : MYTEST
> 17/12/22 20:36:48 ERROR tool.ExportTool: Encountered IOException running export job:
java.io.IOException: MYTEST is not a valid Netezza table.  Please make sure that you have
connected to the Netezza DB and the table name is right.   The current values are
> 	  connection string : jdbc:netezza://10.10.20.14:5480/Test
> 	  table owner : admin
> 	  table name : MYTEST
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message