sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Robson <David.Rob...@software.dell.com>
Subject RE: Oracle Import Error Using --direct Option
Date Tue, 17 Nov 2015 20:13:51 GMT
It provides a large performance boost over the standard connector. In addition it also places
lower load on the target Oracle database.

There is more details in the blog post: http://blog.cloudera.com/blog/2014/11/how-apache-sqoop-1-4-5-improves-oracle-databaseapache-hadoop-integration/


-----Original Message-----
From: Deepak Vohra [mailto:dvohra09@yahoo.com] 
Sent: Wednesday, 18 November 2015 3:19 AM
To: user@sqoop.apache.org
Subject: RE: Oracle Import Error Using --direct Option

Does the --direct provide significant performance gain?
--------------------------------------------
On Mon, 11/16/15, David Robson <David.Robson@software.dell.com> wrote:

 Subject: RE: Oracle Import Error Using --direct Option
 To: "user@sqoop.apache.org" <user@sqoop.apache.org>
 Date: Monday, November 16, 2015, 2:45 PM
 
 Sqoop 1.4.5 and above includes the
 Data Connector for Oracle and Hadoop via the "--direct"
 flag. This was previously known as OraOop.
 
 -----Original Message-----
 From: Deepak Vohra [mailto:dvohra09@yahoo.com]
 
 Sent: Tuesday, 17 November 2015 8:45 AM
 To: user@sqoop.apache.org
 Cc: tdong@splicemachine.com
 Subject: Re: Oracle Import Error Using --direct Option
 
 Oracle Database does not support the --direct mode.
 
 Database    version
 --direct support?    connect string matches  Oracle    10.2.0+  No    jdbc:oracle:*//
 
 https://sqoop.apache.org/docs/1.4.0-incubating/SqoopUserGuide.html#id1773570
 
  
 --------------------------------------------
 On Mon, 11/16/15, Ted Dong <tdong@splicemachine.com>
 wrote:
 
  Subject: Oracle Import Error Using --direct Option
  To: user@sqoop.apache.org
  Date: Monday, November 16, 2015, 1:14 PM
  
  Hi,
  I was able to use the --direct
  option to import data from an Oracle database to HDFS.
 The  Oracle database was a development instance and  the  credentials I was using had full
read/write  privileges.
  I then switched to another Oracle
  database which is a QA instance. The credentials I was  given  contains only read-only
access.  When I run  Sqoop with the --direct  option I get this error:
  15/11/16
  13:06:09 WARN oracle.OraOopManagerFactory: Unable to  determine whether the Oracle table
 "IAPPS"."MF_PROFILE" is an  index-organized
 table.Error:ORA-00942: table  or view does not
 exist15/11/16
  13:06:09 ERROR sqoop.Sqoop: Got exception running Sqoop:
  java.lang.RuntimeException:
  java.sql.SQLSyntaxErrorException: ORA-00942: table or  view  does not exist
  java.lang.RuntimeException:
  java.sql.SQLSyntaxErrorException: ORA-00942: table or  view  does not exist
     
      at
  org.apache.sqoop.manager.oracle.OraOopConnManager.getColumnNamesInOracleTable(OraOopConnManager.java:133)
       at
  org.apache.sqoop.manager.oracle.OraOopConnManager.getSelectedColumnNamesInOracleTable(OraOopConnManager.java:144)
       at
  org.apache.sqoop.manager.oracle.OraOopConnManager.getColTypesQuery(OraOopConnManager.java:216)
       at
  org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:225)
       at
  org.apache.sqoop.manager.oracle.OraOopConnManager.getColumnTypes(OraOopConnManager.java:504)
       at
  org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
       at
  org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
       at
  org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
       at
  org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
       at
  org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
       at
  org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
       at
  org.apache.sqoop.Sqoop.run(Sqoop.java:143)        at
  org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at
  org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
 at
  org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
 at
  org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
 at  org.apache.sqoop.Sqoop.main(Sqoop.java:236)Caused
 by:
  java.sql.SQLSyntaxErrorException: ORA-00942: table or  view  does not exist
     
      at
  oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
       at
  oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
       at
  oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
       at
  oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
       at
  oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
     at
  oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
     at
  oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
       at
  oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
       at
  oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
       at
  oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
       at
  oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
       at
  oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
       at
  oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
       at
  org.apache.sqoop.manager.oracle.OraOopOracleQueries.getTableColumns(OraOopOracleQueries.java:658)
       at
  org.apache.sqoop.manager.oracle.OraOopOracleQueries.getTableColumns(OraOopOracleQueries.java:767)
       at
  org.apache.sqoop.manager.oracle.OraOopOracleQueries.getTableColumnNames(OraOopOracleQueries.java:624)
       at
  org.apache.sqoop.manager.oracle.OraOopConnManager.getColumnNamesInOracleTable(OraOopConnManager.java:124)
       ...
  16 more
  I confirmed that the
  credentials I am using to access the QA database can see  and  read from the table I am
trying to import to HDFS  (confirmed  using toad).
  Are there some other
  user account settings that is required on the Oracle side  to  get the --direct option
working from Sqoop?
  Thanks!
  
Mime
View raw message