sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Robson <David.Rob...@software.dell.com>
Subject RE: Sqoop Oracle and MS Access
Date Fri, 06 Nov 2015 23:32:09 GMT

Are you using Sqoop 1.4.6? Have you tried using the “—direct” option for Oracle in order
to utilise the Data Connector for Oracle and Hadoop?

You can get more details in the documentation here: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_data_connector_for_oracle_and_hadoop

This has much better support for Oracle data types and also better performance. By default
it will convert timestamp with timezone to String as Sqoop normally stores dates as a number
which loses the timezone information. There is a parameter documented here if you prefer to
store the data as a number http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_java_sql_timestamp

In regards to Access I have never tried this with Sqoop but did you try the JDBC to ODBC bridge
driver? From memory the easysoft driver lets you run a bridge on a Windows server that you
can connect to from the Linux server so it may work.


From: Giovanni M.L.G. [mailto:giovanni.gargioni@gmail.com]
Sent: Friday, 6 November 2015 7:33 PM
To: user@sqoop.apache.org
Subject: Sqoop Oracle and MS Access


I've been using sqoop for a while and have recently come across two issues:

  1.  We have a use case where we import data as avro from an oracle database (so it's a standard
use case).

The problem is that there are really a lot of tables, and most of these tables contain fields
with "strange" types such as "Timestamp(6) with Timezone".

By default sqoop complains about these fields and breaks, unless one manually specifies how
to convert these columns with "--map-column-java". (We tried this for a few tables, mapping
unsupported types to String).

As I mentioned the number of tables is so large that using this workaround is not a viable

Is there a way to tell sqoop to convert unknown types to String when importing as avro? [I
couldn't find anything on the documentation].

  2.  Another use case we are facing is importing data from Microsoft Access Dbs. Is there
a way to do this with sqoop? [I tried using a few jdbc drivers for Access, but with no success]

Thanks for the support!


View raw message