sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Joshi <visit2ra...@gmail.com>
Subject Re: Handling CLOBs in Sqoop - Hive Import
Date Wed, 06 Nov 2013 18:15:50 GMT
Hi,



We are trying to use Sqoop for importing data from Oracle. The table has
CLOB as one of its column type which contains newline characters at many
places. Tried using --hive-drop- import-delims option but somehow it’s not
working. The data still contains newlines, and so Hive table doesn’t read
them properly. Found that this works with SQL Server tables smoothly. The
table / commands / sqoop options are more or less similar (except
connection strings etc), not sure why it’s not working with Oracle. In case
of import from Oracle,  the delims are not getting droped where for SQL
Server, it modifies the data on HDFS.



Sqoop also has a way to treat CLOB as external file (setting --inline-lob-limit
to 0), wanted to know how this can be used along with Hive. Could import
the data using this option, but the import fails if this option is used
with –hive-import option. Is there any known way of dealing with such
external CLOB data in Hive?



Please let us know if anyone has any suggestions.



Regards,

--Rahul Joshi.


On 6 November 2013 13:09, Rahul Joshi <visit2rahul@gmail.com> wrote:

> Hi,
>
>
>
> We are trying to use Sqoop for importing data from Oracle. The table has
> CLOB as one of its column type which contains newline characters at many
> places. Tried using --hive-drop- import-delims option but somehow it’s not
> working. The data still contains newlines, and so Hive table doesn’t read
> them properly. Found that this works with SQL Server tables smoothly. The
> table / commands / sqoop options are more or less similar (except
> connection strings etc), not sure why it’s not working with Oracle. In case
> of import from Oracle,  the delims are not getting droped where for SQL
> Server, it modifies the data on HDFS.
>
>
>
> Sqoop also has a way to treat CLOB as external file (setting --inline-lob-limit
> to 0), wanted to know how this can be used along with Hive. Could import
> the data using this option, but the import fails if this option is used
> with –hive-import option. Is there any known way of dealing with such
> external CLOB data in Hive?
>
>
>
> Please let us know if anyone has any suggestions.
>
>
>
> Regards,
>
> --Rahul Joshi.
>



-- 
Regards,
Rahul R. Joshi.

Mime
View raw message