sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manish Gupta 8 <mgupt...@sapient.com>
Subject RE: Error while Sqoop Export Hive (AvroSerDe) to MSSQL using Hcat
Date Fri, 11 Dec 2015 11:09:02 GMT
Thanks Jercec. But the problem is that job never even reaches to the MR stage. It fails right
after loading the hive metastore and running tool.ExportTool.
Hence, no logs to check.

.
.
.
15/12/11 16:33:28 DEBUG orm.CompilationManager: Could not rename /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.java
to /root/./tblOthrBus.java
15/12/11 16:33:28 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.jar
15/12/11 16:33:28 DEBUG orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117
15/12/11 16:33:28 DEBUG orm.CompilationManager: Got classfile: /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.class
-> tblOthrBus.class
15/12/11 16:33:28 DEBUG orm.CompilationManager: Finished writing jar file /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.jar
15/12/11 16:33:28 INFO mapreduce.ExportJobBase: Beginning export of tblOthrBus
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Checking for existing class: tblOthrBus
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Attempting to load jar through URL: jar:file:/tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.jar!/
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Previous classloader is sun.misc.Launcher$AppClassLoader@66d2e7d9
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Testing class in jar: tblOthrBus
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: jar:file:/tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.jar!/
15/12/11 16:33:28 DEBUG util.ClassLoaderStack: Added classloader for jar /tmp/sqoop-root/compile/bb46fc39dfc3f2d7a1fbc6df6a2b9117/tblOthrBus.jar:
java.net.FactoryURLClassLoader@e041f0c
15/12/11 16:33:28 INFO mapreduce.ExportJobBase: Configuring HCatalog for export job
15/12/11 16:33:28 INFO hcat.SqoopHCatUtilities: Configuring HCatalog specific details for
job
15/12/11 16:33:28 WARN hcat.SqoopHCatUtilities: Provided HCatalog table name tblOthrBus will
be mapped to  tblOthrBus
15/12/11 16:33:28 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM
[tblOthrBus] AS t WHERE 1=0
15/12/11 16:33:28 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
15/12/11 16:33:28 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [tblOthrBus]
AS t WHERE 1=0
15/12/11 16:33:28 DEBUG manager.SqlManager: Found column RowID of type [-5, 19, 0]
15/12/11 16:33:28 DEBUG manager.SqlManager: Found column indvlPK of type [12, 100, 0]
15/12/11 16:33:28 DEBUG manager.SqlManager: Found column desc of type [12, 2147483647, 0]
15/12/11 16:33:28 INFO hcat.SqoopHCatUtilities: Database column names projected : [rowid,
indvlpk, desc]
15/12/11 16:33:28 INFO hcat.SqoopHCatUtilities: Database column name - info map :
        indvlpk : [Type : 12,Precision : 100,Scale : 0]
        rowid : [Type : -5,Precision : 19,Scale : 0]
        desc : [Type : 12,Precision : 2147483647,Scale : 0]

15/12/11 16:33:29 INFO hive.metastore: Trying to connect to metastore with URI thrift://devmaster01.mydomain.com:9083
15/12/11 16:33:29 INFO hive.metastore: Connected to metastore.
15/12/11 16:33:29 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@66d2e7d9
15/12/11 16:33:29 ERROR tool.ExportTool: Encountered IOException running export job: java.io.IOException:
java.lang.NullPointerException


Regards,
Manish


-----Original Message-----
From: Jarek Jarcec Cecho [mailto:jarcec@gmail.com] On Behalf Of Jarek Jarcec Cecho
Sent: Thursday, December 10, 2015 3:06 PM
To: user@sqoop.apache.org
Subject: Re: Error while Sqoop Export Hive (AvroSerDe) to MSSQL using Hcat

I’m not an hcatalog expert, but I would took a look into mapreduce task logs (especially
into those that are failing).

Jarcec

> On Dec 10, 2015, at 10:17 AM, Manish Gupta 8 <mgupta50@sapient.com<mailto:mgupta50@sapient.com>>
wrote:
>
> Can someone please provide any pointers I should look into? Basically I want to sqoop
export avro data via Hive tables.
>
> Thanks
>
>
> From: Manish Gupta 8 [mailto:mgupta50@sapient.com]
> Sent: Wednesday, December 09, 2015 5:32 PM
> To: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
> Subject: Error while Sqoop Export Hive (AvroSerDe) to MSSQL using Hcat
>
> Hi,
>
> I have an external table in hive created using Avro SerDe. On this, I have created couple
of normalized views. Now, when I try to export any of the normalized views/tables from hive
to sql server using sqoop, I get error: “ERROR tool.ExportTool: Encountered IOException
running export job: java.io.IOException: java.lang.NullPointerException”
>
> My sqoop export looks like following:
> sqoop export --connect "jdbc:sqlserver://*****\****:1433;databaseName=****;user=*****;password=*****;"
--table tblOthrBus --hcatalog-database brokdb --hcatalog-table tblOthrBus --verbose
>
> Please note:
> ·         Table has data and I tested with hive as well as spark jdbc server.
> ·         Database connection works fine. Tested with eval and export works fine with
export-dir option.
> ·         Hcat details are correct. hcat.SqoopHCatUtilities is able to read table metadata
> ·         Hive metastore also gets connected.
>
> Can someone please guide me into what should I look into, any logs etc.
>
> Thanks
>
> Regards,
> Manish Gupta
> Specialist, Platform| Sapient Global Markets
> Noida, India
>
> Tel: +91 (120) 479 5000 x73766
> Mobile: +91 981 059 1361
> Email: mgupta50@sapient.com<mailto:mgupta50@sapient.com>
> sapient.com


Mime
View raw message