sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu, Qian A" <qian.a...@intel.com>
Subject RE: Sqoop Parquet incremental import “Cannot append files to target dir”
Date Thu, 15 Jan 2015 15:42:18 GMT
HI Thomas,

Sqoop Parquet support uses Kite SDK. Kite handles hive access differently than Sqoop.

Hive table does not allow dot in name, so any dot will be replaced with splash. This was a
known behavior of Kite 0.16. https://issues.cloudera.org/browse/CDK-650

I’m not sure if SQOOP 1.4.5 uses Kite 0.17 already. Please fire a jira, I will take a look
into it.

--Stanley (Qian) Xu

From: Thomas Lété [mailto:thomas.lete@soprism.com]
Sent: Thursday, January 15, 2015 4:42 PM
To: user@sqoop.apache.org
Subject: Sqoop Parquet incremental import “Cannot append files to target dir”

Hi Everyone !
I have a problem while importing data from MySQL to Hive as Parquet using Sqoop… (I had
no problem with text files)
This query :

        sqoop import --connect jdbc:mysql://xx.xx.xx.xx/database --username sqoop --password
sqoop --table datatable --target-dir /home/cloudera/user/hive/warehouse/database.db/datatable
--as-parquetfile -m 1 --append
Return this error :

        15/01/14 16:27:28 WARN util.AppendUtils: Cannot append files to target dir; no such
directory: _sqoop/14162350000000781_32315_servername.ip-xx-xx-xx.eu<http://xx-xx-xx.eu>_datatable
Files are located in /user/root/_sqoop/ this way : /user/root/_sqoop/14162350000000781_32315_servername/ip-xx-xx-xx/eu_datatable/
Is it normal that dots from the hostname are replaced by slashes ? It seems to be the problem
but noone is complaining about this problem...
Or there is an other way to append data to a parquet file ?

PS : I’m using CDH5.3 which has Sqoop 1.4.5 built in.
Thank you !
View raw message