sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Lété <thomas.l...@soprism.com>
Subject Re: Sqoop Parquet incremental import “Cannot append files to target dir”
Date Thu, 15 Jan 2015 16:59:36 GMT
Hi Qian,

Thank you for your reply, unfortunately, this table is named datatable (--table datatable).

The file sqoop attempts to create is simply named using the hostname of the server, which
seems to be the problem.

I can send you a complete verbose log if this can help…


> Le 15 janv. 2015 à 16:42, Xu, Qian A <qian.a.xu@intel.com> a écrit :
> HI Thomas,
> Sqoop Parquet support uses Kite SDK. Kite handles hive access differently than Sqoop.
> Hive table does not allow dot in name, so any dot will be replaced with splash. This
was a known behavior of Kite 0.16. https://issues.cloudera.org/browse/CDK-650
> I’m not sure if SQOOP 1.4.5 uses Kite 0.17 already. Please fire a jira, I will take
a look into it.
> --Stanley (Qian) Xu
> From: Thomas Lété [mailto:thomas.lete@soprism.com] 
> Sent: Thursday, January 15, 2015 4:42 PM
> To: user@sqoop.apache.org
> Subject: Sqoop Parquet incremental import “Cannot append files to target dir”
> Hi Everyone !
> I have a problem while importing data from MySQL to Hive as Parquet using Sqoop… (I
had no problem with text files)
> This query :
>         sqoop import --connect jdbc:mysql://xx.xx.xx.xx/database --username sqoop --password
sqoop --table datatable --target-dir /home/cloudera/user/hive/warehouse/database.db/datatable
--as-parquetfile -m 1 --append
> Return this error :
>         15/01/14 16:27:28 WARN util.AppendUtils: Cannot append files to target dir; no
such directory: _sqoop/14162350000000781_32315_servername.ip-xx-xx-xx.eu_datatable
> Files are located in /user/root/_sqoop/ this way : /user/root/_sqoop/14162350000000781_32315_servername/ip-xx-xx-xx/eu_datatable/
> Is it normal that dots from the hostname are replaced by slashes ? It seems to be the
problem but noone is complaining about this problem...
> Or there is an other way to append data to a parquet file ?
> PS : I’m using CDH5.3 which has Sqoop 1.4.5 built in.
> Thank you !

View raw message