sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Valle <marcelo.va...@ktech.com>
Subject Re: Exporting parquet, issues with schema
Date Fri, 24 May 2019 13:05:31 GMT
This assumes I have a hive table. Then I have to generate one for my
parquet.
Parquet files have schemas inside themselves, why should I need a metadata
folder or hive metadata? Couldn't sqoop read the parquet metadata directly
from the file?

On Fri, 24 May 2019 at 13:40, Markus Kemper <markus@cloudera.com> wrote:

>
> Hello Marcelo,
>
> Have you tried using hcatalog to export Parquet?
>
> $ sqoop export --connect $MYSQL_CONN --username $MYSQL_USER --password
> $MYSQL_PWD --table $MYSQL_TABLE --num-mappers 1 --hcatalog-database
> $HIVE_DATABASE --hcatalog-table $HIVE_PARQUET_TABLE
>
> Markus Kemper
> Cloudera Support
>
>
>
>
> On Fri, May 24, 2019 at 7:58 AM Marcelo Valle <marcelo.valle@ktech.com>
> wrote:
>
>> Hi,
>>
>> I know this thread is a little old, I found the original discussion here:
>> https://grokbase.com/t/sqoop/user/15c94s4vjx/exporting-parquet-issues-with-schema
>>
>> But I am having the same problem now, more than 3 years later, and scoop
>> 2 seems to have been deprecated.
>>
>> Are there any plans of not needing this metadata folder anymore?
>>
>> Thanks,
>> Marcelo.
>>
>>
>>
>> This email is confidential [and may be protected by legal privilege]. If
>> you are not the intended recipient, please do not copy or disclose its
>> content but contact the sender immediately upon receipt.
>>
>> KTech Services Ltd is registered in England as company number 10704940.
>>
>> Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE,
>> United Kingdom
>>
>

This email is confidential [and may be protected by legal privilege]. If you are not the intended
recipient, please do not copy or disclose its content but contact the sender immediately upon
receipt.

KTech Services Ltd is registered in England as company number 10704940.

Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE, United Kingdom

Mime
View raw message