sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richa Sharma <mailtorichasha...@gmail.com>
Subject Re: Sqoop import into a Hive table with 2 columns as partition keys
Date Thu, 13 Mar 2014 06:28:31 GMT
Thanks for the inputs.

In my case the partition columns are dynamic.. so I guess it will make
sense to load it into 2 hops.

Venkat, thanks for sharing the ppt on Hcatalog - I couldn't find much
online so it is very helpful :-)

Richa


On Thu, Mar 13, 2014 at 11:33 AM, Venkat Ranganathan <
vranganathan@hortonworks.com> wrote:

> In Sqoop 1.4.4 we have Hcatalog Integration.   This allows importing data
> into hive partition tables with multiple partition keys (but only can be a
> static partitioning key).    See
> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_sqoop_hcatalog_integration
>
> There is a presentation done at Hadoop World on Hcatalog integration that
> you can refer to for more information
>
>
> https://cwiki.apache.org/confluence/download/attachments/27361435/SqoopHCatIntegration-HadoopWorld2013.pptx
>
> Venkat
>
>
> On Wed, Mar 12, 2014 at 9:19 PM, bejoy ks <bejoyks@gmail.com> wrote:
>
>> AFAIK. Sqoop 1x imports supports hive single column partitions.
>>
>> If it is multi column partitions you might have to take a two hop approach
>> 1. Load the data into a single/non partitioned hive tabe
>> 2. From the above staging table load to the final one.
>>
>>
>> On Tue, Mar 11, 2014 at 7:23 AM, Richa Sharma <
>> mailtorichasharma@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am trying to run sqoop import into a Hive table partitioned on 2
>>> columns (col1 string, col2 string).
>>>
>>> I looked at the documentation for both --hive-import and --hcatalog and
>>> could not find any such option.
>>>
>>> It works successfully when table is partitioned on 1 column but not when
>>> i increase number of partition columns.
>>>
>>> Has anyone tried it and knows how to achieve it ?
>>>
>>> Thanks,
>>> Richa
>>>
>>
>>
>>
>> --
>> Regards
>>        Bejoy
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Mime
View raw message