sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bejo...@gmail.com
Subject Re: [sqoop-user] Dynamic Partition & Sqoop
Date Mon, 08 Aug 2011 18:12:34 GMT
Hi Arvind
        With this option is it feasible to accomplish dynamic partitions with a single SQOOP
command? 
It does well for populating a static partition with values. It'd be great if you just brief
on this implementation.

ie with the step as mentioned, in Sqoop import we need to provide the where clause to fetch
the relevant data from db and load it to some pre partition defined hive table, where the
partitioned hive column is defined by '--hive-partition-key' and the value defined by '--hive-partition-value'.
With this approach. In order to implement DYNAMIC partitions we need to analyze and identify
the different values corresponding to the partition column of hive from db table. If there
are n different values identified then n different SQOOP imports with appropriate where clauses
 need to be fired . Is my understanding right?

Thank You

Regards
Bejoy K S

-----Original Message-----
From: "arvind@cloudera.com" <arvind@cloudera.com>
Date: Mon, 8 Aug 2011 10:00:20 
To: <sqoop-user@incubator.apache.org>
Reply-To: sqoop-user@cloudera.org
Subject: Re: [sqoop-user] Dynamic Partition & Sqoop

[bcc:sqoop-user@cloudera.org (deprecated), to:
sqoop-user@incubator.apache.org. Please continue the discussion on
Apache mailing lists - for more information see [1]].

Hi,

Sqoop does support populating Hive partitions. The relevant command
line arguments you need to supply are --hive-partition-key <key-name>,
--hive-partition-value <value-string>. Please refer to the user-guide
for more details.

Thanks,
Arvind

[1] https://cwiki.apache.org/confluence/display/SQOOP/Mailing+Lists

On Mon, Aug 8, 2011 at 5:17 AM, Cyberlord <kkrishnan88@gmail.com> wrote:
> I’d like to know if Dynamic Partition is supported by Sqoop.
>
> For eg. If I have a database(RDBMS) Informix and want to import data
> into Hive,
> Will it be possible for me to import the data through sqoop into Hive
> after creating dynamic partitions (supported by Hive) on-the-fly ?
>
> I possibly thought of loading the data into a temporary table (through
> Sqoop)
> and then load it into the required table through Hive interface.
>
> I wanted to know if I could migrate the data directly into Hive
> through Sqoop while creating dynamic partitions in Hive without
> using a temporary table.
>
> If it is possible, please let me know how to do it.
>
> Thanks :)
>
> --
> NOTE: The mailing list sqoop-user@cloudera.org is deprecated in favor of Apache Sqoop
mailing list sqoop-user@incubator.apache.org. Please subscribe to it by sending an email to
incubator-sqoop-user-subscribe@apache.org.
>

-- 
NOTE: The mailing list sqoop-user@cloudera.org is deprecated in favor of Apache Sqoop mailing
list sqoop-user@incubator.apache.org. Please subscribe to it by sending an email to incubator-sqoop-user-subscribe@apache.org.
Mime
View raw message