sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yogesh kumar <yogeshsq...@gmail.com>
Subject Re: Sqoop incremental import ( can any just help me out)
Date Mon, 30 Dec 2013 19:27:11 GMT
Thanks Chalcy, I got your point, let me try a simple test for it..   but
the situation here is for incremental import i have to change the column
for split by

Its a kind of risk..   can not take a chance.  just want to be sure that.

it will not affect the hive table and data into it after
being incremental import. my incremental  import will directly pull data
and put it at where my old sqooped data resides

Want suggestion from champions of sqoop
Pls hep me out





On Tue, Dec 31, 2013 at 12:30 AM, Chalcy <chalcy@gmail.com> wrote:

> I have not tried this but I believe you can change the split by as you
> wish.  The split by is used to split the jobs while --check-column and
> --last-value are used for incremental import.
>
> I do not know exact scenario but if empno gives a better split, you still
> can use that for incremental import instead of changing the split-by field.
>
> I would suggest you do a very simple test to find out.
>
> Hope this helps,
> Chalcy
>
>
> On Mon, Dec 30, 2013 at 1:18 PM, yogesh kumar <yogeshsqoop@gmail.com>wrote:
>
>> Hello all,
>>
>> I have done sqoop import for a particluar table first time say table
>> Employee..
>>
>> sqoop import -libjars .....
>> --query "select empno, name, date, loc from table Employee where
>> \$CONDITIONS ..  "
>> *--split-by empno*
>> --fields-terminated-by ','
>> .
>> .
>> .
>> .
>>
>> I have created an external table on hive,
>>
>> *Now I want to pull data on daily basis by using incremental pull.  can
>> I specify the different column for --split-by*
>>
>> like
>>
>> sqoop import -libjars .....
>> --query "select empno, name, date, loc from table Employee where
>> \$CONDITIONS ..  "
>> --check-column date
>> --incremental append
>> --last-value 2013-05-01
>> *--split-by date*
>> --split-by empno
>>
>>
>> Can I change the column for *split by in incremental sqoop*, if not then
>> how to do it.
>>
>> Pls suggest
>>
>
>

Mime
View raw message