sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Kemper <mar...@cloudera.com>
Subject Re: sqoop import for UUID(primary key)
Date Sun, 25 Sep 2016 16:35:08 GMT
Hello Selvam,

You can use a single maptask (no split).  If you are ingesting from Oracle you can us --direct
which does not use column keys to generate splits.

Thanks, Markus

> On Sep 25, 2016, at 10:14, Selvam Raman <selmna@gmail.com> wrote:
> 
> I have 1 TB of data in databse. Primary key are alphanumeric.
> Now how can I use sqoop.
> 
> Is it possible to use sqoop to import.
> 
> Thanks,
> Selvam R
> +91-97877-87724
> 
>> On Sep 23, 2016 3:17 PM, "Markus Kemper" <markus@cloudera.com> wrote:
>> As Ravi noted, non-numeric keys are not reliable and can result in both duplicate
as well as missing rows.  When using a non-numeric key for split-by you should observe a warning
in the debug console output.
>> 
>> 
>> Markus Kemper
>> Customer Operations Engineer
>> 
>> 
>> 
>>> On Fri, Sep 23, 2016 at 10:11 AM, Ravi, Chandramouli <Chandramouli.Ravi@vantiv.com>
wrote:
>>> It won't work well when Primary key is alpha numeric. I think data will be skewed
or won't come back as expected creating non-balanced split files.
>>> 
>>> Specify different numeric index as Split key if numeric primary key is not present.
>>> 
>>>  
>>> 
>>> From: Selvam Raman [mailto:selmna@gmail.com] 
>>> Sent: Friday, September 23, 2016 10:09 AM
>>> To: user@sqoop.apache.org
>>> Subject: sqoop import for UUID(primary key)
>>> 
>>>  
>>> 
>>> Hi,
>>> 
>>>  
>>> 
>>> In Sqoop If i am having primary key (Number value) and number of parallel task
then it will work (max-min/number of task), to pull the data from table to hdfs.
>>> 
>>>  
>>> 
>>> suppose if i have the primary key as UUID(alpha numeric value), how the load
will be distributed.
>>> 
>>>  
>>> 
>>> Thank you for your help.
>>> 
>>>  
>>> 
>>> --
>>> 
>>> Selvam Raman
>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>> 
>>> 
>>>  **NOTICE: This e-mail message, including any attachments hereto, is for the
sole use of the intended recipient(s) and may contain confidential and/or privileged information.
 If you are not the intended recipient(s), any unauthorized review, use, copying, disclosure
or distribution is prohibited.  If you are not the intended recipient(s), please contact the
sender by reply e-mail immediately and destroy the original and all copies (including electronic
versions) of this message and any of its attachments.
>>> 
>> 

Mime
View raw message