sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Kemper <mar...@cloudera.com>
Subject Re: Upsert option in Sqoop export command.
Date Sun, 03 Dec 2017 21:41:40 GMT
Hey Alana,

—lines-* and —fields-* are for (sqoop import) and —input-lines-* and —input-fields-*
are for (sqoop export).

If you do not specify it will use the default which may or may not be compatible with your
data files.  

As a best practice I always recommend explicitly setting all options to avoid any confusion
or assumptions.

Thanks, Markus

> On Dec 3, 2017, at 12:46, Ajana Chandiruthil Sathian <ajanacs@gmail.com> wrote:
> 
> Hello Markus,
> 
> Can I ask you one question? Does that make any change? Both are for the same purpose,
right? Please correct me if I am wrong. 
> 
>> On Sun, Dec 3, 2017 at 10:39 AM, Markus Kemper <markus@cloudera.com> wrote:
>> Hello Ajana,
>> 
>> Have you tried using (--input-lines-terminated-by '\n') instead of (--lines-terminated-by
'\n')?
>> 
>> 
>> Markus Kemper
>> Customer Operations Engineer
>> 
>> 
>> 
>>> On Sun, Dec 3, 2017 at 8:45 AM, Ajana Chandiruthil Sathian <ajanacs@gmail.com>
wrote:
>>> 
>>> ---------- Forwarded message ----------
>>> From: Ajana Chandiruthil Sathian <ajanacs@gmail.com>
>>> Date: Wed, Nov 15, 2017 at 11:09 AM
>>> Subject: Upsert option in Sqoop export command.
>>> To: user@sqoop.apache.org
>>> 
>>> 
>>> To whom ever it may concern,
>>> 
>>> I have a csv file in Hadoop and I did Sqoop export to Oracle. The column data
type order in ODS is number,date, date,float,varchar and the column data type in the csv file
is number,date, float,date,varchar and I used the --columns sqoop argument to get control
in column ordering and it worked. But I could not control the column ordering when I was doing
the upsert operation in sqoop export. It is giving  me misalignment in ODS after Sqoop( please
see the attached image). The below given is the sqoop command:
>>> 
>>> sqoop export --connect ConnectionString \
>>> --username xxx \
>>> --password xxxx \
>>> --table tableName \
>>> --export-dir /dir/TestUpdate.txt \
>>> --input-fields-terminated-by ',' \
>>> --lines-terminated-by '\n' \
>>> --update-key column_name \
>>> --update-mode allowinsert \
>>> --columns "id,START_DT,VAL,end_DT,QUALITY" \
>>> -m 4
> 

Mime
View raw message