sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Oozie + Sqoop + MySql UPSERT
Date Fri, 08 May 2015 18:22:59 GMT
I'd get rid of --driver and see if that works.

On Fri, May 8, 2015 at 11:14 AM, Aravindakshan Srinivasan <
s_aravind@yahoo.com> wrote:

> Hi Abraham,
>
> Here's the Sqoop command that works.
>
> sqoop export --connect jdbc:mysql://myhost.rds.amazonaws.com/DB1
> --username XXX -password YYY --table test_table --columns
> "name,ts,age,salary" --update-key "name" --export-dir
> /user/aravind/test/year=2015/month=5/day=7/hour=19 --fields-terminated-by
> "\t" --update-mode allowinsert -m 1
>
> The following workflow does not work.
>
> <workflow-app xmlns="uri:oozie:workflow:0.2" name="ara_sqoop-wf">
>   <start to="ara_move_to_mysql"/>
>   <action name="ara_move_to_mysql">
>     <sqoop xmlns="uri:oozie:sqoop-action:0.3">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <arg>export</arg>
> <arg>-libjars</arg>
> <arg>${wf_folder}lib/mysql-connector-java.jar</arg>
> <arg>--connect</arg>
> <arg>jdbc:mysql://myhost.rds.amazonaws.com/${mysql_db}
> <http://myhost.rds.amazonaws.com/$%7Bmysql_db%7D></arg>
> <arg>--driver</arg>
> <arg>com.mysql.jdbc.Driver</arg>
> <arg>--username</arg>
> <arg>${mysql_usern}</arg>
> <arg>-password</arg>
> <arg>${mysql_pswd}</arg>
> <arg>--table</arg>
> <arg>${table_name}</arg>
> <arg>--columns</arg>
> <arg>"name,ts,age,salary"</arg>
> <arg>--update-key</arg>
> <arg>"name"</arg>
> <arg>--export-dir</arg>
> <arg>/user/aravind/test/year=2015/month=5/day=7/hour=19</arg>
> <arg>--fields-terminated-by</arg>
> <arg>"\t"</arg>
> <arg>--update-mode</arg>
> <arg>allowinsert</arg>
> <arg>-m</arg>
> <arg>1</arg>
>    </sqoop>
>    <ok to="end"/>
>    <error to="fail"/>
>   </action>
>   <kill name="fail">
>       <message>Sqoop failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>   </kill>
>   <end name="end"/>
> </workflow-app>
>
> Thanks,
> Aravind
>
>   ------------------------------
>  *From:* Abraham Elmahrek <abe@cloudera.com>
> *To:* "user@sqoop.apache.org" <user@sqoop.apache.org>; Aravindakshan
> Srinivasan <s_aravind@yahoo.com>
> *Sent:* Friday, May 8, 2015 10:36 PM
> *Subject:* Re: Oozie + Sqoop + MySql UPSERT
>
> Can you send over your oozie job or sqoop command?
>
>
>
> On Fri, May 8, 2015 at 8:30 AM, Aravindakshan Srinivasan <
> s_aravind@yahoo.com> wrote:
>
> Friends,
>
> I have an odd problem. I am trying to run an MySql upsert via Sqoop. It
> works fine from the edge node that has Sqoop 1.4.5 installed. When I try to
> run via Oozie, I get this error.
>
> org.apache.sqoop.tool.ExportTool - Error during export: Mixed
> update/insert is not supported against the target database
>
> This feature is available in the newer versions of Sqoop starting with
> Sqoop 1.4.3, which is what my Oozie sharelib contains. Still, my first
> reaction was that the Sqoop jars in Oozie's sharelib are different and
> hence throwing this error. As you know, the Sqoop via Oozie runs on the
> Hadoop compute nodes. I tried copying the Sqoop jars from my edge node to
> the workflow/lib dir. It still didn't work.
>
>
>
> Since then, I have tried everything I can think of, but can't get past
> this error. I have even rebuilt the Oozie sharelib with Sqoop 1.4.5 and
> tried both  oozie.libpath and oozie.action.sharelib.for.sqoop pointing to
> my rebuilt lib. Still no luck. What am I missing?
>
> Thanks,
> Aravind
>
>
> --------------------------------------------------------------------------------------------
> Aravind Srinivasan
>
> http://goodcynicsports.blogspot.com
>
> I have decided to stick with love. Hate is too great a burden to bear.
> - MLK, Jr.
>
> --------------------------------------------------------------------------------------------
>
>
>
>
>
>
>

Mime
View raw message