sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Martin Pampliega <jpampli...@gmail.com>
Subject Last value in incremental import is not being updated
Date Fri, 06 Mar 2015 14:55:03 GMT

I am using *sqoop-1.4.5.bin__hadoop-1.0.0* in Amazon EMR.

I have compiled it for hadoop 2 with *ant clean package -Dhadoopversion=200*

I am running an incremental job which I have saved in the metastore with
the following configuration:

/home/hadoop/sqoop/bin/sqoop job --create import-events-inc2 \
--meta-connect jdbc:hsqldb:hsql://localhost:16000/sqoop \
-- import \
--connect 'jdbc:mysql://mysql01:3306/db' \
--username user \
--password pass \
--table events \
--check-column id \
--incremental append \
--last-value 215885892 \
--direct \
-m 10 \
--hive-import \
--target-dir /data/events \
-- \
--skip-lock-tables --single-transaction --quick

When I execute the job with:

/home/hadoop/sqoop/bin/sqoop job --exec events-inc \
--meta-connect jdbc:hsqldb:hsql://localhost:16000/sqoop

It works fine and brings the data to HDFS and does not throw any ERRORS.

The problem is that the property incremental.last.value in the
SQOOP_SESSIONS table is not being modified.

I tried moving the metastore to an outside database (MySQL) and I keep
having the same problem.

Any idea what the problem is or things I should try to fix it?


View raw message