sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop incremental.last.value does not update to hsqldb when incremental append to HBase
Date Sat, 22 Nov 2014 04:03:29 GMT
Sorry, I think I made a logical error :).

Could you try adding --verbose to your --exec command? Copy/paste the
output to this thread.

-Abe

On Fri, Nov 21, 2014 at 7:46 PM, Abraham Elmahrek <abe@cloudera.com> wrote:

> Not sure. Could you check your task logs for any exceptions? It's possible
> that your Sqoop job is failing before the --last-value is updated.
>
> -Abe
>
> On Fri, Nov 21, 2014 at 6:33 PM, Wen, Dongrong <
> dongrong.wen@ventana.roche.com> wrote:
>
>> Hi,
>>
>> On CentOS 6.5, I am using Sqoop Version 1.4.4-cdh5.1.0 and HBase Version
>> 0.98.1-cdh5.1.0.
>>
>> Here is my Sqoop job of incremental append from Intersystems Cache to
>> HBase.
>> -------------------------
>> sudo -u hdfs sqoop job --create JobName \
>> -- \
>> import \
>> --verbose \
>> -m 1 \
>> --connect jdbc:Cache://xx.xx.xx.xx:1972/USER \
>> --driver com.intersys.jdbc.CacheDriver \
>> --username username \
>> --password password \
>> --column-family CFNAME \
>> --hbase-table TABLENAME \
>> --hbase-row-key NUM \
>> --query "select * from TABLENAME where \$CONDITIONS" \
>> --incremental append \
>> --check-column NUM \
>> --last-value 9
>> -------------------------
>>
>> I can see the new data rows appended successfully to the table in HBase.
>> But Sqoop's "incremental.last.value" does not get updated to the new
>> last-value on hsqldb. I tried it several times and it is 9 all the time as
>> given during the sqoop job creation. Below is the last few lines of Sqoop
>> output message.
>> >>>>>>>>>>>>>>>>>
>> 14/11/20 15:28:55 INFO mapreduce.ImportJobBase: Transferred 0 bytes in
>> 24.6684 seconds (0 bytes/sec)
>> 14/11/20 15:28:55 INFO mapreduce.ImportJobBase: Retrieved 7 records.
>> 14/11/20 15:28:55 DEBUG util.ClassLoaderStack: Restoring classloader:
>> sun.misc.Launcher$AppClassLoader@5ac524dd
>> 14/11/20 15:28:55 ERROR tool.ImportTool: Imported Failed: Can not create
>> a Path from a null string
>> 14/11/20 15:28:55 DEBUG hsqldb.HsqldbJobStorage: Flushing current
>> transaction
>> 14/11/20 15:28:55 DEBUG hsqldb.HsqldbJobStorage: Closing connection
>> >>>>>>>>>>>>>>>>>>
>>
>> However, if I change the target to HDFS. It works perfectly. Sqoop's
>> "incremental.last.value" does update to hsqldb without any problem. Here is
>> my Sqoop incremental job to HDFS.
>> -------------------------
>> sudo -u hdfs sqoop job --create jobname \
>> -- \
>> import \
>> --verbose \
>> -m 1 \
>> --connect jdbc:Cache://xx.xx.xx.xx:1972/USER \
>> --driver com.intersys.jdbc.CacheDriver \
>> --username username \
>> --password password \
>> --query "select * from tablename where \$CONDITIONS" \
>> --target-dir /user/hdfs/tablename \
>> --incremental append \
>> --check-column NUM \
>> --last-value 9
>> -------------------------
>>
>>
>> Is this a bug? Or did I miss anything else?  Thanks.
>>
>>
>> -Dongrong
>>
>>
>

Mime
View raw message