sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pratik khadloya <tispra...@gmail.com>
Subject Re: Java API to Import Tables from RDBMs to Hive using Sqoop
Date Thu, 25 Sep 2014 01:06:47 GMT
You can access the incremental mode property in SqoopOptions class by
calling setIncrementalMode(IncrementalMode mode).

  public enum IncrementalMode {
    None,
    AppendRows,
    DateLastModified,
  }

Also you can set the last value using setIncrementalLastValue(String
lastVal)


~Pratik

On Wed, Sep 24, 2014 at 5:50 AM, Radhe Radhe <radhe.krishna.radhe@live.com>
wrote:

> Hi Pratik,
>
> I had a look at the below references.
>
> *#1* uses SqoopOptions and finally invoking Sqoop.runTool() method in
> Java Program.
> This is since Sqoop 1.x does not have an official Java API.
>
> I this is a viable solution for performing an increment update using
> sqoop ?
>
> *#2* uses Sqoop 2 Client Java API and is not currently ready for
> production deployment.
> Latest cut of Sqoop2 is 1.99.3 (download
> <http://www.apache.org/dyn/closer.cgi/sqoop/1.99.3>, documentation
> <http://sqoop.apache.org/docs/1.99.3/index.html>). Note that 1.99.3 is
> not compatible with 1.4.5 and not feature complete, it is not intended
> for production deployment.
> I believe that leaves me to use Sqoop 1.x ONLY.
>
>
> Basically I want to do an increment update using sqoop through Java
> Program.
>
> I was thinking of invoking the entire bin/sqoop import commands with
> increment mode options from Java code using a ProcessBuilder is a correct
> option.
>
> Or is there is a better way of doing that programatically ?
>
>
> Thanks,
> -RR
>
> ------------------------------
> From: radhe.krishna.radhe@live.com
> To: user@sqoop.apache.org
> Subject: RE: Java API to Import Tables from RDBMs to Hive using Sqoop
> Date: Tue, 16 Sep 2014 10:56:48 +0530
>
> Thanks Pratik,
>
> I will have a look into.
>
> -RR
>
> ------------------------------
> From: tispratik@gmail.com
> Date: Fri, 12 Sep 2014 10:14:08 -0700
> Subject: Re: Java API to Import Tables from RDBMs to Hive using Sqoop
> To: user@sqoop.apache.org
>
> This may help you
> http://stackoverflow.com/questions/9229611/how-to-use-sqoop-in-java-program
> http://devslogics.blogspot.com/2013/09/sqoop-java-client.html
>
> ~Pratik
>
> On Thu, Sep 11, 2014 at 7:47 PM, Radhe Radhe <radhe.krishna.radhe@live.com
> > wrote:
>
> Hello everyone,
>
> As per my experience till now I have used the  direct *sqoop commands* to
> import Tables from RDBMs to Hive
> The commands itself has a lot more switches and options.
>
> Is there an easy way to use JAVA APIs for importing Tables using Sqoop.
>
> Need suggestions if it's possible and viable to use a JAVA Program to
> import tables from RDBMs (*dynamically based on the source Tables
> Metadata*) to Hive using Sqoop
> Thanks,
> -RR
>
>
>

Mime
View raw message