sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kunal Sachdeva <kunal...@gmail.com>
Subject How to invoke extractor during IMPORT Job Submition process- Sqoop version 1.99.1 Connector
Date Wed, 09 Oct 2013 13:01:22 GMT

*Subject: Want to invoke custom extractor class run method for printing
data- MySql table's columns values data in Import job*.

I am newby to Sqoop2 (*Currently working on connector version 1.99.1*) and
want some help from you. I have developed Sqoop2 connector similar to
generic connector 1.99.1 provided by apache, here

In other words my connector is 99% similar to generic connector 1.99.1 but
have different name for packages and classes.I have tested with my own
connector  Import and Export both are working fine. But I want to print
data during Import job submission and before it writes data to
given/specific file, I mean when we import MySql Database table and submit
the Import job then whatever data Sqoop2 is importing from that table, I
want all rows,columns to be appended with my name or some other value 'abc'
through connector.

I know, Each connector in Sqoop2 extends class SqoopConnector by providing
classes that should be called. If I need to run custom extractor, I will
need to create custom connector that will have different extractor. On the
client side, I will have to create connection for your new connector
instead of the build-in Generic JDBC Connector.

I have already done these above steps and My connector is working fine for
both Import,Export jobs and I have seen result in desired location.

But, as said in Import job, to append anything in table rows data before
writing them to any file , we have to code in extractor. My extractor is
100% same as generic connector extractor. But when and how to invoke this
custom conector extractor is the main thing which I do not know and I want
to invoke this.

Please help me which class and method I need to add/modify so that I can
get invoke my connector extractor.

I have followed all steps mentioned here
http://sqoop.apache.org/docs/1.99.1/ but invoking extractor is hard thing
for me. I do not know which value I have to give in command prompt and all.

Thanks in advance.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message