sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <gshap...@cloudera.com>
Subject Re: SQOOP customize Generated Class and customize flow
Date Thu, 09 Apr 2015 17:54:02 GMT
It looks like Sqoop2 may be a good fit for you.
It allows you to write your own connector - so you can basically copy
the HDFS connector, add your custom code to transform the data and use
your new connector instead.

On Thu, Apr 9, 2015 at 10:49 AM, KayVajj <vajjalak009@gmail.com> wrote:
> Hi,
> I have a question if we can override some methods in the QueryResult.java in
> the sqoop generated Java file and use it during the import to customize a
> few things. What I'm exactly trying to do is some clean up and
> transformations before I write it to HDFS. I feel if this is possible it
> will efficient rather doing it after the fact as we have to do expensive
> reads from HDFS.
> Also I wanted to see if we can plugin some code in the process of writing it
> to HDFS. Lets say If I want to write the record to some other store the only
> way I see write another MR job which reads the files written by SQOOP and
> process it the way I need.
> I really appreciate your inputs.
> Thanks
> Kay

View raw message