sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anna Szonyi <szo...@cloudera.com>
Subject Re: Review Request 57499: Sqoop incremental import - NULL column updates are not pulled into HBase table
Date Thu, 13 Jul 2017 17:14:30 GMT


> On July 11, 2017, 6:52 p.m., Boglarka Egyed wrote:
> > Hi Jilani,
> > 
> > Thanks for the patch updates, I have found one last minor problem with your change,
please fix it.
> > Otherwise it looks good to me.
> > 
> > Thanks for your contribution!
> > 
> > Cheers,
> > Bogi
> 
> Jilani Shaik wrote:
>     Just checking, can we add this to upcoming release i.e 1.4.7?
>     
>     Thanks,
>     Jilani
> 
> Boglarka Egyed wrote:
>     Hi Jilani,
>     
>     Could you please go directly to the 1.4.7 release tracking JIRA with your request?
>     Link: https://issues.apache.org/jira/browse/SQOOP-3199
>     
>     Thanks,
>     Bogi

Hi Jilani,

I'm not sure if it should make it into 1.4.7, as while this corrects a behaviour to what would
be your natural expectation for the feature, it also changes the current behaviour. So if
there are customers relying on the current (incorrect?) behaviour their jobs will start failing.

I think the best way of going about it would be:
Bring this up on the dev@ list, so others can chime in with their opinion on the matter and
a) If the dev list agrees we should add it to 1.4.7 we can potentially add a flag for turning
this on/off and add it to 1.4.7.
b) If the dev list agrees that this is the correct behaviour, we might add it to a bigger
upcoming release as a "breaking change" and document this.

Regards,
Anna


- Anna


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57499/#review180233
-----------------------------------------------------------


On July 12, 2017, 5:20 a.m., Jilani Shaik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57499/
> -----------------------------------------------------------
> 
> (Updated July 12, 2017, 5:20 a.m.)
> 
> 
> Review request for Sqoop, Boglarka Egyed, Attila Szabo, and Anna Szonyi.
> 
> 
> Bugs: SQOOP-3149
>     https://issues.apache.org/jira/browse/SQOOP-3149
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> HBase delete is added as part of incremental data import, Modified the return type of
method which is responsible for holding list of Put objects to List of Mutation objects and
at the time of execution of insert or delete using the type of object in Mutation, executed
the actaul operation either insert or delete.
> 
> Jira ticket for the same: https://issues.apache.org/jira/browse/SQOOP-3149
> 
> Similar ticket to above:  SQOOP-3125
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java fdbe1276 
>   src/java/org/apache/sqoop/hbase/PutTransformer.java 533467e5 
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 363e1456 
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java 58ccee7b 
>   src/test/com/cloudera/sqoop/hbase/HBaseImportTest.java fa14a013 
>   src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java a054eb66 
>   src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java 6310a398 
> 
> 
> Diff: https://reviews.apache.org/r/57499/diff/9/
> 
> 
> Testing
> -------
> 
> Executed with jar prepared with these changes into hadoop cluster and tested bot import
and then incremental import.
> 
> 
> File Attachments
> ----------------
> 
> HBase delete support for incremental import
>   https://reviews.apache.org/media/uploaded/files/2017/03/11/5b1895fd-4c6b-42fa-8a92-4e093153e370__hbase_delete_support_in_incremental_import
> updated with unit test and based on below suggestions
>   https://reviews.apache.org/media/uploaded/files/2017/03/22/708f63ba-2d8a-4a47-ab67-c1d2776354fd__hbase_delete_support_in_incremental_unit_test_also
> 
> 
> Thanks,
> 
> Jilani Shaik
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message