sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jilani Shaik <jilani2...@gmail.com>
Subject Re: Review Request 57499: Sqoop incremental import - NULL column updates are not pulled into HBase table
Date Mon, 27 Mar 2017 09:53:20 GMT


> On March 18, 2017, 11:06 a.m., Boglarka Egyed wrote:
> > Hi Jilani,
> > 
> > Thanks for your contribution!
> > 
> > I have applied your patch and got some warnings because of the incorrect indentation
and trailing whitespaces which I mentioned in my review below too, please find my further
findings there too. I could run 'ant clean test' susseccfully with your patch.
> > 
> > On the top of the findings, could you please add a test case for your change?
> > 
> > Thanks,
> > Bogi
> 
> Boglarka Egyed wrote:
>     Hi Jilani,
>     
>     I think you could add test(s) to com.cloudera.sqoop.hbase.HBaseImportNullTest class
as it tests the null import into HBase cases.
>     
>     Thanks,
>     Bogi
> 
> Jilani Shaik wrote:
>     Hi Bogi,
>     
>     I updated unit test and code chnages based on below suggestions. But unit test in
my local not able to validate due to HBase mini cluster init error from command line in windows.
details of the error I have sent in mail along with error file. I am hoping that it may work
for you to validate. 
>     
>     Please let me know what I am missing, so that I can run the unit test in local.
>     
>     Thanks,
>     Jilani
> 
> Boglarka Egyed wrote:
>     Hi Jilani,
>     
>     Could you please upload the latest version of your patch to the review board (via
Update Diff button)? Your review request says "Latest diff uploaded 2 weeks, 2 days ago".
>     
>     Also, Sqoop is predominantly operated and tested on Linux which means it's behavior
is not so deterministic on Windows. I would recommend to use Linux based VM for testing or
maybe someone else could have other suggestions who is working on Windows too.
>     
>     Thanks,
>     Bogi

Hi Bogi,

I have moved the project from Windows to MacOS, Some unit tests are still failing there, atleast
the HBase unit tests are passed, where my modifications are there. So I validated my unit
tests to support Null update.

Can I modify the PutTransformer to MutationTransformer, entire java class name? Is it acceptable
to modify the class name and resppective changes?

Thanks,
Jilani


- Jilani


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57499/#review169361
-----------------------------------------------------------


On March 27, 2017, 9:42 a.m., Jilani Shaik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57499/
> -----------------------------------------------------------
> 
> (Updated March 27, 2017, 9:42 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3149
>     https://issues.apache.org/jira/browse/SQOOP-3149
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> HBase delete is added as part of incremental data import, Modified the return type of
method which is responsible for holding list of Put objects to List of Mutation objects and
at the time of execution of insert or delete using the type of object in Mutation, executed
the actaul operation either insert or delete.
> 
> Jira ticket for the same: https://issues.apache.org/jira/browse/SQOOP-3149
> 
> Similar ticket to above:  SQOOP-3125
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java fdbe1276 
>   src/java/org/apache/sqoop/hbase/PutTransformer.java 533467e5 
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 363e1456 
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java 58ccee7b 
>   src/test/com/cloudera/sqoop/hbase/HBaseImportTest.java fa14a013 
>   src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java a054eb66 
>   src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java 6310a398 
> 
> 
> Diff: https://reviews.apache.org/r/57499/diff/2/
> 
> 
> Testing
> -------
> 
> Executed with jar prepared with these changes into hadoop cluster and tested bot import
and then incremental import.
> 
> 
> File Attachments
> ----------------
> 
> HBase delete support for incremental import
>   https://reviews.apache.org/media/uploaded/files/2017/03/11/5b1895fd-4c6b-42fa-8a92-4e093153e370__hbase_delete_support_in_incremental_import
> updated with unit test and based on below suggestions
>   https://reviews.apache.org/media/uploaded/files/2017/03/22/708f63ba-2d8a-4a47-ab67-c1d2776354fd__hbase_delete_support_in_incremental_unit_test_also
> 
> 
> Thanks,
> 
> Jilani Shaik
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message