sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <vasas.szabo...@gmail.com>
Subject Re: Review Request 52426: row key not added into column family using --hbase-bulkload
Date Mon, 10 Oct 2016 13:47:28 GMT


> On Oct. 7, 2016, 1:26 p.m., Attila Szabo wrote:
> > Hi Szabi,
> > 
> > The general idea behind this solution seems to be correct. Also the solution seems
to be valid for the first shot.
> > However I was not able to run the HBaseImportAddRowKeyTest with your changes successfully
(it's hanging in an infinite state).
> > I also checked that test without your changes and seems to have the very same effect.
> > I suspect here the HBase version in the upstream ivy.xml is older than the version
you'd tested with, and thus I'd like to kindly suggest you to deliver a fixed version where
both the before/after states are running with a successful state.
> > I would like to ask you also to double check if the scenario with asking Zookeeper
to open an arbitrary port is right and sustainable.
> > 
> > Regardless these things, I'm pretty sure this modification will work, you just need
to invest a bit more on the testing/validation front.
> > 
> > Many thanks in advance!
> > Attila
> > 
> > p.s.: If you deliver a new verison, could you please also ensure that the order
of the dependencies are unchanged? Thanks!

Hi Attila,

Thank you for reviewing it! 
I have created a new profile with upgraded HBase and Hadoop versions, if you run the test
with this profile (ant clean test -Dtestcase=HBaseImportAddRowKeyTest -Dhadoopversion=260
-Dhbaseprofile=95) it runs successfully. As we discussed offline we should create another
JIRA for cleaning up these Hadoop profiles, shall I go ahead and create it?

I have checked the method which generates the random Zookeeper port and its documentation
says the following:


   The random port is selected from the
   range between 49152 to 65535. These ports cannot be registered with IANA
   and are intended for dynamic allocation (see http://bit.ly/dynports).

This seems to be pretty good solution to me, at least it is as good as the previous solution
was IMHO.


- Szabolcs


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52426/#review151787
-----------------------------------------------------------


On Oct. 10, 2016, 1:42 p.m., Szabolcs Vasas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52426/
> -----------------------------------------------------------
> 
> (Updated Oct. 10, 2016, 1:42 p.m.)
> 
> 
> Review request for Sqoop and Attila Szabo.
> 
> 
> Bugs: SQOOP-2952
>     https://issues.apache.org/jira/browse/SQOOP-2952
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> row key not added into column family using --hbase-bulkload
> 
> 
> Diffs
> -----
> 
>   build.xml 97e5502 
>   ivy.xml a502530 
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java b2431ac 
>   src/java/org/apache/sqoop/hbase/PutTransformer.java 8d6bcac 
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java b5cad1d 
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java 363b5d7 
>   src/test/com/cloudera/sqoop/hbase/HBaseImportAddRowKeyTest.java cfbb1d3 
>   src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 37dc004 
> 
> Diff: https://reviews.apache.org/r/52426/diff/
> 
> 
> Testing
> -------
> 
> New unit test cases are added.
> 
> HBaseImportAddRowKeyTest can be run with the following command:
> 
> ant clean test -Dtestcase=HBaseImportAddRowKeyTest -Dhadoopversion=260 -Dhbaseprofile=95
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message