sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexandre Normand" <alexandre.norm...@gmail.com>
Subject Re: Review Request 14240: SQOOP-1032: Add the --bulk-load-dir option to support the HBase doBulkLoad function
Date Thu, 03 Oct 2013 16:54:11 GMT


> On Oct. 1, 2013, 7:49 p.m., Jarek Cecho wrote:
> > Thank you Alexandre for taking up this ticket!
> > 
> > Would you mind adding automated tests to ensure that the functionality is indeed
working?

I'd like to be able to integration test bulk load but this is still not a possibility in non-distributed
mode. We currently have the same problems in some of our map/reduce jobs that use bulk loading.
The way we've approached it with those is that we have two modes for each job: bulk load enabled/bulk
load disabled. The integration tests always run with bulk load disabled. The "bulk load mode"
is tested on a real cluster with automated tests that run from jenkins. 

For what it's worth, I've done the manual testing on our cluster for this patch. 


- Alexandre


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14240/#review26576
-----------------------------------------------------------


On Oct. 2, 2013, 9:21 p.m., Alexandre Normand wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14240/
> -----------------------------------------------------------
> 
> (Updated Oct. 2, 2013, 9:21 p.m.)
> 
> 
> Review request for Sqoop, Jarek Cecho, Jean-Marc Spaggiari, and vasanthkumar.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> This is the patch to address some of the comments on Zhancheng Deng's review request
(https://reviews.apache.org/r/13052/) which was basically the change to add the --bulk-load-dir
option to support the HBase doBulkLoad function.
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/SqoopOptions.java 01805f9 
>   src/java/org/apache/sqoop/hbase/HBasePutProcessor.java 9ceb5bd 
>   src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 5ccf311 
>   src/java/org/apache/sqoop/manager/SqlManager.java 2a4992d 
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java PRE-CREATION 
>   src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java PRE-CREATION 
>   src/java/org/apache/sqoop/mapreduce/ImportJobBase.java ab7f21e 
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 0eca991 
> 
> Diff: https://reviews.apache.org/r/14240/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Alexandre Normand
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message