hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugeny Morozov <emoro...@griddynamics.com>
Subject Re: Does TotalOrderPartitioner refresh its partitions selection tree
Date Tue, 09 Oct 2012 07:45:23 GMT

In this case nothing scared actually happens.

* If partitions are the same, then HBase simply copies all your HFiles
during bulkloading procedure.
* If partitions are changed, then it still copies them, but in addition,
some of these files (according to number of split regions) would be also
split and the whole procedure would be slightly longer.

To understand the idea, imagine you have only one HFile, but  some regions
in HBase.

On Mon, Oct 8, 2012 at 9:33 PM, chris Behin <chrisbehin01@gmail.com> wrote:

> Hil all,
> I am reading TotalOrderPartitioner code to understand its structure,
> As I understood ( correct me if I am wrong ) TotalOrderPartitioner.setConf
> method reads splitPoints from partFile via readPartitions method
> ( TotalOrderPartitioner.java line 82 )
> and creates a buildTrie or BinarySearchNode according to this partition
> information ( TotalOrderPartitioner.java line 97 and 108 )
> I wonder what happens if regions are changed during the execution of
> mapreduce job. Is it a problem?
> Because TotalOrderPartitioner does not refresh its partitioner during its
> execution ( as I understand ).
> You may say that this is an exception case and the possibility is low,
> I just ask for understanding the mechanism..
> best regards...

Evgeny Morozov
Developer Grid Dynamics
Skype: morozov.evgeny

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message