incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aimee Cheng <chengsj....@gmail.com>
Subject Re: S4 Load Balancing Mechanism
Date Fri, 12 Oct 2012 01:55:19 GMT
I can answer your second question: no. In current S4 piper, when there are no standby nodes,
if one node crushes, the remaining nodes will not pick up the task and still focus on their
own tasks.

-Aimee
 
On Oct 12, 2012, at 9:36 AM, Frank Zheng wrote:

> Hi,
> 
> Thanks for your reply.
> So the dynamic load balancing is not implemented yet, right?
> In case of static load balancing, if there are two working PE node without any standby
nodes, 
> when one fails, does the workload for the remaining working PE increase?
> 
> Besides, can zeno prototype do dynamic load balancing? 
> 
> Thanks.
> 
> On Thu, Oct 11, 2012 at 7:00 PM, Matthieu Morel <mmorel@apache.org> wrote:
> On 10/11/12 12:35 PM, Daniel Gómez Ferro wrote:
> 
> 
> On Thu Oct 11 05:52:01 2012, Frank Zheng wrote:
> Hi All,
> 
> I want to figure out the mechanism of load balancing in S4.
> In the paper and web site, it says ZooKeeper takes this
> responsibility, but the description is not very clear.
> I want to figure out that how ZooKeeper does load balancing.
> For example, if there are two working PE nodes, and 120 messages per
> second. The adapter distributes these messages evenly, 60 per PE node,
> right?
> When one PE node is added into the cluster, does it mean ZooKeeper
> balances the messages, 40 per PE node?
> And how does ZooKeeper decide which message to be distributed to which
> node?
> 
> Hi,
> 
> Currently load balancing is static. When you define your cluster you
> specify the number of tasks (say, 2) which won't change for this
> cluster. If you have more than 2 nodes, the rest will be backup nodes.
> In case one of the serving nodes fails, a backup node will pick up the
> orphaned task and start serving requests, but the load will always be
> spread among the 2 tasks that you defined at the beginning.
> 
> Having dynamic load balancing would be really cool, but I don't know if
> there are currently any efforts in that direction.
> 
> There is the zeno prototype https://github.com/s4/zeno , which is actually independent
of S4. It also relies on ZooKeeper for metadata about partitions.
> 
> Matthieu
> 
> 
> 
> -- 
> Sincerely,
> Zheng Yu
> Mobile:  (852) 60670059
> Email:    bearzheng2011@gmail.com
> 
> 
> 


Mime
View raw message