incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: S4 Load Balancing Mechanism
Date Fri, 12 Oct 2012 11:42:28 GMT
On 10/12/12 3:36 AM, Frank Zheng wrote:
>
> Besides, can zeno prototype do dynamic load balancing?

Yes that's the idea. It's not integrated with S4 yet though. But it is 
definitely a nice approach to the problem.

Matthieu

>
> Thanks.
>
> On Thu, Oct 11, 2012 at 7:00 PM, Matthieu Morel <mmorel@apache.org
> <mailto:mmorel@apache.org>> wrote:
>
>     On 10/11/12 12:35 PM, Daniel Gómez Ferro wrote:
>
>
>
>         On Thu Oct 11 05:52:01 2012, Frank Zheng wrote:
>
>             Hi All,
>
>             I want to figure out the mechanism of load balancing in S4.
>             In the paper and web site, it says ZooKeeper takes this
>             responsibility, but the description is not very clear.
>             I want to figure out that how ZooKeeper does load balancing.
>             For example, if there are two working PE nodes, and 120
>             messages per
>             second. The adapter distributes these messages evenly, 60
>             per PE node,
>             right?
>             When one PE node is added into the cluster, does it mean
>             ZooKeeper
>             balances the messages, 40 per PE node?
>             And how does ZooKeeper decide which message to be
>             distributed to which
>             node?
>
>
>         Hi,
>
>         Currently load balancing is static. When you define your cluster you
>         specify the number of tasks (say, 2) which won't change for this
>         cluster. If you have more than 2 nodes, the rest will be backup
>         nodes.
>         In case one of the serving nodes fails, a backup node will pick
>         up the
>         orphaned task and start serving requests, but the load will
>         always be
>         spread among the 2 tasks that you defined at the beginning.
>
>         Having dynamic load balancing would be really cool, but I don't
>         know if
>         there are currently any efforts in that direction.
>
>
>     There is the zeno prototype https://github.com/s4/zeno , which is
>     actually independent of S4. It also relies on ZooKeeper for metadata
>     about partitions.
>
>     Matthieu
>
>
>
>
> --
> Sincerely,
> Zheng Yu
> Mobile:  (852) 60670059
> Email: bearzheng2011@gmail.com <mailto:bearzheng2011@gmail.com>
>
>
>


Mime
View raw message