helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HELIX-470) Add performant IPC (Helix actors)
Date Tue, 26 Aug 2014 20:15:58 GMT

    [ https://issues.apache.org/jira/browse/HELIX-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111240#comment-14111240

ASF GitHub Bot commented on HELIX-470:

GitHub user brandtg opened a pull request:


    [HELIX-470] Netty-based IPC layer


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/brandtg/helix master

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2
commit f2475fa9a6123052fea2588cdd4e439ddc7af020
Author: Greg Brandt <brandt.greg@gmail.com>
Date:   2014-08-26T20:14:36Z

    [HELIX-470] Netty-based IPC layer


> Add performant IPC (Helix actors)
> ---------------------------------
>                 Key: HELIX-470
>                 URL: https://issues.apache.org/jira/browse/HELIX-470
>             Project: Apache Helix
>          Issue Type: Improvement
>          Components: helix-core
>    Affects Versions: 0.7.1, 0.6.4
>            Reporter: Greg Brandt
> Helix is missing a high-performance way to exchange messages among resource partitions,
with a user-friendly API.
> Currently, the Helix messaging service relies on creating many nodes in ZooKeeper, which
can lead to ZooKeeper outages if messages are sent too frequently.
> In order to avoid this, high-performance NIO-based {{HelixActors}} should be implemented
(in rough accordance with the actor model). {{HelixActors}} exchange messages asynchronously
without waiting for a response, and are partition/state-addressable.
> The API would look something like this:
> {code}
> public interface HelixActor<T> {
>     void send(Partition partition, String state, T message);
>     void register(String resource, HelixActorCallback<T> callback);
> }
> public interface HelixActorCallback<T> {
>     void onMessage(Partition partition, State state, T message);
> }
> {code}
> {{#send}} should likely support wildcards for partition number and state, or its method
signature might need to be massaged a little bit for more flexibility. But that's the basic
> Nothing is inferred about the format of the messages - the only metadata we need to be
able to interpret is (1) partition name and (2) state. The user provides a codec to encode
/ decode messages, so it's nicer to implement {{HelixActor#send}} and {{HelixActorCallback#onMessage}}.
> {code}
> public interface HelixActorMessageCodec<T> {
>     byte[] encode(T message);
>     T decode(byte[] message);
> }
> {code}
> Actors should support somewhere around 100k to 1M messages per second. The Netty framework
is a potential implementation candidate, but should be thoroughly evaluated w.r.t. performance.

This message was sent by Atlassian JIRA

View raw message