helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Brandt (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HELIX-470) Add performant IPC (Helix actors)
Date Fri, 11 Jul 2014 17:52:05 GMT
Greg Brandt created HELIX-470:

             Summary: Add performant IPC (Helix actors)
                 Key: HELIX-470
                 URL: https://issues.apache.org/jira/browse/HELIX-470
             Project: Apache Helix
          Issue Type: Improvement
          Components: helix-core
    Affects Versions: 0.7.1, 0.6.4
            Reporter: Greg Brandt

Helix is missing a high-performance way to exchange messages among resource partitions, with
a user-friendly API.

Currently, the Helix messaging service relies on creating many nodes in ZooKeeper, which can
lead to ZooKeeper outages if messages are sent too frequently.

In order to avoid this, high-performance NIO-based {{HelixActors}} should be implemented (in
rough accordance with the actor model). {{HelixActors}} exchange messages asynchronously without
waiting for a response, and are partition/state-addressable.

The API would look something like this:

public interface HelixActor<T> {
    void send(Partition partition, String state, T message);
    void register(String resource, HelixActorCallback<T> callback);

public interface HelixActorCallback<T> {
    void onMessage(Partition partition, State state, T message);

{{#send}} should likely support wildcards for partition number and state, or its method signature
might need to be massaged a little bit for more flexibility. But that's the basic idea.

Nothing is inferred about the format of the messages - the only metadata we need to be able
to interpret is (1) partition name and (2) state. The user provides a codec to encode / decode
messages, so it's nicer to implement {{HelixActor#send}} and {{HelixActorCallback#onMessage}}.

public interface HelixActorMessageCodec<T> {
    byte[] encode(T message);
    T decode(byte[] message);

Actors should support somewhere around 100k to 1M messages per second. The Netty framework
is a potential implementation candidate, but should be thoroughly evaluated w.r.t. performance.

This message was sent by Atlassian JIRA

View raw message