hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10641) Introduce Coordination Engine interface
Date Tue, 26 Aug 2014 00:40:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110070#comment-14110070
] 

Konstantin Shvachko commented on HADOOP-10641:
----------------------------------------------

Steve, thanks for thorough code review.
The question about "what's next" has been discussed in this jira (see Aaron's and others comments),
on the meetup on July 15 and in person. The decision everybody agreed on is to continue the
CE and CNode development on a branch.
- I would be glad to commit it to hadoop-common as you propose. That way we can see faster
adoption.
- I don't think the interface should be tied to ZooKeeper itself or higher level projects
on top of it like Curator. Because CE is intended to be have implementations for different
consensus algorithms.

> Introduce Coordination Engine interface
> ---------------------------------------
>
>                 Key: HADOOP-10641
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10641
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch,
HADOOP-10641.patch, NNThroughputBenchmark Results.pdf, ce-tla.zip, hadoop-coordination.patch,
zkCEBenchmark.pdf, zkbench.pdf
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of events in
a distributed system. In order to be reliable CE should be distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and
have different implementations, depending on use cases, reliability, availability, and performance
requirements.
> CE should have a common API, so that it could serve as a pluggable component in different
projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message