hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anu Engineer <aengin...@hortonworks.com>
Subject Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk
Date Fri, 20 Oct 2017 15:19:03 GMT
Hi Steve,

In addition to everything Weiwei mentioned (chapter 3 of user guide), if you really want to
drill down to REST protocol you might want to apply this patch and build ozone.


This will generate an Open API (https://www.openapis.org , http://swagger.io) based specification
which can be accessed from KSM UI or just as a json file.
Unfortunately, this patch is still at code review stage, so you will have to apply the patch
and build it yourself. 


On 10/20/17, 6:09 AM, "Yang Weiwei" <cheersyang@hotmail.com> wrote:

    Hi Steve
    The code is available in HDFS-7240 feature branch, public git repo here<https://github.com/apache/hadoop/tree/HDFS-7240>.
    I am not sure if there is a "public" API for object stores, but the design doc<https://issues.apache.org/jira/secure/attachment/12799549/ozone_user_v0.pdf>
uses most common syntax so I believe it should be compliance. You can find the rest API doc
(with some example usages), and commandline API here<https://github.com/apache/hadoop/blob/HDFS-7240/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/OzoneCommandShell.md>.
    Look forward for your feedback!
    发件人: Steve Loughran <stevel@hortonworks.com>
    发送时间: 2017年10月20日 11:49
    收件人: Yang Weiwei
    抄送: hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
    主题: Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk
    Wow, big piece of work
    1. Where is a PR/branch on github with rendered docs for us to look at?
    2. Have you made any public APi changes related to object stores? That's probably something
I'll have opinions on more than implementation details.
    > On 19 Oct 2017, at 02:54, Yang Weiwei <cheersyang@hotmail.com> wrote:
    > Hello everyone,
    > I would like to start this thread to discuss merging Ozone (HDFS-7240) to trunk.
This feature implements an object store which can co-exist with HDFS. Ozone is disabled by
default. We have tested Ozone with cluster sizes varying from 1 to 100 data nodes.
    > The merge payload includes the following:
    >  1.  All services, management scripts
    >  2.  Object store APIs, exposed via both REST and RPC
    >  3.  Master service UIs, command line interfaces
    >  4.  Pluggable pipeline Integration
    >  5.  Ozone File System (Hadoop compatible file system implementation, passes all
FileSystem contract tests)
    >  6.  Corona - a load generator for Ozone.
    >  7.  Essential documentation added to Hadoop site.
    >  8.  Version specific Ozone Documentation, accessible via service UI.
    >  9.  Docker support for ozone, which enables faster development cycles.
    > To build Ozone and run ozone using docker, please follow instructions in this wiki
page. https://cwiki.apache.org/confluence/display/HADOOP/Dev+cluster+with+docker.
    Dev cluster with docker - Hadoop - Apache Software Foundation<https://cwiki.apache.org/confluence/display/HADOOP/Dev+cluster+with+docker>
    First, it uses a much more smaller common image which doesn't contains Hadoop. Second,
the real Hadoop should be built from the source and the dist director should be ...
    > We have built a passionate and diverse community to drive this feature development.
As a team, we have achieved significant progress in past 3 years since first JIRA for HDFS-7240
was opened on Oct 2014. So far, we have resolved almost 400 JIRAs by 20+ contributors/committers
from different countries and affiliations. We also want to thank the large number of community
members who were supportive of our efforts and contributed ideas and participated in the design
of ozone.
    > Please share your thoughts, thanks!
    > -- Weiwei Yang

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message