tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyunsik Choi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-88) Separate Tajo's cluster resource manager from Yarn (Umbrella)
Date Mon, 22 Jul 2013 15:34:49 GMT

    [ https://issues.apache.org/jira/browse/TAJO-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715294#comment-13715294
] 

Hyunsik Choi commented on TAJO-88:
----------------------------------

I'm sorry for frequently updating the issue title and contents.
                
> Separate Tajo's cluster resource manager from Yarn (Umbrella)
> -------------------------------------------------------------
>
>                 Key: TAJO-88
>                 URL: https://issues.apache.org/jira/browse/TAJO-88
>             Project: Tajo
>          Issue Type: Improvement
>          Components: master, worker
>            Reporter: Hyunsik Choi
>             Fix For: 0.2-incubating
>
>
> In the current implementation, the resource management of Tajo is tightly coupled to
Hadoop Yarn. Hadoop YARN has emerged and a nice resource manager for large clusters, but Yarn
is not the best choice for low latency query executions.
> The main purpose of this issue is to separate Tajo's resource manager from Yarn and add
a standby mode. In the standby mode, Tajo workers will be on standby, and they will provide
low-latency responses for query tasks.
> This issue is an umbrella issue, and our subtasks are as follows:
> * We will separate QueryMaster from TajoMaster.
> * We will provide a resource manager interface for Yarn and Tajo's one.
> * We will implement our own resource manager and worker containers for the standby mode.
> Finally, we will provide two modes:
>  (1) On-demand mode (using Yarn): in this mode,  Tajo shares the cluster resource with
another Yarn applications. Per query, Tajo will request necessary cluster resources.
>  (2) Standby mode: in this mode, Tajo master and workers will be on standby.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message