kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Long (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3810) Kylin service nodes discovery with Zookeeper
Date Tue, 26 Feb 2019 02:52:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777515#comment-16777515
] 

Chao Long commented on KYLIN-3810:
----------------------------------

Recently, I added a curator-based scheduler in kylin and I found some work duplicate with
yours. In my work, I use the curator-x-discovery component to implement kylin service auto
discovery and failover and implement the automatic election and monitoring of the job node
leader through zk. So, I want to discuss with you guys which realization is better.:D KYLIN-3820

> Kylin service nodes discovery with Zookeeper 
> ---------------------------------------------
>
>                 Key: KYLIN-3810
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3810
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Metadata, Others
>    Affects Versions: v2.3.1
>            Reporter: zhangwei
>            Assignee: zhangwei
>            Priority: Minor
>         Attachments: zookeeperCacheSync.patch, 同步阻塞.png, 引入zookeeper.png
>
>
> 由于Kylin的元数据同步使用了Boardcast类中的线程池,所以一旦负载均衡查询节点中一台机器岩机,Kylin的同步请求线程会堵塞在岩机的查询节点,最后将线程资源全部占满,如图同步到1,2节点的请求堵塞在线程队列中导致查询节点获取不到最新元数据从而无法查询到实时构建好的数据
> 我们的方案:
> 引进了服务发现组建zookeeper



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message