kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hongbin ma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KYLIN-1094) improve performance of spark cubing
Date Wed, 13 Jan 2016 05:10:39 GMT

     [ https://issues.apache.org/jira/browse/KYLIN-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

hongbin ma updated KYLIN-1094:
------------------------------
    Assignee: Dong Li  (was: ZhouQianhao)

> improve performance of spark cubing
> -----------------------------------
>
>                 Key: KYLIN-1094
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1094
>             Project: Kylin
>          Issue Type: Sub-task
>          Components: Spark Engine
>    Affects Versions: v2.0
>            Reporter: ZhouQianhao
>            Assignee: Dong Li
>             Fix For: v2.1
>
>
> POC result of spark cubing shows that, on a dataset of 150 million records, MR is about
100% faster than Spark, however we believe that Spark could be at least at same speed as MR,
so optimization is needed here.
> We are asking Spark community for help now.
> the cluster info:
> vm: 8 nodes * (128G mem + 64 core)
> hadoop cluster: hdp 2.2.6
> spark running mode: yarn-client
> spark version: 1.5.1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message