kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hongbin ma (JIRA)" <>
Subject [jira] [Updated] (KYLIN-745) Generic Data Reader
Date Wed, 13 Jan 2016 05:11:39 GMT


hongbin ma updated KYLIN-745:
    Assignee: Dong Li  (was: ZhouQianhao)

> Generic Data Reader
> -------------------
>                 Key: KYLIN-745
>                 URL:
>             Project: Kylin
>          Issue Type: Wish
>          Components: Job Engine, Spark Engine
>            Reporter: Luke Han
>            Assignee: Dong Li
> When data be stored on existing DW like Oracle, it's not be able to read directly through
Kylin to build cube. 
> There are many requirements coming from different teams like Candor about this.
> There are two options:
> #1, copy your data to Hive and then build cube through Kylin. There are some cases are
running this model to bring data into Hive from DW and leveraging Kylin very well. 
> #2, rewrite  data read module to pull data from Oracle directly. Actually, the first
step of cube build is generate Hive Query to read data and generate one temp table in Hive,
so it should be not too complicated to do this (but it depends network and others, otherwise,
#1 will be more efficient one). Then process cube build as normal. Using generical reader
to read data from any SQL rdbms through JDBC or other protocol will be perfect solution since
cube could be built without ETL process
> Scope:
> Only ready data directly from existing RDBMs and store jointed result in Hive (temp table)
for further processing, no any other transfomation. 
> By design, Kylin is OLAP system not ETL one.

This message was sent by Atlassian JIRA

View raw message