kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhong Yanghong (JIRA)" <>
Subject [jira] [Issue Comment Deleted] (KYLIN-1082) Hive dependencies should be add to tmpjars
Date Thu, 28 Jan 2016 05:38:39 GMT


Zhong Yanghong updated KYLIN-1082:
    Comment: was deleted

(was: Maybe the previous clause is not good. For example, in the Class "CubeHFileJob", we
imported some classes related to hbase, 
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.mapreduce.HFileOutputFormat;
import org.apache.hadoop.hbase.mapreduce.KeyValueSortReducer;

They are responsible for convert Cuboid to HBase. However, they have been included in the
hadoop-common jars.)

> Hive dependencies should be add to tmpjars
> ------------------------------------------
>                 Key: KYLIN-1082
>                 URL:
>             Project: Kylin
>          Issue Type: Bug
>          Components: Environment , Job Engine
>            Reporter: liyang
>            Assignee: Zhong Yanghong
>              Labels: newbie
>             Fix For: v2.1, v1.3
>         Attachments: auto_hive_tmpjars_1_x_staging.patch, auto_hive_tmpjars_2_x_staging.patch
> Currently kylin assume all data nodes have hive deployment at exact same FS location.
However, a better position is to think hive as a client side app. Then we need to ship hive
jar with MR job every time.
> This make deploy kylin a lot easier in cluster that does not have hive on all data nodes.

This message was sent by Atlassian JIRA

View raw message