lens-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "amareshwarisr ." <amareshw...@gmail.com>
Subject Re: Cube Creation for Hive Tables
Date Tue, 16 Jun 2015 12:45:43 GMT
Srinivasan Hariharan,

I will try to explain how city table is created in examples. Let me see if
that helps you?

If you see
https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city.xml
:

----

<x_dimension name="city" xmlns="uri:lens:cube:0.1" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
  <attributes>
    <dim_attribute name="id" type="INT"/>
    <dim_attribute name="name" type="STRING"/>
    <dim_attribute name="POI" type="ARRAY&lt;STRING&gt;" description="Point
of interests"/>
    <dim_attribute name="population" type="BIGINT" />
  </attributes>
  <properties>
    <property name="dimension.city.timed.dimension" value="dt"/>
  </properties>
</x_dimension>
----

Above of definition of city defines dimension city and its attribute. This
is only logical table, not linked to any physical table.

Then we have
https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city_table.xml
- Which defines the dimtable associated with dimension city

----
<x_dimension_table dimension_name="city" table_name="city_table"
weight="10.0" xmlns="uri:lens:cube:0.1" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
  <columns>
    <column comment="ID" name="id" type="INT"/>
    <column comment="name" name="name" type="STRING"/>
    <column comment="Point of interests" name="POI"
type="ARRAY&lt;STRING&gt;"/>
    <column comment="city population" name="population" type="BIGINT"/>
  </columns>
  <properties>
    <property name="city.prop" value="d1"/>
  </properties>
  <storage_tables>
    <storage_table>
      <update_periods>
        <update_period>HOURLY</update_period>
      </update_periods>
      <storage_name>local</storage_name>
      <table_desc external="true" field_delimiter=","
collection_delimiter=":" table_location="/tmp/examples/city">
        <part_cols>
          <column comment="Time column" name="dt" type="STRING"/>
        </part_cols>
        <time_part_cols>dt</time_part_cols>
      </table_desc>
    </storage_table>
  </storage_tables>
</x_dimension_table>
----

Here the above definition will be mapped to table on HDFS - which is Text
formatted with delimiters specified above.

Are you looking for something which uses existing tables to create facts
and dimensions - https://issues.apache.org/jira/browse/LENS-340 ? Right
now, that feature is not there. Until then you have to map lens created
table as external table to your existing table.

For ex:
You have Hive table Table1 : col1, col2;

Create Dimension1 : col1, col2 as dim attributes - give schema for them.
Create DimTable1 : col1, col2 associated with Dimension1.

Then you should be good.

Let us know if above answers your query. Or let us know if you are looking
for something else.

Thanks
Amareshwari



On Tue, Jun 16, 2015 at 1:02 PM, Srinivasan Hariharan02 <
Srinivasan_H02@infosys.com> wrote:

>  Hi,
>
>
>
> I have all my Dim tables in Hive, I want to create cube on top of these
> tables using lens. Can anyone provide how ca I start doing that. I have
> checked the examples in the resources directory under client folder. It
> doesn’t help much for creation of storage dim, facts from hive tables. Help
> is much appreciated.
>
>
>
> *Regards,*
>
> *Srinivasan Hariharan*
>
> *Mob +91-9940395830 <%2B91-9940395830>*
>
>
>

Mime
View raw message