lens-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Yan <t...@linkedin.com>
Subject Re: Dimension Table Divisions
Date Wed, 14 Sep 2016 17:03:32 GMT
Thanks!

On Wed, Sep 14, 2016 at 1:15 AM, amareshwarisr . <amareshwari@gmail.com>
wrote:

> Tao,
>
> hive view with two dimension joins may not work here, as there is no such
> support. It would work for other storages.
>
> Thanks
>
> On Thu, Sep 8, 2016 at 10:49 AM, Puneet Gupta <puneet.gupta@inmobi.com>
> wrote:
>
>> +Amareshwari
>>
>> Thanks,
>> Puneet Gupta
>>
>> On Thu, Sep 1, 2016 at 10:16 PM, Tao Yan <tyan@linkedin.com> wrote:
>>
>>> If the two division tables are on HDFS or in Hive and I create a hive
>>> view which joins two divisions, can I use the view as lens dimension table?
>>>
>>> Thanks!
>>>
>>> On Thu, Sep 1, 2016 at 12:59 AM, Puneet Gupta <puneet.gupta@inmobi.com>
>>> wrote:
>>>
>>>> We currently don't see the need to support dimension table sub
>>>> divisions.  Thats why this jira is parked since a long time.
>>>>
>>>> You can do the following
>>>> 1. Create two dimensions and two dimension tables. Dimension to
>>>> Dimension joins can be mentioned in join chains
>>>> or
>>>> 2.Create a view in MYSQL which you join the two sub divisions . Use the
>>>> view as lens dimension table instead.
>>>>
>>>> Thanks,
>>>> Puneet Gupta
>>>>
>>>> On Fri, Aug 26, 2016 at 10:33 PM, Tao Yan <tyan@linkedin.com> wrote:
>>>>
>>>>> Any plan to support this feature in the next version?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> On Fri, Aug 26, 2016 at 3:41 AM, Rajat Khandelwal <
>>>>> rajatgupta59@gmail.com> wrote:
>>>>>
>>>>>> Seems you are hitting a known issue. https://issues.apache.o
>>>>>> rg/jira/browse/LENS-453
>>>>>> The feature is not supported in the current version.
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 26, 2016 at 3:20 AM Tao Yan <tyan@linkedin.com>
wrote:
>>>>>>
>>>>>>> Hi Lens Developers,
>>>>>>>
>>>>>>> I am trying to define a dimension as follows:
>>>>>>>
>>>>>>> *dimension1.xml:*
>>>>>>> *<x_dimension name="dimension1" xmlns="uri:lens:cube:0.1"
>>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance
>>>>>>> <http://www.w3.org/2001/XMLSchema-instance>"*
>>>>>>> *  xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">*
>>>>>>> *  <attributes>*
>>>>>>> *    <dim_attribute name="primary_key" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr1" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr2" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr3" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr4" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr5" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr6" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr7" _type="BIGINT"/>*
>>>>>>> *    <dim_attribute name="attr8" _type="BIGINT"/>*
>>>>>>> *  </attributes>*
>>>>>>> *  <properties>*
>>>>>>> *    <property name="dimension.dimension1.timed.dimension"
>>>>>>> value="dt"/>*
>>>>>>> *  </properties>*
>>>>>>> *</x_dimension>*
>>>>>>>
>>>>>>> And defined two dimtables under this dimension:
>>>>>>>
>>>>>>> *dimension1_division1.xml*
>>>>>>> <x_dimension_table dimension_name="dimension1"
>>>>>>> table_name="dimension1_division1" weight="10.0"
>>>>>>> xmlns="uri:lens:cube:0.1"
>>>>>>>   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>> xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
>>>>>>>   <columns>
>>>>>>>     <column name="primary_key" _type="BIGINT"/>
>>>>>>> *    <column name="attr1" _type="BIGINT"/>*
>>>>>>> *    <column name="attr2" _type="BIGINT"/>*
>>>>>>> *    <column name="attr3" _type="BIGINT"/>*
>>>>>>> *    <column name="attr4" _type="BIGINT"/>*
>>>>>>>   </columns>
>>>>>>>   <properties>
>>>>>>>     <property name="dimension1.prop" value="d1"/>
>>>>>>>   </properties>
>>>>>>>   <storage_tables>
>>>>>>> ...
>>>>>>>   </storage_tables>
>>>>>>> </x_dimension_table>
>>>>>>>
>>>>>>>
>>>>>>> *dimension1_division2.xml*
>>>>>>> <x_dimension_table dimension_name="dimension1"
>>>>>>> table_name="dimension1_division2" weight="10.0"
>>>>>>> xmlns="uri:lens:cube:0.1"
>>>>>>>   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>> xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
>>>>>>>   <columns>
>>>>>>>     <column name="primary_key" _type="BIGINT"/>
>>>>>>> *    <column name="attr5" _type="BIGINT"/>*
>>>>>>> *    <column name="attr6" _type="BIGINT"/>*
>>>>>>> *    <column name="attr7" _type="BIGINT"/>*
>>>>>>> *    <column name="attr8" _type="BIGINT"/>*
>>>>>>>   </columns>
>>>>>>>   <properties>
>>>>>>>     <property name="dimension1.prop" value="d2"/>
>>>>>>>   </properties>
>>>>>>>   <storage_tables>
>>>>>>> ...
>>>>>>>   </storage_tables>
>>>>>>> </x_dimension_table>
>>>>>>>
>>>>>>> And added some sample data for both dimtables. When I run queries
>>>>>>> like:
>>>>>>> *cube select primary_key, attr1 from dimension1*
>>>>>>> OR
>>>>>>> *cube select primary_key, attr5 from dimension1*
>>>>>>> It returns result correctly, however, when I run the following
query:
>>>>>>>
>>>>>>> l*ens-shell>cube select primary_key, attr1, attr5 from dimension1*
>>>>>>> It gives the following error:
>>>>>>> Request Id: 80676c43-e3e8-455b-b3da-4d38b9f093d2
>>>>>>> Error Code: 3023
>>>>>>> Error Message: No dimension table has the queried columns for
>>>>>>> dimension1, columns: [primary_key, attr1, attr5]
>>>>>>>
>>>>>>> I expect Lens will figure out attr1 and attr5 is not in the same
>>>>>>> logical table, but can be obtained by joining logical tables
>>>>>>> 'dimension1_division1' and 'dimension1_division2' on 'primary_key',
so, I
>>>>>>> also tried to define primary key as join key:
>>>>>>>
>>>>>>> *<dim_attribute name="primary_key" _type="BIGINT" join_key="true"/>*
>>>>>>>
>>>>>>> And updating the dimension failed with no error logs at all:
>>>>>>>
>>>>>>> *lens-shell>update dimension --name dimension1 --path
>>>>>>> /export/home/dev_svc/lens/schemas/dimension1.xml*
>>>>>>> *failed*
>>>>>>>
>>>>>>> Is it possible to achieve this goal, or Lens currently does not
>>>>>>> support it?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> --
>>>>>>>
>>>>>>> *Tao Yan*
>>>>>>> Software Engineer
>>>>>>> Data Analytics Infrastructure Tools and Services
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 206.250.5345
>>>>>>> tyan@linkedin.com
>>>>>>> https://www.linkedin.com/in/taousc
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Tao Yan*
>>>>> Software Engineer
>>>>> Data Analytics Infrastructure Tools and Services
>>>>>
>>>>>
>>>>>
>>>>> 206.250.5345
>>>>> tyan@linkedin.com
>>>>> https://www.linkedin.com/in/taousc
>>>>>
>>>>
>>>>
>>>> _____________________________________________________________
>>>> The information contained in this communication is intended solely for
>>>> the use of the individual or entity to whom it is addressed and others
>>>> authorized to receive it. It may contain confidential or legally privileged
>>>> information. If you are not the intended recipient you are hereby notified
>>>> that any disclosure, copying, distribution or taking any action in reliance
>>>> on the contents of this information is strictly prohibited and may be
>>>> unlawful. If you have received this communication in error, please notify
>>>> us immediately by responding to this email and then delete it from your
>>>> system. The firm is neither liable for the proper and complete transmission
>>>> of the information contained in this communication nor for any delay in its
>>>> receipt.
>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Tao Yan*
>>> Software Engineer
>>> Data Analytics Infrastructure Tools and Services
>>>
>>>
>>>
>>> 206.250.5345
>>> tyan@linkedin.com
>>> https://www.linkedin.com/in/taousc
>>>
>>
>>
>> _____________________________________________________________
>> The information contained in this communication is intended solely for
>> the use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify
>> us immediately by responding to this email and then delete it from your
>> system. The firm is neither liable for the proper and complete transmission
>> of the information contained in this communication nor for any delay in its
>> receipt.
>>
>
>


-- 

*Tao Yan*
Software Engineer
Data Analytics Infrastructure Tools and Services



206.250.5345
tyan@linkedin.com
https://www.linkedin.com/in/taousc

Mime
View raw message