lens-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajat Khandelwal <rajatgupt...@gmail.com>
Subject Re: Difference between Select and Cube Select
Date Fri, 26 Aug 2016 10:36:26 GMT
It's part of the driver interface. Each driver has its own estimate method.
The methods have already been implemented for HiveDriver and JDBCDriver,
for adding drivers externally, one has to implement the method.

https://github.com/apache/lens/blob/master/lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java#L69


On Fri, Aug 26, 2016 at 1:46 AM Tao Yan <tyan@linkedin.com> wrote:

> Thanks! Is there a general estimation algorithm for all drivers? Or
> developer needs to implement estimation algorithm for each new drivers he
> added?
>
> On Thu, Aug 25, 2016 at 11:38 AM, Rajat Khandelwal <rajatgupta59@gmail.com
> > wrote:
>
>> All drivers are asked to estimate. In the process of estimation, a driver
>> can fail, which lens server takes to mean that this driver cannot run this
>> query.
>>
>> On Thu, Aug 25, 2016 at 10:32 PM Tao Yan <tyan@linkedin.com> wrote:
>>
>>> If Lens find the table is not a cube/dimension and pass the query
>>> through to drivers, then, how does Lens decide which driver to use?  What
>>> if the driver picked is not the right one? For example, a query contains
>>> hive UDF is passed through to a MySQL database.
>>>
>>> Thanks,
>>>
>>> On Wed, Aug 24, 2016 at 11:47 PM, Rajat Khandelwal <
>>> rajatgupta59@gmail.com> wrote:
>>>
>>>> That doesn't happen. One driver is picked to run the query. The drivers
>>>> bid by providing their estimate of cost for running this query.
>>>>
>>>> On Thu, Aug 25, 2016 at 1:47 AM Tao Yan <tyan@linkedin.com> wrote:
>>>>
>>>>> If the query is executed in multiple drivers, and more than one
>>>>> drivers return the result, then, how does Lens decide which result to
>>>>> return to user? And if one driver return result before others, will Lens
>>>>> wait for other drivers?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> On Wed, Aug 24, 2016 at 12:00 AM, Rajat Khandelwal <
>>>>> rajatgupta59@gmail.com> wrote:
>>>>>
>>>>>> Yes, it can be any standard sql. The logic in lens is to check the
>>>>>> from clause, and if the entity in the from-clause is either a cube
or a
>>>>>> dimension, then treat it as cube query, and do the translation before
>>>>>> running this query. If it's not a cube or a dimension, it's treated
as a
>>>>>> native query and all the drivers are forwarded this query as it is
without
>>>>>> any translation.
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 24, 2016 at 5:43 AM Tao Yan <tyan@linkedin.com>
wrote:
>>>>>>
>>>>>>> When using "select" (not "cube select"), can it be any standard
SQL
>>>>>>> (not bounded to Lens QL)?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Tue, Aug 23, 2016 at 4:57 PM, amareshwarisr . <
>>>>>>> amareshwari@gmail.com> wrote:
>>>>>>>
>>>>>>>> cube select expects the from clause to be on cube/dimension.
select
>>>>>>>> is the one in SQL.
>>>>>>>>
>>>>>>>> Going forward from 2.6 onwards, cube keyword is optional
for
>>>>>>>> cube/dimension queries.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Wed, Aug 24, 2016 at 3:53 AM, Tao Yan <tyan@linkedin.com>
wrote:
>>>>>>>>
>>>>>>>>> Hi Lens Developer,
>>>>>>>>>
>>>>>>>>> I found the following example in
>>>>>>>>> https://github.com/apache/lens/blob/master/lens-examples/src/main/resources/dimension-queries.sql
>>>>>>>>>
>>>>>>>>> select * from (cube select sample_dim.name name1,
>>>>>>>>> sample_dim2_chain.name name2 from sample_dim where sample_dim.name
>>>>>>>>> !='first') a
>>>>>>>>>
>>>>>>>>> I tested "cube select * " and it didn't work, and the
above
>>>>>>>>> example shows "select *". What is the difference between
"select" and "cube
>>>>>>>>> select"?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> *Tao Yan*
>>>>>>>>> Software Engineer
>>>>>>>>> Data Analytics Infrastructure Tools and Services
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 206.250.5345
>>>>>>>>> tyan@linkedin.com
>>>>>>>>> https://www.linkedin.com/in/taousc
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Tao Yan*
>>>>>>> Software Engineer
>>>>>>> Data Analytics Infrastructure Tools and Services
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 206.250.5345
>>>>>>> tyan@linkedin.com
>>>>>>> https://www.linkedin.com/in/taousc
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Tao Yan*
>>>>> Software Engineer
>>>>> Data Analytics Infrastructure Tools and Services
>>>>>
>>>>>
>>>>>
>>>>> 206.250.5345
>>>>> tyan@linkedin.com
>>>>> https://www.linkedin.com/in/taousc
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *Tao Yan*
>>> Software Engineer
>>> Data Analytics Infrastructure Tools and Services
>>>
>>>
>>>
>>> 206.250.5345
>>> tyan@linkedin.com
>>> https://www.linkedin.com/in/taousc
>>>
>>
>
>
> --
>
> *Tao Yan*
> Software Engineer
> Data Analytics Infrastructure Tools and Services
>
>
>
> 206.250.5345
> tyan@linkedin.com
> https://www.linkedin.com/in/taousc
>

Mime
View raw message