drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <jasbir.s...@accenture.com>
Subject RE: Drill Hangout 2/7/2017
Date Tue, 07 Feb 2017 09:04:06 GMT
Thanks for the reply. 

Attached in a query in which we have a member SUBDMA which has around 300 members in IN CLAUSE.
If you run this type of query fetching data from parquet file, drill gives no result. And
if we reduce IN CLAUSE to less than 19 it works. 

Can this be solved by changing any parameter?


Regards,
Jasbir Singh

-----Original Message-----
From: Jinfeng Ni [mailto:jni@apache.org] 
Sent: Tuesday, February 07, 2017 1:01 PM
To: user <user@drill.apache.org>
Cc: dev <dev@drill.apache.org>; Sareen, Nitin A. <nitin.a.sareen@accenture.com>;
Kothari, Maneesh <maneesh.kothari@accenture.com>; Kumar, H. P. <h.p.kumar@accenture.com>
Subject: Re: Drill Hangout 2/7/2017

Regarding item 2 & 3,  IIKC, there is no upper limit of IN clause. An IN-list predicate
 could be converted to a JOIN between T1 and VALUES operator, when the number of values in
IN-LIST is beyond certain threshold. An Aggregate operator is applied to remove possible duplicates
values in the list. That's why you may see HashAgg in the query plan.

If the number of values in IN-list is under the threshold, IN-list is evaluated as a OR-ed
predicates.

The default threshold is 20 [1]. But you can change it, by running following:

alter session set `planner.in_subquery_threshold` = some_number;


From T1
WHERE  T1.expression in (value1, value2, ..., value_n)?

==>
           Join
          /        \
       T1         Agg
                      \
                      Values (values1, values2, ..., values_n)


[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_blob_master_exec_java-2Dexec_src_main_java_org_apache_drill_exec_planner_physical_PlannerSettings.java-23L99-2DL100&d=DwIFaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=pJ2Zg_dw2Uqydx8WOtKp7fRJ2_93ggT4MlawkE_77d4&s=XfXo4Gh524K0A_hwKSgiQA2Kzt7L2C-x0RawQFmlpDE&e=


On Mon, Feb 6, 2017 at 9:16 PM,  <jasbir.sing@accenture.com> wrote:
> Hi,
>
> I am interested in joining this session.
>
> My area of interest would be -
>
> 1. Future roadmap of Apache Drill
> 2. How apache drill creates HashAgg when there are lot of IN members 
> in a where clause 3. What is the upper limit of IN CLAUSE.
>
> Regards,
> Jasbir Singh
>
> -----Original Message-----
> From: Jinfeng Ni [mailto:jni@apache.org]
> Sent: Tuesday, February 07, 2017 1:19 AM
> To: dev <dev@drill.apache.org>; user <user@drill.apache.org>
> Subject: Drill Hangout 2/7/2017
>
> Hi drillers,
>
> We are going to have Drill Hangout tomorrow (02/07/2017, 10 AM PT). If you have any suggestions
for hangout topics, you can add them to this thread. We will also ask around at the beginning
of the hangout for topics.
>
> Thank you,
>
> Jinfeng
>
> ________________________________
>
> This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.
> ______________________________________________________________________
> ________________
>
> www.accenture.com
Mime
View raw message