drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua" <ku...@apache.org>
Subject Re: This query is so slow
Date Mon, 06 Aug 2018 17:41:01 GMT
Hi Peter

What does the profile for the query indicate?

Take a look at the operator overview. It will indicate which operator is using the most CPU
cycles. If the average and max processing times vary wildly, it might be a problem of skew,
where some fragments are doing a relatively excessive work.
If that is the case, within that operator's profile segment, you can see the distribution
of the fragments and see which ones are the long pole in your run.

~ KK 

On 8/6/2018 9:01:25 AM, Peter Edike <peter.edike@interswitchgroup.com> wrote:
Hello all,
 
The following query takes ages to complete in drill and more often that not, fails
 
select * from dfs.`/iswdata/rj/201805` where unique_key in
    (
         select unique_key from dfs.`/iswdata/rj/201805` group by unique_key having
count(unique_key) > 1
    )
limit 40
 
Please what can I do to improve the performance of this query
 
Kind Regards
 
Peter Edike
Senior Software Engineer
Research and Development, ENG
Engineering

Office  NO:  
Mobile NO:
Email: peter.edike@interswitchgroup.com [mailto:peter.edike@interswitchgroup.com]
Interswitch Limited
1648C Oko-Awo Street, Victoria Island Lagos
Customer Contact Centre 0700-9065000
ü http://www.interswitchgroup.com [http://www.interswitchgroup.com/]
[https://www.quickteller.com/loan-request]
This e-mail and all  attachments transmitted with it remain the property of Interswitch Limited
, the information contained herein  are private  confidential and intended solely for the
use of the addressee. If you have received this e-mail in error, kindly notify the sender.
If you are not the addressee, you should not disseminate, distribute or copy this e-mail.
Kindly notify Interswitch immediately by email if you have received this email in error and
delete this email and any attachment from your system  Emails cannot be guaranteed to be
secure or error free as the message and any attachments could be intercepted, corrupted, lost,
delayed, incomplete or amended. the contents of this email or its attachments have been scanned
for all viruses and all reasonable measures have been taken to ensure that no viruses are
present. Interswitch Limited and its subsidiaries do not accept liability for damage caused
by this email or any attachments.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message