hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-25159) Remove support for ordered results in llap external client library
Date Sat, 31 Jul 2021 00:09:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-25159?focusedWorklogId=631917&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631917
]

ASF GitHub Bot logged work on HIVE-25159:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 31/Jul/21 00:08
            Start Date: 31/Jul/21 00:08
    Worklog Time Spent: 10m 
      Work Description: github-actions[bot] commented on pull request #2318:
URL: https://github.com/apache/hive/pull/2318#issuecomment-890260557


   This pull request has been automatically marked as stale because it has not had recent
activity. It will be closed if no further activity occurs.
   Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 631917)
    Time Spent: 20m  (was: 10m)

> Remove support for ordered results in llap external client library
> ------------------------------------------------------------------
>
>                 Key: HIVE-25159
>                 URL: https://issues.apache.org/jira/browse/HIVE-25159
>             Project: Hive
>          Issue Type: Bug
>          Components: Clients, Hive
>            Reporter: Shubham Chaurasia
>            Assignee: Shubham Chaurasia
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-25159.01.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently when querying via llap external client framework, in case of order by queries
-
> 1. Due to the fact that spark-llap used to wrap actual query in a subquery as mentioned
in [HIVE-19794|https://issues.apache.org/jira/browse/HIVE-19794]
> a) We had to detect order by like - 
> {code}
> orderByQuery = plan.getQueryProperties().hasOrderBy() || plan.getQueryProperties().hasOuterOrderBy();
> {code}
> Due to this we recently saw an exception like below for one of the queries that did not
have an outer order by (It was having an order by in a subquery)
> {code}
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.IOException:
> java.lang.IllegalStateException: Requested to generate single split. Paths and fileStatuses
are expected to be 1. Got paths: 1 fileStatuses: 7
> {code}
> b) Also we had to disable following optimization - 
> {code}
> HiveConf.setBoolVar(conf, ConfVars.HIVE_REMOVE_ORDERBY_IN_SUBQUERY, false);
> {code}
> 2. By default we have {{hive.llap.external.splits.order.by.force.single.split=true}}
which forces us to generate single split leading to performance bottleneck. 
> We should remove ordering support altogether from llap external client repo and let clients
handle it at their end.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message