hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carter Shanklin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16789) Query with window function fails when where clause returns no records
Date Thu, 01 Jun 2017 14:47:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033094#comment-16033094
] 

Carter Shanklin commented on HIVE-16789:
----------------------------------------

This doesn't reproduce on HDP 2.5.3 (version = hive_2_5_3_0_37-1.2.1000.2.5.3.0-37) or on
later versions. I didn't try going any further back.

{code}
hive> select
    > a,b,c,d,e,
    > min(c) over (partition by a,b order by d rows between 1 preceding and current row)
prev_c
    > from
    > (
    > select '1234' a ,'abc' b,10 c,2 d,'test1' e
    > union all
    > select '1234' a ,'abc' b,9 c,1 d,'test2' e
    > union all
    > select '1234' a ,'abc' b,11 c,3 d,'test2' e
    > union all
    > select '1234' a ,'abcd' b,1 c,5 d,'test2' e
    > union all
    > select '1234' a ,'abcd' b,6 c,9 d,'test1' e
    > )X
    > where e='test3'
    > ;
Query ID = vagrant_20170601144546_60ec95f0-a4a1-4ee5-a265-3219a9cae780
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1496327868636_0004)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
Map 4 ..........   SUCCEEDED      1          1        0        0       0       0
Map 5 ..........   SUCCEEDED      1          1        0        0       0       0
Map 6 ..........   SUCCEEDED      1          1        0        0       0       0
Map 7 ..........   SUCCEEDED      1          1        0        0       0       0
Reducer 3 ......   SUCCEEDED      1          1        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 06/06  [==========================>>] 100%  ELAPSED TIME: 0.83 s
--------------------------------------------------------------------------------
OK
Time taken: 1.958 seconds
hive>
{code}

Is it possible for you to upgrade?

> Query with window function fails when where clause returns no records
> ---------------------------------------------------------------------
>
>                 Key: HIVE-16789
>                 URL: https://issues.apache.org/jira/browse/HIVE-16789
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 1.0.1, 1.2.1
>         Environment: OS: CentOS release 6.6 (Final)
> HDP: 2.7.3
> Hive: 1.0.1,1.2.1
>            Reporter: Narayana
>
> When outer where clause found no matching records, queries like these are failing
> {code}
> select 
> a,b,c,d,e,
> min(c) over (partition by a,b order by d rows between 1 preceding and current row) prev_c
> from
> (
> select '1234' a ,'abc' b,10 c,2 d,'test1' e
> union all 
> select '1234' a ,'abc' b,9 c,1 d,'test2' e
> union all
> select '1234' a ,'abc' b,11 c,3 d,'test2' e
> union all  
> select '1234' a ,'abcd' b,1 c,5 d,'test2' e
> union all 
> select '1234' a ,'abcd' b,6 c,9 d,'test1' e
> )X
> where e='test3'
> ;
> {code}
> Error:
> Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: null
> 	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:295)
> 	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Unknown Source)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.NoSuchElementException
> 	at java.util.ArrayDeque.getFirst(Unknown Source)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMax$MaxStreamingFixedWindow.terminate(GenericUDAFMax.java:280)
> 	at org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.finishPartition(WindowingTableFunction.java:413)
> 	at org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:337)
> 	at org.apache.hadoop.hive.ql.exec.PTFOperator.closeOp(PTFOperator.java:95)
> 	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:598)
> 	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
> 	at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> 	... 7 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message