hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13245) VectorDeserializeRow throws IndexOutOfBoundsException
Date Wed, 09 Mar 2016 19:44:40 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187767#comment-15187767
] 

Matt McCline commented on HIVE-13245:
-------------------------------------

[~prasanth_j] No, I have not seen this before.  And, it could be what Gopal said because it
is going through Native Vector MapJoin but it isn't clear if the new Fast hash table is being
used (HIVE-12896) or just the regular BytesBytesMultiHashMap is being used.

Is hive.vectorized.execution.mapjoin.native.fast.hashtable.enabled ON or OFF?

> VectorDeserializeRow throws IndexOutOfBoundsException
> -----------------------------------------------------
>
>                 Key: HIVE-13245
>                 URL: https://issues.apache.org/jira/browse/HIVE-13245
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.1.0
>            Reporter: Prasanth Jayachandran
>
> When running following query on TPCDS 1000 scale, VectorDeserializeRow threw ArrayIndexOutOfBoundsException
> {code:title=Query}
> SELECT `customer_address`.`ca_zip`                   AS `ca_zip`, 
>        `customer_demographics`.`cd_education_status` AS `cd_education_status`, 
>        Sum(`store_sales`.`ss_net_paid`)              AS `SUM:SS_NET_PAID:ok` 
> FROM   `store_sales` `store_sales` 
>        INNER JOIN `customer` `customer` 
>                ON ( `store_sales`.`ss_customer_sk` = 
>                   `customer`.`c_customer_sk` ) 
>        INNER JOIN `customer_address` `customer_address` 
>                ON ( `customer`.`c_current_addr_sk` = 
>                   `customer_address`.`ca_address_sk` ) 
>        INNER JOIN `customer_demographics` `customer_demographics` 
>                ON ( `customer`.`c_current_cdemo_sk` = 
>                     `customer_demographics`.`cd_demo_sk` ) 
> WHERE  ( `customer`.`c_first_sales_date_sk` > 2452300 
>          AND `customer_demographics`.`cd_gender` = 'F' 
>          AND `customer`.`c_current_addr_sk` IS NOT NULL 
>          AND `store_sales`.`ss_sold_date_sk` IS NOT NULL 
>          AND `customer`.`c_current_cdemo_sk` IS NOT NULL ) 
> GROUP  BY `ca_zip`, 
>           `cd_education_status`;
> {code}
> {code:title=Exception}
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:354)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:356)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
> 	... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:62)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> 	... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:392)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:143)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:121)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:115)
> 	at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:170)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:55)
> 	... 18 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:152)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow$StringReaderByValue.apply(VectorDeserializeRow.java:362)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeByValue(VectorDeserializeRow.java:710)
> 	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultSingleValue(VectorMapJoinGenerateResultOperator.java:189)
> 	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerGenerateResultOperator.finishInner(VectorMapJoinInnerGenerateResultOperator.java:186)
> 	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:380)
> 	... 26 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message