hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-15437) avro tables join fails when - tbl join tbl_postfix
Date Thu, 29 Dec 2016 23:42:58 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thejas M Nair updated HIVE-15437:
---------------------------------
    Description: 
The following queries return good results:
select * from table1 where col1=key1; 
select * from table1_1 where col1=key1; 
When join them together, it gets following error:
{noformat}
Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long, expecting union
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}

The two avro tables both is defined by using avro schema, and the first table's name is the
second table name's prefix. 

Note that this happens when single map is reading input from both tables. If map-join is used
with MR execution engine, or if Tez execution engine is used, this issue is not seen.


  was:
The following queries return good results:
select * from table1 where col1=key1; 
select * from table1_1 where col1=key1; 
When join them together, it gets following error:
{noformat}
Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long, expecting union
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}

The two avro tables both is defined by using avro schema, and the first table's name is the
second table name's prefix. 


> avro tables join fails when - tbl join tbl_postfix
> --------------------------------------------------
>
>                 Key: HIVE-15437
>                 URL: https://issues.apache.org/jira/browse/HIVE-15437
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Yongzhi Chen
>            Assignee: Yongzhi Chen
>             Fix For: 2.2.0
>
>         Attachments: HIVE-15437.1.patch
>
>
> The following queries return good results:
> select * from table1 where col1=key1; 
> select * from table1_1 where col1=key1; 
> When join them together, it gets following error:
> {noformat}
> Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long, expecting
union
>         at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141)
~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {noformat}
> The two avro tables both is defined by using avro schema, and the first table's name
is the second table name's prefix. 
> Note that this happens when single map is reading input from both tables. If map-join
is used with MR execution engine, or if Tez execution engine is used, this issue is not seen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message