hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ziyang Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-13965) Empty resultset run into Exception when using Thrift Binary Serde
Date Tue, 07 Jun 2016 21:06:21 GMT

     [ https://issues.apache.org/jira/browse/HIVE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ziyang Zhao updated HIVE-13965:
-------------------------------
    Description: 
This error can be reproduced by enabling thrift binary serde, using beeline connect to hiveserver2
and executing the following commands:
>create table test3(num1 int);
>create table test4(num1 int);
>insert into test3 values(1);
>insert into test4 values(2);
>select * from test3 join test4 on test3.num1=test4.num1;

The result should be empty, but it gives an exception:

Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:206)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1029)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:641)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:195)
        ... 8 more

This error is caused in FileSinkOperator.java. 
If the resultset is empty, function process() will not be called, so variable "fpaths" will
not be set. When run into CloseOp(), 
if (conf.isHiveServerQuery() && HiveConf.getBoolVar(hconf,
         HiveConf.ConfVars.HIVE_SERVER2_THRIFT_RESULTSET_SERIALIZE_IN_TASKS) &&
         serializer.getClass().getName().equalsIgnoreCase(ThriftJDBCBinarySerDe.class.getName()))
{
         try {
           recordValue = serializer.serialize(null, inputObjInspectors[0]);
           rowOutWriters = fpaths.outWriters;
           rowOutWriters[0].write(recordValue);
         } catch (SerDeException | IOException e) {
           throw new HiveException(e);
         }
     }
Here fpaths is null.


  was:
This error can be reproduced by enabling thrift binary serde, and using beeline connect to
hiveserver2, executing the following commands:
>create table test3(num1 int);
>create table test4(num1 int);
>insert into test3 values(1);
>insert into test4 values(2);
>select * from test3 join test4 on test3.num1=test4.num1;

The result should be empty, but it gives an exception:

Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:206)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1029)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:641)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:195)
        ... 8 more

This error is caused in FileSinkOperator.java. If the resultset is empty, function process()
will not be called. So when run into CloseOp(), 
if (conf.isHiveServerQuery() && HiveConf.getBoolVar(hconf,
         HiveConf.ConfVars.HIVE_SERVER2_THRIFT_RESULTSET_SERIALIZE_IN_TASKS) &&
         serializer.getClass().getName().equalsIgnoreCase(ThriftJDBCBinarySerDe.class.getName()))
{
         try {
           recordValue = serializer.serialize(null, inputObjInspectors[0]);
           rowOutWriters = fpaths.outWriters;
           rowOutWriters[0].write(recordValue);
         } catch (SerDeException | IOException e) {
           throw new HiveException(e);
         }
     }
Here fpaths is null.



> Empty resultset run into Exception when using Thrift Binary Serde
> -----------------------------------------------------------------
>
>                 Key: HIVE-13965
>                 URL: https://issues.apache.org/jira/browse/HIVE-13965
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 2.1.0
>            Reporter: Ziyang Zhao
>
> This error can be reproduced by enabling thrift binary serde, using beeline connect to
hiveserver2 and executing the following commands:
> >create table test3(num1 int);
> >create table test4(num1 int);
> >insert into test3 values(1);
> >insert into test4 values(2);
> >select * from test3 join test4 on test3.num1=test4.num1;
> The result should be empty, but it gives an exception:
> Diagnostic Messages for this Task:
> Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
>         at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:206)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.lang.NullPointerException
>         at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1029)
>         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:641)
>         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
>         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
>         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
>         at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:195)
>         ... 8 more
> This error is caused in FileSinkOperator.java. 
> If the resultset is empty, function process() will not be called, so variable "fpaths"
will not be set. When run into CloseOp(), 
> if (conf.isHiveServerQuery() && HiveConf.getBoolVar(hconf,
>          HiveConf.ConfVars.HIVE_SERVER2_THRIFT_RESULTSET_SERIALIZE_IN_TASKS) &&
>          serializer.getClass().getName().equalsIgnoreCase(ThriftJDBCBinarySerDe.class.getName()))
{
>          try {
>            recordValue = serializer.serialize(null, inputObjInspectors[0]);
>            rowOutWriters = fpaths.outWriters;
>            rowOutWriters[0].write(recordValue);
>          } catch (SerDeException | IOException e) {
>            throw new HiveException(e);
>          }
>      }
> Here fpaths is null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message