hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aihua Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11980) Follow up on HIVE-11696, exception is thrown from CTAS from the table with table-level serde is Parquet while partition-level serde is JSON
Date Thu, 01 Oct 2015 15:56:26 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940004#comment-14940004
] 

Aihua Xu commented on HIVE-11980:
---------------------------------

The tests seem not related to the patch.

[~szehon] can you help review the change? 

> Follow up on HIVE-11696, exception is thrown from CTAS from the table with table-level
serde is Parquet while partition-level serde is JSON
> -------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-11980
>                 URL: https://issues.apache.org/jira/browse/HIVE-11980
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>         Attachments: HIVE-11980.patch
>
>
> When we create a new table from the table with table-level serde to be Parquet and partition-level
serde to be JSON, currently the following exception will be thrown if there are struct fields.
> Apparently, getStructFieldsDataAsList() also needs to handle the case of List in addition
to ArrayWritable similar to getStructFieldData.
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: Cannot inspect java.util.ArrayList
>         at org.apache.hadoop.hive.ql.io.parquet.serde.ArrayWritableObjectInspector.getStructFieldsDataAsList(ArrayWritableObjectInspector.java:172)
>         at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:354)
>         at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:257)
>         at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:241)
>         at org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:55)
>         at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:720)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:813)
>         at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:813)
>         at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
>         at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:162)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message