hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mithun Radhakrishnan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21897) Setting serde / serde properties for partitions
Date Thu, 20 Jun 2019 21:38:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868949#comment-16868949
] 

Mithun Radhakrishnan commented on HIVE-21897:
---------------------------------------------

bq. Mithun Radhakrishnan just for sure I've inserted two rows, one with dt=1, one with dt=2,
and checked the files in HDFS. They are both ORC files.

Pardon me, but this does not sound right. How exactly did you insert values into {{dt=1}}
and {{dt=2}}? At what point did you write to each of the partitions? If you {{INSERT OVERWRITE}}
*after* the partitions were created, then I can understand how you see what you see. But,
consider this sequence:

{code:sql}
-- Create the table.
CREATE TABLE foobar ( foo STRING, bar STRING ) PARTITIONED BY (dt STRING) STORED AS TEXTFILE;
-- 1.
INSERT OVERWRITE TABLE foobar PARTITION( dt='1' ) VALUES ( "foo1", "value1" ); -- SerDe ==
LazySimpleSerDe.
-- Describe the partition, to confirm.
DESC FORMATTED TABLE foobar PARTITION (dt='1');

-- Alter format.
ALTER TABLE foobar SET FILEFORMAT ORCFILE; -- (No CASCADE)

-- 2.
INSERT OVERWRITE TABLE foobar PARTITION( dt='2' ) VALUES ( "foo2", "value2" ); -- SerDe ==
OrcSerDe.
-- Describe the partition, to confirm.
DESC FORMATTED TABLE foobar PARTITION (dt='2');
{code}

In this case, if {{dt='1'}} doesn't retain its SerDe setting, it will be rendered unreadable
after the table-format is changed. Please correct me if I'm wrong.

> Setting serde / serde properties for partitions
> -----------------------------------------------
>
>                 Key: HIVE-21897
>                 URL: https://issues.apache.org/jira/browse/HIVE-21897
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 3.1.1
>            Reporter: Miklos Gergely
>            Assignee: Miklos Gergely
>            Priority: Major
>             Fix For: 4.0.0
>
>
> According to [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AddSerDeProperties] the
SerDe and the SerDe properties can be set for a partition too, so
>  
> {code:java}
> ALTERT TABLE table PARTITION (partition_col='partition_value') SET SERDE 'serde.class.name';{code}
> Is a valid statement. In fact it is not rejected, but it is not doing anything at all.
The execution is successful, everything remains the same. The same is true for setting the
serde properties:
> {code:java}
> ALTER TABLE table PARTITION (partition_col='partition_value') SET SERDEPROPERTIES ('property_name'='property_value');{code}
> is also a valid statement, and not doing anything.
> I suggest to modify the parser, and reject these statements. SerDe is for a table, and
not for a partition.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message