hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ádám Szita (Jira) <j...@apache.org>
Subject [jira] [Assigned] (HIVE-22931) HoS dynamic partitioning fails with blobstore optimizations off
Date Wed, 26 Feb 2020 09:18:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-22931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ádám Szita reassigned HIVE-22931:
---------------------------------


> HoS dynamic partitioning fails with blobstore optimizations off
> ---------------------------------------------------------------
>
>                 Key: HIVE-22931
>                 URL: https://issues.apache.org/jira/browse/HIVE-22931
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ádám Szita
>            Assignee: Ádám Szita
>            Priority: Major
>
> Reproduction steps:
>  - Create s3a backed table and normal table.
> {code:java}
> CREATE TABLE source (
>   a string,
>   b int,
>   c int);
>   
> CREATE TABLE target (
>   a string)
> PARTITIONED BY (
>   b int,
>   c int)
> STORED AS parquet
> LOCATION
>   's3a://somepath';
> {code}
>  - Insert values into normal table.
> {code:java}
> INSERT INTO TABLE source VALUES ("a", "1", "1");
> {code}
>  - Do an insert overwrite with dynamic partitions:
> {code:java}
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.blobstore.optimizations.enabled=false;
> set hive.execution.engine=spark;
> INSERT OVERWRITE TABLE target partition (b,c)
> SELECT *
> FROM source;{code}
> This fails only with Spark execution engine + blobstorage optimizations being turned
off with:
> 2020-01-16 15:24:56,064 ERROR hive.ql.metadata.Hive: [load-dynamic-partitions-5]: Exception
when loading partition with parameters  partPath=hdfs://nameservice1/tmp/hive/hive/6bcee075-b637-429e-9bf0-a2658355415e/hive_2020-01-16_15-24-01_156_4299941251929377815-4/-mr-10000/.hive-staging_hive_2020-01-16_15-24-01_156_4299941251929377815-4/-ext-10002,
 table=email_click_base,  partSpec={b=null, c=null},  replace=true,  listBucketingEnabled=false,
 isAcid=false,  hasFollowingStatsTask=trueorg.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Partition spec is incorrect. {companyid=null, eventmonth=null})
>         at org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:1666)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message