hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22040) Drop partition throws exception with 'Failed to delete parent: File does not exist' when the partition's parent path does not exists
Date Fri, 26 Jul 2019 08:36:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893465#comment-16893465
] 

Hive QA commented on HIVE-22040:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12975804/HIVE-22040.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/18172/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18172/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18172/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ date '+%Y-%m-%d %T.%3N'
2019-07-26 08:35:16.271
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18172/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-07-26 08:35:16.274
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at f7f7c58 HIVE-12971: Hive Support for Kudu (Grant Henke, reviewed by Jesus Camacho
Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at f7f7c58 HIVE-12971: Hive Support for Kudu (Grant Henke, reviewed by Jesus Camacho
Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-07-26 08:35:17.510
+ rm -rf ../yetus_PreCommit-HIVE-Build-18172
+ mkdir ../yetus_PreCommit-HIVE-Build-18172
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18172
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18172/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java:
does not exist in index
error: standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestDropPartitions.java:
does not exist in index
error: src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java: does not exist in index
error: src/test/java/org/apache/hadoop/hive/metastore/client/TestDropPartitions.java: does
not exist in index
error: main/java/org/apache/hadoop/hive/metastore/Warehouse.java: does not exist in index
error: test/java/org/apache/hadoop/hive/metastore/client/TestDropPartitions.java: does not
exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18172
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12975804 - PreCommit-HIVE-Build

> Drop partition throws exception with 'Failed to delete parent: File does not exist' when
the partition's parent path does not exists
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-22040
>                 URL: https://issues.apache.org/jira/browse/HIVE-22040
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 1.2.1, 2.0.0, 3.0.0
>            Reporter: xiepengjie
>            Assignee: xiepengjie
>            Priority: Major
>         Attachments: HIVE-22040.patch
>
>
> I create a manage table with multi partition columns, when i try to drop partition throws
exception with 'Failed to delete parent: File does not exist' when the partition's parent
path does not exist. The partition's metadata in mysql has been deleted, but the exception
is still thrown. it will fail if  connecting hiveserver2 with jdbc by java, this problem
also exists in master branch, I  think it is very unfriendly and we should fix it.
> Example:
> – First, create manage table with nulti partition columns, and add partitions:
> {code:java}
> drop table if exists t1;
> create table t1 (c1 int) partitioned by (year string, month string, day string);
> alter table t1 add partition(year='2019', month='07', day='01');{code}
> – Second, delete the path of partition 'month=07':
> {code:java}
> hadoop fs -rm -r /user/hadoop/xiepengjietest.db/drop_partition/year=2019/month=07{code}
> --  Third, when i try to drop partition, the metastore throws exception with 'Failed
to delete parent: File does not exist' .
> {code:java}
> alter table t1 drop partition partition(year='2019', month='07', day='01');
> {code}
> exception like this:
> {code:java}
> Error: Error while processing statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. Failed to delete parent: File does not exist: /user/hadoop/xiepengjietest.db/drop_partition/year=2019/month=07
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummaryInt(FSDirStatAndListingOp.java:493)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummary(FSDirStatAndListingOp.java:140)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:3995)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:1202)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:883)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2115)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2111)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1867)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2111) (state=08S01,code=1)
>  {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message