hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergio Peña (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HIVE-15385) Failure to inherit permissions when running HdfsUtils.setFullFileStatus(..., false) causes queries to fail
Date Fri, 09 Dec 2016 17:51:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15735896#comment-15735896
] 

Sergio Peña commented on HIVE-15385:
------------------------------------

Thanks [~stakiar] for the explanation. I spent some time investigating the history of this
permission issue, and as you mentioned, the places where the IOException is not ignored could
have been by accident.

Based on the history that permissions should not throw an exception and make the query to
fail, I agree on just send a warning to the log inside the setFullFileStatus() instead of
throwing an exception. This is confusing for people that want to use such method.

+1

I don't think tests failures are related, but could you re-attach the patch to see if the
others stop failing?

> Failure to inherit permissions when running HdfsUtils.setFullFileStatus(..., false) causes
queries to fail
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-15385
>                 URL: https://issues.apache.org/jira/browse/HIVE-15385
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>         Attachments: HIVE-15385.1.patch, HIVE-15385.2.patch
>
>
> According to https://cwiki.apache.org/confluence/display/Hive/Permission+Inheritance+in+Hive,
failure to inherit permissions should not cause queries to fail.
> It looks like this was the case until HIVE-13716, which added some code to use {{fs.setOwner}},
{{fs.setAcl}}, and {{fs.setPermission}} to set permissions instead of shelling out and running
{{-chgrp -R ...}}.
> When shelling out, the return status of each command is ignored, so if there are any
failures when inheriting permissions, a warning is logged, but the query still succeeds.
> However, when invoked the {{FileSystem}} API, any failures will be propagated up to the
caller, and the query will fail.
> This is problematic because {{setFulFileStatus}} shells out when the {{recursive}} parameter
is set to {{true}}, and when it is false it invokes the {{FileSystem}} API. So the behavior
is inconsistent depending on the value of {{recursive}}.
> We should decide whether or not permission inheritance should fail queries or not, and
then ensure the code consistently follows that decision.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message