spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jey Kottalam <...@cs.berkeley.edu>
Subject Re: HDFS user permissions
Date Fri, 20 Sep 2013 00:43:36 GMT
Hi Koert,

Here's a pending pull request from jerryshao that may be relevant:
https://github.com/mesos/spark/pull/750

Hope that helps,
-Jey

On Thursday, September 19, 2013, Koert Kuipers wrote:

> hello all,
> currently spark runs tasks as the user that runs the spark worker daemon.
> at least, this is what i observer with standalone spark.
>
> as a result spark does not respect user permissions on HDFS.
>
> i guess this could be fixed by running the tasks as a different user, or
> even just by using hadoop's proxy mechanism to access HDFS as that user.
>
> are there plans to fix this? is there a JIRA tickets for this? i couldn't
> find it but would like to track this issue.
>
> and what about secure HDFS (kerberos)?
>
> thanks! koert
>

Mime
View raw message