spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jey Kottalam <>
Subject Re: HDFS user permissions
Date Fri, 20 Sep 2013 00:43:36 GMT
Hi Koert,

Here's a pending pull request from jerryshao that may be relevant:

Hope that helps,

On Thursday, September 19, 2013, Koert Kuipers wrote:

> hello all,
> currently spark runs tasks as the user that runs the spark worker daemon.
> at least, this is what i observer with standalone spark.
> as a result spark does not respect user permissions on HDFS.
> i guess this could be fixed by running the tasks as a different user, or
> even just by using hadoop's proxy mechanism to access HDFS as that user.
> are there plans to fix this? is there a JIRA tickets for this? i couldn't
> find it but would like to track this issue.
> and what about secure HDFS (kerberos)?
> thanks! koert

View raw message