spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jey Kottalam <...@cs.berkeley.edu>
Subject Re: HDFS user permissions
Date Fri, 20 Sep 2013 00:46:47 GMT
Please note that this branch has a more recent version of the patch that
you should use if applying to 0.8 or master:
https://github.com/jey/spark/tree/hdfs-auth

On Thursday, September 19, 2013, Jey Kottalam wrote:

> Hi Koert,
>
> Here's a pending pull request from jerryshao that may be relevant:
> https://github.com/mesos/spark/pull/750
>
> Hope that helps,
> -Jey
>
> On Thursday, September 19, 2013, Koert Kuipers wrote:
>
>> hello all,
>> currently spark runs tasks as the user that runs the spark worker daemon.
>> at least, this is what i observer with standalone spark.
>>
>> as a result spark does not respect user permissions on HDFS.
>>
>> i guess this could be fixed by running the tasks as a different user, or
>> even just by using hadoop's proxy mechanism to access HDFS as that user.
>>
>> are there plans to fix this? is there a JIRA tickets for this? i couldn't
>> find it but would like to track this issue.
>>
>> and what about secure HDFS (kerberos)?
>>
>> thanks! koert
>>
>

Mime
View raw message