spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manu Zhang <>
Subject Re: Unable to see completed application in Spark 2 history web UI
Date Wed, 15 Aug 2018 14:11:30 GMT
Hi Fawze,

A) The file permission is currently hard coded to 770 (
B) I think add all users (including UI) to the group like Spark will do.

On Wed, Aug 15, 2018 at 6:38 PM Fawze Abujaber <> wrote:

> Hi Manu,
> Thanks for your response.
> Yes, i see but still interesting to know how i can see these applications
> from the spark history UI.
> How i can know with which user i'm  logged in when i'm navigating the
> spark history UI.
> The Spark process is running with cloudera-scm and the events written in
> the spark2history folder at the HDFS written with the user name who is
> running the application and group spark (770 permissions).
> I'm interesting to see if i can force these logs to be written with 774 or
> 775 permission or finding another solutions that enable Rnd or anyone to be
> able to investigate his application logs using the UI.
> for example : can i use such spark conf : spark.eventLog.permissions=755
> The 2 options i see here:
> A) find a way to enforce these logs to be written with other permissions.
> B) Find the user that the UI running with as creating LDAP groups and user
> that can handle this.
> for example creating group called Spark and create the user that the UI
> running with and add this user to the spark group.
> not sure if this option will work as i don't know if these steps
> authenticate against the LDAP.

View raw message