spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ZHANG Wei <wezh...@outlook.com>
Subject Re: Is there any way to set the location of the history for the spark-shell per session?
Date Thu, 16 Apr 2020 07:20:47 GMT
>From my understanding, you are talking about spark-shell command history, aren't you?

If yes, you can try adding `--conf 'spark.driver.extraJavaOptions=-Dscala.shell.histfile=<history
file you'd like to set>` into spark-shell command arguments since Spark shell is leveraging
Scala REPL JLine file backend history settings.

Cheers,
-z

________________________________________
From: Yeikel <email@yeikel.com>
Sent: Wednesday, April 15, 2020 2:20
To: user@spark.apache.org
Subject: Is there any way to set the location of the history for the spark-shell per session?

In my team , we get elevated access to our Spark cluster using a common
username which means that we all share the same history. I am not sure if
this is common , but unfortunately there is nothing I can do about it.

Is there any option to set the location of the history?  I am looking for
something like spark-shell --history=...path or something similar that can
be reused for other sessions.

By default , history seems to be stored in $HOME/.scala_history and I did
not see anything in the documentation about it , but maybe there is an
undocumented way to do it.

Thanks for your help!



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message