spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianluca Privitera <>
Subject Re: Spark History Server pointing to S3
Date Tue, 16 Jun 2015 15:53:49 GMT
It gives me an exception with org.apache.spark.deploy.history.FsHistoryProvider , a problem
with the file system. I can reproduce the exception if you want.
It perfectly works if I give a local path, I tested it in 1.3.0 version.


On 16 Jun 2015, at 15:08, Akhil Das <<>>

Not quiet sure, but try pointing the spark.history.fs.logDirectory to your s3

Best Regards

On Tue, Jun 16, 2015 at 6:26 PM, Gianluca Privitera <<>>
In Spark website it’s stated in the View After the Fact section (
that you can point the script to a directory in order do view the
Web UI using the logs as data source.

Is it possible to point that script to S3? Maybe from a EC2 instance?


To unsubscribe, e-mail:<>
For additional commands, e-mail:<>

View raw message