spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Rovner <alex.rov...@magnetic.com>
Subject Re: [Spark on YARN] Multiple Auxiliary Shuffle Service Versions
Date Wed, 07 Oct 2015 00:27:21 GMT
Thank you all for your help.

*Alex Rovner*
*Director, Data Engineering *
*o:* 646.759.0052

* <http://www.magnetic.com/>*

On Tue, Oct 6, 2015 at 11:17 AM, Steve Loughran <stevel@hortonworks.com>
wrote:

>
> On 6 Oct 2015, at 01:23, Andrew Or <andrew@databricks.com> wrote:
>
> Both the history server and the shuffle service are backward compatible,
> but not forward compatible. This means as long as you have the latest
> version of history server / shuffle service running in your cluster then
> you're fine (you don't need multiple of them).
>
>
> FWIW I've just created a JIRA on tracking/reporting version mismatch on
> history server playback better:
> https://issues.apache.org/jira/browse/SPARK-10950
>
> Even though the UI can't be expected to playback later histories, it could
> be possible to report the issue in a way that users can act on "run a later
> version", rather than raise support calls.
>
>

Mime
View raw message