spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Templeton <>
Subject ATSv2 Integration
Date Tue, 21 Apr 2020 20:57:24 GMT
Hi!  I see that SPARK-1537 is now closed as won't fix.  What was the 
reason the Spark community decided to skip ATSv2 support?

My org is currently looking at how we introduce Spark into YARN clusters 
that are primarily used for Scalding today.  For the Scalding UI we use 
an internal tool that's backed by ATSv2.  It seems like it would make 
good sense for us to have Spark also storing metrics in ATSv2, rather 
than maintaining two separate metrics stores, which would also make it 
easier for us to have a single common UI.

If we were willing to take up the work to integrate Spark with ATSv2, 
would the community be willing to support that effort?


To unsubscribe e-mail:

View raw message