spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Zsolt Piros <api...@cloudera.com.INVALID>
Subject Re: Spark on Yarn, is it possible to manually blacklist nodes before running spark job?
Date Tue, 22 Jan 2019 10:30:08 GMT
Hi,

>> Is it this one: https://github.com/apache/spark/pull/23223 ?

No. My old development was https://github.com/apache/spark/pull/21068,
which is closed.

This would be a new improvement with a new Apache JIRA issue (
https://issues.apache.org) and with a new Github pull request.

>> Can I try to reach you through Cloudera Support portal?

It is not needed. This would be an improvement into the Apache Spark which
details can be discussed in the JIRA / Github PR.

Attila


On Mon, Jan 21, 2019 at 10:18 PM Serega Sheypak <serega.sheypak@gmail.com>
wrote:

> Hi Apiros, thanks for your reply.
>
> Is it this one: https://github.com/apache/spark/pull/23223 ?
> Can I try to reach you through Cloudera Support portal?
>
> пн, 21 янв. 2019 г. в 20:06, attilapiros <apiros@cloudera.com>:
>
>> Hello, I was working on this area last year (I have developed the
>> YarnAllocatorBlacklistTracker) and if you haven't found any solution for
>> your problem I can introduce a new config which would contain a sequence
>> of
>> always blacklisted nodes. This way blacklisting would improve a bit again
>> :)
>>
>>
>>
>> --
>> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>

Mime
View raw message