spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Zsolt Piros <api...@cloudera.com.INVALID>
Subject Re: Spark on Yarn, is it possible to manually blacklist nodes before running spark job?
Date Tue, 22 Jan 2019 11:25:11 GMT
The new issue is https://issues.apache.org/jira/browse/SPARK-26688.


On Tue, Jan 22, 2019 at 11:30 AM Attila Zsolt Piros <apiros@cloudera.com>
wrote:

> Hi,
>
> >> Is it this one: https://github.com/apache/spark/pull/23223 ?
>
> No. My old development was https://github.com/apache/spark/pull/21068,
> which is closed.
>
> This would be a new improvement with a new Apache JIRA issue (
> https://issues.apache.org) and with a new Github pull request.
>
> >> Can I try to reach you through Cloudera Support portal?
>
> It is not needed. This would be an improvement into the Apache Spark which
> details can be discussed in the JIRA / Github PR.
>
> Attila
>
>
> On Mon, Jan 21, 2019 at 10:18 PM Serega Sheypak <serega.sheypak@gmail.com>
> wrote:
>
>> Hi Apiros, thanks for your reply.
>>
>> Is it this one: https://github.com/apache/spark/pull/23223 ?
>> Can I try to reach you through Cloudera Support portal?
>>
>> пн, 21 янв. 2019 г. в 20:06, attilapiros <apiros@cloudera.com>:
>>
>>> Hello, I was working on this area last year (I have developed the
>>> YarnAllocatorBlacklistTracker) and if you haven't found any solution for
>>> your problem I can introduce a new config which would contain a sequence
>>> of
>>> always blacklisted nodes. This way blacklisting would improve a bit
>>> again :)
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>
>>>

Mime
View raw message