spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabor Somogyi <gabor.g.somo...@gmail.com>
Subject Re: Spark on YARN, HowTo kill executor or individual task?
Date Sun, 10 Feb 2019 15:57:23 GMT
Hi Serega,

If I understand your problem correctly you would like to kill one executor
only and the rest of the app has to be untouched.
If that's true yarn -kill is not what you want because it stops the whole
application.

I've done similar thing when tested/testing Spark's HA features.
- jps -vlm | grep
"org.apache.spark.executor.CoarseGrainedExecutorBackend.*applicationid"
- kill -9 pidofoneexecutor

Be aware if it's a multi-node cluster check whether at least one process
runs on a specific node(it's not required).
Happy killing...

BR,
G


On Sun, Feb 10, 2019 at 4:19 PM Jörn Franke <jornfranke@gmail.com> wrote:

> yarn application -kill applicationid ?
>
> > Am 10.02.2019 um 13:30 schrieb Serega Sheypak <serega.sheypak@gmail.com
> >:
> >
> > Hi there!
> > I have weird issue that appears only when tasks fail at specific stage.
> I would like to imitate failure on my own.
> > The plan is to run problematic app and then kill entire executor or some
> tasks when execution reaches certain stage.
> >
> > Is it do-able?
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message