spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sung Hwan Chung <>
Subject Executor shutdown hooks?
Date Wed, 06 Apr 2016 19:24:18 GMT

I'm looking for ways to add shutdown hooks to executors : i.e., when a Job
is forcefully terminated before it finishes.

The scenario goes likes this : executors are running a long running job
within a 'map' function. The user decides to terminate the job, then the
mappers should perform some cleanups before going offline.

What would be the best way to do this?

View raw message