flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LINZ, Arnaud" <AL...@bouyguestelecom.fr>
Subject RE: How to stop job through java API
Date Mon, 03 Oct 2016 08:31:21 GMT

I have a similar issue. Here is how I deal with programmatically stopping permanent streaming
jobs, and I’m interested in knowing if there is a better way now.

Currently, I use hand-made streaming sources that periodically check for some flag and end
if a stop request was made. Stopping the source allow the on-going streaming jobs to finish
That way, by using a “live file” presence check in my sources, and creating a “stopped
ok file” when the streaming execution ends ok, I’m able to make  “a stopping shell script”
that notify the streaming chain of a user cancellation by deleting the “live file” and
checking for the “stopped ok file” in a waiting loop before ending.

This stopping shell script allow me to automate “stop/start” scripts when I need to perform
some task (like a reference table refresh) that requires the streaming job to be stopped (without
cancelling it from the rest interface, so without relying on the snapshot system to prevent
the loss of intermediate data, and allowing me to implement clean-up code if needed).

The drawback of this method is not being able to use “off-the-shelf” sources directly,
I always have to wrap or patch them to implement the live file check.

Best regards,

De : Aljoscha Krettek [mailto:aljoscha@apache.org]
Envoyé : mercredi 21 septembre 2016 14:54
À : user@flink.apache.org; Max Michels <mxm@apache.org>
Objet : Re: How to stop job through java API

right now this is not possible, I'm afraid.

I'm looping in Max who has done some work in that direction. Maybe he's got something to say.


On Wed, 14 Sep 2016 at 03:54 Will Du <willddy@gmail.com<mailto:willddy@gmail.com>>
Hi folks,
How to stop link job given job_id through java API?


L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice
ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation
ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message,
merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent
this message cannot therefore be held liable for its content nor attachments. Any unauthorized
use or dissemination is prohibited. If you are not the intended recipient of this message,
then please delete it and notify the sender.
View raw message