spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priya Matpadi <pmatp...@gmail.com>
Subject Re: How to update structured streaming apps gracefully
Date Tue, 18 Dec 2018 17:29:08 GMT
Using checkpointing for graceful updates is my understanding as well, based
on the writeup in
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing,
and some prototyping. Have you faced any missed events?

On Mon, Dec 17, 2018 at 6:56 PM Yuta Morisawa <yu-morisawa@kddi-research.jp>
wrote:

> Hi
>
> Now I'm trying to update my structured streaming application.
> But I have no idea how to update it gracefully.
>
> Should I stop it, replace a jar file then restart it?
> In my understanding, in that case, all the state will be recovered if I
> use checkpoints.
> Is this correct?
>
> Thank you,
>
>
> --
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message