spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Zhou <zhou10...@gmail.com>
Subject Question on streaming job wait and re-run
Date Fri, 06 Sep 2019 21:07:19 GMT
Hi,
My streaming job consumes data from kafka and writes them into Cassandra.

Current status:
Cassandra is not stable. Streaming job crashed when it can't write data
into Cassandra. Streaming job has check point. Usually, the Cassandra
cluster will come back in 4 hours. Finally, I start the streaming job
again, and it will start from check point and run.

Expectation:
Streaming job can enter a status like thread wait, not crash, and it will
be notified automatically when Cassandra comes back.

Does Streaming job support this feature? I searched by Google, can't find
related solutions.

Thanks.
David

Mime
View raw message