spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jahagirdar, Madhu" <>
Subject Dstream Transformations
Date Sat, 04 Oct 2014 23:49:37 GMT
In my spark streaming program I have created kafka utils to receive data and store data in
elastic search and in flume. Storing function is applied on same dstream. My question what
is the behavior of spark if after storing data in elastic search the worker node dies before
storing in flume? Does it  restart worker and then again store the data in elastic search
and then flume or does it only run functions to store in flume.

Madhu Jahagirdar

The information contained in this message may be confidential and legally protected under
applicable law. The message is intended solely for the addressee(s). If you are not the intended
recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction
of this message is strictly prohibited and may be unlawful. If you are not the intended recipient,
please contact the sender by return e-mail and destroy all copies of the original message.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message