spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: kafka pipeline exactly once semantics
Date Mon, 01 Dec 2014 03:44:44 GMT
Josh,

On Sun, Nov 30, 2014 at 10:17 PM, Josh J <joshjdevl@gmail.com> wrote:
>
> I would like to setup a Kafka pipeline whereby I write my data to a single
> topic 1, then I continue to process using spark streaming and write the
> transformed results to topic2, and finally I read the results from topic 2.
>

Not really related to your question, but you may also want to look into
Samza <http://samza.incubator.apache.org/> which was built exactly for this
kind of processing.

Tobias

Mime
View raw message