spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Spark Streaming 1.3 & Kafka Direct Streams
Date Wed, 01 Apr 2015 17:01:02 GMT
https://github.com/koeninger/kafka-exactly-once/blob/master/blogpost.md

The kafka consumers run in the executors.

On Wed, Apr 1, 2015 at 11:18 AM, Neelesh <neeleshs@gmail.com> wrote:

> With receivers, it was pretty obvious which code ran where - each receiver
> occupied a core and ran on the workers. However, with the new kafka direct
> input streams, its hard for me to understand where the code that's reading
> from kafka brokers runs. Does it run on the driver (I hope not), or does it
> run on workers?
>
> Any help appreciated
> thanks!
> -neelesh
>

Mime
View raw message