spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Hemnani <>
Subject Re: How to use FlumeInputDStream in spark cluster?
Date Fri, 21 Mar 2014 13:46:15 GMT
I'll start with Kafka implementation.

Thanks for all the help.
On Mar 21, 2014 7:00 PM, "anoldbrain [via Apache Spark User List]" <> wrote:

> It is my understanding that there is no way to make FlumeInputDStream work
> in a cluster environment with the current release. Switch to Kafka, if you
> can, would be my suggestion, although I have not used KafkaInputDStream.
> There is a big difference between Kafka and Flume InputDstream:
> KafkaInputDStreams are consumers (clients). FlumeInputDStream, which needs
> to listen on a specific address:port so other flume agent can send messages
> to. This may also give Kafka an advantage on performance too.
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>  To unsubscribe from How to use FlumeInputDStream in spark cluster?, click
> here<>
> .
> NAML<>

View this message in context:
Sent from the Apache Spark User List mailing list archive at
View raw message