spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <tathagata.das1...@gmail.com>
Subject Re: When will Spark Streaming supports Kafka-simple consumer API?
Date Thu, 05 Feb 2015 04:26:40 GMT
1. There is already a third-party low-level kafka receiver -
http://spark-packages.org/package/5
2. There is a new experimental Kafka stream that will be available in Spark
1.3 release. This is based on the low level API, and might suffice your
purpose. JIRA - https://issues.apache.org/jira/browse/SPARK-4964

Can you elaborate on why you have to use SimpleConsumer in your environment?

TD


On Wed, Feb 4, 2015 at 7:44 PM, Xuelin Cao <xuelincao2014@gmail.com> wrote:

> Hi,
>
>      In our environment, Kafka can only be used with simple consumer API,
> like storm spout does.
>
>      And, also, I found there are suggestions that " Kafka connector of
> Spark should not be used in production
> <http://markmail.org/message/2lb776ta5sq6lgtw> because it is based on the
> high-level consumer API of Kafka."
>
>     So, my question is, when will spark streaming supports Kafka simple
> consumer API?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message