spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dibyendu Bhattacharya <dibyendu.bhattach...@gmail.com>
Subject Re: Low Level Kafka Consumer for Spark
Date Fri, 16 Jan 2015 06:20:50 GMT
Hi Kidong,

No , I have not tried yet with Spark 1.2 yet. I will try this out and let
you know how this goes.

By the way, is there any change in Receiver Store method happened in Spark
1.2 ?



Regards,
Dibyendu



On Fri, Jan 16, 2015 at 11:25 AM, mykidong <mykidong@gmail.com> wrote:

> Hi Dibyendu,
>
> I am using kafka 0.8.1.1 and spark 1.2.0.
> After modifying these version of your pom, I have rebuilt your codes.
> But I have not got any messages from ssc.receiverStream(new
> KafkaReceiver(_props, i)).
>
> I have found, in your codes, all the messages are retrieved correctly, but
> _receiver.store(_dataBuffer.iterator())  which is spark streaming abstract
> class's method does not seem to work correctly.
>
> Have you tried running your spark streaming kafka consumer with kafka
> 0.8.1.1 and spark 1.2.0 ?
>
> - Kidong.
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Low-Level-Kafka-Consumer-for-Spark-tp11258p21180.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message