spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "A.K.M. Ashrafuzzaman" <ashrafuzzaman...@gmail.com>
Subject Having problem with Spark streaming with Kinesis
Date Wed, 26 Nov 2014 12:23:05 GMT
Hi guys,
When we are using Kinesis with 1 shard then it works fine. But when we use more that 1 then
it falls into an infinite loop and no data is processed by the spark streaming. In the kinesis
dynamo DB, I can see that it keeps increasing the leaseCounter. But it do start processing.

I am using,
scala: 2.10.4
java version: 1.8.0_25
Spark: 1.1.0
spark-streaming-kinesis-asl: 1.1.0

A.K.M. Ashrafuzzaman
Lead Software Engineer
NewsCred

(M) 880-175-5592433
Twitter | Blog | Facebook

Check out The Academy, your #1 source
for free content marketing resources


Mime
View raw message