spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kelly, Jonathan" <>
Subject Re: kinesis multiple records adding into stream
Date Fri, 16 Jan 2015 18:05:39 GMT
Are you referring to the PutRecords method, which was added in 1.9.9?  (See
 If so, can't you just depend upon this later version of the SDK in your app even though spark-streaming-kinesis-asl
is depending upon this earlier 1.9.3 version that does not yet have the PutRecords call?

Also, could you please explain your use case more fully?  spark-streaming-kinesis-asl is for
*reading* data from Kinesis in Spark, not for writing data, so I would expect that the part
of your code that would be writing to Kinesis would be a totally separate app anyway (unless
you are reading from Kinesis using spark-streaming-kinesis-asl, transforming it somehow, then
writing it back out to Kinesis).

~ Jonathan

From: Aniket Bhatnagar <<>>
Date: Friday, January 16, 2015 at 9:13 AM
To: Hafiz Mujadid <<>>,
"<>" <<>>
Subject: Re: kinesis multiple records adding into stream

Sorry. I couldn't understand the issue. Are you trying to send data to kinesis from a spark
batch/real time job?

- Aniket

On Fri, Jan 16, 2015, 9:40 PM Hafiz Mujadid <<>>
Hi Experts!

I am using kinesis dependency as follow
groupId = org.apache.spark
 artifactId = spark-streaming-kinesis-asl_2.10
 version = 1.2.0

in this aws sdk version 1.8.3 is being used. in this sdk multiple records
can not be put in a single request. is it possible to put multiple records
in a single request ?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:<>
For additional commands, e-mail:<>

View raw message