spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Fregly <ch...@fregly.com>
Subject Re: Spark Streaming source from Amazon Kinesis
Date Tue, 22 Jul 2014 16:30:11 GMT
i took this over from parviz.

i recently submitted a new PR for Kinesis Spark Streaming support:
https://github.com/apache/spark/pull/1434

others have tested it with good success, so give it a whirl!

waiting for it to be reviewed/merged.  please put any feedback into the PR
directly.

thanks!

-chris


On Mon, Apr 21, 2014 at 2:39 PM, Matei Zaharia <matei.zaharia@gmail.com>
wrote:

> No worries, looking forward to it!
>
> Matei
>
> On Apr 21, 2014, at 1:59 PM, Parviz Deyhim <pdeyhim@gmail.com> wrote:
>
> sorry Matei. Will definitely start working on making the changes soon :)
>
>
> On Mon, Apr 21, 2014 at 1:10 PM, Matei Zaharia <matei.zaharia@gmail.com>
> wrote:
>
>> There was a patch posted a few weeks ago (
>> https://github.com/apache/spark/pull/223), but it needs a few changes in
>> packaging because it uses a license that isn’t fully compatible with
>> Apache. I’d like to get this merged when the changes are made though — it
>> would be a good input source to support.
>>
>> Matei
>>
>>
>> On Apr 21, 2014, at 1:00 PM, Nicholas Chammas <nicholas.chammas@gmail.com>
>> wrote:
>>
>> I'm looking to start experimenting with Spark Streaming, and I'd like to
>> use Amazon Kinesis <https://aws.amazon.com/kinesis/> as my data source.
>> Looking at the list of supported Spark Streaming sources
>> <http://spark.apache.org/docs/latest/streaming-programming-guide.html#linking>,
>> I don't see any mention of Kinesis.
>>
>> Is it possible to use Spark Streaming with Amazon Kinesis? If not, are
>> there plans to add such support in the future?
>>
>> Nick
>>
>>
>> ------------------------------
>> View this message in context: Spark Streaming source from Amazon Kinesis
>> <http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-source-from-Amazon-Kinesis-tp4550.html>
>> Sent from the Apache Spark User List mailing list archive
>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com
>> <http://nabble.com/>.
>>
>>
>>
>
>

Mime
View raw message