spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <tathagata.das1...@gmail.com>
Subject Re: Spark Streaming source from Amazon Kinesis
Date Wed, 23 Jul 2014 05:06:21 GMT
I will take a look at it tomorrow!

TD


On Tue, Jul 22, 2014 at 9:30 AM, Chris Fregly <chris@fregly.com> wrote:

> i took this over from parviz.
>
> i recently submitted a new PR for Kinesis Spark Streaming support:
> https://github.com/apache/spark/pull/1434
>
> others have tested it with good success, so give it a whirl!
>
> waiting for it to be reviewed/merged.  please put any feedback into the PR
> directly.
>
> thanks!
>
> -chris
>
>
> On Mon, Apr 21, 2014 at 2:39 PM, Matei Zaharia <matei.zaharia@gmail.com>
> wrote:
>
>> No worries, looking forward to it!
>>
>> Matei
>>
>> On Apr 21, 2014, at 1:59 PM, Parviz Deyhim <pdeyhim@gmail.com> wrote:
>>
>> sorry Matei. Will definitely start working on making the changes soon :)
>>
>>
>> On Mon, Apr 21, 2014 at 1:10 PM, Matei Zaharia <matei.zaharia@gmail.com>
>> wrote:
>>
>>> There was a patch posted a few weeks ago (
>>> https://github.com/apache/spark/pull/223), but it needs a few changes
>>> in packaging because it uses a license that isn’t fully compatible with
>>> Apache. I’d like to get this merged when the changes are made though — it
>>> would be a good input source to support.
>>>
>>> Matei
>>>
>>>
>>> On Apr 21, 2014, at 1:00 PM, Nicholas Chammas <
>>> nicholas.chammas@gmail.com> wrote:
>>>
>>> I'm looking to start experimenting with Spark Streaming, and I'd like to
>>> use Amazon Kinesis <https://aws.amazon.com/kinesis/> as my data source.
>>> Looking at the list of supported Spark Streaming sources
>>> <http://spark.apache.org/docs/latest/streaming-programming-guide.html#linking>,
>>> I don't see any mention of Kinesis.
>>>
>>> Is it possible to use Spark Streaming with Amazon Kinesis? If not, are
>>> there plans to add such support in the future?
>>>
>>> Nick
>>>
>>>
>>> ------------------------------
>>> View this message in context: Spark Streaming source from Amazon Kinesis
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-source-from-Amazon-Kinesis-tp4550.html>
>>> Sent from the Apache Spark User List mailing list archive
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com
>>> <http://nabble.com/>.
>>>
>>>
>>>
>>
>>
>

Mime
View raw message