spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Maelstrom: Kafka integration with Spark
Date Wed, 24 Aug 2016 01:49:40 GMT
Were you aware that the spark 2.0 / kafka 0.10 integration also reuses
kafka consumer instances on the executors?

On Tue, Aug 23, 2016 at 3:19 PM, Jeoffrey Lim <jeoffreyl@gmail.com> wrote:
> Hi,
>
> I have released the first version of a new Kafka integration with Spark
> that we use in the company I work for: open sourced and named Maelstrom.
>
> It is unique compared to other solutions out there as it reuses the
> Kafka Consumer connection to achieve sub-milliseconds latency.
>
> This library has been running stable in production environment and has
> been proven to be resilient to numerous production issues.
>
>
> Please check out the project's page in github:
>
> https://github.com/jeoffreylim/maelstrom
>
>
> Contributors welcome!
>
>
> Cheers!
>
> Jeoffrey Lim
>
>
> P.S. I am also looking for a job opportunity, please look me up at Linked In

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message