I have released the first version of a new Kafka integration with Spark 
that we use in the company I work for: open sourced and named Maelstrom.

It is unique compared to other solutions out there as it reuses the 
Kafka Consumer connection to achieve sub-milliseconds latency.

This library has been running stable in production environment and has 
been proven to be resilient to numerous production issues.

Please check out the project's page in github: 


Contributors welcome!


Jeoffrey Lim

P.S. I am also looking for a job opportunity, please look me up at Linked In