samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paris Carbone <par...@kth.se>
Subject Re: Help on issue SAMOA-40
Date Mon, 01 Feb 2016 11:07:16 GMT
Here’s a pointer to Kafka consumers/producers in Apache Flink, in case it helps

https://github.com/apache/flink/tree/master/flink-streaming-connectors

cheers
Paris

On 31 Jan 2016, at 14:32, Gianmarco De Francisci Morales <gdfm@apache.org<mailto:gdfm@apache.org>>
wrote:

Hi Tarush,
Indeed, we do not have support for Kafka yet.
It would be great to add it!

For the reading part of the code itself, the best place to look into is
other projects that use Kafka, e.g., Flink. Or even Kafka's examples.
For the adaptation to SAMOA, the best place to start from is the code for
the HSDF and local files (HDFSFileStreamSource and LocalFileStreamSource).

One tough design point is how to reuse the readers available in each
platform while keeping the API towards SAMOA unified, without needing to
rewrite all the code.

Hope it helps!

-- Gianmarco

On 19 January 2016 at 21:05, tarush grover <tarushapptech@gmail.com<mailto:tarushapptech@gmail.com>>
wrote:

Hi All,

Needed guidelines for starting on this issue. I am not seeing any code for
Kafka in the master branch.

Regards,
Tarush


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message