kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Eichner <m...@sebastian-eichner.de>
Subject Kafka+Avro+Hadoop
Date Mon, 13 Feb 2012 09:18:57 GMT

We want to use Kafka to transport Avro messages, store them in
Avro-Format on HDFS and process then with Hadoop.

So far i have the Hadoop-Incremental-Consumer from Felix and a simple
Avro-Producer working. My next step would be to make the
Hadoop-Consumer receive Binary Avro and store it in files, but before
diving into this i wanted to ask:

Is there any other code for this scenario already available? So far i
could not find anything from the list archives and google. In the
archives i read that LinkedIn does something similiar but afaik it's
not yet released.


View raw message