kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Kafka/Hadoop consumers and producers
Date Wed, 03 Jul 2013 00:01:53 GMT
We currently have a contrib package for consuming and producing messages
from mapreduce (

We keep running into problems (e.g. KAFKA-946) that are basically due to
the fact that the Kafka committers don't seem to mostly be Hadoop
developers and aren't doing a good job of maintaining this code (keeping it
tested, improving it, documenting it, writing tutorials, getting it moved
over to the more modern apis, getting it working with newer Hadoop
versions, etc).

A couple of options:
1. We could try to get someone in the Kafka community (either a current
committer or not) who would adopt this as their baby (it's not much code).
2. We could just let Camus take over this functionality. They already have
a more sophisticated consumer and the producer is pretty minimal.

So are there any people who would like to adopt the current Hadoop contrib

Conversely would it be possible to provide the same or similar
functionality in Camus and just delete these?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message