kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Kafka Hadoop Consumer for multiple brokers
Date Tue, 04 Jun 2013 15:10:34 GMT
The idea is that each mapper is only connecting to a single Kafka broker.
Each line in the input file specifies broker uri, topic, partition and
offset.

The hadoop consumer in contrib is probably a bit outdated. The one that
LinkedIn uses now can be found at https://github.com/linkedin/camus

Thanks,

Jun


On Tue, Jun 4, 2013 at 7:29 AM, Samir Madhavan <
samir.madhavan@fluturasolutions.com> wrote:

> Hi,
>
> I was going through the hadoop-consumer in the contrib folder. There is
> property that asks for the kafka server URI. This might sound silly but
> from looking at it, it seems to be only for a single kafka broker.
>
> What we have multiple brokers, how do we implement the hadoop-consumer for
> it?
>
> Regards,
> Samir
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message