kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Help with kafka-hadoop loader pipeline ???
Date Tue, 04 Jun 2013 14:53:52 GMT
I am not familiar with the hadoop loader that you are using. At LinkedIn,
we are using https://github.com/linkedin/camus. It has an 0.8
implementation too: https://github.com/linkedin/camus/tree/camus-kafka-0.8/

Thanks,

Jun


On Tue, Jun 4, 2013 at 3:32 AM, Chitra Raveendran <
chitra.raveendran@fluturasolutions.com> wrote:

> Hi
>
> We were trying to use the kafka-hadoop loader for loading messages from
> kafka into the hadoop ecosystem
> https://github.com/michal-harish/kafka-hadoop-loader
>
> Our consumer is hadoop in this case. The code runs saying job successfull
> and a _SUCESS file is created in hdfs.
>
> But no output file is created in the given hdfs path, we tried sending both
> json as well as protobuf files through kafka.
>
> Please help if anyone has any idea on loading protobuf files into hadoop.
>
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura Decision Sciences & Analytics*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message