kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin <co...@clark.ws>
Subject Re: Resilient Producer
Date Wed, 28 Jan 2015 18:51:26 GMT
Logstash

--
Colin Clark 
+1 612 859 6129
Skype colin.p.clark

> On Jan 28, 2015, at 10:47 AM, Gwen Shapira <gshapira@cloudera.com> wrote:
> 
> It sounds like you are describing Flume, with SpoolingDirectory source
> (or exec source running tail) and Kafka channel.
> 
>> On Wed, Jan 28, 2015 at 10:39 AM, Fernando O. <fotero@gmail.com> wrote:
>> Hi all,
>>    I'm evaluating using Kafka.
>> 
>> I liked this thing of Facebook scribe that you log to your own machine and
>> then there's a separate process that forwards messages to the central
>> logger.
>> 
>> With Kafka it seems that I have to embed the publisher in my app, and deal
>> with any communication problem managing that on the producer side.
>> 
>> I googled quite a bit trying to find a project that would basically use
>> daemon that parses a log file and send the lines to the Kafka cluster
>> (something like a tail file.log but instead of redirecting the output to
>> the console: send it to kafka)
>> 
>> Does anyone knows about something like that?
>> 
>> 
>> Thanks!
>> Fernando.

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message