kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bao Thai Ngo <baothai...@gmail.com>
Subject Re: Tail or stdin producer?
Date Mon, 03 Oct 2011 06:48:32 GMT
Hi,

I am totally new to Kafka and we are evaluating Kafka like Felix. I'm
looking for a utility to pipe data from a file (or multiple files) to a
producer (like command tail -f in linux). This is an our typical use-case. I
saw the patch (KAFKA-130) you provided but not sure if this will work well
without loosing data if the file (or multiple files) will be rotated.

I would care about plugin that will such as filter message at producer;
modify message body in order to mark some critical kind of information, etc.
It would be good if you provide documentation about how to write a plugin
for Kafka.

Thanks,
~Thai

On Sun, Oct 2, 2011 at 9:51 PM, Jay Kreps <jay.kreps@gmail.com> wrote:

> Hi,
>
> Currently all the patch does is take input from stdin. Can you help me
> understand what you are looking for?
>
> What kind of plugin are you trying to write?
>
> -Jay
>
>
> On Thu, Sep 29, 2011 at 10:41 PM, Bao Thai Ngo <baothaingo@gmail.com>wrote:
>
>> Jay,
>>
>> Does the patch already support log rotation (file rolling)?
>>
>> Also, I wonder if there is any documentation about how to write a plug-in
>> for Kafka.
>>
>> Thanks,
>> ~Thai
>>
>> On Mon, Sep 12, 2011 at 10:17 AM, Jay Kreps <jay.kreps@gmail.com> wrote:
>>
>> > I added a patch for this:
>> > https://issues.apache.org/jira/browse/KAFKA-130
>> >
>> > Felix, if you are still interested in this, let me know if it works for
>> > you.
>> >
>> > -Jay
>> >
>> > On Thu, Sep 1, 2011 at 1:22 PM, Felix Giguere Villegas <
>> > felix.giguere@mate1inc.com> wrote:
>> >
>> > > Hi :)
>> > >
>> > > Thanks for the reply :) . I have created
>> > > KAFKA-130<https://issues.apache.org/jira/browse/KAFKA-130>
>> > > .
>> > >
>> > > I'm not sure I have respected the proper standard for JIRA issues.
>> Please
>> > > let me know if I haven't.
>> > >
>> > > --
>> > > Felix
>> > >
>> > >
>> > >
>> > > On Thu, Sep 1, 2011 at 4:04 PM, Jun Rao <junrao@gmail.com> wrote:
>> > >
>> > > > Hi, Felix,
>> > > >
>> > > > Currently, we don't have a util to pipe data from a file to a
>> producer.
>> > I
>> > > > agree that it's a very convenient tool to have. Could you open a
>> jira
>> > for
>> > > > that?
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jun
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message