kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philip O'Toole <philip.oto...@yahoo.com.INVALID>
Subject Re: Apache webserver access logs + Kafka producer
Date Thu, 07 Aug 2014 19:01:21 GMT
Fluentd might work or simply configure rsyslog or syslog-ng on the box to watch the Apache
log files, and send them to a suitable Producer (for example I wrote something that will accept
messages from a syslog client, and stream them to Kafka.  https://github.com/otoolep/syslog-gollector)


More ideas here:

https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

  Philip
-----------------------------------------
http://www.philipotoole.com 
On Tuesday, August 5, 2014 2:48 PM, Florian Dambrine <florian@gumgum.com> wrote:
 


You might be interested by something like Logstash http://logstash.org for
logs and event processing.

Regards,

Florian

Le 5 août 2014 23:17, "Jonathan Weeks" <jonathanbweeks@gmail.com> a écrit :

> You can look at something like:
>
> https://github.com/harelba/tail2kafka
>
> (although I
 don’t know what the effort would be to update it, as it
> doesn’t look like it has been updated in a couple years)
>
> We are using flume to gather logs, and then sending them to a kafka
> cluster via a flume kafka sink — e.g..
>
> https://github.com/thilinamb/flume-ng-kafka-sink
>
> -Jonathan
>
>
> On Aug 5, 2014, at 1:40 PM, mvs.sree@gmail.com wrote:
>
> > Hi,
> >
> > I want to collect apache web server logs in real time and send it to
> Kafka
> > server. Is there any existing Producer available to do this operation, If
> > not can you please provide a way to implement it.
> >
> > Regards,
> > Sree.
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message