kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Banias H <banias4sp...@gmail.com>
Subject Re: Suggestions of pulling local application logs into Kafka
Date Tue, 03 May 2016 20:00:40 GMT
I should add Flume is not an option for various reasons.

On Tue, May 3, 2016 at 2:49 PM, Banias H <banias4spark@gmail.com> wrote:

> We use Kafka (0.9.x) internally in our pipeline and now we would like to
> ingest application logs sitting in local file system of servers external to
> the Kafka cluster.
> We could write a producer program running on the application servers to
> push files to Kafka. However we wonder if we can leverage Kafka Connect to
> pull files into Kafka. It requires a connector to access local file systems
> and I am not sure if I can make use of existing connectors or I need to
> write one.
> Any thoughts or reference would be helpful.
> Thanks,
> B

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message