kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Latency Tracking Across All Kafka Component
Date Mon, 05 Jan 2015 23:33:15 GMT
Hi,

That sounds a bit like needing a full, cross-app, cross-network
transaction/call tracing, and not something specific or limited to Kafka,
doesn't it?

Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Jan 5, 2015 at 2:43 PM, Bhavesh Mistry <mistry.p.bhavesh@gmail.com>
wrote:

> Hi Kafka Team/Users,
>
> We are using Linked-in Kafka data pipe-line end-to-end.
>
> Producer(s) ->Local DC Brokers -> MM -> Central brokers -> Camus Job ->
> HDFS
>
> This is working out very well for us, but we need to have visibility of
> latency at each layer (Local DC Brokers -> MM -> Central brokers -> Camus
> Job ->  HDFS).  Our events are time-based (time event was produce).  Is
> there any feature or any audit trail  mentioned at (
> https://github.com/linkedin/camus/) ?  But, I would like to know
> in-between
> latency and time event spent in each hope? So, we do not know where is
> problem and what t o optimize ?
>
> Any of this cover in 0.9.0 or any other version of upcoming Kafka release
> ?  How might we achive this  latency tracking across all components ?
>
>
> Thanks,
>
> Bhavesh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message