kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhavesh Mistry <mistry.p.bhav...@gmail.com>
Subject Latency Tracking Across All Kafka Component
Date Mon, 05 Jan 2015 19:43:02 GMT
Hi Kafka Team/Users,

We are using Linked-in Kafka data pipe-line end-to-end.

Producer(s) ->Local DC Brokers -> MM -> Central brokers -> Camus Job ->
HDFS

This is working out very well for us, but we need to have visibility of
latency at each layer (Local DC Brokers -> MM -> Central brokers -> Camus
Job ->  HDFS).  Our events are time-based (time event was produce).  Is
there any feature or any audit trail  mentioned at (
https://github.com/linkedin/camus/) ?  But, I would like to know in-between
latency and time event spent in each hope? So, we do not know where is
problem and what t o optimize ?

Any of this cover in 0.9.0 or any other version of upcoming Kafka release
?  How might we achive this  latency tracking across all components ?


Thanks,

Bhavesh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message