kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Backhouse <BackhouseAnd...@gmail.com>
Subject Kafka - Networking considerations
Date Fri, 06 May 2016 11:07:42 GMT

I trust you are well.

There's a wealth of great articles and presentations relating to the Kafka
logic and design, thank you for these. But, I'm unable to find information
relating to the network infrastructure that underpins Kafka. What I am
trying to understand and put my Network Engineers at ease when it comes to
sizing our Kafka cluster is as follows;

Looking at an example deployment e.g. LinkedIn and their Kafka deployment,
it processes 18.5Gbit/sec inbound.

What design considerations were made to;

   1. validate the network can support such throughput (inbound,
   inter-cluster & outbound)
   2. identify what network infrastructure config/layout can be used or
   avoided to optimise for this level of traffic

To help your answer, we're looking at potentially 16GBit/sec inbound which
concerns our network team.

If you can please share pointers to existing materials or specific details
of your deployment, that will be great.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message