kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From István <lecc...@gmail.com>
Subject Re: Sizing Cluster
Date Tue, 21 Oct 2014 18:28:46 GMT
One thing that you have to keep in mind is that moving 10T between nodes
takes long time. If you have a node failure and you need to rebuild
(resync) the data your system is going to be vulnerable against the second
node failure. You could mitigate this with using raid. I think generally
speaking 3 node clusters are better for production purposes.


On Tue, Oct 21, 2014 at 11:12 AM, Pete Wright <pwright@rubiconproject.com>

> Hi There,
>         I have a question regarding sizing disk for kafka brokers.  Let's
> say I
> have systems capable of providing 10TB of storage, and they act as Kafka
> brokers.  If I were to deploy two of these nodes, and enable replication
> in Kafka, would I actually have 10TB available for my producers to write
> to?  Is there any overhead I should be concerned with?
> I guess I am just wanting to make sure that there are not any major
> pitfalls in deploying a two-node cluster, versus say a 3-node cluster.
> Any advice or best-practices would be very helpful!
> Thanks in advance,
> -pete
> --
> Pete Wright
> Systems Architect
> Rubicon Project
> pwright@rubiconproject.com
> 310.309.9298

the sun shines for all

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message