kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Borowiecki <michal.borowie...@openbet.com>
Subject Re: Zookeeper on same server as Kafka
Date Sat, 03 Jun 2017 07:20:32 GMT
I'm not an expert but I prefer keeping zookeepers on the same hosts as 
kafka brokers and mimic each-others topology. The reason is to minimize 
the chance of e.g. kafka brokers being able to talk to one another but 
zookeepers not, or vice-versa. So, I'd say I /do/ want my kafka broker 
and the co-located zookeeper to go down together - for simplicity - 
prefer that to some asymmetric failures to debug. This comes from past 
experience, albeit using other technologies, when relying on 2 different 
clustering mechanism made failures in one but not the other very 
difficult to debug.

Also, I think I read this advice somewhere a long time ago (don't recall 
where) and it made sense to me (given the prior experience) and we've 
never tried a different arrangement.

As to the overheads, I believe it's mostly disk IO and can hopefully be 
addressed by separate disks for each but it's never been a bottleneck 
for us, so can't really say.

Thanks,

MichaƂ


On 02/06/17 21:47, Mohammed Manna wrote:
> Usually, the overhead comes when you have kafka and zookeeper doing the
> housekeeping (i.e. Disk IO) on the same server. ZK even suggests that you
> should keep their logs on totally different physical machine for better
> performance. Furthermore, if a mechanical failure occurs, you might not
> want both zookeeper and broker going down together.
>
> Can anyone else chime in for some better points?
>
>
> On 2 Jun 2017 7:57 pm, "Meghana Narasimhan" <mnarasimhan@bandwidth.com>
> wrote:
>
> Hi,
> What are the pros and cons of setting up Zookeeper on the same server as
> the Kafka broker ? Earlier offsets were being written to zookeeper which
> was a major overhead but with offsets being written to Kafka now, what
> other requirements should be taken into consideration for setting up
> Zookeeper on the same server as Kafka vs having a separate zookeeper
> cluster ?
>
> Thanks,
> Meghana
>

-- 
Signature
<http://www.openbet.com/> 	Michal Borowiecki
Senior Software Engineer L4
	T: 	+44 208 742 1600

	
	+44 203 249 8448

	
	
	E: 	michal.borowiecki@openbet.com
	W: 	www.openbet.com <http://www.openbet.com/>

	
	OpenBet Ltd

	Chiswick Park Building 9

	566 Chiswick High Rd

	London

	W4 5XT

	UK

	
<https://www.openbet.com/email_promo>

This message is confidential and intended only for the addressee. If you 
have received this message in error, please immediately notify the 
postmaster@openbet.com <mailto:postmaster@openbet.com> and delete it 
from your system as well as any copies. The content of e-mails as well 
as traffic data may be monitored by OpenBet for employment and security 
purposes. To protect the environment please do not print this e-mail 
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company 
registered in England and Wales. Registered no. 3134634. VAT no. 
GB927523612


Mime
View raw message