kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsha <ka...@harsha.io>
Subject Re: Securing Multi-Node single broker kafka instance
Date Wed, 01 Mar 2017 21:33:47 GMT
Here is the recommended way to setup a 3-node Kafka cluster. Its always recommended to keep
zookeeper nodes on different set of nodes than the one you are running Kafka. To go with your
current 3-node installation.
1. Install 3-node zookeeper make sure they are forming the quorum (https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html)
2. Install apache kafka binaries on all 3 nodes.
3. Make sure you keep the same zookeeper.connect in server.properties on all 3 nodes for your
kafka broker.
4. Start Kafka brokers
5. For sanity check, make sure you create a topic with 3-replication factor and see if you
can produce & consume messages

Before stepping into security make sure your non-secure Kafka cluster works ok. Once you’ve
a stable & working cluster
follow instructions in the doc to enable SSL.

-Harsha

On Mar 1, 2017, 1:08 PM -0800, IT Consultant <0binarybuddha1@gmail.com>, wrote:
> Hi Harsha ,
>
> Thanks a lot .
>
> Let me explain where am i stuck ,
>
> i have three machines on which i am running apache kafka with single broker
> but zookeeper of each machine is configured with other machine.
>
> Example : node1=zk1,zk2,zk3
> node2=zk1,zk2,zk3
> node3=zk1,zk2,zk3
>
> This is done for HA .
>
> Now i need to secure this deployment using SSL .
>
> *Things tried so far :*
>
> Create a key and certificate for each of these nodes and configure broker
> according to the documentation .
>
> However , i see following error when i run console producer and consumer
> with client certificate or client properties file .
>
> WARN Error while fetching metadata for topic
>
>
> How do i make each broker work with other broker ?
> How do i generate and store certificate for this ? because online document
> seems to be confusing for me.
> How do i make zookeepers sync with each other and behave as earlier ?
>
>
>
> On Thu, Mar 2, 2017 at 2:25 AM, Harsha Chintalapani <kafka@harsha.io> wrote:
>
> > For inter broker communication over SSL all you need is to add
> > security.inter.broker.protocol to SSL.
> > "How do i make zookeeper talk to each other and brokers?"
> > Not sure I understand the question. You need to make sure zookeeper hosts
> > and port are reachable from your broker nodes.
> > -Harsha
> >
> > On Wed, Mar 1, 2017 at 12:45 PM IT Consultant <0binarybuddha1@gmail.com
> > wrote:
> >
> > > Hi Team ,
> > >
> > > Can you please help me understand ,
> > >
> > > 1. How can i secure multi-node (3 machine) single broker (1 broker )
> > Apache
> > > Kafka deployment secure using SSL ?
> > >
> > > i tried to follow instructions here but found pretty confusing .
> > >
> > > https://www.confluent.io/blog/apache-kafka-security-authoriz
> > > ation-authentication-encryption/
> > >
> > > http://docs.confluent.io/2.0.0/kafka/security.html
> > >
> > > Currently , i have kafka running on 3 different machines .
> > > 2. How do i make them talk to each other over SSL ?
> > > 3. How do i make zookeeper talk to each other and brokers?
> > >
> > > Requesting your help .
> > >
> > > Thanks in advance.
> > >
> >

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message