kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From IT Consultant <0binarybudd...@gmail.com>
Subject Re: Securing Multi-Node single broker kafka instance
Date Wed, 01 Mar 2017 21:43:28 GMT
Hi Harsha ,

Just looked at the URL you shared .

I have ensured that zookeeper.properties file is same across all nodes .
Just like it's shown here .
As i stated earlier , its working well for quite sometime .

tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

Generation of a key and certificate is enough or should i do anything
on zookeeper front to make it work with

kafka brokers ?


Am i missing anything here?


On Thu, Mar 2, 2017 at 3:08 AM, IT Consultant <0binarybuddha1@gmail.com>
wrote:

> Sure Harsha . I shall follow recommended method .
>
> However , i would like to add to the discussion that current deployment
> worked just fine .
>
> People were using it for quite sometime with no security .
>
> Do i need to create topics and all again if am enabling security ?
>
> On Thu, Mar 2, 2017 at 3:03 AM, Harsha <kafka@harsha.io> wrote:
>
>> Here is the recommended way to setup a 3-node Kafka cluster. Its always
>> recommended to keep zookeeper nodes on different set of nodes than the one
>> you are running Kafka. To go with your current 3-node installation.
>> 1. Install 3-node zookeeper make sure they are forming the quorum (
>> https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html)
>> 2. Install apache kafka binaries on all 3 nodes.
>> 3. Make sure you keep the same zookeeper.connect in server.properties on
>> all 3 nodes for your kafka broker.
>> 4. Start Kafka brokers
>> 5. For sanity check, make sure you create a topic with 3-replication
>> factor and see if you can produce & consume messages
>>
>> Before stepping into security make sure your non-secure Kafka cluster
>> works ok. Once you’ve a stable & working cluster
>> follow instructions in the doc to enable SSL.
>>
>> -Harsha
>>
>> On Mar 1, 2017, 1:08 PM -0800, IT Consultant <0binarybuddha1@gmail.com>,
>> wrote:
>> > Hi Harsha ,
>> >
>> > Thanks a lot .
>> >
>> > Let me explain where am i stuck ,
>> >
>> > i have three machines on which i am running apache kafka with single
>> broker
>> > but zookeeper of each machine is configured with other machine.
>> >
>> > Example : node1=zk1,zk2,zk3
>> > node2=zk1,zk2,zk3
>> > node3=zk1,zk2,zk3
>> >
>> > This is done for HA .
>> >
>> > Now i need to secure this deployment using SSL .
>> >
>> > *Things tried so far :*
>> >
>> > Create a key and certificate for each of these nodes and configure
>> broker
>> > according to the documentation .
>> >
>> > However , i see following error when i run console producer and consumer
>> > with client certificate or client properties file .
>> >
>> > WARN Error while fetching metadata for topic
>> >
>> >
>> > How do i make each broker work with other broker ?
>> > How do i generate and store certificate for this ? because online
>> document
>> > seems to be confusing for me.
>> > How do i make zookeepers sync with each other and behave as earlier ?
>> >
>> >
>> >
>> > On Thu, Mar 2, 2017 at 2:25 AM, Harsha Chintalapani <kafka@harsha.io>
>> wrote:
>> >
>> > > For inter broker communication over SSL all you need is to add
>> > > security.inter.broker.protocol to SSL.
>> > > "How do i make zookeeper talk to each other and brokers?"
>> > > Not sure I understand the question. You need to make sure zookeeper
>> hosts
>> > > and port are reachable from your broker nodes.
>> > > -Harsha
>> > >
>> > > On Wed, Mar 1, 2017 at 12:45 PM IT Consultant <
>> 0binarybuddha1@gmail.com
>> > > wrote:
>> > >
>> > > > Hi Team ,
>> > > >
>> > > > Can you please help me understand ,
>> > > >
>> > > > 1. How can i secure multi-node (3 machine) single broker (1 broker
)
>> > > Apache
>> > > > Kafka deployment secure using SSL ?
>> > > >
>> > > > i tried to follow instructions here but found pretty confusing .
>> > > >
>> > > > https://www.confluent.io/blog/apache-kafka-security-authoriz
>> > > > ation-authentication-encryption/
>> > > >
>> > > > http://docs.confluent.io/2.0.0/kafka/security.html
>> > > >
>> > > > Currently , i have kafka running on 3 different machines .
>> > > > 2. How do i make them talk to each other over SSL ?
>> > > > 3. How do i make zookeeper talk to each other and brokers?
>> > > >
>> > > > Requesting your help .
>> > > >
>> > > > Thanks in advance.
>> > > >
>> > >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message