kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajini Sivaram <rajinisiva...@gmail.com>
Subject Re: Kafka SSL encryption plus external CA
Date Wed, 21 Dec 2016 10:46:20 GMT
Stephane,

I believe that should work, though I haven't tried it myself.

On Wed, Dec 21, 2016 at 12:11 AM, Stephane Maarek <
stephane@simplemachines.com.au> wrote:

> Thanks Rajini.
>
> I used a CNAME broker-bootstrap-A.example.com that round robins to the
> actual brokers broker-1.example.com, broker-2.example.com (etc etc).
> Therefore no brokers advertises the bootstrap DNS name we’re using. Is
> that an issue? The SSL certificate wildcard will match both boostrap CNAME
> and advertised hostnames
>
> We basically have the CNAME in order to cover all the brokers only using 3
> DNS records, but the bootstrap CNAME is never advertised by any of the
> broker. Is it an issue?
>
> Kind regards,
> Stephane
>
> [image: Simple Machines]
>
> *Stephane Maarek* | Developer
>
> +61 416 575 980 <+61%20416%20575%20980>
> stephane@simplemachines.com.au
> simplemachines.com.au
> Level 2, 145 William Street, Sydney NSW 2010
>
> On 21 December 2016 at 12:22:54 am, Rajini Sivaram (
> rajinisivaram@gmail.com) wrote:
>
> Stephane,
>
> Bootstrap brokers are also verified by the client in exactly the same way,
> so they should also match the wildcard of their certificate. Basically,
> clients need to make a secure SSL connection to one of the bootstrap
> brokers to obtain advertised hostnames of brokers, so they need to complete
> hostname verification of the bootstrap brokers.
>
>
> On Tue, Dec 20, 2016 at 12:21 AM, Stephane Maarek <
> stephane@simplemachines.com.au> wrote:
>
>> Thanks Rajini!
>>
>> Also, I currently have each broker advertising as broker1.mydomain.com,
>> broker2.mydomain.com broker6.mydomain.com etc…
>> I have setup CNAME with round robin fashion to group brokers by
>> availability zone i.e. broker-a.mydomain.com broker-b.mydomain.com
>> broker-c.mydomain.com. I use them for setting up the bootstrap such as I
>> got high resiliency and don’t need to change the client code if I had or
>> remove or change brokers.
>>
>> Do I need the bootstrap servers to match the wildcard of the certificate,
>> or is the SSL verification happening after we get the advertised hostnames
>> from the brokers?
>>
>> Kind regards,
>> Stephane
>>
>> [image: Simple Machines]
>>
>> *Stephane Maarek* | Developer
>>
>> +61 416 575 980 <+61%20416%20575%20980>
>> stephane@simplemachines.com.au
>> simplemachines.com.au
>> Level 2, 145 William Street, Sydney NSW 2010
>>
>> On 20 December 2016 at 4:27:28 am, Rajini Sivaram (
>> rajinisivaram@gmail.com) wrote:
>>
>> Stephane,
>>
>> If you are using a trusted CA like Verisign, clients don't need to specify
>> a truststore. The host names specified in advertised.listeners in the
>> broker must match the wildcard DNS names in the certificates if clients
>> configure ssl.endpoint.identification.algorithm=https. If
>> ssl.endpoint.identification.algorithm is not specified, by default
>> hostname
>> is not validated. It should be set to https however to prevent
>> man-in-the-middle attacks. There is an open JIRA to make this the default
>> in Kafka.
>>
>> It makes sense to enable SSL in dev and prod to ensure that the code path
>> being run in dev is the same as in prod.
>>
>>
>>
>> On Mon, Dec 19, 2016 at 3:50 AM, Stephane Maarek <
>> stephane@simplemachines.com.au> wrote:
>>
>> > Hi,
>> >
>> > I have read the docs extensively but yet there are a few answers I can’t
>> > find. It has to do with external CA
>> > Please confirm my understanding if possible:
>> >
>> > I can create my own CA to sign all the brokers and clients certificates.
>> > Pros:
>> > - cheap, easy, automated. I need to find a way to access that CA
>> > programatically for new brokers if I want to automated their deployment,
>> > but I could use something like credstash or vault for that.
>> > Cons:
>> > - all of my clients needs to trust the CA. That means somehow find a way
>> > for my clients to get access to the CA using ca-cert and add it to their
>> > truststore… correct?
>> >
>> > I don’t really like the fact that I need to provide the CA cert file to
>> > every client. That seems quite hard to achieve, and prevents my users
>> from
>> > using the Kafka cluster directly. What’s the best way for the Kafka
>> clients
>> > to get access to the CA, while my users are doing dev, etc? Most of our
>> > applications run in Docker, which means we usually pass stuff around
>> using
>> > environment variables.
>> >
>> >
>> > My next idea was to use an external CA (like Verisign) to sign my
>> > certificate with a wildcard *.kafka.mydomain.com (A records pointing to
>> > internal IPs - the DNS name would be the advertised kafka hostname). My
>> > goal was then for the clients not to require to trust the CA because it
>> > would be automatically trusted? Do I have the correct understanding? Or
>> do
>> > I still need to add the external CA to the truststore of my clients?
>> > (basically I’m trying to reproduce the behaviour of what a web browser
>> > does).
>> >
>> >
>> > Finally, is it recommended to enable SSL in my dev Kafka cluster vs my
>> prod
>> > Kafka cluster, or to have SSL on each cluster?
>> >
>> > Thanks!
>> >
>> > Kind regards,
>> > Stephane
>> >
>>
>>
>>
>> --
>> Regards,
>>
>> Rajini
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message