kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Kafka 0.8/VIP/SSL
Date Wed, 25 Jun 2014 18:22:39 GMT
One possible issue: the brokers need to talk directly to each other,
broker-to-broker, right? And they will try to talk to each other via the
VIP endpoints (vip1a, vip2a)?

The brokers communicate with each other and they use the
advertised.host.name for the same. So you will need to ensure that is
possible in your setup.

Thanks,
Neha


On Tue, Jun 24, 2014 at 3:35 PM, Reiner Stach <reiner.stach@gmail.com>
wrote:

> I'm looking for advice on running Kafka 0.8 behind VIPs. The goal is to
> support SSL traffic, with encryption and decryption being performed by
> back-to-back VIPs at the client and in front of the broker. That is:
>
> Kafka client
>     --> vip1a.myco.com:8080 (SSL encrypt) --- WAN ---> VIP 1b (SSL
> decrypt)
> --> Kafka broker 1
>     --> vip2a.myco.com:8080 (SSL encrypt) --- WAN ---> VIP 2b (SSL
> decrypt)
> --> Kafka broker 2
>
> Note there is a separate VIP pipeline per broker, no multiplexing involved.
> We don't care about ordering of messages within partitions or elsewhere.
>
> I've configured each broker to register in ZK and advertise itself as being
> the VIP endpoint:
> broker 1:
>     advertised.host.name=vip1a.myco.com
>     advertised.port=8080
> broker 2:
>     advertised.host.name=vip2a.myco.com
>     advertised.port=8080
>
> Should this work? Any issues?
>
> And since it *isn't* working for me yet, any idea why I might consistently
> see a LeaderNotAvailableException when trying to publish or consume?
>
> One possible issue: the brokers need to talk directly to each other,
> broker-to-broker, right? And they will try to talk to each other via the
> VIP endpoints (vip1a, vip2a)? Seems like I will need to either make the VIP
> address routeable from the Kafka broker hosts to allow the broker-to-broker
> comms, or else hack the Kafka code so the brokers present the advertised
> host/port in response to metadata requests from clients - but talk to each
> other on their 'real' addresses.
>
> Many thanks
> Reiner
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message