kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Wilkinson <aa...@modopayments.com>
Subject Massive SSL performance degredation
Date Thu, 17 Nov 2016 23:56:09 GMT
Pardon if this is a oft repeated issue, but all the information I could
find said I should expect a 20-50% performance hit when using SSL with
kafka, and I am seeing closer to 2000-3000%

I'm trying to get kafka to behave like a fast, secured message bus.  So I
am sending small messages, one at a time.  I have set up a simple, 2
machine experiment in AWS with 1 client machine and 1 zookeeper/broker
machine and I'm an running a very linear test.

There are 2 topics: "request" and "response" and 2 threads on the client
machine each of which connects to those 2 topics.  Thread 1 produces a
"request", thread 2 consumes it and then produces a "response" which thread
1 then consumes.  At that point thread 1 proceeds to send the next
"request" and the process repeats.

So there are a total of 4 connections to the broker.

I can run a sustained test without SSL and see 1 to 1.5 ms per message hop
(where a "hop" means the message has traveled across 1 of the 4
connections- either a production or a consumption of either the request or
the response).

Each connection for which I turn on SSL increases the hop time 35 to 45 ms.

Now, the problem could be with the stack I'm using (PHP 7 talking to the
broker via the librdkafka C library).  But before I go about trying to
reproduce this with a java client (which is not my forte) I was wondering
if anyone else has run into a similar issue either with PHP or any other
language / library.  Or does anyone know a direct way to figure out whether
this slow down is at the broker or at the client?

Thanks in advance for your help!

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message