kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <dav...@spiceworks.com>
Subject Reactive Kafka performance
Date Thu, 28 Jul 2016 19:16:58 GMT
Our team is evaluating KStreams and Reactive Kafka (version 0.11-M4)  on a confluent 3.0 cluster.
 Our testing is very simple (pulling from one topic, doing a simple transform) and then writing
out to another topic.

The performance for the two systems is night and day. Both applications were running on a
laptop and connecting to kafka over a wifi network.  Here are the numbers:

KStreams: ~14K messages per second
Reactive Kafka: ~110 messages per second

Both the input, and output topic had 54 partitions.  I’m fairly certain I’m not using
Reactive kafka with good configuration.  Here is some stubbed out code: https://gist.github.com/anduill/2e17cd7a40d4a86fefe19870d1270f5b

One note, I am using the confluent stack (hence the CachedSchemaRegistryClient)

I like the flexibility of Reactive Kafka, so we’d really like to use it…but if performance
is going to be like this, I can’t really justify it.  I’m a scala/akka/streaming-akka
newbie, so I’m sure there are better ways to use the API.  Any help is appreciated.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message