kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Quinlivan <rquinli...@signal.co>
Subject Java 8 stream consumer pattern
Date Wed, 17 May 2017 20:45:39 GMT

I've been looking at writing a Java 8 streams API wrapper for the Kafka
consumer. Since this seems like a common use case I was wondering if
someone in the user community had already begun a project like this.

My goal is to be able to get back a Stream<ConsumerRecord<K, V>> wrapping
the results of #poll() which can then be passed into a map/filter pipeline.
I am using an underlying blocking queue data structure to buffer in memory
and using Stream.generate() to pull records. Any recommendations on a best
approach here?

Robert Quinlivan
Software Engineer, Signal

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message