kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liam Clarke-Hutchinson <liam.cla...@adscale.co.nz>
Subject Re: Send several Serialized Java objects under one Kafka Topic
Date Sun, 31 Jan 2021 22:32:26 GMT
Hi Peter,

The biggest issue with Java serialization has always been compatibility
between versions of the classes between producer and consumer, it can be
avoided via very careful management, but it's a lot more painful (IMO) than
other serialisation formats.

If you're looking to avoid JSON, I recommend Avro and/or Protobuf, as the
Schema Registry developed by Confluent and associated Kafka producer
serializers/ consumer deserializers  explicitly support JSON, Avro and
Protobuf, and (once again, in my opinion) using the schema registry really
takes the pain out of schema version compatibility management.

Kind regards,

Liam Clarke-Hutchinson

On Mon, 1 Feb. 2021, 3:49 am Peter Penzov, <peter.penzov@gmail.com> wrote:

> Hello All,
>      I'm working on proof of concept for sending several Java Objects
> under one Kafka Topic. More about the requirements:
> https://stackoverflow.com/questions/65811681/design-kafka-consumers-and-producers-for-scalability
> I managed to implement this working concept:
> https://github.com/rcbandit111/skyobject_engine
> In this code example I send several Java Objects under one Kafka Topic
> and response is sent back into another Kafka Topic.
> I would like to ask you what are the advantages and disadvantages in
> this design of sending data between Consumer and Producer? I want to
> use a serialized Java object, not JSON because I think it's more
> optimized for performance.
> Can you share your feedback, please?
> Also is there some better way to implement this? Like for example
> using Streaming API?
> BR,
> Peter

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message