kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tauzell, Dave" <Dave.Tauz...@surescripts.com>
Subject RE: Best practice for publishing byte messages to Kafka
Date Thu, 11 Jan 2018 15:16:48 GMT
Whatever you use I recommend some sort of wrapper since Kafka doesn't support any sort of metadata
(like the version of the serialization format).
-Dave

-----Original Message-----
From: Matt Farmer [mailto:matt@frmr.me]
Sent: Thursday, January 11, 2018 8:56 AM
To: users@kafka.apache.org
Subject: Re: Best practice for publishing byte messages to Kafka

We use Thrift and Kryo, and we haven’t done any formal analysis recently. Performance numbers
for those should be easy to find by googling around.

I can say that we push tens of thousands of messages per second at peak and serialization
hasn’t been a cause of production lag in the entire year that I’ve been working on this
application. That’s an imprecise analysis, but I would readily recommend either library
in a case where performance is your only concern. (Though, Avro may prove to be a better choice
than Thrift these days.)

> On Jan 9, 2018, at 11:17 PM, Ali Nazemian <alinazemian@gmail.com> wrote:
>
> Thanks, Matt. Have you done any benchmarking to see how using
> different Serializers may impact throughput/latency?
>
> Regards,
> Ali
>
> On Wed, Jan 10, 2018 at 7:55 AM, Matt Farmer <matt@frmr.me> wrote:
>
>> We use the default byte array serializer provided with Kafka and it
>> works great for us.
>>
>>> On Jan 9, 2018, at 8:12 AM, Ali Nazemian <alinazemian@gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> I was wondering whether there is any best practice/recommendation
>>> for publishing byte messages to Kafka. Is there any specific
>>> Serializer that
>> is
>>> recommended for this matter?
>>>
>>> Cheers,
>>> Ali
>>
>>
>
>
> --
> A.Nazemian

This e-mail and any files transmitted with it are confidential, may contain sensitive information,
and are intended solely for the use of the individual or entity to whom they are addressed.
If you have received this e-mail in error, please notify the sender by reply e-mail immediately
and destroy all copies of the e-mail and any attachments.
Mime
View raw message