kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sybrandy, Casey" <Casey.Sybra...@Six3Systems.com>
Subject RE: Binary Data and Kafka
Date Wed, 08 May 2013 15:29:10 GMT
That's what I would have assumed.  And no, we're not using compression.


From: Jun Rao [mailto:junrao@gmail.com]
Sent: Wednesday, May 08, 2013 11:26 AM
To: users@kafka.apache.org
Cc: Sybrandy, Casey
Subject: Re: Binary Data and Kafka

No. Kafka broker stores the binary data as it is. The binary data may be compressed, if compression
is enabled at the producer.



On Wed, May 8, 2013 at 5:57 AM, Sybrandy, Casey <Casey.Sybrandy@six3systems.com<mailto:Casey.Sybrandy@six3systems.com>>

Does the Kafka broker Base64 encode the messages?  We are sending binary data to the brokers
and I looked at the logs to confirm that they data was being stored, however all of the data,
with a few exceptions, looks to be Base64 encoded.  I didn't expect this, so I wanted to ask
and confirm what I'm seeing.

If this is true, does this affect the size of the message when fetching?  In other words,
if I send a 100K message, do I have to make sure I can fetch a 300K message since the message
can now be 300K in size because of the encoding?

Casey Sybrandy MSWE
Cyber and Enterprise Systems Group
301-206-6000<tel:301-206-6000> (Office)
301-206-6020<tel:301-206-6020> (Fax)
11820 West Market Place
Suites N-P
Fulton, MD. 20759

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message