kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evan Chan ...@ooyala.com>
Subject Re: Kafka and Binary Data
Date Thu, 01 Dec 2011 21:11:32 GMT
A couple quick questions:

1) What prevents one from changing the V type in the Producer API to any
arbitrary type I want, such as String, or my own Thrift struct, for example?
(And I suppose one can define custom serializers to serialize any type)

2) When will the docs be updated to fit 0.7?   I don't see where the docs
document the [K, V] typing, for example.

thanks,
Evan

On Thu, Dec 1, 2011 at 12:10 PM, Sybrandy, Casey <
CaseySybrandy@noviidesign.com> wrote:

> Perfect, thanks!
>
> ________________________________________
> From: Neha Narkhede [neha.narkhede@gmail.com]
> Sent: Thursday, December 01, 2011 2:57 PM
> To: kafka-users@incubator.apache.org
> Subject: Re: Kafka and Binary Data
>
> Hi Casey,
>
> Yes, you can send messages as byte arrays. If you look at the constructor
> of the Message class, you will see -
>
> def this(bytes: Array[Byte])
>
> Now, looking at the Producer send() API -
>
> def send(producerData: ProducerData[K,V]*)
>
> You can set V to be of type Message and K to what you want your key to be.
> If you don't care about partitioning using a key, then set that to Message
> type as well.
>
> Thanks,
> Neha
>
> On Thu, Dec 1, 2011 at 11:46 AM, Sybrandy, Casey <
> CaseySybrandy@noviidesign.com> wrote:
>
> > All,
> >
> > Can one send byte arrays as messages to Kafka without having to encode
> > them, such as using Base64?  We're trying to be as space efficient as
> > possible, however any gains made by using binary data formats, such as
> > BSON, are lost when we Base64 encode the data.  I tried once before, but
> > got an error with a conversion to a string, so I wanted to check and see
> if
> > anyone else has done this.
> >
> > Thanks.
> >
> > Casey
>



-- 
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message