kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Kafka and Binary Data
Date Thu, 01 Dec 2011 21:43:26 GMT
1) V can be any type as long as you have the right encoder defined.

2) We will do that once 0.7 is officially released.

Thanks,

Jun

On Thu, Dec 1, 2011 at 1:11 PM, Evan Chan <ev@ooyala.com> wrote:

> A couple quick questions:
>
> 1) What prevents one from changing the V type in the Producer API to any
> arbitrary type I want, such as String, or my own Thrift struct, for
> example?
> (And I suppose one can define custom serializers to serialize any type)
>
> 2) When will the docs be updated to fit 0.7?   I don't see where the docs
> document the [K, V] typing, for example.
>
> thanks,
> Evan
>
> On Thu, Dec 1, 2011 at 12:10 PM, Sybrandy, Casey <
> CaseySybrandy@noviidesign.com> wrote:
>
> > Perfect, thanks!
> >
> > ________________________________________
> > From: Neha Narkhede [neha.narkhede@gmail.com]
> > Sent: Thursday, December 01, 2011 2:57 PM
> > To: kafka-users@incubator.apache.org
> > Subject: Re: Kafka and Binary Data
> >
> > Hi Casey,
> >
> > Yes, you can send messages as byte arrays. If you look at the constructor
> > of the Message class, you will see -
> >
> > def this(bytes: Array[Byte])
> >
> > Now, looking at the Producer send() API -
> >
> > def send(producerData: ProducerData[K,V]*)
> >
> > You can set V to be of type Message and K to what you want your key to
> be.
> > If you don't care about partitioning using a key, then set that to
> Message
> > type as well.
> >
> > Thanks,
> > Neha
> >
> > On Thu, Dec 1, 2011 at 11:46 AM, Sybrandy, Casey <
> > CaseySybrandy@noviidesign.com> wrote:
> >
> > > All,
> > >
> > > Can one send byte arrays as messages to Kafka without having to encode
> > > them, such as using Base64?  We're trying to be as space efficient as
> > > possible, however any gains made by using binary data formats, such as
> > > BSON, are lost when we Base64 encode the data.  I tried once before,
> but
> > > got an error with a conversion to a string, so I wanted to check and
> see
> > if
> > > anyone else has done this.
> > >
> > > Thanks.
> > >
> > > Casey
> >
>
>
>
> --
> --
> *Evan Chan*
> Senior Software Engineer |
> ev@ooyala.com | (650) 996-4600
> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> @ooyala<http://www.twitter.com/ooyala>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message