kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang <teddyyyy...@gmail.com>
Subject Re: generics type for Producer and Consumer do not need to match?
Date Thu, 05 Feb 2015 20:25:49 GMT
thanks, I just noticed the consumerconnector.createMessageStream Api, it
has 2 versions

 public <K,V> Map<String, List<KafkaStream<K,V>>>
    createMessageStreams(Map<String, Integer> topicCountMap, Decoder<K>
keyDecoder, Decoder<V> valueDecoder);

  /**
   *  Create a list of message streams of type T for each topic, using the
default decoder.
   */
  public Map<String, List<KafkaStream<byte[], byte[]>>>
createMessageStreams(Map<String, Integer> topicCountMap);


essentially the byte[], byte[] version is a default, kind of a back door ,
in which case the generics declaration of the consumer type chain is
useless indeed.



On Thu, Feb 5, 2015 at 12:18 PM, Joel Koshy <jjkoshy.w@gmail.com> wrote:

> There has to be an implicit contract between the producer and
> consumer. The K, V pairs don't _need_ to match but generally _should_.
> If producer sends with <PK, PV> the consumer may receive <CK, CV> as
> long as it knows how to convert those raw bytes to <CK, CV>. In the
> example if CK == byte[] and CV == byte[] it is effectively a no-op
> conversion.
>
> On Thu, Feb 05, 2015 at 12:11:03PM -0800, Yang wrote:
> > in the example
> > https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
> >
> > we use a String,String for <K,V>
> >
> > in the same set of example
> > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> >
> > on the consumer side we use byte[], byte[] for K,V
> >         ConsumerIterator<byte[], byte[]> it = m_stream.iterator();
> >
> >
> >
> > we tested in our own code that the above producer-consumer pair does
> work.
> >
> >
> > so the K,V types on the 2 sides do NOT need to match? if we argue that
> > "everything has to come down to byte[], then maybe we don't need the
> > generics types on the consumer side? in other words, how do we determine
> > the K,V types on consumer side?
> >
> > thanks
> > Yang
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message