kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Menke <jme...@numerix.com>
Subject RE: Match Producer and RecordMetadata with Consumer and ConsumerRecord
Date Wed, 09 Dec 2015 19:29:04 GMT
Gwen, thanks for the reply:

Just to confirm - are we talking about the combination of the IDs that I layed out as being
the "CorrelationID" ?  

I guess the alternative is to pass our own CorrelationID as part of the data.   Would you
agree with this?

I do see something at this address that talks about a "user defined" CorrelationID

https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProduceResponse

CorrelationId:	

This is a user-supplied integer. It will be passed back in the response by the server, unmodified.
It is useful for matching request and response between the client and server.



-----Original Message-----
From: Gwen Shapira [mailto:gwen@confluent.io] 
Sent: Wednesday, December 09, 2015 12:50 PM
To: users@kafka.apache.org
Subject: Re: Match Producer and RecordMetadata with Consumer and ConsumerRecord

Correlation ID is for a request (i.e. separate ID for produce request and a fetch request),
not a record. So it can't be used in the way you are trying to.

On Wed, Dec 9, 2015 at 9:30 AM, John Menke <jmenke@numerix.com> wrote:

> Can a correlationID be created from a ConsumerRecord that will allow 
> for identification of the corresponding RecordMetaData instance that 
> was returned from the Producer.send() method?
>
> I am Looking at the JavaDocs and the Producer returns RecordMetadata 
> which has the following signature:
>
> RecordMetadata(TopicPartition topicPartition, long baseOffset, long
> relativeOffset)
>
> I am not sure if this can match to
>
> ConsumerRecord(java.lang.String topic, int partition, long offset, K 
> key, V value)
>
> Can we match producer requests to Consumer Records by matching these 
> values?
>
> RecordMetaData.TopicPartion.topic = ConsumerRecord.topic 
> RecordMetaData.TopicPartion.partition = ConsumerRecord.partition 
> RecordMetaData(baseOffset + relativeOffset) = ConsumerRecord.offset
>
> In other words, can a CorrelationID be created from these values that 
> will allow Consumers to link back to the Producer send() event.  (In 
> the client calling code)
>
>
>
>
>
>
>
Mime
View raw message