kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hutashan Chandrakar <hutas...@gmail.com>
Subject How to get size (metadata) of a message without fetch request in Kafka through SimpleConsumer?
Date Fri, 17 Jul 2015 21:21:00 GMT
Hello All,

Requirement :- Process/Read (Consumer) data only if kafka message size
greater then specified size (configured).

I am using SimpleConsumer and trying to pull size wise (GB) message using
spark(kafka RDD).I am able to get earliest and latest offset using metadata
request, but not able to get any size related metadata information.

Is there way to get size of a message without fetchrequest in Kafka through
SimpleConsumer/highlevel api ? Is there any API/method to get size related
information in metadata request/response in kafka api.

Hutashan Chandrakar

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message