kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayesh Thakrar <j_thak...@yahoo.com.INVALID>
Subject Re: Kafka getMetadata api
Date Fri, 02 Jan 2015 19:56:37 GMT
Just wondering Mukesh - the reason you want this feature is because your value payload is not
small (tens of kb). Don't know if that is the right usage of kafka. It might be worthwhile
to store the avro files in a filesystem (regular, cluster fs, hdfs or even hbase) and the
value in your kafka message can be the reference or uri for the avro file. 

That way you make the best use of each system's features and strengths. 

Kafka does have api to get metadata - the topics, partitions and  primary for partition etc.
If we consider a key-value pair as a "record" than what you are looking for is to get a part
of the record (ie key only) and not the whole record - so i would still consider that a data
query/api.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message