kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: list of pattern processed topic list
Date Tue, 20 Aug 2019 18:17:18 GMT
#listTopic() is not related to your subscription/assignment as indicated
by its JavaDocs:

> Get metadata about partitions for all topics that the user is authorized to view. This
method will issue a
> remote call to the server.


-Matthias

On 8/19/19 7:02 AM, Kamal Chandraprakash wrote:
> You can use the KafkaConsumer#assignment() method to get all the assigned
> topic-partitions for that consumer instance.
> But, you've to periodically call poll method to get the latest assignment
> which may return records.
> This shortcoming is actively discussed in the below threads.
> 
> https://mail-archives.apache.org/mod_mbox/kafka-dev/201908.mbox/<08030a68-3f0b-42db-9b79-dfcd3200cf25%40www.fastmail.com>
> <https://mail-archives.apache.org/mod_mbox/kafka-dev/201908.mbox/%3C08030a68-3f0b-42db-9b79-dfcd3200cf25%40www.fastmail.com%3E>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-505%3A+Add+new+public+method+to+only+update+assignment+metadata+in+consumer
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484
> 
> 
> On Mon, Aug 19, 2019 at 6:33 PM M. Manna <manmedia@gmail.com> wrote:
> 
>> Kafka doesn’t have regex subscription. You could get a list of available
>> topic using admin client. From there, you can decide what topics you would
>> like to subscribe to (or process using your logic).
>>
>> Check documentation for AdminClient.listTopics and ListTopicResult holder.
>>
>> I hope this helps.
>>
>> Regards,
>>
>> On Mon, 19 Aug 2019 at 13:22, Upendra Yadav <upendra1024@gmail.com> wrote:
>>
>>> Hi,
>>> I have initialised kafka consumer:
>>> KafkaConsumer<byte[], byte[]> consumer = new KafkaConsumer<byte[],
>>> byte[]>(consumerConfig);
>>>
>>> and subscribed with topic pattern:
>>> consumer.subscribe(Pattern.compile(topicRegex), listener);
>>>
>>> now, I'm trying to get the list of topics.
>>> Map<String, List<PartitionInfo>> topicsPartitions =
>> consumer.listTopics();
>>>
>>> here topic list is not following given pattern. it is giving all topic
>>> list.
>>> Is this expected behaviour? Is there any API to get subscribed topic list
>>> with consumer instance?
>>> or Do I need to write simple prog to process this list with same pattern?
>>>
>>
> 


Mime
View raw message