kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sat <sathish.al...@gmail.com>
Subject Re: Kafka Consumer poll
Date Thu, 04 Aug 2016 00:46:38 GMT
Hi,


Thanks for your reply Kamal and Oleg.


Thanks and Regards

A.SathishKumar

>Also keep in mind that unfortunately KafkaConsumer.poll(..) will deadlock regardless of
the
>timeout if connection to the broker can not be established and won't react to thread interrupts.
>This essentially means that the only way to exit is to kill jvm. This is all because Kafka
>fetches topic metadata synchronously before timeout takes effect.
>While it is my understanding that the reason for it is there is a background thread attempting
>to reconnect in the event of temporary broker outage, it doesn't help if you accidentally
>specified wrong broker url.

Oleg

> On Aug 2, 2016, at 10:27, Kamal C <kamaltarget@gmail.com> wrote:
>
> See the answers inline.
>
>> On Tue, Aug 2, 2016 at 12:23 AM, sat <sathish.alwar@gmail.com> wrote:
>>
>> Hi,
>>
>> I am new to Kafka. We are planning to use Kafka messaging for our
>> application. I was playing with Kafka 0.9.0.1 version and i have following
>> queries. Sorry for asking basic questions.
>>
>>
>> 1) I have instantiated Kafka Consumer and invoked
>> consumer.poll(Long.MAX_VALUE). Although i have specified timeout as
>> Long.MAX_VALUE, i observe my consumer to fetch records whenever the
>> publisher publishes a message to a topic. This makes me wonder whether
>> Kafka Consumer is push or pull mechanism. Please help us understand the
>> logic of consumer.poll(timeout).
>
> Fetches the data from the topic, waiting up to the specified wait time *if
> necessary *for a record to become available.
> Kafka Consumer by design is pull mechanism.
>
> Take a look into Kafka Consumer java docs[1]. It's explained in detail.
>
>
>> 2) What are the pros and cons of poll for long timeout vs short timeout.
>>
>> Short Timeout
>
> Pros:
> - On shutdown, if no data available in the topic -- Shutdown will be quick
>
> Cons:
> - Number of network trips will be high
>
>
>>
>> Thanks and Regards
>> A.SathishKumar
>
> [1]:
> https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
>
> -- Kamal


On Mon, Aug 1, 2016 at 11:53 AM, sat <sathish.alwar@gmail.com> wrote:

> Hi,
>
> I am new to Kafka. We are planning to use Kafka messaging for our
> application. I was playing with Kafka 0.9.0.1 version and i have following
> queries. Sorry for asking basic questions.
>
>
> 1) I have instantiated Kafka Consumer and invoked
> consumer.poll(Long.MAX_VALUE). Although i have specified timeout as
> Long.MAX_VALUE, i observe my consumer to fetch records whenever the
> publisher publishes a message to a topic. This makes me wonder whether
> Kafka Consumer is push or pull mechanism. Please help us understand the
> logic of consumer.poll(timeout).
>
> 2) What are the pros and cons of poll for long timeout vs short timeout.
>
>
> Thanks and Regards
> A.SathishKumar
>
>


-- 
A.SathishKumar
044-24735023

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message