kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navneet Kumar <navneetkumar.in...@gmail.com>
Subject Re: Java Client for Kafka Consumer(0.9) not polling the records from brokers instantly
Date Fri, 27 May 2016 02:35:38 GMT
Hi
We are facing a issue where our Consumer component is not instantly
logging the records in DB polled from the Brokers. We have following
theOur architecture diagram is attached herewith. Following are the
properties
configured:

Producer.properties

bootstrap.servers=xx.xxx.xxx.140:9092,xx.xxx.xxx.140:9093,xx.xxx.xxx.140:9094,xx.xxx.xxx.141:9092,xx.xxx.xxx.141:9093,xx.xxx.xxx.141:9094,xx.xxx.xxx.142:9092,xx.xxx.xxx.142:9093,xx.xxx.xxx.142:9094
acknowledgement=all
group_id=kafkaClusterVLink
retries=3
batch_size=6384
linger_ms=1
buffer_memory=54432
topic=visualize-link-redirect
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.StringSerializer

We send the Json String to from Producer to Borkers(Approxs 250Byte/Messsage)

Consumer.properties

bootstrap.servers=xx.xxx.xxx.140:9092,xx.xxx.xxx.140:9093,xx.xxx.xxx.140:9094,xx.xxx.xxx.141:9092,xx.xxx.xxx.141:9093,xx.xxx.xxx.141:9094,xx.xxx.xxx.142:9092,xx.xxx.xxx.142:9093,xx.xxx.xxx.142:9094
group.id=kafkaClusterVLink
enable.auto.commit=true
auto.commit.interval.ms=10000
session.timeout.ms=30000
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
topic=visualize-link-redirect


Please help me to resolve this issue as soon as possible. Will be
grateful to you.




Thanks and Regards,
Navneet Kumar


On Wed, May 25, 2016 at 4:27 PM, Navneet Kumar
<navneetkumar.india@gmail.com> wrote:
> Hi
> We are facing a issue where our Consumer component is not instantly
> logging the records polled from the Brokers. We have following the
> below architecture as attached. Following are the properties
> configured:
>
> Producer.properties
>
> bootstrap.servers=xx.xxx.xxx.140:9092,xx.xxx.xxx.140:9093,xx.xxx.xxx.140:9094,xx.xxx.xxx.141:9092,xx.xxx.xxx.141:9093,xx.xxx.xxx.141:9094,xx.xxx.xxx.142:9092,xx.xxx.xxx.142:9093,xx.xxx.xxx.142:9094
> acknowledgement=all
> group_id=kafkaClusterVLink
> retries=3
> batch_size=6384
> linger_ms=1
> buffer_memory=54432
> topic=visualize-link-redirect
> key.serializer=org.apache.kafka.common.serialization.StringSerializer
> value.serializer=org.apache.kafka.common.serialization.StringSerializer
>
> We send the Json String to from Producer to Borkers(Approxs 250Byte/Messsage)
>
> Consumer.properties
>
> bootstrap.servers=xx.xxx.xxx.140:9092,xx.xxx.xxx.140:9093,xx.xxx.xxx.140:9094,xx.xxx.xxx.141:9092,xx.xxx.xxx.141:9093,xx.xxx.xxx.141:9094,xx.xxx.xxx.142:9092,xx.xxx.xxx.142:9093,xx.xxx.xxx.142:9094
> group.id=kafkaClusterVLink
> enable.auto.commit=true
> auto.commit.interval.ms=10000
> session.timeout.ms=30000
> key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
> value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
> topic=visualize-link-redirect
>
>
> Please help me to resolve this issue as soon as possible. Will be
> grateful to you.
>
>
>
>
>
>
>
> Thanks and Regards,
> Navneet Kumar

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message