kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject RE: Kafka consumer issue
Date Tue, 20 Mar 2018 19:30:05 GMT
So the documentation says "advertised.host.name" and "advertised.port" are 
used only when "listeners" or "advertised.listeners" are not set.
Sorry I missed that you had already set them. Though, they are deprecated 
and it is recommended to use "advertised.listeners" instead.

I just tested this successfully for a simple consumption and only set 
"listeners" (starting with the default servers.properties), as mentioned 
earlier.
If it's possible, I'd suggest you try that too, to hopefully narrow down 
the cause of the issue.

--Vahid




From:   "Anand, Uttam" <Uttam.Anand@BNSF.com>
To:     "users@kafka.apache.org" <users@kafka.apache.org>
Date:   03/20/2018 12:15 PM
Subject:        RE: Kafka consumer issue



Getting below repeated warnings and messages are not consumed -:

[2018-03-20 14:09:56,787] WARN [Consumer clientId=consumer-1, 
groupId=console-consumer-35712] Connection to node -1 could not be 
established. Broker may not be available. 
(org.apache.kafka.clients.NetworkClient)

-----Original Message-----
From: Vahid S Hashemian [mailto:vahidhashemian@us.ibm.com] 
Sent: Tuesday, March 20, 2018 1:55 PM
To: users@kafka.apache.org
Subject: RE: Kafka consumer issue

<center><p style="color:red; font-size:24px;"><b>EXTERNAL 
EMAIL</b></p></center> Yes, without the spaces: 
listeners=PLAINTEXT://kfk03.mp.com:9092




From:   "Anand, Uttam" <Uttam.Anand@BNSF.com>
To:     "users@kafka.apache.org" <users@kafka.apache.org>
Date:   03/20/2018 11:50 AM
Subject:        RE: Kafka consumer issue



You meant 

 listeners=PLAINTEXT:// kfk03.mp.com:9092


-----Original Message-----
From: Vahid S Hashemian [mailto:vahidhashemian@us.ibm.com]
Sent: Tuesday, March 20, 2018 1:45 PM
To: users@kafka.apache.org
Subject: RE: Kafka consumer issue

<center><p style="color:red; font-size:24px;"><b>EXTERNAL 
EMAIL</b></p></center> Thanks.

Have you tried setting the listeners property to the actual domain name of 
the broker? That might be the issue.

--Vahid



From:   "Anand, Uttam" <Uttam.Anand@BNSF.com>
To:     "users@kafka.apache.org" <users@kafka.apache.org>
Date:   03/20/2018 11:34 AM
Subject:        RE: Kafka consumer issue



Server.properties

broker.id=0
listeners=PLAINTEXT://:9092
port=9092
host.name=kfk03.mp.com
advertised.host.name=kfk03.mp.com
advertised.port=9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/bnsf/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
log.retention.hours=1440
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=zkp02.mp.com:2181
zookeeper.connection.timeout.ms=60000

-----Original Message-----
From: Vahid S Hashemian [mailto:vahidhashemian@us.ibm.com]
Sent: Tuesday, March 20, 2018 1:31 PM
To: users@kafka.apache.org
Subject: RE: Kafka consumer issue

<center><p style="color:red; font-size:24px;"><b>EXTERNAL 
EMAIL</b></p></center> Could you paste the server.properties content?
It wasn't attached to the original note.

Thanks.
--Vahid




From:   "Anand, Uttam" <Uttam.Anand@BNSF.com>
To:     "users@kafka.apache.org" <users@kafka.apache.org>
Date:   03/20/2018 11:10 AM
Subject:        RE: Kakfa consumer issue



Yes checked everything. No errors. Replication factor is 1 only.



-----Original Message-----

From: Manikumar [mailto:manikumar.reddy@gmail.com] 

Sent: Tuesday, March 20, 2018 12:59 PM

To: users@kafka.apache.org

Subject: Re: Kakfa consumer issue



<center><p style="color:red; font-size:24px;"><b>EXTERNAL 
EMAIL</b></p></center> check broker logs for any errors. also enable 
consumer debug logs.

check the health of the __consumer_offsets topic. make sure to set

offsets.topic.replication.factor=1 for single node cluster.



On Tue, Mar 20, 2018 at 11:21 PM, Anand, Uttam <Uttam.Anand@bnsf.com>
wrote:



> I don’t want to use --new-consumer as it is the default, so this

> option is deprecated and will be removed in a future release.

>

> -----Original Message-----

> From: Anand, Uttam

> Sent: Tuesday, March 20, 2018 12:43 PM

> To: 'users@kafka.apache.org' <users@kafka.apache.org>

> Subject: RE: Kakfa consumer issue

>

> You mean by executing below command ?

>

> /kafka/bin/kafka-console-consumer.sh --new-consumer --bootstrap-server

> kfk03.mp.com:2181 --topic test --from-beginning

>

> -----Original Message-----

> From: Zakee [mailto:kzakee1@netzero.net]

> Sent: Tuesday, March 20, 2018 12:35 PM

> To: users@kafka.apache.org

> Subject: Re: Kakfa consumer issue

>

> <center><p style="color:red; font-size:24px;"><b>EXTERNAL

> EMAIL</b></p></center> Did you try with  --new-consumer  ?

>

> -Zakee

>

> > On Mar 20, 2018, at 10:26 AM, Anand, Uttam <Uttam.Anand@BNSF.com>
wrote:

> >

> > I am facing an issue while consuming message using the

> > bootstrap-server

> i.e. Kafka server. Any idea why is it not able to consume messages

> without zookeeper?

> >

> > Kafka Version -: kafka_2.11-1.0.0

> > Zookeeper Version -: kafka_2.11-1.0.0 Server.properties -: Attached

> > Zookeeper Host and port -: z

> > <
https://urldefense.proofpoint.com/v2/url?u=http-3A__zkp02.mp.com-3A2181_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=HtCECVj6e6Vg8GxNoHQofxvBWbemYOT7QtAedYnV1LU&e=



>kp02.mp.com:2181

> > <
https://urldefense.proofpoint.com/v2/url?u=http-3A__zkp02.mp.com-3A2181_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=HtCECVj6e6Vg8GxNoHQofxvBWbemYOT7QtAedYnV1LU&e=



> Kafka Host and port -: kfk03.mp.com:9092

> > <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com-3A9092_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=0ULoijWNSqOP_En8AvhlnQvd_6LbG71ko-c1anZhnHo&e=



>

> >

> > Producing some message  -:

> >

> > [kfk03.mp.com <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=84h4-GitFCbi7J_IhfqzIRTgV3_7sGoLIdevdfQKW2g&e=



> ~]$

> > /bnsf/kafka/bin/kafka-console-producer.sh --broker-list kfk03.mp.com

> > <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com-3A9092_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=0ULoijWNSqOP_En8AvhlnQvd_6LbG71ko-c1anZhnHo&e=



>:9092 <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com-3A9092_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=0ULoijWNSqOP_En8AvhlnQvd_6LbG71ko-c1anZhnHo&e=



> --topic

> > test

> > >hi

> > >hi

> >

> > Consumer not able to consume message if I give –bootstrap-server  -:

> >

> > [kfk03.mp.com <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=84h4-GitFCbi7J_IhfqzIRTgV3_7sGoLIdevdfQKW2g&e=



> ~]$

> > /bnsf/kafka/bin/kafka-console-consumer.sh --bootstrap-server

> > kfk03.mp.com <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com-3A9092_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=0ULoijWNSqOP_En8AvhlnQvd_6LbG71ko-c1anZhnHo&e=



>:9092

> > <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com-3A9092_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=0ULoijWNSqOP_En8AvhlnQvd_6LbG71ko-c1anZhnHo&e=



> --topic test --from-beginning

> >

> > Consumer able to consume messages when zookeeper server is given

> > instead

> of bootstrap-server -:

> >

> > [kfk03.mp.com <
https://urldefense.proofpoint.com/v2/url?u=http-3A__kfk03.mp.com_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=84h4-GitFCbi7J_IhfqzIRTgV3_7sGoLIdevdfQKW2g&e=



> ~]$

> > /bnsf/kafka/bin/kafka-console-consumer.sh --zookeeper

> > zkp02.mp.com:2181

> <
https://urldefense.proofpoint.com/v2/url?u=http-3A__zkp02.mp.com-3A2181_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=HtCECVj6e6Vg8GxNoHQofxvBWbemYOT7QtAedYnV1LU&e=



> --topic test --from-beginning Using the

> ConsoleConsumer with old consumer is deprecated and will be removed in

> a future major release. Consider using the new consumer by passing

> [bootstrap-server] instead of [zookeeper].

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > hi

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > {"properties": {"messageType": "test", "sentDateTime":

> > "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}

> > hi hi uttam hi hi hi hello hi ^CProcessed a total of 17 messages

> >

> > Thanks

> > Uttam

>

> ____________________________________________________________

> How To Remove Eye Bags & Lip Lines Fast (Watch) Fit Mom Daily

> 
https://urldefense.proofpoint.com/v2/url?u=http-3A__thirdpartyoffers.netzero.net_TGL3231_5ab14671c27946702f74st04du&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=kfMgTkQ_OnAIXEkdDqz7juDsY7PVQGYY7dxzFMHrNEQ&s=COXBDQQFOwGnm4FrhhwQi7n_qbBsSK9AjFPCY5Rx_QQ&e=





> c

>


















Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message