kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SenthilKumar K <senthilec...@gmail.com>
Subject Re: Required guidelines for kafka upgrade
Date Fri, 03 May 2019 19:05:56 GMT
You have to set the same endpoint algorithm (empty) in consumer client
properties.



On Sat, May 4, 2019, 12:15 AM ASHOK MACHERLA <iAshok7@outlook.com> wrote:

> Dear Senthil
>
> as you suggested I follow, Kafka Cluster is fine ISR showing 0,1,2
>
> but getting SSL error logs
>
> [2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
>
>
> when I tried to send some messages to one topic
> Producer::
> $ sh bin/kafka-console-producer.sh --broker-list 192.168.175.128:9092
> --topic otp-sms
> aaaaaaaaaaaaaaaaaaa
> bbbbbbbbbbbbbbbbbbbb
> ccccccccccccccccccccccccccc
>
> Consumer::
> $ sh bin/kafka-console-consumer.sh --bootstrap-server 192.168.175.128:9092
> --topic otp-sms
>
> it's not printing any messages , even when I try to pull messages from
> beginning also not working
>
> same above error getting
>
> Please help us Senthil, to fix this issue
>
> Thanks for your support...
>
>
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: ASHOK MACHERLA <iAshok7@outlook.com>
> Sent: 03 May 2019 23:52
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> Dear Senthil
>
> first of all thanks for help,
>
> after I set like ssl.endpoint.identification.algorithm =   and then
> restart it's working fine.
>
> after that I changed below parameters in all brokers
>
> inter.broker.protocol.version=2.2.0
> log.message.format.version=2.2.0
>
> after that restarted one by one. ISR is showing 0,1,2 it's good,
> but getting some SSL error, please look into these logs
>
> [2019-05-03 11:00:37,789] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
> [2019-05-03 11:01:19,611] INFO [SocketServer brokerId=0] Failed
> authentication with /192.168.175.128<http://192.168.175.128/> (SSL
> handshake failed) (org.apache.kafka.common.network.Selector)
>
> getting logs all brokers,
>
> I tried to produce sample messages to topic,
> consumer is not print that messages .
>
> please help us to fix this
>
> Thanks a lot  senthill
>
>
> Sent from Outlook<http://aka.ms/weboutlook>
> ________________________________
> From: SenthilKumar K <senthilec566@gmail.com>
> Sent: 03 May 2019 22:26
> To: users@kafka.apache.org
> Subject: Re: Required guidelines for kafka upgrade
>
> Hi Ashok , From the logs its clear that problem with identification
> algorithm.
>
> at
>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> ... 15 more
> Caused by: java.security.cert.CertificateException: Unknown identification
> algorithm: " "
>
> Set empty and restart your broker.
>
> --Senthil
>
> On Fri, May 3, 2019 at 10:20 PM SenthilKumar K <senthilec566@gmail.com>
> wrote:
>
> > Here is my server.properties.
> >
> >
> > reserved.broker.max.id = 2147483647
> > log.retention.bytes = 68719476736
> > listeners = SSL://xxxxxx:9093
> > socket.receive.buffer.bytes = 102400
> > broker.id = xxx
> > ssl.truststore.password = xxxxx
> > auto.create.topics.enable = true
> > ssl.enabled.protocols = TLSv1.2
> > zookeeper.connect = xxxxx
> > default.replication.factor = 2
> > offsets.topic.replication.factor = 2
> > log.message.timestamp.type = CreateTime
> > min.insync.replicas = 2
> > transaction.state.log.replication.factor = 2
> > security.inter.broker.protocol = SSL
> > socket.send.buffer.bytes = 102400
> > num.partitions = 40
> > ssl.secure.random.implementation = xxxxx
> > ssl.key.password = xxxx
> > log.retention.ms = 3600000
> > log.cleaner.delete.retention.ms = 300000
> > message.max.bytes = 67108864
> > ssl.endpoint.identification.algorithm =
> > log.roll.ms = 1800000
> > log.message.timestamp.difference.max.ms = 14400000
> > ssl.keystore.location = xxxx
> > log.retention.hours = 168
> > log.retention.check.interval.ms = 180000
> > inter.broker.protocol.version = 2.2.0
> > socket.request.max.bytes = 104857600
> > log.dirs = xxxxx
> > ssl.keystore.password = xxxx
> > ssl.truststore.location = xxxx
> >
> > Pls set `ssl.endpoint.identification.algorithm =  ` and restart your
> > broker.
> >
> >
> > On Fri, May 3, 2019 at 10:09 PM ASHOK MACHERLA <iAshok7@outlook.com>
> > wrote:
> >
> >> Dear
> >>
> >>
> >> Please find this below error
> >>
> >> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> >> failed
> >> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine
> problem
> >> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
> >> at
> sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
> >> at
> sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
> >> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
> >> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:448)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:313)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:265)
> >> at
> >>
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:170)
> >> at
> >>
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:536)
> >> at org.apache.kafka.common.network.Selector.poll(Selector.java:472)
> >> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
> >> at
> >>
> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)
> >> at
> >>
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:95)
> >> at
> >>
> kafka.server.ReplicaFetcherThread.fetchFromLeader(ReplicaFetcherThread.scala:193)
> >> at
> >>
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:280)
> >> at
> >>
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:132)
> >> at
> >>
> kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:131)
> >> at scala.Option.foreach(Option.scala:274)
> >> at
> >>
> kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:131)
> >> at
> >>
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
> >> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
> >> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine
> problem
> >> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> >> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
> >> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
> >> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
> >> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
> >> at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
> >> at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:402)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:484)
> >> at
> >>
> org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:340)
> >> ... 15 more
> >> Caused by: java.security.cert.CertificateException: Unknown
> >> identification algorithm: " "
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
> >> at
> >>
> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
> >> at
> >>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
> >> ... 24 more
> >> [2019-05-03 06:36:23,840] INFO [ReplicaFetcher replicaId=0, leaderId=2,
> >> fetcherId=0] Failed authentication with /192.168.175.130 (SSL handshake
> >> failed) (org.apache.kafka.common.network.Selector)
> >> [2019-05-03 06:36:23,842] ERROR [ReplicaFetcher replicaId=0, leaderId=2,
> >> fetcherId=0] Connection to node 2 (/192.168.175.130:9092) failed
> >> authentication due to: SSL handshake failed
> >> (org.apache.kafka.clients.NetworkClient)
> >> Sent from Outlook<http://aka.ms/weboutlook>
> >> ________________________________
> >> From: Harper Henn <harper.henn@datto.com>
> >> Sent: 03 May 2019 21:35
> >> To: users@kafka.apache.org
> >> Subject: Re: Required guidelines for kafka upgrade
> >>
> >> What specific errors are you seeing in the server logs of the broker you
> >> upgraded (can you copy/paste them)?
> >>
> >> On Fri, May 3, 2019 at 7:29 AM ASHOK MACHERLA <iAshok7@outlook.com>
> >> wrote:
> >>
> >> > *Dear Senthil*
> >> >
> >> > As you suggested , I follow but I’m facing errors
> >> >
> >> > This is my old configurations which is Kafka (0.10.1) version
> >> >
> >> > *****************************************************************
> >> >
> >> > broker.id=0
> >> >
> >> > port=9092
> >> >
> >> > delete.topic.enable=true
> >> >
> >> > message.max.bytes=100000
> >> >
> >> > listeners=SSL://192.168.175.128:9092
> >> >
> >> > advertised.listeners=SSL://192.168.175.128:9092
> >> >
> >> > num.network.threads=3
> >> >
> >> > num.io.threads=8
> >> >
> >> > socket.send.buffer.bytes=102400
> >> >
> >> > socket.receive.buffer.bytes=102400
> >> >
> >> > socket.request.max.bytes=104857600
> >> >
> >> > log.dirs=/opt/kafka/kafka-logs
> >> >
> >> > num.partitions=3
> >> >
> >> > default.replication.factor=3
> >> >
> >> > auto.topic.creation.enable=false
> >> >
> >> > num.recovery.threads.per.data.dir=1
> >> >
> >> > log.retention.hours=168
> >> >
> >> > log.segment.bytes=1073741824
> >> >
> >> > log.retention.check.interval.ms=300000
> >> >
> >> > ssl.keystore.location=/opt/kafka/certificate/kafka.keystore.jks
> >> >
> >> > ssl.keystore.password=Sbi#123
> >> >
> >> > ssl.key.password=Sbi#123
> >> >
> >> > ssl.truststore.location=/opt/kafka/certificate/kafka.truststore.jks
> >> >
> >> > ssl.truststore.password=Sbi#123
> >> >
> >> > security.inter.broker.protocol=SSL
> >> >
> >> > zookeeper.connect=192.168.175.128:2181,192.168.175.129:2181,
> >> > 192.168.175.130:2181
> >> >
> >> > zookeeper.connection.timeout.ms=6000
> >> >
> >> > *****************************************************************
> >> >
> >> > After that i added three parameters into server.properties which is
> *new
> >> > kafka version (2.2.0)*
> >> >
> >> > inter.broker.protocol.version=0.10.1
> >> >
> >> > log.message.format.version=0.10.1
> >> >
> >> > ssl.endpoint.identification.algorithm=""
> >> >
> >> > After that I stopped one Kafka node, and then I started new Kafka
> >> (2.2.0)
> >> > version in same node.
> >> >
> >> > in this port is opening , it's showing 9092 port number
> >> >
> >> > but getting errors due to SSL issues
> >> >
> >> > I this position first node is running with new version (2.2.0) and
> >> > remaining two nodes are running with previous version (0.10.1)
> >> >
> >> > I checked topic describe command on second node, here ISR is not
> syncing
> >> > with new version,
> >> >
> >> > it's showing only 1,2, here "0" is missing it means first nodes was
> not
> >> > syncing with remaining nodes,
> >> >
> >> > it’s should show like 0,1,2.
> >> >
> >> > Please help Senthil
> >> >
> >> > I tried so many options like below ,
> >> >
> >> > ssl.endpoint.identification.algorithm=""
> >> >
> >> > ssl.endpoint.identification.algorithm=" "
> >> >
> >> > ssl.endpoint.identification.algorithm="none"
> >> >
> >> > ssl.endpoint.identification.algorithm="null"
> >> >
> >> > ssl.endpoint.identification.algorithm=null
> >> >
> >> > ssl.endpoint.identification.algorithm=https
> >> >
> >> > please tell what correct value I should mention, and port is is
> showing
> >> > but why it's ISR showing only 1,2 instead of 0,1,2
> >> >
> >> > is there any firewall settings problems?
> >> >
> >> > Please help us to fix this Senthil
> >> >
> >> > thanks
> >> >
> >> >
> >> > Sent from Outlook <http://aka.ms/weboutlook>
> >> > ------------------------------
> >> > *From:* ASHOK MACHERLA <iashok7@outlook.com>
> >> > *Sent:* 02 May 2019 13:28
> >> > *To:* users@kafka.apache.org
> >> > *Subject:* Re: Required guidelines for kafka upgrade
> >> >
> >> > OK Senthil
> >> >
> >> > Thanks for your support and cooperation
> >> >
> >> > Sent from Outlook
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message