kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhu v <prabhuvrajp...@gmail.com>
Subject Re: Consumer - Failed to find leader
Date Wed, 20 Jan 2016 09:48:26 GMT
Hi Harsh/Ismael,

Any suggestions or inputs for the above issue?

When i run the producer client, I still get this error

./kafka-console-producer.sh --broker-list hostname:9094 --topic topic3


*[2016-01-05 10:16:20,272] ERROR Error when sending message to topic test
with key: null, value: 5 bytes with error: Failed to update metadata after
60000 ms.
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)*

Also, i can see the below error in controller.log

*[2016-01-20 09:39:33,408] DEBUG [Controller 0]: preferred replicas by
broker Map(1 -> Map([topic3,0] -> List(1, 0)), 0 -> Map([topic3,1] ->
List(0, 1), [topic2,0] -> List(0), [topic1,0] -> List(0)))
(kafka.controller.KafkaController)*
*[2016-01-20 09:39:33,408] DEBUG [Controller 0]: topics not in preferred
replica Map() (kafka.controller.KafkaController)*
*[2016-01-20 09:39:33,408] TRACE [Controller 0]: leader imbalance ratio for
broker 1 is 0.000000 (kafka.controller.KafkaController)*
*[2016-01-20 09:39:33,408] DEBUG [Controller 0]: topics not in preferred
replica Map() (kafka.controller.KafkaController)*
*[2016-01-20 09:39:33,409] TRACE [Controller 0]: leader imbalance ratio for
broker 0 is 0.000000 (kafka.controller.KafkaController)*


Tried reinstalling kafka, but no luck:(


Checked telnet also, I am able to connect to that port.
[root@blrd-cmgvapp46 logs]# telnet hostname 9094
Trying 172.31.31.186...
Connected to hostname (172.31.31.186).
Escape character is '^]'.

I can see the topic is created properly.

[root@hostname bin]# ./kafka-topics.sh --describe --zookeeper hostname:2181
--topic topic3
Topic:topic3    PartitionCount:2        ReplicationFactor:2     Configs:
        Topic: topic3   Partition: 0    Leader: 1       Replicas: 1,0
Isr: 1,0
        Topic: topic3   Partition: 1    Leader: 0       Replicas: 0,1
Isr: 0,1


Thanks in advance,


On Tue, Jan 5, 2016 at 3:17 PM, prabhu v <prabhuvrajpblr@gmail.com> wrote:

> Hi Harsha,
>
> This is my Kafka_server_jaas.config file. This is passed as JVM param to
> the Kafka broker while start up.
>
> =============
> KafkaServer {
>     com.sun.security.auth.module.Krb5LoginModule required
>       useKeyTab=true
>        storeKey=true
>       serviceName="kafka"
>        keyTab="/etc/security/keytabs/kafka1.keytab"
>         useTicketCache=true
>         principal="kafka/hostname@realmname";
> };
>
> zkclient{
>
> com.sun.security.auth.module.Krb5LoginModule required
>       useKeyTab=true
>        storeKey=true
>       serviceName="zookeeper"
>        keyTab="/etc/security/keytabs/kafka1.keytab"
>         useTicketCache=true
>         principal="kafka@realmname";
>
> };
> =============
>
> Note: For security reasons, changed my original FQDN to hostname and
> original realm name to realm name in the below output.
>
> I am able to view the ticket using klist command as well. Please find
> below output.
>
> [root@localhost config]# kinit -k -t /etc/security/keytabs/kafka1.keytab
> kafka/hostname@realmname
> [root@localhost config]# klist
> Ticket cache: FILE:/tmp/krb5cc_0
> Default principal: kafka/hostname@realmname
>
> Valid starting     Expires            Service principal
> 01/05/16 08:14:28  01/06/16 08:14:28  krbtgt/realm@realm
>         renew until 01/05/16 08:14:28
>
>
>
>
>
>
> For(topics,producer and consumer) clients, I am using the below JAAS
> Config:
>
> =============
>
> Client {
> com.sun.security.auth.module.Krb5LoginModule required
> useKeyTab=true
> keyTab="/etc/security/keytabs/kafka_client.keytab"
> storeKey=true
> useTicketCache=true
> serviceName="kafka"
> principal="kafkaclient/hostname@realmname";
> };
>
> =============
>
> I am able to view the ticket using klist command as well. Please find
> below output.
>
> [root@localhost config]# kinit -k -t
> /etc/security/keytabs/kafka_client.keytab kafkaclient/hostname@realmname
> [root@localhost config]# klist
> Ticket cache: FILE:/tmp/krb5cc_0
> Default principal: kafkaclient/hostname@realmname
>
> Valid starting     Expires            Service principal
> 01/05/16 08:14:28  01/06/16 08:14:28  krbtgt/realm@realm
>         renew until 01/05/16 08:14:28
>
> Error when running producer client:
>
> ./kafka-console-producer.sh --broker-list hostname:9095 --topic test
>
>
> [2016-01-05 10:16:20,272] ERROR Error when sending message to topic test
> with key: null, value: 5 bytes with error: Failed to update metadata after
> 60000 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
>
> Error when running topics.sh:
>
> [root@localhost bin]# ./kafka-topics.sh --list --zookeeper hostname:2181
> [2015-12-28 12:41:32,589] WARN SASL configuration failed:
> javax.security.auth.login.LoginException: No key to store Will continue
> connection to Zookeeper server without SASL authentication, if Zookeeper
> server allows it. (org.apache.zookeeper.ClientCnxn)
> ^Z
>
> Please let me know if i am missing anything.
>
>
>
>
> Thanks,
> Prabhu
>
>
>
>
> On Wed, Dec 30, 2015 at 9:28 PM, Harsha <kafka@harsha.io> wrote:
>
>> can you add your jass file details. Your jaas file might have
>> useTicketCache=true and storeKey=true as well
>> example of KafkaServer jass file
>>
>> KafkaServer {
>>
>> com.sun.security.auth.module.Krb5LoginModule required
>>
>> useKeyTab=true
>>
>> storeKey=true
>>
>> serviceName="kafka"
>>
>> keyTab="/vagrant/keytabs/kafka1.keytab"
>>
>> principal="kafka/kafka1.witzend.com@WITZEND.COM";
>> };
>>
>> and KafkaClient
>> KafkaClient {
>>
>> com.sun.security.auth.module.Krb5LoginModule required
>>
>> useTicketCache=true
>>
>> serviceName="kafka";
>>
>> };
>>
>> On Wed, Dec 30, 2015, at 03:10 AM, prabhu v wrote:
>>
>> Hi Harsha,
>>
>> I have used the Fully qualified domain name. Just for security concerns,
>> Before sending this mail,i have replaced our FQDN hostname to localhost.
>>
>> yes, i have tried KINIT and I am able to view the tickets using klist
>> command as well.
>>
>> Thanks,
>> Prabhu
>>
>> On Wed, Dec 30, 2015 at 11:27 AM, Harsha <kafka@harsha.io> wrote:
>>
>> Prabhu,
>>            When using SASL/kerberos always make sure you give FQDN of
>>            the hostname . In your command you are using --zookeeper
>>            localhost:2181 and make sure you change that hostname.
>>
>> "avax.security.auth.login.LoginException: No key to store Will continue
>> > connection to Zookeeper server without SASL authentication, if
>> Zookeeper"
>>
>> did you try  kinit with that keytab at the command line.
>>
>> -Harsha
>> On Mon, Dec 28, 2015, at 04:07 AM, prabhu v wrote:
>> > Thanks for the input Ismael.
>> >
>> > I will try and let you know.
>> >
>> > Also need your valuable inputs for the below issue:)
>> >
>> > i am not able to run kafka-topics.sh(0.9.0.0 version)
>> >
>> > [root@localhost bin]# ./kafka-topics.sh --list --zookeeper
>> localhost:2181
>> > [2015-12-28 12:41:32,589] WARN SASL configuration failed:
>> > javax.security.auth.login.LoginException: No key to store Will continue
>> > connection to Zookeeper server without SASL authentication, if Zookeeper
>> > server allows it. (org.apache.zookeeper.ClientCnxn)
>> > ^Z
>> >
>> > I am sure the key is present in its keytab file ( I have cross verified
>> > using kinit command as well).
>> >
>> > Am i missing anything while calling the kafka-topics.sh??
>> >
>> >
>> >
>> > On Mon, Dec 28, 2015 at 3:53 PM, Ismael Juma <ismaelj@gmail.com> wrote:
>> >
>> > > Hi Prabhu,
>> > >
>> > > kafka-console-consumer.sh uses the old consumer by default, but only
>> the
>> > > new consumer supports security. Use --new-consumer to change this.
>> > >
>> > > Hope this helps.
>> > >
>> > > Ismael
>> > > On 28 Dec 2015 05:48, "prabhu v" <prabhuvrajpblr@gmail.com> wrote:
>> > >
>> > > > Hi Experts,
>> > > >
>> > > > I am getting the below error when running the consumer
>> > > > "kafka-console-consumer.sh" .
>> > > >
>> > > > I am using the new version 0.9.0.1.
>> > > > Topic name: test
>> > > >
>> > > >
>> > > > [2015-12-28 06:13:34,409] WARN
>> > > >
>> > > >
>> > >
>> [console-consumer-61657_localhost-1451283204993-5512891d-leader-finder-thread],
>> > > > Failed to find leader for Set([test,0])
>> > > > (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)
>> > > > kafka.common.BrokerEndPointNotAvailableException: End point
>> PLAINTEXT not
>> > > > found for broker 0
>> > > >         at kafka.cluster.Broker.getBrokerEndPoint(Broker.scala:136)
>> > > >
>> > > >
>> > > > Please find the current configuration below.
>> > > >
>> > > > Configuration:
>> > > >
>> > > >
>> > > > [root@localhost config]# grep -v "^#" consumer.properties
>> > > > zookeeper.connect=localhost:2181
>> > > > zookeeper.connection.timeout.ms=60000
>> > > > group.id=test-consumer-group
>> > > > security.protocol=SASL_PLAINTEXT
>> > > > sasl.kerberos.service.name="kafka"
>> > > >
>> > > >
>> > > > [root@localhost config]# grep -v "^#" producer.properties
>> > > > metadata.broker.list=localhost:9094,localhost:9095
>> > > > producer.type=sync
>> > > > compression.codec=none
>> > > > serializer.class=kafka.serializer.DefaultEncoder
>> > > > security.protocol=SASL_PLAINTEXT
>> > > > sasl.kerberos.service.name="kafka"
>> > > >
>> > > > [root@localhost config]# grep -v "^#" server1.properties
>> > > >
>> > > > broker.id=0
>> > > > listeners=SASL_PLAINTEXT://localhost:9094
>> > > > delete.topic.enable=true
>> > > > num.network.threads=3
>> > > > num.io.threads=8
>> > > > socket.send.buffer.bytes=102400
>> > > > socket.receive.buffer.bytes=102400
>> > > > socket.request.max.bytes=104857600
>> > > > log.dirs=/data/kafka_2.11-0.9.0.0/kafka-logs
>> > > > num.partitions=1
>> > > > num.recovery.threads.per.data.dir=1
>> > > > log.retention.hours=168
>> > > > log.segment.bytes=1073741824
>> > > > log.retention.check.interval.ms=300000
>> > > > log.cleaner.enable=false
>> > > > zookeeper.connect=localhost:2181
>> > > > zookeeper.connection.timeout.ms=60000
>> > > > inter.broker.protocol.version=0.9.0.0
>> > > > security.inter.broker.protocol=SASL_PLAINTEXT
>> > > > allow.everyone.if.no.acl.found=true
>> > > >
>> > > >
>> > > > [root@localhost config]# grep -v "^#" server4.properties
>> > > > broker.id=1
>> > > > listeners=SASL_PLAINTEXT://localhost:9095
>> > > > delete.topic.enable=true
>> > > > num.network.threads=3
>> > > > num.io.threads=8
>> > > > socket.send.buffer.bytes=102400
>> > > > socket.receive.buffer.bytes=102400
>> > > > socket.request.max.bytes=104857600
>> > > > log.dirs=/data/kafka_2.11-0.9.0.0/kafka-logs-1
>> > > > num.partitions=1
>> > > > num.recovery.threads.per.data.dir=1
>> > > > log.retention.hours=168
>> > > > log.segment.bytes=1073741824
>> > > > log.retention.check.interval.ms=300000
>> > > > log.cleaner.enable=false
>> > > > zookeeper.connect=localhost:2181
>> > > > zookeeper.connection.timeout.ms=60000
>> > > > inter.broker.protocol.version=0.9.0.0
>> > > > security.inter.broker.protocol=SASL_PLAINTEXT
>> > > > zookeeper.sasl.client=zkclient
>> > > >
>> > > > [root@localhost config]# grep -v "^#" zookeeper.properties
>> > > > dataDir=/data/kafka_2.11-0.9.0.0/zookeeper
>> > > > clientPort=2181
>> > > > maxClientCnxns=0
>> > > > requireClientAuthScheme=sasl
>> > > >
>> > >
>> authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
>> > > > jaasLoginRenew=3600000
>> > > >
>> > > >
>> > > > Need your valuable inputs on this issue.
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Prabhu.V
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Prabhu.V
>>
>>
>>
>>
>> --
>> Regards,
>>
>> Prabhu.V
>>
>>
>>
>>
>
>
>
> --
> Regards,
>
> Prabhu.V
>
>



-- 
Regards,

Prabhu.V

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message