kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sivananda Reddys Thummala Abbigari <sthumm...@salesforce.com>
Subject Weird error on machine restart when creating ConsumerConnector reference
Date Fri, 09 Oct 2015 06:30:47 GMT
Hi,

*Following is the consumer related code*:

Line#1 ----------------> ConsumerConnector consumer =
kafka.consumer.Consumer.createJavaConsumerConnector(createConsumerConfig(zkConnect,
consumerGroupId, offSetStorage));

private ConsumerConfig createConsumerConfig(String zookeeperConnectString,
String consumerGroupId, String offSetStorage) {
    Properties props = new Properties();
    props.put("zookeeper.connect", zookeeperConnectString);
    props.put("group.id", consumerGroupId);
    props.put("offsets.storage", offSetStorage);
    return new ConsumerConfig(props);
}

Kafka/ZK/Client processes are run on a single machine(dev environment).
Client process created bunch of topics(*with replication factor of 1*),
performed R/W operations on them. Then I re-started the machine and then
started all the three processes(Kafka/ZK/Client), I see the following error
in the kafka server logs:

*kafka.admin.AdminOperationException: replication factor: 3 larger than
available brokers: 1*
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171)
at kafka.server.KafkaApis$$anonfun$19.apply(KafkaApis.scala:513)
at kafka.server.KafkaApis$$anonfun$19.apply(KafkaApis.scala:503)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Set$Set1.foreach(Set.scala:74)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at
scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
at scala.collection.SetLike$class.map(SetLike.scala:93)
at scala.collection.AbstractSet.map(Set.scala:47)
at kafka.server.KafkaApis.getTopicMetadata(KafkaApis.scala:503)
at kafka.server.KafkaApis.handleConsumerMetadataRequest(KafkaApis.scala:607)
at kafka.server.KafkaApis.handle(KafkaApis.scala:69)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
at java.lang.Thread.run(Thread.java:745)

This exception occurs when Line#1 is executed, there is no topic with
replication factor of 3.

*Kafka Version*: kafka_2.10-0.8.2.1

So I re-started Kafka + ZK + client processes again and the error is gone.
Was wondering what could the reason for this behavior?. Please advice.

Thank you,
Siva.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message