kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Reply:Re: Reply:Bad user experience when out of box
Date Tue, 07 Feb 2012 17:03:49 GMT
Hi,

The producer closes the socket connection to the broker and
reconnects. The frequency of this reconnect is based on the
reconnect.interval config parameter in the producer.
By default, it is set to 30 sec, if you want you can increase it. This
is mainly to ensure better load balancing on the Kafka servers. It
shouldn't cause any overhead on the Kafka cluster, is there any
particular reason you would like the Producer to not reconnect to the
brokers ?

Thanks,
Neha

2012/2/7 SuoNayi <suonayi2006@163.com>:
> Yes, zk server and kafka server are now running normally.
> My client sends messages successfully now.
> Why kafka sever keeps closing connections from my client all the time,
> I can see information of log like this:
> INFO Closing socket connection to /2x3.1x9.x0.x. (kafka.network.Processor)
> Whichever configuration such as zk  or static configuration to discovery kafka sever
my client uses ,
> kafka server will log these informations all the time.
> I know that kafka client will create a connection pool keeping connect to brokers,
> each broker each connection, in my case, my client has connected to the single broker,
> so why kafka server keeps closing connection from my client?
>
> At 2012-02-07 18:42:17,"Joe Stein" <cryptcom@gmail.com> wrote:
>>SuoNayi,
>>
>>Do you have the Kafka Server running?   The Producer will need to find a
>>Broker that is registered within ZK for it to publish messages.
>>
>>So like in the Quick Start this is still a required step
>>
>>*bin/kafka-server-start.sh config/server.properties*
>>
>>
>>On Tue, Feb 7, 2012 at 5:00 AM, SuoNayi <suonayi2006@163.com> wrote:
>>
>>> Sorry for that I have found that hostname of the test server is of
>>> misconfigure.
>>> Here is new question when I send messages with 20 producers that each
>>> workes in it's own thread,
>>>
>>> My client throws exception of timeout:
>>> Exception in thread "Thread-2" java.net.ConnectException: Connection timed
>>> out: connect
>>> at sun.nio.ch.Net.connect(Native Method)
>>> at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
>>> at kafka.producer.SyncProducer.connect(SyncProducer.scala:172)
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:101)
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> at
>>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>>> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>>> at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>>> at kafka.producer.Producer.configSend(Producer.scala:167)
>>> at kafka.producer.Producer.send(Producer.scala:106)
>>> at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>>> at test.kafka.TestOutboundKfa$Sender.run(TestOutboundKfa.java:87)
>>>
>>> At 2012-02-07 10:26:43,SuoNayi <suonayi2006@163.com> wrote:
>>> >Hi all,
>>> >   I'm new to kafka.I set up zooker server and kafka server on my test
>>> server by following the quickstart.
>>> >   I also success running the example list in the quickstart,such as step
>>> 3 and step 4.
>>> >   Now I set up a project on my pc to run examples of directory named
>>> kafka-0.7.0-incubating-src\examples.
>>> >   Of course, I changed addresses  and ports of zk server and kafka that
>>> both pointed at my test server.
>>> >   When start the main function of class KafkaConsumerProducerDemo,it
>>> throws the following exception:
>>> >Exception in thread "main"
>>> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
>>> zookeeper server within timeout: 6000
>>> >at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:876)
>>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>>> >at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>>> >at
>>> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:63)
>>> >at kafka.producer.Producer.<init>(Producer.scala:53)
>>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>>> >at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>>> >at kafka.examples.Producer.<init>(Producer.java:35)
>>> >at
>>> kafka.examples.KafkaConsumerProducerDemo.main(KafkaConsumerProducerDemo.java:23)
>>> >   I have to specify the timeout value for zk,like this,
>>> zk.connectiontimeout.ms=10000 and the client could connect to zk server
>>> successfully.
>>> >   But another new exception is thrown again:
>>> >Exception in thread "Thread-0" java.nio.channels.ClosedChannelException
>>> >at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source)
>>> >at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
>>> >at
>>> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:48)
>>> >at kafka.network.Send$class.writeCompletely(Transmission.scala:76)
>>> >at
>>> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
>>> >at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:88)
>>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:87)
>>> >at kafka.producer.SyncProducer.send(SyncProducer.scala:117)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:116)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> >at
>>> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102)
>>> >at
>>> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>>> >at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>>> >at kafka.producer.ProducerPool.send(ProducerPool.scala:102)
>>> >at kafka.producer.Producer.zkSend(Producer.scala:143)
>>> >at kafka.producer.Producer.send(Producer.scala:105)
>>> >at kafka.javaapi.producer.Producer.send(Producer.scala:104)
>>> >at kafka.examples.Producer.run(Producer.java:45)
>>> >  Hope someone could help me, thanks very much!
>>> >
>>> >
>>> >SuoNayi
>>> >
>>> >
>>> >
>>>
>>
>>
>>
>>--
>>
>>/*
>>Joe Stein
>>http://www.linkedin.com/in/charmalloc
>>Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>*/

Mime
View raw message