kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Andruff <matt.andr...@gmail.com>
Subject Re: Kafka : To meny files open
Date Wed, 03 Jan 2018 16:10:33 GMT
You should check the kafka docs.  You need to increase your open file
descriptors.

Here's a link that describes  how to do
it.  (Ignore that this is for hdp the ulimit part is what you need to pay
attend to.)

You can also check kafka docs too they say to change open file descriptors
to 100,000 I believe.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_security/content/kerb-config-limits.html


On Jan 3, 2018 10:48 AM, "daniel brown" <daniel.w.brown1@gmail.com> wrote:

Hello,



Currently Zookeeper and kafka do not  work appropriately and the logs are
not streaming to logstash. The var-log partitions across the zookeeper
cluster are filled up to 100 percent.  We are getting the following
messages when we investigated the logs and the services status:



1 . Zookeeper logs

2018-01-03 06:16:55,001 [myid:1] - WARN
[SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread

2018-01-03 06:23:08,943 [myid:1] - WARN
[RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id
3, my id = 1, error = java.io.EOFException

        at java.io.DataInputStream.readInt(DataInputStream.java:392)

        at
org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(
QuorumCnxManager.java:795)

2018-01-03 06:23:08,943 [myid:1] - WARN
[RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker

2018-01-03 06:23:08,943 [myid:1] - WARN  [QuorumPeer[myid=1]/0.0.0.0:2181
:Follower@87] - Exception when following the leader java.io.EOFException

        at java.io.DataInputStream.readInt(DataInputStream.java:392)

        at org.apache.





**Consumer check output



kafka-console-consumer --bootstrap-server localhost:2181 --topic
RCCSWA_DHCP_1 --from-beginning

[2018-01-03 07:39:27,559] WARN Bootstrap broker localhost:2181 disconnected
(org.apache.kafka.clients.NetworkClient)

[2018-01-03 07:39:27,660] WARN Bootstrap broker localhost:2181 disconnected
(org.apache.kafka.clients.NetworkClient)

[2018-01-03 07:39:27,761] WARN Bootstrap broker localhost:2181 disconnected
(org.apache.kafka.clients.NetworkClient)



 2. Kafka Logs

tail -f server.log.2018-01-02-13

        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)

        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)

        at kafka.network.Acceptor.accept(SocketServer.scala:326)

        at kafka.network.Acceptor.run(SocketServer.scala:269)

        at java.lang.Thread.run(Thread.java:748)

[2018-01-02 13:30:01,124] ERROR Error while accepting connection
(kafka.network.Acceptor)

java.io.IOException: Too many open files

        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)

        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)

        at sun.nio.ch.ServerSocketChan





Thanks.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message