kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sa Li <sal...@gmail.com>
Subject Re: "java.io.IOException: Too many open files" error
Date Fri, 16 Jan 2015 00:02:30 GMT
Thanks for the reply, I have change the configuration and running to see if
any errors come out.

SL

On Thu, Jan 15, 2015 at 3:34 PM, István <leccine@gmail.com> wrote:

> Hi Sa Li,
>
> Depending on your system that configuration entry needs to be modified. The
> first parameter after the insert is the username what you use to run kafka.
> It might be your own username or something else, in the following example
> it is called kafkauser. On the top of that I also like to use soft and hard
> limits, when you hit the soft limit the system will log a meaningful
> message in dmesg so you can see what is happening.
>
> kafkauser soft nofile 80000
> kafkauser hard nofile 100000
>
> Hope that helps,
> Istvan
>
> On Thu, Jan 15, 2015 at 12:30 PM, Sa Li <salicn@gmail.com> wrote:
>
> > Hi, all
> >
> > We test our production kafka, and getting such error
> >
> > [2015-01-15 19:03:45,057] ERROR Error in acceptor
> (kafka.network.Acceptor)
> > java.io.IOException: Too many open files
> >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at sun.nio.ch.ServerSocketChannelImpl.accept(
> > ServerSocketChannelImpl.java:241)
> >         at kafka.network.Acceptor.accept(SocketServer.scala:200)
> >         at kafka.network.Acceptor.run(SocketServer.scala:154)
> >         at java.lang.Thread.run(Thread.java:745)
> >
> > I noticed some other developers had similar issues, one suggestion was "
> >
> > Without knowing the intricacies of Kafka, i think the default open file
> > descriptors is 1024 on unix. This can be changed by setting a higher
> ulimit
> > value ( typically 8192 but sometimes even 100000 ).
> > Before modifying the ulimit I would recommend you check the number of
> > sockets stuck in TIME_WAIT mode. In this case, it looks like the broker
> has
> > too many open sockets. This could be because you have a rogue client
> > connecting and disconnecting repeatedly.
> > You might have to reduce the TIME_WAIT state to 30 seconds or lower.
> >
> > "
> >
> > We increase the open file handles by doing this:
> >
> > insert "kafka - nofile 100000" in /etc/security/limits.conf
> >
> > Is that right to change the open file descriptors?  In addition, it says
> to
> > reduce the TIME_WAIT, where about to change this state? Or any other
> > solution for this issue?
> >
> > thanks
> >
> >
> >
> > --
> >
> > Alec Li
> >
>
>
>
> --
> the sun shines for all
>



-- 

Alec Li

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message