kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ziliang Chen <zlchen....@gmail.com>
Subject Kafka Connect high CPU / Memory usage
Date Tue, 23 Jan 2018 21:57:52 GMT
Hi,

I run Kafka Connect 0.11.2 (Kafka version) with only one instance and 6
tasks in it for a while, it then showed very high CPU usage ~250 % and
memory consumption 10 Gb memory and it reported consume group rebalance
warning. After the Kafka connect was killed and restarted, it showed the
same symptom, very high cpu usage and quickly consume 10 GB memory even
there is no tasks running there.  The start log showed below.

...
Jan 23, 2018 1:07:58 AM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The
(sub)resource method createConnector in org.apache.kafka.connect.
runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectors in
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains
empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in
org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource
contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.
runtime.rest.resources.RootResource contains empty path annotation.

[2018-01-23 01:07:58,388] INFO Started
o.e.j.s.ServletContextHandler@5b936410{/,null,AVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:744)
[2018-01-23 01:07:58,404] INFO Started ServerConnector@33bef845{HTTP/1.1}{
0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266)
[2018-01-23 01:07:58,405] INFO Started @3854ms (org.eclipse.jetty.server.
Server:379)
[2018-01-23 01:07:58,406] INFO REST server listening at
http://172.17.0.2:8083/, advertising URL http://172.17.0.2:8083/
 (org.apache.kafka.connect.runtime.rest.RestServer:150)
[2018-01-23 01:07:58,406] INFO Kafka Connect started
(org.apache.kafka.connect.runtime.Connect:55)

screenshot about the CPU and memory usage is attached as below.

[image: Inline image 1]

Could you please shed some light here ?

Thank you !
-- 
Regards, Zi-Liang

Mail:zlchen.ken@gmail.com

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message