nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: High CPU usage in FileSystemRepository.java
Date Thu, 19 Nov 2015 19:16:05 GMT
Adam, thanks for doing all the research and pointing out where the problem is. It is definitely
a bug.

Joe I’ve raised the https://issues.apache.org/jira/browse/NIFI-1200

Cheers
Oleg

On Nov 19, 2015, at 1:18 PM, Joe Percivall <joepercivall@yahoo.com<mailto:joepercivall@yahoo.com>>
wrote:

Hello Adam,


Are you still seeing high cpu usage?

Sorry no has gotten back to you sooner, we are all working very hard to get 0.4.0 out.

Joe

- - - - - -
Joseph Percivall
linkedin.com/in/Percivall<http://linkedin.com/in/Percivall>
e: joepercivall@yahoo.com




On Friday, November 13, 2015 4:10 PM, Adam Lamar <adamonduty@gmail.com> wrote:



Mark,

For this development system, I'm running the packaged OpenJDK from
Ubuntu 14.04:

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)

Interestingly, I tried another system running the Oracle JDK (same
version) and didn't see the same issue. Though it does seem to benefit
from the additional sleep in that loop, but just barely (maybe 1% CPU
difference - I could be making that up).

I hadn't uncommented those values, but I tried with no noticeable
difference on the OpenJDK system.

Hope that helps,
Adam


On 11/13/15 5:28 AM, Mark Payne wrote:
Adam,

What version of Java are you running?

Do you have the following lines from conf/bootstrap.conf uncommented, or are they still commented
out?

java.arg.7=-XX:ReservedCodeCacheSize=256m
java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m
java.arg.9=-XX:+UseCodeCacheFlushing
java.arg.11=-XX:PermSize=128M
java.arg.12=-XX:MaxPermSize=128M

Thanks
-Mark


On Nov 13, 2015, at 12:28 AM, Joe Witt <joe.witt@gmail.com> wrote:

sorry - i see now :-)

Thanks for the analysis.  Will dig in.

Joe

On Fri, Nov 13, 2015 at 12:28 AM, Joe Witt <joe.witt@gmail.com> wrote:
Adam,

Are you on a recent master build?

Thanks
Joe

On Fri, Nov 13, 2015 at 12:27 AM, Adam Lamar <adamonduty@gmail.com> wrote:
Hi everybody!

I'm following up from my previous thread about high CPU usage in GetSQS. I
ran into high CPU usage while developing a patch for that processor, and
while investigating with "top", I noticed one NiFi thread in particular
showed high CPU usage, even after turning off all processors and restarting
NiFi.

A jstack showed this thread was busy at FileSystemRepository.java line 1287
[1]. Since that is a continue statement, it suggests that the thread was
churning in the surrounding for loop.

I didn't debug any further, but I did add a sleep statement just before the
continue, and CPU usage dropped wildly, settling around 2-4%.

I hope this is useful information, and I'm happy to debug further if needed.

Cheers,
Adam

[1]
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java#L1287


Mime
View raw message