nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 李磊 <l2254...@163.com>
Subject Issue with nifi startup performance
Date Sat, 17 Mar 2018 05:21:55 GMT
Hi all,

I use KYLO with NIFI-1.3.0 now. In KYLO, I create 100 feed with data-ingest template to transform
data form RDBMS to HDFS, which provided by kylo.Finally, more than 1000 processors were created
and there are 29 processors in the reuseable_templates. NIFI uses 6g memory.

The problem is that when I restart the NIFI, more than 7 mins cost. By modify the source code
and print log, I find that nearly 6 min costs in org.apache.nifi.controller.StandardFlowSynchronizer
Class<sync>, especially method <addProcessGroup>. By read the source code, I think
the function of <addProcessGroup> is analyze flow.xml.gz and restore the NIFI status
before turned off.

Is there any ways to improve the performance of <addProcessGroup>? We will create more
than 1000 feeds in KYLO in production, means more than 10000 processors will be existed in
NIFI.

Thanks!
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message