nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Testing a nifi flow via junit
Date Tue, 05 Jan 2016 22:40:31 GMT
Vincent,

I would be vary wary about trusting performance results that you obtain by using the Mock
Framework.
The mock framework is intended to be used only for testing correctness, not performance. It
has very
different threading characteristics than the "actual" NiFi framework, and it uses very different
FlowFile, 
Content, and Provenance Repositories. Processor A may perform far better than Processor B
in the 
mock framework, but that does not by any means guarantee that it will also perform better
in a live
environment.

Thanks
-Mark

> On Jan 5, 2016, at 5:17 PM, Vincent Russell <vincent.russell@gmail.com> wrote:
> 
> Hello Aldrin,
> 
> Thanks for the response.
> 
> My current use case is that I would like to chain several processors together and write
a performance test against that mini flow and then be free to modify the processors that are
in the chain and see how performance changes.    I think I may be able to chain several TestRunners
together to achieve my goal, although this isn't ideal.
> 
> Ideally I'd be able to provide the TestRunner with multiple Processors and identify how
the processors are chained together.
> 
> Thanks,
> 
> 
> 
> On Mon, Jan 4, 2016 at 3:26 PM, Aldrin Piri <aldrinpiri@gmail.com <mailto:aldrinpiri@gmail.com>>
wrote:
> Hello Vincent,
> 
> This is something that does not exist and there have been a few threads on this topic
[1][2].
> 
> Summarily, these tools do not currently exist due to the preference of using the interactive
and real-time command and control over the flow as well as the increasing difficulty of maintaining
flows as they grow and evolve.
> 
> There are some good tips on how other people have tackled the problem in the linked message
threads.  One alternative suggestion is making use of NiFi's template functionality [3] to
work on stubbing out flows on a different instance and use that to promote an entire flow
or segment to another system.  Templates are an area we are planning to both enhance and mature
as laid out in some of our feature proposals [4][5].
> 
> Please let us know if this is accomplishes the functionality you are looking for or if
we are coming up a bit short on some of what your needs are for integration level testing.
 Your case is common and certainly one we need to execute on well.  Any feedback you can provide
from your perspective both in view of the current state of templates and the path forward
as laid out in the proposals would be much appreciated!
> 
> Thanks!
> 
> --aldrin  
> 
> [1] https://mail-archives.apache.org/mod_mbox/nifi-dev/201502.mbox/%3CBAY406-EAS4223C0996501AE0F51DEE77CE160@phx.gbl%3E
<https://mail-archives.apache.org/mod_mbox/nifi-dev/201502.mbox/%3CBAY406-EAS4223C0996501AE0F51DEE77CE160@phx.gbl%3E>
> [2] http://apache-nifi-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
<http://apache-nifi-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811>
> [3] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates <https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates>
> [4] https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry
<https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry>
> [5] https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows
<https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows>
> 
> 
> On Mon, Jan 4, 2016 at 1:37 PM, Vincent Russell <vincent.russell@gmail.com <mailto:vincent.russell@gmail.com>>
wrote:
> All,
> 
> I see that there is a way to test a single processor with the TestRunner (StandardProcessorTestRunner)
class, but is there a way to set up an integration test to test a complete flow or a subset
of a flow?
> 
> Thank you,
> Vincent
> 
> 


Mime
View raw message