nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dweep Sharma <dweep.sha...@redbus.com>
Subject Nifi capabilities
Date Mon, 05 Aug 2019 05:10:57 GMT
Hi All,

I have been using Nifi to setup some pipelines now. Before I can absorb
more use cases into this, I need to understand a few capabilities

1) Can we setup an user authentication before the web application. If yes,
is there a way we can have role based access for processor groups. I would
like certain teams working on only specific groups and not control all.

2) If the major use case would only involve reading from RMQ, KAFKA convert
to parquet and store in S3, does it make sense to setup a cluster or just
vertical scaling is good ?

3) Are the flow files in the queues (connections between processors)
persisted?. Any machine failure or restart would cause a loss of data ? For
instance messages are dequeued form RMQ and lost due to failure. Which
would be a best way to handle this ? I think maintaining a low back
pressure (threshold) can help mitigate the loss

4) Does the Kafka consumer, by default consume all partitions or is there a
way to control that.

5) Can we have some of the metrics of processors pushed out as
notifications or alerts (flow file count in / out or errors etc)

It would be great, if someone could share resources that address these.

Thanks in advance.

-Dweep

-- 
*::DISCLAIMER::

----------------------------------------------------------------------------------------------------------------------------------------------------


The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
<http://redbus.in/>com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

Mime
View raw message