nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Doran <>
Subject Re: API to get all Policies
Date Fri, 09 Nov 2018 17:38:20 GMT
Hi Lars, 

I think as long as the following are true (it sounds like they are from what you have looked

1. the proposed endpoint does not require adding any additional Authorizable or policy to
protect, and 
2. the proposed endpoint does not expose any information that the authenticated client/user
would not already have access to view, and is merely acting as a convenience method to return
a list of things they could fetch individually

then in that case this is probably fine. No objection from me.

Any time we are adding a collection endpoint, my main concern is if pagination of the results
also needs to be added (i.e., if for typical usage of NiFi, the response size of the JSON
result would be larger than is reasonable to transmit in a single HTTP round trip, or if creating
the response would be unreasonable load on the server). In typical usage of NiFi, I don't
think the number of policies is that large (perhaps others can chime in if they feel differently?),
so it would come down to what is the size of a policy element when returned in a list. If
it is very large, you may also want to introduce a summary view/perspective of the policy
that reduces the amount of information to the minimal that is required for a list view...
I think that may already exist for NiFi in the AccessPolicySummary object, but it's been a
while since I've looked at the code so I may be forgetting the details or confusing it with
the NiFi Registry implementation, which does have a get all policies endpoint.

Lastly, take care that the Swagger annotations that are used to drive the Rest API documentation
are accurate. If you have any questions on that let me know. Happy to help review a PR if
you submit one.


On 11/9/18, 06:23, "Lars Francke" <> wrote:

    I've just tried implementing my new resource and it seems to work fine and
    as I expect it to. Also in regards to authorization. Users cannot see
    anything that they are not allowed to do anyway.
    Regarding your other comments: I agree that it's probably not a super
    common use case.
    Either way I'd love to use a API that I can access remotely as I need to
    connect to other systems as well (e.g. Kafka, HBase etc.) so I don't want
    to colocate my service on one of the NiFi machines.
    But yes I could probably get a list of all resources somehow using the API
    and then send one request per resource. But that seems overly complicated.
    So if you don't object I'd create a Jira.
    On Fri, Nov 9, 2018 at 10:01 AM Lars Francke <> wrote:
    > Andy,
    > that's a good question. I have to admit that I thought about it and then
    > saw that there is already an Authorizable for this scenario so I assumed
    > that part was already taken care of. So whoever has the permission to view
    > "access all policies" would also be able to use the API? Were you thinking
    > of something different?
    > Cheers,
    > Lars
    > On Fri, Nov 9, 2018 at 12:35 AM Andy LoPresto <>
    > wrote:
    >> Lars,
    >> What access controls do you anticipate putting on this API endpoint and
    >> what potential issues do you see? I could see this being abused if not
    >> secured very carefully, and it doesn’t seem like a common use case
    >> (notwithstanding your current requirement). Is this something that can be
    >> done by using the NiFi CLI to iterate/recurse through the various PGs and
    >> retrieve these policies?
    >> Andy LoPresto
    >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >> > On Nov 9, 2018, at 3:31 AM, Lars Francke <>
    >> wrote:
    >> >
    >> > Hi,
    >> >
    >> > I was tasked with writing a tool to generate a kind of "audit report".
    >> For
    >> > that I need to get all policies that people have across various systems.
    >> > NiFi is one of them.
    >> >
    >> > I see that we have a REST API for Policies but that doesn't expose a
    >> method
    >> > to expose _all_ policies. I'd like to add a REST endpoint that allows
    >> > retrieving all policies.
    >> > Before I open a Jira I wanted to get feedback whether this addition
    >> would
    >> > be acceptable.
    >> >
    >> > Implementation notes
    >> > This is how I see the current flow of requests from the
    >> > AccessPolicyResource to the actual AccessPolicyProider:
    >> >
    >> > AccessPolicyResource -> NiFiServiceFacade (StandardNiFiServiceFacade)
    >> > AccessPolicyDAO (StandardPolicyBasedAuthorizerDAO) ->
    >> AccessPolicyProvider
    >> >
    >> > Fortunately the AccessPolicyProvider already has a method to retrieve
    >> all
    >> > policies. Should there be custom implementations by third-parties they
    >> > already support the necessary methods and I believe the classes that
    >> need
    >> > to be changed are all NiFi internal:
    >> >
    >> > * AccessPolicyResource
    >> > * NiFiServiceFacade
    >> > * StandardNiFiServiceFacade
    >> > * AccessPolicyDAO
    >> > * StandardPolicyBasedAuthorizerDAO
    >> > * And probably a bunch of others especially test classes
    >> >
    >> > If I don't hear any objections I will open a Jira issue and would try
    >> and
    >> > provide a patch.
    >> >
    >> > Cheers,
    >> > Lars
View raw message