metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?
Date Fri, 07 Jul 2017 18:50:17 GMT
Yep, I think you are right, Matt.

Create a Configuration API that is called from wherever it is needed; the
REST API or a Stellar function.  The logic is more cohesive, simplifies
testing.  That's what we've done in most places.

On Fri, Jul 7, 2017 at 2:36 PM, Matt Foley <mattf@apache.org> wrote:

> Hi all,
> At the risk of getting suddenly unpopular (:-) I would like to argue the
> other side.
> Architecturally I disagree with having REST invoke Stellar, or in general
> making Stellar the single point of contact for management functionality.
> Several reasons:
>
> 1. The architectural component properly at the center of this discussion
> is Configuration.  The Metron configuration model is different from Hadoop,
> and different from Storm, and different from NiFi.  We want Stellar to be
> usable in multiple environments.  Hence our Configuration model should be
> external from Stellar, not intrinsic to it.
>
> 2. Currently our Configuration model, while fairly explicit, lacks
> easily-usable APIs.  We should fix that, but not by making them be
> Stellar.  If you’re ready to make them be Stellar, it means you now know
> the APIs that Configuration should have, and we should implement those – in
> Java.
>
> 3. REST APIs should be lightweight and stateless.  There’s no benefit that
> I see in having them call a language interpreter, when they should just be
> calling the Configuration Java APIs.
>
> 4. Currently the Stellar REPL is the easiest way for a human to interact
> with Configuration.  I would claim that situation was evolved, not
> designed.  It makes sense for us Linux-heads to have a CLI for
> Configuration.  But having REST call a CLI?  No.  Backwards.
>
> 5. I think the only reason the issue came up is because there isn’t
> already a Config API that is simple to call from the REST layer.  It is
> correct that the REST layer shouldn’t have to “fix” that, but neither
> should it hack the solution by invoking Stellar.  The correct architectural
> place for a simple Config API is Configuration.
>
> Thanks,
> --Matt
>
> On 7/7/17, 10:01 AM, "Nick Allen" <nick@nickallen.org> wrote:
>
>     > Are we talking about exposing an endpoint that just executes a
> stellar
>     statement?
>
>     No, that is not the case AFAIK, Ryan.
>
>     I see Otto's PR as more of a discussion around how to go about
> implementing
>     Management-ish functionality in the REST API.  I think the goal here is
>     just to get consensus on an approach, not necessarily code.
>
>     At least that's my understanding because there is no mention of
> specific
>     management functionality in the JIRA that needs changed or added.
> Correct
>     me, if I misstated Otto.
>
>
>
>
>
>     On Fri, Jul 7, 2017 at 12:42 PM, Ryan Merriman <merrimanr@gmail.com>
> wrote:
>
>     > Makes sense to me.  Are we talking about exposing an endpoint that
> just
>     > executes a stellar statement?  We already have one but it's scope is
>     > limited to applying stellar transformations to a sample message.
> Assuming
>     > we would just add on to that controller.  What Jiras are going to
> come out
>     > of this discussion?
>     >
>     > I'm all for adding as many rest endpoints as possible.  It makes our
>     > platform much easier to understand and use for people who are not
> experts
>     > on Metron internals.
>     >
>     > > On Jul 7, 2017, at 11:07 AM, Otto Fowler <ottobackwards@gmail.com>
>     > wrote:
>     > >
>     > > Anyone else have feelings?
>     > >
>     > >
>     > > On July 7, 2017 at 11:06:32, Nick Allen (nick@nickallen.org)
> wrote:
>     > >
>     > > Like you mentioned, Otto, I think it makes more sense to have a
> REST API
>     > > that is backed by Stellar functions executed in a JVM. That is,
> the REST
>     > > API simply executes the right Stellar functions in a JVM. This
> makes it
>     > > very simple to reuse the same implementation (Stellar functions)
> across
>     > > multiple contexts; the REPL, Streaming, and in a REST API.
>     > >
>     > >
>     > >
>     > > On Sun, Jul 2, 2017 at 10:06 AM, Otto Fowler <
> ottobackwards@gmail.com>
>     > > wrote:
>     > >
>     > >> Bump
>     > >>
>     > >>
>     > >>> On June 13, 2017 at 00:11:52, Otto Fowler (
> ottobackwards@gmail.com)
>     > >> wrote:
>     > >>
>     > >> I have opened METRON–994 <https://issues.apache.org/
>     > jira/browse/METRON-994
>     > >> : STELLAR Shell and management should front the METRON REST api
>     > >>
>     > >> As Stellar management functions start overlapping with the REST
> api, we
>     > > may
>     > >> want have stellar management backed by rest, and have a single
> main api
>     > -
>     > >> rest.
>     > >>
>     > >> Nick brings up an excellent point that we should consider doing
> the
>     > >> opposite, back the rest api with the stellar implementation.
>     > >>
>     > >> After a little thought, I believe that this approach may have the
>     > > greatest
>     > >> pay off long term, as it will result (hopefully) in an increase
> of the
>     > >> number of STELLAR commands, that may be leveraged in different
> contexts.
>     > >>
>     > >> But, I think this issue warrants more discussion from the group.
>     > >>
>     > >> The questions as I can see them (please add more or correct me )
> are:
>     > >>
>     > >> - Should Stellar have a api which is fronted by multiple front
> ends?
>     > >> - If so, which is better suited, REST, STELLAR or other?
>     > >> - What are some trade offs | pay offs with each approach?
>     > >>
>     >
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message