metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?
Date Fri, 07 Jul 2017 18:47:06 GMT
This was my original inclination and Casey’s as well when we spoke.  I
think Nick has some good points however, so I created this thread.

Thanks for replying!


On July 7, 2017 at 14:36:51, Matt Foley (mattf@apache.org) wrote:

Hi all,
At the risk of getting suddenly unpopular (:-) I would like to argue the
other side.
Architecturally I disagree with having REST invoke Stellar, or in general
making Stellar the single point of contact for management functionality.
Several reasons:

1. The architectural component properly at the center of this discussion is
Configuration. The Metron configuration model is different from Hadoop, and
different from Storm, and different from NiFi. We want Stellar to be usable
in multiple environments. Hence our Configuration model should be external
from Stellar, not intrinsic to it.

2. Currently our Configuration model, while fairly explicit, lacks
easily-usable APIs. We should fix that, but not by making them be Stellar.
If you’re ready to make them be Stellar, it means you now know the APIs
that Configuration should have, and we should implement those – in Java.

3. REST APIs should be lightweight and stateless. There’s no benefit that I
see in having them call a language interpreter, when they should just be
calling the Configuration Java APIs.

4. Currently the Stellar REPL is the easiest way for a human to interact
with Configuration. I would claim that situation was evolved, not designed.
It makes sense for us Linux-heads to have a CLI for Configuration. But
having REST call a CLI? No. Backwards.

5. I think the only reason the issue came up is because there isn’t already
a Config API that is simple to call from the REST layer. It is correct that
the REST layer shouldn’t have to “fix” that, but neither should it hack the
solution by invoking Stellar. The correct architectural place for a simple
Config API is Configuration.

Thanks,
--Matt

On 7/7/17, 10:01 AM, "Nick Allen" <nick@nickallen.org> wrote:

> Are we talking about exposing an endpoint that just executes a stellar
statement?

No, that is not the case AFAIK, Ryan.

I see Otto's PR as more of a discussion around how to go about implementing
Management-ish functionality in the REST API. I think the goal here is
just to get consensus on an approach, not necessarily code.

At least that's my understanding because there is no mention of specific
management functionality in the JIRA that needs changed or added. Correct
me, if I misstated Otto.





On Fri, Jul 7, 2017 at 12:42 PM, Ryan Merriman <merrimanr@gmail.com> wrote:

> Makes sense to me. Are we talking about exposing an endpoint that just
> executes a stellar statement? We already have one but it's scope is
> limited to applying stellar transformations to a sample message. Assuming
> we would just add on to that controller. What Jiras are going to come out
> of this discussion?
>
> I'm all for adding as many rest endpoints as possible. It makes our
> platform much easier to understand and use for people who are not experts
> on Metron internals.
>
> > On Jul 7, 2017, at 11:07 AM, Otto Fowler <ottobackwards@gmail.com>
> wrote:
> >
> > Anyone else have feelings?
> >
> >
> > On July 7, 2017 at 11:06:32, Nick Allen (nick@nickallen.org) wrote:
> >
> > Like you mentioned, Otto, I think it makes more sense to have a REST
API
> > that is backed by Stellar functions executed in a JVM. That is, the
REST
> > API simply executes the right Stellar functions in a JVM. This makes it
> > very simple to reuse the same implementation (Stellar functions) across
> > multiple contexts; the REPL, Streaming, and in a REST API.
> >
> >
> >
> > On Sun, Jul 2, 2017 at 10:06 AM, Otto Fowler <ottobackwards@gmail.com>
> > wrote:
> >
> >> Bump
> >>
> >>
> >>> On June 13, 2017 at 00:11:52, Otto Fowler (ottobackwards@gmail.com)
> >> wrote:
> >>
> >> I have opened METRON–994 <https://issues.apache.org/
> jira/browse/METRON-994
> >> : STELLAR Shell and management should front the METRON REST api
> >>
> >> As Stellar management functions start overlapping with the REST api,
we
> > may
> >> want have stellar management backed by rest, and have a single main
api
> -
> >> rest.
> >>
> >> Nick brings up an excellent point that we should consider doing the
> >> opposite, back the rest api with the stellar implementation.
> >>
> >> After a little thought, I believe that this approach may have the
> > greatest
> >> pay off long term, as it will result (hopefully) in an increase of the
> >> number of STELLAR commands, that may be leveraged in different
contexts.
> >>
> >> But, I think this issue warrants more discussion from the group.
> >>
> >> The questions as I can see them (please add more or correct me ) are:
> >>
> >> - Should Stellar have a api which is fronted by multiple front ends?
> >> - If so, which is better suited, REST, STELLAR or other?
> >> - What are some trade offs | pay offs with each approach?
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message