metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: [DISCUSS] Metron Release - 0.7.1 next steps
Date Thu, 02 May 2019 15:12:13 GMT
I remember this now, but I’m not sure how I would have related this to a
parser aggregation pr honestly.


On May 2, 2019 at 07:54:13, Shane Ardell (shane.m.ardell@gmail.com) wrote:

Here's a link to the ngrx discussion thread from a few months back:
https://lists.apache.org/thread.html/06a59ea42e8d9a9dea5f90aab4011e44434555f8b7f3cf21297c7c87@%3Cdev.metron.apache.org%3E

On Thu, May 2, 2019 at 1:17 PM Otto Fowler <ottobackwards@gmail.com> wrote:

> If you can find a link in the archives for that thread, it would really
> help.
>
> I don’t think sending them up as one sensor would work…. as something
> quick. I think it is an interesting idea from a higher level that would
> need some more thought though ( IE: what if every sensor in the ui was a
> sensor group, and the existing entries where just groups of 1 ).
>
> As far as I can see, we have brought up the idea of a release ourselves,
I
> don’t see why we don’t just swarm this issue and get it right then
release.
>
>
>
> On May 2, 2019 at 04:16:31, Tamás Fodor (ftamas.mail@gmail.com) wrote:
>
> In PR#1360 we introduced a new state management strategy involving a new
> module called Ngrx. We had a discussion thread on this a few months ago
and
> we successfully convinced you about the benefits. This is one of the
> reasons why this PR is going to be still huge after cleaning up the
commit
> history. After you having a look at the changes and the feature itself,
> there's likely have questions about why certain parts work as they do.
The
> thing what I'd like to point out is that, yes, it probably takes more
time
> to get it in.
>
> In order to being able to release the RC, wouldn't it be an easy and
quick
> fix on the backend if it sent the aggregated parsers to the client as
they
> were one sensor? It's just an idea, it might be wrong, but at least we
> shouldn't have to wait until the aforementioned PR gets ready to be
merged
> to the master.
>
> On Wed, May 1, 2019 at 4:16 PM Justin Leet <justinjleet@gmail.com> wrote:
>
> > Short version: I'm in favor of #2 of 0.7.1 and #1 as a blocker for
0.8.0.
> > #3 seems like a total waste of time and effort.
> >
> > The wall of text version:
> > I agree this isn't "just the wrong thing shown", but for completely
> > different reasons.
> >
> > To be extremely clear about what the problem is: Our "dev" environment
> > (whose very name implies the audience is develops) uses a
> performance-based
> > advanced feature to ensure that all our of sample flows are regularly
run
> > and produce data. This feature has a bare minimal implementation to be
> > enabled via Ambari, which it currently is by default. This is because
of
> > the limited resources available that previously resulted in us turning
> off
> > Yaf, and therefore testing it during regular full dev runs. Right now
> > however, this feature is not exposed through the management UI, and
> > therefore it isn't obvious what the implications are. Am I missing
> anything
> > here?
> >
> > For users actually choosing to use the parser aggregation feature in a
> > non-full-dev environment, I'd expect substantially more care to be
> involved
> > given the lack of easy configuration for it (after all, why would you
> > bother running the aggregated parser alongside the regular parser? This
> > could be more explicitly stated, but again that feels like a doc
problem.
> > Right now I could essentially provide two of the same parser and create
> the
> > same problem, so right now aggregation is only special because it runs
on
> > dev by default). This is, in my opinion, primarily a first impression
> > problem and likely one of many areas that could use improved
> documentation.
> >
> > Quite frankly, I think the issue pointed out here could mostly be
> resolved
> > by documenting how the current aggregation is done in dev, and telling
> how
> > to change it. Especially for a 0.x.1 release, which is primarily bug
> > fixes. As can be inferred from my vote, I don't think this problem is a
> > problem that needs solving in a point release. I would support
improving
> > the documentation, both full-dev and for aggregation in general for the
> > 0.7.1 point release, while making a 0.8.0 release contingent upon the
> > outstanding PRs to enable it in the management UI.
> >
> > There are a couple deeper issues, imo, that I care substantially more
> about
> > than this in particular
> > * The dev environment is being used as our intro for users, because
it's
> > convenient for us to not maintain more environments (which has been a
> major
> > pain point in the past). Worse, the dev environment strongly implies
it's
> > for Metron developers, rather than people looking to build on top of
> > Metron. We need an actual strategy for providing end users a clean
> > impression of Metron (this could be clarifying what the expectations of
> > full dev are, renaming it to something like "full-demo", something more
> > involved, etc.). This is something that we've needed for awhile in
> general,
> > and includes larger topics like improving our website, potentially
> > improving the site book, actually publishing our Javadocs somewhere so
> > people can develop things easier, publishing out info about Stellar
> > functions in a better manner, etc.
> > * The fact that parsers are handled in Ambari at all. It's awful and
> leads
> > to situations like this. To the best of my knowledge, once we can do
> > chaining and aggregation in the Management UI, we should be able to
> > entirely divorce these two overlapping domains. I'd love to see parsers
> > ripped out of Ambari, then full-dev manages all the setup via REST. At
> that
> > point, we can easily tell everyone to just use the management UI.
> >
> > On Wed, May 1, 2019 at 7:23 AM Otto Fowler <ottobackwards@gmail.com>
> > wrote:
> >
> > > I think it would help if the full consequences of having the UI show
> the
> > > wrong status where listed.
> > >
> > > Someone trying metron, will, by default , see the wrong thing in the
UI
> > for
> > > the ONLY sensors they have that are running and doing data.
> > >
> > > What happens when they try to start them to make them work? One, two
or
> > > all?
> > > What happens when he edits them or try to add transformations? One,
two
> > or
> > > all?
> > > What other things can you do with the sensors in the ui? What
happens?
> > >
> > > Are we recommending aggregation on the list and elsewhere for users?
> Are
> > > we recommending something that is going to ensure they get into this
> > > situation?
> > >
> > > I think this is more than ‘just the wrong thing shown’ in the ui.
> > >
> > >
> > >
> > >
> > > On April 30, 2019 at 20:48:10, Michael Miklavcic (
> > > michael.miklavcic@gmail.com) wrote:
> > >
> > > The vote for RC1 did not pass and I'd like to kickstart some
discussion
> > > about what we should do.
> > >
> > > I started taking a look at PR#1360 and it looks like this isn't quite
> as
> > > close to being able go in as I had originally expected. I want to
talk
> > > about options here. It seems to me that we can:
> > >
> > > 1. Wait for PR#1360 to go in, but this is likely going to take more
> time
> > > than originally anticipated
> > > 2. Accept the issue in full dev, but add some notes in the developer
> > > docs about the current feature gap and why sensors aren't showing
> status
> > in
> > > the management UI when aggregation is enabled.
> > > 3. Find some other workable UI solution.
> > > 4. Other option?
> > >
> > > All things considered, I'm personally leaning towards #2 in the
> > short-term,
> > > but I think we should probably talk about this a bit before deciding
> what
> > > RC2 should be.
> > >
> > > Best,
> > > Mike
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message