metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Leet <>
Subject Re: [DISCUSS] Metron Release - 0.7.1 next steps
Date Wed, 01 May 2019 14:16:33 GMT
Short version: I'm in favor of #2 of 0.7.1 and #1 as a blocker for 0.8.0.
#3 seems like a total waste of time and effort.

The wall of text version:
I agree this isn't "just the wrong thing shown", but for completely
different reasons.

To be extremely clear about what the problem is: Our "dev" environment
(whose very name implies the audience is develops) uses a performance-based
advanced feature to ensure that all our of sample flows are regularly run
and produce data. This feature has a bare minimal implementation to be
enabled via Ambari, which it currently is by default.  This is because of
the limited resources available that previously resulted in us turning off
Yaf, and therefore testing it during regular full dev runs. Right now
however, this feature is not exposed through the management UI, and
therefore it isn't obvious what the implications are. Am I missing anything

For users actually choosing to use the parser aggregation feature in a
non-full-dev environment, I'd expect substantially more care to be involved
given the lack of easy configuration for it (after all, why would you
bother running the aggregated parser alongside the regular parser? This
could be more explicitly stated, but again that feels like a doc problem.
Right now I could essentially provide two of the same parser and create the
same problem, so right now aggregation is only special because it runs on
dev by default).  This is, in my opinion, primarily a first impression
problem and likely one of many areas that could use improved documentation.

Quite frankly, I think the issue pointed out here could mostly be resolved
by documenting how the current aggregation is done in dev, and telling how
to change it.  Especially for a 0.x.1 release, which is primarily bug
fixes. As can be inferred from my vote, I don't think this problem is a
problem that needs solving in a point release.  I would support improving
the documentation, both full-dev and for aggregation in general for the
0.7.1 point release, while making a 0.8.0 release contingent upon the
outstanding PRs to enable it in the management UI.

There are a couple deeper issues, imo, that I care substantially more about
than this in particular
* The dev environment is being used as our intro for users, because it's
convenient for us to not maintain more environments (which has been a major
pain point in the past). Worse, the dev environment strongly implies it's
for Metron developers, rather than people looking to build on top of
Metron. We need an actual strategy for providing end users a clean
impression of Metron (this could be clarifying what the expectations of
full dev are, renaming it to something like "full-demo", something more
involved, etc.). This is something that we've needed for awhile in general,
and includes larger topics like improving our website, potentially
improving the site book, actually publishing our Javadocs somewhere so
people can develop things easier, publishing out info about Stellar
functions in a better manner, etc.
* The fact that parsers are handled in Ambari at all. It's awful and leads
to situations like this. To the best of my knowledge, once we can do
chaining and aggregation in the Management UI, we should be able to
entirely divorce these two overlapping domains.  I'd love to see parsers
ripped out of Ambari, then full-dev manages all the setup via REST. At that
point, we can easily tell everyone to just use the management UI.

On Wed, May 1, 2019 at 7:23 AM Otto Fowler <> wrote:

> I think it would help if the full consequences of having the UI show the
> wrong status where listed.
> Someone trying metron, will, by default , see the wrong thing in the UI for
> the ONLY sensors they have that are running and doing data.
> What happens when they try to start them to make them work? One, two or
> all?
> What happens when he edits them or try to add transformations? One, two or
> all?
> What other things can you do with the sensors in the ui?  What happens?
> Are we recommending aggregation on the list and elsewhere for users?  Are
> we recommending something that is going to ensure they get into this
> situation?
> I think this is more than ‘just the wrong thing shown’ in the ui.
> On April 30, 2019 at 20:48:10, Michael Miklavcic (
> wrote:
> The vote for RC1 did not pass and I'd like to kickstart some discussion
> about what we should do.
> I started taking a look at PR#1360 and it looks like this isn't quite as
> close to being able go in as I had originally expected. I want to talk
> about options here. It seems to me that we can:
> 1. Wait for PR#1360 to go in, but this is likely going to take more time
> than originally anticipated
> 2. Accept the issue in full dev, but add some notes in the developer
> docs about the current feature gap and why sensors aren't showing status in
> the management UI when aggregation is enabled.
> 3. Find some other workable UI solution.
> 4. Other option?
> All things considered, I'm personally leaning towards #2 in the short-term,
> but I think we should probably talk about this a bit before deciding what
> RC2 should be.
> Best,
> Mike

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message