metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Elliston Ball <si...@simonellistonball.com>
Subject Re: [DISCUSS] Metron Management UI
Date Sat, 18 Feb 2017 10:42:42 GMT
At the moment the UI relies on Kafka topics already existing, and having data flowing into
them to pull samples for the parser config panel. However, you can paste alternative samples
if kafka isn’t already setup. As such, the assumption is that the topics would already have
data flowing from something like nifi, or some other producer before you got round to configuring
a sensor in this UI. Would that work, or would you expect to be creating the parser config
before the data was flowing into kafka? In the later case, we could probably rely on copy-pasted
samples to design the parser, and then auto-create topics if not already existing on parser
save.

Simon


> On 17 Feb 2017, at 23:42, Zeolla@GMail.com <zeolla@gmail.com> wrote:
> 
> Well, I think that a UI that does what you show is a huge step forward and
> I definitely wouldn't want to stop it from getting into the code because of
> any of the additional features that we are talking about here.
> 
> That said, I really would like to see the Kafka topic creation as a part of
> the UI because without it, data can't move.  Templates would be a second
> priority that I see as important, but depending on how Otto's thread goes
> how that could would may change and I wouldn't push to have that in a MVP
> UI.
> 
> Aside from that, I still agree with my previous comments regarding nice to
> have features, but definitely not needed for a first run.
> 
> Jon
> 
> On Fri, Feb 17, 2017, 1:24 PM Houshang Livian <hlivian@hortonworks.com>
> wrote:
> 
>> Hi Dima and Jon,
>> 
>> Do you think the UI:
>> 
>> http://imgur.com/a/QAQyu?
>> 
>> 
>> ...Is a reasonable place to start as it is?… or do we need to have the
>> Elastic Search Templates right away?
>> Are the templates a MUST HAVE in order to add this UI to Metron?
>> 
>> As Simon said, I think we can file a JIRA on the templates and start
>> working on what that looks like.
>> 
>> What are your thoughts?
>> 
>> 
>> 
>> On 2/17/17, 10:10 AM, "Simon Elliston Ball" <simon@simonellistonball.com>
>> wrote:
>> 
>>> Seems like the UI discussion has fallen off the inboxes.
>>> 
>>> We should definitely open up a separate Jira and discussion on the best
>> way to process elastic search template management. This feels like a non-UI
>> question to me, since it would apply equally to any other parser config
>> method. I’ll start a separate discuss for that.
>>> 
>>> In the meantime it would be great to hear more from the community on the
>> UI at http://imgur.com/a/QAQyu?
>>> 
>>> Is there anything you see missing around sensor config here?
>>> Does it look useful?
>>> Does the flow match the way you would currently think about setting up
>> sensors?
>>> 
>>> It would be great to get some thoughts.
>>> 
>>> Simon
>>> 
>>> 
>>>> On 1 Feb 2017, at 12:07, Simon Elliston Ball <
>> Simon@simonellistonball.com> wrote:
>>>> 
>>>> The MPack creates elastic templates for the standard parsers included
>> in the pack.
>>>> 
>>>> However, when creating new sensors we should be able to infer Elastic
>> templates from the schema. There are certain fields that we need to mark
>> types for to generate those templates (e.g. date / time fields). This goes
>> back to the debate we had a while ago about, for example, typing in the
>> grok parser, but applies for others too. I suspect that since we should be
>> able to infer all the types needed for templates, this is more an excellent
>> feature enhancement for the REST api layer, unless anyone can think of a
>> reason we would need explicit type definition in the
>>>> 
>>>> 
>>>> 
>>>>> On 1 Feb 2017, at 19:54, Nick Allen <nick@nickallen.org> wrote:
>>>>> 
>>>>> The Index templates are currently handled in the Ambari MPack (unless
>> you
>>>>> are suggesting some additional functionality beyond what the Ambari
>> MPack
>>>>> gives us today.)
>>>>> 
>>>>> I think there is always going to be a fuzzy, gray line between what the
>>>>> Ambari MPack should do and what the Management UI should do.  I think
>> it
>>>>> works fine in Ambari today, but I could be convinced otherwise.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Wed, Feb 1, 2017 at 9:51 AM, Dima Kovalyov <Dima.Kovalyov@sstech.us
>>> 
>>>>> wrote:
>>>>> 
>>>>>> Thank you Houshang,
>>>>>> 
>>>>>> That looks absolutely great, loved it!
>>>>>> Will it make sense to also add ElasticSearch index  templates
>>>>>> querying/creation here?
>>>>>> 
>>>>>> - Dima
>>>>>> 
>>>>>> On 02/01/2017 02:06 AM, Houshang Livian wrote:
>>>>>>> Hello Metron Community,
>>>>>>> 
>>>>>>> We have constructed a Management Module UI, built on top of
>> METRON-503
>>>>>> (REST API) (currently under review). This Module gives users the
>> ability to
>>>>>> setup and administer much of the product through the UI.
>>>>>>> 
>>>>>>> Here are some screens to show you what we are thinking. Please
take a
>>>>>> look:
>>>>>>> http://imgur.com/a/QAQyu?
>>>>>>> 
>>>>>>> Does this look like a reasonable place to start?
>>>>>>> Is there anything that is an absolute MUST have or MUST NOT have?
>>>>>>> 
>>>>>>> Houshang Livian
>>>>>>> Senior User Experience Designer
>>>>>>> Hortonworks
>>>>>>> 
>>>>>>> www.hortonworks.com<http://www.hortonworks.com/>
>>>>>>> ________________________________
>>>>>>> 
>>>>>>> Mobile: (831) 521-4176<tel:(831)%20521-4176>
>>>>>>> HLivian@hortonworks.com<mailto:HLivian@hortonworks.com>
>>>>>>> ________________________________
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Nick Allen <nick@nickallen.org>
>>>> 
>>> 
>> 
> -- 
> 
> Jon
> 
> Sent from my mobile device


Mime
View raw message