metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: log parsers-
Date Mon, 19 Sep 2016 22:23:49 GMT
It may.  We may get to the point where we can handle more complex objects.
Until then, I made the approach pluggable and put up a quick JIRA/PR  for
people to tinker with here
<https://github.com/apache/incubator-metron/pull/261>.  I had this dude
already done in a long languishing branch, so I figure I might as well see
if it's useful.

On Mon, Sep 19, 2016 at 10:25 AM, David Lyle <dlyle65535@gmail.com> wrote:

> Does Elasticsearch Nested Objects [1] help with that?
>
> [1]
> https://www.elastic.co/guide/en/elasticsearch/guide/
> current/nested-objects.html
>
> On Mon, Sep 19, 2016 at 9:43 AM, Casey Stella <cestella@gmail.com> wrote:
>
> > So, just curious, what kind of behavior would you expect if the JSON had
> a
> > complex map inside of it (e.g. { "foo" : { "bar" : 1 }, "numeric" : 7 }
> )?
> > As it is now, our indices in ES do not handle complex structures.  Would
> > you want those fields dropped, folded in to the larger structure (e.g. {
> > "foo.bar" : 1, "numeric" : 7 }) or an error to occur?  Or, would you want
> > that to be pluggable?
> >
> > Casey
> >
> > On Mon, Sep 19, 2016 at 3:56 AM, Egon Kidmose <kidmose@gmail.com> wrote:
> >
> > > +1 on the pass through parser that just sends JSON onwards
> > >
> > >
> > >
> > > Mvh. / BR
> > > Egon Kidmose
> > >
> > > On Thu, Sep 15, 2016 at 6:08 PM, Casey Stella <cestella@gmail.com>
> > wrote:
> > >
> > > > Just to tack onto the parser thread (love it, btw :).  I'd love to
> see
> > a
> > > > couple of general ones:
> > > >
> > > >    - Arbitrary XML with the ability to map xpaths to columns in the
> > JSON
> > > >    - Pass through parser (in the situation where your data is a JSON
> > map
> > > >    already)
> > > >
> > > >
> > > > On Thu, Sep 15, 2016 at 11:36 AM, Zeolla@GMail.com <zeolla@gmail.com
> >
> > > > wrote:
> > > >
> > > > > I would love to tack onto this thread - we are also working on some
> > > > parsers
> > > > > for various technologies and plan to contribute them back.  If
> others
> > > are
> > > > > not working on it we will do it ourselves, but it would be great
to
> > > speed
> > > > > things up with help from the community.
> > > > >
> > > > > - Shibboleth v2 (link
> > > > > <https://wiki.shibboleth.net/confluence/display/SHIB2/IdPLogging>)
> > > > > - 389 Directory Server (link
> > > > > <https://wiki.shibboleth.net/confluence/display/SHIB2/IdPLogging>)
> > > > > - OpenLDAP (link <http://www.openldap.org/>)
> > > > > - Aruba ClearPass
> > > > > - sshd
> > > > > - FreeRADIUS
> > > > >
> > > > > Jon
> > > > >
> > > > > On Thu, Sep 15, 2016 at 9:57 AM Joe Gumke <joegumke@gmail.com>
> > wrote:
> > > > >
> > > > > > Let me know if I can be of any assistance. Ill need documentation
> > and
> > > > > such
> > > > > > to help build the parsers.
> > > > > >
> > > > > > On Sep 14, 2016 17:58, "Satish Abburi" <Satish.Abburi@sstech.us>
> > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Thanks, timelines are 2 weeks from now. Thanks.
> > > > > > >
> > > > > > > From: Poornima Ravindra Mulukutla <gprmulukutla@gmail.com<
> > mailto:
> > > > > > > gprmulukutla@gmail.com>>
> > > > > > > Reply-To: "user@metron.incubator.apache.org<mailto:user@metron
> .
> > > > > > > incubator.apache.org>" <user@metron.incubator.apache.org
> <mailto:
> > > > > > > user@metron.incubator.apache.org>>
> > > > > > > Date: Wednesday, September 14, 2016 at 3:26 PM
> > > > > > > To: "user@metron.incubator.apache.org<mailto:user@metron.
> > > > > > > incubator.apache.org>" <user@metron.incubator.apache.org
> <mailto:
> > > > > > > user@metron.incubator.apache.org>>
> > > > > > > Cc: "dev@metron.incubator.apache.org<mailto:dev@metron.
> > > > > > > incubator.apache.org>" <dev@metron.incubator.apache.org
> <mailto:
> > > > > > dev@metron.
> > > > > > > incubator.apache.org>>
> > > > > > > Subject: Re: log parsers-
> > > > > > >
> > > > > > > Thank you
> > > > > > >
> > > > > > > I am happy to take up ASA log file analyser, what is the
> timeline
> > > you
> > > > > are
> > > > > > > looking for so that I will plan accordingly?
> > > > > > >
> > > > > > > In the past I have done BlueCoat log analyser when I was
doing
> > > > research
> > > > > > on
> > > > > > > HTTP specification (published a patent has created big
change
> in
> > > HTTP
> > > > > > > designs), recently adopted for the Microsoft IE 11
> > > > > > >
> > > > > > > On Wed, Sep 14, 2016 at 6:54 PM, Satish Abburi <
> > > > > Satish.Abburi@sstech.us<
> > > > > > > mailto:Satish.Abburi@sstech.us>> wrote:
> > > > > > >
> > > > > > > Hi, we are trying to build parsers for our Phase1 demo
on
> Metron
> > > > > > platform.
> > > > > > > Would like to find, if anyone already has these parsers
> > developed.
> > > > > > > We already started working on  Windows parser, rest planning
to
> > > start
> > > > > > this
> > > > > > > week. We can leverage if some thing avaialble or collaborate
> > > > > > appropriately.
> > > > > > >
> > > > > > >
> > > > > > >   *   ASA (Firewall) Metron-363
> > > > > > >   *   Windows (Desktop) - METRON-165
> > > > > > >   *   Unix (OS) Metron-175
> > > > > > >   *   Email
> > > > > > >   *   BlueCoat(Proxy) METRON-162
> > > > > > >
> > > > > > > Thanks for your help!
> > > > > > > Satish
> > > > > > >
> > > > > > >
> > > > > >
> > > > > --
> > > > >
> > > > > Jon
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message