metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zeolla@GMail.com" <zeo...@gmail.com>
Subject Re: [DISCUSS] The bro kafka plugin
Date Wed, 05 Apr 2017 17:29:19 GMT
Okay great, thanks.  Would you mind throwing in an INFRA ticket for the new
repo?  I can take it all from there.

Does anybody know if we have ASF resources to help answer the above legal
question?

Jon

On Wed, Apr 5, 2017 at 1:26 PM Nick Allen <nick@nickallen.org> wrote:

> (1) I am not sure if licensing is a problem here.
>
> (2) I am OK with whatever we need to get this effort done and under ASF.
>
>
> On Wed, Apr 5, 2017 at 1:12 PM, Zeolla@GMail.com <zeolla@gmail.com> wrote:
>
> > I'm working on this
> > <https://github.com/JonZeolla/incubator-metron/tree/METRON-348> in
> > preparation for the new repo and migration to a package.  It looks like
> in
> > bro-plugins COPYING
> > <https://github.com/bro/bro-plugins/blob/master/kafka/COPYING>
> attributes
> > to Nick, but in our version COPYING
> > <https://github.com/apache/incubator-metron/tree/master/
> > metron-sensors/bro-plugin-kafka/COPYING>
> > points to the Apache License.  Same with MAINTAINER (this
> > <https://github.com/apache/incubator-metron/blob/master/
> > metron-sensors/bro-plugin-kafka/MAINTAINER>
> > vs this <https://github.com/bro/bro-plugins/blob/master/kafka/MAINTAINER
> > >).
> > I assume when we package this up and host it in Apache we need to give it
> > the Apache license, and point to Metron for MAINTAINER.  My questions
> are:
> >
> > 1.  Is there any legal/licensing concern here?  I am taking changes from
> > the bro-plugins version and pulling it into the Apache-hosted code.
> IANAL
> > 2.  Nick - are you OK with these changes?
> >
> > Jon
> >
> > On Mon, Apr 3, 2017 at 3:50 PM Zeolla@GMail.com <zeolla@gmail.com>
> wrote:
> >
> > > Can someone on the PMC submit a ticket to INFRA?  It looks like
> > > <https://www.apache.org/dev/infra-contact> committers aren't supposed
> > to.
> > >
> > > Jon
> > >
> > > On Fri, Mar 31, 2017 at 4:23 PM Zeolla@GMail.com <zeolla@gmail.com>
> > wrote:
> > >
> > > I would be happy to try it again but I attempted to do that before with
> > > bro packages and it failed to be able to handle it.  I also tried using
> > > branches of a repo with bro but that similarly failed (and was a pretty
> > bad
> > > idea to start with).
> > >
> > > Jon
> > >
> > > On Fri, Mar 31, 2017, 3:24 PM Matt Foley <mattf@apache.org> wrote:
> > >
> > > We should be able to request just one alternate repo from INFRA, and
> put
> > a
> > > top hierarchical level in it that doesn’t include a maven pom.  As far
> as
> > > maven and clients are concerned, it
> > >
> > > just increases by 1 the path length to the root of the repo.
> > >
> > > On 3/31/17, 10:30 AM, "Zeolla@GMail.com" <zeolla@gmail.com> wrote:
> > >
> > >     Once we agree on a repo location to host this, I would be happy to
> > put
> > >     together the package and update our environments to use bro-pkg to
> > > install
> > >     the plugin.  I have created METRON-813
> > >     <https://issues.apache.org/jira/browse/METRON-813> to track this
> and
> > >     changed METRON-348 <
> https://issues.apache.org/jira/browse/METRON-348
> > >
> > > to be
> > >     a sub-task.
> > >
> > >     Otto - the bro packages model doesn't allow colocation with
> anything
> > > else.
> > >     That said, if we have two similar situations, and given the INFRA
> > > example
> > >     <https://issues.apache.org/jira/browse/INFRA-7060> Casey linked to
> > > before
> > >     was requesting 9 repos, perhaps we just request two repos.  Would
> > > someone
> > >     else mind putting that request in?
> > >
> > >     Jon
> > >
> > >     On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler <
> > ottobackwards@gmail.com>
> > >     wrote:
> > >
> > >     Could we create a separate repo for more than on thing?  like put …
> > um
> > >     let’s say
> > >     a maven plugin and the bro plugin?
> > >
> > >
> > >
> > >     On March 31, 2017 at 12:30:25, Nick Allen (nick@nickallen.org)
> > wrote:
> > >
> > >     I agree with everything that I've read.
> > >
> > >     One of the guys from Bro had contacted me a while back, letting me
> > know
> > >     that the packaging mechanism in Bro was ready for public
> > consumption. I
> > >     just have not had cycles to do anything with it yet. They are not
> > > wanting
> > >     to host any of the plugins.
> > >
> > >     I thought the package mechanism requires that a package live within
> > > its own
> > >     repo (which Casey confirmed). This put me in a bind on how to
> tackle
> > >     this. I don't want to personally host the plugin in my own Github
> > > repo. I
> > >     would prefer that we host it in a community repo; either Bro or
> > Metron.
> > >     Since Bro is moving away from hosting their own plugins, that
> leaves
> > >     Metron.
> > >
> > >     It would be great if we could create a separate repo for the
> plugin.
> > > That
> > >     solves the challenge of using the packaging mechanism.
> > >
> > >     We do need to reconcile what is in bro/bro-plugins and what is in
> > > Metron.
> > >     There are some enhancements that I and others have made that never
> > > made it
> > >     back into Metron. They never made it back, because the original
> plan
> > > was
> > >     just to switch to using the plugin from bro/bro-plugins before the
> > > idea of
> > >     a packaging mechanism hit Bro. Reconciling should be fairly easy to
> > > see by
> > >     just doing a diff.
> > >
> > >     It would be great if others want to take on any of that work. I
> would
> > > be
> > >     glad to offer any support that you need. Thanks, Jon!
> > >
> > >
> > >
> > >
> > >
> > >     On Thu, Mar 30, 2017 at 11:20 PM, Zeolla@GMail.com <
> zeolla@gmail.com
> > >
> > >     wrote:
> > >
> > >     > Ok, great.
> > >     >
> > >     > I agree, I definitely want to hear from Nick on the topic. My
> team
> > is
> > >     > currently looking into enhancing the plugin as well to
> potentially
> > > allow
> > >     > sending to multiple clusters, investigating some issues we see
> when
> > > our
> > >     bro
> > >     > cluster is under load, turn it into a package, etc.
> > >     >
> > >     > The work you just did was on our to do list as well so I'm very
> > > excited
> > >     to
> > >     > see it come through.
> > >     >
> > >     > Jon
> > >     >
> > >     > On Thu, Mar 30, 2017, 11:16 PM Casey Stella <cestella@gmail.com>
> > > wrote:
> > >     >
> > >     > I *think* it's possible. People do ask for mirrors of directories
> > > from
> > >     > time to time (see
> https://issues.apache.org/jira/browse/INFRA-7060
> > ).
> > > If
> > >     > we
> > >     > think this is a good idea, we can pose it to INFRA as a request.
> > I'd
> > > love
> > >     > to see us be able to use the bro packaging infrastructure and get
> > > more
> > >     > visibility for the plugin.
> > >     >
> > >     > I'd be particularly interested in Nick's opinion on this, though.
> > >     >
> > >     > On Thu, Mar 30, 2017 at 11:12 PM, Zeolla@GMail.com <
> > zeolla@gmail.com
> > > >
> > >     > wrote:
> > >     >
> > >     > > You can version packages -
> > >     > > http://bro-package-manager.readthedocs.io/en/stable/
> > >     > package.html#package-
> > >     > > versioning
> > >     > >
> > >     > > I agree that having a separate repo provided by Apache would
be
> > >     optimal,
> > >     > I
> > >     > > just don't know the process for that or if it was even
> reasonable
> > > to
> > >     > > suggest.
> > >     > >
> > >     > > Jon
> > >     > >
> > >     > > On Thu, Mar 30, 2017, 11:01 PM Casey Stella <
> cestella@gmail.com>
> > > wrote:
> > >     > >
> > >     > > > Looking at the bro packages, it appears that bro is expecting
> > > things
> > >     to
> > >     > > be
> > >     > > > its own git repository. I wonder if we could either request
> > INFRA
> > >     > > provide
> > >     > > > another repo for the bro-kafka plugin and integrate it into
> > > metron as
> > >     a
> > >     > > git
> > >     > > > submodule *or* if we could request INFRA to create a github
> > > mirror of
> > >     > the
> > >     > > > metron-sensors/bro-kafka-plugin directory. I'm not sure
how
> > > viable
> > >     > > either
> > >     > > > of those options are, frankly.
> > >     > > >
> > >     > > > One thing that I didn't see is how do you specify a
> particular
> > >     release
> > >     > of
> > >     > > > the plugin that you want to install? For us, we'd want to
> > > release the
> > >     > > > plugin along with the product. I didn't quite see how you'd
> > push
> > >     > > releases
> > >     > > > for bro plugins.
> > >     > > >
> > >     > > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella <
> > > cestella@gmail.com>
> > >     > > wrote:
> > >     > > >
> > >     > > > > So, I do agree with the concern. Is there a way to
host the
> > > package
> > >     > > > > within Metron? I definitely would like to see the
> > > modifications at
> > >     > > > > https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db
> > >     > > > > 065348da0a5043a8353b4a0a8 brought back into Metron
and I'd
> > > love for
> > >     > us
> > >     > > to
> > >     > > > > host the plugin.
> > >     > > > >
> > >     > > > > Thoughts?
> > >     > > > >
> > >     > > > >
> > >     > > > > On Thu, Mar 30, 2017 at 9:09 PM, Zeolla@GMail.com <
> > > zeolla@gmail.com>
> > >
> > >     > > > > wrote:
> > >     > > > >
> > >     > > > >> Today I was taking a look at METRON-812
> > >     > > > >> <https://issues.apache.org/jira/browse/METRON-812>,
which
> > > made me
> > >     > > > recall
> > >     > > > >> some conversations from a while back regarding
where the
> bro
> > > kafka
> > >     > > > plugin
> > >     > > > >> should ultimately live, and how to update it.
> > >     > > > >>
> > >     > > > >> Back in METRON-348 <https://issues.apache.org/
> > >     > jira/browse/METRON-348>
> > >     > > I
> > >     > > > >> brought up the fact that some important changes
> > >     > > > >> <https://github.com/bro/bro-plugins/commit/
> > b9f1f35415cb0db06
> > >     > > > >> 5348da0a5043a8353b4a0a8>
> > >     > > > >> were made to the externally hosted version of the
kafka
> > > plugin,
> > >     and
> > >     > > were
> > >     > > > >> never introduced to Metron's hosted version (i.e.
the one
> we
> > > use
> > >     > > > >> <https://github.com/apache/incubator-metron/blob/master/
> > metr
> > >     > > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml>
> > >     > > > >> in vagrant when bro is installed). The conversation
went
> > down
> > > the
> > >     > > route
> > >     > > > >> of
> > >     > > > >> discussing whether or not the bro kafka plugin
code should
> > >     continue
> > >     > to
> > >     > > > >> live
> > >     > > > >> in Metron in the first place. Now, with METRON-812,
I see
> us
> > >     > further
> > >     > > > >> muddying the waters of where to go for the right
plugin,
> as
> > > our
> > >     > > version
> > >     > > > is
> > >     > > > >> still missing the public changes but adds some
very
> > important
> > > new
> > >     > > > >> functionality.
> > >     > > > >>
> > >     > > > >> I'd like to bring up the idea of using bro's packages
> > >     > > > >> <https://github.com/bro/packages> framework,
released in
> > > late 2016
> > >     > > > >> <
> > > http://blog.bro.org/2016/10/introducing-bro-package-manager.html>
> > >     > > > >> (additional
> > >     > > > >> documentation here <
> > >     > > > http://bro-package-manager.readthedocs.io/en/stable/
> > >     > > > >> >),
> > >     > > > >> as a potential place for this to be hosted/referenced.
> This
> > > is a
> > >     > > simple
> > >     > > > >> and supported method (funded by Mozilla
> > >     > > > >> <https://blog.mozilla.org/blog/2015/12/10/mozilla-open-
> > sourc
> > >     > > > >> e-support-first-awards-made/>)
> > >     > > > >> to install and uninstall bro scripts, plugins,
etc., and
> it
> > > also
> > >     > > allows
> > >     > > > us
> > >     > > > >> to continue to have enough control over updates
to the
> > plugin
> > > so
> > >     > that
> > >     > > it
> > >     > > > >> will not slow down Metron development by having
it as a
> > > dependency
> > >     > > > >> (resolving both of Casey's concerns noted here
> > >     > > > >> <https://issues.apache.org/jira/browse/METRON-348?
> > focusedCom
> > >     > > > >> mentId=15391865&page=com.atlassian.jira.plugin.system.
> > >     > > > >> issuetabpanels:comment-tabpanel#comment-15391865>,
> > >     > > > >> and I think this solution is supported by Nick's
comments
> > here
> > >     > > > >> <https://issues.apache.org/jira/browse/METRON-348?
> > focusedCom
> > >     > > > >> mentId=15391872&page=com.atlassian.jira.plugin.system.
> > >     > > > >> issuetabpanels:comment-tabpanel#comment-15391872>
> > >     > > > >> as
> > >     > > > >> well).
> > >     > > > >>
> > >     > > > >> The only thing I'm not sure about is where to host
the
> > plugin
> > >     itself
> > >     > -
> > >     > > > my
> > >     > > > >> first thought would be Nick's github <
> > > https://github.com/nickwallen
> > >     > >,
> > >     > > > as
> > >     > > > >> he
> > >     > > > >> really kicked off this effort, but maybe we can
think of
> > > something
> > >     > > > better.
> > >     > > > >>
> > >     > > > >> Is this approach of interest to anybody? It is
extremely
> > > simple to
> > >     > > put
> > >     > > > >> together - I was able to throw one together
> > >     > > > >> <https://github.com/bro/packages/blob/master/
> > >     > jonzeolla/bro-pkg.index>
> > >     > > > and
> > >     > > > >> get it working with a fresh bro 2.5 install when
attending
> > the
> > >     > brocon
> > >     > > > talk
> > >     > > > >> <
> https://www.bro.org/brocon2016/brocon2016_abstracts.html#
> > >     > > > >> bro-packagemanager>
> > >     > > > >> (recording <https://www.youtube.com/watch?v=9RFfPJeGkcE>,
> > > slides
> > >     > > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>)
> that
> > >     > introduced
> > >     > > > >> this
> > >     > > > >> to me in the first place.
> > >     > > > >>
> > >     > > > >> Jon
> > >     > > > >> --
> > >     > > > >>
> > >     > > > >> Jon
> > >     > > > >>
> > >     > > > >
> > >     > > > >
> > >     > > >
> > >     > > --
> > >     > >
> > >     > > Jon
> > >     > >
> > >     >
> > >     > --
> > >     >
> > >     > Jon
> > >     >
> > >
> > >     --
> > >
> > >     Jon
> > >
> > >
> > >
> > > --
> > >
> > > Jon
> > >
> > > --
> > >
> > > Jon
> > >
> > --
> >
> > Jon
> >
>
-- 

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message