metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zeolla@GMail.com" <zeo...@gmail.com>
Subject Re: [DISCUSS] The bro kafka plugin
Date Wed, 05 Apr 2017 17:12:34 GMT
I'm working on this
<https://github.com/JonZeolla/incubator-metron/tree/METRON-348> in
preparation for the new repo and migration to a package.  It looks like in
bro-plugins COPYING
<https://github.com/bro/bro-plugins/blob/master/kafka/COPYING> attributes
to Nick, but in our version COPYING
<https://github.com/apache/incubator-metron/tree/master/metron-sensors/bro-plugin-kafka/COPYING>
points to the Apache License.  Same with MAINTAINER (this
<https://github.com/apache/incubator-metron/blob/master/metron-sensors/bro-plugin-kafka/MAINTAINER>
vs this <https://github.com/bro/bro-plugins/blob/master/kafka/MAINTAINER>).
I assume when we package this up and host it in Apache we need to give it
the Apache license, and point to Metron for MAINTAINER.  My questions are:

1.  Is there any legal/licensing concern here?  I am taking changes from
the bro-plugins version and pulling it into the Apache-hosted code.  IANAL
2.  Nick - are you OK with these changes?

Jon

On Mon, Apr 3, 2017 at 3:50 PM Zeolla@GMail.com <zeolla@gmail.com> wrote:

> Can someone on the PMC submit a ticket to INFRA?  It looks like
> <https://www.apache.org/dev/infra-contact> committers aren't supposed to.
>
> Jon
>
> On Fri, Mar 31, 2017 at 4:23 PM Zeolla@GMail.com <zeolla@gmail.com> wrote:
>
> I would be happy to try it again but I attempted to do that before with
> bro packages and it failed to be able to handle it.  I also tried using
> branches of a repo with bro but that similarly failed (and was a pretty bad
> idea to start with).
>
> Jon
>
> On Fri, Mar 31, 2017, 3:24 PM Matt Foley <mattf@apache.org> wrote:
>
> We should be able to request just one alternate repo from INFRA, and put a
> top hierarchical level in it that doesn’t include a maven pom.  As far as
> maven and clients are concerned, it
>
> just increases by 1 the path length to the root of the repo.
>
> On 3/31/17, 10:30 AM, "Zeolla@GMail.com" <zeolla@gmail.com> wrote:
>
>     Once we agree on a repo location to host this, I would be happy to put
>     together the package and update our environments to use bro-pkg to
> install
>     the plugin.  I have created METRON-813
>     <https://issues.apache.org/jira/browse/METRON-813> to track this and
>     changed METRON-348 <https://issues.apache.org/jira/browse/METRON-348>
> to be
>     a sub-task.
>
>     Otto - the bro packages model doesn't allow colocation with anything
> else.
>     That said, if we have two similar situations, and given the INFRA
> example
>     <https://issues.apache.org/jira/browse/INFRA-7060> Casey linked to
> before
>     was requesting 9 repos, perhaps we just request two repos.  Would
> someone
>     else mind putting that request in?
>
>     Jon
>
>     On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler <ottobackwards@gmail.com>
>     wrote:
>
>     Could we create a separate repo for more than on thing?  like put … um
>     let’s say
>     a maven plugin and the bro plugin?
>
>
>
>     On March 31, 2017 at 12:30:25, Nick Allen (nick@nickallen.org) wrote:
>
>     I agree with everything that I've read.
>
>     One of the guys from Bro had contacted me a while back, letting me know
>     that the packaging mechanism in Bro was ready for public consumption. I
>     just have not had cycles to do anything with it yet. They are not
> wanting
>     to host any of the plugins.
>
>     I thought the package mechanism requires that a package live within
> its own
>     repo (which Casey confirmed). This put me in a bind on how to tackle
>     this. I don't want to personally host the plugin in my own Github
> repo. I
>     would prefer that we host it in a community repo; either Bro or Metron.
>     Since Bro is moving away from hosting their own plugins, that leaves
>     Metron.
>
>     It would be great if we could create a separate repo for the plugin.
> That
>     solves the challenge of using the packaging mechanism.
>
>     We do need to reconcile what is in bro/bro-plugins and what is in
> Metron.
>     There are some enhancements that I and others have made that never
> made it
>     back into Metron. They never made it back, because the original plan
> was
>     just to switch to using the plugin from bro/bro-plugins before the
> idea of
>     a packaging mechanism hit Bro. Reconciling should be fairly easy to
> see by
>     just doing a diff.
>
>     It would be great if others want to take on any of that work. I would
> be
>     glad to offer any support that you need. Thanks, Jon!
>
>
>
>
>
>     On Thu, Mar 30, 2017 at 11:20 PM, Zeolla@GMail.com <zeolla@gmail.com>
>     wrote:
>
>     > Ok, great.
>     >
>     > I agree, I definitely want to hear from Nick on the topic. My team is
>     > currently looking into enhancing the plugin as well to potentially
> allow
>     > sending to multiple clusters, investigating some issues we see when
> our
>     bro
>     > cluster is under load, turn it into a package, etc.
>     >
>     > The work you just did was on our to do list as well so I'm very
> excited
>     to
>     > see it come through.
>     >
>     > Jon
>     >
>     > On Thu, Mar 30, 2017, 11:16 PM Casey Stella <cestella@gmail.com>
> wrote:
>     >
>     > I *think* it's possible. People do ask for mirrors of directories
> from
>     > time to time (see https://issues.apache.org/jira/browse/INFRA-7060).
> If
>     > we
>     > think this is a good idea, we can pose it to INFRA as a request. I'd
> love
>     > to see us be able to use the bro packaging infrastructure and get
> more
>     > visibility for the plugin.
>     >
>     > I'd be particularly interested in Nick's opinion on this, though.
>     >
>     > On Thu, Mar 30, 2017 at 11:12 PM, Zeolla@GMail.com <zeolla@gmail.com
> >
>     > wrote:
>     >
>     > > You can version packages -
>     > > http://bro-package-manager.readthedocs.io/en/stable/
>     > package.html#package-
>     > > versioning
>     > >
>     > > I agree that having a separate repo provided by Apache would be
>     optimal,
>     > I
>     > > just don't know the process for that or if it was even reasonable
> to
>     > > suggest.
>     > >
>     > > Jon
>     > >
>     > > On Thu, Mar 30, 2017, 11:01 PM Casey Stella <cestella@gmail.com>
> wrote:
>     > >
>     > > > Looking at the bro packages, it appears that bro is expecting
> things
>     to
>     > > be
>     > > > its own git repository. I wonder if we could either request INFRA
>     > > provide
>     > > > another repo for the bro-kafka plugin and integrate it into
> metron as
>     a
>     > > git
>     > > > submodule *or* if we could request INFRA to create a github
> mirror of
>     > the
>     > > > metron-sensors/bro-kafka-plugin directory. I'm not sure how
> viable
>     > > either
>     > > > of those options are, frankly.
>     > > >
>     > > > One thing that I didn't see is how do you specify a particular
>     release
>     > of
>     > > > the plugin that you want to install? For us, we'd want to
> release the
>     > > > plugin along with the product. I didn't quite see how you'd push
>     > > releases
>     > > > for bro plugins.
>     > > >
>     > > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella <
> cestella@gmail.com>
>     > > wrote:
>     > > >
>     > > > > So, I do agree with the concern. Is there a way to host the
> package
>     > > > > within Metron? I definitely would like to see the
> modifications at
>     > > > > https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db
>     > > > > 065348da0a5043a8353b4a0a8 brought back into Metron and I'd
> love for
>     > us
>     > > to
>     > > > > host the plugin.
>     > > > >
>     > > > > Thoughts?
>     > > > >
>     > > > >
>     > > > > On Thu, Mar 30, 2017 at 9:09 PM, Zeolla@GMail.com <
> zeolla@gmail.com>
>
>     > > > > wrote:
>     > > > >
>     > > > >> Today I was taking a look at METRON-812
>     > > > >> <https://issues.apache.org/jira/browse/METRON-812>,
which
> made me
>     > > > recall
>     > > > >> some conversations from a while back regarding where the
bro
> kafka
>     > > > plugin
>     > > > >> should ultimately live, and how to update it.
>     > > > >>
>     > > > >> Back in METRON-348 <https://issues.apache.org/
>     > jira/browse/METRON-348>
>     > > I
>     > > > >> brought up the fact that some important changes
>     > > > >> <https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db06
>     > > > >> 5348da0a5043a8353b4a0a8>
>     > > > >> were made to the externally hosted version of the kafka
> plugin,
>     and
>     > > were
>     > > > >> never introduced to Metron's hosted version (i.e. the one
we
> use
>     > > > >> <https://github.com/apache/incubator-metron/blob/master/metr
>     > > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml>
>     > > > >> in vagrant when bro is installed). The conversation went
down
> the
>     > > route
>     > > > >> of
>     > > > >> discussing whether or not the bro kafka plugin code should
>     continue
>     > to
>     > > > >> live
>     > > > >> in Metron in the first place. Now, with METRON-812, I see
us
>     > further
>     > > > >> muddying the waters of where to go for the right plugin,
as
> our
>     > > version
>     > > > is
>     > > > >> still missing the public changes but adds some very important
> new
>     > > > >> functionality.
>     > > > >>
>     > > > >> I'd like to bring up the idea of using bro's packages
>     > > > >> <https://github.com/bro/packages> framework, released
in
> late 2016
>     > > > >> <
> http://blog.bro.org/2016/10/introducing-bro-package-manager.html>
>     > > > >> (additional
>     > > > >> documentation here <
>     > > > http://bro-package-manager.readthedocs.io/en/stable/
>     > > > >> >),
>     > > > >> as a potential place for this to be hosted/referenced. This
> is a
>     > > simple
>     > > > >> and supported method (funded by Mozilla
>     > > > >> <https://blog.mozilla.org/blog/2015/12/10/mozilla-open-sourc
>     > > > >> e-support-first-awards-made/>)
>     > > > >> to install and uninstall bro scripts, plugins, etc., and
it
> also
>     > > allows
>     > > > us
>     > > > >> to continue to have enough control over updates to the plugin
> so
>     > that
>     > > it
>     > > > >> will not slow down Metron development by having it as a
> dependency
>     > > > >> (resolving both of Casey's concerns noted here
>     > > > >> <https://issues.apache.org/jira/browse/METRON-348?focusedCom
>     > > > >> mentId=15391865&page=com.atlassian.jira.plugin.system.
>     > > > >> issuetabpanels:comment-tabpanel#comment-15391865>,
>     > > > >> and I think this solution is supported by Nick's comments
here
>     > > > >> <https://issues.apache.org/jira/browse/METRON-348?focusedCom
>     > > > >> mentId=15391872&page=com.atlassian.jira.plugin.system.
>     > > > >> issuetabpanels:comment-tabpanel#comment-15391872>
>     > > > >> as
>     > > > >> well).
>     > > > >>
>     > > > >> The only thing I'm not sure about is where to host the plugin
>     itself
>     > -
>     > > > my
>     > > > >> first thought would be Nick's github <
> https://github.com/nickwallen
>     > >,
>     > > > as
>     > > > >> he
>     > > > >> really kicked off this effort, but maybe we can think of
> something
>     > > > better.
>     > > > >>
>     > > > >> Is this approach of interest to anybody? It is extremely
> simple to
>     > > put
>     > > > >> together - I was able to throw one together
>     > > > >> <https://github.com/bro/packages/blob/master/
>     > jonzeolla/bro-pkg.index>
>     > > > and
>     > > > >> get it working with a fresh bro 2.5 install when attending
the
>     > brocon
>     > > > talk
>     > > > >> <https://www.bro.org/brocon2016/brocon2016_abstracts.html#
>     > > > >> bro-packagemanager>
>     > > > >> (recording <https://www.youtube.com/watch?v=9RFfPJeGkcE>,
> slides
>     > > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>)
that
>     > introduced
>     > > > >> this
>     > > > >> to me in the first place.
>     > > > >>
>     > > > >> Jon
>     > > > >> --
>     > > > >>
>     > > > >> Jon
>     > > > >>
>     > > > >
>     > > > >
>     > > >
>     > > --
>     > >
>     > > Jon
>     > >
>     >
>     > --
>     >
>     > Jon
>     >
>
>     --
>
>     Jon
>
>
>
> --
>
> Jon
>
> --
>
> Jon
>
-- 

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message