metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [DISCUSS] Dockerize Metron
Date Sat, 01 Oct 2016 17:28:47 GMT
It's definitely possible to debug in a Docker environment using a remote
debugger.  I remember reading something along these lines a few weeks ago.

https://blog.docker.com/2016/09/java-development-using-docker/

On Sat, Oct 1, 2016 at 1:15 PM, Casey Stella <cestella@gmail.com> wrote:

> I agree with that concern. Being able to debug a running topology has
> helped in so many circumstances. Not sure how to accomplish this in a
> dockerized environment.
>
> On Sat, Oct 1, 2016 at 13:11 Michael Miklavcic <
> michael.miklavcic@gmail.com>
> wrote:
>
> > Any ideas on how debugging works when leveraging Docker? In spite of the
> > classpath troubles, one of the benefits of the current single-JVM
> approach
> > is that you can easily debug, set break points, etc within an IDE. Is
> that
> > still doable? Seems like there might need to be some remote debugging
> magic
> > done to accomplish this.
> >
> > On Fri, Sep 30, 2016 at 6:39 PM, James Sirota <jsirota@apache.org>
> wrote:
> >
> > > I agree with making an effort to create containers.  I suggest doing it
> > on
> > > a feature branch until we are feature complete and are able to migrate
> > our
> > > integration tests into a dockerized environment.
> > >
> > > 30.09.2016, 14:52, "Nick Allen" <nick@nickallen.org>:
> > > >  Relieve dependency version conflict issues? Umm, yes please. I'll
> > take a
> > > >  second helping of that too.
> > > >
> > > >  On Fri, Sep 30, 2016 at 5:00 PM, Ryan Merriman <merrimanr@gmail.com
> >
> > > wrote:
> > > >
> > > >>   I would like to open up a discussion around creating Docker images
> > for
> > > >>   Metron. Having this available would provide a leaner alternative
> to
> > > the
> > > >>   ansible/vagrant environment for development tesing (and even
> demoing
> > > or
> > > >>   exploring features). It could also relieve some of the dependency
> > > version
> > > >>   conflict issues that we've been experiencing when running
> > integration
> > > tests
> > > >>   in a single JVM.
> > > >>
> > > >>   I would suggest the initial version be intended only for
> development
> > > and
> > > >>   testing purposes. The general approach could be to create an image
> > for
> > > >>   each service we depend on and use something like Docker compose
to
> > > package
> > > >>   them together. A Dockerfile would either install the service from
> > > scratch
> > > >>   or extend a community image then add any Metron related
> dependencies
> > > or
> > > >>   configurations on top. The metron-deployment project code could
be
> > > used as
> > > >>   a guide.
> > > >>
> > > >>   I would like to see these images added initially to support
> > > development and
> > > >>   testing:
> > > >>
> > > >>      - Kafka with topics preconfigured
> > > >>      - Storm with metron topology assets installed
> > > >>      - Zookeeper with paths created and sample configs loaded
> > > >>      - HBase with sample enrichments and threat intel loaded
> > > >>      - Elasticsearch configured for Metron
> > > >>      - MySQL with databases/tables/users created and geo data loaded
> > > >>
> > > >>   Other images that could also be useful:
> > > >>
> > > >>      - Images for each sensor
> > > >>      - Ambari?
> > > >>      - Solr
> > > >>
> > > >>   Looking forward to hearing what everyone thinks.
> > > >>
> > > >>   Ryan Merriman
> > > >
> > > >  --
> > > >  Nick Allen <nick@nickallen.org>
> > >
> > >
> > > -------------------
> > > Thank you,
> > >
> > > James Sirota
> > > PPMC- Apache Metron (Incubating)
> > > jsirota AT apache DOT org
> > >
> > > 30.09.2016, 14:52, "Nick Allen" <nick@nickallen.org>:
> > > > Relieve dependency version conflict issues? Umm, yes please. I'll
> take
> > a
> > > > second helping of that too.
> > > >
> > > > On Fri, Sep 30, 2016 at 5:00 PM, Ryan Merriman <merrimanr@gmail.com>
> > > wrote:
> > > >
> > > >>  I would like to open up a discussion around creating Docker images
> > for
> > > >>  Metron. Having this available would provide a leaner alternative
to
> > the
> > > >>  ansible/vagrant environment for development tesing (and even
> demoing
> > or
> > > >>  exploring features). It could also relieve some of the dependency
> > > version
> > > >>  conflict issues that we've been experiencing when running
> integration
> > > tests
> > > >>  in a single JVM.
> > > >>
> > > >>  I would suggest the initial version be intended only for
> development
> > > and
> > > >>  testing purposes. The general approach could be to create an image
> > for
> > > >>  each service we depend on and use something like Docker compose to
> > > package
> > > >>  them together. A Dockerfile would either install the service from
> > > scratch
> > > >>  or extend a community image then add any Metron related
> dependencies
> > or
> > > >>  configurations on top. The metron-deployment project code could be
> > > used as
> > > >>  a guide.
> > > >>
> > > >>  I would like to see these images added initially to support
> > > development and
> > > >>  testing:
> > > >>
> > > >>     - Kafka with topics preconfigured
> > > >>     - Storm with metron topology assets installed
> > > >>     - Zookeeper with paths created and sample configs loaded
> > > >>     - HBase with sample enrichments and threat intel loaded
> > > >>     - Elasticsearch configured for Metron
> > > >>     - MySQL with databases/tables/users created and geo data loaded
> > > >>
> > > >>  Other images that could also be useful:
> > > >>
> > > >>     - Images for each sensor
> > > >>     - Ambari?
> > > >>     - Solr
> > > >>
> > > >>  Looking forward to hearing what everyone thinks.
> > > >>
> > > >>  Ryan Merriman
> > > >
> > > > --
> > > > Nick Allen <nick@nickallen.org>
> > >
> > > -------------------
> > > Thank you,
> > >
> > > James Sirota
> > > PPMC- Apache Metron (Incubating)
> > > jsirota AT apache DOT org
> > >
> >
>



-- 
Nick Allen <nick@nickallen.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message