metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: ML features for Metron
Date Thu, 09 Jun 2016 19:44:59 GMT
+1 on the google doc idea.

I think any solution should include a framework that allows the user to

   - Manage the training of their models
   - Manage the deployment of their models without stopping the topologies
   (i.e. hot loading of models)
   - Application of their models

I'd also very much like to see support for

   - both small data ML libraries (i.e. scikit-learn) and big-data ML
   libraries (i.e. MLLib)
   - The popular non-java language support (i.e. Python and R)


On Thu, Jun 9, 2016 at 3:33 PM, Debo Dutta (dedutta) <dedutta@cisco.com>
wrote:

> Haven't seen one. Hence I started a thread.
>
> Metron is a community project so please feel free to start a google doc.
>
> And then we can get feedback from the users.
>
> Thx
> Debo
>
> Sent from my iPhone
>
> > On Jun 9, 2016, at 12:28 PM, Yazan Boshmaf <boshmaf@ece.ubc.ca> wrote:
> >
> > Do we have a roadmap for ML support in Metron? If not, how someone reach
> > out to existing users of Metron and get more input so that we at least
> > collect functional requirements?
> >
> > From my side, I can share some of the nice-to-have features from a
> research
> > perspective (i.e., feature that would make Metron a better platform to
> > conduct cybersecurity research).
> >
> > All the best,
> > Yazan
> >
> >> On Mon, Jun 6, 2016 at 10:12 AM, Debojyoti Dutta <ddutta@gmail.com>
> wrote:
> >>
> >> Thx Egon. The idea of labeled data collection is awesome, else we have
> to
> >> resort to unsupervised alone. Maybe one of the things the website could
> do
> >> is to point to labeled data contributed by users of Metron.
> >>
> >>> On Mon, Jun 6, 2016 at 12:03 AM, Egon Kidmose <kidmose@gmail.com>
> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I'd be interested in joining that discussion.
> >>>
> >>> I'm a phd student applying ML in the security monitoring domain.
> >>> It is my expectation that I'll be able to contribute with some event
> >>> correlation and alert filtering methods.
> >>> (Corelation: Finding events that are relevant to each other. Filtering:
> >>> Suppressing false alerts from e.g. IDSs, or picking out the relevant
> >> ones)
> >>> You'll see a PR as soon as I have something that is somewhat ready.
> >>>
> >>> A particularly interesting issue (to me at least) is the possibilities
> of
> >>> using a real, running SOC as the the "label factory" for labelled data.
> >>> Getting real data with labels for supervised methods is one of the
> great
> >>> challenges, and I see quite some potential for Metron here.
> >>>
> >>>
> >>> Mvh. / BR
> >>> Egon Kidmose
> >>>
> >>>> On Sat, Jun 4, 2016 at 5:02 PM, Yazan Boshmaf <boshmaf@ece.ubc.ca>
> >>> wrote:
> >>>
> >>>> One use case of Apache Metron (or OpenSOC) is to analyze amplification
> >>> DDoS
> >>>> attacks <https://www.internetsociety.org/sites/default/files/01_5.pdf
> >>> .
> >>>>
> >>>> With honeypots as information sources (e.g., AmptPot
> >>>> <http://www.christian-rossow.de/publications/amppot-raid2015.pdf>),
> >> you
> >>>> have the typical UDP/IP features (IP addresses, timestamps, protocols,
> >>>> ports, payload, etc.), which get enriched with reverse IP data,
> >>>> geolocation, etc. Some of these attributes can be used as features to
> >>>> identify and characterize types of reflection attacks (e.g.,
> exploiting
> >>>> NTP, DNS resolvers, or even RIPv1). Also, it is important to
> >> distinguish
> >>>> attackers from scanners, using certain features like timestamp
> >>>> synchronization across honeypots, as scanner tend to go through IP
> >>> blocks,
> >>>> one by one, as compared to actual attacks.
> >>>>
> >>>> These are some of the attributes one might consider for this use case.
> >> It
> >>>> would be nice to have something that does online learning and
> >> analytics,
> >>> so
> >>>> clustering / classification is done in real-time. Maybe Apache Spark's
> >>>> MLlib?
> >>>>
> >>>> All the best,
> >>>> Yazan
> >>>>
> >>>>> On Sat, Jun 4, 2016 at 4:59 PM, Zeolla@GMail.com <zeolla@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> I'm in
> >>>>>
> >>>>>> On Sat, Jun 4, 2016, 09:53 Yazan Boshmaf <boshmaf@ece.ubc.ca>
> wrote:
> >>>>>>
> >>>>>> Me too.
> >>>>>>
> >>>>>>> On Sat, Jun 4, 2016 at 9:43 AM, Franck Vervial <vervial@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> hi,
> >>>>>>>
> >>>>>>> i am interested.
> >>>>>>>
> >>>>>>> regards
> >>>>>>> On Fri, 3 Jun 2016 at 3:43 PM, Debo Dutta (dedutta) <
> >>>> dedutta@cisco.com
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> Wondering if anyone is interested in starting a discussion
on
> >>> what
> >>>>> kind
> >>>>>>> of
> >>>>>>>> machine learning based features would be good for Metron
….
> >> Would
> >>>>> love
> >>>>>> to
> >>>>>>>> have the SOC users chime in on the dev list.
> >>>>>>>>
> >>>>>>>> The result of the discussion could lead to JIRA items.
> >>>>>>>>
> >>>>>>>> thx
> >>>>>>>> debo
> >>>>> --
> >>>>>
> >>>>> Jon
> >>
> >>
> >>
> >> --
> >> -Debo~
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message