spot-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christos Mathas <>
Subject Re: Re: Evaluating the ML for netflow, proxy, dns
Date Sat, 20 Jan 2018 16:27:21 GMT

I'm currently evaluating spot and I'm also using ROC-AUC, but I'm not 
experienced in these evaluation methods. If you could share some of the 
data you mention or set up a meeting as you say to explain how you're 
conducting the evaluation process it would be really useful.

Also, I would like to ask about the topics parameter:

How can I calculate an approximation of the number of topics based on 
the setup/traffic of my network?

Thanks in advance

On 06/05/2017 05:40 PM, Lujan Moreno, Gustavo wrote:
> Hi,
> I have plenty of information about how to compute spot-ml performance 
> in terms of ROC-AUC and we are getting good results with internal 
> data. In order to compute the AUC we are injecting artificial attacks 
> and track their rank on the final score. Once we know the ranks of the 
> attacks we can compute the AUC. On proxy we are getting > 0.90 on AUC 
> and on net flow >0.99. I understand improvements on DNS are being done 
> and results of AUC should be computed soon.
> The data is internal and I don’t think I can share it. The attacks are 
> being generated using breaking point. I will ask if I can share this.
> We can also set up a meeting to explain how I’m conducting the 
> evaluation process.
> Best,
> Gustavo
> From: Giacomo Bernardi < <>>
> Reply-To: " 
> <>" 
> < <>>
> Date: Saturday, June 3, 2017 at 1:28 PM
> To: " 
> <>" 
> < <>>
> Subject: Re: Evaluating the ML for netflow, proxy, dns
> +1
> On the same boat.
> I've been trying Spot on two different networks (a large office, and a 
> large event venue) but I haven't been able to extract useful anomalies 
> so far. I understand the algorithm, but to be honest all the results I 
> get from spot-ml->spot-oa seem a bit random and not "anomalous" at all.
> I played a bit with the LDA parameters but that hasn't helped.
> Thanks.
> ps: i'm only looking at the netflow feed, no dns/proxy.
> On 3 June 2017 at 17:48, D.Anil < 
> <>> wrote:
>     Hi,
>     I would like to know if any of us have test data (for netflow,
>     proxy, dns) that have certain events to be detected as
>     threat/anomalies such that we can run ML on top of the test data
>     to evaluate the ML algorithm.
>     I would like to understand the concept of threshold in the score
>     calculation of LDA algorithm and how can we fine tune the
>     threshold to give best results for the model.
>     Looking forward to hear the experiences from the community.
>     Thanks,
>     Anil.

View raw message