metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Sumbul <>
Subject Re: [DISCUSS] Pcap panel architecture
Date Fri, 04 May 2018 12:19:44 GMT

(Youhouuu my first reply on this kind of mail chain^^)

If I may, I would like to share my view on the following 3 points.

- Backend:

The current metron-api is totally seperate, it will be logic for me to have
it at the same place as the others rest api. Especially when more security
will be added, it will not be needed to do the job twice.
The current implementation send back a pcap object which still need to be
decoded. In the opensoc, the decoding was done with tshard on the frontend.
It will be good to have this decoding happening directly on the backend to
not create a load on frontend. An option will be to install tshark on the
rest server and to use to convert the pcap to xml and then to a json that
will be send to the frontend.

I tried to start directly the map/reduce job to search over all the pcap
data from the rest server and as Ryan mention it, we had trouble. I will
try to find back the error.

Then in the POC, what we tried is to use the pcap_query script and this
work fine. I just modified it that he sends back directly the job_id of
yarn and not waiting that the job is finished. Then it will allow the UI
and the rest server to know what the status of the research by querying the
yarn rest api. This will allow the UI and the rest server to be async
without any blocking phase. What do you think about that?

Having the job submitted directly from the code of the rest server will be
perfect, but it will need a lot of investigation I think (but I'm not the
expert so I might be completely wrong ^^).

We know that the pcap_query scritp work fine so why not calling it? Is it
that bad? (maybe stupid question, but I really don’t see a lot of drawback)

- Front end:

Adding the the pcap search to the alert UI is, I think, the easiest way to
move forward. But indeed, it will then be the “Alert UI and pcapquery”.
Maybe the name of the UI should just change to something like “Monitoring &
Investigation UI” ?

Is there any roadmap or plan for the different UI? I mean did you already
had discussion on how you see the ui evolving with the new feature that
will come in the future?

- Microservices:

What do you mean exactly by microservices? Is it to separate all the
features in different projects? Or something like having the different
components in container like kubernet? (again maybe stupid question, but I
don’t clearly understand what you mean J )


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message