metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammed Irshad <irshadkt....@gmail.com>
Subject Re: HBaseDao and IndexDao abstraction
Date Tue, 23 Oct 2018 10:40:29 GMT
Hi All,

I have got a solution for this using SHEW ( Simple HBase Enrichment Writer
) which is documented in confluence
<https://cwiki.apache.org/confluence/display/METRON/2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichment>
but not in metron current book documentation
<https://metron.apache.org/current-book/index.html>. I am going to give
this a try and see how it goes. Thanks a lot for Simon Elliston Ball
<simon@simonellistonball.com> & Metron slack channel :)

On Thu, Oct 18, 2018 at 10:51 AM Muhammed Irshad <irshadkt.mec@gmail.com>
wrote:

> Mike,
>
> Thanks for replying. I had gone through it already and we are indexing our
> Active Directory logs to hdfs by streaming from Splunk. But I have a
> requirement of maintaining Active Directory asset inventory ( Just list of
> asset and their status not historic data) along with AD event indexing. So
> I thought of using HBase and was thinking the best place to put this logic
> ( Enrichment by writing a custom stellar which populate HBase column family
> for assets or In indexing layer ) . Then I saw the HBaseDao in
> documentation and wanted to understand what it is and weather it can be
> used to meet my use case.
>
> On Tue, Oct 16, 2018 at 7:41 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
>> Hi Muhammed,
>>
>> I think you probably want to start with our parser infrastructure rather
>> than the DAO's for what you're doing. This series of blog posts gives a
>> use
>> case driven walkthrough that should help shed some light on things:
>> Part 1 (start here) -
>>
>> https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%3A+Creating+a+New+Telemetry
>> TOC of the 7-part series -
>>
>> https://cwiki.apache.org/confluence/display/METRON/2016/06/22/Metron+Tutorial+-+Fundamentals+Part+7%3A+Dashboarding+with+Kibana
>>
>> Here's some details about our parser infrastructure -
>>
>> https://github.com/apache/metron/tree/master/metron-platform/metron-parsers
>> ...which feeds into the data enrichment topology -
>>
>> https://github.com/apache/metron/tree/master/metron-platform/metron-enrichment
>> ...which feeds into the indexing topology, which you've already found
>>
>> Hope this helps for a start!
>>
>> Best,
>> Mike Miklavcic
>>
>>
>> On Tue, Oct 16, 2018 at 12:05 AM Muhammed Irshad <irshadkt.mec@gmail.com>
>> wrote:
>>
>> > Hi all,
>> >
>> > What is the actual use of HBaseDao documented in metron indexing
>> > documentation
>> > <
>> >
>> https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html
>> > >
>> > under section 'The IndexDao Abstraction' ? From my reading I understand
>> it
>> > as a HBase indexing implementation which can be clubbed to hdfs for
>> updated
>> > data. What is the use of it as we cannot chose to index in HBase / hdfs
>> > dynamically ? Can some one explain an example about how to configure and
>> > use it ( More documentation link or reference is fine) ? I have a use
>> case
>> > where I need to maintain an Active Directory inventory, Using AD event
>> logs
>> > being indexed via metron. Is HBaseDao can be used for this use case ?
>> >
>> > --
>> > Muhammed Irshad K T
>> > Senior Software Engineer
>> > +919447946359
>> > irshadkt.mec@gmail.com
>> > Skype : muhammed.irshad.k.t
>> >
>>
>
>
> --
> Muhammed Irshad K T
> Senior Software Engineer
> +919447946359
> irshadkt.mec@gmail.com
> Skype : muhammed.irshad.k.t
>


-- 
Muhammed Irshad K T
Senior Software Engineer
+919447946359
irshadkt.mec@gmail.com
Skype : muhammed.irshad.k.t

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message