metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mmiklavc <...@git.apache.org>
Subject [GitHub] incubator-metron pull request #218: Metron 383 Create Ambari Service Definit...
Date Fri, 19 Aug 2016 21:46:43 GMT
GitHub user mmiklavc opened a pull request:

    https://github.com/apache/incubator-metron/pull/218

    Metron 383 Create Ambari Service Definition for Metron Parsers

    More detailed instructions for testing are coming soon, but I wanted to get community
eyes on this asap.
    
    This PR addresses [https://issues.apache.org/jira/browse/METRON-383](https://issues.apache.org/jira/browse/METRON-383)
    
    To test this PR:
    - You need to build the rpm's and make them available in a "/localrepo" directory on the
node you plan to use as Master for the parsers.
    - Copy or mount the services and stacks content into the correct ambari-server resources
directories. For example
      - incubator-metron/metron-deployment/packaging/ambari/src/main/resources/stacks/HDP/2.3/services/PARSERS
-> /var/lib/ambari-server/resources/stacks/HDP/2.3/services/PARSERS
      - incubator-metron/metron-deployment/packaging/ambari/src/main/resources/common-services/PARSERS
-> /var/lib/ambari-server/resources/common-services/PARSERS
    - Use Ambari 2.4
    - Install on a node with a Kafka Broker and ZK client (service advisor recommendations
to come in a separate PR)
    - You can check both a full install as well as an install of HDFS, Kakfa, Zookeeper, and
Storm followed after by Metron Parsers.
    
    Cleanup/delete service
    - delete the service via Ambari UI or REST API
    Run the following commands from the master node, swapping out the zookeeper quorum as
appropriate:
    
    ```
    #!/bin/bash
    
    # kafka
    for topic in bro parser_error parser_invalid snort yaf
    do
        /usr/hdp/current/kafka-broker/bin/kafka-topics.sh \
                --zookeeper amb1.service.consul:2181,amb2.service.consul:2181,amb3.service.consul:2181
\
                --delete \
                --topic $topic
    done
    
    # storm kill
    for i in bro yaf snort
    do
        storm kill $i
    done
    
    # yum delete
    yum -y erase metron-parsers metron-common
    
    # final file delete
    rm -rf /usr/metron
    
    # zookeeper
    /usr/hdp/current/zookeeper-client/bin/zkCli.sh -server 127.0.0.1:2181
    
    # once connected to the client do this:
    rmr /metron
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mmiklavc/incubator-metron METRON-383

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/218.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #218
    
----
commit d4b8ea8c148f1753932f799e3c553847cdb3b05c
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-05T13:23:12Z

    sample

commit 0193f467f6b79a81479d30798525d981757d3146
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-09T22:14:10Z

    add parser and common-services

commit 67096076717087126bdcf1e07d5e28d5b55a52e7
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-10T22:12:04Z

    finish commands. refactor

commit 09dc18b9266702e90f4a7f02ea33edac31be0317
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-12T16:00:13Z

    fixes

commit b447901c078c2b943608e2b5304fb87fb2aaabad
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-12T16:18:18Z

    config fixes

commit 724070de4696059b98a358d585ff724b2f2226f9
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-16T14:10:27Z

    incremental progress. quicklinks. kafka home

commit edd7b60c0565ef725858aa32e299c1b20d1f481b
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-16T21:01:46Z

    incremental fixes

commit 955680b20c6684c30631257a1068791a52e92a0a
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-17T21:09:44Z

    check service configured on start. fix zk load. fix topology start

commit 310550291051f7d649f8f4a51a80ab722758837c
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-18T20:21:33Z

    service check

commit 2a308376fb57ceaf2c875c0f0fb754c372d74f6f
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-18T21:40:12Z

    trying to fix service check

commit e9abe68bba0b1b95302c99a29f5fb872dd60f1c0
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-19T15:25:07Z

    cleanup/refactor. fix service check ordering

commit 8c1a093807c2ee18a6c9f315510bce2c2d0baf69
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-19T16:07:02Z

    fix service check

commit ffae457b34e94f5fdbf22a638c62bf69d032057a
Author: Michael Miklavcic <michael.miklavcic@gmail.com>
Date:   2016-08-19T21:34:14Z

    METRON-383 Ambari install for Metron Parser topologies.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message