metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nickwallen <>
Subject [GitHub] incubator-metron pull request #427: METRON-676 Create Zeppelin Notebook for ...
Date Thu, 26 Jan 2017 21:29:41 GMT
GitHub user nickwallen opened a pull request:

    METRON-676 Create Zeppelin Notebook for YAF Telemetry

    Created a Zeppelin Notebook that serves as a basic template for a situational awareness
dashboard for the YAF flow telemetry produced by Metron. 
    In practice, this notebook should be enhanced and customized to leverage enrichments specific
to your production environment.  The notebook provides a fair introduction into the mechanics
of using Zeppelin/Spark to work with the telemetry that is archived by Metron in HDFS.  
    The Zeppelin Notebook is deployed with Metron through the MPack and can be installed by
using the "Metron" > "Service Actions" > "Zeppelin Notebook Import" action in Ambari.
    [METRON-676]( contains a screen capture
of the dashboard when it is run with roughly 7 days of archived telemetry data.
    ### Dependency
    This change is dependent on #423 , which is why you will see those commits included here.
 Once #423 hits master, I will rebase on master.
    ### Testing
    I tested this change by following these steps.
    #### Build It
    * Build Metron
    cd incubator-metron
    mvn clean package -DskipTests
    * Start Docker on your build machine
    * Build Metron RPMs
    cd metron-deployment
    mvn clean package -Pbuild-rpms -DskipTests
    * Build Ambari MPack
    cd metron-deployment
    mvn clean package
    #### Setup Test VM
    * Launch Vagrant and install Ambari only
    cd metron-deployment/vagrant/quick-dev-platform
    vagrant --ansible-tags=ambari up
    * Copy artifacts to VM
    scp metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/*.rpm vagrant@node1:/tmp
    scp metron-deployment/packaging/ambari/metron-mpack/target/metron_mpack-
    * Stage RPMS
    vagrant ssh
    sudo su -
    mkdir /localrepo
    cp /tmp/metron*.rpm /localrepo
    #### Install Ambari MPack
    * Install MPack
    ambari-server install-mpack --mpack=/tmp/metron_mpack- --verbose
    * Restart Ambari
    service ambari-server restart
    #### Deploy Metron with MPack
    * You may need to clear the browser cache to see the additional options installed by the
    * Login to Ambari at http://node1:8080
    * Click "Actions" -> "Add Services", then choose "Metron"
    #### Install Zeppelin Notebooks
    * Login to Ambari at http://node1:8080.
    * In Ambari, click "Metron" > "Service Actions" > "Zeppelin Notebook Import"
    * Wait for the action to complete in Ambari.
    * Login to Zeppelin at http://node1:9995
    * Search for the notebook named "Metron - YAF Telemetry"  

You can merge this pull request into a Git repository by running:

    $ git pull METRON-676

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #427


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message