metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From justinleet <>
Subject [GitHub] incubator-metron issue #437: METRON-692: Update for 0.3.0 -> 0....
Date Mon, 06 Feb 2017 13:38:22 GMT
Github user justinleet commented on the issue:
    @cestella Let me know if there's anythign to change or adjust that you see in here. I'm
not sure if you can get the raw version in GitHub like in Jira, so ping me if this isn't grabbable
in the current form and I'll get it to you differently.
    ### [Migrate Geo Enrichment outside of MySQL](
    #### Description
    As of 0.3.0, a MySQL database was used for storage and retrieval of GeoIP information
during enrichment.
    As of 0.3.1, the MySQL database is removed in favor of using MaxMind's binary GeoIP files
and stored on HDFS
    After initial setup, this change is transparent and existing enrichment definitions will
run as-is.
    #### Migration
    While new installs will not require any additional steps, in an existing install a script
must be run to retrieve and load the initial data.
    The shell script ``  will retrieve MaxMind GeoLite2 data and load
data into HDFS, and update the configuration to point to this data.
    In most cases the following usage will grab the data appropriately:
    $METRON_HOME/bin/ -z <zk_server>:<zk_port>
    Additional options, including changing the source file location (which can be a file://
location if the GeoIP data is already downloaded), are available with the
    `-h` flag and are also detailed in the metron-data-management file.
    One caveat is that this script will NOT update on disk config files.  It is recommended
to retrieve the configuration using
    $METRON_HOME/bin/ -z <zk_server>:<zk_port> -m DUMP
    The new config will be `geo.hdfs.file` in the global section of the configuration.  Append
this key-value into the global.json in the config directory. A PUSH is unnecessary

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message