metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cestella <...@git.apache.org>
Subject [GitHub] incubator-metron pull request #415: METRON-652: Extract indexing config from...
Date Mon, 16 Jan 2017 17:58:02 GMT
GitHub user cestella reopened a pull request:

    https://github.com/apache/incubator-metron/pull/415

    METRON-652: Extract indexing config from enrichment config

    Currently, the indexing configuration is bound and coupled with the sensor enrichment
configuration.  This was done historically because indexing and enrichment were part of the
same topology.  When the topologies were separated, the configurations were never separated,
leaving a confusing section about indexes in the middle of the enrichment configuration. 
This effort will separate out the configuration.  
    
    Because of the configuration's simplicity, we are treating the config as just a `Map<String,
Object>` and supporting the two existing configurations:
    * `batchSize`
    * `index`
    
    Because this change is fairly deep, a few non-obvious things were also updated for consistency
and necessity:
    * The Stellar Management functions `ENRICHMENT_SET_BATCH` and `ENRICHMENT_SET_INDEX` functions
have been renamed to `INDEXING_SET_BATCH` and `INDEXING_SET_INDEX`
    * Support in `CONFIG_GET` and `CONFIG_PUT` for retrieving/storing indexing configurations
was added.  
    * Support for indexing configurations were added to the zookeeper utility function
    * The indexing RPM was updated to include the new configs for the shipped sensors.
    
    I have *not* updated the management pack to create a new panel for managing indexing configs.
 That will be part of a follow-on JIRA (METRON-653).
    
    I have done smoke testing on quickdev, but will include more comprehensive testing explanation
in the comments.  I feel that this has the possibility of causing regressions, so I want to
be explicit in testing methodology.
    
    Furthermore, I plan on creating a wiki section about about migrations, because this will
necessitate migrating configs when upgrading from 0.3.0 to the next version.
    
    EDIT: Please review, but do NOT commit this PR until METRON-653 is complete and reviewed.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cestella/incubator-metron METRON-652

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/415.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #415
    
----
commit 1d03bc57235af2c1b90c901e7895f3c037a054c2
Author: cstella <cestella@gmail.com>
Date:   2017-01-10T14:25:33Z

    initial update

commit 4c9227cb618a62bfea63b147c32054bfeadf7d08
Author: cstella <cestella@gmail.com>
Date:   2017-01-10T15:48:33Z

    Updating testing components.

commit 3681edb32dca2876ecb2c0895c58a392ce558ada
Author: cstella <cestella@gmail.com>
Date:   2017-01-10T20:32:47Z

    second round of changes.

commit facb6518b27fd19fd5c3f40671b57b84013c4076
Author: cstella <cestella@gmail.com>
Date:   2017-01-11T15:03:14Z

    Merge branch 'master' into METRON-652

commit ec91c279af078c94c4d762bd8ed166a18fa78da3
Author: cstella <cestella@gmail.com>
Date:   2017-01-11T15:03:40Z

    Merge branch 'master' into METRON-652

commit ed98341e904575f4d1e89533440cb920ee7a6e3d
Author: cstella <cestella@gmail.com>
Date:   2017-01-11T17:54:59Z

    updating.

commit 375e0260ce2d12e38e66e8ca05740bf6f71e348c
Author: cstella <cestella@gmail.com>
Date:   2017-01-11T21:10:16Z

    Updating documentation.

commit 3736294bfe9bb4f3f6c3862af300da49960da81f
Author: cstella <cestella@gmail.com>
Date:   2017-01-11T21:15:24Z

    Updating RPM.

commit 3834178117f8af23ebeff95d69f4eaba965b617c
Author: cstella <cestella@gmail.com>
Date:   2017-01-16T16:11:35Z

    Whoops, wrong README.md

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message