metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [metron] merrimanr commented on issue #1389: METRON-2087: Remove Storm dependency from metron-indexing
Date Fri, 17 May 2019 20:11:34 GMT
merrimanr commented on issue #1389: METRON-2087: Remove Storm dependency from metron-indexing
URL: https://github.com/apache/metron/pull/1389#issuecomment-493584143
 
 
   After another round of testing I realized that we also need to separate `metron-elasticsearch`
and `metron-solr` into separate `common` and `storm` modules.  This is because the indexing
topology depends on uber jars built from the these modules.  The `storm` modules for each
now contain flux property files, start scripts, and indexing integration tests.  The benefit
of this is that the awkwardness of modules in `metron-indexing` that only contain a single
test goes away.  The `metron-indexing-storm-elasticsearch` module in `metron-indexing` is
now `metron-elasticsearch-storm` and likewise for `metron-indexing-storm-solr`.
   
   The changes were fairly straightforward except for the `pom.xml` files.  I ran into similar
class version issues that we commonly run into when refactoring.  The `metron-elasticsearch-storm`
module was particularly tricky.  I believe this is caused by our use of a dedicated `elasticsearch-shaded`
module that makes the dependency tree more difficult to follow.  The solution was to just
include this module dependency at the top of the `metron-elasticsearch-storm` pom.  
   
   Other changes included updating paths in various tests since the module structure changed
and modules containing these tests are now a level deeper.  Other than that the only changes
involved were just moving files.
   
   I ran through another round of tests (based on instructions in https://github.com/apache/metron/pull/1368)
and everything seems to be working well.  I also tested the various meta alert functions in
the Alerts UI.  I found a bug (https://issues.apache.org/jira/browse/METRON-2119) but I believe
it is preexisting and unrelated to this work.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message