metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1869) Unable to Sort an Escalated Meta Alert
Date Mon, 03 Dec 2018 15:08:01 GMT

    [ https://issues.apache.org/jira/browse/METRON-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707332#comment-16707332
] 

ASF GitHub Bot commented on METRON-1869:
----------------------------------------

GitHub user nickwallen reopened a pull request:

    https://github.com/apache/metron/pull/1280

    METRON-1869 Unable to Sort an Escalated Meta Alert

    This fixes a bug that causes meta-alerts to not be visible in the Alerts UI when the UI
is sorted by 'alert_status' and a meta-alert has been escalated.  This is only a problem when
indexing into Elasticsearch.
    
    The root cause is that the 'alert_status' field needs to be defined as a keyword in the
metaalert index template. This field only exists in the meta-alert index after a meta-alert
has changed status, like when an alert is escalated.
    
    ## Changes
    
    * Fixed the Elasticsearch meta-alerts template that is deployed with the MPack.
    * Added an integration test for this bug.
    
    ## Testing
    
    Follow the "steps to reproduce" outlined in the JIRA to ensure that the bug has been squashed.
    
    1. Create a meta-alert.
    2. Escalate the meta-alert.
    3. Submit another search that filters the results to only show meta-alerts.
    4. Sort the results by “alert_status” field.
    5. The meta-alert should still be visible.
    
    ## Pull Request Checklist
    - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at
[Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
    - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are
trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target branch (typically
master)?
    - [x] Have you included steps to reproduce the behavior or problem that is being changed
or addressed?
    - [x] Have you included steps or a guide to how the change may be verified and tested
manually?
    - [x] Have you ensured that the full suite of tests and checks have been executed in the
root metron folder via:
    - [x] Have you written or updated unit tests and or integration tests to verify your changes?
    - [x] If adding new dependencies to the code, are these dependencies licensed in a way
that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
    - [x] Have you verified the basic functionality of the build by building and running locally
with Vagrant full-dev environment or the equivalent?
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nickwallen/metron METRON-1869

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/1280.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1280
    
----
commit d2f56b4f41bead7b3490cb81bdce553359ce09fc
Author: Nick Allen <nick@...>
Date:   2018-11-14T13:34:17Z

    Attempting to create test that highlights the problem

commit 70d1dc4bea3992f8e94eb855c36c99fde7f6cad9
Author: Nick Allen <nick@...>
Date:   2018-11-26T19:41:21Z

    METRON-1869 Fix missing alert_status field in meta-alert template

commit 09558fb5fa5eb24cebfc974170fd14abf047e2e5
Author: Nick Allen <nick@...>
Date:   2018-11-26T20:51:43Z

    Completed integration test that highlights the problem

commit afb87ba0b4efbc74913698e123c5661a42ac7f45
Author: Nick Allen <nick@...>
Date:   2018-11-26T20:58:47Z

    Testing sorted search results under 3 scenarios

commit 60c3af90061ce82a89696c96b9f07b3e4fb35324
Author: Nick Allen <nick@...>
Date:   2018-11-26T20:59:49Z

    Reset ordering

commit 8c5e21d9c0d0051ed85aeac5318ce362f10eacdf
Author: Nick Allen <nick@...>
Date:   2018-11-27T17:52:51Z

    Merge remote-tracking branch 'apache/master' into METRON-1869

commit a592e12595510a94cef7127f278e53ddcf40d242
Author: Nick Allen <nick@...>
Date:   2018-11-27T20:04:34Z

    Hopefully this is as clear without the javadoc

commit c7855508cfb9ea4404320f9cd6709880296e107e
Author: Nick Allen <nick@...>
Date:   2018-11-29T01:13:15Z

    Merge remote-tracking branch 'apache/master' into METRON-1869

----


> Unable to Sort an Escalated Meta Alert
> --------------------------------------
>
>                 Key: METRON-1869
>                 URL: https://issues.apache.org/jira/browse/METRON-1869
>             Project: Metron
>          Issue Type: Bug
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>            Priority: Major
>         Attachments: Can Sort on Metaalert Before Escalation.png, Cannot View Metaalert
Sorted by Status After Escalation.png
>
>
> Steps to Replicate:
>  1. Create a meta-alert.
>  2. Escalate the meta-alert.
>  3. Submit another search that filters the results to only show meta-alerts.
>  4. Sort the results by “alert_status” field.
>  5. The search will not return any meta-alerts in the results.
>  6. The REST log will contain the exception shown above.
> {code:java}
> 18/11/12 22:25:50 ERROR dao.ElasticsearchRequestSubmitter: Shard search failure [1/1];
reason=ElasticsearchException: Elasticsearch exception [type=illegal_argument_exception, reason=Fielddata
is disabled on text fields by default. Set fielddata=true on [alert_status] in order to load
fielddata in memory by uninverting the inverted index. Note that this can however use significant
memory. Alternatively use a keyword field instead.], index=metaalert_index, shard=0, status=INTERNAL_SERVER_ERROR,
nodeId=uuG7-lZoSiaowJK_cUD4IQ
> ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Fielddata
is disabled on text fields by default. Set fielddata=true on [alert_status] in order to load
fielddata in memory by uninverting the inverted index. Note that this can however use significant
memory. Alternatively use a keyword field instead.]]
> 	at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:490)
> 	at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:406)
> 	at org.elasticsearch.action.search.ShardSearchFailure.fromXContent(ShardSearchFailure.java:205)
> 	at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:297)
> 	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:526)
> 	at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$2(RestHighLevelClient.java:382)
> 	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:413)
> 	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:382)
> 	at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:323)
> 	at org.apache.metron.elasticsearch.dao.ElasticsearchRequestSubmitter.submitSearch(ElasticsearchRequestSubmitter.java:62)
> 	at org.apache.metron.elasticsearch.dao.ElasticsearchSearchDao.search(ElasticsearchSearchDao.java:128)
> 	at org.apache.metron.elasticsearch.dao.ElasticsearchDao.search(ElasticsearchDao.java:191)
> 	at org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertSearchDao.search(ElasticsearchMetaAlertSearchDao.java:81)
> 	at org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.search(ElasticsearchMetaAlertDao.java:209)
> 	at org.apache.metron.rest.service.impl.SearchServiceImpl.search(SearchServiceImpl.java:92)
> 	at org.apache.metron.rest.controller.SearchController.search(SearchController.java:54){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message