metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-1879) Allow Elasticsearch to Auto-Generate the Document ID
Date Wed, 05 Dec 2018 15:54:00 GMT

    [ https://issues.apache.org/jira/browse/METRON-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710259#comment-16710259
] 

ASF GitHub Bot commented on METRON-1879:
----------------------------------------

Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1269
  
    Prior to this change we use the Metron-generated GUID as the documentID.  They were always
the same.  There were various places in the code where the name `guid`, `docID`, or `id` were
used that implicitly meant the Metron GUID.
    
    With this change, the Metron-generated guid will never be equal to the Elasticsearch-generated
doc ID. You can view and use either the guid or the doc ID in the UI.  Both are available
as separate fields in the index.
    
    This PR has pulled on those strings and tried to make clear what is a Metron GUID and
what is a document ID.  In the Alerts UI you can see and/or use either the GUID or the document
ID.  It is up to the user which one they care to see.  Although most users will not care what
the document ID is.
    
    From my PR description:
    
    > This change is backwards compatible. The Alerts UI should continue to work no matter
if some of the underlying indices were written with the Metron GUID as the document ID and
some are written with the auto-generated document ID.
    
    There is no option to continue to use the GUID as the documentID.  I can't think of a
use case worthy of the additional effort and testing needed to support that.
    
    It is backwards compatible in that the Alerts UI will work when searching over both "legacy"
indices where guid = docID and "new" indices where guid != docID.  All places in the code
where a docID is needed, that docID is actually first retrieved from Elasticsearch, rather
than making an assumption about what it is.
    
    
    



> Allow Elasticsearch to Auto-Generate the Document ID
> ----------------------------------------------------
>
>                 Key: METRON-1879
>                 URL: https://issues.apache.org/jira/browse/METRON-1879
>             Project: Metron
>          Issue Type: Sub-task
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>            Priority: Major
>
> The document ID should not be set so that Elasticsearch can auto-generate the document
ID.  This should significantly improve write performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message