atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Mestry (Jira)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-3799) Migration Import: EntityConsumer Adds GUIDs to Result that Will Not Get Cleared
Date Thu, 14 May 2020 22:07:00 GMT
Ashutosh Mestry created ATLAS-3799:
--------------------------------------

             Summary: Migration Import: EntityConsumer Adds GUIDs to Result that Will Not
Get Cleared
                 Key: ATLAS-3799
                 URL: https://issues.apache.org/jira/browse/ATLAS-3799
             Project: Atlas
          Issue Type: Bug
          Components:  atlas-core
    Affects Versions: trunk
            Reporter: Ashutosh Mestry
            Assignee: Ashutosh Mestry
             Fix For: trunk


*Background*

Zip file-based migration import introduced _EntityConsumer_. This uses _StatusReporter_
that is used to determine which which of the produced items are processed. This, in turn,
helps determine the right position to set in case of resuming the import.

The _EntityConumer_ class is responsible for adding the right GUIDs to the result that helps _StatusReporter_
determine the right GUIDs. This is done correctly except that it also adds additional GUIDs
that were updated or deleted. This causes _StatusReporter_._processedSet_ to be filled with
GUIDs that will never be cleared. This will grow over time and can potentially lead to memory
pressure.

*Solution*

Add only the GUIDs that were processed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message