atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-3799) Migration Import: EntityConsumer Adds GUIDs to Result that Will Not Get Cleared
Date Fri, 15 May 2020 19:05:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108573#comment-17108573
] 

ASF subversion and git services commented on ATLAS-3799:
--------------------------------------------------------

Commit 3de30f55d808df6245f60fe01fed4b6cf894a31d in atlas's branch refs/heads/master from Ashutosh
Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=3de30f5 ]

ATLAS-3799: EntityConumer only adds entity GUIDs that are added when they were produced.


> Migration Import: EntityConsumer Adds GUIDs to Result that Will Not Get Cleared
> -------------------------------------------------------------------------------
>
>                 Key: ATLAS-3799
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3799
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: trunk
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>             Fix For: trunk
>
>         Attachments: ATLAS-3799-EntityConumer-only-adds-entity-GUIDs-that.patch
>
>
> *Background*
> Zip file-based migration import introduced _EntityConsumer_. This uses _StatusReporter_
that is used to determine which which of the produced items are processed. This, in turn,
helps determine the right position to set in case of resuming the import.
> The _EntityConumer_ class is responsible for adding the right GUIDs to the result that
helps _StatusReporter_ determine the right GUIDs. This is done correctly except that it also
adds additional GUIDs that were updated or deleted. This causes _StatusReporter_._processedSet_
to be filled with GUIDs that will never be cleared. This will grow over time and can potentially
lead to memory pressure.
> *Solution*
> Add only the GUIDs that were processed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message