atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-3796) Migration Import Display: StatusReporter Does not Remove Processed Items
Date Fri, 15 May 2020 19:04:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108571#comment-17108571
] 

ASF subversion and git services commented on ATLAS-3796:
--------------------------------------------------------

Commit 935d0ab6465775e16db62896375d96f6a567b884 in atlas's branch refs/heads/master from Ashutosh
Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=935d0ab ]

ATLAS-3796: Status Reporter display fix.


> Migration Import Display: StatusReporter Does not Remove Processed Items
> ------------------------------------------------------------------------
>
>                 Key: ATLAS-3796
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3796
>             Project: Atlas
>          Issue Type: Bug
>          Components: atlas-intg
>    Affects Versions: trunk
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>             Fix For: trunk
>
>         Attachments: ATLAS-3796-Status-Reporter-processed-set-cleanup.patch
>
>
> *Background*
> _Migration Import_ implementation introduced _StatusReporter_. The class maintains GUIDs
of entities that are to be processed and entities that are processed. When the entity is marked
as processed, it needs to be removed from the processed set.
> *Steps to Duplicate*
> _Pre-requisites_
> Large migration import dataset (over 50 M entities).
> _Steps to Duplicate_
>  * Start importing the dataset.
>  * After over 60%, the Atlas server will crash with out of memory exception.
> _Expected results_: Migration Import should continue without problems.
> _Actual results_: Atlas server crashes with out of memory exception, hprof dump indicates
large number of items allocated by _StatusReporter.processedSet_.
> _Additional information_: This bug does not affect status reporting.
> _Work around_: Resume migration import. It will continue without problems.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message