atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bolke de Bruin (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-3755) Allow system attributes to be updated when policy allows
Date Sun, 03 May 2020 07:58:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098286#comment-17098286
] 

Bolke de Bruin commented on ATLAS-3755:
---------------------------------------

[~madhan] I have made a couple of updates:

1. I removed the defaults in the AtlasEntity to ensure that at json deserialisation no defaults
are set. This was a change very limited in scope as the deserialiser uses the default constructor
and the default constructor isn't used anywhere else. All other constructors use "init()"
that do set the default values.
2. The update of the system attributes is now behind a feature flag "atlas.store.system_attribute.enable".
This defaults to false and disables updating/creation of system attributes if not in "import"
mode.
3. Access requests are done in one go having virtually no impact on processing time
4. Unit tests have been updated to verify the feature flags
5. Removed the side effect in preCreateOrUpdate that was setting system attributes, all this
is now contained in createOrUpdate

*Use case*
We intend to receive about 0.5 million entity updates per day from other meta data systems.
We use Kafka to handle the backpressure hence the need to have it managed by policy. Also
Glossary (which relies on the entity system) needs to expose homeId and other system attributes
as well as Glossary items can be created other meta data systems. This needs to be enabled
in a follow PR.

> Allow system attributes to be updated when policy allows
> --------------------------------------------------------
>
>                 Key: ATLAS-3755
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3755
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: 2.0.0, 2.1.0
>            Reporter: Bolke de Bruin
>            Assignee: Bolke de Bruin
>            Priority: Critical
>         Attachments: 0001-ATLAS-3755-Allow-system-attributes-to-be-updated-by-.patch,
0001-ATLAS-3755-Allow-system-attributes-to-be-updated-by-.patch, 0001-ATLAS-3755-Allow-system-attributes-to-be-updated-by-.patch,
feature.patch, feature.patch
>
>
> Atlas does not operate in a isolated environment, this is one of the reasons the "homeId"
system attribute was introduced. Unfortunately system attributes can only be updated when
importing. This means any integration with other services is significantly limited (Kafka,
Rest API will not work). (See also ATLAS-3754)
> To resolve this I propose to make it possible to update the system attributes when policy
allows it. This introduces new AtlasPrivilege.ENTITY_UPDATE_SYSTEM_ATTRIBUTE and AtlasPrivilege.ENTITY_CREATE_SYSTEM_ATTRIBUTE
next to AtlasPrivilege.ENTITY_UPDATE_ATTRIBUTE and AtlasPrivilege.ENTITY_CREATE_ATTRIBUTE
rather than just checking on the entity level. In certain places we will then drop the requirement
for an import to be active as this can now happen through other channels as well.
> This allows operators to specify policies that allow granular controls over attributes
and system attributes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message