falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-2182) Add support for Falcon user extensions
Date Wed, 16 Nov 2016 11:23:59 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670199#comment-15670199

Srikanth Sundarrajan commented on FALCON-2182:

I have reservations with the following :

1. The fact that entities generated can be tampered with
2. Entities can be submitted / deleted / updated independently

The benefit of an extension being a black box with a defined functionality is somehow lost
in this scheme. I do, however appreciate the concern in trying to make this any more complex
than it ought to be (by treating the whole thing as a single atomic unit). Please do consider
the following tweak in the approach

* Don't allow the entities to be tampered with before submission
* All entities will be created / updated / deleted through the extension, but not directly
(it should be possible to lock them down preventing independent update)
* While instances can be managed independently, entity submission, updates or deletes should
be disallowed directly.
* Create a simple overlay mechanism (similar to oozie.launcher.*) to override configuration
to tune (not alter) the functioning.

The primary motivation behind this is that Instance management at best can stall the pipeline,
but direct entity management can actually alter the functionality itself.

> Add support for Falcon user extensions
> --------------------------------------
>                 Key: FALCON-2182
>                 URL: https://issues.apache.org/jira/browse/FALCON-2182
>             Project: Falcon
>          Issue Type: New Feature
>          Components: extensions
>            Reporter: Pallavi Rao
>         Attachments: FalconUserExtensions.pdf
> Currently server-side trusted extensions are supported (see FALCON-634 for more details).
> Need to extend the Falcon Extensions to support extensions built by users. For example,
lets say a user wants to add an extension for "data-quality check". This shouldn't require
it to be compiled and tested with Falcon build. User should be able to add it on-the-fly (while
the Falcon Server is running) and other users should be able to discover it, and "instantiate"
this new extension.

This message was sent by Atlassian JIRA

View raw message