flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Piotr Nowojski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-11953) Introduce Plugin/Loading system and integrate it with FileSystem
Date Tue, 09 Apr 2019 09:49:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-11953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813194#comment-16813194
] 

Piotr Nowojski commented on FLINK-11953:
----------------------------------------

Yes thanks for pointing it out [~uce] :) Actually Presto's plugins mechanism was what sparked
this idea here to solve dependency convergence and shading issues in Flink. We were also looking
at [Apache NiFi Nars|https://medium.com/hashmapinc/nifi-nar-files-explained-14113f7796fd].

Our proposed solution at its core work the same as both of those before mentioned, although
it's at the moment more simplified, since it's trying to address only the {{FileSystems}}
in Flink. But if everything works out and idea will catch on, we might decide to evolve it
more towards Presto style of centralised Plugins providing various services like metrics,
connectors, internal storage etc - not only {{FileSystems}}.

> Introduce Plugin/Loading system and integrate it with FileSystem
> ----------------------------------------------------------------
>
>                 Key: FLINK-11953
>                 URL: https://issues.apache.org/jira/browse/FLINK-11953
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / FileSystem
>    Affects Versions: 1.9.0
>            Reporter: Stefan Richter
>            Assignee: Stefan Richter
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message