falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-2299) Adding custom engine in Apache Falcon
Date Wed, 02 Aug 2017 08:22:02 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110514#comment-16110514

Pallavi Rao commented on FALCON-2299:

[~Aniketmo], Falcon's design is quite well-structured in terms ease of adding a new execution
engine. However, it does not follow a plugin/extension model, where users can just plug a
new engine in. You will have to make changes to the core components of Falcon. If you have
a good use case for supporting a generic engine, you can branch off the code and work on the
same. You will get the community's input/help along the way.

Also, just wanted to point out that pig, hive and spark are also internally translated to
corresponding Oozie actions. So, when you say an execution engine, I'm not sure if you are
planning add another type of action supported by Oozie (going by your example) or an engine
that is parallel to Oozie. Either way, the design is extensible.

> Adding custom engine in Apache Falcon
> -------------------------------------
>                 Key: FALCON-2299
>                 URL: https://issues.apache.org/jira/browse/FALCON-2299
>             Project: Falcon
>          Issue Type: Improvement
>          Components: general
>    Affects Versions: 0.9
>            Reporter: Aniket Rajendra More
>              Labels: features
>             Fix For: 0.9
> Falcon currently supports 4 engines - oozie, pig, hive, spark. Is it possible to configure
Falcon to include another engine apart from the ones already provided ? 
> We might be able to contribute to the Falcon project source code in order to support
custom engine. But is there any other way using which we can plug custom engine to an existing
Falcon build instead of creating our own Falcon build which supports our engine ?
> Has falcon opened its api for letting developers integrate their own tools/engines ?

This message was sent by Atlassian JIRA

View raw message