flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-454) Add ProgramInput/OutputFormats
Date Mon, 08 Sep 2014 15:19:29 GMT

     [ https://issues.apache.org/jira/browse/FLINK-454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fabian Hueske updated FLINK-454:
--------------------------------
    Priority: Minor  (was: Major)

> Add ProgramInput/OutputFormats
> ------------------------------
>
>                 Key: FLINK-454
>                 URL: https://issues.apache.org/jira/browse/FLINK-454
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: GitHub Import
>            Priority: Minor
>              Labels: github-import
>             Fix For: pre-apache
>
>
> It would be nice to be able to plug existing Stratosphere programs together. 
> This eases the use of program libraries, such as for machine learning or spatial data.
> Right now a library algorithm would be used as follows:
> 1. Run a program that preprocessed data, brings it into the correct format for the library
algorithm and writes it to a FS.
> 1. Run the algorithm, which reads its input from FS and write the result back.
> 1. Maybe have a postprocessing job, which reads again from FS.
> By providing ProgramInput/OutputFormats, these programs could be directly connected,
allowing for: 
> 1. pipelined processing
> 1. cross program optimization 
> 1. elimination of a driver program
> 1. combination of different programming abstraction in one job (Spargel, Stratosphere
Java, etc.)
> 1. ...
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/454
> Created by: [fhueske|https://github.com/fhueske]
> Labels: enhancement, user satisfaction, 
> Created at: Mon Feb 03 21:14:05 CET 2014
> State: open



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message