flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "niraj rai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-992) Create CollectionDataSets by reading (client) local files.
Date Tue, 28 Jul 2015 18:39:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644819#comment-14644819
] 

niraj rai commented on FLINK-992:
---------------------------------

Hi Fabian,
Can you please provide more details about this feature? My understanding is, if we need to
read the data from local file system.
Are you suggesting, we should read the data from local file system and and pass it to collection
data sets? 
Thanks again.
Niraj

> Create CollectionDataSets by reading (client) local files.
> ----------------------------------------------------------
>
>                 Key: FLINK-992
>                 URL: https://issues.apache.org/jira/browse/FLINK-992
>             Project: Flink
>          Issue Type: New Feature
>          Components: Java API, Python API, Scala API
>            Reporter: Fabian Hueske
>            Assignee: niraj rai
>            Priority: Minor
>              Labels: starter
>
> {{CollectionDataSets}} are a nice way to feed data into programs.
> We could add support to read a client-local file at program construction time using a
FileInputFormat, put its data into a CollectionDataSet, and ship its data together with the
program.
> This would remove the need to upload small files into DFS which are used together with
some large input (stored in DFS).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message