beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Łukasz Gajowy (JIRA) <j...@apache.org>
Subject [jira] [Updated] (BEAM-3267) Return file names from TFRecordIO write
Date Wed, 29 Nov 2017 02:28:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Łukasz Gajowy updated BEAM-3267:
--------------------------------
    Description: In TFRecordIO we cannot actually retrurn filename set while doing the write.
It prevents us from creating pipelines that write and then read in the same pipeline. It was
acomplished in TextIO - there's even an example use case in [TextIOIT](https://github.com/apache/beam/blob/f8d8ff14c49e4dfb15541f4b73aa66513c9a9d23/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java#L114).
Maybe we can do the same in TFRecordIO?  (was: I mean an analogous version of the TextIO's
method of the same name. Currently there's no way to perform write and then read in one pipeline.
A method like this would allow to create such pipelines easily. )

> Return file names from TFRecordIO write
> ---------------------------------------
>
>                 Key: BEAM-3267
>                 URL: https://issues.apache.org/jira/browse/BEAM-3267
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Łukasz Gajowy
>            Assignee: Kenneth Knowles
>            Priority: Minor
>
> In TFRecordIO we cannot actually retrurn filename set while doing the write. It prevents
us from creating pipelines that write and then read in the same pipeline. It was acomplished
in TextIO - there's even an example use case in [TextIOIT](https://github.com/apache/beam/blob/f8d8ff14c49e4dfb15541f4b73aa66513c9a9d23/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java#L114).
Maybe we can do the same in TFRecordIO?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message