beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Łukasz Gajowy (JIRA) <j...@apache.org>
Subject [jira] [Updated] (BEAM-3267) Return file names from TFRecordIO write
Date Wed, 29 Nov 2017 02:30:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Łukasz Gajowy updated BEAM-3267:
--------------------------------
    Description: 
In TFRecordIO we cannot actually retrurn filename set while doing the write. It prevents us
from creating pipelines that write and then read in the same pipeline. It was acomplished
in TextIO - there's even an example use case in TextIOIT. Maybe we can do the same in TFRecordIO?

See: https://github.com/apache/beam/blob/f8d8ff14c49e4dfb15541f4b73aa66513c9a9d23/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java#L114

  was:In TFRecordIO we cannot actually retrurn filename set while doing the write. It prevents
us from creating pipelines that write and then read in the same pipeline. It was acomplished
in TextIO - there's even an example use case in TextIOIT. Maybe we can do the same in TFRecordIO?


> Return file names from TFRecordIO write
> ---------------------------------------
>
>                 Key: BEAM-3267
>                 URL: https://issues.apache.org/jira/browse/BEAM-3267
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Łukasz Gajowy
>            Assignee: Kenneth Knowles
>            Priority: Minor
>
> In TFRecordIO we cannot actually retrurn filename set while doing the write. It prevents
us from creating pipelines that write and then read in the same pipeline. It was acomplished
in TextIO - there's even an example use case in TextIOIT. Maybe we can do the same in TFRecordIO?
> See: https://github.com/apache/beam/blob/f8d8ff14c49e4dfb15541f4b73aa66513c9a9d23/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java#L114



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message