beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pei He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1045) Windows OS compatibilities
Date Wed, 23 Nov 2016 22:54:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691580#comment-15691580
] 

Pei He commented on BEAM-1045:
------------------------------

Other usages of * are:
https://github.com/apache/incubator-beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java#L68

https://github.com/apache/incubator-beam/blob/master/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/TFIDF.java#L146

https://github.com/apache/incubator-beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L628

> Windows OS compatibilities
> --------------------------
>
>                 Key: BEAM-1045
>                 URL: https://issues.apache.org/jira/browse/BEAM-1045
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-java-core, sdk-java-gcp
>            Reporter: Pei He
>
> One known issue is "*" is not allowed in Windows OS.
> For example, Paths.get("tempDir/*") might throw when code runs in Windows OS.
> http://stackoverflow.com/questions/27522581/asterisks-in-java-path
> This affecting IOChannelFactory.match(String glob).
> However, since match only requires support globs in the final component of a path. (local)
FileIOChannelFactory could do things similar as GcsIOChannelFactory:
> First, list all files under the directory path (this won't contain glob, such as *).
> Then, check each returned files whether it matches glob.
> In this way, glob (*) stays within Apache Beam's code.
> From match()'s javadoc:
> """
> Glob handling is dependent on the implementation.  Implementations should
>    * all support globs in the final component of a path (eg /foo/bar/*.txt),
>    * however they are not required to support globs in the directory paths.
> """



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message