beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rafal Wojdyla (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-1497) Refactor Hadoop/HDFS IO
Date Thu, 16 Feb 2017 06:11:41 GMT
Rafal Wojdyla created BEAM-1497:
-----------------------------------

             Summary: Refactor Hadoop/HDFS IO
                 Key: BEAM-1497
                 URL: https://issues.apache.org/jira/browse/BEAM-1497
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-extensions
    Affects Versions: 0.5.0
            Reporter: Rafal Wojdyla
            Assignee: Rafal Wojdyla


[Scio|https://github.com/spotify/scio/tree/master/scio-hdfs/src] includes refactored version
of Hadoop/HDFS IO, primarily:
 * single source/sink for all hadoop formats (no separate avro specific io)
 * no separate simple authentication - UGI in sink
 * use builder pattern instead of CTORs

This issue is to report push of this refactor to upstream/beam. Also tracked in [scio#416|https://github.com/spotify/scio/issues/416]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message