beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chamikara Jayalath (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2531) Improve efficiency of reading compressed text files
Date Wed, 28 Jun 2017 16:34:00 GMT
Chamikara Jayalath created BEAM-2531:
----------------------------------------

             Summary: Improve efficiency of reading compressed text files
                 Key: BEAM-2531
                 URL: https://issues.apache.org/jira/browse/BEAM-2531
             Project: Beam
          Issue Type: Improvement
          Components: sdk-py
            Reporter: Chamikara Jayalath


Reading compressed text files using textio.ReadFromText() is about five times slower compared
to the corresponding transform in Java SDK. This is specially noticeable when using DirectRunner.
We should look into how we can improve efficiency of this so that Python SDK is more comparable
to Java SDK in this regard.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message