spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Rosen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-4741) Do not destroy and re-create FileInputStream
Date Tue, 09 Dec 2014 20:26:12 GMT

     [ https://issues.apache.org/jira/browse/SPARK-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Rosen updated SPARK-4741:
------------------------------
    Component/s: Spark Core
                 Shuffle

> Do not destroy and re-create FileInputStream
> --------------------------------------------
>
>                 Key: SPARK-4741
>                 URL: https://issues.apache.org/jira/browse/SPARK-4741
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle, Spark Core
>            Reporter: Liang-Chi Hsieh
>            Priority: Minor
>
> The FileInputStream in DiskMapIterator is destroyed and recreate after each batch reading.
However, since we can change the reading position on that stream, it is no need and inefficient
to destroy and recreate it every time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message