spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-4873) WriteAheadLogBasedBlockHandler improvement
Date Wed, 17 Dec 2014 09:55:13 GMT
Shixiong Zhu created SPARK-4873:
-----------------------------------

             Summary: WriteAheadLogBasedBlockHandler improvement
                 Key: SPARK-4873
                 URL: https://issues.apache.org/jira/browse/SPARK-4873
             Project: Spark
          Issue Type: Improvement
          Components: Streaming
            Reporter: Shixiong Zhu
            Priority: Minor


1. Move the block store work to a separate thread pool. IO work usually runs longer than CPU
work. Running IO work in `Implicits.global` may block other work that want to run in `Implicits.global`.
2. Use `Future.zip` instead of `Future.flatMap`(for-loop). `zip` implies these two  Futures
will run concurrently, while `flatMap` usually means one Future depends on the other one.
3. Fix a unit test in `ReceivedBlockTrackerSuite` which may fail occasionally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message