spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <>
Subject [jira] [Created] (SPARK-4873) WriteAheadLogBasedBlockHandler improvement
Date Wed, 17 Dec 2014 09:55:13 GMT
Shixiong Zhu created SPARK-4873:

             Summary: WriteAheadLogBasedBlockHandler improvement
                 Key: SPARK-4873
             Project: Spark
          Issue Type: Improvement
          Components: Streaming
            Reporter: Shixiong Zhu
            Priority: Minor

1. Move the block store work to a separate thread pool. IO work usually runs longer than CPU
work. Running IO work in `` may block other work that want to run in ``.
2. Use `` instead of `Future.flatMap`(for-loop). `zip` implies these two  Futures
will run concurrently, while `flatMap` usually means one Future depends on the other one.
3. Fix a unit test in `ReceivedBlockTrackerSuite` which may fail occasionally.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message