spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Tootoonchian <>
Subject Re: Improving system design logging in spark
Date Thu, 21 Apr 2016 20:51:57 GMT

My point for #2 is distinguishing between how long does it take for each
task to read a data from disk and transfer it through network to targeted
node. As I know (correct me if I'm wrong) block time to fetch data includes
both reading a data by remote node and transferring it to requested node. If
the block time is bigger than our expectation, from system design, we cannot
identify which component is weakest link, storage or network. 

View this message in context:
Sent from the Apache Spark Developers List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message