-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7883/
-----------------------------------------------------------
Review request for giraph.
Description
-------
Having a lot of maps in SendMessageCache still makes it slow, so here is another step towards
making it faster.
Here are the results on PageRankBenchmark:
10m vertices, 100 edges per vertex, 10 workers
1 thread: Total superstep time: 54s -> 35s
20m vertices, 100 edges per vertex, 12 workers
4 threads: Computation time: 26s -> 17s
Also tested on one of our real applications, speedup was a bit smaller, about 20-25%.
This addresses bug GIRAPH-404.
https://issues.apache.org/jira/browse/GIRAPH-404
Diffs
-----
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessageCollection.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStoreByPartition.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/requests/SendWorkerMessagesRequest.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/utils/PairList.java
PRE-CREATION
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/utils/PairListWritable.java
PRE-CREATION
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
1405925
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestTest.java
1405925
Diff: https://reviews.apache.org/r/7883/diff/
Testing
-------
mvn clean verify, pseudo-distributed tests
PageRankBenchmark (results above)
Tried it out with a real application
Thanks,
Maja Kabiljo
|