hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "BSPModel" by thomasjungblut
Date Fri, 24 Feb 2012 16:15:41 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "BSPModel" page has been changed by thomasjungblut:
http://wiki.apache.org/hama/BSPModel?action=diff&rev1=15&rev2=16

      peer.reopenInput()
    }
  }}}
+ 
+ You must not consume the whole input to reopen it.
  
  === Custom Inputformat ===
  
@@ -266, +268 @@

  
  Counters are in 0.4.0 not usable for flow controls, since they are not synced during sync
phase. Watch [[https://issues.apache.org/jira/browse/HAMA-515|HAMA-515]] for details.
  
- == Setup and Cleanup ==
+ = Setup and Cleanup =
  
+ 
+ Since 0.4.0 you can use Setup and Cleanup methods in your BSP code. 
+ They can be inherited from BSP class like this:
+ 
+ {{{
+  public class MyEstimator extends
+       BSP<NullWritable, NullWritable, Text, DoubleWritable, DoubleWritable> {
+ 
+     @Override
+     public void setup(
+         BSPPeer<NullWritable, NullWritable, Text, DoubleWritable, DoubleWritable>
peer)
+         throws IOException {
+       //Setup: Choose one as a master
+       this.masterTask = peer.getPeerName(peer.getNumPeers() / 2);
+     }
+ 
+     @Override
+     public void cleanup(
+         BSPPeer<NullWritable, NullWritable, Text, DoubleWritable, DoubleWritable>
peer)
+         throws IOException {
+       // your cleanup here
+     }
+ 
+     @Override
+     public void bsp(
+         BSPPeer<NullWritable, NullWritable, Text, DoubleWritable, DoubleWritable>
peer)
+         throws IOException, SyncException, InterruptedException {
+       // your computation here
+     }
+   } 
+ }}}
+ 
+ Setup is called before bsp method, and cleanup is executed at the end after bsp.
+ You can do everything in setup and cleanup: sync, send, increment counters, write output
or even read from the input.
+ 
- == Combiners ==
+ = Combiners =
  Combiners are used for performing message aggregation to reduce communication overhead in
cases when messages can be summarized arithmetically e.g., min, max, sum, and average at the
sender side. Suppose that you want to send the integer messages to a specific processor from
0 to 1000 and sum all received the integer messages from all processors.
  
  {{{

Mime
View raw message