hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From edwardy...@apache.org
Subject svn commit: r1231787 - in /incubator/hama/trunk/src/site: resources/css/site.css site.xml xdoc/downloads.xml xdoc/getting_started_with_hama.xml xdoc/hama_bsp_tutorial.xml xdoc/hama_on_clouds.xml xdoc/index.xml xdoc/team-list.xml
Date Sun, 15 Jan 2012 23:02:07 GMT
Author: edwardyoon
Date: Sun Jan 15 23:02:06 2012
New Revision: 1231787

URL: http://svn.apache.org/viewvc?rev=1231787&view=rev
Log:
Update website

Added:
    incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml   (with props)
    incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml   (with props)
    incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml   (with props)
Modified:
    incubator/hama/trunk/src/site/resources/css/site.css
    incubator/hama/trunk/src/site/site.xml
    incubator/hama/trunk/src/site/xdoc/downloads.xml
    incubator/hama/trunk/src/site/xdoc/index.xml
    incubator/hama/trunk/src/site/xdoc/team-list.xml

Modified: incubator/hama/trunk/src/site/resources/css/site.css
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/resources/css/site.css?rev=1231787&r1=1231786&r2=1231787&view=diff
==============================================================================
--- incubator/hama/trunk/src/site/resources/css/site.css (original)
+++ incubator/hama/trunk/src/site/resources/css/site.css Sun Jan 15 23:02:06 2012
@@ -20,22 +20,28 @@ p {
 }
 
 body a:link {
-        color: #74240f;
+  color: #74240f;
 }
 body a:visited {
-        color: #74240f;
+  color: #74240f;
 }
 body a:hover {
-        text-decoration: underline;
-        color: #888800;
+  text-decoration: underline;
+  color: #888800;
 }
 
 h2, h3, h4 {
-	color: #74240f;
+  color: #74240f;
+}
+
+ul {
+ margin-left:10px; 
 }
 
 .green {
  color: #0f7355;
+ width: 95%;
+ margin: 20px 20px 20px 20px;
  border: 1px solid #0f7355;
  background: #edf7f4;
  background-color: #edf7f4;
@@ -43,6 +49,8 @@ h2, h3, h4 {
 .green a:link, .green a:active, .green a:visited { color: #0a4d39; }
 .green a:hover { color: #888800; }
 .green h3 {
+  margin-bottom:-5px;
+  margin-top:-2px;
   padding: 0px 0px 0px 0px;
   border: 0px;
   color: #74240f;
@@ -66,3 +74,9 @@ h2, h3, h4 {
   padding: 0;
   font-size: small;
 }
+
+ul {
+  margin-left: 20px;
+  padding-left: 1em;
+  list-style-position: inside;
+}

Modified: incubator/hama/trunk/src/site/site.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/site.xml?rev=1231787&r1=1231786&r2=1231787&view=diff
==============================================================================
--- incubator/hama/trunk/src/site/site.xml (original)
+++ incubator/hama/trunk/src/site/site.xml Sun Jan 15 23:02:06 2012
@@ -65,7 +65,10 @@
     </menu>
 
     <menu name="Documentation">
-      <item name="Quick Start" href="http://wiki.apache.org/hama/GettingStarted"/>
+      <item name="Getting Started" href="getting_started_with_hama.html" />
+      <item name="Hama on Clouds" href="hama_on_clouds.html" />
+      <item name="Hama BSP tutorial" href="hama_bsp_tutorial.html" />
+      <item name="Benchmarks" href="http://wiki.apache.org/hama/Benchmarks" />
       <item name="Presentations" href="http://wiki.apache.org/hama/Presentations" />
       <item name="Research Papers" href="http://wiki.apache.org/hama/Articles" />
       <item name="Books" href="http://wiki.apache.org/hama/Books" />

Modified: incubator/hama/trunk/src/site/xdoc/downloads.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/downloads.xml?rev=1231787&r1=1231786&r2=1231787&view=diff
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/downloads.xml (original)
+++ incubator/hama/trunk/src/site/xdoc/downloads.xml Sun Jan 15 23:02:06 2012
@@ -41,10 +41,10 @@ xsi:schemaLocation="http://maven.apache.
     which is located within each download directory.
     <br/><br/>
     Always use the signature files to verify the authenticity of the distribution, e.g.,
-    <pre>  % pgpk -a KEYS
+    <pre class="green">  % pgpk -a KEYS
   % pgpv hama-x.x.x.tar.gz.asc</pre>
     or,
-    <pre>  % gpg --import KEYS
+    <pre class="green">  % gpg --import KEYS
   % gpg --verify hama-x.x.x.tar.gz.asc</pre>
     We offer MD5 hashes as an alternative to validate the integrity of the downloaded files.
     A unix program called md5 or md5sum is included in many unix distributions.

Added: incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml?rev=1231787&view=auto
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml (added)
+++ incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml Sun Jan 15 23:02:06 2012
@@ -0,0 +1,125 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<document xmlns="http://maven.apache.org/XDOC/2.0"
+xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
+  <properties></properties>
+  <body>
+    <section name="Getting Started with Hama"></section>
+    <p>This document describes how to install, configure and manage Hama clusters ranging
from a few nodes to extremely large clusters with thousands of nodes.</p>
+
+    <subsection name="Requirements"></subsection>
+    <p>1. Make sure all required software is installed on all nodes in your cluster:</p>
+    <ul>
+      <li>hadoop-0.20.x (non-secure version)</li>
+      <li>Sun Java JDK 1.6.x or higher version</li>
+      <li>SSH access to manage BSP deamons</li>
+    </ul>
+    <p>2. <a href="downloads.html">Download</a> Hama from the release page.
+    <br /><br />
+    For additional information consult our 
+    <a href="http://wiki.apache.org/hama/CompatibilityTable">Compatibility Table</a></p>
+    
+    <subsection name="Mode"></subsection>
+    <p>Just like Hadoop, we distinct between three modes:</p>
+    <ul>
+      <li>Local Mode - <font style="font-size: 11px;">This mode is the default
mode if you download Hama (>= 0.3.0) and install it. When submitting a job it will run
a local multithreaded BSP Engine on your server. It can be configured via the bsp.master.address
property to local. You can adjust the number of threads used in this utility by setting the
<b>bsp.local.tasks.maximum property</b>. See the Settings step how and where to
configure this.</font></li>
+      <li>Pseudo Distributed Mode - <font style="font-size: 11px;">This mode
is when you just have a single server and want to launch all the deamon processes (BSPMaster,
Groom and Zookeeper). It can be configured when you set the bsp.master.address to a host address
e.g., <b>localhost</b> and put the same address into the groomservers file in
the configuration directory. As stated it will run a BSPMaster, a Groom and a Zookeeper on
your machine.</font></li>
+      <li>Distributed Mode - <font style="font-size: 11px;">This mode is just
like the "Pseudo Distributed Mode", but you have multiple machines, which are mapped in the
groomservers file.</font></li>
+    </ul>
+    
+    <subsection name="Configuration Files"></subsection>
+    <p>The $HAMA_HOME/conf directory contains some configuration files for Hama. These
are:</p>
+    <ul>
+      <li>hama-env.sh - This file contains some environment variable settings used
by Hama. You can use these to affect some aspects of Hama daemon behavior, such as where log
files are stored, the maximum amount of heap used etc. The only variable you should need to
change in this file is JAVA_HOME, which specifies the path to the Java 1.5.x installation
used by Hama.</li>
+      <li>groomservers - This file lists the hosts, one per line, where the GroomServer
daemons will run. By default this contains the single entry localhost</li>
+      <li>hama-default.xml - This file contains generic default settings for Hama daemons.
Do not modify this file.</li>
+      <li>hama-site.xml - This file contains site specific settings for all Hama daemons
and BSP jobs. This file is empty by default. Settings in this file override those in hama-default.xml.
This file should contain settings that must be respected by all servers and clients in a Hama
installation.</li>
+    </ul>
+    
+    <subsection name="Startup Scripts"></subsection>
+    <p>The $HAMA_HOME/bin directory contains some script used to start up the Hama
daemons.</p>
+    <ul><li>start-bspd.sh - Starts all Hama daemons, the BSPMaster, GroomServers
and Zookeeper.</li></ul>
+    <p>Note: You have to start Hama with the same user which is configured for Hadoop.</p>
+    
+    <subsection name="Setting up Hama"></subsection>
+    <ul><li>BSPMaster and Zookeeper settings - Figure out where to run your HDFS
namenode and BSPMaster. Set the variable bsp.master.address to the BSPMaster's intended host:port.
Set the variable fs.default.name to the HDFS Namenode's intended host:port.</li></ul>
+    <p>Here's an example of a hama-site.xml file:</p>
+
+<pre class="green">
+  &lt;?xml version="1.0"?&gt;
+  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
+  &lt;configuration&gt;
+    &lt;property&gt;
+      &lt;name&gt;bsp.master.address&lt;/name&gt;
+      &lt;value&gt;host1.mydomain.com:40000&lt;/value&gt;
+      &lt;description&gt;The address of the bsp master server. Either the
+      literal string "local" or a host:port for distributed mode
+      &lt;/description&gt;
+    &lt;/property&gt;
+
+    &lt;property&gt;
+      &lt;name&gt;fs.default.name&lt;/name&gt;
+      &lt;value&gt;hdfs://host1.mydomain.com:9000/&lt;/value&gt;
+      &lt;description&gt;
+        The name of the default file system. Either the literal string
+        "local" or a host:port for HDFS.
+      &lt;/description&gt;
+    &lt;/property&gt;
+
+    &lt;property&gt;
+      &lt;name&gt;hama.zookeeper.quorum&lt;/name&gt;
+      &lt;value&gt;host1.mydomain.com,host2.mydomain.com&lt;/value&gt;
+      &lt;description&gt;Comma separated list of servers in the ZooKeeper Quorum.
+      For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
+      By default this is set to localhost for local and pseudo-distributed modes
+      of operation. For a fully-distributed setup, this should be set to a full
+      list of ZooKeeper quorum servers. If HAMA_MANAGES_ZK is set in hama-env.sh
+      this is the list of servers which we will start/stop zookeeper on.
+      &lt;/description&gt;
+    &lt;/property&gt;
+  &lt;/configuration&gt;
+</pre>
+
+<p>If you are managing your own ZooKeeper, you have to specify the port number as below:</p>
+
+<pre class="green">
+  &lt;property&gt;
+    &lt;name&gt;hama.zookeeper.property.clientPort&lt;/name&gt;
+    &lt;value&gt;2181&lt;/value&gt;
+  &lt;/property&gt;
+</pre>
+<p>See all <a href="http://wiki.apache.org/hama/GettingStarted/Properties">Configuration
Properties</a></p>
+    
+    <subsection name="Starting a Hama cluster"></subsection>
+    <p>NOTE: Skip this step if you're in Local Mode.
+    <br/>Run the command:</p>
+    
+  <pre class="green">
+  % $HAMA_HOME/bin/start-bspd.sh</pre>
+    <p>This will startup a BSPMaster, GroomServers and Zookeeper on your machine.</p>
+    <p>Run the command:</p>
+  <pre class="green">
+  % $HAMA_HOME/bin/stop-bspd.sh</pre>
+
+    <p>to stop all the daemons running on your cluster.</p>
+    <subsection name="Execute Hama Examples"></subsection>
+  <pre class="green">
+  % $HAMA_HOME/bin/hama jar hama-examples-0.x.0-incubating.jar [args]</pre>
+    </body>
+</document>

Propchange: incubator/hama/trunk/src/site/xdoc/getting_started_with_hama.xml
------------------------------------------------------------------------------
    svn:eol-style = native

Added: incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml?rev=1231787&view=auto
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml (added)
+++ incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml Sun Jan 15 23:02:06 2012
@@ -0,0 +1,30 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<document xmlns="http://maven.apache.org/XDOC/2.0"
+xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
+  <properties></properties>
+  <body>
+    <section name="Hama BSP Tutorial"></section>
+    <p>This document describes the Hama BSP framework and serves as a tutorial.</p>
+    <subsection name="Overview"></subsection>
+    <subsection name="Inputs and Outputs"></subsection>
+    <subsection name="BSP user interfaces"></subsection>
+    <subsection name="Example: Pi calculation"></subsection>
+  </body>
+</document>

Propchange: incubator/hama/trunk/src/site/xdoc/hama_bsp_tutorial.xml
------------------------------------------------------------------------------
    svn:eol-style = native

Added: incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml?rev=1231787&view=auto
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml (added)
+++ incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml Sun Jan 15 23:02:06 2012
@@ -0,0 +1,45 @@
+<?xml version="1.0" encoding="iso-8859-1"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<document xmlns="http://maven.apache.org/XDOC/2.0"
+xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
+  <properties></properties>
+  <body>
+    <section name="Hama on Clouds"></section>
+    <p>This document describes how to deploy Hama clusters on Clouds e.g., EC2, Rackspace
using Whirr.</p>
+    
+    <subsection name="Deploy a Hama cluster on EC2"></subsection>
+    <p>The following commands install Whirr and start a 5 node Hama cluster on Amazon
EC2 in 5 minutes or less.
+    <pre class="green">
+  % curl -O http://www.apache.org/dist/whirr/whirr-0.x.0/whirr-0.x.0.tar.gz
+  % tar zxf whirr-0.x.0.tar.gz; cd whirr-0.x.0
+
+  % export AWS_ACCESS_KEY_ID=YOUR_ID
+  % export AWS_SECRET_ACCESS_KEY=YOUR_SECKEY
+  % ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa_whirr
+
+  % bin/whirr launch-cluster --config recipes/hama-ec2.properties --private -key-file ~/.ssh/id_rsa_whirr
+    </pre>
+    </p>
+    <subsection name="Execute Hama Examples"></subsection>
+    <pre class="green">
+  % cd /usr/local/hama-0.x.0-incubating
+  % bin/hama jar hama-examples-0.x.0-incubating.jar [args]
+    </pre>
+    </body>
+</document>

Propchange: incubator/hama/trunk/src/site/xdoc/hama_on_clouds.xml
------------------------------------------------------------------------------
    svn:eol-style = native

Modified: incubator/hama/trunk/src/site/xdoc/index.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/index.xml?rev=1231787&r1=1231786&r2=1231787&view=diff
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/index.xml (original)
+++ incubator/hama/trunk/src/site/xdoc/index.xml Sun Jan 15 23:02:06 2012
@@ -22,34 +22,51 @@ xsi:schemaLocation="http://maven.apache.
   <body>
 
     <section name="Apache Hama"></section>
-       <div align="center"><img src="images/hama_art_arthur_300x220.png"/></div>
+      <img src="images/hama_art_arthur_300x220.png" align="right" style="margin-top:-20px;margin-left:20px;margin-right:20px;"
/>
       <p>
-      <strong>Apache Hama</strong> is a distributed computing framework based
on 
-      BSP (Bulk Synchronous Parallel) computing techniques for massive scientific computations,

-      e.g., matrix, graph and network algorithms. 
-      It was inspired by Google's Pregel, but different in the sense 
-      that it's purely BSP and common model, not just for graph.
+      <strong>Apache Hama</strong> is a Pure BSP computing framework on top of
Hadoop HDFS for massive scientific computations 
+      e.g., matrix, graph and network algorithms. Currently, it has follow features:
       </p>
-      
+      <ul>
+        <li>Job submission and management interface.</li>
+        <li>Multiple tasks per node.</li>
+        <li>Input/Output Formatter.</li>
+        <li>Checkpoint recovery.</li>
+        <li>Supports to run in the Clouds using <a href="hama_on_clouds.html">Apache
Whirr</a>.</li>
+        <li>Supports to run with Hadoop YARN.</li>
+      </ul>
+      <br/>
       <table class="green" width="100%" cellspacing="0" cellpadding="5" border="1">

       <tr><td> 
           <h3 align="center">Recent News</h3>
           <ul>
+            <li>Jan 30, 2012: release 0.4.0 available</li>
             <li>July 28, 2011: release 0.3.0 available</li>
             <li>June 2, 2011: release 0.2.0 available</li>
             <li>Apr 30, 2010: Introduced in the <a href="http://www.bsp-worldwide.org/bspww3000.html">BSP
Worldwide</a></li>
-            <li>May 20, 2008: Accept Hama to be a Apache incubator project</li>
           </ul>
       </td></tr></table>
-      
+    <subsection name="Why Hama and BSP?"></subsection>
+    <p>Today, many practical data processing applications requires a good scalability,
a flexible model, a high compatibility with existing data systems e.g., HDFS, HBase, and especially
a communication capability which allows to exchange information using message-passing paradigm
beyond MapReduce.
+    Here, Bulk Synchronous Parallel (BSP) model fills the bill nicely and besides, it has
several main advantages over MapReduce and MPI:</p>
+    <ul>
+      <li>Supports message passing paradigm style of application development</li>
+      <li>Provides a flexible, simple, and easy-to-use small APIs</li>
+      <li>Enables to perform better than MPI for communication-intensive applications</li>
+      <li>Guarantees impossibility of deadlocks or collisions in the communication
mechanisms</li>
+    </ul>
+    
     <subsection name="Getting Started"></subsection>
       <p>
       Start by installing Hama on a Hadoop cluster.
       <br/>
       </p>
       <ul>
-        <li><a href="http://wiki.apache.org/hama/">Learn about</a> Hama
by reading the documentation.</li>
         <li><a href="downloads.html">Download</a> Hama from the release
page.</li>
+        <li><a href="getting_started_with_hama.html">Getting Started</a>
with Hama.</li>
+        <li><a href="hama_on_clouds.html">Launch a Hama cluster</a> on
Clouds.</li>
+        <li>Hama <a href="hama_bsp_tutorial.html">BSP Tutorial.</a></li>
+        <li><a href="http://wiki.apache.org/hama">Learn about</a> Hama
and BSP by reading the documentation.</li>
       </ul>
     <subsection name="Getting Involved"></subsection>
       <p>

Modified: incubator/hama/trunk/src/site/xdoc/team-list.xml
URL: http://svn.apache.org/viewvc/incubator/hama/trunk/src/site/xdoc/team-list.xml?rev=1231787&r1=1231786&r2=1231787&view=diff
==============================================================================
--- incubator/hama/trunk/src/site/xdoc/team-list.xml (original)
+++ incubator/hama/trunk/src/site/xdoc/team-list.xml Sun Jan 15 23:02:06 2012
@@ -96,7 +96,7 @@ xsi:schemaLocation="http://maven.apache.
        <tr valign="top">
           <td align="center">chl501</td>
           <td align="center">Chia-Hung Lin</td>
-          <td align="center"></td>
+          <td align="center">.</td>
           <td align="center">committer</td>
         </tr>
     </table>



Mime
View raw message