vcl-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject svn commit: r1430372 [11/11] - /vcl/site/trunk/content/confluence_export/
Date Tue, 08 Jan 2013 16:37:56 GMT
Added: vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext
--- vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext (added)
+++ vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext Tue Jan  8 16:37:53 2013
@@ -0,0 +1,502 @@
+Title: VMware Configuration
+<a name="VMwareConfiguration-Terminology"></a>
+## Terminology
+<a name="VMwareConfiguration-VMHost"></a>
+#### VM Host
+* A VM host is a physical computer running a VMware hypervisor
+* A VCL computer entry must be added for each VM
+host&nbsp;({color:#000000}{_}Manage Computers > Edit Computer
+* {color:#000000}After the computer has been added to VCL, it is designated
+as a VM host by changing the computer state to vmhostinuse{color}
+{color:#000000}_(Manage Computers > Computer Utilities)_{color}
+<a name="VMwareConfiguration-VM"></a>
+#### VM
+* A VM is a virtual machine managed by VCL
+* A computer entry must be added to VCL for each VM
+({color:#000000}{_}Manage Computers > Edit Computer Information)_{color}
+* {color:#000000}Each VM must be assigned to a VM host{color}
+{color:#000000}_(Virtual Hosts > VM Hosts tab > Configure Host)_{color}
+* VMs do not need to be created manually in VMware, VCL automatically
+creates and deletes VMs
+<a name="VMwareConfiguration-VMHostProfile"></a>
+#### VM Host Profile
+* A VM host profile contains&nbsp;several parameters which describe how a
+VM host is configured so that VCL knows how to manage it
+* Each VM host is assigned a VM host profile
+* A VM host profile may be assigned to multiple VM hosts if they are
+configured identically
+* VM host profiles may be added or modified via _Virtual Hosts > VM Host
+Profiles tab_
+<a name="VMwareConfiguration-VMwareProductsSupported"></a>
+## VMware Products Supported
+* VMware Server 2.x
+* VMware ESX 3.5 - 4.x
+* VMware ESXi 4.x
+* VMware ESXi 5.x
+<a name="VMwareConfiguration-VMHost&nbsp;ManagementOptions"></a>
+### VM Host&nbsp;Management Options
+&nbsp;The VCL&nbsp;management node must be able to control the VM host and
+the VMs running on it.&nbsp; VMware provides several different&nbsp;ways of
+doing this.&nbsp; VCL currently supports the following methods for remote
+VM host management:
+* VMware vSphere SDK
+* Use SSH to execute&nbsp;commands directly on the VM (_not officially
+supported by VMware_)
+The vSphere SDK can only&nbsp;be used if management is not&nbsp;restricted
+due to the VMware license key installed on the host.&nbsp; This mainly
+affects hosts running the free version of ESXi.&nbsp; Remote management
+using any of the&nbsp;methods supported by VMware&nbsp;is restricted once a
+free license key is entered.
+If remote management is restricted, the VM host can be managed if SSH is
+enabled on it.&nbsp; VCL will execute vim-cmd and other commands on the VM
+host via SSH.&nbsp;
+<a name="VMwareConfiguration-HowtoenableSSHontheVMhost:"></a>
+#### How to enable SSH on the VM host:
+<a name="VMwareConfiguration-VMwareServer2.x"></a>
+##### VMware Server 2.x
+Enable the SSH daemon and configure identity key authentication according
+to the underlying VM host OS
+<a name="VMwareConfiguration-ESX/ESXi3.5&4.0"></a>
+##### ESX/ESXi 3.5 & 4.0
+* Connect to the console of the ESX/ESXi host
+* Press *ALT-F1* \- you should see a black screen with the VMware product
+name at the top
+* Type the word *unsupported* and press *Enter* (you won't see the letters
+appear as you type them)
+* You should see a password prompt, type&nbsp;in the *root
+password*&nbsp;and press *Enter*
+* Edit the file: *vi /etc/inetd.conf*
+* Uncomment the first line beginning with _\#ssh_ by deleting the #
+* Save the file - press *Esc* and then *:wq*
+* Kill the inetd process
+** Determine the PID of the inetd process: *ps \| grep inetd*
+You should see a line that looks like: _5065 5065 busybox inetd_
+** Kill the process (enter the PID from the output of the previous
+command): *kill \-HUP 5065*
+<a name="VMwareConfiguration-ESXi4.1"></a>
+##### ESXi 4.1
+Beginning with *ESXi 4.1*, SSH&nbsp;can be enabled using the vSphere
+* Select the ESXi host
+* Select the *Configuration* tab
+* Select&nbsp;*Security Profile* under Software
+* Click *Properties*
+* Select *Remote Tech Support (SSH)*
+* Click *Options*
+* Select *Start automatically*
+* Click *Start*
+* Click *OK*
+<a name="VMwareConfiguration-ESX5.0"></a>
+##### ESX 5.0
+In the case of *ESX 5.0*:
+* Select the ESXi host
+* Select the *Configuration* tab
+* Select&nbsp;*Security Profile* under Software
+* Click *Properties*
+* Select *SSH Server*
+* Click *Options*
+* Confirm that *Start automatically* is selected
+* Click *OK*
+<a name="VMwareConfiguration-HowtoconfigureESX/ESXitouseSSHidentitykeyauthentication:"></a>
+#### How to configure ESX/ESXi to use SSH identity key authentication:
+SSH identity key authentication&nbsp;must be configured if SSH is used to
+manage the VM host.
+* Create an SSH key pair on the management node (or use a key you
+previously created):
+ssh-keygen \-t rsa \-f /etc/vcl/vcl.key \-N '' \-b 1024 \-C 'VCL root
+* Log into the ESX host via SSH (password authentication should work) and
+create the directory:
+ssh <ESXi host> 'mkdir /.ssh'
+* Copy the public key to the ESXi host:
+ESXi 4.x:
+scp /etc/vcl/ <ESXi host>:/.ssh/authorized_keys
+ESXi 5.x:
+scp /etc/vcl/ <ESXi host>:/etc/ssh/keys-root/authorized_keys
+* Test making an SSH connection using the key:
+ssh \-i /etc/vcl/vcl.key <ESXi host>
+{color:#ff0000}{*}IMPORTANT:*{color} Under ESXi 4.x, the authorized_keys
+file is erased when the ESXi VM host is rebooted.&nbsp;Complete the
+following steps to&nbsp;make&nbsp;the authorized_keys file&nbsp;persistent:
+_Note: VCL will perform these steps automatically when the 1st reservation
+assigned to the host is processed._
+* Create a compressed tarball file containing the /.ssh directory:
+tar \-C / \-czf bootbank/vcl.tgz .ssh
+* Edit the /bootbank/boot.cfg file and append *_' \--\- vcl.tgz'_* to
+modules line as shown in the following example:
+modules=k.z --- s.z --- c.z --- oem.tgz --- license.tgz --- m.z ---
+state.tgz --- vcl.tgz
+Optionally you can run the following two commands:
+tar \-C / \-czf vcl.tgz .ssh \--add=vcl.tgz \--verbose
+<a name="VMwareConfiguration-VMHostProfileParameters"></a>
+## VM Host Profile Parameters
+<a name="VMwareConfiguration-GeneralParameters"></a>
+### General Parameters
+<a name="VMwareConfiguration-"></a>
+<a name="VMwareConfiguration-*Name*"></a>
+##### *Name*
+* Descriptive name of the VM host&nbsp;profile
+<a name="VMwareConfiguration-*Type*_(deprecated)_"></a>
+##### *Type* _(deprecated)_
+* Removed in VCL 2.3
+<a name="VMwareConfiguration-*Image*_(optional)_"></a>
+##### *Image* _(optional)_
+* VCL hypervisor image installed on VM host computers using xCAT
+{info}{_}xCAT is not required. &nbsp;VM host computers may be installed
+manually or by some other means._{info}
+* If xCAT is not used, select "No Image"
+* VCL has the ability to install a hypervisor image on bare-metal computers
+using xCAT. &nbsp;If the image property is configured, the image is
+installed when a computer's state is changed to vmhostinuse via _Manage
+Computers > Computer Utilities_
+<a name="VMwareConfiguration-*Username/Password*_(optional)_"></a>
+##### *Username/Password* _(optional)_
+* Name and password&nbsp;of the&nbsp;administrative or root user residing
+on the VM host
+* This account is used to manage the VM host and VMs assigned to the host
+* The username and password are currently only used if the vSphere SDK is
+used to manage the VM host and VMs
+<a name="VMwareConfiguration-StorageParameters"></a>
+### Storage Parameters
+<a name="VMwareConfiguration-ResourcePath_(optional)_"></a>
+##### Resource Path _(optional)_
+Resource Path only needs to be configured if VMware vCenter is used. It
+defines the location where VMs will be created in the vCenter inventory
+tree. The inventory tree contains at least one Datacenter, and may also
+contain Folders, Clusters, and Resource Pools.
+Example: /DatacenterA/Folder1/Cluster2/ResourcePool3
+<a name="VMwareConfiguration-*RepositoryPath*_(optional)_"></a>
+##### *Repository Path* _(optional)_
+* Path where master copies of images are stored which are&nbsp;used to
+transfer images to VM host datastores or to other repositories:
+** If a reservation is assigned to a host but the image does not  exist in
+that host's datastore, it is copied from the repository to the virtual disk
+path when the VM is loaded
+** If the VCL environment contains multiple management nodes and the  image
+does not exist in the repository or the host's datastore, the  image will
+be retrieved from another management node's repository by  copying it via
+* The Repository Path parameter does not need to be configured if the  VCL
+environment contains a single management node and all VM hosts	share the
+same Virtual Disk Path
+* Example:&nbsp;_/vmfs/volumes/nfs-repository1_
+* VMs do not run directly off of the images stored in the repository
+* Setting the Repository Path parameter determines whether or not an 
+additional copy of an image is created when an image is captured
+** If repository path is not configured then only a single copy of the
+image will exist in the virtual disk path after an image is captured
+** If repository path is configured then two copies of the image will 
+exist after an image is captured - one in the virtual disk path and one in
+the  repository
+* Repository Path location can refer to and be mounted on either the
+management node or VM host
+** It is highly recommended that the repository be mounted on the VM host
+*** When mounted on the VM host, vmdk operations can be done directly on
+the VM host in a single step
+* Images in the repository are stored in the 2 GB sparse vmdk format
+** The size of the vmdk files will approximately be equal to the  amount of
+actual data saved in the image regardless of the size of the  VM's hard
+** Storing images in the 2 GB sparse format is necessary to allow  images
+to be transferred via SCP without having to transfer data equal  to the
+entire size of the VM's hard drive
+<a name="VMwareConfiguration-RepositoryImageType"></a>
+##### Repository Image Type
+Virtual disk file format for images stored in the repository.
+<a name="VMwareConfiguration-*VirtualDiskPath(previouslyDatastorePath)*"></a>
+##### *Virtual Disk Path (previously Datastore Path)*
+* Location where master copies of images are stored which are used by
+running VMs
+* Example:&nbsp;_/vmfs/volumes/nfs-datastore1_
+{info}For ESXi, the path configured in the profile may simply be the short
+datastore name as it appears in the vSphere Client: nfs-datastore1{info}
+* Storage location should be large enough to store all of the images which
+may be loaded on the VM host _(from 100's of GB to several TB)_
+* VCL creates a directory for each image in the Virtual Disk Path
+* Images are stored in the [vmfs thin vmdk format](
+* Virtual Disk Path may either reside on local or network storage
+* Multiple VM hosts can share the same datastore if network storage is used
+** A single datastore may be used by all VM hosts if performance is
+** Multiple VMs on different hosts may access the same Virtual Disk Path
+image at the same time
+** It is recommended that datastores are shared among hosts so that fewer
+copies of each image have to be stored
+** The underlying storage hardware and network connectivity from the hosts
+to the storage must be adequate
+** Storage where the datastore is located should be optimized for read
+* VCL configures VMs to access images stored in the Virtual Disk Path in
+read-only mode
+** Changes made to the VM's hard drive are written to delta files located
+in the VM Working Directory Path dedicated for the VM
+<a name="VMwareConfiguration-VirtualDiskImageType"></a>
+##### Virtual Disk Image Type
+Virtual disk file format for images stored in the virtual disk path.
+<a name="VMwareConfiguration-VirtualDiskMode(previously*VMDisk)*"></a>
+##### Virtual Disk Mode (previously *VM Disk)*
+* Defines whether the storage where the VM host's Virtual Disk Path 
+resides is dedicated to a single host or shared among multiple hosts:
+** dedicated (previously localdisk)
+*** The VM host's Virtual Disk Path&nbsp;is located on local disks or
+dedicated network storage
+*** The VM host is the only host which accesses the Virtual Disk Path
+*** Repository Path must be configured
+** shared (previously networkdisk)
+*** The VM host's Virtual Disk Path&nbsp;is located on network storage
+which is shared by other VM hosts
+*** Repository Path is optional
+The Virtual Disk Mode (VM Disk) parameter does *not* determine whether or
+...images are copied from the datastore to the repository during image
+...images are copied from the repository to the datastore during image load
+These are determined by whether or not Repository Path is configured in the
+<a name="VMwareConfiguration-VMWorkingDirectoryPath_(optional)_(previously*VMPath*)"></a>
+##### VM Working Directory Path _(optional)_ (previously *VM Path*)
+* Defines path on VM host where VM working directories will reside
+(contains .vmx, delta, .vswp, nvram files)
+* If not configured, the Virtual Disk Path location will be used
+* VCL creates a directory under the VM Working Directory Path for each VM
+it creates
+** Contains the .vmx file which defines the VM
+** Contains delta vmdk files which are written to as changes are made to
+the VM's hard drive
+* VM Working Directory Path may either reside on local or network storage
+* Location should be dedicated for each VM host
+** Multiple VM hosts should not share the same VM Working Directory Path
+location for performance and image safety reasons
+** VM Working Directory Paths of multiple hosts may reside on the same
+volume but a subdirectory should be created for each host
+* Storage where the VM Working Directory Path is located should be
+optimized for read-write performance
+<a name="VMwareConfiguration-"></a>
+<a name="VMwareConfiguration-NetworkingParameters"></a>
+#### Networking Parameters
+<a name="VMwareConfiguration-*VMNetwork(previouslyVirtualSwitch)*"></a>
+##### *VM Network (previously Virtual Switch)*
+* VM Network 0 (previously Virtual Switch 0) - private VCL
+* VM Network 1 (previously Virtual Switch 1) - public network used by user
+making reservation&nbsp;to access the VMs
+* The VM Network parameters should match the network names configured on
+the VM host
+** For ESXi, the _VM Network_ parameters must match the _Virtual Machine
+Port Group Network Labels_ configured in the vSphere Client, example:
+*** VM Network 0: Public
+*** VM Network 1: Private
+&nbsp; !vmware-network-labels.gif|border=1!
+* For VMware Server 2.x, the _VM Network_ parameters must match the
+_Network Names_ configured&nbsp;by running&nbsp;
+<a name="VMwareConfiguration-*Generateeth0/eth1MAC*"></a>
+##### *Generate eth0/eth1 MAC*
+* New in VCL 2.3
+* Determines whether VMs are assigned MAC addresses defined in the VCL
+database or if random MAC addresses should be assigned
+<a name="VMwareConfiguration-ConfigurationExamples"></a>
+## Configuration Examples
+<a name="VMwareConfiguration-LocalDiskOnly-RepositoryMountedviaNFS"></a>
+### Local Disk Only - Repository Mounted via NFS
+<a name="VMwareConfiguration-!local-only-nfs.gif!"></a>
+### !local-only-nfs.gif!
+The diagram above shows a simple VCL configuration with 1 management node
+and 2 VMware ESXi hosts.&nbsp; Network storage is not used.
+The local disks on the VM hosts are used to store all of the files used by
+running VMs including the VM's working directory and the master vmdk image.
+A directory on the local disk on the management node is used to as the
+image repository.&nbsp;  This directory is exported via NFS.&nbsp; VM hosts
+mount this directory  as a datastore named "repository".&nbsp; Mounting the
+repository directly on	the VM hosts allows the vmkfstools utility to be
+used on the VM hosts to  copy and convert images directly from the
+repository to the local  datastore in a single step.
+If an image is to be loaded on a VM host and that image does not already
+exist in the VM host's local datastore (Virtual Disk Path), it is
+automatically copied from the repository to the VM host's local datastore
+(Virtual Disk Path) at the beginning of the load process.
+During image capture, images are automatically copied to from the VM host's
+local datastore (Virtual Disk Path) to the repository.&nbsp; This allows
+images captured on a VM host to be loaded on any other VM host.
+The VM host profile Virtual Disk Mode parameter is set to dedicated.&nbsp;
+This indicates to the load process that the VM host's Virtual Disk Path is
+dedicated to the VM host and not shared by other VM hosts.&nbsp; This
+allows images to be deleted from the VM host's local datastore (Virtual
+Disk Path) if another image must be copied from the repository and not
+enough space is available.
+<a name="VMwareConfiguration-LocalDiskOnly-RepositoryNotAvailableviaNFS"></a>
+### Local Disk Only - Repository Not Available via NFS
+This example is identical to the one above except that the repository
+located on the management node's local disk is not exported via NFS.&nbsp;
+Because of this, images must be transferred using SCP instead of
+vmkfstools.&nbsp; This is less desirable than mounting the repository
+directly on the VM hosts because images cannot be copied and converted in a
+single step.&nbsp; Images are stored in the repository in the 2GB sparse
+format.&nbsp; This allows the images to be copied via SCP while only
+transferring the data stored in the image, not the entire size of the hard
+drive stored in the image.&nbsp; VMware ESXi cannot run VMs using vmdk
+images stored in the 2GB sparse format.&nbsp; Images are converted to the
+vmfs thin format so that they can be loaded on VMware ESXi.&nbsp; This adds
+extra time to the load process if an image does not exist in the VM's local
+datastore (Virtual Disk Path) and must be copied from the repository.&nbsp;
+It also requires additional space in the VM host's local datastore (Virtual
+Disk Path) becuase 2 copies of the image exist while it is being converted.
+Note that the VM host profile Repository Path parameter is set to the path
+on the management node's hard drive.&nbsp; The code first checks if the
+path exists on the VM host.&nbsp; If not, it assumes the repository is not
+mounted directly on the VM host and the Repository Path value refers to a
+location on the management node.
+<a name="VMwareConfiguration-NetworkStorageOnly-NoRepository"></a>
+### Network Storage Only - No Repository
+This is an example of a simple configuration where the network storage is
+A repository is not used in this configuration. &nbsp;This implies that all
+VM hosts which will ever be added to this VCL environment will need to be
+able to connect to the network storage.
+A datastore to be used as the Virtual Disk Path named "datastore" is
+mounted on every VM host. &nbsp;Each of these mounts points to the same
+location on the network storage. &nbsp;The datastore will contain the
+master vmdk images. &nbsp;VMs loaded on the VM hosts will read from these
+master vmdk images.
+A datastore to be used as the VM Working Directory Path named "vmpath" is
+also mounted on each VM host. &nbsp;However, the location to which each VM
+host points should be different. &nbsp;In the example above, vmhost-a-01
+points to th the /vmpath01 directory on the network storage and vmhost-a-02
+points to the /vmpath02 directory. &nbsp;These locations may be different
+network storage filesystems or may be different directories on the same
+network filesystem. &nbsp;Even though the mounts on the VM hosts point to
+different locations, the datastore names configured under ESXi are
+identical. &nbsp;This allows you to use the same VCL VM host profile for
+all of the VM hosts.
+The VM host profile Virtual Disk Mode parameter is set to shared.&nbsp;
+This indicates to the load process that the VM host's Virtual Disk Path is
+shared by other VM hosts.

Added: vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext
--- vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext (added)
+++ vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext Tue Jan  8 16:37:53 2013
@@ -0,0 +1,78 @@
+Title: VMware Provisioning Module Information
+<a name="VMwareProvisioningModuleInformation-VMwareProvisioningModule"></a>
+## VMware Provisioning Module
+* Source file: managementnode/lib/VCL/Module/Provisioning/VMware/
+<a name="VMwareProvisioningModuleInformation-SupportedVMwareProducts"></a>
+#### Supported VMware Products
+* VMware Server 2.0
+* VMware ESX/ESXi 3.5
+* VMware ESX/ESXi 4.0
+<a name="VMwareProvisioningModuleInformation-SupportedVMware&nbsp;APIs,SDKs,&CLIs"></a>
+#### Supported VMware&nbsp;APIs, SDKs, & CLIs
+There are several different methods to control VMware hosts and virtual
+machines.&nbsp; The main module should remain as
+SDK/API/CLI-agnostic as possible.&nbsp; All code which interacts with a
+VMware remote control method should reside in utility modules located in
+the lib/VCL/Module/Provisioning/VMware directory.
+* vSphere SDK - lib/VCL/Module/Provisioning/VMware/
+** Provides full VM and VM host remote access functionality
+** Provides VM host datastore file access
+** Can only be used if the VMware&nbsp;host is not running with a
+restricted license
+** SSH access is not required to the VM host
+** Installed by default on ESX/ESXi hosts
+** Can be installed on Server 2.0 hosts
+* -VIX API - lib/VCL/Module/Provisioning/VMware/
+_Although a utility module has been written to use the VIX API, it is not
+supported and will not be maintained.&nbsp; Everything that the VIX API can
+do can be done by the vim-cmd CLI and SSH access to the VM host is required
+for both methods.&nbsp; Therefore, there is no benefit to maintain&nbsp;the
+VIX API module._
+** -Provides basic VM control functions-
+** -Does not provide any file access-
+** -SSH access is required to the VM host in order to manipulate the file
+** -Can only be used if the VMware&nbsp;host is not running with a
+restricted license-
+** -Installed by default on Server 2.0 hosts-
+* vim-cmd CLI via SSH - lib/VCL/Module/Provisioning/VMware/
+** Provides full VM host remote access functionality
+** Does not provide file access
+** SSH access is required to the VM host in order to manipulate the file
+** Can be used if the the VMware host is running with a restricted license
+<a name="VMwareProvisioningModuleInformation-RequiredUtilityModuleSubroutines"></a>
+### Required Utility Module Subroutines
+* get_registered_vms
+* vm_register
+* vm_unregister
+* get_vm_power_state
+* vm_power_on
+* vm_power_off
+* copy_virtual_disk
+* move_virtual_disk
+* get_virtual_disk_controller_type
+* get_virtual_disk_hardware_version
+* get_virtual_disk_type
+<a name="VMwareProvisioningModuleInformation-RequiredOSModuleSubroutines"></a>
+### Required OS Module Subroutines
+* find_files
+* copy_file
+* copy_file_to
+* copy_file_from
+* delete_file
+* move_file
+* file_exists
+* get_available_space
+* create_directory
+* get_file_contents
+* get_file_size

Added: vcl/site/trunk/content/confluence_export/web-code-installation.mdtext
--- vcl/site/trunk/content/confluence_export/web-code-installation.mdtext (added)
+++ vcl/site/trunk/content/confluence_export/web-code-installation.mdtext Tue Jan  8 16:37:53 2013
@@ -0,0 +1,159 @@
+Title: Web Code Installation
+{excerpt}This page describes how to install and configure the frontend VCL
+web code components including the web server prerequisites and frontend VCL
+web code. It also describes how to add local web accounts, configure LDAP
+authentication, and set the timezone correctly.{excerpt}
+<a name="WebCodeInstallation-*Prerequisites*"></a>
+## *Prerequisites*
+Your web server should meet the following criteria before installing the
+frontend VCL code:
+* Apache HTTP Server v1.3 or v2.x with SSL enabled - while VCL may run
+under another webserver capable of running PHP code, it has only been
+tested to work with Apache HTTP Server
+* PHP 5
+* php modules that should be installed (depending on your Linux distro,
+some of these may be compiled in to php instead of being a separate
+** php-gd
+** php-json (if your PHP version is < 5.2, this is not required)
+** php-mcrypt
+** php-mysql
+** php-openssl
+** php-sysvsem
+** php-xml
+** php-xmlrpc
+* useful to have the server set up to be able to send debugging emails
+* php-mcrypt requires libmcrypt and mcrypt libraries as dependencies. 
+These may need to be installed first.
+<a name="WebCodeInstallation-*InstallingVCLFrontendWebCode*"></a>
+## *Installing VCL Frontend Web Code*
+1. If you haven't already done so, download and extract a copy of the latest
+release. There is a link to it under the Project Resources section on our
+wiki [home page](apache-vcl.html)
+. Look for "Current version".
+1. copy the "web" directory to a location somewhere under the web root of
+your web server:
+    cp -r web/ /var/www/html/vcl
+1. modify vcl/.ht-inc/secrets.php
+1. * set $vclhost, $vcldb, $vclusername, and $vclpassword to match your
+database setup
+1. * create random passwords for $mcryptkey, $mcryptiv, and $pemkey -
+$mcryptiv must be 8 hex characters
+1. run the script from within vcl/.ht-inc and give it $pemkey
+from secrets.php as the passphrase (3 times, copy/paste is a good idea
+1. modify vcl/.ht-inc/conf.php to match your site - COOKIEDOMAIN needs to be
+the domain name your web server is using, or left blank if you are
+accessing it by IP only.
+*NOTE:* There is a misconfiguration in conf.php in VCL 2.1.  To correct it,
+change affiliationid for "Local Account" in the $authMechs array from 4 to
+1. \**NOTICE*\* JpGraph 2.x is no longer available.  JpGraph 3.x is released
+under a dual license. QPL 1.0 (Qt Free Licensee).  Free for non-commercial,
+open-source or educational use (JpGraph Professional License for commercial
+use).  If you are planning to use this for commercial use and don't want to
+pay for JpGraph, you can safely skip this step with the only side effect of
+not being able to display a few graphs on the statistics page.
+Download JpGraph from [](
+1. * For PHP5, download the 3.x series, extract it, and copy the src
+directory from it to vcl/.ht-inc/jpgraph
+1. download version 0.4.0 of Dojo Toolkit: [](
+1. * extract it under the vcl directory and rename "dojo-0.4.0-ajax" to
+1. download version 1.1.0 of Dojo Toolkit: [](
+1. * extract it under the vcl directory and rename "dojo-release-1.1.0" to
+1. go into the themes directory (vcl/themes) and run "./
+default" to copy parts of dojo's css into the "default" theme
+1. if you want to be able to edit any of the documentation that comes
+bundled with the vcl web code, download fckeditor from [](
+ (most people can skip this step)
+1. * extract it under the vcl directory
+1. open a browser and open the testsetup.php page
+1. * i.e. if you set up your site to be [](
+ open []
+1. debug any issues reported by testsetup.php
+1. now, open the index.php page in your browser
+1. select Local Account and use 'admin' as the user and 'adminVc1passw0rd'
+as the password
+1. click the "Management Nodes" link
+1. enter the hostname and IP of your management node
+1. click Add
+1. fill in "Install Path" - this is parent directory under which image files
+will be stored
+1. enter "/etc/vcl/vcl.key" for "End Node SSH Identity Key Files"
+1. click "Confirm Management Node"
+1. click Submit
+1. click the "Management Nodes" link
+1. select "Edit Management Node Grouping"
+1. click Submit
+1. select the checkbox for your management node
+1. click Submit
+1. click "Manage Computers"
+1. select the "Add Single Computer" radio button
+1. click the Submit
+1. fill in Hostname, IP Address, owner (admin@Local), RAM, Proc Speed,
+Network Speed, select "blade" for Type, select "xCAT 1.x Provisioning" for
+"Provisioning Engine", and click the checkbox under "allcomputers", and
+&nbsp;&nbsp; &nbsp;Note: if using using vmware, select "virtualmachine" for
+Type and "VMWare Server Provisioning" for "Provisioning Engine"
+1. click Confirm Computer
+1. click Submit (don't worry about the fact that the computer you just added
+isn't listed after clicking Submit)
+1. after you've configured your image library and your management node has
+started checking in, you should be able to make a reservation
+<a name="WebCodeInstallation-*Addingextralocalaccounts*"></a>
+## *Adding extra local accounts*
+There's not currently a tool for this.	You will need to add entries
+directly to the database.
+1. add entry to user table
+    INSERT INTO user (unityid, firstname, lastname, email, lastupdated) VALUES
+('myusername', 'myfirstname', 'mylastname', 'myemailaddr', NOW());
+1. find out the id generated for that user
+    SELECT id, unityid FROM user WHERE unityid = 'myusername';
+1. add entry to the localauth table
+    INSERT INTO localauth (userid, salt, passhash, lastupdated) VALUES
+('place1', 'place2', 'place3', NOW())
+with place1 = id from step 2
+place2 = an 8 char random string
+place3 = sha1sum( desired password with place2 stuck on the end )
+this can be generated under linux like this (using 'thedog' as the password
+and 11111111 as place2):
+echo \-n 'thedog11111111' \| sha1sum
+Once a user has been added, the user can go to User Preferences to change
+his/her password
+<a name="WebCodeInstallation-*AddingLDAPauthentication*"></a>
+## *Adding LDAP authentication*
+1. fill in the necessary information in vcl/.ht-inc/conf.php
+1. add an entry to the affiliation table and use the id for that entry as
+'affiliationid' for your new entry in vcl/.ht-inc/conf.php
+1. uncomment the 'require_once(".ht-inc/authmethods/ldapauth.php");' line in
+in vcl/.ht-inc/conf.php
+<a name="WebCodeInstallation-*SettingTimeZone*"></a>
+## *Setting Time Zone*
+1. Edit /var/www/html/vcl/.ht-inc/php5extras.php to indicate the correct
+time zone:
+1. Edit /var/www/html/vcl/.ht-inc/requests.php (line 3301 currently) to
+indicate the correct time zone (purely cosmetic):
+print "<small>(Pacific Time Zone)</small>";

Added: vcl/site/trunk/content/confluence_export/web-code-overview.mdtext
--- vcl/site/trunk/content/confluence_export/web-code-overview.mdtext (added)
+++ vcl/site/trunk/content/confluence_export/web-code-overview.mdtext Tue Jan  8 16:37:53 2013
@@ -0,0 +1,454 @@
+Title: Web Code Overview
+<a name="WebCodeOverview-*GeneralOverview*"></a>
+## *General Overview*
+The code is broken up into multiple files, based on sections of the site
+(as divided up by the navigation area). There are also several general
+purpose files:
+* index.php -  This is the only file used in any URLs. It includes other
+files based on the passed in "mode".
+* states.php - builds an array of modes, which function should be called
+for each mode, and the part of the site each mode belongs to
+* utils.php - contains many functions that are common to several sections
+* errors.php - contains an array of all errors that can be reported by the
+site. I'm not sure how useful it was do set it up this way, but it's where
+things currently are. errors.php also contains the debug function that is
+used if the current user has an adminlevel of ADMIN_DEVELOPER in the user
+* secrets.php - contains all passwords/passphrases and secret and shared
+* conf.php - contains all configuration variables that might need to be
+modified for each install base
+* php5extras.php - anything that needs to be done differently when using
+php5 vs php4
+<a name="WebCodeOverview-*SiteSecurity*"></a>
+## *Site Security*
+The initial handling of form data within the code was quite insecure, and
+several areas of the site are still this way. After learning more about web
+security, I developed a security model based on "continuations". All of the
+pages have been converted to using continuations.
+Deep linking into the site is only allowed for modes in the $actions\['entry'\]('entry'\.html)
+ array in states.php. Anything else requires the submission of a
+continuation. For the most part, access to different parts of the site is
+controlled by what privileges you have in the Privileges section of the
+site. However, there are a few things controlled by a user's adminlevel
+field in the user table. The very earliest form of authorization was
+handled by the user's adminlevel field, and it has continued to be useful
+in a few situations.
+All form data passed in to the site should be verified very strictly.
+Unfortunately, that is not currently the case. All of the main pages
+available to the average user should have been updated to have strict
+validation, though other parts of the site have not made it yet. Most
+sections of the site have a single function (or a very small number of
+similar functions) that handle the processing of form data. This will make
+it easier to add better checks throughout the site as the number of
+locations needing to be modified is fairly small.
+<a name="WebCodeOverview-*GeneralProcessingFlow*"></a>
+## *General Processing Flow*
+Every time someone views the site, it is through index.php. This file
+defines several global variables and includes conf.php, states.php,
+errors.php, and utils.php. It then creates a database connection and calls
+initGlobals(), which among other things, determines $mode. Based on $mode,
+index.php determines which function needs to be called and assigns it to
+$actionFunction. Next, checkAccess() is called, followed by sendHeaders()
+and printHTMLHeader() (which doesn't actually print the header if $mode is
+in $noHTMLwrappers). set_error_handler is called if the current user has an
+adminlevel of ADMIN_DEVELOPER. Next, $actionFunction is finally called,
+followed by disconnecting from the database, a call to printHTMLFooter(),
+and a call to semUnlock() to make sure the semaphore was unlocked if it was
+locked somewhere in the process.
+One thing worth noting that initGlobals() does is to include other files
+based on which mode is being processed. This helps to prevent the php
+script engine from having to process unnecessary files. It also adds a
+small layer of security because a section of code cannot be attacked if it
+has not been included.
+<a name="WebCodeOverview-*Continuations*"></a>
+## *Continuations*
+Continuations are how the site handles sequences of pages. It also helps
+keep people from getting to parts of the site they aren't allowed to access
+or shouldn't jump in to the middle of (i.e. by using the browser's back
+button). Continuations are created by calling addContinuationsEntry, which
+accepts up to 6 arguments:
+* $nextmode - the mode that should be used when the continuation is
+* $data - an array of any data that you would like to have available when
+the continuation is submitted
+* $duration - how long from "now" that the continuation should be valid (in
+* $deleteFromSelf - boolean - 1 if this is the start of a new sequence of
+pages, 0 if it will be a continuation of a sequence. If it is set to 0,
+then the "parent" continuation for this one is the continuation that the
+site is currently processing.
+* $multicall - boolean - whether or not the continuation may be submitted
+more than once
+* $repeatProtect - boolean - used in cases where a "sequence loop" can
+addContinuationsEntry returns a long encrypted string to be used as an
+identifier to be submitted (either as a hidden form field or in the URL for
+a GET link). What gets encrypted is a salt value, the id of the
+continuation that was created (or updated), the user's numeric id, and a
+timestamp. If the string gets tampered with, it will not decrypt properly.
+If someone tries to submit a continuation given to another user, their user
+ids won't match; so, it will be considered invalid.
+When a continuation is submitted, some checks are run and, if everything
+passes, whatever was submitted as $nextmode is the mode for which the site
+functions. One of those checks is that $duration has not expired; if it
+has, the user is given a notice that he has submitted expired data. Any
+data submitted as $data can be accessed by calling getContinuationsData()
+with a single argument being the index of the array that was passed to
+addContinuationsEntry. Additionally, getContinuationsData can be called
+with no arguments to get all of $data as a single array. If $multicall was
+set to 0, then the continuation is deleted. If $deleteFromSelf was also set
+to 0, then this continuation's parent will also be deleted. If
+$deleteFromSelf was set to 0 for the parent, it's parent will be deleted,
+and so on until a continuation is reached that had $deleteFromSelf set to
+<a name="WebCodeOverview-*JavascriptandAJAX*"></a>
+## *Javascript and AJAX*
+Efforts have been made to ensure the pages required for making and
+connecting to a reservation work without requiring any javascript. However,
+enhancements have been made to enrich those parts of the site if javascript
+is enabled. For some of the administrative parts of the site, javascript
+and AJAX have been used heavily, particularly the Privileges page. The [Dojo Toolkit](
+ is what is being used for javascript widgets and to handle some of the
+<a name="WebCodeOverview-*AFewExamples*"></a>
+## *A Few Examples*
+<a name="WebCodeOverview-*Addingalinktothenavigationarea*"></a>
+### *Adding a link to the navigation area*
+Here are the steps that would need to be done to add a new section of the
+First, modify states.php to add a new mode.
+1. create a new $actions\['mode'\]('mode'\.html)
+ with the name of your mode set to the name of the function that should be
+1. create a new $actions\['pages'\]('pages'\.html)
+ with the name of your mode set to the name of the section this mode
+belongs to. This is only an internal identifier used to associate modes
+So, if your mode is named "examplemode", you could end up with these two
+lines being added:
+    $actions['mode']
+['examplemode'] = "exampleFunc1";
+    $actions['pages']
+['examplemode'] = "exampleSection";
+While we're editing states.php, lets jump to the top and add our new mode to $actions\['entry'\]('entry'\.html)
+ so that it can be called directly without having to already be on the
+site. Just add 'examplemode' as a new item at the end of the array.
+The next thing to do is to actually add the functions. Lets place them in a
+new file called 'examples.php' in the .ht-inc directory. Our first function
+can be really simple and just print out some text. So, create examples.php
+with this in it:
+    <?php
+    function exampleFunc1() {
+       print "exampleFunc1 successfully called.<br>\n";
+    }
+    ?>
+There's one last thing we need to do before linking this in on the site. As
+described in the "General Processing Flow" section, initGlobals includes
+the required files based on the current mode's section. So, edit utils.php
+and scroll toward the end of it where files are included (using
+require_once) within a switch statement. In the switch statement, before
+the "default" case, add
+    case 'exampleSection':
+       require_once('.ht-inc/examples.php');
+       break;
+Now, we're ready to actually add a link for this example function to the
+navigation area (of course, not all modes are linked to from the navigation
+area, but it is an easy example). To do this, edit utils.php and find the
+getNavMenu function close to the bottom of the file. We'll add our new mode
+to the end; so, find the "logout" part which should look something like
+    if($inclogout) {
+       $rt .= menulistLI('authentication');
+       $rt .= "<a href=\"" . BASEURL . SCRIPT . "?mode=logout\">";
+       $rt .= "Logout</a></li>\n";
+    }
+We'll basically duplicate that (without the if conditional), changing a few
+things so that we add this right below it:
+    $rt .= menulistLI('exampleSection');
+    $rt .= "<a href=\"" . BASEURL . SCRIPT . "?mode=examplemode\">";
+    $rt .= "Example Section</a></li>\n";
+Viewing the site should now show "Example Section" right under "Logout" in
+the navigation area. Clicking "Example Section" should cause "exampleFunc1
+successfully called." to be displayed in the main content area of the site.
+<a name="WebCodeOverview-*Usingcontinuationswhensubmittingformdata*"></a>
+### *Using continuations when submitting form data*
+Let's modify examplefunc1 so that it prints some form data that gets
+submitted with a continuation.
+So, change the contents of examplefunc1 to be:
+    $options = array(0 => "option1",
+    		 1 => "option2");
+    print "<FORM action=\"" . BASEURL . SCRIPT . "\" method=post>\n";
+    print "Select one of these options:";
+    printSelectInput("theoption", $options);
+    $cont = addContinuationsEntry("submitFunc1Form", $options);
+    print "<INPUT type=hidden name=continuation value=\"$cont\">\n";
+    print "<INPUT type=submit value=Submit>\n";
+    print "</FORM>\n";
+Now, we have a form that gets displayed when "Example Section" is clicked. 
+Now, we need to add a function to process that form.  Add this function to
+    function submitFunc1Form() {
+       $data = getContinuationVar();
+       $theoption = processInputVar("theoption", ARG_NUMERIC);
+       if(! array_key_exists($theoption, $data)) {
+          print "invalid option submitted\n";
+          return;
+       }
+       print "The option you selected was: ";
+       print "{$data\[$theoption\]
+    }
+Next, we add this function to states.php:
+    $actions['mode']
+['submitFunc1Form'] = "submitFunc1Form";
+    $actions['pages']
+['submitFunc1Form'] = "exampleSection";
+Now, click the "Example Section" link, select one of the options, and click
+Submit to see if it works.
+<a name="WebCodeOverview-*UsingAJAXtodynamicallyupdateapage*"></a>
+### *Using AJAX to dynamically update a page*
+AJAX is very useful for dynamically updating page content.  Let's add
+something to examplefunc1 that can be updated with an AJAX call.
+Add this to the end of examplefunc1:
+    print "<br><br>\n";
+    print "<div id=examplediv>\n";
+    print "This will get dynamically changed.<br>\n";
+    print "</div>\n";
+    $cont = addContinuationsEntry("AJexample");
+    print "<a onclick=\"JS_AJexample('$cont');\">Click to ";
+    print "update</a><br>\n";
+Next, we need to add the javascript function we just referenced to code.js
+(in .ht-inc's parent directory) as well as a callback function that will be
+called when the results of the AJAX call are returned:
+    function JS_AJexample(contid) {
+       dojo.xhrPost({
+          url:'index.php',
+          load: JS_AJexampleCB,
+          error: errorHandler,
+          content: {continuation: contid},
+          timeout: 15000
+       });
+    }
+    function JS_AJexampleCB(data, ioArgs) {
+       eval(data);
+    }
+Then, we need to add a few things to states.php:
+to the $actions array:
+    $actions['mode']
+['AJexample'] = "exampleFunc2";
+    $actions['pages']
+['AJexample'] = "exampleSection";
+to the $noHTMLwrappers array:
+    'AJexample'
+Now, we need to create exampleFunc2 (in examples.php):
+    function exampleFunc2() {
+       print "document.getElementById('examplediv').innerHTML = 'Dynamic
+    }
+Then, we do something we haven't seen before.  getDojoHTML (in utils.php)
+must be modified so that the correct dojo requires get set when we are in
+mode examplefunc1.  For a simple AJAX update, we only need the dojo.parser
+module to be loaded.  Since this is already loaded for some of the Group
+modes, just add another case statement under submitDeleteGroup so we have:
+    case 'viewGroups':
+    case 'submitEditGroup':
+    case 'submitAddGroup':
+    case 'submitDeleteGroup':
+    case 'examplemode':
+       $dojoRequires = array('dojo.parser');
+       break;
+We also have to add a case statement a little further down where the HTML
+is actually generated. Find "case 'submitDeleteGroup':" in the switch
+statement following the one we just modified, and add another case
+statement for examplemode so we have:
+    case 'viewGroups':
+    case 'submitEditGroup':
+    case 'submitAddGroup':
+    case 'submitDeleteGroup':
+    case 'examplemode':
+       $rt .= "<style type=\"text/css\">\n";
+Since we modified code.js, to test this, you will need to hold Shift and
+click the reload button in your browser to force your browser to reload
+code.js.  "Click to update" will not show up as a normal link, but it
+should still be clickable.  You can use CSS to make it look like a normal
+link or give it an href="#"  if you want.
+<a name="WebCodeOverview-*Awordaboutusingtabsforcodeindentation*"></a>
+## *A word about using tabs for code indentation*
+All of the code has been indented using tabs (rather than spaces) except
+where code wraps to more than one line, in which case a combination of tabs
+and spaces are used.  This allows someone editing the code to set his tab
+width to whatever he prefers, but still allows code that wraps to multiple
+lines to line up correctly.  To do this, when you want to wrap code to
+another line, put a newline at the end of the one you are on just as you
+normally would.  Next, tab over from the beginning of the line until you
+are even with where the initial line of code started, then use spaces to
+line up this line with something in the line above it that makes sense. 
+I'll give an example.  Almost all of the queries are written on multiple
+lines like this:
+    =====------------------------------
+         $query = "SELECT id, "
+    	    .	     "name, "
+    	    .	     "prettyname "
+    	    . "FROM image "
+    	    . "WHERE id < 400 AND "
+    	    .	    "OSid = 3";
+    =====------------------------------
+In this example, whitespace indicated by the = marks should be made of
+tabs, whitespace indicated by the - marks should be made of spaces.
+<a name="WebCodeOverview-*InlineCodeDocumentation*"></a>
+## *Inline Code Documentation*
+Online documentation of the code is generated using Doxygen.  Each file
+that should be parsed to generate docs needs to have the following toward
+the top of it:
+    /**
+     * \file
+     */
+All functions should have a header above them that documents what it does. 
+The header should include the name of the function, a description of any
+parameters passed in, a description of anything returned, and a brief
+description of what the function does.	The format of the header is:
+    /////////////////////////////////////////////////////////////////////
+    ///
+    /// \fn nameOfFunction($param1, $param2)
+    ///
+    /// \param $param1 - description of param1
+    /// \param $param2 - description of param2
+    ///
+    /// \return description of datastructure returned
+    ///
+    /// \brief short description of what the function does
+    ///
+    /////////////////////////////////////////////////////////////////////
+If the function doesn't receive any parameters or doesn't return anything,
+those sections can be omitted.
+<a name="WebCodeOverview-*Authentication*"></a>
+## *Authentication*
+Authentication has been somewhat modularized.  When a user loads the site,
+the initGlobals function checks to see if the user is logged in.  If not,
+the mode in which the site should function gets set to "selectauth".  Here,
+the user is prompted to select an authentication method to use to log in. 
+Authentication methods are configured in $authMethods which is in conf.php.
+ There are currently three authentication types that can be handled:
+redirect, ldap, and local.  Redirect types send the user to another site to
+handle their authentication, with the expectation that a cookie will be set
+and that knowledge of how to interpret that cookie is handled in
+initGlobals.  If an ldap or local method is chosen, the user is directed to
+a page displayed by printLoginPageWithSkin() (in authentication.php as are
+most of the following functions).  This function sets up the skin for the
+page to match the affiliation defined in $authMethods, and then calls
+printLoginPage().  printLoginPage() displays a form for the user to enter a
+userid and password.  The form is submitted and then processed by
+submitLogin().	If the authentication method is ldap based, ldapLogin() is
+called; if it is the local system, localLogin() is called.  Each of these
+functions tries to validate the user.  If it succeeds, a cookie named
+VCLAUTH is set, and the user is redirected to the main page of the site. 
+If it fails, the user is redirected back to the login page. _If you enter
+valid credentials, but still get redirected back to the login page, the
+first thing to check is whether or not the setcookie() call was reached,
+and if so, was the cookie actually set in your browser._

View raw message