libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject svn commit: r1196915 - in /libcloud/site/trunk: content/docs/components.mdtext content/docs/compute-deployment.mdtext content/docs/using-libcloud-in-multi-threaded-and-async-environments.mdtext templates/blocks/menu_docs.html templates/documentation.html
Date Thu, 03 Nov 2011 01:39:37 GMT
Author: tomaz
Date: Thu Nov  3 01:39:36 2011
New Revision: 1196915

More docs.


Added: libcloud/site/trunk/content/docs/components.mdtext
--- libcloud/site/trunk/content/docs/components.mdtext (added)
+++ libcloud/site/trunk/content/docs/components.mdtext Thu Nov  3 01:39:36 2011
@@ -0,0 +1,23 @@
+title: Components
+## Components ##
+Libcloud is composed of multiple components, currently those are:
+* [Compute](/docs/compute-overview.html) - libcloud.compute.*
+* [Storage](/docs/storage-overview.html) -*
+* [Load balancers](/docs/loadbalancer-overview.html) - libcloud.loadbalancer.*
+Each component is relatively self-sustainable and exposes a simple and easy
+to use base API which works across all the supported providers.
+In places where it makes sense a component is also aware of other components.
+Good example of this is a `balancer_attach_node` method in the load balancer
+API. This method takes `Node` object from the compute API as an argument
+and automatically reads its first public IP address before adding a member
+to the load balancer.
+Besides the Base API some drivers also implement driver-specific or so called
+extension methods and arguments. Those methods implement functionality which
+is specific to a provider or a subset of providers. They can be easily recognized
+because they are prefixed with an `ex_`.

Modified: libcloud/site/trunk/content/docs/compute-deployment.mdtext
--- libcloud/site/trunk/content/docs/compute-deployment.mdtext (original)
+++ libcloud/site/trunk/content/docs/compute-deployment.mdtext Thu Nov  3 01:39:36 2011
@@ -76,6 +76,7 @@ tool so it shouldn't be used as such.
     # deploy_node takes the same base keyword arguments as create_node.
     node = conn.deploy_node(name='test', image=image, size=size, deploy=script,
-                            ssh_key=key_path, ex_keyname=key_name)
+                            ssh_username='ubuntu', ssh_key=key_path,
+                            ex_keyname=key_name)
     # <Node: uuid=..., name=test, state=3, public_ip=[''], provider=EC2 ...>
     # the node is now booted, with puppet installed.

Added: libcloud/site/trunk/content/docs/using-libcloud-in-multi-threaded-and-async-environments.mdtext
--- libcloud/site/trunk/content/docs/using-libcloud-in-multi-threaded-and-async-environments.mdtext
+++ libcloud/site/trunk/content/docs/using-libcloud-in-multi-threaded-and-async-environments.mdtext
Thu Nov  3 01:39:36 2011
@@ -0,0 +1,70 @@
+title: Using Libcloud in multi-threaded and async environments
+## Using Libcloud in multi-threaded and async environments ##
+Libcloud's primary task is to communicate with different provider APIs using HTTP.
+This means most of the work is not CPU intensive, but performing all those HTTP
+requests includes a lot of waiting which makes the library I/O bound.
+Most of the time you want to perform more operations in parallel or just want
+your code to finish faster (for example starting a lot of servers or periodically
+polling for node status).
+Problems like this are usually solved using threads or async libraries such as
+Twisted, Tornado or gevent.
+This page contains some information and tips about how to use Libcloud in
+such environments.
+### Libcloud and thread-safety ###
+Important thing to keep in mind when dealing with threads is
+thread-safety. Libcloud driver instance is **not** thread safe. This means if
+you don't want to deal with complex (and usually inefficient) locking the
+easiest solution is to create a new driver instance inside each thread.
+### Using Libcloud with gevent ###
+gevent has an ability to monkey patch and replace functions in the Python `socket`
+and `time` module with its own functions which don't block.
+The only thing you need to use to make Libcloud efficiently work with gevent is
+to enable this monkey-patching:
+    ::python
+    from gevent import monkey
+    monkey.patch_all()
+### Using Libcloud with Twisted ###
+Libcloud has no Twisted support included in the core which means you need
+to be careful when you use it with Twisted and some other async frameworks.
+If you don't use it properly it can block the whole reactor (similar as any
+other blocking library or a long CPU-intensive task) which means the execution 
+of other pending tasks in the event queue will be blocked.
+A simple solution to prevent blocking the reactor is to run Libcloud calls
+inside a thread. In Twisted this can be achieved using `threads.deferToThread`
+which runs a provided method inside the Twisted thread pool.
+The example bellow demonstrates how to create a new node inside a thread
+without blocking the whole reactor.
+    ::python
+    from twisted.internet import defer, threads, reactor
+    from libcloud.compute.types import Provider
+    from libcloud.compute.providers import get_driver
+    @defer.inlineCallbacks
+    def create_node():
+        node = yield threads.deferToThread(_thread_create_node,
+                                           name=name)
+    def _thread_create_node(name):
+        Driver = get_driver(Provider.RACKSPACE)
+        conn = Driver('username', 'api key')
+        image = conn.list_images()[0]
+        size = conn.list_sizes()[0]
+        node = conn.create_node(name=name, image=image, size=size)
+        return node

Modified: libcloud/site/trunk/templates/blocks/menu_docs.html
--- libcloud/site/trunk/templates/blocks/menu_docs.html (original)
+++ libcloud/site/trunk/templates/blocks/menu_docs.html Thu Nov  3 01:39:36 2011
@@ -1,7 +1,7 @@
 <ul id="menu">
     <li><a href="/index.html">Website home</a></li>
     <li><a href="/docs/index.html">Documentation home</a></li>
-    <li>Components</li>
+    <li><a href="/docs/components.html">Components</a></li>
         <li class="title">Compute</li>
@@ -16,6 +16,7 @@
             <li><a href="debugging.html">Debugging</a></li>
+            <li><a href="using-libcloud-in-multi-threaded-and-async-environments.html">Using
Libcloud in multi-threaded and async environments</a></li>
     <li><a href="/apidocs">Generated API docs</a></li>

Modified: libcloud/site/trunk/templates/documentation.html
--- libcloud/site/trunk/templates/documentation.html (original)
+++ libcloud/site/trunk/templates/documentation.html Thu Nov  3 01:39:36 2011
@@ -1,7 +1,7 @@
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "">
-  <title>Apache Libcloud Python library - {% block title %}{{ headers.title }}{% endblock
+  <title>{% block title %}{{ headers.title }}{% endblock %} | Apache Libcloud Documentation</title>
   <meta http-equiv="Content-Type" content="text/html;charset=utf-8">
   <script src="" type="text/javascript"></script>
   <script src="" type="text/javascript"></script>
@@ -16,7 +16,7 @@
   <div class="container">
     <div id="header" class="span-24 last">
-        {% block header %}
+      {% block header %}
       <h1>Apache Libcloud <span>a unified interface to the cloud</span></h1>
       {% endblock %}

View raw message