trafficserver-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jpe...@apache.org
Subject [17/51] trafficserver git commit: Documentation reorganization
Date Tue, 03 Nov 2015 06:09:53 GMT
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-headers/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-headers/index.en.rst b/doc/developer-guide/plugins/http-headers/index.en.rst
new file mode 100644
index 0000000..05b0adb
--- /dev/null
+++ b/doc/developer-guide/plugins/http-headers/index.en.rst
@@ -0,0 +1,139 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-headers:
+
+HTTP Headers
+************
+
+.. toctree::
+   :maxdepth: 2
+
+   trafficserver-http-header-system.en
+   header-functions.en
+   mime-headers.en
+   marshal-buffers.en
+   urls.en
+
+An **HTTP message** consists of the following:
+
+-  HTTP header
+-  body
+-  trailer
+
+The **HTTP header** consists of:
+
+-  A request or response line
+
+   -  An HTTP **request line** contains a method, URL, and version
+   -  A **response line** contains a version, status code, and reason
+      phrase
+
+-  A MIME header
+
+A **MIME header** is comprised of zero or more MIME fields. A **MIME
+field** is composed of a field name, a colon, and (zero or more) field
+values. The values in a field are separated by commas. An HTTP header
+containing a request line is usually referred to as a **request**. The
+following example shows a typical request header:
+
+.. code-block:: http
+
+   GET http://www.tiggerwigger.com/ HTTP/1.0
+   Proxy-Connection: Keep-Alive
+   User-Agent: Mozilla/5.0 [en] (X11; I; Linux 2.2.3 i686)
+   Host: www.tiggerwigger.com
+   Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*
+   Accept-Encoding: gzip
+   Accept-Language: en
+   Accept-Charset: iso-8859-1, *, utf-8
+
+The **response header** for the above request might look like the
+following:
+
+.. code-block:: http
+
+   HTTP/1.0 200 OK
+   Date: Fri, 13 Nov 2009 06:57:43 GMT
+   Content-Location: http://locutus.tiggerwigger.com/index.html
+   Etag: "07db14afa76be1:1074"
+   Last-Modified: Thu, 05 Nov 2009 20:01:38 GMT
+   Content-Length: 7931
+   Content-Type: text/html
+   Server: Microsoft-IIS/4.0
+   Age: 922
+   Proxy-Connection: close
+
+The following figure illustrates an HTTP message with an expanded HTTP
+header.
+
+**Figure 10.1. HTTP Request/Response and Header Structure**
+
+.. figure:: /static/images/sdk/http_header_struct.jpg
+   :alt: HTTP Request/Response and Header Structure
+
+   HTTP Request/Response and Header Structure
+
+The figure below shows example HTTP request and response headers.
+
+**Figure 10.2. Examples of HTTP Request and Response Headers**
+
+.. figure:: /static/images/sdk/http_headers.jpg
+   :alt: Examples of HTTP Request and Response Headers
+
+   Examples of HTTP Request and Response Headers
+
+The marshal buffer or ``TSMBuffer`` is a heap data structure that stores
+parsed URLs, MIME headers, and HTTP headers. You can allocate new
+objects out of marshal buffers and change the values within the marshal
+buffer. Whenever you manipulate an object, you must require the handle
+to the object (``TSMLoc``) and the marshal buffer containing the object
+(``TSMBuffer``).
+
+**Figure 10.3. Marshal Buffers and Header Locations**
+
+.. figure:: /static/images/sdk/marshall_buffers.jpg
+   :alt: Marshal Buffers and Header Locations
+
+   Marshal Buffers and Header Locations
+
+The figure above shows the following:
+
+-  The marshal buffer containing the HTTP request, ``reqest_bufp``
+
+-  ``TSMLoc`` location pointer for the HTTP header (``http_hdr_loc``)
+
+-  ``TSMLoc`` location pointer for the request URL (``url_loc``)
+
+-  ``TSMLoc`` location pointers for the MIME header (``mime_hdr_loc``)
+
+-  ``TSMLoc`` location pointers for MIME fields (``fieldi_loc``)
+
+-  ``TSMLoc`` location pointer for the next duplicate MIME field
+   (``next_dup_loc``)
+
+The diagram also shows that an HTTP header contains pointers to the URL
+location and the MIME header location. You can obtain the URL location
+from an HTTP header using the function ``TSHttpHdrUrlGet``. To work with
+MIME headers, you can pass either a MIME header location or an HTTP
+header location to MIME header functions . If you pass an HTTP header to
+a MIME header function, then the system locates the associated MIME
+header and executes the MIME header function on the MIME header
+location.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-headers/marshal-buffers.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-headers/marshal-buffers.en.rst b/doc/developer-guide/plugins/http-headers/marshal-buffers.en.rst
new file mode 100644
index 0000000..99fafad
--- /dev/null
+++ b/doc/developer-guide/plugins/http-headers/marshal-buffers.en.rst
@@ -0,0 +1,55 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-headers-marshal-buffers:
+
+Marshal Buffers
+***************
+
+A *marshal buffer*, or ``TSMBuffer``, is a heap data structure that
+stores parsed URLs, MIME headers, and HTTP headers. You can allocate new
+objects out of marshal buffers and change the values within a marshal
+buffer. Whenever you manipulate an object, you require the handle to the
+object (``TSMLoc``) and the marshal buffer containing the object
+(``TSMBuffer``).
+
+Routines exist for manipulating the object based on these two pieces of
+information. For examples, see one of the following:
+
+-  :ref:`developer-plugins-http-headers`
+-  :ref:`developer-plugins-http-headers-urls`
+-  :ref:`developer-plugins-http-headers-mime-headers`
+
+The **marshal buffer functions** enable you to create and destroy
+Traffic Server's marshal buffers, which are the data structures that
+hold parsed URLs, MIME headers, and HTTP headers.
+
+.. caution::
+   Any marshal buffer fetched by ``TSHttpTxn*Get`` will be used by other
+   parts of the system. Be careful not to destroy these shared transaction
+   marshal buffers in functions such as those below:
+
+-  `TSHttpTxnCachedReqGet <http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a889b626142157077f4f3cfe479e8b8e2>`_
+-  `TSHttpTxnCachedRespGet <http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#ae8f24b8dabb5008ad11620a11682ffd6>`_
+-  `TSHttpTxnClientReqGet <http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#acca66f22d0f87bf8f08478ed926006a5>`_
+-  `TSHttpTxnClientRespGet <http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a92349c8363f72b1f6dfed3ae80901fff>`_
+-  `TSHttpTxnServerReqGet <http://people.apache.org/~amc/ats/doc/html/ts_8h.html#aac2343a8b47bf9150f3ff7cd4e692d57>`_
+-  `TSHttpTxnServerRespGet <http://people.apache.org/~amc/ats/doc/html/ts_8h.html#a39e8bfb199eadabb54c067ff25a9a400>`_
+-  `TSHttpTxnTransformRespGet <http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a20367f5469e8b7e73621c1316091d578>`_
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-headers/mime-headers.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-headers/mime-headers.en.rst b/doc/developer-guide/plugins/http-headers/mime-headers.en.rst
new file mode 100644
index 0000000..3787858
--- /dev/null
+++ b/doc/developer-guide/plugins/http-headers/mime-headers.en.rst
@@ -0,0 +1,441 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-headers-mime-headers:
+
+MIME Headers
+************
+
+The Traffic Server \*\*MIME header functions \*\* enable you to retrieve
+and modify information about HTTP MIME fields.
+
+An HTTP request or response consists of a header, body, and trailer. The
+**HTTP** **header** contains a request (or response) line and a MIME
+header. A **MIME** **header** is composed of zero or more MIME fields. A
+**MIME** **field** is composed of a field name, a colon, and zero or
+more field values (values in a field are separated by commas).
+
+In the example below: ``Foo`` is the MIME field name, ``bar`` is the
+first MIME field value, and ``car`` is the second MIME field value.
+
+::
+
+      Foo: bar, car
+
+The following example is an augmented **Backus-Naur Form** (BNF) for the
+form of a MIME header - it specifies exactly what was described above. A
+**header** consists of zero or more **fields** that contain a name,
+separating colon, and zero or more values. A **name** or **value** is
+simply a string of tokens that is potentially zero length; a **token**
+is any character except certain control characters and separators (such
+as colons). For the purpose of retrieving a field, field names are not
+case-sensitive; therefore, the field names ``Foo``, ``foo`` and ``fOO``
+are all equivalent.
+
+::
+
+    MIME-header = *MIME-field
+    MIME-field = field-name ":" #field-value
+    field-name = *token
+    field-value = *token
+
+The MIME header data structure is a parsed version of a standard
+Internet MIME header. The MIME header data structure is similar to the
+URL data structure (see :doc:`URLs <urls.en>`). The actual data is stored in a
+marshal buffer; the MIME header functions operate on a marshal buffer
+and a location (``TSMLoc``) within the buffer.
+
+After a call to ``TSMimeHdrFieldDestroy``, ``TSMimeHdrFieldRemove``, or
+``TSUrlDestroy`` is made, you must deallocate the ``TSMLoc`` handle with
+a call to ``TSHandleMLocRelease``. You do not need to deallocate a
+``NULL`` handles. For example: if you call
+``TSMimeHdrFieldValueStringGet`` to get the value of the content type
+field and the field does not exist, then it returns ``TS_NULL_MLOC``. In
+such a case, you wouldn't need to deallocate the handle with a call to
+``TSHandleMLocRelease``.
+
+The location (``TSMLoc``) in the :ref:`MIME header
+functions <MimeHeaderFxns>` can be either an HTTP header location or
+a MIME header location. If an HTTP header location is passed to these
+functions, then the system locates the MIME header associated with that
+HTTP header and executes the corresponding MIME header operations
+specified by the functions (see the example in the description of
+:c:func:`TSMimeHdrCopy`).
+
+**Note:** MIME headers may contain more than one MIME field with the
+same name. Previous versions of Traffic Server joined multiple fields
+with the same name into one field with composite values, but this
+behavior came at a performance cost and caused compatability issues with
+older clients and servers. Hence, the current version of Traffic Server
+does not coalesce duplicate fields. Correctly-behaving plugins should
+check for the presence of duplicate fields and iterate over the
+duplicate fields by using ``TSMimeHdrFieldNextDup``.
+
+To facilitate fast comparisons and reduce storage size, Traffic Server
+defines several pre-allocated field names. These field names correspond
+to the field names in HTTP and NNTP headers.
+
+``TS_MIME_FIELD_ACCEPT``
+    "Accept"
+    ``TS_MIME_LEN_ACCEPT``
+
+``TS_MIME_FIELD_ACCEPT_CHARSET``
+    "Accept-Charset"
+    ``TS_MIME_LEN_ACCEPT_CHARSET``
+
+``TS_MIME_FIELD_ACCEPT_ENCODING``
+    "Accept-Encoding"
+    ``TS_MIME_LEN_ACCEPT_ENCODING``
+
+``TS_MIME_FIELD_ACCEPT_LANGUAGE``
+    "Accept-Language"
+    ``TS_MIME_LEN_ACCEPT_LANGUAGE``
+
+``TS_MIME_FIELD_ACCEPT_RANGES``
+    "Accept-Ranges"
+    ``TS_MIME_LEN_ACCEPT_RANGES``
+
+``TS_MIME_FIELD_AGE``
+    "Age"
+    ``TS_MIME_LEN_AGE``
+
+``TS_MIME_FIELD_ALLOW``
+    "Allow"
+    ``TS_MIME_LEN_ALLOW``
+
+``TS_MIME_FIELD_APPROVED``
+    "Approved"
+    ``TS_MIME_LEN_APPROVED``
+
+``TS_MIME_FIELD_AUTHORIZATION``
+    "Authorization"
+    ``TS_MIME_LEN_AUTHORIZATION``
+
+``TS_MIME_FIELD_BYTES``
+    "Bytes"
+    ``TS_MIME_LEN_BYTES``
+
+``TS_MIME_FIELD_CACHE_CONTROL``
+    "Cache-Control"
+    ``TS_MIME_LEN_CACHE_CONTROL``
+
+``TS_MIME_FIELD_CLIENT_IP``
+    "Client-ip"
+    ``TS_MIME_LEN_CLIENT_IP``
+
+``TS_MIME_FIELD_CONNECTION``
+    "Connection"
+    ``TS_MIME_LEN_CONNECTION``
+
+``TS_MIME_FIELD_CONTENT_BASE``
+    "Content-Base"
+    ``TS_MIME_LEN_CONTENT_BASE``
+
+``TS_MIME_FIELD_CONTENT_ENCODING``
+    "Content-Encoding"
+    ``TS_MIME_LEN_CONTENT_ENCODING``
+
+``TS_MIME_FIELD_CONTENT_LANGUAGE``
+    "Content-Language"
+    ``TS_MIME_LEN_CONTENT_LANGUAGE``
+
+``TS_MIME_FIELD_CONTENT_LENGTH``
+    "Content-Length"
+    ``TS_MIME_LEN_CONTENT_LENGTH``
+
+``TS_MIME_FIELD_CONTENT_LOCATION``
+    "Content-Location"
+    ``TS_MIME_LEN_CONTENT_LOCATION``
+
+``TS_MIME_FIELD_CONTENT_MD5``
+    "Content-MD5"
+    ``TS_MIME_LEN_CONTENT_MD5``
+
+``TS_MIME_FIELD_CONTENT_RANGE``
+    "Content-Range"
+    ``TS_MIME_LEN_CONTENT_RANGE``
+
+``TS_MIME_FIELD_CONTENT_TYPE``
+    "Content-Type"
+    ``TS_MIME_LEN_CONTENT_TYPE``
+
+``TS_MIME_FIELD_CONTROL``
+    "Control"
+    ``TS_MIME_LEN_CONTROL``
+
+``TS_MIME_FIELD_COOKIE``
+    "Cookie"
+    ``TS_MIME_LEN_COOKIE``
+
+``TS_MIME_FIELD_DATE``
+    "Date"
+    ``TS_MIME_LEN_DATE``
+
+``TS_MIME_FIELD_DISTRIBUTION``
+    "Distribution"
+    ``TS_MIME_LEN_DISTRIBUTION``
+
+``TS_MIME_FIELD_ETAG``
+    "Etag"
+    ``TS_MIME_LEN_ETAG``
+
+``TS_MIME_FIELD_EXPECT``
+    "Expect"
+    ``TS_MIME_LEN_EXPECT``
+
+``TS_MIME_FIELD_EXPIRES``
+    "Expires"
+    ``TS_MIME_LEN_EXPIRES``
+
+``TS_MIME_FIELD_FOLLOWUP_TO``
+    "Followup-To"
+    ``TS_MIME_LEN_FOLLOWUP_TO``
+
+``TS_MIME_FIELD_FROM``
+    "From"
+    ``TS_MIME_LEN_FROM``
+
+``TS_MIME_FIELD_HOST``
+    "Host"
+    ``TS_MIME_LEN_HOST``
+
+``TS_MIME_FIELD_IF_MATCH``
+    "If-Match"
+    ``TS_MIME_LEN_IF_MATCH``
+
+``TS_MIME_FIELD_IF_MODIFIED_SINCE``
+    "If-Modified-Since"
+    ``TS_MIME_LEN_IF_MODIFIED_SINCE``
+
+``TS_MIME_FIELD_IF_NONE_MATCH``
+    "If-None-Match"
+    ``TS_MIME_LEN_IF_NONE_MATCH``
+
+``TS_MIME_FIELD_IF_RANGE``
+    "If-Range"
+    ``TS_MIME_LEN_IF_RANGE``
+
+``TS_MIME_FIELD_IF_UNMODIFIED_SINCE``
+    "If-Unmodified-Since"
+    ``TS_MIME_LEN_IF_UNMODIFIED_SINCE``
+
+``TS_MIME_FIELD_KEEP_ALIVE``
+    "Keep-Alive"
+    ``TS_MIME_LEN_KEEP_ALIVE``
+
+``TS_MIME_FIELD_KEYWORDS``
+    "Keywords"
+    ``TS_MIME_LEN_KEYWORDS``
+
+``TS_MIME_FIELD_LAST_MODIFIED``
+    "Last-Modified"
+    ``TS_MIME_LEN_LAST_MODIFIED``
+
+``TS_MIME_FIELD_LINES``
+    "Lines"
+    ``TS_MIME_LEN_LINES``
+
+``TS_MIME_FIELD_LOCATION``
+    "Location"
+    ``TS_MIME_LEN_LOCATION``
+
+``TS_MIME_FIELD_MAX_FORWARDS``
+    "Max-Forwards"
+    ``TS_MIME_LEN_MAX_FORWARDS``
+
+``TS_MIME_FIELD_MESSAGE_ID``
+    "Message-ID"
+    ``TS_MIME_LEN_MESSAGE_ID``
+
+``TS_MIME_FIELD_NEWSGROUPS``
+    "Newsgroups"
+    ``TS_MIME_LEN_NEWSGROUPS``
+
+``TS_MIME_FIELD_ORGANIZATION``
+    "Organization"
+    ``TS_MIME_LEN_ORGANIZATION``
+
+``TS_MIME_FIELD_PATH``
+    "Path"
+    ``TS_MIME_LEN_PATH``
+
+``TS_MIME_FIELD_PRAGMA``
+    "Pragma"
+    ``TS_MIME_LEN_PRAGMA``
+
+``TS_MIME_FIELD_PROXY_AUTHENTICATE``
+    "Proxy-Authenticate"
+    ``TS_MIME_LEN_PROXY_AUTHENTICATE``
+
+``TS_MIME_FIELD_PROXY_AUTHORIZATION``
+    "Proxy-Authorization"
+    ``TS_MIME_LEN_PROXY_AUTHORIZATION``
+
+``TS_MIME_FIELD_PROXY_CONNECTION``
+    "Proxy-Connection"
+    ``TS_MIME_LEN_PROXY_CONNECTION``
+
+``TS_MIME_FIELD_PUBLIC``
+    "Public"
+    ``TS_MIME_LEN_PUBLIC``
+
+``TS_MIME_FIELD_RANGE``
+    "Range"
+    ``TS_MIME_LEN_RANGE``
+
+``TS_MIME_FIELD_REFERENCES``
+    "References"
+    ``TS_MIME_LEN_REFERENCES``
+
+``TS_MIME_FIELD_REFERER``
+    "Referer"
+    ``TS_MIME_LEN_REFERER``
+
+``TS_MIME_FIELD_REPLY_TO``
+    "Reply-To"
+    ``TS_MIME_LEN_REPLY_TO``
+
+``TS_MIME_FIELD_RETRY_AFTER``
+    "Retry-After"
+    ``TS_MIME_LEN_RETRY_AFTER``
+
+``TS_MIME_FIELD_SENDER``
+    "Sender"
+    ``TS_MIME_LEN_SENDER``
+
+``TS_MIME_FIELD_SERVER``
+    "Server"
+    ``TS_MIME_LEN_SERVER``
+
+``TS_MIME_FIELD_SET_COOKIE``
+    "Set-Cookie"
+    ``TS_MIME_LEN_SET_COOKIE``
+
+``TS_MIME_FIELD_SUBJECT``
+    "Subject"
+    ``TS_MIME_LEN_SUBJECTTS_MIME_LEN_SUBJECT``
+
+``TS_MIME_FIELD_SUMMARY``
+    "Summary"
+    ``TS_MIME_LEN_SUMMARY``
+
+``TS_MIME_FIELD_TE``
+    "TE"
+    ``TS_MIME_LEN_TE``
+
+``TS_MIME_FIELD_TRANSFER_ENCODING``
+    "Transfer-Encoding"
+    ``TS_MIME_LEN_TRANSFER_ENCODING``
+
+``TS_MIME_FIELD_UPGRADE``
+    "Upgrade"
+    ``TS_MIME_LEN_UPGRADE``
+
+``TS_MIME_FIELD_USER_AGENT``
+    "User-Agent"
+    ``TS_MIME_LEN_USER_AGENT``
+
+``TS_MIME_FIELD_VARY``
+    "Vary"
+    ``TS_MIME_LEN_VARY``
+
+``TS_MIME_FIELD_VIA``
+    "Via"
+    ``TS_MIME_LEN_VIA``
+
+``TS_MIME_FIELD_WARNING``
+    "Warning"
+    ``TS_MIME_LEN_WARNING``
+
+``TS_MIME_FIELD_WWW_AUTHENTICATE``
+    "Www-Authenticate"
+    ``TS_MIME_LEN_WWW_AUTHENTICATE``
+
+``TS_MIME_FIELD_XREF``
+    "Xref"
+    ``TS_MIME_LEN_XREF``
+
+The header field names above are defined in ``ts.h`` as ``const char*``
+strings. When Traffic Server sets the name portion of a header field (or
+any portion for that matter), it quickly checks to see if the new value
+is one of the known values. If it is, then Traffic Server stores a
+pointer into a global table instead of storing the known value in the
+marshal buffer. The header field names listed above are also pointers
+into this table, which enables simple pointer comparison of the value
+returned from ``TSMimeHdrFieldNameGet`` with one of the values listed
+above. It is recommended that you use the above values when referring to
+one of the known header field names to avoid the possibility of a
+spelling error.
+
+Traffic Server adds one important feature to MIME fields that you may
+not know about: Traffic Server does not print a MIME field if the field
+name begins with the '``@``\ ' symbol. For example: a plugin can add the
+field "``@My-Field``\ " to a header. Even though Traffic Server never
+sends that field out in a request to an origin server or in a response
+to a client, it can be printed to Traffic Server logs by defining a
+custom log configuration file that explicitly logs such fields. This
+provides a useful mechanism for plugins to store information about an
+object in one of the MIME headers associated with the object.
+
+.. _MimeHeaderFxns:
+
+The MIME header functions are listed below:
+
+-  :c:func:`TSMimeHdrFieldAppend`
+-  :c:func:`TSMimeHdrFieldClone`
+-  :c:func:`TSMimeHdrFieldCopy`
+-  :c:func:`TSMimeHdrFieldCopyValues`
+-  :c:func:`TSMimeHdrFieldCreate`
+-  :c:func:`TSMimeHdrFieldDestroy`
+-  :c:func:`TSMimeHdrFieldLengthGet`
+-  :c:func:`TSMimeHdrFieldNameGet`
+-  :c:func:`TSMimeHdrFieldNameSet`
+-  :c:func:`TSMimeHdrFieldNext`
+-  :c:func:`TSMimeHdrFieldNextDup`
+-  :c:func:`TSMimeHdrFieldValueAppend`
+-  :c:func:`TSMimeHdrFieldValueAppend`
+-  :c:func:`TSMimeHdrFieldValueDateGet`
+-  :c:func:`TSMimeHdrFieldValueDateInsert`
+-  :c:func:`TSMimeHdrFieldValueDateSet`
+-  :c:func:`TSMimeHdrFieldValueIntGet`
+-  :c:func:`TSMimeHdrFieldValueIntSet`
+-  :c:func:`TSMimeHdrFieldValueStringGet`
+-  :c:func:`TSMimeHdrFieldValueStringInsert`
+-  :c:func:`TSMimeHdrFieldValueStringSet`
+-  :c:func:`TSMimeHdrFieldValueUintGet`
+-  :c:func:`TSMimeHdrFieldValueUintInsert`
+-  :c:func:`TSMimeHdrFieldValueUintSet`
+-  :c:func:`TSMimeHdrFieldValuesClear`
+-  :c:func:`TSMimeHdrFieldValuesCount`
+-  :c:func:`TSMimeHdrClone`
+-  :c:func:`TSMimeHdrCopy`
+-  :c:func:`TSMimeHdrCreate`
+-  :c:func:`TSMimeHdrDestroy`
+-  :c:func:`TSMimeHdrFieldFind`
+-  :c:func:`TSMimeHdrFieldGet`
+-  :c:func:`TSMimeHdrFieldRemove`
+-  :c:func:`TSMimeHdrFieldsClear`
+-  :c:func:`TSMimeHdrFieldsCount`
+-  :c:func:`TSMimeHdrLengthGet`
+-  :c:func:`TSMimeHdrParse`
+-  :c:func:`TSMimeParserClear`
+-  :c:func:`TSMimeParserCreate`
+-  :c:func:`TSMimeParserDestroy`
+-  :c:func:`TSMimeHdrPrint`
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-headers/trafficserver-http-header-system.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-headers/trafficserver-http-header-system.en.rst b/doc/developer-guide/plugins/http-headers/trafficserver-http-header-system.en.rst
new file mode 100644
index 0000000..fa84861
--- /dev/null
+++ b/doc/developer-guide/plugins/http-headers/trafficserver-http-header-system.en.rst
@@ -0,0 +1,185 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-headers-system:
+
+Traffic Server HTTP Header System
+*********************************
+
+.. toctree::
+   :maxdepth: 2
+
+No Null-Terminated Strings
+==========================
+
+It's not safe to assume that string data contained in marshal buffers
+(such as URLs and MIME fields) is stored in null-terminated string
+copies. Therefore, your plugins should always use the length parameter
+when retrieving or manipulating these strings. You **cannot** pass in
+``NULL`` for string-length return values; string values returned from
+marshall buffers are not null-terminated. If you need a null-terminated
+value, then use ``TSstrndup`` to automatically null-terminate a string.
+The strings that come back and are not null-terminated **cannot** be
+passed into the common ``str*()`` routines
+
+.. note::
+   Values returned from a marshall buffer can be ``NULL``, which means the
+   field or object requested does not exist.
+
+For example (from the ``blacklist-1`` sample)
+
+.. code-block:: c
+
+   char *host_string;
+   int host_length;
+   host_string = TSUrlHostGet (bufp, url_loc, &host_length);
+   for (i = 0; i < nsites; i++) {
+   if (strncmp (host_string, sites[i], host_length) == 0) {
+      // ...
+   }
+
+See the sample plugins for additional examples.
+
+Duplicate MIME Fields Are Not Coalesced
+=======================================
+
+MIME headers can contain more than one MIME field with the same name.
+Earlier versions of Traffic Server joined multiple fields with the same
+name into one field with composite values. This behavior came at a
+performance cost and caused interoperability problems with older clients
+and servers. Therefore, this version of Traffic Server does not coalesce
+duplicate fields.
+
+Properly-behaving plugins should check for the presence of duplicate
+fields and then iterate over the duplicate fields via
+:c:func:`TSMimeHdrFieldNextDup`.
+
+MIME Fields Always Belong to an Associated MIME Header
+======================================================
+
+When using Traffic Server, you cannot create a new MIME field without an
+associated MIME header or HTTP header; MIME fields are always seen as
+part of a MIME header or HTTP header.
+
+To use a MIME field, you must specify the MIME header or HTTP header to
+which it belongs - this is called the field's **parent header**. The
+``TSMimeField*`` functions in older versions of the SDK have been
+deprecated, as they do not require the parent header as inputs. The
+current version of Traffic Server uses new functions, the
+**``TSMimeHdrField``** series, which require you to specify the location
+of the parent header along with the location of the MIME field. For
+every deprecated *``TSMimeField``* function, there is a new, preferred
+``TSMimeHdrField*`` function. Therefore, you should use the
+**``TSMimeHdrField``** functions instead of the deprecated
+*``TSMimeField``* series. Examples are provided below.
+
+Instead of:
+
+.. code-block:: c
+
+    TSMLoc TSMimeFieldCreate (TSMBuffer bufp)
+
+You should use:
+
+.. code-block:: c
+
+    TSMLoc TSMimeHdrFieldCreate (TSMBuffer bufp, TSMLoc hdr)
+
+Instead of:
+
+.. code-block:: c
+
+    void TSMimeFieldCopyValues (TSMBuffer dest_bufp, TSMLoc dest_offset,
+       TSMBuffer src_bufp, TSMLoc src_offset)
+
+You should use:
+
+.. code-block:: c
+
+    void TSMimeHdrFieldCopyValues (TSMBuffer dest_bufp, TSMLoc dest_hdr,
+       TSMLoc dest_field, TSMBuffer src_bufp, TSMLoc src_hdr, TSMLoc
+       src_field)
+
+In the ``TSMimeHdrField*`` function prototypes, the ``TSMLoc`` field
+corresponds to the ``TSMLoc`` offset used the deprecated
+``TSMimeField*`` functions (see the discussion of parent ``TSMLoc`` in
+the following section).
+
+Release Marshal Buffer Handles
+==============================
+
+When you fetch a component object or create a new object, you get back a
+handle to the object location. The handle is either an ``TSMLoc`` for an
+object location or ``char *`` for a string location. You can manipulate
+the object through these handles, but when you are finished you need to
+release the handle to free up system resources.
+
+The general guideline is to release all ``TSMLoc`` and string handles
+you retrieve. The one exception is the string returned by
+``TSUrlStringGet``, which must be freed by a call to ``TSfree``.
+
+The handle release functions expect three arguments: the marshal buffer
+containing the data, the location of the parent object, and the location
+of the object to be released. The parent location is usually clear from
+the creation of the ``TSMLoc`` or string. For example, if your plugin
+had the following calls:
+
+.. code-block:: c
+
+   url_loc = TSHttpHdrUrlGet (bufp, hdr_loc);
+   host_string = TSUrlHostGet (bufp, url_loc, &host_length);
+
+then your plugin would have to call:
+
+.. code-block:: c
+
+   TSHandleMLocRelease (bufp, hdr_loc, url_loc);
+
+If an ``TSMLoc`` is obtained from a transaction, then it does not have a
+parent ``TSMLoc``. Use the null ``TSMLoc`` constant ``TS_NULL_MLOC`` as
+its parent. For example, if your plugin calls:
+
+.. code-block:: c
+
+   TSHttpTxnClientReqGet (txnp, &bufp, &hdr_loc);
+
+then you must release ``hdr_loc`` with:
+
+.. code-block:: c
+
+   TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);
+
+You need to use ``TS_NULL_MLOC`` to release any ``TSMLoc`` handles
+retrieved by the ``TSHttpTxn*Get`` functions.
+
+Here's an example using a new ``TSMimeHdrField`` function:
+
+.. code-block:: c
+
+   TSHttpTxnServerRespGet( txnp, &resp_bufp, &resp_hdr_loc );
+   new_field_loc = TSMimeHdrFieldCreate (resp_bufp, resp_hdr_loc);
+   TSHandleMLocRelease ( resp_bufp, resp_hdr_loc, new_field_loc);
+   TSHandleMLocRelease ( resp_bufp, TS_NULL_MLOC, resp_hdr_loc);
+
+See the sample plugins for many more examples.
+
+.. tip::
+
+   You should release handles before reenabling the HTTP transaction.
+   In other words, call ``TSHandleMLocRelease`` before ``TSHttpTxnReenable``.

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-headers/urls.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-headers/urls.en.rst b/doc/developer-guide/plugins/http-headers/urls.en.rst
new file mode 100644
index 0000000..3d06633
--- /dev/null
+++ b/doc/developer-guide/plugins/http-headers/urls.en.rst
@@ -0,0 +1,136 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-headers-urls:
+
+URLs
+****
+
+API URL functions provide access to URL data stored in marshal buffers.
+The URL functions can create, copy, retrieve or delete entire URLs; they
+can also retrieve or modify parts of URLs, such as port or scheme
+information.
+
+The general form of an Internet URL is:
+
+::
+
+       scheme://user:password@host:port/stuff
+
+The URL data structure includes support for two specific types of
+internet URLs. HTTP URLs have the form:
+
+::
+
+       http://user:password@host:port/path;params?query#fragment
+
+The URL port is stored as integer. All remaining parts of the URL
+(scheme, user, etc.) are stored as strings. Traffic Server URL functions
+are named according to the portion of the URL on which they operate. For
+instance, the function that retrieves the host portion of a URL is named
+``TSUrlHostGet``.
+
+To facilitate fast comparisons and reduce storage size, Traffic Server
+defines several preallocated scheme names.
+
+``TS_URL_SCHEME_FILE``
+    "file"
+    ``TS_URL_LEN_FILE``
+
+``TS_URL_SCHEME_FTP``
+    "ftp"
+    ``TS_URL_LEN_FTP``
+
+``TS_URL_SCHEME_GOPHER``
+    "gopher"
+    ``TS_URL_LEN_GOPHER``
+
+``TS_URL_SCHEME_HTTP``
+    "http"
+    ``TS_URL_LEN_HTTP``
+
+``TS_URL_SCHEME_HTTPS``
+    "https"
+    ``TS_URL_LEN_HTTPS``
+
+``TS_URL_SCHEME_MAILTO``
+    "mailto"
+    ``TS_URL_LEN_MAILTO``
+
+``TS_URL_SCHEME_NEWS``
+    "news"
+    ``TS_URL_LEN_NEWS``
+
+``TS_URL_SCHEME_NNTP``
+    "nntp"
+    ``TS_URL_LEN_NNTP``
+
+``TS_URL_SCHEME_PROSPERO``
+    "prospero"
+    ``TS_URL_LEN_PROSPERO``
+
+``TS_URL_SCHEME_TELNET``
+    "telnet"
+    ``TS_URL_LEN_TELNET``
+
+``TS_URL_SCHEME_WAIS``
+    "wais"
+    ``TS_URL_LEN_WAIS``
+
+The scheme names above are defined in ``ts.h`` as ``const`` ``char*``
+strings. When Traffic Server sets the scheme portion of the URL (or any
+portion for that matter), it quickly checks to see if the new value is
+one of the known values. If it is, then it stores a pointer into a
+global table (instead of storing the known value in the marshal buffer).
+The scheme values listed above are also pointers into this table. This
+allows simple pointer comparison of the value returned from
+``TSUrlSchemeGet`` with one of the values listed above. You should use
+the Traffic Server-defined values when referring to one of the known
+schemes, since doing so can prevent the possibility of spelling errors.
+
+Traffic Server **URL functions** are listed below:
+
+:c:func:`TSUrlClone`
+:c:func:`TSUrlCopy`
+:c:func:`TSUrlCreate`
+:c:func:`TSUrlDestroy`
+:c:func:`TSUrlPrint`
+:c:func:`TSUrlFtpTypeGet`
+:c:func:`TSUrlFtpTypeSet`
+:c:func:`TSUrlHostGet`
+:c:func:`TSUrlHostSet`
+:c:func:`TSUrlHttpFragmentGet`
+:c:func:`TSUrlHttpFragmentSet`
+:c:func:`TSUrlHttpParamsGet`
+:c:func:`TSUrlHttpParamsSet`
+:c:func:`TSUrlHttpQueryGet`
+:c:func:`TSUrlHttpQuerySet`
+:c:func:`TSUrlLengthGet`
+:c:func:`TSUrlParse`
+:c:func:`TSUrlPasswordGet`
+:c:func:`TSUrlPasswordSet`
+:c:func:`TSUrlPathGet`
+:c:func:`TSUrlPathSet`
+:c:func:`TSUrlPortGet`
+:c:func:`TSUrlPortSet`
+:c:func:`TSUrlSchemeGet`
+:c:func:`TSUrlSchemeSet`
+:c:func:`TSUrlStringGet`
+:c:func:`TSUrlUserGet`
+:c:func:`TSUrlUserSet`

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-transformations/append-transform-plugin.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-transformations/append-transform-plugin.en.rst b/doc/developer-guide/plugins/http-transformations/append-transform-plugin.en.rst
new file mode 100644
index 0000000..2207278
--- /dev/null
+++ b/doc/developer-guide/plugins/http-transformations/append-transform-plugin.en.rst
@@ -0,0 +1,148 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-transformations-append:
+
+Append-Transform Plugin
+***********************
+
+The append-transform plugin appends text to the body of an HTTP
+response. It obtains this text from a file; the name of the file
+containing the append text is a parameter you specify in
+``plugin.config``, as follows:
+
+::
+
+    append-transform.so path/to/file
+
+The append-transform plugin is based on ``null-transform.c``. The only
+difference is that after the plugin feeds the document through the
+transformation, it adds text to the response.
+
+Below is a list of the functions in ``append-transform.c``, in the order
+they appear in the source code. Below each entry is a description of
+what the function does:
+
+-  **``my_data_alloc``**
+
+   Allocates and initializes a ``MyData`` structure. The plugin defines
+   a struct, ``MyData``, as follows:
+
+   .. code-block:: c
+
+       typedef struct {
+           TSVIO output_vio;
+           TSIOBuffer output_buffer;
+           TSIOBufferReader output_reader;
+           int append_needed;
+       } MyData;
+
+   The ``MyData`` structure is used to represent data that the
+   transformation (vconnection) needs. The transformation's data pointer
+   is set to a ``MyData`` pointer using ``TSContDataSet`` in the
+   ``handle_transform`` routine.
+
+-  **``my_data_destroy``**
+
+   Destroys objects of type ``MyData``. To deallocate the transform's
+   data, the ``append_transform`` routine (see below) calls
+   ``my_data_destroy`` when the transformation is complete.
+
+-  **``handle_transform``**
+
+   This function does the actual data transformation. The transformation
+   is created in ``transform_add`` (see below). ``handle_transform`` is
+   called by ``append_transform``.
+
+-  **``append_transform``**
+
+   This is the handler function for the transformation vconnection
+   created in ``transform_add``. It is the implementation of the
+   vconnection.
+
+   -  If the transformation vconnection has been closed, then
+      ``append_transform`` calls ``my_data_destroy`` to destroy the
+      vonnection.
+
+   -  If ``append_transform`` receives an error event, then it calls
+      back the continuation to let it know it has completed the write
+      operation.
+
+   -  If it receives a ``WRITE_COMPLETE`` event, then it shuts down the
+      write portion of its vconnection.
+
+   -  If it receives a ``WRITE_READY`` or any other event (such as
+      ``TS_HTTP_RESPONSE_TRANSFORM_HOOK``), then it calls
+      ``handle_transform`` to attempt to transform more data.
+
+-  **``transformable``**
+
+   The plugin transforms only documents that have a content type of
+   ``text/html``. This function examines the ``Content-Type`` MIME
+   header field in the response header. If the value of the MIME field
+   is ``text/html``, then the function returns 1; otherwise, it returns
+   zero.
+
+-  **``transform_add``**
+
+   Creates the transformation for the current transaction and sets up a
+   transformation hook. The handler function for the transformation is
+   ``append_transform``.
+
+-  **``transform_plugin``**
+
+   This is the handler function for the main continuation for the
+   plugin. Traffic Server calls this function whenever it reads an HTTP
+   response header. ``transform_plugin`` does the following:
+
+   -  Gets a handle to the HTTP transaction being processed
+
+   -  Calls ``transformable`` to determine whether the response document
+      content is of type ``text/html``
+
+   -  If the content is transformable, then it calls ``transform_add``
+      to create the transformation.
+
+   -  Calls ``TSHttpTxnReenable`` to continue the transaction
+
+-  **``load``**
+
+   Opens the file containing the text to be appended and loads the
+   contents of the file into an ``TSIOBuffer`` called ``append_buffer``.
+
+-  **``TSPluginInit``**
+
+   Does the following:
+
+   -  Checks to make sure that the required configuration information
+      (the append text filename) is entered in ``plugin.config``
+      correctly.
+
+   -  If there is a filename, then ``TSPluginInit`` calls load to load
+      the text.
+
+   -  Creates a continuation for the plugin. The handler for this
+      continuation is ``transform_plugin``.
+
+   -  Adds the plugin's continuation to
+      ``TS_HTTP_READ_RESPONSE_HDR_HOOK``. In other words, it sets up a
+      callback of the plugin's continuation when Traffic Server reads
+      HTTP response headers.
+
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-transformations/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-transformations/index.en.rst b/doc/developer-guide/plugins/http-transformations/index.en.rst
new file mode 100644
index 0000000..34fabc6
--- /dev/null
+++ b/doc/developer-guide/plugins/http-transformations/index.en.rst
@@ -0,0 +1,185 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-transformations:
+
+HTTP Transformations
+********************
+
+Transform plugins examine or transform HTTP message body content. For
+example, transform plugins can:
+
+-  Append text to HTML documents
+
+-  Compress images
+
+-  Do virus checking (on client ``POST`` data or server response data)
+
+-  Do content-based filtering (filter out HTML documents that contain
+   certain terms or expressions)
+
+This chapter explains how to write transform plugins. The following
+examples are discussed in detail:
+
+.. toctree::
+   :maxdepth: 2
+
+   sample-null-transformation-plugin.en
+   append-transform-plugin.en
+   sample-buffered-null-transformation-plugin.en
+
+.. _WritingContentTransformPlugin:
+
+Writing Content Transform Plugins
+=================================
+
+Content transformation plugins transform HTTP response content (such as
+images or HTML documents) and HTTP request content (such as client
+``POST`` data). Because the data stream to be transformed is of variable
+length, these plugins must use a mechanism that passes data from buffer
+to buffer *and* checks to see if the end of the data stream is reached.
+This mechanism is provided by virtual connections (``VConnection``\ s)
+and virtual IO descriptors (``VIO``\ s).
+
+A ``VConnection`` is an abstraction for a data pipe that allows its
+users to perform asynchronous reads and writes without knowing the
+underlying implementation. A transformation is a specific type of
+``VConnection``. A **transformation** connects an input data source and
+an output data sink; this feature enables it to view and modify all the
+data passing through it.
+
+Transformations can be chained together, one after the other, so that
+multiple transformations can be performed on the same content. The
+``VConnection`` type, ``TSVConn``, is actually a subclass of ``TSCont``,
+which means that ``VConnection``\ s (and transformations) are
+continuations. ``VConnection``\ s and transformations can thus exchange
+events, informing one another that data is available for reading or
+writing, or that the end of a data stream is reached.
+
+A ``VIO`` is a description of an IO operation that is in progress.
+Every ``VConnection`` has an associated *input VIO* and an associated
+*output VIO*. When ``VConnection``\ s are transferring data to one
+another, one ``VConnection``'s input ``VIO`` is another
+``VConnection``'s output ``VIO``. A ``VConnection``'s input ``VIO`` is
+also called its **write ``VIO``** because the input ``VIO`` refers to a
+write operation performed on the ``VConnection`` itself. Similarly, the
+output ``VIO`` is also called the **read ``VIO``**. For transformations,
+which are designed to pass data in one direction, you can picture the
+relationship between the transformation ``VConnection`` and its
+``VIO``\ s as follows:
+
+.. _transformationAndItsVIOs:
+
+.. figure:: /static/images/sdk/vconnection.jpg
+   :alt: A Transformation and its VIOs
+   :align: center
+
+   **A Transformation and its VIOs**
+
+Because the Traffic Server API places transformations directly in the
+response or request data stream, the transformation ``VConnection`` is
+responsible only for reading the data from the input buffer,
+transforming it, and then writing it to the output buffer. The upstream
+``VConnection`` writes the incoming data to the transformation's input
+buffer. In the figure above, :ref:`TransformationAndItsVIOs`, the input ``VIO`` describes the
+progress of the upstream ``VConnection``'s write operation on the
+transformation, while the output ``VIO`` describes the progress of the
+transformation's write operation on the output (downstream)
+``VConnection``. The **nbytes** value in the ``VIO`` is the total number
+of bytes to be written. The **ndone** value is the current progress, or
+the number of bytes that have been written at a specific point in time.
+
+When writing a transformation plugin, you must understand implementation
+as well as the use of ``VConnection``\ s. The *implementor's side*
+refers to how to implement a ``VConnection`` that others can use. At
+minimum, a transform plugin creates a transformation that sits in the
+data stream and must be able to handle the events that the upstream and
+downstream ``VConnection``\ s send to it. The *user's side* refers to
+how to use a ``VConnection`` to read or write data. At the very least,
+transformations output (write) data.
+
+.. _transformations:
+
+Transformations
+---------------
+
+VIOs
+----
+
+A ``VIO``*or virtual IO is a description of an in progress IO
+operation. The ``VIO`` data structure is used by ``VConnection`` users
+to determine how much progress has been made on a particular IO
+operation, and to reenable an IO operation when it stalls due to buffer
+space. ``VConnection`` implementors use ``VIO``\ s to determine the
+buffer for an IO operation, how much work to do on the IO operation, and
+which continuation to call back when progress on the IO operation is
+made.
+
+The ``TSVIO`` data structure itself is opaque, but it might have been
+defined as follows:
+
+.. code-block:: c
+
+   typedef struct {
+      TSCont continuation;
+      TSVConn vconnection;
+      TSIOBufferReader reader;
+      TSMutex mutex;
+      int nbytes;
+      int ndone;
+   } *TSVIO;
+
+IO Buffers
+----------
+
+The **IO buffer** data structure is the building block of the
+``VConnection`` abstraction. An IO buffer is composed of a list of
+buffer blocks which, in turn, point to buffer data. Both the *buffer
+block* (``TSIOBufferBlock``) and *buffer data* (``TSIOBufferData``) data
+structures are reference counted so they can reside in multiple buffers
+at the same time. This makes it extremely efficient to copy data from
+one IO buffer to another using ``TSIOBufferCopy``, since Traffic Server
+only needs to copy pointers and adjust reference counts appropriately
+(instead of actually copying any data).
+
+The IO buffer abstraction provides for a single writer and multiple
+readers. In order for the readers to have no knowledge of each other,
+they manipulate IO buffers through the\ ``TSIOBufferReader`` data
+structure. Since only a single writer is allowed, there is no
+corresponding ``TSIOBufferWriter`` data structure. The writer simply
+modifies the IO buffer directly.
+
+Transaction Data Sink
+~~~~~~~~~~~~~~~~~~~~~
+
+The hook `TS_HTTP_RESPONSE_CLIENT_HOOK` is a hook that supports a special type of transformation, one with only input and no output.
+Although the transformation doesn't provide data back to Traffic Server it can do anything else with the data, such as writing it
+to another output device or process. It must, however, consume all the data for the transaction. There are two primary use cases.
+
+#. Tap in to the transaction to provide the data for external processing.
+#. Maintain the transaction.
+
+For the latter it is important to note that if all consumers of a transaction (primarily the user agent) shut down the transaction is also
+terminated, including the connection to the origin server. A data sink transform, unlike a standard transform, is considered to be a consumer
+and will keep the transaction and the origin server connection up. This is useful when the transaction is in some way expensive and should
+run to completion even if the user agent disconnects. Examples would be a standard transform that is expensive to initiate, or expensive
+origin server connections that should be :ts:cv:`shared <proxy.config.http.server_session_sharing.match>`.
+
+There is an `example plugin <https://github.com/apache/trafficserver/blob/master/example/txn-data-sink/txn-data-sink.c>`_ that demonstrates
+this used as a pure data sink to keep the transaction up regardless of whether the user agent disconnects.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-transformations/sample-buffered-null-transformation-plugin.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-transformations/sample-buffered-null-transformation-plugin.en.rst b/doc/developer-guide/plugins/http-transformations/sample-buffered-null-transformation-plugin.en.rst
new file mode 100644
index 0000000..2e7c18e
--- /dev/null
+++ b/doc/developer-guide/plugins/http-transformations/sample-buffered-null-transformation-plugin.en.rst
@@ -0,0 +1,216 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-transformations-buffered-null:
+
+Sample Buffered Null Transform Plugin
+*************************************
+
+The buffered null transform, ``bnull-transform.c``, reads the response
+content into a buffer and then writes the full buffer out to the client.
+Many examples of transformations, such as compression, require you to
+gather the full response content in order to perform the transformation.
+
+The buffered null transform uses a state variable to keep track of when
+it is (a) reading data into the buffer and (b) writing the data from the
+buffer to the downstream vconnection.
+
+The following is a step-by-step walk through the buffered null
+transform:
+
+#.  Gets a handle to HTTP transactions.
+
+    .. code-block:: c
+
+       void
+          TSPluginInit (int argc, const char *argv[]) {
+             TSHttpHookAdd (TS_HTTP_READ_RESPONSE_HDR_HOOK,
+                TSContCreate (transform_plugin, NULL)); }
+
+    With this ``TSPluginInit`` routine, the plugin is called back every
+    time Traffic Server reads a response header.
+
+#.  Checks to see if the transaction response is transformable.
+
+    .. code-block:: c
+
+       static int transform_plugin (TSCont contp, TSEvent event, void *edata) {
+          TSHttpTxn txnp = (TSHttpTxn) edata;
+          switch (event) {
+             case TS_EVENT_HTTP_READ_RESPONSE_HDR:
+                if (transformable (txnp)) {
+                   transform_add (txnp);
+                }
+
+    The default behavior for transformations is to cache the transformed
+    content (if desired, you also can tell Traffic Server to cache
+    untransformed content). Therefore, only responses received directly
+    from an origin server need to be transformed. Objects served from
+    the cache are already transformed. To determine whether the response
+    is from the origin server, the routine transformable checks the
+    response header for the "200 OK" server response.
+
+    .. code-block:: c
+
+       {
+          TSMBuffer bufp;
+          TSMLoc hdr_loc;
+          TSHttpStatus resp_status;
+
+          TSHttpTxnServerRespGet (txnp, &bufp, &hdr_loc);
+
+          if(TS_HTTP_STATUS_OK==
+             (resp_status=TSHttpHdrStatusGet(bufp,hdr_loc)))
+          {
+             return 1;
+          }
+          else {
+             return 0;
+          }
+       }
+
+#. If the response is transformable, then the plugin creates a
+   transformation vconnection that gets called back when the response
+   data is ready to be transformed (as it is streaming from the origin
+   server).
+
+   .. code-block:: c
+
+      static void transform_add (TSHttpTxn txnp)
+      {
+         TSVConn connp;
+         connp = TSTransformCreate (bnull_transform, txnp);
+         TSHttpTxnHookAdd (txnp, TS_HTTP_RESPONSE_TRANSFORM_HOOK, connp);
+      }
+
+   The previous code fragment shows that the handler function for the
+   transformation vconnection is ``bnull_transform``.
+
+#. The ``bnull_transform`` function has to handle ``ERROR``,
+   ``WRITE_COMPLETE``, ``WRITE_READY``, and ``IMMEDIATE`` events. If
+   the transform is just beginning, the event received is probably
+   ``IMMEDIATE``. The ``bnull_transform`` function calls
+   ``handle_transform`` to handle ``WRITE_READY`` and ``IMMEDIATE``.
+
+#. The ``handle_transform`` function examines the data parameter for
+   the continuation passed to it (the continuation passed to
+   ``handle_transform`` is the transformation vconnection). The data
+   structure keeps track of two states: copying the data into the
+   buffer (``STATE_BUFFER_DATA``) and writing the contents of the
+   buffer to the output vconnection (``STATE_OUTPUT_DATA``).
+
+#. Get a handle to the input VIO (see the ``handle_buffering``
+   function). ``input_vio = TSVConnWriteVIOGet (contp);`` This is so
+   that the transformation can get information about the upstream
+   vconnection's write operation to the input buffer.
+
+#. Copy data from the input buffer to the output buffer. See the
+   ``handle_buffering`` function for the following code fragment:
+
+   .. code-block:: c
+
+      TSIOBufferCopy (data->output_buffer,
+         TSVIOReaderGet (write_vio), towrite, 0);
+
+#. Tell the input buffer that the transformation has read the data. See
+   the ``handle_buffering`` function for the following code fragment:
+
+   .. code-block:: c
+
+      TSIOBufferReaderConsume (TSVIOReaderGet (write_vio), towrite);
+
+#. Modify the input VIO to tell it how much data has been read
+   (increase the value of ``ndone``). See the ``handle_buffering``
+   function for the following code fragment:
+
+   .. code-block:: c
+
+      TSVIONDoneSet (write_vio, TSVIONDoneGet (write_vio) + towrite); }
+
+#. If there is more data left to read ( if ndone < nbytes), then the
+   ``handle_buffering`` function wakes up the upstream vconnection by
+   sending it ``WRITE_READY``:
+
+   .. code-block:: c
+
+      if (TSVIONTodoGet (write_vio) > 0) {
+         if (towrite > 0) {
+            TSContCall (TSVIOContGet (write_vio),
+               TS_EVENT_VCONN_WRITE_READY, write_vio);
+         }
+      } else {
+
+   The process of passing data through the transformation is
+   illustrated in the following diagram. The transformation sends
+   ``WRITE_READY`` events when it needs more data; when data is
+   available, the upstream vconnection reenables the transformation
+   with an ``IMMEDIATE`` event.
+
+   The following diagram illustrates the read from an input
+   vconnection:
+
+   **Reading Data Into the Buffer (the ``STATE_BUFFER_DATA`` State)**
+   {#ReadingDataIntoBuffer}
+
+   .. figure:: /static/images/sdk/vconn_buffer.jpg
+      :alt: Reading Data Into the Buffer the STATE\_BUFFER\_DATA State
+
+      Reading Data Into the Buffer the STATE\_BUFFER\_DATA State
+
+#. When the data is read into the output buffer, the
+   ``handle_buffering`` function sets the state of the transformation's
+   data structure to ``STATE_OUTPUT_DATA`` and calls the upstream
+   vconnection back with the ``WRITE_COMPLETE`` event.
+
+   .. code-block:: c
+
+      data->state = STATE_OUTPUT_DATA;
+      TSContCall (TSVIOContGet (write_vio),
+         TS_EVENT_VCONN_WRITE_COMPLETE, write_vio);
+
+#. The upstream vconnection will probably shut down the write operation
+   when it receives the ``WRITE_COMPLETE`` event. The handler function
+   of the transformation, ``bnull_transform``, receives an
+   ``IMMEDIATE`` event and calls the ``handle_transform`` function.
+   This time, the state is ``STATE_OUTPUT_DATA``, so
+   ``handle_transform`` calls ``handle_output``.
+
+#. The ``handle_output`` function gets a handle to the output
+   vconnection: ``output_conn = TSTransformOutputVConnGet (contp);``
+
+#. The ``handle_output`` function writes the buffer to the output
+   vconnection:
+
+   .. code-block:: c
+
+      data->output_vio =
+         TSVConnWrite (output_conn, contp, data->output_reader,
+         TSIOBufferReaderAvail (data->output_reader) );
+
+   The following diagram illustrates the write to the output
+   vconnection:
+
+   **Writing the Buffered Data to the Output Vconnection**
+   {#WritingBufferedtDataIntoVConnection}
+
+   .. figure:: /static/images/sdk/vconn_buf_output.jpg
+      :alt: Writing the Buffered Data to the Output Vconnection
+
+      Writing the Buffered Data to the Output Vconnection
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/http-transformations/sample-null-transformation-plugin.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/http-transformations/sample-null-transformation-plugin.en.rst b/doc/developer-guide/plugins/http-transformations/sample-null-transformation-plugin.en.rst
new file mode 100644
index 0000000..8a0dd1a
--- /dev/null
+++ b/doc/developer-guide/plugins/http-transformations/sample-null-transformation-plugin.en.rst
@@ -0,0 +1,221 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../../common.defs
+
+.. _developer-plugins-http-transformations-null-transform:
+
+Sample Null Transform Plugin
+****************************
+
+This section provides a step-by-step description of what the null
+transform plugin does, along with sections of code that apply. For
+context, you can find each code snippet in the complete source code.
+Some of the error checking details are left out - to give the
+description a step-by-step flow, only the highlights of the transform
+are included.
+
+Below is an overview of the null transform plugin:
+
+1.  Gets a handle to HTTP transactions.
+
+    .. code-block:: c
+
+        void
+        TSPluginInit (int argc, const char *argv[]) {
+            TSHttpHookAdd (TS_HTTP_READ_RESPONSE_HDR_HOOK,
+                    TSContCreate (transform_plugin, NULL));
+
+    With this ``TSPluginInit`` routine, the plugin is called back every
+    time Traffic Server reads a response header.
+
+2.  Checks to see if the transaction response is transformable.
+
+    .. code-block:: c
+
+        static int transform_plugin (TSCont contp, TSEvent event, void *edata) {
+            TSHttpTxn txnp = (TSHttpTxn) edata;
+            switch (event) {
+                case TS_EVENT_HTTP_READ_RESPONSE_HDR:
+                    if (transformable (txnp)) {
+                        transform_add (txnp);
+                    }
+
+    The default behavior for transformations is to cache the transformed
+    content (you can also tell Traffic Server to cache untransformed
+    content, if you want). Therefore, only responses received directly
+    from an origin server need to be transformed. Objects served from
+    cache are already transformed. To determine whether the response is
+    from the origin server, the routine ``transformable`` checks the
+    response header for the "200 OK" server response.
+
+    .. code-block:: c
+
+        static int transformable (TSHttpTxn txnp)
+        {
+            TSMBuffer bufp;
+            TSMLoc hdr_loc;
+            TSHttpStatus resp_status;
+            TSHttpTxnServerRespGet (txnp, &bufp, &hdr_loc);
+
+            if (TS_HTTP_STATUS_OK == (resp_status =
+                        TSHttpHdrStatusGet (bufp, hdr_loc)) ) {
+                return 1;
+            } else {
+                return 0;
+            }
+        }
+
+3.  If the response is transformable, then the plugin creates a
+    transformation vconnection that gets called back when the response
+    data is ready to be transformed (as it is streaming from the origin
+    server).
+
+    .. code-block:: c
+
+        static void transform_add (TSHttpTxn txnp)
+        {
+            TSVConn connp;
+            connp = TSTransformCreate (null_transform, txnp);
+            TSHttpTxnHookAdd (txnp, TS_HTTP_RESPONSE_TRANSFORM_HOOK, connp);
+        }
+
+    The previous code fragment shows that the handler function for the
+    transformation vconnection is ``null_transform``.
+
+4.  Get a handle to the output vconnection (that receives data from the
+    tranformation).
+
+    .. code-block:: c
+
+        output_conn = TSTransformOutputVConnGet (contp);
+
+5.  Get a handle to the input VIO. (See the ``handle_transform``
+    function.)
+
+    .. code-block:: c
+
+        input_vio = TSVConnWriteVIOGet (contp);
+
+    This is so that the transformation can get information about the
+    upstream vconnection's write operation to the input buffer.
+
+6.  Initiate a write to the output vconnection of the specified number
+    of bytes. When the write is initiated, the transformation expects to
+    receive ``WRITE_READY``, ``WRITE_COMPLETE``, or ``ERROR`` events
+    from the output vconnection. See the ``handle_transform`` function
+    for the following code fragment:
+
+    .. code-block:: c
+
+        data->output_vio = TSVConnWrite (output_conn, contp,
+            data->output_reader, TSVIONBytesGet (input_vio));
+
+7.  Copy data from the input buffer to the output buffer. See the
+    ``handle_transform`` function for the following code fragment:
+
+    .. code-block:: c
+
+        TSIOBufferCopy (TSVIOBufferGet (data->output_vio),
+                TSVIOReaderGet (input_vio), towrite, 0);
+
+8.  Tell the input buffer that the transformation has read the data. See
+    the ``handle_transform`` function for the following code fragment:
+
+    .. code-block:: c
+
+        TSIOBufferReaderConsume (TSVIOReaderGet (input_vio), towrite);
+
+9.  Modify the input VIO to tell it how much data has been read
+    (increase the value of ``ndone``). See the ``handle_transform``
+    function for the following code fragment:
+
+    .. code-block:: c
+
+        TSVIONDoneSet (input_vio, TSVIONDoneGet (input_vio) + towrite);
+
+10. If there is more data left to read ( if ndone < nbytes), then the
+    ``handle_transform`` function wakes up the downstream vconnection
+    with a reenable and wakes up the upstream vconnection by sending it
+    ``WRITE_READY``:
+
+    .. code-block:: c
+
+        if (TSVIONTodoGet (input_vio) > 0) {
+            if (towrite > 0) {
+                TSVIOReenable (data->output_vio);
+
+                TSContCall (TSVIOContGet (input_vio),
+                        TS_EVENT_VCONN_WRITE_READY, input_vio);
+            }
+            } else {
+
+    The process of passing data through the transformation is
+    illustrated in the following diagram. The downstream vconnections
+    send ``WRITE_READY`` events when they need more data; when data is
+    available, the upstream vconnections reenable the downstream
+    vconnections. In this instance, the ``TSVIOReenable`` function sends
+    ``TS_EVENT_IMMEDIATE``.
+
+    **Passing Data Through a Transformation**
+    {#PassingDataThroughaTransformation}
+
+.. figure:: /static/images/sdk/vconnection1.jpg
+      :alt: Passing Data Through a Transformation
+
+      Passing Data Through a Transformation
+
+11. If the ``handle_transform`` function finds there is no more data to
+    read, then it sets ``nbytes`` to ``ndone`` on the output
+    (downstream) VIO and wakes up the output vconnection with a
+    reenable. It then triggers the end of the write operation from the
+    upstream vconnection by sending the upstream vconnection a
+    ``WRITE_COMPLETE`` event.
+
+    .. code-block:: c
+
+      TSVIONBytesSet (data->output_vio, TSVIONDoneGet (input_vio));
+         TSVIOReenable (data->output_vio);
+         TSContCall (TSVIOContGet (input_vio),
+            TS_EVENT_VCONN_WRITE_COMPLETE, input_vio);
+      }
+
+    When the upstream vconnection receives the ``WRITE_COMPLETE`` event,
+    it will probably shut down the write operation.
+
+12. Similarly, when the downstream vconnection has consumed all of the
+    data, it sends the transformation a ``WRITE_COMPLETE`` event. The
+    transformation handles this event with a shut down (the
+    transformation shuts down the write operation to the downstream
+    vconnection). See the ``null_plugin`` function for the following
+    code fragment:
+
+    .. code-block:: c
+
+        case TS_EVENT_VCONN_WRITE_COMPLETE:
+            TSVConnShutdown (TSTransformOutputVConnGet (contp), 0, 1
+            break;
+
+    The following diagram illustrates the flow of events:
+
+    **Ending the Transformation** {#EndingTransformation}
+
+    .. figure:: /static/images/sdk/vconnection2.jpg
+       :alt: Ending the Transformation
+
+       Ending the Transformation
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/index.en.rst b/doc/developer-guide/plugins/index.en.rst
new file mode 100644
index 0000000..323ebde
--- /dev/null
+++ b/doc/developer-guide/plugins/index.en.rst
@@ -0,0 +1,154 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-plugins:
+
+Plugin Development
+******************
+
+.. toctree::
+   :maxdepth: 2
+
+   introduction.en
+   getting-started/index.en
+   building-plugins.en
+   configuration.en
+   plugin-management/index.en
+   actions/index.en
+   hooks-and-transactions/index.en
+   continuations/index.en
+   mutexes.en
+   io/index.en
+   http-headers/index.en
+   http-transformations/index.en
+   new-protocol-plugins.en
+   plugin-interfaces.en
+   adding-statistics.en
+   example-plugins/index.en
+
+Traffic Server HTTP State Machine
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Traffic Server performs sophisticated HTTP caching and proxying.
+Important features include checking for alternates and document
+freshness, filtering, supporting cache hierarchies, and hosting. Traffic
+Server handles thousands of client requests at a time and each request
+is handled by an HTTP state machine. These machines follow a complex
+state diagram that includes all of the states required to support
+Traffic Server's features. The Traffic Server API provides hooks to a
+subset of these states, chosen for their relevance to plugins. You can
+view the API hooks and corresponding HTTP states in the
+:ref:`http-txn-state-diagram`.
+
+The example in this section (below) explains how a plugin typically
+intervenes and extends Traffic Server's processing of an HTTP
+transaction. Complete details about hooking on to Traffic Server
+processes are provided in :ref:`developer-plugins-hooks-and-transactions`.
+
+HTTP Transaction
+^^^^^^^^^^^^^^^^
+
+An HTTP transaction consists of a client request for a web document and
+Traffic Server's response. The response could be the requested web
+server content or it could be an error message. The content could come
+from the Traffic Server cache or Traffic Server might fetch it from the
+origin server. The following diagram shows some states in a typical
+transaction - specifically, the scenario wherein content is served from
+cache.
+
+**Simplified HTTP Transaction**
+
+.. _SimplifiedHTTPTransaction:
+
+.. figure:: /static/images/sdk/transact75.jpg
+   :alt: Simplified HTTP Transaction
+
+   Simplified HTTP Transaction
+
+In the diagram above, Traffic Server accepts the client connection,
+reads the request headers, looks up the origin server's IP address, and
+looks for the requested content in the cache. If the content is not in
+the cache (a "miss"), then Traffic Server opens a connection to the
+origin server and issues a request for the content. If the content is in
+the cache (a "hit"), then Traffic Server checks it for freshness.
+
+If the content is fresh, then Traffic Server sends a reply header to the
+client. If the content is stale, then Traffic Server opens a connection
+to the origin server and requests the content. The figure above,
+:ref:`SimplifiedHTTPTransaction`, does *not*
+show behavior in the event of an error. If there is an error at a any
+stage, then the HTTP state machine jumps to the "send reply header"
+state and sends a reply. If the reply is an error, then the transaction
+closes. If the reply is not an error, then Traffic Server first sends
+the response content before it closes the transaction.
+
+**API Hooks Corresponding to States**
+
+.. _APIHooksCorrespondingtoStates:
+
+.. figure:: /static/images/sdk/transact_hook75.jpg
+   :alt: API Hooks Corresponding to States Listed in
+
+   API Hooks Corresponding to States Listed in
+
+You use hooks as triggers to start your plugin. The name of a hook
+reflects the Traffic Server state that was *just completed*. For
+example, the "OS DNS lookup" hook wakes up a plugin right *after* the
+origin server DNS lookup. For a plugin that requires the IP address of
+the requested origin server, this hook is the right one to use. The
+Blacklist plugin works in this manner, as shown in the :ref:`BlackListPlugin`
+diagram below.
+
+**Blacklist Plugin**
+
+.. _BlackListPlugin:
+
+.. figure:: /static/images/sdk/blacklist75.jpg
+   :alt: Blacklist Plugin
+
+   Blacklist Plugin
+
+Traffic Server calls the Blacklist plugin right after the origin server
+DNS lookup. The plugin checks the requested host against a list of
+blacklisted servers; if the request is allowed, then the transaction
+proceeds. If the host is forbidden, then the Blacklist plugin sends the
+transaction into an error state. When the HTTP state machine gets to the
+"send reply header" state, it then calls the Blacklist plugin to provide
+the error message that's sent to the client.
+
+Types of Hooks
+^^^^^^^^^^^^^^
+
+The Blacklist plugin's hook to the origin server DNS lookup state is a *global
+hook*, meaning that the plugin is called every time there's an HTTP transaction
+with a DNS lookup event. The plugin's hook to the send reply header state is a
+*transaction hook*, meaning that this hook is only invoked for specified
+transactions (in the :ref:`developer-plugins-examples-blacklist` example, it's
+only used for requests to blacklisted servers). Several examples of setting up
+hooks are provided in :ref:`developer-plugins-header-based-examples` and
+:ref:`developer-plugins-http-transformations`.
+
+*Header manipulation plugins*, such as filtering, basic authorization,
+or redirects, usually have a global hook to the DNS lookup or the read
+request header states. If specific actions need to be done to the
+transaction further on, then the plugin adds itself to a transaction
+hook. *Transformation plugins* require a global hook to check
+all transactions for transformability followed by a transform hook,
+which is a type of transaction hook used specifically for transforms.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/plugins/introduction.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/plugins/introduction.en.rst b/doc/developer-guide/plugins/introduction.en.rst
new file mode 100644
index 0000000..22f63ce
--- /dev/null
+++ b/doc/developer-guide/plugins/introduction.en.rst
@@ -0,0 +1,308 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-plugins-introduction:
+
+Plugin Development Introduction
+*******************************
+
+.. toctree::
+   :maxdepth: 1
+
+This chapter provides a foundation for designing and writing plugins.
+Reading this chapter will help you to understand:
+
+-  The asynchronous event mode. This is the design paradigm used
+   throughout Traffic Server; plugins must also follow this design. It
+   includes the callback mechanism for Traffic Server to "wake up" your
+   plugin and put it to work.
+
+-  Traffic Server's HTTP processing, with an overview of the HTTP state
+   machine.
+
+-  How plugins can hook onto and modify/extend Traffic Server's HTTP
+   processing.
+
+-  A :ref:`developer-plugins-roadmap` with an overview of the functionality
+   provided by the Traffic Server API.
+
+.. _developer-plugins-roadmap:
+
+Roadmap
+=======
+
+This chapter has provided an overview of Traffic Server's HTTP
+processing, API hooks, and the asynchronous event model. Next, you must
+understand the capabilities of Traffic Server API functions. These are
+quite broad:
+
+-  **HTTP header manipulation functions**
+
+   Obtain information about and manipulate HTTP headers, URLs, & MIME
+   headers.
+
+-  **HTTP transaction functions**
+
+   Get information about and modify HTTP transactions (for example: get
+   the client IP associated to the transaction; get the server IP; get
+   parent proxy information)
+
+-  **IO functions**
+
+   Manipulate vconnections (virtual connections, used for network and
+   disk I/O)
+
+-  **Network connection functions**
+
+   Open connections to remote servers.
+
+-  **Statistics functions**
+
+   Define and compute statistics for your plugin's activity.
+
+-  **Traffic Server management functions**
+
+   Obtain values for Traffic Server configuration and statistics
+   variables.
+
+Below are some guidelines for creating a plugin:
+
+#. Decide what you want your plugin to do, based on the capabilities of
+   the API and |TS|. The two kinds of example plugins
+   provided with this SDK are HTTP-based (includes header-based and
+   response transform plugins), and non-HTTP-based (a protocol plugin).
+   These examples are discussed in the following chapters.
+
+#. Determine where your plugin needs to hook on to Traffic Server's HTTP
+   processing (view the :ref:`http-txn-state-diagram`.
+
+#. Read :ref:`developer-plugins-header-based-examples` to learn the basics of
+   writing plugins: creating continuations and setting up hooks. If you
+   want to write a plugin that transforms data, then read
+   :ref:`developer-plugins-http-transformations`.
+
+#. Figure out what parts of the Traffic Server API you need to use and
+   then read about the details of those APIs in this manual's reference
+   chapters.
+
+#. Compile and load your plugin (see :ref:`developer-plugins-getting-started`.
+
+#. Depending on your plugin's functionality, you might start testing it
+   by issuing requests by hand and checking for the desired behavior in
+   Traffic Server log files. See the ***Traffic Server Administrator's
+   Guide*** for information about Traffic Server logs.
+
+Asynchronous Event Model
+========================
+
+Traffic Server is a multi-threaded process. There are two main reasons
+why a server might use multiple threads:
+
+-  To take advantage of the concurrency available with multiple CPUs and
+   multiple I/O devices.
+
+-  To manage concurrency from having many simultaneous client
+   connections. For example, a server could create one thread for each
+   connection, allowing the operating system (OS) to control switching
+   between threads.
+
+Traffic Server uses multiple threads for the first reason. However,
+Traffic Server does not use a separate OS thread per transaction because
+it would not be efficient when handling thousands of simultaneous
+connections.
+
+Instead, Traffic Server provides special event-driven mechanisms for
+efficiently scheduling work: the event system and continuations. The
+**event system** is used to schedule work to be done on threads. A
+**continuation** is a passive, event-driven state machine that can do
+some work until it reaches a waiting point; it then sleeps until it
+receives notification that conditions are right for doing more work. For
+example, HTTP state machines (which handle HTTP transactions) are
+implemented as continuations.
+
+Continuation objects are used throughout Traffic Server. Some might live
+for the duration of the Traffic Server process, while others are created
+(perhaps by other continuations) for specific needs and then destroyed.
+:ref:`TSInternals` (below) shows how the major
+components of Traffic Server interact. Traffic Server has several
+**processors**, such as *cache processor* and *net processor*, that
+consolidate cache or network I/O tasks. Processors talk to the event
+system and schedule work on threads. An executing thread calls back a
+continuation by sending it an event. When a continuation receives an
+event, it wakes up, does some work, and either destroys itself or goes
+back to sleep & waits for the next event.
+
+**Traffic Server Internals**
+
+.. _TSInternals:
+
+.. figure:: /static/images/sdk/event_sys80.jpg
+   :alt: Traffic Server Internals
+
+   Traffic Server Internals
+
+Plugins are typically implemented as continuations. All of the sample
+code plugins (except ``hello-world``) are continuations that are created
+when Traffic Server starts up; they then wait for events that trigger
+them into activity.
+
+**Traffic Server with Plugins**
+
+.. _TSwithPlugins:
+
+.. figure:: /static/images/sdk/evt_plugin120.jpg
+   :alt: Traffic Server with Plugins
+
+   Traffic Server with Plugins
+
+A plugin may consist of just one static continuation that is called
+whenever certain events happen. Examples of such plugins include
+``blacklist-1.c``, ``basic-auth.c``, and ``redirect-1.c``.
+Alternatively, a plugin might dynamically create other continuations as
+needed. Transform plugins are built in this manner: a static parent
+continuation checks all transactions to see if any are transformable;
+when a transaction is transformable, the static continuation creates a
+type of continuation called a **vconnection**. The vconnection lives as
+long as it takes to complete the transform and then destroys itself.
+This design can be seen in all of the sample transform plugins. Plugins
+that support new protocols also have this architecture: a static
+continuation listens for incoming client connections and then creates
+transaction state machines to handle each protocol transaction.
+
+When you write plugins, there are several ways to send events to
+continuations. For HTTP plugins, there is a "hook" mechanism that
+enables the Traffic Server HTTP state machine to send your plugin wakeup
+calls when needed. Additionally, several Traffic Server API functions
+trigger Traffic Server sub-processes to send events to plugins:
+``TSContCall``, ``TSVConnRead``, ``TSCacheWrite``, and
+``TSMgmtUpdateRegister``, to name a few.
+
+Naming Conventions
+==================
+
+Traffic Server HTTP State Machine
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Traffic Server performs sophisticated HTTP caching and proxying.
+Important features include checking for alternates and document
+freshness, filtering, supporting cache hierarchies, and hosting. Traffic
+Server handles thousands of client requests at a time and each request
+is handled by an HTTP state machine. These machines follow a complex
+state diagram that includes all of the states required to support
+Traffic Server's features. The Traffic Server API provides hooks to a
+subset of these states, chosen for their relevance to plugins. You can
+view the API hooks and corresponding HTTP states in the
+:ref:`http-txn-state-diagram`.
+
+The example in this section (below) explains how a plugin typically
+intervenes and extends Traffic Server's processing of an HTTP
+transaction. Complete details about hooking on to Traffic Server
+processes are provided in :ref:`developer-plugins-hooks-and-transactions`.
+
+HTTP Transaction
+^^^^^^^^^^^^^^^^
+
+An HTTP transaction consists of a client request for a web document and
+Traffic Server's response. The response could be the requested web
+server content or it could be an error message. The content could come
+from the Traffic Server cache or Traffic Server might fetch it from the
+origin server. The following diagram shows some states in a typical
+transaction - specifically, the scenario wherein content is served from
+cache.
+
+**Simplified HTTP Transaction**
+
+.. _SimplifiedHTTPTransaction:
+
+.. figure:: /static/images/sdk/transact75.jpg
+   :alt: Simplified HTTP Transaction
+
+   Simplified HTTP Transaction
+
+In the diagram above, Traffic Server accepts the client connection,
+reads the request headers, looks up the origin server's IP address, and
+looks for the requested content in the cache. If the content is not in
+the cache (a "miss"), then Traffic Server opens a connection to the
+origin server and issues a request for the content. If the content is in
+the cache (a "hit"), then Traffic Server checks it for freshness.
+
+If the content is fresh, then Traffic Server sends a reply header to the
+client. If the content is stale, then Traffic Server opens a connection
+to the origin server and requests the content. The figure above,
+:ref:`SimplifiedHTTPTransaction`, does not
+show behavior in the event of an error. If there is an error at a any
+stage, then the HTTP state machine jumps to the "send reply header"
+state and sends a reply. If the reply is an error, then the transaction
+closes. If the reply is not an error, then Traffic Server first sends
+the response content before it closes the transaction.
+
+**API Hooks Corresponding to States**
+
+.. _APIHooksCorrespondingtoStates:
+
+.. figure:: /static/images/sdk/transact_hook75.jpg
+   :alt: API Hooks Corresponding to States Listed in
+
+   API Hooks Corresponding to States Listed in
+
+You use hooks as triggers to start your plugin. The name of a hook
+reflects the Traffic Server state that was *just completed*. For
+example, the "OS DNS lookup" hook wakes up a plugin right *after* the
+origin server DNS lookup. For a plugin that requires the IP address of
+the requested origin server, this hook is the right one to use. The
+Blacklist plugin works in this manner, as shown in the :ref:`BlackListPlugin`
+diagram below.
+
+**Blacklist Plugin**
+
+.. _BlackListPlugin:
+
+.. figure:: /static/images/sdk/blacklist75.jpg
+   :alt: Blacklist Plugin
+
+   Blacklist Plugin
+
+Traffic Server calls the Blacklist plugin right after the origin server
+DNS lookup. The plugin checks the requested host against a list of
+blacklisted servers; if the request is allowed, then the transaction
+proceeds. If the host is forbidden, then the Blacklist plugin sends the
+transaction into an error state. When the HTTP state machine gets to the
+"send reply header" state, it then calls the Blacklist plugin to provide
+the error message that's sent to the client.
+
+Types of Hooks
+^^^^^^^^^^^^^^
+
+The Blacklist plugin's hook to the origin server DNS lookup state is a *global
+hook*, meaning that the plugin is called every time there's an HTTP transaction
+with a DNS lookup event. The plugin's hook to the send reply header state is a
+*transaction hook*, meaning that this hook is only invoked for specified
+transactions (in the :ref:`developer-plugins-examples-blacklist` example, it's
+only used for requests to blacklisted servers). Several examples of setting up
+hooks are provided in :ref:`developer-plugins-header-based-examples` and
+:ref:`developer-plugins-http-transformations`.
+
+*Header manipulation plugins*, such as filtering, basic authorization,
+or redirects, usually have a global hook to the DNS lookup or the read
+request header states. If specific actions need to be done to the
+transaction further on, then the plugin adds itself to a transaction
+hook. *Transformation plugins* require a global hook to check
+all transactions for transformability followed by a *transform hook*,
+which is a type of transaction hook used specifically for transforms.
+


Mime
View raw message