trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owens, Steve" <Steve.Ow...@disney.com>
Subject Re: Api Questions
Date Fri, 25 Jan 2013 20:01:45 GMT
Otto,

I have started mapping code examples from the null transform example to
the diagram to try to connect the dots see attached.

Steve Owens



On 1/25/13 11:44 AM, "Owens, Steve" <Steve.Owens@disney.com> wrote:

>Otto, yes I have seen them and read them several times.  The second link
>explains in rather abstract terms what VIO's do and how they relate to
>buffers.
>
>But I personally could use a little more amplification on what the
>specific methods are that manipulate these structures and what effect that
>calls to these methods have on the structures depicted in the images.
>
>The null transform plugin re-uses variables and takes enough shortcuts
>that what it is doing is seems obscured and I am trying to pierce that
>obscurity.  I will continue to try to pierce the veil but after having
>done so I would love to write up additional info to make it clearer to
>dunderheads such as myself what these methods actually do in order to
>reduce the slope of the learning curve for this technology.
>
>Thank you for re-directing me to these I am sure that given enough time
>and patience I will crack the code.
>
>
>
>On 1/25/13 11:25 AM, "Otto van der Schaaf" <oschaaf@gmail.com> wrote:
>
>>Hi Steve,
>>
>>Did you get a chance already to have a look at
>>http://trafficserver.apache.org/docs/v2/sdk/images/vconnection.jpg ?
>>That image is from
>>http://trafficserver.apache.org/docs/v2/sdk/HTTPTransformationPlugins.htm
>>l
>>#VIOs
>>Both helped my understanding of the transformation api's
>>
>>Regards,
>>
>>Otto
>>
>>
>>2013/1/25 Owens, Steve <Steve.Owens@disney.com>:
>>> One step closer
>>>
>>> VIO *
>>>
>>> PluginVC::do_io_write(Continuation * c, int64_t nbytes, IOBufferReader
>>>*
>>> abuffer, bool owner)
>>>
>>> {
>>>
>>>
>>>   ink_assert(!closed);
>>>
>>>   ink_assert(magic == PLUGIN_VC_MAGIC_ALIVE);
>>>
>>>
>>>   if (abuffer) {
>>>
>>>     ink_assert(!owner);
>>>
>>>     write_state.vio.buffer.reader_for(abuffer);
>>>
>>>   } else {
>>>
>>>     write_state.vio.buffer.clear();
>>>
>>>   }
>>>
>>>
>>>   // Note: we set vio.op last because process_write_side looks at it to
>>>
>>>   //  tell if the VConnection is active.
>>>
>>>   write_state.vio.mutex = c->mutex;
>>>
>>>   write_state.vio._cont = c;
>>>
>>>   write_state.vio.nbytes = nbytes;
>>>
>>>   write_state.vio.ndone = 0;
>>>
>>>   write_state.vio.vc_server = (VConnection *) this;
>>>
>>>   write_state.vio.op = VIO::WRITE;
>>>
>>>
>>>   Debug("pvc", "[%u] %s: do_io_write for %"PRId64" bytes", PVC_ID,
>>>PVC_TYPE,
>>> nbytes);
>>>
>>>
>>>   // Since reentrant callbacks are not allowed on from do_io
>>>
>>>   //   functions schedule ourselves get on a different stack
>>>
>>>   need_write_process = true;
>>>
>>>   setup_event_cb(0, &sm_lock_retry_event);
>>>
>>>
>>>   return &write_state.vio;
>>>
>>> }
>>>
>>>
>>> It would appear that the value passed in to the third argument of
>>> TSVConnWrite ends up being assigned to write_state.vio.nbytes
>>>
>>> the docs on write_state.vio.nbytes state:
>>>
>>> /**
>>>
>>>     Number of bytes to be done for this operation.
>>>
>>>
>>>     The total number of bytes this operation must complete.
>>>
>>>
>>>   */
>>>
>>>   int64_t nbytes;
>>>
>>>
>>> So far clear as mud.  Apologies in advance for my mental density.
>>>
>>>
>>>
>>> From: "Owens, Steve" <Steve.Owens@disney.com>
>>> Reply-To: "users@trafficserver.apache.org"
>>><users@trafficserver.apache.org>
>>> Date: Fri, 25 Jan 2013 11:09:08 -0800
>>> To: "users@trafficserver.apache.org" <users@trafficserver.apache.org>
>>> Subject: Re: Api Questions
>>>
>>> An addendum to the previous message I have resorted to reading the
>>>traffic
>>> server code in order to try to discern what it does:
>>>
>>> I am examining
>>>
>>> TSVIO
>>>
>>> TSVConnWrite(TSVConn connp, TSCont contp, TSIOBufferReader readerp,
>>>int64_t
>>> nbytes)
>>>
>>> {
>>>
>>>   sdk_assert(sdk_sanity_check_iocore_structure(connp) == TS_SUCCESS);
>>>
>>>   sdk_assert(sdk_sanity_check_iocore_structure(contp) == TS_SUCCESS);
>>>
>>>   sdk_assert(sdk_sanity_check_iocore_structure(readerp) == TS_SUCCESS);
>>>
>>>   sdk_assert(nbytes >= 0);
>>>
>>>
>>>   FORCE_PLUGIN_MUTEX(contp);
>>>
>>>   VConnection *vc = (VConnection *) connp;
>>>
>>>
>>>   return reinterpret_cast<TSVIO>(vc->do_io_write((INKContInternal *)
>>>contp,
>>> nbytes, (IOBufferReader *) readerp));
>>>
>>> }
>>>
>>>
>>> It would appear that the answer to what this method does lies in what
>>> happens in the function "vc->do_io_write" yet when I search the
>>> trafficserver 3.2.0 codebase for any file that actually contains an
>>> implementation of "vc->do_io_write" I find that "do_io_write" is
>>>declared in
>>> several .h files but not defined in any .c or .cpp files.
>>>
>>>
>>> Does anyone know where in the code base the implementation of
>>>"do_io_write"
>>> resides?
>>>
>>>
>>>
>>>
>>> From: "Owens, Steve" <Steve.Owens@disney.com>
>>> Reply-To: "users@trafficserver.apache.org"
>>><users@trafficserver.apache.org>
>>> Date: Fri, 25 Jan 2013 10:48:11 -0800
>>> To: "users@trafficserver.apache.org" <users@trafficserver.apache.org>
>>> Subject: Api Questions
>>>
>>> I am trying to understand the relationship between the following
>>>functions
>>> as they are used within a transformation plugin and so far no matter
>>>how
>>> hard I dig the mystery remains.  I am hoping that people who have a
>>>better
>>> understanding of ATS than I do would be willing to help clear up any
>>> misunderstandings I have.
>>>
>>> Here are the functions I find confusing:
>>>
>>> TSVIONBytesSet
>>>
>>> (TSIOBufferCopy and or TSIOBufferWrite)
>>>
>>> And finally
>>>
>>> TSVConnWrite
>>>
>>>
>>> Each of the above functions takes an int_64 argument at the end to
>>>indicate
>>> the number of bytes but I don't fully understand the purpose of this
>>> parameter with respect to each of the functions.
>>>
>>>
>>> Suppose I have a transformation which is intended to write a prefix
>>>spew the
>>> response and then write a suffix to the down stream I would imagine it
>>>looks
>>> something like this:
>>>
>>>
>>>
>>>
>>> void handleTransformOperation(TSCont contp) {
>>>
>>> TSIOBuffer buf_test;
>>>
>>> int_64 written = 0;
>>>
>>> TransormationData* pData = (TransformationData*) TSContDataGet(contp);
>>>
>>> TSVIO input_vio = TSTransformOutputVConnGet(contp);
>>>
>>>
>>> if(data->state == STATE_BEGIN) {
>>>
>>>     // Initialize the output_buffer and output_reader
>>>
>>>     data->output_buffer = TSIOBufferCreate();
>>>
>>>     data->output_reader = TSIOBufferReaderAlloc(data->output_buffer);
>>>
>>>
>>>
>>>    // Write out a prefix
>>>
>>>    written += TSIOBufferWrite(data->output_buffer, "{Hello I am a
>>>prefix}",
>>> strlen(prefix) * sizeof(char));
>>>
>>>    data->state == STATE_MIDDLE;
>>>
>>>
>>>    // Figure out if the upstream wants to stop receiving WRITEREADY and
>>> WRITE_COMPLETE events
>>>
>>>    buf_test = TSVIOBufferGet(input_vio);
>>>
>>>    if (!buf_test) {
>>>
>>>        int64_t doneget = TSVIONDoneGet(input_vio);
>>>
>>>        TSVIONBytesSet(data->output_vio, /*What goes here doneget or
>>>written
>>> + doneget or just written?  Bonus question: why? /* ???);
>>>
>>>        data->state = STATE_END;
>>>
>>>    } else
>>>
>>>        data->state = STATE_MIDDLE;
>>>
>>> }
>>>
>>>
>>> // Notice that the above if falls through to the next state so it is
>>> possible that the following code may or may not
>>>
>>> // get executed based on the buf_test result above.
>>>
>>> if(data->state == STATE_MIDDLE) {
>>>
>>>    // Figure out if the upstream wants to stop receiving WRITEREADY and
>>> WRITE_COMPLETE events
>>>
>>>    buf_test = TSVIOBufferGet(input_vio);
>>>
>>>    if (!buf_test) {
>>>
>>>        int64_t doneget = TSVIONDoneGet(input_vio);
>>>
>>>        TSVIONBytesSet(data->output_vio, /*What goes here doneget or
>>>written
>>> + doneget or just written?  Bonus question: why? /* ???);
>>>
>>>        data->state = STATE_END;
>>>
>>>    } else {
>>>
>>>        // Figure out how much data is left to read from the upstream
>>>
>>>        int64_t upstreamBytesRemaining = TSVIONTodoGet(input_vio);
>>>
>>>        int64_t upstreamBytesAvail =
>>> TSIOBufferReaderAvail(TSVIOReaderGet(input_vio));
>>>
>>>        if(upstreamBytesRemaining > 0) {
>>>
>>>             int64_t bytesToWrite = upstreamBytesRemaining;
>>>
>>>             if(bytesToWrite > upstreamBytesAvail)
>>>
>>>                  bytesToWrite = upstreamBytesAvail;
>>>
>>>
>>>
>>>             /* Copy the data from the read buffer to the output buffer.
>>>*/
>>>
>>>     TSIOBufferCopy(TSVIOBufferGet(data->output_vio),
>>> TSVIOReaderGet(input_vio), bytesToWrite, 0);
>>>
>>>
>>>            /* Tell the read buffer that we have read the data and are
>>>no
>>>
>>>     * longer interested in it.
>>>
>>>     */
>>>
>>>     TSIOBufferReaderConsume(TSVIOReaderGet(input_vio), bytesToWrite);
>>>
>>>
>>>            /* Modify the input VIO to reflect how much data we've
>>>
>>>     * completed.
>>>
>>>     */
>>>
>>>            int64_t doneget = TSVIONDoneGet(input_vio);
>>>
>>>    TSVIONDoneSet(input_vio, /* This is where things get really muddy
>>>what
>>> goes here?  doneget or doneget + bytesToWrite or doneget + bytesToWrite
>>>+
>>> written */ ??);
>>>
>>>            /* What is the impact of calling this later in the handler
>>>if it
>>> has already been called here?
>>>
>>>        }
>>>
>>>
>>>        /* Now we check the input VIO to see if there is data left to
>>>
>>> * read.
>>>
>>> */
>>>
>>>         upstreamBytesRemaining = TSVIONTodoGet(input_vio);
>>>
>>> if (upstreamBytesRemaining > 0) {
>>>
>>>     if (upstreamBytesAvail > 0) {
>>>
>>>       /* If there is data left to read, then we reenable the output
>>>
>>>        * connection by reenabling the output VIO. This will wake up
>>>
>>>        * the output connection and allow it to consume data from the
>>>
>>>        * output buffer.
>>>
>>>        */
>>>
>>>        TSVIOReenable(data->output_vio);
>>>
>>>
>>>        /* Call back the input VIO continuation to let it know that we
>>>
>>>         * are ready for more data.
>>>
>>>         */
>>>
>>>        TSContCall(TSVIOContGet(input_vio),
>>>
>>> TS_EVENT_VCONN_WRITE_READY, input_vio);
>>>
>>>     }
>>>
>>> } else {
>>>
>>>     /* There is no data left to read so set state to STATE_END and */
>>>
>>>     data->state = STATE_END;
>>>
>>> }
>>>
>>>    }
>>>
>>>
>>>    // The above can fall through to this if block
>>>
>>>    if(data->state == STATE_END) {
>>>
>>>        /* Spew a suffix */
>>>
>>>        const char* suffix = "{i am a suffix}";
>>>
>>>        int64_t localWritten = TSIOBufferWrite(data->output_buffer,
>>>suffix,
>>> (strlen(suffix) - 1) * sizeof(char));
>>>
>>>
>>>
>>>        int64_t doneget = TSVIONDoneGet(input_vio);
>>>
>>>        TSVIONBytesSet(data->output_vio, /* What goes here? Why? */??);
>>>
>>>        TSVIOReenable(data->output_vio);
>>>
>>>
>>>       /* Call back the input VIO continuation to let it know that we
>>>
>>>        * have completed the write operation.
>>>
>>>        */
>>>
>>>        TSContCall(TSVIOContGet(input_vio),
>>>TS_EVENT_VCONN_WRITE_COMPLETE,
>>> input_vio);
>>>
>>>    }
>>>
>>>
>>>    /* I am presuming we only want to call this once per invocation of
>>> handlerOperation  But I am not sure */
>>>
>>>   data->output_vio = TSVConnWrite(output_conn, contp,
>>>data->output_reader,
>>> /* What to put here hmmm and why? */ ?? );
>>>
>>> }
>>>
>>>
>>>
>>> I really want to understand how these methods relate and what they do.
>>>I
>>> would be willing to help update/contribute to the ATS docs but first I
>>>would
>>> need to understand what they do and so far the examples that are
>>>available
>>> have not made it clear to me what is going on as yet so any help would
>>>be
>>> appreciated.
>>>
>>>
>>>
>>>
>


Mime
View raw message