trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaun mcginnity <shaun.mcginn...@gmail.com>
Subject Re: API Question
Date Fri, 15 Mar 2013 17:35:01 GMT
Yes, it allows the VIO to know when it is done. We only set it when we have
read all the data from the input. I suspect if you continually set it the
VIO might think it is done and send a WRITE_COMPLETE but you actually still
have more to write.

Shaun


On Fri, Mar 15, 2013 at 5:20 PM, Owens, Steve <Steve.Owens@disney.com>wrote:

> Shaun,
>
> Looking at the code for TSVIONBytesSet we have
>
> *void*281 <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#281>TSVIONBytesSet
<http://code.metager.de/source/s?refs=TSVIONBytesSet&project=apache>(TSVIO <http://code.metager.de/source/s?defs=TSVIO&project=apache>
viop <http://code.metager.de/source/s?refs=viop&project=apache>, int64_t <http://code.metager.de/source/s?defs=int64_t&project=apache>
nbytes <http://code.metager.de/source/s?refs=nbytes&project=apache>)282 <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#282>{283
<http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#283>
 sdk_assert <http://code.metager.de/source/s?defs=sdk_assert&project=apache>(sdk_sanity_check_iocore_structure
<http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#sdk_sanity_check_iocore_structure>(viop
<http://code.metager.de/source/s?defs=viop&project=apache>) == TS_SUCCESS <http://code.metager.de/source/s?defs=TS_SUCCESS&project=apache>);284
<http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#284>
 sdk_assert <http://code.metager.de/source/s?defs=sdk_assert&project=apache>(nbytes
<http://code.metager.de/source/s?defs=nbytes&project=apache> >= 0);285 <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#285>286
<http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#286>
 VIO <http://code.metager.de/source/s?defs=VIO&project=apache> *vio <http://code.metager.de/source/s?refs=vio&project=apache>
= (VIO <http://code.metager.de/source/s?defs=VIO&project=apache> *)viop <http://code.metager.de/source/s?defs=viop&project=apache>;287
<http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#287>
 vio <http://code.metager.de/source/s?defs=vio&project=apache>->nbytes <http://code.metager.de/source/s?defs=nbytes&project=apache>
= nbytes <http://code.metager.de/source/s?defs=nbytes&project=apache>;288 <http://code.metager.de/source/xref/apache/trafficserver/trafficserver/proxy/InkIOCoreAPI.cc#288>}
>
> It looks like this method updates the nbytes of the VIO.  But it doesn't explain what
the implications of changing this value are.  I am presuming that when I called TSVConnWrite(output_conn,
contp, data->output_reader, INT64_MAX);  That the value of vio->nbytes got set to INT64_MAX.
 Since my transform may or may not know how much upstream data it needs to process I don't
see any way for it to update this value to anything meaningful and it makes me wonder why
I would want to.
>
> In my STATE_EXIT processing I am calling: TSContCall(TSVIOContGet(input_vio), TS_EVENT_VCONN_WRITE_COMPLETE,
input_vio); But I don't call: TSVConnShutdown(TSTransformOutputVConnGet(contp), 0, 1); until
I receive the TS_EVENT_VCONN_WRITE_COMPLETE event in my callbackHandler.
>
> What would happen if I continuously update TSVIONBytesSet(output_vio, total_written)
to indicate how much I have written thus far?
> Steve Owens
>
>
> From: Shaun mcginnity <shaun.mcginnity@gmail.com>
> Reply-To: "users@trafficserver.apache.org" <users@trafficserver.apache.org
> >
> Date: Fri, 15 Mar 2013 09:48:05 -0700
>
> To: "users@trafficserver.apache.org" <users@trafficserver.apache.org>
> Subject: Re: API Question
>
> Hi Steve,
>
> I believe that in a transform you should only call TSVConnWrite once.
>
> You say you are shutting down the VIOs. Are you doing this when you
> receive a WRITE_COMPLETE event? If not you may be closing the VIO early.
> You can tell the output VIO that all the data has been written to it using
> TSVIONBytesSet(output_vio, total_written);
>
> Regards,
>
> Shaun
>
>
> On Fri, Mar 15, 2013 at 4:25 PM, Owens, Steve <Steve.Owens@disney.com>wrote:
>
>> Shaun,
>>
>> This is extremely helpful.  In fact it would make a great addition to the
>> traffic server on line documentation.  Personally I prefer to understand
>> what the methods I am calling do rather than not.
>>
>> The one question that remains unanswered is whether or not this method
>> needs to be called more than once.
>>
>> The reason for my question is this:
>>
>> I have a transform which essentially prepends and postpends content to
>> the upstream content being served.  It works in states
>>
>> STATE_START = write the prefix content downstream and shift to
>> STATE_MIDDLE.
>>         STATE_MIDDLE = copy upstream content to downstream until there is
>> no more content and then shift to STATE_END.
>>         STATE_END = write the suffix content downstream and shift to
>> STATE_EXIT.
>>         STATE_EXIT = clean up the transaction and shut down the VIO's
>> because were done.
>>
>> This plugin works with smaller upstream content as expected.  However
>> when the upstream content gets larger than about 30K, what I am seeing is
>> that from the transform plugin's perspective it is working as expected.
>>  The expected amount of data is being copied downstream and the suffix is
>> being written.  I can tell this because I am logging the values of
>> upstream_avail, upstream_todo, bytesWritten etc.  And everything adds up to
>> all content being delivered.  However the client is receiving truncated
>> content and I am baffled as to why this my be happening.  I thought that
>> maybe I was not properly using the TSVConnWrite method but from your
>> explanation of the method below, it would seem that this is possibly not
>> the case.
>>
>> At any rate,
>>
>> Thank you for the explanation.
>>
>> Best Regards,
>>
>> Steve Owens
>>
>> From: Shaun mcginnity <shaun.mcginnity@gmail.com>
>> Reply-To: "users@trafficserver.apache.org" <
>> users@trafficserver.apache.org>
>> Date: Fri, 15 Mar 2013 07:09:12 -0700
>> To: "users@trafficserver.apache.org" <users@trafficserver.apache.org>
>> Subject: Re: API Question
>>
>> Hi Steve,
>>
>> TSVConnWrite initiates a write operation on the transform's downstream
>> VConnection. The last parameter is the total amount of data you are going
>> to write. INT64_MAX is effectively unlimited.
>>
>> When you have data to write you get the VIO's buffer
>> (TSVIOBufferGet(output_vio)), and copy the data into it. Then the VIO is
>> reenabled. Reenabling the VIO causes an IMMEDIATE event to be sent to the
>> VIO's VConnection. When the VConnection's event handler has finished with
>> the data it will call your continuation back. If it expects you to send
>> more data it will pass a WRITE_READY event. If you have given the VIO the
>> total amount of data as set in TSVConnWrite it will call you back with a
>> WRITE_COMPLETE event. Thisbehaviour is the same as what the transform
>> should do to its input VIO.
>>
>> Since the downstream VConnection is the client and / or cache, if you do
>> a finite write, i.e. not INT64_MAX, then the HTTP transaction will close
>> when the limit is reached. Issuing another write in this case will probably
>> not work. You can do multiple writes on a NetVConnection though, e.g. as
>> provided by TSNetConnect.
>>
>> Hope that helps.
>>
>> Shaun
>>
>>
>>
>> -----Original Message-----
>>
>> From: James Peach [mailto:jpeach@apache.org <jpeach@apache.org>]
>>
>> Sent: Friday, March 15, 2013 3:12 AM
>>
>> To: users@trafficserver.apache.org
>>
>> Subject: Re: API Question
>>
>>
>>
>> On 14/03/2013, at 5:13 PM, "Owens, Steve" <Steve.Owens@disney.com> wrote:
>>
>>
>>
>> > In looking at the following documentation:
>> http://trafficserver.apache.org/docs/trunk/sdk/io-guide/transformations.en.html
>>
>> >
>>
>> > It would seem that the only mention of TSVConnWrite is to a single
>> parameter method.
>>
>> >
>>
>> > Yet in my plugin I am using
>>
>> >
>>
>> > data->output_vio = TSVConnWrite(output_conn, contp, data->output_reader,
>>
>> >                                 INT64_MAX);
>>
>> >
>>
>> > What has changed about this method?
>>
>> >
>>
>> > What does it actually do?
>>
>>
>>
>> It schedules data to be written to the VConnection.
>>
>>
>>
>> >
>>
>> > Do I need to call it every time the plugin is re-enabled?  Or can I
>> just call it once?
>>
>>
>>
>> I don't know that I really understand the question, but if the length is
>> INT64_MAX you would only call it once. If you want you cancall it a
>> multiple times with shorter lengths.
>>
>>
>>
>> J
>>
>
>

Mime
View raw message