trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Miles Libbey <mlib...@apache.org>
Subject Re: Downtime when traffic_ctl config reload
Date Fri, 11 Jan 2019 22:31:36 GMT
First thing I'd check is proxy.config.http.parent_proxy_routing_enable
on the children :)

On Fri, Jan 11, 2019 at 2:15 PM Hobin Yoon <hobinyoon@gmail.com> wrote:
>
> It is disabled. I just tried with both on and off, but none of them worked. What bothers
me is ATS doesn't seem to be reading parent.config at all since I don't get any error when
I intentionally made a typo in parent.config.
>
> Hobin
>
> On Fri, Jan 11, 2019 at 4:38 PM Leif Hedstrom <zwoop@apache.org> wrote:
>>
>>
>>
>> On Jan 11, 2019, at 12:54 PM, Hobin Yoon <hobinyoon@gmail.com> wrote:
>>
>> Ok, I added the rule in remap.config. Now, the requests are going to "origin". But,
they bypasses parent nodes; I don't see any requests on either of the parent nodes.
>>
>> Here is my system setup for test. "child" is a pure load balancer and doesn't cache
any data items. parent1 and parent2 do cache.
>> Client (web browser) - child (ATS)--- parent1 (ATS) --- origin (Apache2)
>>                                    └- parent2 (ATS) -┘
>>
>> Here is the configuration in "child" node.
>>
>> remap.config
>> map / http://origin
>>
>> parent.config
>> dest_domain=origin scheme=http parent="parent1:80;parent2:80" round_robin=consistent_hash
>>
>>
>>
>> Do you have “pristine” host headers enabled? The mapping in parent selection
happens on the “remapped” URL, but I’m pretty sure it’d be different if you have pristine
host headers disabled.
>>
>> That much said, I’d run with a diagnostics (-T) on “parent”.
>>
>> — Leif
>>
>>
>>
>> records.config
>> CONFIG proxy.config.http.parent_proxy_routing_enable INT 1
>>
>> Hobin
>>
>> On Fri, Jan 11, 2019 at 1:08 PM Leif Hedstrom <zwoop@apache.org> wrote:
>>>
>>>
>>>
>>> On Jan 11, 2019, at 10:00 AM, Hobin Yoon <hobinyoon@gmail.com> wrote:
>>>
>>> Should you still specify the mapping in remap.config when you have the mapping
in parent.config?
>>>
>>>
>>> Yes. Unless you run an open proxy, and then you have to turn off “require remap”.
Not recommended for almost all cases.
>>>
>>> — leif
>>>
>>>
>>>
>>> On Fri, Jan 11, 2019 at 11:33 AM Randall Meyer <randallmeyer@yahoo.com>
wrote:
>>>>
>>>>
>>>> Looking at your remap.config in that repo, I see that all the mappings are
all commented out. Was that intentional?
>>>> On Thursday, January 10, 2019, 8:28:53 PM PST, Hobin Yoon <hobinyoon@gmail.com>
wrote:
>>>>
>>>>
>>>> Yes, they are also ATS nodes and they worked fine when requests were made
to them directly.
>>>>
>>>> I uploaded the config files here, if you could take a look: https://gitlab.com/hobinyoon/trafficserver-config
>>>>
>>>> Hobin
>>>>
>>>>
>>>>
>>>> On Thu, Jan 10, 2019 at 7:31 PM Miles Libbey <mlibbey@apache.org> wrote:
>>>>
>>>> Typically that error means the request doesn't match a rule in
>>>> remap.config. Is the error coming from the first hop or one of the
>>>> 123.123.123.[1-4] nodes? That is, if 123.123.123.[1-4] are ATS nodes,
>>>> are they configured to accept the requests they are getting?
>>>>
>>>> On Thu, Jan 10, 2019 at 2:51 PM Hobin Yoon <hobinyoon@gmail.com> wrote:
>>>> >
>>>> > With parent.config
>>>> >   dest_domain=. scheme=http parent="123.123.123.1:80,123.123.123.2:80"
round_robin=consistent_hash go_direct=false
>>>> >
>>>> > I'm getting
>>>> >
>>>> > Not Found on Accelerator
>>>> > ________________________________
>>>> > Description: Your request on the specified host was not found. Check
the location and try again.
>>>> >
>>>> > I must be missing something ...
>>>> >
>>>> > Hobin
>>>> >
>>>> > On Thu, Jan 10, 2019 at 1:18 PM Hobin Yoon <hobinyoon@gmail.com>
wrote:
>>>> >>
>>>> >> Alan, that is the only map rule we have with the varying number
of cache nodes. During the down time, ATS doesn't return "HTTP/1.1 200 OK" for the requests.
I didn't check what it returned.
>>>> >>
>>>> >> Miles, I'll check out parent plugin!
>>>> >>
>>>> >> Hobin
>>>> >>
>>>> >> On Thu, Jan 10, 2019 at 1:13 PM Alan Carroll <solidwallofcode@oath.com>
wrote:
>>>> >>>
>>>> >>> It could be an artifact of reloading plugin configurations if
you have a lot of remap rules with plugins, although internally ATS should do the load and
then swap the configuration. During the down time, does ATS process any traffic, or there
is traffic but no caching?
>>>> >>>
>>>> >>> On Thu, Jan 10, 2019 at 10:53 AM Miles Libbey <mlibbey@apache.org>
wrote:
>>>> >>>>
>>>> >>>> We don't experience downtime when using traffic_ctl config
reload (we
>>>> >>>> use that ~daily).
>>>> >>>>
>>>> >>>> We don't use the balancer plugin. Instead, we use parent.config
>>>> >>>> (https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/files/parent.config.en.html)
>>>> >>>> to achieve the same consistent hash. Your config would translate
to
>>>> >>>>
>>>> >>>> remap.config
>>>> >>>> map / http://127.0.0.1 @plugin=cachekey.so
>>>> >>>> @pparam=--include-params=p0,p1 @pparam=--sort-params=true
>>>> >>>>
>>>> >>>> parent.config
>>>> >>>> dest_domain=. scheme=http
>>>> >>>> parent="123.123.123.1:80,23.123.123.2:80,123.123.123.3:80,123.123.123.4:80"
>>>> >>>> round_robin=consistent_hash go_direct=false
>>>> >>>> dest_domain=. scheme=https
>>>> >>>> parent="123.123.123.1:443,23.123.123.2:443,123.123.123.3:443,123.123.123.4:443"
>>>> >>>> round_robin=consistent_hash go_direct=false
>>>> >>>>
>>>> >>>> miles
>>>> >>>>
>>>> >>>> On Wed, Jan 9, 2019 at 10:53 PM Hobin Yoon <hobinyoon@gmail.com>
wrote:
>>>> >>>> >
>>>> >>>> > Hi,
>>>> >>>> >
>>>> >>>> > We are noticing there is quite a bit of delay when
we reload the config with traffic_ctl config reload. The delay is up to about 30 seconds,
during which period we don't get any caching. We are using consistency hashing plugin. The
number of nodes changes dynamically between 5 to 30.
>>>> >>>> >
>>>> >>>> > Here is an example balancer (consistent hash) configuration
in remap.config.
>>>> >>>> >
>>>> >>>> > map / http://127.0.0.1 @plugin=cachekey.so @pparam=--include-params=p0,p1
@pparam=--sort-params=true @plugin=balancer.so @pparam=--policy=hash,key @pparam=123.123.123.1
@pparam=123.123.123.2 @pparam=123.123.123.3 @pparam=123.123.123.4 ...
>>>> >>>> >
>>>> >>>> > Is this downtime normal? How do you guys avoid the
service downtime while reconfiguring the cache nodes in the cluster?
>>>> >>>> >
>>>> >>>> > Hobin
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> Beware the fisherman who's casting out his line in to a dried
up riverbed.
>>>> >>> Oh don't try to tell him 'cause he won't believe. Throw some
bread to the ducks instead.
>>>> >>> It's easier that way. - Genesis : Duke : VI 25-28
>>>
>>>
>>

Mime
View raw message