trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randeep <randeep...@gmail.com>
Subject Re: Query strings are not forwarded
Date Wed, 02 Nov 2016 12:59:20 GMT
Hi,

Its true. I started using this plugin for removing the query strings from
the requests. But when I checked the documentation, I found that the same
plugin can be used to whitelist user-agents also.

https://docs.trafficserver.apache.org/en/6.2.x/admin-guide/plugins/cachekey.en.html?highlight=cachekey

I have whitelisted only Stagefright. But for AppleWebkit also requests are
served.

cat remap.config
map http://jitp1.domain.com/ http://bglrcache1.domain.com/
@plugin=cachekey.so
@pparam=--ua-whitelist=lukupplayer:/opt/ats/etc/trafficserver/randeep.config
@pparam=--remove-all-params=true

cat /opt/ats/etc/trafficserver/randeep.config
^stagefright/1.*

@Miles Libbey
I have enabled Xdebug plugin.

[root@cachetest trafficserver]# cat plugin.config
background_fetch.so --config
/opt/ats/etc/trafficserver/background_fetch.config
*xdebug.so*
[root@cachetest trafficserver]#

[Nov  2 18:12:58.353] {0x2aafed658a40} STATUS: opened
/opt/ats/var/log/trafficserver/diags.log
[Nov  2 18:12:58.376] Server {0x2aafed658a40} NOTE: <Plugin.cc:84
(plugin_load)> loading plugin
'/opt/ats/libexec/trafficserver/background_fetch.so'
*[Nov  2 18:12:58.377] Server {0x2aafed658a40} NOTE: <Plugin.cc:84
(plugin_load)> loading plugin '/opt/ats/libexec/trafficserver/xdebug.so'*

curl -v  "http://jitp1.domain.com/dashw/zoom/manifest.mpd" -H 'User-Agent:
AppleWebKit'  -H 'X-Debug: X-Cache-Key'
* About to connect() to jitp1.domain.com port 80 (#0)
*   Trying 172.16.0.51... connected
* Connected to jitp1.domain.com (172.16.0.51) port 80 (#0)
> GET /dashw/zoom/manifest.mpd HTTP/1.1
> Host: jitp1.domain.com
> Accept: */*
> User-Agent: AppleWebKit
> X-Debug: X-Cache-Key
>
< HTTP/1.1 200 OK
< Content-Length: 2084
< Content-Type: application/dash+xml
< Access-Control-Allow-Origin: *
< Cache-Control: max-age=5
< Date: Wed, 02 Nov 2016 12:43:23 GMT
< Age: 22
< Server: ATS/6.2.0
< Connection: keep-alive
*< X-Cache-Key: /jitp1.domain.com/80/dashw/zoom/manifest.mpd
<http://jitp1.domain.com/80/dashw/zoom/manifest.mpd>*
<
<?xml version="1.0" encoding="UTF-8"?><MPD
*** Data Truncated***

Can we get any specific log for cachekey whitelist check?

Regards,
Randeep

On Fri, Oct 28, 2016 at 10:23 PM, Miles Libbey <mlibbey@apache.org> wrote:

> On Fri, Oct 28, 2016 at 6:04 AM, Sudheer Vinukonda
> <sudheervinukonda@yahoo.com.invalid> wrote:
> > Not sure to follow - what exactly are you trying to do? Afaik, cachekey
> plugin only manipulates the otherwise default cache key (which is the full
> request URL) used against the configured cache object(s) and modifies it as
> specified.
> >
> > I don't believe it can "block" any requests or user-agents.
> >
> >> On Oct 28, 2016, at 4:17 AM, Randeep <randeep123@gmail.com> wrote:
> >>
> >> Hi Sudheer,
> >>
> >> Sorry. I had added it. While pasting here somehow it was not selected. I
> >> was was checking the whitelisting of a user agent. Query string was
> >> working.
> >>
> >> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> >> @plugin=cachekey.so
> >> @pparam=--ua-whitelist=player:/opt/ats/etc/trafficserver/randeep.config
> >> @pparam=--ua-capture=(stagefright\/[^\s]*).*
> >> @pparam=--remove-all-params=true
> >>
> >> Still its not working. It is not blocking the other user agents.
> >>
> >> [root@cachetest ~]# curl -v  "
> >> http://jitp1.domain.com/dashw/zoom/manifest.mpd" -H 'User-Agent:
> >> AppleWebKit'  -H 'X-Debug: X-Cache-Key'
> >> * About to connect() to jitp1.domain.com port 80 (#0)
> >> *   Trying 172.16.0.51... connected
> >> * Connected to jitp1.domain.com (172.16.0.51) port 80 (#0)
> >>> GET /dashw/zoom/manifest.mpd HTTP/1.1
> >>> Host: jitp1.domain.com
> >>> Accept: */*
> >>> User-Agent: AppleWebKit
> >>> X-Debug: X-Cache-Key
>
> It doesn't look like you have the X-Debug plugin enabled, so the
> Cache-Key isn't being printed here, making it hard to figure out.
> Could you enable it?
>
>
> >>>
> >> < HTTP/1.1 200 OK
> >> < Content-Length: 2082
> >> < Content-Type: application/dash+xml
> >> < Access-Control-Allow-Origin: *
> >> < Cache-Control: max-age=5
> >> < Date: Fri, 28 Oct 2016 11:14:31 GMT
> >> < Age: 19
> >> < Server: ATS/6.2.0
> >> < Connection: keep-alive
> >> <
> >> <?xml version="1.0" encoding="UTF-8"?><MPD xmlns:xsi="
> >> http://www.w3.org/2001/XMLSchema-instance"
> >> xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:cenc="urn:mpeg:cenc:2013"
> >> ***Truncated***
> >>
> >> Regards,
> >> Randeep
> >>
> >> On Fri, Oct 28, 2016 at 4:34 PM, Sudheer Vinukonda <
> >> sudheervinukonda@yahoo.com> wrote:
> >>
> >>> Hmm...where's the pparam to strip query string?
> >>>
> >>> Please make sure to read through the documentation on cachekey.
> >>> Specifically, the query param section you are looking for and the
> examples.
> >>>
> >>> Here's a relevant extract -
> >>>
> >>> URI query parameters
> >>> Ignore the query string (all query parameters)
> >>>
> >>> The following added to the remap rule will ignore the query, removing
> it
> >>> from the cache key.
> >>>
> >>> @plugin=cachekey.so @pparam=--remove-all-params=true
> >>>
> >>>
> >>> On Oct 28, 2016, at 2:27 AM, Randeep <randeep123@gmail.com> wrote:
> >>>
> >>> Hi Miles Libbey,
> >>>
> >>> Thanks for pointing it out.  I changed it as below.
> >>> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> >>> @plugin=cachekey.so  @pparam=--ua-whitelist=player:
> >>> /opt/ats/etc/trafficserver/randeep.config  @pparam=--ua-capture=(
> >>> stagefright\/[^\s]*).*
> >>>
> >>> Still now luck.
> >>>
> >>> [root@cachetest ~]# curl -v  "http://jitp1.domain.com/
> >>> dashw/zoom/manifest.mpd" -H 'User-Agent: stagefright/1.2 (Linux;Android
> >>> 4.2.109)'  -H 'X-Debug: X-Cache-Key'
> >>> * About to connect() to jitp1.domain.com port 80 (#0)
> >>> *   Trying 172.16.0.51... connected
> >>> * Connected to jitp1.domain.com (172.16.0.51) port 80 (#0)
> >>>> GET /dashw/zoom/manifest.mpd HTTP/1.1
> >>>> Host: jitp1.domain.com
> >>>> Accept: */*
> >>>> User-Agent: stagefright/1.2 (Linux;Android 4.2.109)
> >>>> X-Debug: X-Cache-Key
> >>>>
> >>> < HTTP/1.1 200 OK
> >>> < Content-Length: 2082
> >>> < Content-Type: application/dash+xml
> >>> < Access-Control-Allow-Origin: *
> >>> < Cache-Control: max-age=5
> >>> < Date: Fri, 28 Oct 2016 09:23:33 GMT
> >>> < Age: 20
> >>> < Server: ATS/6.2.0
> >>> < Connection: keep-alive
> >>> <
> >>> <?xml version="1.0" encoding="UTF-8"?><MPD xmlns:xsi="
> http://www.w3.org/
> >>> 2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011"
> >>> ***Truncated****
> >>>
> >>> Regards,
> >>> Randeep
> >>>
> >>>> On Fri, Oct 28, 2016 at 1:24 PM, Miles Libbey <mlibbey@apache.org>
> wrote:
> >>>>
> >>>>> On Thu, Oct 27, 2016 at 11:41 PM, Randeep <randeep123@gmail.com>
> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I used the cachekey plugin as below. But its not working. Am I doing
> it
> >>>>> wrong?
> >>>>>
> >>>>> My remap.config
> >>>>> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> >>>>> @pparam=--ua-whitelist=lukupplayer:/opt/ats/etc/
> trafficserver/randeep.config
> >>>>> @pparam=--ua-capture=(stagefright\/[^\s]*).*
> >>>>>
> >>>>
> >>>> shouldn't you have
> >>>> @plugin=cachekey.so in there?
> >>>> eg
> >>>> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> >>>> @plugin=cachekey.so @pparam=--ua-whitelist=...
> >>>>
> >>>>
> >>>> cat /opt/ats/etc/trafficserver/randeep.config
> >>>>> ^stagefright/1.*
> >>>>>
> >>>>> I am requesting the like this
> >>>>> curl "http://jitp1.domain.com/dashw/abc/manifest.mpd" -H
> 'User-Agent:
> >>>>> stagefright/1.2 (Linux;Android 4.2.109)'  -H 'X-Debug: X-Cache-Key'
> >>>>>
> >>>>> But its not rejected!
> >>>>> [root@cachetest ~]# tail -f /opt/ats/var/log/trafficserver/squid.log
> >>>>> |grep abc
> >>>>> 1477636406.439 3 172.16.0.51 TCP_MISS/404 199 GET
> >>>>> http://bglrcache1.domain.com/dashw/abc/manifest.mpd - PARENT_HIT/
> >>>>> 172.16.0.40 text/plain
> >>>>>
> >>>>> Please guide.
> >>>>>
> >>>>> Regards,
> >>>>> Randeep
> >>>>>
> >>>>>> On Thu, Oct 27, 2016 at 5:02 PM, Randeep <randeep123@gmail.com>
> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
> >>>>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?
> starttime=1477454400000&stoptime=1477456200000> will
> >>>>>> give the live content
> >>>>>>
> >>>>>> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?startt
> >>>>>> ime=1477454400000&stoptime=1477456200000
> >>>>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?
> starttime=1477454400000&stoptime=1477456200000>
> >>>>>> will give the recorded content from the starttime to endtime.
The
> manifest
> >>>>>> file will be for the specified time.
> >>>>>>
> >>>>>> but in that case if we cache the response without query strings
for
> the
> >>>>>> above, it might conflict with the live requests!
> >>>>>>
> >>>>>> I think its better not cache the manifest url with query strings.
> Only
> >>>>>> the video chunks and manifest file without query string we will
> cache.
> >>>>>> Is that possible?
> >>>>>>
> >>>>>> Regards,
> >>>>>> Randeep
> >>>>>>
> >>>>>> On Thu, Oct 27, 2016 at 4:50 PM, Reindl Harald <
> h.reindl@thelounge.net>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Am 27.10.2016 um 13:05 schrieb Randeep:
> >>>>>>>>
> >>>>>>>> Our urls are like
> >>>>>>>>
> >>>>>>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=
> >>>>>>>> 1477454400000&stoptime=1477456200000
> >>>>>>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
> >>>>>>>> =1477454400000&stoptime=1477456200000>
> >>>>>>>>
> >>>>>>>> We need to cache only
> >>>>>>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
> >>>>>>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
> >>>>>>>> =1477454400000&stoptime=1477456200000>
> >>>>>>>>
> >>>>>>>
> >>>>>>> but *why* are your URLs than that way?
> >>>>>>>
> >>>>>>> if that params don't do anything remove them
> >>>>>>>
> >>>>>>> if you configrue your proxy to ignore params and always
deliver the
> >>>>>>> same content that will for sure break something else and
in doubt
> it will
> >>>>>>> break terrible
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Randeep
> >>>>>> Mob: +919447831699[kerala]
> >>>>>> Mob: +919880050349[B'lore]
> >>>>>> http://twitter.com/Randeeppr
> >>>>>> http://in.linkedin.com/in/randeeppr
> >>>>>>
> >>>>>>
> >>>>>> [image: --]
> >>>>>> Randeep Raman
> >>>>>> [image: http://]about.me/Randeeppr
> >>>>>> <http://about.me/Randeeppr>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Randeep
> >>>>> Mob: +919447831699[kerala]
> >>>>> Mob: +919880050349[B'lore]
> >>>>> http://twitter.com/Randeeppr
> >>>>> http://in.linkedin.com/in/randeeppr
> >>>>>
> >>>>>
> >>>>> [image: --]
> >>>>> Randeep Raman
> >>>>> [image: http://]about.me/Randeeppr
> >>>>> <http://about.me/Randeeppr>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Randeep
> >>> Mob: +919447831699[kerala]
> >>> Mob: +919880050349[B'lore]
> >>> http://twitter.com/Randeeppr
> >>> http://in.linkedin.com/in/randeeppr
> >>>
> >>>
> >>> [image: --]
> >>> Randeep Raman
> >>> [image: http://]about.me/Randeeppr
> >>> <http://about.me/Randeeppr>
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Randeep
> >> Mob: +919447831699[kerala]
> >> Mob: +919880050349[B'lore]
> >> http://twitter.com/Randeeppr
> >> http://in.linkedin.com/in/randeeppr
> >>
> >>
> >> [image: --]
> >> Randeep Raman
> >> [image: http://]about.me/Randeeppr
> >> <http://about.me/Randeeppr>
> >
>



-- 
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr


[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>

Mime
View raw message