www-modproxy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kwindla Hultman Kramer <kwin...@allafrica.com>
Subject [PATCH][1.3] Header directives
Date Thu, 03 May 2001 16:29:48 GMT


Chuck Murcko forwarded my message about a patch to add Header
set/unset/add/append and Expires/Cache-Control directives yesterday
(thanks, Chuck). I thought that I should follow up with a short
explanation of the rationale for this functionality.

But first, since this is my initial post to this list, I want to say
thank you to everyone here. Mod_proxy is such a nice thing to have as
part of Apache, and I am tremendously impressed by all the development
work going into the 2.0 rewrite (I read through the list archives,
last night.) We depend on mod_proxy as a caching front-end to our
public web servers (http://allafrica.com -- we distribute news and
information from 85 or so African publishers). We serve about 300,000
dynamically-built pages a day (50 request/second at peak times,
including image files) using just a couple of dual-processor Linux
boxen, one for mod_perl and one for mod_proxy!

As our traffic has expanded, we've tried to offload more and more of
the work onto the mod_proxy box. We needed to do two things that
mod_proxy didn't support: 1) send some "connection" information to the
mod_perl backend based on request host/uri, 2) fine-tune mod_proxy
caching, so that the "public" information about expiration times of
documents could differ from the "private" caching information that
mod_proxy uses.

1) involved adding a directive:

 ProxyRequestHeader [set|unset|add|append] Header [value] [match-pattern]

This directive allows the manipulation of a header, with an optional
pattern to match against the request uri controlling application of
the directive. A simple example of use would be:

 ServerName          en.allafrica.com
 ProxyRequestHeader  set  Req-Language  "en"

While I was in the code, I also added a symmetrical directive applying
to responses rather than requests:

 ProxyResponseHeader [set|unset|add|append] Header [value] [pattern]

This works exactly the same way. We don't use it, yet, in our
production code, but I've been playing with it as a way to quickly set
headers that give hints to some of our commercial, non-browser clients
doing feed-retrieval via HTTP (we do bulk commercial feeds for
Lexis-Nexis, Comtex, ISyndicate, etc.).

2) involved a pair of directives:

 ProxyResponseExpiresVector [seconds] [match-pattern]
 CacheFreshenDate           On|Off    [match-pattern]

We wanted to try to spread out some of the requests for pages that,
perhaps because of downstream caching, were clumping together. We also
needed to be able to cache documents inside mod_proxy, but discourage
caching downstream, for some parts of our site (because we are
contractually obligated to attempt to provide "more accurate" usage
logging). An Example of these directives in use:

 CacheFreshenDate            On        "^/publishers/dailynation"
 # daily-nation index page expires immediately
 ProxyResponseExpiresVector  0         "^/publishers/dailynation/$"
 # rest of daily-nation pages ttl 30 seconds
 ProxyResponseExpiresVector  30        "^/publishers/dailynation"


I would like to know whether this functionality would be useful to
other folks using mod_proxy, and what I can do to make it more
useful. An obvious way to increase the power of the
header-manipulation directives would be to allow some dynamic
substitutions (variables ala mod_rewrite, or something similar). Ian
has mentioned a notes-table interface, as well, which is an
interesting idea.

I've put the patches up at: 


as well as appending them, here. And my "rolled-in" documentation is




View raw message