trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Peach <jpe...@apache.org>
Subject Re: Migrating from squid
Date Fri, 27 Feb 2015 16:03:48 GMT

> On Feb 27, 2015, at 7:50 AM, Leif Hedstrom <zwoop@apache.org> wrote:
> 
>> 
>> On Feb 27, 2015, at 12:51 AM, J David <j.david.lists@gmail.com> wrote:
>> 
>> (Not sure if this is best for the user list or the dev list, as it's
>> development, but not of ATS per se.)
>> 
>> Squid offers a feature called url_rewrite_program that can be used to
>> change its behavior based on client IP, requested URL, and various
>> other criteria.  It runs a bunch of copies of an external program and
>> passes information to and from them over pipes.
>> 
>> Overall, ATS is a much better fit for our environment, but we are
>> heavily dependent on this feature of Squid.
>> 
>> The short version of what we need to do is this:
>> 
>> Based on a (ClientIP,RequestURL) pair, either choose a backend (by
>> name or IP) to handle the request *or* issue an arbitrary redirect
>> *or* return an HTTP error.
>> 
>> According to the "squid config translation" docs, the equivalent for
>> url_rewrite_program is remap.config.  Since that does not actually
>> support external programs, it appears the intent is to use the
>> built-in remapping abilities to replicate the functionality of the
>> external program.  However, that is not always possible.
>> 
>> The messy part that prevents us from doing that or hacking up a module
>> is that the external program we use with squid is ridiculously
>> complicated, pulling together information from multiple sources, all
>> of them dynamic, and gets linked to all kinds of libraries like MySQL,
>> memcached, and distributed message passing stuff.
>> 
>> What is the best way to migrate this functionality to ATS?  The whole
>> big ball of wax seems like a really poor candidate for a built-in ATS
>> module due to all the external dependencies, many of which run
>> asynchronously and can block.  It seems really advantageous to seal
>> that off So what we probably really need is a pretty efficient way to
>> emulate squid's ability to call out to an external program.
> 
> 
> Dealing with external squid helpers is a bit wonky, but you probably could implement
something in a plugin that does it. The fact that you are doing so much weirdness (MySql,
Memcached) makes it particularly tough, I’m not sure how Squid deals with that?
> 
> Dealing with synchronous APIs such as MySQL is tricky to say the least.
> 
> I’m not sure we have a good answer here, other than you probably need to try to implement
this as a plugin.

You could do this as a server intercept plugin. You could exec the helper, wire up pipes to
it's standard I/O, then use TSVConnFdCreate to suck the response back into Traffic Server.

https://docs.trafficserver.apache.org/en/latest/reference/api/TSVConnFdCreate.en.html
https://github.com/apache/trafficserver/blob/master/example/intercept/intercept.cc

J
Mime
View raw message