thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garrett Smith...@rrett.us.com>
Subject Re: Python server over HTTP, HTTPS -- How?
Date Fri, 03 Apr 2009 06:30:47 GMT
----- "Brian Hammond" <brian@brianhammond.com> wrote:

> On Apr 2, 2009, at 11:26 PM, Garrett Smith wrote:

>> Unless you need an asynchronous server side framework for high
>> concurrency and low memory footprint, I would stay clear of
>> Twisted.
> 
> It turns out that I need a highly efficient server.  I'm a one-man  
> shop and am limited in the number of servers I can afford to deploy.  
> 
> I plan on starting with a bare minimum of two load-balanced VPS  
> instances so memory is tight.  I do also need high concurrency.  I'm 
> developing a turn-based game server and have a very large user base  
> already (iPhone app) and would like to license my solution to other  
> similar iPhone developers ... of course I can enlarge my cluster of  
> servers linearly with the number of licensees.  I digress...
> 
>> I think a standard threaded wsgi server would work fine.
> 
> Suggestions?  CherryPy?

For threaded wsgi apps, IMO, the two strongest options are CherryPy or
mod_wsgi for Apache worker MPM.

CherryPy is extremely fast, especially for a pure Python server. But
you have to fall back on wsgi middleware for things like auth. Apache
provides similar functionality via mods.

You can easily play around with both since your app code will likely not
change.

Unless your connections are long running -- to the point that you
*really* need to support thousands of concurrent connections (this is
very rarely the case with HTTP as you soon become CPU bound), I'd stick
with threads -- async can be a serious pain, esp if it starts leaking
into your app (coroutines, Twisted deferreds, etc.)

>> If you're inclined to use a mod_wsgi, I recommend Graham Dumpleton's
>> outstanding wsgi implementation for Apache. The Nginx wsgi
>> interface is good as well, but beware if your app needs to block -- 
>> you'll be serializing your requests.
> 
> True.  Nginx is indeed single-threaded.  I'm not leaning in any way to
> any particular serving tech. at this point actually.  I just want to 
> ensure that whatever tech. I choose is as efficient as possible.
> 
> I actually don't have any points of blocking in the front-end  
> actually, not on disk I/O at least.  My datastore is a file-backed
> key-value database that runs in a separate process and writes to disk on 
> every Nth database modification.

All of the above options are quite lean -- and it sounds like you could
get by with Nginx if you wanted to go that route.

>> Alas, message signing is another application layer measure -- it
>> would
>> be sweet to see auth work its way into the Thrift spec.
 
> Auth in Thrift would be wonderful but I wonder if that's feature
> creep?

Yes, definite is. I've been looking at AMQP and their authentication
scheme, using SASL, is quite simple and still useful. This could serve
possibly as a model for Thrift. Maybe this has been hashed out before and
duly rejected though.

Mime
View raw message