thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phillip B Oldham <phillip.old...@gmail.com>
Subject Re: Short question re. threading with a Python server
Date Fri, 06 Aug 2010 13:08:21 GMT
Seems my error was due to threading, specifically with the MySQL
client. I had to alter the handler class so that it extended
`threading.local` otherwise a MySQL query would block the whole
process, causing error responses for other clients.

On Tue, Aug 3, 2010 at 2:41 PM, Phillip B Oldham
<phillip.oldham@gmail.com> wrote:
> Thanks Mark, that's really helpful. We're seeing a very odd bottleneck
> and thought it could've been down to issues with threading in our
> Python server. We've no shared state except what is persisted through
> memcached and MySQL, so it mustn't be that. I'll continue to dig
> around.
>
> Cheers!
>
> On Mon, Aug 2, 2010 at 8:57 PM, Mark Slee <mslee@facebook.com> wrote:
>> Generally, no. Python's threading.local is a specific way of offering thread-local
storage.
>>
>> You should NOT use threading.local if you're trying to safely share data between
threads. That's not what it does -- rather it actually exposes a *different* copy of the data
to each thread.
>>
>> So, you should use it if you actually want N copies of some data (where N is the
number of threads), but you want a simple interface to refer to them all by the same name
(with the clear understanding that you are actually accessing N different pieces of data).
>>
>> There are certainly situations in which this can be useful in a threaded Thrift server,
but it's by no means a general rule.
>>
>> -----Original Message-----
>> From: Phillip B Oldham [mailto:phillip.oldham@gmail.com]
>> Sent: Friday, July 30, 2010 2:39 AM
>> To: thrift-user
>> Subject: Short question re. threading with a Python server
>>
>> Should Python classes that are run via the
>> TThreaded/TThreadPool/TNonBlocking server inherit from
>> `threading.local`, in order to be thread safe?
>>
>> --
>> Phillip B Oldham
>> phillip.oldham@gmail.com
>> +44 (0) 7525 01 09 01
>>
>
>
>
> --
> Phillip B Oldham
> phillip.oldham@gmail.com
> +44 (0) 7525 01 09 01
>



-- 
Phillip B Oldham
phillip.oldham@gmail.com
+44 (0) 7525 01 09 01

Mime
View raw message