thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rush Manbert <>
Subject Re: Server acting as a proxy ?
Date Wed, 16 May 2012 15:26:40 GMT
Hi Glenn,

On May 16, 2012, at 1:49 AM, Glenn Pierce wrote:

> Hi. I am having a little trouble with a server / client design and
> wonder if anyone had any advice.
> I have a Thrift server that abstracts a data store.  The idea is that
> there will be a number of clients that are essentially
> out of process plugins that use the interface provided by the server
> to receive, manipulate the underlying data store and also provide
> their own data.
> There will be a number of other clients which simply access the data
> provided by the server and its "plugins".
> The  problem case is when one of these "plugins" wishes to provide its
> own data and provide an interface to that data.
> The server should have no knowledge of the plugins data or interface.
> I would ideally like all clients to access functionality through the
> main thrift server so it acts as a facade for the plugins. If a client
> requested some data provided by a plugin the main server could
> delegate to the plugin to provide that data. I guess this would mean
> have each plugin being a thrift client and server. I have written the
> server in python so could probably handle thrift calls that are not
> yet defined but would it be possible to forward these calls another
> thrift server IE act as a proxy ?

Assuming that you really can "handle thrift calls that are not yet defined" (which you couldn't
in C++, which is what I use), then having the main server act as a proxy for "server" plugins
is fine. We have a daemon process that presents a Thrift service API to the local system on
a Unix domain socket. The calls that clients make are sometimes handled locally, and other
times passed on to our servers that present a Thrift service interface to the internet, and
other times are handled by a combination of local and remote processing. Internally, the daemon
also uses a third Thrift service that serves a local database. IN fact, there's a fourth Thrift
server buried inside the daemon too, but all clients see on the local system is the one daemon

> An alternative is maybe have the plugins be clients only and push data
> to the server. But the format of these messages
> would unknown to the server and would have to be generic enough to
> accommodate different types of data. I not sure how I would provide a
> useful interface to this data to other clients.
> As far as I can see only the plugins knows how to store and manipulate
> the data it owns so this idea probably would not work.

The only way I know of (from direct experience) to make something like this work would be
if your real server had a RPC call that was defined something like this (written off the top
of my head - not compiled, etc.):

struct PassThroughCallParams
	1: list<byte> RealRpcStruct,

struct PassThroughReturnParams
	1: list<byte> RealRpcReturnStruct,

struct ClientPassThroughReturnParams
	1: bool wasHandled,
	2: PassThroughReturnParams actualReturnParams,

service MyService
	PassThroughReturnParams  passThroughCall (1: PassThroughCallParams),

Then your server could always field this call. It would need to know that each of its clients
has a server interface that defines a passThroughCall() that takes the same calling structure
and returns the ClientPassThroughReturnParams. The server would call this on each client until
the wasHandled struct member came back true, or the call wasn't handled.

To pull this off, you need to have a standardized way of flattening Thrift structures to buffers,
with some sort of identification. We wrote a variant on the TMemoryBuffer transport to do
this. If you get a flattened Thrift structure you can try to unflatten it into a struct type
that you expect. The flattened buffer has a checksum block and an ID in a header that catches
when things don't match up.

> Thanks for any advice. Any suggestions welcomed.
> -- 
> Regards
> Glenn P Pierce

Best regards,
View raw message