thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Abernethy ...@apache.org>
Subject Re: Thrift Transport LifeCycle
Date Sun, 31 Jan 2016 20:22:05 GMT
Connections are expensive and every transport open makes a connection. If
you can manage it, making a single connection and then issuing multiple
calls over that connection will save a considerable amount of overhead.

On Sun, Jan 31, 2016 at 10:56 AM, j.barrett Strausser <
j.barrett.strausser@gmail.com> wrote:

> What is the best practice for lifecycle of a  transport?
>
> I'm finding that opening a transport per call is not effective. In
> particular the below code seems to spend ~5 seconds opening up the
> transport then ~300s per call for my actual service invocation. The 300ms
> is more than acceptable.
>
> Should I simply open the transport once per client and close at the end of
> the application? If I do that an I experience transient network
> disconnects, what will be client side exceptions thrown?
>
> My client batches request so that the client is not overly chatty. Meaning
> every 30s or so a message of a 1MB or so will be sent. Then approx ~2 mins
> I will send a large message of size 15MB to 50MB
>
> def make_transport_context(transport_context, rpc_type):
>     # Make socket
>
>
>     transport = TSSLSocket.TSSLSocket(transport_context.host,
> transport_context.port,
>
> ca_certs=transport_context.ca_certs,
>                                           validate=False)
>
>     transport.setTimeout(ms=transport_context.timeout_in_millis)
>     # Buffering is critical. Raw sockets are very slow
>     transport = TTransport.TBufferedTransport(transport)
>
>     # Wrap in a protocol
>     protocol = TBinaryProtocol.TBinaryProtocol(transport)
>
>     # Create a client to use the protocol encoder
>     client = rpc_type.Client(protocol)
>     return client, transport
>
> @contextlib.contextmanager
> def rpc_thrift_context(transport_context):
>     client, transport = make_transport_context(transport_context, DataRPC)
>     try:
>         transport.open()
>         yield client
>     finally:
>         transport.close()
>
>
> Also asked at :
>
> http://stackoverflow.com/questions/35117943/proper-lifecycle-of-python-thrift-client-transport
> If you want those sweet SO points.
>
> --
>
>
> https://github.com/bearrito
> @deepbearrito
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message