thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "j.barrett Strausser" <j.barrett.straus...@gmail.com>
Subject Thrift Transport LifeCycle
Date Sun, 31 Jan 2016 18:56:16 GMT
What is the best practice for lifecycle of a  transport?

I'm finding that opening a transport per call is not effective. In
particular the below code seems to spend ~5 seconds opening up the
transport then ~300s per call for my actual service invocation. The 300ms
is more than acceptable.

Should I simply open the transport once per client and close at the end of
the application? If I do that an I experience transient network
disconnects, what will be client side exceptions thrown?

My client batches request so that the client is not overly chatty. Meaning
every 30s or so a message of a 1MB or so will be sent. Then approx ~2 mins
I will send a large message of size 15MB to 50MB

def make_transport_context(transport_context, rpc_type):
    # Make socket


    transport = TSSLSocket.TSSLSocket(transport_context.host,
transport_context.port,
                                          ca_certs=transport_context.ca_certs,
                                          validate=False)

    transport.setTimeout(ms=transport_context.timeout_in_millis)
    # Buffering is critical. Raw sockets are very slow
    transport = TTransport.TBufferedTransport(transport)

    # Wrap in a protocol
    protocol = TBinaryProtocol.TBinaryProtocol(transport)

    # Create a client to use the protocol encoder
    client = rpc_type.Client(protocol)
    return client, transport

@contextlib.contextmanager
def rpc_thrift_context(transport_context):
    client, transport = make_transport_context(transport_context, DataRPC)
    try:
        transport.open()
        yield client
    finally:
        transport.close()


Also asked at :
http://stackoverflow.com/questions/35117943/proper-lifecycle-of-python-thrift-client-transport
If you want those sweet SO points.

-- 


https://github.com/bearrito
@deepbearrito

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message