ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Daschinskiy (Jira)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-13967) Refactor and imrpove performance of python thin client marshaller
Date Mon, 11 Jan 2021 11:42:00 GMT
Ivan Daschinskiy created IGNITE-13967:

             Summary: Refactor and imrpove performance of python thin client marshaller
                 Key: IGNITE-13967
                 URL: https://issues.apache.org/jira/browse/IGNITE-13967
             Project: Ignite
          Issue Type: Improvement
          Components: thin client
            Reporter: Ivan Daschinskiy
            Assignee: Ivan Daschinskiy

Currently implemented serialization has questionable design and suffers from some problems
1. It is tightly coupled with Client object
2. It doesn't use protocol feature that total length of message is in the header,
thus it constantly load from Client some data instead of iteration over byte array.
3. It uses some tricky hacks and sometimes new connection is created when deserializing object.
4. It constantly allocates bytes (immutable data structure).

I suggest to rewrite serialization and deserialization:
1. Pass to corresponding methods specific SerDe context + BytesIO
2. Context can be sync and async and contains specific flags and methods for loading/uploading
binary object schemas
3. Refactor Client in order to retrieve full packet from socket at once then pass full packet

These steps can significantly improve performance, reduce amount of allocations and give
foundation for implementing asyncio version of client.

This message was sent by Atlassian Jira

View raw message