ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yakov Zhdanov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-4646) Try to unmarshall direct messages in striped pool
Date Thu, 02 Feb 2017 10:46:51 GMT
Yakov Zhdanov created IGNITE-4646:

             Summary: Try to unmarshall direct messages in striped pool
                 Key: IGNITE-4646
                 URL: https://issues.apache.org/jira/browse/IGNITE-4646
             Project: Ignite
          Issue Type: Improvement
            Reporter: Yakov Zhdanov
             Fix For: 2.0

During marshalling in NIO thread the following should be added to the write buffer and sent
to peer:
1. chunk size - 16 bits (this probably puts limitation on max write buffer size of 64k or
will require some changes to direct writer)
2. last  chunk - 1 bit
3. pool policy - 8 bits

Here is the scheme to explain how this should work.
[chunk size] [pool policy] [partition] [last flag] [chunk data] X <-- no more space in
write buffer
[next chunk size] [last flag] [chunk data] <<-- we write next chunk once some space
is available in write buffer, but we skip partition and policy flags and maybe others that
should be sent only once.
[next chunk size] [last flag] [chunk data] <<-- last flag is true here

Write buffer - 64k
Message - 84k

# sender reserves space for chunk size
# reserves space for policy and last chunk flag
# marshalls message to buffer while it has free space (64k - SPACE will be written to buffer)
# puts size and flags to reserved space in the beginning
# sends buffer or part of it which makes some space available to further writes
# reserves space for next chunk size and flags
# marshalls message to buffer while it has free space (lets assume the rest of message fits)
# puts size and last=true to the reserved space and sends

# reads chunk size, stores the target pool and partition
# allocates heap buffer and copies chunk data to it from read buffer
# once all message chunks are fully read message should be submitted to a pool where it will
be unmarshalled and processed

This message was sent by Atlassian JIRA

View raw message