ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: IGNITE-4302 discovery messages to exchange binary metadata - design proposal
Date Fri, 13 Jan 2017 09:28:51 GMT
Vladimir,

Given that metadata updates will be initiated with DiscoveryCustomEvent, I
see no problems neither with total message ordering (the order will be the
same on all nodes and consistent with discovery events) nor with failover -
the event is marked with @TcpDiscoveryEnsureDelivery, so the event should
be fired even in the case of coordinator failure.

I think handling metadata on clients might be tricky, however fore-fetch
approach is proven to work in the scope of IGNITE-4157.

2017-01-13 12:04 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:

> Sergey,
>
> In my understanding the only difficult thing here is how to disallow usage
> of not-yet-confirmed metadata during serialization. As we assume metadata
> updates to be relatively rare, can we always initiate message send from
> coordinator node? In this case we will have total order of all metadata
> updates, what will make conflict resolution trivial.
>
> Alex, how do you think - will this approach work? I am concerned about
> coordinator failover.
>
> On Thu, Jan 12, 2017 at 4:09 PM, Sergey Chugunov <
> sergey.chugunov@gmail.com>
> wrote:
>
> > Hello Ignite devs,
> >
> > I came up with the following design proposal for IGNITE-4302
> > <https://issues.apache.org/jira/browse/IGNITE-4302>, which is very
> similar
> > to described here
> > <http://apache-ignite-developers.2346864.n4.nabble.
> com/IGNITE-4157-design-
> > proposal-td11996.html>
> > proposal for IGNITE-4157 <https://issues.apache.org/
> > jira/browse/IGNITE-4157>,
> > "*Use discovery custom messages instead of marshaller cache*".
> >
> > Protocol is very similar to IGNITE-4157 and based on exchanging two
> > messages: *MetadataProposed*/*MetadataAccepted*.
> >
> >    1. When a node wants to add or update existing metadata, it sends
> >    *MetadataProposed* message with the update.
> >    2. Coordinator node checks for any conflicts with existing metadata
> for
> >    this typeId. In case of no conflicts coordinator assigns a version
> > number
> >    to new metadata and sends Proposed message further.
> >    In case of conflict it simply marks this message as IN_CONFLICT and
> >    sends it to the original node.
> >    3. All nodes upon receiving Proposed message update local metadata
> >    including version number and put it to "pending_acceptance" status.
> [1]
> >    4. Coordinator acknowledges proposed message with *MetadataAccepted*
> >    message, marking it with the version number from proposed message.
> >    5. Each node receiving accepted message checks local version number
> and
> >    the one from the message. If they are equal, metadata is considered as
> >    accepted by the cluster. [2]
> >
> > New nodes joining the cluster receive metadata on discovery phase.
> >
> > As discovery messages are delivered to clients asynchronously, there is a
> > possibility that cluster may consider and agree upon update to metadata
> > before client receives initial proposed message.
> > In that case I suggest to attach version number of metadata BinaryObject
> > was serialized with to that BinaryObject so client may check whether it
> can
> > deserialize this object safely or need to request some updated version of
> > metadata.
> >
> > [1] When a metadata is marked as "pending_acceptance" it should be
> > prohibited to use it to serialize binary objects but it still can be used
> > for deserialization.
> > At the same time from code perspective I didn't find a clear distinction
> > like "this method is used only for serialization and that method is
> called
> > only during deserialization". So for now the only way I see is to block
> all
> > operations for a given typeId until MetadataAccepted message for this
> > typeId is arrived.
> >
> > [2] If MetadataAccepted message has a version number less than currently
> > pending, it is ignored and all threads waiting for this metadata to be
> > accepted remain blocked.
> > It may be possible for new joining node to receive a MetadataAccepted
> > message with version number greater than local copy, in that case it is
> > fine to apply metadata update and declare metadata accepted.
> >
> >
> > Please share with me your thoughts about suggested design. Any
> > improvements, missed corner cases or other drawbacks are really
> > appreciated.
> >
> > Thanks,
> > Sergey.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message