ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Daschinsky <ivanda...@gmail.com>
Subject Re: IEP-61 Technical discussion
Date Mon, 23 Nov 2020 07:45:43 GMT
Also, here is some interesting reading about gossip, SWIM etc.

1 -- http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/SWIM.pdf
2 --
http://www.antonkharenko.com/2015/09/swim-distributed-group-membership.html
3 -- https://github.com/hashicorp/memberlist (Foundation library of
hashicorp serf)
4 -- https://github.com/scalecube/scalecube-cluster -- (Java implementation
of SWIM)

чт, 19 нояб. 2020 г. в 16:35, Ivan Daschinsky <ivandasch@gmail.com>:

> >> Friday, Nov 27th work for you? If ok, let's have an open call then.
> Yes, great
> >> As for the protocol port - we will not be dealing with the
> concurrency...
> >>Judging by the Rust port, it seems fairly straightforward.
> Yes, they chose split transport and logic. But original Go package from
> etcd (see raft/node.go) contains some  heartbeats mechanism etc.
> I agree with you, this seems not to be a huge deal to port.
>
> чт, 19 нояб. 2020 г. в 16:13, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
>> Ivan,
>>
>> Agree, let's have a call to discuss the IEP. I have some more thoughts
>> regarding how the replication infrastructure works with
>> atomic/transactional caches, will put this info to the IEP. Does next
>> Friday, Nov 27th work for you? If ok, let's have an open call then.
>>
>> As for the protocol port - we will not be dealing with the concurrency
>> model if we choose this way, this is what I like about their code
>> structure. Essentially, the raft module is a single-threaded automata
>> which
>> has a callback to process a message, process a tick (timeout) and produces
>> messages that should be sent and log entries that should be persisted.
>> Judging by the Rust port, it seems fairly straightforward. Will be happy
>> to
>> discuss this and other alternatives on the call as well.
>>
>> чт, 19 нояб. 2020 г. в 14:41, Ivan Daschinsky <ivandasch@gmail.com>:
>>
>> > > Any existing library that can be used to avoid re-implementing the
>> > protocol ourselves? Perhaps, porting the existing implementation to Java
>> > Personally, I like this idea. Go libraries (either raft module of etcd
>> or
>> > serf by Hashicorp) are famous for clean code, good design, stability,
>> not
>> > enormous size.
>> > But, on other side, Go has different model for concurrency and porting
>> > probably will not be so straightforward.
>> >
>> >
>> >
>> > чт, 19 нояб. 2020 г. в 13:48, Ivan Daschinsky <ivandasch@gmail.com>:
>> >
>> > > I'd suggest to discuss this IEP and technical details in open ZOOM
>> > > meeting.
>> > >
>> > > чт, 19 нояб. 2020 г. в 13:47, Ivan Daschinsky <ivandasch@gmail.com>:
>> > >
>> > >>
>> > >>
>> > >> ---------- Forwarded message ---------
>> > >> От: Ivan Daschinsky <ivandasch@gmail.com>
>> > >> Date: чт, 19 нояб. 2020 г. в 13:02
>> > >> Subject: Re: IEP-61 Technical discussion
>> > >> To: Alexey Goncharuk <alexey.goncharuk@gmail.com>
>> > >>
>> > >>
>> > >> Alexey, let's arise another question. Specifically, how nodes
>> initially
>> > >> find each other (discovery) and how they detect failures.
>> > >>
>> > >> I suppose, that gossip protocol is an ideal candidate. For example,
>> > >> consul [1] uses this approach, using serf [2] library to discover
>> > members
>> > >> of cluster.
>> > >> Then consul forms raft ensemble (server nodes) and client use raft
>> > >> ensemble only as lock service.
>> > >>
>> > >> PacificA suggests internal heartbeats mechanism for failure
>> detection of
>> > >> replicated group, but it says nothing about initial discovery of
>> nodes.
>> > >>
>> > >> WDYT?
>> > >>
>> > >> [1] -- https://www.consul.io/docs/architecture/gossip
>> > >> [2] -- https://www.serf.io/
>> > >>
>> > >> чт, 19 нояб. 2020 г. в 12:46, Alexey Goncharuk <
>> > >> alexey.goncharuk@gmail.com>:
>> > >>
>> > >>> Following up the Ignite 3.0 scope/development approach threads,
>> this is
>> > >>> a separate thread to discuss technical aspects of the IEP.
>> > >>>
>> > >>> Let's reiterate one more time on the questions raised by Ivan and
>> also
>> > >>> see if there are any other thoughts on the IEP:
>> > >>>
>> > >>>    - *Whether to deploy metastorage on a separate subset of the
>> nodes
>> > >>>    or allow Ignite to choose these nodes automatically.* I think
it
>> is
>> > >>>    feasible to maintain both modes: by default, Ignite will choose
>> > >>>    metastorage nodes automatically which essentially will provide
>> the
>> > same
>> > >>>    seamless user experience as TCP discovery SPI - no separate
>> roles,
>> > >>>    simplistic deployment. For deployments where people want to
have
>> > more
>> > >>>    fine-grained control over the nodes' assignments, we will
>> provide a
>> > runtime
>> > >>>    configuration which will allow pinning metastorage group to
>> certain
>> > nodes,
>> > >>>    thus eliminating the latency concerns.
>> > >>>    - *Whether there are any TLA+ specs for the PacificA protocol.*
>> Not
>> > >>>    to my knowledge, but it is known to be used in production by
>> > Microsoft and
>> > >>>    other projects, e.g. [1]
>> > >>>
>> > >>> I would like to collect general feedback on the IEP, as well as
>> > feedback
>> > >>> on specific parts of it, such as:
>> > >>>
>> > >>>    - Metastorage API
>> > >>>    - Any existing library that can be used to avoid re-implementing
>> the
>> > >>>    protocol ourselves? Perhaps, porting the existing implementation
>> to
>> > Java
>> > >>>    (the way TiKV did with etcd-raft [2] [3]? This is a very neat
way
>> > btw in my
>> > >>>    opinion because I like the finite automata-like approach of
the
>> > replication
>> > >>>    module, and, additionally, we could sync bug fixes and
>> improvements
>> > from
>> > >>>    the upstream project)
>> > >>>
>> > >>>
>> > >>> Thanks,
>> > >>> --AG
>> > >>>
>> > >>> [1]
>> > >>>
>> https://cwiki.apache.org/confluence/display/INCUBATOR/PegasusProposal
>> > >>> [2] https://github.com/etcd-io/etcd/tree/master/raft
>> > >>> [3] https://github.com/tikv/raft-rs
>> > >>>
>> > >>
>> > >>
>> > >> --
>> > >> Sincerely yours, Ivan Daschinskiy
>> > >>
>> > >>
>> > >> --
>> > >> Sincerely yours, Ivan Daschinskiy
>> > >>
>> > >
>> > >
>> > > --
>> > > Sincerely yours, Ivan Daschinskiy
>> > >
>> >
>> >
>> > --
>> > Sincerely yours, Ivan Daschinskiy
>> >
>>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>


-- 
Sincerely yours, Ivan Daschinskiy

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message