ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: IEP-61 Technical discussion
Date Thu, 26 Nov 2020 09:29:31 GMT
Hi Ivan,

Unfortunately, the earliest window available for us is 12:00 MSK (1 hour
slot), or after 14:30 MSK. Let me know what time works best for you.

ср, 25 нояб. 2020 г. в 21:38, Ivan Daschinsky <ivandasch@gmail.com>:

> Alexey, I kindly ask you to move the meeting a little bit earlier, ideal
> variant -- in the morning.
>
> ср, 25 нояб. 2020 г. в 20:10, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > Folks, let's have the call on Friday, Nov 27th at 18:00 MSK? We can use
> the
> > following waiting room link:
> >  https://zoom.us/j/99450012496?pwd=RWZmOGhCNWlRK0ZpamdOOTZsYTJ0dz09
> >
> > Let me know if this time works for everybody.
> >
> > ср, 25 нояб. 2020 г. в 16:42, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > Folks,
> > >
> > > I've made some edits in IEP-61 [1] regarding the group membership
> service
> > > and transaction protocol interaction with the replication
> infrastructure,
> > > please take a look before our Friday call.
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-61%3A+Common+Replication+Infrastructure
> > >
> > > пн, 23 нояб. 2020 г. в 13:28, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > >
> > >> Thanks, Ivan,
> > >>
> > >> Another protocol for group membership worth checking out is RAPID [1]
> (a
> > >> recent one). Not sure though if there are any available
> implementations
> > for
> > >> it already.
> > >>
> > >> [1]
> > https://www.usenix.org/system/files/conference/atc18/atc18-suresh.pdf
> > >>
> > >> пн, 23 нояб. 2020 г. в 10:46, Ivan Daschinsky <ivandasch@gmail.com>:
> > >>
> > >>> Also, here is some interesting reading about gossip, SWIM etc.
> > >>>
> > >>> 1 --
> > >>>
> http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/SWIM.pdf
> > >>> 2 --
> > >>>
> > >>>
> >
> http://www.antonkharenko.com/2015/09/swim-distributed-group-membership.html
> > >>> 3 -- https://github.com/hashicorp/memberlist (Foundation library of
> > >>> hashicorp serf)
> > >>> 4 -- https://github.com/scalecube/scalecube-cluster -- (Java
> > >>> implementation
> > >>> of SWIM)
> > >>>
> > >>> чт, 19 нояб. 2020 г. в 16:35, Ivan Daschinsky <ivandasch@gmail.com>:
> > >>>
> > >>> > >> Friday, Nov 27th work for you? If ok, let's have an open
call
> > then.
> > >>> > Yes, great
> > >>> > >> As for the protocol port - we will not be dealing with
the
> > >>> > concurrency...
> > >>> > >>Judging by the Rust port, it seems fairly straightforward.
> > >>> > Yes, they chose split transport and logic. But original Go package
> > from
> > >>> > etcd (see raft/node.go) contains some  heartbeats mechanism etc.
> > >>> > I agree with you, this seems not to be a huge deal to port.
> > >>> >
> > >>> > чт, 19 нояб. 2020 г. в 16:13, Alexey Goncharuk <
> > >>> alexey.goncharuk@gmail.com
> > >>> > >:
> > >>> >
> > >>> >> Ivan,
> > >>> >>
> > >>> >> Agree, let's have a call to discuss the IEP. I have some more
> > thoughts
> > >>> >> regarding how the replication infrastructure works with
> > >>> >> atomic/transactional caches, will put this info to the IEP.
Does
> > next
> > >>> >> Friday, Nov 27th work for you? If ok, let's have an open call
> then.
> > >>> >>
> > >>> >> As for the protocol port - we will not be dealing with the
> > concurrency
> > >>> >> model if we choose this way, this is what I like about their
code
> > >>> >> structure. Essentially, the raft module is a single-threaded
> > automata
> > >>> >> which
> > >>> >> has a callback to process a message, process a tick (timeout)
and
> > >>> produces
> > >>> >> messages that should be sent and log entries that should be
> > persisted.
> > >>> >> Judging by the Rust port, it seems fairly straightforward.
Will be
> > >>> happy
> > >>> >> to
> > >>> >> discuss this and other alternatives on the call as well.
> > >>> >>
> > >>> >> чт, 19 нояб. 2020 г. в 14:41, Ivan Daschinsky <
> ivandasch@gmail.com
> > >:
> > >>> >>
> > >>> >> > > Any existing library that can be used to avoid re-implementing
> > the
> > >>> >> > protocol ourselves? Perhaps, porting the existing implementation
> > to
> > >>> Java
> > >>> >> > Personally, I like this idea. Go libraries (either raft
module
> of
> > >>> etcd
> > >>> >> or
> > >>> >> > serf by Hashicorp) are famous for clean code, good design,
> > >>> stability,
> > >>> >> not
> > >>> >> > enormous size.
> > >>> >> > But, on other side, Go has different model for concurrency
and
> > >>> porting
> > >>> >> > probably will not be so straightforward.
> > >>> >> >
> > >>> >> >
> > >>> >> >
> > >>> >> > чт, 19 нояб. 2020 г. в 13:48, Ivan Daschinsky
<
> > ivandasch@gmail.com
> > >>> >:
> > >>> >> >
> > >>> >> > > I'd suggest to discuss this IEP and technical details
in open
> > ZOOM
> > >>> >> > > meeting.
> > >>> >> > >
> > >>> >> > > чт, 19 нояб. 2020 г. в 13:47, Ivan Daschinsky
<
> > >>> ivandasch@gmail.com>:
> > >>> >> > >
> > >>> >> > >>
> > >>> >> > >>
> > >>> >> > >> ---------- Forwarded message ---------
> > >>> >> > >> От: Ivan Daschinsky <ivandasch@gmail.com>
> > >>> >> > >> Date: чт, 19 нояб. 2020 г. в 13:02
> > >>> >> > >> Subject: Re: IEP-61 Technical discussion
> > >>> >> > >> To: Alexey Goncharuk <alexey.goncharuk@gmail.com>
> > >>> >> > >>
> > >>> >> > >>
> > >>> >> > >> Alexey, let's arise another question. Specifically,
how nodes
> > >>> >> initially
> > >>> >> > >> find each other (discovery) and how they detect
failures.
> > >>> >> > >>
> > >>> >> > >> I suppose, that gossip protocol is an ideal
candidate. For
> > >>> example,
> > >>> >> > >> consul [1] uses this approach, using serf [2]
library to
> > discover
> > >>> >> > members
> > >>> >> > >> of cluster.
> > >>> >> > >> Then consul forms raft ensemble (server nodes)
and client use
> > >>> raft
> > >>> >> > >> ensemble only as lock service.
> > >>> >> > >>
> > >>> >> > >> PacificA suggests internal heartbeats mechanism
for failure
> > >>> >> detection of
> > >>> >> > >> replicated group, but it says nothing about
initial discovery
> > of
> > >>> >> nodes.
> > >>> >> > >>
> > >>> >> > >> WDYT?
> > >>> >> > >>
> > >>> >> > >> [1] -- https://www.consul.io/docs/architecture/gossip
> > >>> >> > >> [2] -- https://www.serf.io/
> > >>> >> > >>
> > >>> >> > >> чт, 19 нояб. 2020 г. в 12:46, Alexey
Goncharuk <
> > >>> >> > >> alexey.goncharuk@gmail.com>:
> > >>> >> > >>
> > >>> >> > >>> Following up the Ignite 3.0 scope/development
approach
> > threads,
> > >>> >> this is
> > >>> >> > >>> a separate thread to discuss technical aspects
of the IEP.
> > >>> >> > >>>
> > >>> >> > >>> Let's reiterate one more time on the questions
raised by
> Ivan
> > >>> and
> > >>> >> also
> > >>> >> > >>> see if there are any other thoughts on the
IEP:
> > >>> >> > >>>
> > >>> >> > >>>    - *Whether to deploy metastorage on a
separate subset of
> > the
> > >>> >> nodes
> > >>> >> > >>>    or allow Ignite to choose these nodes
automatically.* I
> > >>> think it
> > >>> >> is
> > >>> >> > >>>    feasible to maintain both modes: by default,
Ignite will
> > >>> choose
> > >>> >> > >>>    metastorage nodes automatically which
essentially will
> > >>> provide
> > >>> >> the
> > >>> >> > same
> > >>> >> > >>>    seamless user experience as TCP discovery
SPI - no
> separate
> > >>> >> roles,
> > >>> >> > >>>    simplistic deployment. For deployments
where people want
> to
> > >>> have
> > >>> >> > more
> > >>> >> > >>>    fine-grained control over the nodes'
assignments, we will
> > >>> >> provide a
> > >>> >> > runtime
> > >>> >> > >>>    configuration which will allow pinning
metastorage group
> to
> > >>> >> certain
> > >>> >> > nodes,
> > >>> >> > >>>    thus eliminating the latency concerns.
> > >>> >> > >>>    - *Whether there are any TLA+ specs for
the PacificA
> > >>> protocol.*
> > >>> >> Not
> > >>> >> > >>>    to my knowledge, but it is known to be
used in production
> > by
> > >>> >> > Microsoft and
> > >>> >> > >>>    other projects, e.g. [1]
> > >>> >> > >>>
> > >>> >> > >>> I would like to collect general feedback
on the IEP, as well
> > as
> > >>> >> > feedback
> > >>> >> > >>> on specific parts of it, such as:
> > >>> >> > >>>
> > >>> >> > >>>    - Metastorage API
> > >>> >> > >>>    - Any existing library that can be used
to avoid
> > >>> re-implementing
> > >>> >> the
> > >>> >> > >>>    protocol ourselves? Perhaps, porting
the existing
> > >>> implementation
> > >>> >> to
> > >>> >> > Java
> > >>> >> > >>>    (the way TiKV did with etcd-raft [2]
[3]? This is a very
> > >>> neat way
> > >>> >> > btw in my
> > >>> >> > >>>    opinion because I like the finite automata-like
approach
> of
> > >>> the
> > >>> >> > replication
> > >>> >> > >>>    module, and, additionally, we could sync
bug fixes and
> > >>> >> improvements
> > >>> >> > from
> > >>> >> > >>>    the upstream project)
> > >>> >> > >>>
> > >>> >> > >>>
> > >>> >> > >>> Thanks,
> > >>> >> > >>> --AG
> > >>> >> > >>>
> > >>> >> > >>> [1]
> > >>> >> > >>>
> > >>> >>
> > https://cwiki.apache.org/confluence/display/INCUBATOR/PegasusProposal
> > >>> >> > >>> [2] https://github.com/etcd-io/etcd/tree/master/raft
> > >>> >> > >>> [3] https://github.com/tikv/raft-rs
> > >>> >> > >>>
> > >>> >> > >>
> > >>> >> > >>
> > >>> >> > >> --
> > >>> >> > >> Sincerely yours, Ivan Daschinskiy
> > >>> >> > >>
> > >>> >> > >>
> > >>> >> > >> --
> > >>> >> > >> Sincerely yours, Ivan Daschinskiy
> > >>> >> > >>
> > >>> >> > >
> > >>> >> > >
> > >>> >> > > --
> > >>> >> > > Sincerely yours, Ivan Daschinskiy
> > >>> >> > >
> > >>> >> >
> > >>> >> >
> > >>> >> > --
> > >>> >> > Sincerely yours, Ivan Daschinskiy
> > >>> >> >
> > >>> >>
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Sincerely yours, Ivan Daschinskiy
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>> Sincerely yours, Ivan Daschinskiy
> > >>>
> > >>
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message