ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Sapego <isap...@apache.org>
Subject Re: Best Effort Affinity for thin clients
Date Mon, 04 Feb 2019 09:48:27 GMT
Guys,

Can you explain why do we want to avoid Cache groups in protocol?

If it's about simplicity of the protocol, then removing cache groups will
not help much with it - we will still need to include "knownCacheIds"
field in request and "cachesWithTheSamePartitioning" field in response.
And also, since when do Ignite prefers simplicity over performance?

If it's about not wanting to show internals of Ignite then it sounds like
a very weak argument to me, since Cache Groups is a public thing [1].

[1] - https://apacheignite.readme.io/docs/cache-groups

Best Regards,
Igor


On Mon, Feb 4, 2019 at 11:47 AM Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Pavel, Igor,
>
> This is not very accurate to say that this will not save memory. In
> practice we observed a number of OOME issues on the server-side due to many
> caches and it was one of motivations for cache groups (another one disk
> access optimizations). On the other hand, I agree that we'd better to avoid
> cache groups in the protocol because this is internal implementation detail
> which is likely (I hope so) to be changed in future.
>
> So I have another proposal - let's track caches with the same affinity
> distribution instead. That is, normally most of PARTITIONED caches will
> have very few variants of configuration: it will be Rendezvous affinity
> function, most likely with default partition number and with 1-2 backups at
> most. So when affinity distribution for specific cache is requested, we can
> append to the response *list of caches with the same distribution*. I.e.:
>
> class AffinityResponse {
>     Object distribution;    // Actual distribution
>     List<Integer> cacheIds; // Caches with similar distribution
> }
>
> Makes sense?
>
> On Sun, Feb 3, 2019 at 8:31 PM Pavel Tupitsyn <ptupitsyn@apache.org>
> wrote:
>
> > Igor, I have a feeling that we should omit Cache Group stuff from the
> > protocol.
> > It is a rare use case and even then dealing with them on client barely
> > saves some memory.
> >
> > We can keep it simple and have partition map per cacheId. Thoughts?
> >
> > On Fri, Feb 1, 2019 at 6:49 PM Igor Sapego <isapego@apache.org> wrote:
> >
> > > Guys, I've updated the proposal once again [1], so please,
> > > take a look and let me know what you think.
> > >
> > > [1] -
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients
> > >
> > > Best Regards,
> > > Igor
> > >
> > >
> > > On Thu, Jan 17, 2019 at 1:05 PM Igor Sapego <isapego@apache.org>
> wrote:
> > >
> > > > Yeah, I'll add it.
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > >
> > > > On Wed, Jan 16, 2019 at 11:08 PM Pavel Tupitsyn <
> ptupitsyn@apache.org>
> > > > wrote:
> > > >
> > > >> >  to every server
> > > >> I did not think of this issue. Now I agree with your approach.
> > > >> Can you please add an explanation of this to the IEP?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> On Wed, Jan 16, 2019 at 2:53 PM Igor Sapego <isapego@apache.org>
> > wrote:
> > > >>
> > > >> > Pavel,
> > > >> >
> > > >> > Yeah, it makes sense, but to me it seems that this approach can
> lead
> > > >> > to more complicated client logic, as it will require to make
> > > additional
> > > >> > call
> > > >> > to every server, that reports affinity topology change.
> > > >> >
> > > >> > Guys, WDYT?
> > > >> >
> > > >> > Best Regards,
> > > >> > Igor
> > > >> >
> > > >> >
> > > >> > On Tue, Jan 15, 2019 at 10:59 PM Pavel Tupitsyn <
> > ptupitsyn@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> > > Igor,
> > > >> > >
> > > >> > > >  It is proposed to add flag to every response, that
shows
> > whether
> > > >> the
> > > >> > > Affinity Topology Version of the cluster has changed since
the
> > last
> > > >> > request
> > > >> > > from the client.
> > > >> > > I propose to keep this flag. So no need for periodic checks.
> Makes
> > > >> sense?
> > > >> > >
> > > >> > > On Tue, Jan 15, 2019 at 4:45 PM Igor Sapego <isapego@apache.org
> >
> > > >> wrote:
> > > >> > >
> > > >> > > > Pavel,
> > > >> > > >
> > > >> > > > This will require from client to send this new request
> > > periodically,
> > > >> > I'm
> > > >> > > > not
> > > >> > > > sure this will make clients simpler. Anyway, let's
discuss it.
> > > >> > > >
> > > >> > > > Vladimir,
> > > >> > > >
> > > >> > > > With current proposal, we will have affinity info in
message
> > > header.
> > > >> > > >
> > > >> > > > Best Regards,
> > > >> > > > Igor
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Jan 15, 2019 at 11:01 AM Vladimir Ozerov <
> > > >> vozerov@gridgain.com
> > > >> > >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Igor,
> > > >> > > > >
> > > >> > > > > I think that "Cache Partitions Request" should
contain
> > affinity
> > > >> > > topology
> > > >> > > > > version. Otherwise we do not know what distribution
is
> > returned
> > > -
> > > >> the
> > > >> > > one
> > > >> > > > > we expected, or some newer one. The latter may
happen in
> case
> > > >> > topology
> > > >> > > > > changed or late affinity assignment happened between
server
> > > >> response
> > > >> > > and
> > > >> > > > > subsequent client partitions request.
> > > >> > > > >
> > > >> > > > > Vladimir.
> > > >> > > > >
> > > >> > > > > On Mon, Jan 14, 2019 at 6:08 PM Igor Sapego <
> > isapego@apache.org
> > > >
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > > > Hello guys,
> > > >> > > > > >
> > > >> > > > > > I've updated IEP page [1] describing proposed
solution in
> > more
> > > >> > > details
> > > >> > > > > and
> > > >> > > > > > proposing some changes for a protocol.
> > > >> > > > > >
> > > >> > > > > > Please, take a look and let me know what
you think.
> > > >> > > > > >
> > > >> > > > > > [1] -
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients
> > > >> > > > > >
> > > >> > > > > > Best Regards,
> > > >> > > > > > Igor
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Tue, Jun 19, 2018 at 11:54 AM Vladimir
Ozerov <
> > > >> > > vozerov@gridgain.com
> > > >> > > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Denis,
> > > >> > > > > > >
> > > >> > > > > > > Yes, in principle we can extend it.
We are going to
> > > implement
> > > >> it
> > > >> > in
> > > >> > > > > > > subsequent phases of this IEP.
> > > >> > > > > > >
> > > >> > > > > > > On Tue, Jun 19, 2018 at 4:30 AM, Dmitriy
Setrakyan <
> > > >> > > > > > dsetrakyan@apache.org>
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > On Mon, Jun 18, 2018 at 11:07 AM,
Denis Magda <
> > > >> > dmagda@apache.org
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > Folks,
> > > >> > > > > > > > >
> > > >> > > > > > > > > Feel that this functionality
can be extended to the
> > > >> automatic
> > > >> > > > > > > reconnect,
> > > >> > > > > > > > > can't it? Presently we require
to provide a static
> > list
> > > of
> > > >> > IPs
> > > >> > > to
> > > >> > > > > be
> > > >> > > > > > > used
> > > >> > > > > > > > > at a reconnect time. By having
a partition map of
> all
> > > the
> > > >> > > nodes,
> > > >> > > > > the
> > > >> > > > > > > thin
> > > >> > > > > > > > > client should be able to automate
this piece.
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Not sure if static IP list can
be avoided. What Igor
> is
> > > >> > > suggesting
> > > >> > > > is
> > > >> > > > > > > that
> > > >> > > > > > > > we try to pick the best node out
of the static IP
> list.
> > > >> > > > > > > >
> > > >> > > > > > > > D.
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message