ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: Custom Affinity Functions proposed for removal?
Date Tue, 03 Nov 2020 08:20:11 GMT
Moti, Raymond,

Could you please describe your use cases in more detail? What are the types
used as cache keys? What is the custom logic that you use for affinity
mapping? What was the exact reason to customize versus using built-in
collocation mechanisms?

Ultimately, I'm sure that custom affinity functions will go away cause they
introduce multiple potential issues. However, I would definitely like to
make sure that your use cases are still supported in Ignite 3.0 via some
other means.

-Val

On Mon, Nov 2, 2020 at 12:21 PM Alexei Scherbakov <
alexey.scherbakoff@gmail.com> wrote:

> Thanks for the clarification.
>
> There was no intention to remove the customizable key to partition mapping.
>
> Difficulties arise when mapping partitions to nodes, so it's desirable to
> have internally tested implementation with a way to customize it's behavior
> without additional coding on the user side.
>
> пн, 2 нояб. 2020 г. в 23:01, Raymond Wilson <raymond_wilson@trimble.com>:
>
> > Just to be clear, the affinity functions we are using convert keys to
> > partitions, we do not map partitions to nodes and leave that to Ignite.
> >
> > On Tue, Nov 3, 2020 at 8:48 AM Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com> wrote:
> >
> > > Hello.
> > >
> > > Custom affinity functions can cause weird bugs and data loss if
> > implemented
> > > wrongly.
> > > There is an intention  to keep a backup filter based on user attributes
> > > (with additional validation logic to ensure correctness) for
> controllable
> > > data placement.
> > >
> > > Can you describe more precisely why you had to implement custom
> affinity
> > > functions and not resort to default rendezvous affinity + backup
> filter ?
> > >
> > >
> > > пн, 2 нояб. 2020 г. в 21:45, Raymond Wilson <
> raymond_wilson@trimble.com
> > >:
> > >
> > > > We also use custom affinity functions (vis the C# client).
> > > >
> > > > The wish list mentions use of a particular annotation
> > > > (@CentralizedAffinityFunction):
> > > > Is the wish to remove just this annotation, or the ability to define
> > > custom
> > > > affinity functions at all?
> > > >
> > > > In our case we use affinity functions to ensure particular
> distribution
> > > of
> > > > spatial data across a processing cluster to ensure even load
> > management.
> > > >
> > > > On Tue, Nov 3, 2020 at 5:31 AM Moti Nisenson <
> moti.nisenson@gmail.com>
> > > > wrote:
> > > >
> > > > > I saw at
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist
> > > > > that custom affinity functions are on the potential wishlist for
> > > removal.
> > > > > The way we're using it's very critical that we be able to control
> the
> > > > > placement of data quite precisely - as part of that we specify
> > > explicitly
> > > > > the partition we want in the key, and then our affinity function
> uses
> > > > that
> > > > > (else delegating to default rendezvous). We don't need all the
> > > > > abilities there, although I think that often others do.
> > > > >
> > > > > This seems to me to be a case that the benefit of removing this is
> > > > minimal
> > > > > and could cause quite a lot of disruption to users.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > <http://www.trimble.com/>
> > > > Raymond Wilson
> > > > Solution Architect, Civil Construction Software Systems (CCSS)
> > > > 11 Birmingham Drive | Christchurch, New Zealand
> > > > +64-21-2013317 Mobile
> > > > raymond_wilson@trimble.com
> > > >
> > > > <
> > > >
> > >
> >
> https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
> >
> > --
> > <http://www.trimble.com/>
> > Raymond Wilson
> > Solution Architect, Civil Construction Software Systems (CCSS)
> > 11 Birmingham Drive | Christchurch, New Zealand
> > +64-21-2013317 Mobile
> > raymond_wilson@trimble.com
> >
> > <
> >
> https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message