ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Harvey <syssoft...@gmail.com>
Subject Re: affinityBackupFilter for AWS Availability Zones
Date Thu, 23 Aug 2018 21:29:14 GMT
Added IGNITE-9365

On Thu, Aug 23, 2018 at 3:56 PM Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:

> Hi David,
>
> With the Docker image you can actually use additional libraries by
> providing URLs to JARs via EXTERNAL_LIBS property. Please refer to this
> page: https://apacheignite.readme.io/docs/docker-deployment
>
> But anyway, I believe that such contribution might be very valuable for
> Ignite. Feel free to create a ticket.
>
> -Val
>
> On Thu, Aug 23, 2018 at 11:58 AM David Harvey <syssoftsol@gmail.com>
> wrote:
>
> > I need an affinityBackupFilter that will prevent backups from running in
> > the same AWS availability zone.  (A single availability zone has the
> > characteristic that some or all of the EC2 instances in that zone can
> fail
> > together due to a single fault.   You have no control over the hosts on
> > which the EC2 instance VMs run on in AWS, except by controlling the
> > availability zone) .
> >
> > I could write a few lines of custom code, but then I have to get it
> > deployed on all nodes in the cluster, and peer class loading will not
> > work.   So I cannot use an of the shelf docker image, for example.   So
> > that code should just be part of Ignite.
> >
> > I was thinking of adding new class along these lines, where the apply
> > function will return true only if none of the node's attributes match
> those
> > of any of the nodes in the list.   This would become part of the code
> base,
> > but would only be used if configured as the backupAffinityFunction
> >
> > ClusterNodeNoAttributesMatchBiPredicate implements
> > IgniteBiPredicate<ClusterNode,
> >
> > List<ClusterNode>> {
> >
> >
> >     ClusterNodeNoAttributesMatchBiPredicate(String[] attributeNames)
> >     {....}
> >
> > For AvailabilityZones, there would be only one attribute examined, but we
> > have some potential use cases for distributing backups across two
> > sub-groups of an AZ.
> >
> > Alternately, we could enhance the RendezvousAffinityFunction to allow one
> > or more arbitrary attributes to be compared  to determine neighbors,
> > rather  than only org.apache.ignite.macs, and to add a setting that
> > controls whether backups should be placed on neighbors if they can't be
> > placed anywhere else.
> >
> > If I have 2 backups and three availability zones (AZ), I want one copy of
> > the data in each AZ.  If all nodes in one AZ fail, I want to be able to
> > decide to try to get to three copies anyway, increasing the per node
> > footprint by 50%, or to only run with one backup.     This would also
> give
> > be a convoluted way to change  the number of backups of a cache
> > dynamically:    Start the cache with a large number of backups, but don't
> > provide a location where the backup would be allowed to run initially.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message