ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: affinityBackupFilter for AWS Availability Zones
Date Thu, 23 Aug 2018 19:55:32 GMT
Hi David,

With the Docker image you can actually use additional libraries by
providing URLs to JARs via EXTERNAL_LIBS property. Please refer to this
page: https://apacheignite.readme.io/docs/docker-deployment

But anyway, I believe that such contribution might be very valuable for
Ignite. Feel free to create a ticket.

-Val

On Thu, Aug 23, 2018 at 11:58 AM David Harvey <syssoftsol@gmail.com> wrote:

> I need an affinityBackupFilter that will prevent backups from running in
> the same AWS availability zone.  (A single availability zone has the
> characteristic that some or all of the EC2 instances in that zone can fail
> together due to a single fault.   You have no control over the hosts on
> which the EC2 instance VMs run on in AWS, except by controlling the
> availability zone) .
>
> I could write a few lines of custom code, but then I have to get it
> deployed on all nodes in the cluster, and peer class loading will not
> work.   So I cannot use an of the shelf docker image, for example.   So
> that code should just be part of Ignite.
>
> I was thinking of adding new class along these lines, where the apply
> function will return true only if none of the node's attributes match those
> of any of the nodes in the list.   This would become part of the code base,
> but would only be used if configured as the backupAffinityFunction
>
> ClusterNodeNoAttributesMatchBiPredicate implements
> IgniteBiPredicate<ClusterNode,
>
> List<ClusterNode>> {
>
>
>     ClusterNodeNoAttributesMatchBiPredicate(String[] attributeNames)
>     {....}
>
> For AvailabilityZones, there would be only one attribute examined, but we
> have some potential use cases for distributing backups across two
> sub-groups of an AZ.
>
> Alternately, we could enhance the RendezvousAffinityFunction to allow one
> or more arbitrary attributes to be compared  to determine neighbors,
> rather  than only org.apache.ignite.macs, and to add a setting that
> controls whether backups should be placed on neighbors if they can't be
> placed anywhere else.
>
> If I have 2 backups and three availability zones (AZ), I want one copy of
> the data in each AZ.  If all nodes in one AZ fail, I want to be able to
> decide to try to get to three copies anyway, increasing the per node
> footprint by 50%, or to only run with one backup.     This would also give
> be a convoluted way to change  the number of backups of a cache
> dynamically:    Start the cache with a large number of backups, but don't
> provide a location where the backup would be allowed to run initially.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message