ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladislav Pyatkov <vpyat...@gridgain.com>
Subject Re: Cache consistency
Date Wed, 02 Nov 2016 08:48:41 GMT
Hi Alisher,

At first I assume the discussion need to be moved to user-forum (
user@ignite.apache.org).

If you have 2 backups (in case you have enough nodes) this mean you can
loose 2 node, without afraid to loose data. In case you have a fail more
than the 2 nodes, it is mean you have incomplete(only part of data has
available) data.

In general, If you are loosing more nodes than backups you are have, this
must be signal: "These inconsistent!".

Also you can divided your nodes by specification: group of backups and
group of primary, using RendezvousAffinityFunction#setAffinityBackupFilter.

On Wed, Nov 2, 2016 at 11:11 AM, Alisher Alimov <alimovalisher@gmail.com>
wrote:

> Hi!
>
> I have a question about data consistency in cluster, if there are any
> mechanism for checking that cache is in consistency state (no lost
> data/partitions)
>
> For example I have a cluster with N nodes, long compute job that calculate
> monthly revenue on huge amount of data. Data is transaction log that stored
> in cache “transactions" <UUID, Float> (where key is transaction id, value
> transaction amount). Cache backups = 2.
>
> First case.
>
> 1. We load huge amount of data in cache "transactions"
> 2. All is fine
> 3. Run simple compute job that sum values in transaction log  ( Float::sum
> )
> 4. Compute job run on all nodes except n1,n2
> 5. We lost n1 node, than n2 node that stored backup of n1 node
> 6. Ignite found that node n1 and n2 are down and rebalancing data
> 7. Compute job was completed
>
> Second case.
>
> 1. We load huge amount of data in cache "transactions"
> 2. All is fine
> 3. We lost n1 node, then n2 node that stored backup of n1 node
> 4. Ignite found that node n1 and n2 are down and rebalancing data
> 5. Run simple compute job that sum values in transaction log  ( (a,b) ->
> a+b )
>
> So questions:
> - what result we can retrieve in first case from compute job?
> - at second case we run compute job on cache that is not in consistency
> state  (partitions that belongs to n1, n2 nodes was lost) but Ignite cache
> will work fine and allow us to run ComputeJob on cache and doesn’t tell us
> that some data was lost, isn’t it?
>
>
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message