ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: Losing data during restarting cluster with persistence enabled
Date Fri, 17 Nov 2017 18:07:18 GMT
Vyacheslav,

CacheStore design assumes that the underlying storage is shared by all the
nodes in topology. Even if you delay rebalancing on node stop (which is
possible via CacheConfiguration#rebalanceDelay), I doubt it will solve all
your consistency issues.

Why don't you use Ignite persistence [1]?

[1] https://apacheignite.readme.io/docs/distributed-persistent-store

-Val

On Fri, Nov 17, 2017 at 4:24 AM, Vyacheslav Daradur <daradurvs@gmail.com>
wrote:

> Hi Andrey! Thank you for answering.
>
> >> Key to partition mapping shouldn't depends on topology, and shouldn't
> changed unstable topology.
> Key to partition mapping doesn't depend on topology in my test
> affinity function. It only depends on partitions number.
> But partition to node mapping depends on topology and at cluster stop,
> when one node left topology, some partitions may be moved to other
> nodes.
>
> >> Does all nodes share same RockDB database or each node has its own copy?
> Each Ignite node has own RocksDB instance.
>
> >> Would you please share configuration?
> It's pretty simple:
>         IgniteConfiguration cfg = new IgniteConfiguration();
>         cfg.setIgniteInstanceName(instanceName);
>
>         CacheConfiguration<Integer, String> cacheCfg = new
> CacheConfiguration<>();
>         cacheCfg.setName(TEST_CACHE_NAME);
>         cacheCfg.setCacheMode(CacheMode.PARTITIONED);
>         cacheCfg.setWriteSynchronizationMode(
> CacheWriteSynchronizationMode.PRIMARY_SYNC);
>         cacheCfg.setBackups(1);
>         cacheCfg.setAffinity(new
> TestAffinityFunction(partitionsNumber, backupsNumber));
>         cacheCfg.setWriteThrough(true);
>         cacheCfg.setReadThrough(true);
>         cacheCfg.setRebalanceMode(CacheRebalanceMode.SYNC);
>         cacheCfg.setCacheStoreFactory(new
> RocksDBCacheStoreFactory<>("/test/path/to/persistence",
> TEST_CACHE_NAME, cfg));
>
>         cfg.setCacheConfiguration(cacheCfg);
>
> Could you give me advice on places which I need to pay attention?
>
>
> On Wed, Nov 15, 2017 at 3:02 PM, Andrey Mashenkov
> <andrey.mashenkov@gmail.com> wrote:
> > Hi Vyacheslav,
> >
> > Key to partition mapping shouldn't depends on topology, and shouldn't
> > changed unstable topology.
> > Looks like you've missed smth.
> >
> > Would you please share configuration?
> > Does all nodes share same RockDB database or each node has its own copy?
> >
> >
> >
> > On Wed, Nov 15, 2017 at 12:22 AM, Vyacheslav Daradur <
> daradurvs@gmail.com>
> > wrote:
> >
> >> Hi, Igniters!
> >>
> >> I’m using partitioned Ignite cache with RocksDB as 3rd party persistence
> >> store.
> >> I've got an issue: if cache rebalancing is switched on, then it’s
> >> possible to lose some data.
> >>
> >> Basic scenario:
> >> 1) Start Ignite cluster and fill a cache with RocksDB persistence;
> >> 2) Stop all nodes
> >> 3) Start Ignite cluster and validate data
> >>
> >> This works fine while rebalancing is switched off.
> >>
> >> If rebalancing switched on: when I call Ignition#stopAll, some nodes
> >> go down sequentially and while one node having gone down another start
> >> rebalancing. When nodes started affinity function works with a full
> >> set of nodes and may define a wrong partition for a key because the
> >> previous state was changed at rebalancing.
> >>
> >> Maybe I'm doing something wrong. How can I avoid rebalancing while
> >> stopping all nodes in the cluster?
> >>
> >> Could you give me any advice, please?
> >>
> >> --
> >> Best Regards, Vyacheslav D.
> >>
> >
> >
> >
> > --
> > Best regards,
> > Andrey V. Mashenkov
>
>
>
> --
> Best Regards, Vyacheslav D.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message