ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ткаленко кирилл <tkalkir...@yandex.ru>
Subject [DISCUSSION] Cache warmup
Date Mon, 27 Jul 2020 12:03:02 GMT
Now, after restarting node, we have only cold caches, which at first requests to them will
gradually load data from disks, which can slow down first calls to them.
If node has more RAM than data on disk, then they can be loaded at start "warmup", thereby
solving the issue of slowdowns during first calls to caches.

I suggest adding a warmup phase after recovery here [1] after [2], before descovery.

I suggest adding a new interface:

package org.apache.ignite.internal.processors.cache;

import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.internal.IgniteInternalFuture;
import org.jetbrains.annotations.Nullable;

/**
 * Interface for warming up cache.
 */
public interface CacheWarmup {
    /**
     * Warmup cache.
     *
     * @param cacheCtx Cache context.
     * @return Future cache warmup.
     * @throws IgniteCheckedException if failed.
     */
    @Nullable IgniteInternalFuture<?> process(GridCacheContext cacheCtx) throws IgniteCheckedException;
}

Which will allow to warm up caches in parallel and asynchronously. Warmup phase will end after
all IgniteInternalFuture for all caches isDone.

Also adding the ability to customize via methods:
org.apache.ignite.configuration.IgniteConfiguration#setDefaultCacheWarmup
org.apache.ignite.configuration.CacheConfiguration#setCacheWarmup

Which will allow for each cache to set implementation of cache warming up, both for a specific
cache, and for all if necessary.

I suggest adding an implementation of SequentialWarmup that will use [3].

Questions, suggestions, comments?

[1] - org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#afterLogicalUpdatesApplied
[2] - org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#restorePartitionStates
[3] - org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManager.CacheDataStore#preload

Mime
View raw message