ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanislav Lukyanov <stanlukya...@gmail.com>
Subject Re: [DISCUSSION] Cache warmup
Date Thu, 06 Aug 2020 17:41:05 GMT
Hi Kirill, Alexey,

On the interface structure.
So, as a user I'll see one interface, WarmUpConfiguration, with no methods.
I choose an implementation and configure it like
    cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
Ignite tries to map the LoadEverythingWarmupConfiguration to an implementations it knows -
either to a built-in one or to a plugin.
If it finds the implementation, it passes the configuration to it and it handles the warmup.
If it doesn't find an existing implementation, it throws an exception.
The implementation will use any internal API of Ignite that it needs to perform the warmup.
It is up to the plugin maintainer to track code changes in Ignite and adjust the plugin to
be compatible from version to version.
Is all of the above correct?
How about 
     cfg.setWarmUpConfiguration(IgniteWarmupStrategies.LOAD_EVERYTHING);
instead of
     cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
?
Or do we expect having POJO instead of a string constant to be beneficial?

Agree about preloadPartition(). Fair enough, let's leave it be.


On IgniteConfiguration vs DataRegionConfiguration.
I like DataRegionConfiguration more because it allows to specify different strategies for
different regions naturally.
We already say that all cache groups in the same region share memory management (e.g. share
space and participate in page replacement together).
So it's natural to say that if I want different memory warmup semantics for two groups then
I should be putting them in different regions.
Do you see a good way to have distinct warmup configuration for different regions while the
config is on IgniteConfiguration level?

Thanks,
Stan

> On 6 Aug 2020, at 15:39, ткаленко кирилл <tkalkirill@yandex.ru> wrote:
> 
> Hello, Alexey!
> 
> Your comments are fair.
> 
> 05.08.2020, 15:51, "Alexey Goncharuk" <alexey.goncharuk@gmail.com>:
>> Kirill,
>> 
>> Thank you for driving this discussion and implementation.
>> 
>> A few points from my side:
>> * Agree that it will be best to keep the strategy interface private because
>> it will be very dependent on the persistent storage implementation. We
>> would need to expose page IDs and types to public API, which is very
>> restrictive. The configuration part obviously needs to be public, and
>> ability to pull the strategy implementation from plugin is a good idea.
>> * I was also thinking of adding the warmup configuration straight to the
>> IgniteConfiguration, but I like Stan's idea of adding it to
>> DataRegionConfiguration. No strong preference here.
>> * I do not think we need to deprecate preloadPartition() method. One of the
>> use-cases for this method was to process partitions sequentially while a
>> node is running. This method is able to fetch the partition from disk much
>> (from times to orders of magnitude) faster than sequential scan.
>> * Being able to cancel the warmup during startup is a great feature. We
>> should be able to support it from control.sh because the warmup runs before
>> discovery which starts the last, so the control.sh handler should be
>> already running.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message