ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ткаленко кирилл <tkalkir...@yandex.ru>
Subject Re: [DISCUSSION] Cache warmup
Date Fri, 07 Aug 2020 12:21:37 GMT
Hi, Stan!

Yes, that's right.
> On the interface structure.
> So, as a user I'll see one interface, WarmUpConfiguration, with no methods.
> I choose an implementation and configure it like
>     cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
> Ignite tries to map the LoadEverythingWarmupConfiguration to an implementations it knows
- either to a built-in one or to a plugin.
> If it finds the implementation, it passes the configuration to it and it handles the
warmup.
> If it doesn't find an existing implementation, it throws an exception.
> The implementation will use any internal API of Ignite that it needs to perform the warmup.
It is up to the plugin maintainer to track code changes in Ignite and adjust the plugin to
be compatible from version to version.
> Is all of the above correct?

If we need to configure heating, a string constant will not work for us.
> How about
>      cfg.setWarmUpConfiguration(IgniteWarmupStrategies.LOAD_EVERYTHING);
> instead of
>      cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
> ?
> Or do we expect having POJO instead of a string constant to be beneficial?

It seems to me that if you need to specify regions for a strategy, you can add them to strategy
settings. It would be more natural for user to do configuration by caches, but due to internal
feature of Ignite, it just doesn't work(yet), so it seems to me that you can add additional
options for each setting.

06.08.2020, 20:41, "Stanislav Lukyanov" <stanlukyanov@gmail.com>:
> Hi Kirill, Alexey,
>
> On the interface structure.
> So, as a user I'll see one interface, WarmUpConfiguration, with no methods.
> I choose an implementation and configure it like
>     cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
> Ignite tries to map the LoadEverythingWarmupConfiguration to an implementations it knows
- either to a built-in one or to a plugin.
> If it finds the implementation, it passes the configuration to it and it handles the
warmup.
> If it doesn't find an existing implementation, it throws an exception.
> The implementation will use any internal API of Ignite that it needs to perform the warmup.
It is up to the plugin maintainer to track code changes in Ignite and adjust the plugin to
be compatible from version to version.
> Is all of the above correct?
> How about
>      cfg.setWarmUpConfiguration(IgniteWarmupStrategies.LOAD_EVERYTHING);
> instead of
>      cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
> ?
> Or do we expect having POJO instead of a string constant to be beneficial?
>
> Agree about preloadPartition(). Fair enough, let's leave it be.
>
> On IgniteConfiguration vs DataRegionConfiguration.
> I like DataRegionConfiguration more because it allows to specify different strategies
for different regions naturally.
> We already say that all cache groups in the same region share memory management (e.g.
share space and participate in page replacement together).
> So it's natural to say that if I want different memory warmup semantics for two groups
then I should be putting them in different regions.
> Do you see a good way to have distinct warmup configuration for different regions while
the config is on IgniteConfiguration level?
>
> Thanks,
> Stan
>
>>  On 6 Aug 2020, at 15:39, ткаленко кирилл <tkalkirill@yandex.ru>
wrote:
>>
>>  Hello, Alexey!
>>
>>  Your comments are fair.
>>
>>  05.08.2020, 15:51, "Alexey Goncharuk" <alexey.goncharuk@gmail.com>:
>>>  Kirill,
>>>
>>>  Thank you for driving this discussion and implementation.
>>>
>>>  A few points from my side:
>>>  * Agree that it will be best to keep the strategy interface private because
>>>  it will be very dependent on the persistent storage implementation. We
>>>  would need to expose page IDs and types to public API, which is very
>>>  restrictive. The configuration part obviously needs to be public, and
>>>  ability to pull the strategy implementation from plugin is a good idea.
>>>  * I was also thinking of adding the warmup configuration straight to the
>>>  IgniteConfiguration, but I like Stan's idea of adding it to
>>>  DataRegionConfiguration. No strong preference here.
>>>  * I do not think we need to deprecate preloadPartition() method. One of the
>>>  use-cases for this method was to process partitions sequentially while a
>>>  node is running. This method is able to fetch the partition from disk much
>>>  (from times to orders of magnitude) faster than sequential scan.
>>>  * Being able to cancel the warmup during startup is a great feature. We
>>>  should be able to support it from control.sh because the warmup runs before
>>>  discovery which starts the last, so the control.sh handler should be
>>>  already running.

Mime
View raw message