ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandr Kuramshin <ein.nsk...@gmail.com>
Subject Empty cache memory overhead
Date Wed, 11 Jan 2017 05:41:34 GMT
Hi community,

I'd like to share my investigations about the subject.

Even if the caches is off-heap and contains no data, the JVM heap memory
consumed. I'm calling this feature "empty cache memory overhead"
("overhead" later for shot).

The size of the memory consumed depends on many factors, and varying from 1
to 50 Mb per cache on every node in the cluster.

There is real systems uses >1000 caches within the cluster. So the heap
memory consumed on each node will be 50 Gb or more.

I've found that overhead mainly depends on this factors:

1) local partitions count assigned to the node by the affinity function;

1.a) total number of partitions of the affinity function;

1.b) number of backups;

2) IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE

3) IGNITE_AFFINITY_HISTORY_SIZE

After analyzing heapdumps and the sources I've found this countable objects
upon overhead depends:

1) First group.

GridDhtPartitionTopologyImpl = cache count

GridDhtLocalPartition = cache count * local partitions count

GridCircularBuffer$Item = cache count * local partitions count * item
factor (default 32).

Local partitions count = affinity function total partitions / node count *
(1 + number of backups)

Item factor = map capacity for storing ->
IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE / affinity function partitions
count, but minimum 20.

Real values:

GridDhtPartitionTopologyImpl = 1000
Affinity function total partitions = 1024
Node count = 16
Number of backups = 3
Local partitions count = 256
GridDhtLocalPartition = 256_000
GridCircularBuffer$Item = 8_192_000

2) Second group.

GridAffinityAssignmentCache = cache count * node count

GridAffinityAssignment = cache count * node count * assignment factor

Assignment factor depends on topology version and
IGNITE_AFFINITY_HISTORY_SIZE, default 6-7.

Real values:

GridAffinityAssignmentCache = 16_000
GridAffinityAssignment  = 112_000

I think the implementation should be changed in the way the object counts
should depends on cache data size. And the small (or empty) caches should
be more lightweight as possible.

-- 
Thanks,
Alexandr Kuramshin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message