ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: [IGNITE-5717] improvements of MemoryPolicy default size
Date Wed, 02 Aug 2017 05:27:34 GMT
Please see original Sergey's message - when persistence is enabled, memory
is not allocated incrementally, maxSize is used.

But this is a problem even for in-memory case. Consider that I wanted to
load several gigs of data as an experiment on my laptop - frequent
scenario. First several seconds it works fine. Then it starts swapping and
machine becomes unresponsive.

Default settings must allow for normal work on developer's environment.

ср, 2 авг. 2017 г. в 1:10, Denis Magda <dmagda@apache.org>:

> > Why not allocate in increments automatically?
>
> This is exactly how the allocation works right now. The memory will grow
> incrementally until the max size is reached (80% of RAM by default).
>
> —
> Denis
>
> > On Aug 1, 2017, at 3:03 PM, dsetrakyan@apache.org wrote:
> >
> > Vova, 1GB seems a bit too small for me, and frankly i do not want t o
> guess. Why not allocate in increments automatically?
> >
> > ⁣D.​
> >
> > On Aug 1, 2017, 11:03 PM, at 11:03 PM, Vladimir Ozerov <
> vozerov@gridgain.com> wrote:
> >> Denis,
> >> No doubts you haven't heard about it - AI 2.1 with persistence, when
> >> 80% of
> >> RAM is allocated right away, was released several days ago. How do you
> >> think, how many users tried it already?
> >>
> >> Guys,
> >> Do you really think allocating 80% of available RAM is a normal thing?
> >> Take
> >> your laptop and check how many available RAM you have right now. Do you
> >> fit
> >> to remaining 20%? If not, then running AI with persistence with all
> >> defaults will bring your machine down. This is insane. We shold
> >> allocate no
> >> more than 1Gb, so that user can play with it without any problems.
> >>
> >> On Tue, Aug 1, 2017 at 10:26 PM, Denis Magda <dmagda@apache.org> wrote:
> >>
> >>> My vote goes for option #1 too. I don’t think that 80% is too
> >> aggressive
> >>> to bring it down.
> >>>
> >>> IGNITE-5717 was created to fix the issue of the 80% RAM allocation on
> >> 64
> >>> bit systems when Ignite works on top of 32 bit JVM. I’ve not heard of
> >> any
> >>> other complaints in regards the default allocation size.
> >>>
> >>> —
> >>> Denis
> >>>
> >>>> On Aug 1, 2017, at 10:58 AM, dsetrakyan@apache.org wrote:
> >>>>
> >>>> I prefer option #1.
> >>>>
> >>>> ⁣D.​
> >>>>
> >>>> On Aug 1, 2017, 11:20 AM, at 11:20 AM, Sergey Chugunov <
> >>> sergey.chugunov@gmail.com> wrote:
> >>>>> Folks,
> >>>>>
> >>>>> I would like to get back to the question about MemoryPolicy
> >> maxMemory
> >>>>> defaults.
> >>>>>
> >>>>> Although MemoryPolicy may be configured with initial and maxMemory
> >>>>> settings, when persistence is used MemoryPolicy always allocates
> >>>>> maxMemory
> >>>>> size for performance reasons.
> >>>>>
> >>>>> As default size of maxMemory is 80% of physical memory it causes
> >> OOME
> >>>>> exceptions of 32 bit platforms (either on OS or JVM level) and
> >> hurts
> >>>>> performance in setups when multiple Ignite nodes are started on
> >> the
> >>>>> same
> >>>>> physical server.
> >>>>>
> >>>>> I suggest to rethink these defaults and switch to other options:
> >>>>>
> >>>>> - Check whether platform is 32 or 64 bits and adapt defaults. In
> >> this
> >>>>> case we still need to address the issue with multiple nodes on one
> >>>>> machine
> >>>>> even on 64 bit systems.
> >>>>>
> >>>>> - Lower defaults for maxMemory and allocate, for instance,
> >> max(0.3 *
> >>>>> availableMemory, 1Gb).
> >>>>> This option allows us to solve all issues with starting on 32 bit
> >>>>> platforms and reduce instability with multiple nodes on the same
> >>>>> machine.
> >>>>>
> >>>>>
> >>>>> Thoughts and/or other options?
> >>>>>
> >>>>> Thanks,
> >>>>> Sergey.
> >>>
> >>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message