drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Omernik <j...@omernik.com>
Subject Re: Guidelines for planner.memory.max_query_memory_per_node
Date Wed, 01 Jun 2016 21:00:22 GMT
So for my Parquet issues it will not likely make a difference, (It appears
to be heap related and/or parquet writer related)  Still, I would be very
interested in guidelines here, keeping it at 2GB with such beefy nodes
seems to be a waste.

John

On Wed, Jun 1, 2016 at 3:38 PM, Abdel Hakim Deneche <adeneche@maprtech.com>
wrote:

> I don't know about any specific guidelines for this options, but what I
> know is that it only affects the sort operator, and it's related to direct
> memory not heap memory.
>
>
>
> On Wed, Jun 1, 2016 at 1:20 PM, John Omernik <john@omernik.com> wrote:
>
> > I am reposting this question here as well. (I posted on the MapR
> Community
> > forums).
> >
> > The default as I understand it, for the setting
> > planner.memory.max_query_memory_per_node
> > is 2G.  The default heap memory settings in drill-env.sh is 4G and the
> > default Direct memory is 8G.
> >
> > I guess, is there any advice on where I should set my
> > planner.memory.max_query_memory_per_node
> > as the other numbers scale? I.e. does this setting coordinate more with
> > heap or direct or both? If I double my direct mem, should I double the
> > setting? Are there any guidelines or methods for tuning this?
> >
> > I am currently running bits at 24 GB of Heap and 84GB of Direct, should I
> > take the planner.memory.max_query_memory_per_node and x 8?  to put 16G?
> > Thoughts?
> >
> > Thanks!
> >
> > John
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message