drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <par0...@yahoo.com.INVALID>
Subject Re: Update planner/configuration keys
Date Wed, 13 Mar 2019 06:03:25 GMT
Hi Kunal and Praveen,

ALTER SYSTEM sets an option value persistently in Zookeeper, which makes the value permanent
within that one cluster.

Sounds like you want to persist values in an embedded Drillbit. In this case, there is no
Zookeeper. System options get written to the file system, but I don't know if they are persistent.
Any embedded Drill users know how this works?

One solution is to start Drill as a server, even if it runs on only one node. I don't know,
however, if the Drill server mode is supported on Windows. Any Windows users know this?

Finally, Kunal is correct. Since about a 18 months ago, the value for system/session options
are defined in Drill's config system. See [1].

Although Drill does not encourage this usage, you can customize these values in your drill-override.conf
as Kunal suggests. You can try this as a workaround for the fact that you are 1) using Embedded
Drill, 2) on Windows.

But, Drill is really a distributed tool, so you'd really want to run Drill on Linux as a server,
then allow the options to persist in Zookeeper.

Thanks,
- Paul

[1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/resources/drill-module.conf#L454


 

    On Tuesday, March 12, 2019, 11:31:59 AM PDT, Kunal Khatua <kunal@apache.org> wrote:
 
 
 That is correct... you'll need to run these on the command prompt for the
very first time. Like profiles, I believe, the ALTER SYSTEM commands also
have a permanent effect.

I'm not sure if there is a way for a user to provide a pre-configured set
of parameters to use during startup. You could see if setting these values
in DRILL_HOME/conf/drill-override.conf helps. A lot of the values you want
are originally set as defaults from an embedded file "drill-module.conf" ,
and drill-override.conf , well... overrides some of those values.

If this doesn't work, you can file an improvement/newFeature JIRA for this,
considering this is a nice feature to have.

~ Kunal


On Tue, Mar 12, 2019 at 3:18 AM PRAVEEN DEVERACHETTY <praveendk@gmail.com>
wrote:

> Hi Kunal, But where should i keep those statements? is there any drill
> startup script that i can run these alter staements? i think these scripts
> only on the drill command prompt right?
>
> Thanks,
> Praveen
>
> On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua <kunal@apache.org> wrote:
>
> > Executing an "alter system set param=value" usually persists the value.
> > Not sure if that works for an embedded mode.
> > Could you try and let us know if that works?
> > On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY <praveendk@gmail.com>
> > wrote:
> > I am using apache drill on windows platform. My requirement is to udpate
> > the following parameters during apache drill startup. These parameter
> > vaules may differ in each apache drill nodes. can you share me an example
> > of how to update these in windows. I know another option is to run on
> webUI
> > or postman? i dont want to do it post installation of apache drill. i
> want
> > these changes to be reflected during apache drill startup.
> >
> > *planner.cpu_load_average - 0.7*
> > *exec.queue.enable - true*
> > *exec.queue.large - 2*
> > *exec.queue.memory_ratio - 10.0*
> > *exec.queue.memory_reserve_ratio - 0.2*
> > *exec.queue.small - 4*
> >
> > Thanks,
> > praveen
> >
>
  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message