drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse Yates <jesse.k.ya...@gmail.com>
Subject Re: deploy dockerized drill cluster
Date Mon, 01 Aug 2016 01:07:19 GMT
Ran into this myself today, so gonna try and close the loop.

Looking at the 1.6 codeline (assuming its the same in 1.7), on startup the
SystemOptionManager on pulls out the sys.options line from the config and
uses that to build the PersistentStoreConfig (which get initialized from
the store in #init()). However, this obviously doesn't take into account
the config level options, only the startup options since those get pulled
out to create everything else.

It seems simple enough to wrap the SystemOptionManager into a
FallbackOptionManager that falls back to the config file, if its not in the
persistent store (or something like that).

Unless someone already filed a JIRA on this?

On Mon, Jul 25, 2016 at 10:05 AM Scott Kinney <scott.kinney@stem.com> wrote:

> it's still not picking up the store.json* config changes
>
> The only way I can see to set these is with running ALTER SYSTEM query
> after drill api is up.
>
>
> ________________________________
> Scott Kinney | DevOps
> stem   |   m  510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: John Omernik <john@omernik.com>
> Sent: Monday, July 25, 2016 8:21 AM
> To: user
> Subject: Re: deploy dockerized drill cluster
>
> Try (for the sake of the conversation here) using host networking, and see
> if it changes how successful your setup is.  (I know bridged is preferred,
> but try the host side and see what happens)
>
> John
>
> On Mon, Jul 25, 2016 at 10:06 AM, Scott Kinney <scott.kinney@stem.com>
> wrote:
>
> > I'm running the docker in bridged network mode.
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem   |   m  510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: John Omernik <john@omernik.com>
> > Sent: Sunday, July 24, 2016 8:28 AM
> > To: user
> > Subject: Re: deploy dockerized drill cluster
> >
> > Are you running Drill in host networking or bridged networking?
> >
> > On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <scott.kinney@stem.com>
> > wrote:
> >
> > > Hm, i must have set those another way in embeded mode. I can't see
> where.
> > > Those settings persist between drill restarts.
> > >
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem   |   m  510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> > > ________________________________________
> > > From: Abhishek Girish <abhishek.girish@gmail.com>
> > > Sent: Friday, July 22, 2016 1:57 PM
> > > To: Drill User List
> > > Subject: Re: deploy dockerized drill cluster
> > >
> > > You can set boot level start-up options in drill-override.conf [1].
> But I
> > > don't think we can do the same with the system options. Someone else
> can
> > > comment if there is a workaround.
> > >
> > > On why it works for you with drill-embedded, is something I'm trying to
> > > understand. I attempted this and couldn't manage to get those options
> to
> > > show up in embedded mode.
> > >
> > > [1] https://drill.apache.org/docs/start-up-options/
> > >
> > > On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <scott.kinney@stem.com>
> > > wrote:
> > >
> > > > I have built a drill docker images very much like
> > > > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> > > >
> > > >
> > > > I volume mount a drill-override.conf file that looks like:
> > > >
> > > > drill.exec:{
> > > >   cluster-id: drill1,
> > > >   zk.connect: 192.1.1.1:2181",
> > > >   sys.store.provider.local.path: "/drill-storage",
> > > >   store.json.all_text_mode: True,
> > > >   store.json.read_numbers_as_double: True
> > > > }
> > > >
> > > > it seems to be connecting to the zookeeper (otherwise it would fail
> > > right?
> > > > i should actually confirm this).
> > > >
> > > > I know it is picking up my s3 plugin that i volume mount inside
> > > > /drill-storage but it's not setting the json all_text_mode and
> > > > read_numbers_as_double.
> > > >
> > > > I have a drill-embeded instance that seems to pick these json setting
> > > from
> > > > the drill-override file just fine.
> > > >
> > > > Can you see what I'm doing wrong?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > Scott Kinney | DevOps
> > > > stem <http://www.stem.com/>   |   m  510.282.1299
> > > > 100 Rollins Road, Millbrae, California 94030
> > > >
> > > > This e-mail and/or any attachments contain Stem, Inc. confidential
> and
> > > > proprietary information and material for the sole use of the intended
> > > > recipient(s). Any review, use or distribution that has not been
> > expressly
> > > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > > intended recipient, please contact the sender and delete all copies.
> > > Thank
> > > > you.
> > > >
> > >
> >
>
-- 
Jesse Yates
Founder/CEO Fineo.io

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message