drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Omernik <j...@omernik.com>
Subject Shutdown Options for Drillbits in Clustered Mode
Date Tue, 03 May 2016 18:24:01 GMT
DRILL-4286 and DRILL-2656 both deal with this, so I won't open another
JIRA.  This feature would be extremely beneficial for enterprise
deployments of Drill, especially where the DRILL cluster may be sharing
resources with other frameworks.   Drill is a wonderful Adhoc query system,
and for example, you may have all nodes running Drill bits during the day
to serve adhoc user loads, but the ability to shutdown say 50-75% of drill
bits at night where the adhoc work load may drop off significantly, would
be a huge benefit. These resources could then be allocated to other batch
drops, etc.

These JIRAs are important in that to make this feasible, there has to be a
way to shutdown drill bits without negatively impacting currently running
queries.  Users would not tolerate the idea that at 9:00pm every night
queries will just fail as some of the drillbits are shutdown.   In a very
binary environment, you could monitor for when there are queries running,
but that's a race condition to identify that no queries are running, and
that you can ID and shutdown the right bits prior to someone submitting a
query.

This feature would be hugely beneficial.

John

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message