drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: query performance with unequal drillbits
Date Thu, 23 Aug 2018 01:23:31 GMT
Cool

On Wed, Aug 22, 2018, 17:07 scott <tcots8888@gmail.com> wrote:

> Thanks Ted and Paul. I've been experimenting with the "hack" method. It
> works somewhat, and I guess will have to do.
>
> On Tue, Aug 21, 2018 at 2:50 PM Ted Dunning <ted.dunning@gmail.com> wrote:
>
> > A cheap hack is to use multiple smaller drillbits. Put more drillbits on
> > the hefty machines and fewer on the weaker ones.
> >
> > This increases overheads, but it might help you out.
> >
> >
> >
> > On Tue, Aug 21, 2018 at 1:48 PM scott <tcots8888@gmail.com> wrote:
> >
> > > Hi community,
> > > I am trying to find a way to tune Drill so that weaker drillbits get
> less
> > > data to work on so that the weak link doesn't drag my performance
> down. I
> > > have drillbits running on a variety of hardware and sometimes these
> > shared
> > > resources get really slow. It seems that the query plan always evenly
> > > divides the data fragments so that each drillbit gets the same data to
> > chew
> > > on. How do I make it give weaker drillbits less data?
> > >
> > > Alternatively, is there a way to limit and queue fragments of the query
> > and
> > > leave them unassigned, then assign to drillbits as their resources
> become
> > > free, similar to MapReduce?
> > >
> > > Thanks for you time,
> > > Scott
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message