drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boaz Ben-Zvi <bben-...@mapr.com>
Subject Re: Drill: Memory Spilling for the Hash Aggregate Operator
Date Sat, 14 Jan 2017 15:34:15 GMT
  Sorry for no attachment (Apache mail rules) -- Here is a link to the document:


DrillSpillmemoryforHashAggregation.pdf - https://drive.google.com/file/d/0ByUg32jfEW16ajNiQlVRczhPTjA/view?usp=sharing

[https://lh3.googleusercontent.com/U9FNbWEBljT-HDRBE1-vhMnE4Ug5YFgutztvbys2UnTiVp-FQX6mzQ=w1200-h630-p]<https://drive.google.com/file/d/0ByUg32jfEW16ajNiQlVRczhPTjA/view?usp=sharing>

DrillSpillmemoryforHashAggregation.pdf<https://drive.google.com/file/d/0ByUg32jfEW16ajNiQlVRczhPTjA/view?usp=sharing>
drive.google.com



    -- Boaz

________________________________
From: Julian Hyde <jhyde@apache.org>
Sent: Friday, January 13, 2017 11:00 PM
To: dev@drill.apache.org
Subject: Re: Drill: Memory Spilling for the Hash Aggregate Operator

The attachment didn't come through. I'm hoping that you settled on a "hybrid" hash algorithm
that can write to disk, or write to memory, and the cost of discovering that is wrong is not
too great. With Goetz Graefe's hybrid hash join (which can be easily adapted to hybrid hash
aggregate) if the input ALMOST fits in memory you could process most of it in memory, then
revisit the stuff you spilled to disk.

> On Jan 13, 2017, at 7:46 PM, Boaz Ben-Zvi <bben-zvi@mapr.com> wrote:
>
>  Hi Drill developers,
>
>      Attached is a document describing the design for memory spilling implementation
for the Hash Aggregate operator.
>
>      Please send me any comments or questions,
>
>         -- Boaz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message