drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Khurram Faraaz <kfar...@mapr.com>
Subject Re: RESOURCE ERROR: External Sort encountered an error while spilling to disk
Date Tue, 12 Mar 2019 20:23:18 GMT
Can you also share the stack trace from drillbit.log for the below error
RESOURCE ERROR: External Sort encountered an error while spilling to disk

And what is the version of Drill that you are running ?
Please share the table definition and the number of rows in that table,
which is being queried.

Thanks,
Khurram

On Tue, Mar 12, 2019 at 12:17 PM Boaz Ben-Zvi <boaz@apache.org> wrote:

>   Hi Giovanni,
>
>      The error given by the External-Sort indicates a problem while
> spilling the excess memory into disk.
>
> When you enlarged the memory (from the default 2GB) to 8GB the LAG query
> may succeeded without spilling, hence circumvented the issue.
>
> Yes, you can keep enlarging the memory and run w/o spilling, but better
> check and fix the root issue.
>
> How is your spilling configured - check the filesystem -
> "drill.exec.spill.fs" and the directories - "drill.exec.spill.directories"
>
> The default is the local filesystem, and into /tmp .  It is possible
> that very little disk space is available for /tmp .
>
>       Thanks,
>
>           -- Boaz
>
> On 3/12/19 6:03 AM, Giovanni Conte wrote:
> > Hello,
> > I am doing a LAG and a SUM query over a PCAP dataset of 300Mb.
> > I get this error:
> > RESOURCE ERROR: External Sort encountered an error while spilling to disk
> >
> > Then I changetd
> > planner.memory.max_query_memory_per_node ---> 8589934592
> > and with this I am able to perform the LAG but not the SUM.
> >
> > I have no problem of memory since I am working on a server with 72 cores
> > and 256 GB of RAM.
> > Which other parameter should I change to avoid the RESOURCE ERROR?
> > Can I go over 8589934592?
> > Thank you very much,
> >
> > Giovanni
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message