drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From scott <tcots8...@gmail.com>
Subject CTAS memory leak
Date Wed, 29 Aug 2018 23:09:59 GMT
Hi all,
I've got a problem using the create table as option I was hoping someone
could help with. I am trying to create parquet files from existing json
files using this method. It works on smaller datasets, but when I try this
on a large dataset, drill will take up all memory on my servers until it
swaps and then crashes. I'm running version 1.12 on centos 7. I've got my
drillbits set to xmx 8G, which seems to work for most queries and it does
not exceed that limit by much, but when I do the CTAS, it just keeps
growing without bounds.
I run 4 drillbits on each server with these settings: -Xms8G -Xmx8G
-XX:MaxDirectMemorySize=10G on a server that has 48G RAM.
Has anyone else experienced this? Are there any workarounds you can suggest?

Thanks for your time,
Scott

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message