spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikhil Goyal <nownik...@gmail.com>
Subject Driver OOM when using writing parquet
Date Mon, 06 Aug 2018 23:59:20 GMT
Hi guys,

I have a simple job which reads LZO thrift files and writes them in Parquet
Format. Driver is going out of memory. Parquet writer does keep some meta
info in memory but that should cause the executor to go out of memory. No
computation is being done on the driver. Any idea what could be the reason?


Thanks
Nikhil

Mime
View raw message