drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avner Levy <avner.levy+dr...@gmail.com>
Subject Controlling number of fragments with Apache Drill CREATE TABLE AS (CTAS) and parquet
Date Wed, 25 Mar 2020 21:45:47 GMT
I'm running Drill 1.17.0 in embedded mode and reading/writing files to S3.
When I create new CTAS tables it creates 6 small fragments parquet files
(1.5MB each).
Is there a way to control the number of fragments?
To limit it to 1 fragment only?
Already tried the following but it didn't help:

ALTER SYSTEM SET store.parquet.block-size = 1073741824;
ALTER SYSTEM SET store.parquet.writer.use_single_fs_block = true;

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message