drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [drill] paul-rogers commented on a change in pull request #1953: Add docs for Drill Metastore
Date Tue, 04 Feb 2020 03:12:29 GMT
paul-rogers commented on a change in pull request #1953: Add docs for Drill Metastore
URL: https://github.com/apache/drill/pull/1953#discussion_r374450331
 
 

 ##########
 File path: _docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md
 ##########
 @@ -0,0 +1,69 @@
+---
+title: "Drill Iceberg Metastore"
+parent: "Drill Metastore"
+date: 2020-01-31
+---
+
+Drill uses Iceberg Metastore implementation based on [Iceberg tables](http://iceberg.incubator.apache.org).
For Drill 1.17,
+ this is default Drill Metastore implementation. For details on how to configure Iceberg
Metastore implementation and
+ its option descriptions, please refer to [Iceberg Metastore docs](https://github.com/apache/drill/blob/master/metastore/iceberg-metastore/README.md).
+
+{% include startnote.html %}
+Iceberg table supports concurrent writes and transactions but they are only effective on
file systems that support
+ atomic rename.
+If the file system does not support atomic rename, it could lead to inconsistencies during
concurrent writes.
 
 Review comment:
   This worries me. S3 does not support atomic renames: renames make a copy. Will metastore
work on S3? If not, we should call out that Drill 1.17 supports only the local file system,
HDFS and similar systems. Drill 1.17 does not support S3 because S3 does not provide atomic
rename.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message