drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bowen Ding (Jira)" <j...@apache.org>
Subject [jira] [Created] (DRILL-7756) Storage IPFS: GroupScan: detect malformed datasets when planning
Date Fri, 26 Jun 2020 14:14:00 GMT
Bowen Ding created DRILL-7756:
---------------------------------

             Summary: Storage IPFS: GroupScan: detect malformed datasets when planning
                 Key: DRILL-7756
                 URL: https://issues.apache.org/jira/browse/DRILL-7756
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Other
            Reporter: Bowen Ding
            Assignee: Bowen Ding


The initial design of a Drill-queriable dataset on IPFS is a MerkleTree, where the leaf nodes
contain the actual data, and the intermediate nodes record the hashes of the leaf nodes. Not
all datasets in the wild are well-formed MerkleTrees, because IPFS allows arbitrary DAGs.


The current implementation in IPFSGroupScan does not check if a dataset is well-formed, and
if it contains recursive links, the planner will have no way to detect it and will loop forever.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message