drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-13) Storage Engine: Define Java Interface
Date Tue, 22 Jan 2013 22:44:13 GMT

    [ https://issues.apache.org/jira/browse/DRILL-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560126#comment-13560126
] 

Jacques Nadeau commented on DRILL-13:
-------------------------------------

New re-thoughts...

The primary interface is the Storage Engine Capabilities API.  It should describe everything
that the particular storage engine supports.  This includes whether the storage engine supports
serialization, deserialization, what types of logical operator capabilities it supports internally.
 It also needs to include a description of statistics capabilities (e.g. supports approximate
row keys, average row size, total data size, data distribution statistics, etc) and metadata
capabilities

Statistics API: Provide the actual statistics information that is utilized during query planning.
Metadata API: Provide information about the available sub data sources (tables, keyspaces,
etc) along with locality information, schema information, type information, primary and secondary
indices types, partitioning information,  etc.  Portions of this information are used in query
parsing.  Others in query planning.  Others portions in Execution planning.
DeserializationAPI - Convert a particular data source into one of our two canonical in-memory
formats.  (row-based or column-based).  Additionally support particular types of logical operation
pushdown.
Serialization - Serialize the in-memory format back into the persistent storage format.

                
> Storage Engine: Define Java Interface
> -------------------------------------
>
>                 Key: DRILL-13
>                 URL: https://issues.apache.org/jira/browse/DRILL-13
>             Project: Apache Drill
>          Issue Type: Task
>            Reporter: Jacques Nadeau
>
> We're going to need to define a storage engine API.  At a minimum, we'll need to generate
a Java one.  We will probably need to also create a CPP one.  This task is for the former.
 Things that are likely to be included in a the Java interface are: reader (scanner), writer,
capabilities interface, schema interface, statistics interface, data layout and ordering

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message