calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <julianh...@gmail.com>
Subject Re: Shared enumerator
Date Wed, 04 Mar 2015 22:38:54 GMT
I presume that you are reading from some source and using ENUMERABLE convention, so that each
table scan ultimately turns into an Enumerable.enumerator() call?

If so, it doesn’t sound too difficult. You can make sure that the Enumerable that implements
your table actually looks in a cache first.

I suggest that you run your query with -Dcalcite.debug=true so that you can see the Java code
that is being generated. Find the piece of that code that reads from the table, and figure
out how to convert it to read from a cache. (You may need to write an implementation of Enumerator
that caches rows in a backing array.) Then you can change generator code, which is probably
the

  Expression getExpression(SchemaPlus schema, String tableName, Class clazz)

method of your Table implementation.

Julian

On Mar 2, 2015, at 2:19 AM, Liam Fisk <liamfisk@xtra.co.nz> wrote:

> Hi,
> 
> Is it possible to have a set of tables backed by a shared enumerator, and let Calcite
do its thing as usual?
> 
> I'm wanting to project a document with a mix of scalars and arrays into say 3 tables,
each with a different cardinality, and each allowing filters to be pushed down. Each table
would have their "own" Calcite enumerator, due to the mismatched cardinalities, but these
enumerators would be backed by a single shared iterator fetching documents. A fetch could
only occur once each enumerator had exhausted the rows extracted from the current document,
and thus all enumerators had signalled "next()".
> 
> This pattern would avoid having 3 tables created from different views of the underlying
document, and this would avoid multiple network round trips. Can this be achieved with Calcite,
or do the operations on each table need to be completely isolated?
> 
> Thanks,
> 
> Liam
> 


Mime
View raw message