calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop Johnson <anoop.k.john...@gmail.com>
Subject Calcite on Partitioned Databases
Date Mon, 07 Mar 2016 18:58:42 GMT
Hello Everyone -

 We have an in-house distributed database with our own custom query engine.
We're considering using Calcite to replace our current query engine.

I looked through the examples and  one of the adapters. One thing I haven't
quite figured out is using Calcite when the data is partitioned and you
need to fan out the query into all the partitions and combine the partial
results.

There are several challenges here - for instance, some operations like AVG
are not partitionable, so they have to be rewritten as COUNT and SUM and
only at the final step.

We could always write custom code to rewrite the query and reassemble the
results. For instance, I saw a planner rule[1] in the Calcite codebase that
does something similar.

Since this is a common problem, I was wondering if there was a standard way
of handling this use case or if there were any example plugin I could look
at.

Thanks,
Anoop


[1]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/AggregateReduceFunctionsRule.java

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message