calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rui Wang <amaliu...@apache.org>
Subject Re: Question of Calcite Dynamic Code Generation Feature
Date Wed, 13 May 2020 19:19:13 GMT
Based on my past experience in distributed computing, there is a difference
between a compilation time that generates java code and then compiles java
code into bytecode, and a runtime that runs bytecode by JVM.

The compilation will take time, but the runtime shouldn't be slower due to
a good generation of bytecode. It works for distributed computation. For
example, the query complies in your local machine, slower compilation time
(a few more minutes?) is ok, and then the bytecode will run on multiple
workers which can execute bytecode directly (so efficient). I don't have a
benchmark to show the difference between running java code and executing
bytecode, but in Apache Beam, after we switched to the dynamic code
generation, we didn't see a performance regression.


However I don't have enough experience on if single machine SQL engine can
benefit from this concept.


-Rui

On Wed, May 13, 2020 at 11:54 AM 徐泷泽 <15258826527@qq.com> wrote:

> Hi buddies !
> Our team is working on developing a&nbsp; mulit data source sql engine use
> Calcite.But something confuse us,&nbsp; why calcite generate Java code
> dynamicly on SQL query, it seen have performance issues in my opinion.
> And building dynamic code object is very hard,the code is unreadable and
> hard to understood, futhermore we can not debug it.
>
>
> i wander why it was designed like that, what advantage&nbsp; of that .we
> search on internet,&nbsp;
> but nobody can realy explain it(may be we not goot at search, and poor
> english).
> so anybody can tell me the history of this, or give us some article.
>
>
> applogize for my poor english,&nbsp; hope you know what i'm taking aboult.
> thank you very very very much !

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message