ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Izhikov <nizhi...@apache.org>
Subject Re: New SQL execution engine
Date Fri, 27 Sep 2019 11:14:14 GMT
Hello, Andrey.

> Ignite SQL layer has some issues that can't be fix with changes in Ignite
> only, and we are blocked with H2.

What are these issues?
Can you make it specific and send a tickets for this issues?

> 3. Replace H2 with smth else.

Actually, I support this decision in general.
But, to make a right choise for H2 replacement we should carefully discuss such huge replacement.

So far, I can't see any written down(in IEP) requirements for SQL engine.
Let's do it and discuss them.

В Пт, 27/09/2019 в 13:39 +0300, Andrey Mashenkov пишет:
> Hi Nikolay,
> 
> Let me add my 5- cent here.
> 
> Ignite SQL layer has some issues that can't be fix with changes in Ignite
> only, and we are blocked with H2.
> To resolve these issues we can:
> 1. Donate some changes to H2 and wait for it's next release. But there are
> more cons than pros and I think we can't rely on H2 project anymore.
> - There is no guarantee our changes will be approved by H2 community.
> - We definitely won't to depend on H2 product lifecycle.
> - New H2 features (like parallel multi-statement query processing in latest
> release) force Ignite for significant changes\refactoring in Ignite SQL
> layer with no visible benefits.
> Every next release it becomes harder to upgrade H2 dependency.
> - Latest H2 versions causes questions about their stability.
> 
> Hot issues are
> - Large intermediate results inside H2 internals can cause OOM for some
> kind of queries. Ignite can't handle this anyhow for now without reworking
> H2 code.
> - HashJoins
> - Ignite can't start multi-step queries, but 2-step (map-reduce) only.
> - It is not possible to apply optimizations on query plan as no logical
> plan actually doen't exists. H2 execution plan is hard-wired with H2
> internals and can't be easily transformed.
> Implementing a new good planner over H2 looks like a huge task.
> 
> 2. Fork H2.
> We already done this in GridGain (you can found H2 module in GridGain
> community edition) as fastest way to unblock work on SQL improvements.
> But this way doesn't look like a good one for Ignite, regarding our
> experience.
> - H2 code can't be included into Ignite at all.
> H2 license are MIT and EPL. From one side they can't be changed to Apache
> Licence. From other side Apache Foundation don't want to host any code
> licensed with other than Apache License.
> GridGain is ok with this, but Apache Foundation won't.
> 
> - We can made separate H2 fork project with it's own lifecycle with full
> control over it and publish it in Maven Central.
> This doen't seem like a big deal. But will causes additional difficulties
> in development, test and release processes of Ignite.
> This way seems bring much pain for every contributor.
> 
> 3. Replace H2 with smth else.
> E.g. with Apache Calcite.
> - Calcite is a framework and it is designed very flexible and extendable.
> - Every it's part can be replaced with our own implementation.
> - Apache License is out of the box =)
> 
> So, summary:
> 1-st way of pain we have now and it slows down Ignite SQL layer developing.
> 2-nd looks few better, but seems bring Ignite to nowhere in prospect.
> 3-rd is a risky, but promissory way.
> 
> 
> On Fri, Sep 27, 2019 at 12:16 PM Nikolay Izhikov <nizhikov@apache.org>
> wrote:
> 
> > Hello, Igor.
> > 
> > Thanks for starting this discussion.
> > 
> > I think we should take a step back in it and answer the following
> > questions:
> > 
> > 1. What the exact issues with the H2 integration?
> > Can you send a tickets links?
> > Can we label all H2 integration issues in JIRA? I propose to use "h2"
> > label.
> > 
> > 2. What are the requirements for the new SQL engine?
> > We should write it down and discuss.
> > 
> > 3. What options do we have?
> > Are there any alternatives to Calcite on the market?
> > We did the wrong choice that looked obvious one time.
> > So we should carefully avoid it at this time.
> > 
> > 4. What is improvements of Ignite we want to make with the new engine?
> > 
> > 
> > В Пт, 27/09/2019 в 08:44 +0000, Igor Seliverstov пишет:
> > > Hi Igniters!
> > > 
> > > As you might know currently we have many open issues relating to current
> > 
> > H2 based engine and its execution flow.
> > > 
> > > Some of them are critical (like impossibility to execute particular
> > 
> > queries), some of them are majors (like impossibility to execute particular
> > queries without pre-preparation your data to have a collocation) and many
> > minors.
> > > 
> > > Most of the issues cannot be solved without whole engine redesign.
> > > 
> > > So, here the proposal:
> > 
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028084
> > > 
> > > I'll appreciate if you share your thoughts on top of that.
> > > 
> > > Regards,
> > > Igor
> 
> 

Mime
View raw message