calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: Introduction and question on lattices and adapters
Date Thu, 21 Jan 2016 16:21:57 GMT
On Thu, Jan 21, 2016 at 7:12 AM, Michael Mior <mmior@uwaterloo.ca> wrote:

>
> Is there anything on the radar for Phoenix to suggest materialized views or
> indexes to users based on queries? This is something I'd be particular
> interested in. Being able to leverage the cost-based optimizer from Calcite
> makes this task much more approachable.
>

Yes, we already use the Calcite optimizer to choose between using an index
versus the data table. We also use it to determine join ordering and
strategy. We don't yet use it to choose between a range scan and a skip
scan[1] - this would be a good, self contained contribution to get your
feet wet. Another derivative of this is to force a skip scan when the
leading column in the primary key isn't being filtered.

[1] https://phoenix.apache.org/skip_scan.html


>
>
> 2016-01-20 18:34 GMT-05:00 James Taylor <jamestaylor@apache.org>:
>
> > Yes, this is a good fit for the current plans of Phoenix + Calcite. We
> > actually model secondary indexes as materialized views. We're interested
> in
> > being able to use materialized views in other situations too (i.e. to
> > service aggregate queries). Are you familiar with Apache Kylin? One idea
> we
> > had (since Kylin uses Calcite as well) is to see if it's possible for
> > Phoenix + Calcite + Kylin to inter-operate.
> >
> > Take a look at the presentations here for more documentation:
> > http://phoenix.apache.org/resources.html
> > Also, in general, our website is a good source for information:
> > http://phoenix.apache.org. Take a look at the pages under the Features
> > menu
> > for more detail.
> >
> > On Wed, Jan 20, 2016 at 2:23 PM, Michael Mior <mmior@uwaterloo.ca>
> wrote:
> >
> > > James,
> > >
> > > Thanks for the reply! I understand HBase has no query language per se,
> > but
> > > if I was trying to create the most basic possible adapter for HBase, it
> > > seems like the most straightforward approach would be to expose HBase
> > > tables and implement scans as a possible query plan operator. This
> would
> > > require the user to specify a schema, but of course this needs to
> happen
> > > with Phoenix anyway.
> > >
> > > That said, given that Phoenix does exist, the approach makes a lot of
> > sense
> > > and obviously adds additional features. In general, I'm interested in
> > > incorporating the use of materialized views when querying databases
> such
> > as
> > > Cassandra and HBase and exploring what needs to change from the
> > relational
> > > world. If there's a way that this fits into the current plans for
> > Phoenix +
> > > Calcite, I'd love to talk more.
> > >
> > > I'm still trying to wrap my head around both code bases. If there's any
> > > particular documentation you can point to that you think would be
> > helpful,
> > > do let me know!
> > >
> > > Cheers,
> > > --
> > > Michael Mior
> > > mmior@uwaterloo.ca
> > >
> > > 2016-01-20 16:37 GMT-05:00 James Taylor <jamestaylor@apache.org>:
> > >
> > > > Hi Michael,
> > > > We'd love for you to get involved on the Phoenix on Calcite work. I'd
> > be
> > > > good to hear more on your thoughts about our implementation being
> "too
> > > > Phoenix-specific to enable the use of vanilla HBase". HBase has no
> > query
> > > > language. We're basically leverage Calcite to do query optimization
> (at
> > > the
> > > > relational algebra level) over HBase.
> > > > Thanks,
> > > > James
> > > >
> > > > On Wed, Jan 20, 2016 at 1:32 PM, Michael Mior <mmior@uwaterloo.ca>
> > > wrote:
> > > >
> > > > > Hello Calcite devs!
> > > > >
> > > > > I'm a PhD student at the database group of the University of
> Waterloo
> > > in
> > > > > Ontario, Canada. My work has focused on schema optimization for
> NoSQL
> > > > > databases (specifically Cassandra for the time being). You can see
> a
> > > > piece
> > > > > of my recent work at the link below where I developed an automated
> > tool
> > > > to
> > > > > design efficient schemas for Cassandra based on a description of
> > > > > application workloads.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://www.researchgate.net/publication/289372760_NoSE_Schema_Design_for_NoSQL_Applications
> > > > >
> > > > > I'm really interested on getting involved in Calcite and I have two
> > > main
> > > > > questions
> > > > >
> > > > > 1) What is the status of lattices in Calcite? I read the
> > documentation
> > > (
> > > > > https://calcite.apache.org/docs/lattice.html) but it's unclear
> > what's
> > > > been
> > > > > implemented or how it works. Specifically, the documentation seems
> to
> > > > > suggest that Calcite will automatically select tiles to create. Are
> > > there
> > > > > any examples of using lattices I can look at?
> > > > >
> > > > > 2) Is there any work being done on an adapter for Cassandra or
> HBase?
> > > > This
> > > > > is something I would be interested in contributing to. I know the
> > > Phoenix
> > > > > folks are working on integrating with Calcite, but it looks like
> > their
> > > > > implementation is too Phoenix-specific to enable the use of vanilla
> > > > HBase.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Cheers,
> > > > > --
> > > > > Michael Mior
> > > > > mmior@uwaterloo.ca
> > > > > http://michael.mior.ca/
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message