cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Corentin Chary (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-12915) SASI: Index intersection can be very inefficient
Date Mon, 12 Dec 2016 16:45:58 GMT


Corentin Chary commented on CASSANDRA-12915:

To give you some perspective, in my specific use case the queries are user generated and our
only safeguard is that we put a limit to the number of generate results (5000). We already
process user-provided globs into CQL queries trying to choose the best indexes. But really
it would be better if the query optimizer was in the database rather than in our code.

I agree that my way of doing things tries to bypass a few things, but it also doesn't seem
to create regressions and provides some benefits. I propose that we split the discussion in
three parts:
1# Do you see a quick way of fixing the 'empty iterator' issue ?
2# Would you agree to chance the format to include proper cardinality estimation to the index
(so we can build upon later)
3# What would be the ETA for a proper query planner ? If O(months), would it make sense to
merge something like what I did to bring some performance improvement in the meantime ?

Thanks !

> SASI: Index intersection can be very inefficient
> ------------------------------------------------
>                 Key: CASSANDRA-12915
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: sasi
>            Reporter: Corentin Chary
>             Fix For: 3.x
> It looks like and be pretty inefficient in some cases.
Let's take the following query:
> SELECT data FROM table WHERE index1 = 'foo' AND index2 = 'bar';
> In this case:
> * index1 = 'foo' will match 2 items
> * index2 = 'bar' will match ~300k items
> On my setup, the query will take ~1 sec, most of the time being spent in disk.TokenTree.getTokenAt().
> if I patch RangeIntersectionIterator so that it doesn't try to do the intersection (and
effectively only use 'index1') the query will run in a few tenth of milliseconds.
> I see multiple solutions for that:
> * Add a static thresold to avoid the use of the index for the intersection when we know
it will be slow. Probably when the range size factor is very small and the range size is big.
> * CASSANDRA-10765

This message was sent by Atlassian JIRA

View raw message