kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Burkert <danburk...@apache.org>
Subject Re: Multi-level partitions question
Date Thu, 11 Oct 2018 20:05:34 GMT
Hi Boris,

The two examples you gave are exactly equivalent; the relative ordering of
hash levels has no effect on query performance, hotspotting, or anything
else.  Given that 60% of your queries don't specify a specific customer_id,
it does make sense to use hash(shop_id), hash(customer_id) instead of
combining them in a single hash level as hash(shop_id, customer_id),
however the trade-off is that the hotspotting resistance isn't as good.  If
the shop_id and customer_id columns aren't skewed to begin with that's not
a concern, though.

- Dan

On Thu, Oct 11, 2018 at 12:14 PM Boris Tyukin <boris@boristyukin.com> wrote:

> Hi guys,
> Read this doc
> https://kudu.apache.org/docs/schema_design.html#multilevel-partitioning
> and I have a question on this particular statement
> "Scans on multilevel partitioned tables can take advantage of partition
> pruning on any of the levels independently"
>
> Does it mean, that both strategies below would be equivalent in terms of
> performance (i.e. minimum scans)
>
> partition by hash(shop_id), hash(customer_id)
> vs.
> partition by hash(customer_id), hash(shop_id)
>
> 60% of the queries are using both shop_id and customer_id but 40% of
> queries need to pull all customers for a specific shop_id. And almost never
> by customer_id alone (customer_id is not unique across shops and is
> assigned per shop).
>
> At the same time, if I partition by customer_id first,  partitions will be
> distributed more evenly.
>
> Thanks!
> Boris
>
>
>
>

Mime
View raw message