kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mac Noland <mcdonaldnol...@gmail.com>
Subject Re: Tablets Per Tablet Server
Date Fri, 30 Nov 2018 22:15:05 GMT
Thanks for the feedback.  We are going to cut our small table partitions by
half with that feature to not require a partition.


On Thu, Nov 29, 2018 at 7:03 PM helifu <hzhelifu@corp.netease.com> wrote:

> Hi Mac,
>
>
>
> To many of the small tables which have the minimum of 2 partitions, there
> is another way to help you to save half the tablets.(Thanks to Andrew, who
> shared this info to me on the slack)
>
> You can find the detail in both link:
>
> https://jira.apache.org/jira/browse/IMPALA-5546   IMPALA-2.10.0
>
>
> https://www.cloudera.com/documentation/enterprise/latest/topics/kudu_impala.html#concept_g51_5vk_ft
>
>
>
> *Note:* In Impala included in CDH 5.13 and higher, the PARTITION BY clause
> is optional for Kudu tables. If the clause is omitted, Impala automatically
> constructs a single partition that is not connected to any column. Because
> such a table cannot take advantage of Kudu features for parallelized
> queries and query optimizations, omitting the PARTITION BY clause is only
> appropriate for small lookup tables.
>
>
>
>
>
> 何李夫
>
> 2018-11-30 09:02:21
>
>
>
> *发件人:* user-return-1546-hzhelifu=corp.netease.com@kudu.apache.org
> <user-return-1546-hzhelifu=corp.netease.com@kudu.apache.org> *代表 *Mac
> Noland
> *发送时间:* 2018年11月30日 1:58
> *收件人:* user@kudu.apache.org
> *主题:* Tablets Per Tablet Server
>
>
>
> Good day,
>
>
>
> We have a number of customers using Kudu for a large number of tables from
> db/warehouse offload use cases and commonly run into situations where our
> tablets are over 2000 per tablet server.  Many of the tables are actually
> small, but with the minimum of 2 partitions and default of 3 replicas, we
> end up chewing up 6 tablets minimally for each table.  Being we have
> customers way over this and are running fine, we understand this is not a
> hard guideline, but overall does cause some concern with some customers
> knowing we are outside of the written guidelines.
>
>
>
> *The maximum number of tablets per tablet server is 2000,
> post-replication, but we recommend 1000 tablets or fewer per tablet server.*
> https://kudu.apache.org/docs/known_issues.html#_scale
>
>
>
> Do we know how this number was picked and if there is any work being done
> to look at what it would take to increase the number?  Or things we could
> contribute to help increase that number?  Not 100% sure what we'd be
> signing up for to do that, but always looking for opportunity to help.
>
>
>
> Thanks in advance.
>
>
>
> Mac
>
>
>
>
>

Mime
View raw message