drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: show tables taking long time on hive plugin
Date Thu, 18 Aug 2016 19:33:30 GMT
You are right that there is improvement room for "Show table" command.

Today, Drill process "Show table" as a query against
INFORMATION_SCHEMA.TABLE WHERE schema = 'Hive_schema', which will not
only get the table names but also table instances. In fact, "Show
Table" should only return table names. If Drill does not ask table
instances from hive meta store, "Show Table" will run much faster.

Could you please file a JIRA for such issue, if there is no one opened yet?




On Thu, Aug 18, 2016 at 8:01 AM, Sungwook Yoon <syoon@maprtech.com> wrote:
> Yes, but...
> It has not much to do with taking 2mins reading tables compared to 1 second
> from beeline...
> Beeline is not fast due to caching..
>
> Sungwook
>
>
>
> On Thu, Aug 18, 2016 at 7:49 AM, Zelaine Fong <zfong@maprtech.com> wrote:
>
>> Have you tried tweaking Drill's Hive metastore cache parameters?  See
>> https://drill.apache.org/docs/hive-metadata-caching/.
>>
>> -- Zelaine
>>
>> On Wed, Aug 17, 2016 at 10:02 PM, Sungwook Yoon <syoon@maprtech.com>
>> wrote:
>>
>> > Hi,
>> >
>> > Drill 1.6.0
>> > MapR 3.1.1
>> > Hive 0.13
>> > HS2 SQL Authorization
>> > Hive Meta storage authorization
>> >
>> > Hive Default database has 1400 tables.
>> >
>> > I see significant slow down in "show tables" when Hive database is bigger
>> > than 500 tables.
>> > It takes 3 minutes to get the resultset.
>> > There is no obvious logs on drillbit or zookeeper to help diagnose the
>> > situation.
>> >
>> > Beeline is snappy, returns 1400 tables in a second or so.
>> >
>> > Why Drill "show tables" on Hive plugin takes this long?
>> >
>> > Thanks,
>> >
>> > Sungwook
>> >
>>

Mime
View raw message