hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20250) Option to allow external tables to use query results cache
Date Wed, 01 Aug 2018 17:53:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565715#comment-16565715
] 

Jason Dere commented on HIVE-20250:
-----------------------------------

Had some offline discussion with [~hagleitn] on this and there is concern that users might
too easily enable hive.query.results.cache.external.tables.enabled/hive.query.results.cache.nontransactional.tables.enabled
without considering the ramifications - that Hive will continue to serve stale cached results
if such tables are modified outside of Hive.

One option we have talked about is replacing hive.query.results.cache.max.entry.lifetime with
a new timeout setting that only applies to non-transactional/external tables, as both [~gopalv]
and [~hagleitn] have argued that cached results from transactional tables do not need to be
timed out (there is other logic to automatically invalidate results on transactional tables
if a transactional table has been updated using a transaction). So hive.query.results.cache.external.tables.enabled/hive.query.results.cache.nontransactional.tables.enabled/hive.query.results.cache.max.entry.lifetime
would be replaced by some new setting like hive.query.results.cache.nontransactional.result.lifetime
(defaulted to 0), where setting this value with a positive value would enable results caching
for non-transactional tables, where the cached result would be considered valid for as long
as the value of that setting.

> Option to allow external tables to use query results cache
> ----------------------------------------------------------
>
>                 Key: HIVE-20250
>                 URL: https://issues.apache.org/jira/browse/HIVE-20250
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Jason Dere
>            Priority: Major
>         Attachments: HIVE-20250.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message