hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20723) Allow per table specification of compaction yarn queue
Date Wed, 10 Oct 2018 19:00:01 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eugene Koifman updated HIVE-20723:
----------------------------------
    Description: 
Currently compactions of full CRUD transactional tables are Map-Reduce jobs submitted to a
yarn queue defined by hive.compactor.job.queue property.

If would be useful to be able to override this on per table basis by putting it into table
properties so that compactions for different tables can use different queues.

 

There is already ability to override other compaction related configs via table props, though
this will need additional handling to set the queue name {{CompactorMr.createBaseJobConf}}

[https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-TableProperties]

 

See {{CopactorMR.COMPACTOR_PREFIX}} and {{Initiator.COMPACTORTHRESHOLD_PREFIX}}

 

 

 

  was:
Currently compactions of full CRUD transactional tables are Map-Reduce jobs submitted to a
yarn queue defined by hive.compactor.job.queue property.

If would be useful to be able to override this on per table basis by putting it into table
properties so that compactions for different tables can use different queues.

 

There is already ability to override other compaction related configs via table props, though
this will need additional handling.

[https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-TableProperties]

 

See {{CopactorMR.COMPACTOR_PREFIX}} and {{Initiator.COMPACTORTHRESHOLD_PREFIX}}

 

 

 


> Allow per table specification of compaction yarn queue
> ------------------------------------------------------
>
>                 Key: HIVE-20723
>                 URL: https://issues.apache.org/jira/browse/HIVE-20723
>             Project: Hive
>          Issue Type: New Feature
>          Components: Transactions
>    Affects Versions: 2.0.0
>            Reporter: Eugene Koifman
>            Priority: Major
>
> Currently compactions of full CRUD transactional tables are Map-Reduce jobs submitted
to a yarn queue defined by hive.compactor.job.queue property.
> If would be useful to be able to override this on per table basis by putting it into
table properties so that compactions for different tables can use different queues.
>  
> There is already ability to override other compaction related configs via table props,
though this will need additional handling to set the queue name {{CompactorMr.createBaseJobConf}}
> [https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-TableProperties]
>  
> See {{CopactorMR.COMPACTOR_PREFIX}} and {{Initiator.COMPACTORTHRESHOLD_PREFIX}}
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message