phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1091) Implement hard ceiling on per-query concurrency
Date Tue, 05 Aug 2014 22:33:13 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086910#comment-14086910
] 

James Taylor commented on PHOENIX-1091:
---------------------------------------

Phoenix has a lot of knobs and dials to impact concurrency and fairness. Phoenix already round
robins between queued work and requires that the all scans that cover the query be submitted
together (as merge sorts are done among them). First, we should build a concurrency test-bed
harness so we can understand better how to use the existing knobs and dials. For example,
experiment with bump up the thread pool queue size (as [~lhofhansl] suggested), and potentially
increase the max & target concurrency to chunk up a query into smaller pieces to improve
on fairness and prevent starvation.

Also related is PHOENIX-180 actively being worked on by [~ramkrishna]. Once this is in place,
work can be chunked up into more or less even sizes.

> Implement hard ceiling on per-query concurrency
> -----------------------------------------------
>
>                 Key: PHOENIX-1091
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1091
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Eli Levine
>
> Phoenix has targetConcurrency and maxConcurrency configuration settings. However, it
seems that they only come into effect when they are higher than the number of regions. In
clients operating over tables with a large number of regions this could result in the consumption
of a large number of client-side threads per query, potentially leading to starvation of other
queries in Phoenix clients that support concurrent queries (such as highly multi-tenant environments).
> Implementation of a hard per-query concurrency limit is suggested to avoid potential
starvation due to each query taking too many client threads.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message