phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoffrey Jacoby (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-4984) PhoenixMRJobSubmitter doesn't prevent rebuild jobs from being started multiple times
Date Fri, 19 Oct 2018 17:15:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Geoffrey Jacoby updated PHOENIX-4984:
-------------------------------------
    Summary: PhoenixMRJobSubmitter doesn't prevent rebuild jobs from being started multiple
times  (was: PhoenixMRJobSubmitter does not get the correct list of jobs to submit for IndexTool
jobs)

> PhoenixMRJobSubmitter doesn't prevent rebuild jobs from being started multiple times
> ------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4984
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4984
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Chinmay Kulkarni
>            Assignee: Geoffrey Jacoby
>            Priority: Major
>
> When trying to get the list of jobs to submit, we get the already scheduled jobs list
from the Yarn Resource Manager and exclude those jobs inside {{PhoenixMRJobSubmitter#getJobsToSubmit}},
however a naming format difference prevents correctly removing already running/submitted jobs.
> In {{IndexTool.java}}, we use the following convention for naming the M/R job:
>  INDEX_JOB_NAME_TEMPLATE = "PHOENIX_<schema name>*.*<data table name>_INDX_<index
name>";
> However, I see the following log lines for candidate jobs:
> _Candidate Indexes to be built as seen from SYSTEM.CATALOG - PHOENIX_<data table name>_INDX_<index
name> ... _
> And the following for already submitted jobs as got from Yarn:
> _Already Submitted/Running MR index build jobs - [PHOENIX_<schema name>.<data
table name>_INDX_<index name>]_
> Due to this naming conflict (no '.'), even though an index build M/R job is running for
a given index, this is not detected correctly and another one can be started for the same
index. This can lead to unnecessary load on the region servers hosting regions for the index



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message