helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Congrui Ji (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HELIX-601) Allow work flow to schedule dependency jobs in parallel
Date Thu, 18 Jun 2015 22:46:00 GMT
Congrui Ji created HELIX-601:

             Summary: Allow work flow to schedule dependency jobs in parallel
                 Key: HELIX-601
                 URL: https://issues.apache.org/jira/browse/HELIX-601
             Project: Apache Helix
          Issue Type: New Feature
            Reporter: Congrui Ji

Currently, Helix won't schedule dependency jobs in a same work flow. For example, if Job2
depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.

However, if some participant is very slow, then all dependency jobs is waiting for that single

Helix should be able to schedule multiple jobs according to a parameter.

1. Introduce parallel count parameter in work flow and job queue.
2. Dependency jobs can be scheduled according to the parameter (Now the parameter is always
1, so no parallel)
3. If Job2 depends on Job1, Job1 is scheduled before Job2.
4. No parallel jobs on the same instance. If a instance is running Job1, it won't run Job2
until Job1 is finished. 

This message was sent by Atlassian JIRA

View raw message