helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HELIX-601) Allow work flow to schedule dependency jobs in parallel
Date Mon, 22 Jun 2015 04:26:00 GMT

    [ https://issues.apache.org/jira/browse/HELIX-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595344#comment-14595344
] 

ASF GitHub Bot commented on HELIX-601:
--------------------------------------

Github user kanakb commented on a diff in the pull request:

    https://github.com/apache/helix/pull/30#discussion_r32904755
  
    --- Diff: helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerParallel.java
---
    @@ -0,0 +1,195 @@
    +package org.apache.helix.integration.task;
    +
    +import java.util.ArrayList;
    +import java.util.Arrays;
    +import java.util.Collections;
    +import java.util.HashMap;
    +import java.util.List;
    +import java.util.Map;
    +
    +import org.apache.helix.AccessOption;
    +import org.apache.helix.HelixDataAccessor;
    +import org.apache.helix.HelixManager;
    +import org.apache.helix.HelixManagerFactory;
    +import org.apache.helix.InstanceType;
    +import org.apache.helix.PropertyKey;
    +import org.apache.helix.TestHelper;
    +import org.apache.helix.integration.ZkIntegrationTestBase;
    +import org.apache.helix.integration.manager.ClusterControllerManager;
    +import org.apache.helix.integration.manager.MockParticipantManager;
    +import org.apache.helix.participant.StateMachineEngine;
    +import org.apache.helix.task.JobConfig;
    +import org.apache.helix.task.JobContext;
    +import org.apache.helix.task.JobQueue;
    +import org.apache.helix.task.Task;
    +import org.apache.helix.task.TaskCallbackContext;
    +import org.apache.helix.task.TaskConstants;
    +import org.apache.helix.task.TaskDriver;
    +import org.apache.helix.task.TaskFactory;
    +import org.apache.helix.task.TaskPartitionState;
    +import org.apache.helix.task.TaskResult;
    +import org.apache.helix.task.TaskState;
    +import org.apache.helix.task.TaskStateModelFactory;
    +import org.apache.helix.task.TaskUtil;
    +import org.apache.helix.task.Workflow;
    +import org.apache.helix.tools.ClusterSetup;
    +import org.apache.helix.tools.ClusterStateVerifier;
    +import org.testng.Assert;
    +import org.testng.annotations.AfterClass;
    +import org.testng.annotations.BeforeClass;
    +import org.testng.annotations.Test;
    +
    +import com.google.common.base.Joiner;
    +import com.google.common.collect.ImmutableMap;
    +
    +public class TestTaskRebalancerParallel extends ZkIntegrationTestBase {
    --- End diff --
    
    Apache license header is missing


> Allow work flow to schedule dependency jobs in parallel
> -------------------------------------------------------
>
>                 Key: HELIX-601
>                 URL: https://issues.apache.org/jira/browse/HELIX-601
>             Project: Apache Helix
>          Issue Type: New Feature
>            Reporter: Congrui Ji
>
> Currently, Helix won't schedule dependency jobs in a same work flow. For example, if
Job2 depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.
> However, if some participant is very slow, then all dependency jobs is waiting for that
single participant.
> Helix should be able to schedule multiple jobs according to a parameter.
> A.C.
> 1. Introduce parallel count parameter in work flow and job queue.
> 2. Dependency jobs can be scheduled according to the parameter (Now the parameter is
always 1, so no parallel)
> 3. If Job2 depends on Job1, Job1 is scheduled before Job2.
> 4. No parallel jobs on the same instance. If a instance is running Job1, it won't run
Job2 until Job1 is finished. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message