[ https://issues.apache.org/jira/browse/TAJO-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13860014#comment-13860014
]
Hudson commented on TAJO-385:
-----------------------------
FAILURE: Integrated in Tajo-trunk-postcommit #660 (See [https://builds.apache.org/job/Tajo-trunk-postcommit/660/])
TAJO-385: Refactoring TaskScheduler to assign multiple fragments. (jihoon) (jihoonson: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=df5727c49edd2e726b99dfa49dcc6b3d5b6f252b)
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/DefaultFragmentScheduleAlgorithm.java
* tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/MergeScanner.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/AbstractTaskScheduler.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/DefaultTaskScheduler.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryUnitAttemptScheduleEvent.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/TaskSchedulerEventFactory.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/FragmentScheduleAlgorithmFactory.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/LazyTaskScheduler.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/FragmentScheduleAlgorithm.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/TaskSchedulerEvent.java
* tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterManagerService.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/FetchScheduleEvent.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/DefaultTaskSchedulerEvent.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/ScheduledFetches.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GreedyFragmentScheduleAlgorithm.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/FragmentScheduleEvent.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
* tajo-core/tajo-core-backend/src/main/resources/tajo-default.xml
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TaskSchedulerContext.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/FragmentPair.java
* CHANGES.txt
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TaskSchedulerFactory.java
> Refactoring TaskScheduler to assign multiple fragments
> ------------------------------------------------------
>
> Key: TAJO-385
> URL: https://issues.apache.org/jira/browse/TAJO-385
> Project: Tajo
> Issue Type: Improvement
> Components: query master
> Affects Versions: 0.8-incubating
> Reporter: Jihoon Son
> Assignee: Jihoon Son
> Attachments: TAJO-385.patch, TAJO-385_2.patch, TAJO-385_3.patch, TAJO-385_4.patch,
TAJO-385_5.patch
>
>
> In the current implementation, each task processes only one fragment.
> However, processing multiple fragments in a task will increase the query processing performance
according to the storage layout and the user queries.
> In this issue, TaskScheduler is refactored to enable assigning multiple fragments to
each task.
> Followings should be contained.
> * Schedule Fragments instead of QueryUnits in TaskScheduler
> ** The QueryUnit creation is postponed until TaskScheduler receives task requests from
workers.
> ** When TaskScheduler receives task requests from workers, it dynamically creates an
QueryUnit and assigns one or more fragments.
> ** The fragment scheduling should take into account the disk load balancing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
|