From commits-return-14977-apmail-beam-commits-archive=beam.apache.org@beam.incubator.apache.org Wed Nov 16 18:13:01 2016 Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A583B19F7D for ; Wed, 16 Nov 2016 18:13:01 +0000 (UTC) Received: (qmail 21619 invoked by uid 500); 16 Nov 2016 18:13:01 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 21569 invoked by uid 500); 16 Nov 2016 18:13:01 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 21560 invoked by uid 99); 16 Nov 2016 18:13:01 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Nov 2016 18:13:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 243E6C0118 for ; Wed, 16 Nov 2016 18:13:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -7.019 X-Spam-Level: X-Spam-Status: No, score=-7.019 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id A4qgq-RNbDHj for ; Wed, 16 Nov 2016 18:13:00 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 575F25F645 for ; Wed, 16 Nov 2016 18:12:59 +0000 (UTC) Received: (qmail 21339 invoked by uid 99); 16 Nov 2016 18:12:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Nov 2016 18:12:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 6C0CD2C4C75 for ; Wed, 16 Nov 2016 18:12:58 +0000 (UTC) Date: Wed, 16 Nov 2016 18:12:58 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-833) Exercise Dynamic Splitting in the DirectRunner MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671180#comment-15671180 ] ASF GitHub Bot commented on BEAM-833: ------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1254 > Exercise Dynamic Splitting in the DirectRunner > ---------------------------------------------- > > Key: BEAM-833 > URL: https://issues.apache.org/jira/browse/BEAM-833 > Project: Beam > Issue Type: Improvement > Components: runner-direct > Reporter: Thomas Groh > Assignee: Thomas Groh > > This primarily means calling the {{BoundedSource#splitAtFraction}} API during the evaluation of a BoundedReadEvaluatorFactory - this can return the work via addUnprocessedElements. > At a later point, the {{MonitorRunnable}} within {{ExecutorServiceParallelExecutor}} can examine currently evaluating work and invoke a {{TransformExecutor#split}} method, which can return to the runner some amount of input, which can be scheduled to use more of the available threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)