From commits-return-26787-apmail-beam-commits-archive=beam.apache.org@beam.apache.org Thu Mar 23 03:09:47 2017 Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9E609197B4 for ; Thu, 23 Mar 2017 03:09:47 +0000 (UTC) Received: (qmail 82080 invoked by uid 500); 23 Mar 2017 03:09:47 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 82018 invoked by uid 500); 23 Mar 2017 03:09:47 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 82009 invoked by uid 99); 23 Mar 2017 03:09:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Mar 2017 03:09:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 26E50C0C6D for ; Thu, 23 Mar 2017 03:09:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.349 X-Spam-Level: X-Spam-Status: No, score=-99.349 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id C0nf7Q1BrC5r for ; Thu, 23 Mar 2017 03:09:46 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 2A6CA5F249 for ; Thu, 23 Mar 2017 03:09:46 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 4CA00E0A2B for ; Thu, 23 Mar 2017 03:09:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 9E11D254EB for ; Thu, 23 Mar 2017 03:09:41 +0000 (UTC) Date: Thu, 23 Mar 2017 03:09:41 +0000 (UTC) From: "Eugene Kirpichov (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (BEAM-1784) DataflowPipelineJob.cancel() should be idempotent MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Eugene Kirpichov created BEAM-1784: -------------------------------------- Summary: DataflowPipelineJob.cancel() should be idempotent Key: BEAM-1784 URL: https://issues.apache.org/jira/browse/BEAM-1784 Project: Beam Issue Type: Bug Components: runner-dataflow Reporter: Eugene Kirpichov Assignee: Eugene Kirpichov Right now DataflowPipelineJob.cancel() issues an RPC to the Dataflow service and, if it fails, checks whether the job is in a terminal state - in that case the failure is ignored. Ideally, the Dataflow-side cancel call itself would be idempotent, and would succeed even if the job is already terminated. However, right now this is not the case. Additionally, Dataflow service can, for a short time, return RUNNING state even for a job that has already been cancelled (i.e. it will reject cancel() calls, but will report state as RUNNING). Because of this, if you call DataflowPipelineJob.cancel() multiple times, it might fail (throw an exception). Calling cancel() multiple times should, logically, be ok. So, to partially work around these issues of the Dataflow service, we can make DataflowPipelineJob.cancel() be idempotent by itself, i.e. absorb duplicate cancel() requests. This, of course, doesn't address the case when the job terminates externally almost concurrently to calling DataflowPipelineJob.cancel() - but it's an improvement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)