From commits-return-4999-apmail-beam-commits-archive=beam.apache.org@beam.incubator.apache.org Wed Jun 15 21:21:12 2016 Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1C4841990E for ; Wed, 15 Jun 2016 21:21:12 +0000 (UTC) Received: (qmail 18600 invoked by uid 500); 15 Jun 2016 21:21:12 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 18545 invoked by uid 500); 15 Jun 2016 21:21:12 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 18536 invoked by uid 99); 15 Jun 2016 21:21:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 21:21:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 9B71F180361 for ; Wed, 15 Jun 2016 21:21:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id xEfNeCl6pCUJ for ; Wed, 15 Jun 2016 21:21:11 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id 84FE95F489 for ; Wed, 15 Jun 2016 21:21:10 +0000 (UTC) Received: (qmail 18096 invoked by uid 99); 15 Jun 2016 21:21:09 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 21:21:09 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id A5C352C1F64 for ; Wed, 15 Jun 2016 21:21:09 +0000 (UTC) Date: Wed, 15 Jun 2016 21:21:09 +0000 (UTC) From: "Daniel Halperin (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Closed] (BEAM-347) Progress updates inaccurate for non-uniform keys in Bigtable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin closed BEAM-347. -------------------------------- > Progress updates inaccurate for non-uniform keys in Bigtable > ------------------------------------------------------------ > > Key: BEAM-347 > URL: https://issues.apache.org/jira/browse/BEAM-347 > Project: Beam > Issue Type: Improvement > Components: sdk-java-gcp > Reporter: Ian Zhou > Assignee: Daniel Halperin > Priority: Minor > > When reading from a Bigtable source with clustered keys, fraction consumed progress updates are inaccurate. For example, for a range spanning ['a', 'z'], a cluster of keys starting with the letter 'm' (e.g. 'me100,' ..., 'me999') will be recorded as ~50% complete upon reading the first key, and will remain at this percentage until the final key has been read. Instead, the start of the range should be changed to the first key read (e.g. new range ['me100', 'z']). The end of the range can be changed over time through dynamic work rebalancing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)