From dev-return-5658-apmail-samza-dev-archive=samza.apache.org@samza.apache.org Tue Mar 22 20:33:08 2016 Return-Path: X-Original-To: apmail-samza-dev-archive@minotaur.apache.org Delivered-To: apmail-samza-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C469419DB7 for ; Tue, 22 Mar 2016 20:33:08 +0000 (UTC) Received: (qmail 77779 invoked by uid 500); 22 Mar 2016 20:33:08 -0000 Delivered-To: apmail-samza-dev-archive@samza.apache.org Received: (qmail 77720 invoked by uid 500); 22 Mar 2016 20:33:08 -0000 Mailing-List: contact dev-help@samza.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@samza.apache.org Delivered-To: mailing list dev@samza.apache.org Received: (qmail 77708 invoked by uid 99); 22 Mar 2016 20:33:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Mar 2016 20:33:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id CE37CC71C7 for ; Tue, 22 Mar 2016 20:33:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.448 X-Spam-Level: * X-Spam-Status: No, score=1.448 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id qJxHf4XcrXSd for ; Tue, 22 Mar 2016 20:33:07 +0000 (UTC) Received: from mail-io0-f177.google.com (mail-io0-f177.google.com [209.85.223.177]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id D285F5FB27 for ; Tue, 22 Mar 2016 20:33:06 +0000 (UTC) Received: by mail-io0-f177.google.com with SMTP id c63so76400561iof.0 for ; Tue, 22 Mar 2016 13:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to; bh=ZI97HqPush2gemEKrn6oyp4kdocS+5u4HsK1YggDl24=; b=DkYN/HusC+0tW2mdvz4WAP02HHzNltDNModQJ1ErbHRj1jW5h+frrAiXqGQ8VkOMyr 43n2guQQQfGm/1A6qoOu2xJiLGfo3OX40MtjrC0j+wmktL6BVZTfSuBkBIFCzW9WUnLG eTUQoBo8i4brLHJsB/t37i1xrFeUxIkuHEIWkVYQv57YiViq4MCJsD4hFonD4l0O0hZI xIwfIOjuzRMYlaWAIPQKEoNU7q48VhiGJbuH2zCHBd6zSNHk5VwLYTb9nX0d75v/OqvJ EwFB18uiXgu3hjnbEiWYIHILuAmhdwMIbkah7CGFKILbDU6Srkckam8fnmExi15vZWH2 pVVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to; bh=ZI97HqPush2gemEKrn6oyp4kdocS+5u4HsK1YggDl24=; b=Knm86VoYKh0tH4PZD+9Ofbgq9t4vFJ5bitFm4SLAk/MC5MD5KG3mFbGNT7wtuJIuNI /ih7QT4T5QRhurrf/irbUVkUzJqKYbF6u1MUMtWo3wbPMl9Cr1a6JYrV9p/VFlfdWb75 ILxVSJnsMLC9KHBEEbxxGNBol/MMpRx7+lsixJSyHvUaLd5genCMElyCFaNIGaNFqVfr mp5Yvf5HoBoCnPMHp8AYw+8p46cQLoWcbICq22RN36cWamxn+68Sfy8UsTG7K33NYJNu 1C9wKVz2ktoc4tmRTWErwZJ0In2t6KwGISYgNmimhQRzrvwIp9JVAUn9lw+pU/gzm+zY BQRw== X-Gm-Message-State: AD7BkJL+A+NE3kMu7Y8AT478nPC4A62jwWam+sbXWU7mq5NByL7L3CZVi/k4kA2AGBp17xZjFqmWzl6bmJUNJw== MIME-Version: 1.0 X-Received: by 10.50.155.1 with SMTP id vs1mr21080663igb.14.1458678786155; Tue, 22 Mar 2016 13:33:06 -0700 (PDT) Received: by 10.64.159.195 with HTTP; Tue, 22 Mar 2016 13:33:06 -0700 (PDT) In-Reply-To: References: Date: Tue, 22 Mar 2016 13:33:06 -0700 Message-ID: Subject: Re: Picking up checkpoint when upgrade to 0.10.0 from 0.9? From: Yi Pan To: dev@samza.apache.org Content-Type: multipart/alternative; boundary=001a11c1e7eae3ef5a052ea91e25 --001a11c1e7eae3ef5a052ea91e25 Content-Type: text/plain; charset=UTF-8 Hi, Yuanchi, Did you check your configuration of task.checkpoint.system? What are the config value you used in 0.9 and what's the current configuration in 0.10? If you can share your config before and after the upgrade, + the container log from 0.10, we can be more helpful. Thanks! -Yi On Tue, Mar 22, 2016 at 1:19 PM, Yuanchi Ning wrote: > Hi All, > > When we test upgrading our existing Samza job from 0.9 to 0.10, we saw our > Kafka Lag metric (KafkaSystemConsumerMetrics > "messages-behind-high-watermark > ") kept zero. > Since we stopped the old job for a while and then restart the job with 0.10 > using the same name, the lag should at least spike at the beginning. In the > application master we did see it's picking up the same checkpoint topic > though. > Any ideas? thanks! > > Yuanchi > > > -- > Yuanchi Ning > --001a11c1e7eae3ef5a052ea91e25--