From user-return-2418-apmail-storm-user-archive=storm.apache.org@storm.incubator.apache.org Wed May 28 18:15:06 2014 Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2DCC3106BE for ; Wed, 28 May 2014 18:15:06 +0000 (UTC) Received: (qmail 17033 invoked by uid 500); 28 May 2014 18:15:05 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 16989 invoked by uid 500); 28 May 2014 18:15:05 -0000 Mailing-List: contact user-help@storm.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.incubator.apache.org Delivered-To: mailing list user@storm.incubator.apache.org Received: (qmail 16981 invoked by uid 99); 28 May 2014 18:15:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 May 2014 18:15:05 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of raffihsieh@gmail.com designates 74.125.82.172 as permitted sender) Received: from [74.125.82.172] (HELO mail-we0-f172.google.com) (74.125.82.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 May 2014 18:15:00 +0000 Received: by mail-we0-f172.google.com with SMTP id k48so11881980wev.31 for ; Wed, 28 May 2014 11:14:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=QwTHW4ajZhh3EeBYqHGx0eSmiADxBTWM4W8xylQLgCk=; b=YQZcZXmgJriOq0c1IczRJN1NZBejWYMkX/yJv5W4XS3sU67GcppIj/IIzLYY2DCA7B VA8O8RazCs3x0MmRydgXYFec+FGH30YtHo4nJvb19IMsvfQ6u5u2iWE5n6UI9ae5wlVY EOV3dEwYOubQokl7UIVq+QdNf1W4+gaVcXyvs8UlKqDbh15akxA6v32TyhlRtYdYH/Jz 1fZjH296LQ9ZQkvUX+Qy5eQkODhqnvbucbSmYYNYknDvnVIo2XhssuTls07hjyV86xsk uTQHPCGM0g+mTb7WSZTdWZRXk4Fz0+YTXeavdLm+0djTrN9P+C7rLpDRPWW4XYSlnRWd FcQw== MIME-Version: 1.0 X-Received: by 10.180.38.107 with SMTP id f11mr28783756wik.59.1401300835986; Wed, 28 May 2014 11:13:55 -0700 (PDT) Received: by 10.194.192.202 with HTTP; Wed, 28 May 2014 11:13:55 -0700 (PDT) Date: Wed, 28 May 2014 11:13:55 -0700 Message-ID: Subject: Trident, ZooKeeper and Kafka From: Raphael Hsieh To: user@storm.incubator.apache.org Content-Type: multipart/alternative; boundary=e89a8f6433548d73ff04fa79c641 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f6433548d73ff04fa79c641 Content-Type: text/plain; charset=UTF-8 If I don't tell trident to start consuming data from the beginning of the Kafka stream, where does it start from? If I were to do: tridentKafkaConfig.forceFromStart = true; Then it will tell the spout to start consuming from the start of the stream. If that is not set, then where does it start consuming from? and How might I go about telling it to start consuming from the very end of the stream? If a disaster were to happen and all my hosts died, when I start my cluster back up, it might start consuming from where it left off. I would rather manually process that old data, and have my storm system start processing the live data. Thanks -- Raphael Hsieh --e89a8f6433548d73ff04fa79c641 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If I don't tell trident to start consuming data from t= he beginning of the Kafka stream, where does it start from?
If I were to= do:
=C2=A0 =C2=A0tridentKafkaConfig.forceFromStart =3D true;
Then i= t will tell the spout to start consuming from the start of the stream. If t= hat is not set, then where does it start consuming from? and How might I go= about telling it to start consuming from the very end of the stream?

If a disaster were to happen and all my hosts died, when I start my clu= ster back up, it might start consuming from where it left off. I would rath= er manually process that old data, and have my storm system start processin= g the live data.

Thanks
--
Raphael Hsieh
=C2=A0
= =C2=A0
=C2=A0
--e89a8f6433548d73ff04fa79c641--