From user-return-27911-apmail-spark-user-archive=spark.apache.org@spark.apache.org Tue Mar 3 10:32:07 2015 Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8B2E110CE0 for ; Tue, 3 Mar 2015 10:32:07 +0000 (UTC) Received: (qmail 51974 invoked by uid 500); 3 Mar 2015 10:32:04 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 51902 invoked by uid 500); 3 Mar 2015 10:32:03 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 51892 invoked by uid 99); 3 Mar 2015 10:32:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2015 10:32:03 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of sowen@cloudera.com designates 74.125.82.170 as permitted sender) Received: from [74.125.82.170] (HELO mail-we0-f170.google.com) (74.125.82.170) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2015 10:31:38 +0000 Received: by wevl61 with SMTP id l61so38916501wev.2 for ; Tue, 03 Mar 2015 02:29:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=ra+3UhZpl45IZS6fVIeQOD0dYRpNsQ8sfsfhxmrDvBU=; b=ID2z1UwWixUZIorxhGKT3rgi6ElPffr7cYjqZJtweIqYIZt0vlfgJc7LLCBnsJ+ysQ QVs+47JKSTCWd8kyRMxoHzyXLXfiU44ahlgM2GGBpgxvcuVUPuAnjuiMiNaR1EXRJ+7z 0ALUiX+WO5FeMiY/jiJQK+yZBNWvxD7/aDBNUhIK0hjcKOrMiZKiGUNpX7YgGqtCuuBU TC0cWOnRTMWym5ZwxVSe5zwszNiBgavuNd3R66KMhQ3RFb31KzTXuhw95Z7ngvMGA0NE WE0hfgB57RsIMFSpi9dBOLfwuBjIZ358dXcAzwdwRIhDdbLS0FSGJfL3Z6BozKNRKTx/ UryA== X-Gm-Message-State: ALoCoQl5F7fS8zgF3YtGGvJclZbFpmVYMa2mGJfPa4oyankKDMZYNglNKDjL4ZxNSUYfYvtW3yK+ X-Received: by 10.180.91.47 with SMTP id cb15mr1333126wib.39.1425378561998; Tue, 03 Mar 2015 02:29:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.27.83.209 with HTTP; Tue, 3 Mar 2015 02:29:01 -0800 (PST) In-Reply-To: References: From: Sean Owen Date: Tue, 3 Mar 2015 10:29:01 +0000 Message-ID: Subject: Re: Is the RDD's Partitions determined before hand ? To: Jeff Zhang Cc: "user@spark.apache.org" Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org An RDD has a certain fixed number of partitions, yes. You can't change an RDD. You can repartition() or coalese() and RDD to make a new one with a different number of RDDs, possibly requiring a shuffle. On Tue, Mar 3, 2015 at 10:21 AM, Jeff Zhang wrote: > I mean is it possible to change the partition number at runtime. Thanks > > > -- > Best Regards > > Jeff Zhang --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org