From user-return-44474-apmail-spark-user-archive=spark.apache.org@spark.apache.org Sat Oct 17 19:58:36 2015 Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EDB8E18459 for ; Sat, 17 Oct 2015 19:58:35 +0000 (UTC) Received: (qmail 18955 invoked by uid 500); 17 Oct 2015 19:58:30 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 18845 invoked by uid 500); 17 Oct 2015 19:58:30 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 18834 invoked by uid 99); 17 Oct 2015 19:58:30 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 Oct 2015 19:58:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id BA23D1A2728 for ; Sat, 17 Oct 2015 19:58:29 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.511 X-Spam-Level: *** X-Spam-Status: No, score=3.511 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_WEB=0.614, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=adobe.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id tHq8Jf-YFsSJ for ; Sat, 17 Oct 2015 19:58:28 +0000 (UTC) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0066.outbound.protection.outlook.com [207.46.100.66]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 35A0E439C9 for ; Sat, 17 Oct 2015 19:58:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=adobe.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=wmkPY/tSMmkfxGbCiw1m7+XJFtrzG5zmuaFY80rfcHA=; b=Jn9RtrYoGPlWUDYLeQJAYxPrD71CCBrnIAYlunCkrqnj+g6WLoG4qtOGl1W71WTiJwNvxgC4NquQ7Oss3EyOPERCjHZYf1ndEEh9Y6s/VXExLk3QuT9Da8T4MbvgMPYPGXkAMZkzqxBD2LHwMmI/yOUN9cT5x4TN7jlNgebUOdA= Received: from BY2PR02MB1702.namprd02.prod.outlook.com (10.163.27.28) by BY2PR02MB1704.namprd02.prod.outlook.com (10.163.27.30) with Microsoft SMTP Server (TLS) id 15.1.300.14; Sat, 17 Oct 2015 19:58:20 +0000 Received: from BY2PR02MB1702.namprd02.prod.outlook.com ([10.163.27.28]) by BY2PR02MB1702.namprd02.prod.outlook.com ([10.163.27.28]) with mapi id 15.01.0300.010; Sat, 17 Oct 2015 19:58:19 +0000 From: Adrian Tanase To: "user@spark.apache.org" Subject: Spark Streaming scheduler delay VS driver.cores Thread-Topic: Spark Streaming scheduler delay VS driver.cores Thread-Index: AQHRCRYqmR61ru+mnEyqf8J9GtQ9Gw== Date: Sat, 17 Oct 2015 19:58:18 +0000 Message-ID: <9E07A36E-30A3-4483-B44D-83EF7810BF2C@adobe.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/15.9.0.150408 authentication-results: spf=none (sender IP is ) smtp.mailfrom=atanase@adobe.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [78.96.220.8] x-microsoft-exchange-diagnostics: 1;BY2PR02MB1704;5:vjLp/8ES12S5UuytYqCvFSiRWS4vZdyF/QtUskTLSz7rZ9tq811BuM3pfTPuxEPW6fplyyV+0Cs/Fjk3EGoW7Prs3WZJtXyOXXYa8Zkq6k6zUWf5iCpRVwDIJzAQw5KBzlYoXvmIfBcMbcPXRZFqUA==;24:59NySpdmIQCIth2MFyjHttaKflIPun+L7P6wDCtVqkukN0zrpTfiCUqQrvZEx6aSbULiKobYbugMv2azMG1si/PKWLnHup2OsZBJzA/ghpI=;20:BZTVk2ov2D9j2lNJ0PS0RiYeh+LuVkgls+f/WvOdsUcF2CPZHA7mENnZnTwPa8QdWJ7UH4iiA2dZmw2adq7BvQ== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR02MB1704; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(61425024)(601004)(2401047)(520078)(5005006)(8121501046)(3002001)(61426024)(61427024);SRVR:BY2PR02MB1704;BCL:0;PCL:0;RULEID:;SRVR:BY2PR02MB1704; x-forefront-prvs: 07326CFBC4 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(189002)(51874003)(199003)(122556002)(189998001)(40100003)(82746002)(5002640100001)(106116001)(2900100001)(83506001)(77096005)(87936001)(102836002)(99286002)(2351001)(229853001)(105586002)(83716003)(106356001)(5004730100002)(5007970100001)(16236675004)(64706001)(66066001)(5008740100001)(5001960100002)(110136002)(10090500001)(107886002)(33656002)(2501003)(101416001)(50986999)(54356999)(4001350100001)(36756003)(5001920100001)(92566002)(97736004)(10400500002)(86362001)(81156007)(450100001)(46102003)(104396002);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR02MB1704;H:BY2PR02MB1702.namprd02.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; received-spf: None (protection.outlook.com: adobe.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_9E07A36E30A34483B44D83EF7810BF2Cadobecom_" MIME-Version: 1.0 X-OriginatorOrg: adobe.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Oct 2015 19:58:18.6624 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: fa7b1b5a-7b34-4387-94ae-d2c178decee1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR02MB1704 --_000_9E07A36E30A34483B44D83EF7810BF2Cadobecom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGksDQoNCknigJl2ZSByZWNlbnRseSBidW1wZWQgdXAgdGhlIHJlc291cmNlcyBmb3IgYSBzcGFy ayBzdHJlYW1pbmcgam9iIOKAkyBhbmQgdGhlIHBlcmZvcm1hbmNlIHN0YXJ0ZWQgdG8gZGVncmFk ZSBvdmVyIHRpbWUuDQppdCB3YXMgcnVubmluZyBmaW5lIG9uIDcgbm9kZXMgd2l0aCAxNCBleGVj dXRvciBjb3JlcyBlYWNoICh2aWEgWWFybikgdW50aWwgSSBidW1wZWQgZXhlY3V0b3IuY29yZXMg dG8gMjIgY29yZXMvbm9kZSAob3V0IG9mIDMyIG9uIEFXUyBjMy54bGFyZ2UsIDI0IGZvciB5YXJu KQ0KDQpUaGUgZHJpdmVyIGhhcyAyIGNvcmVzIGFuZCAyIEdCIHJhbSAodXNhZ2UgaXMgYXQgemVy bykuDQoNCkZvciByZWFsbHkgbG93IGRhdGEgdm9sdW1lIGl0IGdvZXMgZnJvbSAxLTIgc2Vjb25k cyBwZXIgYmF0Y2ggdG8gNC01IHMvYmF0Y2ggYWZ0ZXIgYWJvdXQgNiBob3VycywgZG9pbmcgYWxt b3N0IG5vdGhpbmcuIEnigJl2ZSBub3RpY2VkIHRoYXQgdGhlIHNjaGVkdWxlciBkZWxheSBpcyAz LTRzLCBldmVuIDUtNiBzZWNvbmRzIGZvciBzb21lIHRhc2tzLiBTaG91bGQgYmUgaW4gdGhlIGxv dyB0ZW5zIG9mIG1pbGxpc2Vjb25kcy4gV2hhdOKAmXMgd2VpcmRlciBpcyB0aGF0IHVuZGVyIG1v ZGVyYXRlIGxvYWQgKHRob3VzYW5kcyBvZiBldmVudHMgcGVyIHNlY29uZCkgLSB0aGUgZGVsYXkg aXMgbm90IGFzIG9idmlvdXMgYW55bW9yZS4NCg0KQWZ0ZXIgdGhpcyBJIHJlZHVjZWQgdGhlIGV4 ZWN1dG9yLmNvcmVzIHRvIDIwIGFuZCBidW1wZWQgZHJpdmVyLmNvcmVzIHRvIDQgYW5kIGl0IHNl ZW1zIHRvIGJlIG9rIG5vdy4NCkhvd2V2ZXIsIHRoaXMgaXMgdG90YWxseSBlbXBpcmljYWwsIEkg aGF2ZSBub3QgZm91bmQgYW55IGRvY3VtZW50YXRpb24sIGNvZGUgc2FtcGxlcyBvciBlbWFpbCBk aXNjdXNzaW9uIG9uIGhvdyB0byBwcm9wZXJseSBzZXQgZHJpdmVyLmNvcmVzLg0KDQpEb2VzIGFu eW9uZSBrbm93Og0KDQogICogICBJZiBJIGFzc2lnbiBtb3JlIGNvcmVzIHRvIHRoZSBkcml2ZXIv YXBwbGljYXRpb24gbWFuYWdlciwgd2lsbCBpdCB1c2UgdGhlbT8NCiAgICAgKiAgIEkgd2FzIGxv b2tpbmcgYXQgdGhlIHByb2Nlc3MgbGlzdCB3aXRoIGh0b3AgYW5kIG9ubHkgb25lIG9mIHRoZSBq dm3igJlzIG9uIHRoZSBkcml2ZXIgd2FzIHJlYWxseSB0YWtpbmcgdXAgQ1BVIHRpbWUNCiAgKiAg IFdoYXQgaXMgYSBkZWNlbnQgcGFyYWxsZWxpc20gZmFjdG9yIGZvciBhIHN0cmVhbWluZyBhcHAg d2l0aCAxMC0yMCBzZWNzIGJhdGNoIHRpbWU/IEkgZm91bmQgaXQgb2RkIHRoYXQgYXQgIDcgeCAy MiA9IDE1NCB0aGUgZHJpdmVyIGlzIGJlY29taW5nIGEgYm90dGxlbmVjaw0KICAgICAqICAgSeKA mXZlIHNlZW4gcGVvcGxlIHJlY29tbWVuZCAzLTQgdGFrcy9jb3JlIG9yIH4xMDAwIHBhcmFsbGVs aXNtIGZvciBjbHVzdGVycyBpbiB0aGUgdGVucyBvZiBub2Rlcw0KDQpUaGFua3MgaW4gYWR2YW5j ZSwNCi1hZHJpYW4NCg== --_000_9E07A36E30A34483B44D83EF7810BF2Cadobecom_ Content-Type: text/html; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1saW5lLWJy ZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAx NHB4OyBmb250LWZhbWlseTogQ2FsaWJyaSwgc2Fucy1zZXJpZjsiPg0KPGRpdj5IaSw8L2Rpdj4N CjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2PknigJl2ZSByZWNlbnRseSBidW1wZWQgdXAgdGhlIHJl c291cmNlcyBmb3IgYSBzcGFyayBzdHJlYW1pbmcgam9iIOKAkyBhbmQgdGhlIHBlcmZvcm1hbmNl IHN0YXJ0ZWQgdG8gZGVncmFkZSBvdmVyIHRpbWUuPC9kaXY+DQo8ZGl2Pml0IHdhcyBydW5uaW5n IGZpbmUgb24gNyBub2RlcyB3aXRoIDE0IGV4ZWN1dG9yIGNvcmVzIGVhY2ggKHZpYSBZYXJuKSB1 bnRpbCBJIGJ1bXBlZCBleGVjdXRvci5jb3JlcyB0byAyMiBjb3Jlcy9ub2RlIChvdXQgb2YgMzIg b24gQVdTIGMzLnhsYXJnZSwgMjQgZm9yIHlhcm4pPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0K PGRpdj5UaGUgZHJpdmVyIGhhcyAyIGNvcmVzIGFuZCAyIEdCIHJhbSAodXNhZ2UgaXMgYXQgemVy bykuPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5Gb3IgcmVhbGx5IGxvdyBkYXRhIHZv bHVtZSBpdCBnb2VzIGZyb20gMS0yIHNlY29uZHMgcGVyIGJhdGNoIHRvIDQtNSBzL2JhdGNoIGFm dGVyIGFib3V0IDYgaG91cnMsIGRvaW5nIGFsbW9zdCBub3RoaW5nLiBJ4oCZdmUgbm90aWNlZCB0 aGF0IHRoZSBzY2hlZHVsZXIgZGVsYXkgaXMgMy00cywgZXZlbiA1LTYgc2Vjb25kcyBmb3Igc29t ZSB0YXNrcy4gU2hvdWxkIGJlIGluIHRoZSBsb3cgdGVucyBvZiBtaWxsaXNlY29uZHMuIFdoYXTi gJlzIHdlaXJkZXINCiBpcyB0aGF0IHVuZGVyIG1vZGVyYXRlIGxvYWQgKHRob3VzYW5kcyBvZiBl dmVudHMgcGVyIHNlY29uZCkgLSB0aGUgZGVsYXkgaXMgbm90IGFzIG9idmlvdXMgYW55bW9yZS48 L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2PkFmdGVyIHRoaXMgSSByZWR1Y2VkIHRoZSBl eGVjdXRvci5jb3JlcyB0byAyMCBhbmQgYnVtcGVkIGRyaXZlci5jb3JlcyB0byA0IGFuZCBpdCBz ZWVtcyB0byBiZSBvayBub3cuJm5ic3A7PC9kaXY+DQo8ZGl2Pkhvd2V2ZXIsIHRoaXMgaXMgdG90 YWxseSBlbXBpcmljYWwsIEkgaGF2ZSBub3QgZm91bmQgYW55IGRvY3VtZW50YXRpb24sIGNvZGUg c2FtcGxlcyBvciBlbWFpbCBkaXNjdXNzaW9uIG9uIGhvdyB0byBwcm9wZXJseSBzZXQgZHJpdmVy LmNvcmVzLjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+RG9lcyBhbnlvbmUga25vdzo8 L2Rpdj4NCjx1bD4NCjxsaT5JZiBJIGFzc2lnbiBtb3JlIGNvcmVzIHRvIHRoZSBkcml2ZXIvYXBw bGljYXRpb24gbWFuYWdlciwgd2lsbCBpdCB1c2UgdGhlbT8NCjx1bD4NCjxsaT5JIHdhcyBsb29r aW5nIGF0IHRoZSBwcm9jZXNzIGxpc3Qgd2l0aCBodG9wIGFuZCBvbmx5IG9uZSBvZiB0aGUganZt 4oCZcyBvbiB0aGUgZHJpdmVyIHdhcyByZWFsbHkgdGFraW5nIHVwIENQVSB0aW1lPC9saT48L3Vs Pg0KPC9saT48bGk+V2hhdCBpcyBhIGRlY2VudCBwYXJhbGxlbGlzbSBmYWN0b3IgZm9yIGEgc3Ry ZWFtaW5nIGFwcCB3aXRoIDEwLTIwIHNlY3MgYmF0Y2ggdGltZT8gSSBmb3VuZCBpdCBvZGQgdGhh dCBhdCAmbmJzcDs3IHggMjIgPSAxNTQgdGhlIGRyaXZlciBpcyBiZWNvbWluZyBhIGJvdHRsZW5l Y2sNCjx1bD4NCjxsaT5J4oCZdmUgc2VlbiBwZW9wbGUgcmVjb21tZW5kIDMtNCB0YWtzL2NvcmUg b3IgfjEwMDAgcGFyYWxsZWxpc20gZm9yIGNsdXN0ZXJzIGluIHRoZSB0ZW5zIG9mIG5vZGVzPC9s aT48L3VsPg0KPC9saT48L3VsPg0KPGRpdj5UaGFua3MgaW4gYWR2YW5jZSw8L2Rpdj4NCjxkaXY+ LWFkcmlhbjwvZGl2Pg0KPGRpdj4NCjxkaXYgaWQ9Ik1BQ19PVVRMT09LX1NJR05BVFVSRSI+PC9k aXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg== --_000_9E07A36E30A34483B44D83EF7810BF2Cadobecom_--