From user-return-73615-apmail-spark-user-archive=spark.apache.org@spark.apache.org Tue Feb 13 08:16:07 2018 Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B6CBF18FE4 for ; Tue, 13 Feb 2018 08:16:07 +0000 (UTC) Received: (qmail 20869 invoked by uid 500); 13 Feb 2018 08:16:02 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 20398 invoked by uid 500); 13 Feb 2018 08:16:02 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 20387 invoked by uid 99); 13 Feb 2018 08:16:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Feb 2018 08:16:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C760A18015D for ; Tue, 13 Feb 2018 08:16:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.889 X-Spam-Level: * X-Spam-Status: No, score=1.889 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=outlook.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id PsHv8iKZlLtn for ; Tue, 13 Feb 2018 08:16:00 +0000 (UTC) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-oln040092001096.outbound.protection.outlook.com [40.92.1.96]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 46E1D5F1ED for ; Tue, 13 Feb 2018 08:15:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=oG7WHjSR4so+G9z6D12u6eP0iYyjWHmX8UwsmTnQy9U=; b=aEQ0DNtCAX6eNSlDA7zHHmk+A0+U+tAQ2JtNsFGgMibRu46EIz7xYVn70Zj/R0+PfaAuV70WCzNBnM8XeBRlv9orni6xqnp7fRjITmUxwBI0LLDIXZy5oBdAGb+HfFRgg/dTPISZkqM7U9LoylNC++RhmVL83/oLK+ZelJK8B54o5Uhw95Wk4VRxziBt/o7cWzcwxsRxGvbwJTvbt+8a9abhFL4wEUdcH246LUvxj/LfcYNUowWgNZKMlO8GCbeaPSromSJskRS34AenIUFtR1c72jDryW2nsJXWJgwhOZG3VAHNqo/UkZrI+RwQUh4TmNCeYJo8HpkInAh1yjvW4w== Received: from BN3NAM01FT058.eop-nam01.prod.protection.outlook.com (10.152.66.57) by BN3NAM01HT159.eop-nam01.prod.protection.outlook.com (10.152.67.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.485.14; Tue, 13 Feb 2018 08:15:52 +0000 Received: from DM5PR03MB2540.namprd03.prod.outlook.com (10.152.66.56) by BN3NAM01FT058.mail.protection.outlook.com (10.152.66.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.444.13 via Frontend Transport; Tue, 13 Feb 2018 08:15:52 +0000 Received: from DM5PR03MB2540.namprd03.prod.outlook.com ([10.168.233.150]) by DM5PR03MB2540.namprd03.prod.outlook.com ([10.168.233.150]) with mapi id 15.20.0506.016; Tue, 13 Feb 2018 08:15:52 +0000 From: Naved Alam To: "user@spark.apache.org" Subject: [Spark-Listener] [How-to] Listen only to specific events Thread-Topic: [Spark-Listener] [How-to] Listen only to specific events Thread-Index: AQHTpKAYw7F0scosCkGGF1MuZebkbg== Date: Tue, 13 Feb 2018 08:15:51 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:38B13EE40B0A99AB789E3EC39A009E47511B2D74F2CCE0392EEB1AD93F4AD302;UpperCasedChecksum:075177FE2022C664F2EC7C07248D3A7DA2A6EBCE3787924CFC4DA9E88C9C4DB9;SizeAsReceived:6744;Count:42 x-tmn: [sKR0rze4uJkikf63oJHcHrEzPQTxFbNu] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;BN3NAM01HT159;6:WKLIe1QKTKqhUkpRs+Qv9p3vFrxUrYaCKsl+fl00+61pZY2WZNvDbD4x9VJsr8cFhFl8H2936JZ+dEu/Sp9X6r6HtKJBV+BEOtZ3AkVtvsJBjbDJ4e46tgY8Fn/SkJtF36mbWTHBOB3Trj3YJInCMjrePhnvyXom2QYjzMlw72O913TeacGYmZnf2qtD7Oo1Y7FKXBw+twVirmfMYqPa57fFT5YyLUHuv1n2wahserrbFFM+c7exFKr11Jwt3jTyFecnqpCAz4gm1kIe8IjxM4vi1q3J8OBjTj/tG9at6qotiMsoF6UOzLNhXetfj/zSnP5Elo0IPDbUGaLvvjYF08CmDLdTkTPB+XIe06xi6A8=;5:1wE6+3jxAa5jmZP4QeBCYm36NuBrqo0serBJBSAqKpL9VZy/UDPlg8JXMrEviNFn+TysHnr8/lLXNK7MtX/oGxEPh5XeN8FaCTXJF+sU2eTDkxaNE5zcKaMIl8YOnaXc/HK5HI2UvqMkoEMg8epZesp3z5N/st217YDhf/occnk=;24:FM1Tw590dZf3MeyKFLvVO4NSiJLOzH1HNs4m7SH5Dgim0x61jrTM79yZaAc3yvAYlKBvCOxpWZVKCkGB3N6Vt8nZbO6tu7dEqWKjInxsKn0=;7:d5QfDKRUMbZIhctCb7YFSHpiOP7zvLq/F/ygRFF6iBdPFLGWLKWsivuOAdBl1Q18HG1YwEmgoNNbsFA0Wa2mBRvWRvA7TUvbsDoBaquHupQLeOYXmerriXtFg7pGw4glV20d0F9ztvQxZ47Xz3ZPlnzZTFBw9m98olVDsDKgQjTvpvqjYIMi2pLKm3To6xc4c+9DJ3diG6XSgnMHZBRcFQ7iLWdfPIh5c4fr6rrOeGujWTVByc5eGO3IHlUzN5eP x-incomingheadercount: 42 x-eopattributedmessage: 0 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125374)(1701031045);SRVR:BN3NAM01HT159; x-ms-traffictypediagnostic: BN3NAM01HT159: x-ms-office365-filtering-correlation-id: 1016f943-66af-4600-a353-08d572b9ff50 x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(444000031);SRVR:BN3NAM01HT159;BCL:0;PCL:0;RULEID:;SRVR:BN3NAM01HT159; x-forefront-prvs: 0582641F53 x-forefront-antispam-report: SFV:NSPM;SFS:(7070007)(98901004);DIR:OUT;SFP:1901;SCL:1;SRVR:BN3NAM01HT159;H:DM5PR03MB2540.namprd03.prod.outlook.com;FPR:;SPF:None;LANG:; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_DM5PR03MB25408143CAC0D22B1D8552948BF60DM5PR03MB2540namp_" MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1016f943-66af-4600-a353-08d572b9ff50 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Feb 2018 08:15:51.9653 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3NAM01HT159 --_000_DM5PR03MB25408143CAC0D22B1D8552948BF60DM5PR03MB2540namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have a spark application which creates multiple sessions. Each of these s= essions can run jobs in parallel. I want to log some details about the exec= ution of these jobs, but want to the tag them with the session they were ca= lled from. I tried creating a listener from within each session (instantiations of the= same class, but with session specific information), but whenever an event = is emmited, it is intercepted by all the listeners (which makes sense since= the listeners are attached to the shared context). Is there a way I can en= sure that the events are only intercepted by the listener from the same ses= sion. If not, then is it possible to add any identifiers to the events whic= h can be used to figure the session which triggered a job? Thanks Naved --_000_DM5PR03MB25408143CAC0D22B1D8552948BF60DM5PR03MB2540namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I have a spark application which creates multiple sessions. Each of these s= essions can run jobs in parallel. I want to log some details about the= execution of these jobs, but want to the tag them with the session th= ey were called from.

I tried creating a listener from within each session (instantiations of the= same class, but with session specific information), but whenever an&n= bsp;event is emmited, it is intercepted by all the listeners (which ma= kes sense since the listeners are attached to the shared context). Is there a way I can ensure that the events are only inte= rcepted by the listener from the same session. If not, then is it possible = to add any identifiers to the events which can be used to figure the s= ession which triggered a job?

Thanks
Naved
--_000_DM5PR03MB25408143CAC0D22B1D8552948BF60DM5PR03MB2540namp_--