From issues-return-189253-apmail-spark-issues-archive=spark.apache.org@spark.apache.org Fri Apr 13 15:45:04 2018 Return-Path: X-Original-To: apmail-spark-issues-archive@minotaur.apache.org Delivered-To: apmail-spark-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B5EBB17A28 for ; Fri, 13 Apr 2018 15:45:04 +0000 (UTC) Received: (qmail 58970 invoked by uid 500); 13 Apr 2018 15:45:04 -0000 Delivered-To: apmail-spark-issues-archive@spark.apache.org Received: (qmail 58940 invoked by uid 500); 13 Apr 2018 15:45:04 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 58931 invoked by uid 99); 13 Apr 2018 15:45:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Apr 2018 15:45:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 354AA180543 for ; Fri, 13 Apr 2018 15:45:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id HDZqVZTpd3Zi for ; Fri, 13 Apr 2018 15:45:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 9BF625F5B3 for ; Fri, 13 Apr 2018 15:45:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id AA428E050B for ; Fri, 13 Apr 2018 15:45:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 54144240C6 for ; Fri, 13 Apr 2018 15:45:00 +0000 (UTC) Date: Fri, 13 Apr 2018 15:45:00 +0000 (UTC) From: "Anirudh Ramanathan (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan reassigned SPARK-22839: ------------------------------------------ Assignee: Matt Cheah > Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction > ------------------------------------------------------------------------------------------------------- > > Key: SPARK-22839 > URL: https://issues.apache.org/jira/browse/SPARK-22839 > Project: Spark > Issue Type: Improvement > Components: Kubernetes > Affects Versions: 2.3.0 > Reporter: Yinan Li > Assignee: Matt Cheah > Priority: Major > Fix For: 3.0.0 > > > As discussed in https://github.com/apache/spark/pull/19954, the current code for configuring the driver pod vs the code for configuring the executor pods are not using the same abstraction. Besides that, the current code leaves a lot to be desired in terms of the level and cleaness of abstraction. For example, the current code is passing around many pieces of information around different class hierarchies, which makes code review and maintenance challenging. We need some thorough refactoring of the current code to achieve better, cleaner, and consistent abstraction. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org