From yarn-dev-return-29381-apmail-hadoop-yarn-dev-archive=hadoop.apache.org@hadoop.apache.org Tue Jan 23 23:43:03 2018 Return-Path: X-Original-To: apmail-hadoop-yarn-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 526A617DCC for ; Tue, 23 Jan 2018 23:43:03 +0000 (UTC) Received: (qmail 28499 invoked by uid 500); 23 Jan 2018 23:43:02 -0000 Delivered-To: apmail-hadoop-yarn-dev-archive@hadoop.apache.org Received: (qmail 28389 invoked by uid 500); 23 Jan 2018 23:43:02 -0000 Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-dev@hadoop.apache.org Received: (qmail 28378 invoked by uid 99); 23 Jan 2018 23:43:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Jan 2018 23:43:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 2EB4D1A0B81 for ; Tue, 23 Jan 2018 23:43:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.711 X-Spam-Level: X-Spam-Status: No, score=-100.711 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Kimhph02tpjg for ; Tue, 23 Jan 2018 23:43:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 4B9E25F640 for ; Tue, 23 Jan 2018 23:43:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B66C2E0F85 for ; Tue, 23 Jan 2018 23:43:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 256CE25C74 for ; Tue, 23 Jan 2018 23:43:00 +0000 (UTC) Date: Tue, 23 Jan 2018 23:43:00 +0000 (UTC) From: "Gour Saha (JIRA)" To: yarn-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (YARN-7799) YARN Service dependency follow up work MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Gour Saha created YARN-7799: ------------------------------- Summary: YARN Service dependency follow up work Key: YARN-7799 URL: https://issues.apache.org/jira/browse/YARN-7799 Project: Hadoop YARN Issue Type: Bug Components: client, resourcemanager Reporter: Gour Saha As per [~jianhe] these are some followup items that make sense to do after YARN-7766. Quoting Jian's comment below - Currently, if user doesn't supply location when run yarn app -enableFastLaunch, the jars will be put under this location {code} hdfs:///yarn-services//service-dep.tar.gz {code} Since API server is embedded in RM, should RM look for this location too if "yarn.service.framework.path" is not specified ? And if "yarn.service.framework.path" is not specified and still the file doesn't exist at above default location, I think RM can try to upload the jars to above default location instead, currently RM is uploading the jars to the location defined by below code. This folder is per app and also inconsistent with CLI location. {code} protected Path addJarResource(String serviceName, Map localResources) throws IOException, SliderException { Path libPath = fs.buildClusterDirPath(serviceName); {code} By doing this, the next time a submission request comes, RM doesn't need to upload the jars again. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-dev-help@hadoop.apache.org