From issues-return-174284-apmail-flink-issues-archive=flink.apache.org@flink.apache.org Mon Jul 2 09:50:02 2018 Return-Path: X-Original-To: apmail-flink-issues-archive@minotaur.apache.org Delivered-To: apmail-flink-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9FCA417978 for ; Mon, 2 Jul 2018 09:50:02 +0000 (UTC) Received: (qmail 74951 invoked by uid 500); 2 Jul 2018 09:50:02 -0000 Delivered-To: apmail-flink-issues-archive@flink.apache.org Received: (qmail 74915 invoked by uid 500); 2 Jul 2018 09:50:02 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 74906 invoked by uid 99); 2 Jul 2018 09:50:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Jul 2018 09:50:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 26C7D185022 for ; Mon, 2 Jul 2018 09:50:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id FkJnjLkfcsYq for ; Mon, 2 Jul 2018 09:50:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 0E0B45F49F for ; Mon, 2 Jul 2018 09:50:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 93C34E103C for ; Mon, 2 Jul 2018 09:50:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 51D0F21843 for ; Mon, 2 Jul 2018 09:50:00 +0000 (UTC) Date: Mon, 2 Jul 2018 09:50:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-9633) Flink doesn't use the Savepoint path's filesystem to create the OuptutStream on Task. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/FLINK-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529625#comment-16529625 ] ASF GitHub Bot commented on FLINK-9633: --------------------------------------- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/6194#discussion_r199425645 --- Diff: flink-core/src/main/java/org/apache/flink/core/fs/FileSystem.java --- @@ -416,6 +417,12 @@ public static FileSystem getUnguardedFileSystem(final URI fsUri) throws IOExcept } } + @VisibleForTesting + public static void setFsFactories(HashMap fsFactories) { --- End diff -- Let's pass in a `Map` instead of a `HashMap`. > Flink doesn't use the Savepoint path's filesystem to create the OuptutStream on Task. > ------------------------------------------------------------------------------------- > > Key: FLINK-9633 > URL: https://issues.apache.org/jira/browse/FLINK-9633 > Project: Flink > Issue Type: Bug > Components: State Backends, Checkpointing > Affects Versions: 1.5.0 > Reporter: Sihua Zhou > Assignee: Sihua Zhou > Priority: Critical > Labels: pull-request-available > Fix For: 1.6.0, 1.5.1 > > > Currently, flink use the Savepoint's filesystem to create the meta output stream in CheckpointCoordinator(JM side), but in StreamTask(TM side) it uses the Checkpoint's filesystem to create the checkpoint data output stream. When the Savepoint & Checkpoint in different filesystem this will lead to problematic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)