flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Lin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10130) How to define two hdfs name-node IPs in flink-conf.yaml file
Date Tue, 14 Aug 2018 12:14:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579710#comment-16579710
] 

Paul Lin commented on FLINK-10130:
----------------------------------

It could be solved by  [HDFS name services|https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html],
and I think it might be better to leave it to the HDFS client. FYI.

> How to define two hdfs name-node IPs in flink-conf.yaml file
> ------------------------------------------------------------
>
>                 Key: FLINK-10130
>                 URL: https://issues.apache.org/jira/browse/FLINK-10130
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Keshav Lodhi
>            Priority: Blocker
>         Attachments: docker-entrypoints.sh
>
>
> Hi Team,
> Here is, what we are looking for:
>  * We have  flink HA dockerized cluster with (3 zookeepers, 2 job-managers, 3 task-managers).
>  * We are using HDFS from the flink to store some data. The problem we are facing is
that, we are not able to pass 2 name-node IPs in config. 
>  * These are the config parameters we want to add two name-node IPs
>  #       "state.checkpoints.dir: hdfs://X.X.X.X:9001/flinkdatastorage"
>  #       "state.backend.fs.checkpointdir: hdfs://X.X.X.X:9001/flinkdatastorage"
>  #       "high-availability.zookeeper.storageDir: hdfs://X.X.X.X:9001/flink/recovery"
> Currently we are passing only one name-node IP
> Please advise. 
> I have attached the sample *docker-entrypoint.sh* file: 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message