airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijayanand Rengarajan <>
Subject using HDFSSensor with Namenode HA
Date Sat, 23 Sep 2017 21:49:45 GMT

I have a question on using HDFSSensor with Namenode HA  ( Create a
Connection in the UI pointing to your namenode).

The question is, what is the value of the Conn id "hdfs_default" , when we
have Namenode HA enabled?

My setup is working fine when I point to active namenode but I need to
change it every time when NN failover happens.

So What should be the value of Conn Type & Host to point Namenode Name
Service instead of pointing to particular Namenode.

I tried with below value but that is not working.

fs.default = hdfs://ClusterName

I set Conn Type = hdfs

        Host           = ClusterName

        Port .         = "" ( empty).

Please guide.

Thanks, Vijayanand.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message