ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 39111: [PREVIEW] Stop-and-Start Upgrade: Handle Core Services
Date Fri, 16 Oct 2015 02:48:03 GMT


> On Oct. 8, 2015, 2:05 p.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java,
line 232
> > <https://reviews.apache.org/r/39111/diff/1/?file=1092632#file1092632line232>
> >
> >     Maybe better >1 ? HDFS Federation feature seems to use multiple namenodes,
not sure if we support it in Ambari

Good catch, thanks.


> On Oct. 8, 2015, 2:05 p.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py,
line 124
> > <https://reviews.apache.org/r/39111/diff/1/?file=1092646#file1092646line124>
> >
> >     This construct is dangerous. When the first part fails (e.g. due to missing
hdfs_binary), pipe breaks and we get non-zero exit code. This code is treated as a grep result,
thus namenode is considered to be in safe mode. It's hard to debug since any error message
is hidden by pipe

I agree we should change it, but we can do that after we get the basics working.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39111/#review101898
-----------------------------------------------------------


On Oct. 8, 2015, midnight, Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39111/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, midnight)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, and
Nate Cole.
> 
> 
> Bugs: AMBARI-12701
>     https://issues.apache.org/jira/browse/AMBARI-12701
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Stop-and-Start Upgrade of the Stack to handle core services: HDFS, YARN, MR, ZK
> 
> * Renamed the functions pre_rolling_upgrade and post_rolling_upgrade to pre_upgrade_restart
and post_upgrade_restart that accept the upgrade_type (rolling|nonrolling) as a param.
> 
> * When NameNode HA is present, the command used to start NameNode is different between
the active and the standby, so changed MasterHostResolver.java to pick one of the NameNodes
to become the active and the other the standby, and pass it in a parameter called "desired_namenode_role"
> 
> * Wrote a function to take a backup of HDFS
> 
> * Changed the "hdfs" command to actually search which one to use depending on the stack
and version to account for Express Upgrade from HDP 2.1 -> 2.3
> 
> * Tested Express Upgrade with and without NameNode HA
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/script/script.py 26fd158

>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
a2f642c 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
ef75d38 
>   ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ed9b3a7

>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
648e7a4 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java
a8e9c43 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
1164335 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
4116101 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
68fec03 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java
92df3b5 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
6c0f3c7 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapper.java
f7cc930 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java
a5813e3 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py
fa68435 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
2e5ac19 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py
dd0dca4 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
b11d7ea 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py
46c7272 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py
e2ebbcb 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
07be496 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
e8c142c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
fb39878 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py
be6f0d5 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
5f6b8fe 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
f5df86f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py
108ef01 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py
7bc0b6a 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
14de094 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py
f3b3d11 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
16e34d4 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
5263d9f 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
fd25651 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
f3914f3 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py
e67f1ce 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py
e58ea3c 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py
973fa0f 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py
02c1006 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py
b7fb578 
>   ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_service.py
685eb6d 
> 
> Diff: https://reviews.apache.org/r/39111/diff/
> 
> 
> Testing
> -------
> 
> * Tested Express Upgrade with and without NameNode HA
> 
> There are still a handful of bugs.
> * YARN history server start fails because NameNode is still in safemode and waiting for
block reports.
> * YARN Service Check fails due to some bad properties in the config pack from 2.1->2.3
> * May need to remove HDP 2.1
> * Taking HDFS backup and report to /tmp folder
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message