hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Singh (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HADOOP-12988) upgrading the HDP stack through Ambari (from 2.3 to 2.4)
Date Thu, 31 Mar 2016 23:35:25 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun Singh reassigned HADOOP-12988:
-----------------------------------

    Assignee: Ali Bajwa

Please let me know if any additional info is needed. We will get from customer/eng. Thx.

> upgrading the HDP stack through Ambari (from 2.3 to 2.4)
> --------------------------------------------------------
>
>                 Key: HADOOP-12988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12988
>             Project: Hadoop Common
>          Issue Type: Bug
>         Environment: SLES11 SP4
>            Reporter: Arun Singh
>            Assignee: Ali Bajwa
>
>  - Issue 3: When upgrading the HDP stack through Ambari (from 2.3 to 2.4), at some point
a YARN smokescreen test is performed. This smoke screen test fails, as it is trying to call
an API command using curl with the --negotiate option. The problem is that on SUSE 11, the
version of curl does not ship with one that understands "--negotiate", grinding the whole
upgrade process to a halt. 
>  
> There are quite a few files in Ambari where this seems to be the case, although I personally
only encountered it during the YARN component: 
> /var/lib/ambari-server/resources/common-services/RANGER/0.4.0/package/scripts/service_check.py:
     
>     Execute(format("curl -s -o /dev/null -w'%{{http_code}}' --negotiate -u: -k {ranger_external_url}/login.jsp
| grep 200"),
> /var/lib/ambari-server/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py:
   
>     Execute(format("curl -s -o /dev/null -w'%{{http_code}}' --negotiate -u: -khttp://{spark_history_server_host}:{spark_history_ui_port}
| grep 200"),
> /var/lib/ambari-server/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py:
     
>     get_app_info_cmd = "curl --negotiate -u : -ksL --connect-timeout " + CURL_CONNECTION_TIMEOUT
+ " " + info_app_url
> /var/lib/ambari-server/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py:
   
>     smoke_cmd = format('curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt -s
-o /dev/null -w "%{{http_code}}"http://{metadata_host}:{metadata_port}/')
> /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh:
>     cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>'  $ttonurl/status
2>&1"
> /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh:
 
>     cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>'  $ttonurl/status?user.name=$smoke_test_user
2>&1"
> /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh:
>     cmd="${kinitcmd}curl --negotiate -u : -s -w 'http_code <%{http_code}>' -d 
\@${destdir}/show_db.post.txt  $ttonurl/ddl 2>&1"
> For example, in this file: /var/lib/ambari-server/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py
> You will see the code as:
> for rm_webapp_address in params.rm_webapp_addresses_list:
>       info_app_url = params.scheme + "://" + rm_webapp_address + "/ws/v1/cluster/apps/"
+ application_name
>       get_app_info_cmd = "curl --negotiate -u : -ksL --connect-timeout " + CURL_CONNECTION_TIMEOUT
+ " " + info_app_url
>       return_code, stdout, _ = get_user_call_output(get_app_info_cmd,
>                                             user=params.smokeuser,
>                                             path='/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin',
>                                             )
>  
>  
> There is no code checking whether RHEL vs SUSE, to run the correct usage of curl. Or
alternatively, there is no code to check for version of curl, and run a "deprecated" version
of the command as a fallback should it detect that the installed curl does not support --negotiate.
This is just blindly assuming to work on SUSE 11 (or any version of curl). 
>  
> Information about the curl installed on the system: 
> hdplab02:~ # curl -V 
> curl 7.45.0 (x86_64-pc-linux-gnu) libcurl/7.45.0 OpenSSL/1.0.2d zlib/1.2.8 
> Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp
smtps telnet tftp 
> Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message