ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Ian.Malo...@tdameritrade.com>
Subject Re: Question about python scripts
Date Wed, 30 Nov 2016 19:49:01 GMT
I’m running Jupyterhub, which requires python 3. When I start up my spark
notebooks (which use yarn-client as the master),
those scripts are called and they use "/usr/bin/env python”. Since they’re
regenerated, I would like to find away to change
that (the first line in each file) to “/usr/bin/python”.

I would be happy to try to create a patch, but am unfamiliar with the code
base.









On 11/30/16, 2:21 PM, "Alejandro Fernandez" <afernandez@hortonworks.com>
wrote:

>Hi Ian,
>
>We would need more information about where that version of python is being
>invoked.
>You're welcome to submit a patch.
>
>Thanks,
>Alejandro
>
>
>On 11/29/16, 2:33 PM, "Ian.Maloney@tdameritrade.com"
><Ian.Maloney@tdameritrade.com> wrote:
>
>>Hi Alejandro,
>>
>>Sorry for the delay. I looked at that script and it didn’t seem like it
>>would allow for me to force the right python. What I need is something
>>like where the scripts get generated, I just need to change the top line
>>of the scripts below, but since they’re regenerated when services are
>>stopped, I need to do it programmatically. Maybe this could be made as a
>>property, which is optionally configurable?
>>
>>
>>/usr/bin/hdp-select
>>/etc/hadoop/conf/topology_script.py
>>
>>
>>
>>Thanks,
>>
>>Ian
>>
>>
>>
>>
>>
>>
>>
>>On 11/18/16, 6:44 PM, "Alejandro Fernandez" <afernandez@hortonworks.com>
>>wrote:
>>
>>>Hi Ian,
>>>
>>>Ambari currently supports Python 2.6 and 2.7.
>>>Take a look at 
>>>https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_a
>>>m
>>>b
>>>ari_blob_trunk_ambari-2Dshell_ambari-2Dpython-2Dshel&d=DgIFAw&c=nulvIAQn
>>>C
>>>0
>>>yOOjC0e0NVa8TOcyq9jNhjZ156R-JJU10&r=CxpqDYMuQy-1uNI-UOyUbaX6BMPCZXH8d8ev
>>>u
>>>C
>>>oP_OA&m=9yDxGmbR83EbujEvcB36vHJPNAosEHXd763_ySz8hk0&s=PbZ99YTBAIx1e3-OmT
>>>8
>>>-
>>>6HZbd9isTV8yXA4SH0Tcxi4&e=
>>>l/conf/unix/ambari-shell to see if you can force it to always use Python
>>>2.#
>>>
>>>Thanks,
>>>Alejandro
>>>
>>>On 11/17/16, 2:22 PM, "Ian.Maloney@tdameritrade.com"
>>><Ian.Maloney@tdameritrade.com> wrote:
>>>
>>>>Hi,
>>>>
>>>>I¹m currently running into an issue, where some of the Ambari related
>>>>python scripts are being invoked from a python 3 environment:
>>>>
>>>>/usr/bin/hdp-select
>>>>/etc/hadoop/conf/topology_script.py
>>>>
>>>>The reason this happens is a bit complicated, I¹m running jupyterhub,
>>>>which requires a python 3 environment on my management node, where
>>>>ambari
>>>>is located. Ambari uses python 2 currently. For most kernels, it works
>>>>fine, but Apache Toree, is for some reason calling those scripts from
>>>>within the python 3 environment. Since the scripts use /usr/bin/env
>>>>python, when they are in the conda environment for jupyterhub, they use
>>>>python 3 and break. So, what I have done is update the scripts to be
>>>>usable in both python 2 and 3. Unfortunately, it appears these scripts
>>>>are recreated when the cluster goes down (or is built), so I¹d have to
>>>>swap the files out.
>>>>
>>>>What I¹m proposing is that my changes for the scripts (or someone
>>>>else¹s), get committed, so with future versions of Ambari, there are
>>>>not
>>>>issues of cross compatibility.
>>>>
>>>>How does this sound?
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message