airavata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Christie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRAVATA-2581) Manually deploy Django version of dev seagrid
Date Fri, 03 Nov 2017 20:32:00 GMT

    [ https://issues.apache.org/jira/browse/AIRAVATA-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238309#comment-16238309
] 

Marcus Christie commented on AIRAVATA-2581:
-------------------------------------------

Current work I've done so far
{noformat}
[Using Daemon process with mod_wsgi](https://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html#delegation-to-daemon-process)
* also has an example virtual host configuration

Steps:
* checkout the django code
* create a virtual environment
* source virtual env and run `pip install -r requirements.txt` in that environment
* create a settings_local.py will have all the necessary settings
* set the STATIC_ROOT in settings_local.py
* run build_js.sh
* run `collectstatic`
* create virtual host config [like this example](https://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html#delegation-to-daemon-process)
* gracefully restart apache
* create a database in MySQL for django database
* migrate database and configure

For the test server:
* connect to dev.seagrid.org like we currently have
* domain name: django.seagrid.org

Do we need to rebuild mod_wsgi every time that the python version is updated?
* No. Python3.4 installed by yum was compiled with `--enable-shared`. See [python patch level
mismatch](https://modwsgi.readthedocs.io/en/develop/user-guides/installation-issues.html#python-patch-level-mismatch)
* verifying
```
[centos@pga-scigap-develop ~]$ python3
Python 3.4.5 (default, May 29 2017, 15:17:55)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CONFIG_ARGS')
"'--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking'
'--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc'
'--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec'
'--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info'
'--enable-ipv6' '--enable-shared' '--with-computed-gotos=yes' '--with-dbmliborder=gdbm:ndbm:bdb'
'--with-system-expat' '--with-system-ffi' '--enable-loadable-sqlite-extensions' '--with-systemtap'
'--with-valgrind' '--without-ensurepip' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu'
'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches   -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv  ' 'LDFLAGS=-Wl,-z,relro
  ' 'CPPFLAGS= ' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'"
```

Actual installation
* sudo yum -y install python34
* sudo yum -y install httpd-devel
* sudo yum -y install python34-devel
* install mod_wsgi from source
```
mkdir mod_wsgi
cd mod_wsgi/
curl -LO https://github.com/GrahamDumpleton/mod_wsgi/archive/4.5.17.tar.gz
tar zxf 4.5.17.tar.gz
cd mod_wsgi-4.5.17/
./configure --with-python=/usr/bin/python3
make
sudo make install
```
* configure Apache to load mod_wsgi
```
sudo vim /etc/httpd/conf.modules.d/00-wsgi.conf
```
* 00-wsgi.conf file contains
```
LoadModule wsgi_module modules/mod_wsgi.so
```
* `sudo apachectl restart`
* Verify line in apache error log showing the mod_wsgi and Python34 loaded:
```
[Thu Aug 24 14:20:20.171560 2017] [mpm_prefork:notice] [pid 6657] AH00163: Apache/2.4.6 (CentOS)
OpenSSL/1.0.1e-fips mod_wsgi/4.5.17 Python/3.4 PHP/5.4.16 configured -- resuming normal operations
```
* Clean up build: `make clean`
* Clone django code and setup virtual environment
```
mkdir django-seagrid
cd django-seagrid/
git clone https://github.com/machristie/django-airavata-gateway.git
python3 -m venv venv
source venv/bin/activate
cd django-airavata-gateway/
pip install -r requirements.txt 
```

* create the settings_local.py file

    """
    Override default Django settings for a particular instance.

    Copy this file to settings_local.py and modify as appropriate. This file will
    be imported into settings.py last of all so settings in this file override any
    defaults specified in settings.py.
    """

    import os

    # Build paths inside the project like this: os.path.join(BASE_DIR, ...)
    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    # Keycloak Configuration
    KEYCLOAK_CLIENT_ID = 'pga'
    KEYCLOAK_CLIENT_SECRET = '5d2dc66a-f54e-4fa9-b78f-80d33aa862c1'
    KEYCLOAK_AUTHORIZE_URL = 'https://iamdev.scigap.org/auth/realms/seagrid/protocol/openid-connect/auth'
    KEYCLOAK_TOKEN_URL = 'https://iamdev.scigap.org/auth/realms/seagrid/protocol/openid-connect/token'
    KEYCLOAK_USERINFO_URL = 'https://iamdev.scigap.org/auth/realms/seagrid/protocol/openid-connect/userinfo'
    KEYCLOAK_LOGOUT_URL = 'https://iamdev.scigap.org/auth/realms/seagrid/protocol/openid-connect/logout'
    KEYCLOAK_CA_CERTFILE = os.path.join(BASE_DIR, "django_airavata", "resources", "incommon_rsa_server_ca.pem")
    KEYCLOAK_VERIFY_SSL = True

    # Airavata API Configuration
    GATEWAY_ID = 'seagrid'
    AIRAVATA_API_HOST = 'apidev.scigap.org'
    AIRAVATA_API_PORT = 9930
    AIRAVATA_API_SECURE = True

    # Sharing API Configuration
    SHARING_API_HOST = 'apidev.scigap.org'
    SHARING_API_PORT = 7878
    SHARING_API_SECURE = False

    STATIC_ROOT = "/var/www/portals/django-seagrid/static/"

* as user pga, ran `python manage.py collectstatic`
* create virtual host config

# 2017-11-03

## Continuing deployment

* update code and virtual environment
```
cd portals/
cd django-seagrid/
cd django-airavata-gateway/
git remote set-url origin https://github.com/apache/airavata-django-portal.git
git pull --ff-only
source ../venv/bin/activate
pip install -r requirements.txt 
pip install --upgrade pip
```
* run build_js.sh: install npm
```
sudo yum update epel-release
sudo yum install npm
```
* run build_js.sh as user pga
```
./build_js.sh
```
* create virtual host config
```xml
<VirtualHost *:80>
    ServerName django.seagrid.org
    
    ## Redirect all http traffic to https
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

<VirtualHost *:443>
    ServerName django.seagrid.org

    Alias /robots.txt /var/www/portals/django-seagrid/static/robots.txt
    Alias /favicon.ico /var/www/portals/django-seagrid/static/favicon.ico

    Alias /static/ /var/www/portals/django-seagrid/static/

    <Directory /var/www/portals/django-seagrid/static>
    Require all granted
    </Directory>

    WSGIDaemonProcess django.seagrid.org python-home=/var/www/portals/django-seagrid/venv
python-path=/var/www/portals/django-seagrid/django-airavata-gateway processes=2
    WSGIProcessGroup django.seagrid.org

    WSGIScriptAlias / /var/www/portals/django-seagrid/django-airavata-gateway/django_airavata/wsgi.py

    <Directory /var/www/portals/django-seagrid/django-airavata-gateway/django_airavata>
        <Files wsgi.py>
        Require all granted
        </Files>
    </Directory>

    ErrorLog /var/log/httpd/django-seagrid.error.log
    CustomLog /var/log/httpd/django-seagrid.requests.log combined

    SSLEngine on
    # Disable SSLv3 which is vulnerable to the POODLE attack
    SSLProtocol All -SSLv2 -SSLv3
    SSLCertificateFile /etc/letsencrypt/live/django.seagrid.org/cert.pem
    SSLCertificateChainFile /etc/letsencrypt/live/django.seagrid.org/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/django.seagrid.org/privkey.pem
</VirtualHost>
```
* 403 Forbidden
```
[Fri Nov 03 18:15:17.798456 2017] [core:error] [pid 6181] (13)Permission denied: [client 149.160.163.103:49683]
AH00035: access to / denied (filesystem path '/var/www/portals/django-seagrid/django-airavata-gateway/django_airavata/wsgi.py')
because search permissions are missing on a component of the path
```
* installed setroubleshoot-server to debug, I think it is an SELinux problem
```
sealert -a /var/log/audit/audit.log
```
* running restorecon
```
restorecon -R /var/www/portals/django-seagrid/
```
* new error:
    ```
    [Fri Nov 03 19:02:46.626623 2017] [wsgi:error] [pid 2738] [remote 149.160.163.103:50169]
ImportError: No module named 'apache.airavata'; 'apache' is not a package
    ```
    and `sealert -a /var/log/audit/audit.log` reports:
    ```
    found 1 alerts in /var/log/audit/audit.log
    --------------------------------------------------------------------------------

    SELinux is preventing /usr/sbin/httpd from read access on the file pyvenv.cfg.

    *****  Plugin catchall_labels (83.8 confidence) suggests   *******************

    If you want to allow httpd to have read access on the pyvenv.cfg file
    Then you need to change the label on pyvenv.cfg
    Do
    # semanage fcontext -a -t FILE_TYPE 'pyvenv.cfg'
    ...
    ```
    But which pyvenv.cfg?
* trying to turn off SELinux so I can make sure everything is installed correctly first
```
setenforce 0
```
* there's some other `apache` object in the Python namespace that is conflicting with `apache.airavata`.
I get this when I print the `apache` object imported in the wsgi.py script:
```
[Fri Nov 03 20:12:39.228053 2017] [wsgi:error] [pid 10885] apache: ['__doc__', '__loader__',
'__name__', '__package__', '__spec__', 'build_date', 'description', 'maximum_processes', 'mpm_name',
'threads_per_process', 'version']
```
* turning SELinux back on
```
setenforce 1
```

{noformat}

> Manually deploy Django version of dev seagrid
> ---------------------------------------------
>
>                 Key: AIRAVATA-2581
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2581
>             Project: Airavata
>          Issue Type: Sub-task
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message