airavata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Christie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRAVATA-2627) Letsencrypt auto renewal is preventing Apache from restarting
Date Wed, 23 May 2018 14:24:00 GMT

    [ https://issues.apache.org/jira/browse/AIRAVATA-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487338#comment-16487338
] 

Marcus Christie commented on AIRAVATA-2627:
-------------------------------------------

Letsencrypt renewal failed on the dreg jetstream instance. From /var/log/httpd/error_log:
{noformat}
[Wed May 23 04:36:15.704698 2018] [mpm_prefork:notice] [pid 29883] AH00171: Graceful restart
requested, doing restart
AH00112: Warning: DocumentRoot [/www/default] does not exist
AH00112: Warning: DocumentRoot [/var/lib/letsencrypt/tls_sni_01_page/] does not exist
AH00112: Warning: DocumentRoot [/var/lib/letsencrypt/tls_sni_01_page/] does not exist
[Wed May 23 04:36:31.015404 2018] [auth_digest:notice] [pid 29883] AH01757: generating secret
for digest authentication ...
[Wed May 23 04:36:31.111576 2018] [lbmethod_heartbeat:notice] [pid 29883] AH02282: No slotmem
from mod_heartmonitor
[Wed May 23 04:36:31.111741 2018] [ssl:emerg] [pid 29883] (2)No such file or directory: AH02201:
Init: Can't open server certificate file /var/lib/letsencrypt/JbZ1--OTKoDFcaH4fCFIxdjQmOmpNE6Win6w4Eclqgw.crt
[Wed May 23 04:36:31.111763 2018] [ssl:emerg] [pid 29883] AH02312: Fatal error initialising
mod_ssl, exiting.
{noformat}

>From /var/log/letsencrypt/letsencrypt.log
{noformat}
2018-05-23 04:36:31,084:DEBUG:certbot.reporter:Reporting to user: The following errors were
reported by the server:

Domain: dreg.dnasequence.org
Type:   connection
Detail: Timeout after connect (your server may be slow or overloaded)

To fix these errors, please make sure that your domain name was entered correctly and the
DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please
check that your computer has a publicly routable IP address and that no firewalls are preventing
the server from communicating with the client. If you're using the webroot plugin, you should
also verify that you are serving files from the webroot path you provided.
2018-05-23 04:36:31,085:INFO:certbot.auth_handler:Cleaning up challenges
2018-05-23 04:36:31,207:ERROR:certbot.util:Error while running apachectl graceful.

Job for httpd.service invalid.

2018-05-23 04:36:31,207:WARNING:certbot.renewal:Attempting to renew cert (dreg.dnasequence.org)
from /etc/letsencrypt/renewal/dreg.dnasequence.org.conf produced an unexpected error: Error
while running apachectl graceful.

Job for httpd.service invalid.
. Skipping.
2018-05-23 04:36:31,311:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/certbot/renewal.py", line 422, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 1102, in renew_cert
    _get_and_save_cert(le_client, config, lineage=lineage)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 113, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/usr/lib/python2.7/site-packages/certbot/renewal.py", line 297, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains)
  File "/usr/lib/python2.7/site-packages/certbot/client.py", line 294, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python2.7/site-packages/certbot/client.py", line 330, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/usr/lib/python2.7/site-packages/certbot/auth_handler.py", line 82, in handle_authorizations
    self._respond(resp, best_effort)
  File "/usr/lib/python2.7/site-packages/certbot/auth_handler.py", line 159, in _respond
    self._cleanup_challenges(active_achalls)
  File "/usr/lib/python2.7/site-packages/certbot/auth_handler.py", line 304, in _cleanup_challenges
    self.auth.cleanup(achalls)
  File "/usr/lib/python2.7/site-packages/certbot_apache/configurator.py", line 2109, in cleanup
    self.restart()
  File "/usr/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1989, in restart
    self._reload()
  File "/usr/lib/python2.7/site-packages/certbot_apache/configurator.py", line 2000, in _reload
    raise errors.MisconfigurationError(str(err))
MisconfigurationError: Error while running apachectl graceful.

Job for httpd.service invalid.


2018-05-23 04:36:31,311:ERROR:certbot.renewal:All renewal attempts failed. The following certs
could not be renewed:
2018-05-23 04:36:31,311:ERROR:certbot.renewal:  /etc/letsencrypt/live/dreg.dnasequence.org/fullchain.pem
(failure)
{noformat}

This matches pretty well with this certbot issue: https://github.com/certbot/certbot/issues/5439

So I tried to fix by upgrading with yum:
{noformat}
	 yum makecache fast
	 yum update python2-certbot-apache
	 yum update certbot
	 systemctl start certbot-renew
{noformat}

The letsencrypt renewal worked successfully this time.


> Letsencrypt auto renewal is preventing Apache from restarting
> -------------------------------------------------------------
>
>                 Key: AIRAVATA-2627
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2627
>             Project: Airavata
>          Issue Type: Bug
>          Components: PGA PHP Web Gateway
>            Reporter: Marcus Christie
>            Assignee: Marcus Christie
>            Priority: Major
>
> The {{certbot renew --quiet}} command in the crontab is apparently causing Apache to
fail to reload:
> From the systemd journal ({{journalctl -xe}}):
> {noformat}
> -- Unit session-34124.scope has begun starting up.
> Jan 09 12:50:01 gridfarm004.ucs.indiana.edu CROND[11610]: (root) CMD (/usr/lib64/sa/sa1
1 1)
> Jan 09 12:52:01 gridfarm004.ucs.indiana.edu systemd[1]: Started Session 34125 of user
root.
> -- Subject: Unit session-34125.scope has finished start-up
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit session-34125.scope has finished starting up.
> -- 
> -- The start-up result is done.
> Jan 09 12:52:01 gridfarm004.ucs.indiana.edu systemd[1]: Starting Session 34125 of user
root.
> -- Subject: Unit session-34125.scope has begun start-up
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit session-34125.scope has begun starting up.
> Jan 09 12:52:01 gridfarm004.ucs.indiana.edu CROND[11692]: (root) CMD (/usr/bin/certbot
renew --quiet)
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu httpd[11725]: AH00112: Warning: DocumentRoot
[/www/default] does not exist
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu httpd[11725]: AH00526: Syntax error on line
10 of /etc/httpd/conf.d/le_tls_sni_01_cert_challenge.conf:
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu httpd[11725]: SSLCertificateFile: file '/var/lib/letsencrypt/YDnHNU3oKDOaT_oO2qXSoXR65gUb7k66KB0dF4nwT-8.crt'
does not exist or is empty
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu systemd[1]: httpd.service: control process
exited, code=exited status=1
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu systemd[1]: Reload failed for The Apache
HTTP Server.
> -- Subject: Unit httpd.service has finished reloading its configuration
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit httpd.service has finished reloading its configuration
> -- 
> -- The result is failed.
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu httpd[11735]: AH00112: Warning: DocumentRoot
[/www/default] does not exist
> Jan 09 12:52:03 gridfarm004.ucs.indiana.edu systemd[1]: Reloaded The Apache HTTP Server.
> -- Subject: Unit httpd.service has finished reloading its configuration
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit httpd.service has finished reloading its configuration
> -- 
> -- The result is done.
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu httpd[11757]: AH00112: Warning: DocumentRoot
[/www/default] does not exist
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu httpd[11757]: AH00526: Syntax error on line
10 of /etc/httpd/conf.d/le_tls_sni_01_cert_challenge.conf:
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu httpd[11757]: SSLCertificateFile: file '/var/lib/letsencrypt/9qLZfLerTerU_bGLYPfXWXq-EXktXgYfNQAEQcdHSpE.crt'
does not exist or is empty
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu systemd[1]: httpd.service: control process
exited, code=exited status=1
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu systemd[1]: Reload failed for The Apache
HTTP Server.
> -- Subject: Unit httpd.service has finished reloading its configuration
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit httpd.service has finished reloading its configuration
> -- 
> -- The result is failed.
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu httpd[11767]: AH00112: Warning: DocumentRoot
[/www/default] does not exist
> Jan 09 12:52:05 gridfarm004.ucs.indiana.edu systemd[1]: Reloaded The Apache HTTP Server.
> -- Subject: Unit httpd.service has finished reloading its configuration
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> -- 
> -- Unit httpd.service has finished reloading its configuration
> -- 
> -- The result is done.
> Jan 09 12:52:07 gridfarm004.ucs.indiana.edu httpd[11796]: AH00112: Warning: DocumentRoot
[/www/default] does not exist
> Jan 09 12:52:07 gridfarm004.ucs.indiana.edu httpd[11796]: AH00526: Syntax error on line
10 of /etc/httpd/conf.d/le_tls_sni_01_cert_challenge.conf:
> Jan 09 12:52:07 gridfarm004.ucs.indiana.edu httpd[11796]: SSLCertificateFile: file '/var/lib/letsencrypt/I69cuV1431Lfk88VjtDFxlBPEnagdg5atz9dhGhsxfY.crt'
does not exist or is empty
> Jan 09 12:52:07 gridfarm004.ucs.indiana.edu systemd[1]: httpd.service: control process
exited, code=exited status=1
> Jan 09 12:52:07 gridfarm004.ucs.indiana.edu systemd[1]: Reload failed for The Apache
HTTP Server.
> ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message