2018/05/30
It looks like nginx was set to use a different set of certificate files than the ones Let's Encrypt was set to renew.
Tentatively, LE goes through all the .conf files in /etc/letsencrypt/renewal and renews each one.
There was only one, and it pointed at files in /etc/letsencrypt/live/tootcat2.hypertwins.net/
I've changed it to point to /etc/letsencrypt/live/toot.cat/
Nginx also looks for 2 cert files in /etc/letsencrypt/live/toot.cat/, so now at least we're matched.
When I try to renew with letsencrypt renew, I get:
root@tootcat2:/# letsencrypt renew Processing /etc/letsencrypt/renewal/toot.cat.conf 2018-05-31 00:11:15,708:ERROR:letsencrypt.error_handler:Encountered exception during recovery 2018-05-31 00:11:15,709:ERROR:letsencrypt.error_handler:Missing --webroot-path for domain: toot.cat Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/letsencrypt/error_handler.py", line 74, in call_registered self.funcs[-1]() File "/usr/lib/python2.7/dist-packages/letsencrypt/auth_handler.py", line 280, in _cleanup_challenges self.dv_auth.cleanup(dv_c) File "/usr/lib/python2.7/dist-packages/letsencrypt/plugins/webroot.py", line 139, in cleanup root_path = self._get_root_path(achall) File "/usr/lib/python2.7/dist-packages/letsencrypt/plugins/webroot.py", line 108, in _get_root_path .format(achall.domain)) PluginError: Missing --webroot-path for domain: toot.cat 2018-05-31 00:11:15,711:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/toot.cat.conf produced an unexpected error: Missing --webroot-path for domain: toot.cat. Skipping. All renewal attempts failed. The following certs could not be renewed: /etc/letsencrypt/live/toot.cat/fullchain.pem (failure) 1 renew failure(s), 0 parse failure(s) root@tootcat2:/#
The key piece of information there seems to be "Missing --webroot-path for domain". What seems to be happening is that Nginx is redirecting from http to https even though the file exists.
Phase 2
It turned out there was an HSTS policy that was forcing the browser to redirect even though Nginx wasn't, I think? But opening a toot.cat URL in an anonymous window fixed that. However it still wasn't finding the test file, so I changed the webroot on both Nginx and Let'sEncrypt to /var/www/challenges, and then was able to access the test file.
...but Let's Encrypt still returns this:
root@tootcat2:/var/www/challenges# letsencrypt renew Processing /etc/letsencrypt/renewal/toot.cat.conf 2018-05-31 00:49:39,460:ERROR:letsencrypt.error_handler:Encountered exception during recovery 2018-05-31 00:49:39,460:ERROR:letsencrypt.error_handler:Missing --webroot-path for domain: toot.cat Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/letsencrypt/error_handler.py", line 74, in call_registered self.funcs[-1]() File "/usr/lib/python2.7/dist-packages/letsencrypt/auth_handler.py", line 280, in _cleanup_challenges self.dv_auth.cleanup(dv_c) File "/usr/lib/python2.7/dist-packages/letsencrypt/plugins/webroot.py", line 139, in cleanup root_path = self._get_root_path(achall) File "/usr/lib/python2.7/dist-packages/letsencrypt/plugins/webroot.py", line 108, in _get_root_path .format(achall.domain)) PluginError: Missing --webroot-path for domain: toot.cat 2018-05-31 00:49:39,462:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/toot.cat.conf produced an unexpected error: Missing --webroot-path for domain: toot.cat. Skipping. All renewal attempts failed. The following certs could not be renewed: /etc/letsencrypt/live/toot.cat/fullchain.pem (failure) 1 renew failure(s), 0 parse failure(s) root@tootcat2:/var/www/challenges#
Phase 3
root@tootcat2:/var/www/challenges/.well-known# letsencrypt renew Processing /etc/letsencrypt/renewal/toot.cat.conf 2018-05-31 01:30:48,656:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/toot.cat.conf produced an unexpected error: urn:acme:error:rateLimited :: There were too many requests of a given type :: Error creating new authz :: too many failed authorizations recently: see https://letsencrypt.org/docs/rate-limits/. Skipping. All renewal attempts failed. The following certs could not be renewed: /etc/letsencrypt/live/toot.cat/fullchain.pem (failure) 1 renew failure(s), 0 parse failure(s) root@tootcat2:/var/www/challenges/.well-known#
root@tootcat2:/var/www/challenges/.well-known# letsencrypt --staging renew Processing /etc/letsencrypt/renewal/toot.cat.conf 2018-05-31 01:36:59,134:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/toot.cat.conf produced an unexpected error: You should register before running non-interactively, or provide --agree-tos and --email <email_address> flags. Skipping. All renewal attempts failed. The following certs could not be renewed: /etc/letsencrypt/live/toot.cat/fullchain.pem (failure) 1 renew failure(s), 0 parse failure(s) root@tootcat2:/var/www/challenges/.well-known#