FreeBSD/APACHE/Let's Encrypt

Interestingly enough I have no problem using let's encrypt on Windows platform and figured it was going to be a breeze on *nix... Running into a slew of issues.  For one I am trying to use certbot to facilitate this.  When I issue the correct commands and webroot I see it builds the .well-known folder but it does not build the acme-challenge folder.  If I try to manually create the acme-challenge folder it deletes it after the sudo certbot certonly --webroot -w /var/www/example -d -d command (with my domain info substituted of course) with the following error:

Detail: Invalid response from
   <title>403 Forbidden</title>
Super frustrated as everything in *nix is always easier, right?  Well not this time LOL.   Any suggestions would be appreciated!

LVL 18
John Gates, CISSPSecurity ProfessionalAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jim RiddlesPrepress/OMS SpecialistCommented:
Out of curiosity, does your config file for these domains force a SSL connection?  If it does, disable that before running your command, and try again.  That was an early gotcha for me.
John Gates, CISSPSecurity ProfessionalAuthor Commented:
It does not.  As a matter of fact, SSL is not even currently enabled.  I am trying to build the cert so that I can enable it.  It's really odd.  The certbot seems to be creating the .well-known folder so I am lost on why it can't create the acme-challenge folder and drop the subject into it...
Jim RiddlesPrepress/OMS SpecialistCommented:
Is there anything in your config that would prevent an outside client from accessing your webroot for that domain?  Are you able to post the actual domains here so that I can attempt to connect from my location?
David FavorLinux/LXD/WordPress/Hosting SavantCommented:
Looks like your Apache config has disallowed directory listing (403 Forbidden).

For Apache-2.4.x Try adding something like this...


      ... ... ...

      DocumentRoot /path-to-your-root-dir

      <Directory /path-to-your-root-dir>
          Options +Indexes +FollowSymLinks
          AllowOverride All 
          Require all granted

      ... ... ...


Open in new window

When  you can create files + directories in .well-known likely your cert generation will work.

Also, be sure...

1) /path-to-your-root-dir is owned by Apache

2) Be sure you have latest certbot-auto installed per instructions.

So arrange for latest version of certbot-auto to supersede any OS versions, as some OSes provide very old versions.

chmod a+x certbot-auto

Open in new window

Old versions of certbot-auto have some gnarly bugs. Best to always start with latest version.
John Gates, CISSPSecurity ProfessionalAuthor Commented:
I will try this a little later and respond.  Thanks!
David FavorLinux/LXD/WordPress/Hosting SavantCommented:
You're welcome!
John Gates, CISSPSecurity ProfessionalAuthor Commented:
Those directives were already in the config.  The certbot is creating the .well-known folder but nothing under it...  Do I have the process down.  Certbot makes the calls to let's encrypt and then is supposed to be staging the challenge in the path <domain>/.well-known/acme-challenge/<file>  this is not happening, it is not building anything beyond .well-known

John Gates, CISSPSecurity ProfessionalAuthor Commented:
also if I manually create the acme-challenge folder and then run the command the folder get's deleted.  I can only assume that the certbot is deleting, and recreating this each time it runs.  It is bombing out when trying to create the subfolder acme-challenge really pulling my hair out now.  This took me 5 minutes on Windows .....
John Gates, CISSPSecurity ProfessionalAuthor Commented:
2017-07-19 02:18:16,398:INFO:certbot.plugins.webroot:Using the webroot path /usr/local/www/apache24/data for all unmatched domains.
2017-07-19 02:18:16,399:DEBUG:certbot.plugins.webroot:Creating root challenges validation dir at /usr/local/www/apache24/data/.well-known/acme-challenge
2017-07-19 02:18:16,420:DEBUG:certbot.plugins.webroot:Creating root challenges validation dir at /usr/local/www/apache24/data/.well-known/acme-challenge
2017-07-19 02:18:16,428:DEBUG:certbot.plugins.webroot:Attempting to save validation to /usr/local/www/apache24/data/.well-known/acme-challenge/Z7yBMzvliHRoK$
2017-07-19 02:18:16,431:DEBUG:certbot.plugins.webroot:Attempting to save validation to /usr/local/www/apache24/data/.well-known/acme-challenge/OgypMbYTLH44K$
2017-07-19 02:18:16,431:INFO:certbot.auth_handler:Waiting for verification...

So it's logging the creation but the creation is never happening.... Not even sure what to do with this.

John Gates, CISSPSecurity ProfessionalAuthor Commented:
What a PITA LOL so I had to add a .htaccess file with the following directive:

RewriteRule "/\.|^\.(?!well-known/)" - [F]

this then allowed browsing to the .well-known directory

It appears that the certbot is state in time so it writes the entries, verifies and then cleans up.  That's what threw me.  Posting this so it helps others as this did solve the issue and allow me to get the certs!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
John Gates, CISSPSecurity ProfessionalAuthor Commented:
I figured it out but Jim was on point that it was not able to access the directory location.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.