Recently we decided to move some of our web applications, as well as this blog, to a new cloud server provided by DreamHost.com and their new DreamCompute OpenStack cluster in Virginia. Starting with a minimal install of CentOS7, we built a fairly standard LAMP server. Once we got the new server stable, we moved the apps over. The web apps hosted on our new server are GeekGlossary.com and jobs.beausanders.org. For the sake of this post, you need to know both of these apps have similar backup routines and logging functions.
After moving our apps, both of them began having issues saving backup files. The logging function stopped working as well. What made this most annoying is that PHP was not throwing any errors when executing the backup or logging scripts. We checked the permissions, paths, and variable names, as we set out to solve this problem. All looked fine.
Seasoned Linux administrators have already jumped to the end of this short story. It turns out that SELinux, which was in Enforcing mode, was blocking writes from httpd (Apache). We determined this by putting SELinux in Permissive mode with the following commands and re-running the backup and logging scripts:
# setenforce 0
# getenforce
Permissive
With SELinux in Permissive mode, CentOS7 and Apache did what was expected and saved the backups and created the log entries.
Now that the cause of the problem was confirmed, fixing it was fairly easy. In preparation, we did a little review online on SELinux. This “how-to” article on the CentOS website is very helpful.
Next we made sure that we had installed the CentOS7 packages to troubleshoot SELinux; these commands will install the necessary packages in CentOS7:
# yum install setroubleshoot-server
# yum install policycoreutils-python
Among other things, these packages provide semanage and sealert, two valuable SELinux command line tools needed to manage, monitor, and fix SElinux.
In order to see in plain English what SELinux is blocking and why, run the following command; your destination file can be whatever you want:
# sealert -a /var/log/audit/audit.log > /path/to/mylogfile.txt
Carefully read through the destination file from the command above and you will not only learn what the problems are, but also how to fix them…complete with commands. Basically, you will be instructed to change SELinux contexts for certain directories on your server, as well as create policy mods to keep the problem(s) from coming back. Cut and paste the suggested commands in to your command line interface and execute them. That’s it. Simple.
This is one of those situations where a little bit of reading and study goes a long way. My recommendation…do an online search for selinux sealert and learn about this very helpful tool.
After you have run the commands as instructed by sealert, be sure to turn SELinux Enforcing mode back on with these commands:
# setenforce 1
# getenforce
Enforcing
Now SELinux is protecting your server, while allowing custom web apps and other software tools you use to do their jobs. Periodically you should run sealert to check for any new problems related to enforcing SELinux.
