Website monitoring (or uptime monitoring) is essential for any commercial website or service and there are plenty of commercial vendors that provide these services. With the recent rollout of GitHub Actions, I’ve put together a simple lightweight website monitor you can implement for free, perfect for side projects and personal websites. In addition to this WordPress blog I’m using it to watch my parked domains and Tweetfave project, making sure they are alive and well.
Summary
The code for this project is on GitHub at bcantoni/sitecheck-example. The basic idea is to treat this as a Python application from the GitHub Actions perspective. (In fact, that’s how I created the workflow at the start, by choosing Python App as the template.)
The core of the code will run a series of checks against the configured website. I’m using it to confirm the following:
- Redirects from www to non-www (or vice versa) are as expected (including 301 vs 302)
- Redirects from http to https are as expected (including 301 vs 302)
- Website is up with some content text found (to ensure site is alive and being served properly)
- SSL certificates are still valid and have an expiration date at least 3 days in the future
All of the above are configurable, including any kind of additional changes that make sense for your environment. The status of each test run is saved persistently to S3 using my bcantoni/s3data project. The persistent data is used to only report “pass” results once after a previous failure (to keep down the noise).
At the conclusion of the test run, in the case of failures a message is sent via SMS/text messaging and also Slack.
The project README has a lot more detail explaining the code and the structure within GitHub actions/workflows.
Instructions
To use this project for your own sites, you’ll need to make a fork of the bcantoni/sitecheck-example project.
Next, follow these steps:
Locally you’ll want a Python 3.7/3.8 environment (probably easiest with a virtual environment). Install dependencies with:
pip install --upgrade pip
pip install -r requirements.txt
Now you can edit sites.py to put your own sites and/or checks you want to make.
To enable SMS/text messaging, you’ll need an account and number in Twilio. Set up these environment variables (secrets) in GitHub: TWILIO_AUTH_TOKEN, TWILIO_ACCOUNT_SID, TWILIO_FROM_NUMBER and TWILIO_TO_NUMBER. If you don’t want to use Twilio, remove the code for send_sms_messages.
If you’re going to use Slack, set up the SLACK_WEBHOOK secret in GitHub. If you don’t want to use Slack, remove the code for send_slack_messages.
S3 persistent storage between runs will require an AWS account with an S3 bucket configured. Set up these secrets in GitHub: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and S3DATA_BUCKET. If you don’t want to use AWS S3, remove all the code related to s3data.
In fact, if you don’t need any of the above (or want to set it up later), adjust pythonapp.yml to run the code without the --ci option (i.e. just: python sites.py -v). Then you can add the other services later if you want.
Now try running the script locally to make sure everything works:
python sites.py -v
Once it looks good, push to GitHub and check the Actions tab to see how it goes. It will run each time you commit, so you can test it by committing a known bad check, then reverting back to a good state.
If you try this project yourself and have any feedback or ideas, please let me know!
Example Failure (Update)
Just one week after publishing this project, I had a good example of a content failure because the Thunderbird website changed their marketing content slightly. It’s a good reminder to use text which is unlikely to change!
Here’s what the failure looks like on the GitHub Actions tab:
And since I’m integrated with my personal Slack account, you can see what the Slack notifications look like:
After a simple fix was pushed, the test suite is passing once again.