Brian Cantoni

How to Add Playwright Automated Tests to an Eleventy Static Blog

Now that this blog has moved to the Eleventy system, I wanted to add some simple smoke tests to make sure the build process is working correctly and catch me if I break anything. I decided to go with Playwright because we use it at work and it's been on my to-learn list. Everything worked well, and I achieved good test coverage with 10 different tests, each containing multiple assertions. The trickiest part was making it all run correctly in the GitHub Actions pipelines.

Learning Playwright

The Playwright docs are very good so I started there. Getting set up, writing some initial tests, learning about assertions and locators were my starting points.

Once I had the basic structure running, I created an outline of the tests I wanted to create. These are all pretty simple for this static site, with a focus on making sure everything was built correctly.

For each test I dropped the outline into a code comment like this:

/*
test home page
- go to home page
- confirm <title> is set to "Brian Cantoni"
- confirm <h1> heading is set to "Brian Cantoni"
- confirm exactly 10 blog posts are listed
*/

Then I would select the comment and ask Copilot to implement it. It did very well at this, particularly when using Agent mode and asking it to implement the test and then run it, iterating until it worked.

From the above outline, Copilot and I created the Playwright test:

test("Home page", async ({ page }) => {
  await expect(page).toHaveTitle(/Brian Cantoni/);
  await expect(
    page.getByRole("heading", { name: "Brian Cantoni" }),
  ).toBeVisible();
  await expect(page.locator("li.postlist-item")).toHaveCount(10);
});

This approach worked well unless I tried creating and selecting multiple comment blocks like this. Copilot got confused and started adding code in random locations in the JavaScript file and then reported the file as "corrupted" :) Another good reminder for having frequent git commits of your work in progress.

Playwright with Netlify Preview Builds

Beyond running tests locally, I wanted them to run against the Netlify preview builds as well. I have Netlify deploy previews configured against my private GitHub repo and every pull request creates a unique "preview" build that goes with it. I'm not using Netlify for the production build, but these preview builds are very helpful especially for checking any tricky changes.

To run tests against the preview build, the GitHub action workflow needs to wait for the preview deploy to finish before running the tests. Luckily there are a few published actions that do this, including JakePartusch/wait-for-netlify-action.

Incorporating this as an action step is quite simple:

- name: Wait for Netlify Deploy Preview
  id: waitForDeployPreview
  uses: jakepartusch/wait-for-netlify-action@v1.4

And then a later step uses the unique preview URL to run the tests:

- name: Run Playwright tests
  run: npx playwright test
  env:
    PLAYWRIGHT_BASE_URL: $

In the end it's a clean solution which works well, but it took so many incremental pushes to test the actions, it was frustrating. It didn't help that I trusted some Google AI search results which gave me a .yml workflow file that looked great but was far from a working example.

One thing I learned that really helped: Copilot very good at reviewing a workflow file and either pointing out improvements or actual problems (some of which were just dumb mistakes on my part).

Playwright with GitHub Build

Once I got the Netlify preview tests running, I realized I should also handle the simpler case of just building and testing on GitHub.

Doing this doesn't require any custom actions. Instead, it's just a sequence of starting the local server, waiting for that server to finish, and then running the tests:

- name: Start local server for Playwright
  run: npx http-server _site -p 8080 &

- name: Wait for server to start
  run: |
    timeout 30 bash -c 'until curl -f http://localhost:8080; do sleep 1; done'
    echo "Server is ready"

- name: Run Playwright tests
  run: npx playwright test
  env:
    PLAYWRIGHT_BASE_URL: http://localhost:8080

Next

Right now, my smoke test coverage is pretty good and should catch any build issues. Also, now that the tests are part of the automatic build pipeline, it will be easy to add incremental tests for any issues that pop up.

Added: Just after publishing this blog post, I realized I had a problem with my .htaccess files (for Apache server redirects). With my new test framework in place, it was easy to add a set of checks to ensure that file is properly deployed.

The Playwright test report is uploaded as a zip file, but I want to adjust that to be viewable online.

This test setup is simple because the static blog is simple. (Really the only interactive part is the search box.) I want to learn and use Playwright for some actual applications too. This would have been great for end-to-end testing of my retired Tweetfave project.