As a WordPress developer, the topic of testing isn’t something that we hear about that often. And, if we do hear about it, it’s more often than not just one specific type of testing: unit testing. (That’s even the one that I’ve thought first!) In fact, it’s not uncommon to associate testing with unit testing because of this.
But software testing is an enormous field. There are a lot of different types of testing. While they might not all be as useful as unit testing, they still all serve a specific purpose.
That said, there are other types of testing that are as useful (or almost as useful!) as unit testing. Acceptance testing is one of them. It’s especially useful if you’re a plugin or theme developer.
Defining acceptance testing
So what is acceptance testing and why is it so useful? Acceptance testing is all about testing whether your code meets the needs of the people who will use your software. (The technical term for them is end user.) For example, if you’re a plugin developer, it tests whether your plugin does what you’ve told people it would do.
This is especially important if someone paid for the code that you worked on. This could be a paid plugin or theme. But it also applies to big projects where you develop a whole WordPress site for a client. Acceptance testing ensures that what you built works for the people that you built it for.
This might seem similar to unit testing. After all, with unit testing, you’re testing whether your code works or not. But the difference with acceptance testing is really who the tests are for.
With acceptance testing, you’re making sure that your code works for the end user. This is quite different from unit testing. Unit testing is only about ensuring that your code works for other developers.
Now, it’s possible for acceptance testing and unit testing to overlap. For example, an acceptance test can find a bug where code that you wrote doesn’t work for other developers. The opposite is true as well. A unit test can find an issue for an end user.
That said, you shouldn’t count on this to happen too often. If anything, this is the result of you having both a strong acceptance and a strong unit test suites. That’s why it’s better to have both if you can!
The challenges of acceptance testing
If you’re familiar with unit testing already, you know that unit testing relies a lot on assertions. An assertion is something like “I expect this method to return a string.” When you’re writing unit tests, you’re often writing a lot of assertions like these.
Like unit tests, acceptance testing also relies on assertions. The big difference is that these assertions aren’t for regular programming things. You’re not testing if a method returns a string. Instead, you’re testing for things like “I expect to log in if I enter the correct user and password.”
The problem with this type of assertion is that they’re not that easy to test. There are quite a few challenges to acceptance testing that you won’t encounter with unit testing. Let’s go over them.
Environment setup
First, acceptance testing requires a much more elaborate testing environment. You can’t test if you can log in to a WordPress site if you don’t have a working one. This means that you at least need a testing environment with a working database and web server.
There’s also the question of JavaScript. Nowadays, most WordPress plugins themes and sites use a lot of JavaScript. This means that the testing environment that you use for acceptance testing also needs to be able to process JavaScript.
Setting up an environment that supports all this is without a doubt one of the biggest challenges with acceptance testing. With unit testing, you add your testing library (most likely PHPUnit), then you write your tests and that’s it. You can also run them on your development machine without much of a hassle.
Acceptance testing requires a much larger upfront investment to get started. And it’s a lot harder for developers to run them on the development machine. This is something to take into account if you want to start doing acceptance testing. That’s also why it’s a better idea to start with unit testing instead of acceptance testing if you’re doing neither.
Putting yourself in the shoes of the end user
Next, let’s go back to our acceptance testing assertion. It said, “I expect to log in if I enter the correct user and password.” This seems like pretty straightforward assertion at a glance.
But it’s actually not that simple in practice. There are a lot of unknowns that our assertion doesn’t really discuss. Here are some examples:
- Where are we trying to log in from?
- How did we get there?
- Where is the login form?
- Do I need to interact with anything for the login form to appear?
These are all things that our end user might need to do to log in to our WordPress site. With acceptance testing, you can’t leave these questions unanswered. You need to specify all the steps that an end user would have to go through to confirm your assertion.
Figuring out all these steps can be a lot of work depending on the complexity of your assertion. That said, it also has its benefits. The most important one is that it forces you to think more about how an end user uses the code that you write.
As developers, this isn’t always easy. We’re not always thinking about the people that will use our code. We’re more preoccupied with the code that we’re writing. (And making sure that it’s amazing!)
Defining how something works for the end user
Last, there’s also the question of how to specify all the steps that an end user needs to do. You need a way to define steps like “I am on the home page.” This will vary depending on the acceptance testing framework you choose to use. (More on that in a second!)
But like it can be hard to think of all the steps that an end user does, it can also be hard to define those steps. Some things like navigating to a specific web page (like a homepage) can be easy. But other things like navigating a menu powered by JavaScript aren’t.
It can be hard to know ahead of time what you’ll find complicated to define and what you won’t. That said, in general, if you just have to interact with HTML elements, it’s pretty easy to do. If it involves JavaScript, you’ll find it more complicated.
This is because JavaScript can create some complex interactions that are hard to define. For example, if you can open or close a menu with JavaScript, you need to define steps for that. You can’t just click on the menu item (even if it’s there in the HTML) because it’s not really visible to the end user.
If you ever worked on web accessibility, this won’t be anything new to you. There’s a correlation between how accessible your WordPress site is and how easy it is to define steps. That’s because a lot of the acceptance testing tools work in a similar way as assistive technologies like screen readers.
So let’s say that you’re writing code that isn’t very accessible. There’s a good chance it won’t be easy (but not impossible!) to define acceptance testing steps for it. And vice versa if you’re writing accessible code.
Picking a framework
Now that you’re all primed and ready to do acceptance testing, what acceptance testing framework should you use? Right now, the two most popular PHP acceptance testing frameworks are Behat and Codeception. Both of them can do more or less the same thing. What really sets Behat and Codeception apart is how you write tests for them.
Behat and Gherkin
Behat tests use a special language called Gherkin. Gherkin comes from Cucumber which is an acceptance testing framework for Ruby. Behat takes a lot of inspiration from Cucumber, but it’s not a PHP port of it. Here’s an example of a Behat test written in Gherkin:
Scenario: I can see the "powered by WordPress" Given I am not logged in When I navigate to the homepage Then I should see "Proudly powered by WordPress"
There’s a lot going on in that example, but we’re not going to go over it in this article. What’s important to notice is that the tests don’t use any code. The Gherkin language is a non-technical and human readable language. This confers it several important benefits.
The most important one is that you can have someone other than a developer write your acceptance tests. This is useful when you have, for example, a designer or a business analyst helping develop new features. They might not have the technical know-how to write acceptance tests using code.
But Gherkin isn’t only useful if you want to have someone non-technical write acceptance tests. It’s also a way for a team to have a ubiquituous language. If you’re not familiar with the idea, a ubiquitous language is a language shared by all members of a team. (Both non-technical and technical.)
A ubiquitous language is a great way to improve team communication. This shared language reduces the potential for confusion and misunderstandings. This, in turn, helps keep everyone on the same page.
On top of that, it also helps acceptance tests written in Gherkin become more like design documents. Since they’re written in this ubiquitous language, anyone can refer to them. For example, if someone isn’t sure how a feature should work, they can just look at the acceptance tests for it.
Codeception and PHP
Unlike Behat, you write Codeception tests in PHP. So there’s no new language like Gherkin to learn. Here’s what the Behat test from the previous section would look like in Codeception (inspired by Steve Henty‘s file format):
<?php
/**
* Test summary: Look the "powered by WordPress" message on the homepage.
*
* Test details:
* - Go to the homepage
* - Look for the "Proudly powered by WordPress" message
*/
$I = new AcceptanceTester( $scenario );
$I->wantTo( 'Test that we can see the "powered by WordPress" message.' );
// Go to the homepage
$I->amOnPage('/');
// Look for the "powered by WordPress" message
$I->see('Proudly powered by WordPress');
Again the goal here isn’t to explain what’s going on with the Codeception test. (This is something for another time.) What we want to look at is the contrast between it and the Behat one.
As we mentioned already, the most obvious change is that you write tests for Codeception in PHP. This is a very appealing thing for developer heavy teams or if you’re a solo developer. Everyone who works with WordPress knows PHP so it’s a lot easier to get up and running with Codeception.
And on the theme of getting started with Codeception, there’s also the question of documentation. At this time, there’s a lot more material on how to use Codeception with WordPress. There are quite a few people that have written articles on it.
That’s not really the case for how to use Behat with WordPress. Information on how to do that is a lot more sparse. So that’s also something to keep in mind as you make your decision.
Once you’ve picked a framework
Now that you’ve picked a framework, what’s next? Well, due to the complexity of setting up an acceptance testing environment, we’re going to leave that for another article. (Boo!) That said, there are few more things worth discussing.
Transitioning to acceptance testing
The first one is how to handle the change to acceptance testing. As we’ve seen with unit testing, it’s one thing to learn about acceptance testing. It’s another to use it every day as you’re writing code.
A lot of this will vary depending on what type of quality assurance you already do. Do you have a dedicated quality assurance person or team? Do you have a spreadsheet or Trello board with all the tests that you want to do with each release? Maybe you even have nothing. (And that’s ok! That’s why this article exists too.)
But if you do already have a quality assurance process, you need to consider that fact. That means investigating how to migrate it to your chosen acceptance testing framework. This might be a pretty time intensive task depending on how big your existing quality assurance process already is.
Using acceptance testing in your development process
Once you’ve migrated your existing quality assurance process (if you had one!), you need to keep the ball rolling. To do that, you need to think about how acceptance testing will fit in your development process. There are a lot of different ways to approach this.
You could choose to have a test-driven development process. There are two types of test-driven development processes with acceptance testing. One is acceptance test-driven development. The other is behaviour-driven development.
The difference between both processes isn’t that important here. It’s the reason for using either process that is. They both strive to make acceptance testing central to your development process.
Often that means that you’ll work on your acceptance tests before writing a single line of code. (Blasphemy!) This has the advantage of allowing other team members (like designers and support staff) to collaborate in the development process. They can help guide the process so that the final result works as expected for everyone.
Depending on the context, this can also extend to clients and people who will use your software. They can also help guide the development process in a constructive way. Your acceptance tests are a gateway for them to do that.
Building trust with the people using your code
The goal of any type of testing is to build trust. It’s the people that you build trust with that changes between each type of testing. Unit testing is about building trust with the other developers that will work with your code. Acceptance testing builds trust with the people that will use your code for what it does.
Both are necessary for different reasons. But, as developers, we tend to focus more on unit testing than acceptance testing. That’s because what we spend our days writing code and working with other people’s code. So we’re more interested in building trust around what we spend our days doing.
Acceptance testing forces you to expand your horizons. You have to think about people that aren’t necessarily other developers. But these people are important none the less.
They’re the people using your finished product. Most of us that make a living from WordPress. So it’s probable that it’s more important to build trust with them than other developers. (They’re paying the bills after all!)
This is why you should look into acceptance testing if you have or work on a paid WordPress plugin or theme. Your customer’s trust is hard to gain and easy to lose. Acceptance testing helps ensure that the latter happens as little as possible.
Photo Credit: José Alejandro Cuffia