# Continuous deployment to the WordPress directory with CircleCI *Note: This article focuses on how to continuously deploy a plugin using CircleCI. But you can also use everything discussed here with a theme as well.*As developers, we all have our preferred tools and distinct way of working. That's why it's not uncommon for us to [write about it](https://carlalexander.ca/2016-macbook-pro-setup/) or create scripts to set up our work computers. But the one tool that most of us tend to agree on is using [git](https://en.wikipedia.org/wiki/Git) for [version control](https://en.wikipedia.org/wiki/Version_control). That said, if you've ever had a plugin or theme on the WordPress directory, you know that it doesn't use git. It uses [subversion](https://en.wikipedia.org/wiki/Apache_Subversion). That's a problem for a lot of us because we don't want to have to deal with both. While there are a lot of resources out there to deal with this problem, it also presents us with an opportunity. We can use this problem to build a [continuous deployment](https://en.wikipedia.org/wiki/Continuous_delivery#Relationship_to_continuous_deployment) workflow for the WordPress directory. This will allow us to not worry about this aspect of WordPress plugin development anymore. # Some background But before we begin looking at all the details of how to do this, let's discuss why I'm writing this article. The initial motivation is exactly what I described in the introduction. Like a lot of WordPress developers, I've never built a plugin that I've published to the [plugin directory](https://wordpress.org/plugins/). But I'd been working on a [small security plugin](https://wordpress.org/plugins/passwords-evolved/) that I wanted to put in the directory. I wasn't super excited at the idea of having to maintain my plugin in two locations. Like most developers, I work with GitHub a lot, and I had a [public repository](https://wordpress.org/plugins/passwords-evolved/) there for the plugin. I didn't want to have to keep track of things in the [plugin's subversion repository](plugins.svn.wordpress.org/passwords-evolved/) as well. At the same time, I'd been working a lot with [CircleCI](https://circleci.com/). I'd used it to build a [continuous integration workflow](https://carlalexander.ca/continuous-integration-wordpress/) on a WordPress project. I'd also used it to manage the continuous deployment of that project to [Pantheon](http://pantheon.io/). I'd also used [Travis CI](https://travis-ci.com/) before. But I think that, at this point, CircleCI is a bit better than Travis CI. The only exception is if you need Travis's [build matrix](https://docs.travis-ci.com/user/customizing-the-build/#Build-Matrix) feature. As we'll see, we can do something a bit similar, but it's definitely not as powerful. So that's why we're going to use CircleCI to build our continuous deployment workflow. That said, if you use Travis CI, there's already a [good article](https://deliciousbrains.com/deploying-wordpress-plugins-travis/) on how to do this with it. In fact, some of the scripts that you'll see today came initially from that article. They were then modified to fit the continuous delivery workflow with CircleCI better. # A note on JavaScript Before we begin, it would be important to discuss JavaScript. It's uncommon for a plugin not to use any JavaScript nowadays. That said, this plugin is one of those rare plugins where we don't use any JavaScript. This means that what we'll see in this article doesn't cover anything related to JavaScript. There's no JavaScript unit testing to do. The setup used to run acceptance tests uses a [headless browser](https://en.wikipedia.org/wiki/Headless_browser) driver. This driver doesn't support the execution of JavaScript. This means that, if you rely a lot on JavaScript and you want to support it, you'll find this setup incomplete. You'll need to add jobs for unit testing JavaScript. You'll also need to modify the acceptance test jobs to support JavaScript as well. That said, you'll still find a lot of useful information in this article anyways. We explain a lot of how the CircleCI platform works. This will help you understand the changes that you need to support JavaScript in your continuous deployment workflow. # Overview of the workflow So speaking of continuous deployment workflow, let's look at what it'll look like in its final form. We'll then go over these jobs or group of jobs and explain what they're doing and why. CircleCI has a screen where you can see all the jobs in a workflow and how they're connected. You can see it below: ![](https://d3nw5tt5i784nr.cloudfront.net/uploads/2018/06/circleci-workflow.png)Complete CircleCI workflowThis is a picture of the CircleCI workflow used by my plugin. It has 12 jobs. But we can break down these jobs into more specific categories. These are: - Build - Code quality checks - Unit tests - Acceptance tests - Deployment These categories are also in the order that CircleCI will go through them. The only exception is the asset deployment job which runs on its own. That's because the WordPress directory manages [plugin assets](https://developer.wordpress.org/plugins/wordpress-org/plugin-assets/) differently. So we don't deploy them at the same time as the rest of the plugin files. (We'll mention this a few times throughout the article.) # CircleCI config file With CircleCI, you configure this entire workflow and its jobs using a [YAML](https://en.wikipedia.org/wiki/YAML) configuration file. (You can find the documentation for it [here](https://circleci.com/docs/2.0/configuration-reference/).) This configuration file is always under `.circleci/config.yml` path in your repository. Here's what the one that I used to create the workflow that you saw in the picture above: ``` version: 2.0 references: # Environment variables # WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander # Default container configuration # container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp attach_workspace: &attach_workspace attach_workspace: at: *workspace_root copy_vendor: ©_vendor run: name: Copy vendor directory command: cp -R /tmp/vendor . install_subversion: &install_subversion run: name: Install subversion command: sudo apt-get install subversion # Default configuration for all behat testing jobs # behat_job: &behat_job <<: *container_config docker: - image: circleci/php:7.2 - image: circleci/mysql:5.7 steps: - checkout - run: name: Add WordPress host to hosts file command: echo "127.0.0.1 ${WP_HOST}" | sudo tee -a /etc/hosts - *attach_workspace - *copy_vendor - run: name: Install MySQL client command: sudo apt-get install mysql-client - run: name: Install MySQL PHP extension command: sudo docker-php-ext-install mysqli - run: name: Setup WordPress command: .circleci/setup-$WP_TYPE.sh - run: name: Start PHP server command: sudo php -S $WP_HOST:80 -t $WP_CORE_DIR background: True - run: name: Run Behat tests command: vendor/bin/behat --config .circleci/behat.yml --format progress --tags=$WP_TYPE # Default configuration for all phpunit testing jobs # phpunit_job: &phpunit_job <<: *container_config steps: - checkout - *attach_workspace - *copy_vendor - run: name: Run PHP unit tests command: vendor/bin/phpunit jobs: build: <<: *container_config steps: - checkout - run: name: Install dependencies command: composer install -n -o - run: name: Install WordPress command: bash .circleci/install-wp.sh - persist_to_workspace: root: . paths: - vendor - persist_to_workspace: root: *workspace_root paths: - wordpress code_quality: <<: *container_config steps: - checkout - *attach_workspace - *copy_vendor - run: name: Run code quality tests command: vendor/bin/grumphp run --testsuite=code_quality test_php72: <<: *phpunit_job docker: - image: circleci/php:7.2 test_php71: <<: *phpunit_job docker: - image: circleci/php:7.1 test_php70: <<: *phpunit_job docker: - image: circleci/php:7.0 test_php56: <<: *phpunit_job docker: - image: circleci/php:5.6 test_php55: <<: *phpunit_job docker: - image: vandries/phpenv:5.5 test_php54: <<: *phpunit_job docker: - image: vandries/phpenv:5.4 acceptance_singlesite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: singlesite acceptance_multisite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: multisite deploy_assets: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy assets to WordPress plugin directory command: .circleci/deploy-assets.sh deploy_plugin: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy new version to WordPress plugin directory command: .circleci/deploy-plugin.sh workflows: version: 2 build_test_deploy: jobs: - build - deploy_assets: filters: branches: only: - master - code_quality: requires: - build - test_php72: requires: - code_quality - test_php71: requires: - code_quality - test_php70: requires: - code_quality - test_php56: requires: - code_quality - test_php55: requires: - code_quality - test_php54: requires: - code_quality - acceptance_singlesite: requires: - test_php72 - test_php71 - test_php70 - test_php56 - test_php55 - test_php54 - acceptance_multisite: requires: - test_php72 - test_php71 - test_php70 - test_php56 - test_php55 - test_php54 - deploy_plugin: filters: branches: only: - master requires: - acceptance_singlesite - acceptance_multisite ``` The primary goal of this article is to explain this entire configuration file. That said, there's a section of the config file above that's not part of a standard CircleCI YAML config file. It's the `references` section at the top. ## YAML references The `references` section is there for us to leverage a lesser known feature of YAML: anchor and references. Anchors (noted by the `&` symbol) let us define nodes that we want to reuse in our YAML file. We then use the `*` symbol to reference these anchors. You can also extend a YAML node using the `<<` notation. This lets us inject the content inside our reference inside another YAML node. You can then add more elements to that YAML node afterwards. In that scenario, our reference is more of a YAML node template that we can reuse instead of content that we're copying in. So what is the link between this YAML feature and our `references` section? It's that we use it to organize all the anchors that we want to use throughout our config file. We have to do this because there's no official way to handle anchors in YAML or with CircleCI. This, at least, keeps things tidy and gives us an easy to maintain solution to that problem. ## Default container configuration At the top of our `references` section, there's a reference called `container_config`. This is one of the most important parts of the entire CircleCI configuration file. This is the default configuration for all the containers that CircleCI will use. Without going into the full details of the CircleCI's architecture, you should know that everything works off [Docker](https://en.wikipedia.org/wiki/Docker_(software)) containers. CircleCI spins up a new Docker container each time that it runs a job. The `container_config` reference is the base template that for those Docker containers that CircleCI spins up. ``` # .circleci/config.yml references: # Environment variables # WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander # Default container configuration # container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved ``` So here's a look at our `container_config` reference. First, we created four other references. Those references are the [environment variables](https://en.wikipedia.org/wiki/Environment_variable) that we need our containers to have. We have to define them first so that we can use them in `container_config` reference. Before moving on, here's a quick overview of each of them: - `WP_CORE_DIR` is the directory where our scripts can find our WordPress installation. - `WP_HOST` is the [hostname](https://en.wikipedia.org/wiki/Hostname) of the WordPress site that we'll use for testing. - `WP_ORG_PLUGIN_NAME` is the name of our plugin in the WordPress plugin directiory. - `WP_ORG_USERNAME` is your [WordPress.org](https://wordpress.org/) username. ### container\_config sections The `container_config` reference itself has three section. First, there's the `docker` section. This is the section where you define the Docker images that the CircleCI container will use. By default, we use the `circleci/php:7.2` image only. `circleci/php` is the Docker image that CircleCI built for PHP projects. CircleCI has a lot of [pre-built Docker images](https://circleci.com/docs/2.0/circleci-images/) that you can use for your jobs. Meanwhile, the `:7.2` specifies the image tag that we want to use. In this scenario, it means that we want the `circleci/php` to use PHP 7.2. So, by default, all our containers will use PHP 7.2 unless specified otherwise. The last two sections are more straightforward than the `docker` section. The `environment` section is where we configure the container's environment variables. We just list references to the four environment variables that went over in the last section. # Configuring our CircleCI jobs The first of two sections of a CircleCI configuration file is the `jobs` section. This is where we define all the jobs in our continuous integration workflow. Here's another look at all the jobs in our workflow: ``` # .circleci/config.yml jobs: build: <<: *container_config steps: - checkout - run: name: Install dependencies command: composer install -n -o - run: name: Install WordPress command: bash .circleci/install-wp.sh - persist_to_workspace: root: . paths: - vendor - persist_to_workspace: root: *workspace_root paths: - wordpress code_quality: <<: *container_config steps: - checkout - *attach_workspace - *copy_vendor - run: name: Run code quality tests command: vendor/bin/grumphp run --testsuite=code_quality test_php72: <<: *phpunit_job docker: - image: circleci/php:7.2 test_php71: <<: *phpunit_job docker: - image: circleci/php:7.1 test_php70: <<: *phpunit_job docker: - image: circleci/php:7.0 test_php56: <<: *phpunit_job docker: - image: circleci/php:5.6 test_php55: <<: *phpunit_job docker: - image: vandries/phpenv:5.5 test_php54: <<: *phpunit_job docker: - image: vandries/phpenv:5.4 acceptance_singlesite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: singlesite acceptance_multisite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: multisite deploy_assets: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy assets to WordPress plugin directory command: .circleci/deploy-assets.sh deploy_plugin: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy new version to WordPress plugin directory command: .circleci/deploy-plugin.sh ``` Each key in the `jobs` node is the name of the job. It needs to be unique because we need to be able to refer to it when we build our workflow later. The content of the node contains details like the `steps` that you want CircleCI to perform. We're going through all these jobs and groups of jobs. We'll explain what the `steps` in them are doing. We'll also talk about other unique aspects of their configuration. ## Build job The first job that we have in our continuous deployment workflow is the `build` job. Its purpose is to check out a copy of our plugin and install its dependencies. We also use the build process to install the copy of WordPress that we'll use throughout our workflow. ``` # .circleci/config.yml references: WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp jobs: build: <<: *container_config steps: - checkout - run: name: Install dependencies command: composer install -n -o - run: name: Install WordPress command: bash .circleci/install-wp.sh - persist_to_workspace: root: . paths: - vendor - persist_to_workspace: root: *workspace_root paths: - wordpress ``` Above is the parts of the `config.yml` file that is relevant to the `build` job. The `build` job starts by extending the `container_config` node. This is the default configuration for all the Docker containers that we just saw. ### Build steps The rest of the `build` job is the `steps` section. We first check out our plugin, then run `composer install` and then the `install-wp.sh` bash script. You can see a copy of the bash script below: ``` #!/usr/bin/env bash WP_CORE_DIR=${WP_CORE_DIR-/tmp/wordpress/} WP_VERSION=${WP_VERSION-latest} download() { if [ `which curl` ]; then curl -s "$1" > "$2"; elif [ `which wget` ]; then wget -nv -O "$2" "$1" fi } set -ex install_wp() { if [ -d $WP_CORE_DIR ]; then return; fi mkdir -p $WP_CORE_DIR if [[ $WP_VERSION == 'nightly' || $WP_VERSION == 'trunk' ]]; then mkdir -p /tmp/wordpress-nightly download https://wordpress.org/nightly-builds/wordpress-latest.zip /tmp/wordpress-nightly/wordpress-nightly.zip unzip -q /tmp/wordpress-nightly/wordpress-nightly.zip -d /tmp/wordpress-nightly/ mv /tmp/wordpress-nightly/wordpress/* $WP_CORE_DIR else if [ $WP_VERSION == 'latest' ]; then local ARCHIVE_NAME='latest' else local ARCHIVE_NAME="wordpress-$WP_VERSION" fi download https://wordpress.org/${ARCHIVE_NAME}.tar.gz /tmp/wordpress.tar.gz tar --strip-components=1 -zxmf /tmp/wordpress.tar.gz -C $WP_CORE_DIR fi download https://raw.github.com/markoheijnen/wp-mysqli/master/db.php $WP_CORE_DIR/wp-content/db.php } install_wp ``` The script is just a partial copy of the script generated by the WP-CLI [`scaffold`](https://developer.wordpress.org/cli/commands/scaffold/) command. This is the part that downloads the version of WordPress specified by the `WP_VERSION` variable. It then installs that version of WordPress in the directory defined by the `WP_CORE_DIR` variable. That said, we didn't define either of those variables. This means that the `install-wp.sh` bash script will use the default values defined inside of it. It'll install the `latest` version of WordPress in the `/tmp/wordpress` directory. ### Saving everything to a workspace Once we installed our dependencies and WordPress, we need to save them so that we can reuse them throughout the workflow. CircleCI has two mechanisms to do that: [caching](https://circleci.com/docs/2.0/caching/) and [workspaces](https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs). If you [read about the difference between them](https://circleci.com/blog/persisting-data-in-workflows-when-to-use-caching-artifacts-and-workspaces/), the correct one to use would be caching. But, as you can see from the `build` job configuration above, we're not. We're using workspaces instead. Why is that? Well, it comes down to how CircleCI implemented its cache feature. With CircleCI, when you save something to a cache, you have to use a key to identify it. The problem is that CircleCI doesn't let you overwrite an existing cache once it's created. This means that you need a good way to generate keys to use the CircleCI cache feature. If you don't, CircleCI will keep fetching the same data over and over forever. That's why we can't use the CircleCI cache feature. Our current setup doesn't have a dependable way to generate cache keys for either composer or WordPress. We could for composer if we used a `composer.lock` file, but we don't here. So, instead of the CircleCI cache feature, we'll use the workspace feature to cache our composer and WordPress files. Those are what the two `persist_to_workspace` steps do. The first one saves the composer `vendor` directory with all our dependencies. The second saves the copy of WordPress that the `install-wp.sh` bash script downloaded earlier. ## Code quality job The second job in our continuous deployment workflow is the `code_quality` job. We use it to run all our code quality checks on our plugin code. We want to do this before we start running the tests on our plugin code. ``` # .circleci/config.yml references: WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp attach_workspace: &attach_workspace attach_workspace: at: *workspace_root copy_vendor: ©_vendor run: name: Copy vendor directory command: cp -R /tmp/vendor . jobs: code_quality: <<: *container_config steps: - checkout - *attach_workspace - *copy_vendor - run: name: Run code quality tests command: vendor/bin/grumphp run --testsuite=code_quality ``` This is the sections of the `config.yml` file used for the `code_quality` job. Like the `build` job that we saw before, the `code_quality` job also extends the `container_config` node. The main difference between the two jobs is the `steps` section. ### Code quality steps This job also starts by checking out our plugin code. In fact, all our jobs will start with the `checkout` step so it won't be the last time that we mention it. But the rest of the steps are different from what we saw earlier. Next, we have the `attach_workspace` step which is a reference from our `references` section. The `attach_workspace` calls the special [`attach_workspace`](https://circleci.com/docs/2.0/configuration-reference/#attach_workspace) step from CircleCI. (Yes, this is a bit confusing!) We then tell the `attach_workspace` step to attach our workspace to the `workspace_root` directory. Once CircleCi attached our workspace, we want to copy back the `vendor` directory that we copied to it earlier. That's what the `copy_vendor` step does. This is another reference from our `references` section. The `copy_vendor` reference uses the [`run`](https://circleci.com/docs/2.0/configuration-reference/#run) step. This is the most common step used in CircleCI job configurations. You use to run a specific command in the container. For our `copy_vendor` step, we just want to run [`cp`](https://en.wikipedia.org/wiki/Cp_(Unix)) command. It copies our vendor directory back to the working directory defined in `container_config`. This allows us not to run `composer install` again which is a time-consuming command. ### Checking code quality with GrumPHP The last step in our `code_quality` job is the one doing the code quality checks. For this, we leverage a tool called [GrumPHP](https://github.com/phpro/grumphp). It's a tool for enforcing code quality whenever you commit code. Now, GrumPHP isn't a tool built with continuous deployment in mind. Its primary use is to enforce code quality on developer's local development environments. That said, you can still use it with continuous deployment. ``` # grumphp.yml parameters: process_timeout: 120 tasks: composer: ~ gherkin: ~ git_commit_message: ~ phpcpd: exclude: - lib - tests - vendor phpcs: whitelist_patterns: - /^src phplint: ~ phpmd: ruleset: ['codesize', 'design', 'naming', 'unusedcode'] exclude: - lib - tests - vendor phpunit: always_execute: true testsuites: code_quality: tasks: - composer - gherkin - phpcpd - phpcs - phplint - phpmd ``` Above is the GrumPHP configuration used by the plugin. It has a quite a few code quality checks enabled. We're not going to go over them in this article, but you can read more about them [here](https://github.com/phpro/grumphp/blob/master/doc/tasks.md). What we're more interested in is the [`testsuites`](https://github.com/phpro/grumphp/blob/master/doc/testsuites.md) section of the configuration file. This section lets us create custom test suites that only run specific code quality checks. The `code_quality` test suite is the test suite that we've created for our CircleCI workflow. We did this because the default behaviour of GrumpmPHP is to run all the code quality checks. We don't want that here since we need to run our unit tests multiple times to test different PHP versions. (More on that in a second!) That's why the `code_quality` test suite contains every code quality check except `phpunit`. So that's why we need this `code_quality` test suite for CircleCI to perform our code quality checks. We use a `run` step to do it. The step runs with GrumPHP with an argument specifying the `code_quality` test suite that we want to run. ## PHPUnit jobs Once we're done with the `code_quality` job, we move on to jobs running our unit tests. This section is a bit different from what we've seen so far. We're going to talk about a group of jobs instead of a specific job. The reason for that is that we need to run our unit tests on different versions of PHP. (For our plugin, we need to run them on PHP 5.4 and newer.) But, with CircleCI, you can only run your unit tests on one version of PHP at a time. So that means that we need a job for each version of PHP that we want to run our unit tests against. ``` # .circleci/config.yml references: WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp attach_workspace: &attach_workspace attach_workspace: at: *workspace_root copy_vendor: ©_vendor run: name: Copy vendor directory command: cp -R /tmp/vendor . phpunit_job: &phpunit_job <<: *container_config steps: - checkout - *attach_workspace - *copy_vendor - run: name: Run PHP unit tests command: vendor/bin/phpunit jobs: test_php72: <<: *phpunit_job docker: - image: circleci/php:7.2 test_php71: <<: *phpunit_job docker: - image: circleci/php:7.1 test_php70: <<: *phpunit_job docker: - image: circleci/php:7.0 test_php56: <<: *phpunit_job docker: - image: circleci/php:5.6 test_php55: <<: *phpunit_job docker: - image: vandries/phpenv:5.5 test_php54: <<: *phpunit_job docker: - image: vandries/phpenv:5.4 ``` This is the sections of the `config.yml` file with all our unit testing jobs. As you can see, it's a lot bigger than the sections that we saw before. That said, it's not a lot more complicated than the jobs that we've seen so far. ### phpunit\_job reference The essence of our unit testing jobs is the `phpunit_job` reference. This is the reference that contains all the steps that our unit test jobs use. Like all the other jobs that we've seen so far, it also extends `container_config`. The steps in `phpunit_job` job reference are almost the same as the ones we saw with the `code_quality` job. We start by checking out the plugin code. We then attach our workspace and copy the `vendor` directory from it. The last step is where things differ from the `code_quality` job. We have a `run` step, but it doesn't make a call to GrumPHP. Instead, we make a direct call to PHPUnit so that it can run the unit test suite for our plugin. ### Testing different PHP versions So why did we need our `phpunit_job` job reference? It's because we need to run our unit test suite with different versions of PHP. CircleCI can't do this [using a build matrix like Travis CI does](https://docs.travis-ci.com/user/customizing-the-build/#Build-Matrix). Because of this limitation with CircleCI, we have to create individual jobs for every PHP version that we want to test. This isn't that problematic since, as you can see, we can leverage anchors and references to do it. Each unit test job then just has to say what PHP version we want to test. That's what we do in the `docker` section of each unit test job. We use it to specify the Docker `image` that we want to use. By default, we're using `circleci/php:7.2` which we defined in the `container_config` reference that we saw that earlier. Each unit test job will use a different `image` with a different PHP version. ### Supporting older PHP versions Now, an issue that every WordPress developer has to contend with is having to support old (if not ancient!) PHP versions. This is going to be a problem because almost all continuous integration services don't support them. The only exception is [Travis CI](https://travis-ci.com/). (It's another reason why it's popular with WordPress developers.) There's a reason why all the continuous integration services don't support old PHP versions. Like CircleCI, they're all built using containers and Docker. But most of them use [Docker's official PHP image](https://hub.docker.com/_/php/). And this image doesn't support any PHP versions older than 5.6. So, to work around this issue, we have to use a different Docker image than the one from CircleCI for older PHP versions. The one that I ended up using is [`vandries/phpenv`](https://hub.docker.com/r/vandries/phpenv/). If you'd rather use another Docker image, feel free to! ## Behat jobs So what happens when our plugin's unit tests pass for all the PHP versions that we want to test? Well, at that point, we want to run our plugin's [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing). Acceptance tests are the tests that we do to ensure that our plugin works with WordPress. In PHP, the most popular framework for doing acceptance testing is [Behat](http://behat.org/). Like our PHPUnit jobs, we're also going to need run our acceptance tests multiple times. That said, we're not trying to test different PHP versions like we did with the PHPUnit jobs. Instead, we want to run our acceptance tests on different types of WordPress installations. Specifically, we want to run them on a regular single site installation of WordPress and on a [multisite](https://codex.wordpress.org/Glossary#Multisite) one. But if you're a plugin shop, you might also want to test against specific older WordPress versions as well. ``` # .circleci/config.yml references: WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp attach_workspace: &attach_workspace attach_workspace: at: *workspace_root copy_vendor: ©_vendor run: name: Copy vendor directory command: cp -R /tmp/vendor . behat_job: &behat_job <<: *container_config docker: - image: circleci/php:7.2 - image: circleci/mysql:5.7 steps: - checkout - run: name: Add WordPress host to hosts file command: echo "127.0.0.1 ${WP_HOST}" | sudo tee -a /etc/hosts - *attach_workspace - *copy_vendor - run: name: Install MySQL client command: sudo apt-get install mysql-client - run: name: Install MySQL PHP extension command: sudo docker-php-ext-install mysqli - run: name: Setup WordPress command: .circleci/setup-$WP_TYPE.sh - run: name: Start PHP server command: sudo php -S $WP_HOST:80 -t $WP_CORE_DIR background: True - run: name: Run Behat tests command: vendor/bin/behat --config .circleci/behat.yml --format progress --tags=$WP_TYPE jobs: acceptance_singlesite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: singlesite acceptance_multisite: <<: *behat_job environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME - WP_TYPE: multisite ``` So here are the relevant sections of the `config.yml` file for our acceptance tests. There's even more going on now than there was in our unit test jobs. That's because we need an actual WordPress site with our plugin installed to do acceptance testing. ### Adding a MySQL database to our container Like our `phpunit_job` reference, the `behat_job` reference is quite big. It contains the `steps` section that our two acceptance test jobs use. We also overwrite the `docker` section of our `container_config` reference. What did we change in the `docker` section? We added the `circleci/mysql` image to it. This will allow our container to have both access to PHP and a MySQL database. ### Behat job steps The `steps` section of our `behat_job` reference is quite different from the other `steps` section that we've seen so far. There's a lot of new steps that we haven't seen so far. We're going to go over all these new steps and why they're there. #### Adding WordPress host to hosts file The first new step is right after the `checkout` step to check out our code. We add our `WP_HOST` environment variable to the [hosts file](https://en.wikipedia.org/wiki/Hosts_(file)) our container. If you're not familiar with what a hosts file is, it's a special operating system file. You use it to map hostnames to [IP addresses](https://en.wikipedia.org/wiki/IP_address). Why do we need to do that? It's because the host defined `WP_HOST` isn't real. There's no [DNS](https://en.wikipedia.org/wiki/Domain_Name_System) record for it anywhere. In fact, [`.test`](https://en.wikipedia.org/wiki/.test) isn't even allowed as a [top-level domain](https://en.wikipedia.org/wiki/Top-level_domain). So that's why we need to add the `WP_HOST` value to the operating system's hosts file. We need the operating system to think that this is a valid address. Otherwise, our Behat tests won't work. #### Installing MySQL dependencies The next new steps are after our `attach_workspace` and `copy_vendor` reference steps. It's related to the `circleci/mysql` image that we added to our Docker container configuration. But doing that doesn't actually do much in practice. The image adds a MySQL database to our Docker container. However, that doesn't mean that it has the tools to use it. So that's what we have to take care of before we can continue. We're going need to separate steps to add support for MySQL to our container. The first one is `install_mysql_client` which is a reference to a `run` step. This step uses [`apt`](https://en.wikipedia.org/wiki/APT_(Debian)) command to install the `mysql-client` dependency. `mysql-client` is the MySQL client library used by the Linux operating system. We need it because it's a requirement for the [`mysqli`](https://secure.php.net/manual/en/book.mysqli.php) PHP extension. This is the second step after `copy_vendor` called `install_mysql_extension`. Meanwhile, we need the `mysqli` PHP extension for WP-CLI. Without the `mysqli` PHP extension, WP-CLI can't set up WordPress for us. And that's what we need to do next. Installing PHP extensions is a bit different with Docker. You need to use the `docker-php-ext-install` helper script instead of following the regular installation instructions. You can read more about `docker-php-ext-install` and other helper scripts [here](https://docs.docker.com/samples/library/php/#how-to-install-more-php-extensions). #### Setting up WordPress After installing our MySQL dependencies is where things start to diverge. At this point, we need to setup WordPress. That said, we need to do different things to setup WordPress as a single site installation vs a multisite one. So how do we know if we want to install WordPress as a single site installation or as a multisite one? Well, we're going to create an environment variable to tell us which type of installation that we want. We'll call it `WP_TYPE`. Our WordPress setup step uses the `WP_TYPE` environment variable to choose which install bash script to run. There's one script for the single site installation and one for the multisite installation. Let's take a look at them. ##### Single site installation Let's start with setting up WordPress as a single site installation. We do this using the `setup-singlesite.sh` bash script. You can see it below: ``` #!/usr/bin/env bash if [[ -z "$CIRCLECI" ]]; then echo "This script can only be run by CircleCI. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_CORE_DIR" ]]; then echo "WordPress core directory isn't set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_HOST" ]]; then echo "WordPress host isn't set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then echo "WordPress.org plugin name not set. Aborting." 1>&2 exit 1 fi # Install WordPress vendor/bin/wp config create --path="${WP_CORE_DIR}" --dbhost="127.0.0.1" --dbname="circle_test" --dbuser="root" vendor/bin/wp core install --path="${WP_CORE_DIR}" --url="http://${WP_HOST}" --title="Passwords Evolved Test" --admin_user="admin" --admin_password="password" --admin_email="admin@example.com" vendor/bin/wp rewrite structure --path="${WP_CORE_DIR}" '/%postname%/' # Copy our plugin to WordPress directory cp -r ./ ${WP_CORE_DIR}/wp-content/plugins/${WP_ORG_PLUGIN_NAME} # Activate our plugin vendor/bin/wp plugin activate --path="${WP_CORE_DIR}" ${WP_ORG_PLUGIN_NAME} ``` The `setup-singlesite.sh` bash script starts with a set of [guard clauses](https://carlalexander.ca/mastering-php-conditionals/). These are there to abort the script if certain requirements are missing. The first requirement that we have is to ensure that we're on the CircleCI environment. We do that by checking for the `CIRCLECI` environment variable is there. Then we check for the three environment variables that our script needs. Those are `WP_CORE_DIR`, `WP_HOST` and `WP_ORG_PLUGIN_NAME`. We can break down the rest of the script into three sections. First, we want to actually install WordPress. Although the name is a bit misleading, our `install-wp.sh` didn't do that. That's because, as we just discussed, our CircleCI container doesn't use the MySQL image by default. But now we can, and we're going to use WP-CLI to do it. Installing WordPress comes down to three commands. In order, they are: 1. [`wp config create`](https://developer.wordpress.org/cli/commands/config/create/) which creates the `wp-config.php` file for our WordPress installation. 2. [`wp core install`](https://developer.wordpress.org/cli/commands/core/install/) which does the actual installation of WordPress 3. [`wp rewrite structure`](https://developer.wordpress.org/cli/commands/rewrite/structure/) which we use to set a more standard [permalink structure](https://codex.wordpress.org/Using_Permalinks#Choosing_your_permalink_structure). (No one uses `?p=` anymore!) After that, we have to copy the plugin from the working directory to the WordPress `plugins` directory. We just use the standard `cp` to copy it. Once that's done, we activate it using the [`wp plugin activate`](https://developer.wordpress.org/cli/commands/plugin/activate/) command. And we're done! ##### Multisite installation Next, we have the `setup-multisite.sh` bash script. This is the bash script used to set up a WordPress multisite installation. You can find it below: ``` #!/usr/bin/env bash if [[ -z "$CIRCLECI" ]]; then echo "This script can only be run by CircleCI. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_CORE_DIR" ]]; then echo "WordPress core directory isn't set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_HOST" ]]; then echo "WordPress host isn't set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then echo "WordPress.org plugin name not set. Aborting." 1>&2 exit 1 fi # Install WordPress vendor/bin/wp config create --path="${WP_CORE_DIR}" --dbhost="127.0.0.1" --dbname="circle_test" --dbuser="root" vendor/bin/wp core multisite-install --path="${WP_CORE_DIR}" --url="http://${WP_HOST}" --title="Passwords Evolved Test" --admin_user="admin" --admin_password="password" --admin_email="admin@example.com" vendor/bin/wp rewrite structure --path="${WP_CORE_DIR}" '/%postname%/' # Copy our plugin to WordPress directory cp -r ./ ${WP_CORE_DIR}/wp-content/plugins/${WP_ORG_PLUGIN_NAME} # Activate our plugin vendor/bin/wp plugin activate --network --path="${WP_CORE_DIR}" ${WP_ORG_PLUGIN_NAME} ``` We won't dive into this script in as much detail as we did with the single site installation one. That's because it's almost the same as the other one. There are only two small changes that we made to it. First, we don't use the `wp core install` command to perform the WordPress installation. Instead, we use the [`wp core multisite-install`](https://developer.wordpress.org/cli/commands/core/multisite-install/). This is a command dedicated to the installing WordPress as a multisite installation. The second change is to the `wp plugin activate` command. We added the `--network` option to it. This tells WP-CLI to activate the plugin across the entire multisite installation and not just a specific site. It's possible that activating your plugin using the `--network` option won't change anything in practice. That said, in the case of this plugin, there's a different admin page if the plugin is network activated. So doing this is important. ## Deployment jobs Once our acceptance tests pass, we can deploy the plugin to the WordPress plugin directory. This is the purpose of the last set of jobs that we'll look at. We broke them down into two jobs due to the structure of the WordPress plugin directory SVN repository. The first job is to deploy plugin assets to WordPress plugin directory. We need this job because plugin assets aren't tied to our code. We might want to add or update our plugin assets without releasing a new version of the plugin. The second job is to deploy the plugin itself. This job is a bit like a build job that we described earlier. We want to prepare a "WordPress directory" version of the plugin. Once done, we commit that new version to WordPress plugins directory SVN repository. ``` references: WP_CORE_DIR: &WP_CORE_DIR /tmp/wordpress WP_HOST: &WP_HOST passwords-evolved.test WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME passwords-evolved WP_ORG_USERNAME: &WP_ORG_USERNAME carlalexander container_config: &container_config docker: - image: circleci/php:7.2 environment: - WP_CORE_DIR: *WP_CORE_DIR - WP_HOST: *WP_HOST - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME - WP_ORG_USERNAME: *WP_ORG_USERNAME working_directory: ~/passwords-evolved workspace_root: &workspace_root /tmp attach_workspace: &attach_workspace attach_workspace: at: *workspace_root install_subversion: &install_subversion run: name: Install subversion command: sudo apt-get install subversion jobs: deploy_assets: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy assets to WordPress plugin directory command: .circleci/deploy_assets.sh deploy_plugin: <<: *container_config steps: - checkout - *attach_workspace - *install_subversion - run: name: Deploy new version to WordPress plugin directory command: .circleci/deploy_plugin.sh ``` And this is the relevant sections of the `config.yml` file for our deployment jobs. There's less to do at this point compared to what we saw for unit tests and acceptance tests. Each job has its bash script which does most of the work. We'll discuss those next. ### Installing subversion But before doing that, let's just go over the one new reference step in the `steps` section. It's the `install_subversion` step. This is step is necessary for similar reasons as the ones we added to install MySQL for acceptance tests. Our Docker container only has PHP installed and nothing else. But to deploy our assets or plugin, we need subversion. So we install it with this step. ### Adding your WordPress.org password to CircleCI At this point, our jobs need our WordPress.org password to commit changes to the plugin's WordPress SVN repository. But you should never store a password (or any other sensitive data) in your git repository. This means that we need another way to store our WordPress.org password. CircleCI lets you define environment variables for a project in the projet's settings. Just follow the instructions [here](https://circleci.com/docs/2.0/env-vars/#setting-an-environment-variable-in-a-project) and add the `WP_ORG_PASSWORD` environment variable with your password. And then you'll be ready to proceed! ### Deployment bash scripts As we said earlier, the rest of the deployment jobs revolves around bash scripts. We have one for each deployment job and they share some things in common. That said, the plugin deployment script is more involved due to a couple of factors that we'll discuss. #### Deploying assets We're going to start with the bash script to deploy our plugin assets since it's the simplest of the two scripts. Plugin assets are things like your plugin logo and the banner image for the WordPress plugin directory. We also mentioned earlier that we don't want to tie those to our code and releases. But also, plugin assets don't get stored in the same place in our plugin's SVN repository. ``` #!/usr/bin/env bash if [[ -z "$CIRCLECI" ]]; then echo "This script can only be run by CircleCI. Aborting." 1>&2 exit 1 fi if [[ -z "$CIRCLE_BRANCH" || "$CIRCLE_BRANCH" != "master" ]]; then echo "Build branch is required and must be 'master' branch. Stopping deployment." 1>&2 exit 0 fi if [[ -z "$WP_ORG_PASSWORD" ]]; then echo "WordPress.org password not set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then echo "WordPress.org plugin name not set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_USERNAME" ]]; then echo "WordPress.org username not set. Aborting." 1>&2 exit 1 fi PLUGIN_SVN_PATH="/tmp/svn" # Checkout the SVN repo svn co -q "http://svn.wp-plugins.org/$WP_ORG_PLUGIN_NAME" $PLUGIN_SVN_PATH # Delete the assets directory rm -rf $PLUGIN_SVN_PATH/assets # Copy our plugin assets as the new assets directory cp -r ./assets $PLUGIN_SVN_PATH/assets # Move to SVN directory cd $PLUGIN_SVN_PATH # Add new files to SVN svn stat | grep '^?' | awk '{print $2}' | xargs -I x svn add x@ # Remove deleted files from SVN svn stat | grep '^!' | awk '{print $2}' | xargs -I x svn rm --force x@ # Commit to SVN svn ci --no-auth-cache --username $WP_ORG_USERNAME --password $WP_ORG_PASSWORD -m "Deploy new assets" ``` You can see our bash script for deploying assets above. It starts similarly to the setup bash scripts that we had for our acceptance test jobs. We have a bunch of guard clauses checking for various environment variables. It's worth noting that there's one guard clause that's a bit different from what we saw in the other bash scripts. It's the `CIRCLE_BRANCH` guard clause. Not only does it check if the variable is there, but it also checks that its value is `master`. That's because we only want our deploy scripts to run on the `master` branch and not other branches such as pull requests. Once we're through the guard clauses, we start by checking out the plugin from the SVN repository. We define the location that we want to check out our plugin with the `PLUGIN_SVN_PATH` variable. We'll reuse this variable throughout the script. Once the plugin checked out, we immediately delete the `assets` directory from it. The `assets` directory is the directory that the WordPress plugin directory wants you to put your assets in. It always uses the current files in that directory so you can't create different versions of it. That's why we want to delete it. We then copy the `assets` directory of the plugin that we checked out from GitHub. This creates a new `assets` directory in our plugin that we checked out from SVN. We can use this new `assets` directory to track the changes that we made to it. ##### Commiting changes to SVN The next two commands (if we can call them that!) are a series of four [piped](https://en.wikipedia.org/wiki/Pipeline_(Unix)) commands. They both start by running the `svn stat` command. This prints the current status of all the files in the checked out SVN repository. After that, both commands differ. Both use [`grep`](https://en.wikipedia.org/wiki/Grep), but the first one checks for new files and the other one for missing files. Those missing files are the files that we removed in our commit. We then pass the results of both `grep` commands to the same `awk` command. [AWK](https://en.wikipedia.org/wiki/AWK) is a programming language, and the `awk` runs whatever program you pass it. The `{print $2}` prints out the paths of the files that we added or removed. These file paths are then piped into two different [`xargs`](https://en.wikipedia.org/wiki/Xargs) commands. One runs `svn add` on all the files that we added in our commit. And the other runs `svn rm --force` on all the files that we removed. Once we've added and removed files using these two sets of piped commands, we can commit these changes. We do that using the `svn ci` command. We tell it not to cache our credentials using the `--no-auth-cache` flag. And then we pass it our WordPress.org username and password manually using the `WP_ORG_USERNAME` and `WP_ORG_PASSWORD` environment variables. It's worth noting that, if the two sets of piped commands didn't add or remove files, the `svn ci` command wouldn't do anything. So you don't have to worry that this will make commits every time CircleCI runs this job. The command only does something if there are changes to push to the repository. #### Deploying new versions of the plugin Now that we've covered deploying assets to the WordPress plugin directory, we can move on to the harder scenario. That's deploying a new version of our plugin. Here's the bash script that does this: ``` #!/usr/bin/env bash if [[ -z "$CIRCLECI" ]]; then echo "This script can only be run by CircleCI. Aborting." 1>&2 exit 1 fi if [[ -z "$CIRCLE_BRANCH" || "$CIRCLE_BRANCH" != "master" ]]; then echo "Build branch is required and must be 'master' branch. Stopping deployment." 1>&2 exit 0 fi if [[ -z "$WP_ORG_PASSWORD" ]]; then echo "WordPress.org password not set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then echo "WordPress.org plugin name not set. Aborting." 1>&2 exit 1 fi if [[ -z "$WP_ORG_USERNAME" ]]; then echo "WordPress.org username not set. Aborting." 1>&2 exit 1 fi PLUGIN_BUILD_DIRECTORIES=(lib resources src) PLUGIN_BUILD_FILES=(index.php LICENSE passwords-evolved.php pluggable.php readme.txt) PLUGIN_BUILD_PATH="/tmp/build" PLUGIN_SVN_PATH="/tmp/svn" # Figure out the most recent git tag LATEST_GIT_TAG=$(git describe --tags `git rev-list --tags --max-count=1`) # Remove the "v" at the beginning of the git tag LATEST_SVN_TAG=${LATEST_GIT_TAG:1} # Check if the latest SVN tag exists already TAG=$(svn ls "https://plugins.svn.wordpress.org/$WP_ORG_PLUGIN_NAME/tags/$LATEST_SVN_TAG") error=$? if [ $error == 0 ]; then # Tag exists, don't deploy echo "Latest tag ($LATEST_SVN_TAG) already exists on the WordPress directory. No deployment needed!" exit 0 fi # Checkout the git tag git checkout tags/$LATEST_GIT_TAG # Create the build directory mkdir $PLUGIN_BUILD_PATH # Copy plugin directories to the build directory for DIRECTORY in "${PLUGIN_BUILD_DIRECTORIES[@]}"; do cp -r $DIRECTORY $PLUGIN_BUILD_PATH/$DIRECTORY done # Copy plugin files to the build directory for FILE in "${PLUGIN_BUILD_FILES[@]}"; do cp $FILE $PLUGIN_BUILD_PATH/$FILE done # Checkout the SVN repo svn co -q "http://svn.wp-plugins.org/$WP_ORG_PLUGIN_NAME" $PLUGIN_SVN_PATH # Move to SVN directory cd $PLUGIN_SVN_PATH # Delete the trunk directory rm -rf ./trunk # Copy our new version of the plugin as the new trunk directory cp -r $PLUGIN_BUILD_PATH ./trunk # Copy our new version of the plugin into new version tag directory cp -r $PLUGIN_BUILD_PATH ./tags/$LATEST_SVN_TAG # Add new files to SVN svn stat | grep '^?' | awk '{print $2}' | xargs -I x svn add x@ # Remove deleted files from SVN svn stat | grep '^!' | awk '{print $2}' | xargs -I x svn rm --force x@ # Commit to SVN svn ci --no-auth-cache --username $WP_ORG_USERNAME --password $WP_ORG_PASSWORD -m "Deploy version $LATEST_SVN_TAG" ``` This is a more elaborate version of the assets deployment bash script that we just saw. The beginning of the bash script starts the same way as the assets one. We have guard clauses checking for the presence of various environment variables. We also ensure that we only run this bash script on the `master` branch. ##### Do we need a new version of the plugin? Once we're done with the default guard clauses, we move on to another new and more complex guard clause. We need to determine if we have to deploy a new version of the plugin. We do that by comparing the latest tag in our git repository to the latest tag in our SVN repository. To get the latest git tag, we use the following command: `git describe --tags `git rev-list --tags --max-count=1``. We store this tag value in the `LATEST_GIT_TAG` variable. The step after that is optional. That's because the plugin uses a tagging format that leads the [semantic version number](https://semver.org/) with a `v`. We can't have that for our SVN tags. So we create a `LATEST_SVN_TAG` variable where we truncate the `v` off the `LATEST_GIT_TAG` value. But, if you don't do your git tags that way, you don't need to do this step and can just use the `LATEST_GIT_TAG` value. Once we have our `LATEST_SVN_TAG` value, we want to use it to check for that tag on the WordPress plugin directory SVN repository. We do that by using the `svn ls` command. This command is pretty much the same as the UNIX [`ls`](https://en.wikipedia.org/wiki/Ls) command. It lists all the files and directories in the target directory in the SVN repository. If there's no directory for our `LATEST_SVN_TAG`, the `svn ls` command will return an error. We store this error in the `error` variable. We will use it to determine if we continue or not. If there wasn't an error, it means our `LATEST_SVN_TAG` does exist. We don't need to do anything else, and we exit with `0`. (This is a bit counter-intuitive, but `0` means that command ran successfully.) Otherwise, an error indicates that our `LATEST_SVN_TAG` doesn't exist and we need to create it, so we continue. ##### Building the directory version of the plugin At this point, we're sure that we need to build a new version of the plugin. The first step is to check out the `LATEST_GIT_TAG` version of the plugin. We do that using the `git checkout` command. Why not just use the current version that CircleCI checked out? This is a safety step in case the code that CircleCI checked out wasn't the same as the `LATEST_GIT_TAG` version. With this step, we know that whatever happens, we're working with the version of the code that we want to deploy to the WordPress plugins directory. After that, we start building the directory version of the plugin. We create the build directory using the [`mkdir`](https://en.wikipedia.org/wiki/Mkdir). We use `PLUGIN_BUILD_PATH` variable that we defined earlier as the directory name. Once we have our build directory, we start the build process. We copy all the directories and files that we want our WordPress directory version of our plugin to have. We defined those in the `PLUGIN_BUILD_DIRECTORIES` and `PLUGIN_BUILD_FILES` variables. We copy these directories and files using a bash [`for`](http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html) loop. The loop for directories copies directory using the `cp -r` command. The `-r` is a modifier that tells the `cp` command to recursively copy all the files from the target. To copy the files, we just use a normal `cp` command without the `-r` modifier. ##### Commiting our new version to SVN The last part of the `deploy-plugin.sh` bash script focuses on committing the new version of the plugin that we just built. This part is almost the same as the one that we used to commit our plugin assets to WordPress plugins directory SVN repository. The differences between both are quite minor. First, we're not working from the `assets` directory. Instead, we're using the [`trunk`](https://en.wikipedia.org/wiki/Trunk_(software)) directory. But like the `assets` directory, we start by deleting everything in it. We then copy our new version of the plugin to replace the deleted `trunk` directory. We also copy our new version of the plugin to the `tags` directory. The `tags` directory is how SVN tracks new versions of our plugin. That's why we copy it to the `tags` directory using the `LATEST_SVN_TAG` variable for the subdirectory. The rest of the script focuses on committing our changes. For this, we reuse the same commands that we used in the `deploy-assets.sh` bash script. The only small change is in the commit message where it mentions the version that we're deploying. # Building our workflow So this wraps up the definition of our jobs in CircleCI. That said, we still need to put these jobs together into the continuous deployment workflow that we saw at the beginning. (That was like ages ago!) This is a lot less complicated than what we just did to create all the CircleCI jobs. ``` workflows: version: 2 build_test_deploy: jobs: - build - deploy_assets: filters: branches: only: - master - code_quality: requires: - build - test_php72: requires: - code_quality - test_php71: requires: - code_quality - test_php70: requires: - code_quality - test_php56: requires: - code_quality - test_php55: requires: - code_quality - test_php54: requires: - code_quality - acceptance_singlesite: requires: - test_php72 - test_php71 - test_php70 - test_php56 - test_php55 - test_php54 - acceptance_multisite: requires: - test_php72 - test_php71 - test_php70 - test_php56 - test_php55 - test_php54 - deploy_plugin: filters: branches: only: - master requires: - acceptance_singlesite - acceptance_multisite ``` This is the section of the `config.yml` file that configures our workflow. The first thing to notice is that it's under its own section named `workflows`. It's not under the `jobs` section that we were working on earlier. Under the `workflows` section, we have two elements. `version` is a required element. The documentation doesn't explain why it's necessary. The other element under the `workflows` section is `build_test_deploy`. This is the name that we gave to our workflow. You could name it whatever you want. You can also have as many workflows as you want. If you run more complicated setups than this one, you could have a workflow for nightly builds and pull request testing for example. But, for us, one is plenty, and we'll take care of special cases right in our workflow configuration. ## Orchestrating our workflow Speaking of that, let's go over our workflow configuration and see how we set things up. How to configure a workflow is pretty simple. We just list all our jobs using the names that we gave them in `jobs` section of the `config.yml` file. What's more important is what configuration information we put under each job. The only job without any is the `build` job. And that's because it's the job at the beginning of our workflow. After that, every job will have the `requires` section except for the `deploy_assets` job. (We'll go over why that is in a bit.) The `requires` section is how we configure the [orchestration](https://en.wikipedia.org/wiki/Orchestration_(computing)) of our jobs with CircleCI. If you're not familiar with the idea of orchestration, it's just a way to automate the ordering of our jobs. This allows CircleCI to run them in the most optimal way possible. CircleCI wants to do this because it supports running jobs [in parallel](https://en.wikipedia.org/wiki/Parallelism_(computing)). This is something that we want our configuration to take advantage of as well. That's because CircleCI gives open source projects 4 containers to use. (And my plugin is open source.) ### Configuring our job dependencies Ok, so let's go back to the `requires` section. We use it to tell CircleCI which job(s) does the job that we're configuring depend on. That's also why the `build` job doesn't have a `requires` section. As the first job in our workflow, it doesn't depend on anything. (The same is true for the `deploy_assets` job as well but it's for a different reason.) The next job that we want to run after the `build` job is the `code_quality` job. So that job has the `build` job in its `requires` section. All pretty simple so far. ### Fanning out and fanning in Things get more complicated with our unit testing jobs. We don't care in what order these jobs get run as. In fact, we can even run them in parallel since they don't depend on each other. CircleCI calls this type of workflow a [fan-out](https://circleci.com/docs/2.0/workflows/#fan-outfan-in-workflow-example). To configure a fan-out workflow, we just need to put the `code_quality` job as the dependency of all our unit test jobs. That's why they all have the `code_quality` job in their `requires` section. After all our unit test jobs have run successfully, we want to run our two acceptance test jobs. To do that, we have to do a fan-in. This is the opposite of a fan-out. A fan-in tells CircleCI that it must complete all our dependent jobs before it can proceed. To do a fan-in, we have to put all our unit test jobs in the `requires` section of our acceptance test jobs. This will tell CircleCI that it can only start our acceptance test jobs once it finishes running all the unit test jobs. We then need to do the same for the `deploy_plugin` job. We want to only run it once CircleCI has finished running both our acceptance test jobs. So we put both jobs under the `requires` section so that CircleCI performs a fan-in once both jobs finish running. ### Only deploying on the master branch The last thing that we want to do is add some extra security for our deployment jobs. Our bash scripts already have guard clauses to prevent bad things from happening. That said, there's nothing wrong with some extra precautions! That's why both our deployment jobs have an extra `filters` section. This section tells CircleCI to only run these jobs when the branch or tag matches a certain condition. In our config, that condition is that the commit is on the `master` branch. ### The deploy\_assets job There's one job that stands out a bit in our entire workflow. It's the `deploy_assets` job which we've alluded a few times so far this section. Everything about this job is a bit special, and the way we integrate it into our workflow isn't any different. First, the `deploy_assets` job is like our `build` job. It doesn't depend on any other job. But it's also different from the `build` job in that it's not a dependency for any other job either. It's a job that's standalone. This goes back to what we said earlier about the `deploy_assets` job. It's a job that's there due to the structure of the WordPress directory SVN repository. WordPress wants plugin assets stored in a different location than our code. On top of that, there's no reason to run our entire workflow for our plugin assets. Plugin assets aren't code. We don't need to perform unit tests or acceptance tests on them. We just need them deployed to the WordPress plugin directory when we update them. That's something that we want to be able to do at any time. The only restriction that we have is on the branch that we want this job to run on. In this way, the `deploy_assets` job is like the `deploy_plugin` job. We only want it to run on commits in the `master` branch. After all, you wouldn't want a commit in a pull request or some other branch to update your assets in the WordPress plugin directory! That's why the `deploy_assets` job has a `filters` section. # And that's it! Whew, that was quite the journey! But this is everything that you need to know to continuously deploy a plugin to the WordPress directory. It's worth pointing out again that, while we did this for a plugin, you can also use this with a theme. With a setup like this, you don't have to worry about deploying your new plugin versions to the WordPress directory. You can instead focus on coding new features, writing tests and just tag a new version when you're ready to go. That's the magic of continuous integration! *Special thanks to [Iain Poulson](http://www.polevaultweb.com/) who wrote about doing the same thing with Travis CI.*