The Man in the Arena

by Carl Alexander

Want to get more articles like this one? Join my newsletter

In DevOps

Continuous deployment to the WordPress directory with CircleCI

Note: This article focuses on how to continuously deploy a plugin using CircleCI. But you can also use everything discussed here with a theme as well.

As developers, we all have our preferred tools and distinct way of working. That’s why it’s not uncommon for us to write about it or create scripts to set up our work computers. But the one tool that most of us tend to agree on is using git for version control.

That said, if you’ve ever had a plugin or theme on the WordPress directory, you know that it doesn’t use git. It uses subversion. That’s a problem for a lot of us because we don’t want to have to deal with both.

While there are a lot of resources out there to deal with this problem, it also presents us with an opportunity. We can use this problem to build a continuous deployment workflow for the WordPress directory. This will allow us to not worry about this aspect of WordPress plugin development anymore.

Some background

But before we begin looking at all the details of how to do this, let’s discuss why I’m writing this article. The initial motivation is exactly what I described in the introduction. Like a lot of WordPress developers, I’ve never built a plugin that I’ve published to the plugin directory.

But I’d been working on a small security plugin that I wanted to put in the directory. I wasn’t super excited at the idea of having to maintain my plugin in two locations. Like most developers, I work with GitHub a lot, and I had a public repository there for the plugin. I didn’t want to have to keep track of things in the plugin’s subversion repository as well.

At the same time, I’d been working a lot with CircleCI. I’d used it to build a continuous integration workflow on a WordPress project. I’d also used it to manage the continuous deployment of that project to Pantheon.

I’d also used Travis CI before. But I think that, at this point, CircleCI is a bit better than Travis CI. The only exception is if you need Travis’s build matrix feature. As we’ll see, we can do something a bit similar, but it’s definitely not as powerful.

So that’s why we’re going to use CircleCI to build our continuous deployment workflow. That said, if you use Travis CI, there’s already a good article on how to do this with it. In fact, some of the scripts that you’ll see today came initially from that article. They were then modified to fit the continuous delivery workflow with CircleCI better.

A note on JavaScript

Before we begin, it would be important to discuss JavaScript. It’s uncommon for a plugin not to use any JavaScript nowadays. That said, this plugin is one of those rare plugins where we don’t use any JavaScript.

This means that what we’ll see in this article doesn’t cover anything related to JavaScript. There’s no JavaScript unit testing to do. The setup used to run acceptance tests uses a headless browser driver. This driver doesn’t support the execution of JavaScript.

This means that, if you rely a lot on JavaScript and you want to support it, you’ll find this setup incomplete. You’ll need to add jobs for unit testing JavaScript. You’ll also need to modify the acceptance test jobs to support JavaScript as well.

That said, you’ll still find a lot of useful information in this article anyways. We explain a lot of how the CircleCI platform works. This will help you understand the changes that you need to support JavaScript in your continuous deployment workflow.

Overview of the workflow

So speaking of continuous deployment workflow, let’s look at what it’ll look like in its final form. We’ll then go over these jobs or group of jobs and explain what they’re doing and why. CircleCI has a screen where you can see all the jobs in a workflow and how they’re connected. You can see it below:

Complete CircleCI workflow

This is a picture of the CircleCI workflow used by my plugin. It has 12 jobs. But we can break down these jobs into more specific categories. These are:

  • Build
  • Code quality checks
  • Unit tests
  • Acceptance tests
  • Deployment

These categories are also in the order that CircleCI will go through them. The only exception is the asset deployment job which runs on its own. That’s because the WordPress directory manages plugin assets differently. So we don’t deploy them at the same time as the rest of the plugin files. (We’ll mention this a few times throughout the article.)

CircleCI config file

With CircleCI, you configure this entire workflow and its jobs using a YAML configuration file. (You can find the documentation for it here.) This configuration file is always under .circleci/config.yml path in your repository. Here’s what the one that I used to create the workflow that you saw in the picture above:

version: 2.0

references:
  # Environment variables
  #
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  # Default container configuration
  #
  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

  attach_workspace: &attach_workspace
    attach_workspace:
      at: *workspace_root

  copy_vendor: &copy_vendor
    run:
      name: Copy vendor directory
      command: cp -R /tmp/vendor .

  install_subversion: &install_subversion
    run:
      name: Install subversion
      command: sudo apt-get install subversion

  # Default configuration for all behat testing jobs
  #
  behat_job: &behat_job
    <<: *container_config
    docker:
      - image: circleci/php:7.2
      - image: circleci/mysql:5.7
    steps:
      - checkout
      - run:
          name: Add WordPress host to hosts file
          command: echo "127.0.0.1 ${WP_HOST}" | sudo tee -a /etc/hosts
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Install MySQL client
          command: sudo apt-get install mysql-client
      - run:
          name: Install MySQL PHP extension
          command: sudo docker-php-ext-install mysqli
      - run:
          name: Setup WordPress
          command: .circleci/setup-$WP_TYPE.sh
      - run:
          name: Start PHP server
          command: sudo php -S $WP_HOST:80 -t $WP_CORE_DIR
          background: True
      - run:
          name: Run Behat tests
          command: vendor/bin/behat --config .circleci/behat.yml --format progress --tags=$WP_TYPE

  # Default configuration for all phpunit testing jobs
  #
  phpunit_job: &phpunit_job
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Run PHP unit tests
          command: vendor/bin/phpunit

jobs:
  build:
    <<: *container_config
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: composer install -n -o
      - run:
          name: Install WordPress
          command: bash .circleci/install-wp.sh
      - persist_to_workspace:
          root: .
          paths:
            - vendor
      - persist_to_workspace:
          root: *workspace_root
          paths:
            - wordpress

  code_quality:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Run code quality tests
          command: vendor/bin/grumphp run --testsuite=code_quality

  test_php72:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.2

  test_php71:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.1

  test_php70:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.0

  test_php56:
    <<: *phpunit_job
    docker:
      - image: circleci/php:5.6

  test_php55:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.5

  test_php54:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.4

  acceptance_singlesite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: singlesite

  acceptance_multisite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: multisite

  deploy_assets:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy assets to WordPress plugin directory
          command: .circleci/deploy-assets.sh

  deploy_plugin:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy new version to WordPress plugin directory
          command: .circleci/deploy-plugin.sh

workflows:
  version: 2
  build_test_deploy:
    jobs:
      - build
      - deploy_assets:
          filters:
            branches:
              only:
                - master
      - code_quality:
          requires:
            - build
      - test_php72:
          requires:
            - code_quality
      - test_php71:
          requires:
            - code_quality
      - test_php70:
          requires:
            - code_quality
      - test_php56:
          requires:
            - code_quality
      - test_php55:
          requires:
            - code_quality
      - test_php54:
          requires:
            - code_quality
      - acceptance_singlesite:
          requires:
            - test_php72
            - test_php71
            - test_php70
            - test_php56
            - test_php55
            - test_php54
      - acceptance_multisite:
          requires:
            - test_php72
            - test_php71
            - test_php70
            - test_php56
            - test_php55
            - test_php54
      - deploy_plugin:
          filters:
            branches:
              only:
                - master
          requires:
            - acceptance_singlesite
            - acceptance_multisite

The primary goal of this article is to explain this entire configuration file. That said, there’s a section of the config file above that’s not part of a standard CircleCI YAML config file. It’s the references section at the top.

YAML references

The references section is there for us to leverage a lesser known feature of YAML: anchor and references. Anchors (noted by the & symbol) let us define nodes that we want to reuse in our YAML file. We then use the * symbol to reference these anchors.

You can also extend a YAML node using the << notation. This lets us inject the content inside our reference inside another YAML node. You can then add more elements to that YAML node afterwards. In that scenario, our reference is more of a YAML node template that we can reuse instead of content that we’re copying in.

So what is the link between this YAML feature and our references section? It’s that we use it to organize all the anchors that we want to use throughout our config file. We have to do this because there’s no official way to handle anchors in YAML or with CircleCI. This, at least, keeps things tidy and gives us an easy to maintain solution to that problem.

Default container configuration

At the top of our references section, there’s a reference called container_config. This is one of the most important parts of the entire CircleCI configuration file. This is the default configuration for all the containers that CircleCI will use.

Without going into the full details of the CircleCI’s architecture, you should know that everything works off Docker containers. CircleCI spins up a new Docker container each time that it runs a job. The container_config reference is the base template that for those Docker containers that CircleCI spins up.

# .circleci/config.yml

references:
  # Environment variables
  #
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  # Default container configuration
  #
  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

So here’s a look at our container_config reference. First, we created four other references. Those references are the environment variables that we need our containers to have. We have to define them first so that we can use them in container_config reference. Before moving on, here’s a quick overview of each of them:

  • WP_CORE_DIR is the directory where our scripts can find our WordPress installation.
  • WP_HOST is the hostname of the WordPress site that we’ll use for testing.
  • WP_ORG_PLUGIN_NAME is the name of our plugin in the WordPress plugin directiory.
  • WP_ORG_USERNAME is your WordPress.org username.

container_config sections

The container_config reference itself has three section. First, there’s the docker section. This is the section where you define the Docker images that the CircleCI container will use.

By default, we use the circleci/php:7.2 image only. circleci/php is the Docker image that CircleCI built for PHP projects. CircleCI has a lot of pre-built Docker images that you can use for your jobs.

Meanwhile, the :7.2 specifies the image tag that we want to use. In this scenario, it means that we want the circleci/php to use PHP 7.2. So, by default, all our containers will use PHP 7.2 unless specified otherwise.

The last two sections are more straightforward than the docker section. The environment section is where we configure the container’s environment variables. We just list references to the four environment variables that went over in the last section.

Configuring our CircleCI jobs

The first of two sections of a CircleCI configuration file is the jobs section. This is where we define all the jobs in our continuous integration workflow. Here’s another look at all the jobs in our workflow:

# .circleci/config.yml

jobs:
  build:
    <<: *container_config
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: composer install -n -o
      - run:
          name: Install WordPress
          command: bash .circleci/install-wp.sh
      - persist_to_workspace:
          root: .
          paths:
            - vendor
      - persist_to_workspace:
          root: *workspace_root
          paths:
            - wordpress

  code_quality:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Run code quality tests
          command: vendor/bin/grumphp run --testsuite=code_quality

  test_php72:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.2

  test_php71:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.1

  test_php70:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.0

  test_php56:
    <<: *phpunit_job
    docker:
      - image: circleci/php:5.6

  test_php55:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.5

  test_php54:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.4

  acceptance_singlesite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: singlesite

  acceptance_multisite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: multisite

  deploy_assets:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy assets to WordPress plugin directory
          command: .circleci/deploy-assets.sh

  deploy_plugin:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy new version to WordPress plugin directory
          command: .circleci/deploy-plugin.sh

Each key in the jobs node is the name of the job. It needs to be unique because we need to be able to refer to it when we build our workflow later. The content of the node contains details like the steps that you want CircleCI to perform.

We’re going through all these jobs and groups of jobs. We’ll explain what the steps in them are doing. We’ll also talk about other unique aspects of their configuration.

Build job

The first job that we have in our continuous deployment workflow is the build job. Its purpose is to check out a copy of our plugin and install its dependencies. We also use the build process to install the copy of WordPress that we’ll use throughout our workflow.

# .circleci/config.yml

references:
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

jobs:
  build:
    <<: *container_config
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: composer install -n -o
      - run:
          name: Install WordPress
          command: bash .circleci/install-wp.sh
      - persist_to_workspace:
          root: .
          paths:
            - vendor
      - persist_to_workspace:
          root: *workspace_root
          paths:
            - wordpress

Above is the parts of the config.yml file that is relevant to the build job. The build job starts by extending the container_config node. This is the default configuration for all the Docker containers that we just saw.

Build steps

The rest of the build job is the steps section. We first check out our plugin, then run composer install and then the install-wp.sh bash script. You can see a copy of the bash script below:

#!/usr/bin/env bash

WP_CORE_DIR=${WP_CORE_DIR-/tmp/wordpress/}
WP_VERSION=${WP_VERSION-latest}

download() {
    if [ `which curl` ]; then
        curl -s "$1" > "$2";
    elif [ `which wget` ]; then
        wget -nv -O "$2" "$1"
    fi
}

set -ex

install_wp() {

    if [ -d $WP_CORE_DIR ]; then
        return;
    fi

    mkdir -p $WP_CORE_DIR

    if [[ $WP_VERSION == 'nightly' || $WP_VERSION == 'trunk' ]]; then
        mkdir -p /tmp/wordpress-nightly
        download https://wordpress.org/nightly-builds/wordpress-latest.zip  /tmp/wordpress-nightly/wordpress-nightly.zip
        unzip -q /tmp/wordpress-nightly/wordpress-nightly.zip -d /tmp/wordpress-nightly/
        mv /tmp/wordpress-nightly/wordpress/* $WP_CORE_DIR
    else
        if [ $WP_VERSION == 'latest' ]; then
            local ARCHIVE_NAME='latest'
        else
            local ARCHIVE_NAME="wordpress-$WP_VERSION"
        fi
        download https://wordpress.org/${ARCHIVE_NAME}.tar.gz  /tmp/wordpress.tar.gz
        tar --strip-components=1 -zxmf /tmp/wordpress.tar.gz -C $WP_CORE_DIR
    fi

    download https://raw.github.com/markoheijnen/wp-mysqli/master/db.php $WP_CORE_DIR/wp-content/db.php
}

install_wp

The script is just a partial copy of the script generated by the WP-CLI scaffold command. This is the part that downloads the version of WordPress specified by the WP_VERSION variable. It then installs that version of WordPress in the directory defined by the WP_CORE_DIR variable.

That said, we didn’t define either of those variables. This means that the install-wp.sh bash script will use the default values defined inside of it. It’ll install the latest version of WordPress in the /tmp/wordpress directory.

Saving everything to a workspace

Once we installed our dependencies and WordPress, we need to save them so that we can reuse them throughout the workflow. CircleCI has two mechanisms to do that: caching and workspaces. If you read about the difference between them, the correct one to use would be caching.

But, as you can see from the build job configuration above, we’re not. We’re using workspaces instead. Why is that?

Well, it comes down to how CircleCI implemented its cache feature. With CircleCI, when you save something to a cache, you have to use a key to identify it. The problem is that CircleCI doesn’t let you overwrite an existing cache once it’s created.

This means that you need a good way to generate keys to use the CircleCI cache feature. If you don’t, CircleCI will keep fetching the same data over and over forever. That’s why we can’t use the CircleCI cache feature.

Our current setup doesn’t have a dependable way to generate cache keys for either composer or WordPress. We could for composer if we used a composer.lock file, but we don’t here. So, instead of the CircleCI cache feature, we’ll use the workspace feature to cache our composer and WordPress files.

Those are what the two persist_to_workspace steps do. The first one saves the composer vendor directory with all our dependencies. The second saves the copy of WordPress that the install-wp.sh bash script downloaded earlier.

Code quality job

The second job in our continuous deployment workflow is the code_quality job. We use it to run all our code quality checks on our plugin code. We want to do this before we start running the tests on our plugin code.

# .circleci/config.yml

references:
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

  attach_workspace: &attach_workspace
    attach_workspace:
      at: *workspace_root

  copy_vendor: &copy_vendor
    run:
      name: Copy vendor directory
      command: cp -R /tmp/vendor .

jobs:
  code_quality:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Run code quality tests
          command: vendor/bin/grumphp run --testsuite=code_quality

This is the sections of the config.yml file used for the code_quality job. Like the build job that we saw before, the code_quality job also extends the container_config node. The main difference between the two jobs is the steps section.

Code quality steps

This job also starts by checking out our plugin code. In fact, all our jobs will start with the checkout step so it won’t be the last time that we mention it. But the rest of the steps are different from what we saw earlier.

Next, we have the attach_workspace step which is a reference from our references section. The attach_workspace calls the special attach_workspace step from CircleCI. (Yes, this is a bit confusing!) We then tell the attach_workspace step to attach our workspace to the workspace_root directory.

Once CircleCi attached our workspace, we want to copy back the vendor directory that we copied to it earlier. That’s what the copy_vendor step does. This is another reference from our references section.

The copy_vendor reference uses the run step. This is the most common step used in CircleCI job configurations. You use to run a specific command in the container.

For our copy_vendor step, we just want to run cp command. It copies our vendor directory back to the working directory defined in container_config. This allows us not to run composer install again which is a time-consuming command.

Checking code quality with GrumPHP

The last step in our code_quality job is the one doing the code quality checks. For this, we leverage a tool called GrumPHP. It’s a tool for enforcing code quality whenever you commit code.

Now, GrumPHP isn’t a tool built with continuous deployment in mind. Its primary use is to enforce code quality on developer’s local development environments. That said, you can still use it with continuous deployment.

# grumphp.yml

parameters:
  process_timeout: 120
  tasks:
    composer: ~
    gherkin: ~
    git_commit_message: ~
    phpcpd:
      exclude:
        - lib
        - tests
        - vendor
    phpcs:
      whitelist_patterns:
        - /^src
    phplint: ~
    phpmd:
      ruleset: ['codesize', 'design', 'naming', 'unusedcode']
      exclude:
        - lib
        - tests
        - vendor
    phpunit:
      always_execute: true
  testsuites:
    code_quality:
      tasks:
        - composer
        - gherkin
        - phpcpd
        - phpcs
        - phplint
        - phpmd

Above is the GrumPHP configuration used by the plugin. It has a quite a few code quality checks enabled. We’re not going to go over them in this article, but you can read more about them here.

What we’re more interested in is the testsuites section of the configuration file. This section lets us create custom test suites that only run specific code quality checks. The code_quality test suite is the test suite that we’ve created for our CircleCI workflow.

We did this because the default behaviour of GrumpmPHP is to run all the code quality checks. We don’t want that here since we need to run our unit tests multiple times to test different PHP versions. (More on that in a second!) That’s why the code_quality test suite contains every code quality check except phpunit.

So that’s why we need this code_quality test suite for CircleCI to perform our code quality checks. We use a run step to do it. The step runs with GrumPHP with an argument specifying the code_quality test suite that we want to run.

PHPUnit jobs

Once we’re done with the code_quality job, we move on to jobs running our unit tests. This section is a bit different from what we’ve seen so far. We’re going to talk about a group of jobs instead of a specific job.

The reason for that is that we need to run our unit tests on different versions of PHP. (For our plugin, we need to run them on PHP 5.4 and newer.) But, with CircleCI, you can only run your unit tests on one version of PHP at a time. So that means that we need a job for each version of PHP that we want to run our unit tests against.

# .circleci/config.yml

references:
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

  attach_workspace: &attach_workspace
    attach_workspace:
      at: *workspace_root

  copy_vendor: &copy_vendor
    run:
      name: Copy vendor directory
      command: cp -R /tmp/vendor .

  phpunit_job: &phpunit_job
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Run PHP unit tests
          command: vendor/bin/phpunit

jobs:
  test_php72:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.2

  test_php71:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.1

  test_php70:
    <<: *phpunit_job
    docker:
      - image: circleci/php:7.0

  test_php56:
    <<: *phpunit_job
    docker:
      - image: circleci/php:5.6

  test_php55:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.5

  test_php54:
    <<: *phpunit_job
    docker:
      - image: vandries/phpenv:5.4

This is the sections of the config.yml file with all our unit testing jobs. As you can see, it’s a lot bigger than the sections that we saw before. That said, it’s not a lot more complicated than the jobs that we’ve seen so far.

phpunit_job reference

The essence of our unit testing jobs is the phpunit_job reference. This is the reference that contains all the steps that our unit test jobs use. Like all the other jobs that we’ve seen so far, it also extends container_config.

The steps in phpunit_job job reference are almost the same as the ones we saw with the code_quality job. We start by checking out the plugin code. We then attach our workspace and copy the vendor directory from it.

The last step is where things differ from the code_quality job. We have a run step, but it doesn’t make a call to GrumPHP. Instead, we make a direct call to PHPUnit so that it can run the unit test suite for our plugin.

Testing different PHP versions

So why did we need our phpunit_job job reference? It’s because we need to run our unit test suite with different versions of PHP. CircleCI can’t do this using a build matrix like Travis CI does.

Because of this limitation with CircleCI, we have to create individual jobs for every PHP version that we want to test. This isn’t that problematic since, as you can see, we can leverage anchors and references to do it. Each unit test job then just has to say what PHP version we want to test.

That’s what we do in the docker section of each unit test job. We use it to specify the Docker image that we want to use. By default, we’re using circleci/php:7.2 which we defined in the container_config reference that we saw that earlier. Each unit test job will use a different image with a different PHP version.

Supporting older PHP versions

Now, an issue that every WordPress developer has to contend with is having to support old (if not ancient!) PHP versions. This is going to be a problem because almost all continuous integration services don’t support them. The only exception is Travis CI. (It’s another reason why it’s popular with WordPress developers.)

There’s a reason why all the continuous integration services don’t support old PHP versions. Like CircleCI, they’re all built using containers and Docker. But most of them use Docker’s official PHP image. And this image doesn’t support any PHP versions older than 5.6.

So, to work around this issue, we have to use a different Docker image than the one from CircleCI for older PHP versions. The one that I ended up using is vandries/phpenv. If you’d rather use another Docker image, feel free to!

Behat jobs

So what happens when our plugin’s unit tests pass for all the PHP versions that we want to test? Well, at that point, we want to run our plugin’s acceptance tests. Acceptance tests are the tests that we do to ensure that our plugin works with WordPress.

In PHP, the most popular framework for doing acceptance testing is Behat. Like our PHPUnit jobs, we’re also going to need run our acceptance tests multiple times. That said, we’re not trying to test different PHP versions like we did with the PHPUnit jobs.

Instead, we want to run our acceptance tests on different types of WordPress installations. Specifically, we want to run them on a regular single site installation of WordPress and on a multisite one. But if you’re a plugin shop, you might also want to test against specific older WordPress versions as well.

# .circleci/config.yml

references:
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

  attach_workspace: &attach_workspace
    attach_workspace:
      at: *workspace_root

  copy_vendor: &copy_vendor
    run:
      name: Copy vendor directory
      command: cp -R /tmp/vendor .

  behat_job: &behat_job
    <<: *container_config
    docker:
      - image: circleci/php:7.2
      - image: circleci/mysql:5.7
    steps:
      - checkout
      - run:
          name: Add WordPress host to hosts file
          command: echo "127.0.0.1 ${WP_HOST}" | sudo tee -a /etc/hosts
      - *attach_workspace
      - *copy_vendor
      - run:
          name: Install MySQL client
          command: sudo apt-get install mysql-client
      - run:
          name: Install MySQL PHP extension
          command: sudo docker-php-ext-install mysqli
      - run:
          name: Setup WordPress
          command: .circleci/setup-$WP_TYPE.sh
      - run:
          name: Start PHP server
          command: sudo php -S $WP_HOST:80 -t $WP_CORE_DIR
          background: True
      - run:
          name: Run Behat tests
          command: vendor/bin/behat --config .circleci/behat.yml --format progress --tags=$WP_TYPE

jobs:
  acceptance_singlesite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: singlesite

  acceptance_multisite:
    <<: *behat_job
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
      - WP_TYPE: multisite

So here are the relevant sections of the config.yml file for our acceptance tests. There’s even more going on now than there was in our unit test jobs. That’s because we need an actual WordPress site with our plugin installed to do acceptance testing.

Adding a MySQL database to our container

Like our phpunit_job reference, the behat_job reference is quite big. It contains the steps section that our two acceptance test jobs use. We also overwrite the docker section of our container_config reference.

What did we change in the docker section? We added the circleci/mysql image to it. This will allow our container to have both access to PHP and a MySQL database.

Behat job steps

The steps section of our behat_job reference is quite different from the other steps section that we’ve seen so far. There’s a lot of new steps that we haven’t seen so far. We’re going to go over all these new steps and why they’re there.

Adding WordPress host to hosts file

The first new step is right after the checkout step to check out our code. We add our WP_HOST environment variable to the hosts file our container. If you’re not familiar with what a hosts file is, it’s a special operating system file. You use it to map hostnames to IP addresses.

Why do we need to do that? It’s because the host defined WP_HOST isn’t real. There’s no DNS record for it anywhere. In fact, .test isn’t even allowed as a top-level domain.

So that’s why we need to add the WP_HOST value to the operating system’s hosts file. We need the operating system to think that this is a valid address. Otherwise, our Behat tests won’t work.

Installing MySQL dependencies

The next new steps are after our attach_workspace and copy_vendor reference steps. It’s related to the circleci/mysql image that we added to our Docker container configuration. But doing that doesn’t actually do much in practice.

The image adds a MySQL database to our Docker container. However, that doesn’t mean that it has the tools to use it. So that’s what we have to take care of before we can continue.

We’re going need to separate steps to add support for MySQL to our container. The first one is install_mysql_client which is a reference to a run step. This step uses apt command to install the mysql-client dependency.

mysql-client is the MySQL client library used by the Linux operating system. We need it because it’s a requirement for the mysqli PHP extension. This is the second step after copy_vendor called install_mysql_extension.

Meanwhile, we need the mysqli PHP extension for WP-CLI. Without the mysqli PHP extension, WP-CLI can’t set up WordPress for us. And that’s what we need to do next.

Installing PHP extensions is a bit different with Docker. You need to use the docker-php-ext-install helper script instead of following the regular installation instructions. You can read more about docker-php-ext-install and other helper scripts here.

Setting up WordPress

After installing our MySQL dependencies is where things start to diverge. At this point, we need to setup WordPress. That said, we need to do different things to setup WordPress as a single site installation vs a multisite one.

So how do we know if we want to install WordPress as a single site installation or as a multisite one? Well, we’re going to create an environment variable to tell us which type of installation that we want. We’ll call it WP_TYPE.

Our WordPress setup step uses the WP_TYPE environment variable to choose which install bash script to run. There’s one script for the single site installation and one for the multisite installation. Let’s take a look at them.

Single site installation

Let’s start with setting up WordPress as a single site installation. We do this using the setup-singlesite.sh bash script. You can see it below:

#!/usr/bin/env bash

if [[ -z "$CIRCLECI" ]]; then
    echo "This script can only be run by CircleCI. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_CORE_DIR" ]]; then
    echo "WordPress core directory isn't set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_HOST" ]]; then
    echo "WordPress host isn't set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then
    echo "WordPress.org plugin name not set. Aborting." 1>&2
    exit 1
fi

# Install WordPress
vendor/bin/wp config create --path="${WP_CORE_DIR}" --dbhost="127.0.0.1" --dbname="circle_test" --dbuser="root"
vendor/bin/wp core install --path="${WP_CORE_DIR}" --url="http://${WP_HOST}" --title="Passwords Evolved Test" --admin_user="admin" --admin_password="password" --admin_email="admin@example.com"
vendor/bin/wp rewrite structure --path="${WP_CORE_DIR}" '/%postname%/'

# Copy our plugin to WordPress directory
cp -r ./ ${WP_CORE_DIR}/wp-content/plugins/${WP_ORG_PLUGIN_NAME}

# Activate our plugin
vendor/bin/wp plugin activate --path="${WP_CORE_DIR}" ${WP_ORG_PLUGIN_NAME}

The setup-singlesite.sh bash script starts with a set of guard clauses. These are there to abort the script if certain requirements are missing.

The first requirement that we have is to ensure that we’re on the CircleCI environment. We do that by checking for the CIRCLECI environment variable is there. Then we check for the three environment variables that our script needs. Those are WP_CORE_DIR, WP_HOST and WP_ORG_PLUGIN_NAME.

We can break down the rest of the script into three sections. First, we want to actually install WordPress. Although the name is a bit misleading, our install-wp.sh didn’t do that. That’s because, as we just discussed, our CircleCI container doesn’t use the MySQL image by default.

But now we can, and we’re going to use WP-CLI to do it. Installing WordPress comes down to three commands. In order, they are:

  1. wp config create which creates the wp-config.php file for our WordPress installation.
  2. wp core install which does the actual installation of WordPress
  3. wp rewrite structure which we use to set a more standard permalink structure. (No one uses ?p= anymore!)

After that, we have to copy the plugin from the working directory to the WordPress plugins directory. We just use the standard cp to copy it. Once that’s done, we activate it using the wp plugin activate command. And we’re done!

Multisite installation

Next, we have the setup-multisite.sh bash script. This is the bash script used to set up a WordPress multisite installation. You can find it below:

#!/usr/bin/env bash

if [[ -z "$CIRCLECI" ]]; then
    echo "This script can only be run by CircleCI. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_CORE_DIR" ]]; then
    echo "WordPress core directory isn't set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_HOST" ]]; then
    echo "WordPress host isn't set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then
    echo "WordPress.org plugin name not set. Aborting." 1>&2
    exit 1
fi

# Install WordPress
vendor/bin/wp config create --path="${WP_CORE_DIR}" --dbhost="127.0.0.1" --dbname="circle_test" --dbuser="root"
vendor/bin/wp core multisite-install --path="${WP_CORE_DIR}" --url="http://${WP_HOST}" --title="Passwords Evolved Test" --admin_user="admin" --admin_password="password" --admin_email="admin@example.com"
vendor/bin/wp rewrite structure --path="${WP_CORE_DIR}" '/%postname%/'

# Copy our plugin to WordPress directory
cp -r ./ ${WP_CORE_DIR}/wp-content/plugins/${WP_ORG_PLUGIN_NAME}

# Activate our plugin
vendor/bin/wp plugin activate --network --path="${WP_CORE_DIR}" ${WP_ORG_PLUGIN_NAME}

We won’t dive into this script in as much detail as we did with the single site installation one. That’s because it’s almost the same as the other one. There are only two small changes that we made to it.

First, we don’t use the wp core install command to perform the WordPress installation. Instead, we use the wp core multisite-install. This is a command dedicated to the installing WordPress as a multisite installation.

The second change is to the wp plugin activate command. We added the --network option to it. This tells WP-CLI to activate the plugin across the entire multisite installation and not just a specific site.

It’s possible that activating your plugin using the --network option won’t change anything in practice. That said, in the case of this plugin, there’s a different admin page if the plugin is network activated. So doing this is important.

Deployment jobs

Once our acceptance tests pass, we can deploy the plugin to the WordPress plugin directory. This is the purpose of the last set of jobs that we’ll look at. We broke them down into two jobs due to the structure of the WordPress plugin directory SVN repository.

The first job is to deploy plugin assets to WordPress plugin directory. We need this job because plugin assets aren’t tied to our code. We might want to add or update our plugin assets without releasing a new version of the plugin.

The second job is to deploy the plugin itself. This job is a bit like a build job that we described earlier. We want to prepare a “WordPress directory” version of the plugin. Once done, we commit that new version to WordPress plugins directory SVN repository.

references:
  WP_CORE_DIR: &WP_CORE_DIR
    /tmp/wordpress
  WP_HOST: &WP_HOST
    passwords-evolved.test
  WP_ORG_PLUGIN_NAME: &WP_ORG_PLUGIN_NAME
    passwords-evolved
  WP_ORG_USERNAME: &WP_ORG_USERNAME
    carlalexander

  container_config: &container_config
    docker:
      - image: circleci/php:7.2
    environment:
      - WP_CORE_DIR: *WP_CORE_DIR
      - WP_HOST: *WP_HOST
      - WP_ORG_PLUGIN_NAME: *WP_ORG_PLUGIN_NAME
      - WP_ORG_USERNAME: *WP_ORG_USERNAME
    working_directory: ~/passwords-evolved

  workspace_root: &workspace_root
    /tmp

  attach_workspace: &attach_workspace
    attach_workspace:
      at: *workspace_root

  install_subversion: &install_subversion
    run:
      name: Install subversion
      command: sudo apt-get install subversion

jobs:
  deploy_assets:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy assets to WordPress plugin directory
          command: .circleci/deploy_assets.sh

  deploy_plugin:
    <<: *container_config
    steps:
      - checkout
      - *attach_workspace
      - *install_subversion
      - run:
          name: Deploy new version to WordPress plugin directory
          command: .circleci/deploy_plugin.sh

And this is the relevant sections of the config.yml file for our deployment jobs. There’s less to do at this point compared to what we saw for unit tests and acceptance tests. Each job has its bash script which does most of the work. We’ll discuss those next.

Installing subversion

But before doing that, let’s just go over the one new reference step in the steps section. It’s the install_subversion step. This is step is necessary for similar reasons as the ones we added to install MySQL for acceptance tests.

Our Docker container only has PHP installed and nothing else. But to deploy our assets or plugin, we need subversion. So we install it with this step.

Adding your WordPress.org password to CircleCI

At this point, our jobs need our WordPress.org password to commit changes to the plugin’s WordPress SVN repository. But you should never store a password (or any other sensitive data) in your git repository. This means that we need another way to store our WordPress.org password.

CircleCI lets you define environment variables for a project in the projet’s settings. Just follow the instructions here and add the WP_ORG_PASSWORD environment variable with your password. And then you’ll be ready to proceed!

Deployment bash scripts

As we said earlier, the rest of the deployment jobs revolves around bash scripts. We have one for each deployment job and they share some things in common. That said, the plugin deployment script is more involved due to a couple of factors that we’ll discuss.

Deploying assets

We’re going to start with the bash script to deploy our plugin assets since it’s the simplest of the two scripts. Plugin assets are things like your plugin logo and the banner image for the WordPress plugin directory. We also mentioned earlier that we don’t want to tie those to our code and releases. But also, plugin assets don’t get stored in the same place in our plugin’s SVN repository.

#!/usr/bin/env bash

if [[ -z "$CIRCLECI" ]]; then
    echo "This script can only be run by CircleCI. Aborting." 1>&2
    exit 1
fi

if [[ -z "$CIRCLE_BRANCH" || "$CIRCLE_BRANCH" != "master" ]]; then
    echo "Build branch is required and must be 'master' branch. Stopping deployment." 1>&2
    exit 0
fi

if [[ -z "$WP_ORG_PASSWORD" ]]; then
    echo "WordPress.org password not set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then
    echo "WordPress.org plugin name not set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_USERNAME" ]]; then
    echo "WordPress.org username not set. Aborting." 1>&2
    exit 1
fi

PLUGIN_SVN_PATH="/tmp/svn"

# Checkout the SVN repo
svn co -q "http://svn.wp-plugins.org/$WP_ORG_PLUGIN_NAME" $PLUGIN_SVN_PATH

# Delete the assets directory
rm -rf $PLUGIN_SVN_PATH/assets

# Copy our plugin assets as the new assets directory
cp -r ./assets $PLUGIN_SVN_PATH/assets

# Move to SVN directory
cd $PLUGIN_SVN_PATH

# Add new files to SVN
svn stat | grep '^?' | awk '{print $2}' | xargs -I x svn add x@

# Remove deleted files from SVN
svn stat | grep '^!' | awk '{print $2}' | xargs -I x svn rm --force x@

# Commit to SVN
svn ci --no-auth-cache --username $WP_ORG_USERNAME --password $WP_ORG_PASSWORD -m "Deploy new assets"

You can see our bash script for deploying assets above. It starts similarly to the setup bash scripts that we had for our acceptance test jobs. We have a bunch of guard clauses checking for various environment variables.

It’s worth noting that there’s one guard clause that’s a bit different from what we saw in the other bash scripts. It’s the CIRCLE_BRANCH guard clause. Not only does it check if the variable is there, but it also checks that its value is master. That’s because we only want our deploy scripts to run on the master branch and not other branches such as pull requests.

Once we’re through the guard clauses, we start by checking out the plugin from the SVN repository. We define the location that we want to check out our plugin with the PLUGIN_SVN_PATH variable. We’ll reuse this variable throughout the script.

Once the plugin checked out, we immediately delete the assets directory from it. The assets directory is the directory that the WordPress plugin directory wants you to put your assets in. It always uses the current files in that directory so you can’t create different versions of it.

That’s why we want to delete it. We then copy the assets directory of the plugin that we checked out from GitHub. This creates a new assets directory in our plugin that we checked out from SVN. We can use this new assets directory to track the changes that we made to it.

Commiting changes to SVN

The next two commands (if we can call them that!) are a series of four piped commands. They both start by running the svn stat command. This prints the current status of all the files in the checked out SVN repository.

After that, both commands differ. Both use grep, but the first one checks for new files and the other one for missing files. Those missing files are the files that we removed in our commit.

We then pass the results of both grep commands to the same awk command. AWK is a programming language, and the awk runs whatever program you pass it. The {print $2} prints out the paths of the files that we added or removed.

These file paths are then piped into two different xargs commands. One runs svn add on all the files that we added in our commit. And the other runs svn rm --force on all the files that we removed.

Once we’ve added and removed files using these two sets of piped commands, we can commit these changes. We do that using the svn ci command. We tell it not to cache our credentials using the --no-auth-cache flag. And then we pass it our WordPress.org username and password manually using the WP_ORG_USERNAME and WP_ORG_PASSWORD environment variables.

It’s worth noting that, if the two sets of piped commands didn’t add or remove files, the svn ci command wouldn’t do anything. So you don’t have to worry that this will make commits every time CircleCI runs this job. The command only does something if there are changes to push to the repository.

Deploying new versions of the plugin

Now that we’ve covered deploying assets to the WordPress plugin directory, we can move on to the harder scenario. That’s deploying a new version of our plugin. Here’s the bash script that does this:

#!/usr/bin/env bash

if [[ -z "$CIRCLECI" ]]; then
    echo "This script can only be run by CircleCI. Aborting." 1>&2
    exit 1
fi

if [[ -z "$CIRCLE_BRANCH" || "$CIRCLE_BRANCH" != "master" ]]; then
    echo "Build branch is required and must be 'master' branch. Stopping deployment." 1>&2
    exit 0
fi

if [[ -z "$WP_ORG_PASSWORD" ]]; then
    echo "WordPress.org password not set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_PLUGIN_NAME" ]]; then
    echo "WordPress.org plugin name not set. Aborting." 1>&2
    exit 1
fi

if [[ -z "$WP_ORG_USERNAME" ]]; then
    echo "WordPress.org username not set. Aborting." 1>&2
    exit 1
fi

PLUGIN_BUILD_DIRECTORIES=(lib resources src)
PLUGIN_BUILD_FILES=(index.php LICENSE passwords-evolved.php pluggable.php readme.txt)
PLUGIN_BUILD_PATH="/tmp/build"
PLUGIN_SVN_PATH="/tmp/svn"

# Figure out the most recent git tag
LATEST_GIT_TAG=$(git describe --tags `git rev-list --tags --max-count=1`)
# Remove the "v" at the beginning of the git tag
LATEST_SVN_TAG=${LATEST_GIT_TAG:1}

# Check if the latest SVN tag exists already
TAG=$(svn ls "https://plugins.svn.wordpress.org/$WP_ORG_PLUGIN_NAME/tags/$LATEST_SVN_TAG")
error=$?
if [ $error == 0 ]; then
    # Tag exists, don't deploy
    echo "Latest tag ($LATEST_SVN_TAG) already exists on the WordPress directory. No deployment needed!"
    exit 0
fi

# Checkout the git tag
git checkout tags/$LATEST_GIT_TAG

# Create the build directory
mkdir $PLUGIN_BUILD_PATH

# Copy plugin directories to the build directory
for DIRECTORY in "${PLUGIN_BUILD_DIRECTORIES[@]}"; do
    cp -r $DIRECTORY $PLUGIN_BUILD_PATH/$DIRECTORY
done

# Copy plugin files to the build directory
for FILE in "${PLUGIN_BUILD_FILES[@]}"; do
    cp $FILE $PLUGIN_BUILD_PATH/$FILE
done

# Checkout the SVN repo
svn co -q "http://svn.wp-plugins.org/$WP_ORG_PLUGIN_NAME" $PLUGIN_SVN_PATH

# Move to SVN directory
cd $PLUGIN_SVN_PATH

# Delete the trunk directory
rm -rf ./trunk

# Copy our new version of the plugin as the new trunk directory
cp -r $PLUGIN_BUILD_PATH ./trunk

# Copy our new version of the plugin into new version tag directory
cp -r $PLUGIN_BUILD_PATH ./tags/$LATEST_SVN_TAG

# Add new files to SVN
svn stat | grep '^?' | awk '{print $2}' | xargs -I x svn add x@

# Remove deleted files from SVN
svn stat | grep '^!' | awk '{print $2}' | xargs -I x svn rm --force x@

# Commit to SVN
svn ci --no-auth-cache --username $WP_ORG_USERNAME --password $WP_ORG_PASSWORD -m "Deploy version $LATEST_SVN_TAG"

This is a more elaborate version of the assets deployment bash script that we just saw. The beginning of the bash script starts the same way as the assets one. We have guard clauses checking for the presence of various environment variables. We also ensure that we only run this bash script on the master branch.

Do we need a new version of the plugin?

Once we’re done with the default guard clauses, we move on to another new and more complex guard clause. We need to determine if we have to deploy a new version of the plugin. We do that by comparing the latest tag in our git repository to the latest tag in our SVN repository.

To get the latest git tag, we use the following command: git describe --tags `git rev-list --tags --max-count=1`. We store this tag value in the LATEST_GIT_TAG variable. The step after that is optional.

That’s because the plugin uses a tagging format that leads the semantic version number with a v. We can’t have that for our SVN tags. So we create a LATEST_SVN_TAG variable where we truncate the v off the LATEST_GIT_TAG value. But, if you don’t do your git tags that way, you don’t need to do this step and can just use the LATEST_GIT_TAG value.

Once we have our LATEST_SVN_TAG value, we want to use it to check for that tag on the WordPress plugin directory SVN repository. We do that by using the svn ls command. This command is pretty much the same as the UNIX ls command. It lists all the files and directories in the target directory in the SVN repository.

If there’s no directory for our LATEST_SVN_TAG, the svn ls command will return an error. We store this error in the error variable. We will use it to determine if we continue or not.

If there wasn’t an error, it means our LATEST_SVN_TAG does exist. We don’t need to do anything else, and we exit with 0. (This is a bit counter-intuitive, but 0 means that command ran successfully.) Otherwise, an error indicates that our LATEST_SVN_TAG doesn’t exist and we need to create it, so we continue.

Building the directory version of the plugin

At this point, we’re sure that we need to build a new version of the plugin. The first step is to check out the LATEST_GIT_TAG version of the plugin. We do that using the git checkout command.

Why not just use the current version that CircleCI checked out? This is a safety step in case the code that CircleCI checked out wasn’t the same as the LATEST_GIT_TAG version. With this step, we know that whatever happens, we’re working with the version of the code that we want to deploy to the WordPress plugins directory.

After that, we start building the directory version of the plugin. We create the build directory using the mkdir. We use PLUGIN_BUILD_PATH variable that we defined earlier as the directory name.

Once we have our build directory, we start the build process. We copy all the directories and files that we want our WordPress directory version of our plugin to have. We defined those in the PLUGIN_BUILD_DIRECTORIES and PLUGIN_BUILD_FILES variables.

We copy these directories and files using a bash for loop. The loop for directories copies directory using the cp -r command. The -r is a modifier that tells the cp command to recursively copy all the files from the target. To copy the files, we just use a normal cp command without the -r modifier.

Commiting our new version to SVN

The last part of the deploy-plugin.sh bash script focuses on committing the new version of the plugin that we just built. This part is almost the same as the one that we used to commit our plugin assets to WordPress plugins directory SVN repository. The differences between both are quite minor.

First, we’re not working from the assets directory. Instead, we’re using the trunk directory. But like the assets directory, we start by deleting everything in it.

We then copy our new version of the plugin to replace the deleted trunk directory. We also copy our new version of the plugin to the tags directory. The tags directory is how SVN tracks new versions of our plugin. That’s why we copy it to the tags directory using the LATEST_SVN_TAG variable for the subdirectory.

The rest of the script focuses on committing our changes. For this, we reuse the same commands that we used in the deploy-assets.sh bash script. The only small change is in the commit message where it mentions the version that we’re deploying.

Building our workflow

So this wraps up the definition of our jobs in CircleCI. That said, we still need to put these jobs together into the continuous deployment workflow that we saw at the beginning. (That was like ages ago!) This is a lot less complicated than what we just did to create all the CircleCI jobs.

workflows:
  version: 2
  build_test_deploy:
    jobs:
      - build
      - deploy_assets:
          filters:
            branches:
              only:
                - master
      - code_quality:
          requires:
            - build
      - test_php72:
          requires:
            - code_quality
      - test_php71:
          requires:
            - code_quality
      - test_php70:
          requires:
            - code_quality
      - test_php56:
          requires:
            - code_quality
      - test_php55:
          requires:
            - code_quality
      - test_php54:
          requires:
            - code_quality
      - acceptance_singlesite:
          requires:
            - test_php72
            - test_php71
            - test_php70
            - test_php56
            - test_php55
            - test_php54
      - acceptance_multisite:
          requires:
            - test_php72
            - test_php71
            - test_php70
            - test_php56
            - test_php55
            - test_php54
      - deploy_plugin:
          filters:
            branches:
              only:
                - master
          requires:
            - acceptance_singlesite
            - acceptance_multisite

This is the section of the config.yml file that configures our workflow. The first thing to notice is that it’s under its own section named workflows. It’s not under the jobs section that we were working on earlier.

Under the workflows section, we have two elements. version is a required element. The documentation doesn’t explain why it’s necessary.

The other element under the workflows section is build_test_deploy. This is the name that we gave to our workflow. You could name it whatever you want.

You can also have as many workflows as you want. If you run more complicated setups than this one, you could have a workflow for nightly builds and pull request testing for example. But, for us, one is plenty, and we’ll take care of special cases right in our workflow configuration.

Orchestrating our workflow

Speaking of that, let’s go over our workflow configuration and see how we set things up. How to configure a workflow is pretty simple. We just list all our jobs using the names that we gave them in jobs section of the config.yml file.

What’s more important is what configuration information we put under each job. The only job without any is the build job. And that’s because it’s the job at the beginning of our workflow.

After that, every job will have the requires section except for the deploy_assets job. (We’ll go over why that is in a bit.) The requires section is how we configure the orchestration of our jobs with CircleCI. If you’re not familiar with the idea of orchestration, it’s just a way to automate the ordering of our jobs. This allows CircleCI to run them in the most optimal way possible.

CircleCI wants to do this because it supports running jobs in parallel. This is something that we want our configuration to take advantage of as well. That’s because CircleCI gives open source projects 4 containers to use. (And my plugin is open source.)

Configuring our job dependencies

Ok, so let’s go back to the requires section. We use it to tell CircleCI which job(s) does the job that we’re configuring depend on. That’s also why the build job doesn’t have a requires section. As the first job in our workflow, it doesn’t depend on anything. (The same is true for the deploy_assets job as well but it’s for a different reason.)

The next job that we want to run after the build job is the code_quality job. So that job has the build job in its requires section. All pretty simple so far.

Fanning out and fanning in

Things get more complicated with our unit testing jobs. We don’t care in what order these jobs get run as. In fact, we can even run them in parallel since they don’t depend on each other.

CircleCI calls this type of workflow a fan-out. To configure a fan-out workflow, we just need to put the code_quality job as the dependency of all our unit test jobs. That’s why they all have the code_quality job in their requires section.

After all our unit test jobs have run successfully, we want to run our two acceptance test jobs. To do that, we have to do a fan-in. This is the opposite of a fan-out.

A fan-in tells CircleCI that it must complete all our dependent jobs before it can proceed. To do a fan-in, we have to put all our unit test jobs in the requires section of our acceptance test jobs. This will tell CircleCI that it can only start our acceptance test jobs once it finishes running all the unit test jobs.

We then need to do the same for the deploy_plugin job. We want to only run it once CircleCI has finished running both our acceptance test jobs. So we put both jobs under the requires section so that CircleCI performs a fan-in once both jobs finish running.

Only deploying on the master branch

The last thing that we want to do is add some extra security for our deployment jobs. Our bash scripts already have guard clauses to prevent bad things from happening. That said, there’s nothing wrong with some extra precautions!

That’s why both our deployment jobs have an extra filters section. This section tells CircleCI to only run these jobs when the branch or tag matches a certain condition. In our config, that condition is that the commit is on the master branch.

The deploy_assets job

There’s one job that stands out a bit in our entire workflow. It’s the deploy_assets job which we’ve alluded a few times so far this section. Everything about this job is a bit special, and the way we integrate it into our workflow isn’t any different.

First, the deploy_assets job is like our build job. It doesn’t depend on any other job. But it’s also different from the build job in that it’s not a dependency for any other job either. It’s a job that’s standalone.

This goes back to what we said earlier about the deploy_assets job. It’s a job that’s there due to the structure of the WordPress directory SVN repository. WordPress wants plugin assets stored in a different location than our code.

On top of that, there’s no reason to run our entire workflow for our plugin assets. Plugin assets aren’t code. We don’t need to perform unit tests or acceptance tests on them.

We just need them deployed to the WordPress plugin directory when we update them. That’s something that we want to be able to do at any time. The only restriction that we have is on the branch that we want this job to run on.

In this way, the deploy_assets job is like the deploy_plugin job. We only want it to run on commits in the master branch. After all, you wouldn’t want a commit in a pull request or some other branch to update your assets in the WordPress plugin directory! That’s why the deploy_assets job has a filters section.

And that’s it!

Whew, that was quite the journey! But this is everything that you need to know to continuously deploy a plugin to the WordPress directory. It’s worth pointing out again that, while we did this for a plugin, you can also use this with a theme.

With a setup like this, you don’t have to worry about deploying your new plugin versions to the WordPress directory. You can instead focus on coding new features, writing tests and just tag a new version when you’re ready to go. That’s the magic of continuous integration!

Special thanks to Iain Poulson who wrote about doing the same thing with Travis CI.

# # # # # #

© 2024 Carl Alexander

This site runs without a server thanks to image/svg+xml

Creative Commons License