Acceptance Testing for Drupal Sites

If you've ever been asked to apply security updates or upgrade a module on a Drupal site you didn't build, you may have relied on the client to "click around and make sure everything works as expected." Or you may have had to click around yourself long after you built a site, not quite remembering what it was supposed to do and thinking that there must be a better way. And there is — Behat.

Behat is a PHP implementation of Ruby's Cucumber testing framework. Behat supports Behavior Driven Development (BDD), a programming philosophy which aims to improve developer understanding of client business needs with plain language descriptions of site behavior. Behat uses Mink, the PHP equivalent of Ruby's Capybara, as the acceptance testing framework that turns the plain language descriptions into actual tests.

Mink provides a single API to simulate the interaction between a web browser and our Drupal site using any of the following tools:

• Goutte, a web crawler which returns output similar to wget. It's fast, but it doesn't render CSS or javascript.
• Zombie.js, which tests client-side javascript in a simulated environment without a browser. It's also quite fast, but there's still no CSS involved in the interaction.
• Selenium or Sahi, which will open and drive different browsers — a more time-consuming process but one which includes CSS and javascript both and most closely resembles the full user experience.

By using a common API, you choose how you want to run an entire suite of tests, an individual feature, or even a single scenario — so you can achieve an optimal balance of speed and thoroughness with a single code base.

Behat forces us to pay attention to the business value of the features we implement, creating documentation of how the site is supposed to behave, without worrying about the technical details behind that behavior. Then, Mink takes that documentation and does the clicking on our behalf. Even for shops that do not embrace BDD, Behat and Mink solve an immediate need for site builders: efficiently ensuring that sites continue to work as expected after security or module updates.

Getting Started

The example in this article focuses on the core Drupal 7 Contact module which has a link called "Contact us" available on the homepage.

The example in this article is based on the following:

• PHP 5.3.1 or later
• Behat — for writing features and scenarios
  http://behat.org
• Mink — (plus its Goutte and Selenium drivers) for testing the steps
  http://mink.behat.org
• MinkExtension — for a built-in library of pre-defined steps
  http://extensions.behat.org/mink/
• Selenium2 — for running browser-driven tests
  http://seleniumhq.org/docs/03_webdriver.html

Behat installation instructions describe how to install Behat, Mink, and MinkExtension together. To install Selenium2, download and run the .jar file as described in the first two bullet points in the Getting Started section of the Selenium php-webdriver documentation. Don't worry about the third bullet point on opening sessions, Mink will handle that for you.

Defining the Base URL: behat.yml

Behat is configured via a 'behat.yml' file, which tells Behat that Mink is installed and has some information about the drivers we might want to use. The most important setting is the 'base_url', which tells Mink which domain we'll be testing on. This domain could be public or just something accessible from your local machine.

behat.yml:

default:
  context:
  extensions:
    Behat\MinkExtension\Extension:
      base_url:  'http://localhost'
      goutte: ~
      selenium2: ~

Features

Next, we describe the feature we plan to test. Each feature is stored in its own file, which begins with a non-technical description of the feature itself. Feature files are written using the Gherkin syntax, a line-oriented language structured by indentation. The skeleton of a feature looks like:

Feature: Terse description of the feature
  In order to (accomplish something of value)
  As a (specific role using the site)
  I want/need (to do something)

The first sentence establishes the business value for the feature, the second identifies who will benefit from that value, and the third describes what that person must be able to do. Our example starts by defining a feature for a contact form:

features/contact_form.feature

Feature: Contact form
  In order to ask questions and provide feedback
    As a site visitor
    I need to use the contact form

The feature provides guidance for developers before anything else is defined, as well as plain language documentation once the feature is implemented.

Note: Feature files (just the first 4 lines without any scenarios) can form the backlog for your project and even be used for estimations. This is kind of a cool side effect — you could plan a project out just by creating all of your features (without scenarios) up front. Since each feature defines its value and who enjoys that value, you can prioritize things by what's most important at each stage or possibly by the most important user at any stage (e.g. frontend anonymous users versus admin user).

Scenarios

For any given feature, there will be one or more scenarios describing what the person will do in a particular situation. Scenarios are at the heart of the Gherkin syntax and begin with the word Scenario: followed by each step involved in the scenario. Steps must start with one of the keywords: Given, When, Then, But, or And. Each should describe — in the non-technical language of the user — how he or she will actually interact with your site. And while each step will be used as part of automated testing, the best scenarios are those that are true to describing the user's experience.

Two scenarios for the Drupal contact form include:

  Scenario: Submits feedback with all required fields
  Scenario: Submits feedback without supplying all required fields

Next, we add the steps detailing how the user interacts with this feature. Steps follow the pattern:

    Given: Put the situation in a known state
    When: Describe an action taken by a user
    Then: Describe the outcome of the action

The first scenario could look like:

features/contact_form.feature

Scenario: Submits feedback when required fields are filled out
  Given I am on "/"
  When I follow "Contact us"
    And I fill in "Your name" with "Test User"
    And I fill in "Your e-mail address" with "visitor@example.com"
    And I fill in "Subject" with "Great new site"
    And I fill in "Message" with "I especially liked the animated gif"
    And I press "Send message"
  Then I should see "Your message has been sent."

Thanks to the MinkExtension, we can run this automatically. Now. Without writing any code. Because most of what users do with a website consists of a finite set of actions — clicking, visiting, pressing, uploading, seeing things — a great deal of what we do can be re-used. To prove it, we'll try running our test scenario with Behat by running the following command:

$ bin/behat features/contact_form.feature

Behat output shown in figure at right, click to enlarge.

When I manually followed these steps in Firefox, I definitely saw "Your message has been sent," but the test failed. Why? This test runs in Goutte by default and, in a text-based browser, no javascript message is presented to the user. The text is not seen.

Refining Steps in the Example Scenario

Using Selenium Webdriver to Run the Test

To force this scenario to run in a browser, we'll add the tag @javascript. This invokes the Selenium Webdriver (which must be running), which opens a browser and clicks through the steps. It's helpful, not only when javascript is involved, but also to gain visibility into how the interaction is playing out. Again, we'll run Behat with the following command:

$ bin/behat features/contact_form.feature

Behat output is shown in figure at right, click to enlarge.

Identifying Page Elements

I particularly appreciate the flexibility Behat and Mink provide for identifying elements on a web page. In the above example, where the client is filling in form fields, there's a choice for how to identify the various form fields.

For example, looking at the HTML from the Drupal contact form:

<div class="form-item" id="edit-mail-wrapper">
  <label for="edit-mail">Your e-mail address: 
     <span class="form-required" title="This field is required.">*</span>
  </label>
  <input type="text" maxlength="255" name="mail" id="edit-mail" 
  size="60" value="seven@bucky.l" class="form-text required" />

I can choose to supply the label, name, or id value of the form element. Any of the following will match:

    And I fill in "visitor@example.com" for "Your e-mail address"
    And I fill in "visitor@example.com" for "mail"
    And I fill in "visitor@example.com" for "#edit-mail"

I can implement more specific matching, of course, but in this case the label is unique on the page, and working with what the client sees (instead of a value visible only in the HTML code), allows the tests to be theme-independent.

A lot of folks I know are familiar with acceptance testing primarily via the Selenium recorder, and they have expressed frustration with how brittle the tests are. Quite often, they tell me, tests fail not because something is actually broken on the site but because markup was changed by an additional module or in an update.

Focusing on what's visible to the user generally helps avoid that sort of breakage, and if a module changes the language presented to users, then breakage should happen, since the change may be significant enough to require re-training. Testing what's visible also means it's immediately clear what is different; the test no longer requires reading HTML, Xpaths, or PHP to update, and writing about what's visible encourages those all-important client conversations.

Making Steps More Readable

The longer the series of steps are, the less likely our clients are to engage them, so we want to strive to keep them short and readable. Behat gives us a way to do that, as shown in the following example in the file features/contact_form.feature:

@javascript

Scenario: Successfully submit feedback when required fields are filled out
  Given I am on "/"
  When I follow "Contact us"
     And I fill in the following:
     |Your name            |Test User                            |
     |Your e-mail address  |visitor@example.com                  |
     |Subject              |Great new site                       |
     |Message              |I especially liked the animated gif! |
    And I press "Send message"
  Then I should see "Your message has been sent"

We've created a table that lists the field label in the left-hand column (or as described above, this could be the id or name of the field). The corresponding values to fill in are in the right-hand column. By removing the redundant step language the fill-in steps are more easily scanned and more readable.

Writing Your Own Step Definitions

When we write Given I am on "/", we are supplying the path to the homepage. We can execute this as a test because it has already been defined by the MinkExtension in the file vendor/behat/mink-extension/src/Behat/MinkExtension/Context/MinkContext.php:

    /**
     * Opens specified page.
     *
     * @Given /^(?:|I )am on "(?P[^"]+)"$/
     */
    public function visit($page)
    {
        $this->getSession()->visit($this->locatePath($page));
    }

The regular expression in the comment makes the magic happen, linking the natural language step in the feature file with the code which will test the site. Behat matches the literal string "am on", takes what follows in quotation marks, and supplies that value to the visit() function.

We can write our own step definition to take readability a step further. Using a "/" isn't a natural way to refer to the homepage, so we'll use this as an opportunity to implement our own step definition. Clients may be uncomfortable referring to pages by their path in general, but I opt to keep the reusability of substitution in all cases except the homepage. First, we'll change the language in the feature file to say: Given I am on the homepage, and run the test. Behat will provide helpful scaffolding for implementing the step definition.

You can implement step definitions for undefined steps with these snippets:

    /**
     * @Given /^I am on the homepage$/
     */
    public function iAmOnTheHomepage()
    {
        throw new PendingException();
    }

Next, we create the FeatureContext file, features/bootstrap/FeatureContext.php to hold new step definitions:

<?php
use Behat\Symfony2Extension\Context\KernelAwareInterface;
use Behat\MinkExtension\Context\MinkContext;
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;

require_once 'vendor/autoload.php';

class FeatureContext extends MinkContext
{
    /** @BeforeFeature */
    public static function prepareForTheFeature() {
    // clean database or do other preparation stuff
    }

    /**
    * @Given /^I am on the homepage$/
    */
    public function iAmOnTheHomepage() {
       $this->getSession()->visit($this->locatePath('/'));
    }
}
?>

This step matches the literal text "I am on the homepage" and supplies a fixed value of "/". The homepage is defined once, here, and can be updated in a single place should the value change. The step is more readable and can be reused in any scenario which begins on the homepage. It's just a very small example of how steps are written and defined. We'll run Behat again with the following command in order to see our new homepage definition in use:

$ bin/behat features/contact_form.feature

Behat output is shown in figure at right, click to enlarge.

The potential for Behat and Mink in the Drupal community is outstanding.

• We can better understand what our clients need.
• We can create living documentation of what a site is supposed to do.
• We expand the roles able to participate in creating acceptance tests.
• We can test that sites continue to behave, after security updates and module upgrades.
• We can support each other by sharing what and how we test.

Interested in learning more? Check out the BDD project for Drupal.org and Behat Integration for Drupal