Dominique Stender A blog about my thoughts and experiences in Information Technology

HOWTO: Continuous Integration for PHP, pt. 3

This is the third article in my series about continuous integration for PHP. You might want to read the first and second article prior to this.

In this article I want to introduce you to the way I run the continuous integration tests, what to run when, and where. If you're unsure how to write unittests, this will not be covered in this article but phpunit.de has an excellent documentation section just covering how to write tests. Another section is there covering how to write tests for SeleniumRC.

Let me start this article with a short summary of what we have installed in the second part: We have a pretty standard LAMP server based on Ubuntu Linux. Then there is PHPUnit for unit and regression testing as well as phing, our integration server. We added SeleniumRC as a frontend test tool, then Xvfb and Firefox to facilitate the frontend tests. xdebug is used by PHPUnit for code coverage reports and phpDocumentor will be used by phing for automatic APIdoc generation.

A Subversion client to fetch latest updates out of the repository completes the setup.

Ok, let's dig a little deeper into the filesystem structure and how to use it, shall we?

We installed three virtual hosts in apache, one for development, one for the integration and one as staging system.

The whole setup is merely to demonstrate the workflow. In reality the integration and staging environment would most likely run on a separate machine, or two.

Each virtual host contains the same initial directory structure, like you see on the left. The image shows the structure of the default server (used for development).

You see the ./conf folder, which stores the Apache configuration file. You can add your own configuration files to this folder if you like.

The ./htdocs folder is what you think it is, the webserver root.

The ./tests and ./buildScripts folders will contain the test classes, as well as all files required by phing to build the project.

This layout is simple, yet effective. It cleanly separates the utilities used for testing and continuous integration from the rest of the project, thus making packaging and deployment comparatively easy.

In the screenshot you also see that the whole default virtual host is under revision control. Experience shows that in the end, you will want everything versioned no matter what you think initially.

Last not least the screenshot shows the other two virtual hosts 'integration' and 'demo', both being under version control as well - but more on that later.

Ok, are you ready to write some tests?

Some of the scripts discussed below are quite long, so I will not post them here in the blog in their full length. You can get a continuous integration test file setup (.zip file) for ease of use.

We start with a simple file which we put in default/htdocs/class.hello.php:

<?php
 class HelloWorld {
   public function hello() {
     return 'Hello World';
   } // end: function hello()
 } // end: class HelloWorld
?>

As you see the class HelloWorld contains a single method 'hello' without any parameters that returns the famous 'Hello World'. The corresponding unittest is equaly simple. We create a second file in default/tests/Test.HelloWorld.php:

<?php
require_once('PHPUnit/Framework.php');
include_once('../htdocs/class.hello.php');

class TestHelloWorld extends PHPUnit_Framework_TestCase {
  private $ciTestObj = null;

  public function setUp() {
    $this->ciTestObj = new HelloWorld();
  } // end: function setUp()

  public function testHello() {
    $this->assertEquals($this->ciTestObj->hello(), 'Hello World');
  } // end: function testHello()
} // end: class TestHelloWorld
?>

Now, the property $ciTestObj contains an instance of the test object which is our HelloWorld class from the first script. The only test we perform is an assertEquals() call that compares the return value of the HelloWorld::hello() method with the static string 'Hello World'.

Save the file and connect to the continuous integration server using your favorite SSH client. Change into the tests directory and execute phpunit:

phpunit Test.HelloWorld.php

You should get a result similar to this:

PHPUnit 3.4.2 by Sebastian Bergmann.
.
Time: 1 second
OK (1 test, 1 assertion)

Hooray, the test passed. Our class behaves as expected.

The second step will be to run a frontend test in SeleniumRC.

Usually you would create a test in Firefox with SeleniumIDE set to write PHPUnit test files instead of the html default. However for a quick success I recommend copy-pasting the test that I prepared.

Before we attempt to run SeleniumRC in the continuous integration server, configure your SSH client to allow X forwarding. If it is currently inactive, enable it. The test won't run without that. Double check your settings are correct.

Ok, once more we start with our test subject. This time it is a very simple html page with a form. The form doesn't do anything except filling the form fields with the values you entered before submit. Create a file default/htdocs/form.php with the following content:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
<head>
  <title>Testable form</title>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  <style type="text/css">
    body {background-color: #fff;}
    label {
      display: block;
      float: left;
      min-width: 140px;
      padding-right: 5px;
    }
    input,textarea {
      border: 1px inset;
      margin: 5px 0;
    }
    textarea {height: 150px;}
    input[type=submit] {border: 1px outset;}
   .long {width: 300px;}
  </style>
</head>
<body>
  <h2>Selenium RC test form</h2>
  <p>
    This form does have absolutelly no purpose other than being a test subject for selenium RC.
  </p>
  <form action="<?php echo $PHP_SELF; ?>" method="post">
    <label for="firstname">firstname, lastname: *</label><input type="text" name="firstname" value="<?php echo $_POST['firstname']; ?>" /> <input type="text" name="lastname" value="<?php echo $_POST['lastname']; ?>" /><br />
    <label for="email">email: *</label><input type="text" name="email" value="<?php echo $_POST['email']; ?>" /><br />
    <label for="url">homepage:</label><input type="text" name="url" value="<?php echo $_POST['url']; ?>" /><br />
    <label for="subject">subject: *</label><input type="text" name="subject" value="<?php echo $_POST['subject']; ?>" /><br />
    <label for="comment">your message: *</label><textarea name="comment"><?php echo $_POST['comment']; ?></textarea><br />
    <label for="">&nbsp;</label><input type="submit" value="submit" />
  </form>
</body>
</html>

Open it in your browser, just to confirm that it is working. If you followed this howto step by step, http://integration.local/form.php should work just fine. You should do the same from the command line on the server to see if local access is also correct:

lynx http://integration.local/form.php

If that is not working, go back to the second part of this series and check the settings in the hosts files. However if it does, we can write our test for the subject. Open a new file in default/tests/TestForm.php and paste this:

<?php
require_once('PHPUnit/Extensions/SeleniumTestCase.php');

class TestForm extends PHPUnit_Extensions_SeleniumTestCase {
  protected $captureScreenshotOnFailure = TRUE;
  protected $screenshotUrl = 'http://integration.local/reports/selenium';

  function setUp() {
    $this->screenshotPath = $_ENV['PWD'] . '/../reports/selenium';
    $this->setBrowser('*firefox');
    $this->setBrowserUrl('http://integration.local/');
  } // end: function setUp()

  function testFormSubmit() {
    $this->open('/form.php');
    $this->waitForPageToLoad('30000');
    $this->type('firstname', 'SeleniumRC');
    $this->type('lastname', 'Testautomation');
    $this->type('email', 'selenium@example.com');
    $this->type('url', 'http://seleniumhq.com');
    $this->type('subject', 'Test for Selenium RC');
    $this->type('comment', 'This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.');

    $this->click('//input[@value="submit"]');
    $this->waitForPageToLoad('30000');
    $this->assertEquals('SeleniumRC', $this->getValue('firstname'));
    $this->assertEquals('Testautomation', $this->getValue('lastname'));
    $this->assertEquals('selenium@example.com', $this->getValue('email'));
    $this->assertEquals('http://seleniumhq.com', $this->getValue('url'));
    $this->assertEquals('Test for Selenium RC', $this->getValue('subject'));
    $this->assertEquals('This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.', $this->getValue('comment'));
  } // end: function testFormSubmit()
} //end: class TestForm
?>

Note that before 30th of November 2009 the above code contained the wrong hostnames for both TestFor::screenShotUrl and the setBrowserUrl() call. Sorry for this! The .zip file has been corrected, as well.

This PHPUnit test will start a firefox browser on the server, hit the form.php on the server and fill out the form. It will submit the form and compare the values in the form fields with those entered previously with assertions.

To check this we run PHPUnit again. Since this is a frontend test it required Xvfb (our X server) and SeleniumRC to run.

Xvfb :1 -screen 0 1280x1024x24 &
env DISPLAY=:1 java -jar /var/www/SeleniumRC/selenium-server-1.0.1/selenium-server.jar  &

Ignore all errors and warnings that Xvfb will probably throw for now - those will be interesting only if the tests fail. It takes some time for both daemons to start up, but after a bit of patience we can  run PHPUnit:

phpunit TestForm.php

The first thing you will notice is that it takes some time before PHPUnit is doing anything - that is while firefox starts - but then it is cluttering the screen with output (actually most of it doesn't come from PHPUnit but from SeleniumRC but you get the point).

The important part is that the output ends with something like this:

.
Time: 24 seconds
OK (1 test, 6 assertions)

You already know what this means... the test passed, the form works as expected. Just to clarify once again: We just tested a website in Firefox by writing and executing a PHPUnit test class.

Now you will want to stop both Xvfb and SeleniumRC in order to continue:

kill -TERM %2
kill -TERM %1

Step three: Automate all tests with phing.

Now that the PHPUnit tests run independently it is time for some more automation with phing, our integration server. Check out the test.xml file in the buildScripts folder contained in the .zip file that contains the test setup. It is over 100 lines long so I will not post it here. If you're familiar with the file structure of a phing build script, you will see that the file consists of a couple of targets which will implement the following workflow:

1. Removing a htdocs/report folder (more on that in a bit)
   <target name="prepare">
2. Prepare a (temporary) xdebug code coverage database
   <target name="coveragePrepare">
3. Start Xvfb and SeleniumRC, just like we did above
   <target name="setUp">
4. Generate an APIdoc using phpDocumentor
   <target name="doc">
5. Running all PHPUnit tests inside the tests folder
   <target name="unittest">
6. Generating the code coverage report
   <target name="coverageReport">
7. Shutting down Xvfb and SeleniumRC again
   (inside the "main" target)
8. In some condition (which we discuss in a bit) the build script will send a mail notification that the build tests have failed.

But first let's do the fun thing, run the script. Once piece of advice though: You might want to go to line no 23 of the test.xml script and set your own mail Id. While running the script in this constellation the mail will not be used but better cover this now than wonder why mails are not reaching you later.

So go ahead... change into the default/buildScripts directory and execute

phing -f test.xml

Phing will read in the .xml file and perform all the tasks mentioned above in that order. Once the script has finished, check the htdocs folder. Now there is a reports subfolder that hasn't been there before. This is where all test results are stored. To be specific:

1. Your PHPUnit test results

   http://integration.local/reports/phpunit/

2. A code coverage report

   http://integration.local/reports/coverage/

3. PHP APIDoc

   http://integration.local/reports/apidocs/

There is an additional folder http://integration.local/reports/selenium/ which contains the STDOUT and STDERR logs generated by SeleniumRC but that is really only of interest if something goes wrong in the execution of the frontend tests.

Open each of the URLs above in your browser to see what you got. I believe it is pretty impressive.

Final note on the text.xml file.

To avoid this article getting even longer, I'll cut things short. The test.xml phing build script is meant to be run by the developer, and manually so. It will only work in the 'default' virtual host unless you change the properties on top. But please keep them as they are for now, as in the fourth article of this series I'll explain how to use another phing build file to automate all things once and for all.

The idea behind the test.xml build file is to make it easy for the developer to get a thorough overview of his working copy - without the need to commit everything just to see what is breaking.

I can only motivate you to try and play around with things. One cool detail for example is that PHPUnit will automatically make a screenshot from the website in firefox whenever a SeleniumRC test assertion fails. So break one assertion and visit the phpunit report file in your browser and you will find a report on what failed along with a URL (sadly it is not linked).

This concludes the third part of the continuous integration for PHP series. Come back in a couple of days for details on how to automate the build process in the integration virtual host.