Dominique Stender A blog about my thoughts and experiences in Information Technology

HOWTO: Continuous Integration for PHP, pt. 2

This is the second in the series of articles on continuous integration for PHP. You might want to start reading the first article, covering the basic tools in the setup before you continue here.

Here I will give you a step-by-step guide how to set up a LAMP server with all the necessary tools to run a thorough set of tests and reports.

As of this writing Ubuntu Linux version 9.10 (Karmic Koala) has just been released. Personally I like Ubuntu for its ease of use, so this howto will focus on that. I assume that the general setup will be slightly different if you use another flavor of Linux, though.

If you don't have a spare computer lying around just yet, install VMWare Server. That will get you going well enough to convince your manager that a few hundred bucks for a medium-spec server will be well invested money.

This step of the tutorial assumes that you're at least somewhat familiar with the Linux command line. If you are not, do yourself a favor and get in touch with one of your system administrators. Make him understand what you want and convince him to divert from the typical company guidelines for server installation. You might want to show him this:

This will be a test server. It will not run in production. It will never be accessible by the public. It'll be a proof of concept and will be replaced by a server compliant to all guidelines once this proof of concept is approved by management.

For those of you who do this on their own: The above paragraph is for you. All aspects of creating a secure, well administered server with backup are ignored in this tutorial. Do not attempt to run this box online.

One more comment. Even if you're convinced that you'll not be using one of the packages or reports described below, just install them for now. The whole setup is well under 2GB and it works. Removing a package might break things.

This being said, are you ready for the installation?

Step one: Set up a LAMP Server.

Go to the Ubuntu Homepage and download the .iso CD image of 9.10 Karmic Koala. I grabbed the 32-bit version but I assume the 64-bit will run just fine. VMWare users: You can directly mount the .iso image as CD in the virtual machine. No need to burn a disc.

Follow the graphical installer, it is pretty straight forward. Write down the user name for the unprivileged user you will be asked to create during the process, we will need it later.

When you reach the package selection, check 'LAMP', 'OpenSSH' and 'Samba', you will need it.

Now go for lunch, the download of the additional packages will take a bit.

Step two: Basic setup.

You should set up your OpenSSH public key authorization for root and one unprivileged user, any bash aliases that you like as well as any Samba shares that you require. Since I can't know your requirements, I won't even try to give a one-size-fits-all description. Push your question in the comments below and I will answer asap if I can.

You will want one Samba share pointing to where the apache environments will be:

vi /etc/samba/smb.conf

Activate the line reading

security = user

Append this section to the end of the file.

#---- start webserver vhost main directory
comment = webserver main directory
read only = no
create mask = 0664
directory mask = 0775
force group = www-data
#---- end webserver vhost main directory

After that, restart Samba

/etc/init.d/samba restart

Step three: Install basic packages.

A few essentials are missing after the initial installation of Ubuntu. For example we need to install quite a few additional PHP packages as well as Subversion (my personal favorite in revision control).

Make sure you're logged in as root user.

apt-get install php5 php5-dev php5-cli php5-curl php5-gd php5-mysql
apt get install php5-sqlite php5-xmlrpc php5-imagick php5-mcrypt
apt-get install php-pear php5-xsl php5-xdebug
apt-get install subversion
apt-get install zip unzip mailutils

I assume that you might need additional PHP packages. Feel free to add them. However the list above is what we'll need for continuous integration.

Step four: Install continuous integration software.

This is where the fun starts. Luckily next to everything we need can be installed thru the handy PEAR installer, so the complexity is down to a minimum.

pear channel-discover pear.phpunit.de
pear channel-discover pear.phing.info
pecl install xdebug
pear install phpunit/PHPUnit-3.4.2
pear install Image_GraphViz PhpDocumentor
pear install http://pear.phing.info/get/phing-2.4.0RC2.tgz
pear install channel://pear.php.net/VersionControl_SVN-0.3.3
pear install channel://pear.php.net/XML_Serializer-0.20.0
pear install PEAR_PackageFileManager_Plugins PEAR_PackageFileManager

Please note that there is a death-grip like dependency between PHPUnit and phing. phing before v2.4 does not run with PHPUnit 3.4.x because one class was replaced in the latter. On the other hand, PHPUnit before version 3.4.x is not able to perform screenshots for failing Selenium tests, something that you'll definitely want if you plan to automate your frontend testing as well.

You might want to check the phing website if 2.4.0 final is released by the time you read this. I didn't run into any difficulties with the RC2 of phing though.

Since almost everything is based on PEAR, we need to adapt the include_path directive in the php.ini files:

Do a

vi /etc/php5/apache2/php.ini

and a

vi /etc/php5/cli/php.ini

and in both files find the line

include_path = ".:/usr/share/php"

and remove the comment.

Step five: Install SeleniumRC.

SeleniumRC or Selenium Remote Control is a solution to automate front end testing without the need for a display. For its installation, follow me:

apt-get install openjdk-6-jre-headless xvfb lynx firefox imagemagick

wget -O /var/www/seleniumRC.zip http://release.seleniumhq.org/selenium-remote-control/1.0.1/selenium-remote-control-1.0.1-dist.zip
unzip /var/www/seleniumRC.zip -d /var/www/
mv /var/www/selenium-remote-control-1.0.1 /var/www/SeleniumRC
rm /var/www/seleniumRC.zip

We now have the SeleniumRC server (a java program) installed in /var/www/SeleniumRC. Also, we installed the Xvfb virtual frame buffer X server and Firefox 3.5, the former enabling the latter to run without an attached monitor, mouse or keyboard.

Step six: Create the 'dev' environment.

As I described in the first article, I typically use three different environments for continuous integration. A 'dev' environment for the developer, an 'int' environment where the continuous integration takes place as well as a third 'stage' environment to show the client the latest working revision.

In order to try out continuous integration we will simply install all three virtual hosts on one and the same machine.

With Ubuntu the default host for apache is in /var/www. While that is nice and well, I thrive for a little more consistency, hence we begin by moving the default host to its new home for its purpose as the 'dev' environment.

mkdir -p /var/www/vhosts/default/conf
mkdir /var/www/vhosts/default/htdocs
mkdir /var/www/vhosts/default/tests
mkdir /var/www/vhosts/default/buildScripts

mv /var/www/index.html /var/www/vhosts/default/htdocs/.

mv /etc/apache2/sites-available/default /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default /etc/apache2/sites-available/default

mv /etc/apache2/sites-available/default-ssl /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default-ssl /etc/apache2/sites-available/default-ssl

vi /var/www/vhosts/default/conf/default

Now change the DocumentRoot directive to

/var/www/vhosts/default/htdocs

and also change

<Directory /var/www>

to

<Directory /var/www/vhosts/default/htdocs>

Step seven: Create the 'int' and 'stage' environments.

The setup for both the integration and staging environments is almost identical to the 'dev' setup. We start with 'int':

mkdir -p /var/www/vhosts/integration/conf
mkdir /var/www/vhosts/integration/htdocs
mkdir /var/www/vhosts/integration/tests
mkdir /var/www/vhosts/integration/buildScripts

cp /var/www/vhosts/default/conf/default /var/www/vhosts/integration/conf/vhost.conf
ln -s /var/www/vhosts/integration/conf/vhost.conf /etc/apache2/sites-available/integration.conf
ln -s /etc/apache2/sites-available/integration.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/integration/conf/vhost.conf

Now add

ServerName    integrate.integration.local

and once more change the DocumentRoot directive, this time to

/var/www/vhosts/integration/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/integration/htdocs>

As said, for the staging environment it is almost the same:

mkdir -p /var/www/vhosts/demo/conf
mkdir /var/www/vhosts/demo/htdocs
mkdir /var/www/vhosts/demo/tests
mkdir /var/www/vhosts/demo/buildScripts

# create virtual host conf file
cp /var/www/vhosts/default/conf/default /var/www/vhosts/demo/conf/vhost.conf
ln -s /var/www/vhosts/demo/conf/vhost.conf /etc/apache2/sites-available/demo.conf
ln -s /etc/apache2/sites-available/demo.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/demo/conf/vhost.conf

Now add

ServerName    demo.integration.local

and one last time change the DocumentRoot directive, this time to

/var/www/vhosts/demo/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/demo/htdocs>

Step eight: Setting permissions, restarting Apache

We simply want to correct the file system permissions so that both the unprivileged user and the webserver have write permissions everywhere. You do remember that I said this server will not be secure, right?

Do you remember the user name for the unprivileged user you sat up during the installation procedure? Fill in that user name below

chown -R YOUR_USERNAME:www-data /var/www
chmod -R g+w /var/www

We need to restart apache in order to 'activate' the three virtual hosts we just created in step seven.

/etc/init.d/apache2 restart

Last step: Making the host names known.

You may have wondered about the unusual host names, ending in .local. Well you could name them whatever you want but I prefer .local personally because that makes it absolutely clear that this is a non public machine.

In order to make them accessible you may ask your system administrators to configure your local DNS zone. Or you simply add the host names to the hosts file of your development machine (probably Windows or Mac) and to the Ubuntu server we're currently configuring. For the latter one all you need to do is

vi /etc/hosts

and append this section

127.0.0.1    integration.local
127.0.0.1    integrate.integration.local
127.0.0.1    demo.integration.local

Note that on your Windows machine you will probably want to do the same, but with the IP address of the virtual machine. Your hosts file for Windows usually sits in c:\WINDOWS\system32\drivers\etc

Alright, that's it!

You've just set up the whole continuous integration environment.

Before I let you wander through the (required, really) manuals and user guides of phing, PHPUnit, Selenium ([1], [2]) and what not, bare with me for a few more moments though.

Let us just summarize what we did.

We've set up an Ubuntu Linux server that runs on your computer, right along with whatever operating system you use to see this website.

That server has Apache, PHP, MySQL, Samba and OpenSSH and thus can be considered a full fledged LAMP server. However it is highly insecure.

We installed a PHP build tool (phing), a regression testing tool (PHPUnit), Xdebug for code coverage. We can generate API docs on the fly (phpDocumentor) and we a graphical environment (Xvfb) and Firefox for frontend tests with Selenium.

All three Apache environments run on the same machine which is different what I mentioned to be my ideal setup in the first article of this series. However, for the purpose of this demo this is perfectly alright. If you want to go into production with continuous integration, simply repeat all the steps above for your two (or three) machines, once for each environment. You can and should leave out the Samba configuration for your integration and staging system however. Oh and get back to your system admin, thank him for his work so far and kindly ask him to secure those boxes!

Okay that concludes the second part in this series. Come back here in a bit to continue with the third part: Automating tests and reports with phing.