Tiki Unit Testing

This page describes how to write and run unit tests for Tiki, using phpunit. These tests are quite different from the kinds of tests that can be done with TikiTest.

Unit tests, Integration tests and GUI level tests

In Tiki, we have three kinds of automated tests. This pages refers to the first and second kind #Unit tests and #Integration tests. For more information on GUI level tests, see Tiki Testing with Selenium.

Unit tests

Unit Tests check the internals of certain classes, by exercising them in isolation from other classes and other infrastructure (eg: apache, mysql). This is what phpunit is especially good for. These tests run REALLY fast (the whole suite can be executed in less than a minute). Note however that those tests are only applicable to classes that are modular enough to be exercised in isolation from everything else. Generally speaking, that means the classes defined in lib/core.

Integration tests

Like #Unit tests, integration tests check the internals of the system. The main difference is that they test one component in the context of the whole system, and, in particular, may need the Tiki DB in order to run. They are slower than Unit tests, but still faster than GUI level tests.

GUI level tests

These tests check the system by exercising it much in the same way that an end user might. This can be done either with TikiTests or Selenium. This kind of test tends to be very slow and the whole suite may take several hours to run. This type of test is not covered by this page. For more details, see Tiki Testing with Selenium.

Installing and configuring

Installing phpunit

To install phpunit, run the following command lines from a shell window:

pear channel-discover pear.phpunit.de
pear install phpunit/PHPUnit

Once phpunit is installed, you may also need to modify your php.ini file to change the include_path variable. You need to add the path of the directory that is just above the PHPUnit directory. For example, on my EasyPHP installation, the PHPUnit directory was created under:

c:\Program Files\EasyPHP 3.0\php\PEAR

so, I appended "c:\Program Files\EasyPHP 3.0\php\PEAR" to the include path as follows:

include_path = ".;c:\Program Files\EasyPHP 3.0\php\includes;c:\Program Files\EasyPHP 3.0\php\PEAR"

You also need to change the following two variables in your php.ini file to:

extension_dir = "C:\Program Files\EasyPHP 3.0\php\ext"

and

upload_tmp_dir = "C:\Program Files\EasyPHP 3.0\tmp"

You may also need to add the phphunit to your path.

Configuring environment variables

Some of the tests (those who restore the Tiki DB to start states) need to invoke mysql command lines. For this to work, you must make sure that the mysql command is part of your PATH.

Configuring a new database

First you have to create a new database. After you have to copy the file lib/test/local.php.dist to lib/test/local.php and edit it changing the default values to the values that match your environment.

The first time you run the test suite it will automatically populate the database. On subsequent runs it will update the database if needed.

Running existing tests

Once you have installed phpunit, you can run all the existing unit tests that have been written for Tiki, as follows.

in Tiki>=8, you need to have a copy of local.php in the directory lib/test

To run all the unit and integration tests:

cd lib/test
phpunit . # with the dot at the end

You may also exclude some tests like the GoogleTranslate tests, or the GUI level tests groups.

• GoogleTranslate tests don't work if you need to specify a web proxy, because they get some data from Google translation tool.
• #GUI level tests require a selenium installation and a configuration file for selenium. You may not have this available.

So, if you don't want to run one of those groups, you just have to use the --exclude-group option, like this:

cd lib/test
phpunit --include-path ../.. --exclude-group gui,GoogleTranslate

Note that this assumes that a phpunit @group directive has been inserted in the source file of those tests.

You can also execute only tests whose names match a certain expression. For example:

# Will only run tests whose name contains string Importer
phpunit --filter Importer .

Sometimes, phpunit fails and does not provide you with sufficient details to allow you to know exactly where the error occured (for example, sometimes it doesn't even tell you in which TestCase the error happened). You can get more details by using the --verbose option. For example:

phpunit --verbose .

To run the test only on the parser for example

phpunit core/WikiParser/

Writing new tests

Adding a new test function to an existing TestCase

If there is already a TestCase class (i.e. a class whose name ends with Test somewhere under lib/test) where your new test might fit in, then all you have to do is to create a new method of that class whose name starts with "test". For example:

public function test_HelloWorld() {
   $this->fail("Forcing failure to see if this TestCase is actually loaded");
}

When you run the tests again, then you should see that test_HelloWorld was run and failed. Just change the content of that test so that it implements the actual tests.

Adding a new TestCase

If there isn't already a TestCase class where your new test might fit in, then you need to create a new TestCase class. This is done slightly differently for the different types of tests.

Let's start with an example for a unit test. Say you want to create a class DummyUnitTest and want to put it under lib/core/Foo/Bar. All you need is to create afile lib/core/Foo/Bar/DummyUnitTest.php with the following content:

<?php
/*
 * @group unit
 */

class Foo_Bar_DummyUnitTest extends TikiTestCase
{
   public function test_HelloWorld() {
      $this->fail("Forcing failure to see if this TestCase is actually loaded");
   }
}

?>

Note:

• Foo_Bar_ prefix for the name of the class
  • This allows phpunit to automatically know that this class is located in a directory Foo/Bar located somewhere on the include path.
• The class name and the file ends with Test
• @group unit
  • This allows exclusion of unit tests using the --exclude-group option of phpunit

Creating a TestCase for an integration test is very similar, except that:

• You would put the file in a subdirectory of lib, instead of lib/core
• Set @group to 'integration' instead of 'unit'

For creation of #GUI level tests, details will follow. For the meantime, if you need help figuring out how to do this, ask Alain Désilets (alain.desilets@nrc-cnrc.gc.ca) or Marta Stojanovic (marta.stojanovic@nrc-cnrc.gc.ca).

Creating a new GUI level test

Details will follow. For the meantime, if you need help figuring out how to do this, ask Alain Désilets (alain.desilets@nrc-cnrc.gc.ca) or Marta Stojanovic (marta.stojanovic@nrc-cnrc.gc.ca).

Troubleshooting

phpunit hangs up

If you invoke phpunit and nothing at all happens, it's probably because your XAMP services are not started. Although you are running the tests from a command line, some of the tests may actually require some services like SQL to run.

Fatal error: Allowed memory size of NNN bytes exhausted

Running thousands of tests tends to consume much more memory than your average web PHP script, especially if you use a version of PHP that does not have garbage collection.

So you might get the above error. The way to address that is to increase your the memory_limit in your php.ini file.

Note that you have to make sure you use the proper php.ini file (there are often more than one on a given computer). To find out which php.ini file is used when you run phpunit, do this in a shell window:

php -i | grep ini

The file mentioned after "Loaded Configuration File =>" is the one you want. Just edit the file and increase the memory limit. If this is a development machine, then you can increase it all you want, but be careful if you are testing on a machine that is also used to deploy a Tiki site, and if the php.ini used by phpunit is also the one used by Apache, then don't increase it too much.

Tried to run an acceptance test without an initial database dump

If you get this error, it means you tried to run some #GUI level tests but are not properly setup to run them. See Tiki Testing with Selenium#Database_Issues? for details.

In order to run #GUI level tests, you must have snapshots of databases pre-configured for different tests. If you do not have those snapshots, the tests will not run, and all other tests will not run either.

A way around this problem is simply to exclude the GUI level tests using the --exclude-group option:

phpunit --exclude-group gui .

Failing tests for Multilingual_MachineTranslation_GoogleTranslateWrapperTest

This is probably due to the fact that you are using a web proxy. For some reason, the Google translation tool does not work from behind a proxy.

If you are running the tests behind a proxy, you can exclude those tests as follows:

phpunit --exclude-group GoogleTranslate .