TDD: Test Driven Development

One problem I have with the two big TDD sets out there for PHP (SimpleTest and PHPUnit) was the incompatibility of working in the setup I am working in. SimpleTest apparently wanted to take over error control, something my environment was already handling, and PHPUnit didn't have a web interface by default. Finally, both wanted to do things their way. None of this is bad, but working in a shop were Unit Testing isn't the norm at the moment (though momentum exists for the change to take place), I didn't have time to play around with getting them setup properly. But I wanted to test.

So I did what any programmer would do in my place: I started building my own simple testing sweet. Testing doesn't have to be complicated. It doesn't have to involved massive amounts of setup and tear down. In reality, at the very heart of it, just getting some form of tests started is part of the path toward redemption.

My current testing platform is very, very simple. It consists of a file that displays all the tests available to run in a directory (or packages of tests in sub-directories of the main test folder). You click a link and it runs the appropriate test. It then prints out a simple page that shows the number of successful tests.

Here is a completed testing screen. As we can see, it ran the test, and seven tests failed, and four passed fine. We can review the test below inside the testing screen, saving us even more time. But all this is fairly useless beyond that. We know quickly that the test failed. Now we have to fix those tests. This gives us a list of things we have to fix, a 'todo' list of things that need to be addressed. But, what exactly do we need to fix?

We can click on the text right above the test results and it will give us this information.

Now you can see where the fun comes in.  Clicking the text expands the details (using jQuery), and now we can see all tests that failed and passed, where they are by line number, and what they received in testing, and what they expected, as well as why the test failed.  We can now trace the errors to each line of code, so we know exactly what code caused this to fail, or more precisely, what test failed.  The same works for the tests that passed.  Of course, even this isn't as efficient as it could be.  Granted, this provides us with enough information to work with, and in some cases, it's all the information you need.  Information overload is something I wanted to avoid.  I don't need to see more than this, except sometimes I do.

Clicking on the top Failed or Passed boxes will expand all the associated tests to further details (again, using jQuery).  We can also click on tests to open them up one by one, allowing us to view only what we want.  This lets us see what the offending code was that caused the failure without having to go and look for it.  This is most useful when we have an unexpected error.

Getting to this point is quick and easy.  Drilling down doesn't require any real effort, and we review things as we go along.  We see the failures, we open them, and we allow ourselves the opportunity to review all the errors without forcing ourselves to become overcome with too much information.  We can focus on what we need to work on at the moment.

Finally, at the top of the page, the name of the test is also set to rerun the test.  This is done on purpose because this testing page sits inside a frame.  I use the frame as a type of bookmark for in-house development tools and sites, frequently used links that I don't want to put in any one browser.  After all, I'm generally working inside Firefox, IE, Chrome, and Safari, usually at least two of the four.

There are some problems with my setup.  The packaging of tests is really dirty at the moment.  Every tests needs to exist in a certain directory.  Each tests also needs to start with 'test_', and a name like 'test_AlwaysFail.php' turns into 'AlwaysFail.'  'test_Always_fail.php' would be run as the test 'Always fail.'   But this also means each test file is self-contained, and considered on test.  I've setup a way to run all the tests at once, but it's cumbersome, and doesn't work like it should: quickly, easily, and efficiently.

I also need a way for tests to better document themselves, and a way to automate this feature.

Tests also run when you click on the initial testing link.  While this makes testing easy, testing isn't always about just running the test.  There should be someway for me to allow for a launcher page, a way to go to the test page and let me do something if I want, even if it's as simple as reviewing the tests that will run.

I should be able to run multiple test pages together in a certain order.  Setting up a test is important, but I want to test that setup, as usually it's using the tools I want to test in the first place to do the setup.  At the same time, I want setting up test requirements to be easy.  This is where the above idea of having a launcher page might be best.  Allow me to go through a series of tests step by step, or all at once.

The general theme here is focusing on using tests as development tools, not just as error checking devices.  Of course, being able to run all the tests and get an easy to use report is critical.

I don't imagine for a second my testing platform is as mature as any of the other ones out there now.  However, I didn't start out with the intent of building one.  I just needed a simple way to run tests.  Then I needed a way to report tests.  As time dragged on, I would add one feature, and then another, and keep adding as I needed them.  I didn't spend time developing this, it just evolved.

This all isn't to suggest that SimpleTest or PHPUnit are bad.  I'm using them as a reference for some of the ideas I'm having.  But I always focus on how to take what they do, and make it fit with what I want.