$pdo = $this->getMockBuilder('PDO')
    ->disableOriginalConstructor()
    ->getMock();

This will work on another class, but with PDO you will get this error:

PDOException: You cannot serialize or unserialize PDO instances

The solution

My solution for this problem is to create a class that derive from PDO. This class has only an empty constructor. Then you can mock this class, making sure that you don’t disable the original constructor. This way the mocked object can be passed to the code to test even if this code does type hinting.

class PDOMock extends \PDO {
    public function __construct() {}
}

class PDOTest extends \PHPUnit_Framework_TestCase {
    public function setup() {
        $pdo = $this->getMockBuilder('PDOMock')
            ->getMock();
    }
}

True North PHP

True North PHP is a new conference that will be held in the Toronto area. It will be on November 2nd and 3rd. I’m very excited as I was chosen to speak. I will give an introduction to Continuous Integration. If you’re curious about CI, come see me.

They managed to get an amazing line up of speakers. And at $200, the tickets are really affordable.

Confoo

Confoo will have its fourth edition from February 27 to March 1st of 2013. Confoo is always great and the next edition promise to be as good as the previous ones. I will not speak there, but I will attend it.

If you’re in the Montreal region, buy your ticket, you won’t regret it.

Reasons to Change

One of the problems I had with watchr, is that it did not see new files. Every time I added a test file, I had to switch window and restart watchr. Then I would add the class to test and have to do the same thing. It’s not a big deal, but I’m lazy. And the main reason to use a tool like this is to have the test runs automatically.

Another concerns, is that watchr is not maintained anymore. The latest commit on GitHub was done over a year ago. And the last time there where any real activity is 2 years ago. There is a couple pull requests waiting, but some of them have been there for a year.

And lastly, Guard comes built in with some functionality that required code in watchr, and others are part of additional Guards.

Guard

Like watchr, Guard watch your file system and trigger some actions when files are modified. It’s also build in Ruby. There are many plugins that will simplify frequent tasks. One of those is guard-phpunit that runs PHPUnit when PHP files are changed. Guard runs only the needed tests when you change a file. It will run all the tests only when you ask for it, or after a failing test finally succeed. It also has a very good notification system.

To install Guard, you need to have ruby already installed. On Ubuntu, you can install it like this

sudo apt-get install ruby1.9

Once you have Ruby installed, you just need to run

gem install guard guard-phpunit

to install Guard with the PHPUnit plugin. If you don’t use RVM to manage your Ruby versions, you will need to run gem install with sudo.

That’s all there is to it. You are now ready to use Guard.

The Guardfile

The Guardfile is where you tell Guard which files to watch, and what it should do when a file is modified. You can generate one with all the plugins installed by running

guard init

If you want to have a file only for one plugin, or add the code for another plugin, just run

guard init phpunit

You can also generate the Guardfile manually, it’s a simple Ruby file. My Guard file for PHPUnit looks like this:

guard 'phpunit', :cli => '--colors', :tests_path => 'tests', 
        :keep_failed => true, :all_after_pass => true do
  watch(%r{^tests/.+Test\.php$})
  watch(%r{^src/(.+)\.php$}) { |m| "tests/#{m[1]}Test.php" }
end

The first line, tells guard to use the PHPUnit plugin. This plugin will execute PHPUnit and check if the tests pass or not. We give it a few options. First we tell guard to pass the colors argument to PHPUnit. keep_failed is to make sure that when a test case fails, guard will run it on every save until it passes. all_after_pass, tells Guard to run all the tests once a failing tests succeed. This way you are sure you didn’t break anything in the process of getting the test to pass.

This file has two calls to watch, the first one makes Guard watch the tests folder and run PHPUnit on any file modified in it.

The second one watches the src folder. When a file is modified in this folder, it makes Guard run the associated file in the tests folder. For this to work, I just have to keep the structure of my src and tests folders identical.

Running Guard

Running Guard is done by simply issuing the ‘guard’ command in your project folder. It will look for the Guard file in the current directory. If there are none, it will look in your home directory for a file called ‘.Guardfile’.

In addition to the ‘.Guardfile’, you can use a file called ‘.guard.rb’, also in your home directory. .Guardfile will be use if you don’t have a Guardfile in the current directory. .guard.rb is appended to your Guardfile. This is useful for configurations that you don’t want to share between developers on the same project. Notifications preferences are a good example of what can go in .guard.rb.

One interesting parameter to Guard is -c or –clear. If you run it with this parameter, Guard will clear the terminal before running the tests.

Running Guard

Notifications

Guard has a comprehensive notification system. It uses Libnotify on Linux, Growl on Mac and Notifu on Windows. To use it on Linux, make sure you have libnotify-bin and the libnotify gem installed.

sudo apt-get install libnotify-bin
gem install libnotify

You can then add this line to your Guardfile

notification :libnotify

Notifications

If you want to turn them off, change the notification value to :off.

notification :off

Other Plugins

There are a lot of plugins for Guard. Here’s a small list of the ones I found interesting:

• guard-coffeescript – Compile your CoffeeScript
• guard-jasmine – Run Jasmine tests
• guard-less – Compile less into css
• guard-sass – Complie sass into css
• guard-puppet – Run Puppet
• guard-remote-sync – rsync your files when you change them
• guard-shell – Run any shell command
• guard-livereload – Reload your browser when you change your views

Inline Guard

When there are no plugin to perform a task you need, you can use guard-shell. Another alternative is to create an inline guard. Just add a class that extends Guard to your Guardfile. In the class you need to at a minimum override the method run_on_change. Here’s what I did for running Behat.

module ::Guard
    class Behat < Guard
        def start 
            run_all
        end

        def run_all
            puts 'Run all Behat tests'
            puts `behat`
        end

        def run_on_change(paths)
            paths.each do |file|
                puts `behat #{file}`
            end
        end
    end
end

guard 'behat' do
    watch %r{^tests/integrationTests/.+\.feature$}
end

This runs Behat every time a .feature file is changed.

I'm working on a plugin for Behat. There is not much yet, it just runs Behat on file changes. It won't parse the results or display notifications yet. I still have a lot of work, but it can be useful as it is now. It's on my GitHub.

Give It a Try

Continuous testing has been a great addition to my toolbox. It took me some time to find the correct tools for me, but now, Guard is making my life easier. If you do Test Driven Development (TDD), you should try it.

If you have some better tools, or improvements I can make to the way I use Guard, please let me know in a comment.

I use Google Reader to subscribe to a lot of blogs, probably too many. I also use twitter to find out what’s happening. I follow people who are very active in the PHP, agile and software craftsmanship communities. When I see something interesting, I add it to Instapaper, so I can read it when I have time.

Aggregators

Aggregators are great, by following a few, you can quickly get a good view of the community.

The main one for PHP is without any doubt PHPDeveloper. This aggregator is maintained by Chris Cornutt. You can check the website or the Twitter feed, but don’t miss this one. Everyday, you will get links to 4 or 5 of the best articles published in the PHP world. You will also get weekly reminder of the best posts of the week and of what was popular a year ago.

Here’s a few other aggregator:

• Zend Developer Zone
• DZone Web Builder Zone
• PHP Quick Fix
• WebSec Quick Fix
• PHP Web Developer

Blogs

Those are some of the PHP blogs I follow:

• Cal Evans blog
• Rob Allen’s DevNotes
• Elizabeth Naramore’s blog
• Lorna Jane’s blog
• Matthew Weier O’Phinney project lead of Zend Framework
• Sebastian Bergmann the creator of PHPUnit
• Fabien Potencier founder Symfony
• 8th Light’s blog, not PHP but interesting

Twitter

• Ilia Alshanetsky
• Day Camp 4 Developers
• Derick Rethans
• Cal Evans
• Rasmus Lerdorf
• PragmaticProgrammers
• Alison Green great career advices
• Chris Cornutt
• Giorgio Sironi
• Chris Hartjes
• #linktuesday
• Uncle Bob Martin

Podcasts And Screencasts

• Voices of the ElePHPant Interviews with the PHP community
• /dev/hell The Development Hell Podcast
• Clean Coders

User Groups

I attempt a few user groups here in Montreal. I go to PHP Québec every months and to JS-Montreal, Montreal.rb and OWASP Montreal every time I can. You probably have some user groups in you region. Check them out. They are great for learning and making contacts with other developers.

Conferences

Conferences are more expensive, but they are really worth it. Here in Montreal, we have ConFoo that is amazing. There is also Agile Tour that is coming in November. This year, I will also be at the first edition of True North PHP in Toronto.

Conferences are a good way to be introduce to many technologies. You get back home with lots of new ideas. And a lot of things to research.

PHP Mentoring

PHP Mentoring is a new project that connects mentors with apprentices. I joined the program as soon as I heard about it and I’m very happy I did. I found two apprentices and a mentor in a few days. This is a great way to improve and learn a lot.

If you haven’t looked at it, I suggest you go read the guidelines and rules and put your name on the signup page on GitHub. It does not matter if you do it as a mentor, an apprentice or both. I’m sure it will help you a lot.

When your name is there, look for someone who matches what you are looking for in the names already there and contact him. I see a lot of names there, and not enough matches.

Those are a few of the resources I check to stay current. There are a lot more in my feed reader and in the list of people I follow on Twitter. But I think those are the most informative when it comes to PHP development.

If you think I missed some good ones, please leave a comment. I’m always happy to find more sources of great content.

Autotest

I discovered continuous testing over a year ago when I watched a video where Corey Haines performed a kata in front of a crowd. He was doing it in Ruby and using Autotest to run his test suite. I tough it was awesome that he immediately saw the result of his changes without having to do anything.

Back then I had used PHPsrc. It’s a great tool, but I didn’t like that I had to take my hand off the keyboard and click on a button to run my tests. So I was mainly running them from the command line.

AutoPHPUnit

When I saw Autotest, I was immediately sold and I looked for something similar in PHP. Sadly I didn’t found anything like it. So after a while I wrote something for myself. I called it AutoPHPUnit. It uses libnotify to watch the file system. Every time a file is changed, it runs PHPUnit. You can also specify a configuration file for PHPUnit.

It worked well for me, I really loved using it. But it was not very flexible.

Watchr

Then I read the book Continuous Testing: with Ruby, Rails, and JavaScript (affiliate link) that I won at the Code Retreat in Quebec city.

In the book, the authors use watchr to monitor the file system and perform any actions when a file changes. Watchr can be used to monitor any kind of file, so it’s more flexible than my solution. And you tell it what to do with a Ruby block so it can react to changes any way you want.

To install it, you need to have Ruby already installed on your machine. Then you use RubyGems to install it. Simply run “gem install watchr” and you will have watchr on your machine. If you are not using RVM to manage your versions of Ruby, you might need to use sudo to install watchr.

You can then create a simple Ruby script. Calling the function watch() with a regexp for the files you want to watch and the block you want to execute when a file changes. Then, you run watchr passing it the script file as a parameter.

Here’s a simple script to run PHPUnit:


watch ('.*\.php$') {|phpFile| system("phpunit -c phpunit.xml")}


I watch every file that ends with ‘.php’. When watchr detects a change, it simply runs PHPUnit for me.

watchr running

Notifications

Using watchr like this is great if you have enough screen space to show a terminal beside your code. But if you can’t have a terminal on the screen at the same time as your editor, you can use notify-send on Ubuntu to get a pop up in the notification section.

watch ('.*\.php$') {|phpFile| run_php_unit(phpFile)} 

def run_php_unit(modified_file)
    system('clear')
    if (system("phpunit -c phpunit.xml")) 
        system("notify-send 'All test passed'")
    else
        system("notify-send 'Test failed'")
    end
end

You can add any logic you want. In the Continuous Testing book they go further. You can have it display notifications only when the tests fails and when they pass again for the first time. They also count the numbers of successful runs and display notifications every 5 consecutive successful run.

watchr notification

Beyond PHP

The main advantage of watchr is that it is not limited to only testing Ruby or PHP. Since you are using a block of Ruby code, you can do almost anything. At work, we are starting a little Node.js project, and I plan on using it to run the Vows tests.

Here’s a few other things you can do with it

• Compile CoffeeScript
• Run PHP Code Sniffer
• Run lint tests
• Minify your JavaScript and CSS

Basically, anything you can automate can be ran every time a file of a certain type is changed. I’ve seen only one drawback to watchr, it does not see new files. So when you add files, you need to restart it if you want them to be monitored.

Update 2012-09-04

I posted a follow up about Continuous Testing. I now use Guard to run my tests automatically.

The Book

In the first chapter he defines professionalism in our career. A lot of it turns around taking responsibility for our actions. He also compare our profession to being a doctor by saying that we should Do no harm. We should strive not to harm the function and the structure of our programs. He then describes the means by which we should ensure that we don’t hurt our code. Those means are the subject of the rest of the book.

The following two chapters are about how we answer requests. The importance of saying no, and of delivering when we say yes. We should never accept to commit to something we can’t deliver. But when we commit to something, we need to make sure we deliver it. Uncle Bob publish a great story taken from the blog of John Blanco that is really worth reading. It’s a great example of how bad it can get when we fail to say no. John also has a great recipe for manipulating developers. Tell them the the app is simple, add features by faulting them for not seeing their necessity and push the deadline to get them to do more. Reading it made me realize that I have seen it in action many times. And I had fell for this trick myself.

He then follow about the actual act of coding. How we should prepare to code. The importance of being rested and free of worries. He takes a stand against “the zone” talks about how we should be more forgiving of interruptions. He also speaks about the importance of helping others and being helped by others.

The next chapter is about Test driven development. This is something I try to practice more and more. I already wrote a post about it.

He then move on practicing. He describes Code Kata and coding dojo. He also talks about how to get more experience by contributing to open source projects.

The next two chapters are about testing. He speaks about acceptance tests. How they should define the requirements and make the definition of done pretty clear. These tests should always be automated and should run pretty quickly. He then describes the testing strategies. The most important thing to remember, is that QA should not find anything. He list the different kinds of tests and their goals.

In chapter 9 and 10, the author speaks about time. How to manage it and how to estimate projects. He mentions meetings, how they can be important, but also useless. We should try to be in the important one, and use the Law of Two Feet when we end up in the others. He also mention The Pomodoro Technique and avoiding getting caught in blind alleys and messes. Then he mention a few techniques for estimating tasks.

Chapter 11 is about pressure. How to avoid it, and how to mitigate it. Developers are often under a lot of pressure. Some of it we cause to ourselves by not saying no when we should. Some of it is caused by the importance of our work for our employers. Learning how to handle this pressure is important.

The following two chapters are about working with others. In chapter 12, the author speaks about collaboration. How important it is for us to collaborate with other developers, but also with other peoples. Then chapter 13 is about teams and projects. He describes the importance of having a good team that works well together.

Chapter 12 is about mentoring, apprenticeship and craftsmanship. He describes how valuable mentoring can be. A mentor can help bring a junior developer up to speed, but it also can help the mentor get a better understanding of what he has to teach. The author then describes an apprenticeship model where developers would start as an apprentice before moving to being a journeyman and finally a master.

The books ends with an appendix on the tools the author uses. He mentions source control, testing tools, editors, issue tracking and continuous integration servers.

My Take On All Of This

The Clean Coder is a very good book. Uncle Bob have a very strong opinion on how software should be developed. I don’t agree with everything he says, but in general he makes good points.

One of the interesting things about the book is that it is full of stories taken from the career of the author. He show us how to become better developers by showing us some of his failures. Seeing how and why he failed can help us avoid some of these mistakes.

I wish I had this book earlier in my career, but I’m not sure I will have understand it’s importance. It his a great handbook on improving as developers. And it’s a great reminder of how far I still need to go if I want to consider myself a professional developer.

In this post I will describe TDD as I understand it. I will also talk about the tools that are available in PHP.

What Is TDD

In TDD, developers write a failing test before writing any production code. The expected behavior of the code to write is defined this way. It is then easy to know when we have reached this goal. This produce a very small feedback loop. It also push the developer to write code that is very loosely coupled to the rest of the system. This code is easier to change, and it can be reused outside without having to bring half of the current system.

One of the greatest advantage TDD comes with maintenance. You can modify your existing code without the fear of breaking anything else. Because the code is loosely coupled, the risk of side effects is practically non existent. If the behavior of the application change in any way, your tests should warn you. You should also always write a failing test before fixing a bug. This way if the bug gets reintroduce, you will know right away.

How To Practice TDD

Practicing TDD can be resumed with the following mantra: red/green/refactor. Red is a failing test. You write a test for the simpler thing you can achieve. Then you write the code to make that test pass and go to green. You should write as little code as possible to get the test passing. Commit any crime you need, the only important thing is to get back to green. Then in the refactor phase, you fix the code you wrote. Remove any duplication, make sure the code is readable, use descriptive names… Just make sure your tests are still passing while you refactor. And then, you go back and write another failing test.

Robert C. Martin (Uncle Bob) wrote The Three Rules Of TDD. It is a great read, but here are the rules:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

You can see the very short feedback loop in those rules. There is also a strong emphasis on writing as little code as possible. Just write what you need and nothing more. This goes with the YAGNI (You ain’t gonna need it) approach.

TDD in PHP

PHP has two main tools for unit testing. Simple Test and PHPUnit. I always used PHPUnit for unit testing in PHP. I love the tool, it integrate with Eclipse and it can generate code coverage reports. Apparently Simple Test is pretty good also, but I have never tried it. You can pick any unit testing framework you like, just start writing test.

To write unit tests with PHPUnit, you simply create a file with a name ending by ‘Test.php’. If you want to test a class named KarateChop, you create the file ‘KarateChopTest.php’. In this file you create a class KarateChopTest that extends ‘PHPUnit_Framework_TestCase’.

You then create your testing methods inside this class. The testing methods must be public and their names must start with test. I just found out that you can use the @test annotation in the docblock instead of prefixing your method name with test. All your test will be run in isolation. An instance of the test class will be created for each test method. Just be careful with global data. Global variable and static properties can make you code very hard to test.

If your methods needs some code to prepare the test, you can create a setup() method. It will be call before every test. You can also add a teardown() method to do any clean up after the test.

Your test methods should be small. They should test for only one behavior, and every test should end with an assertion that verify that the system under test (SUT) behaved has expected. Don’t forget to write the test before the actual code. This way you can see the test fail first, and when it will pass, you will know that the system act as intended.

To verify the results of your test, PHPUnit has a wide range of assertions. They go from the simple assertTrue to more elaborates ones like assertEqualXMLStructure and assertGreaterThanOrEqual. I counted 36 assertions types in the documentation. It even have an assertThat method to write tests in the Behavior Driven Development style.

All your tests could simply use assertTrue to verify a condition, but the intent of your code is more obvious with the verbose assertions. Those 2 lines are the same:

$this->assertStringStartsWith($prefix, $string);
$this->assertTrue(0 === strpos($string, $prefix));

But the first one says what it is I’m testing.

To run your tests, all you need to do is run phpunit, passing it the file with the tests as a parameter. If you pass it a folder, PHPUnit will run the test in every files with a name ending by ‘Test.php’ in that folder and any sub folders.

Bootstrapping

Sometimes, you will need to have some code run before all the test cases. You might need to alter the require path, add an autoloader or set some environment variables. PHPUnit allow us to pass it a bootstrap file. This file will be run before your tests to prepare your testing environment.

phpunit --bootstrap Tests/testBootstrap.php .


Testing Databases

One of the common issue when doing TDD, or writing unit tests in general is how should we test code that interact with the database. The short answer to that, is don’t. Your database should be tested during your integration tests, not in unit tests. You should try to keep your data access layer isolated from the rest of the code. That will help you testing, but also make changing the way you store your data easier.

However, we work in the real world and we sometimes have to test code that access a database. To make it easier, you should not create the connection, but require it in the object constructor or as a parameter to the method that use it. If you can’t do this, consider creating a setter that allow your testing code to inject a different connection.

Make sure you use PDO connections and try to stick to standard SQL. This way in your tests, you can create a SQLite database and pass it to your tests. Make sure that you create your database in memory, not in a file. This way each test will have his own database and they will stay isolated. Creating a database like this can become very cumbersome if you use a lot of tables, or need to populate them with a lot of fake data.

If you need to connect to a real database to run your tests, you can pass your code a different configuration so at least it does not connect to your production server. You can also add an entry to your hosts file so when you try to connect to production, you end up in the test database. This can be dangerous though, someone might end up running your tests without the entry in the hosts file and alter your production database.

Connecting to a real database cause problems with the isolation of the tests. Changes made by a test can alter the result of another test. Testing against a database can also be very slow. So you should do this only if you have no other choices.

PHPsrc

I already wrote about PHPsrc in a previous post. This tool will allow you to run your tests from inside Eclipse.

PHPsrc PHPUnit Configuration

The PHPUnit integration will allow you to easily run the tests. It can also jump between the tests and the class being tested. You can give it a bootstrap file in the configuration. Either in the global settings, or by project. It will underline failing tests as errors, and it can also show code that is not covered by your tests.

Where To Start

Beginning TDD is not an easy task. At first it will slow you down. You might appear to lose some productivity, but you should catch up pretty fast. Especially when you will maintain your code. You should then be able to make changes while being confident that you didn’t break anything.

If you want to be good at anything, you need to practice. There are many resources available to practice TDD. You can perform some Code Kata, attempt a Code Retreat, find a Coding Dojo or try CyberDojo. All these are excellent, just go and practice.

If you need any more information about TDD, I would recommend reading Test Driven Development: By Example (affiliate link) by Kent Beck. It starts with an in-depth explanation of TDD and finishes by implementing an xUnit framework in TDD.

The Setup

The event took place at the Germain-Dominion Hotel. It is located in the old port district of Quebec City, close to the fortifications.

The retreat started at 8 AM with a breakfast and finished at 5 PM after 6 sessions of coding. With had 2 coffee breaks and a nice diner. The room was nice, with plenty of plugs for the laptop and a good WiFi connection. The organizers did a great job setting this up. Everything was ready for a full day of learning.

How The Day Went

After the breakfast, the actual retreat started by an introduction from Corey. He describes the goals of the retreat, how it got started and the problem we where going to work on. If you are interested in his introduction, you can look at the introduction he gave in Cleveland in January.

Cleveland Code Retreat Introduction from Corey Haines on Vimeo.

After that, Corey explained that we where going to focus on the The Four Elements of Simple Design. They are in order of importance:

• Passes its tests
• Minimizes duplication
• Maximizes clarity
• Has fewer elements

Then we started the first session. Each sessions lasted 45 minutes. During this time we had to do Pair programming to work on the problem. Every time we tried to produce the best code we could. And at the end of the session we had to delete the code. Then Corey did a brief recap of the session and gave some tips to help us move forward. Then we had to find another partner for the next session.

At the end of the day, we took pictures, then we formed a circle and everyone had to answer 3 questions. What we learned, what surprised us the most and what we will change at work.

My Impressions

The first thing I notice is that there where people programming in many different languages. We where a few PHP developers, but not all of us had a testing environment ready. I was happy to show some people how to do TDD in PHP with PHPUnit.

During the day, I paired with six different developers. We covered Java, C#, PHP and Python. I got to work with good developers. The pairing is probably the greatest thing about the retreat. During each session we got to exchange ideas with someone else. Every one brought a different way to attack the problem. This is a great chance to try to code differently. Especially in a setup where we can take the time to do it correctly, with no pressure to deliver. Pairing is also a great way to transfer knowledge. While working in pair, you can learn things such as language features, better design, IDE features and keyboard shortcuts. All this just by looking at the other developer.

Practicing Test-driven development during an entire day reminded me of how much I love it. Even in an event like this one, not every one I paired with was doing TDD. Some started by writing some code before adding a test to verify it. But when I was doing TDD, it really felt like the correct thing to do. I know I still need a lot of practice, but when I did TDD, programming was just more fun.

What I Got Out Of It

I think the greatest lesson I got from the code retreat is a new perspective on programming. Now I will try to look at my projects for another angle. Corey and J. B. made me realize that even if we are all against the waterfall approach, we still tend to over-think our design before we start coding. On the first 3 sessions, I started with a Cell class. It seemed like it was a good starting point because it will be needed for the game. But it was a dead end.

Corey told us that when we are ready to start coding something, we should ask why? Why are we coding this? Then take the answer and ask why again. After a few time we will reach a point where we cannot ask it anymore, this is where we should start.

With the Cell class example, I was starting there because I needed to know if a cell was alive or not. I needed to know this because I needed to check if a cell needed to be killed or not. I needed this because… In the end, what I needed is a board to play the game of life. Starting with the board seemed counter intuitive to me, but when I tried it everything fell into place.

I also got to see some Python. This is a language I barely know. Pairing with a Python developer was really fun. I think I could really like this language. I especially liked the list comprehension. This is something I really wish we had in PHP.

What Now?

The code retreat was fun, I got to experiment with practices that are not accepted where I work. It is now up to me to try and change that. I already used TDD on some project. But I really need to get back to doing it as much as I can. If it does not slow me down, and I end up producing better code, nobody can complain about it. It would also be great if I could get the other members of the team to try it. I gave a little presentation about it at work over a year ago. Sadly as far as I know, no one else tried it.

Pair programming is harder to sell. I still think it’s an amazing way to transfer knowledge. It can help bring up to speed a new developer. Pairing a junior developer with someone with more experience can help both of them. The junior will get an access to the knowledge and techniques of an experienced developers. And the experienced one should consolidate is knowledge by explaining it. And he may also learn something from the other developer.

I also have to thanks the organizers, the sponsors and the facilitator. They made this amazing event possible. For only 30$, I got to spend a day coding with better developers than me. I hope it made me a better programmer, and I am looking forward to the next event.

Update

Karl Metivier has posted the videos of the introduction that Corey Haines did at the Quebec code retreat.

However, when I make a mistake, produce sub-optimal code or if I write code that does not respect our coding standards I want to know before I commit. I would rather fix my errors before they break the build. I can run all the tools that the server runs. But running them on my machine and analyzing the results every time I save will be painful. Fortunately, there is an easy solution.

PHPsrc

PHPsrc is a plugin that allow you to run PHP_CodeSniffer, PHPUnit, PHP Depend and PHP Copy/Paste Detector directly in Eclipse. The site also says that more tools should come. As you work, you will see any transgression you make. That will save you from breaking the build, but it also makes it easier to fix the problem. After all, you just wrote the faulty lines of code.

You install it like any other Eclipse plugin. Go to the Help menu and click on “Install new software…” In the Install dialog, click on “Add…”, choose a name for the repository and enter “http://www.phpsrc.org/eclipse/pti/” in the location. After you click on OK, Eclipse will detect “PHP Tool integration. Select it and click on Next twice. Accept the condition and click on Finish. After the installation is completed, you will have to restart Eclipse.

After the installation, in the preferences you will have a new “PHP Tools” menu. Make sure that every tools have an PHP executable and a PEAR library selected. For PHP CodeSniffer, you will also need to pick a standard.

PHPsrc Configuration

Your Eclipse should now have new buttons in the toolbar and a new “PHP Tools” sub menu in the PHP Explorer.

PHPsrc Toolbar

PHPsrc Menu

PHPsrc will now check your code every time you save. Errors and warnings will be added to the Problems tab and the Type will be the tool that found the problem. The lines that causes problems will also be underlined like a syntax error. The copy paste detector will send the results to the console.

Here’s what Eclipse looks like with PHPsrc:

PHPsrc Results

If you develop PHP code in eclipse, you should definitely give PHPsrc a try. If you use a continuous integration server, being warned about potential issues before you commit to your source control repository can save you from becoming the one who broke the build.

implements continuous processes of applying quality control — small pieces of effort, applied frequently.

In simple terms, you verify that your project meets the quality standards frequently. This way, you can catch any deviation early. Doing the integration in small increments makes it easier. Implementing a continuous integration server can look difficult when you’ve never done it. In this post I will try to document how to set it up. I take for granted that you already have a LAMP server configured.

The Tools

PHP has many tools that can be used to analyse your code. There are tools for running unit tests, detecting copy/paste, checking the coding standards and more. We have to thank Sebastian Bergmann for this because he wrote most of those tools. I’ll cover some of them here, but there are more.

PHPUnit

PHPUnit is the PHP version of the xUnit framework. It allows running automated tests on code. To install it, use those commands:

sudo apt-get install php5-curl php-pear php5-dev
sudo pear upgrade pear

sudo pear install phpunit/PHPUnit


After you installed PHPUnit, you can run it with this command:

phpunit .

It will look for test cases in the current folder and sub folders.

You can also use PHPUnit to generate a code coverage report.

phpunit --coverage-html ../CodeCoverage .

This will generate an HTML report of the code that is covered by the tests. The files will be located in the path you passed it. In this case, a CodeCoverage folder located in the parent folder. The reports will look like this:

Coverage index page

If you click on a file, you will get the coverage by function and the content of the file. The code will be highlighted to show what is covered and what is not.

Coverage of a file

Code not covered

PHP CodeSniffer

PHP Code Sniffer analyse you code and detect violations to coding standards. It comes bundle with a few standards like the Zend and PEAR standards. But you can write your own standard by expanding an existing standard.

It’s a simple PEAR install. Than you can run it by calling phpcs with the standard to use and the path to check.

sudo pear install PHP_CodeSniffer
phpcs --standard=Zend .

You will get a result like this:

CodeSniffer results

If you don’t want to specify the standard to use on every call, you can set a default with:

phpcs --config-set default_standard Zend


PHP Depend

PHP Depend is a tool that will perform a static analysis of you code base. It generates software metrics that can be used to evaluate the complexity and quality of your code. Install it with PEAR:

sudo pear channel-discover pear.pdepend.org
sudo pear install pdepend/PHP_Depend-beta

To execute it, you call pdepend and pass it the type of reports you want it to generate. Execute it without parameters to view a brief description of the possible parameters.

pdepend --jdepend-xml=../jdepend.xml --jdepend-chart=../dependencies.svg --overview-pyramid=../overview-pyramid.svg .

This will generate an XML with the packages dependencies, a diagram of the dependencies and the overview pyramid of the project.

Abstraction Instability Chart

The Abstraction Instability Chart was invented by Robert C. Martin (uncle Bob) as a mean of measuring the instability and abstraction of the packages of a project. You should try to keep your packages as close as possible to the green line. An abstract package should have more classes depending on it, so it should be very stable. Concrete packages can be unstable, because nothing should directly depends on them if you code against abstractions. Having said that, I am not sure how it breaks my code into packages. I would guess it uses folders.

Overview Pyramid

The Overview Pyramid contains the metrics extracted by PHP Depend. The PHP Depend documentation has a great explanation of the pyramid, but I’ll try to give a brief overview.

The top part displays the metrics about inheritance. They are Average Number of Derived Classes (ANDC) and Average Hierarchy Height (AHH). ANDC tells you how many of your classes are derived from other classes. AHH is a measure of how deep your hierarchy is.

The bottom right part shows the coupling metrics. NOM represents the numbers of methods in your project. CALLS is the number of methods calls. FANOUT counts the types that are references by your classes. It counts only references that are not part of the same class hierarchy.

The bottom left part of the pyramid contains the metrics that are used the most. NOP is the number of packages, NOC the number of classes, NOM the number of methods, LOC the lines of code and CYCLO is the cyclomatic complexity.

The numbers in the middle represents the actual count and the numbers on either sides are averages of the metrics. They represent the value of the row under it divided by the value of the current row. So in my graph, I have a total of 168 lines of code and 26 methods, for an average of 6.462 lines per method.

PHP Mess Detector

PHPMD is used to detect problems in your code. It uses the same metrics as PHP Depends to give you feedback on your code. It can detect possible bugs and common issues. Install it with PEAR.

sudo pear channel-discover pear.phpmd.org
sudo pear channel-discover pear.pdepend.org
sudo pear install --alldeps phpmd/PHP_PMD


To execute PHPMD you need to give it the files to parse, a format for the output and the rule sets to use.

phpmd . html codesize,unusedcode,naming,design --reportfile ../messdetector.html --exclude Tests/

This will run it on the current folder and generate html. It will use all 4 rule sets install with PHPMD. The reportfile option tells PHPMD to send the output to the specified file instead of stdout and the exclude options tells it to ignore files in the Tests folder. It generates something like this:

PHP Mess Detector Output

PHP Copy/Paste Detector

As the name implies, PHPCPD detects when code has been copied over in your code. This gives you great candidates for refactoring your code.

Install it with PEAR. If you already installed PHPUnit, you should not need the two channel-discover commands.

sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no

sudo pear install phpunit/phpcpd

When it’s installed you can just run phpcpd by giving it the path to check. It will returns something like this:

PHPCPD Output

PHP Dead Code Detector

PHPDCD scan you project and detect functions that are not called. This might not look like a big deal, but dead code can really slow down the maintenance of a project. Developers have to scan and understand this code, and they might end up fixing bugs in code that is never called. Delete this code, you can get it back from your source control if you need it.

sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no

sudo pear install phpunit/phpdcd-beta

Run it by calling phpdcd with the path to scan. I make it ignore my tests. If not, it will flag them all because they are not called anywhere in the code, phpunit runs them.

PHPDCD Output

Putting It All Together With Jenkins

All those tools are very good, but having to call them one by one is painful. Jenkins is the continuous integration server that will manage continuously running all of those tools. It will be a central point of access for all the reports and send notifications when a build failed.

Building Your Project

Jenkins can build a project in many ways. It can use a Windows batch file, execute a shell script, Ant or Maven build files.

For my project I used Ant. To install Ant, make sure you have the Java JDK installed, then you can install it with apt.

sudo apt-get install default-jdk
sudo apt-get install ant

Then, you need to create your build file. The Ant site has a great introduction on creating build files. Here’s what my build file looks like:

<project name="Test" default="build" basedir=".">
	<property name="output" location="${basedir}/buildOutput/"/>

	<target name="init">
		<mkdir dir="${output}"/>
		<mkdir dir="${output}/phpcs/"/>
		<mkdir dir="${output}/pdepend/"/>
	</target>

	<target name="build" depends="init, test, phpcs, phpmd, phpcpd, pdepend">
	</target>

	<target name="test">
		<exec executable="phpunit" failonerror="true">
			<arg line="--coverage-clover ${output}/CodeCoverage/clover.xml 
				--coverage-html ${output}/CodeCoverage/ 
				."/>
		</exec>
	</target>

	<target name="phpcs">
		<exec executable="phpcs"> 
			<arg line="--report=checkstyle
              			--report-file=${output}/phpcs/checkstyle.xml
              			--standard=Zend
              			${basedir}" />
		</exec>
	</target>

	<target name="phpmd">
		<exec executable="phpmd">
			<arg line="
				 . xml codesize,unusedcode,naming,design --reportfile ${output}/messdetector.xml --exclude Tests/
			" />
		</exec>
	</target>

	<target name="phpcpd">
		<exec executable="phpcpd">
			<arg line="
				 --log-pmd ${output}/phpcpd.xml .
			" />
		</exec>
	</target>

	<target name="pdepend">
		<exec executable="pdepend">
			<arg line="
				--jdepend-xml=${output}/pdepend/jdepend.xml 
				--jdepend-chart=${output}/pdepend/dependencies.svg 
				--overview-pyramid=${output}/pdepend/overview-pyramid.svg 
				--ignore=Tests/ 
				.
			" />
		</exec>
	</target>
</project>

It defines a default target that make sure the needed folders for the output are there, then it calls PHPUnit, PHP Code Sniffer, PHP Mess detector, PHP Copy Paste Detector and PHP Depend.

Installing Jenkins

Installing it it pretty easy. There are package for many OS. For Ubuntu, just follow the instructions from the Jenkins site. Download the key and add it to apt.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -

Add this line to /etc/apt/sources.list

deb http://pkg.jenkins-ci.org/debian binary/

Update your apt index and install Jenkins

sudo apt-get update
sudo apt-get install jenkins

This will install Jenkins, set it as a daemon and start it. After, you can navigate to http://localhost:8080 to access your Jenkins installation.

Jenkins Home Page

Configuring Jenkins

Jenkins has many configuration options. I won’t go through all of them, but I’ll try to cover those I thinks are important. First click on “Manage Jenkins” then “Configure System”. Go through the settings there and configure Jenkins according to your needs. You should probably enable the security and pick how your users will log in. You should also configure the SMTP servers so Jenkins can send you emails.

After, look at the plugins and enable those you need. I use Git for my source control, so I enabled the Git plugin. All I needed to do was select the plugin, click on the Install button at the bottom of the list and click on the restart button when it was done.

You will also need a bunch of plugins to manage the data produced by the tools. Clover will display a code coverage graph from with the data built by PHPUnit. Checkstyle to display the results of PHP Code Sniffer. PMD for the PHP Mess Detector results. DRY to show duplicated code. And finally JDepend to show the metrics produces by PHP Depend.

Creating You Project

This is where you will configure Jenkins to build your project. Click on the “New Job” link. Give your job a name, select the free style project and click on OK. You will be taken the Configure page of the project. There are many options available and they all have a description if you click on the question mark icon.

First thing I did is a trick I got from Sebastian Bergmann’s Template for Jenkins Jobs for PHP Projects. I added an embed tags with links to the svg files created by PHP Depend. Since they are not showed by the plugins and they contains valuable information, it’s nice to see them on my project page.

Next, I needed to configure the source control repository. I gave Jenkins the path to my Git repository on the same machine. I left the branch blank so it check for changes in all the branches and build them. I entered the URL to my Gitweb interface so my builds can contains links to the repository and the commit that triggered the build.

Git requires that you set a user name and an email. The advanced section of the Git repository setup have fields for them, but they are not saved correctly, so you will have to do it manually.

su - jenkins
git config --global user.name "Jenkins"
git config --global user.email "jenkins@yourdomain.com"

This problem should be fixed in the next version.

You needs to tell Jenkins when to build your project. I decided to have Jenkins poll Git every minutes and launch a build every time something changed. I your project has many commits and takes time to build, you might want to make interval between checks longer.

In the build section, I just told Jenkins to invoke Ant. Since I want it to launch the default action in my build file, I did not need to give it a target.

Post Build Actions

This is where most of the configuration for the project is needed. All the plugins we installed for displaying the results of the build need to be configured. Make sure, you select all the plugins you have installed.

Many plugins will requires a results file. This is the files produce by the tests tools. Enter the relative path the results files so Jenkins can read them. There are also “Run always” check-boxes. By default, some plugins don’t run on failed build. When “Run always” is checked, the plugin will run no matter the status of the build.

Some plugins also requires some thresholds. Those are the values that tell Jenkins when to consider a build unstable, or failed. Configure them to values that make sense for your project. For the code coverage, set the conditionals to 0%, because they are not reported.

At the bottom of the project settings, you can also configure Jenkins to send emails on failed build. It will ask you for a list of addresses where to send the email and you can make it send another email to those who break the build.

Jenkins Project Settings

The End Result

Keep Jenkins running for a while. Work on your project and make a few commits. After some time, you will have a project page that looks like this:

Jenkins Project Page

If you click on a build in the Build History, you will get a page like this one:

Jenkins Build Results

If you have a team that commit early and often, Jenkins will provide some invaluable information. You can use it to look for part of your code where you need improvements. You can always know if your new developer is not following the team coding standards. And you can pinpoint area that might be too complicated and should be refactored. With a good test suite in place that automatically runs on every commit, you can detect regressions as soon as they are committed.

Wrapping It Up

The subject of Continuous Integration is broad and this is already a long post. And there is still much I need to learn about it. Creating the build script and configuring the Jenkins project was a lot of work. It was fun to do and I learned a lot, but doing it manually for every project would be daunting. I already mentioned it earlier, there is a great template job for Jenkins. It has all the instruction for installing it, a build script and the Jenkins project.