Continuous testing is a way to automate the execution of your tests while you work. This makes the feedback loop very short. As soon as you save a file, the tests are run and you know right away if anything fails.
Autotest
I discovered continuous testing over a year ago when I watched a video where Corey Haines performed a kata in front of a crowd. He was doing it in Ruby and using Autotest to run his test suite. I tough it was awesome that he immediately saw the result of his changes without having to do anything.
Back then I had used PHPsrc. It’s a great tool, but I didn’t like that I had to take my hand off the keyboard and click on a button to run my tests. So I was mainly running them from the command line.
AutoPHPUnit
When I saw Autotest, I was immediately sold and I looked for something similar in PHP. Sadly I didn’t found anything like it. So after a while I wrote something for myself. I called it AutoPHPUnit. It uses libnotify to watch the file system. Every time a file is changed, it runs PHPUnit. You can also specify a configuration file for PHPUnit.
It worked well for me, I really loved using it. But it was not very flexible.
Watchr
Then I read the book Continuous Testing: with Ruby, Rails, and JavaScript (affiliate link) that I won at the Code Retreat in Quebec city.
In the book, the authors use watchr to monitor the file system and perform any actions when a file changes. Watchr can be used to monitor any kind of file, so it’s more flexible than my solution. And you tell it what to do with a Ruby block so it can react to changes any way you want.
To install it, you need to have Ruby already installed on your machine. Then you use RubyGems to install it. Simply run “gem install watchr” and you will have watchr on your machine. If you are not using RVM to manage your versions of Ruby, you might need to use sudo to install watchr.
You can then create a simple Ruby script. Calling the function watch() with a regexp for the files you want to watch and the block you want to execute when a file changes. Then, you run watchr passing it the script file as a parameter.
Here’s a simple script to run PHPUnit:
watch ('.*\.php$') {|phpFile| system("phpunit -c phpunit.xml")}
I watch every file that ends with ‘.php’. When watchr detects a change, it simply runs PHPUnit for me.
watchr running
Notifications
Using watchr like this is great if you have enough screen space to show a terminal beside your code. But if you can’t have a terminal on the screen at the same time as your editor, you can use notify-send on Ubuntu to get a pop up in the notification section.
watch ('.*\.php$') {|phpFile| run_php_unit(phpFile)}
def run_php_unit(modified_file)
system('clear')
if (system("phpunit -c phpunit.xml"))
system("notify-send 'All test passed'")
else
system("notify-send 'Test failed'")
end
end
You can add any logic you want. In the Continuous Testing book they go further. You can have it display notifications only when the tests fails and when they pass again for the first time. They also count the numbers of successful runs and display notifications every 5 consecutive successful run.
watchr notification
Beyond PHP
The main advantage of watchr is that it is not limited to only testing Ruby or PHP. Since you are using a block of Ruby code, you can do almost anything. At work, we are starting a little Node.js project, and I plan on using it to run the Vows tests.
Here’s a few other things you can do with it
- Compile CoffeeScript
- Run PHP Code Sniffer
- Run lint tests
- Minify your JavaScript and CSS
Basically, anything you can automate can be ran every time a file of a certain type is changed. I’ve seen only one drawback to watchr, it does not see new files. So when you add files, you need to restart it if you want them to be monitored.
Tags: Best Practices, PHP, PHPUnit, Testing
Yesterday I finished reading The Clean Coder: A Code of Conduct for Professional Programmers (affiliate link) by Robert C. Martin. In this book, Uncle Bob expose his view of what it is to be a professional programmer. The author have been programming for 40 years and he have very strong opinion about the subject of professionalism in programming.
The Book
In the first chapter he defines professionalism in our career. A lot of it turns around taking responsibility for our actions. He also compare our profession to being a doctor by saying that we should Do no harm. We should strive not to harm the function and the structure of our programs. He then describes the means by which we should ensure that we don’t hurt our code. Those means are the subject of the rest of the book.
The following two chapters are about how we answer requests. The importance of saying no, and of delivering when we say yes. We should never accept to commit to something we can’t deliver. But when we commit to something, we need to make sure we deliver it. Uncle Bob publish a great story taken from the blog of John Blanco that is really worth reading. It’s a great example of how bad it can get when we fail to say no. John also has a great recipe for manipulating developers. Tell them the the app is simple, add features by faulting them for not seeing their necessity and push the deadline to get them to do more. Reading it made me realize that I have seen it in action many times. And I had fell for this trick myself.
He then follow about the actual act of coding. How we should prepare to code. The importance of being rested and free of worries. He takes a stand against “the zone” talks about how we should be more forgiving of interruptions. He also speaks about the importance of helping others and being helped by others.
The next chapter is about Test driven development. This is something I try to practice more and more. I already wrote a post about it.
He then move on practicing. He describes Code Kata and coding dojo. He also talks about how to get more experience by contributing to open source projects.
The next two chapters are about testing. He speaks about acceptance tests. How they should define the requirements and make the definition of done pretty clear. These tests should always be automated and should run pretty quickly. He then describes the testing strategies. The most important thing to remember, is that QA should not find anything. He list the different kinds of tests and their goals.
In chapter 9 and 10, the author speaks about time. How to manage it and how to estimate projects. He mentions meetings, how they can be important, but also useless. We should try to be in the important one, and use the Law of Two Feet when we end up in the others. He also mention The Pomodoro Technique and avoiding getting caught in blind alleys and messes. Then he mention a few techniques for estimating tasks.
Chapter 11 is about pressure. How to avoid it, and how to mitigate it. Developers are often under a lot of pressure. Some of it we cause to ourselves by not saying no when we should. Some of it is caused by the importance of our work for our employers. Learning how to handle this pressure is important.
The following two chapters are about working with others. In chapter 12, the author speaks about collaboration. How important it is for us to collaborate with other developers, but also with other peoples. Then chapter 13 is about teams and projects. He describes the importance of having a good team that works well together.
Chapter 12 is about mentoring, apprenticeship and craftsmanship. He describes how valuable mentoring can be. A mentor can help bring a junior developer up to speed, but it also can help the mentor get a better understanding of what he has to teach. The author then describes an apprenticeship model where developers would start as an apprentice before moving to being a journeyman and finally a master.
The books ends with an appendix on the tools the author uses. He mentions source control, testing tools, editors, issue tracking and continuous integration servers.
My Take On All Of This
The Clean Coder is a very good book. Uncle Bob have a very strong opinion on how software should be developed. I don’t agree with everything he says, but in general he makes good points.
One of the interesting things about the book is that it is full of stories taken from the career of the author. He show us how to become better developers by showing us some of his failures. Seeing how and why he failed can help us avoid some of these mistakes.
I wish I had this book earlier in my career, but I’m not sure I will have understand it’s importance. It his a great handbook on improving as developers. And it’s a great reminder of how far I still need to go if I want to consider myself a professional developer.
Tags: Best Practices, Books
Test driven development (TDD) is at the core of the Agile Methodology and Extreme Programming. This practice has been known for a while and a lot have been written on it. However, I still meet developers that don’t know what it is. I understand that many employers won’t let their employees write tests, but we should at least know about the best practices of our industry.
In this post I will describe TDD as I understand it. I will also talk about the tools that are available in PHP.
What Is TDD
In TDD, developers write a failing test before writing any production code. The expected behavior of the code to write is defined this way. It is then easy to know when we have reached this goal. This produce a very small feedback loop. It also push the developer to write code that is very loosely coupled to the rest of the system. This code is easier to change, and it can be reused outside without having to bring half of the current system.
One of the greatest advantage TDD comes with maintenance. You can modify your existing code without the fear of breaking anything else. Because the code is loosely coupled, the risk of side effects is practically non existent. If the behavior of the application change in any way, your tests should warn you. You should also always write a failing test before fixing a bug. This way if the bug gets reintroduce, you will know right away.
How To Practice TDD
Practicing TDD can be resumed with the following mantra: red/green/refactor. Red is a failing test. You write a test for the simpler thing you can achieve. Then you write the code to make that test pass and go to green. You should write as little code as possible to get the test passing. Commit any crime you need, the only important thing is to get back to green. Then in the refactor phase, you fix the code you wrote. Remove any duplication, make sure the code is readable, use descriptive names… Just make sure your tests are still passing while you refactor. And then, you go back and write another failing test.
Robert C. Martin (Uncle Bob) wrote The Three Rules Of TDD. It is a great read, but here are the rules:
1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
You can see the very short feedback loop in those rules. There is also a strong emphasis on writing as little code as possible. Just write what you need and nothing more. This goes with the YAGNI (You ain’t gonna need it) approach.
TDD in PHP
PHP has two main tools for unit testing. Simple Test and PHPUnit. I always used PHPUnit for unit testing in PHP. I love the tool, it integrate with Eclipse and it can generate code coverage reports. Apparently Simple Test is pretty good also, but I have never tried it. You can pick any unit testing framework you like, just start writing test.
To write unit tests with PHPUnit, you simply create a file with a name ending by ‘Test.php’. If you want to test a class named KarateChop, you create the file ‘KarateChopTest.php’. In this file you create a class KarateChopTest that extends ‘PHPUnit_Framework_TestCase’.
You then create your testing methods inside this class. The testing methods must be public and their names must start with test. I just found out that you can use the @test annotation in the docblock instead of prefixing your method name with test. All your test will be run in isolation. An instance of the test class will be created for each test method. Just be careful with global data. Global variable and static properties can make you code very hard to test.
If your methods needs some code to prepare the test, you can create a setup() method. It will be call before every test. You can also add a teardown() method to do any clean up after the test.
Your test methods should be small. They should test for only one behavior, and every test should end with an assertion that verify that the system under test (SUT) behaved has expected. Don’t forget to write the test before the actual code. This way you can see the test fail first, and when it will pass, you will know that the system act as intended.
To verify the results of your test, PHPUnit has a wide range of assertions. They go from the simple assertTrue to more elaborates ones like assertEqualXMLStructure and assertGreaterThanOrEqual. I counted 36 assertions types in the documentation. It even have an assertThat method to write tests in the Behavior Driven Development style.
All your tests could simply use assertTrue to verify a condition, but the intent of your code is more obvious with the verbose assertions. Those 2 lines are the same:
$this->assertStringStartsWith($prefix, $string);
$this->assertTrue(0 === strpos($string, $prefix));
But the first one says what it is I’m testing.
To run your tests, all you need to do is run phpunit, passing it the file with the tests as a parameter. If you pass it a folder, PHPUnit will run the test in every files with a name ending by ‘Test.php’ in that folder and any sub folders.
Bootstrapping
Sometimes, you will need to have some code run before all the test cases. You might need to alter the require path, add an autoloader or set some environment variables. PHPUnit allow us to pass it a bootstrap file. This file will be run before your tests to prepare your testing environment.
phpunit --bootstrap Tests/testBootstrap.php .
Testing Databases
One of the common issue when doing TDD, or writing unit tests in general is how should we test code that interact with the database. The short answer to that, is don’t. Your database should be tested during your integration tests, not in unit tests. You should try to keep your data access layer isolated from the rest of the code. That will help you testing, but also make changing the way you store your data easier.
However, we work in the real world and we sometimes have to test code that access a database. To make it easier, you should not create the connection, but require it in the object constructor or as a parameter to the method that use it. If you can’t do this, consider creating a setter that allow your testing code to inject a different connection.
Make sure you use PDO connections and try to stick to standard SQL. This way in your tests, you can create a SQLite database and pass it to your tests. Make sure that you create your database in memory, not in a file. This way each test will have his own database and they will stay isolated. Creating a database like this can become very cumbersome if you use a lot of tables, or need to populate them with a lot of fake data.
If you need to connect to a real database to run your tests, you can pass your code a different configuration so at least it does not connect to your production server. You can also add an entry to your hosts file so when you try to connect to production, you end up in the test database. This can be dangerous though, someone might end up running your tests without the entry in the hosts file and alter your production database.
Connecting to a real database cause problems with the isolation of the tests. Changes made by a test can alter the result of another test. Testing against a database can also be very slow. So you should do this only if you have no other choices.
PHPsrc
I already wrote about PHPsrc in a previous post. This tool will allow you to run your tests from inside Eclipse.
PHPsrc PHPUnit Configuration
The PHPUnit integration will allow you to easily run the tests. It can also jump between the tests and the class being tested. You can give it a bootstrap file in the configuration. Either in the global settings, or by project. It will underline failing tests as errors, and it can also show code that is not covered by your tests.
Where To Start
Beginning TDD is not an easy task. At first it will slow you down. You might appear to lose some productivity, but you should catch up pretty fast. Especially when you will maintain your code. You should then be able to make changes while being confident that you didn’t break anything.
If you want to be good at anything, you need to practice. There are many resources available to practice TDD. You can perform some Code Kata, attempt a Code Retreat, find a Coding Dojo or try CyberDojo. All these are excellent, just go and practice.
If you need any more information about TDD, I would recommend reading Test Driven Development: By Example (affiliate link) by Kent Beck. It starts with an in-depth explanation of TDD and finishes by implementing an xUnit framework in TDD.
Tags: Best Practices, PHP, PHPUnit, TDD
Last Saturday I had the privilege of attempting a code retreat in Quebec City. I have been hoping for this for a while, so I registered as soon as I found out about it. When I learned that Corey Haines and J. B. Rainsberger where going to be there I was really thrilled.
The Setup
The event took place at the Germain-Dominion Hotel. It is located in the old port district of Quebec City, close to the fortifications.
The retreat started at 8 AM with a breakfast and finished at 5 PM after 6 sessions of coding. With had 2 coffee breaks and a nice diner. The room was nice, with plenty of plugs for the laptop and a good WiFi connection. The organizers did a great job setting this up. Everything was ready for a full day of learning.
How The Day Went
After the breakfast, the actual retreat started by an introduction from Corey. He describes the goals of the retreat, how it got started and the problem we where going to work on. If you are interested in his introduction, you can look at the introduction he gave in Cleveland in January.
Cleveland Code Retreat Introduction from Corey Haines on Vimeo.
After that, Corey explained that we where going to focus on the The Four Elements of Simple Design. They are in order of importance:
- Passes its tests
- Minimizes duplication
- Maximizes clarity
- Has fewer elements
Then we started the first session. Each sessions lasted 45 minutes. During this time we had to do Pair programming to work on the problem. Every time we tried to produce the best code we could. And at the end of the session we had to delete the code. Then Corey did a brief recap of the session and gave some tips to help us move forward. Then we had to find another partner for the next session.
At the end of the day, we took pictures, then we formed a circle and everyone had to answer 3 questions. What we learned, what surprised us the most and what we will change at work.
My Impressions
The first thing I notice is that there where people programming in many different languages. We where a few PHP developers, but not all of us had a testing environment ready. I was happy to show some people how to do TDD in PHP with PHPUnit.
During the day, I paired with six different developers. We covered Java, C#, PHP and Python. I got to work with good developers. The pairing is probably the greatest thing about the retreat. During each session we got to exchange ideas with someone else. Every one brought a different way to attack the problem. This is a great chance to try to code differently. Especially in a setup where we can take the time to do it correctly, with no pressure to deliver. Pairing is also a great way to transfer knowledge. While working in pair, you can learn things such as language features, better design, IDE features and keyboard shortcuts. All this just by looking at the other developer.
Practicing Test-driven development during an entire day reminded me of how much I love it. Even in an event like this one, not every one I paired with was doing TDD. Some started by writing some code before adding a test to verify it. But when I was doing TDD, it really felt like the correct thing to do. I know I still need a lot of practice, but when I did TDD, programming was just more fun.
What I Got Out Of It
I think the greatest lesson I got from the code retreat is a new perspective on programming. Now I will try to look at my projects for another angle. Corey and J. B. made me realize that even if we are all against the waterfall approach, we still tend to over-think our design before we start coding. On the first 3 sessions, I started with a Cell class. It seemed like it was a good starting point because it will be needed for the game. But it was a dead end.
Corey told us that when we are ready to start coding something, we should ask why? Why are we coding this? Then take the answer and ask why again. After a few time we will reach a point where we cannot ask it anymore, this is where we should start.
With the Cell class example, I was starting there because I needed to know if a cell was alive or not. I needed to know this because I needed to check if a cell needed to be killed or not. I needed this because… In the end, what I needed is a board to play the game of life. Starting with the board seemed counter intuitive to me, but when I tried it everything fell into place.
I also got to see some Python. This is a language I barely know. Pairing with a Python developer was really fun. I think I could really like this language. I especially liked the list comprehension. This is something I really wish we had in PHP.
What Now?
The code retreat was fun, I got to experiment with practices that are not accepted where I work. It is now up to me to try and change that. I already used TDD on some project. But I really need to get back to doing it as much as I can. If it does not slow me down, and I end up producing better code, nobody can complain about it. It would also be great if I could get the other members of the team to try it. I gave a little presentation about it at work over a year ago. Sadly as far as I know, no one else tried it.
Pair programming is harder to sell. I still think it’s an amazing way to transfer knowledge. It can help bring up to speed a new developer. Pairing a junior developer with someone with more experience can help both of them. The junior will get an access to the knowledge and techniques of an experienced developers. And the experienced one should consolidate is knowledge by explaining it. And he may also learn something from the other developer.
I also have to thanks the organizers, the sponsors and the facilitator. They made this amazing event possible. For only 30$, I got to spend a day coding with better developers than me. I hope it made me a better programmer, and I am looking forward to the next event.
Update
Karl Metivier has posted the videos of the introduction that Corey Haines did at the Quebec code retreat.
In a previous post, I talked about Continuous Integration. If your Continuous Integration server runs on every commits, it will help you keep your code quality high. It will also make integration a non issue.
However, when I make a mistake, produce sub-optimal code or if I write code that does not respect our coding standards I want to know before I commit. I would rather fix my errors before they break the build. I can run all the tools that the server runs. But running them on my machine and analyzing the results every time I save will be painful. Fortunately, there is an easy solution.
PHPsrc
PHPsrc is a plugin that allow you to run PHP_CodeSniffer, PHPUnit, PHP Depend and PHP Copy/Paste Detector directly in Eclipse. The site also says that more tools should come. As you work, you will see any transgression you make. That will save you from breaking the build, but it also makes it easier to fix the problem. After all, you just wrote the faulty lines of code.
You install it like any other Eclipse plugin. Go to the Help menu and click on “Install new software…” In the Install dialog, click on “Add…”, choose a name for the repository and enter “http://www.phpsrc.org/eclipse/pti/” in the location. After you click on OK, Eclipse will detect “PHP Tool integration. Select it and click on Next twice. Accept the condition and click on Finish. After the installation is completed, you will have to restart Eclipse.
After the installation, in the preferences you will have a new “PHP Tools” menu. Make sure that every tools have an PHP executable and a PEAR library selected. For PHP CodeSniffer, you will also need to pick a standard.
PHPsrc Configuration
Your Eclipse should now have new buttons in the toolbar and a new “PHP Tools” sub menu in the PHP Explorer.
PHPsrc Toolbar
PHPsrc Menu
PHPsrc will now check your code every time you save. Errors and warnings will be added to the Problems tab and the Type will be the tool that found the problem. The lines that causes problems will also be underlined like a syntax error. The copy paste detector will send the results to the console.
Here’s what Eclipse looks like with PHPsrc:
PHPsrc Results
If you develop PHP code in eclipse, you should definitely give PHPsrc a try. If you use a continuous integration server, being warned about potential issues before you commit to your source control repository can save you from becoming the one who broke the build.
Tags: Best Practices, PHP, PHPUnit
According to Wikipedia, continuous integration
implements continuous processes of applying quality control — small pieces of effort, applied frequently.
In simple terms, you verify that your project meets the quality standards frequently. This way, you can catch any deviation early. Doing the integration in small increments makes it easier. Implementing a continuous integration server can look difficult when you’ve never done it. In this post I will try to document how to set it up. I take for granted that you already have a LAMP server configured.
The Tools
PHP has many tools that can be used to analyse your code. There are tools for running unit tests, detecting copy/paste, checking the coding standards and more. We have to thank Sebastian Bergmann for this because he wrote most of those tools. I’ll cover some of them here, but there are more.
PHPUnit
PHPUnit is the PHP version of the xUnit framework. It allows running automated tests on code. To install it, use those commands:
sudo apt-get install php5-curl php-pear php5-dev
sudo pear upgrade pear
sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear channel-discover pear.symfony-project.com
sudo pear install phpunit/PHPUnit
After you installed PHPUnit, you can run it with this command:
phpunit .
It will look for test cases in the current folder and sub folders.
You can also use PHPUnit to generate a code coverage report.
phpunit --coverage-html ../CodeCoverage .
This will generate an HTML report of the code that is covered by the tests. The files will be located in the path you passed it. In this case, a CodeCoverage folder located in the parent folder. The reports will look like this:
Coverage index page
If you click on a file, you will get the coverage by function and the content of the file. The code will be highlighted to show what is covered and what is not.
Coverage of a file
Code not covered
PHP CodeSniffer
PHP Code Sniffer analyse you code and detect violations to coding standards. It comes bundle with a few standards like the Zend and PEAR standards. But you can write your own standard by expanding an existing standard.
It’s a simple PEAR install. Than you can run it by calling phpcs with the standard to use and the path to check.
sudo pear install PHP_CodeSniffer
phpcs --standard=Zend .
You will get a result like this:
CodeSniffer results
If you don’t want to specify the standard to use on every call, you can set a default with:
phpcs --config-set default_standard Zend
PHP Depend
PHP Depend is a tool that will perform a static analysis of you code base. It generates software metrics that can be used to evaluate the complexity and quality of your code. Install it with PEAR:
sudo pear channel-discover pear.pdepend.org
sudo pear install pdepend/PHP_Depend-beta
To execute it, you call pdepend and pass it the type of reports you want it to generate. Execute it without parameters to view a brief description of the possible parameters.
pdepend --jdepend-xml=../jdepend.xml --jdepend-chart=../dependencies.svg --overview-pyramid=../overview-pyramid.svg .
This will generate an XML with the packages dependencies, a diagram of the dependencies and the overview pyramid of the project.
Abstraction Instability Chart
The Abstraction Instability Chart was invented by Robert C. Martin (uncle Bob) as a mean of measuring the instability and abstraction of the packages of a project. You should try to keep your packages as close as possible to the green line. An abstract package should have more classes depending on it, so it should be very stable. Concrete packages can be unstable, because nothing should directly depends on them if you code against abstractions. Having said that, I am not sure how it breaks my code into packages. I would guess it uses folders.
Overview Pyramid
The Overview Pyramid contains the metrics extracted by PHP Depend. The PHP Depend documentation has a great explanation of the pyramid, but I’ll try to give a brief overview.
The top part displays the metrics about inheritance. They are Average Number of Derived Classes (ANDC) and Average Hierarchy Height (AHH). ANDC tells you how many of your classes are derived from other classes. AHH is a measure of how deep your hierarchy is.
The bottom right part shows the coupling metrics. NOM represents the numbers of methods in your project. CALLS is the number of methods calls. FANOUT counts the types that are references by your classes. It counts only references that are not part of the same class hierarchy.
The bottom left part of the pyramid contains the metrics that are used the most. NOP is the number of packages, NOC the number of classes, NOM the number of methods, LOC the lines of code and CYCLO is the cyclomatic complexity.
The numbers in the middle represents the actual count and the numbers on either sides are averages of the metrics. They represent the value of the row under it divided by the value of the current row. So in my graph, I have a total of 168 lines of code and 26 methods, for an average of 6.462 lines per method.
PHP Mess Detector
PHPMD is used to detect problems in your code. It uses the same metrics as PHP Depends to give you feedback on your code. It can detect possible bugs and common issues. Install it with PEAR.
sudo pear channel-discover pear.phpmd.org
sudo pear channel-discover pear.pdepend.org
sudo pear install --alldeps phpmd/PHP_PMD
To execute PHPMD you need to give it the files to parse, a format for the output and the rule sets to use.
phpmd . html codesize,unusedcode,naming,design --reportfile ../messdetector.html --exclude Tests/
This will run it on the current folder and generate html. It will use all 4 rule sets install with PHPMD. The reportfile option tells PHPMD to send the output to the specified file instead of stdout and the exclude options tells it to ignore files in the Tests folder. It generates something like this:
PHP Mess Detector Output
PHP Copy/Paste Detector
As the name implies, PHPCPD detects when code has been copied over in your code. This gives you great candidates for refactoring your code.
Install it with PEAR. If you already installed PHPUnit, you should not need the two channel-discover commands.
sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear install phpunit/phpcpd
When it’s installed you can just run phpcpd by giving it the path to check. It will returns something like this:
PHPCPD Output
PHP Dead Code Detector
PHPDCD scan you project and detect functions that are not called. This might not look like a big deal, but dead code can really slow down the maintenance of a project. Developers have to scan and understand this code, and they might end up fixing bugs in code that is never called. Delete this code, you can get it back from your source control if you need it.
sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear install phpunit/phpdcd-beta
Run it by calling phpdcd with the path to scan. I make it ignore my tests. If not, it will flag them all because they are not called anywhere in the code, phpunit runs them.
PHPDCD Output
Putting It All Together With Jenkins
All those tools are very good, but having to call them one by one is painful. Jenkins is the continuous integration server that will manage continuously running all of those tools. It will be a central point of access for all the reports and send notifications when a build failed.
Building Your Project
Jenkins can build a project in many ways. It can use a Windows batch file, execute a shell script, Ant or Maven build files.
For my project I used Ant. To install Ant, make sure you have the Java JDK installed, then you can install it with apt.
sudo apt-get install default-jdk
sudo apt-get install ant
Then, you need to create your build file. The Ant site has a great introduction on creating build files. Here’s what my build file looks like:
<project name="Test" default="build" basedir=".">
<property name="output" location="${basedir}/buildOutput/"/>
<target name="init">
<mkdir dir="${output}"/>
<mkdir dir="${output}/phpcs/"/>
<mkdir dir="${output}/pdepend/"/>
</target>
<target name="build" depends="init, test, phpcs, phpmd, phpcpd, pdepend">
</target>
<target name="test">
<exec executable="phpunit" failonerror="true">
<arg line="--coverage-clover ${output}/CodeCoverage/clover.xml
--coverage-html ${output}/CodeCoverage/
."/>
</exec>
</target>
<target name="phpcs">
<exec executable="phpcs">
<arg line="--report=checkstyle
--report-file=${output}/phpcs/checkstyle.xml
--standard=Zend
${basedir}" />
</exec>
</target>
<target name="phpmd">
<exec executable="phpmd">
<arg line="
. xml codesize,unusedcode,naming,design --reportfile ${output}/messdetector.xml --exclude Tests/
" />
</exec>
</target>
<target name="phpcpd">
<exec executable="phpcpd">
<arg line="
--log-pmd ${output}/phpcpd.xml .
" />
</exec>
</target>
<target name="pdepend">
<exec executable="pdepend">
<arg line="
--jdepend-xml=${output}/pdepend/jdepend.xml
--jdepend-chart=${output}/pdepend/dependencies.svg
--overview-pyramid=${output}/pdepend/overview-pyramid.svg
--ignore=Tests/
.
" />
</exec>
</target>
</project>
It defines a default target that make sure the needed folders for the output are there, then it calls PHPUnit, PHP Code Sniffer, PHP Mess detector, PHP Copy Paste Detector and PHP Depend.
Installing Jenkins
Installing it it pretty easy. There are package for many OS. For Ubuntu, just follow the instructions from the Jenkins site. Download the key and add it to apt.
wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
Add this line to /etc/apt/sources.list
deb http://pkg.jenkins-ci.org/debian binary/
Update your apt index and install Jenkins
sudo apt-get update
sudo apt-get install jenkins
This will install Jenkins, set it as a daemon and start it. After, you can navigate to http://localhost:8080 to access your Jenkins installation.
Jenkins Home Page
Configuring Jenkins
Jenkins has many configuration options. I won’t go through all of them, but I’ll try to cover those I thinks are important. First click on “Manage Jenkins” then “Configure System”. Go through the settings there and configure Jenkins according to your needs. You should probably enable the security and pick how your users will log in. You should also configure the SMTP servers so Jenkins can send you emails.
After, look at the plugins and enable those you need. I use Git for my source control, so I enabled the Git plugin. All I needed to do was select the plugin, click on the Install button at the bottom of the list and click on the restart button when it was done.
You will also need a bunch of plugins to manage the data produced by the tools. Clover will display a code coverage graph from with the data built by PHPUnit. Checkstyle to display the results of PHP Code Sniffer. PMD for the PHP Mess Detector results. DRY to show duplicated code. And finally JDepend to show the metrics produces by PHP Depend.
Creating You Project
This is where you will configure Jenkins to build your project. Click on the “New Job” link. Give your job a name, select the free style project and click on OK. You will be taken the Configure page of the project. There are many options available and they all have a description if you click on the question mark icon.
First thing I did is a trick I got from Sebastian Bergmann’s Template for Jenkins Jobs for PHP Projects. I added an embed tags with links to the svg files created by PHP Depend. Since they are not showed by the plugins and they contains valuable information, it’s nice to see them on my project page.
Next, I needed to configure the source control repository. I gave Jenkins the path to my Git repository on the same machine. I left the branch blank so it check for changes in all the branches and build them. I entered the URL to my Gitweb interface so my builds can contains links to the repository and the commit that triggered the build.
Git requires that you set a user name and an email. The advanced section of the Git repository setup have fields for them, but they are not saved correctly, so you will have to do it manually.
su - jenkins
git config --global user.name "Jenkins"
git config --global user.email "[email protected]"
This problem should be fixed in the next version.
You needs to tell Jenkins when to build your project. I decided to have Jenkins poll Git every minutes and launch a build every time something changed. I your project has many commits and takes time to build, you might want to make interval between checks longer.
In the build section, I just told Jenkins to invoke Ant. Since I want it to launch the default action in my build file, I did not need to give it a target.
Post Build Actions
This is where most of the configuration for the project is needed. All the plugins we installed for displaying the results of the build need to be configured. Make sure, you select all the plugins you have installed.
Many plugins will requires a results file. This is the files produce by the tests tools. Enter the relative path the results files so Jenkins can read them. There are also “Run always” check-boxes. By default, some plugins don’t run on failed build. When “Run always” is checked, the plugin will run no matter the status of the build.
Some plugins also requires some thresholds. Those are the values that tell Jenkins when to consider a build unstable, or failed. Configure them to values that make sense for your project. For the code coverage, set the conditionals to 0%, because they are not reported.
At the bottom of the project settings, you can also configure Jenkins to send emails on failed build. It will ask you for a list of addresses where to send the email and you can make it send another email to those who break the build.
Jenkins Project Settings
The End Result
Keep Jenkins running for a while. Work on your project and make a few commits. After some time, you will have a project page that looks like this:
Jenkins Project Page
If you click on a build in the Build History, you will get a page like this one:
Jenkins Build Results
If you have a team that commit early and often, Jenkins will provide some invaluable information. You can use it to look for part of your code where you need improvements. You can always know if your new developer is not following the team coding standards. And you can pinpoint area that might be too complicated and should be refactored. With a good test suite in place that automatically runs on every commit, you can detect regressions as soon as they are committed.
Wrapping It Up
The subject of Continuous Integration is broad and this is already a long post. And there is still much I need to learn about it. Creating the build script and configuring the Jenkins project was a lot of work. It was fun to do and I learned a lot, but doing it manually for every project would be daunting. I already mentioned it earlier, there is a great template job for Jenkins. It has all the instruction for installing it, a build script and the Jenkins project.
Tags: Best Practices, Jenkins, PHP, PHPUnit
Late static binding is one of the new features of PHP 5.3. It came out almost 2 years ago, but it to me that many programmers around me have no idea about it. Myself, I have learned about it around 6 months ago.
The PHP documentation defines late static binding as a way to “reference the called class in a context of static inheritance.” This definition didn’t really help me the first time I read it. Fortunately, there are more explanations in the documentation, and there are good examples. If you haven’t, you should read it.
How Does It Work
To use late static binding, you need a class that inherits from another one and some static methods that are overridden. Take the following code:
class ParentClass {
public static function normalCall() {
self::calledMethod();
}
public static function lateStaticCall() {
static::calledMethod();
}
public static function calledMethod() {
echo "Called in Parent\n";
}
}
class ChildClass extends ParentClass {
public static function calledMethod() {
echo "Called in Child\n";
}
}
The method normallCall() represent the traditional way of using static functions. The self keyword will call the function in the current class. So no matter if I call it with ParentClass::normalCall() or ChildClass::normalCall(), the calledMethod() of the ParentClass will be called and “Called in Parent” will be printed.
However, the lateStaticCall() method uses the new “static” keyword. With this keyword, the call will be forwarded to the class on witch the original method was called. So ParentClass::lateStaticCall() will end up calling ParentClass::calledMethod() and print “Called in Parent”. But ChildClass::lateStaticCall() will call calledMethod() in the ChildClass and print “Called in Child”.
When Should I Use It
I am really not sure. Personally, I try to avoid static methods. They are often better ways to do it. However, if you have a class hierarchy with static methods and you need to override how they act for some child classes, it can be a solution.
If you used in production code, or know a good reason to use it, I would really appreciate if you leave a comment with it.
Many developers I know never attempt any tech event. I am lucky to live in Montréal where we have plenty of them. The Montreal NewTech calendar shows that we have activities for everyone. And it does not have everything. Some meetings cover programming languages, others are targeted at startups, or tools like WordPress. Even with all those choices, most of the developers around me never go to any of those events.
User Group Meetups
Most user group have monthly meetings. They meetup at a certain day every month to discuss about their subject. I have a family, so I cannot attempt to all the meeting I would like, but I try to go to PHPQuébec,
Conferences
Tech conferences are bigger events. They are usually held once a year. Some are always presented in the same city, and other moves from city to city. Those events bring together many great developers. You get to be around very intelligent people. Not only the speakers, but the audience is also very knowledgeable. When you spend a day learning about 5 or 6 different subjects, you go home tired, but with many ideas on how to improve as a developer. This year, I was lucky enough to have my employer sends the entire team to Confoo, but last year I paid for my ticket, and it was worth every dollars.
Conferences can appear a little expensive, but developers should be able to afford it. Most of them offers an early bird rate that allow to save a few hundred dollars. And some of them are really not that expensive. Last year, I attempted Agile Tour Montreal. For 50$, I had a a full day of training. There is also Day Camp 4 Developers, an online conference that should be held 2 or 3 time a year. On this one, you get 5 sessions of an hour for 35$, and you can also download the videos after the event.
For those who don’t have local conferences, it’s harder. You have to add the price of the plane tickets and a few night at the hotel. Even with this, I think it’s worth the price. I am saving some money to go to an event outside of Montreal next year. I haven’t decided yet, but the PHP Community Conference and Tek are pretty tempting.
What Do I Get Out Of It
The obvious answer is learning. When I go to these types of events, I learn a lot. I am exposed to knowledge I would never come across at work. But it’s only a small part of it. To me, the social aspect of it is as important. Conferences and meetups gave me a chance to meet many developers that share the same passion for programming and learning. During the discussions in the pauses or around a beer after the events, I have had the chance to debate about ideas and learn even more. They also offers great opportunities to add to your network. Who knows, the next time you will need a job, maybe someone you met at an event will remember you.
And recently, I realized that it changed me. I have never been a social person. I don’t like talking to people I don’t know. But this changed a little. I now look forward to those events. They offer me a chance to talk to new peoples, chance I use to dread. And it’s not only for technical stuff, I am not as reluctant as I use to be to talk to strangers.
When developing web applications, we often run into performance issues. People often blame PHP or MySQL for bad performance, but in most case the answer is not that easy. Blindly trying to optimize random parts of our applications can lead to some uneven results.
There are many available tools to profile a PHP application. Learning how to use them can help us pinpoint which parts are slow. With this information we can pick the optimizations that will give us the best results.
This post describes the installation and configuration of some of them. I tested them in a Ubuntu 10.10 virtual machine. If you want to try those tools, don’t forget that they can greatly impact the performance of you web server when they are active. Installing a tool like Xdebug on a production server is not recommended.
Benchmark Your Application
First you need to benchmark your application. If you don’t, you won’t be able to know if you improve the performance or degrade it. If you make some changes, a gut feeling that the site is faster is not enough, you need to have numbers. Siege is a load testing tool. It will tell you how many request a second your application handle and the response time.
Installing Siege is pretty straightforward, download it, untar it and compile it with make.
tar -xvf siege-latest.tar.gz
cd siege-2.70/
./configure
make
sudo make install
siege.config
Running siege.config will generate a .siegerc file. You might need to edit it and change where the log file is generated. By default, the logs where stored in /var/siege.log and I didn’t want to run it as root. So I uncommented the logfile line to store it in my home folder. You might want to set verbose to false also.
When Siege is install, you can use the following command to execute it for 30 seconds with 5 concurrent connections.
siege -c 5 -b -t30s http://localhost
** SIEGE 2.70
** Preparing 5 concurrent users for battle.
The server is now under siege...
Lifting the server siege.. done.
Transactions: 143 hits
Availability: 100.00 %
Elapsed time: 29.75 secs
Data transferred: 0.27 MB
Response time: 1.02 secs
Transaction rate: 4.81 trans/sec
Throughput: 0.01 MB/sec
Concurrency: 4.92
Successful transactions: 143
Failed transactions: 0
Longest transaction: 1.54
Shortest transaction: 0.94
The results will be appended to the log file. This makes it easy to compare the numbers after you make some changes. You can also see when the performances really start to degrade if you add users.
Xdebug
Xdebug is a debugger for PHP. I already talked about it in my post on Debugging PHP In Eclipse. It can also be used for profiling. You can install it with Apt.
sudo apt-get install php5-xdebug
To configure it, open the Xdebug ini file, on standard Ubuntu it’s in ‘/etc/php5/conf.d/xdebug.ini’. Add these lines:
xdebug.default_enable=1
xdebug.scream=1
xdebug.profiler_enable=1
xdebug.profiler_output_dir=/tmp/xdebug
This will make sure that Xdebug append a stacktrace to errors. And the scream option will disable the PHP error control operator (@). This way all errors will be displayed. It enable the profiler and change it’s output folder. Make sure the folder exits and that apache can write in it.
When Xdebug is installed and configured, restart Apache and browse to your web application. Xdebug will create some files in the folder you picked. You can then use KCachegrind to view those files. You can install it with this command:
sudo apt-get install kcachegrind
On a Gnome machine, this will install lots of dependencies. But after, you will be able to open the output file from Xdebug. You should get a window that looks like this:
KCachegrind Output
In KCachegrind you can see where your program spent most of it’s time. How many time a function is called. This information should help identify where you should start optimizing to get the most out of your time.
If you prefer a web front end, you can use Webgrind to inspect the result of Xdebug.
XHProf
Xdebug is a very good tool to check where your bottlenecks are in development. However, you should not use it in production. Facebook developed XHProf a profiling tool that can be used on a production server.
To install it, download the latest version and run those commands:
tar -xvf xhprof-0.9.2.tgz
cd xhprof-0.9.2/extension/
phpize
./configure --with-php-config=/usr/bin/php-config5
make
make test
sudo make install
Edit your php.ini and add those line at the end:
[xhprof]
extension=xhprof.so
;
; directory used by default implementation of the iXHProfRuns
; interface (namely, the XHProfRuns_Default class) for storing
; XHProf runs.
;
xhprof.output_dir=/tmp/xhprof
Make sure to change the output dir to something appropriate on your machine.
To profile your application, you now need to add some code at the beginning and at the end of you page execution. Add this at the beginning:
xhprof_enable();
And this at the end:
$data = xhprof_disable();
$XHPROF_ROOT = '/home/testing/Downloads/xhprof-0.9.2';
include_once $XHPROF_ROOT . "/xhprof_lib/utils/xhprof_lib.php";
include_once $XHPROF_ROOT . "/xhprof_lib/utils/xhprof_runs.php";
$xhprof_runs = new XHProfRuns_Default();
// Save the run under a namespace "xhprof".
$run_id = $xhprof_runs->save_run($data, "xhprof");
XHGui
XHGui is a tool that allow saving the data from XHProf in a database and displaying it in a nice UI. It also comes with two files that facilitates adding XHProf to your silte. To install it, clone the repository from GitHub. And follow the instructions in the INSTALL file. Here are the instructions for a machine using MySQL for storage.
git clone https://github.com/preinheimer/xhprof.git
cd xhprof
ln -s xhprof_lib/utils/xhprof_runs_mysql.php xhprof_runs.php
mv xhprof_lib/config.sample.php xhprof_lib/config.php
Look in the xhprof_runs.php file. There is a create table query. Create a database for XHGui and execute the query. Edit the file xhprof_lib/config.php and enter the information for your system. Don’t forget to set doprofile to true. Also, you need to uncomment one of the section for the paths where XHGui should put its files. There are default values for Windows and Linux.
XHGui comes with two files that makes profiling easier. A header file to start profiling and a footer that will end it and persist the information in your database. To automatically insert them in all your site, add the following lines to you virtual host:
php_admin_value auto_prepend_file "/home/testing/Downloads/xhprof/external/header.php"
php_admin_value auto_append_file "/home/testing/Downloads/xhprof/external/footer.php"
You will also need to create a virtual host for XHGui. Then after your site have been visited a few times, you can navigate to you XHGui installation and you will have a page that list the last 25 runs of your sites. Pick one and you should see a page like this one.
From this page you can view a lot of information about your pages. It contains statistics about the time the page took, the CPU and memory usage and the number of functions calls. At the bottom, there is a list of functions with the same information. It makes it easy for you to see where the page takes the most time or resources. This will help you find targets for optimizations that will really impact the use experience on your site or the machine where it runs. You can click on any function and view a break down of the functions it calls.
On the main page of a run and in the functions pages, you can generate a call graph with just on click.
It gives you a nice visual of the functions called when you display a page. The functions that can be problematics are in red or yellow, so you can spot the easily.
This Is Just The Beginning
This is only an introduction to some of the tools that are available for PHP developers. I want to play more with them, and maybe post more information when I get more familiar with them. If you don’t do any profiling, you should probably try them too.
One important rule when optimizing, always measure. Benchmark before and after every change to make sure you are going in the right direction.
Update 2011-04-01
php|architect announced yesterday their first Annual Impact Awards. They nominated quite a few amazing projects, the choices are not easy. However, I think XDebug is an outstanding tool that impact the work of many PHP developers every day. Derick Rethans is doing some important work with it. He has my vote for the award. If you are a php| architect subscriber, go cast your vote.
Tags: performance, PHP, profiling, seige, Xdebug, xhgui, xhprof
I have been programming in PHP for 5 years now. I did not choose this language, I had to learn it to work on the code of the company my bosses bought. Since I started, I heard many rants about how bad PHP is. Some where valid, but a lot of them are just blaming the language for bad code written by bad programmers.
I am by no mean an expert in programming languages or PHP, but here is my take on the language I use every day.
What I Love
The first thing that comes to my mind about PHP is it’s ease of use. When I started PHP, I had experience in C++, Delphi, VB and C#. The low barrier of entry allowed me to be productive in it quickly. I lacked a lot of knowledge, especially in web development, but I was able to maintain the existing code right away. I no time I could add new functionalities to the existing code.
The PHP documentation is an amazing resource. It’s available online, but it can also be downloaded in html of chm format. The manual contains everything there is to know about the language fundamentals. It has descriptions of every functions, their parameters and return values. It also contains examples on almost every pages to help understand what the page is about. The manual is maintain by the community, and most of the pages have notes from users that clarifies the subject even more.
Over the last five years I have become a web developer and PHP is targeted at web development. It can be embedded in html to produce dynamic pages quickly. I know this can lead to spaghetti code, but if used correctly if a powerful tool. PHP has many features that makes the life of web programmers easy. Things like writing and reading cookies, sessions handling and file uploading are easy to do.
PHP also has an interactive shell. I learned about this about a month ago. For a long time I hoped that PHP had one. I used the interactive Ruby prompt, I saw it it other languages and every time I was thinking it was a neat thing. But I never though of looking for one in PHP. Then last month I saw it in the manual and now I feel stupid for never searching for it. It’s a great reminder that I know so little and that because of that I am not always using the best solutions for the problems I have. You cannot use a solution you don’t know about. I don’t use the interactive shell much, but it’s a great tool to make a quick test before implementing something.
The community around PHP is fantastic. It is very friendly and forthcoming. There are many use groups around the world. Here in Montréal we have PHP Québec that organize monthly meetups. Once a year, they hold a family barbecue in a park an they occasional do nights in a bar where we can speak with other developers around a beer. And last year with other Montréal user groups they started the new Confoo conference.
This is just the Montréal PHP community. If you look on the web, the worldwide community is nice and helping. In Coder at Work, Joshua Bloch says that when you choose a language, you choose a community. He compares it to choosing a bar, you want good beer, but you also want it to have nice people that hang around. I did not pick the PHP bar, but I am glad I ended up with that bunch.
What I Hate
PHP has a few fiction point. The first thing that comes to my mind is again the low barrier of entry. This is both a blessing and a curse. Because it’s easy to code in PHP, many peoples who don’t know anything about programming are making PHP web sites. Because of this, there is a tremendous amount of bad PHP code out there. Most of the web pages they build have security vulnerabilities. As a result, many people consider PHP a bad language, and think that all PHP programmers are bad. This is far from the true, there are very good PHP developers out there. And there are bad programmers in every languages. PHP just tends to attract the beginners.
One thing that can be confusing in PHP is that the order of some parameters changes between functions. The needle/haystack example is known to all PHP developers. The strpos function takes the haystack parameter first and the needle second. The array_search takes them in the opposite order. The array_map and array_filter have the same kind of inconsistencies with the callback parameter.
Another annoying things to me is that the constructor of a derived class does not call the constructor of the base class. This can lead to some problems because if you forgot to call it explicitly, the properties of the base class might not be initialized. The base constructor should probably be implicitly called on every instantiation. But we would also need a way to pass parameters to the base class constructor. Maybe something like in C++ would do it.
Conclusion
PHP is not a perfect language, there is no such thing. However, I think it is a very good multi purpose language. I was able to be productive with it in a few days when we inherited from a bad code base. Since then I have learn much about it and I still have a lot to learn. The language keeps improving, with version 5 we got decent object oriented programming. With 5.3, closures and namespaces And with 5.4 we will get traits.
PHP as a lot of detractors, it also has some fan boys, don’t believe any of them. Ask for more details, what they think is so bad or good about it. And go ahead and try it for yourself and make your own opinion.
Tags: PHP
Recent Comments