Eric Hogue on January 11th, 2011

When I need to do something like a search and replace across multiple files, I usually write a PHP script. I always knew I could do it faster with grep and sed, but I was too lazy to research how to do it. Last week I finally decided to find a better way. I’m entering it here so I can easily find it back.

You can use this technique to replace any string you want. For example, to change a class name you would do something like this:

grep -RlZ OldClassName * | xargs -0l sed -i -e 's/OldClassName/NewClassName/g'

grep

grep will search through files for a regular expression. The R option make the search recursive so it will go down in the the directory structure. The l option tells grep to output only the names of files where it found a match. And with Z, grep will separate the results with \0 instead of \n, we will see later why this is necessary. You can also add the i option if you want to make a case insensitive search. After the options, we give grep the expression to search for and the files we want to search. In our case, we only search for a class name, but we could have used a regular expression.

xargs

xargs is a tool used to build a command from the standard input. So instead of writing a loop to go through the results from grep and passing them as parameters to another command, we let xargs take care of it. It will execute the given command on all the files returned by grep. We give xargs the -0 option to tell it that a \0 instead of the whitespace it expect. This will allow us to process files with whitespace in the name. The l option is used so xargs will pass the file names 1 line at the time to sed.

sed

sed is a stream editor. It take text from a stream (a file in this case), edit it in some way and output it back. In this case, we will pass it all the files found by grep and ask it to replace all occurrence of OldClassName with NewClassName. To make sed act on the files found by grep, we pass then using a pipe and xargs. We use the i option to make sed edit the file. If you give -i an extension, it will create a back up of the original file. The e options tells sed to add the string between quotes to the command. This is the regular expression to be executed. In this case, the expression is pretty simple. The s tells sed to replace OldClassName with NewClassName. The g at the ends make it global, so it replace all occurrences.

Other Uses

The value of learning this technique is that it can be used to replace any string you want. With regular expressions, you can use it to replace a phone number across multiple html pages. It doesn’t matter if there are hundreds of files, and if the phone number is repeated many times in each file. Just build the correct regex and you’re set. And with the flexibility of regular expressions, you can replace one phone number, or any phone number found on your pages. If your files are under source control, you can easily look at the diff before committing the changes made by sed. If they are not, start by putting them in source control.

To me, this command shows the power of command line tools. They are built in a way that makes it easy to use them together, passing the result of a command to the next one. I really need to work on my knowledge of them and start using them more in my daily tasks.

Tags: , , , , ,

Eric Hogue on January 8th, 2011

Here are my goals for 2011. I am making them public to increase my motivation. Having them visible to everyone makes me more committed to them than when I was keeping them private. It will also leave a trace so I can make a retrospection at the end of the year.

So here they are in no particular order:

Start A Blog And Keep It Alive

I already started the blog. That was the easy part. Now hopefully I will keep on posting through the year and after. I aim to post a least once a week.

Learn Ruby and Rails

The pragmatic programmers wrote that we should learn at least one language a year. For the past couple of years I have been pretty lazy on this. I tried Ruby a little bit last year and I liked it, but I did not pushed enough. This year I want to really give Ruby a go. I want to use it to build a few breakable toys and then give Rails a try. Once I’ll start to feel good programming in Ruby with Rails, I should try to pick an open source project and start coding on a real Rail application.

Start Doing Continuous Integration

I have read a lot about continuous integration, but I have never gave it a try. This kind of practice is not encourage where I work. But this is not an excuse, anyway even at work it has begun to change.

I try to use TDD as much as I can, and I want to push on this more this year. I want to set up a Hudson server at home and explore all I can do with it. Automate the unit tests, learn to automate the deployment of the build, check the code coverage, validate that the coding standard are respected…

Read 20 Technical Books

This should be an easy one. I love to read and I am trying hard to learn new things and open my mind to new ideas. My Shelfari bookshelf already have 32 books I want to read. And every books I read has more recommendations. I add them to my list faster then I can read them.

Experiment With a NoSQL Database

For my storage solutions, I rely only on relational databases. Since I have not try any NoSQL solution if I come across a problem where it would be the best solution I probably won’t even think about it. I have read a little bit about CouchDB, but I need need to actually start playing with it or another NoSQL database.

Become a Zend Certified Engineer

I will keep my reasons for getting the Zend certification for another post, but I think it’s about time I do it. And I want to do it early this year. I aim at getting it for the end of February.

I have already begin my preparation. I started reading the PHP manual and I want to read the Zend PHP 5 Certification Study Guide from PHP Architect. I also installed the PhpRiot iPhone app and I want to do as many practice tests as possible. I am aware that most of this material is for the PHP 5 certification, but most of it is still relevant to the PHP 5.3 certification.

Learn To Use Vim Effectively

Vim is a tool I use daily. However, I use it like any other editor. I use the arrow key to navigate, I use the mouse, use only one buffer at the time… I know Vim can be way more powerful than this. So this year I will learn to use many more of it’s functionalities.

I already started with this. I found some Kata that I practice every day. I installed Vrapper to emulate Vim in Eclipse. And whenever I edit a file, I try to be aware of how I use Vim functionalities. I already see a difference in my editing, and some of this new knowledge of Vim commands is starting to make me more productive. The downside of this is that I’m beginning to miss Vim when I work in other editors, especially for navigation.

Learn The Basics Of Web Design

Web design is one of my greatest weakness. As a web developer I know the basics of css. If someone gives me a Photoshop of a web page to reproduce in html I will make it. I might struggle with some part, especially making it work in IE. But in the end the result should look like what I was asked to do.

But if someone ask me to design a web page it will be ugly. I’m a back developer, so the page will be functional. Everything should work as expected, but the result will be scary. For a long time I just blame this on a lack of artistic talent, but it’s not an excuse. I am working on web pages everyday. I should at least know the basic rules about fonts. I should be able to differentiate a good design from a bad one. Understand why a design is good.

Knowledge of the basics should help me avoid the common mistakes. It will not give me talent, but at least if I have to make a web page from scratch, it should at least be presentable.

This Will Be a Busy Year

I have given myself quite a few goals for this year, but I think I should be able to meet them all. I have to be careful though. Since I decided on those goals, I have a tendency to tackle them all immediately. If I do this, I set myself for a big failure. If I attack them one or two at the time I will get great results.

Hopefully, at the end of 2011, I will be a better developer than I am now by far. I am aware of my many weakness and I want to work hard on them. So next year I will give a shot at other weaknesses.

Tags: ,

Eric Hogue on January 5th, 2011

Welcome to my new blog. I intend on using this blog to write about various subjects that should be of interest for developers. I should blog about things I learn while developing at work or on my personal projects. I might also write about the life and career of a developer.

Why Am I Staring This Blog?

Last November I attended Day Camp For Developers hosted by Cal Evans. During the first 2 sessions, Lorna Jane and Elizabeth Naramore gave some pretty convincing talks about how good blogging his and how important it is.

On that day I decided it was about time I started my own blog. But work have been crazy for the last 3 months and with all the overtime I kept postponing it. With the new year, I decided that starting and maintaining a blog would be one of my top priorities. But again I was not doing anything about it. Then yesterday, I saw this post from Lorna Jone and I decided that if I waited after the rush, I was never going to do it. So here it is, I’m starting my blog.

I want to use it has a way to keep track of things I learn. Forcing myself to put what I know in writing should force me to push a little deeper and research more. Also, since it is public, I might get comments that will correct any mistakes I might makes or add more information on the subjects I will discuss. And if what I will write helps someone else, it even better.

It should also improve my writing even if English is not my primary language.

Where Do I Go From Here?
This is the hardest part. Now the I have the blog, I have to make sure I write on it. I already have a few ideas for future posts, but I have to be disciplined and take the time to write them. I also have to get in the habit of noting any idea I get so I can come back to it later and blog about it.