ack: powerful command line search

ack is a fantastic tool to do recursive searches in text files, such as in your source code. It is simple to use and amazingly fast. See the ack home page for installation instructions. I am on OS X without Macport and installed ack with:

sudo perl -MCPAN -e 'install App::Ack'

Ignoring certain files

Out of the box, ack will ignore "most of the crap you don't want to search" (quoting from their documentation!), but you might want to set it up to ignore more files. For instance, for every JavaScript file (gaga.js) I have a minimized version of that file (gaga-min.js) automatically generated for me. I am never interested in searching in those files. ack doesn't have an option for you to say which files you want to ignore, but you can specify what files you want it to include: -G. This option takes a Perl regular expression. How to write a regular expression that matches any file except those ending with -min.js? With a negative look-behind regexp:

(?<!-min\.js)$

To have this used by default, add the following two lines to your ~/.ackrc:

-G
(?<!-min\.js)$

Adding file types

ack only search in files that are of a "type" it knows about. In general, files are mapped to types by extension, but in some cases they can be mapped by their content. I ran into a case where ack would find what I look for in certain .xpl files¹, but not others. The reason was that ack didn't know about the .xpl extension, so it would ignore those files, except those that started with an XML declaration. For those files, even if it didn't recognize the extension, it would recognize the content and know it is dealing with an XML file.

If you run into a similar situation and suspect this is your problem, run the following command which will list the files ack will search in, starting from the current directory:

ack -f .

After you find which file are ignored, add those to your .ackrc so ack recognizes those files. For instance, the following parameters added to your .ackrc will get ack to recognize .xpl and .xsl files as XML files:

--type-add
xml=.xpl,.xsl

¹ .xpl are the pipeline files we use in Orbeon Forms.

Git Tips

Last updated: March 15, 2010

A flurry of excellent documentation and great blog posts has been published about Git. But who doesn't need some additional tips on how to better use your source code management tool of choice? So here are a few that I found useful:

Table of contents

Ignoring files
Getting rid of the warnings when pushing
Finding when and why that code was changed
Mac OS X: colors for your terminal
Handling pull requests
Avoid "Merge Branch..." in the commit history

Ignoring files

I always have a number of files in my projects that I have no intention of checking in, such as a TextMate project file (orbeon-forms.tmproj) or scratch files I use for testing (gaga.xhtml). Those files pollute the output of your git status. Sure, you are smart enough to ignore them, but why not let Git do the ignoring for you? Do so by listing the files you want Git to ignore for this current repository in .git/info/exclude.

Getting rid of the warnings when pushing

When running a git push without specifying what branch to push, you'll get the warning message You did not specify any refspecs to push. By default git pushes all the local branches for which there is a remote branch with the same name. It does so, but still tells you that this may not be what you want. To have git push only the current branch (and also get rid of the error message every time you do a push), run:

git config --global push.default tracking

You could use current instead of tracking, but it won't have the desired effect when the name of your local branch doesn't match the name of the remote branch you want to push to. Really, this should be the default.

Finding when and why that code was changed

Run the following command and locate the piece of code you are interested in:

git blame your-file.ext

Copy the revision number (rev) showing at the beginning of the line and run:

git log -1 -p rev

The -1 tells Git you're only interested in that particular commit, and none of the previous commits. The -p as in patch gives you the changes to the code done in that commit.

Mac OS X: colors for your terminal

First, for you terminal to support colors, add the following two lines to your .bash_profile:

export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagacad

Then enable colors in diff and status with:

git config --global color.diff true
git config --global color.status true

Handling pull requests

(This is based on the GitHub documentation and example.) Start by adding the remote repository. The first defunkt is the local name you give to the repository at the URL that directly follows. The -f is to immediately fetch that repository (fetching retrieves the data from the remote repository to your local repository):

git remote add -f defunkt git://github.com/defunkt/grit.git

Checkout the code from your local repository to your working directory and at the same time create a branch named defunkt:

git checkout -b defunkt

Now that your working directory is in the branch defunkt, pull the latest code from the master branch of the repository defunkt to the working directory (if you just did the git remote add -f, most likely there will be nothing new to pull):

git pull defunkt master

To avoid having to specify defunkt master on the command line every time you do a pull (of course you want to pull from the same repository and branch you originally got the data from!), run the two following commands. The first one defines defunkt as the default repository for the your branch defunkt. The second one defines the remote master branch as the one you want to merge with the current branch. (It is a config for merge because pull runs a fetch and a merge.)

git config branch.defunkt.remote defunkt
git config branch.defunkt.merge refs/heads/master

Switch to your master branch:

git checkout master

Merge the branch defunkt into your current branch (master). The merged commits from defunkt will go to your local repository.

git merge defunkt

Before you push the changes, you'll want to review and test them. You can see what you would send to the remote repository if you were to do a push now with:

git diff origin/master..master

If you make changes to the files in your working directory (say, to amend the changes made by the developer who sent you the pull request), you can see a diff between your working directory and the remote repository (which will include the changes your pulled plus your changes) with:

git diff origin/master

To only view a summary of what was changed (the diffstat showing files names and how much was removed/added), add the --stat parameter:

git diff --stat origin/master

Avoid Merge Branch... in the commit history

You do this by rewriting your local commits to be applied to the new state of your local branch as it will be after the changes you pulled from a remote source have been applied. You can do this in a single command:

git pull --rebase

You always want the rebase to be done when pulling changes, and would like to avoid having to put that --rebase every time you do a pull? No problem, run the following just once (assuming you want to enable this for the master branch):

git config branch.master.rebase true

Also see Rebasing is Editing Commits, which provides background information on the subject.

Delete All Files With Exceptions Using find

Gavin suggested a clever trick to recursively get the list of all the files in in a directory, except those under .svn. It goes like this:

find WEB-INF -path "*/.svn" -prune -o -print

I used this quite a bit, but never thought about how this really works, until today:

• The -o is an or operator. So either -path "*/.svn" -prune or -print is true. Let's look at each part.
• The first part -path "*/.svn" is true for the .svn directories and with -prune we skip the content of those directories.
• If you just had the first part, you would be skipping over the content of the .svn directories, but not over the directory itself. The second part -print prints what isn't matched by the first part (because of the or). So it won't print the .svn directories.

For sure, this is not the most intuitive expression I have ever seen! Now we can extend this to do other things. For instance the expression below will delete all the files in the current directory, but it won't delete:

• directories;
  
• files in the CVS directories.

find . -path "*/CVS" -prune -o -type f -exec rm {} \;

This can be useful for instance when you have files checked out from CVS or Subversion and want to replace all your files by a copy of those files that you receive from someone in an archive.


Recursively Excluding Directories with Info-ZIP

You want to compress the content of a directory, ignoring all the .svn files created by Subversion (or the .cvs files from CVS). Info-ZIP unfortunately doesn't have a simple options to do that. So instead, here is what you can do:

find . -name .svn -exec echo {}/* \; > /tmp/exclude.txt; zip -q9r archive.zip * -x@/tmp/exclude.txt

With find, you first output the path to all the .svn directories into a temporary file, adding "/*" at the end of the path. Then you pass that file to zip with the -x@ option.

This works just fine, but is there a better way, maybe without using a temporary file, with everything in one line?

Update November 1, 2007 - The following, suggested by Gavin in the comments of this post, works like a charm and fits on one line. Here we are compressing the content of a WEB-INF directory to create a WAR file:

find WEB-INF -path "*/.svn" -prune -o -print | xargs zip myapp.war

Update February 16, 2009 - Tastenmetzger suggested in the comment an even simpler way of achieving the same result, using just the zip command:

zip -r myzip.zip * -x *.DS_Store *.svn*

Isn't that beautiful?

Where Is That Which?

Yesterday afternoon I installed Oracle 10g on my Windows notebook. After doing so, I notice that typing java on the command line run Java 1.4.2_03, while I had version 1.4.2_07 just a moment ago. So I type which java; it tells me that the java.exe is the one in \windows\system32 (apparently Oracle decided to overwrite a pre-existing copy of java.exe that I had in \windows\system32). At this point you might be wondering: What is this which command? Did't the guy say he was running on Windows? Is he using cygwin? This is no cygwin, just a batch file written by my friend and Windows guru Dan Small, and published here with his permission. How to install? Copy the code below into a file called which.bat and put the file somewhere in your PATH. Enjoy!

@echo off
setlocal ENABLEDELAYEDEXPANSION

if "%1" equ "" (
 echo %0 file to find on path
) else if "%~nx1" neq "%1" (
 echo %1
) else (
 set exts=%~x1
 if "!exts!" equ "" set exts=%pathext:;= %
 for %%i in ( !exts! ) do (
   set name=%~n1%%%i
   for %%j in ( !name! ) do (
     set fqn=%%~$PATH:j
     if "!fqn!" neq "" (
         echo !fqn!
         goto :EOF
     )
   )
 )
)