ls –color considered harmful

Posted at 2008-01-26 18.57

This is a rant I have made many times in person: far too many to count. Since I can't possibly warn everybody who might be affected by this, face to face, I'm hoping you will all spread the word.

The problem

To begin with, I should say that ls --color is useful. It can show you which items in your directory listing are directories, symlinks, broken symlinks, device nodes, named pipes, whatever. I am glad that it exists.

However, let’s consider the basic action of ls. It is, of course, to display which files and directories exist in a directory. This is really quick and easy in Unix. Simply use the libc function opendir() then just readdir() till it returns NULL and errno is 0. In terms of syscalls, this is an open(), one or two read()s and a close().

When you turn on the color option, every directory item incurs the weight of an extra stat() call to see what it is.

Worse still, if it’s a symlink, ls will obligingly bound across to the thing linked to, to check it exists. If this causes your automounter to wake up and mount something remote, that’s quite an overhead. If your mount hangs because the remote filesystem is unavailable, you can say goodbye to that process; it'll be stuck blocking on a syscall that will never return.

A partial solution

To fix the symlink problem, here’s what you do. ls was written to avoid chasing symlinks if configured not to. Check your environment variables for the LS_COLORS variable with:

$ env | grep ^LS_COLORS

If it exists, you have a config file for colour ls: you should edit /etc/DIR_COLORS* or copy it to $HOME/.dircolors and edit it there. Otherwise, type:

$ dircolors --print-database > $HOME/.dircolors

The edit you need to do is to remove or comment out the ORPHAN line from the file.

Unless you were able to edit the file in /etc, you now need to make sure that this new $HOME/.dircolors file is used. In your shell login script (.bash_login for bash and .login for (t)csh, put the line:

eval `dircolors $HOME/.dircolors`

(Note that these are backticks, not apostrophes.)

Next time you log in, you'll be able to use colourful ls without fear of a remote filesystem freezing your directory listings.

In conclusion

Use ls --color when you need it, but as with all powerful tools, use it with care.

Corrections: James pointed out that to dump the default config it is –print-database not –print-directory.

6 Comments for 'ls –color considered harmful'

    Matthew W. S. Bell
    2008-01-27 | 06.16 +0000

    Nah, you're confusing the symptom with the problem(s):

    - network file-systems are rubbish
    - auto-mounting is a slightly broken idea
    - file/directory API is not ideal for high-latency file-systems

    2008-01-27 | 09.25 +0000

    Whilst I agree with your bullet points Matthew, I would say that the default configuration of ls –color exacerbates the situation greatly. The other day, a colleague was puzzled that ls ~user1 worked when ls ~user2 hung. (His system had 'alias ls=ls –color=tty' in its defaults). Turns out, user2 had a symlink in his homedir to a filesystem that was unavailable.

    This action at a distance is a clear violation of the principle of least surprise.

    2008-01-27 | 10.51 +0000

    To quote the dircolors manpage on Debian: "FILES
    (Slackware, SuSE and RedHat only; ignored by GNU dircolors(1) and thus Debian.) System-wide configuration file.

    (Slackware, SuSE and RedHat only; ignored by GNU dircolors(1) and thus Debian.) Per-user configuration file."

    Also dircolors –print-directory doesn't work on Debian.

    2008-01-27 | 16.32 +0000

    James: I've updated the article in light of your final point.

    2008-01-28 | 18.12 +0000

    Do you also consider ls -l harmful?

    Symlinks to automatic mountpoints are broken, period.

    2008-01-28 | 21.32 +0000

    Marius: no, it’s not hugely harmful. Likewise, ls –color, with disabling orphan checking is similarly harmless. However, ls -lL is dangerous. Fortunately, people don't alias ls to ls -lL. Interestingly, I've just discovered, ls -L stats all sorts of things yet doesn't change its output on the basis of it!

Leave a comment



Information for comment users
Line and paragraph breaks are implemented automatically. Your e-mail address is never displayed. Please consider what you're posting.

Use the buttons below to customise your comment.

RSS feed for comments on this post | TrackBack URI