Criteria making software useful long-term & worth learning in detail (Haskell, predictions)
created: 26 Sep 2008; modified: 01 Dec 2013; status: in progress; belief: possible

It’s not always clear what software to pick in the open source world.

For some tasks, the choice is easy since there’s one dominating player - for web browsing you will almost certainly want to go with Mozilla Firefox when in X.org, and ELinks when on the command-line.

But with other applications, the array of choices is quite astounding. For an editor, Vim or Emacs or… For a shell, do you use Bash, Fish, Zsh, Tcsh, Ksh, or what? Each option has many complex pros and cons, and often partisans have strong feelings about the issues.

Myself? I’ve found it useful to try and classify all the options by a few considerations. Since we’re interested in Unix applications, one can safely say that we’re all “power users” and are interested in the best we can get. So the first desiderata is that:

# Maintenance

• It should be maintained.

Life is too short to waste dealing with unmaintained software. Such software is probably already ridden with bit rot, and it may well have been abandoned for a reason.

The point of a program is to make something easier or to do something you could not do before; programs are generally hard enough to use as it is, and a package being unmaintained almost guarantees that you will spend more time on it than an equivalent maintained package. There are usually plenty of choices, so go use one of them.

A commitment to maintenance is also a useful thing. Take GNU Emacs, or the GNU command line utilities in general. There are quite a few of them, they are complex and powerful, and take a while to learn. To profitably use them, you will need to invest time - I had learned enough about the Coreutils to use them comfortably within a few months, and GNU Emacs took about half a year to really learn to any satisfactory degree. Other alternatives like ‘nano’ would have taken less time to learn, but the GNU tools offered me more power.

More importantly, the whole reason I was willing to spend so much time learning Emacs was because Emacs has a history. It was begun in 1984 and has been in constant development in the more than 2 decades since. It is likely to continue to be developed for decades (if only by the Doomsday Argument) by the same gnarled old hackers who wrote it originally.

Even better, it has the institutional support of being the GNU Project’s official text editor. In other words, learning Emacs is indeed a steep price to pay but that’s alright since it is entirely plausible that you could be using it for the rest of your computing life. Six months of not particularly productive text editing may sound like a lot, but when set against 20 or 30 years of use, the productivity gains seem worthwhile.

Being maintained is just the most basic criterion though, and many tools will meet that basic level. Unmaintained projects tend to be forgotten, and you usually won’t see them discussed in lists of projects, so in a sense you can just assume tools are maintained. The second, more difficult feature is:

# Extensible

• It should be extensible.

There are very few tools which you will want to use, now and forever, exactly the way they come. Even a command as humble as ‘cat’ nevertheless has >9 options! For something as complex as a browser or text editor, you will want to customize and most probably extend it. Rather than using nasty hacks to get around this fact, just use a tool which can at least be scripted.

• However, scripting isn’t always the best way. Probably the best way is the Emacs-style program architecture1: a simple, efficient, and performant core which supports a panoply of extensions - the sum of which is the application. If the bulk of the application is being written as extensions, you get numerous benefits which include:
1. Modularity; and the ability to disable whatever you do not need (with attendant benefits in performance, reliability, and security)
2. Ease of extension. Obviously, since the developers would have striven to make the large body of code - which they will spend most of their time editing - clean and easy to change. If you have to do everything in Perl or Emacs Lisp, for example, you are obviously going to have nice libraries and features to use. These features and libraries will also be available to you.
3. Portability. The small core is all that needs to be ported, sometimes. Portability is always a good thing.
4. Features. Emacs contains the proverbial kitchen sink; Irssi has more extensions and skins than I can count; and Firefox has precisely 2.89 million add-ons. If users can add features without waiting for the official developers and release cycles, they will. And odds are pretty good they may have already done exactly what you needed, or solved a need you didn’t even realize you had.

It’s lovely to be able to make use of other people’s extensions in your shiny new toy, but do you know what to do with it now? Obviously:

# Comprehensibility

• You should be able to understand it.

This is a valuable property for a number of reasons also. An example I like to use is the XMonad tiled windowing manager for X. Before, I had used Ratpoison and StumpWM, but I was never really satisfied because I didn’t understand C or Common Lisp so many things were and remained a mystery to me. This made extending them, debugging them, even just reading the documentation to use them - it made them all more difficult than they had to be.

With XMonad, I understand most of it. I see how the extensions work, and have written my own. The imperfect mirror of the documentation does not confuse, as I can easily turn to the code which is the ultimate documentation. This point ties into the first and second points. If you understand the code and the architecture, then you are in a much better position to extend it, and potentially maintain it.

This goes back to the first point; you should never need to maintain it, but given your investment in it, you may want to and it’s better to be able to maintain it to the degree you need it to function - sometimes maintenance is just a matter of updating a few files or the packaging system.

# Popularity

• Try to use standard stuff.

For example, Bash is by all the former criterion a good choice for one’s shell. It is maintained by the GNU Project, it is certainly extensible and powerful, and it’s relatively understandable. But the really nice thing about Bash is how very standard it is: every Linux distribution includes Bash, and even set it as the default shell.

And if an operating system doesn’t run Bash as default, it is basically guaranteed that Bash will run on it (or there will at least be shells with Bash-compatibility). This is an advantage you should not underestimate. Because of this ubiquity, there are powerful network effects operating in favor of Bash guaranteeing that it will be useful for you for a very long time. Standard means you will be able to get your hands on it, standard means there will be documentation and people working to solve bugs and other people taking advantage of the extensibility, and so on. Standard means that new implementations and forks are that much easier. Standard means many things, but from a user’s perspective, standard is generally a good thing for software and languages to be.

The final major consideration was left to last. It should come as no surprise to modern programmer - sinfully awash amongst high-level garbage-collected interpreted languages as they are - that the least important factor to consider is the level of:

# Efficiency

• efficiency of a program.

This is the final concern because applications spend little time doing inefficient things. The rationale here is that, perhaps Emacs may be somewhat slower than a less capable or portable editor - but you will not save so much time on quicker screen updates &etc. as to make up for real hours lost in porting the other editor or working around its lack of features (see Amdahl’s Law).

There are other concerns here: it is easier to make an extensible/powerful/portable editor more efficient in the bottlenecks than it is to hack in those features to an efficient editor never intended for those features2. But if you are truly faced with two choices - each of whom seems equally solidly maintained, extensible, standard, etc. - then go with the more efficient one. After all, efficiency is a virtue and does make the experience more pleasant and gives you the option of using it on slower computers.

Alas, just marking down these features doesn’t solve every problem. For example, choosing between Vim and Emacs was easy for me - Vim’s supposed efficiency didn’t (in my reckoning) outweigh the increased standardness and comprehensibility of Emacs. Or take Bash compared with Zsh. Bash is clearly more standard and is clearly better maintained, being even older than Zsh and the official shell of the GNU Project, and it is probably marginally more efficient. But I nevertheless chose to switch to Zsh.

Why? I just liked it better. It has more features, yes (but not significantly more); it has Bash compatibility (sure, but not as good as the real thing); it makes my configuration files cleaner and easier to maintain, sure (but they were difficult to convert, and are now no longer quite so portable). Basically every point adumbrated argues strongly for Bash and at best, weakly for Zsh. But I liked it better, and in the end that matters.

# Short configurations

What other things should we keep in mind? Well, as time passes, I’ve found configuration files tend to grow. A good thing to do when a configuration file has grown particularly elaborate is to simplify it.

Slim it done, remove everything you haven’t needed in the last several months. If you still have a huge file, see whether a large chunk of it makes sense to split out.

For example, for the Zsh I have: ~/.zshenv, for environmental variables and shell settings that all Zsh invocations need; ~/.zaliases, hand-written aliases and functions; ~/.ztypos, containing all aliases automatically written to correct typographical errors I have made in the past; ~/.zlogout and ~/.zprofile, for logout/login stuff; and finally, ~/.zshrc, which contains everything needed for pleasant interactive use. Clearly it’d be absurd and extremely awkward to stuff all of that into a single file, as well as potentially slow.

But suppose you have removed cruft, written stuff as well and concisely as you could, and split files out, and you still have too much? Now it comes time to give back to the free software community. Consider carefully your settings: perhaps they are very common and you could file a bug suggesting they be made the default? That would then be a line you could remove. Or perhaps you have on your hands a de facto useful extension to the program you could package up and offer online somewhere, or even offer to the developers as something to add to all the other extensions included by default.

# Take one, leave one

There is one last piece of advice:

• For every new tree planted, let one be uprooted.

This is a good rule to follow with all possessions. If you buy a new pair of shoes, give away or junk the old pair which you no longer need or can use. If you purchase a new book, sell or donate that old one you will never read again. If you add one program to your system, try to remove another - perhaps one you never used, or perhaps one made redundant by the new one. If you install ELinks, remove Lynx.

Redundancy wastes resources, it imposes mental burdens, it increases inertia and resistance to change. The more stuff you have that you do not need, the stupider your possessions get. If you have a room bursting at the gills: you cannot freely travel. You cannot easily change your place of residence. You are no longer sure what do you and do not have, you are increasingly mentally stressed. You literally can no longer freely move about. In extreme cases, people have been known to die because they refused to let go of things3.

Try to keep matters elegant and uncluttered. There is a difference between elegance and competence - and this difference makes all the difference in the world.

1. For my purposes, the many variants such as the XMonad model are immaterial. Yes, it is a very interesting model (the program compiles a fresh binary with the user’s new config hardwired in, serializes out the current global state, and restarts itself with the new binary and passes it the serialized state) - but the details of how the extensibility is accomplished are only important if the method is crappy and unreliable.

2. As a bigoted Emacs user, I of course see Vim as the classic example of attempting to shoehorn in a feature like a first-class extension language - with humorous results!

3. Often due to mental disease; see Pathological hoarding