Choosing Software

Criteria making software useful long-term & worth learning in detail
Haskell, predictions
2008-09-262012-10-28 in progress certainty: possible importance: 4

There are so many tools one could use, it’s not always clear what soft­ware to pick in the open source world.

For some tasks, the choice is easy since there’s one dom­i­nat­ing play­er—­for web brows­ing you will almost cer­tainly want to go with Mozilla Fire­fox when in, and ELinks when on the com­mand-line.

But with other appli­ca­tions, the array of choices is quite astound­ing. For an edi­tor, Vim or Emacs or… For a shell, do you use Bash, Fish, Zsh, Tcsh, Ksh, or what? Each option has many com­plex pros and cons, and often par­ti­sans have strong feel­ings about the issues.

Myself? I’ve found it use­ful to try and clas­sify all the options by a few con­sid­er­a­tions. Since we’re inter­ested in Unix appli­ca­tions, one can safely say that we’re all “power users” and are inter­ested in the best we can get. So the first desider­a­tum is that:


  • It should be main­tained.

Life is too short to waste deal­ing with unmain­tained soft­ware. Such soft­ware is prob­a­bly already rid­den with bit rot, and it may well have been aban­doned for a rea­son.

The point of a pro­gram is to make some­thing eas­ier or to do some­thing you could not do before; pro­grams are gen­er­ally hard enough to use as it is, and a pack­age being unmain­tained almost guar­an­tees that you will spend more time on it than an equiv­a­lent main­tained pack­age. There are usu­ally plenty of choic­es, so go use one of them.

A com­mit­ment to main­te­nance is also a use­ful thing. Take GNU Emacs, or the GNU com­mand line util­i­ties in gen­er­al. There are quite a few of them, they are com­plex and pow­er­ful, and take a while to learn. To profitably use them, you will need to invest time—I had learned enough about the Core­utils to use them com­fort­ably within a few months, and GNU Emacs took about half a year to really learn to any sat­is­fac­tory degree. Other alter­na­tives like ‘nano’ would have taken less time to learn, but the GNU tools offered me more pow­er.

More impor­tant­ly, the whole rea­son I was will­ing to spend so much time learn­ing Emacs was because Emacs has a his­tory. It was begun in 1984 and has been in con­stant devel­op­ment in the more than 2 decades since. It is likely to con­tinue to be devel­oped for decades (if only by the ) by the same gnarled old hack­ers who wrote it orig­i­nal­ly.

Even bet­ter, it has the insti­tu­tional sup­port of being the GNU Pro­jec­t’s offi­cial text edi­tor. In other words, learn­ing Emacs is indeed a steep price to pay but that’s alright since it is entirely plau­si­ble that you could be using it for the rest of your com­put­ing life. Six months of not par­tic­u­larly pro­duc­tive text edit­ing may sound like a lot, but when set against 20 or 30 years of use, the pro­duc­tiv­ity gains seem worth­while.

Being main­tained is just the most basic cri­te­rion though, and many tools will meet that basic lev­el. Unmain­tained projects tend to be for­got­ten, and you usu­ally won’t see them dis­cussed in lists of pro­jects, so in a sense you can just assume tools are main­tained. The sec­ond, more diffi­cult fea­ture is:


  • It should be exten­si­ble.

There are very few tools which you will want to use, now and forever, exactly the way they come. Even a com­mand as hum­ble as ‘cat’ nev­er­the­less has >9 options! For some­thing as com­plex as a browser or text edi­tor, you will want to cus­tomize and most prob­a­bly extend it. Rather than using nasty hacks to get around this fact, just use a tool which can at least be script­ed.

  • How­ev­er, script­ing isn’t always the best way. Prob­a­bly the best way is the Emac­s-style pro­gram archi­tec­ture1: a sim­ple, effi­cient, and per­for­mant core which sup­ports a panoply of exten­sion­s—the sum of which is the appli­ca­tion. If the bulk of the appli­ca­tion is being writ­ten as exten­sions, you get numer­ous ben­e­fits which include:

    1. Mod­u­lar­i­ty; and the abil­ity to dis­able what­ever you do not need (with atten­dant ben­e­fits in per­for­mance, reli­a­bil­i­ty, and secu­ri­ty)
    2. Ease of exten­sion. Obvi­ous­ly, since the devel­op­ers would have striven to make the large body of code—which they will spend most of their time edit­ing—­clean and easy to change. If you have to do every­thing in Perl or Emacs Lisp, for exam­ple, you are obvi­ously going to have nice libraries and fea­tures to use. These fea­tures and libraries will also be avail­able to you.
    3. Porta­bil­i­ty. The small core is all that needs to be port­ed, some­times. Porta­bil­ity is always a good thing.
    4. Fea­tures. Emacs con­tains the prover­bial kitchen sink; Irssi has more exten­sions and skins than I can count; and Fire­fox has pre­cisely 2.89 mil­lion add-ons. If users can add fea­tures with­out wait­ing for the offi­cial devel­op­ers and release cycles, they will. And odds are pretty good they may have already done exactly what you need­ed, or solved a need you did­n’t even real­ize you had.

It’s lovely to be able to make use of other peo­ple’s exten­sions in your shiny new toy, but do you know what to do with it now? Obvi­ous­ly:


  • You should be able to under­stand it.

This is a valu­able prop­erty for a num­ber of rea­sons also. An exam­ple I like to use is the XMonad tiled win­dow­ing man­ager for X. Before, I had used Rat­poi­son and Stump­WM, but I was never really sat­is­fied because I did­n’t under­stand C or Com­mon Lisp so many things were and remained a mys­tery to me. This made extend­ing them, debug­ging them, even just read­ing the doc­u­men­ta­tion to use them—it made them all more diffi­cult than they had to be.

With XMon­ad, I under­stand most of it. I see how the exten­sions work, and have writ­ten my own. The imper­fect mir­ror of the doc­u­men­ta­tion does not con­fuse, as I can eas­ily turn to the code which is the ulti­mate doc­u­men­ta­tion. This point ties into the first and sec­ond points. If you under­stand the code and the archi­tec­ture, then you are in a much bet­ter posi­tion to extend it, and poten­tially main­tain it.

This goes back to the first point; you should never need to main­tain it, but given your invest­ment in it, you may want to and it’s bet­ter to be able to main­tain it to the degree you need it to func­tion—­some­times main­te­nance is just a mat­ter of updat­ing a few files or the pack­ag­ing sys­tem.


  • Try to use stan­dard stuff.

For exam­ple, Bash is by all the for­mer cri­te­rion a good choice for one’s shell. It is main­tained by the GNU Pro­ject, it is cer­tainly exten­si­ble and pow­er­ful, and it’s rel­a­tively under­stand­able. But the really nice thing about Bash is how very stan­dard it is: every Linux dis­tri­b­u­tion includes Bash, and even set it as the default shell.

And if an oper­at­ing sys­tem does­n’t run Bash as default, it is basi­cally guar­an­teed that Bash will run on it (or there will at least be shells with Bash-com­pat­i­bil­i­ty). This is an advan­tage you should not under­es­ti­mate. Because of this ubiq­ui­ty, there are pow­er­ful net­work effects oper­at­ing in favor of Bash guar­an­tee­ing that it will be use­ful for you for a very long time. Stan­dard means you will be able to get your hands on it, stan­dard means there will be doc­u­men­ta­tion and peo­ple work­ing to solve bugs and other peo­ple tak­ing advan­tage of the exten­si­bil­i­ty, and so on. Stan­dard means that new imple­men­ta­tions and forks are that much eas­i­er. Stan­dard means many things, but from a user’s per­spec­tive, stan­dard is gen­er­ally a good thing for soft­ware and lan­guages to be.

The final major con­sid­er­a­tion was left to last. It should come as no sur­prise to mod­ern pro­gram­mer—s­in­fully awash amongst high­-level garbage-col­lected inter­preted lan­guages as they are—that the least impor­tant fac­tor to con­sider is the level of:


  • effi­ciency of a pro­gram.

This is the final con­cern because appli­ca­tions spend lit­tle time doing ineffi­cient things. The ratio­nale here is that, per­haps Emacs may be some­what slower than a less capa­ble or portable edi­tor—but you will not save so much time on quicker screen updates &etc. as to make up for real hours lost in port­ing the other edi­tor or work­ing around its lack of fea­tures (see ).

There are other con­cerns here: it is eas­ier to make an extensible/powerful/portable edi­tor more effi­cient in the bot­tle­necks than it is to hack in those fea­tures to an effi­cient edi­tor never intended for those fea­tures2. But if you are truly faced with two choic­es—each of whom seems equally solidly main­tained, exten­si­ble, stan­dard, etc.—then go with the more effi­cient one. After all, effi­ciency is a virtue and does make the expe­ri­ence more pleas­ant and gives you the option of using it on slower com­put­ers.

Alas, just mark­ing down these fea­tures does­n’t solve every prob­lem. For exam­ple, choos­ing between Vim and Emacs was easy for me—Vim’s sup­posed effi­ciency did­n’t (in my reck­on­ing) out­weigh the increased stan­dard­ness and com­pre­hen­si­bil­ity of Emacs. Or take Bash com­pared with Zsh. Bash is clearly more stan­dard and is clearly bet­ter main­tained, being even older than Zsh and the offi­cial shell of the GNU Pro­ject, and it is prob­a­bly mar­gin­ally more effi­cient. But I nev­er­the­less chose to switch to Zsh.

Why? I just liked it bet­ter. It has more fea­tures, yes (but not sig­nifi­cantly more); it has Bash com­pat­i­bil­ity (sure, but not as good as the real thing); it makes my con­fig­u­ra­tion files cleaner and eas­ier to main­tain, sure (but they were diffi­cult to con­vert, and are now no longer quite so portable). Basi­cally every point adum­brated argues strongly for Bash and at best, weakly for Zsh. But I liked it bet­ter, and in the end that mat­ters.

Short configurations

What other things should we keep in mind? Well, as time pass­es, I’ve found con­fig­u­ra­tion files tend to grow. A good thing to do when a con­fig­u­ra­tion file has grown par­tic­u­larly elab­o­rate is to sim­plify it.

Slim it down, remove every­thing you haven’t needed in the last sev­eral months. If you still have a huge file, see whether a large chunk of it makes sense to split out.

For exam­ple, for the Zsh I have: ~/.zshenv, for envi­ron­men­tal vari­ables and shell set­tings that all Zsh invo­ca­tions need; ~/.zaliases, hand-writ­ten aliases and func­tions; ~/.ztypos, con­tain­ing all aliases auto­mat­i­cally writ­ten to cor­rect typo­graph­i­cal errors I have made in the past; ~/.zlogout and ~/.zprofile, for logout/login stuff; and final­ly, ~/.zshrc, which con­tains every­thing needed for pleas­ant inter­ac­tive use. Clearly it’d be absurd and extremely awk­ward to stuff all of that into a sin­gle file, as well as poten­tially slow.

But sup­pose you have removed cruft, writ­ten stuff as well and con­cisely as you could, and split files out, and you still have too much? Now it comes time to give back to the free soft­ware com­mu­ni­ty. Con­sider care­fully your set­tings: per­haps they are very com­mon and you could file a bug sug­gest­ing they be made the default? That would then be a line you could remove. Or per­haps you have on your hands a de facto use­ful exten­sion to the pro­gram you could pack­age up and offer online some­where, or even offer to the devel­op­ers as some­thing to add to all the other exten­sions included by default.

Take one, leave one

There is one last piece of advice:

  • For every new tree plant­ed, let one be uproot­ed.

This is a good rule to fol­low with all pos­ses­sions. If you buy a new pair of shoes, give away or junk the old pair which you no longer need or can use. If you pur­chase a new book, sell or donate that old one you will never read again. If you add one pro­gram to your sys­tem, try to remove anoth­er—per­haps one you never used, or per­haps one made redun­dant by the new one. If you install ELinks, remove Lynx.

Redun­dancy wastes resources, it imposes men­tal bur­dens, it increases iner­tia and resis­tance to change. The more stuff you have that you do not need, the stu­pider your pos­ses­sions get. If you have a room burst­ing at the gills: you can­not freely trav­el. You can­not eas­ily change your place of res­i­dence. You are no longer sure what do you and do not have, you are increas­ingly men­tally stressed. You lit­er­ally can no longer freely move about. In extreme cas­es, peo­ple have been known to die because they refused to let go of things3.

Try to keep mat­ters ele­gant and unclut­tered. There is a differ­ence between ele­gance and com­pe­tence—and this differ­ence makes all the differ­ence in the world.

  1. For my pur­pos­es, the many vari­ants such as the XMonad model are imma­te­r­i­al. Yes, it is a very inter­est­ing model (the pro­gram com­piles a fresh binary with the user’s new con­fig hard­wired in, seri­al­izes out the cur­rent global state, and restarts itself with the new binary and passes it the seri­al­ized state)—but the details of how the exten­si­bil­ity is accom­plished are only impor­tant if the method is crappy and unre­li­able.↩︎

  2. As a big­oted Emacs user, I of course see Vim as the clas­sic exam­ple of attempt­ing to shoe­horn in a fea­ture like a first-class exten­sion lan­guage—with humor­ous results!↩︎

  3. Often due to men­tal dis­ease; see ↩︎