Skip to main content

Fuzz Testing

Catalogue of errors or problems caused by cats randomly doing things (ie. real-life fuzz testing).

Software is poorly tested, and not robust even after decades of development using techniques like “fuzz testing” (bruteforcing random inputs to trigger problems).

As an example, I log all instances of a different kind of fuzz testing—when my cat walks over my computer keyboard and causes problem! This has done everything from frozen my monitor to deleted files to segfaulted statistical software to both DoSed & crashed X.

Other cats have shut down datacenters, crashed Mac/Window/Ubuntu systems by pressing a button too long, broken screens & printers (somehow), and watched cat videos on YouTube.

If builders built houses the way programmers built programs, the first woodpecker to come along would destroy civilization.

Gerald Weinberg (attributed in Conrad Schneiker1975)

Things my cat has done by walking across my computer keyboard over the years:

My Cat

  • ordered extra socks from Muji

  • retweeted1 tweets, ‘favorited’ tweets, and deleted draft tweets on Twitter

  • clipped whole pages in Evernotes

  • deleted a dozen photos

  • turned off sound in MATE

  • chatted on IRC

  • killed Emacs

  • killed Firefox

    • disabled Firefox by laying against the side of the keyboard & holding down some no-op key so long that Firefox hung for the next hour at 100% CPU, apparently trying to process all the key-down events.

    • crashed Firefox by similarly laying & pressing the Escape key

  • crashed R inside Urxvt

    • crashed Stan inside R inside Urxvt

  • made Urxvt unusable (terminal escapes?)

  • deleted directories of docs (but fortunately, versioned)

  • hard-locked XMonad (by opening arbitrarily many windows)

  • permanently disabled weather forecast widget in MATE status bar, which remained broken even after being manually re-added (?)

  • crashed X.org (the Synaptic driver for my Acer laptop would segfault whenever he stepped on the trackpad; I never figured out why but chalk it up to perhaps he was touching it in too many spots and triggered bugs)

  • turned off monitor via keyboard

    replicating this, I disabled my monitor a second time but it didn’t come back up when I pushed the hotkey apparently responsible; I had to push the power button on the PC itself to bring it back up, and network was disabled afterwards! A check of dmesg system logs suggests that the system had fully suspended and apparently that can’t be undone from the keyboard?

  • made my external monitor a ‘mirror’ display

  • turned off external monitor, forcing reboot of MATE to restore access to both external & laptop monitors

  • shut down laptop

Other Cats

Things other peoples’ cats have done:

Stan segfaulting. Somehow.

Stan segfaulting. Somehow.

Fuzz Testing

This ‘fuzz testing’, aside from demonstrating that the first AI with the intelligence of a cat will take over civilization (crashes like Stan/R/X simply should not happen in the first place much less be triggered by legal input like random keypresses), also shows a lot of bad user-experience (UX) design in FLOSS programs.

Here, the problematic software includes Emacs, Urxvt, MATE shutdown interface, MATE external monitor handling, and the Twitter UI. Destructive irreversible operations like shutting down a computer should require prompts which cannot be faked by a cat sitting on a keyboard; in the case of MATE’s external monitor handling, it is reversible by the same key pressed—but only partially, as MATE forgets the external display settings like the orientation & relative positioning of the two monitors, requiring me to again manually set all the options.

Perhaps more tech companies & software developers should do fuzz testing (given that between cats and real fuzz testing, we are still finding so many serious issues in software)…

See Also

Similar Links

[Similar links by topic]