On Having Enough Socks

Personal experience and surveys on running out of socks; discussion of socks as small example of human procrastination and irrationality, caused by lack of explicit deliberative thought where no natural triggers or habits exist.
statistics, R, survey, Bayes, psychology, technology, decision-theory, design, insight-porn
2017-11-222019-06-12 finished certainty: possible importance: 4


After run­ning out of socks one day, I reflected on how ordi­nary tasks get neglect­ed. Anec­do­tally and in 3 online sur­veys, peo­ple report often not hav­ing enough socks, a prob­lem which cor­re­lates with rar­ity of sock pur­chases and demo­graphic vari­ables, con­sis­tent with a neglect/procrastination inter­pre­ta­tion: because there is no spe­cific time or trig­ger­ing fac­tor to replen­ish a shrink­ing sock stock­pile, it is easy to run out.

This reminds me of akra­sia on minor tasks, ‘yak shav­ing’, and the nature of dis­as­ter in com­plex sys­tems: lack of hard rules lets errors accu­mu­late, with­out any ‘global’ under­stand­ing of the drift into dis­as­ter (or at least ineffi­cien­cy). Humans on a smaller scale also ‘drift’ when they engage in Sys­tem I reac­tive think­ing & action for too long, result­ing in cog­ni­tive bias­es. An exam­ple of drift is the gen­er­al­ized human fail­ure to explore/experiment ade­quate­ly, result­ing in overly greedy exploita­tive behav­ior of the cur­rent local opti­mum. Gro­cery shop­ping pro­vides a case study: despite large gains, most peo­ple do not explore, per­haps because there is no estab­lished rou­tine or prac­tice involv­ing exper­i­men­ta­tion. Fixes for these things can be seen as ensur­ing that Sys­tem II delib­er­a­tive cog­ni­tion is peri­od­i­cally invoked to review things at a global lev­el, such as devel­op­ing a habit of max­i­mum explo­ration at first pur­chase of a food pro­duct, or annu­ally review­ing pos­ses­sions to note prob­lems like a lack of socks.

While socks may be small things, they may reflect big things.

Socks pos­sess the mys­te­ri­ous pow­er, like cats, of van­ish­ing; unlike cats, they don’t get hun­gry and come back. So I found myself one day in sum­mer 2013 doing laun­dry a week early and wast­ing time schlep­ping back & forth solely because I had run out of socks entirely and could­n’t bear walk­ing around in dirty socks. I sud­denly real­ized that this was a ridicu­lous prob­lem to have in an age awash with cheap tex­tiles (so cheap that clothes must be shipped to Africa or incin­er­ated lest the thrift stores burst at the seam­s), and imme­di­ately went on Ama­zon & bought a pack of 30 pairs to refill my ‘sock­pile’.1 This made me curi­ous: how many other peo­ple don’t have enough socks, and why not?

I began ask­ing peo­ple if they thought they had enough socks and quite a few peo­ple would say that they did­n’t, but they had­n’t quite got­ten around to it. (Although some insist changed their lives for­ev­er.)

So I began run­ning polls, and I am not alone.

Sock Surveys

An oth­er­wise-un­pub­lished Sam­sung sock sur­vey finds that “Brits lose an aver­age of 1.3 socks each month (and more than 15 in a year)”, imply­ing an annual loss of ~8 pairs in the best case sce­nar­io: where you either don’t need exact matches (be­cause all socks are the same kind) or don’t mind mis­match­es.2 If each pair is unique and one goes miss­ing from each, then in the worst case an annual loss of 15 indi­vid­ual socks implies one must buy another 15 pairs. This appears to be in addi­tion to wear-and-tear or changes in nec­es­sary type, which must be made up for.

In a Twit­ter sur­vey 2019-01-20–2019-01-27 of my fol­low­ers, I asked:

  1. Do you have enough pairs of socks?

    • Yes: 64% (n = 689)
    • No: 37% (n = 405)
  2. How many pairs of socks do you have?

    • 0–10: 18% (n = 118)
    • 11–20: 46% (n = 302)
    • 21–30: 27% (n = 177)
    • 31+: 9% (n = 59)
  3. How often do you buy replace­ment socks?

    • Month­ly: 2% (n = 15)
    • Semi­-an­nu­al­ly: 33% (n = 254)
    • Annu­al­ly: 37% (n = 285)
    • Less or nev­er: 28% (n = 216)
  4. Who buys your socks?

    • Me: 75% (n = 580)
    • Spouse/significant-other: 7% (n = 54)
    • Rel­a­tive: 16% (n = 123)
    • Oth­er: 2% (n = 15)

At least among my Twit­ter social cir­cle, not hav­ing enough socks is com­mon and a fair num­ber of peo­ple are on the verge of sock bank­rupt­cy. An answer to why sug­gests itself by the pur­chase details: most peo­ple are respon­si­ble for their own sock main­te­nance, but buy on per­haps not even an annual basis (a plu­ral­ity buy ‘annu­ally’, and the ‘semi­-an­nu­ally’ may be more than off­set by the ‘less or never’ respon­dents); so it’s easy to for­get and not buy socks.

Is sock­less­ness con­cen­trated among those who must buy their own socks & do so rarely? Twit­ter responses are inde­pen­dent and not linked by user­name (only the aggre­gate %s and total n are report­ed), so there’s no way to know from the respons­es, so there’s no way to see the inter­cor­re­la­tions.

To do that, I set up a sur­vey on 2019-01-20 (CSV), ask­ing all 4 ques­tions in a sin­gle sur­vey with n = 130 US responses cost­ing $100. (This is more expen­sive than my usual trick of ask­ing only 1 ques­tion, and costs $1/response rather than $0.10/response, but a set of 4 sin­gle-ques­tion sur­veys would be the same as the Twit­ter sur­vey.) Eric Jor­gensen also ran a ver­sion of the sur­vey on a per­son­al­ity quiz web­site with an inter­na­tional audi­ence (ODS/CSV), with n = 455. They have the same ques­tions (with the excep­tion of the sock count ques­tion, where his sur­vey asked for a numeric rather than ordi­nal respon­se, so I con­vert it back to ordi­nal), so I pool them for analy­sis:

socks <- read.csv("https://www.gwern.net/docs/psychology/2019-01-20-gs-socks.csv")
socks <- subset(socks, select=c("Question..1.Answer", "Question..2.Answer", "Question..3.Answer", "Question..4.Answer"))
socks <- socks[socks$Question..3.Answer!="",] # rm NAs
socks <- socks[socks$Question..4.Answer!="",] # rm NAs
socks$Question..1.Answer <- socks$Question..1.Answer=="Yes"
socks$Question..2.Answer <- as.ordered(socks$Question..2.Answer)
socks$Question..3.Answer <- ordered(socks$Question..3.Answer, levels=c("monthly", "semi-annually", "annually", "less/never"))
socks$Question..4.Answer <- ordered(socks$Question..4.Answer, levels=c("me", "spouse or significant other", "relative", "other"))
socksI <- with(socks, data.frame(Enough=as.integer(Question..1.Answer), Count=as.integer(Question..2.Answer), Frequency=as.integer(Question..3.Answer), Purchaser=as.integer(Question..4.Answer)))

eric <- read.csv("https://www.gwern.net/docs/psychology/2019-01-21-eric-socksurvey.csv")
eric$Count <- as.integer(ordered(sapply(eric$Count, function(c) { if (c<=10) { "0-10"; } else { if (c<=20) { "11-20"; } else { if (c<=30) { "21-30"; } else { "31+"; }}}})))
ericI <- subset(eric, select=c("Enough", "Count", "Frequency", "Purchaser"))

socksAllI <- rbind(socksI, ericI)

## Descriptive:
library(skimr)
skim(socksAllI)
# Skim summary statistics
#  n obs: 599
#  n variables: 4
#
# ── Variable type:integer
#   variable missing complete   n mean   sd p0 p25 p50 p75 p100     hist
#      Count       0      599 599 2.04 0.91  1   1   2   3    4 ▆▁▇▁▁▃▁▂
#     Enough       0      599 599 0.84 0.37  0   1   1   1    1 ▂▁▁▁▁▁▁▇
#  Frequency       0      599 599 2.76 0.87  1   2   3   3    4 ▁▁▇▁▁▇▁▅
#  Purchaser       0      599 599 1.63 0.96  1   1   1   3    4 ▇▁▁▁▁▂▁▁

#  n obs: 600
#  n variables: 4
#
# ── Variable type:integer
#   variable missing complete   n mean   sd p0 p25 p50 p75 p100     hist
#      Count       0      600 600 2.04 0.92  1   1   2   3    4 ▆▁▇▁▁▃▁▂
#     Enough       0      600 600 0.84 0.37  0   1   1   1    1 ▂▁▁▁▁▁▁▇
#  Frequency       1      599 600 2.76 0.87  1   2   3   3    4 ▁▁▇▁▁▇▁▅
#  Purchaser       0      600 600 1.63 0.96  1   1   1   3    4 ▇▁▁▁▁▂▁▁

## Bivariate correlations:
library(psych)
polychoric(socksAllI)
# Polychoric correlations
#           Enogh Count Frqnc Prchs
# Enough     1.00
# Count      0.29  1.00
# Frequency -0.23 -0.08  1.00
# Purchaser  0.16 -0.01  0.17  1.00
#
#  with tau of
#               1     2     3    4
# Enough    -0.99   Inf   Inf  Inf
# Count      -Inf -0.48  0.62 1.38
# Frequency  -Inf -1.60 -0.21 0.74
# Purchaser  -Inf  0.45  0.64 1.71

The GS respon­dents have less of an issue with sock short­ages than my Twit­ter respon­dents (un­sur­pris­ing­ly) with 15% rather than 37% sock­less, and the bivari­ate poly­choric cor­re­la­tions3 make sense to me: count and hav­ing enough cor­re­late strong­ly, of course, while fre­quency & pur­chaser dis­tance pre­dict less socks/more risk of not hav­ing enough.

What about joint rela­tion­ships? con­ve­niently sup­ports ordi­nal pre­dic­tors via “monot­o­nic effects” in addi­tion to sup­port­ing ordi­nal regres­sion for ordi­nal out­comes, so there’s no prob­lem mod­el­ing any of the vari­ables in any com­bi­na­tion; given the over­lap of sock count & hav­ing enough, it does­n’t make much sense to use one as a pre­dic­tor of the other (although extract­ing a fac­tor might make sense). So to do regres­sion from Frequency & Purchaser onto Enough & Count:

library(brms)
brm(Enough ~ mo(Frequency) + mo(Purchaser), family="bernoulli", data=socksAllI)
# ...Population-Level Effects:
#             Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept       2.26      0.28     1.79     2.91       1851 1.00
# moFrequency    -0.93      0.37    -1.71    -0.24       1948 1.00
# moPurchaser    -0.58      0.39    -1.38     0.17       3365 1.00
# ...
brm(Count ~ mo(Frequency) + mo(Purchaser), family="cumulative", data=socksAllI)
# ...Population-Level Effects:
#              Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept[1]    -1.03      0.17    -1.39    -0.72       2488 1.00
# Intercept[2]     0.77      0.17     0.41     1.06       2860 1.00
# Intercept[3]     2.17      0.20     1.75     2.56       3505 1.00
# moFrequency     -0.52      0.25    -1.02    -0.03       2558 1.00
# moPurchaser      0.02      0.29    -0.49     0.68       3142 1.00

While differ­ent para­me­ter­i­za­tions, the mes­sage remains the same: a fair num­ber of peo­ple do not have socks (it’s not only me), and this par­tic­u­larly cor­re­lates with not fre­quently pur­chas­ing socks.

Demographics

Inci­den­tal­ly, both the GS & Eric Jor­gensen polls include some demo­graph­ics data: esti­mated gender/age/location for GS, and ESL-speaker/country/gender for Eric Jor­gensen. Those aren’t my main inter­est here, but how do they look?

One could make some pre­dic­tions based on stereo­types: women will have more socks than men, older peo­ple will be more likely to have enough socks than younger peo­ple, and there will prob­a­bly be cross-coun­try differ­ences. Check­ing, older peo­ple are indeed more like­ly, cross-coun­try differ­ences are not so large as to be infer­able, and there appears to be incon­sis­tency in gen­der effects: men have more prob­lems with socks in the US than inter­na­tion­al­ly?

Jor­gensen’s data first; because of the large num­ber of coun­tries, heavy reg­u­lar­iza­tion must be used:

polychoric(subset(eric, select=c(Gender.Int, Enough, Frequency, Purchaser)))
# Polychoric correlations
#            Gnd.I Enogh Frqnc Prchs
# Gender.Int  1.00
# Enough      0.02  1.00
# Frequency  -0.01 -0.25  1.00
# Purchaser   0.20  0.24  0.15  1.00
brm(Enough ~ Gender + Country + mo(Frequency) + mo(Purchaser), family=bernoulli, prior=c(set_prior("horseshoe(1, par_ratio=0.05)")), control = list(max_treedepth = 15, adapt_delta=0.95), chains=30, iter=10000, data=eric)
# ...Population-Level Effects:
#             Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept       1.85      0.25     1.52     2.54      10770 1.00
# GenderMale      0.00      0.04    -0.07     0.07     131457 1.00
# CountryAU      -0.00      0.08    -0.10     0.08     113988 1.00
# CountryAZ       0.00      0.18    -0.10     0.12      95067 1.00
# CountryBE       0.01      0.18    -0.09     0.11      84197 1.00
# CountryBG       0.01      0.16    -0.08     0.13      62989 1.00
# CountryBO       0.00      0.16    -0.10     0.12      85775 1.00
# CountryBR       0.00      0.16    -0.09     0.11      80827 1.00
# CountryBS      -0.09      0.53    -1.28     0.05      29590 1.00
# CountryCA       0.00      0.08    -0.08     0.10     118251 1.00
# CountryCH      -0.02      0.26    -0.20     0.07      55249 1.00
# CountryCZ       0.01      0.18    -0.08     0.15      54813 1.00
# CountryDE       0.01      0.18    -0.07     0.16      72813 1.00
# CountryDK      -0.00      0.11    -0.11     0.09     111953 1.00
# CountryEE       0.00      0.15    -0.10     0.11     109200 1.00
# CountryES       0.01      0.17    -0.09     0.12      59172 1.00
# CountryFI       0.00      0.15    -0.10     0.11      90646 1.00
# CountryFR      -0.00      0.11    -0.11     0.09     115618 1.00
# CountryGB      -0.00      0.06    -0.09     0.08     101507 1.00
# CountryGR       0.01      0.18    -0.08     0.13      33231 1.00
# CountryHK       0.01      0.15    -0.09     0.12      73429 1.00
# CountryHR      -0.01      0.12    -0.13     0.08      98295 1.00
# CountryHU       0.01      0.17    -0.09     0.12      85208 1.00
# CountryID       0.00      0.10    -0.08     0.11      91982 1.00
# CountryIE      -0.01      0.15    -0.15     0.07      55181 1.00
# CountryIL      -0.04      0.28    -0.46     0.06      35464 1.00
# CountryIN      -0.02      0.13    -0.20     0.06      69378 1.00
# CountryIR      -0.02      0.25    -0.19     0.07      53285 1.00
# CountryIS      -0.03      0.29    -0.23     0.07      44140 1.00
# CountryIT       0.00      0.16    -0.10     0.11      71643 1.00
# CountryJE       0.00      0.16    -0.09     0.11      65529 1.00
# CountryJM       0.00      0.11    -0.09     0.11      91020 1.00
# CountryJP       0.00      0.10    -0.09     0.09     118871 1.00
# CountryKE       0.01      0.16    -0.09     0.12     104434 1.00
# CountryKR       0.01      0.16    -0.09     0.12      93687 1.00
# CountryLB       0.01      0.17    -0.08     0.14      51394 1.00
# CountryLT       0.01      0.15    -0.09     0.12      97039 1.00
# CountryMD       0.00      0.16    -0.09     0.11      79148 1.00
# CountryMK      -0.01      0.17    -0.16     0.07      14656 1.00
# CountryMM       0.00      0.15    -0.09     0.11     107831 1.00
# CountryMX       0.01      0.16    -0.08     0.12      84871 1.00
# CountryMY       0.01      0.19    -0.08     0.15      57119 1.00
# CountryNL       0.04      0.31    -0.05     0.48      44071 1.00
# CountryNO       0.00      0.10    -0.08     0.11     101486 1.00
# CountryNONE     0.03      0.28    -0.06     0.33      36810 1.00
# CountryPH      -0.01      0.13    -0.20     0.06      63309 1.00
# CountryPL       0.00      0.10    -0.10     0.10      97421 1.00
# CountryPT       0.01      0.17    -0.09     0.12      69037 1.00
# CountryQA       0.01      0.16    -0.09     0.11      66602 1.00
# CountryRO       0.01      0.16    -0.09     0.11      82836 1.00
# CountryRU       0.00      0.15    -0.09     0.11      99649 1.00
# CountrySA       0.01      0.17    -0.09     0.11      70890 1.00
# CountrySE      -0.00      0.08    -0.10     0.08     105770 1.00
# CountrySG       0.02      0.21    -0.06     0.20      48360 1.00
# CountryTR      -0.01      0.17    -0.16     0.07      56410 1.00
# CountryTT       0.01      0.17    -0.08     0.14      68940 1.00
# CountryUA       0.01      0.16    -0.08     0.13      60610 1.00
# CountryUS      -0.01      0.06    -0.14     0.05      73991 1.00
# CountryVE       0.01      0.17    -0.09     0.13      35998 1.00
# CountryVN       0.00      0.17    -0.09     0.11      54781 1.00
# CountryXK       0.00      0.16    -0.10     0.12     101146 1.00
# CountryZA       0.01      0.16    -0.08     0.13      59905 1.00
# moFrequency    -0.17      0.40    -1.40     0.02       7944 1.00
# moPurchaser    -0.01      0.08    -0.17     0.05      52232 1.00
# ...
brm(Count ~ Gender + Country + mo(Frequency) + mo(Purchaser), family=cumulative, prior=c(set_prior("horseshoe(1, par_ratio=0.05)")), control = list(max_treedepth = 15, adapt_delta=0.95), chains=30, iter=10000, data=eric)
# ...Population-Level Effects:
#              Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept[1]    -0.91      0.31    -1.57    -0.33     103560 1.00
# Intercept[2]     1.07      0.32     0.42     1.68     105615 1.00
# Intercept[3]     2.68      0.35     1.98     3.37     113296 1.00
# GenderMale      -0.13      0.17    -0.50     0.16     125937 1.00
# CountryAU       -1.03      0.69    -2.48     0.08     124545 1.00
# CountryAZ       -0.53      1.12    -3.33     1.18     158180 1.00
# CountryBE        0.11      0.81    -1.54     1.95     215427 1.00
# CountryBG       -0.07      0.61    -1.45     1.20     224586 1.00
# CountryBO        0.12      0.80    -1.51     1.97     208130 1.00
# CountryBR       -0.55      1.13    -3.36     1.16     158105 1.00
# CountryBS        0.27      0.91    -1.41     2.47     194178 1.00
# CountryCA        0.98      0.57    -0.04     2.09      82595 1.00
# CountryCH       -0.51      1.10    -3.26     1.21     161968 1.00
# CountryCZ        0.37      0.74    -0.90     2.12     154331 1.00
# CountryDE        0.29      0.59    -0.73     1.68     156133 1.00
# CountryDK        0.62      0.73    -0.47     2.27     113453 1.00
# CountryEE       -0.60      1.15    -3.44     1.12     155992 1.00
# CountryES       -0.12      0.75    -1.84     1.38     219013 1.00
# CountryFI        1.27      1.56    -0.79     4.96     123980 1.00
# CountryFR       -0.17      0.55    -1.46     0.88     202919 1.00
# CountryGB        0.17      0.28    -0.33     0.81      90166 1.00
# CountryGR        0.02      0.62    -1.32     1.37     214245 1.00
# CountryHK       -0.96      1.24    -4.01     0.65     144472 1.00
# CountryHR        0.01      0.62    -1.34     1.37     222467 1.00
# CountryHU        0.08      0.80    -1.61     1.88     212960 1.00
# CountryID       -1.81      0.92    -3.80    -0.14     138380 1.00 # Indonesia
# CountryIE       -0.92      1.23    -3.93     0.67     145126 1.00
# CountryIL        0.22      0.62    -0.93     1.67     181101 1.00
# CountryIN       -2.25      1.38    -5.43    -0.09     138907 1.00 # India
# CountryIR        0.22      0.84    -1.40     2.21     194840 1.00
# CountryIS        0.05      0.80    -1.64     1.84     210993 1.00
# CountryIT        0.17      0.82    -1.47     2.07     206471 1.00
# CountryJE        1.27      1.56    -0.79     4.96     123390 1.00
# CountryJM        0.76      0.64    -0.23     2.10      86077 1.00
# CountryJP       -0.42      0.60    -1.82     0.52     156615 1.00
# CountryKE       -0.56      0.82    -2.53     0.69     164055 1.00
# CountryKR        0.07      0.76    -1.54     1.76     219641 1.00
# CountryLB       -0.32      0.65    -1.87     0.79     178249 1.00
# CountryLT        0.44      0.79    -0.86     2.31     157516 1.00
# CountryMD        0.77      1.12    -0.94     3.41     137492 1.00
# CountryMK       -0.27      0.77    -2.13     1.12     199321 1.00
# CountryMM       -0.46      1.08    -3.19     1.24     164795 1.00
# CountryMX        0.36      0.68    -0.78     1.96     158241 1.00
# CountryMY       -1.91      1.39    -5.10     0.06     133497 1.00
# CountryNL        0.49      0.46    -0.22     1.46      90797 1.00
# CountryNO        1.22      0.69    -0.02     2.56      91436 1.00
# CountryNONE     -0.66      0.60    -1.95     0.25     127296 1.00
# CountryPH       -0.25      0.51    -1.44     0.63     177716 1.00
# CountryPL        0.09      0.45    -0.82     1.11     171907 1.00
# CountryPT        0.92      0.98    -0.52     3.07     118487 1.00
# CountryQA       -0.46      1.09    -3.17     1.27     166932 1.00
# CountryRO        0.02      0.78    -1.70     1.71     216995 1.00
# CountryRU       -0.60      1.15    -3.45     1.12     154146 1.00
# CountrySA       -1.01      1.26    -4.07     0.61     142090 1.00
# CountrySE        0.59      0.53    -0.23     1.72      85612 1.00
# CountrySG       -0.67      0.71    -2.29     0.37     143570 1.00
# CountryTR        0.14      0.67    -1.21     1.69     214560 1.00
# CountryTT        0.65      0.87    -0.69     2.63     121131 1.00
# CountryUA       -0.59      0.84    -2.59     0.67     150001 1.00
# CountryUS        0.79      0.28     0.26     1.35      72791 1.00
# CountryVE        1.35      1.14    -0.35     3.72     101866 1.00
# CountryVN       -0.65      1.18    -3.56     1.09     153337 1.00
# CountryXK       -0.59      1.15    -3.44     1.12     150475 1.00
# CountryZA        0.01      0.62    -1.35     1.35     212747 1.00
# moFrequency     -0.89      0.36    -1.61    -0.19      96417 1.00
# moPurchaser     -0.06      0.26    -0.64     0.45     179817 1.00

Noth­ing of note emerges here, except per­haps a ten­dency for males to have fewer socks (al­beit they appear to be con­tent with few­er); there might be coun­try-level effects as even the horse­shoe reg­u­lar­iza­tion does­n’t pull them tightly to zero, but there is far too lit­tle data to be con­fi­dent in what the effects might be.

In the GS US sur­vey data, there is only one coun­try, of course, but in exchange an inferred age bracket is avail­able:

socks <- read.csv("https://www.gwern.net/docs/psychology/2019-01-20-gs-socks.csv")
socks <- subset(socks, select=c("Question..1.Answer", "Question..2.Answer", "Question..3.Answer", "Question..4.Answer", "Gender", "Age"))
socks <- socks[socks$Question..3.Answer!="",] # rm NAs
socks <- socks[socks$Question..4.Answer!="",] # rm NAs

socks$Question..1.Answer <- socks$Question..1.Answer=="Yes"
socks$Question..2.Answer <- as.ordered(socks$Question..2.Answer)
socks$Question..3.Answer <- ordered(socks$Question..3.Answer, levels=c("monthly", "semi-annually", "annually", "less/never"))
socks$Question..4.Answer <- ordered(socks$Question..4.Answer, levels=c("me", "spouse or significant other", "relative", "other"))
socks <- socks[socks$Age!="Unknown" & socks$Gender!="Unknown",]

socksI <- with(socks, data.frame(Enough=as.integer(Question..1.Answer), Count=as.integer(Question..2.Answer), Frequency=as.integer(Question..3.Answer), Purchaser=as.integer(Question..4.Answer), Age=as.integer(Age), Gender=as.integer(Gender=="Male")))
skim(socksI)
# Skim summary statistics
#  n obs: 114
#  n variables: 6
#
# ── Variable type:integer
#   variable missing complete   n mean   sd p0 p25 p50 p75 p100     hist
#        Age       0      114 114 3.92 1.57  1   3   4   5    6 ▃▂▁▇▅▁▇▆
#      Count       0      114 114 2.34 0.94  1   2   2   3    4 ▃▁▇▁▁▅▁▂
#     Enough       0      114 114 0.79 0.41  0   1   1   1    1 ▂▁▁▁▁▁▁▇
#  Frequency       0      114 114 2.8  0.84  1   2   3   3    4 ▁▁▅▁▁▇▁▃
#     Gender       0      114 114 0.72 0.45  0   0   1   1    1 ▃▁▁▁▁▁▁▇
#  Purchaser       0      114 114 1.39 0.88  1   1   1   1    4 ▇▁▁▁▁▁▁▁
polychoric(socksI)
# Polychoric correlations
#           Enogh Count Frqnc Prchs Age   Gendr
# Enough     1.00
# Count      0.19  1.00
# Frequency -0.21  0.12  1.00
# Purchaser -0.23 -0.24 -0.25  1.00
# Age        0.17  0.08  0.09 -0.20  1.00
# Gender    -0.49 -0.14  0.11  0.55  0.26  1.00
brm(Enough ~ Gender + mo(Age) + mo(Frequency) + mo(Purchaser), family="bernoulli", data=socksI)
# ...Population-Level Effects:
#             Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept       2.42      1.13     0.28     4.70       2064 1.00
# Gender         -1.81      0.91    -3.82    -0.27       2802 1.00
# moAge           1.08      0.84    -0.52     2.81       2342 1.00
# moFrequency    -0.03      1.00    -1.87     2.10       2111 1.00
# moPurchaser    -0.99      0.75    -2.50     0.45       3191 1.00
brm(Count ~ Gender + mo(Age) + mo(Frequency) + mo(Purchaser), family="cumulative", data=socksI)
# ...Population-Level Effects:
#              Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
# Intercept[1]    -0.65      0.88    -2.21     1.20       2351 1.00
# Intercept[2]     1.45      0.90    -0.14     3.39       2295 1.00
# Intercept[3]     2.94      0.93     1.31     4.91       2392 1.00
# Gender           0.12      0.41    -0.72     0.91       5283 1.00
# moAge           -0.49      0.57    -1.60     0.65       3793 1.00
# moFrequency      1.44      0.92    -0.19     3.34       2553 1.00
# moPurchaser      1.35      0.74     0.02     2.86       4586 1.00

There are pos­si­ble age effects in the expected direc­tion; older peo­ple appear to be bet­ter at man­ag­ing sock lev­els.

Curi­ous­ly, there may be differ­ent gen­der effects in the two sur­vey datasets: in the Jor­gensen inter­na­tional sur­vey, gen­der is largely inert (ex­cept for a cor­re­la­tion with Purchaser) while in the US GS sur­vey, gen­der cor­re­lates with every­thing and men appear much less likely to have enough socks (but to have more sock­s). Pok­ing at the data, there appears to be another con­nec­tion: in the US, men are more likely to do their own sock pur­chas­ing. I won­der if this reflects a differ­ent in sex roles, with women doing more cloth­ing shop­ping in non-US coun­tries and tak­ing care of sock needs along the way?

Christmas advice

“What do you see when you look in the Mir­ror [of Erised]?”
“I? I see myself hold­ing a pair of thick, woollen socks.”
Harry stared.
“One can never have enough socks”, said Dum­b­le­dore. “Another Christ­mas has come and gone and I did­n’t get a sin­gle pair. Peo­ple will insist on giv­ing me books.”

J.K. Rowl­ing, Harry Pot­ter and the Philoso­pher’s Stone

Given that con­sis­tently >15% of respon­dents don’t have enough socks, and in the US, younger males are espe­cially likely to not have enough socks, here’s some Christ­mas advice: if you don’t know what to buy them, why not buy them some really good socks?

Socks make a great gift. Every­one will need replace­ment socks, sooner or lat­er, and it seems lots of peo­ple don’t get them. Unlike the feared ‘ugly sweater from Grandma’ pre­sent, they aren’t on pub­lic dis­play so if they’re ugly, it’s not too big a deal. Nor do they take up too much space, and can be used for more of the year. An annual gift of socks is about the opti­mal tem­po, given the sur­veys about how often peo­ple lose socks or buy socks, and Christ­mas is an excel­lent Schelling point, since it’s already asso­ci­ated with socks. Final­ly, socks may be a cun­ning gift as they can be eas­ily eval­u­ated as supe­rior, and so seem pre­mium despite not cost­ing all that much in absolute terms.4

Who Moved My Sock?

How had I run out of socks? Well, like the joke about going bank­rupt, I did it one day at a time: with a sock qui­etly dis­ap­pear­ing one day, and a sock being tossed out due to holes & thin­ning out another day… At no point did I ever delib­er­ately try to econ­o­mize on socks or go with­out socks or explic­itly think that it was­n’t worth the bother of pick­ing up some socks next time I was in a cloth­ing store or doing an Ama­zon order—it just hap­pened on its own.

The Importance Of The Unimportant

In the case of socks, there is never a ‘Sock­nik moment’. There is only a slippery-slope/sorites—there’s no hard and fast line between enough and too-few socks, socks slowly wear out or lose mates, and if you have 20 and now have 19, well, that’s not a big deal, and then when you are down to 18, that’s not a big deal either why go shop­ping, and soon you’ll be down to 17… And if you don’t buy socks reg­u­larly as part of a clothes shop­ping trip, when will you? Even­tu­ally you’re wear­ing uncom­fort­able socks or being cold or being forced to do laun­dry runs ear­ly, with­out there ever being a clear ‘I need to buy some socks!’ trig­ger point. Even a habit like buy­ing replace­ment socks once a year as part of spring clean­ing would be enough, but one still needs to instill a habit.

Some might object that this is over­think­ing socks, and one should never think about socks at all. This is short­-sight­ed. If we were all per­fectly ratio­nal and omni­scient and pos­sessed of infi­nite com­put­ing pow­er, all our prob­lems would already be solved and we would buy socks at the exact opti­mal moment as part of the grand plan; but we are not. Deal­ing with our bounded ratio­nal­ity is the cen­tral con­cern of all dis­cus­sions of ratio­nal­ity & opti­miz­ing & bias­es.

It may not seem impor­tant to think about socks at any par­tic­u­lar moment, and socks are prob­a­bly not the most press­ing thing at this instant for me either, com­pared to tasks like ‘write an essay’ or ‘exer­cise’ or ‘answer emails’. But if it is bet­ter to wear socks than not, and one does not wish to go bare­foot for the rest of one’s life, then it must be opti­mal at some moment to think about socks. Per­haps a few months from now when one’s ‘sock­pile’ has worn down, dur­ing down­time, but there must be one.

Sim­i­lar­ly, one could scoff at all of the neces­si­ties of life like get­ting gro­ceries, or fil­ing a tax return, or get­ting life insur­ance: surely at that instant there is always some­thing more impor­tant one could be work­ing on doing, like get­ting a col­lege degree or found­ing a star­tup? But this argu­ment must have some flaw or by induc­tion you would never do them and so you would starve to death while being audited by the IRS and your heirs are ren­dered home­less. For exam­ple, the value of these tasks increases over time: you don’t really need to do your taxes early before the dead­line, but you do want to get it done by the dead­line. With gro­ceries, as long as you have enough to eat, it’s not much of a prob­lem to be low on food—per­haps it reduces your vari­ety a bit, but it’s not like you’ll starve, except if you run out of food in which case you will. And fail­ure to get life insur­ance incurs a small loss each and every day (be­cause of the risk of you dying that day and fail­ing to pro­vide for what­ever you wanted life insur­ance for).

Fur­ther, one’s life is a com­plex sys­tem: one’s house, one’s career, one’s com­put­er, all of these are com­plex sys­tems with inter­act­ing, cas­cad­ing fail­ures. All com­plex sys­tems (, Cook 2000) oper­ate in a , where minor errors must be reg­u­larly repaired in order to pre­vent a large-s­cale fail­ure cas­cad­ing through the whole sys­tem. When a steel fur­nace explodes, killing peo­ple, it does­n’t hap­pen out of the blue, but reflects a long series of choices & grad­u­ally esca­lat­ing issues & near-miss­es, and is a . When I lost weeks of time and money to a lap­top & backup fail­ure, it was­n’t because only one thing went wrong: it required at least 3 unusual fail­ures simul­ta­ne­ously in my lap­top & backup sys­tems, any of which not hap­pen­ing would have pre­vented the full acci­dent. Each slip may seem rel­a­tively minor and extra­or­di­nar­ily unlikely to have any seri­ous con­se­quences, but, like the “indiffer­ence of the indi­ca­tor”, they add up over a life­time and even­tu­ally a tail risk mate­ri­al­izes. Chance dis­fa­vors the unpre­pared mind—­time and chance hap­peneth to all, and indeed do many things come to pass.

Because fail­ures inter­act and mul­ti­ply, they resem­ble a : each indi­vid­ual fac­tor can block the acci­dent so the final dam­age of the out­come is the mul­ti­pli­ca­tion of the indi­vid­ual fac­tors. The log-nor­mal implies that a small sys­tem­atic increase or decrease in each fac­tor, anal­o­gous to being more care­ful & proac­tive in gen­eral about main­te­nance and risk, can cause a large differ­ence in final out­come (). One must expect the unex­pect­ed, and a fail­ure to ‘sweat the small stuff’ means you are allow­ing brush to pile up in the forest: one match could set it ablaze. Peo­ple who do not sweat the small stuff have a remark­able ten­dency to have ‘bad luck’ and some­how keep get­ting into trou­ble, much like the less intel­li­gent suffer more ‘acci­dents’ or nat­ural dis­as­ters have death tolls almost entirely deter­mined by pover­ty—cer­tain­ly, time & chance may hap­peneth to us all, but our prepa­ra­tions & reac­tions play an even greater role in deter­min­ing how far things go. A lack of the bour­geoisie virtues is a lack of fore­sight, prepa­ra­tions, and reserves/insurance/slack. Con­sider how care­less some peo­ple are in mat­ters of every­day life.5 It’s not hard to see how such care­less­ness in, say, get­ting drunk and mak­ing rental pay­ments can quickly esca­late.

‘Yak Shaving’ as a Failure Cascade

Seth Godin explains yak shav­ing as a sto­ry:

“I want to wax the car today.”

“Oops, the hose is still bro­ken from the win­ter. I’ll need to buy a new one at Home Depot.”

“But Home Depot is on the other side of the Tap­pan Zee bridge and get­ting there with­out my EZ Pass is mis­er­able because of the tolls.”

“But, wait! I could bor­row my neigh­bor’s EZ Pass…”

“Bob won’t lend me his EZ Pass until I return the mooshi pil­low my son bor­rowed, though.”

“And we haven’t returned it because some of the stuffing fell out and we need to get some yak hair to restuff it.”

And the next thing you know, you’re at the zoo, shav­ing a yak, all so you can wax your car.

God­in’s take-away is that yak shav­ing is mis­guided per­fec­tion­ism: once one real­izes one is yak shav­ing, one should decide “Don’t go to Home Depot for the hose. The minute you start walk­ing down a path toward a yak shav­ing par­ty, it’s worth mak­ing a com­pro­mise. Doing it well now is much bet­ter than doing it per­fectly lat­er.”

I inter­pret yak shav­ing entirely differ­ent­ly. At least when I feel I am trapped in yak shav­ing, it more often reflects a fail­ure cas­cade in the com­plex sys­tem I am cur­rently part of: either men­tally I have got­ten trapped into a local min­ima and have failed to reflect peri­od­i­cally on what the best way is, or the sys­tem really is bro­ken and once the yak is shaved, requires to find out how to fix the fun­da­men­tal prob­lems and how to pre­vent them from recur­ring.

I see ‘yak-shav­ing’ as a descrip­tion of a sit­u­a­tion where you are nested so deep in sub­goals that you’ve for­got­ten your orig­i­nal goal, at which point a good heuris­tic is to wake up and say “this is a lot of yak-shav­ing!” and think about what is going on that has led to an unde­sir­able sit­u­a­tion.

Think­ing about my own appli­ca­tions of the term, I think there are 3 differ­ent kinds of prob­lems which can lead to yak-shav­ing: avoid­ance, lack of mind­ful­ness, and cas­cad­ing problems/system fail­ures.

  1. you are pro­cras­ti­nat­ing or being akratic or falling into per­fec­tion­ism (closely related to pro­cras­ti­na­tion), by delib­er­ately over­com­pli­cat­ing some­thing or try­ing to use fancy or shiny new tech­niques, which of course fre­quently lead to new sub­goals because you aren’t famil­iar with them yet.

    This is fine some­times (you have to learn those new tech­niques some­when) or if it’s a kind of ‘struc­tured pro­cras­ti­na­tion’ (where the yak-shav­ing is itself valu­able eg because it makes a neat blog post or use­ful soft­ware pack­age), but often isn’t. The usual akrasia/procrastination equa­tion stuff, except it’s being hid­den under a gloss of super­fi­cial pro­duc­tiv­ity. (“I can’t write my nov­el, I have to clean my desk which requires […solv­ing 15 deeper nested issues…] which will take up all the rest of the day; I sure am a hard-work­ing writer.”)

    By call­ing it yak-shav­ing, you admit you are faffing around and you then solve your prob­lem the way you knew you should all along; or you can deal with why you are avoid­ing fin­ish­ing, or whether you really want to do it at all. If you refuse to acknowl­edge the yak-shav­ing, then even if you ‘shave the yaks’ you’ll just find another way to over­com­pli­cate things or a differ­ent thing to waste time on or switch to pro­cras­ti­nat­ing on social media etc.

  2. you have been fol­low­ing a greedy strat­egy of tak­ing the quick­est option at each deci­sion node; that you have now stacked up so many tasks to com­plete sug­gests that the greedy strat­egy has failed and you have fallen into a local pes­si­ma.

    Like with , it’s time to stop being so mind­less, step back, think about it more glob­al­ly, and ask if there’s some bet­ter approach. Was there some entirely differ­ent strat­egy which seemed too expen­sive com­pared to your cur­rent path (which has actu­ally turned out to be far more costly than pre­dict­ed) and now looks cheap? Or are there any inter­me­di­ate mid­dle steps which are expen­sive but cut out a large num­ber of other steps? Or per­haps all the paths are so costly that the top-level goal now no longer looks worth both­er­ing with and you should drop all the exist­ing tasks & stop shav­ing the yak entire­ly.

    Pro­gram­mers are par­tic­u­larly sus­cep­ti­ble to this because the line between use­ful automa­tion and immensely com­pli­cated time-wast­ing tin­ker­ing is a fine one indeed. This can be com­mon in pro­gram­ming where you can say, build up a Rube Gold­berg col­lec­tion of shell scripts and Emacs func­tions and man­ual edits to text because you wanted to avoid writ­ing a SQL func­tion (be­cause it would take 20 min­utes of con­sult­ing the SQL doc­u­men­ta­tion to get it right); but by the time you’re con­sult­ing the Bash FAQ or reset­ting IFS vari­ables to deal with a prob­lem half an hour lat­er, it’s good to wake up and ask ‘am I yak-shav­ing?’—and then you might real­ize that the data or prob­lem has turned out to be suffi­ciently painful (eg lots of spe­cial char­ac­ters or odd­ity in data for­mat­ting) that you can’t catch all the spe­cial cases and you would’ve been bet­ter off writ­ing the SQL query in the first place. In God­in’s exam­ple, per­haps one should sim­ply return the yak pil­low and hope the neigh­bor won’t notice the miss­ing stuffing, or they will pre­fer to sim­ply have it back rather than wait for you to fix it when­ev­er, or sim­ply upset them a lit­tle; or order the hose on Ama­zon even if it costs $5 more, to get it done; or, pay the damn toll like any­one else; or final­ly, is wax­ing the car worth­while at all (who notices)?

    Here ‘yak-shav­ing’ serves as a use­ful men­tal trig­ger which can break you out of the myopic prob­lem-solv­ing loop. This sort of yak-shav­ing is usu­ally quite bad, and if you don’t break out of it soon enough, can lead to con­sid­er­able exhaus­tion and waste of time, and lock you into bad long-term deci­sions. So it’s good to peri­od­i­cally ask, if you aren’t mak­ing progress on a prob­lem of intrin­sic inter­est to you, “so all this work, what’s it for any­way? If I were start­ing over from scratch—­know­ing what I do now—is this really how I would approach this prob­lem?”

  3. what you are doing is the best way to solve the prob­lem over­all, it’s just that things have been going wrong and you’ve been run­ning into con­tin­ual prob­lems, so you find your­self nested many lay­ers deep deal­ing with the cas­cade of prob­lems and doc­u­men­ta­tion

    …all your (en­crypt­ed) back­ups are bro­ken because you can’t get the most recent decryp­tion key because your drive is cor­rupted because you were run­ning the GPU 24/7 (to name a recent exam­ple of mine) so you’re in a LiveCD try­ing to mount the drive try­ing pass­words try­ing…

    In this case, in addi­tion to sim­ply shav­ing the yak, you need to do root-cause analy­sis—you are expe­ri­enc­ing what might be called —and in addi­tion to fig­ur­ing out how to solve each prox­i­mate prob­lem on the way, fig­ure out why they hap­pened & how to pre­vent them in the future. In pro­gram­ming, this fre­quently entails fil­ing bug reports & doc­u­ment patch­es, for­mal­iz­ing your recov­ery meth­ods as scripts or pro­grams, adding tests or redun­dancy or upgrad­ing hard­ware, and writ­ing post-mortems.

    So God­in’s inter­pre­ta­tion of a stack of nested related prob­lems here is sim­ply a form of this. But here, sim­ply yak shav­ing may solve the fur prob­lem & allow pop­ping, but it’s not enough. It’s not enough to sim­ply close those open loops, or have a sys­tem for record­ing open loops. Root-cause analy­sis is need­ed.

    Why did the yak fur fall out of the pil­low in the first place and how can it be pre­vented ever again? Why did­n’t he have his EZ Pass in the first place? Why was­n’t the hose put on the weekly shop­ping list (there is a shop­ping list right?) and replaced long before? And so on.

    With­out attack­ing prob­lems at the root, you might as well buy a sea­sonal pass to the zoo, because you are merely apply­ing bandaids to a com­plex sys­tem fail­ing, and if you don’t do any root-cause fix­es, even­tu­ally your prob­lems will seri­ously stack up and you’ll find your­self hit by a so-called ‘per­fect storm’ (ac­tu­ally per­fectly fore­see­able & inevitable) and then you’ll really be sor­ry.

So, ‘yak-shav­ing’ is a use­ful heuris­tic for keep­ing plan­ning stacks nested not too deeply by peri­od­i­cally ask­ing whether one is falling prey to one of those 3 fail­ure mod­es, and need to break out of the yak-shav­ing by an appro­pri­ate coun­ter­mea­sure of either: inter­ro­gat­ing the rea­sons for the akrasia; find­ing a bet­ter approach; or pri­or­i­tiz­ing fix­ing the root-causes of need­ing to yak-shave (rather than focus­ing on the yak-shav­ing).

The Ur Cognitive Bias

“I started eat­ing with them [the chemists] for a while. And I started ask­ing, ‘What are the impor­tant prob­lems of your field?’ And after a week or so, ‘What impor­tant prob­lems are you work­ing on?’ And after some more time I came in one day and said, ‘If what you are doing is not impor­tant, and if you don’t think it is going to lead to some­thing impor­tant, why are you at Bell Labs work­ing on it?’ I was­n’t wel­comed after that; I had to find some­body else to eat with­!…In the fall, Dave McCall stopped me in the hall and said, ‘Ham­ming, that remark of yours got under­neath my skin. I thought about it all sum­mer, i.e. what were the impor­tant prob­lems in my field. I haven’t changed my research’, he says, ‘but I think it was well worth­while.’ And I said, ‘Thank you Dave’, and went on. I noticed a cou­ple of months later he was made the head of the depart­ment. I noticed the other day he was a Mem­ber of the National Acad­emy of Engi­neer­ing. I noticed he has suc­ceed­ed. I have never heard the names of any of the other fel­lows at that table men­tioned in sci­ence and sci­en­tific cir­cles.”

,

“A mule who has car­ried a pack for ten cam­paigns under will be no bet­ter a tac­ti­cian for it, and it must be con­fessed, to the dis­grace of human­i­ty, that many men grow old in an oth­er­wise respectable pro­fes­sion with­out mak­ing any greater progress than this mule.”

, “Thoughts on Tac­tics”6

One prob­lem here is that the unim­por­tant becomes impor­tant, slowly and sub­tly. There is no IRS clock tick­ing on one’s wall, any more than there is a real­time dis­play of one’s sock­pile with defined red dan­ger zones upon which one orders new socks.

For many things, there is never any hard dead­line or sched­uled event or reminder which would bring a need to mind. So nec­es­sary things suffer from what a com­puter sci­en­tist might call : when a back­ground task, like run­ning a back­up, which has a low pri­or­ity (eg a backup can wait a few min­utes with­out much risk), is con­tin­u­ously pushed out by higher pri­or­ity tasks and never gets to run; while it may not have been urgent that it run imme­di­ate­ly, it is urgent that it run even­tu­al­ly. (Any­one who dis­agrees about back­ups not being impor­tant is free to imple­ment that advice and see how it works for them in the long run.)

Star­va­tion reflects bad plan­ning: the pri­or­i­ties of starv­ing tasks are not increased over time to reflect their pri­or­i­ty, or starv­ing tasks may not be con­sid­ered at all by a myopic plan­ner. And for humans, ‘out of sight is out of mind’, so myopia is easy.

Many human cog­ni­tive biases can be con­sid­ered as reflec­tions of a sin­gle ur-cog­ni­tive bias (Stanovich 2010, Deci­sion Mak­ing and Ratio­nal­ity in the Mod­ern World), a fail­ure to acti­vate diffi­cult, delib­er­ate, explicit Sys­tem II think­ing when appro­pri­ate, ‘wak­ing up’ from the usual fast fru­gal Sys­tem I think­ing, per­haps from time to time just to re-e­val­u­ate things. “Humans are not auto­mat­i­cally strate­gic.” Instead, Sys­tem I is always invoked, regard­less of Sys­tem II is need­ed, and the fast fru­gal cheap reflex­ive think­ing of Sys­tem I takes over. When Sys­tem I runs unim­ped­ed, work tends to degen­er­ate into what Google SRE terms “toil”; Beyer et al 2016:

…toil is the kind of work tied to run­ning a pro­duc­tion ser­vice that tends to be:

  • Man­ual
  • Repet­i­tive
  • Automat­able and not requir­ing human judg­ment
  • Inter­rup­t-driven and reac­tive
  • Of no endur­ing value

One works hard, but that a few bucks will get you a cup of coffee. Elim­i­nat­ing toil requires step­ping back to take an out­side view and pos­si­bly re-engi­neer things.7

Of course Sys­tem II can’t run all the time, any more than we can pon­der every day whether today we should re-engi­neer our sock­-buy­ing sys­tem or buy more socks. We hardly ever do—but that’s not quite the same as nev­er. It needs to run occa­sion­ally to check the fun­da­men­tals, to look for tasks starv­ing in the back­ground for lack of salien­cy, and to reflect on what is being done that ought not to be done at all, and con­sider entirely new alter­na­tives.

I think appar­ent instances of ‘sunk cost’ are bet­ter described as thought­less­ness. To give an exam­ple: when chess or play­ers con­tinue throw­ing pieces into a doomed posi­tion, is that because they explic­itly real­ize it is doomed but feel they must per­se­vere any­way, or is it due to the fact that chess ama­teurs com­mit more 8 than mas­ters () and don’t real­ize that the posi­tions are in fact irre­triev­able? When one engages in spring-clean­ing, one may wind up throw­ing or giv­ing away a great many things which one has owned for months or years but had not dis­posed of before; is this an instance of sunk cost where you over-val­ued them sim­ply because you have invested into hold­ing onto them for X months, an instance of where it is more valu­able because it’s yours (a bias which does­n’t change with addi­tional invest­men­t)—or is this an instance of you sim­ply never before devot­ing a few sec­onds to pon­der­ing whether you gen­uinely liked that check­ered scarf & if you haven’t worn it in years how likely are you to ever wear it again? When we see an appar­ent sunk cost, might we not be see­ing a well-de­vel­oped which made sense when it was devel­oped and per­haps has sim­ply never been crit­i­cally re-ex­am­ined in the light of cur­rent cir­cum­stances? Habits are invalu­able, but they are also invis­i­ble and indurate except at times of cri­sis where one is re-pri­or­i­tiz­ing things.9 Even in cor­po­ra­tions, where sunk cost think­ing is at its worst, many of the instances (eg the new CEO who rad­i­cally over­hauls the com­pany by cut­ting prod­ucts & divi­sions & employ­ees) are often sim­ply exe­cut­ing changes that the rest of the com­pany knows are long over­due but could never quite rise to a pri­or­ity with­out the Schelling point of a new CEO brought on to shake things up. (Or indeed, in gen­er­al: “never let a cri­sis go to waste.”)

Few peo­ple per­se­vere in a mis­taken choice of col­lege degree because they truly value that they have obtained irra­tionally more solely because they have already spent a lot of money on it, which is the clas­sic ‘sunk cost bias’. Usu­al­ly, it’s more that they are so busy with classes & stu­dent life & projects & hob­bies that they don’t think about it, con­tin­u­ing with the orig­i­nal plan is the path of least reflec­tion, the occa­sional stray thoughts ‘maybe this is the wrong path’ are too painful to pur­sue more than briefly, and they have not sat down and pon­dered even 5 min­utes the costs/benefits or how well it’s been going and seri­ously opened up inter­nally to the pos­si­bil­ity of quit­ting. One con­tin­ues because one con­tin­ues. Nor is there nec­es­sar­ily any point at which they will be forced to con­sider this before grad­u­a­tion, as col­lege sys­tems are geared to usher one from enroll­ment to grad­u­a­tion, and one does­n’t have to make an extra­or­di­nary effort at any point to con­tinue on that path. (One does for grad­u­ate school, which is for­tu­nate, con­sid­er­ing how much stu­dent debt that can entail, but then the same dynamic will kick in once one is in grad school.) Or at what point does a com­muter real­ize that the trade­off isn’t that great? Any doubts may sim­ply starve for lack of thought to feed them, until one day, one sud­denly ‘wakes up’.

Finding New Socks

“It is a pro­foundly erro­neous tru­ism, repeated by all copy­-books and by emi­nent peo­ple when they are mak­ing speech­es, that we should cul­ti­vate the habit of think­ing of what we are doing. The pre­cise oppo­site is the case. Civ­i­liza­tion advances by extend­ing the num­ber of impor­tant oper­a­tions which we can per­form with­out think­ing about them. Oper­a­tions of thought are like cav­alry charges in a bat­tle—they are strictly lim­ited in num­ber, they require fresh hors­es, and must only be made at deci­sive moments…It is inter­est­ing to note how impor­tant for the devel­op­ment of sci­ence a mod­est-look­ing sym­bol may be.”

, An Intro­duc­tion to Math­e­mat­ics (1911)

Many of the best anti-bias mech­a­nisms or ‘life hacks’ or ‘habits’ are about strate­gic appli­ca­tion of our lim­ited Sys­tem II resources, often employ­ing exter­nal sys­tems to fight star­va­tion.

The sim­plest wake-up mech­a­nism is hav­ing a habit to occa­sion­ally review the past, like review­ing one’s ledgers at the end of every month.10 The hum­ble check­list, for exam­ple; or , reminder or note-tak­ing soft­ware, spreadsheets/double-entry ledgers, emails with timers, ‘lint’ tools, many ‘life hacks’ in gen­er­al…11 I am heav­ily reliant on my cal­en­dar soft­ware to remind me to check in on var­i­ous papers or peo­ple, do exports/backups which can’t be eas­ily auto­mat­ed, update pages, and re-e­val­u­ate things peri­od­i­cal­ly; in writ­ing things, I have found it worth­while to develop my own check­list and am con­stantly expand­ing my writ­ing lin­ter, markdown-lint & my site build/sync script, with new errors to watch out for.

Such sys­tems effi­ciently inter­vene only at crit­i­cal moments, and sys­tem­at­i­cally cover avail­able options to over­come Sys­tem I inertia/forgetting: a check­list reminds one of every nec­es­sary step, while poka-yoke error-proofing remove error cases or at least add them to check­lists, and point­ing-and-call­ing is a phys­i­cal imple­men­ta­tion of the men­tal process of check­list­ing, while time-based tools like cal­en­dars can be sched­uled in advance to fire only at the crit­i­cal moment to save all the cog­ni­tion from now to then. And suffi­ciently reli­able auto­mated tools can go one bet­ter and only inter­rupt one, wak­ing up Sys­tem II, only if there is actu­ally an error which needs to be fixed.

Exploration

Under­use of Sys­tem II par­tic­u­larly man­i­fests as over-exploitation/under-exploration, where large poten­tial improve­ments are fore­gone because of a lack of a habit or other sys­tem­atic fac­tor which would trig­ger explo­ration. (By explo­ration, I don’t mean spend­ing hours read­ing reviews on Ama­zon or on social media, or read­ing yet another book on a top­ic, which is largely about feed­ing idle curios­ity & is infor­ma­tion super-s­tim­uli, but actual exper­i­men­ta­tion and try­ing.)

One way to mea­sure under­-ex­plo­ration is not­ing instances where exoge­nous ran­dom­iza­tion or destruc­tion of the sta­tus quo option leads to per­ma­nent changes or net effi­ciency gains after the shock is removed, indi­cat­ing learn­ing or that the sta­tus quo was sub­op­ti­mal all along12. (One area where under­-ex­plo­ration is espe­cially rife is in ran­dom­ized exper­i­ments in sci­ence, where what every­one ‘knows’ based on cor­re­la­tion , yet despite the large implied regrets, it is still held to be ‘uneth­i­cal’ to run more ran­dom­ized exper­i­ments.)

Har­vard econ­o­mist Send­hil Mul­lainathan asks “Why Try­ing New Things Is So Hard to Do”, putting it well with a famil­iar exam­ple, gro­cery shop­ping:

I drink a lot of Diet Coke: two liters a day, almost six cans’ worth. I’m not proud of the habit, but I really like the taste of Diet Coke. As a fru­gal econ­o­mist, I’m well aware that switch­ing to a generic brand would save me mon­ey, not just once but dai­ly, for weeks and years to come. Yet I only drink Diet Coke. I’ve never even sam­pled generic soda.

Why not? I’ve cer­tainly thought about it. And I tell myself that the dol­lars involved are incon­se­quen­tial, real­ly, that I’m happy with what I’m already drink­ing and that I can afford to be pas­sive about this lit­tle extrav­a­gance. Yet I’m clearly mak­ing an error, one that reveals a deeper deci­sion-mak­ing bias whose cumu­la­tive cost is siz­able: Like most peo­ple, I con­duct rel­a­tively few exper­i­ments in my per­sonal life, in both small and big things.

This is a pity because exper­i­men­ta­tion can pro­duce out­size rewards. For exam­ple, I would­n’t be risk­ing much by try­ing a generic soda, and if I liked it enough to switch, the pay­out could be big: All my future sodas would be cheap­er. When the same choice is made over and over again, the down­side of try­ing some­thing differ­ent is lim­ited and fixed—that one soda is unap­peal­ing—while the poten­tial gains are dis­pro­por­tion­ately large. One study esti­mated that 47% of human behav­iors are of this habit­ual vari­ety.

Yet many peo­ple per­sist in buy­ing branded prod­ucts even when equiv­a­lent gener­ics are avail­able. These choices are note­wor­thy for drugs, when gener­ics and branded options are chem­i­cally equiv­a­lent. Why con­tinue to buy a name-brand aspirin when the same chem­i­cal com­pound sits nearby at a cheaper price?

Gro­cery shop­ping is a great exam­ple because it is some­thing every­one does, often, which rep­re­sents a sub­stan­tial por­tion of per­sonal bud­gets, with clear & unam­bigu­ous costs, where the diffi­culty of exper­i­men­ta­tion is so min­i­mal that it feels weird to even call activ­i­ties like ‘com­pare prices & try differ­ent foods’ by a term as fancy as “exper­i­men­ta­tion”, where the ben­e­fits of learn­ing are large & can last decades. (Aldi isn’t going to sud­denly become more expen­sive than Whole Foods, and rank-order­ing of prices remains rel­a­tively con­stan­t—that’s the whole point of hav­ing brands, after all). Yet, we still don’t.

And the ben­e­fits are large. As Mul­lainathan notes, while the cost in a sin­gle instance may be small, the total loss (“regret”) is much larger because it is repeated across a life­time. If you choose to drink Diet Coke and it costs +$0.25/can (let’s say the generic costs $0.75/can and Diet Coke $1/can, and if you dis­like the generic you’ll throw it away), you haven’t lost $0.25, you have lost much more than that, because it is not a one-off deci­sion about a sin­gle drink—you are buy­ing infor­ma­tion for all your future choic­es, and the “” of the exper­i­ment is far higher than the triv­ial upfront cost.

Sup­pose you drink 1 coke a day. The differ­ence is $0.25/day, or, $91 a year. The gain from switch­ing does not stop after a year, it goes on indefi­nite­ly, so at a fairly psy­cho­log­i­cally nor­mal dis­count rate of 5%, the of the gain is $187113. In order for your exper­i­ment to not cover $0.75 and not be profitable, you would have to assign a prior prob­a­bil­ity of <0.013% to the generic being as good (or bet­ter!) and you switch­ing and reap­ing a gain of $1871. Which would be crazy because as Mul­lainathan also notes, every­one knows that often the generic ver­sion is fine, and indeed, fre­quently is lit­er­ally the same as the brand name, either because they use the same man­u­fac­tur­ers or because the seller is imple­ment­ing .

And let’s not pre­tend that this is any great heroic effort, requir­ing advanced sta­tis­tics or long-term exper­i­men­ta­tion or blind­ing.14 It takes a sec­ond to grab the generic soda from off the shelf next to the Diet Coke, and a few sec­onds later in the kitchen to try them side by side; are they about the same? Then great! You can enjoy the sav­ings from buy­ing generic thence­forth, oth­er­wise, toss the generic soda; either way, there’s no need to think about it fur­ther.

This applies as well to any other sta­ples you might buy. Is King Arthur flour really worth pay­ing twice as much as Gold Medal flour? (Not that I’ve ever noticed in my bak­ing.) Per­haps if you tried all 6 kinds of apple­sauce you’d find one of the cheaper ones tastes bet­ter than the expen­sive ones. (I did. It does­n’t add sweet­en­er, and I think most apple­sauces are over­sweet­ened. I want it to taste like apples, not corn syrup.) Is ‘scrap bacon’ ter­ri­ble in some way that makes cost­ing half as much as reg­u­lar bacon a lie? (Nope: tastes as deli­cious to me, and I can buy twice as much.) Can you tell the differ­ence between the expen­sive imported Finnish/Irish but­ter and the generic Wal­mart but­ter? (I can… eat­ing it straight while con­cen­trat­ing care­ful­ly. But I can’t on bread or any­where I would use said but­ter.) And is Smuck­er’s “nat­ural peanut but­ter” any bet­ter than your ordi­nary Jiff or generic peanut but­ter? (Trick ques­tion—I actu­ally think it tastes much bet­ter than reg­u­lar peanut but­ter & that’s what I buy. But, I only know this because I tried them all; oth­er­wise, I would­n’t’ve bought some­thing as weird-look­ing as peanut but­ter which still has its orig­i­nal peanut oil.)

Per­son­al­ly, I make a point of, when­ever try­ing some­thing new like food, to buy 1 of every­thing, to the extent pos­si­ble, and sim­ply try­ing them all. I am no longer sur­prised when I find that the generic is as good or bet­ter at a third or less the cost (how on earth do brands main­tain their profits when it’s so easy to com­pare?), or that I pre­fer some­thing I did­n’t expect to pre­fer. (Par­tic­u­larly in tea this has paid off in learn­ing about strange things like twig tea.) I think it’s crazy how peo­ple will buy the same thing for­ever and over­spend on brand names, and, while they’re at it, never try another gro­cery store (switch­ing to Wal­mart saved me >10%, and then switch­ing to Aldi another >10%), and pass up bulk sav­ings to buy the small­est pos­si­ble quan­ti­ties. And then they com­plain their monthly gro­cery bill is $400 and they won­der where all the money goes… It is waste­ful to not be waste­ful.

If we so often under­-ex­plore in gro­ceries, we surely under­-ex­plore else­where too. What can help ame­lio­rate this is delib­er­ate forc­ing of explo­ration. With gro­ceries, my rule of buy­ing mul­ti­ples the first time is a sim­ple eas­i­ly-im­ple­mented heuris­tic to force explo­ration of gro­cery options. With music, I try to avoid my tastes ‘freez­ing’ into what­ever I lis­tened to as a teenager by lis­ten­ing to large musi­cal dumps rather than rec­om­men­da­tions (eg com­pi­la­tion­s), and avoid­ing the band­wagon effects of pop­u­lar media.15 With research, sys­tem­atic read­ing of all papers on a given topic rather than the most-cited ones can lead to many inter­est­ing but still obscure papers.

We can try to com­pen­sate for our lack of mind­ful­ness in other areas too. With socks, my new heuris­tic is expand my annual pho­to­graphic inven­tory of my per­sonal pos­ses­sions (mak­ing a record of every­thing I own in case of dis­as­ter) to include clothes too16; in con­sid­er­ing my clothes, I expect that I will notice when I get low on sock­s—or any other kind of cloth­ing—and can take action before too many years pass and my sock­pile becomes inad­e­quate. I will surely dis­cover other inad­e­qua­cies in the future, but, if I am mind­ful of my lim­its, fewer and few­er, and they will get less in the way of more impor­tant things.

See Also

Appendix

Grocery shopping advice

To expand on the topic of exper­i­ment­ing & gro­cery shop­ping, I would sum­ma­rize good gro­cery shop­ping as involv­ing, (in descend­ing order of mar­ginal return­s), advance plan­ning to select effi­cient tar­gets, selec­tion of gro­cery stores by total cost (in­clud­ing travel time), selec­tion of cheap­est ver­sion (ex­per­i­men­ta­tion up front, then select­ing by unit cost), avoid­ing gro­cery store trick­ery like coupons, and using assis­tance like a stan­dard gro­cery store shop­ping list to main­tain cor­rect­ness of deci­sions.

  1. plan recipes ahead to avoid impulse shop­ping and food wastage while puz­zling how to eat some­thing. Resources on fru­gal cook­ing are every­where and you can find tons of advice on eg cook­ing soup or stew. You should empha­size min­i­mally unprocessed goods which are com­modi­ties and so cheap, with fewer lay­ers of bogus prod­uct differentiation/overhead/advertising.

    Frozen veg­eta­bles, for exam­ple, are under­rated in terms of both con­ve­nience (why spend that time clean­ing & chop­ping when indus­trial machines are so much more effi­cient at that?) and also cost, as they don’t go bad; and depend­ing on the veg­etable & time of year, can eas­ily be bet­ter-qual­ity as well. (Car­rots are a stand­out case: you can go through the has­sle of clean­ing, peel­ing, and chop­ping a sad bowl of fresh car­rots which you must eat before they go bad, or you can eat some deli­cious pre-chopped car­rots when­ever you feel like pulling them out of your freez­er, pos­si­bly a year lat­er.)

    (I would­n’t take ‘health’ too seri­ously as a cri­te­ri­on. Diet/nutrition research is one of the worst fields in all med­i­cine. Don’t sac­ri­fice your qual­ity of life now for some small late-late QALYs which may not exist at all.)

  2. inves­ti­gate all local gro­ceries. The aver­age price can differ con­sid­er­ably between stores.

    In my own fea­si­ble shop­ping area, I have Wal­mart, Tar­get, Shop­pers, Aldi, Giant and some oth­ers (BJ’s is the major alter­na­tive but I’ve never been con­vinced I would be able to buy enough to ben­e­fit). When I switched from NEX to Wal­mart, I saved a good 10%; when I switched (most of) my shop­ping to Aldi, I saved another good 10%. (The cost sav­ings had I started with Whole Foods or Har­ris Teeter hardly bear think­ing on.) There are some dis­ad­van­tages to shop­ping at Aldi (more restricted selec­tion, dis­or­ga­nized store, hav­ing to remem­ber to bring a quar­ter for the shop­ping carts) but sav­ing $20 or $30 is a good salve for the annoy­ances. It may take some time to get famil­iar with a store (I take about an hour to thor­oughly walk through a store, look­ing at where every­thing is and not­ing prices for things I often buy), but con­sider the Value of Infor­ma­tion: if you spend 2 or 3 hours to find a new gro­cery store and save 10–20%, that’s a sav­ings of eas­ily $120+ a year for a NPV of some­thing like $2k. () And there’s not that many to check.

  3. in choos­ing a gro­cery store and what to buy, remem­ber the costs also of travel and time spent shop­ping.

    The goal is to get your gro­ceries for a total cost which min­i­mizes mon­ey, time, and effort. Every sec­ond spent shop­ping is a waste-–cer­tainly I don’t par­tic­u­larly enjoy it. The cost of dri­ving to a store is some­where around $0.10-$0.50 per mile, and then there is the risk of acci­dents and your own time; adding up the mileage and time, I get ~$15 per gro­cery trip. This is a sub­stan­tial frac­tion of the total cost of my gro­ceries, and so I keep that in mind when plan­ning: I shop once a mon­th, stock­ing up as much as pos­si­ble. I’d much rather make one trip to buy a lot of food at $120+$15=$135 than two trips at $60+$60+$15+$15=$150! (In this respect, Aldi is a wash for me: I have to spend some­what longer dri­ving to it, but it’s so much more com­pact and tiny that I spend much less time walk­ing around it and check­ing out.) Travel time is also why it makes a lot of sense to occa­sion­ally buy from the local dol­lar store about 3 min­utes away-–when a sin­gle trip costs $15, then even if a bot­tle of ketchup or what­ever costs twice as much as at Aldi, it’s still a lot cheap­er. (Although if you find your­self resort­ing to that too often, it sug­gests you are mak­ing mis­takes fur­ther upstream.)

  4. in buy­ing a spe­cific ingre­di­ent, always start with the *unit* cost.

    Many foods keep a long time and you can eas­ily make use of a larger quan­ti­ty. It’s some­what unusual for some­thing to be too big to buy and a bad idea due to spoilage/opportunity cost (usu­ally some­thing either per­ish­able, like fruit, or ridicu­lously long-last­ing and more expen­sive in oppor­tu­nity cost than up front; eg a few months ago, I fin­ished off a bot­tle of molasses which dat­ed, as best as I could infer from the copy­rights on the label, from ~1995, and it would not be a good idea to buy a big bot­tle of molasses if you only use it once in a while like I do, for bak­ing rye bread).

  5. when buy­ing a new ingre­di­ent, start with the gener­ic.

  6. If you have doubts about buy­ing gener­ic, test it: require the much more expen­sive brand-name goods to jus­tify their exis­tence.

    My pref­er­ence is to take into account Value of Infor­ma­tion: by the same logic as choos­ing gro­ceries, reject­ing a cheap generic food in favor of an expen­sive one is an expen­sive mis­take as you incur it indefi­nite­ly.

    One of my pet peeves is how much money peo­ple waste on brand-name goods rather than default­ing to gener­ics or off-brands, when there is rarely a notice­able taste differ­ence to me. So my sug­ges­tion is that when­ever you try some­thing new, buy 1 of every­thing and try them out side by side to see what you like and if the brand-name qual­ity can pos­si­bly jus­tify pay­ing so much more. I’ve done this with but­ter, milk, apple­sauce, cere­al, bacon, sausage, mus­tard, ice cream, etc. It baffles me how few peo­ple appar­ently take advan­tage of this—­like at Wal­mart, the ‘irreg­u­lar bacon’ tastes lit­er­ally iden­ti­cal and is not that differ­ent from the reg­u­lar bacon and yet is always almost half-price per ounce! Half! If I spend $8 a month rather than $4 on bacon, that’s a NPV of -$983. Quite an expen­sive mis­take to make over a life­time.

    I don’t advise rein­force­ment learn­ing-style approaches like Thomp­son sam­pling. Why? Because the VoI for test­ing all options is so high, you can sam­ple them all simul­ta­ne­ously (mak­ing it more of a mul­ti­ple-play MAB), there is large cog­ni­tive costs to main­tain­ing options (the point is to get in and out as fast as pos­si­ble, remem­ber, to min­i­mize time-cost) and so each sam­ple has a fixed cost (which is ignored in the usual MAB for­mu­la­tion where it’s assumed you have to choose each round any­way), and in my expe­ri­ence sam­pling food­stuffs, not many things are ‘acquired tastes’ where mul­ti­ple tastes will yield a differ­ent result, and there is not that much noise in taste com­par­isons of this sort. Typ­i­cal­ly, I try some­thing and I imme­di­ately can tell that the generics/brand-name are equiv­a­lent or which one is much supe­ri­or. (And if the differ­ence is sub­tle, then it does­n’t mat­ter much, but typ­i­cally the price differ­ence is not sub­tle.) If there is no noise, the EV is highly pos­i­tive, and you can take mul­ti­ple actions simul­ta­ne­ous­ly, tak­ing a Thomp­son sam­pling or sequen­tial test­ing approach is merely incur­ring unnec­es­sary regret and com­plex­ity com­pared to a sin­gle-trial deci­sion approach.

    So it’s best to do a sin­gle pre­cise test of all avail­able con­tenders, and then buy the top-ranked item from then on with­out think­ing about it fur­ther. Does the opti­mal buy change? May­be, but food prices are fairly sta­ble in a rel­a­tive rank-order sense (eg when bacon spiked in price ~2017, all the bacons did simul­ta­ne­ous­ly, so I wound up buy­ing the exact same dis­count bacon, but less), so the deci­sions don’t seem to need to be revis­ited more. Even if the infor­ma­tion decays, the tests are still worth run­ning because aside from learn­ing about the spe­cific food type you’re test­ing, you ben­e­fit from get­ting an idea of the gen­eral range of vari­a­tion in food taste/quality and how much a brand-name is worth (ie ‘lit­tle’).

  7. skip coupons and sales.

    They are neg­a­tive sum games after tak­ing into account the time wasted sort­ing through the gim­micks and all the options, intended to get things you did­n’t want to buy in the first place, even at the dis­counted price, even occa­sional mis­takes will wipe out the sav­ings, and they dis­cour­age exper­i­men­ta­tion and com­par­i­son (you would­n’t want to buy the other apple­sauce which you don’t have a coupon for, would you? why, that would be a ripoff!); worse, they are by defi­n­i­tion ephemeral (mak­ing any effort spent on them “toil”), so your gained knowl­edge and effort becomes imme­di­ately worth­less, as com­pared to sta­ble long-term knowl­edge like which gro­cery store is cheap­est, where all items are locat­ed, which gener­ics to buy etc. Like credit card churn­ing or fre­quen­t-flier miles, they should be avoided as traps. Life is far too short.

  8. Gro­cery lists should be kept reg­u­larly and reused as tem­plates to avoid for­get­ting about impor­tant things or indulging in impulse or spree pur­chas­ing.

  9. Track­ing expen­di­tures can be help­ful in find­ing cat­e­gories of food which have been get­ting imbal­anced spend­ing and review­ing enjoyment/$ trade­offs.

  10. After eval­u­at­ing stores, learn­ing where items are, fin­ish­ing taste com­par­isons, pick­ing recipes, mak­ing tem­plate lists, the whole process should­n’t occupy more than an hour or so a mon­th: you take your tem­plate, mod­ify slightly for cur­rent recipes, drive there, dash in to the pre­spec­i­fied items, buy those, and get out.

There are doubt­less fur­ther opti­miza­tions which could be made, but by that point, I believe they truly are into the realm of ‘over­think­ing it’, and one should move one’s scarce delib­er­a­tive capac­ity onto other top­ics (like career plan­ning).

To recap:

  1. plan sen­si­ble cheap meals
  2. find the cheap­est local gro­cery store
  3. buy as rarely as pos­si­ble, in bulk, and generic (un­less a food is proven in taste-test­ing to be supe­ri­or); get in and out and don’t be tempt­ed.

In terms of opti­miz­ing, keep in mind the Pareto prin­ci­ple: quan­ti­ta­tive­ly, I think the biggest wins comes in this order:

  1. choice of foods (<10x differ­ence in cost)
  2. generic vs brand-name (1–3x)
  3. choice of gro­cery store (<=1.3x)
  4. buy­ing bulk (1–1.5x)
  5. loca­tion and fre­quency of vis­its (1–1.1x)
  6. in-s­tore shop­ping effi­ciency (1–1.05x)

  1. All the same type, of course, because who wants to spend time match­ing up socks? Have as few types as pos­si­ble so you can throw them in a drawer or some­thing.↩︎

  2. Where do all those miss­ing socks go? Sam­sung’s sock survey/interviews sug­gests no par­tic­u­lar rea­son:

    There are many prac­ti­cal rea­sons for sock loss rather than super­nat­ural dis­ap­pear­ances. Research inter­views found the com­mon causes included items falling behind radi­a­tors or under fur­ni­ture with­out any­one real­is­ing, stray items being added to the wrong coloured wash and becom­ing sep­a­rated from its match­ing sock, not being secured to a wash­ing line securely so they fall off and blow away—or they are sim­ply care­lessly paired up.

    To which I would add, in mul­ti­-per­son house­holds, socks have a ten­dency to migrate to other peo­ple’s rooms (ac­ci­den­tally or not), flow­ing along a sock gra­di­ent. (I lost a lot of socks to my broth­er. I know because we labeled them with mark­ers and I’d reg­u­larly find them in his draw­er.) Some­times they get phys­i­cally lost in the dry­er. In clut­tered house­holds, it’s easy for a sock to fall out of the dryer or the bas­ket when you’re mov­ing a big load, or fall behind drawers/beds and get lost there. Pet ani­mals can steal them: I’ve seen fer­rets mak­ing off with socks to hide in cor­ners (or behind the dry­er), and sup­pos­edly often have a just for socks & woolens (“wool suck­ing”). And in some cas­es, there may be things man was not meant to know.↩︎

  3. We can’t use Pear­son’s r because these responses are cat­e­gor­i­cal, not con­tin­u­ous num­bers. (While most sur­vey soft­ware sup­ports free response or con­tin­u­ous num­bers, they are typ­i­cally a bad idea because peo­ple will take any chance they get to feed in bad data or wild respons­es.) To han­dle them prop­er­ly, I use the . Poly­choric cor­re­la­tions han­dle ordi­nal data by assum­ing a latent nor­mal vari­able which is dis­cretized as the observed vari­able (sim­i­lar to the ), and to cor­re­late 2 ordi­nal vari­ables, asks what the cor­re­la­tion of the 2 latent nor­mal vari­ables is, which is then the same as the famil­iar r. (It’s more prin­ci­pled than the com­mon approach of sim­ply turn­ing the ordi­nal scale into inte­gers, and using them as-is, any­way.)

    In this case, sock count and sock pur­chase fre­quency might not be nor­mally dis­trib­ut­ed, but I’m happy to assume there’s a latent vari­able; the pur­chaser vari­able is more ques­tion­able but I think the options can be ranked by a kind of ‘social dis­tance’ and it’s rea­son­able to use the poly­choric here as well.↩︎

  4. Real­is­ti­cal­ly, how much could a top-notch pair of wool socks cost? $30 a pair? If you bought 5 pairs at $150 for a recip­i­ent who needs them, that would amaze them; on the other hand, if you bought them, say, a lap­top at the same cost, it’s prob­a­bly going to be a crummy lap­top they don’t want to use and they’ll feel a mix of anger & guilt at being given a white ele­phant.↩︎

  5. I am think­ing of exam­ples like the items on the DOI/A-DMC scales, such as: lock­ing keys in one’s car/throwing out gro­ceries which went bad/never wear­ing pur­chased clothes/missing an air­plane flight/getting an STD/overdrawing an account/declaring bank­rupt­cy.↩︎

  6. Trans­lated on pg47, “The Sov­er­eign and the Study of War”, Fred­er­ick the Great on the Art of War, Jay Luvaas 1966/1999 ISBN 0-306-80908-7; Luvaas cites it to “Réflex­ions sur la tac­tique et sur quelques par­ties de la guer­re, ou Réflex­ions sur quelques changes­ments dans la facon de fairre la guerre”, Oeu­vres 28, pg153–154 [Réflex­ions sur la tac­tique], of Oeu­vres de Fred­eric le Grand (30 vol­umes, 1846–1856).↩︎

  7. But not all toil can or should be elim­i­nat­ed, as the attempt to do so can all too eas­ily back­fire & waste time on net, in a ver­sion of Brian Kernighan’s quote, “Every­one knows that debug­ging is twice as hard as writ­ing a pro­gram in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?” Sim­i­lar­ly, I avoid sock sub­scrip­tion ser­vices, because they are expen­sive and I expect that the has­sle of deal­ing with them—changes in terms, reg­u­lar deliv­er­ies, refunds or billing prob­lems, ser­vices clos­ing etc—ex­ceeds any sup­posed con­ve­nience or time-sav­ings.↩︎

  8. Not fal­si­fy­ing one’s own beliefs or moves is the nat­ural habit of humans, and real effort is reserved for beliefs of other peo­ple and espe­cially ene­mies—sim­ply imag­ing that one’s belief is held by an imag­i­nary friend aids fal­si­fi­ca­tion! See Cow­ley & Byrne 2005, “When Fal­si­fi­ca­tion is the Only Path to Truth”; for exten­sive back­ground on the the­ory that rea­son is prin­ci­pally about argu­ing & per­sua­sion, see Mercier & Sper­ber 2010’s “Why do humans rea­son? Argu­ments for an argu­men­ta­tive the­ory”.↩︎

  9. An inter­est­ing exam­ple is afforded by the NY Times’s “How Com­pa­nies Learn Your Secrets”:

    …two col­leagues from the mar­ket­ing depart­ment stopped by his desk to ask an odd ques­tion: “If we wanted to fig­ure out if a cus­tomer is preg­nant, even if she did­n’t want us to know, can you do that?” As the mar­keters explained to Pole—and as Pole later explained to me, back when we were still speak­ing and before Tar­get told him to stop—new par­ents are a retail­er’s holy grail. Most shop­pers don’t buy every­thing they need at one store. Instead, they buy gro­ceries at the gro­cery store and toys at the toy store, and they visit Tar­get only when they need cer­tain items they asso­ciate with Tar­get—­clean­ing sup­plies, say, or new socks or a six-month sup­ply of toi­let paper. But Tar­get sells every­thing from milk to stuffed ani­mals to lawn fur­ni­ture to elec­tron­ics, so one of the com­pa­ny’s pri­mary goals is con­vinc­ing cus­tomers that the only store they need is Tar­get. But it’s a tough mes­sage to get across, even with the most inge­nious ad cam­paigns, because once con­sumers’ shop­ping habits are ingrained, it’s incred­i­bly diffi­cult to change them. There are, how­ev­er, some brief peri­ods in a per­son’s life when old rou­tines fall apart and buy­ing habits are sud­denly in flux. One of those moments—the moment, real­ly—is right around the birth of a child, when par­ents are exhausted and over­whelmed and their shop­ping pat­terns and brand loy­al­ties are up for grabs. But as Tar­get’s mar­keters explained to Pole, tim­ing is every­thing. Because birth records are usu­ally pub­lic, the moment a cou­ple have a new baby, they are almost instan­ta­neously bar­raged with offers and incen­tives and adver­tise­ments from all sorts of com­pa­nies. Which means that the key is to reach them ear­lier, before any other retail­ers know a baby is on the way. Specifi­cal­ly, the mar­keters said they wanted to send spe­cially designed ads to women in their sec­ond trimester, which is when most expec­tant moth­ers begin buy­ing all sorts of new things, like pre­na­tal vit­a­mins and mater­nity cloth­ing. “Can you give us a list?” the mar­keters asked. “We knew that if we could iden­tify them in their sec­ond trimester, there’s a good chance we could cap­ture them for years,” Pole told me. “As soon as we get them buy­ing dia­pers from us, they’re going to start buy­ing every­thing else too. If you’re rush­ing through the store, look­ing for bot­tles, and you pass orange juice, you’ll grab a car­ton. Oh, and there’s that new DVD I want. Soon, you’ll be buy­ing cereal and paper tow­els from us, and keep com­ing back.”…As the abil­ity to ana­lyze data has grown more and more fine-grained, the push to under­stand how daily habits influ­ence our deci­sions has become one of the most excit­ing top­ics in clin­i­cal research, even though most of us are hardly aware those pat­terns exist. One study from Duke Uni­ver­sity esti­mated that habits, rather than con­scious deci­sion-mak­ing, shape 45% of the choices we make every day, and recent dis­cov­er­ies have begun to change every­thing from the way we think about diet­ing to how doc­tors con­ceive treat­ments for anx­i­ety, depres­sion and addic­tions.

    Habits are thought­less:

    …The first time a rat was placed in the maze, it would usu­ally wan­der slowly up and down the cen­ter aisle after the bar­rier slid away, sniffing in cor­ners and scratch­ing at walls. It appeared to smell the choco­late but could­n’t fig­ure out how to find it. There was no dis­cernible pat­tern in the rat’s mean­der­ings and no indi­ca­tion it was work­ing hard to find the treat. The probes in the rats’ heads, how­ev­er, told a differ­ent sto­ry. While each ani­mal wan­dered through the maze, its brain was work­ing furi­ous­ly. Every time a rat sniffed the air or scratched a wall, the neu­rosen­sors inside the ani­mal’s head exploded with activ­i­ty. As the sci­en­tists repeated the exper­i­ment, again and again, the rats even­tu­ally stopped sniffing cor­ners and mak­ing wrong turns and began to zip through the maze with more and more speed. And within their brains, some­thing unex­pected occurred: as each rat learned how to com­plete the maze more quick­ly, its men­tal activ­ity decreased. As the path became more and more auto­mat­ic—as it became a habit—the rats started think­ing less and less. This process, in which the brain con­verts a sequence of actions into an auto­matic rou­tine, is called . There are dozens, if not hun­dreds, of behav­ioral chunks we rely on every day. Some are sim­ple: you auto­mat­i­cally put tooth­paste on your tooth­brush before stick­ing it in your mouth. Some, like mak­ing the kids’ lunch, are a lit­tle more com­plex. Still oth­ers are so com­pli­cated that it’s remark­able to real­ize that a habit could have emerged at all…What Gray­biel and her col­leagues found was that, as the abil­ity to nav­i­gate the maze became habit­u­al, there were two spikes in the rats’ brain activ­i­ty—once at the begin­ning of the maze, when the rat heard the click right before the bar­rier slid away, and once at the end, when the rat found the choco­late. Those spikes show when the rats’ brains were fully engaged, and the dip in neural activ­ity between the spikes showed when the habit took over. From behind the par­ti­tion, the rat was­n’t sure what waited on the other side, until it heard the click, which it had come to asso­ciate with the maze. Once it heard that sound, it knew to use the “maze habit”, and its brain activ­ity decreased. Then at the end of the rou­tine, when the reward appeared, the brain shook itself awake again and the choco­late sig­naled to the rat that this par­tic­u­lar habit was worth remem­ber­ing, and the neu­ro­log­i­cal path­way was carved that much deep­er.

    ↩︎
  10. To bal­ance review with exe­cu­tion, one could allo­cate a fixed per­cent­age of time at mul­ti­ple time scales for review & meta-re­view : at each lev­el, allo­cate X%, which will con­verg­ing to a finite bounded X.X…% for all meta lev­els. For exam­ple, if one allo­cates 5% for reg­u­lar review, one would spend 1 ‘meta’ day for every 20 days of work; and 1 ‘meta-meta’ day for every 400 days of work; and 1 ‘meta-meta-meta’ day every 8000 days… (Or: month­ly, year­ly, bidece­nial­ly). This is loosely anal­o­gous to .↩︎

  11. If life hacks like never reach fix­a­tion, per­haps that rep­re­sents an inverse .↩︎

  12. Some inter­est­ing links on local optima/greediness/risk-aversion (eg the “” or the )

    An amus­ing fic­tional exam­ple might be the My Lit­tle Pony episode “Apple­jack’s ‘Day’ Off”. An inter­est­ing paper, although not clearly estab­lish­ing sub­op­ti­mal explo­ration, is , which uses an extremely large dataset of restau­rant orders by indi­vid­u­als from ; also inter­est­ing is Alessan­dretti et al 2016, which finds that peo­ple visit few phys­i­cal loca­tions, and while they do con­tin­u­ously explore new loca­tions & shift, there appears to be an equiv­a­lent of a , sug­gest­ing lim­its to human abil­i­ties to eas­ily plan/remember activ­i­ties cov­er­ing more than ~25 loca­tions reg­u­lar­ly.↩︎

  13. Using the usual approx­i­ma­tion for the NPV of an indefi­nite dis­counted income stream: .↩︎

  14. As much fun as I find run­ning blinded exper­i­ments like , I acknowl­edge it may not be many other peo­ple’s idea of fun.↩︎

  15. I am think­ing par­tic­u­larly of the Yahoo exper­i­ments: Sal­ganik et al 2006//Sal­ganik & Watts 2009.↩︎

  16. Pho­to­graphic inven­to­ries are some­times sug­gested for renters as part of renters insur­ance, in order to claim reim­burse­ment from the insur­er, but I orig­i­nally started doing pho­to­graphic inven­to­ries as a dis­as­ter pre­pared­ness thing. Where I live, hur­ri­canes and flood­ing are seri­ous con­cerns: a few years after I moved in, I had to install insu­la­tion under the floor of my bed­room because the pre­vi­ous insu­la­tion had been washed out by flood­ing from a hur­ri­cane sev­eral years pre­vi­ous­ly. I also came uncom­fort­ably close to being flooded out by high tide dur­ing another hur­ri­cane. In addi­tion, the elec­tri­cal wiring in this place was done by an ama­teur like 50 years ago. And I’m also in the evac­u­a­tion zone of a nuclear power plant. All things con­sid­ered, I decided it was a good idea to put together an evac­u­a­tion kit with iodine pills, food bars etc, a fire safe, remote Inter­net back­ups, and take pho­tos of every­thing else to assist in dis­as­ter recov­ery. Just in case.

    More imme­di­ate­ly, I find it to be use­ful dur­ing spring clean­ing. When you pho­to­graph every­thing, it forces you to say men­tally ‘oh this thing! when was the last time I used it, any­way? This turned out to be a waste. Maybe some­one else would find it more use­ful’ or reminds you to use it for some­thing.↩︎