notes/Competence (Link Bibliography)

“notes/​Competence” links:

  1. https://www.amazon.com/Frederick-Great-Art-War-Luvaas/dp/0306809087

  2. 1921-thorndike-educationalpsychology-v2-thepsychologyoflearning.pdf#page=188

  3. 1935-wechsler-rangeofhumancapacities.pdf: ⁠, David Wechsler (1935; iq):

    saw that the subjects who did well at the start of the training also improved faster as the training progressed compared with the subjects who began more slowly. “As a matter of fact”, ⁠, “in this experiment the larger individual differences increase with equal training, showing a positive correlation with high initial ability with ability to profit by training.” The passage from the Bible doesn’t quite capture Thorndike’s results accurately because every subject improved, but the rich got relatively richer. Everyone learned, but the learning rates were consistently different.

    When World War I erupted, Thorndike became a member of the Committee on Classification of Personnel, a group of psychologists commissioned by the U.S. Army to evaluate recruits [see ]. It was there that Thorndike rubbed off on a young man named ⁠, who had just finished his master’s degree in psychology. Wechsler, who would become a famous psychologist, developed a lifelong fascination with tracing the boundaries of humanity, from lower to upper limits.

    In 1935, Wechsler compiled essentially all of the credible data in the world he could find on human measurements. He scoured measures of everything from vertical jump to the duration of pregnancies to the weight of the human liver and the speeds at which card punchers at a factory could punch their cards. He organized it all in the first edition of a book with the aptly momentous title The Range of Human Capacities.

    Wechsler found that the ratio of the smallest to biggest, or best to worst, in just about any measure of humanity, from to hosiery looping [knitting], was between 2 to one and 3 to one. To Wechsler, the ratio appeared so consistent that he suggested it as a kind of universal rule of thumb.

    Phillip Ackerman, a psychologist and skill acquisition expert, is a sort of modern-day Wechsler, having combed the world’s skill-acquisition studies in an effort to determine whether practice makes equal, and his conclusion is that it depends on the task. In simple tasks, practice brings people closer together, but in complex ones, it often pulls them apart. Ackerman has designed computer simulations used to test air traffic controllers, and he says that people converge on a similar skill level with practice on the easy tasks—like clicking buttons to get planes to take off in order—but for the more complex simulations that are used for real-life controllers, “the individual differences go up”, he says, not down, with practice. In other words, there’s a on skill acquisition.

    Even among simple motor skills, where practice decreases individual differences, it never drowns them entirely. “It’s true that doing more practice helps”, Ackerman says, “but there’s not a single study where variability between subjects disappears entirely.”

    “If you go to the grocery store”, he continues, “you can look at the checkout clerk, who is using mostly perceptual motor skill. On average, the people who’ve been doing it for 10 years will get through 10 customers in the time the new people get across one. But the fastest person with 10 years’ experience will still be about 3 times faster than the slowest person with 10 years’ experience.”

  4. 1989-chambliss.pdf: “The Mundanity of Excellence: An Ethnographic Report on Stratification and Olympic Swimmers”⁠, Daniel F. Chambliss

  5. ⁠, José Luis Ricón (2019-07-28):

    Is Bloom’s “Two Sigma” phenomenon real? If so, what do we do about it?

    Educational psychologist Benjamin Bloom found that one-on-one tutoring using mastery learning led to a two sigma(!) improvement in student performance. The results were replicated. He asks in his paper that identified the “2 Sigma Problem”: how do we achieve these results in conditions more practical (ie., more scalable) than one-to-one tutoring?

    In a related vein, this large-scale shows large (>0.5 Cohen’s d) effects from direct instruction using mastery learning. “Yet, despite the very large body of research supporting its effectiveness, DI has not been widely embraced or implemented.”

    • The literatures examined here are full of small sample, non-randomized trials, and highly heterogeneous results.
    • Tutoring in general, most likely, does not reach the 2-sigma level that Bloom suggested. Likewise, it’s unlikely that mastery learning provides a 1-sigma improvement.
      • But high quality tutors, and high quality software are likely able to reach a 2-sigma improvement and beyond.
    • All the methods (mastery learning, direct instruction, tutoring, software tutoring, ⁠, and spaced repetition) studied in this essay are found to work to various degrees, outlined below.
    • This essay covers many kinds of subjects being taught, and likewise many groups (special education vs regular schools, college vs K-12). The reported here are averages that serve as general guidance.
    • The methods studied tend to be more effective for lower skilled students relative to the rest.
    • The methods studied work at all levels of education, with the exception of direct instruction: There is no evidence to judge its effectiveness at the college level.
    • The methods work substantially better when clear objectives and facts to be learned are set. There is little evidence of learning transfer: Practicing or studying X subject does not improve much performance outside of X.
    • There is some suggestive evidence that the underlying reasons these methods work are increased and repeated exposure to the material, the ⁠, and fine-grained feedback on performance in the case of tutoring.
    • Long term studies tend to find evidence of a fade-out effect, effect sizes decrease over time. This is likely due to the skills being learned not being practiced.

    Bloom noted that mastery learning had an effect size of around 1 (one sigma); while tutoring leads to d = 2. This is mostly an outlier case.

    Nonetheless, Bloom was on to something: Tutoring and mastery learning do have a degree of experimental support, and fortunately it seems that carefully designed software systems can completely replace the instructional side of traditional teaching, achieving better results, on par with one to one tutoring. However, designing them is a hard endeavour, and there is a motivational component of teachers that may not be as easily replicable purely by software.

    Overall, it’s good news that the effects are present for younger and older students, and across subjects, but the effect sizes of tutoring, mastery learning or DI are not as good as they would seem from Bloom’s paper. That said, it is true that tutoring does have large effect sizes, and that properly designed software does as well. The case study shows what is possible with software tutoring, in the case the effect sizes went even beyond Bloom’s paper.

  6. 1997-gottfredson.pdf

  7. 1997-gordon.pdf: ⁠, Robert A. Gordon (1997-01-01; iq):

    To show why the importance of intelligence is often misperceived, an analogy between single test items and single nontest actions in everyday life is drawn. 3 requirements of good test items are restated, and the analogy is employed to account for underrecognition of the importance of general intelligence in everyday actions, which often fail to meet the requirements and thus fail as intelligence measures for reasons that have little to do with their dependence on intelligence. A new perspective on the role of intelligence in nontest actions is introduced by considering its operation at 3 levels: that of the individual, that of the near context of the individual, and that of entire populations. Social scientists have misunderstood the operation and impact of IQ in populations by confining attention to the individual level. A population-IQ-outcome model is explained that tests for the pooled effects of intelligence at all 3 levels on differences between 2 populations in prevalences of certain outcomes. When the model fits, the difference between 2 populations in the outcome measured is found commensurate with the difference in their IQ or general intelligence distributions. The model is tested on and found to fit prevalences of juvenile delinquency, adult crime, single parenthood, HIV infection, poverty, belief in conspiracy rumors, and key opinions from polls about the O. J. Simpson trial and the earlier Tawana Brawley case. A deviance principle is extracted from empirical findings to indicate kinds of outcome the model will not fit. Implications for theories of practical and multiple intelligences are discussed. To understand the full policy implications of intelligence, such a fundamentally new perspective as that presented here will be needed.

  8. ⁠, Robin Hanson (2009-07-04):

    A better intuition for common abilities can be found by browsing the US National Assessment of Adult Literacy sample questions⁠.

    For example, in 1992 out of a random sample of US adults, 7% could not do item SCOR300, which is to find the expiration date on a driver’s license. 26% could not do item AB60303, which is to check the “Please Call” box on a phone message slip when they’ve been told:

    James Davidson phones and asks to speak with Ann Jones, who is at a meeting. He needs to know if the contracts he sent are satisfactory and requests that she call before 2:00PM. His number is 259-3860. Fill in the message slip below.

    Only 52% could do item AB30901, which is to look at a table on page 118 of the 1980 World Almanac and answer:

    According to the chart, did U.S. exports of oil (petroleum) increase or decrease between 1976 and 1978?

  9. McNamara

  10. ⁠, David Sirlin (2006):

    [Summary of game designer & former Street Fighter player David Sirlin’s book on the psychology of competition, Playing to Win.

    Sirlin diagnoses one of the most problematic mindsets as that of the “scrub”: the scrub is not just a bad player, they are a bad player who refuses to get better and takes pride in not getting better, in pretending as if parts of the game did not exist and any player who plays differently is immoral for doing so and they are to blame for the scrub losing. A scrub is self-handicapping, self-sabotaging, and can never get better because that would violate their made-up fantasy rules.

    Aside from ensuring that they will predictably keep losing, scrubs typically are playing a far inferior & less fun game: their imaginary rules typically ban mechanics which are critical parts of balancing the game-design—every move should have an equal and opposite move, to create a rock-paper-scissors dynamic. As new subtleties are discovered, new tactics and strategies evolve, in an ever shifting landscape of expertise. If there was simply one ‘right’ move, that would be boring and allow for no skill. (As game designer Sid Meier has famously said in a variety of ways, “Games are a series of meaningful choices.”)

    This can apply to life in general: those who will do what it takes to reach their goals (whatever those may be), and scrubs, who won’t because of imagined scruples and insistence on being handed success on a silver platter and will resentfully blame everyone but themselves for their failure.]

    If you play in such a way as to maximize your chance of winning, it means abusing everything “cheap” that you can. It means frustrating the opponent, using bugs, and anything else you can think of that’s legal to do. When all this comes together, it gives you a deeper kind of fun than is possible at lower skill levels.

    …It’s also totally fine to mess around with no intention of ever becoming really good. You don’t have to try to be the best at every game you play. I certainly don’t try that, it would be exhausting. But when I see someone else trying to be the best, I admire it, rather than condemn it. If that makes the game fall apart, I hold the game developer responsible, not the player.

    But if you want to win—if that’s your intention—then you need to leave behind whatever mental baggage you have that would prevent you from making the moves that actually help you win. By doing that and practicing and learning, you can walk the path of continuous self-improvement that Playing to Win is really about.

  11. ⁠, Dan Luu (2020-02-07):

    Reaching 95%-ile isn’t very impressive because it’s not that hard to do…most people can become (relatively) good at most things…Personally, in every activity I’ve participated in where it’s possible to get a rough percentile ranking, people who are 95%-ile constantly make mistakes that seem like they should be easy to observe and correct. “Real world” activities typically can’t be reduced to a percentile rating, but achieving what appears to be a similar level of proficiency seems similarly easy. We’ll start by looking at (a video game) in detail because it’s an activity I’m familiar with where it’s easy to get ranking information and observe what’s happening, and then we’ll look at some “real world” examples where we can observe the same phenomena, although we won’t be able to get ranking information for real world examples1.

    Overwatch: At 90%-ile and 95%-ile ranks in Overwatch, the vast majority of players will pretty much constantly make basic game losing mistakes. These are simple mistakes like standing next to the objective instead of on top of the objective while the match timer runs out, turning a probable victory into a certain defeat. See the attached footnote if you want enough detail about specific mistakes that you can decide for yourself if a mistake is “basic” or not…When I first started playing Overwatch (which is when I did that experiment), I ended up getting rated slightly above 50%-ile…Some things you’ll regularly see at slightly above 50%-ile are:

    • Supports (healers) will heal someone who’s at full health (which does nothing) while a teammate who’s next to them is dying and then dies
    • Players will not notice someone who walks directly behind the team and kills people one at a time until the entire team is killed
    • Players will shoot an enemy until only one more shot is required to kill the enemy and then switch to a different target, letting the 1-health enemy heal back to full health before shooting at that enemy again
    • After dying, players will not wait for their team to respawn and will, instead, run directly into the enemy team to fight them 1v6. This will repeat for the entire game (the game is designed to be 6v6, but in ranks below 95%-ile, it’s rare to see a 6v6 engagement after one person on one team dies)
    • Players will clearly have no idea what character abilities do, including for the character they’re playing
    • Players go for very high risk but low reward plays (for Overwatch players, a classic example of this is Rein going for a meme pin when the game opens on 2CP defense, very common at 50%-ile, rare at 95%-ile since players who think this move is a good idea tend to have generally poor decision making).
    • People will have terrible aim and will miss four or five shots in a row when all they need to do is hit someone once to kill them
    • If a single flanking enemy threatens a healer who can’t escape plus a non-healer with an escape ability, the non-healer will probably use their ability to run away, leaving the healer to die, even though they could easily kill the flanker and save their healer if they just attacked while being healed.

    Having just one aspect of your gameplay be merely bad instead of atrocious is enough to get to 50%-ile…Another basic situation that the vast majority of 90%-ile to 95%-ile players will get wrong is when you’re on offense, waiting for your team to respawn so you can attack as a group. Even at 90%-ile, maybe 1⁄4 to 1⁄3 of players won’t do this and will just run directly at the enemy team…For anyone who isn’t well into the 99%-ile, reviewing recorded games will reveal game-losing mistakes all the time. For myself, usually ranked 90%-ile or so, watching a recorded game will reveal tens of game losing mistakes in a close game (which is maybe 30% of losses, the other 70% are blowouts where there isn’t a single simple mistake that decides the game).

    It’s generally not too hard to fix these since the mistakes are like the example above: simple enough that once you see that you’re making the mistake, the fix is straightforward because the mistake is straightforward…if you look at the median time played at 50%-ile, people who are stably ranked there have put in hundreds of hours (and the median time played at higher ranks is higher). Given how simple the mistakes we’re discussing are, not having put in enough time cannot be the case for most players. A common complaint among low-ranked Overwatch players in Overwatch forums is that they’re just not talented and can never get better. Most people probably don’t have the talent to play in a professional league regardless of their practice regimen, but when you can get to 95%-ile by fixing mistakes like “not realizing that you should stand on the objective”, you don’t really need a lot of talent to get to 95%-ile.

    …One thing that’s curious about this is that Overwatch makes it easy to spot basic mistakes (compared to most other activities). After you’re killed, the game shows you how you died from the viewpoint of the player who killed you, allowing you to see what led to your death. Overwatch also records the entire game and lets you watch a replay of the game, allowing you to figure out what happened and why the game was won or lost. In many other games, you’d have to set up recording software to be able to view a replay. If you read Overwatch forums, you’ll see a regular stream of posts that are basically “I’m SOOOOOO FRUSTRATED! I’ve played this game for 1200 hours and I’m still ranked 10%-ile, [some Overwatch specific stuff that will vary from player to player]”. Another user will inevitably respond with something like “we can’t tell what’s wrong from your text, please post a video of your gameplay”. In the cases where the original poster responds with a recording of their play, people will post helpful feedback that will immediately make the player much better if they take it seriously. If you follow these people who ask for help, you’ll often see them ask for feedback at a much higher rank (eg., moving from 10%-ile to 40%-ile) shortly afterwards. It’s nice to see that the advice works, but it’s unfortunate that so many players don’t realize that watching their own recordings or posting recordings for feedback could have saved 1198 hours of frustration.

    It appears to be common for Overwatch players (well into 95%-ile and above) to:

    • Want to improve
    • Not get feedback
    • Improve slowly when getting feedback would make improving quickly easy

    Overwatch provides the tools to make it relatively easy to get feedback, but people who very strongly express a desire to improve don’t avail themselves of these tools.

  12. ⁠, Gheed (2011-10-03):

    [First in a famous series of posts about trolling StarCraft 2 players (2⁠/​​​​3⁠/​​​​4⁠/​​​​5⁠/​​​​6⁠/​​​​7⁠/​​​​8⁠/​​​​9⁠/​​​​10⁠/​​​​11). In a sudden fit of perversity, the author, Gheed, decides to test players in the lowest-ranked group of players (‘bronze’).

    His test is to, at the start of every game, use the ‘worker rush’ strategy: immediately send all of your initial units (workers) over to the enemy to attack them. A worker rush is one of the best-known & easiest strategies in all of SC to defeat: all the opponent has to do is literally a single keyboard command (‘A’ then mouse click, to instruct their workers to counter-attack), which is taught in the tutorial, and victory is guaranteed.

    His (already limited) faith in humanity is undermined as he is appalled to discover that this works a considerable fraction of the time, that losing players employ spastic actions and inexplicable ‘strategies’ which do not and could never work, that players get really angry when they lose to it despite it being their fault because worker rushes are trivially defeated, that it works on the same players multiple times, that it works on players who Gheed informs how to defeat it and also players who he had tutored to some degree, on players that claim to spend many hours watching e-sports SC matches and reading SC forums (and know the relevant jargon), and it even works on players who the online Battle.net statistics show have played thousands of games or come from higher leagues.]

  13. https://tl.net/blogs/271998-worker-rush-part-2-bm-rising?view=all

  14. https://tl.net/blogs/272765-worker-rush-nuts-and-bolts?view=all

  15. https://tl.net/blogs/281817-worker-rush-part-3-a-new-approach?view=all

  16. https://tl.net/blogs/283221-worker-rush-part-4-rising-up?view=all

  17. https://tl.net/blogs/286351-worker-rush-part-5-live-to-win?view=all

  18. https://tl.net/blogs/304674-worker-rush-part-6-at-a-loss?view=all

  19. https://tl.net/blogs/308882-bronze-delving-deeper?view=all

  20. https://tl.net/blogs/313577-bronze-part-2-hell-is-other-people?view=all

  21. https://tl.net/blogs/319375-bronze-part-3-casually-cruel?view=all

  22. https://tl.net/blogs/328804-bronze-part-4-a-legendary-league?view=all

  23. ⁠, Tyler Cowen (2021-02-22):

    …I thought I would add a few remarks on my very first job as chess teacher, which I did at ages 14–15.

    1. Chess teaching isn’t mainly about chess. A chess teacher has to have a certain mystique above all, while at the same time being approachable. Even at 14 this is possible. Your students are hiring you at least as much for your mystique as for the content of your lessons.
    2. Not everyone…wanted to be a better chess player. For some, taking the lesson was a substitute for hard work on chess, not a complement to it. The lesson for them was a fun social experience, and it kept the game of chess salient in their minds. They became “the kind of person who takes chess lessons.” I understood this well at the time. Some of the students wanted to show you their chess games, so that someone else would be sharing in their triumphs and tragedies. That is an OK enough way to proceed with a chess lesson, but often the students were more interested in “showing” than in listening and learning and hearing the hard truths about their play.
    3. Students are too interested in asking your opinion of particular openings. At lower-tier amateur levels of chess, the opening just doesn’t matter that much, provided you don’t get into an untenable position too quickly. Nonetheless openings are a fun thing to learn about, and discussing openings can give people the illusion of learning something important…
    4. What I really had to teach was methods for hard work to improve your game consistently over time. That might include for instance annotating a game or position “blind”, and then comparing your work to the published analysis of a world-class player, a la Think Like a Grandmaster. [The book is not concerned with advising where pieces should be placed on the board, or tactical motifs, but rather with the method of thinking that should be employed during a game. Kotov’s advice to identify and methodically examine them to build up an remains well known today.] I did try to teach that, but the demand for this service was not always so high.
    5. The younger chess prodigy I taught was quite bright and also likable. But he had no real interest in improving his chess game. Instead, hanging out with me was more fun for him than either doing homework or watching TV, and I suspect his parents understood that. In any case, early on I was thinking keenly about talent and the determinants of ultimate success, and obsessiveness seemed quite important. All of the really good chess players had it, and without it you couldn’t get far above expert level.
  24. https://marginalrevolution.com/marginalrevolution/2019/07/how-i-practice-at-what-i-do.html

  25. ⁠, Ashton Anderson, Jon Kleinberg, Sendhil Mullainathan (2016-06-15):

    An increasing number of domains are providing us with detailed trace data on human decisions in settings where we can evaluate the quality of these decisions via an algorithm. Motivated by this development, an emerging line of work has begun to consider whether we can characterize and predict the kinds of decisions where people are likely to make errors.

    To investigate what a general framework for human error prediction might look like, we focus on a model system with a rich history in the behavioral sciences: the decisions made by chess players as they select moves in a game. We carry out our analysis at a large scale, employing datasets with several million recorded games, and using chess tablebases to acquire a form of ground truth for a subset of chess positions that have been completely solved by computers but remain challenging even for the best players in the world.

    We organize our analysis around three categories of features that we argue are present in most settings where the analysis of human error is applicable: the skill of the decision-maker, the time available to make the decision, and the inherent difficulty of the decision. We identify rich structure in all three of these categories of features, and find strong evidence that in our domain, features describing the inherent difficulty of an instance are statistically-significantly more powerful than features based on skill or time.

  26. 2017-silver.pdf#page=3&org=deepmind: “Mastering the game of Go without human knowledge”⁠, David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, Demis Hassabis

  27. {#linkBibliography-yorker)-2014 .docMetadata}, James Surowiecki () (2014-11-03):

    [Discussion of the creation of modern sports training: professional athletes, even NBA stars, typically did not ‘train’. Practice was about getting into shape and working with teammates, if even that much—one simply took one’s skills for granted. Coaches focused on strategy, not coaching.

    A harbinger of the professionalization of professional athletes was basketball player Kermit Washington, on the verge of washing out of the NBA early on until he swallowed his pride and began tutoring with coach Pete Newell, who drilled Kermit on the basics repeatedly. Kermit eventually became an All-Star player and influenced other NBA players to engage in coaching and deliberate practice to improve their fundamentals. The modern paradigm is a ruthless quest for perfection in every dimension, quantified, and applying the latest science and technology to eek out even the slightest fraction of a second improvement; athletes are projects, with many different specialists examining them constantly for potential improvements, and as importantly, when not to practice lest they be injured.

    And the results speak for themselves—performance has never been higher, the impossible is now done routinely by many professionals, this continuous improvement trend has spread to other domains too, including chess, classical music, business. Equally striking are domains which don’t see trends like this, particular American education.]

    “You need to have the best PhDs onboard as well”, McClusky says. This technological and analytical arms race is producing the best athletes in history.

    The arms race centers on an obsessive scrutiny of every aspect of training and performance. Trainers today emphasize sports-specific training over generalized conditioning: if you’re a baseball player, you work on rotational power; if you’re a sprinter, on straight-line explosive power. All sorts of tools have been developed to improve vision, reaction time, and the like. The Dynavision D2 machine is a large board filled with flashing lights, which ballplayers have to slap while reading letters and math equations that the board displays. Football players use Nike’s Vapor Strobe goggles, which periodically cloud for tenth-of-a-second intervals, in order to train their eyes to focus even in the middle of chaos. Training is also increasingly personalized. Players are working not just with their own individual conditioning coaches but also with their own individual skills coaches. In non-team sports, such as tennis and golf, coaches were rare until the seventies. Today, tennis players such as Novak Djokovic have not just a single coach but an entire entourage. In team sports, meanwhile, there’s been a proliferation of gurus. George Whitfield has built a career as a “quarterback whisperer”, turning college quarterbacks into NFL-ready prospects. Ron Wolforth, a pitching coach, is known for resurrecting pitchers’ careers—he recently transformed the Oakland A’s Scott Kazmir from a has-been into an All-Star by revamping his mechanics and motion. Then there’s the increasing use of biometric sensors, equipped with heart-rate monitors, G.P.S., and gyroscopes, to measure not just performance (how fast a player is accelerating or cutting) but also fatigue levels. And since many studies show that getting more sleep leads to better performance, teams are now worrying about that, too. The N.B.A.’s Dallas Mavericks have equipped players with Readiband monitors to measure how much, and how well, they’re sleeping.

    All this effort may sound a bit nuts. But it’s how you end up with someone like Chris Hoy, the British cyclist who won two gold medals at the London Olympics in 2012, trailed by a team of scientists, nutritionists, and engineers. Hoy ate a carefully designed diet of five thousand calories a day. His daily workouts—two hours of lifting in the morning, three hours in the velodrome in the afternoon, and an easy one-hour recovery ride in the evening—had been crafted to maximize both his explosive power and his endurance. He had practiced in wind tunnels at the University of Southampton. He had worn biofeedback sensors that delivered exact data to his trainers about how his body was responding to practice. The eighty-thousand-dollar carbon-fibre bike he rode helped, too. Hoy was the ultimate product of an elaborate and finely tuned system designed to create the best cyclist possible. And—since his competitors weren’t slacking, either—he still won by only a fraction of a second.

  28. {#linkBibliography-yorker)-2011 .docMetadata}, Atul Gawande (New Yorker) (2011-10-26):

    [Meditation by doctor interested in medical improvement/​​​​progress (elsewhere, checklists). In tennis, he had improved his performance enormously after just minutes of coaching from a young man who pointed out his errors. Coaches are used in many areas and often spot problems that highly-competent trained professionals continue to make. A good coach is emotionally supportive, careful, speaks with credibility so they are not reflexively dismissed, brings an independent eye to highlight blind spots, and always finds a way they can push themselves to improve and deliberately practice.

    Gawande, having noticed his surgery success rates plateaued, considers a ‘medical coach’. Doctors are intensively taught up until they become full-fledged doctors, at which point they are cut loose to act as little gods in their domains, with no supervision. Yet, they are almost surely not perfect, and their skills may degrade over time. In domains far less important, like entertainment (arts/​​​​athletics), no individual believes they are perfect and they use personal coaches to constantly critique themselves, spot errors that untrained eyes would not, and strive for improvement. Why don’t we do the same thing in important things like surgeries? Why not coaches for doctors? Does the mystique of doctors intimidate themselves (and patients) away from acknowledging error and fallibility and improving?

    Gawande talks a former medical professor into coaching him. Gawande, while proud of his surgical technique, is surprised how many flaws his coach notes, and embarrassed; he had become used to working on his own, with no accountability or external critique. Other doctors made fun of the idea of coaching (coaching for thee, not for me). But he worked on his errors, and feels positive about his improvements and the possibility of breaking out of his plateau.]

  29. https://web.archive.org/web/20140725112211/http://www.wired.com:80/2012/06/ff_superhumans/

  30. https://medium.com/conversations-with-tyler/nicholas-bloom-tyler-cowen-productivity-economics-b5714b05fc2b

  31. https://marginalrevolution.com/marginalrevolution/2018/04/lessons-from-the-profit.html

  32. 2012-bloom.pdf: ⁠, Nicholas Bloom, Benn Eifert, Aprajit Mahajan, David McKenzie, John Roberts (2012-11-18; economics):

    A long-standing question is whether differences in management practices across firms can explain differences in productivity, especially in developing countries where these spreads appear particularly large. To investigate this, we ran a management field experiment on large Indian textile firms. We provided free consulting on management practices to randomly chosen treatment plants and compared their performance to a set of control plants. We find that adopting these management practices raised productivity by 17% in the first year through improved quality and efficiency and reduced inventory, and within three years led to the opening of more production plants. Why had the firms not adopted these profitable practices previously? Our results suggest that informational barriers were the primary factor explaining this lack of adoption. Also, because reallocation across firms appeared to be constrained by limits on managerial time, competition had not forced badly managed firms to exit.

  33. 2017-bloom-2.pdf: ⁠, Nicholas Bloom, Raffaella Sadun, John Van Reenen (2017-10-08; economics):

    Are some management practices akin to a technology that can explain firm and national productivity, or do they simply reflect contingent management styles?

    We collect data on core management practices from over 11,000 firms in 34 countries.

    We find large cross-country differences in the adoption of management practices, with the US having the highest size-weighted average management score.

    We present a formal model of “Management as a Technology”, and structurally estimate it using panel data to recover parameters including the depreciation rate and adjustment costs of managerial capital (both found to be larger than for tangible non-managerial capital). Our model also predicts (1) a positive impact of management on firm performance; (2) a positive relationship between product market competition and average management quality (part of which stems from the larger covariance between management with firm size as competition strengthens); and (3) a rise in the level and a fall in the dispersion of management with firm age.

    We find strong empirical support for all of these predictions in our data.

    Finally, building on our model, we find that differences in management practices account for about 30% of total factor productivity differences both between countries and within countries across firms.

  34. 2018-bruhn.pdf: ⁠, Miriam Bruhn, Dean Karlan, Antoinette Schoar (2018-03-07; economics):

    A with 432 small and medium enterprises in Mexico shows positive impact of access to 1 year of management consulting services on total factor productivity and return on assets. Owners also had an increase in “entrepreneurial spirit” (an index that measures entrepreneurial confidence and goal setting). Using Mexican social security data, we find a persistent large increase (about 50%) in the number of employees and total wage bill even 5 years after the program. We document large heterogeneity in the specific managerial practices that improved as a result of the consulting, with the most prominent being marketing, financial accounting, and long-term business planning.

  35. 2012-grinblatt.pdf: ⁠, Mark Grinblatt, Matti Keloharju, Juhani T. Linnainmaa (2012-05; iq):

    We analyze whether IQ influences trading behavior, performance, and transaction costs. The analysis combines equity return, trade, and limit order book data with two decades of scores from an intelligence (IQ) test administered to nearly every Finnish male of draft age. Controlling for a variety of factors, we find that high-IQ investors are less subject to the disposition effect, more aggressive about tax-loss trading, and more likely to supply liquidity when stocks experience a one-month high. High-IQ investors also exhibit superior market timing, stock-picking skill, and trade execution.

    Figure 1: Cumulative distribution of the cross-section of investors’ annualized portfolio returns. This figure plots the cumulative distribution (CDF) of the cross-section of investors’ annualized returns for subgroups of investors sorted by IQ (stanines 1–4 or stanine 9). The sample excludes investors who held stocks for fewer than 252 trading days in the sample period. Returns for each investor are annualized from the average daily portfolio returns computed over days the investor held stocks. The daily portfolio return is the portfolio-weighted average of the portfolio’s daily stock returns. The latter are close-to-close returns unless a trade takes place in the stock, in which case execution prices replace closing prices in the calculation. The returns are adjusted for dividends, stock splits, and mergers. IQ data [n = 87,914] are from 1/​​​​1982 to 12/​​​​2001. Remaining data are from 1/​​​​1995–11/​​​​2002.
  36. 2019-huising.pdf: ⁠, Ruthanne Huising (2019-06-26; economics):

    This paper examines how employees become simultaneously empowered and alienated by detailed, holistic knowledge of the actual operations of their organization, drawing on an inductive analysis of the experiences of employees working on organizational change teams. As employees build and scrutinize process maps of their organization, they develop a new comprehension of the structure and operation of their organization. What they had perceived as purposively designed, relatively stable, and largely external is revealed to be continuously produced through social interaction. I trace how this altered comprehension of the organization’s functioning and logic changes employees’ orientation to and place within the organization. Their central roles are revealed as less efficacious than imagined and, in fact, as reproducing the organization’s inefficiencies. Alienated from their central operational roles, they voluntarily move to peripheral change roles from which they feel empowered to pursue organization-wide change. The paper offers two contributions. First, it identifies a new means through which central actors may become disembedded, that is, detailed comprehensive knowledge of the logic and operations of the surrounding social system. Second, the paper problematizes established insights about the relationship between social position and challenges to the status quo. Rather than a peripheral social location creating a desire to challenge the status quo, a desire to challenge the status quo may encourage central actors to choose a peripheral social location.

    …Some held out hope that one or two people at the top knew of these design and operation issues; however, they were often disabused of this optimism. For example, a manager walked the CEO through the map, presenting him with a view he had never seen before and illustrating for him the lack of design and the disconnect between strategy and operations. The CEO, after being walked through the map, sat down, put his head on the table, and said, “This is even more fucked up than I imagined.” The CEO revealed that not only was the operation of his organization out of his control but that his grasp on it was imaginary.

    [See HBR popularization: “Can You Know Too Much About Your Organization?”⁠, Huising 2019.:

    But as the projects ended and the teams disbanded, a puzzle emerged. Some team members returned, as intended by senior management, to their roles and careers in the organization. Some, however, chose to leave these careers entirely, abandoning what had been to that point successful and satisfying work to take on organizational change roles elsewhere. Many took new jobs with responsibility for organizational development, Six Sigma, total quality management (TQM), business process re-engineering (BPR), or lean projects. Others assumed temporary contract roles to manage BPR project teams within their own or other organizations.

    …Despite being experienced managers, what they learned was eye-opening. One explained that “it was like the sun rose for the first time….I saw the bigger picture.” They had never seen the pieces—the jobs, technologies, tools, and routines—connected in one place, and they realized that their prior view was narrow and fractured. A team member acknowledged, “I only thought of things in the context of my span of control.”…The maps of the organization generated by the project teams also showed that their organizations often lacked a purposeful, integrated design that was centrally monitored and managed. There may originally have been such a design, but as the organization grew, adapted to changing markets, brought on new leadership, added or subtracted divisions, and so on, this animating vision was lost. The original design had been eroded, patched, and overgrown with alternative plans. A manager explained, “Everything I see around here was developed because of specific issues that popped up, and it was all done ad hoc and added onto each other. It certainly wasn’t engineered.” Another manager described how local, off-the-cuff action had contributed to the problems observed at the organizational level:

    “They see problems, and the general approach, the human approach, is to try and fix them….Functions have tried to put band-aids on every issue that comes up. It sounds good, but when they are layered one on top of the other they start to choke the organization. But they don’t see that because they are only seeing their own thing.”

    Finally, analyzing a particular work process, another manager explained that she had been “assuming that somebody did this [the process] on purpose. And it wasn’t done on purpose. It was just a series of random events that somehow came together.”]

  37. https://www.lifetime-reliability.com/tutorials/reliability-engineering-tutorials/human_error_rate_table_insights/

  38. ⁠, Jeff Atwood (2007-02-26):

    I was incredulous when I read this observation from Reginald Braithwaite:

    Like me, the author Imran Ghory is having trouble with the fact that 199 out of 200 applicants for every programming job can’t write code at all. I repeat: they can’t write any code whatsoever.

    Dan Kegel had a similar experience hiring entry-level programmers.

    Maybe it’s foolish to begin interviewing a programmer without looking at their code first. At Vertigo, we require a code sample before we even proceed to the phone interview stage…It’s a shame you have to do so much pre-screening to have the luxury of interviewing programmers who can actually program. It’d be funny if it wasn’t so damn depressing.

  39. https://www.nngroup.com/articles/computer-skill-levels/

  40. Socks

  41. Local-optima

  42. Small-groups