created: 17 March 2018; modified: 22 Mar 2018; status: in progress; confidence: highly likely; importance: 5
Joel Spolsky in 2002 identified a major pattern in technology business & economics: the pattern ofcommoditizing your complement, an alternative to vertical integration, where companies seek to secure a chokepoint or quasi-monopoly in products composed of many necessary & sufficient layers by dominating one layer while fostering so much competition in another layer above or below its layer that no competing monopolist can emerge, prices are driven down to marginal costs elsewhere in the stack, total price drops & increases demand, and the majority of the consumer surplus of the final product can be diverted to the quasi-monopolist. A classic example is the commodification of PC hardware by the Microsoft OS monopoly, to the detriment of IBM & benefit of MS. This pattern explains many otherwise odd or apparently self-sabotaging ventures by large tech companies into apparently irrelevant fields, such as the high rate of releasing open-source contributions by many Internet companies or the intrusion of advertising companies into smartphone manufacturing & web browser development & statistical software & fiber-optic networks & municipal WiFi & radio spectrum auctions (Google): they are pre-emptive attempts to commodify another company elsewhere in the stack, or defenses against it being done to them.
Smart companies try to commoditize their products’ complements
Former MS PM Joel Spolsky’s 20021
Strategy Letter V: The Economics of Open Source (./; HN) discusses a pattern he sees in technology companies and software in particular (emphasis in original, most links added):
Every product in the marketplace has substitutes and complements. A substitute is another product you might buy if the first product is too expensive. Chicken is a substitute for beef. If you’re a chicken farmer and the price of beef goes up, the people will want more chicken, and you will sell more.
A complement is a product that you usually buy together with another product. Gas and cars are complements. Computer hardware is a classic complement of computer operating systems. And babysitters are a complement of dinner at fine restaurants. In a small town, when the local five star restaurant has a two-for-one Valentine’s day special, the local babysitters double their rates. (Actually, the nine-year-olds get roped into early service.)
All else being equal, demand for a product increases when the prices of its complements decrease.
…In general, a company’s strategic interest is going to be to get the price of their complements as low as possible. The lowest theoretically sustainable price would be thecommodity price- the price that arises when you have a bunch of competitors offering indistinguishable goods. So:
Smart companies try to commoditize their products’ complements.
If you can do this, demand for your product will increase and you will be able to charge more and make more.
When IBM designed the PC architecture, they used off-the-shelf parts instead of custom parts, and they carefully documented the interfaces between the parts in the (revolutionary) IBM-PC Technical Reference Manual. Why? So that other manufacturers could join the party. As long as you match the interface, you can be used in PCs. IBM’s goal was to commoditize the add-in market, which is a complement of the PC market, and they did this quite successfully. Within a short time scrillions of companies sprung up offering memory cards, hard drives, graphics cards, printers, etc. Cheap add-ins meant more demand for PCs.
When IBM licensed the operating system PC-DOS from Microsoft, Microsoft was very careful not to sell an exclusive license. This made it possible for Microsoft to license the same thing to Compaq and the other hundreds of OEMs who had legally cloned the IBM PC using IBM’s own documentation. Microsoft’s goal was to commoditize the PC market. Very soon the PC itself was basically a commodity, with ever decreasing prices, consistently increasing power, and fierce margins that make it extremely hard to make a profit. The low prices, of course, increase demand. Increased demand for PCs meant increased demand for their complement, MS-DOS. All else being equal, the greater the demand for a product, the more money it makes for you. And that’s why Bill Gates can buy Sweden and you can’t.
Spolsky provides 8 examples (IBM commoditizing the add-on manufacturers, MS commoditizing IBM+PC manufacturers, IBM/Transmeta/Sun/HP funding FLOSS/Linux, Netscape open-sourcing Navigator, Sun developing Java & the JVM):
Understanding this strategy actually goes a long, long way in explaining why many commercial companies are making big contributions to open source. Let’s go over these.
- Headline: IBM Spends Millions to Develop Open Source Software.
- Myth: They’re doing this because Lou Gerstner read the GNU Manifesto and decided he doesn’t actually like capitalism.
Reality: They’re doing this because IBM is becoming an IT consulting company. IT consulting is a complement of enterprise software. Thus IBM needs to commoditize enterprise software, and the best way to do this is by supporting open source. Lo and behold, their consulting division is winning big with this strategy. …
- Headline: Netscape Open Sources Their Web Browser.
- Myth: They’re doing this to get free source code contributions from people in cybercafes in New Zealand.
Reality: They’re doing this to commoditize the web browser. This has been Netscape’s strategy from day one. Have a look at the very first Netscape press release: the browser isfreeware. Netscape gave away the browser so they could make money on servers. Browsers and servers are classic complements. The cheaper the browsers, the more servers you sell. This was never as true as it was in October 1994. …
- Headline: Transmeta Hires Linus, Pays Him To Hack on Linux.
- Myth: They just did it to get publicity. Would you have heard of Transmeta otherwise?
Reality: Transmeta is a CPU company. The natural complement of a CPU is an operating system. Transmeta wants OSs to be a commodity.
- Headline: Sun and HP Pay Ximian To Hack on Gnome.
- Myth: Sun and HP are supporting free software because they like Bazaars, not Cathedrals.
Reality: Sun and HP are hardware companies. They make boxen. In order to make money on the desktop, they need for windowing systems, which are a complement of desktop computers, to be a commodity. Why don’t they take the money they’re paying Ximian and use it to develop a proprietary windowing system? They tried this (Sun had NeWS and HP had New Wave), but these are really hardware companies at heart with pretty crude software skills, and they need windowing systems to be a cheap commodity, not a proprietary advantage which they have to pay for. So they hired the nice guys at Ximian to do this for the same reason that Sun bought Star Office and open sourced it: to commoditize software and make more money on hardware.
The bytecode idea is not new - programmers have always tried to make their code run on as many machines as possible. (That’s how you commoditize your complement). For years Microsoft had its own p-code compiler and portable windowing layer which let Excel run on Mac, Windows, and OS/2, and on Motorola, Intel, Alpha, MIPS and PowerPC chips. Quark [QuarkXPress?] has a layer which runs Macintosh code on Windows. The C programming language is best described as a hardware-independent assembler language. It’s not a new idea to software developers.
If you can run your software anywhere, that makes hardware more of a commodity. As hardware prices go down, the market expands, driving more demand for software (and leaving customers with extra money to spend on software which can now be more expensive.)
Sun’s enthusiasm for WORA is, um, strange, because Sun is a hardware company. Making hardware a commodity is the last thing they want to do. Oooooooooooooooooooooops! Sun is the loose cannon of the computer industry. Unable to see past their raging fear and loathing of Microsoft, they adopt strategies based on anger rather than self-interest. Sun’s two strategies are (a) make software a commodity by promoting and developing free software (Star Office, Linux, Apache, Gnome, etc), and (b) make hardware a commodity by promoting Java, with its bytecode architecture and WORA. OK, Sun, pop quiz: when the music stops, where are you going to sit down? Without proprietary advantages in hardware or software, you’re going to have to take the commodity price, which barely covers the cost of cheap factories in Guadalajara, not your cushy offices in Silicon Valley. [The remains of Sun Microsystems were purchased by Oracle in 2009.]
A way I would express it as: Any product is the joint outcome of a large number of individual components, each of which layers is necessary but not sufficient to the final valuable use of the entire stack put together; a smartphone is not much good without a power-efficient sensitive radio, but the radio is not much good without a good OS on top of it, and a good OS is not much good either without great apps like web browsers (and is a web browser all that useful if there aren’t useful websites to use in it, and where are the languages & compilers for all this coming from anyway…?).
The end product of a Symbian, iOS, or Android smartphone is without a doubt fantastically valuable to the user, but what is the fair division of the revenue among the countless people, technologies, manufacturers who created each of the many critically-important layers in the full tech stack? Certainly contemporary intellectual property law (eg
software patents) does not provide a socially-efficient distribution like the Shapley value to all the participants! There are constraints in that the final product cannot cost more than the consumer surplus (otherwise consumers simply wouldn’t buy it) and the companies in the commoditized layers can’t be forced down to below marginal costs (otherwise they would go bankrupt & exit the market), but these are weak, and do not give any good hints as to who will capture the majority of the value: the chip fab manufacturers? the chip designers? the device manufacturers? the OS developers? the userland application developers? the ISPs? the website owners?
Vertical integration can be an effective way of resolving the intractable market dispute with top-down dictatorships, but can require lax anti-monopoly regulations, high capital investment, massive corporation overextension & empire-building, and risks being outcompeted at every level by nimbler competitors; this makes it difficult for any up-and-coming company to implement, and often ineffective. Commoditizing the complements, in contrast, permits a company to remain (relatively) small & lean, can often be accomplished with small strategic investments in releasing intellectual property or other investments, can be done incrementally focusing on specific layers without the
Big Bang orientation of vertical integration and permitting
defeat in detail, retains the general facade of competition, and ensures the extreme competition remains confined to other layers of the stack where the complement can benefit from the cost reductions but is not at any risk.
In practice, the division winds up being due to power plays and market dynamics, and who can most effectively erect a moat while sabotaging competitors, exploiting tactics like lawsuits & software patent trolling, proprietary APIs, cross-business subsidies, kickbacks, DRM, deliberate incompatibility or
embrace and extend, FUD, operating at a loss indefinitely, etc. (
There’s An App For That is why you buy an iPhone - but it’s Apple with the $930b market cap & not any of the app developers.) Done correctly, this is quite effective at perpetuating incumbents’ long-term control of markets & justifies their enormous valuations - by definition, the competitors elsewhere in the stack, who might develop a chokepoint, are too numerous, fragmented, and low-margin to invest substantially into threatening R&D2 or long-term strategic initiatives, and any upstart startups can be relatively easily bought out or suppressed.
- IBM’s 1956 consent decree settling a 1952 antitrust lawsuit: IBM was required to sell as well as lease its devices; more importantly, its business services arm was spun off and IBM had to license software/patents/manuals/training to competitors. Previously, IBM (like many other hardware manufacturers) included all necessary software with its hardware for
free, particularly for the OS/360, strangling any independent software market; the decree and the eventual
unbundlingis credited with sparking a vibrant (and highly profitable) market for IBM mainframe software. (It is also credited with putting the fear of God into IBM & protecting smaller companies like Microsoft.)
- credit card companies/businesses: interchange fees in particular
- Lisp machines vs x86/SPARC/Sun
- Netscape vs Windows (in Metcalfe’s infamous expression, cross-platform web browsers & the Internet would reduce Windows to a
poorly debugged set of device drivers)
FLOSS: Red Hat, Google etc
- XEmacs represented an early example of a company trying to improve a FLOSS version of a genre of software previously typically sold commercially (text editors) in order to support sales of more niche tooling (a C++ IDE)
- GNU and compiler/interpreter companies: GCC; commercial C++ implementations would themselves be cannibalized by GCC, and IDEs by other FLOSS IDEs (apropos of IBM and Java, Eclipse)
- Valve: Steam vs game developers/studios
- ISPS vs tech companies via Net neutrality (eg Google Fiber, zero-rating)
Windows Mobile/Android/iOS vs smartphone manufacturers
- iOS/apps: to put it bluntly,
Apple Doesn’t Want Your iPhone App to Make Moneyand if your app does anyway, it’ll take a 30% cut.
- incompatibility of video call apps across platforms
- iOS/apps: to put it bluntly,
pausesin the famous graph of genome sequencing cost dropping exponentially over time); in response to the high prices, heavy users of genome sequencing have launched desperate attempts to escape the Illumina monopoly, such as BGI’s ill-fated acquisition of Complete Genomics whose sequencers ultimately proved inadequate after sowing internal chaos & wrecking many projects; rumor has it that 23andMe launched its own very expensive internal attempt to develop replacement genome sequencers, and this was a major contributor to its financial woes after the FDA debacle.
Uber/Lyft vs drivers vs self-driving cars
Vocaloid vs singers: 00:14 <@gwern> MicaiahC: the issue is whether Vocaloid software is a substitute or complement to singers, and songwriters. this is like Hanson’s em economics paper - as software is stupid, it serves as a complement to humans, enabling them all to do better as the smartest software is still nowhere near the stupidest human. but as the software gets better, the
waterlinecreeps up and soon some humans are no longer complements, but are substituted away 00:16 < MicaiahC> Yeah, IA sounds almost as good as Lia now, granted IA still has some problems and can’t do certain types of songs 00:16 <@gwern> MicaiahC: so it could be that Vocaloid currently commoditizes songwriters by massively expanding the market of people who can demonstrate their songs work using Vocaloid but serves as a complement for singers, making their human touch more valuable. but as the Vocaloid software improves, eventually top end singers will drown 00:17 <@gwern> we can probably monitor the trend by watching how many Japanese sales go to music directly using Vocaloid, not just based on relatively popular Vocaloid songs or by people who got started in Vocaloid
- FB’s Open Compute Project: https://marco.org/2011/04/09/facebooks-open-compute-project ?
- DNMs providing Grams API when small then revoking it when large; Vendors want markets to be commodities that they can smoothly move their customers and listings between as they go up or down; markets want vendors to be trapped and locked into them.
- Valve: Vive vs Oculus (21:51 <@gwern> valve will be fine if vive doesn’t win as long as no one else wins too / no one makes money on computer games except… Steam) 21:53 <@gwern> for Oculus, the danger is in becoming just an expensive hardware peripheral, whose parts they don’t manufacture but merely assemble against a standard software interface 21:53 < ivan> they’re already being emulated 21:53 < ivan> just like people emulate all of amazon’s APIs 21:54 <@gwern> but gamers and devs are fairly alert to lockin and walled gardens, which makes it a tightrope for Oculus. they can’t push exclusives too hard or be as aggressive in fighting outsiders as they want 21:55 <@gwern> so it’s interesting. both vive and Oculus have incentives to cooperate… for now 21:56 <@gwern> but there’s also a pressure for
curse your unexpected but inevitable betrayal!moment 21:56 <@gwern> like MS releasing IE 21:56 * adiabatic passes the popcorn bucket around 21:57 <@gwern> in this respect, vive is a prescient move by valve to preserve its near-PC monopoly
Imgur vs Reddit 12:04 <@gwern> http://minimaxir.com/2017/06/imgur-decline/
commoditize your complement12:04 < feepbot> The Decline of Imgur on Reddit and the Rise of Reddit’s Native Image Hosting (Before Reddit added native image hosting, Imgur accounted for 15% of all submissions to Reddit. Now it’s below 9%.)
- Samsung Bixby/Google assistant & services
- Qualcomm/Apple with Intel as wedge https://www.nytimes.com/2018/03/15/business/with-one-battle-over-a-bigger-one-looms-for-qualcomm-apple.html
Nvidia/deep learning/ TensorFlow & TPUs
note the lack of contributions by cloud providers to libcloud: 12:54 <@gwern> sbaugh: why would the cloud providers want to help commoditize themselves like that? 12:54 <@niska> https://github.com/apache/libcloud 12:55 < sbaugh> it’s true it would commoditize them, but there are plenty of people who want to commoditize the providers; I’m not sure how they kept it from happening 12:55 <@niska> https://libcloud.apache.org/supported_providers.html 1
22:56 < pie_> someone explain to my why amazon doesn’t accept PayPal :/
23:13 <@gwern> Madplatypus: even spun off, amazon will have enmity for PayPal. it competes with
amazon payments or whatever it’s called , and payments & sellers are always enemies in trying to extract profits from the consumer; amazon wishes to vertically integrate, while PayPal wishes to choke out the life of any seller and increase its fees… 23:14 <@gwern> commoditize your complement or be commoditized, such is the iron law in silicon valley 23:15 < mgin> i was just wondering, surely amazon has their own version directly competing
It was republished in the 2004 anthology, Joel on Software: And on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those Who, Whether by Good Fortune or Ill Luck, Work with Them in Some Capacity, but I only spotted minor formatting changes like removing some hyperlinks, so the 2002 blog post is the canonical version.↩
Of course, this can pose a problem of its own: if the other layers of the stack have near-zero profits, how will they afford to develop new technology that the incumbent might desire? Apple, for example, apparently has to extend large loans and provide other life-support for hardware makers to get the new materials & manufacturing techniques it wants. This does not always end well, like in the case of the ultra-hard sapphire screens for the iPhone (TechCrunch; WSJ).↩