“Smart companies try to commoditize their products’ complements”

Spolsky provides 8 examples (IBM commoditizing the add-on manufacturers, MS commoditizing IBM+PC manufacturers, IBM/Transmeta/Sun/HP funding FLOSS/Linux, Netscape open-sourcing Navigator, Sun developing Java & the JVM):

Understanding this strategy actually goes a long, long way in explaining why many commercial companies are making big contributions to open source. Let’s go over these.

• Headline: IBM Spends Millions to Develop Open Source Software.

• Myth: They’re doing this because Lou Gerstner read the GNU Manifesto and decided he doesn’t actually like capitalism.

• Reality: They’re doing this because IBM is becoming an IT consulting company. IT consulting is a complement of enterprise software. Thus IBM needs to commoditize enterprise software, and the best way to do this is by supporting open source. Lo and behold, their consulting division is winning big with this strategy. …

• Headline: Netscape Open Sources Their Web Browser⁠.

• Myth: They’re doing this to get free source code contributions from people in cybercafes in New Zealand.

• Reality: They’re doing this to commoditize the web browser. This has been Netscape’s strategy from day one. Have a look at the very first Netscape press release: the browser is “freeware”. Netscape gave away the browser so they could make money on servers⁠. Browsers and servers are classic complements. The cheaper the browsers, the more servers you sell. This was never as true as it was in October 1994. …

• Headline: Transmeta Hires Linus⁠, Pays Him To Hack on Linux⁠.

• Myth: They just did it to get publicity. Would you have heard of Transmeta otherwise?

• Reality: Transmeta is a CPU company. The natural complement of a CPU is an operating system. Transmeta wants OSs to be a commodity.

• Headline: Sun and HP Pay Ximian To Hack on Gnome⁠.

• Myth: Sun and HP are supporting free software because they like Bazaars, not Cathedrals⁠.

• Reality: Sun and HP are hardware companies. They make boxen. In order to make money on the desktop, they need for windowing systems⁠, which are a complement of desktop computers, to be a commodity. Why don’t they take the money they’re paying Ximian and use it to develop a proprietary windowing system? They tried this (Sun had NeWS and HP had New Wave), but these are really hardware companies at heart with pretty crude software skills, and they need windowing systems to be a cheap commodity, not a proprietary advantage which they have to pay for. So they hired the nice guys at Ximian to do this for the same reason that Sun bought Star Office and open sourced it: to commoditize software and make more money on hardware.

• Headline: Sun Develops Java⁠; New “Bytecode” System Means Write Once, Run Anywhere [WORA]. [See also Microsoft’s .NET Framework⁠.]

  The bytecode idea is not new—programmers have always tried to make their code run on as many machines as possible. (That’s how you commoditize your complement). For years Microsoft had its own p-code compiler and portable windowing layer which let Excel run on Mac, Windows, and OS / 2⁠, and on Motorola, Intel⁠, Alpha, MIPS and PowerPC chips. Quark [QuarkXPress?] has a layer which runs Macintosh code on Windows. The C programming language is best described as a hardware-independent assembler language. It’s not a new idea to software developers.

  If you can run your software anywhere, that makes hardware more of a commodity. As hardware prices go down, the market expands, driving more demand for software (and leaving customers with extra money to spend on software which can now be more expensive.)

  Sun’s enthusiasm for WORA is, um, strange, because Sun is a hardware company. Making hardware a commodity is the last thing they want to do. Oooooooooooooooooooooops! Sun is the loose cannon of the computer industry. Unable to see past their raging fear and loathing of Microsoft, they adopt strategies based on anger rather than self-interest. Sun’s two strategies are (a) make software a commodity by promoting and developing free software (Star Office, Linux, Apache⁠, Gnome, etc), and (b) make hardware a commodity by promoting Java, with its bytecode architecture and WORA. OK, Sun, pop quiz: when the music stops, where are you going to sit down? Without proprietary advantages in hardware or software, you’re going to have to take the commodity price, which barely covers the cost of cheap factories in Guadalajara, not your cushy offices in Silicon Valley.³

“Open Source as a Strategic Weapon”

In Eric S. Raymond’s famous 1999 The Cathedral and the Bazaar, he includes a section, “Open Source as a Strategic Weapon” on commercial motivations for sponsorship of FLOSS along the lines of Spolsky’s examples, listing Apache, the X window system, and Netscape Mozilla as examples of strategic uses of FLOSS which look like “commoditize your complement” examples:

Cost-sharing as a competitive weapon

Earlier, we considered Apache as an example of better and cheaper infrastructure development through cost-sharing in an open-source project. For software and systems vendors competing against Microsoft and its IIS web server⁠, the Apache project is also a competitive weapon. It would be difficult, perhaps impossible, for any other single web server vendor to completely offset the advantages of Microsoft’s huge war chest and desktop-monopoly market power. But Apache enables each corporate participant in the project to offer a webserver that is both technically superior to IIS and reassures customers with a majority market share—at far lower cost. This improves the market position and cost of production for value-added electronic-commerce products (like IBM’s WebSphere).

…Resetting the competition

When the development of the open-source X Window System was funded by DEC in the 1980s, their explicit goal was to “reset the competition”. At the time there were several competing alternative graphics environments for Unix in play, notably including Sun Microsystems’s NeWS system. DEC strategists believed (probably correctly) that if Sun were able to establish a proprietary graphics standard it would get a lock on the booming Unix-workstation market. By funding X and lending it engineers, and by allying with many smaller vendors to establish X as a de-facto standard, DEC was able to neutralize advantages held by Sun and other competitors with more in-house expertise in graphics. This moved the focus of competition in the workstation market towards hardware, where DEC was historically strong.

Preventing a choke hold

In explaining the loss-leader/market-positioner business model above, I described how Netscape’s open-sourcing of the Mozilla browser was a (successful) maneuver aimed at preventing Microsoft from effectively locking up HTML markup and the HTTP protocol

Generalizing

A way I would express it as: Any product is the joint outcome of a large number of individual components, each of which layers is necessary but not sufficient to the final valuable use of the entire stack put together; a smartphone is not much good without a power-efficient sensitive radio, but the radio is not much good without a good OS on top of it, and a good OS is not much good either without great apps like web browsers (and is a web browser all that useful if there aren’t useful websites to use in it, and where are the languages & compilers for all this coming from anyway…?). Many products are formed by a stack of two-sided markets.

The end product of a Symbian⁠, iOS⁠, or Android smartphone is without a doubt fantastically valuable to the user, but what is the fair division of the revenue among the countless people, technologies, manufacturers who created each of the many critically-important layers in the full tech stack? Certainly contemporary intellectual property law (eg “software patents”) does not provide a socially-efficient distribution like the Shapley value to all the participants! There are constraints in that the final product cannot cost more than the value to the user (otherwise consumers simply wouldn’t buy it, and if the consumer surplus isn’t at least considerably above zero, no one would bother to learn about it) and the companies in the commoditized layers can’t be forced down to below marginal costs (otherwise they would go bankrupt & exit the market), but these are weak, and do not give any good hints as to who will capture the majority of the value: the chip fab manufacturers? the chip designers? the device manufacturers? the OS developers? the userland application developers? the ISPs? the website owners?

Vertical integration can be an effective way of resolving the intractable market dispute with top-down dictatorships, but can require lax anti-monopoly regulations, high capital investment, massive corporation overextension & empire-building, and risks being outcompeted at every level by nimbler competitors; this makes it difficult for any up-and-coming company to implement, and often ineffective. Commoditizing the complements, in contrast, permits a company to remain (relatively) small & lean, can often be accomplished with small strategic investments in releasing intellectual property or other investments, can be done incrementally focusing on specific layers without the “Big Bang” orientation of vertical integration and permitting “defeat in detail”, retains the general facade of competition, and ensures the extreme competition remains confined to other layers of the stack where the product can benefit from the cost reductions in the complement but is not itself at any risk.

In practice, the division winds up being due to power plays and market dynamics, and who can most effectively erect a moat while sabotaging competitors, exploiting tactics like lawsuits & software patent trolling, proprietary APIs, cross-business subsidies, kickbacks, DRM, deliberate incompatibility or “embrace and extend”, FUD, operating at a loss indefinitely, etc. (“There’s An App For That” is why you buy an iPhone—but it’s Apple with the $930 billion market cap & not the app developers.)

Done correctly, this is effective at perpetuating incumbents’ long-term control of markets & justifies their enormous valuations—by definition, the competitors elsewhere in the stack, who might develop a chokepoint, are too numerous, fragmented, and low-margin to invest substantially into threatening R&D⁴ or long-term strategic initiatives, and any upstart startups can be relatively easily bought out or suppressed (eg Instagram or WhatsApp). Nor does this require convoluted explanations like “they are pretending to not be monopolists” or fully general unfalsifiable claims like “it’s good PR” for why big companies like Google steadily fund so many apparently oddball projects like new foreign language fonts (or free TrueType fonts & TrueType itself) or open source TCP/IP protocol replacements, which are neither directly profitable nor well-known nor impressively charitable—but do have clear explanations in terms of business objectives like “driving more mobile web browsing” (thus allowing Google to show them more ads, because the complement, mobile web browsing, has become cheaper/easier). I wonder if this also explains some of the striking copycat behavior we see sometimes—as entities get worried something might be a commodifier, either because it is crucial but was formerly considered ‘neutral’ or because they assume the other entity knows something they don’t. (Google cared little about an also-ran code-hosting site like GitLab other than some VC investment—well, right up until Microsoft outbid it for Github & Github became free for individual developers & acquired NPM⁠, and then suddenly GitLab becomes a unicorn with even more VC from Google & others.) As ucaetano puts it:

Another way that I like to express that is “create a desert of profitability around you”. I once had a strategy professor define the Google business model somewhat like that, where “Google tries to make every other business around it free or irrelevant”…A desert of profitability shifts consumers to you, and keeps competitors away.

Examples

A list of examples I think reflect this dynamic to some extent:

• hardware vs software: IBM’s 1956 consent decree settling a 1952 antitrust lawsuit: IBM was required to sell as well as lease its devices; more importantly, its business services arm was spun off and IBM had to license software/patents/manuals/training to competitors.

  Previously, IBM (like many other hardware manufacturers) included all necessary software with its hardware for ‘free’, particularly for the OS / 360⁠, strangling any independent software market; the decree and the eventual “unbundling” is credited with sparking a vibrant (and highly profitable) market for IBM mainframe software. (It is also credited with putting the fear of God into IBM & protecting smaller companies like Microsoft; in an ironic repetition, Microsoft’s own antitrust travails in the 1990s are credited for causing it to back off its classic ruthless tactics and enable Google’s own rise to dominance⁵⁠.)

  IBM’s patent-free release of the PowerPC ISA in 2019 might be another example.

• banks vs merchants: credit card companies/small businesses: interchange fees in particular (part of why the credit card industry is one of the highest-profit margin ones after academic publishing—itself a highly suspicious example). Amazon vs PayPal is a big example.

• integrated vs open architectures: Lisp machines vs x86/SPARC/Sun

• apps vs OSes: Netscape vs Windows (in Robert Metcalfe’s infamous expression, cross-platform web browsers & the Internet would reduce Windows to a “poorly debugged set of device drivers”); MS’s initial response was to… license IE free for all users, not just noncommercial users like Netscape, killing a major revenue stream for Netscape early on⁶

• FLOSS: Red Hat⁠, Google etc

  • editors vs IDEs: XEmacs represented an early example of a company trying to improve a FLOSS version of a genre of software previously typically sold commercially (text editors) in order to support sales of more niche tooling (their C++ IDE)
  • IDEs vs software devs: GNU and compiler/interpreter companies: GCC; commercial C++ implementations would themselves be cannibalized by GCC, and IDEs by other FLOSS IDEs (apropos of IBM and Java, Eclipse)
  • programming language ecosystems vs users: in programming communities (especially functional languages like Haskell/OCaml/Scala/Clojure), it is common for a lot of work on compilers/libraries/tutorials/books to be sponsored by web development or consultancies, serving both as advertisements for their capabilities and also removing pain points to use of said programming languages and thus increasing demand for their services. (If nobody uses Haskell because GHC has a major bug, nobody is going to hire a Haskell consultancy like Well-Typed, either. In the sales funnel, you have to have customers entering the funnel to get any customers out.)
  • art tools vs artists: media design tools like Autodesk’s AutoCAD⁠/ Maya vs FLOSS alternatives like Blender⁠, the latter of which are supported by commercial users such as the animation studio Khara feeling the pain. Other examples include Netflix sponsoring development of 4k HDR anime (which would be highly-bandwidth-intensive to stream)

• TTRPG game publishers vs themselves’ “open gaming”:

  In 2000, Wizards of the Coast (owner of Dungeons & Dragons) executive Ryan Dancey masterminded the creation of the Open Game License⁠, modeled on the GPL⁠, and the release of the 3rd Edition’s core d20 System rules under the Open Game License; Dancey’s reasoning was explicitly based on the idea that by open-sourcing D&D, that would help it prevail over the constant stream of tabletop RPG competitors through network effects, and drive sales of the (expensive) core rulebook, the Player Handbook. The system was rapidly adopted by other companies (Lecocq & Demil 2006). This bet appears to have paid off: it is difficult to imagine the 2010s D&D renaissance, particularly driven by Kickstarter⁠, without the open-sourcing.

• game portals vs game devs: Valve: Steam vs game developers/studios such as Epic Games⁷

  Steam can be seen as a ‘console maker’, and exemplifying console makers vs game devs: the success of Epic Games’s Fortnite is attributed in part to its use of the cross-platform Unreal game engine, allowing it to run seamlessly on all major PC, mobile, and consoles (“Microsoft Windows, macOS, Nintendo Switch, PlayStation 4, Xbox One, iOS, Android”); when Sony tried to maintain the usual PlayStation 4 walled-garden by breaking interoperability with Fortnite players on other platforms, the backlash forced it to announce it would compromise, and Epic Games simply opted out of Android’s walled garden & Google’s cut of revenue, saving Epic Games more than $50 million annually.

  • apps vs OSes: Linux (“SteamOS”) vs Microsoft Windows (“Gabe I think saw it as a stick to beat Microsoft with—and he was absolutely correct, it worked.”)

  • games vs game hardware: VR: Valve’s Vive vs Facebook Oculus⁠.

    Valve will be fine if Vive doesn’t win the VR market as long as no one else wins too—because no one makes money on computer games except… Steam. While for Oculus, the danger is in becoming just an expensive hardware peripheral, whose parts they don’t manufacture but merely assemble against a standard software interface, where they are forced to compete against cut-rate Chinese manufacturing giants for near-zero profit margin, a rerun of the smartphone market (a similar market, as they even use the same screens). The danger of lockin to an Oculus/Facebook walled garden is clear to gamers, and has made life difficult for Oculus: they can’t push exclusives too hard or be as aggressive in fighting outsiders as they want, lest they spark a backlash or cripple the VR market as a whole. Which makes it interesting—both Vive and Oculus have incentives to cooperate… for now. But there’s constant pressure for a betrayal when a player gets desperate or decides the market has matured and it’s time to break for the finish line, like Microsoft releasing IE.

  • publishers vs esports teams: game publishers, due to DRM & forcing matches to go through publishers, have a death-grip over their platforms, and publishers like Riot Games or Blizzard Entertainment have been careful to maintain control over their esports lest the tail wag the dog, weakening teams by downplaying team merchandise & regularly adding in IP requirements & taking revenue cuts upfront

• telecoms vs users: ISPS vs tech companies via Net neutrality (eg spectrum auctions⁠, Google Fiber⁠, zero-rating)

  eg Jio Reliance, an Indian telecom ISP, disrupted the Indian mobile Internet market by cannibalizing existing margins, under the logic that after commoditizing data & voice, it’ll profit (China/Africa-style) off broadband & “content, financial services and advertising”⁸

• networking hardware manufacturers vs Internet companies: special-purpose networking hardware (especially Cisco Systems) vs Software-defined networking (as employed by Google⁠/ Microsoft⁠/Amazon/cloud giants)

  People are always surprised how much custom silicon goes into a datacenter server or a smartphone (it’s computers all the way down), but a company like Apple or Amazon cannot risk being held up by a would-be monopolist like Intel, thus the perennial interest in fabbing their own custom chips or dabbling in chips from Intel rivals ARM & AMD—cutting-edge CPUs, however, remain hard and expensive enough to design & make that no one has yet succeeded in commoditizing top-end x86 CPUs, and these moves remain largely negotiating ploys or improvements for niche use-cases

  • RISC-V vs x86⁠/ ARM: pretty much the only reason the tech industry funds RISC-V is to keep the incumbents on their toes
  • Machine Learning: Nvidia’s GPUs/ CUDA vs AMD⁠/ OpenCL vs Google TensorFlow⁠/ TPUs (exposed to end-users as Kaggle & Google Colab)

• smartphones (lots):

  • OS vs Manufacturers: Windows Mobile/Android/iOS vs smartphone manufacturers

  • OS vs apps: iOS/apps: to put it bluntly⁠, “Apple Doesn’t Want Your iPhone App to Make Money” and if your app does anyway, it’ll take a 30% cut. Note for example: the incompatibility of video call apps across platforms, or the fight between Samsung Bixby⁠/ Google Assistant & services

    • Google Maps vs OpenStreetMap (OSM) vs Mapillary: the dominance of Google Maps has led to dissatisfaction with its steep price increases and limits, Apple replaced Google Maps on iPhones with Apple Maps in 2012 despite serious quality problems (similar moves were made by FourSquare & Craigslist), and rivals like Yahoo!/ Microsoft / Facebook / DigitalGlobe / Telenav have supported OSM as a counterweight and many corporations actively contribute to OSM improvements (eg FB deep learning efforts). Facebook outright bought Mapillary in June 2020, apparently to… release their data for free commercial use⁠.

  • Designer vs Manufacturers: Qualcomm vs Apple with Intel as wedge

  • Designer vs Telecoms: iPhone vs telecom ISPs: demand for the iPhone was so enormous it was used as a wedge to force cellphone networks into allowing things they didn’t want—such as letting a large fraction of all smartphones on their network out of their control and extracting unusual concessions

• genome sequencer manufacturers vs doctors/researchers: Illumina vs 23andMe & BGI:

  The Illuminati have a notorious stranglehold on the genome sequencing market, with enough of an Intel-style lead in consumables efficiency that they can drop prices just enough to suppress competitors (this is may what is behind the occasional inexplicable “pauses” in the famous graph of genome sequencing cost dropping exponentially over time); in response to the high prices, heavy users of genome sequencing have launched desperate attempts to escape the Illumina monopoly, such as BGI’s ill-fated acquisition of Complete Genomics whose sequencers ultimately proved inadequate after sowing internal chaos & wrecking many projects; rumor has it that 23andMe launched its own very expensive internal attempt to develop replacement genome sequencers, and this was a major contributor to its financial woes after the FDA debacle⁠.

• ride-sharing service vs drivers: Uber⁠/ Lyft vs self-employed drivers vs self-driving cars

  • Car Software Devs vs Car Manufacturers: self-driving car developers like Google Waymo vs car manufacturers like Honda (the Honda-Waymo partnership fell apart in 2018 reportedly because Honda wanted access to the AI & software of Waymo⁠, and Waymo, for reasons that should be clear now, refused; Honda was forced to invest in rival GM’s Cruise). As the WSJ describes automakers’ thinking:

    The global auto industry thinks it sees the future, and it will require a transformation without precedent in business history: The giant industrial sector has to turn itself into a nimble provider of software and services…Auto executives say they need to avoid a nightmare tech scenario that’s become a common refrain at industry gatherings. They don’t want to become the next “handset makers”—commodity suppliers of hardware, helplessly watching all the profits flow to software makers like Apple Inc. and Alphabet Inc., the parent of Google. Both companies are investing in software for driverless cars.

• voice synthesizers vs singers: Crypton’s Vocaloid vs singers: the voice-synthesizer software sparked an amateur explosion of song production, as apparently even finicky software made high-quality singing far more accessible; Crypton sells the software and maintains control over specific characters like Hatsune Miku (sampled from minor voice actress Saki Fujita), particularly over all licensing and commercial revenue, which formerly would have to be shared with the human idol.

  Note the segmentation—software like Vocaloid synthesizers empower the musicians (who can now produce high-quality, competitive with humans given enough tweaking, vocals to go along with their music) at the expense of the more-specialized singers. (A singer can produce their own version of a Vocaloid hit, but are no longer a chokepoint.)

  • An emerging variant of this is the Virtual YouTuber genre.

• Internet services vs manufacturers: FB’s Open Compute Project providing free designs like Open Rack for building efficient datacenters, and acquisitions of Instagram⁠/ WhatsApp

  • FLOSS vs SaaS: Elastic vs Amazon/Netflix/ Expedia: Elasticsearch is text-search software for large-scale document searches; it is commercialized by Elastic, which follows a split-FLOSS model where the core Elasticsearch is FLOSS but features critical for large-scale commercial use (like ‘any security’) are closed-source & must be licensed. In response to Elastic de-emphasizing the FLOSS and increasingly switching development to “source-available”/closed-source-only, on 2019-03-11, Amazon / Netflix / Expedia—who offer Elasticsearch as part of their cloud offerings or rely on it internally—announced a fork, “Open Distro for Elasticsearch”; Elastic has further modified its licensing in response, and Amazon forked back harder⁠. So it goes… Conversely, consider Amazon AWS vs Kubernetes⁠/ Docker⁠. (As one of the main cloud providers, Amazon is merely one the most visible & criticized practitioners of this kind of enclosure, but, rhetoric about ‘strip-mining’ aside, of course this sort of thing is why much FLOSS software is funded in the first place by entities large and small & those complaining often benefit enormously from commoditize-your-complement fights elsewhere—what OSes do they all run on, for example?)

• image hosts vs social media services: Imgur vs Reddit: “The Decline of Imgur on Reddit and the Rise of Reddit’s Native Image Hosting”

• media companies vs the Internet: Vevo vs YouTube (“a former YouTube exec” is quoted as saying “Huge huge success for YouTube…YouTube needed Vevo to exist for just long enough to become so popular that the labels had no leverage anymore.”) , Hulu vs Netflix/Amazon/BitTorrent…

• search engines vs market operators:

  • Darknet Markets: many DNMs provided an API for the DNM search engine “Grams” when small⁹⁠, then neglected or revoked it at some point; this has an easy interpretation: DNM sellers and buyers want markets to be commodities that they can smoothly move their reputations & business between as DNMs go up or down (as they historically have); while of course, markets want vendors to be trapped and locked into them, and the market able to charge high commissions for the privilege of being where buyers go to find drugs

  • Booking Agencies vs Hotels/Airlines: exemplified by the story of Room Key and Booking.com

    • more recently, they are now all pitted against search engines as that is where customers start

• Restaurants:

  • Restaurants vs Reservation Booking Companies: eg OpenTable⁠/SevenRooms
  • Restaurants vs Food Order Apps: Internet/mobile app companies like Seamless⁠/ Grubhub⁠/ Uber Eats⁠/ DoorDash aggregate customers for ordering food locally, but of course, they take a large slice of revenue in an industry with already razor-thin, driving restaurants to smaller more-specialized competitors like Slice

• Venture Capital: Y Combinator: running Hacker News⁠, releasing “SAFE” notes & sales templates⁠, developing Startup School

See Also

• Tragedy of the anticommons
• Exit, Voice, and Loyalty
• Zero to One: Notes on Startups, or How to Build the Future
• “Tech Holy Wars are Coordination Problems”

External Links

• Many examples can be seen in Jimmy Maher’s PC gaming history “The Digital Antiquarian”:

  • eg the creation of & struggle over the IBM PC: 1⁠, 2⁠, 3⁠, 4⁠, 5

• “Better Than Free”⁠, Kevin Kelly

• Steve Yegge’s Google Platforms Rant

• “Digital Sharecropping”

• “The Wheel of Reincarnation” in computer displays

• “In the Beginning was the Command Line”⁠, Neal Stephenson

• “Aggregation Theory”⁠, Ben Thompson

• FLOSS:

  • “The Grand Unified Theory On The Economics Of Free”
  • “The Internet Was Built on the Free Labor of Open Source Developers. Is That Sustainable?”
  • “Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure”⁠, Nadia Eghbal 2016; “Is npm worth $2.6MM?”
  • “Open Source Archetypes: A Framework For Purposeful Open Source”

• Discussion: HN: 1⁠/ 2⁠/ 3⁠;  / r / SSC

Appendix