Reimagining the PhD
November 15, 2019
I recently decided to wrap up my time at Protocol Labs, and along with it, my time in open source research.
I’ve spent the past 4+ years looking at how open source software is produced, from an economic and anthropological lens. The last thing I’ve been working on is a book, and now that the manuscript is nearly done, I’ve decided it’s time for something new.
A few friends have commented that writing a book seemed, for me, like the equivalent of writing a dissertation. Reflecting on the past few years, I realized that I’d (somewhat unintentionally) made my own version of a PhD program. [1] As I’m seeing more independent researchers crop up around town, I thought it might be helpful to share details on how I made this work. If you’re interested in exploring your own research inquiry through non-traditional means, here’s how I did mine.
(This is a long piece, so if you’re less interested in tactics, and more curious about the theoretical idea of making your own PhD, skip to the final section.)
I’d describe my time from 2015-2020 (when the book will be published) in three distinct phases:
- Phase I: 2015-2016: Discovery, exploration, initial research, hypothesis generation (funded by the Ford Foundation)
- Phase II: 2016-2018: Experience, experimenting, hypothesis testing (working at GitHub)
- Phase III: 2018-2020: Refining, summarizing, consolidation (funded by Protocol Labs)
Phase I: Discovery
My initial attraction to this space was exploring alternative funding mechanisms for technology. I started with a question (“What technology is widely used, but not venture backable in tech?”), explored the problem (through lots of reading and conversations), and eventually formulated a hypothesis (“Open source maintainers are undervalued by the market”). I didn’t intend to study open source at all, and initially felt I didn’t have enough credibility to write about it.
I had zero background or connections to open source when I started. A number of people early on encouraged me to get more hands-on experience first. I’m glad I didn’t, not because I think it wouldn’t been helpful, but because I think not having that experience became an advantage. Most open source developers only know their own experience, which is often limited to one project or one language ecosystem. But as an outsider, I could look for patterns across lots of different projects.
At this stage, I kept an open mind and was focused on asking lots of questions, absorbing information, forming relationships, and cold emailing the hell out of everybody. I embraced being curious, and accepted that I was going to be a little naive. I also tried to read primary material first (blog posts and conference talks by developers, issue threads and mailing lists on open source projects) before moving onto any books or papers on open source.
I didn’t write anything until I felt like I had something to say, which came after roughly six months of research. I published a blog post about it with my hypothesis. One of my best decisions was to start a mailing list, which seems more obvious today given the prevalence of newsletters, but at the time, was a last-minute addition before hitting publish.
Building relationships was, in parallel with hypothesis generation, the most important part of this phase. I got a strong response to my initial post, which I think was an unintentional effect of spending a lot of time trying to get to know open source developers beforehand. Once I started publishing, those conversations meant that there were a bunch of people who wanted to support my work.
I tried to embrace “learning in public” as much as I could. In addition to writing blog posts and tweeting, I created a bunch of other artifacts, like conference talks (developer conferences are a thing, which was new for me!), interviews, a podcast series, and lists published as GitHub repositories. I tried to use content formats that were appropriate to my target audience (software developers). Finally, I published a 143-page report called Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure, which capped off this first phase of research.
Fundamentally, I think of that report as a problem statement: it was a hypothesis, based on the conversations I’d had, and my goal was to get more people asking questions. I tried to reflect the voices of the people I’d been speaking to. I still felt like an outsider in open source, so I wanted to amplify the existing conversation, rather than inject too much of myself into it.
I funded my work during this time with two grants from the Ford Foundation. We met by chance, through an interviewee I’d also cold emailed. Before that, I’d been doing this work out-of-pocket for a few months. I had already done a bunch of interviews before talking to Ford, and I think having something to “sell” (highly specific knowledge that most other people hadn’t taken the time to dig into) was probably helpful.
The money was materially useful, but not substantial. Getting Ford’s involvement was more important to me as a source of validation. Knowing that someone cares about what you’re doing is one of those things that seems totally trivial unless you’re currently in that flailing-about sort of situation. Ford provided guidance and mentorship around how to think about “making a thing a Thing”. I’d felt isolated in tech, because my peers mostly felt that starting a company was the best way to effect change in the world (this was especially true pre-2016 IMO), so Ford had a broader social lens that I appreciated. In addition to moral support, like a great funder, they made sure that everyone in their world knew what I was doing (including Darren Walker, Ford’s president!).
Phase II: Testing
After drumming up some initial interest in the problem, I had to figure out where to take it. I’d gotten to know a few folks at GitHub, and after I started writing in public, we began to discuss the possibility of working there.
It’s hard to relate to this feeling now, but I’d felt conflicted about working at GitHub. I knew I’d stumbled upon an interesting problem, but I didn’t exactly know where to go from there. I could’ve gone the advocacy route, lobbying for more financial investment in open source. I could’ve gone the research route, digging up more data and stories and publishing them out to the world.
In the end, though, I wanted to maximize for impact. I cared about staying close to developers, and I felt that working at GitHub would be the fastest way to learn, because I’d get to test my ideas in a pragmatic setting.
It took awhile to figure out where exactly I would fit in at GitHub. I was adamant about not wanting to be in a “strategic” or research role, because the whole reason I wanted to be there was to get hands-on experience. I also didn’t want to be an evangelism role, because I felt like I’d already done a lot of that in Phase I. We worked together to craft a job description that everyone was happy with.
Rather than publishing content, my output at GitHub focused on building programs that would help GitHub, and the maintainers who used it, talk to each other more regularly, then taking those insights back into product. (One exception was Open Source Guides, the first project I worked on at GitHub.) My interest in this work was still influenced by research. To me, research was about collecting, curating, and amplifying others’ stories. If part of the problem was a widespread lack of awareness about what it was like to be an open source maintainer, it seemed that more people internally should have an opportunity to talk to, and learn from, maintainers.
I can’t overemphasize the value of getting hands-on experience, combined with the time I’d previously spent on research. I think it’s unlikely I could’ve gotten to my current point of view just by reading and talking to developers. Through my initial work, I’d become very familiar with what developers think about open source. But in trying to turn those needs into product, I was forced to make rhetoric meet reality. At the same time, having taken time before GitHub to deeply understand the problem greatly accelerated the work I was able to do there.
Phase III: Refining
Somewhere during my time at GitHub, I realized that my views had evolved quite a bit. In translating some of my earlier hypotheses into practice, not all of them had stood up to the test. I’d also felt frustrated by the many conversations I’d been having about open source, feeling like there was the same set of misconceptions being referenced over and over again, such as what a contributor was, or the idea that open source is necessarily participatory.
One of the reasons I write is to create external reference points for my own thinking (including this post!). Roads & Bridges did some of the legwork in providing the vocabulary I wanted, but not enough. Even among people where we were all on the same page about the problem, some second order of vocabulary was missing.
Because my long-term interest was to help shape public conversation on this topic, I decided to leave GitHub and go back to synthesizing the things I’d learned since Phase I. Luckily, Juan Benet (another person I’d met via cold email early on) offered me the time and space to do that research at Protocol Labs.
My setup was an independent researcher’s dream. As a full-time employee, I had all the freedom and resources I needed to explore, at a company that both valued research culture, and encouraged me to publish my work in public. I’d also just begun a research fellowship with Harvard, which gave me the support I needed to simulate some of the academic experience, like having access to journals and Harvard’s staff, library and information resources.
However, with unlimited freedom comes a fresh set of questions about how to structure one’s day-to-day. This past year and a half was perhaps materially the most comfortable, but emotionally the most grueling phase of my research, because it was so isolating.
I didn’t have a manager at Protocol Labs when I started, and the company is fully distributed, so I had to find other ways to create accountability for myself. I started using my newsletter as one mechanism, publishing my notes and bits of research every month. I also started an internal Google Doc (a trick I picked up at GitHub, another remote-friendly company!), where I’d write down all the major things I did that week. In the end, it was mostly just me who read it, but knowing that I’d have to externally log what I did or didn’t do made me feel more accountable.
The first six months or so were quite fun. I’d pick a research question to do a “sprint” on, usually lasting 2-4 weeks, and then publish a blog post with my findings. But, after continuing to have trouble expressing myself when talking to others about open source, I decided to write up a longer piece. My goal was to unwind some of the underlying theory and frameworks that I thought would be useful to have in these conversations. Eventually, this turned into a book.
I hadn’t planned on writing a book; in fact, after talking to a bunch of authors and publishers, I’d explicitly decided not to write one. But it felt like there were too many concepts that needed to be untangled, more than I could fit into one essay. I also felt like blog posts were too incremental to signify what I wanted to articulate. Although people would read those and share my writing, publishing them under my personal domain didn’t feel as “serious” as publishing, say, an academic paper. [2] It almost felt like I was doing a disservice to the work I’d put in. At the same time, I didn’t think that publishing papers would help me reach my intended audience (software developers), even if it made me feel more like a “real” researcher. [3]
I’d also seen how publishing Roads & Bridges as a report gave it a surprising amount of legitimacy. Back then, I genuinely didn’t think anyone would read it, and almost felt embarrassed to have spent so much time writing a long, self-indulgent piece. Three years later, people cite that report more than any of the blog posts that I wrote.
After talking to a few publishers, I finally found one that I loved, who prized moving quickly, and whose audience and interests aligned with mine. (I’m excited to share more details about it early next year!) Having a publisher, as opposed to self-publishing, also gave me the constraints and accountability that I craved, given the total freedom of my work environment.
I also sought the help of a friend, Josh Greenberg at the Sloan Foundation, whom I’d informally collaborated with since the early days, and who I felt really understood my work. After Roads and Bridges, Ford and Sloan had jointly funded a $1.3M research fund, in addition to their grantmaking, to support more researchers tackling these questions. Josh essentially served the role of a PhD advisor, making himself available every two weeks (or as often as I needed) to review my writing and give me feedback. I generally don’t like sharing drafts with other people, but for a project of this length, I realized that outside involvement was necessary. Working with Josh, as well as with a publisher and an editor, made my work significantly better.
Writing alone for a year sucks, especially when your thoughts are rough and jumbled. It didn’t feel like I was building upon my work this past year, so much as taking a “social sabbatical”. I felt withdrawn from the rest of the world, including my peers in open source. In previous phases, I’d been writing in public, talking to developers, organizing events. In this phase, even though I was working just as hard, none of my output was publicly visible, and that felt (still feels!) pretty crappy.
I found solace in talking to people who’d gone through PhD programs. (The “Acknowledgements” section of books also became a unique source of therapy for me. If you’re looking for it, a surprising number of authors talk about how miserable the writing process is.) Apparently, it’s not uncommon for PhD dropouts to drop out in their third year, given the lack of structure and accountability to their days. I would not recommend it in the same way that I would not recommend going through 7th grade, or rushing a fraternity. It’s not fun, but after it’s over, you kinda find yourself feeling nostalgic about it.
Sometimes, it feels like I’ve been looking at the same thing for a year, and that nothing has changed. But when I look back on my notes, I realize I’m in a totally different place. In a surprising way, I think this might have been the phase where I learned the most. I still feel a little too close to this past year, so I might be able to process it better sometime in the future.
A PhD for independent research
Underappreciated phenomenon:
Working at a high-growth, margin-focused startup is a real-life MBA where you get paid six figures plus equity to learn how to build a business instead of paying six figures and going into debt to learn how to build a theoretical one.
Over the course of my work, several professors reached out about considering a PhD. I seriously entertained the idea for a bit. What I most cared about was having an impact on public discourse, so I was definitely open to doing a PhD if I thought it would be a better path to getting there. In the end, I decided not to go the PhD route, but even considering the option meant that I had to articulate to myself why I thought the independent path made more sense than going into academia.
I’m not anti-PhD. In fact, I think it can be the better option, depending on what you’re studying, and what kind of support you’d receive. In my case, I didn’t see any practical benefit to continuing my work at a university. One professor told me that, after 4-6 years in the program, I’d be “considered a credible expert” in this topic. But my research focus was so narrow that there weren’t really many other experts in my field to begin with. I didn’t see why I needed to spend half a decade studying before anyone would take me seriously.
I was also told by one professor that it’d be “expected” for me to join faculty full-time after getting a PhD, which made me uncomfortable, because I knew I didn’t want to work in academia long-term. My goal was to deeply understand a topic, then bring it back into the real world. I worried about running into conflicts of interest.
Finally, I was concerned that pursuing a PhD would only drag me further away from my subject matter. If I wanted to reach software developers and companies, it didn’t seem that academia would offer particularly strong ties to either of these groups, compared to my current world in tech.
The most important thing that I think academia has to offer is giving researchers a stable career path to explore long-term questions. I’ve come to appreciate this more after my last phase of work. Working independently was useful to me in earlier stages of research. As public discourse starts to mature, it became more important to have a community of peers to go deeper with.
I had people I could talk to about different aspects of my work. With philanthropists, I could talk about field building. With academics, I could talk about theory. With GitHubbers, I could talk about the platform. With open source developers, I could talk about the day-to-day. But I didn’t know anybody else whose full-time job was to research this topic. Most of the time, I felt incredibly lonely.
If you are able to fund your research independently, and you’re willing to build your own support network, I think independent research is at least worth considering for those who might otherwise have done a PhD. If you’re thinking about this route, here are some of the building blocks that I found important to include.
Write your own curriculum
You have the freedom to mix and match and draw inspiration from wherever you’d like. Keep a list of the ideas that you keep coming back to. Some of my most cherished concepts came from conversations that I picked up from friends, spanning politics, religion, civil engineering, economics, anthropology, social media, and computer science. In the past month, my sources of inspiration included taking a Chevron refinery tour and watching Clueless.
Stay close to your subject
Have a target audience in mind that you’re trying to reach (it might be other academics!), and go spend time with them. Visit the places you’re writing about instead of reading about them, and make your audience care about what you’re doing. Talking to real developers in their “natural environment” helped me understand them in a way that a purely academic environment could not have; it’s shocking how many researchers don’t take the time to do this.
Work in public
I think of research as an act of service; anybody taking the time to deeply understand a subject is contributing back to the public repository of knowledge. Document what you learn in public, even if you’re not sure anybody’s paying attention. You’ll never know who might stumble across it. Longer-term, working in public also reduces the amount of repeat conversations you need to have, because you can just point people to a list or blog post. (Note: I don’t think this is necessarily at odds with, e.g. charging subscriptions. By “public”, I mean not keeping knowledge locked up inside your head, or proprietary to a specific company or client.)
Build your own support network
You need people to bounce ideas off of, to amplify your work, and sometimes just to tell you that what you’re doing isn’t stupid or a waste of time. You’re going to repeat your thesis a zillion times to a lot of people who think you’re an odd little bird. Build a cohort of people you don’t have to do that with, and who just “get” what you care about. I think of it like having a board (you don’t have to tell them that, though). Mine includes open source developers, GitHub employees, fellow researchers, academics, and philanthropists. I find myself naturally wanting to talk to different people for different things.
Find ways to hold yourself accountable
For better or worse, academia has its own reputation system built in, but independent research doesn’t come with success metrics. Don’t just research for the sake of research, or soon you’ll just feel like a person who reads all day (which might sound cool, but I promise is actually soul-crushing). Early explorations are naturally less unstructured, but if you’re doubling down for the long haul, setting goals will help focus your thinking. Know what sorts of outcomes would give you satisfaction (even if it’s “I want more people talking about X”). Your accountability mechanisms might be Twitter or a Google Doc, or a specific person that you show your work to.
Create artifacts that work for your audience
One frustration I’ve heard from academic researchers, especially those who identify as junior, is that they have to publish papers, because that’s how they’ll be measured. As an independent researcher, you don’t have those constraints, so you can focus on creating whatever artifacts you think will best reach your target audience (which might also mean academic papers, by the way!). I was mildly afraid of GitHub when I started, but I learned that open source developers use GitHub repositories for discussions, so I published some of my work that way. (A few other examples I like: Nicky Case publishes her ideas as games, and Michael Nielsen and Andy Matuschak are writing about quantum computing in an interactive readable format designed to improve retention.)
Resist the temptation to play “research dress-up”
Protocol Labs was my first full-time, salaried researcher position. Between that and the Harvard fellowship, I felt tempted to do things like read a lot of academic papers or write in an overly formal style. Ultimately I scrapped a lot of that work, because I realized it wasn’t me. If I want to cite a developer’s tweet or Taylor Swift lyrics instead of an academic paper, I can just do that. Try not to overthink what “doing research” means. You’re just a person, learning in public, about a topic that other people find interesting.
I’m gonna wrap this up by talking a little bit about the money side of things.
If you’re not on a tenure track, getting paid to do research (even for people who work in academia!) is frequently gig-based, like being a musician or an actor. It can be a nervewracking setup, because you tend to move laterally, rather than vertically, between organizations. (One person I spoke to referred to this as “research Frogger”: you’re hopping from place to place, living in mortal fear of the day that some CFO type starts to run the numbers and wonder what the heck you’re doing there.)
At each point between phases, I didn’t know where my next source of funding was going to come from. What I can say is that if you’re writing thoughtfully, in public, about a topic of niche interest to a certain set of people, and getting those people to engage with your work, it’s very likely that someone will come along and offer to pay you to keep doing that thing. And yes, it’s something you have to believe more on faith than logic. You still have to sell yourself, publish your work, and convince people that you know what you’re talking about. It’s both as hard, and not as hard, as it seems.
The upside of a gig-based lifestyle is that it gives you constraints, and when you’re doing something that’s somewhat illegible to others, constraints are a good thing. (A musician who plays in their bedroom is different from a musician who plays a show.) Independent research affords a incredible amount of freedom, but getting a blank check can be surprisingly scary. Gigs are less stable than tenure, but they also help you make meaning out of your limited time.
On a personal note, I still think of myself as a researcher. My desire to work is primarily motivated by my curiosity about a certain set of questions about the world. I’m not independently wealthy, so I try to find ways to get paid to explore these questions. Sometimes that means working at a company, and sometimes that means going off on my own. I’m not really advocating one over the other; I think you can be a researcher in any setting! (Here’s a fun example: astrophysics PhDs who love working at Stitch Fix.) I think what works for me is a sort of “cycling”: practical, hands-on experience to test out ideas, alternating with deep dives of thinking and exploring and synthesizing.
One of my favorite things about working in tech is that people aren’t afraid to experiment with traditional education, whether it’s K-12, college, or the MBA. In that same spirit, I’d love for us to give more thought as to how we might reinvent the PhD. [4]
Silicon Valley tends to favor practice over theory, often to the bemusement of outsiders, but it’s this strength that makes Y Combinator more attractive than an MBA, or the Thiel Fellowship more attractive than college. It’s also strongly influenced how I think about research. When I think about alternatives to a PhD, I imagine combining the best of both worlds: injecting theoretical concepts directly into the veins of practitioners.
Research, as a discipline, is a valuable public service. If you’re interested in pursuing a question that nobody else has answered to your satisfaction, I encourage you to give it a shot. I think a lot of people are discouraged from doing this because it doesn’t fit into a clear career path outside of academia. But there’s no shortage of interesting questions to explore, many of which, if someone took the time to tackle them, could have an enormous impact on the world.
Notes
[1] Although I’m not brave enough to try, I’ve been told that you can apparently submit a dissertation for a doctorate without going through a PhD program.
[2] One workaround I’ve seen, from Michael Nielsen’s Neural Networks and Deep Learning, is to add a footnote to your website with instructions on how to cite it in an academic context.
[3] If I were to do this again, I probably would’ve made a separate blog for my research, which might’ve made it feel more like a proper place for sharing my output. For example, Jason Crawford has both a personal website and a dedicated one for his research, Roots of Progress.
[4] A few more great sources of inspiration on this topic: Robin Hanson’s “Alternative Institutions” page, and David Klaing’s musings on a “personal PhD”.