In Stevenson and Kording (2011), the authors estimated that every 7.4 years, the number of neurons we can record with doubles. Think of it as Moore’s law for brain recordings. Since then, Stevenson has updated the estimate, which now stands at 6 years. Could it be that progress itself is accelerating?
Matteo Carandini raised a question: why should progress be log-linear anyway? Technological phenomena have been argued to follow a double-exponential curve: the pace of progress itself accelerates over time. This is only noticeable when we look over a very long time horizon, for instance, when we look at computation per dollars over more than a century:
But we have 60 years to look at, so we can make these inferences! I took the data in Urai et al. (2021) – generously released under a CC-BY license – and fit a Bayesian Poisson regression model over time (code here). I fit only the electrophysiology data. It’s clear here that early times are underfit by the line. The doubling time estimated here is shorter than what has been noted in the literature – 4.5 years.
On a technical note, a Poisson regression model will tend to give larger weight to higher numbers – hence, it focuses on fitting the right-hand side of the graph, while the linear regression model that’s conventionally used gives equal weight everywhere. With an accelerating trend, that means the Poisson regression model give a shorter doubling time.
We can do one better – fit a double-exponential model. This is only a few lines of code in PyMC3 – a miracle of automatic differentiation and Hamiltonian Monte Carlo. Here’s what that looks like:
You can see visually this is a much better fit, and it implies something pretty dramatic: progress itself is accelerating. That means that doubling time itself has changed over time – and it currently stands at 3.6 years under this model [95% CI 3.5-3.7].
These results project a 1M neuron average recording capability by 2045 – of course, this discounts ceiling effects and potential paradigm shifts, which could adjust these bounds far upward or downward. What about optical methods? It turns out that the Poisson model works poorly because of overdispersion. I used a negative binomial to model the noise in the curve. I tried to let the overdispersion parameter be free, but I was getting convergence problems. Hence I fixed it to 2.0.
The implied doubling rate is a little less than 2 years. These numbers could swing wildly as we add more data, but we see that the doubling rate for imaging is at least twice the current rate for electrodes. This is due to the market pressures in cellphone sensors and telecommunications (fiber optics and LiDAR), making good sensors very cheap. Many in neurotech have taken note, including Facebook, which is building light-based BCIs, and Paradromics, which is adapting some of the fabrication methods from imaging sensors to electrophysiology.
Thus, this generalized Moore’s law of recordings is likely to continue decreasing in doubling time over the foreseeable future. Does this mean recording from every cell in the brain ($10^{11}$ cells) in the next 25 years? Probably not with electrodes – but if progress with light-based sensing continues at the same pace, perhaps. There is the vexing issue of scatter – and some of the people in this thread have some ideas on how to solve this.
Regardless of the exact course of progress, I think that 7 years is far too long a doubling time – perhaps 3.5 years for ephys, 2 years for imaging. The future ain’t what it used to be, and it’s coming far faster than we’ve perhaps imagined. What will we do with all this data? There’s some great hints in the Urai paper. An interesting research question si how holographic the brain is – perhaps we will get most of the understanding with far less than 100% coverage. Regardless, I think Adam Calhoun put it best:
Update: Ian Stevenson re-did the analysis with slightly different models, and found some slightly different results (longer doubling times than those reported here), from 4.5 to 5.6 years, depending on the assumptions. These are doubling times are nevertheless shorter than the ones reported previously in the literature so far, so the larger point still stands: the future is happening faster than we thought just a few years ago. Read the thread here.
[…] become commonplace to record from hundreds of neurons simultaneously. If past trends extrapolate, we might commonly record 10k neurons by 2030. What are we going to do with all this […]