White matter tracts form the structural basis of large-scale functional networks in the human brain. We applied brain-wide tractography to diffusion images from 30,810 adult participants (UK Biobank), and found significant heritability for 90 regional connectivity measures and 851 tract-wise connectivity measures. Multivariate genome-wide association analyses identified 355 independently associated lead SNPs across the genome, of which 77% had not been previously associated with human brain metrics. Enrichment analyses implicated neurodevelopmental processes including neurogenesis, neural differentiation, neural migration, neural projection guidance, and axon development, as well as prenatal brain expression especially in stem cells, astrocytes, microglia and neurons. We used the multivariate association profiles of lead SNPs to identify 26 genomic loci implicated in structural connectivity between core regions of the left-hemisphere language network, and also identified 6 loci associated with hemispheric left-right asymmetry of structural connectivity. Polygenic scores for schizophrenia, bipolar disorder, autism spectrum disorder, attention-deficit hyperactivity disorder, left-handedness, Alzheimer’s disease, amyotrophic lateral sclerosis, and epilepsy showed significant multivariate associations with structural connectivity, each implicating distinct sets of brain regions with trait-relevant functional profiles. This large-scale mapping study revealed common genetic contributions to the structural connectome of the human brain in the general adult population, highlighting links with polygenic disposition to brain disorders and behavioural traits.
Brain size and IQ are positively correlated. However, multiple meta-analyses have led to considerable differences in summary effect estimations, thus failing to provide a plausible effect estimate.
Here we aim at resolving this issue by providing the largest meta-analysis and systematic review so far of the brain volume and IQ association (86 studies; 454 effect sizes from k = 194 independent samples; n > 26 000) in 3 cognitive ability domains (full-scale, verbal, performance IQ).
By means of competing meta-analytical approaches as well as combinatorial and specification curve analyses, we show that most reasonable estimates for the brain size and IQ link yield r-values in the mid-0.20s, with the most extreme specifications yielding rs of 0.10 and 0.37.
Summary effects appeared to be somewhat inflated due to selective reporting, and cross-temporally decreasing effect sizes indicated a confounding decline effect, with 3⁄4th of the summary effect estimations according to any reasonable specification not exceeding r = 0.26, thus contrasting effect sizes were observed in some prior related, but individual, meta-analytical specifications. Brain size and IQ associations yielded r = 0.24, with the strongest effects observed for more g-loaded tests and in healthy samples that generalize across participant sex and age bands.
[Keywords: multiverse analysis, in vivo brain volume, systematic review, intelligence, specification curve analysis, meta-analysis]
Weight loss diets often restrict either fat or carbohydrate, macronutrients that are sensed via distinct gut-brain pathways and differentially affect peripheral hormones and metabolism. To investigate whether reductions in dietary fat versus carbohydrate alter brain reward circuitry, we measured dopamine D2/3 receptor binding potential (D2BP) using PET and neural activity in response to visual food cues using fMRI in 17 inpatient adults with obesity during an eucaloric baseline diet and on the fifth day of isocaloric diets selectively reduced in either dietary fat or carbohydrate, in random order. Reduction of dietary fat, but not carbohydrate, decreased D2BP and decreased neural activity to food cues in brain reward regions. After the reduced fat diet, ad libitum intake shifted towards foods high in both fat and carbohydrates. These results suggest that dietary fat restriction increases tonic dopamine in brain reward regions thereby affecting food choice in ways that may hamper diet adherence.
More than a hundred genes have been identified that, when disrupted, impart large risk for autism spectrum disorder (ASD). Current knowledge about the encoded proteins—although incomplete—points to a very wide range of developmentally dynamic and diverse biological processes. Moreover, the core symptoms of ASD involve distinctly human characteristics, presenting challenges to interpreting evolutionarily distant model systems. Indeed, despite a decade of striking progress in gene discovery, an actionable understanding of pathobiology remains elusive.
Increasingly, convergent neuroscience approaches have been recognized as an important complement to traditional uses of genetics to illuminate the biology of human disorders. These methods seek to identify intersection among molecular-level, cellular-level and circuit-level functions across multiple risk genes and have highlighted developing excitatory neurons in the human mid-gestational prefrontal cortex as an important pathobiological nexus in ASD. In addition, neurogenesis, chromatin modification and synaptic function have emerged as key potential mediators of genetic vulnerability.
The continued expansion of foundational ‘omics’ data sets, the application of higher-throughput model systems and incorporating developmental trajectories and sex differences into future analyses will refine and extend these results. Ultimately, a systems-level understanding of ASD genetic risk holds promise for clarifying pathobiology and advancing therapeutics.
We revealed the neural nature of abstract WM representations
Distinct visual stimuli were recoded into a shared abstract memory format
Memory formats for orientation and motion direction were recoded into a line-like pattern
Such formats are more efficient and proximal to the behaviors they guide
Working memory (WM) enables information storage for future use, bridging the gap between perception and behavior.
We hypothesize that WM representations are abstractions of low-level perceptual features. However, the neural nature of these putative abstract representations has thus far remained impenetrable.
Here, we demonstrate that distinct visual stimuli (oriented gratings and moving dots) are flexibly recoded into the same WM format in visual and parietal cortices when that representation is useful for memory-guided behavior. Specifically, the behaviorally relevant features of the stimuli (orientation and direction) were extracted and recoded into a shared mnemonic format that takes the form of an abstract line-like pattern.
We conclude that mnemonic representations are abstractions of percepts that are more efficient than and proximal to the behaviors they guide.
[Twitter] Understanding the molecular anatomy and neural connectivity of the brain requires imaging technologies that can map the 3D nanoscale distribution of specific proteins in the context of brain ultrastructure. Light and electron microscopy (EM) enable visualization of either specific labels or anatomical ultrastructure, but combining molecular specificity with anatomical context is challenging.
Here, we present pan-Expansion Microscopy of tissue (pan-ExM-t), an all-optical mouse brain imaging method that combines ~24× linear expansion of biological samples with fluorescent pan-staining of protein densities (providing EM-like ultrastructural context), and immunolabeling of protein targets (for molecular imaging).
We demonstrate the versatility of this approach by imaging the established synaptic markers Homer1, Bassoon, PSD-95, Synaptophysin, the astrocytic protein GFAP, myelin basic protein (MBP), and anti-GFP antibodies in dissociated neuron cultures and mouse brain tissue sections. pan-ExM-t reveals these markers in the context of ultrastructural features such as pre and postsynaptic densities, 3D nanoarchitecture of neuropil, and the fine structures of cellular organelles.
pan-ExM-t is adoptable in any neurobiological laboratory with access to a confocal microscope and has therefore broad applicability in the research community.
Human brain structure changes throughout the lifespan. Altered brain growth or rates of decline are implicated in a vast range of psychiatric, developmental and neurodegenerative diseases.
In this study, we identified common genetic variants that affect rates of brain growth or atrophy in what is, to our knowledge, the first genome-wide association meta-analysis of changes in brain morphology across the lifespan.
Longitudinal magnetic resonance imaging data from 15,640 individuals [in 40 cohorts] were used to compute rates of change for 15 brain structures.
The most robustly identified genes GPR139, DACH1 and APOE are associated with metabolic processes. We demonstrate global genetic overlap with depression, schizophrenia, cognitive functioning, insomnia, height, body mass index and smoking. Gene set findings implicate both early brain development and neurodegenerative processes in the rates of brain changes.
Identifying variants involved in structural brain changes may help to determine biological pathways underlying optimal and dysfunctional brain development and aging.
Magnetic resonance imaging (MRI) has transformed our understanding of the human brain through well-replicated mapping of abilities to specific structures (for example, lesion studies) and functions (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI. A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping4 (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brain-behavioural phenotype associations.
Here we used 3 of the largest neuroimaging datasets currently available—with a total sample size of around 50,000 individuals—to quantify BWAS effect sizes and reproducibility as a function of sample size. [Adolescent Brain Cognitive Development (ABCD) study, n = 11,874; Human Connectome Project (HCP), n = 1,200; and UK Biobank (UKB), n = 35,73]
BWAS associations were smaller than previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes. As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate).
Smaller than expected brain-phenotype associations and variability across population subsamples can explain widespread BWAS replication failures. In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals.
[Twitter] Structural variants (SVs), which are genomic rearrangements of more than 50 base pairs, are an important source of genetic diversity and have been linked to many diseases. However, it remains unclear how they modulate human brain function and disease risk.
Here we report 170,996 SVs discovered using 1,760 short-read whole genomes from aged adults and individuals with Alzheimer’s disease. By applying quantitative trait locus (SV-xQTL) analyses, we quantified the impact of cis-acting SVs on histone modifications, gene expression, splicing and protein abundance in postmortem brain tissues.
More than 3,200 SVs were associated with at least one molecular phenotype. We found reproducibility of 65–99% SV-eQTLs across cohorts and brain regions. SV associations with mRNA and proteins shared the same direction of effect in more than 87% of SV-gene pairs. Mediation analysis showed ~8% of SV-eQTLs mediated by histone acetylation and ~11% by splicing. Additionally, associations of SVs with progressive supranuclear palsy identified previously known and novel SVs.
Recent work in imaging genetics suggests high levels of genetic overlap within cortical regions for cortical thickness (CT) and surface area (SA).
We model this relationship by applying Genomic Structural Equation Modeling (Genomic SEM) to parsimoniously define 5 genomic brain factors for both CT and SA. We reify these factors by demonstrating the generalizability of the model in a semi-independent sample and show that the factors align with biologically and functionally relevant parcellations of the cortex. We apply Stratified Genomic SEM to identify specific categories of genes (eg. neuronal cell types) that are disproportionately associated with pleiotropy across specific subclusters of brain regions, as indexed by the genomic factors. Finally, we examine genetic associations with psychiatric and cognitive correlates, finding that SA is associated with both broad aspects of cognitive function and specific risk pathways for psychiatric disorders.
These analyses provide key insights into the multivariate genomic architecture of two critical features of the cerebral cortex.
Machine learning studies have shown that various phenotypes can be predicted from structural and functional brain images. However, in most such studies, prediction performance ranged from moderate to disappointing. It is unclear whether prediction performance will substantially improve with larger sample sizes or whether insufficient predictive information in brain images impedes further progress. Here, we systematically assess the effect of sample size on prediction performance using sample sizes far beyond what is possible in common neuroimaging studies. We project 3–9× improvements in prediction performance for behavioral and mental health phenotypes when moving from one thousand to one million samples. Moreover, we find that moving from single imaging modalities to multimodal input data can lead to further improvements in prediction performance, often on par with doubling the sample size. Our analyses reveal considerable performance reserves for neuroimaging-based phenotype prediction. Machine learning models may benefit much more from extremely large neuroimaging datasets than currently believed.
The question of how much sleep is best for the brain attracts scientific and public interest, and there is concern that insufficient sleep leads to poorer brain health. However, it is unknown how much sleep is sufficient and how much is too much. We analyzed 51,295 brain magnetic resonance images from 47,039 participants, and calculated the self-reported sleep duration associated with the largest regional volumes and smallest ventricles relative to intracranial volume (ICV) and thickest cortex. 6.8 hours of sleep was associated with the most favorable brain outcome overall. Critical values, defined by 95% confidence intervals, were 5.7 and 7.9 hours. There was regional variation, with for instance the hippocampus showing largest volume at 6.3 hours. Moderately long sleep (> 8 hours) was more strongly associated with smaller relative volumes, thinner cortex and larger ventricles than even very short sleep (< 5 hours), but effect sizes were modest. People with larger ICV reported longer sleep (7.5 hours), so not correcting for ICV yielded longer durations associated with maximal volume. Controlling for socioeconomic status, body mass index and depression symptoms did not alter the associations. Genetic analyses showed that genes related to longer sleep in short sleepers were related to shorter sleep in long sleepers. This may indicate a genetically controlled homeostatic regulation of sleep duration. Mendelian randomization analyses did not suggest sleep duration to have a causal impact on brain structure in the analyzed datasets. The findings challenge the notion that habitual short sleep is negatively related to brain structure.
[previously: Schrimpf et al 2021/Caucheteux & King 2021; Twitter] Deep learning algorithms trained to predict masked words from large amount of text have recently been shown to generate activations similar to those of the human brain. However, what drives this similarity remains currently unknown.
Here, we systematically compare a variety of deep language models to identify the computational principles that lead them to generate brain-like representations of sentences. Specifically, we analyze the brain responses to 400 isolated sentences in a large cohort of 102 subjects, each recorded for 2 hours with functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We then test where and when each of these algorithms maps onto the brain responses. Finally, we estimate how the architecture, training, and performance of these models independently account for the generation of brain-like representations.
Our analyses reveal 2 main findings. First, the similarity between the algorithms and the brain primarily depends on their ability to predict words from context. Second, this similarity reveals the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region.
Overall, this study shows that modern language algorithms partially converge towards brain-like solutions, and thus delineates a promising path to unravel the foundations of natural language processing.
Over the last decade, numerous studies have shown that deep neural networks exhibit sensory representations similar to those of the mammalian brain, in that their activations linearly map onto cortical responses to the same sensory inputs. However, it remains unknown whether these artificial networks also learn like the brain. To address this issue, we analyze the brain responses of two ferret auditory cortices recorded with functional UltraSound imaging (fUS), while the animals were presented with 320 10 s sounds. We compare these brain responses to the activations of Wav2vec 2.0, a self-supervised neural network pretrained with 960 h of speech, and input with the same 320 sounds. Critically, we evaluate Wav2vec 2.0 under two distinct modes: (1) “Pretrained”, where the same model is used for all sounds, and (2) “Continuous Update”, where the weights of the pretrained model are modified with back-propagation after every sound, presented in the same order as the ferrets. Our results show that the Continuous-Update mode leads Wav2Vec 2.0 to generate activations that are more similar to the brain than a Pretrained Wav2Vec 2.0 or than other control models using different training modes. These results suggest that the trial-by-trial modifications of self-supervised algorithms induced by back-propagation aligns with the corresponding fluctuations of cortical responses to sounds. Our finding thus provides empirical evidence of a common learning mechanism between self-supervised models and the mammalian cortex during sound processing.
[Previously: studio diip’s“Fish on Wheels”; rodent operated vehicle; rat VR using foam spheres rolling in place] Navigation is a critical ability for animal survival and is important for food foraging, finding shelter, seeking mates and a variety of other behaviors. Given their fundamental role and universal function in the animal kingdom, it makes sense to explore whether space representation and navigation mechanisms are dependent on the species, ecological system, brain structures, or whether they share general and universal properties.
One way to explore this issue behaviorally is by domain transfer methodology, where one species is embedded in another species’ environment and must cope with an otherwise familiar (in our case, navigation) task. Here we push this idea to the limit by studying the navigation ability of a fish in a terrestrial environment.
For this purpose, we trained goldfish to use a Fish Operated Vehicle (FOV), a wheeled terrestrial platform that reacts to the fish’s movement characteristics, location and orientation in its water tank to change the vehicle’s; i.e., the water tank’s, position in the arena. The fish were tasked to “drive” the FOV towards a visual target in the terrestrial environment, which was observable through the walls of the tank.
The fish were indeed able to operate the vehicle, explore the new environment, and reach the target regardless of the starting point, all while avoiding dead-ends and correcting location inaccuracies.
These results demonstrate how a fish was able to transfer its space representation and navigation skills to a wholly different terrestrial environment, thus supporting the hypothesis that the former possess an universal quality that is species-independent.
Intelligence describes the general cognitive ability level of a person. It is one of the most fundamental concepts in psychological science and is crucial for the effective adaption of behavior to varying environmental demands. Changing external task demands have been shown to induce reconfiguration of functional brain networks. However, whether neural reconfiguration between different tasks is associated with intelligence has not yet been investigated.
We used functional magnetic resonance imaging data from 812 subjects to show that higher scores of general intelligence are related to less brain network reconfiguration between resting state and 7 different task states as well as to network reconfiguration between tasks. This association holds for all functional brain networks except the motor system and replicates in 2 independent samples (n = 138 and n = 184).
Our findings suggest that the intrinsic network architecture of individuals with higher intelligence scores is closer to the network architecture as required by various cognitive demands. Multitask brain network reconfiguration may, therefore, represent a neural reflection of the behavioral positive manifold—the essence of the concept of general intelligence. Finally, our results support neural efficiency theories of cognitive ability and reveal insights into human intelligence as an emergent property from a distributed multitask brain network.
…Here, we use fMRI data from a large sample of healthy adults (n = 812) assessed during different cognitive states, that is, during resting state and during 7 different task states, to test the hypothesis that higher levels of general intelligence relate to less brain network reconfiguration.
Specifically, we expected this association to manifest in reaction to different cognitive demands and on various spatial scales. We used a straight-forward operationalization of brain network reconfiguration and implemented our analyses on a whole-brain level as well as on the level of 7 and 17 canonical functional brain networks.
The results confirm our hypotheses and suggest that functional brain networks of more intelligent people may require less adaption when switching between different cognitive states, thus pointing toward the existence of an advantageous intrinsic brain network architecture.
Furthermore, we show that although the different cognitive states were induced by different demanding tasks, their relative contribution to the observed effect was nearly identical; a finding that supports the assumption of a task-general neural correlate—a neural-positive manifold.
Finally, the involvement of multiple brain networks suggests intelligence as an emergent property of a widely distributed multitask brain network.
Genes control cortical surface area: Humans exhibit heritable variation in brain structure and function. To identify how gene variants affect the cerebral cortex, Makowski et al 2022 performed genome-wide association studies in almost 40,000 adults and 9,000 children. They identified more than 400 loci associated with brain surface area and cortical thickness that could be observed through magnetic resonance imaging analyses. Examining biological pathways linking gene variants to phenotypes identified region-specific enrichments of neurodevelopmental functions, some of which were associated with psychiatric disorders. Partitioning genes with heritable variants relative to evolutionary conservation helped to identify a hierarchy of brain development. This analysis identified a human-specific gene-phenotype association related to speech and informs what genes can be studied in various model organisms.
To determine the impact of genetic variants on the brain, we used genetically informed brain atlases in genome-wide association studies of regional cortical surface area and thickness in 39,898 adults and 9,136 children.
We uncovered 440 genome-wide statistically-significant loci in the discovery cohort and 800 from a post hoc combined meta-analysis. Loci in adulthood were largely captured in childhood, showing signatures of negative selection, and were linked to early neurodevelopment and pathways associated with neuropsychiatric risk. Opposing gradations of decreased surface area and increased thickness were associated with common inversion polymorphisms. Inferior frontal regions, encompassing Broca’s area, which is important for speech, were enriched for human-specific genomic elements.
Thus, a mixed genetic landscape of conserved and human-specific features is concordant with brain hierarchy and morphogenetic gradients.
Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate representations capable of harnessing the underlying symmetries of their inputs is thus crucial in machine learning and neuroscience. Convolutional neural networks, for example, were designed to exploit translation symmetry and their capabilities triggered the first wave of deep learning successes. However, learning convolutions directly from translation-invariant data with a fully-connected network has so far proven elusive.
Here, we show how initially fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs, resulting in localized, space-tiling receptive fields. These receptive fields match the filters of a convolutional network trained on the same task.
By carefully designing data models for the visual scene, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs, which has long been recognised as the hallmark of natural images. We provide an analytical and numerical characterisation of the pattern-formation mechanism responsible for this phenomenon in a simple model, which results in an unexpected link between receptive field formation and the tensor decomposition of higher-order input correlations.
These results provide a new perspective on the development of low-level feature detectors in various sensory modalities, and pave the way for studying the impact of higher-order statistics on learning in neural networks.
For decades, researchers like Cantlon have been studying how animals understand quantities, and they have considered factors ranging from their social group size to diet to total brain volume. Now, drawing from published work on dozens of species, a large team led by Cantlon has found a striking pattern: The density of neurons that an animal has in their cortex predicts its quantitative sense better than any other factor. The work, published in December in Philosophical Transactions of the Royal Society B, shows constraints from evolution—rather than learning or behavior—on cognition. They found that phylogeny, or evolutionary “distance” between species, predicts how well they do at estimating quantities compared to each other. Closely related species tend to have similar levels of skill. Distantly related ones may vary widely. “It’s an impressive study because of the enormous amount of data and all the different factors that they took into account”, says Sarah Brosnan, who researches animal decision-making at Georgia State University.
To Brosnan, the results justify a new wave of research into why some species evolved different cognition—and what that might say about humans. Maybe the reason we’re good at understanding quantities isn’t simply that we are primates. If neural density is indeed the critical factor, that trait might be shared by vastly different species with vastly different brains. “Just because you’re a primate doesn’t mean you’re the brightest”, Brosnan says. And if having a primate brain isn’t the gold standard for abstract skills that it was once made out to be, she asks, “What is it that’s driving intelligence and cognition?”
It has not been long since researchers discovered that animals can compare quantities of things. “Thirty or 40 years ago, people were curious: Could animals do it at all?” Cantlon says…But it’s hard to compare skills across animal species. Study methodologies vary, so they are not always scientifically compatible, especially the more elaborate ones. For their own analysis, Cantlon’s team needed to find a task common enough to have been repeated in experiments among a diverse set of species. They settled on a simple task in which researchers offer animals 2 piles of treats. One pile contains more than the other, like the olive baboon’s peanuts. This type of task has appeared in 49 different studies from around the world, involving 672 individual animals across 33 species. If a parrot, dolphin, horse, or whatever statistically favors piles with more items, then researchers conclude that they likely are able to estimate those quantities. The average sensitivity across species seems to be around 2:1 ratios—they will choose 10 over 5, but 7 versus 5 gets fuzzier.
…Slowly, a picture began to emerge: Animals who were closer together on the phylogenetic tree tended to perform similarly well in the experiments. Chimps were among the top performers, for example. Their close relatives, bonobos, were too. Lemurs, which are more distantly related to them, performed about average.
But non-primate species clustered on other branches of the phylogenetic tree did well too. Grey parrots and rock doves performed about as well as the chimps, and better than many primates. Overall, the study showed, a key predictor of quantitative skills is being closely related to other animals with those skills—not being a primate or even a mammal. “It means that you can pluck any individual animal out of the world and predict something about how sensitive it is to quantity, just by knowing what species it belongs to.” Cantlon says, “That’s new.”
Phylogeny can only tell scientists so much, though. The team wondered if differences might come down to the animals’ neurophysiology. But they weren’t sure which aspect of the brain to measure.
In the past, researchers often used an animal’s total brain volume as a proxy for cognitive power. Basically, the bigger the better. But when Bryer and Koopman pulled the data, they found a weak correlation between brain size and quantitative sensitivity. They turned to a relatively new metric—cortical neuron density—which tells scientists how many neurons a brain has in its cortex. (The cortex is the outer layer of tissue in mammalian brains and is associated with complex cognition.) Let’s not mince words: To quickly count the number of neurons per milligram of brain, a researcher has to liquefy it. (“She calls it ‘brain soup,’” Cantlon says of neuroscientist Suzana Herculano-Houzel of Vanderbilt University, who developed the method. “It is literally melting it in chemicals.”) In this case, the researchers used data sets from Herculano-Houzel’s lab, pulling published figures on neuron density for 12 species. Here, the correlation was clear: Neuron density had the biggest effect on quantitative sensitivity among all metrics tested, including traits like home range size and social group size. Since neuron density is largely constrained by a species’ genes, the team sees that as bonus proof that evolution plays a huge role.
The magic of neuron density is that it has consequences for cognition, yet it is surprisingly independent of brain size. For some mammals, larger brains might have larger neurons and thus lower density. But that is by no means a general rule. It’s simply its own thing. Smaller neurons, with smaller branches, can pack together tighter and give a brain a more fine-grained sense of the world. “Think of the number of pixels in a camera: The more pixels, the more resolution”, says Herculano-Houzel, who was not involved in this study. The new findings are valuable as the field of cognitive science breaks away from old assumptions about evolution, she says. Scientists have historically explained away interspecies variations in cognition with differences in body size, brain volume, or the problematic notion that humans and primates are more evolved than other animals. “There’s no one way in nature to build a brain and a body around it”, says Herculano-Houzel. “There is no ideal brain. There’s no better brain.”
…Still, when it comes to estimating quantities, humans are the top performers. We can do it with around 10% precision. Cantlon suspects that the neurological process is very similar for all species, but humans can just do it with a greater degree of sensitivity. It’s a skill that may have led to our ability to count—and perhaps to our symbolic representations of numbers and letters.
We introduce Active Predictive Coding Networks (APCNs), a new class of neural networks that solve a major problem posed by Hinton and others in the fields of artificial intelligence and brain modeling: how can neural networks learn intrinsic reference frames for objects and parse visual scenes into part-whole hierarchies by dynamically allocating nodes in a parse tree? APCNs address this problem by using a novel combination of ideas: (1) hypernetworks are used for dynamically generating recurrent neural networks that predict parts and their locations within intrinsic reference frames conditioned on higher object-level embedding vectors, and (2) reinforcement learning is used in conjunction with backpropagation for end-to-end learning of model parameters. The APCN architecture lends itself naturally to multilevel hierarchical learning and is closely related to predictive coding models of cortical function. Using the MNIST, Fashion-MNIST and Omniglot datasets, we demonstrate that APCNs can (a) learn to parse images into part-whole hierarchies, (b) learn compositional representations, and (c) transfer their knowledge to unseen classes of objects. With their ability to dynamically generate parse trees with part locations for objects, APCNs offer a new framework for explainable AI that leverages advances in deep learning while retaining interpretability and compositionality.
Cognitive variation is common among-individuals and can be consistent across time and context. From an evolutionary perspective, among-individual variation is important as a pre-requisite for natural selection and adaptive evolution. Selection is widely hypothesized to favor high cognitive performance but directional selection should erode variation over time, how then is cognitive variation maintained? As selection does not act on traits in isolation, covariance among specific cognitive traits and/or other aspects of phenotype (eg. personality) could result in fitness trade-offs that are important in shaping evolutionary dynamics. Here we test this using Trinidadian guppies (Poecilia reticulata), using a multivariate approach by characterizing the correlation structure among task-specific cognitive performance measures and a personality trait. We estimate the among-individual correlation matrix (ID) in performance across three cognitive tasks; colour association learning task; motor learning task; reversal learning task, and the personality trait, boldness, measured as emergence time from a shelter. We found no support for trade-offs among performance in these tasks. Nor do we find evidence of hypothesised speed-accuracy trade-offs within the association learning task. Rather we find strong positive correlation structure in ID, with 57% of variation explained by the leading eigen vector. While noting that non-cognitive factors and assay composition may affect the structure of ID, we suggest our findings are consistent with the g-model of cognitive performance variation, in which a dominant axis of variation loads positively on all performance measures. Thus, we add to a growing body of support for general variation among individuals in animal cognitive ability.
While large-scale, genome-wide association studies (GWAS) have identified hundreds of loci associated with brain-related traits, identification of the variants, genes and molecular mechanisms underlying these traits remains challenging. Integration of GWAS with expression quantitative trait loci (eQTLs) and identification of shared genetic architecture have been widely adopted to nominate genes and candidate causal variants. However, this approach is limited by sample size, statistical power and linkage disequilibrium.
We developed the multivariate multiple QTL approach and performed a large-scale, multi-ancestry eQTL meta-analysis to increase power and fine-mapping resolution.
Analysis of 3,983 RNA-sequenced samples from 2,119 donors, including 474 non-European individuals, yielded an effective sample size of 3,154. Joint statistical fine-mapping of eQTL and GWAS identified 329 variant-trait pairs for 24 brain-related traits driven by 204 unique candidate causal variants for 189 unique genes.
Guided by gut sensory cues, humans and animals prefer nutritive sugars over non-caloric sweeteners, but how the gut steers such preferences remains unknown. In the intestine, neuropod cells synapse with vagal neurons to convey sugar stimuli to the brain within seconds.
Here, we found that cholecystokinin (CCK)-labeled duodenal neuropod cells differentiate and transduce luminal stimuli from sweeteners and sugars to the vagus nerve using sweet taste receptors and sodium glucose transporters. The 2 stimulus types elicited distinct neural pathways: while sweetener stimulated purinergic neurotransmission, sugar stimulated glutamatergic neurotransmission.
To probe the contribution of these cells to behavior, we developed optogenetics for the gut lumen by engineering a flexible fiberoptic. We showed that preference for sugar over sweetener in mice depends on neuropod cell glutamatergic signaling.
By swiftly discerning the precise identity of nutrient stimuli, gut neuropod cells serve as the entry point to guide nutritive choices.
The human visual ability to recognize objects and scenes is widely thought to rely on representations in category-selective regions of visual cortex. These representations could support object vision by specifically representing objects, or, more simply, by representing complex visual features regardless of the particular spatial arrangement needed to constitute real world objects. That is, by representing visual textures.
To discriminate between these hypotheses, we leveraged an image synthesis approach that, unlike previous methods, provides independent control over the complexity and spatial arrangement of visual features. We found that human observers could easily detect a natural object among synthetic images with similar complex features that were spatially scrambled. However, observer models built from BOLD responses from category-selective regions, as well as a model of macaque inferotemporal cortex and Imagenet-trained deep convolutional neural networks, were all unable to identify the real object. This inability was not due to a lack of signal-to-noise, as all of these observer models could predict human performance in image categorization tasks.
How then might these texture-like representations in category-selective regions support object perception? An image-specific readout from category-selective cortex yielded a representation that was more selective for natural feature arrangement, showing that the information necessary for object discrimination is available. Thus, our results suggest that the role of human category-selective visual cortex is not to explicitly encode objects but rather to provide a basis set of texture-like features that can be infinitely reconfigured to flexibly learn and identify new object categories.
Background: Psilocybin, a psychoactive serotonin receptor partial agonist, has been reported to acutely reduce clinical symptoms of depressive disorders. Psilocybin’s effects on cognitive function have not been widely or systematically studied.
Aim: The aim of this study was to explore the safety of simultaneous administration of psilocybin to healthy participants in the largest randomised controlled trial of psilocybin to date. Primary and secondary endpoints assessed the short-term and longer-term change in cognitive functioning, as assessed by a Cambridge Neuropsychological Test Automated Battery (CANTAB) Panel, and emotional processing scales. Safety was assessed via endpoints which included cognitive function, assessed by CANTAB global composite score, and treatment-emergent adverse event (TEAE) monitoring.
Methods: In this phase 1, randomised, double-blind, placebo-controlled study, healthy participants (n = 89; mean age 36.1 years; 41 females, 48 males) were randomised to receive a single oral dose of 10 or 25 mg psilocybin, or placebo, administered simultaneously to up to 6 participants, with one-to-one psychological support—each participant having an assigned, dedicated therapist available throughout the session.
Results: In total, 511 TEAEs were reported, with a median duration of 1.0 day; 67% of all TEAEs started and resolved on the day of administration. There were no serious TEAEs, and none led to study withdrawal. There were no clinically relevant between-group differences in CANTAB global composite score, CANTAB cognitive domain scores, or emotional processing scale scores.
Conclusions: These results indicate that 10 mg and 25 mg doses of psilocybin were generally well tolerated when given to up to 6 participants simultaneously and did not have any detrimental short-term or long-term effects on cognitive functioning or emotional processing.
The goal of perception is to infer the most plausible source of sensory stimulation. Unisensory perception of temporal order, however, appears to require no inference, because the order of events can be uniquely determined from the order in which sensory signals arrive.
Here, we demonstrate a novel perceptual illusion that casts doubt on this intuition: In three experiments (n = 607), the experienced event timings were determined by causality in real time. Adult participants viewed a simple three-item sequence, ACB, which is typically remembered as ABC in line with principles of causality. When asked to indicate the time at which events B and C occurred, participants’ points of subjective simultaneity shifted so that the assumed cause B appeared earlier and the assumed effect C later, despite participants’ full attention and repeated viewings.
This first demonstration of causality reversing perceived temporal order cannot be explained by postperceptual distortion, lapsed attention, or saccades.
Brain imaging genetics is an emerging research field aiming to reveal the genetic basis of brain traits captured by imaging data. Inspired by heritability analysis, the concept of morphometricity was recently introduced to assess trait association with whole brain morphology.
In this study, we extend the concept of morphometricity from its original definition at the whole brain level to a more focal level based on a region of interest (ROI). We propose a novel framework to identify the SNP-ROI association via regional morphometricity estimation of each studied single nucleotide polymorphism (SNP).
We perform an empirical study on the structural MRI and genotyping data from a landmark Alzheimer’s disease (AD) biobank; and yield promising results.
Our findings indicate that the AD-related SNPs have higher overall regional morphometricity estimates than the SNPs not yet related to AD. This observation suggests that the variance of AD SNPs can be explained more by regional morphometric features than non-AD SNPs, supporting the value of imaging traits as targets in studying AD genetics. Also, we identified 11 ROIs, where the AD/non-AD SNPs and statistically-significant/insignificant morphometricity estimation of the corresponding SNPs in these ROIs show strong dependency. Supplementary motor area (SMA) and dorsolateral prefrontal cortex (DPC) are enriched by these ROIs.
Our results also demonstrate that using all the detailed voxel-level measures within the ROI to incorporate morphometric information outperforms using only a single average ROI measure, and thus provides improved power to detect imaging genetic associations.
Sleep deprivation (SD) induces systemic inflammation that promotes neuronal pyroptosis.
The purpose of this study was to investigate the effect of an antioxidant modafinil on neuronal pyroptosis and cognitive decline following SD.
Using a mouse model of SD, we found that modafinil improved learning and memory, reduced proinflammatory factor (IL-1β, TNF-α, and IL-6) production, and increased the expression of anti-inflammatory factors (IL-10). Modafinil treatment attenuated inflammasome activity and reduced neuronal pyroptosis involving the NLRP3/NLRP1/NLRC4-caspase-1-IL-1β pathway. In addition, modafinil induced an upregulation of brain-derived neurotrophic factor (BDNF) and synaptic activity.
These results suggest that modafinil reduces neuronal pyroptosis and cognitive decline following SD. These effects should be further investigated in future studies to benefit patients with sleep disorders.
We feel that we perceive our environment in real-time, despite the constraints imposed by neural transmission delays.
Due to these constraints, the intuitive view of perception in real-time is impossible to implement.
I propose a new way of thinking about real-time perception, in which perceptual mechanisms represent a timeline, rather than a single timepoint.
In this proposal, predictive mechanisms predict ahead to compensate for neural delays, and work in tandem with postdictive mechanisms that revise the timeline as additional sensory information becomes available.
Building on recent theoretical, computational, psychophysical, and functional neuroimaging evidence, this conceptualization of real-time perception for the first time provides an integrated explanation for how we can experience the present.
[cf. Bachmann 2013] We feel that we perceive events in the environment as they unfold in real-time. However, this intuitive view of perception is impossible to implement in the nervous system due to biological constraints such as neural transmission delays. I propose a new way of thinking about real-time perception: at any given moment, instead of representing a single timepoint, perceptual mechanisms represent an entire timeline. On this timeline, predictive mechanisms predict ahead to compensate for delays in incoming sensory input, and reconstruction mechanisms retroactively revise perception when those predictions do not come true. This proposal integrates and extends previous work to address a crucial gap in our understanding of a fundamental aspect of our everyday life: the experience of perceiving the present.
[Keywords: perception, time, prediction, real-time, neural delays]
…Postdiction reconstructs the perceptual past: A key feature of this proposal is that the perceptual timeline can be updated, revised, reinterpreted, and overwritten as new information (sensory or otherwise) becomes available. This means that the subjective experience of past events can be affected by later events. Importantly, in this account, these postdictive mechanisms do not violate the law of causality because it is the represented past, not the physical past, that is revised.
Figure 1 illustrates how this allows the presentation of a second disc to affect the perception of events leading up that event in the Colour Phi effect. In this phenomenon, observers view 2 differently coloured discs presented in different positions in quick succession (Figure 1A). This creates the percept of a single disc jumping from one position to the other, changing colour midway. As in Box 1, rows in Figure 1B indicate the contents of the perceptual timeline at the 3 (physical) timepoints t0, t1, and t2. Broken squares indicate timepoints for which sensory input is not yet available, and asterisks mark the represented present.
At t0, the first available sensory evidence indicates that a disc has been detected. This is represented at the appropriate moment. Future representations may also be activated, depending on prior expectations of the disc’s duration. At t1, subsequent sensory evidence suggests the disc was an isolated flash. Any earlier prediction is discarded and empty space is represented for the moments following the flash. When the second disc is detected at t2, the timeline as a whole is postdictively reinterpreted as a moving disc. The timeline is revised, such that the disc is represented in intervening locations at intermediate moments.
…A final implication of the current proposal is that there is no hard natural boundary between perception and memory. Rather, there is a continuum between the 2: as perceptual representations become older, they become degraded, compressed, or summarised, gradually becoming experiences of a past event in a way that is typically called episodic memory. This continuum between perception and memory is consistent with previous discussions of consciousness more broadly 26 and postdiction specifically , where retroactive revisions of past events are known to take place on timescales ranging from subsecond [14,27] to months or years .
We analysed quantity discrimination data from 672 subjects across 33 bird and mammal species, using a novel Bayesian model that combined phylogenetic regression with a model of number psychophysics and random effect components. This allowed us to combine data from 49 studies and calculate the Weber fraction (a measure of quantity representation precision) for each species. We then examined which cognitive, socioecological and biological factors were related to variance in Weber fraction.
We found contributions of phylogeny to quantity discrimination performance across taxa. Of the neural, socioecological and general cognitive factors we tested, cortical neuron density and domain-general cognition were the strongest predictors of Weber fraction, controlling for phylogeny. Our study is a new demonstration of evolutionary constraints on cognition, as well as of a relation between species-specific neuron density and a particular cognitive ability.
…Quantitative sensitivity is an aspect of cognition that is ubiquitous among many species, and many researchers debate the nature of its evolutionary basis across taxa, including in humans and other primates.1–6Baboons use numerical estimation to guide troop movement,7,8desert ants and fiddler crabs navigate by keeping track of the number of steps they have taken,9,10 and social species like hyenas and lions vocalize or approach other conspecific groups only when their group has a numerical advantage.11–17 A diverse range of animals—from primates to reptiles, fish and insects—can discriminate numerical quantities in laboratory tasks, for example, comparing computerized arrays or sequences of pure tones to peck, touch or approach the numerically larger set.2,4,18–22 Moreover, animals represent numerical values cross-modally23–27 and under conditions where dimensions such as area, density and duration are equated, uncorrelated with numerical value or otherwise controlled.2,4,20,28–30
…This is the first study to measure the origins of quantitative cognition with these methods. We found contributions of phylogeny to quantity discrimination performance across taxa, indicating evolutionary constraints on quantitative cognition. Additionally, a subset of neuronal and cognitive variables predicted species’ quantitative sensitivity—the strongest predictors were neuron density and general cognitive ability. The results indicate that when selecting an animal from the world at random, we can roughly predict its Weber fraction by knowing its species.
An individual’s Weber fraction was related to its species-typical cortical neuron density. Individuals from species with higher cortical neuron density had more precise Weber fractions. Thus, one constraint on an individual’s quantitative cognition is the biological capacity for information processing in their brain, as determined genetically and developmentally for each species.74–81 Additionally, quantitative precision was related to neuron density in the cerebellum, a brain structure that has been overlooked in studies of vertebrate brain evolution and cognition.89–91 We found that density of neurons was a more accurate proxy for quantitative ability than brain volume when comparing species across taxa. Caveats to the interpretation of the neuron density findings include (1) the number of species with cortical and cerebellar neuron density values is lower than for neuronal number, and (2) the relationship between neuron number and neuron density differs across animal groups. An increase in number of neurons in a primate brain structure does not mean larger neurons (and lower density), whereas in most non-primate mammals more neurons means larger neurons and lower density.75,115 Our study is a rare demonstration of a relation between neuron number or neuron density and a particular cognitive ability. Previous within-species comparisons showed that neuron number in multiple brain regions did not predict performance on a battery of behavioural tasks in mice,135 and though raccoons who performed best on a puzzle box task had more cells in their hippocampus than lower performing individuals, this difference may have been driven by glial cells.136 However, cross-species comparisons in primates and birds suggest that neuron number has more behavioural explanatory power than cranial capacity, based on the correlation between cortical or pallial neuron number and performance on a self-control task.71 Our cross-species finding from birds and mammals implies that quantitative sensitivity is yoked to species-specific developmental programmes for neuronal density; therefore, some species are well-equipped to develop precise quantitative sensitivity whereas others may be unable to do so.
Our finding that primate species’ quantitative sensitivity improved with their domain-general cognition score indicates that general cognitive functions, perhaps in tandem with specialized quantitative functions, impacted the evolution of quantitative precision across species.
…Our novel analysis shows that biological features of a species’ evolutionary history likely modulate the development of individuals’ numerical cognition, a crucial finding that emphasizes the importance of phylogenetic constraints on cognition. natural selection has biologically prepared some species to develop high neuronal densities and general cognitive capacities that yield precise quantitative representations. These data begin to reveal the evolutionary pressures that shaped numerical cognition across species and bring us closer to understanding the evolutionary precursors that sparked human mathematical cognition.
The principled design and discovery of biologically/physically-informed models of neuronal dynamics has been advancing since the mid-twentieth century. Recent developments in artificial intelligence (AI) have accelerated this progress. This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis/data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.
Larger brains should be adaptive because they support numerous ecological and socio-cognitive benefits, but these benefits explain only a modest part of the interspecific variation in brain size. Notably underexplored are the high energetic costs of developing brains, and thus the possible role of parental provisioning in the evolution of adult brain size.
We explore this idea in birds, which show considerable variation in both socio-ecological traits and the energy transfer from parents to offspring. Comparative analyses of 1,176 bird species show that the combination of adult body mass, mode of development at hatching, relative egg mass, and the time spent provisioning the young in combination strongly predict relative brain size across species. Adding adult ecological and socio-cognitive predictors only marginally adds explanatory value.
We therefore conclude that parental provisioning enabled bird species to evolve into skill-intensive niches, reducing interspecific competition and consequently promoting survival prospects and population stability. Critically, parental provisioning also explains why precocial bird species have smaller brains than altricial ones. Finally, these results suggest that the cognitive adaptations that provide the behavioral flexibility to improve reproductive success and survival are intrinsically linked to successful parental provisioning. Our findings also suggest that the traditionally assessed cognitive abilities may not predict relative brain size.
Predictive coding is an unifying framework for understanding perception, action and neocortical organization. In predictive coding, different areas of the neocortex implement a hierarchical generative model of the world that is learned from sensory inputs. Cortical circuits are hypothesized to perform Bayesian inference based on this generative model. Specifically, the Rao-Ballard hierarchical predictive coding model assumes that the top-down feedback connections from higher to lower order cortical areas convey predictions of lower-level activities. The bottom-up, feedforward connections in turn convey the errors between top-down predictions and actual activities. These errors are used to correct current estimates of the state of the world and generate new predictions. Through the objective of minimizing prediction errors, predictive coding provides a functional explanation for a wide range of neural responses and many aspects of brain organization.
Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggest that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed “representational drift”. In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift most often occurs along directions that have the most variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, eg. preventing overfitting.
WM increases until puberty but puberty occurs at half the age for Pan as for humans
Claims for extraordinary working memory in Pan are not supported by data
WM increase during hominin evolution parallels complexity increase in stone artifacts
Cumulative WM changes in Homo sapiens evolution led to qualitative cognitive changes
In this article we review publications relevant to addressing widely reported claims in both the academic and popular press that chimpanzees working memory (WM) is comparable to, if not exceeding, that of humans. WM is a complex multidimensional construct with strong parallels in humans to prefrontal cortex and cognitive development. These parallels occur in chimpanzees, but to a lesser degree.
We review empirical evidence and conclude that the size of WM in chimpanzees is 2 ± 1 versus Miller’s famous 7 ± 2 in humans. Comparable differences occur in experiments on chimpanzees relating to strategic and attentional WM subsystems. Regardless of the domain, chimpanzee WM performance is comparable to that of humans around the age of 4 or 5.
Next, we review evidence showing parallels among the evolution of WM capacity in hominins ancestral to Homo sapiens, the phylogenetic evolution of hominins leading to Homo sapiens, and evolution in the complexity of stone tool technology over this time period.
[Keywords: working memory, human evolution, cognitive evolution, comparative psychology, chimpanzee, hominin evolution, theory of mind, planning]
Almost all animals must make decisions on the move. Here, employing an approach that integrates theory and high-throughput experiments (using state-of-the-art virtual reality), we reveal that there exist fundamental geometrical principles that result from the inherent interplay between movement and organisms’ internal representation of space. Specifically, we find that animals spontaneously reduce the world into a series of sequential binary decisions, a response that facilitates effective decision-making and is robust both to the number of options available and to context, such as whether options are static (eg. refuges) or mobile (eg. other animals). We present evidence that these same principles, hitherto overlooked, apply across scales of biological organization, from individual to collective decision-making.
Choosing among spatially distributed options is a central challenge for animals, from deciding among alternative potential food sources or refuges to choosing with whom to associate. Using an integrated theoretical and experimental approach (employing immersive virtual reality), we consider the interplay between movement and vectorial integration during decision-making regarding 2, or more, options in space.
In computational models of this process, we reveal the occurrence of spontaneous and abrupt “critical” transitions (associated with specific geometrical relationships) whereby organisms spontaneously switch from averaging vectorial information among, to suddenly excluding one among, the remaining options. This bifurcation process repeats until only one option—the one ultimately selected—remains. Thus, we predict that the brain repeatedly breaks multi-choice decisions into a series of binary decisions in space-time.
Experiments with fruit flies, desert locusts, and larval zebrafish reveal that they exhibit these same bifurcations, demonstrating that across taxa and ecological contexts, there exist fundamental geometric principles that are essential to explain how, and why, animals move the way they do.
[Keywords: ring attractor, movement ecology, navigation, collective behavior, embodied choice]
Alchemy is a new meta-learning environment rich enough to contain interesting abstractions, yet simple enough to make fine-grained analysis tractable. Further, Alchemy provides an optional symbolic interface that enables meta-RL research without a large compute budget. In this work, we take the first steps toward using Symbolic Alchemy to identify design choices that enable deep-RL agents to learn various types of abstraction. Then, using a variety of behavioral and introspective analyses we investigate how our trained agents use and represent abstract task variables, and find intriguing connections to the neuroscience of abstraction. We conclude by discussing the next steps for using meta-RL and Alchemy to better understand the representation of abstract variables in the brain.
Language is an epiphenomenon of human’s subjective world which is noted as the conceptual network. Human beings realized communication of knowledge, experience, and symbolic entity of subjective psyche, across time and space by language which included but not limited to spoken or writing systems. From the perspective of computational linguistics, one concept in the conceptual network would be identically activated despite variations of modalities (ie. comprehension, generation or production).
In the current study, we conducted a semantic-access word reading task (language comprehension) and a word imagining task (language generation) in Chinese native speakers during fMRI scanning. Part-of-speech category and lexicon of stimuli in word imagining task were predicted by brain responses in the word reading task.
Importantly, our learning model, which was trained from brain activation of word reading, achieved decoding both imagined words and semantically transferred imagined words.
To our knowledge, this is the first report of cross-modality and semantics transferring decoding of imagined speech. Given the huge processing discrepancies between language comprehension and generation, our results demonstrated a stable conceptual network in the human brain and flexible access from linguistic ways to conceptual network, which shed light on understanding brain mechanisms of the relationship between language and thought.
The brain is a metabolically fragile organ as compromises in fuel availability rapidly degrade cognitive function. Nerve terminals are likely loci of this vulnerability as they do not store sufficient AT molecules, needing to synthesize them during activity or suffer acute degradation in performance. The ability of on-demand ATP synthesis to satisfy activity-driven ATP hydrolysis will depend additionally on the magnitude of local resting metabolic processes.
We show here that synaptic vesicle (SV) pools are a major source of presynaptic basal energy consumption. This basal metabolic processes arises from SV-resident V-ATPases compensating for a hidden resting H⁺ efflux from the SV lumen. We show that this steady-state H⁺ efflux (1) is mediated by vesicular neurotransmitter transporters, (2) is independent of the SV cycle, (3) accounts for up to 44% of the resting synaptic energy consumption, and (4) contributes substantially to nerve terminal intolerance of fuel deprivation.
Integrating neurons into digital systems to leverage their innate intelligence may enable performance infeasible with silicon alone, along with providing insight into the cellular origin of intelligence.
We developed DishBrain, a system which exhibits natural intelligence by harnessing the inherent adaptive computation of neurons in a structured environment. In vitro neural networks from human or rodent origins, are integrated with in silico computing via high-density multielectrode array. Through electrophysiological stimulation and recording, cultures were embedded in a simulated game-world, mimicking the arcade game Pong.
Applying a previously untestable theory of active inference via the Free Energy Principle, we found that learning was apparent within 5 minutes of real-time gameplay, not observed in control conditions. Further experiments demonstrate the importance of closed-loop structured feedback in eliciting learning over time.
Cultures display the ability to self-organize in a goal-directed manner in response to sparse sensory information about the consequences of their actions.
Large, open datasets have emerged as important resources in the field of human connectomics. In this review, the evolution of data sharing involving magnetic resonance imaging is described.
A summary of the challenges and progress in conducting reproducible data analyses is provided, including description of recent progress made in the development of community guidelines and recommendations, software and data management tools, and initiatives to enhance training and education. Finally, this review concludes with a discussion of ethical conduct relevant to analyses of large, open datasets and a researcher’s responsibility to prevent further stigmatization of historically marginalized racial and ethnic groups.
Moving forward, future work should include an enhanced emphasis on the social determinants of health, which may further contextualize findings among diverse population-based samples. Leveraging the progress to date and guided by interdisciplinary collaborations, the future of connectomics promises to be an impressive era of innovative research, yielding a more inclusive understanding of brain structure and function.
[Keywords: connectomics, large open datasets, neuroimaging data sharing, reproducible analytics]
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
Computational Deep Language Models (DLMs) have been shown to be effective in predicting neural responses during natural language processing. This study introduces a novel computational framework, based on the concept of fine-tuning (Hinton, 2007), for modeling differences in interpretation of narratives based on the listeners9 perspective (ie. their prior knowledge, thoughts, and beliefs). We draw on an fMRI experiment conducted by Yeshurun et al 2017, in which two groups of listeners were listening to the same narrative but with two different perspectives (cheating versus paranoia). We collected a dedicated dataset of ~3000 stories, and used it to create two modified (fine-tuned) versions of a pre-trained DLM, each representing the perspective of a different group of listeners. Information extracted from each of the two fine-tuned models was better fitted with neural responses of the corresponding group of listeners. Furthermore, we show that the degree of difference between the listeners9 interpretation of the story—as measured both neurally and behaviorally—can be approximated using the distances between the representations of the story extracted from these two fine-tuned models. These models-brain associations were expressed in many language-related brain areas, as well as in several higher-order areas related to the default-mode and the mentalizing networks, therefore implying that computational fine-tuning reliably captures relevant aspects of human language comprehension across different levels of cognitive processing.
Recent work suggests that feature constraints in the training datasets of deep neural networks (DNNs) drive robustness to adversarial noise (Ilyas et al 2019). The representations learned by such adversarially robust networks have also been shown to be more human perceptually-aligned than non-robust networks via image manipulations (Santurkar et al 2019, Engstrom et al 2019). Despite appearing closer to human visual perception, it is unclear if the constraints in robust DNN representations match biological constraints found in human vision. Human vision seems to rely on texture-based/summary statistic representations in the periphery, which have been shown to explain phenomena such as crowding (Balas et al 2009) and performance on visual search tasks (Rosenholtz et al 2012). To understand how adversarially robust optimizations/representations compare to human vision, we performed a psychophysics experiment using a metamer task similar to Freeman & Simoncelli, 2011, Wallis et al 2016 and Deza et al 2019 where we evaluated how well human observers could distinguish between images synthesized to match adversarially robust representations compared to non-robust representations and a texture synthesis model of peripheral vision (Texforms a la Long et al 2018). We found that the discriminability of robust representation and texture model images decreased to near chance performance as stimuli were presented farther in the periphery. Moreover, performance on robust and texture-model images showed similar trends within participants, while performance on non-robust representations changed minimally across the visual field. These results together suggest that (1) adversarially robust representations capture peripheral computation better than non-robust representations and (2) robust representations capture peripheral computation similar to current state-of-the-art texture peripheral vision models. More broadly, our findings support the idea that localized texture summary statistic representations may drive human invariance to adversarial perturbations and that the incorporation of such representations in DNNs could give rise to useful properties like adversarial robustness.
Self-supervised learning is a powerful paradigm for representation learning on unlabeled images. A wealth of effective new methods based on instance matching rely on data augmentation to drive learning, and these have reached a rough agreement on an augmentation scheme that optimizes popular recognition benchmarks. However, there is strong reason to suspect that different tasks in computer vision require features to encode different (in)variances, and therefore likely require different augmentation strategies. In this paper, we measure the invariances learned by contrastive methods and confirm that they do learn invariance to the augmentations used and further show that this invariance largely transfers to related real-world changes in pose and lighting. We show that learned invariances strongly affect downstream task performance and confirm that different downstream tasks benefit from polar opposite (in)variances, leading to performance loss when the standard augmentation strategy is used. Finally, we demonstrate that a simple fusion of representations with complementary invariances ensures wide transferability to all the diverse downstream tasks considered.
Patterns of endogenous activity in the brain reflect a stochastic exploration of the neuronal state space that is constrained by the underlying assembly organization of neurons. Yet it remains to be shown that this interplay between neurons and their assembly dynamics indeed suffices to generate whole-brain data statistics.
Here we recorded the activity from ~40,000 neurons simultaneously in zebrafish larvae, and show that a data-driven network model of neuron-assembly interactions can accurately reproduce the mean activity and pairwise correlation statistics of their spontaneous activity. This model, the compositional Restricted Boltzmann Machine, unveils ~200 neural assemblies, which compose neurophysiological circuits and whose various combinations form successive brain states. From this, we mathematically derived an interregional functional connectivity matrix, which is conserved across individual animals and correlates well with structural connectivity.
This novel, assembly-based generative model of brain-wide neural dynamics enables physiology-bound perturbation experiments in silico.
The mosaic brain evolution hypothesis, stating that brain regions can evolve relatively independently during cognitive evolution, is an important idea to understand how brains evolve with potential implications even for human brain evolution.
Here, we provide the first experimental evidence for this hypothesis through an artificial selection experiment in the guppy (Poecilia reticulata).
After 4 generations of selection on relative telencephalon volume (relative to brain size), we found substantial changes in telencephalon size but no changes in other regions. Further comparisons revealed that up-selected lines had larger telencephalon, while down-selected lines had smaller telencephalon than wild Trinidadian populations.
Our results support that independent evolutionary changes in specific brain regions through mosaic brain evolution can be important facilitators of cognitive evolution…The rate of evolution shown here is similar to that seen when brain size was the target of selection (24); a 9% difference in brain mass between the large-brain and small-brain lines was found after 2 generations of artificial selection and a 15% difference after 5 generations of selection (50). In contrast to the aforementioned artificial selection experiment on relative brain size in guppies, which found a clear reduction in fecundity (ie. lower offspring number) in the large-brain lines (24), we did not find any evidence for any reproductive trade-off with telencephalon size
The biophysical properties of neurons are the foundation for computation in the brain. Neuronal size is a key determinant of single neuron input-output features and varies substantially across species. However, it is unknown whether different species adapt neuronal properties to conserve how single neurons process information.
In 9 of the 10 species, we observe conserved rules that control the conductance of voltage-gated potassium and HCN channels. Species with larger neurons, and therefore a decreased surface-to-volume ratio, exhibit higher membrane ionic conductances. This relationship produces a conserved conductance per unit brain volume. These size-dependent rules result in large but predictable changes in somatic and dendritic integrative properties. Human neurons do not follow these allometric relationships, exhibiting much lower voltage-gated potassium and HCN conductances.
Together, our results in layer 5 neurons identify conserved evolutionary principles for neuronal biophysics in mammals as well as notable features of the human cortex.
In order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream.
To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age.
Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells.
Together our results imply that optimizing the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain.
Genetically encoded DNA recorders noninvasively convert transient biological events into durable mutations in a cell’’s genome, allowing for the later reconstruction of cellular experiences using high-throughput DNA sequencing. Existing DNA recorders have achieved high-information recording, durable recording, prolonged recording over multiple timescales, multiplexed recording of several user-selected signals, and temporally resolved signal recording, but not all at the same time. We present a DNA recorder called peCHYRON (prime editing Cell HistorY Recording by Ordered iNsertion) that does. In peCHYRON, prime editor guide RNAs (pegRNAs) insert a variable triplet DNA sequence alongside a constant propagation sequence that deactivates the previous and activates the next step of insertion. This process results in the sequential accumulation of regularly spaced insertion mutations at a synthetic locus. Accumulated insertions are permanent throughout editing because peCHYRON uses a prime editor that avoids cutting both DNA strands, which risks deletions. Editing continues indefinitely because each insertion adds the complete sequence needed to initiate the next step. Constitutively expressed pegRNAs generate insertion patterns that support straightforward reconstruction of cell lineage relationships. Pulsed expression of different pegRNAs enables the reconstruction of pulse sequences, which may be coupled to biological stimuli for temporally-resolved multiplexed event recording.
Measurements of gene expression and signal transduction activity are conventionally performed with methods that require either the destruction or live imaging of a biological sample within the timeframe of interest.
Here we demonstrate an alternative paradigm, termed ENGRAM (ENhancer-driven Genomic Recording of transcriptional Activity in Multiplex), in which the activity and dynamics of multiple transcriptional reporters are stably recorded to DNA. ENGRAM is based on the prime editing-mediated insertion of signal-specific or enhancer-specific barcodes to a genomically encoded recording unit.
We show how this strategy can be used to concurrently genomically record the relative activity of at least hundreds of enhancers with high fidelity, sensitivity and reproducibility. Leveraging synthetic enhancers that are responsive to specific signal transduction pathways, we further demonstrate time-dependent and concentration-dependent genomic recording of Wnt, NF-κB, and Tet-On activity.
Finally, by coupling ENGRAM to sequential genome editing, we show how serially occurring molecular events can potentially be ordered.
Looking forward, we envision that multiplex, ENGRAM-based recording of the strength, duration and order of enhancer and signal transduction activities has broad potential for application in functional genomics, developmental biology and neuroscience.
DNA is naturally well-suited to serve as a digital medium for in vivo molecular recording. However, DNA-based memory devices described to date are constrained in terms of the number of distinct signals that can be concurrently recorded and/or by a failure to capture the precise order of recorded events.
Here we describe DNA Ticker Tape, a general system for in vivo molecular recording that largely overcomes these limitations. Blank DNA Ticker Tape consists of a tandem array of partial CRISPR-Cas9 target sites, with all but the first site truncated at their 59 ends, and therefore inactive. Signals of interest are coupled to the expression of specific prime editing guide RNAs. Editing events are insertional, and record the identity of the guide RNA mediating the insertion while also shifting the position of the “write head” by one unit along the tandem array, i.e. sequential genome editing.
In this proof-of-concept of DNA Ticker Tape, we demonstrate the recording and decoding of complex event histories or short text messages; evaluate the performance of dozens of orthogonal tapes; and construct “long tape” potentially capable of recording the order of as many as 20 serial events. Finally, we demonstrate how DNA Ticker Tape simplifies the decoding of cell lineage histories.
[brain video] Imaging intact human organs from the organ to the cellular scale in 3 dimensions is a goal of biomedical imaging.
To meet this challenge, we developed hierarchical phase-contrast tomography (HiP-CT), an X-ray phase propagation technique using the European Synchrotron Radiation Facility (ESRF)’s Extremely Brilliant Source (EBS). The spatial coherence of the ESRF-EBS combined with our beamline equipment, sample preparation and scanning developments enabled us to perform non-destructive, 3-dimensional (3D) scans with hierarchically increasing resolution at any location in whole human organs.
We applied HiP-CT to image 5 intact human organ types: brain, lung, heart, kidney and spleen. HiP-CT provided a structural overview of each whole organ followed by multiple higher-resolution volumes of interest, capturing organotypic functional units and certain individual specialized cells within intact human organs. We demonstrate the potential applications of HiP-CT through quantification and morphometry of glomeruli in an intact human kidney and identification of regional changes in the tissue architecture in a lung from a deceased donor with coronavirus disease 2019 (COVID-19).
Cognitive variation is common among-individuals within populations, and this variation can be consistent across time and context. From an evolutionary perspective, among-individual variation is important and required for natural selection. Selection has been hypothesised to favour high cognitive performance, however directional selection would be expected to erode variation over time. Additionally, while variation is a prerequisite for natural selection, it is also true that selection does not act on traits in isolation. Thus, the extent to which performance covaries among specific cognitive domains, and other aspects of phenotype (eg. personality traits) is expected to be an important factor in shaping evolutionary dynamics. Fitness trade-offs could shape patterns of variation in performance across different cognitive domains, however positive correlations between cognitive domains and personality traits are also known to occur. Here we aimed to test this idea using a multivariate approach to characterise and test hypothesised relationships of cognitive performance across multiple domains and personality, in the Trinidadian guppy (Poecilia reticulata). We estimate the among-individual correlation matrix (ID) in performance across three cognitive domains; association learning in a colour discrimination task; motor cognition in a novel motor task and cognitive flexibility in a reversal learning task, and the personality trait boldness, measured as time to emerge. We found no support for trade-offs occurring, but the presence of strong positive domain-general correlations in ID, where 57% of the variation is explained by the leading eigen vector. While highlighting caveats of how non-cognitive factors and assay composition may affect the structure of the ID-matrix, we suggest that our findings are consistent with a domain-general axis of cognitive variation in this population, adding to the growing body of support for domain-general variation among-individuals in animal cognitive ability.
Cerebellar volume is highly heritable and associated with neurodevelopmental and neurodegenerative disorders. Understanding the genetic architecture of cerebellar volume may improve our insight into these disorders. This study aims to investigate the convergence of cerebellar volume genetic associations in close detail. A genome-wide associations study for cerebellar volume was performed in a sample of 27,486 individuals from UK Biobank, resulting in 29 genome-wide statistically-significant loci and a SNP heritability of 39.82%. We pinpoint variants that have effects on amino acid sequence or cerebellar gene-expression. Additionally, 85 genome-wide statistically-significant genes were detected and tested for convergence onto biological pathways, cerebellar cell types or developmental stages. Local genetic correlations between cerebellar volume and neurodevelopmental and neurodegenerative disorders reveal shared loci with Parkinson’’s disease, Alzheimer’s disease and schizophrenia. These results provide insights into the heritable mechanisms that contribute to developing a brain structure important for cognitive functioning and mental health.
Food-caching birds use spatial cognition to recover food stores and survive winter
Variation in cognitive phenotypes is associated with variation across the genome
Top outlier genes are associated with hippocampal development and function
Results link cognitive and genetic variation, making it available for selection
Spatial cognition is used by most organisms to navigate their environment. Some species rely particularly heavily on specialized spatial cognition to survive, suggesting that a heritable component of cognition may be under natural selection. This idea remains largely untested outside of humans, perhaps because cognition in general is known to be strongly affected by learning and experience.
We investigated the genetic basis of individual variation in spatial cognition used by non-migratory food-caching birds to recover food stores and survive harsh montane winters.
Comparing the genomes of wild, free-living birds ranging from best to worst in their performance on a spatial cognitive task revealed statistically-significant associations with genes involved in neuron growth and development and hippocampal function.
These results identify candidate genes associated with differences in spatial cognition and provide a critical link connecting individual variation in spatial cognition with natural selection.
…We used whole-genome sequencing, combining traditional genome-wide association studies (GWASs) and a Random Forest machine learning approach, to compare the genomes of wild, free-living birds. We sampled birds from high and low elevations that performed the best on a spatial cognitive task, all of whom survived more than one year (n = 22), to those that performed the worst on the task and generally did not survive more than 1 year (n = 15⁄20)—the group with better spatial cognition was associated with a statistically-significant survival advantage (Fisher’s exact test, p < 0.001). Birds from both high and low elevations were selected for each performance group to ensure that the strongest signal between groups was variation in cognition and not a correlate of elevation…We sampled 42 mountain chickadees across 3 years of testing from the extremes of the cognitive performance range: 22 were chosen as the best and 20 were chosen as the worst. Performance scores of individuals in the best and worst groups did not overlap, but individual variation within each group provided a continuous distribution from best to worst (Figure 1F). We intentionally chose individuals with means in the tails of the cognitive performance distribution (Figures 1E and 1F) to amplify the signal of genetic associations, although we acknowledge that this design could inflate associations from loci with the largest effect to the detriment of small-effect polygenes. There was a statistically-significant difference in the mean number of errors per trial between best (mean errors/trial: 0.16 ± 0.045) and worst (0.60 ± 0.05) performers (cognitive category [best versus worst]: F1,38 = 72.91, p < 0.0001; Figure 1F), but there was not a statistically-significant effect of elevation, as we selectively picked the best and worst performers at each elevation (elevation [high versus low]: F1,38 = 1.38, p = 0.247; total trials completed [covariate]: F1,38 = 5.26, p = 0.028).
Recording the activity of the same neurons over the adult life of an animal is important to neuroscience research and biomedical applications. Current implantable devices cannot provide stable recording on this time scale. Here, we introduce a method to precisely implant nanoelectronics with an open, unfolded mesh structure across multiple brain regions in the mouse. The open mesh structure forms a stable interwoven structure with the neural network, preventing probe drifting and showing no immune response and neuron loss during the yearlong implantation. Using the implanted nanoelectronics, we can track single-unit action potentials from the same neurons over the entire adult life of mice. Leveraging the stable recordings, we build machine learning algorithms that enable automated spike sorting, noise rejection, stability validation, and generate pseudotime analysis, revealing aging-associated evolution of the single-neuron activities.
Focal cortical lesions lead to local, not global, deficits.
Measurement models to explain the positive manifold are causal models with unique predictions going beyond model fit statistics.
Correlated factor, network, process sampling, mutualism, investment models, make causal predictions inconsistent with lesion evidence.
Hierarchical and bifactor models are consistent with the pattern of lesion effects, as well as possibly one form of bonds sampling models.
Future models and explanations of the positive manifold have to accommodate focal lesions leading to local not global deficits.
Here we examine 3 classes of models regarding the structure of human cognition: common cause models, sampling/network models, and interconnected models. That disparate models can accommodate one of the most globally replicated psychological phenomena—namely, the positive manifold—is an extension of underdetermination of theory by data. Statistical fit indices are an insufficient and sometimes intractable method of demarcating between the theories; strict tests and further evidence should be brought to bear on understanding the potential causes of the positive manifold. The cognitive impact of focal cortical lesions allows testing the necessary causal connections predicted by competing models. This evidence shows focal cortical lesions lead to local, not global (across all abilities), deficits. Only models that can accommodate a deficit in a given ability without effects on other covarying abilities can accommodate focal lesion evidence. After studying how different models pass this test, we suggest bifactor models (class: common cause models) and bond models (class: sampling models) are best supported. In short, competing psychometric models can be informed when their implied causal connections and predictions are tested.
[Keywords: human intelligence, structural models, causality, statistical model fit, cortical lesions]
[This would seem to explain the failure of dual n-back & WM training in general.
Training the specific ability of WM could only cause g increases in models with ‘upwards causation’ like hierarchical models or dynamic mutual causation like mutualism/investment models; these are ruled out by the lesion literature which finds that physically-tiny lesions damage specific abilities but not g, and if decreasing a specific ability cannot decrease g, then it’s hard to see how increasing that ability could ever increase g. See also Lee et al 2019.]
In their comprehensive review of sex differences in the brain, Eliot et al 2021 conclude that (1) men and women substantially differ in global brain size, but this “mostly parallels the divergence of male/female body size during development” and that (2) “once we account for individual differences in brain size, there is almost no difference in the volume of specific cortical or subcortical structures between men and women”. In sum, almost all brain differences would directly or indirectly follow from differences in body size.
In a recent study that does not have the same limitations as most studies reviewed by Eliot et al 2021, we find that sex differences in total brain volume are not accounted for by sex differences in height and weight, and that once global brain size is taken into account, there remain numerous regional sex differences in both directions (Williams et al 2021a).
[Keywords: sex differences, brain volumes, cortical mean thicknesses, cortical surface areas]
Parrots are well-known for their exceptionally long lives and cognitive complexity. While previous studies have demonstrated a correlation between longevity and brain size in a variety of taxa, little research has been devoted to understanding this link in parrots.
Here we employed a large-scale comparative analysis that investigated the influence of brain size and life history variables on patterns of longevity. Specifically, we addressed two hypotheses for evolutionary drivers of longevity: the Cognitive Buffer Hypothesis, which proposes that increased cognitive abilities enable longer life spans, and the Expensive Brain Hypothesis, which holds that the increase in life span is caused by prolonged developmental time of and increased parental investment in, large brained offspring.
We estimated life expectancy from detailed zoo records for 133,818 individuals across 244 parrot species. Using Bayesian structural equation models, we found a consistent correlation between relative brain size and life expectancy in parrots. This correlation was best explained by a direct effect of relative brain size. Notably, we found no effects of developmental time, clutch size, or age at first reproduction.
Our results provide support for the Cognitive Buffer Hypothesis, and demonstrate a principled Bayesian approach that addresses data uncertainty and imputation of missing values.
Urbanicity is a growing environmental challenge for mental health.
Here, we investigate correlations of urbanicity with brain structure and function, neuropsychology and mental illness symptoms in young people from China and Europe (total n = 3,867). We developed a remote-sensing satellite measure (UrbanSat) to quantify population density at any point on Earth.
UrbanSat estimates of urbanicity were correlated with brain volume, cortical surface area and brain network connectivity in the medial prefrontal cortex and cerebellum. UrbanSat was also associated with perspective-taking and depression symptoms, and this was mediated by neural variables. Urbanicity effects were greatest when urban exposure occurred in childhood for the cerebellum, and from childhood to adolescence for the prefrontal cortex.
As UrbanSat can be generalized to different geographies, it may enable assessments of correlations of urbanicity with mental illness and resilience globally.
Fruit flies are capable of sophisticated behaviors, including navigating diverse landscapes, tussling with rivals and serenading potential mates. And their speck-size brains are tremendously complex, containing some 100,000 neurons and tens of millions of connections, or synapses, between them.
Since 2014, a team of scientists at Janelia, in collaboration with researchers at Google, have been mapping these neurons and synapses in an effort to create a comprehensive wiring diagram, also known as a connectome, of the fruit fly brain.
The work, which is continuing, is time-consuming and expensive, even with the help of state-of-the-art machine-learning algorithms. But the data they have released so far is stunning in its detail, composing an atlas of tens of thousands of gnarled neurons in many crucial areas of the fly brain.
By analyzing the connectome of just a small part of the fly brain—the central complex, which plays an important role in navigation—Dr. Jayaraman and his colleagues identified dozens of new neuron types and pinpointed neural circuits that appear to help flies make their way through the world. The work could ultimately help provide insight into how all kinds of animal brains, including our own, process a flood of sensory information and translate it into appropriate action.
It is also a proof of principle for the young field of modern connectomics, which was built on the promise that constructing detailed diagrams of the brain’s wiring would pay scientific dividends. “It’s really extraordinary”, Dr. Clay Reid, a senior investigator at the Allen Institute for Brain Science in Seattle, said of the new paper. “I think anyone who looks at it will say connectomics is a tool that we need in neuroscience—full stop.”
…Several teams at Janelia have embarked on fly connectome projects in the years since, but the work that led to the new paper began in 2014, with the brain of a single, 5-day-old female fruit fly. Researchers cut the fly brain into slabs and then used a technique known as focused-ion beamscanning electron microscopy to image them, layer by painstaking layer. The microscope essentially functioned like a very tiny, very precise nail file, filing away an exceedingly thin layer of the brain, snapping a picture of the exposed tissue and then repeating the process until nothing remained…The team then used computer vision software to stitch the millions of resulting images back together into a single, 3-dimensional volume and sent it off to Google. There, researchers used advanced machine-learning algorithms to identify each individual neuron and trace its twisting branches. Finally, the Janelia team used additional computational tools to pinpoint the synapses, and human researchers proofread the computers’ work, correcting errors and refining the wiring diagrams.
…Last year, the researchers published the connectome for what they called the “hemibrain”, a large portion of the central fly brain, which includes regions and structures that are crucial for sleep, learning and navigation.
The connectome, which is accessible free online, includes about 25,000 neurons and 20 million synapses, far more than the C. elegans connectome. “It’s a dramatic scaling up”, said Cori Bargmann, a neuroscientist at the Rockefeller University in New York. “This is a tremendous step toward the goal of working out the connectivity of the brain.”
…For instance, Hannah Haberkern, a postdoctoral associate in Dr. Jayaraman’s lab, analyzed the neurons that send sensory information to the ellipsoid body, a doughnut-shape structure that acts as the fly’s internal compass. Dr. Haberkern found that neurons that are known to transmit information about the polarization of light—a global environmental cue that many animals use for navigation—made more connections to the compass neurons than did neurons that transmit information about other visual features and landmarks.
…Other members of the research team identified specific neural pathways that seem well suited to helping the fly keep track of its head and body orientation, anticipate its future orientation and traveling direction, calculate its current orientation relative to another desired location and then move in that direction.
…Creating connectomes of larger, more complex brains will be enormously challenging. The mouse brain contains roughly 70 million neurons, the human brain a whopping 86 billion. But the central complex paper is decidedly not an one-off; detailed studies of regional mouse and human connectomes are currently in the pipeline, Dr. Reid said: “There’s a lot more to come.” Journal editors, consider yourselves warned.
Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representations increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contrast, there are no image-computable models of the dorsal stream with comparable explanatory power.
We hypothesized that the properties of dorsal stream neurons could be explained by a simple learning objective: the need for an organism to orient itself during self-motion. To test this hypothesis, we trained a 3D ResNet to predict an agent’s self-motion parameters from visual stimuli in a simulated environment. We found that the responses in this network accounted well for the selectivity of neurons in a large database of single-neuron recordings from the dorsal visual stream of non-human primates. In contrast, ANNs trained on an action recognition dataset through supervised or self-supervised learning could not explain responses in the dorsal stream, despite also being trained on naturalistic videos with moving objects.
These results demonstrate that an ecologically relevant cost function can account for dorsal stream properties in the primate brain.
[media] Flexible behaviors over long timescales are thought to engage recurrent neural networks in deep brain regions, which are experimentally challenging to study. In insects, recurrent circuit dynamics in a brain region called the central complex (CX) enable directed locomotion, sleep, and context/experience-dependent spatial navigation.
We describe the first complete electron-microscopy-based connectome of the Drosophila CX, including all its neurons and circuits at synaptic resolution.
We identified new CX neuron types, novel sensory and motor pathways, and network motifs that likely enable the CX to extract the fly’s head-direction, maintain it with attractor dynamics, and combine it with other sensorimotor information to perform vector-based navigational computations. We also identified numerous pathways that may facilitate the selection of CX-driven behavioral patterns by context and internal state. The CX connectome provides a comprehensive blueprint necessary for a detailed understanding of network dynamics underlying sleep, flexible navigation, and state-dependent action selection.
…Here we analyzed the arborizations and connectivity of the ~3,000 CX neurons in version 1.1 of the ‘hemibrain’ connectome—a dataset with 25,000 semi-automatically reconstructed neurons and 20 million synapses from the central brain of a 5-day-old female fly (Scheffer et al 2020) (see Materials and Methods).
…EM circuit reconstruction: how complete is complete enough? The value of EM-level connectomes in understanding the function of neural circuits in small and large brains is widely appreciated (Abbott et al 2020; Litwin-Kumar & Turaga 2019; Schlegel et al 2017). Although recent technical advances have made it possible to acquire larger EM volumes (Scheffer et al 2020; Zheng et al 2018) and improvements in machine learning have enabled high-throughput reconstruction of larger neural circuits (Dorkenwald et al 2020; Januszewski et al 2018), the step from acquiring a volume to obtaining a complete connectome still requires considerable human proofreading and tracing effort (Scheffer et al 2020).
As part of our analysis of the CX connectome, we found that although increased proofreading led to an expected increase in the number of synaptic connections between neurons, it did not necessarily lead to substantial changes in the relative weight of connections between different neuron types (Figures 3–4). While it is important to note that we made comparisons between the hemibrain connectome at fairly advanced stages of proofreading in the CX, our results do suggest that it may be possible to obtain an accurate picture of neural circuit connectivity from incomplete reconstructions. It may be useful for future large scale connectomics efforts to incorporate similar validation steps of smaller sample volumes into reconstruction pipelines to determine appropriate trade-offs between accuracy and cost of proofreading.
The human brain is organised into networks of interconnected regions that have highly correlated volumes. In this study, we aim to triangulate insights into brain organisation and its relationship with cognitive ability and ageing, by analysing genetic data.
We estimated general genetic dimensions of human brain morphometry within the whole brain, and 9 predefined canonical brain networks of interest. We did so based on principal components analysis (PCA) of genetic correlations among grey-matter volumes for 83 cortical and subcortical regions (nparticipants = 36,778).
We found that the corresponding general dimension of brain morphometry accounts for 40% of the genetic variance in the individual brain regions across the whole brain, and 47–65% within each network of interest. This genetic correlation structure of regional brain morphometry closely resembled the phenotypic correlation structure of the same regions. Applying a novel multivariate methodology for calculating SNP effects for each of the general dimensions identified, we find that general genetic dimensions of morphometry within networks are negatively associated with brain age (rg = −0.34) and profiles characteristic of age-related neurodegeneration, as indexed by cross-sectional age-volume correlations (r = −0.27). The same genetic dimensions were positively associated with a genetic general factor of cognitive ability (rg = 0.17–0.21 for different networks).
We have provided a statistical framework to index general dimensions of shared genetic morphometry that vary between brain networks, and report evidence for a shared biological basis underlying brain morphometry, cognitive ability, and brain ageing, that are underpinned by general genetic factors.
…This indicates that the genetic association between brain morphometry and cognitive ability was not driven by specific network configurations. Instead, dimensions of shared genetic morphometry in general indexed genetic variance relevant to larger brain volumes and a brain organisation that is advantageous for better cognitive performance. This was regardless of how many brain regions and from which regions the measure of shared genetic morphometry was extracted. This lack of differentiation between networks, in how strongly they correlate with cognitive ability, is in line with the suggestion that the total number of neurons in the mammalian cortex, which should at least partly correspond to its volume, is a major predictor of higher cognitive ability.37 These findings suggest that highly shared brain morphometry between regions, and its genetic analogue, indicate a generally bigger, and cognitively better-functioning brain.
Predictive processing is emerging as a common computational hypothesis to account for diverse psychological functions subserved by a brain, providing a systems-level framework for characterizing structure-function relationships of its distinct substructures. Here, we contribute to this framework by examining gradients of functional connectivity as a low dimensional spatial representation of functional variation in the brain and demonstrating their computational implications for predictive processing. Specifically, we investigated functional connectivity gradients in the cerebral cortex, the cerebellum, and the hippocampus using resting-state functional MRI data collected from large samples of healthy young adults. We then evaluated the degree to which these structures share common principles of functional organization by assessing the correspondence of their gradients. We show that the organizing principles of these structures primarily follow two functional gradients consistent with the existing hierarchical accounts of predictive processing: A model-error gradient that describes the flow of prediction and prediction error signals, and a model-precision gradient that differentiates regions involved in the representation and attentional modulation of such signals in the cerebral cortex. Using these gradients, we also demonstrated triangulation of functional connectivity involving distinct subregions of the three structures, which allows characterization of distinct ways in which these structures functionally interact with each other, possibly subserving unique and complementary aspects of predictive processing. These findings support the viability of computational hypotheses about the functional relationships between the cerebral cortex, the cerebellum, and the hippocampus that may be instrumental for understanding the brain’s dynamics within its large-scale predictive architecture.
A core taken in a tree today can reveal climate events from centuries past. Here we adapt this idea to record histories of neural activation. We engineered slowly growing intracellular protein fibers which can incorporate diverse fluorescent marks during growth to store linear ticker tape-like histories. An embedded HaloTag reporter incorporated user-supplied HaloTag-ligand dyes, leading to colored stripes whose boundaries mapped fiber growth to wall-clock time. A co-expressed eGFP tag driven by the cFos immediate early gene promoter recorded the history of neural activity. High-resolution multispectral imaging on fixed samples read the cellular histories. We demonstrated recordings of cFos activation in ensembles of cultured neurons with a single-cell absolute accuracy of ~39 min over a 12-hour interval. Protein-based ticker tapes have the potential to achieve massively parallel single-cell recordings of multiple physiological modalities.
Brain decoding aims to infer cognitive states from recordings of brain activity. Current literature has mainly focused on isolated brain regions engaged in specific experimental conditions, but ignored the integrative nature of cognitive processes recruiting distributed brain networks. To tackle this issue, we propose a connectome-based graph neural network to integrate distributed patterns of brain activity in a multiscale manner, ranging from localized brain regions, to a specific brain circuit/network and towards the full brain. We evaluate the decoding model using a large task-fMRI database from the human connectome project. By implementing connectomic constraints and multiscale interactions in deep graph convolutions, the model achieves high accuracy of decoding 21 cognitive states (93%, chancel level: 4.8%) and shows high robustness against adversarial attacks on the graph architecture. Our study bridges human connectomes with deep learning techniques and provides new avenues to study the underlying neural substrates of human cognition at scale.
Highlights: A full-brain integrative model is critical for cognitive decoding
Human connectomes and long-range connections accelerate the propagation of activity
ChebNet decoding is robust to random attacks on brain connectomes and regions
Observing cellular physiological histories is key to understanding normal and disease-related processes, but longitudinal imaging is laborious and equipment-intensive. A tantalizing possibility is that cells could record such histories in the form of digital biological information within themselves, for later high-throughput readout. Here we show that this concept can be realized through information storage in the form of growing protein chains made out of multiple self-assembling subunits bearing different labels, each corresponding to a different cellular state or function, so that the physiological history of the cell can be visually read out along the chain of proteins. Conveniently, such protein chains are fully genetically encoded, and easily readable with simple, conventional optical microscopy techniques, compatible with visualization of cellular shape and molecular content. We use such expression recording islands (XRIs) to record gene expression timecourse downstream of pharmacological and physiological stimuli, in cultured neurons and in living mouse brain.
Humans can often handle daunting tasks with ease by developing a set of strategies to reduce decision making into simpler problems. The ability to use heuristic strategies demands an advanced level of intelligence and has not been demonstrated in animals. Here, we trained macaque monkeys to play the classic video game Pac-Man. The monkeys9 decision-making may be described with a strategy-based hierarchical decision-making model with over 90% accuracy. The model reveals that the monkeys adopted the take-the-best heuristic by using one dominating strategy for their decision-making at a time and formed compound strategies by assembling the basis strategies to handle particular game situations. With the model, the computationally complex but fully quantifiable Pac-Man behavior paradigm provides a new approach to understanding animals9 advanced cognition.
In neural autopilot theory, habits save cognitive effort by repeating reliably-rewarding choices.
Strong habits are marked by insensitivity to reward change, but large-scale field data do not show this effect.
Habits are predictable from context variables using machine learning.
Predictable habits can be identified in everyday behavior using machine learning.
Identifying contextual cues, and using information about reward reliability, could personalize and improve our ability to change behavior.
This paper is about the background of 2 new ideas from neuroeconomics for understanding habits. The main idea is a 2-process ‘neural autopilot’ model. This model hypothesizes that contextually cued habits occur when the reward from the habitual behavior is numerically reliable (as in related models with an ‘arbitrator’). This computational model is lightly parameterized, has the essential ingredients established in animal learning and cognitive neuroscience, and is simple enough to make nonobvious predictions. An interesting set of predictions is about how consumers react to different kinds of changes in prices and qualities of goods (‘elasticities’). Elasticity analysis expands the habit marker of insensitivity to reward devaluation, and other types of sensitivities. The second idea is to use machine learning to discover which contextual variables seem to cue habits, in field data.
Most neuroimaging experiments are under-powered, limited by the number of subjects and cognitive processes that an individual study can investigate. Nonetheless, over decades of research, neuroscience has accumulated an extensive wealth of results. It remains a challenge to digest this growing knowledge base and obtain new insights since existing meta-analytic tools are limited to keyword queries. In this work, we propose Text2Brain, a neural network approach for coordinate-based meta-analysis of neuroimaging studies to synthesize brain activation maps from open-ended text queries. Combining a transformer-based text encoder and a 3D image generator, Text2Brain was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published neuroimaging studies. We demonstrate that Text2Brain can synthesize anatomically-plausible neural activation patterns from free-form textual descriptions of cognitive concepts. Text2Brain is available at https://braininterpreter.com as a web-based tool for retrieving established priors and generating new hypotheses for neuroscience research.
Human visual perception carves a scene at its physical joints, decomposing the world into objects, which are selectively attended, tracked, and predicted as we engage our surroundings. Object representations emancipate perception from the sensory input, enabling us to keep in mind that which is out of sight and to use perceptual content as a basis for action and symbolic cognition. Human behavioral studies have documented how object representations emerge through grouping, amodal completion, proto-objects, and object files. Deep neural network (DNN) models of visual object recognition, by contrast, remain largely tethered to the sensory input, despite achieving human-level performance at labeling objects.
Here, we review related work in both fields and examine how these fields can help each other. The cognitive literature provides a starting point for the development of new experimental tasks that reveal mechanisms of human object perception and serve as benchmarks driving development of deep neural network models that will put the object into object recognition.
In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing the full picture. Such systems have inspired development of artificial intelligence algorithms in areas such as swarm optimization and cellular automata. Motivated by the emergence of collective behavior from complex cellular systems, we build systems that feed each sensory input from the environment into distinct, but identical neural networks, each with no fixed relationship with one another. We show that these sensory networks can be trained to integrate information received locally, and through communication via an attention mechanism, can collectively produce a globally coherent policy. Moreover, the system can still perform its task even if the ordering of its inputs is randomly permuted several times during an episode. These permutation invariant systems also display useful robustness and generalization properties that are broadly applicable. Interactive demo and videos of our results: https://attentionneuron.github.io/
Understanding how the brain learns may lead to machines with human-like intellectual capacities. However, learning mechanisms in the brain are still not well understood. Here we demonstrate that the ability of a neuron to predict its future activity may provide an effective mechanism for learning in the brain. We show that comparing a neuron’s predicted activity with the actual activity provides an useful learning signal for modifying synaptic weights. Interestingly, this predictive learning rule can be derived from a metabolic principle, where neurons need to minimize their own synaptic activity (cost), while maximizing their impact on local blood supply by recruiting other neurons. This reveals an unexpected connection that learning in neural networks could result from simply maximizing the energy balance by each neuron. We show how this mathematically derived learning rule can provide a theoretical connection between diverse types of brain-inspired algorithms, such as: Hebb’s rule, BCM theory, temporal difference learning and predictive coding. Thus, this may offer a step toward development of a general theory of neuronal learning. We validated this predictive learning rule in neural network simulations and in data recorded from awake animals. We found that in the sensory cortex it is indeed possible to predict a neuron’s activity ~10–20ms into the future. Moreover, in response to stimuli, cortical neurons changed their firing rate to minimize surprise: i.e. the difference between actual and expected activity, as predicted by our model. Our results also suggest that spontaneous brain activity provides “training data” for neurons to learn to predict cortical dynamics. Thus, this work demonstrates that the ability of a neuron to predict its future inputs could be an important missing element to understand computation in the brain.
We learn in a variety of ways: through direct sensory experience, by talking with others, and by thinking. Disentangling how these sources contribute to what we know is challenging. A wedge into this puzzle was suggested by empiricist philosophers, who hypothesized that people born blind would lack deep knowledge of “visual” phenomena such as color. We find that, contrary to this prediction, congenitally blind and sighted individuals share in-depth understanding of object color. Blind and sighted people share similar intuitions about which objects will have consistent colors, make similar predictions for novel objects, and give similar explanations. Living among people who talk about color is sufficient for color understanding, highlighting the efficiency of linguistic communication as a source of knowledge.
Empiricist philosophers such as Locke famously argued that people born blind might learn arbitrary color facts (eg. marigolds are yellow) but would lack color understanding.
Contrary to this intuition, we find that blind and sighted adults share causal understanding of color, despite not always agreeing about arbitrary color facts. Relative to sighted people, blind individuals are less likely to generate “yellow” for banana and “red” for stop sign but make similar generative inferences about real and novel objects’ colors, and provide similar causal explanations. For example, people infer that 2 natural kinds (eg. bananas) and 2 artifacts with functional colors (eg. stop signs) are more likely to have the same color than 2 artifacts with nonfunctional colors (eg. cars).
People develop intuitive and inferentially rich “theories” of color regardless of visual experience. Linguistic communication is more effective at aligning intuitive theories than knowledge of arbitrary facts.
Predictive coding offers a potentially unifying account of cortical function—postulating that the core function of the brain is to minimize prediction errors with respect to a generative model of the world. The theory is closely related to the Bayesian brain framework and, over the last two decades, has gained substantial influence in the fields of theoretical and cognitive neuroscience. A large body of research has arisen based on both empirically testing improved and extended theoretical and mathematical models of predictive coding, as well as in evaluating their potential biological plausibility for implementation in the brain and the concrete neurophysiological and psychological predictions made by the theory. Despite this enduring popularity, however, no comprehensive review of predictive coding theory, and especially of recent developments in this field, exists. Here, we provide a comprehensive review both of the core mathematical structure and logic of predictive coding, thus complementing recent tutorials in the literature. We also review a wide range of classic and recent work within the framework, ranging from the neurobiologically realistic microcircuits that could implement predictive coding, to the close relationship between predictive coding and the widely-used backpropagation of error algorithm, as well as surveying the close relationships between predictive coding and modern machine learning techniques.
Whole-brain mesoscale mapping in primates has been hindered by large brain sizes and the relatively low throughput of available microscopy methods.
Here, we present an approach that combines primate-optimized tissue sectioning and clearing with ultrahigh-speed fluorescence microscopy implementing improved volumetric imaging with synchronized on-the-fly-scan and readout technique, and is capable of completing whole-brain imaging of a rhesus monkey at 1 × 1 × 2.5 µm3 voxel resolution within 100 h.
We also developed a highly efficient method for long-range tracing of sparse axonal fibers in datasets numbering hundreds of terabytes. This pipeline, which we call serial sectioning and clearing, 3-dimensional microscopy with semiautomated reconstruction and tracing (SMART), enables effective connectome-scale mapping of large primate brains. With SMART, we were able to construct a cortical projection map of the mediodorsal nucleus of the thalamus and identify distinct turning and routing patterns of individual axons in the cortical folds while approaching their arborization destinations.
Few neuroimaging studies are sufficiently large to adequately describe population-wide variations.
This study’s primary aim was to generate neuroanatomical norms and individual markers that consider age, sex, and brain size, from 629 cerebral measures in the UK Biobank (n = 40,028). The secondary aim was to examine the effects and interactions of sex, age, and brain allometry—the nonlinear scaling relationship between a region and brain size (eg. total brain volume)—across cerebral measures.
Allometry was a common property of brain volumes, thicknesses, and surface areas (83%) and was largely stable across age and sex. Sex differences occurred in 67% of cerebral measures (median |β| = 0.13): 37% of regions were larger in males and 30% in females. Brain measures (49%) generally decreased with age, although aging effects varied across regions and sexes. While models with an allometric or linear covariate adjustment for brain size yielded similar statistically-significant effects, omitting brain allometry influenced reported sex differences in variance. Finally, we contribute to the reproducibility of research on sex differences in the brain by replicating previous studies examining cerebral sex differences.
This large-scale study advances our understanding of age, sex, and brain allometry’s impact on brain structure and provides data for future UK Biobank studies to identify the cerebral regions that covary with specific phenotypes, independently of sex, age, and brain size.
Background: Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that decodes words and sentences directly from the cerebral cortical activity of such patients may represent an advancement over existing methods for assisted communication.
Methods: We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke. Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words. We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them.
Results: We decoded sentences from the participant’s cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period.
Conclusions: In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model. (Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149.)
As identifying proteins is of paramount importance for cell biology and applications, it is of interest to develop a protein sequencer with the ultimate sensitivity of decoding individual proteins. Here, we demonstrate a nanopore-based single-molecule sequencing approach capable of reliably detecting single amino-acid substitutions within individual peptides. A peptide is linked to a DNA molecule that is pulled through the biological nanopore MspA by a DNA helicase in single amino-acid steps. The peptide sequence yields clear stepping ion current signals which allows to discriminate single-amino-acid substitutions in single reads. Molecular dynamics simulations show these signals to result from size exclusion and pore binding. Notably, we demonstrate the capability to ‘rewind’ peptide reads, obtaining indefinitely many independent reads of the same individual molecule, yielding virtually 100% read accuracy in variant identification, with an error rate less than 10−6. These proof-of-concept experiments constitute a promising basis for developing a single-molecule protein sequencer.
This paper presents proof-of-concept experiments and simulations of a nanopore-based approach to sequencing individual proteins.
Inferotemporal cortex (IT) in humans and other primates is topo-graphically organized, containing multiple hierarchically-organized areas selective for particular domains, such as faces and scenes. This organization is commonly viewed in terms of evolved domain-specific visual mechanisms. Here, we develop an alternative, domain-general and developmental account of IT cortical organization. The account is instantiated as an Interactive Topographic Network (ITN), a form of computational model in which a hierarchy of model IT areas, subject to connectivity-based constraints, learns high-level visual representations optimized for multiple domains. We find that minimizing a wiring cost on spatially organized feedforward and lateral connections within IT, combined with constraining the feedforward processing to be strictly excitatory, results in a hierarchical, topographic organization. This organization replicates a number of key properties of primate IT cortex, including the presence of domain-selective spatial clusters preferentially involved in the representation of faces, objects, and scenes, columnar responses across separate excitatory and inhibitory units, and generic spatial organization whereby the response correlation of pairs of units falls off with their distance. We thus argue that domain-selectivity is an emergent property of a visual system optimized to maximize behavioral performance while minimizing wiring costs.
We introduce the Interactive Topographic Network, a framework for modeling high-level vision, to demonstrate in computational simulations that the spatial clustering of domains in late stages of the primate visual system may arise from the demands of visual recognition under the constraints of minimal wiring costs and excitatory between-area neuronal communication. The learned organization of the model is highly specialized but not fully modular, capturing many of the properties of organization in primates. Our work is significant for cognitive neuroscience, by providing a domain-general developmental account of topo-graphic functional specialization, and for computational neuroscience, by demonstrating how well-known biological details can be successfully incorporated into neural network models in order to account for critical empirical findings.
The last quarter century of cognitive neuroscience has revealed numerous cortical regions in humans with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what other people are thinking. But it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception, using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects the computational requirements of the task. We find that networks trained on generic object recognition perform poorly on face recognition and vice versa, and further that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. Thus, generic visual features that suffice for object recognition are apparently suboptimal for face recognition and vice versa. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.
[supplement; poster; code; Colab] Unrolled computation graphs arise in many scenarios, including training RNNs, tuning hyperparameters through unrolled optimization, and training learned optimizers. Current approaches to optimizing parameters in such computation graphs suffer from high variance gradients, bias, slow updates, or large memory usage.
We introduce a method called Persistent Evolution Strategies (PES), which divides the computation graph into a series of truncated unrolls, and performs an evolution strategies-based (Rechenberg 1973; Nesterov & Spokoiny 2017) update step after each unroll. PES eliminates bias from these truncations by accumulating correction terms over the entire sequence of unrolls. PES allows for rapid parameter updates, has low memory usage, is unbiased, and has reasonable variance characteristics.
We experimentally demonstrate the advantages of PES compared to several other methods for gradient estimation on synthetic tasks, and show its applicability to training learned optimizers and tuning hyperparameters.
…We introduce a method called Persistent Evolution Strategies (PES) to obtain unbiased gradient estimates for the parameters of an unrolled system from partial unrolls of the system.
We prove that PES is an unbiased gradient estimate for a smoothed version of the loss, and an unbiased estimate of the true gradient for quadratic losses. We provide theoretical and empirical analyses of its variance.
We demonstrate the applicability of PES in several illustrative scenarios: (1) we apply PES to tune hyperparameters including learning rates and momentums, by estimating hypergradients through partial unrolls of optimization algorithms; (2) we use PES to meta-train a learned optimizer; (3) we use PES to learn policy parameters for a continuous control task
Astrocytes and microglia can secrete tau, feeding into the prion-like hypothesis of tau spread.
Better understanding of glia cell contribution to tau propagation may be beneficial for therapeutic intervention.
Dementia is one of the leading causes of death worldwide, with tauopathies, a class of diseases defined by pathology associated with the microtubule-enriched protein, tau, as the major contributor. Although tauopathies, such as Alzheimer’s disease and Frontotemporal dementia, are common amongst the ageing population, current effective treatment options are scarce, primarily due to the incomplete understanding of disease pathogenesis. The mechanisms via which aggregated forms of tau are able to propagate from one anatomical area to another to cause disease spread and progression is yet unknown.
The prion-like hypothesis of tau propagation proposes that tau can propagate along neighbouring anatomical areas in a similar manner to prion proteins in prion diseases, such as Creutzfeldt-Jacob disease. This hypothesis has been supported by a plethora of studies that note the ability of tau to be actively secreted by neurons, propagated and internalised by neighbouring neuronal cells, causing disease spread. Surfacing research suggests a role of reactive astrocytes and microglia in early pre-clinical stages of tauopathy through their inflammatory actions. Furthermore, both glial types are able to internalise and secrete tau from the extracellular space, suggesting a potential role in tau propagation; although understanding the physiological mechanisms by which this can occur remains poorly understood.
This review will discuss the current literature around the prion-like propagation of tau, with particular emphasis on glial-mediated neuroinflammation and the contribution it may play in this propagation process.
Using blood-based epigenome-wide analyses of general cognitive function (g; n = 9,162) we show that individual differences in DNA methylation (DNAm) explain 35.0% of the variance in g. A DNAm predictor explains ~4% of the variance in g, independently of a polygenic score, in two external cohorts. It also associates with circulating levels of neurology-related and inflammation-related proteins, global brain imaging metrics, and regional cortical volumes. As sample sizes increase, our ability to assess cognitive function from DNAm data may be informative in settings where cognitive testing is unreliable or unavailable.
The visual system of mammals is comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. In particular, the clearest example is the specialization of the ventral (“what”) and dorsal (“where”) pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. To-date, deep neural networks have mostly been used as models of the ventral, recognition pathway. However, it is unknown whether both pathways can be modelled with a single deep ANN. Here, we ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. We explore this question using data from mice, who like other mammals, have specialized pathways that appear to support recognition and movement behaviours. We show that when we train a deep neural network architecture with two parallel pathways using a self-supervised predictive loss function, we can outperform other models in fitting mouse visual cortex. Moreover, we can model both the dorsal and ventral pathways. These results demonstrate that a self-supervised predictive learning approach applied to parallel pathway architectures can account for some of the functional specialization seen in mammalian visual systems.
Fitting network models to neural activity is an important tool in neuroscience. A popular approach is to model a brain area with a probabilistic recurrent spiking network whose parameters maximize the likelihood of the recorded activity. Although this is widely used, we show that the resulting model does not produce realistic neural activity. To correct for this, we suggest to augment the log-likelihood with terms that measure the dissimilarity between simulated and recorded activity. This dissimilarity is defined via summary statistics commonly used in neuroscience and the optimization is efficient because it relies on back-propagation through the stochastically simulated spike trains. We analyze this method theoretically and show empirically that it generates more realistic activity statistics. We find that it improves upon other fitting algorithms for spiking network models like GLMs (Generalized Linear Models) which do not usually rely on back-propagation. This new fitting algorithm also enables the consideration of hidden neurons which is otherwise notoriously hard, and we show that it can be crucial when trying to infer the network connectivity from spike recordings.
Hippocampal neurons encode physical variables such as space or auditory frequency in cognitive maps. In addition, functional magnetic resonance imaging studies in humans have shown that the hippocampus can also encode more abstract, learned variables. However, their integration into existing neural representations of physical variables is unknown.
Here, using 2-photon calcium imaging, we show that individual neurons in the dorsal hippocampus jointly encode accumulated evidence with spatial position in mice performing a decision-making task in virtual reality. Nonlinear dimensionality reduction showed that population activity was well-described by approximately 4 to 6 latent variables, which suggests that neural activity is constrained to a low-dimensional manifold. Within this low-dimensional space, both physical and abstract variables were jointly mapped in an orderly manner, creating a geometric representation that we show is similar across mice.
The existence of conjoined cognitive maps suggests that the hippocampus performs a general computation—the creation of task-specific low-dimensional manifolds that contain a geometric representation of learned knowledge.
Depression is a multifaceted illness with large interindividual variability in clinical response to treatment. In the era of digital medicine and precision therapeutics, new personalized treatment approaches are warranted for depression.
Here, we use a combination of longitudinal ecological momentary assessments of depression, neurocognitive sampling synchronized with electroencephalography, and lifestyle data from wearables to generate individualized predictions of depressed mood over a 1-month time period. This study, thus, develops a systematic pipeline for N-of-1 personalized modeling of depression using multiple modalities of data. In the models, we integrate 7 types of supervised machine learning (ML) approaches for each individual, including ensemble learning and regression-based methods. All models were verified using 4× nested cross-validation.
The best-fit as benchmarked by the lowest mean absolute percentage error, was obtained by a different type of ML model for each individual, demonstrating that there is no one-size-fits-all strategy. The voting regressor, which is a composite strategy across ML models [which simply averages the model predictions], was best performing on-average across subjects. However, the individually selected best-fit models still showed statistically-significantly less error than the voting regressor performance across subjects. For each individual’s best-fit personalized model, we further extracted top-feature predictors using Shapley statistics. Shapley values revealed distinct feature determinants of depression over time for each person ranging from co-morbid anxiety, to physical exercise, diet, momentary stress and breathing performance, sleep times, and neurocognition.
In future, these personalized features can serve as targets for a personalized ML-guided, multimodal treatment strategy for depression.
We propose that the entirety of the prefrontal cortex can be seen as fundamentally premotor in nature. By this, we mean that the prefrontal cortex consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal selection, which we posit is inherently a form of abstract action optimization. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Our view, therefore, positions the orbitofrontal cortex as the central site for the optimization of goal selection policies, and suggests that other proposed roles are aspects of this more general function. We conclude by proposing that the dynamical systems approach, which works well in motor systems, can be extended to the rest of prefrontal cortex. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry, and will align theories of prefrontal function with evolutionary principles.
[media] Different species of animals can discriminate numerosity, the countable number of objects in a set. The representations of countable numerosities have been deciphered down to the level of single neurons. However, despite its importance for human number theory, a special numerical quantity, the empty set (numerosity zero), has remained largely unexplored. We explored the behavioral and neuronal representation of the empty set in carrion crows.
Crows were trained to discriminate small numerosities including the empty set. Performance data showed a numerical distance effect for the empty set in one crow, suggesting that the empty set and countable numerosities are represented along the crows’ “mental number line.” Single-cell recordings in the endbrain region nidopallium caudolaterale (NCL) showed a considerable proportion of NCL neurons tuned to the preferred numerosity zero. As evidenced by neuronal distance and size effects, NCL neurons integrated the empty set in the neural number line. A subsequent neuronal population analysis using a statistical classifier approach showed that the neuronal numerical representations were predictive of the crows’ success in the task. These behavioral and neuronal data suggests that the conception of the empty set as a cognitive precursor of a zero-like number concept is not an exclusive property of the cerebral cortex of primates.
Zero as a quantitative category cannot only be implemented in the layered neocortex of primates, but also in the anatomically distinct endbrain circuitries of birds that evolved based on convergent evolution.
The conception of “nothing” as number “zero” is celebrated as one of the greatest achievements in mathematics. To explore whether precursors of zero-like concepts can be found in vertebrates with a cerebrum that anatomically differs starkly from our primate brain, we investigated this in carrion crows. We show that crows can grasp the empty set as a null numerical quantity that is mentally represented next to number one. Moreover, we show that single neurons in an associative avian cerebral region specifically respond to the empty set and show the same physiological characteristics as for countable quantities. This suggests that zero as a quantitative category can also be implemented in the anatomically distinct endbrain circuitries of birds that evolved based on convergent evolution.
Language and music are two human-unique capacities whose relationship remains debated. Some argue for overlap in processing mechanisms, especially for structure processing, but others fail to find overlap. Using fMRI, we examined the responses of language brain regions to diverse music stimuli, and also probed the musical abilities of individuals with severe aphasia. Across four experiments, we obtained a clear answer: music does not recruit nor requires the language system. The language regions’ responses to music are generally low and never exceed responses elicited by non-music auditory conditions, like animal sounds. Further, the language regions are not sensitive to music structure: they show low responses to both intact and scrambled music, and to melodies with vs. without structural violations. Finally, individuals with aphasia who cannot judge sentence grammaticality perform well on melody well-formedness judgments. Thus the mechanisms that process structure in language do not appear to support music processing.
Meta-synthesis of 3 decades of human brain sex difference findings.
Few male/female differences survive correction for brain size.
When present, sex accounts for about 1% of variance in structure or laterality.
Male and female brains are monomorphic, not dimorphic, in structure and function.
With the explosion of neuroimaging, differences between male and female brains have been exhaustively analyzed.
Here we synthesize 3 decades of human MRI and postmortem data, emphasizing meta-analyses and other large studies, which collectively reveal few reliable sex/gender differences and a history of unreplicated claims.
Males’ brains are larger than females’ from birth, stabilizing around 11% in adults. This size difference accounts for other reproducible findings: higher white matter/gray matter ratio, intrahemispheric versus interhemispheric connectivity, and regional cortical and subcortical volumes in males. But when structural and lateralization differences are present independent of size, sex/gender explains only about 1% of total variance. Connectome differences and multivariate sex/gender prediction are largely based on brain size, and perform poorly across diverse populations. Task-based fMRI has especially failed to find reproducible activation differences between men and women in verbal, spatial or emotion processing due to high rates of false discovery.
Overall, male/female brain differences appear trivial and population-specific. The human brain is not “sexually dimorphic.”
[Blog] We acquired a rapidly preserved human surgical sample from the temporal lobe of the cerebral cortex. We stained a 1 mm3 volume with heavy metals, embedded it in resin, cut more than 5000 slices at ~30 nm and imaged these sections using a high-speed multibeam scanning electron microscope. We used computational methods to render the 3-dimensional structure of 50,000 cells, hundreds of millions of neurites and 130 million synaptic connections. The 1.3 petabyte electron microscopy volume, the segmented cells, cell parts, blood vessels, myelin, inhibitory and excitatory synapses, and 100 manually proofread cells are available to peruse online.
Despite the incompleteness of the automated segmentation caused by split and merge errors, many interesting features were evident. Glia outnumbered neurons 2:1 and oligodendrocytes were the most common cell type in the volume. The E:I balance of neurons was 69:31%, as was the ratio of excitatory versus inhibitory synapses in the volume. The E:I ratio of synapses was statistically-significantly higher on pyramidal neurons than inhibitory interneurons.
We found that deep layer excitatory cell types can be classified into subsets based on structural and connectivity differences, that chandelier interneurons not only innervate excitatory neuron initial segments as previously described, but also each others’ initial segments, and that among the thousands of weak connections established on each neuron, there exist rarer highly powerful axonal inputs that establish multi-synaptic contacts (up to ~20 synapses) with target neurons. Our analysis indicates that these strong inputs are specific, and allow small numbers of axons to have an outsized role in the activity of some of their postsynaptic partners.
Understanding the neural basis of the remarkable human cognitive capacity to learn novel concepts from just one or a few sensory experiences constitutes a fundamental problem. We propose a simple, biologically plausible, mathematically tractable, and computationally powerful neural mechanism for few-shot learning of naturalistic concepts.
We posit that the concepts that can be learnt from few examples are defined by tightly circumscribed manifolds in the neural firing rate space of higher order sensory areas. We further posit that a single plastic downstream readout neuron learns to discriminate new concepts based on few examples using a simple plasticity rule.
We demonstrate the computational power of our proposal by showing it can achieve high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network models of these representations, and can even learn novel visual concepts specified only through linguistic descriptors. Moreover, we develop a mathematical theory of few-shot learning that links neurophysiology to behavior by delineating several fundamental and measurable geometric properties of high-dimensional neural representations that can accurately predict the few-shot learning performance of naturalistic concepts across all our numerical simulations.
We discuss testable predictions of our theory for psychophysics and neurophysiological experiments.
Engineered psychLight—a genetically encoded 5-HT sensor based on the 5-HT2AR
PsychLight can measure 5-HT dynamics in behaving mice
A psychLight-based cellular imaging platform predicts hallucinogenic potential
Identified a non-hallucinogenic psychedelic analog with antidepressant properties
Ligands can induce G protein-coupled receptors (GPCRs) to adopt a myriad of conformations, many of which play critical roles in determining the activation of specific signaling cascades associated with distinct functional and behavioral consequences. For example, the 5-hydroxytryptamine 2A receptor (5-HT2AR) is the target of classic hallucinogens, atypical antipsychotics, and psychoplastogens. However, currently available methods are inadequate for directly assessing 5-HT2AR conformation both in vitro and in vivo.
Here, we developed psychLight, a genetically encoded fluorescent sensor based on the 5-HT2AR structure. PsychLight detects behaviorally relevant serotonin release and correctly predicts the hallucinogenic behavioral effects of structurally similar 5-HT2AR ligands. We further used psychLight to identify a non-hallucinogenic psychedelic analog, which produced rapid-onset and long-lasting antidepressant-like effects after a single administration.
The advent of psychLight will enable in vivo detection of serotonin dynamics, early identification of designer drugs of abuse, and the development of 5-HT2AR-dependent non-hallucinogenic therapeutics.
[cf. Barlow twins; an AGI architecture sketch] Recent self-supervised methods for image representation learning are based on maximizing the agreement between embedding vectors from different views of the same image. A trivial solution is obtained when the encoder outputs constant vectors. This collapse problem is often avoided through implicit biases in the learning architecture, that often lack a clear justification or interpretation.
In this paper, we introduce VICReg (Variance-Invariance-Covariance Regularization), a method that explicitly avoids the collapse problem with a simple regularization term on the variance of the embeddings along each dimension individually. VICReg combines the variance term with a decorrelation mechanism based on redundancy reduction and covariance regularization.
VICReg achieves results on par with the state of the art on several downstream tasks. In addition, we show that incorporating our new variance term into other methods helps stabilize the training and leads to performance improvements.
Greater Vasa parrots are prospectively highly intelligent but critically understudied
An individual is found to spontaneously solve the string-pulling problem, and to be able to re-solve it after 7 years
This replicates previous findings, and expands on them
A general insight factor is identified among 14 parrot species
Covariance associates strongly with fission-fusion intensity, weakly with brain size and species differences
Spontaneous solving of an insight-based means-end reasoning task (the string-pulling problem) is observed in an adult male captive bred Greater Vasa parrot (Coracopsis vasa (Shaw 1812)), with an efficiency of 66%, replicating previous work in a singleton context. This case report adds to the existing literature on this species by also demonstrating longitudinal retention, specifically the same bird was found to be able to re-solve the simple form of the problem after a period of 7 years (the bird was first tested in 2013, and re-tested in 2020), with an efficiency of 43% (the difference between efficiencies was not statistically-significant, χ2 = 0.991, p = 0.319).
In a second analysis, species-level data across 5 patterned string-pulling tasks involving 14 parrot species were reanalysed, revealing that the Greater Vasa parrot exhibited the greatest general competence among those evaluated. A ‘general insight factor’ (GIF) was also found across taxa, the loadings onto which exhibit positive and large-magnitude associations with the correlation between fission-fusion flocking intensity and indicator level performance (r = 0.831), and also positive small and modest-magnitude associations with the correlation between relative brain size and indicator-level performance, and the magnitude of average pair-wise species differences in performance across indicators (r = 0.219 and 0.365 respectively).
Finally, the theoretical implications of these findings are discussed.
In Stevenson & Kording 2011, the authors estimated that every 7.4 years, the number of neurons we can record with doubles. Think of it as Moore’s law for brain recordings. Since then, Stevenson has updated the estimate, which now stands at 6 years. Could it be that progress itself is accelerating?
…We can do one better—fit a double-exponential model. This is only a few lines of code in PyMC3—a miracle of automatic differentiation and Hamiltonian Monte Carlo. Here’s what that looks like:
You can see visually this is a much better fit, and it implies something pretty dramatic: progress itself is accelerating. That means that doubling time itself has changed over time—and it currently stands at 3.6 years under this model [95% CI 3.5–3.7]…These results project a 1M neuron average recording capability by 2045—of course, this discounts ceiling effects and potential paradigm shifts, which could adjust these bounds far upward or downward.
Cognitive enhancement interventions aimed at boosting human fluid intelligence (gf) have targeted executive functions (EFs), such as updating, inhibition, and switching, in the context of transfer-inducing cognitive training. However, even though the link between EFs and gf has been demonstrated at the psychometric level, their neurofunctional overlap has not been quantitatively investigated. Identifying whether and how EFs and gf might share neural activation patterns could provide important insights into the overall hierarchical organization of human higher-order cognition, as well as suggest specific targets for interventions aimed at maximizing cognitive transfer.
We present the results of a quantitative meta-analysis of the available fMRI and PET literature on EFs and gf in humans, showing the similarity between gf and (1) the overall global EF network, as well as (2) specific maps for updating, switching, and inhibition. Results highlight a higher degree of similarity between gf and updating (80% overlap) compared with gf and inhibition (34%), and gf and switching (17%). Moreover, 3 brain regions activated for both gf and each of the 3 EFs also were identified, located in the left middle frontal gyrus, left inferior parietal lobule, and anterior cingulate cortex. Finally, resting-state functional connectivity analysis on 2 independent fMRI datasets showed the preferential behavioural correlation and anatomical overlap between updating and gf.
These findings confirm a close link between gf and EFs, with implications for brain stimulation and cognitive training interventions.
Biological cognition is based on self-generated learning objectives. However, the mechanism by which this epistemic autonomy is realized by the neuronal substrate is not understood.
Artificial neural networks based on error backpropagation lack epistemic autonomy because they are mostly trained in a supervised fashion. In this respect, they face the symbol grounding problem of artificial intelligence.
We propose that the entorhinal-hippocampal complex, a brain structure located in the medial temporal lobe and central to memory, combines epistemic autonomy with intrinsically generated error gradients akin to error backpropagation.
We present evidence supporting the hypothesis that the counter-current inhibitory projections of the entorhinal-hippocampal complex implement a continuous self-supervised error minimization between network input and output.
Biological cognition is based on the ability to autonomously acquire knowledge, or epistemic autonomy.
Such self-supervision is largely absent in artificial neural networks (ANN) because they depend on externally set learning criteria. Yet training ANN using error backpropagation has created the current revolution in artificial intelligence, raising the question of whether the epistemic autonomy displayed in biological cognition can be achieved with error backpropagation-based learning.
We present evidence suggesting that the entorhinal-hippocampal complex combines epistemic autonomy with error backpropagation. Specifically, we propose that the hippocampus minimizes the error between its input and output signals through a modulatory counter-current inhibitory network. We further discuss the computational emulation of this principle and analyze it in the context of autonomous cognitive systems.
Sleep loss impairs cognitive function, immunological responses and general well-being in humans. However, sleep requirements in mammals and birds vary dramatically. In circumpolar regions with continuous summer light, daily sleep duration is reduced, particularly in breeding birds. The effect of an anti-narcolepsy drug (modafinil) to putatively extend wakefulness was examined in two species of closely related arctic-breeding passerine birds: Lapland longspurs (Calcarius lapponicus) and snow buntings (Plectrophenax nivalis). Free-living adult males were implanted during the nestling phase on day 4 (D4; 4 days post-hatching) with osmotic pumps containing either vehicle or modafinil to extend the active period for 72 h. Nestlings were weighed on D2 and D7 to measure growth rates. Additionally, focal observations were conducted on D6. Male longspurs receiving modafinil made fewer feeding visits and spent less time at the nest but tended to spend more time near the nest than controls. We observed no change in longspur nestling growth rates, but fledging occurred significantly later when males received modafinil, suggesting a fitness cost. In contrast, modafinil had no measurable impact on male or female snow bunting behavior, nestling growth rates or time to fledging. We suggest male longspurs compromise and maintain vigilance at their nests in lieu of sleeping because of the increased predation risk that is characteristic of their tundra nesting habitat. Snow buntings are cavity nesters, and their nests do not require the same vigilance, allowing males to presumably rest following provisioning. These life-history differences between species highlight the role of predation risk in mediating behavioral modifications to prolonged wakefulness in arctic-breeding songbirds.
Exploration, consolidation and planning depend on the generation of sequential state representations. However, these algorithms require disparate forms of sampling dynamics for optimal performance. We theorize how the brain should adapt internally generated sequences for particular cognitive functions and propose a neural mechanism by which this may be accomplished within the entorhinal-hippocampal circuit.
Specifically, we demonstrate that the systematic modulation along the medial entorhinal cortex dorsoventral axis of grid population input into the hippocampus facilitates a flexible generative process that can interpolate between qualitatively distinct regimes of sequential hippocampal reactivations.
By relating the emergent hippocampal activity patterns drawn from our model to empirical data, we explain and reconcile a diversity of recently observed, but apparently unrelated, phenomena such as generative cycling, diffusive hippocampal reactivations and jumping trajectory events.
Neuroscientists today can measure activity from more neurons than ever before, and are facing the challenge of connecting these brain-wide neural recordings to computation and behavior.
Here, we first describe emerging tools and technologies being used to probe large-scale brain activity and new approaches to characterize behavior in the context of such measurements. We next highlight insights obtained from large-scale neural recordings in diverse model systems, and argue that some of these pose a challenge to traditional theoretical frameworks. Finally, we elaborate on existing modelling frameworks to interpret these data, and argue that interpreting brain-wide neural recordings calls for new theoretical approaches that may depend on the desired level of understanding at stake.
These advances in both neural recordings and theory development will pave the way for critical advances in our understanding of the brain.
Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). These models are trained to generate appropriate linguistic responses in a given context using a self-supervised prediction task. We provide empirical evidence that the human brain and autoregressive DLMs share two computational principles: (1) both are engaged in continuous prediction; (2) both represent words as a function of the previous context. Behaviorally, we demonstrate a match between humans and DLM’s next-word predictions given sufficient contextual windows during the processing of a real-life narrative. Neurally, we demonstrate that the brain, like autoregressive DLMs, constantly predicts upcoming words in natural speech, hundreds of milliseconds before they are perceived. Finally, we show that DLM’s contextual embeddings capture the neural representation of context-specific word meaning better than arbitrary or static semantic embeddings. Our findings suggest that autoregressive DLMs provide a novel and biologically feasible computational framework for studying the neural basis of language.
The dominant view in neuroscience is that changes in synaptic weights underlie learning. It is unclear, however, how the brain is able to determine which synapses should change, and by how much. This uncertainty stands in sharp contrast to deep learning, where changes in weights are explicitly engineered to optimize performance. However, the main tool for doing that, backpropagation, is not biologically plausible, and networks trained with this rule tend to forget old tasks when learning new ones. Here we introduce the Dendritic Gated Network (DGN), a variant of the Gated Linear Network [1, 2], which offers a biologically plausible alternative to backpropagation. DGNs combine dendritic “gating” (whereby interneurons target dendrites to shape neuronal response) with local learning rules to yield provably efficient performance. They are statistically-significantly more data efficient than conventional artificial networks and are highly resistant to forgetting, and we show that they perform well on a variety of tasks, in some cases better than backpropagation. The DGN bears similarities to the cerebellum, where there is evidence for shaping of Purkinje cell responses by interneurons. It also makes several experimental predictions, one of which we validate with in vivo cerebellar imaging of mice performing a motor task.
Introduction: Head injury is associated with substantial morbidity and mortality. Long-term associations of head injury with dementia in community-based populations are less clear.
Methods: Prospective cohort study of 14,376 participants (mean age 54 years at baseline, 56% female, 27% Black, 24% with head injury) enrolled in the Atherosclerosis Risk in Communities (ARIC) Study. Head injury was defined using self-report and International Classification of Diseases, Ninth/Tenth Revision (ICD-9/10) codes. Dementia was defined using cognitive assessments, informant interviews, and ICD-9/10 and death certificate codes.
Results: Head injury was associated with risk of dementia (hazard ratio [HR] = 1.44, 95% confidence interval [CI] = 1.3–1.57), with evidence of dose-response (1 head injury: HR = 1.25, 95% CI = 1.13–1.39, 2+ head injuries: HR = 2.14, 95% CI = 1.86–2.46). There was evidence for stronger associations among female participants (HR = 1.69, 95% CI = 1.51–1.90) versus male participants (HR = 1.15, 95% CI = 1.00–1.32), p-for-interaction < 0.001, and among White participants (HR = 1.55, 95% CI = 1.40–1.72) versus Black participants (HR = 1.22, 95% CI = 1.02–1.45), p-for-interaction = 0.008.
Discussion: In this community-based cohort with 25-year follow-up, head injury was associated with increased dementia risk in a dose-dependent manner, with stronger associations among female participants and White participants.
On Jan. 27, 1986, Allan McDonald stood on the cusp of history. McDonald directed the booster rocket project at NASA contractor Morton Thiokol. He was responsible for the two massive rockets, filled with explosive fuel, that lifted space shuttles skyward. He was at the Kennedy Space Center in Florida for the launch of the Challenger “to approve or disapprove a launch if something came up”, he told me in 2016, 30 years after Challenger exploded. His job was to sign and submit an official form. Sign the form, he believed, and he’d risk the lives of the 7 astronauts set to board the spacecraft the next morning. Refuse to sign, and he’d risk his job, his career and the good life he’d built for his wife and 4 children. “And I made the smartest decision I ever made in my lifetime”, McDonald told me. “I refused to sign it. I just thought we were taking risks we shouldn’t be taking.”
…Now, 35 years after the Challenger disaster, McDonald’s family reports that he died Saturday in Ogden, Utah, after suffering a fall and brain damage. He was 83 years old.
Intersecting neuroscience and deep learning has brought benefits and developments to both fields for several decades, which help to both understand how learning works in the brain, and to achieve the state-of-the-art performances in different AI benchmarks. Backpropagation (BP) is the most widely adopted method for the training of artificial neural networks, which, however, is often criticized for its biological implausibility (eg. lack of local update rules for the parameters). Therefore, biologically plausible learning methods (eg. inference learning (IL)) that rely on predictive coding (a framework for describing information processing in the brain) are increasingly studied.
Recent works prove that IL can approximate BP up to a certain margin on multilayer perceptrons (MLPs), and asymptotically on any other complex model, and that zero-divergence inference learning (Z-IL), a variant of IL, is able to exactly implement BP on MLPs. However, the recent literature shows also that there is no biologically plausible method yet that can exactly replicate the weight update of BP on complex models.
To fill this gap, in this paper, we generalize (IL and) Z-IL by directly defining them on computational graphs.
To our knowledge, this is the first biologically plausible algorithm that is shown to be equivalent to BP in the way of updating parameters on any neural network, and it is thus a great breakthrough for the interdisciplinary research of neuroscience and deep learning.
The backpropagation of error algorithm (backprop) has been instrumental in the recent success of deep learning. However, a key question remains as to whether backprop can be formulated in a manner suitable for implementation in neural circuitry. The primary challenge is to ensure that any candidate formulation uses only local information, rather than relying on global signals as in standard backprop. Recently several algorithms for approximating backprop using only local signals have been proposed. However, these algorithms typically impose other requirements which challenge biological plausibility: for example, requiring complex and precise connectivity schemes, or multiple sequential backwards phases with information being stored across phases. Here, we propose a novel algorithm, Activation Relaxation (AR), which is motivated by constructing the backpropagation gradient as the equilibrium point of a dynamical system. Our algorithm converges rapidly and robustly to the correct backpropagation gradients, requires only a single type of computational unit, utilises only a single parallel backwards relaxation phase, and can operate on arbitrary computation graphs. We illustrate these properties by training deep neural networks on visual classification tasks, and describe simplifications to the algorithm which remove further obstacles to neurobiological implementation (for example, the weight-transport problem, and the use of nonlinear derivatives), while preserving performance.
Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function.
We present the Natural Scenes Dataset (NSD), in which high-resolution fMRI responses to tens of thousands of richly annotated natural scenes [MS-COCO] are measured while participants perform a continuous recognition task. To optimize data quality, we develop and apply novel estimation and denoising techniques.
Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we use NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory.
Given its unprecedented scale, quality, and breadth, NSD opens new avenues of inquiry in cognitive and computational neuroscience.
Predictive coding represents a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring a preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modelling to demonstrate that such architectural hard-wiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimize their energy consumption while operating in predictive environments, the networks self-organize into prediction and error units with appropriate inhibitory and excitatory interconnections, and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down driven predictions, we demonstrate via virtual lesioning experiments that networks perform predictions on two timescales: fast lateral predictions among sensory units, and slower prediction cycles that integrate evidence over time.
Intelligence predicts important life and health outcomes, but the biological mechanisms underlying differences in intelligence are not yet understood. The use of genetically determined metabotypes (GDMs) to understand the role of genetic and environmental factors, and their interactions, in human complex traits has been recently proposed. However, this strategy has not been applied to human intelligence.
Here we implemented a 2-sample Mendelian randomization (MR) analysis using GDMs to assess the causal relationships between genetically determined metabolites and human intelligence. The standard inverse-variance weighted (IVW) method was used for the primary MR analysis and 3 additional MR methods (MR-Egger, weighted median, and MR-PRESSO) were used for sensitivity analyses.
Using 25 genetic variants as instrumental variables (IVs), our study found that 5-oxoproline was associated with better performance in human intelligence tests (pIVW = 9.25 × 10−5). The causal relationship was robust when sensitivity analyses were applied (pMR-Egger = 0.0001, pWeighted median = 6.29 × 10−6, PMR-PRESSO = 0.0007), and repeated analysis yielded consistent result (pIVW = 0.0087). Similarly, also dihomo-linoleate (20:2n6) and p-acetamidophenylglucuronide showed robust association with intelligence.
Our study provides novel insight by integrating genomics and metabolomics to estimate causal effects of genetically determined metabolites on human intelligence, which help to understanding of the biological mechanisms related to human intelligence.
As the brain develops, neurons build new connections that are refined by pruning. Gour et al 2020 used electron microscopy to build a high-resolution study of mouse postnatal brain development. The survey reveals the details of how circuits are built to incorporate inhibitory neurons in the somatosensory cortex.
Brain circuits in the neocortex develop from diverse types of neurons that migrate and form synapses. Here we quantify the circuit patterns of synaptogenesis for inhibitory interneurons in the developing mouse somatosensory cortex. We studied synaptic innervation of cell bodies, apical dendrites, and axon initial segments using 3-dimensional electron microscopy focusing on the first 4 weeks postnatally (postnatal days P5 to P28). We found that innervation of apical dendrites occurs early and specifically: Target preference is already almost at adult levels at P5. Axons innervating cell bodies, on the other hand, gradually acquire specificity from P5 to P9, likely via synaptic overabundance followed by antispecific synapse removal. Chandelier axons show first target preference by P14 but develop full target specificity almost completely by P28, which is consistent with a combination of axon outgrowth and off-target synapse removal. This connectomic developmental profile reveals how inhibitory axons in the mouse cortex establish brain circuitry during development.
Introduction: The establishment of neuronal circuits in the cerebral cortex of mammals is an important developmental process extending over embryonic and postnatal periods, from the first occurrence of differentiated neurons to the final formation of precise synaptic innervation patterns, which are further shaped by experience. Of special interest is the establishment of inhibitory circuits, constituted by nerve cells that produce γ-aminobutyric acid as a neurotransmitter (GABAergic interneurons), which are known to form intricate neuronal networks with a distinctive degree of synaptic preference for the types of postsynaptic structures to innervate. While the time course of neuronal migration and integration of interneurons is beginning to be understood and the first molecular cues for selectively enhancing and suppressing synaptic innervation have been identified, a comprehensive mapping of cortical inhibitory innervation during postnatal development is missing.
Rationale: With the development of high-throughput 3-dimensional electron microscopy (3D EM) imaging and analysis of nervous tissue, the goal of systematically mapping neuronal connectivity in ever-increasing volumes of brain tissue has become possible. This methodological approach, connectomics, has so far been primarily aimed at comprehensive circuit mapping in complete smaller animals’ brains or parts of larger brains. An additional advantage of higher-throughput connectomic analysis, however, is the opportunity to repeat similar experiments under many experimental conditions. This advantage is particularly relevant for the study of developmental processes, which naturally require the measurement of multiple time points. In this study, we made use of these technological advances to map neuronal connectivity in 13 3D EM datasets with a focus on the primary somatosensory cortex of mouse during postnatal development.
Results: We acquired and analyzed data from layers 4 and 2–3 of mouse cortex over the period during which synaptic networks are formed within the neocortex. We studied data from mice at 5, 7, 9, 14, 28, and 56 days of age, corresponding to the development from baby to adult. We analyzed the formation of interneuronal synaptic preference for subsections of neurons, their cell bodies, their initial part of the axon, and apical dendrites. We found that only axons with preference for apical dendrites already show high target preference in the early time points measured. By contrast, preference for innervation of cell bodies was gradually established, with a peak in developmental change between postnatal days 7 and 9. During this time, preference for cell bodies increased almost 3×, and the density of synapses along these axons dropped by almost 2×. With this, we found that while for apical dendrite-preferring interneurons, mechanisms of ab initio target choice are plausible, cell body innervation could be established by the removal of inadequately placed synapses along the presynaptic axon. For the innervation of the initial section of axons, we found that axo-axonic innervation initially constitutes only a minor fraction of the innervation and develops to provide ~50% of the synaptic input to the axon initial segment. Our data indicate that synaptic preference for axon initial segments develops before the formation of special vertically oriented axonal configurations called cartridges.
Conclusion: The first comprehensive mapping of inhibitory circuit development in mammalian cortex provides quantitative insights into the formation of circuits and the precise time course for the establishment of synaptic target preference. The approach of connectomic screening also may prove useful for future studies of experimental interference with relevant genetic and environmental conditions of circuit formation in the mammalian brain.
Introduction: The inner voice is experienced during thinking in words (inner speech) and silent reading and evokes brain activity that is highly similar to that associated with external voices. Yet while the inner voice is experienced in internal space (inside the head), external voices (one’s own and those of others) are experienced in external space. In this paper, we investigate the neural basis of this differential spatial localization.
Methods: We used fMRI to examine the difference in brain activity between reading silently and reading aloud. As the task involved reading aloud, data were first denoised by removing independent components related to head movement. They were subsequently processed using finite impulse response basis function to address the variations of the hemodynamic response. Final analyses were carried out using permutation-based statistics, which is appropriate for small samples. These analyses produce spatiotemporal maps of brain activity.
Conclusions: These pilot data suggest that internal space localization of the inner voice depends on the same neural resources as that for external space localization of external voices—the “where” auditory pathway. We discuss the implications of these findings on the possible mechanisms of abnormal experiences of the inner voice as is the case in verbal hallucinations.
Nearly one billion people worldwide suffer from obsessive-compulsive behaviors, yet our mechanistic understanding of these behaviors is incomplete, and effective therapeutics are unavailable. An emerging perspective characterizes obsessive-compulsive behaviors as maladaptive habit learning, which may be associated with abnormal beta-gamma neurophysiology of the orbitofrontal-striatal circuitry during reward processing.
We target the orbitofrontal cortex with alternating current, personalized to the intrinsic beta-gamma frequency of the reward network, and show rapid, reversible, frequency-specific modulation of reward-guided but not punishment-guided choice behavior and learning, driven by increased exploration in the setting of an actor-critic architecture. Next, we demonstrate that chronic application of the procedure over 5 days robustly attenuates obsessive-compulsive behavior in a non-clinical population for 3 months, with the largest benefits for individuals with more severe symptoms. Finally, we show that convergent mechanisms underlie modulation of reward learning and reduction of obsessive-compulsive symptoms.
The results contribute to neurophysiological theories of reward, learning and obsessive-compulsive behavior, suggest an unifying functional role of rhythms in the beta-gamma range, and set the groundwork for the development of personalized circuit-based therapeutics for related disorders.
Deep Learning has recently led to major advances in natural language processing. Do these models process sentences similarly to humans, and is this similarity driven by specific principles? Using a variety of artificial neural networks, trained on image classification, word embedding, or language modeling, we evaluate whether their architectural and functional properties lead them to generate activations linearly comparable to those of 102 human brains measured with functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We show that image, word and contextualized word embeddings separate the hierarchical levels of language processing in the brain. Critically, we compare 3,600 embeddings in their ability to linearly map onto these brain responses. The results show that (1) the position of the layer in the network and (2) the ability of the network to accurately predict words from context are the main factors responsible for the emergence of brain-like representations in artificial neural networks. Together, these results show how perceptual, lexical and compositional representations precisely unfold within each cortical region and contribute to uncovering the governing principles of language processing in brains and algorithms.
Differences in human general intelligence or reasoning ability can be quantified with the psychometric factor g, because individual performance across cognitive tasks is positively correlated. g also emerges in mammals and birds, is correlated with brain size and may similarly reflect general reasoning ability and behavioural flexibility in these species. To exclude the alternative that these positive cross-correlations may merely reflect the general biological quality of an organism or an inevitable by-product of having brains it is paramount to provide solid evidence for the absence of g in at least some species. Here, we show that wild-caught cleaner fish Labroides dimidiatus, a fish species otherwise known for its highly sophisticated social behaviour, completely lacks g when tested on ecologically non-relevant tasks. Moreover, performance in these experiments was not or negatively correlated with an ecologically relevant task, and in none of the tasks did fish caught from a high population density site outperform fish from a low-density site. g is thus unlikely a default result of how brains are designed, and not an automatic consequence of variation in social complexity. Rather, the results may reflect that g requires a minimal brain size, and thus explain the conundrum why the average mammal or bird has a roughly 10× larger brain relative to body size than ectotherms. Ectotherm brains and cognition may therefore be organized in fundamentally different ways compared to endotherms.
Human intelligence has always been a fascinating subject for scientists. Since the inception of Spearman’s general intelligence in the early 1900s, there has been significant progress towards characterizing different aspects of intelligence and its relationship with structural and functional features of the brain. In recent years, the invention of sophisticated brain imaging devices using Diffusion-Weighted Imaging (DWI) and functional Magnetic Resonance Imaging (fMRI) has allowed researchers to test hypotheses about neural correlates of intelligence in humans.This review summarizes recent findings on the associations of human intelligence with neuroimaging data. To this end, first, we review the literature that has related brain morphometry to intelligence. Next, we elaborate on the applications of DWI and resting state fMRI on the investigation of intelligence. Then, we provide a survey of literature that has used multimodal DWI-fMRI to shed light on intelligence. Finally, we discuss the state-of-the-art of individualized prediction of intelligence from neuroimaging data and point out future strategies. Future studies hold promising outcomes for machine learning-based predictive frameworks using neuroimaging features to estimate human intelligence.
Evidence from model organisms and clinical genetics suggests coordination between the developing brain and face, but the role of this link in common genetic variation remains unknown. We performed a multivariate genome-wide association study of cortical surface morphology in 19,644 individuals of European ancestry, identifying 472 genomic loci influencing brain shape, of which 76 are also linked to face shape. Shared loci include transcription factors involved in craniofacial development, as well as members of signaling pathways implicated in brain-face cross-talk. Brain shape heritability is equivalently enriched near regulatory regions active in either forebrain organoids or facial progenitors. However, we do not detect significant overlap between shared brain-face genome-wide association study signals and variants affecting behavioral-cognitive traits. These results suggest that early in embryogenesis, the face and brain mutually shape each other through both structural effects and paracrine signaling, but this interplay may not impact later brain development associated with cognitive function.
The present research (total n = 2,057) tested whether people’s folk conception of consciousness aligns with the notion of a “Cartesian theater” (Dennett, 1991).
More precisely, we tested the hypotheses that people believe that consciousness happens in a single, confined area (vs. multiple dispersed areas) in the human brain, and that it (partly) happens after the brain finished analyzing all available information. Further, we investigated how these beliefs are related to participants’ neuroscientific knowledge as well as their reliance on intuition, and which rationale they use to explain their responses.
Using a computer-administered drawing task, we found that participants located consciousness, but not unrelated neurological processes (Studies 1a and 1b) or unconscious thinking (Study 2) in a single, confined area in the prefrontal cortex, and that they considered most of the brain not involved in consciousness. Participants mostly relied on their intuitions when responding, and they were not affected by prior knowledge about the brain. Additionally, they considered the conscious experience of sensory stimuli to happen in a spatially more confined area than the corresponding computational analysis of these stimuli (Study 3). Furthermore, participants’ explicit beliefs about spatial and temporal localization of consciousness (ie. consciousness happening after the computational analysis of sensory information is completed) are independent, yet positively correlated beliefs (Study 4). Using a more elaborate measure for temporal localization of conscious experience, our final study confirmed that people believe consciousness to partly happen even after information processing is done (Study 5).
[Keywords: Cartesian Theater, neuropsychology, consciousness, lay theories, philosophy of mind]
Many concepts have been proposed for meta learning with neural networks (NNs), eg. NNs that learn to control fast weights, hyper networks, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VS-ML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VS-ML called VS-ML RNN allows for implementing the backpropagation LA solely by running an RNN in forward-mode. It can even meta-learn new LAs that improve upon backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta-learned LAs learn qualitatively different from gradient descent through fast association.
Human children show unique cognitive skills for dealing with the social world but their cognitive performance is paralleled by great apes in many tasks dealing with the physical world. Recent studies suggested that members of a songbird family—corvids—also evolved complex cognitive skills but a detailed understanding of the full scope of their cognition was, until now, not existent. Furthermore, relatively little is known about their cognitive development.
Here, we conducted the first systematic, quantitative large-scale assessment of physical and social cognitive performance of common ravens with a special focus on development. To do so, we fine-tuned one of the most comprehensive experimental test-batteries, the Primate Cognition Test Battery (PCTB), to raven features enabling also a direct, quantitative comparison with the cognitive performance of 2 great ape species. Full-blown cognitive skills were already present at the age of 4 months with subadult ravens’ cognitive performance appearing very similar to that of adult apes in tasks of physical (quantities, and causality) and social cognition (social learning, communication, and theory of mind).
These unprecedented findings strengthen recent assessments of ravens’ general intelligence, and aid to the growing evidence that the lack of a specific cortical architecture does not hinder advanced cognitive skills. Difficulties in certain cognitive scales further emphasize the quest to develop comparative test batteries that tap into true species rather than human specific cognitive skills, and suggest that socialization of test individuals may play a crucial role.
We conclude to pay more attention to the impact of personality on cognitive output, and a currently neglected topic in Animal Cognition—the linkage between ontogeny and cognitive performance.
Here we review the motivation for creating the enhancing neuroimaging genetics through meta-analysis (ENIGMA) Consortium and the genetic analyses undertaken by the consortium so far. We discuss the methodological challenges, findings, and future directions of the genetics working group. A major goal of the working group is tackling the reproducibility crisis affecting “candidate gene” and genome-wide association analyses in neuroimaging. To address this, we developed harmonized analytic methods, and support their use in coordinated analyses across sites worldwide, which also makes it possible to understand heterogeneity in results across sites. These efforts have resulted in the identification of hundreds of common genomic loci robustly associated with brain structure. We have found both pleiotropic and specific genetic effects associated with brain structures, as well as genetic correlations with psychiatric and neurological diseases.
Why Did We Build The Enigma Consortium? The consortium was formed in 2009, largely in response to the growing evidence of a lack of reproducibility dubbed “the replication crisis” in imaging genetics. At this time, the first major works of the Psychiatric Genomics Consortium were being presented at conferences (Neale et al 2010; The Schizophrenia Psychiatric Genome-Wide Association Study [GWAS] Consortium, 2011a, 2011b), and we had observed the improvement in statistical power and increase in reproducibility that could be achieved through large-scale meta-analysis. In late 2009, we were beginning to see a series of GWAS publications using phenotypes derived from magnetic resonance imaging (MRI) attempting to answer complex and important questions in psychiatry and neurology. At that time, it was common to see GWAS papers reporting not only main effect analyses but also interactions with diagnosis or putative risk variables in sample sizes of less than 1,000 people.
…In response to these issues, Thompson and Martin sent an email to neuro-imaging groups around the world asking for interest in being part of a collaborative meta-analysis consortium focusing on imaging genetics. The key points in this email were that, although every group would understandably want to publish its own paper reporting their own findings, (a) the power calculations do not change just because the phenotype acquisition is expensive, (b) it was likely that the individual studies would not be large enough to find statistically-significant genetic effects, and (c) even if they did, it would still be necessary to replicate these findings in independent samples. From these beginnings, the ENIGMA consortium now involves more than 2,000 scientists from over 400 institutions in more than 40 countries.
Temporal difference (TD) error is a powerful teaching signal in machine learning
Teleport and speed manipulations are used to characterize dopamine signals in mice
Slowly ramping as well as phasic dopamine responses convey TD errors
Dopamine neurons compute TD error or changes in value on a moment-by-moment basis
Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide an unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.
Interest in deciphering the fundamental mechanisms and processes of the human mind represents a central driving force in modern neuroscience research. Activities in support of this goal rely on advanced methodologies and engineering systems that are capable of interrogating and stimulating neural pathways, from single cells in small networks to interconnections that span the entire brain. Recent research establishes the foundations for a broad range of creative neurotechnologies that enable unique modes of operation in this context.
This review focuses on those systems with proven utility in animal model studies and with levels of technical maturity that suggest a potential for broad deployment to the neuroscience community in the relatively near future. We include a brief summary of existing and emerging neuroscience techniques, as background for a primary focus on device technologies that address associated opportunities in electrical, optical and microfluidic neural interfaces, some with multimodal capabilities.
Examples of the use of these technologies in recent neuroscience studies illustrate their practical value.
The vibrancy of the engineering science associated with these platforms, the interdisciplinary nature of this field of research and its relevance to grand challenges in the treatment of neurological disorders motivate continued growth of this area of study.
To study the dynamics of neural processing across timescales, we require the ability to follow the spiking of thousands of individually separable neurons over weeks and months, during unrestrained behavior. To address this need, we introduce the Neuropixels 2.0 probe together with novel analysis algorithms. The new probe has over 5,000 sites and is miniaturized such that two probes plus a headstage, recording 768 sites at once, weigh just over 1 g, suitable for implanting chronically in small mammals. Recordings with high quality signals persisting for at least two months were reliably obtained in two species and six different labs. Improved site density and arrangement combined with new data processing methods enable automatic post-hoc stabilization of data despite brain movements during behavior and across days, allowing recording from the same neurons in the mouse visual cortex for over 2 months. Additionally, an optional configuration allows for recording from multiple sites per available channel, with a penalty to signal-to-noise ratio. These probes and algorithms enable stable recordings from >10,000 sites during free behavior in small animals such as mice.
Current state-of-the-art object recognition models are largely based on convolutional neural network (CNN) architectures, which are loosely inspired by the primate visual system. However, these CNNs can be fooled by imperceptibly small, explicitly crafted perturbations, and struggle to recognize objects in corrupted images that are easily recognized by humans. Here, by making comparisons with primate neural data, we first observed that CNN models with a neural hidden layer that better matches primate primary visual cortex (V1) are also more robust to adversarial attacks. Inspired by this observation, we developed VOneNets, a new class of hybrid CNN vision models. Each VOneNet contains a fixed weight neural network front-end that simulates primate V1, called the VOneBlock, followed by a neural network back-end adapted from current CNN vision models. The VOneBlock is based on a classical neuroscientific model of V1: the linear-nonlinear-Poisson model, consisting of a biologically-constrained Gabor filter bank, simple and complex cell nonlinearities, and a V1 neuronal stochasticity generator. After training, VOneNets retain high ImageNet performance, but each is substantially more robust, outperforming the base CNNs and state-of-the-art methods by 18% and 3%, respectively, on a conglomerate benchmark of perturbations comprised of white box adversarial attacks and common image corruptions. Finally, we show that all components of the VOneBlock work in synergy to improve robustness. While current CNN architectures are arguably brain-inspired, the results presented here demonstrate that more precisely mimicking just one stage of the primate visual system leads to new gains in ImageNet-level computer vision applications.
The neuroscience of perception has recently been revolutionized with an integrative reverse-engineering approach in which computation, brain function, and behavior are linked across many different datasets and many computational models. We here present a first systematic study taking this approach into higher-level cognition: human language processing, our species’ signature cognitive skill.
We find that the most powerful ‘transformer’ networks predict neural responses at nearly 100% and generalize across different datasets and data types (fMRI, ECoG). Across models, statistically-significant correlations are observed among all three metrics of performance: neural fit, fit to behavioral responses, and accuracy on the next-word prediction task (but not other language tasks), consistent with the long-standing hypothesis that the brain’s language system is optimized for predictive processing.
Model architectures with initial weights further perform surprisingly similar to final trained models, suggesting that inherent structure—and not just experience with language—crucially contributes to a model’s match to the brain.
Hearing aids are the only available treatment for mild-to-moderate sensorineural hearing loss, but often fail to improve perception in difficult listening conditions. To identify the reasons for this failure, we studied the underlying neural code using large-scale single-neuron recordings in gerbils, a common animal model of human hearing. We found that a hearing aid restored the sensitivity of neural responses, but failed to restore their selectivity. The low selectivity of aided responses was not a direct effect of hearing loss per se, but rather a consequence of the strategies used by hearing aids to restore sensitivity: compression, which decreases the spectral and temporal contrast of incoming sounds, and amplification, which produces high intensities that distort the neural code even with normal hearing. To improve future hearing aids, new processing strategies that avoid this tradeoff between neural sensitivity and selectivity must be developed.
Basic principles of bird and mammal brains: Mammals can be very smart. They also have a brain with a cortex. It has thus often been assumed that the advanced cognitive skills of mammals are closely related to the evolution of the cerebral cortex. However, birds can also be very smart, and several bird species show amazing cognitive abilities. Although birds lack a cerebral cortex, they do have pallium, and this is considered to be analogous, if not homologous, to the cerebral cortex. An outstanding feature of the mammalian cortex is its layered architecture. In a detailed anatomical study of the bird pallium, Stacho et al describe a similarly layered architecture. Despite the nuclear organization of the bird pallium, it has a cyto-architectonic organization that is reminiscent of the mammalian cortex.
Although the avian pallium seems to lack an organization akin to that of the cerebral cortex, birds exhibit extraordinary cognitive skills that are comparable to those of mammals.
We analyzed the fiber architecture of the avian pallium with three-dimensional polarized light imaging and subsequently reconstructed local and associative pallial circuits with tracing techniques.
We discovered an iteratively repeated, column-like neuronal circuitry across the layer-like nuclear boundaries of the hyperpallium and the sensory dorsal ventricular ridge. These circuits are connected to neighboring columns and, via tangential layer-like connections, to higher associative and motor areas.
Our findings indicate that this avian canonical circuitry is similar to its mammalian counterpart and might constitute the structural basis of neuronal computation.
Introduction: For more than a century, the avian forebrain has been a riddle for neuroscientists. Birds demonstrate exceptional cognitive abilities comparable to those of mammals, but their forebrain organization is radically different. Whereas mammalian cognition emerges from the canonical circuits of the six-layered neocortex, the avian forebrain seems to display a simple nuclear organization. Only one of these nuclei, the Wulst, has been generally accepted to be homologous to the neocortex. Most of the remaining pallium is constituted by a multinuclear structure called the dorsal ventricular ridge (DVR), which has no direct counterpart in mammals. Nevertheless, one long-standing theory, along with recent scientific evidence, supports the idea that some parts of the sensory DVR could display connectivity patterns, physiological signatures, and cell type-specific markers that are reminiscent of the neocortex. However, it remains unknown if the entire Wulst and sensory DVR harbor a canonical circuit that structurally resembles mammalian cortical organization.
Rationale: The mammalian neocortex comprises a columnar and laminar organization with orthogonally organized fibers that run in radial and tangential directions. These fibers constitute repetitive canonical circuits as computational units that process information along the radial domain and associate it tangentially. In this study, we first analyzed the pallial fiber architecture with three-dimensional polarized light imaging (3D-PLI) in pigeons and subsequently reconstructed local sensory circuits of the Wulst and the sensory DVR in pigeons and barn owls by means of in vivo or in vitro applications of neuronal tracers. We focused on two distantly related bird species to prove the hypothesis that a canonical circuit comparable to the neocortex is a genuine feature of the avian sensory forebrain.
Results: The 3D-PLI fiber analysis showed that both the Wulst and the sensory DVR display an orthogonal organization of radially and tangentially organized fibers along their entire extent. In contrast, nonsensory components of the DVR displayed a complex mosaic-like arrangement with patches of fibers with different orientations. Fiber tracing revealed an iterative circuit motif that was present across modalities (somatosensory, visual, and auditory), brain regions (sensory DVR and Wulst), and species (pigeon and barn owl). Although both species showed a comparable column-like and lamina-like circuit organization, small species differences were discernible, particularly for the Wulst, which was more subdifferentiated in barn owls, which fits well with the processing of stereopsis, combined with high visual acuity in the Wulst of this species. The primary sensory zones of the DVR were tightly interconnected with the intercalated nidopallial layers and the overlying mesopallium. In addition, nidopallial and some hyperpallial lamina-like areas gave rise to long-range tangential projections connecting sensory, associative, and motor structures.
Conclusion: Our study reveals a hitherto unknown neuroarchitecture of the avian sensory forebrain that is composed of iteratively organized canonical circuits within tangentially organized lamina-like and orthogonally positioned column-like entities. Our findings suggest that it is likely that an ancient microcircuit that already existed in the last common stem amniote might have been evolutionarily conserved and partly modified in birds and mammals. The avian version of this connectivity blueprint could conceivably generate computational properties reminiscent of the neocortex and would thus provide a neurobiological explanation for the comparable and outstanding perceptual and cognitive feats that occur in both taxa.
Humans have tended to believe that we are the only species to possess certain traits, behaviors, or abilities, especially with regard to cognition. Occasionally, we extend such traits to primates or other mammals—species with which we share fundamental brain similarities. Over time, more and more of these supposed pillars of human exceptionalism have fallen. Nieder et al 2020 now argue that the relationship between consciousness and a standard cerebral cortex is another fallen pillar (see the Perspective by Herculano-Houzel). Specifically, carrion crows show a neuronal response in the palliative end brain during the performance of a task that correlates with their perception of a stimulus. Such activity might be a broad marker for consciousness.
Subjective experiences that can be consciously accessed and reported are associated with the cerebral cortex. Whether sensory consciousness can also arise from differently organized brains that lack a layered cerebral cortex, such as the bird brain, remains unknown.
We show that single-neuron responses in the pallial endbrain of crows performing a visual detection task correlate with the birds’ perception about stimulus presence or absence and argue that this is an empirical marker of avian consciousness. Neuronal activity follows a temporal 2-stage process in which the first activity component mainly reflects physical stimulus intensity, whereas the later component predicts the crows’ perceptual reports.
These results suggest that the neural foundations that allow sensory consciousness arose either before the emergence of mammals or independently in at least the avian lineage and do not necessarily require a cerebral cortex.
The term “birdbrain” used to be derogatory. But humans, with their limited brain size, should have known better than to use the meager proportions of the bird brain as an insult. Part of the cause for derision is that the mantle, or pallium, of the bird brain lacks the obvious layering that earned the mammalian pallium its “cerebral cortex” label.
On page 1626 of this issue, Nieder et al (4) show that the bird pallium has neurons that represent what it perceives—a hallmark of consciousness. And on page 1585 of this issue, Stacho et al (5) establish that the bird pallium has similar organization to the mammalian cortex.
Research unveiled on Thursday in Science finds that crows know what they know and can ponder the content of their own minds, a manifestation of higher intelligence and analytical thought long believed the sole province of humans and a few other higher mammals.
A second study, also in Science, looked in unprecedented detail at the neuroanatomy of pigeons and barn owls, finding hints to the basis of their intelligence that likely applies to corvids’, too.
“Together, the two papers show that intelligence/consciousness are grounded in connectivity and activity patterns of neurons” in the most neuron-dense part of the bird brain, called the pallium, neurobiologist Suzana Herculano-Houzel of Vanderbilt University, who wrote an analysis of the studies for Science, told STAT. “Brains can appear diverse, and at the same time share profound similarities. The extent to which similar properties present themselves might be simply a matter of scale: how many neurons are available to work.”
…A second study looked in unprecedented detail at the neuroanatomy of pigeons and barn owls, finding hints to the basis of their intelligence that likely applies to corvids’, too. Scientists have long known that crows and ravens have unusually large forebrains, but unlike mammals’ forebrains—the neocortex—corvids’ do not have the 6 connected layers thought to produce higher intelligence. But theirs do have “connectivity patterns … reminiscent of the neocortex”, scientists led by Martin Stacho of Ruhr-University in Germany reported.
Specifically, the pigeons’ and owls’ neurons meet at right angles, forming computational circuits organized in columns. “The avian version of this connectivity blueprint could conceivably generate computational properties reminiscent of the [mammalian] neocortex”, they write. “Similar microcircuits … achieve largely identical cognitive outcomes from seemingly vastly different forebrains.” That is, evolution invented connected, circuit-laden brain structure at least twice.
“In theory, any brain that has a large number of neurons connected into associative circuitry … could be expected to add flexibility and complexity to behavior”, said Herculano-Houzel. “That is my favorite operational definition of intelligence: behavioral flexibility.”
That enables pigeons to home, count, and be as trainable as monkeys. But for sheer smarts we’re still in the corvid camp. A 2014 study showed that New Caledonian crows, rooks, and European jays can solve an Aesop’s Fable challenge, dropping stones into a water-filled tube to bring a floating bit of food within reach, something kids generally can’t do until age 7. These birds were the first nonhuman animals to solve the task.
The human brain remains active in the absence of explicit tasks and forms networks of correlated activity. Resting-state functional magnetic resonance imaging (rsfMRI) measures brain activity at rest, which has been linked with both cognitive and clinical outcomes. The genetic variants influencing human brain function are largely unknown.
Here we utilized rsfMRI from 44,190 individuals of multiple ancestries (37,339 in the UK Biobank) to discover and validate the common genetic variants influencing intrinsic brain activity.
We identified hundreds of novel genetic loci associated with intrinsic functional signatures (p < 2.8 × 10−11), including associations to the central executive, default mode, and salience networks involved in the triple network model of psychopathology. A number of intrinsic brain activity associated loci colocalized with brain disorder GWAS (eg. Alzheimer’s disease, Parkinson’s disease, schizophrenia) and cognition, such as 19q13.32, 17q21.31, and 2p16.1. Particularly, we detected a colocalization between one (rs429358) of the two variants in the APOE ε4 locus and function of the default mode, central executive, attention, and visual networks. Genetic correlation analysis demonstrated shared genetic influences between brain function and brain structure in the same regions. We also detected statistically-significant genetic correlations with 26 other complex traits, such as ADHD, major depressive disorder, schizophrenia, intelligence, education, sleep, subjective well-being, and neuroticism.
Common variants associated with intrinsic brain activity were enriched within regulatory element in brain tissues.
Large scientific projects in genomics and astronomy are influential not because they answer any single question but because they enable investigation of continuously arising new questions from the same data-rich sources. Advances in automated mapping of the brain’s synaptic connections (connectomics) suggest that the complicated circuits underlying brain function are ripe for analysis. We discuss benefits of mapping a mouse brain at the level of synapses.
An Unbiased Catalog of Cells and Their Synaptic Connections
Connections and Projections in the Same Animal
A Path toward Learning the Structure of Long-Term Memory
A Path toward Describing the Neuropathology of Brain Disorders
A Path toward Designing Non-biological Thinking Systems
Cognitive control is the ability to withhold a default, prepotent response in favor of a more adaptive choice. Control deficits are common across mental disorders, including depression, anxiety, and addiction. Thus, a method for improving cognitive control could be broadly useful in disorders with few effective treatments.
Here, we demonstrate closed-loop enhancement of one aspect of cognitive control by direct brain stimulation in humans. We stimulated internal capsule/striatum in participants undergoing intracranial epilepsy monitoring as they performed a cognitive control/conflict task. Stimulation enhanced performance, with the strongest effects from dorsal capsule/striatum stimulation.
We then developed a framework to detect control lapses and stimulate in response. This closed-loop approach produced larger behavioral changes than open-loop stimulation, with a slight improvement in performance change per unit of energy delivered. Finally, we decoded task performance directly from activity on a small number of electrodes, using features compatible with existing closed-loop brain implants.
Our findings are proof of concept for a new approach to treating severe mental disorders, based on directly remediating underlying cognitive deficits.
Open Philanthropy is interested in when AI systems will be able to perform various tasks that humans can perform (“AI timelines”). To inform our thinking, I investigated what evidence the human brain provides about the computational power sufficient to match its capabilities. I consulted with more than 30 experts, and considered four methods of generating estimates [simulating neurons, comparing brain region sizes to similarly powerful algorithms, laws of physics limits, & IO bandwidth/latency], focusing on floating point operations per second (FLOP/s) as a metric of computational power.
In brief, I think it more likely than not that 1015 FLOP/s is enough to perform tasks as well as the human brain (given the right software, which may be very hard to create). And I think it unlikely (<10%) that more than 1021 FLOP/s is required. [The probabilities reported here should be interpreted as subjective levels of confidence or “credences”, not as claims about objective frequencies, statistics, or “propensities” (see Peterson (2009), Chapter 7, for discussion of various alternative interpretations of probability judgments). See Muehlhauser (2017a), section 2, for discussion of some complexities involved in using these probabilities in practice.] But I’m not a neuroscientist, and the science here is very far from settled. [My academic background is in philosophy.] I offer a few more specific probabilities, keyed to one specific type of brain model, in the report’s appendix.
For context: the Fugaku supercomputer (~$1 billion) performs ~4×1017 FLOP/s, and a V100 GPU (~$10,000) performs up to ~1014 FLOP/s. [Google’s TPU supercomputer, which recently broke records in training ML systems, can also do ~4×1017 FLOP/s. NVIDIA’s newest SuperPOD can deliver ~7×1017 of AI performance. The A100, for ~$200,000, can do 5×1015 FLOP/s.] But even if my best-guesses are right, this doesn’t mean we’ll see AI systems as capable as the human brain anytime soon. In particular: actually creating/training such systems (as opposed to building computers that could in principle run them) is a substantial further challenge.
Mechanistic estimates suggesting that 1013–1017 FLOP/s would be enough to match the human brain’s task-performance seem plausible to me. Some considerations point to higher numbers; some, to lower numbers. Of these, the latter seem to me stronger.
I give less weight to functional method estimates. However, I take estimates based on the visual cortex as some weak evidence that 1013–1017 FLOP/s isn’t much too low. Some estimates based on deep neural network models of retinal neurons point to higher numbers, but I take these as even weaker evidence.
I think it unlikely that the required number of FLOP/s exceeds the bounds suggested by the limit method. However, I don’t think the method itself airtight.
Communication method estimates may well prove informative, but I haven’t vetted them.
Cognitive fatigue and boredom are two phenomenological states widely associated with limitations in cognitive control. In this paper, we present a rational analysis of the temporal structure of controlled behavior, which provides a new framework for providing a formal account of these phenomena. We suggest that in controlling behavior, the brain faces competing behavioral and computational imperatives, and must balance them by tracking their opportunity costs over time. We use this analysis to flesh out previous suggestions that feelings associated with subjective effort, like cognitive fatigue and boredom, are the phenomenological counterparts of these opportunity cost measures, rather then reflecting the depletion of resources as has often been assumed. Specifically, we propose that both fatigue and boredom reflect the competing value of particular options that require foregoing immediate reward but can improve future performance: Fatigue reflects the value of offline computation (internal to the organism) to improve future decisions, while boredom signals the value of exploratory actions (external in the world) to gather information. We demonstrate that these accounts provide a mechanistically explicit and parsimonious account for a wide array of findings related to cognitive control, integrating and re-imagining them under a single, formally rigorous framework.
Due to advances in automated image acquisition and analysis, new whole-brain connectomes beyond C. elegans are finally on the horizon. Proofreading of whole-brain automated reconstructions will require many person-years of effort, due to the huge volumes of data involved. Here we present FlyWire, an online community for proofreading neural circuits in a fly brain, and explain how its computational and social structures are organized to scale up to whole-brain connectomics. Browser-based 3D interactive segmentation by collaborative editing of a spatially chunked supervoxel graph makes it possible to distribute proofreading to individuals located virtually anywhere in the world. Information in the edit history is programmatically accessible for a variety of uses such as estimating proofreading accuracy or building incentive systems. An open community accelerates proofreading by recruiting more participants, and accelerates scientific discovery by requiring information sharing. We demonstrate how FlyWire enables circuit analysis by reconstructing and analysing the connectome of mechanosensory neurons.
Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory and activity regulation.
Here we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs.
Our results provide a foundation for further theoretical and experimental work.
Magnetic resonance imaging (MRI) continues to drive many important neuroscientific advances. However, progress in uncovering reproducible associations between individual differences in brain structure/function and behavioral phenotypes (eg. cognition, mental health) may have been undermined by typical neuroimaging sample sizes (median n = 25)1,2.
Leveraging the Adolescent Brain Cognitive Development (ABCD) Study3 (n = 11,878), we estimated the effect sizes and reproducibility of these brain-wide associations studies (BWAS) as a function of sample size.
The very largest, replicable brain-wide associations for univariate and multivariate methods werer = 0.14 andr = 0.34, respectively. In smaller samples, typical for brain-wide association studies (BWAS), irreproducible, inflated effect sizes were ubiquitous, no matter the method (univariate, multivariate).
Until sample sizes started to approach consortium-levels, BWAS were underpowered and statistical errors assured. Multiple factors contribute to replication failures4–6; here, we show that the pairing of small brain-behavioral phenotype effect sizes with sampling variability is a key element in wide-spread BWAS replication failure. Brain-behavioral phenotype associations stabilize and become more reproducible with sample sizes of N⪆2,000. While investigator-initiated brain-behavior research continues to generate hypotheses and propel innovation, large consortia are needed to usher in a new era of reproducible human brain-wide association studies.
Mammals localize sounds using information from their two ears. Localization in real-world conditions is challenging, as echoes provide erroneous information, and noises mask parts of target sounds. To better understand real-world localization we equipped a deep neural network with human ears and trained it to localize sounds in a virtual environment. The resulting model localized accurately in realistic conditions with noise and reverberation, outperforming alternative systems that lacked human ears. In simulated experiments, the network exhibited many features of human spatial hearing: sensitivity to monaural spectral cues and interaural time and level differences, integration across frequency, and biases for sound onsets. But when trained in unnatural environments without either reverberation, noise, or natural sounds, these performance characteristics deviated from those of humans. The results show how biological hearing is adapted to the challenges of real-world environments and illustrate how artificial neural networks can extend traditional ideal observer models to real-world domains.
Understanding of the evolved biological function of sleep has advanced considerably in the past decade. However, no equivalent understanding of dreams has emerged. Contemporary neuroscientific theories generally view dreams as epiphenomena, and the few proposals for their biological function are contradicted by the phenomenology of dreams themselves. Now, the recent advent of deep neural networks (DNNs) has finally provided the novel conceptual framework within which to understand the evolved function of dreams. Notably, all DNNs face the issue of overfitting as they learn, which is when performance on one data set increases but the network’s performance fails to generalize (often measured by the divergence of performance on training vs. testing data sets). This ubiquitous problem in DNNs is often solved by modelers via “noise injections” in the form of noisy or corrupted inputs. The goal of this paper is to argue that the brain faces a similar challenge of overfitting, and that nightly dreams evolved to combat the brain’s overfitting during its daily learning. That is, dreams are a biological mechanism for increasing generalizability via the creation of corrupted sensory inputs from stochastic activity across the hierarchy of neural structures. Sleep loss, specifically dream loss, leads to an overfitted brain that can still memorize and learn but fails to generalize appropriately. Herein this “overfitted brain hypothesis” is explicitly developed and then compared and contrasted with existing contemporary neuroscientific theories of dreams. Existing evidence for the hypothesis is surveyed within both neuroscience and deep learning, and a set of testable predictions are put forward that can be pursued both in vivo and in silico.
Lifelong learning and adaptability are two defining aspects of biological agents. Modern reinforcement learning (RL) approaches have shown significant progress in solving complex tasks, however once training is concluded, the found solutions are typically static and incapable of adapting to new information or perturbations. While it is still not completely understood how biological brains learn and adapt so efficiently from experience, it is believed that synaptic plasticity plays a prominent role in this process.
Inspired by this biological mechanism, we propose a search method that, instead of optimizing the weight parameters of neural networks directly, only searches for synapse-specific Hebbian learning rules that allow the network to continuously self-organize its weights during the lifetime of the agent.
We demonstrate our approach on several reinforcement learning tasks with different sensory modalities and more than 450K trainable plasticity parameters. We find that starting from completely random weights, the discovered Hebbian rules enable an agent to navigate a dynamical 2D-pixel environment; likewise they allow a simulated 3D quadrupedal robot to learn how to walk while adapting to morphological damage not seen during training and in the absence of any explicit reward or error signal in less than 100× steps.
Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. To date, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping1–5 or point-and-click typing with a 2D computer cursor6,7. However, rapid sequences of highly dexterous behaviors, such as handwriting or touch typing, might enable faster communication rates. Here, we demonstrate an intracortical BCI that can decode imagined handwriting movements from neural activity in motor cortex and translate it to text in real-time, using a novel recurrent neural network decoding approach. With this BCI, our study participant (whose hand was paralyzed) achieved typing speeds that exceed those of any other BCI yet reported: 90 characters per minute at >99% accuracy with a general-purpose autocorrect. These speeds are comparable to able-bodied smartphone typing speeds in our participant’s age group (115 characters per minute)8 and significantly close the gap between BCI-enabled typing and able-bodied typing rates. Finally, new theoretical considerations explain why temporally complex movements, such as handwriting, may be fundamentally easier to decode than point-to-point movements. Our results open a new approach for BCIs and demonstrate the feasibility of accurately decoding rapid, dexterous movements years after paralysis.
Interest in biologically inspired alternatives to backpropagation is driven by the desire to both advance connections between deep learning and neuroscience and address backpropagation’s shortcomings on tasks such as online, continual learning. However, local synaptic learning rules like those employed by the brain have so far failed to match the performance of backpropagation in deep networks.
In this study, we employ meta-learning to discover networks that learn using feedback connections and local, biologically inspired learning rules. Importantly, the feedback connections are not tied to the feedforward weights, avoiding biologically implausible weight transport. Our experiments show that meta-trained networks effectively use feedback connections to perform online credit assignment in multi-layer architectures. Surprisingly, this approach matches or exceeds a state-of-the-art gradient-based online meta-learning algorithm on regression and classification tasks, excelling in particular at continual learning.
Analysis of the weight updates employed by these models reveals that they differ qualitatively from gradient descent in a way that reduces interference between updates. Our results suggest the existence of a class of biologically plausible learning mechanisms that not only match gradient descent-based learning, but also overcome its limitations.
Large-scale simulations of spiking neural network models are an important tool for improving our understanding of the dynamics and ultimately the function of brains. However, even small mammals such as mice have on the order of 1 × 1012 synaptic connections which, in simulations, are each typically characterized by at least one floating-point value. This amounts to several terabytes of data—an unrealistic memory requirement for a single desktop machine. Large models are therefore typically simulated on distributed supercomputers which is costly and limits large-scale modelling to a few privileged research groups. In this work, we describe extensions to GeNN—our Graphical Processing Unit (GPU) accelerated spiking neural network simulator—that enable it to ‘procedurally’ generate connectivity and synaptic weights ‘on the go’ as spikes are triggered, instead of storing and retrieving them from memory. We find that GPUs are well-suited to this approach because of their raw computational power which, due to memory bandwidth limitations, is often under-utilised when simulating spiking neural networks. We demonstrate the value of our approach with a recent model of the Macaque visual cortex consisting of 4.13 × 106 neurons and 24.2 × 109 synapses. Using our new method, it can be simulated on a single GPU—a significant step forward in making large-scale brain modelling accessible to many more researchers. Our results match those obtained on a supercomputer and the simulation runs up to 35% faster on a single high-end GPU than previously on over 1000 supercomputer nodes.
Can the human brain, a complex interconnected structure of over 80 billion neurons learn to control itself at the most elemental scale—a single neuron. We directly linked the firing rate of a single (direct) neuron to the position of a box on a screen, which participants tried to control. Remarkably, all subjects upregulated the firing rate of the direct neuron in memory structures of their brain. Learning was accompanied by improved performance over trials, simultaneous decorrelation of the direct neuron to local neurons, and direct neuron to beta frequency oscillation phase-locking. Such previously unexplored neuroprosthetic skill learning within memory related brain structures, and associated beta frequency phase-locking implicates the ventral striatum. Our demonstration that humans can volitionally control neuronal activity in mnemonic structures, may provide new ways of probing the function and plasticity of human memory without exogenous stimulation.
Modafinil and methylphenidate are medications that inhibit the neuronal reuptake of dopamine, a mechanism shared with cocaine. Their use as “smart drugs” by healthy subjects poses health concerns and requires investigation. We show that methylphenidate, but not modafinil, maintained intravenous self-administration in Sprague-Dawley rats similar to cocaine. Both modafinil and methylphenidate pretreatments potentiated cocaine self-administration. Cocaine, at self-administered doses, stimulated mesolimbic dopamine levels. This effect was potentiated by methylphenidate, but not by modafinil pretreatments, indicating dopamine-dependent actions for methylphenidate, but not modafinil. Modafinil is known to facilitate electrotonic neuronal coupling by actions on gap junctions. Carbenoxolone, a gap junction inhibitor, antagonized modafinil, but not methylphenidate potentiation of cocaine self-administration. Our results indicate that modafinil shares mechanisms with cocaine and methylphenidate but has an unique pharmacological profile that includes facilitation of electrotonic coupling and lower abuse liability, which may be exploited in future therapeutic drug design for cocaine use disorder.
[Later: Levy & Calvert 2021] Darwinian evolution tends to produce energy-efficient outcomes. On the other hand, energy limits computation, be it neural and probabilistic or digital and logical.
After establishing an energy-efficient viewpoint, we define computation and construct an energy-constrained, computational function that can be optimized.
This function implies a specific distinction between ATP-consuming processes, especially computation per se vs action potentials and other costs of communication. As a result, the partitioning of ATP-consumption here differs from earlier work. A bits/J optimization of computation requires an energy audit of the human brain. Instead of using the oft-quoted 20 watts of glucose available to the brain1, 2, the partitioning and audit reveals that cortical computation consumes 0.2 watts of ATP while long-distance communication costs are over 20× greater. The bits/joule computational optimization implies a transient information rate of more than 7 bits/sec/neuron.
Significance Statement: Engineers hold up the human brain as a low energy form of computation. However from the simplest physical viewpoint, a neuron’s computation cost is remarkably larger than the best possible bits/joule—off by a factor of 108.
Here we explicate, in the context of energy consumption, a definition of neural computation that is optimal given explicit constraints. The plausibility of this definition as Nature’s perspective is supported by an energy-audit of the human brain.
The audit itself requires certain novel perspectives and calculations revealing that communication costs are 20× computational costs.
Deep neural networks (DNNs) are being increasingly used to make predictions from functional magnetic resonance imaging (fMRI) data. However, they are widely seen as uninterpretable “black boxes”, as it can be difficult to discover what input information is used by the DNN in the process, something important in both cognitive neuroscience and clinical applications. A saliency map is a common approach for producing interpretable visualizations of the relative importance of input features for a prediction. However, methods for creating maps often fail due to DNNs being sensitive to input noise, or by focusing too much on the input and too little on the model. It is also challenging to evaluate how well saliency maps correspond to the truly relevant input information, as ground truth is not always available. In this paper, we review a variety of methods for producing gradient-based saliency maps, and present a new adversarial training method we developed to make DNNs robust to input noise, with the goal of improving interpretability. We introduce two quantitative evaluation procedures for saliency map methods in fMRI, applicable whenever a DNN or linear model is being trained to decode some information from imaging data. We evaluate the procedures using a synthetic dataset where the complex activation structure is known, and on saliency maps produced for DNN and linear models for task decoding in the Human Connectome Project (HCP) dataset. Our key finding is that saliency maps produced with different methods vary widely in interpretability, in both in synthetic and HCP fMRI data. Strikingly, even when DNN and linear models decode at comparable levels of performance, DNN saliency maps score higher on interpretability than linear model saliency maps (derived via weights or gradient). Finally, saliency maps produced with our adversarial training method outperform those from other methods.
Visual illusions have been widely used to compare visual perception among birds and mammals to assess whether animals interpret and alter visual inputs like humans, or if they detect them with little or no variability.
Here, we investigated whether a nonavian reptile (Pogona vitticeps) perceives the Müller-Lyer illusion, an illusion that causes a misperception of the relative length of 2 line segments. We observed the animals’ spontaneous tendency to choose the larger food quantity (the longer line). In test trials, animals received the same food quantity presented in a spatial arrangement eliciting the size illusion in humans; control trials presented them with 2 different-sized food portions.
Bearded dragons statistically-significantly selected the larger food quantity in control trials, confirming that they maximized food intake. Group analysis revealed that in the illusory test trials, they preferentially selected the line length estimated as longer by human observers. Further control trials excluded the possibility that their choice was based on potential spatial bias related to the illusory pattern.
Our study suggests that a nonavian reptile species has the capability to be sensitive to the Müller-Lyer illusion, raising the intriguing possibility that the perceptual mechanisms underlying size estimation might be similar across amniotes.
TL;DR: We built a physical simulation of a rodent, trained it to solve a set of tasks, and analyzed the resulting networks.
Parallel developments in neuroscience and deep learning have led to mutually productive exchanges, pushing our understanding of real and artificial neural networks in sensory and cognitive systems. However, this interaction between fields is less developed in the study of motor control. In this work, we develop a virtual rodent as a platform for the grounded study of motor activity in artificial models of embodied control. We then use this platform to study motor activity across contexts by training a model to solve four complex tasks. Using methods familiar to neuroscientists, we describe the behavioral representations and algorithms employed by different layers of the network using a neuroethological approach to characterize motor activity relative to the rodent’s behavior and goals. We find that the model uses two classes of representations which respectively encode the task-specific behavioral strategies and task-invariant behavioral kinematics. These representations are reflected in the sequential activity and population dynamics of neural subpopulations. Overall, the virtual rodent facilitates grounded collaborations between deep reinforcement learning and motor neuroscience.
For many traits, males show greater variability than females, with possible implications for understanding sex differences in health and disease.
Here, the ENIGMA (Enhancing Neuro Imaging Genetics through Meta-Analysis) Consortium presents the largest-ever mega-analysis of sex differences in variability of brain structure, based on international data spanning nine decades of life.
Subcortical volumes, cortical surface area and cortical thickness were assessed in MRI data of 16,683 healthy individuals 1–90 years old (47% females).
We observed patterns of greater male than female between-subject variance for all brain measures. This pattern was stable across the lifespan for 50% of the subcortical structures, 70% of the regional area measures, and nearly all regions for thickness. Our findings that these sex differences are present in childhood implicate early life genetic or gene-environment interaction mechanisms.
The findings highlight the importance of individual differences within the sexes, that may underpin sex-specific vulnerability to disorders.
Rats can learn the complex task of navigating a car to a desired goal area.
Enriched environments enhance competency in a rodent driving task.
Driving rats maintained an interest in the car through extinction.
Tasks incorporating complex skill mastery are important for translational research.
Although rarely used, long-term behavioral training protocols provide opportunities to shape complex skills in rodent laboratory investigations that incorporate cognitive, motor, visuospatial and temporal functions to achieve desired goals.
In the current study, following preliminary research establishing that rats could be taught to drive a rodent operated vehicle (ROV) in a forward direction, as well as steer in more complex navigational patterns, male rats housed in an enriched environment were exposed to the rodent driving regime.
Compared to standard-housed rats, enriched-housed rats demonstrated more robust learning in driving performance and their interest in the ROV persisted through extinction trials. Dehydroepiandrosterone/corticosterone (DHEA/CORT) metabolite ratios in fecal samples increased in accordance with training in all animals, suggesting that driving training, regardless of housing group, enhanced markers of emotional resilience.
These results confirm the importance of enriched environments in preparing animals to engage in complex behavioral tasks. Further, behavioral models that include trained motor skills enable researchers to assess subtle alterations in motivation and behavioral response patterns that are relevant for translational research related to neurodegenerative disease and psychiatric illness.
[Keywords: enriched environment, motor skill, emotional resilience, animal learning]
The neural circuits responsible for behavior remain largely unknown. Previous efforts have reconstructed the complete circuits of small animals, with hundreds of neurons, and selected circuits for larger animals. Here we (the FlyEM project at Janelia and collaborators at Google) summarize new methods and present the complete circuitry of a large fraction of the brain of a much more complex animal, the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses, and proofread such large data sets; new methods that define cell types based on connectivity in addition to morphology; and new methods to simplify access to a large and evolving data set. From the resulting data we derive a better definition of computational compartments and their connections; an exhaustive atlas of cell examples and types, many of them novel; detailed circuits for most of the central brain; and exploration of the statistics and structure of different brain compartments, and the brain as a whole. We make the data public, with a web site and resources specifically designed to make it easy to explore, for all levels of expertise from the expert to the merely curious. The public availability of these data, and the simplified means to access it, dramatically reduces the effort needed to answer typical circuit questions, such as the identity of upstream and downstream neural partners, the circuitry of brain regions, and to link the neurons defined by our analysis with genetic reagents that can be used to study their functions.
Note: In the next few weeks, we will release a series of papers with more involved discussions. One paper will detail the hemibrain reconstruction with more extensive analysis and interpretation made possible by this dense connectome. Another paper will explore the central complex, a brain region involved in navigation, motor control, and sleep. A final paper will present insights from the mushroom body, a center of multimodal associative learning in the fly brain.
Google “hormesis” + an adversary in nature, and you will see positive benefits: bitter plant toxins, extreme heat/cold, intense exercise, ethanol, hypoxia, nicotine, even ionizing radiation.4,5Almost every stress that evolving humans inevitably encountered has a favorable effect in small doses. But one unavoidable “toxin”, encountered by most of us in the emergency department, is accused of being harmful in all cases: Sleep deprivation. …what if sleep deprivation were not always bad?
…Depression also responds to acute sleep deprivation [see Boland et al 2017], with robust evidence that one all-nighter elevates the mood.6 Sleep deprivation may prophylax against PTSD after a fear-inducing situation.7 Sleep deprivation mitigates inflammation and ischemic insult in brain cells, protecting hippocampal neurons from damage.8 12 hours of lost sleep appears to not just protect the hippocampus, but also induces neurogenesis that persists 15–30 days later.9 Yes, sleep loss increases oxidative stress and free radical formation,10 but so do exercise, fasting, and plant polyphenols.5
Sleep researchers allow a biased hypothesis to direct research. Most protocols test individuals immediately after deprivation, neglecting measurements after adequate recovery sleep. Elite athletes immediately after a competition meet criteria for ICU admission. Lactate, creatine kinase, free radicals, electrolyte abnormalities, cortisol levels and other markers appear dangerously deranged. Similarly, subjects’ psychomotor vigilance and emotional liability after staying up all night suggest severe acute stress.
…Human subjects allowed ample recovery sleep resemble subjects who did not experience sleep deprivation, trending toward better response time and less sleepiness12. What if this paradigm were applied to shift workers? What if people who undergo small doses of sleep deprivation respond like athletes—stronger? By conducting systematic and longitudinal studies on effects of sleep deprivation and optimization of the recovery process, new studies could elucidate the complete picture of human resilience.
Backpropagation (BP) has been the most successful algorithm used to train artificial neural networks. However, there are several gaps between BP and learning in biologically plausible neuronal networks of the brain (learning in the brain, or simply BL, for short), in particular, (1) it has been unclear to date, if BP can be implemented exactly via BL, (2) there is a lack of local plasticity in BP, i.e., weight updates require information that is not locally available, while BL utilizes only locally available information, and (3) there is a lack of autonomy in BP, i.e., some external control over the neural network is required (eg. switching between prediction and learning stages requires changes to dynamics and synaptic plasticity rules), while BL works fully autonomously. Bridging such gaps, i.e., understanding how BP can be approximated by BL, has been of major interest in both neuroscience and machine learning. Despite tremendous efforts, however, no previous model has bridged the gaps at a degree of demonstrating an equivalence to BP, instead, only approximations to BP have been shown.
Here, we present for the first time a framework within BL that bridges the above crucial gaps. We propose a BL model that (1) produces exactly the same updates of the neural weights as BP, while (2) employing local plasticity, i.e., all neurons perform only local computations, done simultaneously. We then modify it to an alternative BL model that (3) also works fully autonomously. Overall, our work provides important evidence for the debate on the long-disputed question whether the brain can perform BP.
Gagliano et al (Learning by association in plants, 2016) reported associative learning in pea plants. Associative learning has long been considered a behavior performed only by animals, making this claim particularly newsworthy and interesting. In the experiment, plants were trained in Y-shaped mazes for 3 days with fans and lights attached at the top of the maze. Training consisted of wind consistently preceding light from either the same or the opposite arm of the maze. When plant growth forced a decision between the two arms of the maze, fans alone were able to influence growth direction, whereas the growth direction of untrained plants was not affected by fans. However, a replication of their protocol failed to demonstrate the same result, calling for further verification and study before mainstream acceptance of this paradigm-shifting phenomenon. This replication attempt used a larger sample size and fully blinded analysis.
Associative learning is a simple learning ability found in most animals, which involves linking together two different cues. For example, the dogs in Pavlov’s famous experiment were trained to associate sound with the arrival of food, and eventually started salivating upon hearing the sound alone. Plants, like animals, are capable of complex behaviors. The snapping leaves of a Venus fly trap or the sun-tracking abilities of sunflowers are examples of instinctive responses to environmental cues that have evolved over many generations. Whether or not plants can learn during their lifetimes has remained unknown. A handful of studies have tested for associative learning in plants, the most convincing of which was published in 2016. In this study, pea plants were exposed to two signals: light, the plant version of dog food, and wind, equivalent to the sound in Pavlov’s experiment. Just as dogs salivate in response to food, plants instinctively grow towards light, whereas air flow does not affect the direction of growth. The plants were grown inside Y-shaped mazes and their ‘selection’ of one particular arm was used as a ‘read-out’ of learned behavior. The experiments trained growing plants by exposing them to wind and light from either the same direction or opposite directions. Once the plants were at the point of ‘choosing’ between the two arms, they were exposed to wind in the absence of light. Wind by itself appeared to influence the direction the trained plants took, with wind attracting plants trained with wind and light together and repelling plants trained with wind and light apart. Untrained plants remained unaffected, making random selections. These observations were interpreted as the strongest evidence of associative learning in plants and if true would have great scientific and philosophical significance. Kasey Markel therefore set out to confirm and expand on these findings by replicating the 2016 study. As many conditions as possible were kept identical, such as the training regime. The new experiments also used more plants and, most importantly, were done ‘blind’ meaning the people recording the data did not know how the plants had been trained. This ensured the expectations of the researcher would not influence the final results. The new study found no evidence for associative learning, but did not rule it out altogether. This is because some experimental details in the first study remained unknown, such as the exact model of lights and fans originally used. This work demonstrates the importance of replicating scientific experiments. In the future, Markel hopes their results will pave the way for further, rigorous testing of the hypothesis that plants can learn.
Serial-section electron microscopy is the method of choice for studying cellular structure and network connectivity in the brain. We have built a pipeline of parallel imaging using transmission electron automated microscopes (piTEAM) that scales this technology and enables the acquisition of petascale datasets containing local cortical microcircuits. The distributed platform is composed of multiple transmission electron microscopes that image, in parallel, different sections from the same block of tissue, all under control of a custom acquisition software (pyTEM) that implements 24/7 continuous autonomous imaging. The suitability of this architecture for large scale electron microscopy imaging was demonstrated by acquiring a volume of more than 1 mm3 of mouse neocortex spanning four different visual areas. Over 26,500 ultrathin tissue sections were imaged, yielding a dataset of more than 2 petabytes. Our current burst imaging rate is 500 Mpixel/s (image capture only) per microscope and net imaging rate is 100 Mpixel/s (including stage movement, image capture, quality control, and post processing). This brings the combined burst acquisition rate of the pipeline to 3 Gpixel/s and the net rate to 600 Mpixel/s with six microscopes running acquisition in parallel, which allowed imaging a cubic millimeter of mouse visual cortex at synaptic resolution in less than 6 months.
It has been repeatedly reported that motivation for listening to music is majorly driven by the latter’s emotional effect. There is a relative opposition to this approach, however, suggesting that music does not elicit true emotions. Counteracting this notion, contemporary research studies indicate that listeners do respond affectively to music providing a scientific basis in differentially approaching and registering affective responses to music as of their behavioral or biological states. Nevertheless, no studies exist that combine the behavioral and neuroscientific research domains, offering a cross-referenced neuropsychological outcome, based on a non-personalized approach specifically using a continuous response methodology with ecologically valid musical stimuli for both research domains. Our study, trying to fill this void for the first time, discusses a relevant proof-of-concept protocol, and presents the technical outline on how to multimodally measure elicited responses on evoked emotional responses when listening to music. Specifically, we showcase how we measure the structural music elements as they vary from the beginning to the end within two different compositions, suggesting how and why to analyze and compare standardized, non-personalized behavioral to electroencephalographic data. Reporting our preliminary findings based on this protocol, we focus on the electroencephalographic data collected from n = 13 participants in two separate studies (ie. different equipment and sample background), cross-referencing and cross-validating the biological side of the protocol’s structure. Our findings suggest (a) that all participants—irrespectively of the study—reacted consistently in terms of hemispheric lateralization for each stimulus (ie. uniform intra-subjective emotional reaction; non-statistically-significant differentiation in individual variability) and (b) that diverse patterns of biological representations emerge for each stimulus between the subjects in the two studies (variable inter-subjective emotional reaction; statistically-significant differentiation in group variability) pointing towards exogenous to the measurements process factors. We conclude discussing further steps and implications of our protocol approach.
In recent years, deep learning has unlocked unprecedented success in various domains, especially in image, text, and speech processing. These breakthroughs may hold promise for neuroscience and especially for brain-imaging investigators who start to analyze thousands of participants. However, deep learning is only beneficial if the data have nonlinear relationships and if they are exploitable at currently available sample sizes.
We systematically profiled the performance of deep models, kernel models, and linear models as a function of sample size on UK Biobank brain images against established machine learning references. On MNIST and Zalando Fashion, prediction accuracy consistently improved when escalating from linear models to shallow-nonlinear models, and further improved when switching to deep-nonlinear models. The more observations were available for model training, the greater the performance gain we saw.
In contrast, using structural or functional brain scans, simple linear models performed on par with more complex, highly parameterized models in age/sex prediction across increasing sample sizes. In fact, linear models kept improving as the sample size approached ~10,000 participants. Our results indicate that the increase in performance of linear models with additional data does not saturate at the limit of current feasibility.
Yet, nonlinearities of common brain scans remain largely inaccessible to both kernel and deep learning methods at any examined scale.
Physical limits do not preclude simultaneous recordings of all spikes in neocortex
Future electrodes need nontraditional materials and fabrication procedures
Challenges for dense recording include heat dissipation from interface electronics
The classic approach to measure the spiking response of neurons involves the use of metal electrodes to record extracellular potentials. Starting over 60 years ago with a single recording site, this technology now extends to ever larger numbers and densities of sites.
We argue, based on the mechanical and electrical properties of existing materials, estimates of signal-to-noise ratios, assumptions regarding extracellular space in the brain, and estimates of heat generation by the electronic interface, that it should be possible to fabricate rigid electrodes to concurrently record from essentially every neuron in the cortical mantle. This will involve fabrication with existing yet nontraditional materials and procedures. We further emphasize the need to advance materials for improved flexible electrodes as an essential advance to record from neurons in brainstem and spinal cord in moving animals.
Magic is the art of producing in the spectator an illusion of impossibility. Although the scientific study of magic is in its infancy, the advent of recent tracking algorithms based on deep learning allow now to quantify the skills of the magician in naturalistic conditions at unprecedented resolution and robustness.
In this study, we deconstructed stage magic into purely motor maneuvers and trained an artificial neural network (DeepLabCut) to follow coins as a professional magician made them appear and disappear in a series of tricks. Rather than using AI as a mere tracking tool, we conceived it as an “artificial spectator”. When the coins were not visible, the algorithm was trained to infer their location as a human spectator would (ie. in the left fist).
This created situations where the human was fooled while AI (as seen by a human) was not, and vice versa.
Magic from the perspective of the machine reveals our own cognitive biases.
Neuroimaging studies of the psychedelic state offer an unique window onto the neural basis of conscious perception and selfhood. Despite well understood pharmacological mechanisms of action, the large-scale changes in neural dynamics induced by psychedelic compounds remain poorly understood. Using source-localized, steady-state MEG recordings, we describe changes in functional connectivity following the controlled administration of LSD, psilocybin and low-dose ketamine, as well as, for comparison, the (non-psychedelic) anticonvulsant drug tiagabine. We compare both undirected and directed measures of functional connectivity between placebo and drug conditions. We observe a general decrease in directed functional connectivity for all three psychedelics, as measured by Granger causality, throughout the brain. These data support the view that the psychedelic state involves a breakdown in patterns of functional organisation or information flow in the brain. In the case of LSD, the decrease in directed functional connectivity is coupled with an increase in undirected functional connectivity, which we measure using correlation and coherence. This surprising opposite movement of directed and undirected measures is of more general interest for functional connectivity analyses, which we interpret using analytical modelling. Overall, our results uncover the neural dynamics of information flow in the psychedelic state, and highlight the importance of comparing multiple measures of functional connectivity when analysing time-resolved neuroimaging data.
The recent availability of large-scale neuroimaging cohorts (here the UK Biobank [UKB] and the Human Connectome Project [HCP]) facilitates deeper characterisation of the relationship between phenotypic and brain architecture variation in humans. We tested the association between 654,386 vertex-wise measures of cortical and subcortical morphology (from T1w and T2w MRI images) and behavioural, cognitive, psychiatric and lifestyle data. We found a statistically-significant association of grey-matter structure with 58 out of 167 UKB phenotypes spanning substance use, blood assay results, education or income level, diet, depression, being a twin as well as cognition domains (UKB discovery sample: n = 9,888). Twenty-three of the 58 associations replicated (UKB replication sample: n = 4,561; HCP, n = 1,110). In addition, differences in body size (height, weight, BMI, waist and hip circumference, body fat percentage) could account for a substantial proportion of the association, providing possible insight into previous MRI case-control studies for psychiatric disorders where case status is associated with body mass index. Using the same linear mixed model, we showed that most of the associated characteristics (eg. age, sex, body size, diabetes, being a twin, maternal smoking, body size) could be significantly predicted using all the brain measurements in out-of-sample prediction. Finally, we demonstrated other applications of our approach including a Region Of Interest (ROI) analysis that retain the vertex-wise complexity and ranking of the information contained across MRI processing options.
Highlights: Our linear mixed model approach unifies association and prediction analyses for highly dimensional vertex-wise MRI data
Grey-matter structure is associated with measures of substance use, blood assay results, education or income level, diet, depression, being a twin as well as cognition domains
Body size (height, weight, BMI, waist and hip circumference) is an important source of covariation between the phenome and grey-matter structure
Grey-matter scores quantify grey-matter based risk for the associated traits and allow to study phenotypes not collected
The most general cortical processing (“fsaverage” mesh with no smoothing) maximises the brain-morphometricity for all UKB phenotypes
In the course of his research, Sestan, an expert in developmental neurobiology, regularly ordered slices of animal and human brain tissue from various brain banks, which shipped the specimens to Yale in coolers full of ice. Sometimes the tissue arrived within three or four hours of the donor’s death. Sometimes it took more than a day. Still, Sestan and his team were able to culture, or grow, active cells from that tissue—tissue that was, for all practical purposes, entirely dead. In the right circumstances, they could actually keep the cells alive for several weeks at a stretch.
When I met with Sestan this spring, at his lab in New Haven, he took great care to stress that he was far from the only scientist to have noticed the phenomenon. “Lots of people knew this”, he said. “Lots and lots.” And yet he seems to have been one of the few to take these findings and push them forward: If you could restore activity to individual post-mortem brain cells, he reasoned to himself, what was to stop you from restoring activity to entire slices of post-mortem brain?
…The technical hurdles were immense: To perfuse a post-mortem brain, you would have to somehow run fluid through a maze of tiny capillaries that start to clot minutes after death. Everything, from the composition of the blood substitute to the speed of the fluid flow, would have to be calibrated perfectly. In 2015, Sestan struck up an email correspondence with John L. Robertson, a veterinarian and research professor in the department of biomedical engineering at Virginia Tech. For years, Robertson had been collaborating with a North Carolina company, BioMedInnovations, or BMI, on a system known as a CaVESWave—a perfusion machine capable of keeping kidneys, hearts and livers alive outside the body for long stretches. Eventually, Robertson and BMI hoped, the machine would replace cold storage as a way to preserve organs designated for transplants.
…By any measure, the contents of the paper Sestan and his team published in Nature this April were astonishing: Not only were Sestan and his team eventually able to maintain perfusion for six hours in the organs, but they managed to restore full metabolic function in most of the brain—the cells in the dead pig brains took oxygen and glucose and converted them into metabolites like carbon dioxide that are essential to life. “These findings”, the scientists write, “show that, with the appropriate interventions, the large mammalian brain retains an underappreciated capacity for normothermic restoration of microcirculation and certain molecular and cellular functions multiple hours after circulatory arrest.”
…“What’s happened, I’d argue”, says Christof Koch, the president and chief scientist at the Allen Institute for Brain Science, “is that a lot of things about the brain that we once thought were irreversible have turned out not necessarily to be so.”
Over the course of development, humans learn myriad facts about items in the world, and naturally group these items into useful categories and structures. This semantic knowledge is essential for diverse behaviors and inferences in adulthood. How is this richly structured semantic knowledge acquired, organized, deployed, and represented by neuronal networks in the brain? We address this question by studying how the nonlinear learning dynamics of deep linear networks acquires information about complex environmental structures. Our results show that this deep learning dynamics can self-organize emergent hidden representations in a manner that recapitulates many empirical phenomena in human semantic development. Such deep networks thus provide a mathematically tractable window into the development of internal neural representations through experience.
An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: What are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences?
We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species.
Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep-learning dynamics to give rise to these regularities.
[Keywords: semantic cognition, deep learning, neural networks, generative models]
Recent work on adversarial examples has demonstrated that most natural inputs can be perturbed to fool even state-of-the-art machine learning systems. But does this happen for humans as well? In this work, we investigate: what fraction of natural instances of speech can be turned into “illusions” which either alter humans’ perception or result in different people having significantly different perceptions? We first consider the McGurk effect, the phenomenon by which adding a carefully chosen video clip to the audio channel affects the viewer’s perception of what is said (McGurk and MacDonald, 1976). We obtain empirical estimates that a significant fraction of both words and sentences occurring in natural speech have some susceptibility to this effect. We also learn models for predicting McGurk illusionability. Finally we demonstrate that the Yanny or Laurel auditory illusion (Pressnitzer et al 2018) is not an isolated occurrence by generating several very different new instances. We believe that the surprising density of illusionable natural speech warrants further investigation, from the perspectives of both security and cognitive science. Supplementary videos are available at: https://www.youtube.com/playlist?list = PLaX7t1K-e_fF2iaenoKznCatm0RC37B_k.
The current state-of-the-art object recognition algorithms, deep convolutional neural networks (DCNNs), are inspired by the architecture of the mammalian visual system, and are capable of human-level performance on many tasks. However, even these algorithms make errors. As they are trained for object recognition tasks, it has been shown that DCNNs develop hidden representations that resemble those observed in the mammalian visual system. Moreover, DCNNs trained on object recognition tasks are currently among the best models we have of the mammalian visual system. This led us to hypothesize that teaching DCNNs to achieve even more brain-like representations could improve their performance. To test this, we trained DCNNs on a composite task, wherein networks were trained to: (1) classify images of objects; while (2) having intermediate representations that resemble those observed in neural recordings from monkey visual cortex. Compared with DCNNs trained purely for object categorization, DCNNs trained on the composite task had better object recognition performance and are more robust to label corruption. Interestingly, we also found that neural data was not required, but randomized data with the same statistics as neural data also boosted performance. Our results outline a new way to train object recognition networks, using strategies in which the brain—or at least the statistical properties of its activation patterns—serves as a teacher signal for training DCNNs.
Technology that translates neural activity into speech would be transformative for people who are unable to communicate as a result of neurological impairments. Decoding speech from neural activity is challenging because speaking requires very precise and rapid multi-dimensional control of vocal tract articulators.
Here we designed a neural decoder that explicitly leverages kinematic and sound representations encoded in human cortical activity to synthesize audible speech. Recurrent neural networks first decoded directly recorded cortical activity into representations of articulatory movement, and then transformed these representations into speech acoustics. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. Intermediate articulatory dynamics enhanced performance even with limited data. Decoded articulatory representations were highly conserved across speakers, enabling a component of the decoder to be transferable across participants. Furthermore, the decoder could synthesize speech when a participant silently mimed sentences.
These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication.
Finding the best stimulus for a neuron is challenging because it is impossible to test all possible stimuli. Here we used a vast, unbiased, and diverse hypothesis space encoded by a generative deep neural network model to investigate neuronal selectivity in inferotemporal cortex without making any assumptions about natural features or categories. A genetic algorithm, guided by neuronal responses, searched this space for optimal stimuli. Evolved synthetic images evoked higher firing rates than even the best natural images and revealed diagnostic features, independently of category or feature selection. This approach provides a way to investigate neural selectivity in any modality that can be represented by a neural network and challenges our understanding of neural coding in visual cortex.
Highlights: A generative deep neural network interacted with a genetic algorithm to evolve stimuli that maximized the firing of neurons in alert macaque inferotemporal and primary visual cortex.
The evolved images activated neurons more strongly than did thousands of natural images.
Distance in image space from the evolved images predicted responses of neurons to novel images.
Adaptive information seeking is critical for goal-directed behavior. Growing evidence suggests the importance of intrinsic motives such as curiosity or need for novelty, mediated through dopaminergic valuation systems, in driving information-seeking behavior. However, valuing information for its own sake can be highly suboptimal when agents need to evaluate instrumental benefit of information in a forward-looking manner.
Here we show that information-seeking behavior in humans is driven by subjective value that is shaped by both instrumental and non-instrumental motives, and that this subjective value of information (SVOI) shares a common neural code with more basic reward value. Specifically, using a task where subjects could purchase information to reduce uncertainty about outcomes of a monetary lottery, we found information purchase decisions could be captured by a computational model of SVOI incorporating utility of anticipation, a form of non-instrumental motive for information seeking, in addition to instrumental benefits.
Neurally, trial-by-trial variation in SVOI was correlated with activity in striatum and ventromedial prefrontal cortex. Furthermore, cross-categorical decoding revealed that, within these regions, SVOI and expected utility of lotteries were represented using a common code. These findings provide support for the common currency hypothesis and shed insight on neurocognitive mechanisms underlying information-seeking behavior.
Some deep artificial neural networks (ANNs) are today’s most accurate models of the primate brain’s ventral visualstream.
Using an ANN-driven image synthesis method, we found that luminous power patterns (ie. images) can be applied to primate retinae to predictably push the spiking activity of targeted V4 neural sites beyond naturally occurring levels. This method, although not yet perfect, achieves unprecedented independent control of the activity state of entire populations of V4 neural sites, even those with overlapping receptive fields.
These results show how the knowledge embedded in today’s ANN models might be used to noninvasively set desired internal brain states at neuron-level resolution, and suggest that more accurate ANN models would produce even more accurate control.
In this paper, we shall use Tulving’s seminal empirical and theoretical research including the ‘Spoon Test’ to explore memory and mental time travel and its origins and role in planning for the future. We will review the comparative research on future planning and episodic foresight in pre-verbal children and non-verbal animals to explore how this may be manifest as wordless thoughts.
[Keywords: mental time travel, episodic memory, convergent evolution of cognition, corvids, child development, subjective experience of thinking.]
Superorganisms such as social insect colonies are very successful relative to their non-social counterparts. Powerful emergent information processing capabilities would seem to contribute to their abundance, as they explore and exploit their environment collectively. In this series of three papers, we develop a Bayesian model of collective information processing, starting here with nest-finding, then examining foraging (part II) and externalized memories (pheromone territory markers) in part III. House-hunting Temnothorax ants are adept at discovering and choosing the best available nest site for their colony. Essentially, we propose that they estimate the probability each choice is best, and then choose the highest probability. Viewed this way, we propose that their behavioural algorithm can be understood as a sophisticated statistical method that predates recent mathematical advances by some tens of millions of years. Here, we develop a model of their nest finding that incorporates insights from approximate Bayesian computation as a model of collective estimation of alternative choices; and Thompson sampling, as an effective regret-minimizing decision-making rule by viewing nest choice in terms of a multi-armed bandit problem (Robbins, 1952). Our Bayesian framework points to the potential for further bio-inspired statistical techniques. It also facilitates the generation of hypotheses regarding individual and collective movement behaviours when collective decisions must be made.
Particular deep artificial neural networks (ANNs) are today’s most accurate models of the primate brain’s ventral visual stream. Here we report that, using a targeted ANN-driven image synthesis method, new luminous power patterns (ie. images) can be applied to the primate retinae to predictably push the spiking activity of targeted V4 neural sites beyond naturally occurring levels. More importantly, this method, while not yet perfect, already achieves unprecedented independent control of the activity state of entire populations of V4 neural sites, even those with overlapping receptive fields. These results show how the knowledge embedded in today’s ANN models might be used to non-invasively set desired internal brain states at neuron-level resolution, and suggest that more accurate ANN models would produce even more accurate control.
The dense circuit structure of the mammalian cerebral cortex is still unknown. With developments in 3-dimensional (3D) electron microscopy, the imaging of sizeable volumes of neuropil has become possible, but dense reconstruction of connectomes from such image data is the limiting step. Here, we report the dense reconstruction of a volume of about 500,000 μm3 from layer 4 of mouse barrel cortex, about 300× larger than previous dense reconstructions from the mammalian cerebral cortex. Using a novel reconstruction technique, FocusEM, we were able to reconstruct a total of 0.9 meters of dendrites and about 1.8 meters of axons investing only about 4,000 human work hours, about 10–25× more efficient than previous dense circuit reconstructions. We find that connectomic data alone allows the definition of inhibitory axon types that show established principles of synaptic specificity for subcellular postsynaptic compartments. We find that also a fraction of excitatory axons exhibit such subcellular target specificity. Only about 35% of inhibitory and 55% of excitatory synaptic subcellular innervation can be predicted from the geometrical availability of membrane surface, revoking coarser models of random wiring for synaptic connections in cortical layer 4. We furthermore find evidence for enhanced variability of synaptic input composition between neurons at the level of primary dendrites in cortical layer 4. Finally, we obtain evidence for Hebbian synaptic weight adaptation in at least 24% of connections; at least 35% of connections show no sign of such previous plasticity. Together, these results establish an approach to connectomic phenotyping of local dense neuronal circuitry in the mammalian cortex.
Background: Recent studies indicate increased autistic traits in musicians with absolute pitch and a higher incidence of absolute pitch in people with autism. Theoretical accounts connect both of these with shared neural principles of local hyper-connectivity and global hypo-connectivity, enhanced perceptual functioning and a detail-focused cognitive style. This is the first study to investigate absolute pitch proficiency, autistic traits and brain correlates in the same study.
Sample and Methods: Graph theoretical analysis was conducted on resting state (eyes closed and eyes open) EEG connectivity (wPLI, weighted Phase Lag Index) matrices obtained from 31 absolute pitch (AP) and 33 relative pitch (RP) professional musicians. Small Worldness, Global Clustering Coefficient and Average Path length were related to autistic traits, passive (tone identification) and active (pitch adjustment) absolute pitch proficiency and onset of musical training using Welch-two-sample-tests, correlations and general linear models.
Results: Analyses revealed increased Path length (delta 2–4 Hz), reduced Clustering (beta 13–18 Hz), reduced Small-Worldness (gamma 30–60 Hz) and increased autistic traits for AP compared to RP. Only Clustering values (beta 13–18 Hz) were predicted by both AP proficiency and autistic traits. Post-hoc single connection permutation tests among raw wPLI matrices in the beta band (13–18 Hz) revealed widely reduced interhemispheric connectivity between bilateral auditory related electrode positions along with higher connectivity between F7-F8 and F8-P9 for AP. Pitch naming ability and Pitch adjustment ability were predicted by Path length, Clustering, autistic traits and onset of musical training (for pitch adjustment) explaining 44% respectively 38% of variance.
Conclusion: Results show both shared and distinct neural features between AP and autistic traits. Differences in the beta range were associated with higher autistic traits in the same population. In general, AP musicians exhibit a widely underconnected brain with reduced functional integration and reduced small-world-property during resting state. This might be partly related to autism-specific brain connectivity, while differences in Path length and Small-Worldness reflect other ability-specific influences. This is further evidence for different pathways in the acquisition and development of absolute pitch, likely influenced by both genetic and environmental factors and their interaction.
How similar is the human mind to the sophisticated machine-learning systems that mirror its performance? Models of object categorization based on convolutional neural networks (CNNs) have achieved human-level benchmarks in assigning known labels to novel images. These advances promise to support transformative technologies such as autonomous vehicles and machine diagnosis; beyond this, they also serve as candidate models for the visual system itself—not only in their output but perhaps even in their underlying mechanisms and principles. However, unlike human vision, CNNs can be “fooled” by adversarial examples—carefully crafted images that appear as nonsense patterns to humans but are recognized as familiar objects by machines, or that appear as one object to humans and a different object to machines. This seemingly extreme divergence between human and machine classification challenges the promise of these new advances, both as applied image-recognition systems and also as models of the human mind. Surprisingly, however, little work has empirically investigated human classification of such adversarial stimuli: Does human and machine performance fundamentally diverge? Or could humans decipher such images and predict the machine’s preferred labels? Here, we show that human and machine classification of adversarial stimuli are robustly related: In eight experiments on five prominent and diverse adversarial imagesets, human subjects reliably identified the machine’s chosen label over relevant foils. This pattern persisted for images with strong antecedent identities, and even for images described as “totaly unrecognizable to human eyes”. We suggest that human intuition may be a more reliable guide to machine (mis)classification than has typically been imagined, and we explore the consequences of this result for minds and machines alike.
Successful behaviour depends on the right balance between maximizing reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimization. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. ‘Hidden state’ exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, ‘model parameter’ exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of ‘Bayes-optimal’ behaviour. Our findings provide a computational framework to understand how distinct levels of uncertainty induce different modes of information-gain in decision-making.
Background: 16p11.2 breakpoint 4 to 5 copy number variants (CNVs) increase the risk for developing autism spectrum disorder, schizophrenia, and language and cognitive impairment. In this multisite study, we aimed to quantify the effect of 16p11.2 CNVs on brain structure.
Methods: Using voxel-based and surface-based brain morphometric methods, we analyzed structural magnetic resonance imaging collected at 7 sites from 78 individuals with a deletion, 71 individuals with a duplication, and 212 individuals without a CNV.
Results: Beyond the 16p11.2-related mirror effect on global brain morphometry, we observe regional mirror differences in the insula (deletion > control > duplication). Other regions are preferentially affected by either the deletion or the duplication: the calcarine cortex and transverse temporal gyrus (deletion > control; Cohen’s d > 1), the superior and middle temporal gyri (deletion < control; Cohen’s d < −1), and the caudate and hippocampus (control > duplication; −0.5 > Cohen’s d > −1). Measures of cognition, language, and social responsiveness and the presence of psychiatric diagnoses do not influence these results.
Conclusions: The global and regional effects on brain morphometry due to 16p11.2 CNVs generalize across site, computational method, age, and sex. effect-sizes on neuroimaging and cognitive traits are comparable. Findings partially overlap with results of meta-analyses performed across psychiatric disorders. However, the lack of correlation between morphometric and clinical measures suggests that CNV-associated brain changes contribute to clinical manifestations but require additional factors for the development of the disorder. These findings highlight the power of genetic risk factors as a complement to studying groups defined by behavioral criteria.
Deep neural networks (DNNs) have recently been applied successfully to brain decoding and image reconstruction from functional magnetic resonance imaging (fMRI) activity. However, direct training of a DNN with fMRI data is often avoided because the size of available data is thought to be insufficient to train a complex network with numerous parameters. Instead, a pre-trained DNN has served as a proxy for hierarchical visual representations, and fMRI data were used to decode individual DNN features of a stimulus image using a simple linear model, which were then passed to a reconstruction module.
Here, we present our attempt to directly train a DNN model with fMRI data and the corresponding stimulus images to build an end-to-end reconstruction model. We trained a generative adversarial network with an additional loss term defined in a high-level feature space (feature loss) using up to 6,000 training data points (natural images and the fMRI responses). The trained deep generator network was tested on an independent dataset, directly producing a reconstructed image given an fMRI pattern as the input. The reconstructions obtained from the proposed method showed resemblance with both natural and artificial test stimuli. The accuracy increased as a function of the training data size, though not outperforming the decoded feature-based method with the available data size. Ablation analyses indicated that the feature loss played a critical role to achieve accurate reconstruction.
Our results suggest a potential for the end-to-end framework to learn a direct mapping between brain activity and perception given even larger datasets.
I apply recent work on “learning to think” (2015) and on PowerPlay (2011) to the incremental training of an increasingly general problem solver, continually learning to solve new tasks without forgetting previous skills. The problem solver is a single recurrent neural network (or similar general purpose computer) called ONE. ONE is unusual in the sense that it is trained in various ways, eg. by black box optimization / reinforcement learning / artificial evolution as well as supervised / unsupervised learning. For example, ONE may learn through neuroevolution to control a robot through environment-changing actions, and learn through unsupervised gradient descent to predict future inputs and vector-valued reward signals as suggested in 1990. User-given tasks can be defined through extra goal-defining input patterns, also proposed in 1990. Suppose ONE has already learned many skills. Now a copy of ONE can be re-trained to learn a new skill, eg. through neuroevolution without a teacher. Here it may profit from re-using previously learned subroutines, but it may also forget previous skills. Then ONE is retrained in PowerPlay style (2011) on stored input/output traces of (a) ONE’s copy executing the new skill and (b) previous instances of ONE whose skills are still considered worth memorizing. Simultaneously, ONE is retrained on old traces (even those of unsuccessful trials) to become a better predictor, without additional expensive interaction with the environment. More and more control and prediction skills are thus collapsed into ONE, like in the chunker-automatizer system of the neural history compressor (1991). This forces ONE to relate partially analogous skills (with shared algorithmic information) to each other, creating common subroutines in form of shared subnetworks of ONE, to greatly speed up subsequent learning of additional, novel but algorithmically related skills.
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes. Here, we address this question by leveraging recent techniques that transfer adversarial examples from computer vision models with known parameters and architecture to other models with unknown parameters and architecture, and by matching the initial processing of the human visual system. We find that adversarial examples that strongly transfer across computer vision models influence the classifications made by time-limited human observers.
Machine learning-based analysis of human functional magnetic resonance imaging (fMRI) patterns has enabled the visualization of perceptual content. However, it has been limited to the reconstruction with low-level image bases (Miyawaki et al 2008; Wen et al 2016) or to the matching to exemplars (Naselaris et al 2009; Nishimoto et al 2011). Recent work showed that visual cortical activity can be decoded (translated) into hierarchical features of a deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features (Horikawa & Kamitani, 2017).
Here, we present a novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers. We found that the generated images resembled the stimulus images (both natural images and artificial shapes) and the subjective visual content during imagery. While our model was solely trained with natural images, our method successfully generalized the reconstruction to artificial shapes, indicating that our model indeed ‘reconstructs’ or ‘generates’ images from brain activity, not simply matches to exemplars.
A natural image prior introduced by another deep neural network effectively rendered semantically meaningful details to reconstructions by constraining reconstructed images to be similar to natural images. Furthermore, human judgment of reconstructions suggests the effectiveness of combining multiple DNN layers to enhance visual quality of generated images. The results suggest that hierarchical visual information in the brain can be effectively combined to reconstruct perceptual and subjective images.
Reconstruction of neural circuits from volume electron microscopy data requires the tracing of complete cells including all their neurites. Automated approaches have been developed to perform the tracing, but without costly human proofreading their error rates are too high to obtain reliable circuit diagrams. We present a method for automated segmentation that, like the majority of previous efforts, employs convolutional neural networks, but contains in addition a recurrent pathway that allows the iterative optimization and extension of the reconstructed shape of individual neural processes. We used this technique, which we call flood-filling networks, to trace neurons in a data set obtained by serial block-face electron microscopy from a male zebra finch brain. Our method achieved a mean error-free neurite path length of 1.1 mm, an order of magnitude better than previously published approaches applied to the same dataset. Only 4 mergers were observed in a neurite test set of 97 mm path length.
General cognitive function is a prominent human trait associated with many important life outcomes1,2, including longevity3. The substantial heritability of general cognitive function is known to be polygenic, but it has had little explication in terms of the contributing genetic variants4,5,6. Here, we combined cognitive and genetic data from the CHARGE and COGENT consortia, and UK Biobank (total n = 280,360; age range = 16 to 102). We found 9,714 genome-wide statistically-significant SNPs (P<5 x 10−8) in 99 independent loci. Most showed clear evidence of functional importance. Among many novel genes associated with general cognitive function were SGCZ, ATXN1, MAPT, AUTS2, and P2RY6. Within the novel genetic loci were variants associated with neurodegenerative disorders, neurodevelopmental disorders, physical and psychiatric illnesses, brain structure, and BMI. Gene-based analyses found 536 genes statistically-significantly associated with general cognitive function; many were highly expressed in the brain, and associated with neurogenesis and dendrite gene sets. Genetic association results predicted up to 4% of general cognitive function variance in independent samples. There was significant genetic overlap between general cognitive function and information processing speed, as well as many health variables including longevity.
Dogs (Canis lupus familiaris) were domesticated from gray wolves between 20–40kya in Eurasia, yet details surrounding the process of domestication remain unclear. The vast array of phenotypes exhibited by dogs mirror numerous other domesticated animal species, a phenomenon known as the Domestication Syndrome. Here, we use signatures persisting in the dog genome to identify genes and pathways altered by the intensive selective pressures of domestication. We identified 37 candidate domestication regions containing 17.5Mb of genome sequence and 172 genes through whole-genome SNP analysis of 43 globally distributed village dogs and 10 wolves. Comparisons with three ancient dog genomes indicate that these regions reflect signatures of domestication rather than breed formation. Analysis of genes within these regions revealed a significant enrichment of gene functions linked to neural crest cell migration, differentiation and development. Genome copy number analysis identified regions of localized sequence and structural diversity, and discovered additional copy number variation at the amylase-2b locus. Overall, these results indicate that primary selection pressures targeted genes in the neural crest as well as components of the minor spliceosome, rather than genes involved in starch metabolism. Smaller jaw sizes, hairlessness, floppy ears, tameness, and diminished craniofacial development distinguish wolves from domesticated dogs, phenotypes of the Domestication Syndrome that can result from decreased neural crest cells at these sites. We propose that initial selection acted on key genes in the neural crest and minor splicing pathways during early dog domestication, giving rise to the phenotypes of modern dogs.
Imaging as a means of scientific data storage has evolved rapidly over the past century from hand drawings, to photography, to digital images. Only recently can sufficiently large datasets be acquired, stored, and processed such that tissue digitization can actually reveal more than direct observation of tissue. One field where this transformation is occurring is connectomics: the mapping of neural connections in large volumes of digitized brain tissue.
[Reply to “Can a biologist fix a radio?”; earlier, Doug the biochemist & Bill the geneticist research how cars work] There is a popular belief in neuroscience that we are primarily data limited, and that producing large, multimodal, and complex datasets will, with the help of advanced data analysis algorithms, lead to fundamental insights into the way the brain processes information. These datasets do not yet exist, and if they did we would have no way of evaluating whether or not the algorithmically-generated insights were sufficient or even correct. To address this, here we take a classical microprocessor as a model organism, and use our ability to perform arbitrary experiments on it to see if popular data analysis methods from neuroscience can elucidate the way it processes information. Microprocessors are among those artificial information processing systems that are both complex and that we understand at all levels, from the overall logical flow, via logical gates, to the dynamics of transistors. We show that the approaches reveal interesting structure in the data but do not meaningfully describe the hierarchy of information processing in the microprocessor. This suggests current analytic approaches in neuroscience may fall short of producing meaningful understanding of neural systems, regardless of the amount of data. Additionally, we argue for scientists using complex non-linear dynamical systems with known ground truth, such as the microprocessor as a validation platform for time-series and structure discovery methods.
Neuroscience is held back by the fact that it is hard to evaluate if a conclusion is correct; the complexity of the systems under study and their experimental inaccessibility make the assessment of algorithmic and data analytic techniques challenging at best. We thus argue for testing approaches using known artifacts, where the correct interpretation is known. Here we present a microprocessor platform as one such test case. We find that many approaches in neuroscience, when used naively, fall short of producing a meaningful understanding.
The overall goal of this work was to measure the efficacy of fMRI for predicting whether a dog would be a successful service dog. The training and imaging were performed in 50 dogs entering advanced training at 17–21 months of age. FMRI responses were measured while each dog observed hand signals indicating either reward or no reward and given by both a familiar handler and a stranger. 49 dogs successfully completed fMRI training and scanning. Of these, 33 eventually completed service training and were matched with a person, while 10 were released for behavioral reasons. Using anatomically defined regions-of-interest in the ventral caudate, amygdala, and visual cortex, we developed a classifier based on the dogs9 outcomes. We found that responses in the stranger condition were sufficient to develop an accurate brain-based classifier. On all data, the classifier had a positive predictive value of 96% with 10% false positives. The area under the receiver operating characteristic curve was 0.90 (0.79 with 4× cross-validation, p = 0.02), indicating a significant diagnostic capability. Within the stranger condition, the differential response to [reward—no reward] in ventral caudate was positively correlated with a successful outcome, while the differential response in the amygdala was negatively correlated to outcome. These results show that successful service dogs transfer knowledge to strangers as indexed by ventral caudate activity without excessive arousal as measured in the amygdala.
Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagating the error backward don’t have to be symmetric with the weights used for propagation the activation forward. In fact, random feedback weights work evenly well, because the network learns how to make the feedback useful.
In this work, the feedback alignment principle is used for training hidden layers more independently from the rest of the network, and from a zero initial condition. The error is propagated through fixed random feedback connections directly from the output layer to each hidden layer. This simple method is able to achieve zero training error even in convolutional networks and very deep networks, completely without error back-propagation.
The method is a step towards biologically plausible machine learning because the error signal is almost local, and no symmetric or reciprocal weights are required. Experiments show that the test performance on MNIST and CIFAR is almost as good as those obtained with back-propagation for fully connected networks. If combined with dropout, the method achieves 1.45% error on the permutation invariant MNIST task.
What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories. Afterwards, we train a Convolutional Neural Network (CNN)-based regressor to project images onto the learned manifold, thus effectively allowing machines to employ human brain-based features for automated visual classification. We use a 32-channel EEG to record brain activity of seven subjects while looking at images of 40 ImageNet object classes. The proposed RNN based approach for discriminating object classes using brain signals reaches an average accuracy of about 40%, which outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain-driven approach obtains competitive performance, comparable to those achieved by powerful CNN models, both on ImageNet and CalTech 101, thus demonstrating its classification and generalization capabilities. This gives us a real hope that, indeed, human mind can be read and transferred to machines.
Birds are remarkably intelligent, although their brains are small. Corvids and some parrots are capable of cognitive feats comparable to those of great apes. How do birds achieve impressive cognitive prowess with walnut-sized brains? We investigated the cellular composition of the brains of 28 avian species, uncovering a straightforward solution to the puzzle: brains of songbirds and parrots contain very large numbers of neurons, at neuronal densities considerably exceeding those found in mammals. Because these “extra” neurons are predominantly located in the forebrain, large parrots and corvids have the same or greater forebrain neuron counts as monkeys with much larger brains. Avian brains thus have the potential to provide much higher “cognitive power” per unit mass than do mammalian brains.
Some birds achieve primate-like levels of cognition, even though their brains tend to be much smaller in absolute size. This poses a fundamental problem in comparative and computational neuroscience, because small brains are expected to have a lower information-processing capacity.
Using the isotropic fractionator [ie. the ‘kitchen blender’ method] to determine numbers of neurons in specific brain regions, here we show that the brains of parrots and songbirds contain on average twice as many neurons as primate brains of the same mass, indicating that avian brains have higher neuron packing densities than mammalian brains. Additionally, corvids and parrots have much higher proportions of brain neurons located in the pallialtelencephalon compared with primates or other mammals and birds.
Thus, large-brained parrots and corvids have forebrain neuron counts equal to or greater than primates with much larger brains. We suggest that the large numbers of neurons concentrated in high densities in the telencephalon substantially contribute to the neural basis of avian intelligence.
[Keywords: intelligence, evolution, brain size, number of neurons, birds]
[rebuttal] How, why, and when consciousnessevolved remain hotly debated topics. Addressing these issues requires considering the distribution of consciousness across the animal phylogenetic tree.
Here we propose that at least one invertebrate clade, the insects, has a capacity for the most basic aspect of consciousness: subjective experience. In vertebrates the capacity for subjective experience is supported by integrated structures in the midbrain that create a neural simulation of the state of the mobile animal in space. This integrated and egocentric representation of the world from the animal’s perspective is sufficient for subjective experience. Structures in the insect brain perform analogous functions. Therefore, we argue the insect brain also supports a capacity for subjective experience.
In both vertebrates and insects this form of behavioral control system evolved as an efficient solution to basic problems of sensory reafference [sensory signals that occur as a result of the movement of the sensory organ] and true navigation. The brain structures that support subjective experience in vertebrates and insects are very different from each other, but in both cases they are basal to each clade. Hence we propose the origins of subjective experience can be traced to the Cambrian.
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a special type of shallow RNN is exactly equivalent to a very deep ResNet with weight sharing among the layers. A direct implementation of such a RNN, although having orders of magnitude fewer parameters, leads to a performance similar to the corresponding ResNet. We propose (1) a generalization of both RNN and ResNet architectures and (2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex. We demonstrate the effectiveness of the architectures by testing them on the CIFAR-10 and ImageNet dataset.
We show how eye-tracking corpora can be used to improve sentence compression models, presenting a novel multi-task learning algorithm based on multi-layer LSTMs. We obtain performance competitive with or better than state-of-the-art approaches.
Is thought possible without language? Individuals with global aphasia, who have almost no ability to understand or produce language, provide a powerful opportunity to find out. Surprisingly, despite their near-total loss of language, these individuals are nonetheless able to add and subtract, solve logic problems, think about another person’s thoughts, appreciate music, and successfully navigate their environments. Further, neuroimaging studies show that healthy adults strongly engage the brain’s language areas when they understand a sentence, but not when they perform other non-linguistic tasks such as arithmetic, storing information in working memory, inhibiting prepotent responses, or listening to music. Together, these two complementary lines of evidence provide a clear answer: many aspects of thought engage distinct brain regions from, and do not depend on, language.
Circuits in the cerebral cortex consist of thousands of neurons connected by millions of synapses. A precise understanding of these local networks requires relating circuit activity with the underlying network structure.
For pyramidal cells in superficial mouse visual cortex (V1), a consensus is emerging that neurons with similar visual response properties excite each other, but the anatomical basis of this recurrent synaptic network is unknown. Here we combined physiological imaging and large-scale electron microscopy to study an excitatory network in V1. We found that layer 2/layer 33 neurons organized into subnetworks defined by anatomical connectivity, with more connections within than between groups. More specifically, we found that pyramidal neurons with similar orientation selectivity preferentially formed synapses with each other, despite the fact that axons and dendrites of all orientation selectivities pass near (<5 μm) each other with roughly equal probability. Therefore, we predict that mechanisms of functionally specific connectivity take place at the length scale of spines. Neurons with similar orientation tuning formed larger synapses, potentially enhancing the net effect of synaptic specificity.
With the ability to study thousands of connections in a single circuit, functional connectomics is proving a powerful method to uncover the organizational logic of cortical networks.
Many attempts have been made to correlate degrees of both animal and human intelligence with brain properties. With respect to mammals, a much-discussed trait concerns absolute and relative brain size, either uncorrected or corrected for body size. However, the correlation of both with degrees of intelligence yields large inconsistencies, because although they are regarded as the most intelligent mammals, monkeys and apes, including humans, have neither the absolutely nor the relatively largest brains.
The best fit between brain traits and degrees of intelligence among mammals is reached by a combination of the number of cortical neurons, neuron packing density, interneuronal distance and axonal conduction velocity–factors that determine general information processing capacity (IPC), as reflected by general intelligence.
The highest IPC is found in humans, followed by the great apes, Old World and New World monkeys. The IPC of cetaceans and elephants is much lower because of a thin cortex, low neuron packing density and low axonal conduction velocity. By contrast, corvid and psittacid birds have very small and densely packed pallial neurons and relatively many neurons, which, despite very small brain volumes, might explain their high intelligence. The evolution of a syntactical and grammatical language in humans most probably has served as an additional intelligence amplifier, which may have happened in songbirds and psittacids in a convergent manner.
Background: Sleep disturbance is associated with inflammatory disease risk and all-cause mortality. Here, we assess global evidence linking sleep disturbance, sleep duration, and inflammation in adult humans.
Methods: A systematic search of English language publications was performed, with inclusion of primary research articles that characterized sleep disturbance and/or sleep duration or performed experimental sleep deprivation and assessed inflammation by levels of circulating markers. Effect sizes (ES) and 95% confidence intervals (CI) were extracted and pooled using a random effect model.
Results: A total of 72 studies (n > 50,000) were analyzed with assessment of C-reactive protein (CRP), interleukin-6 (IL-6), and tumor necrosis factor α (TNFα). Sleep disturbance was associated with higher levels of CRP (ES .12; 95% CI = 0.05-.19) and IL-6 (ES .20; 95% CI = 0.08-.31). Shorter sleep duration, but not the extreme of short sleep, was associated with higher levels of CRP (ES .09; 95% CI = 0.01-.17) but not IL-6 (ES .03; 95% CI: -.09 to .14). The extreme of long sleep duration was associated with higher levels of CRP (ES .17; 95% CI = 0.01-.34) and IL-6 (ES .11; 95% CI = 0.02–20). Neither sleep disturbances nor sleep duration was associated with TNFα. Neither experimental sleep deprivation nor sleep restriction was associated with CRP, IL-6, or TNFα. Some heterogeneity among studies was found, but there was no evidence of publication bias.
Conclusions: Sleep disturbance and long sleep duration, but not short sleep duration, are associated with increases in markers of systemic inflammation.