- See Also
- Gwern
-
Links
- “Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs”, Misra & Mahowald 2024
- “The ‘After You’ Gesture in a Bird”, Suzuki & Sugita 2024
- “[On Gricean Maxims]”, Munroe 2024
- “Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong et al 2024
- “A Philosophical Introduction to Language Models—Part I: Continuity With Classic Debates”, Millière & Buckner 2024
- “Large Language Models Are Able to Downplay Their Cognitive Abilities to Fit the Persona They Simulate”, Milička et al 2024
- “Generalization in Sensorimotor Networks Configured With Natural Language Instructions”, Riveland & Pouget 2023
- “The Architecture of a Biologically Plausible Language Organ”, Mitropolsky & Papadimitriou 2023
- “TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”, Eldan & Li 2023
- “Approaching an Unknown Communication System by Latent Space Exploration and Causal Inference”, Beguš et al 2023
- “When Watching TV Shows or Movies in Your Native Language, Do You Generally Prefer to Have the Subtitles on or Off? § By Age”, YouGov 2023
- “Human-Like Systematic Generalization through a Meta-Learning Neural Network”, Lake & Baroni 2023
- “Can Language Models Handle Recursively Nested Grammatical Structures? A Case Study on Comparing Models and Humans”, Lampinen 2022
- “Semantic Projection Recovers Rich Human Knowledge of Multiple Object Features from Word Embeddings”, Grand et al 2022
- “Exact Number Concepts Are Limited to the Verbal Count Range”, Pitt et al 2022
- “Maternal Judgments of Child Numeracy and Reading Ability Predict Gains in Academic Achievement and Interest”, Parker et al 2021
- “SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant's Perspective”, Sullivan et al 2021
- “The Impact of Bilingualism on Executive Functions in Children and Adolescents: A Systematic Review Based on the PRISMA Method”, Giovannoli et al 2020
- “Cultural Influences on Word Meanings Revealed through Large-Scale Semantic Alignment”, Thompson et al 2020
- “Can Neural Networks Acquire a Structural Bias from Raw Linguistic Data?”, Warstadt & Bowman 2020
- “Generative Adversarial Phonology: Modeling Unsupervised Phonetic and Phonological Learning With Neural Networks”, Beguš 2020
- “CiwGAN and FiwGAN: Encoding Information in Acoustic Data to Model Lexical Learning With Generative Adversarial Networks”, Beguš 2020
- “Building the Perfect Curse Word: A Psycholinguistic Investigation of the Form and Meaning of Taboo Words”, Reilly et al 2020
- “How Many Words Do We Read per Minute? A Review and Meta-Analysis of Reading Rate”, Brysbaert 2019
- “Different Languages, Similar Encoding Efficiency: Comparable Information Rates across the Human Communicative Niche”, Coupé et al 2019
- “What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models”, Ettinger 2019
- “Human Few-Shot Learning of Compositional Instructions”, Lake et al 2019
- “Humans Store about 1.5 Megabytes of Information during Language Acquisition”, Mollica & Piantadosi 2019
- “Loyal to the Group of 17’s Story—The Just Man”, Wolfe 2018
- “Alarm Calls Evoke a Visual Search Image of a Predator in Birds”, Suzuki 2018
- “Unsupervised Machine Translation Using Monolingual Corpora Only”, Lample et al 2017
- “Word Translation Without Parallel Data”, Conneau et al 2017
- “Linguistically Deprived Children: Meta-Analysis of Published Research Underlines the Importance of Early Syntactic Language Use for Normal Brain Development”, Vyshedskiy et al 2017
- “Language and Thought Are Not the Same Thing: Evidence from Neuroimaging and Neurological Patients”, Fedorenko & Varley 2016
- “The Failure of Analysis and the Nature of Concepts”, Huemer 2015
- “The Idiom Principle Revisited”, Siyanova-Chanturia & Martinez 2014
- “Ontogeny and Phylogeny of Language”, Yang 2013
- “International Art English”, Rule & Levine 2012
- “A Cross-Language Perspective On Speech Information Rate”, Pellegrino et al 2011
- “Universal Entropy of Word Ordering Across Linguistic Families”, Montemurro & Zanette 2011
- “Précis of How Children Learn the Meanings of Words”, Bloom 2010
- “Language Evolution in the Laboratory”, Scott-Phillips & Kirby 2010
- “Franz Boas and Inuktitut Terminology for Ice and Snow: From the Emergence of the Field to the ‘Great Eskimo Vocabulary Hoax’”, Krupnik & Müller-Wille 2010
- “The Structure of Individual Differences in the Cognitive Abilities of Children and Chimpanzees § Table 1. Primate Cognition Test Battery: Description of Tasks and Mean Proportion (With Standard Deviation) of Correct Responses by Chimpanzees and Human Children”, Herrmann et al 2010 (page 4)
- “Diversity in Saami Terminology for Reindeer, Snow, and Ice”, Magga 2006
- “Least Effort and the Origins of Scaling in Human Language”, Cancho & Sole 2003
- “Data Compression and Entropy Estimates by Non-Sequential Recursive Pair Substitution”, Grassberger 2002
- “Estimating and Comparing Entropy across Written Natural Languages Using PPM Compression”, Behr et al 2002
- “What Do You Care What Other People Think § It’s As Simple As One, Two, Three”, Feynman 2001
- “The Distribution of N-Grams”, Egghe 2000
- “Text Compression As a Test for Artificial Intelligence”, Mahoney 1999
- “THE ENTROPY OF ENGLISH USING PPM-BASED MODELS—Data Compression Conference, 1996. DCC '96. Proceedings”
- “Processing Linguistic Probabilities: General Principles and Empirical Evidence”, Budescu & Wallsten 1995
- “Entropy of Natural Languages: Theory and Experiment”, Levitin & Reingold 1994
- “The Perception of Rhythm in Language”, Cutler 1994
- “When It Is Better to Receive Than to Give: Syntactic and Conceptual Constraints on Vocabulary Growth”, Fisher et al 1994
- “How Could a Child Use Verb Syntax to Learn Verb Semantics?”, Pinker 1994
- “Building a Large Annotated Corpus of English: The Penn Treebank”, Marcus et al 1993
- “A Silent Childhood”, Rymer 1992
- “Writing Is a Technology That Restructures Thought”, Ong 1992
- “Epistemology and Cognition”, Fetzer 1991
- “A Dynamic Systems Model of Cognitive and Language Growth”, Geert 1991
- “An Extended Family With a Dominantly Inherited Speech Disorder”, Hurst et al 1990
- “Literacy Theory and Research: Analyses from Multiple Paradigms. Proceedings of the Annual Meeting of the National Reading Conference (39th, AUstin, Texas, November 28–December 2, 1989)”, Zutell & McCormick 1990
- “How Much Do People Remember? Some Estimates of the Quantity of Learned Information in Long-Term Memory”, Landauer 1986
- “Comparative Patterns of Reading Eye Movement in Chinese and English”
- “A Convergent Gambling Estimate Of The Entropy Of English”, Cover & King 1978
- “Interstellar Communication: Scientific Perspectives”, Ponnamperuma & Cameron 1974
- “Distributional Structure”, Harris 1954
- “Prediction and Entropy of Printed English”, Shannon 1951
- “Chance Remarks”, Pierce 1949
- “The World I Live In § XI. Before The Soul Dawn”, Keller 1904
- “An Estimate of an Upper Bound for the Entropy of English”
- Wikipedia
- Miscellaneous
- Link Bibliography
See Also
Gwern
“Subscripts For Citations”, Gwern 2020
“On the Existence of Powerful Natural Languages”, Gwern 2016
Links
“Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs”, Misra & Mahowald 2024
Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs
“The ‘After You’ Gesture in a Bird”, Suzuki & Sugita 2024
“[On Gricean Maxims]”, Munroe 2024
View External Link:
“Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong et al 2024
Grounded language acquisition through the eyes and ears of a single child
“A Philosophical Introduction to Language Models—Part I: Continuity With Classic Debates”, Millière & Buckner 2024
A Philosophical Introduction to Language Models—Part I: Continuity With Classic Debates
“Large Language Models Are Able to Downplay Their Cognitive Abilities to Fit the Persona They Simulate”, Milička et al 2024
“Generalization in Sensorimotor Networks Configured With Natural Language Instructions”, Riveland & Pouget 2023
Generalization in Sensorimotor Networks Configured with Natural Language Instructions
“The Architecture of a Biologically Plausible Language Organ”, Mitropolsky & Papadimitriou 2023
“TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”, Eldan & Li 2023
TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
“Approaching an Unknown Communication System by Latent Space Exploration and Causal Inference”, Beguš et al 2023
Approaching an unknown communication system by latent space exploration and causal inference
“When Watching TV Shows or Movies in Your Native Language, Do You Generally Prefer to Have the Subtitles on or Off? § By Age”, YouGov 2023
“Human-Like Systematic Generalization through a Meta-Learning Neural Network”, Lake & Baroni 2023
Human-like systematic generalization through a meta-learning neural network
“Can Language Models Handle Recursively Nested Grammatical Structures? A Case Study on Comparing Models and Humans”, Lampinen 2022
“Semantic Projection Recovers Rich Human Knowledge of Multiple Object Features from Word Embeddings”, Grand et al 2022
Semantic projection recovers rich human knowledge of multiple object features from word embeddings
“Exact Number Concepts Are Limited to the Verbal Count Range”, Pitt et al 2022
“Maternal Judgments of Child Numeracy and Reading Ability Predict Gains in Academic Achievement and Interest”, Parker et al 2021
“SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant's Perspective”, Sullivan et al 2021
SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant's Perspective
“The Impact of Bilingualism on Executive Functions in Children and Adolescents: A Systematic Review Based on the PRISMA Method”, Giovannoli et al 2020
“Cultural Influences on Word Meanings Revealed through Large-Scale Semantic Alignment”, Thompson et al 2020
Cultural influences on word meanings revealed through large-scale semantic alignment
“Can Neural Networks Acquire a Structural Bias from Raw Linguistic Data?”, Warstadt & Bowman 2020
Can neural networks acquire a structural bias from raw linguistic data?
“Generative Adversarial Phonology: Modeling Unsupervised Phonetic and Phonological Learning With Neural Networks”, Beguš 2020
“CiwGAN and FiwGAN: Encoding Information in Acoustic Data to Model Lexical Learning With Generative Adversarial Networks”, Beguš 2020
“Building the Perfect Curse Word: A Psycholinguistic Investigation of the Form and Meaning of Taboo Words”, Reilly et al 2020
“How Many Words Do We Read per Minute? A Review and Meta-Analysis of Reading Rate”, Brysbaert 2019
How many words do we read per minute? A review and meta-analysis of reading rate
“Different Languages, Similar Encoding Efficiency: Comparable Information Rates across the Human Communicative Niche”, Coupé et al 2019
“What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models”, Ettinger 2019
What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models
“Human Few-Shot Learning of Compositional Instructions”, Lake et al 2019
“Humans Store about 1.5 Megabytes of Information during Language Acquisition”, Mollica & Piantadosi 2019
Humans store about 1.5 megabytes of information during language acquisition
“Loyal to the Group of 17’s Story—The Just Man”, Wolfe 2018
“Alarm Calls Evoke a Visual Search Image of a Predator in Birds”, Suzuki 2018
Alarm calls evoke a visual search image of a predator in birds
“Unsupervised Machine Translation Using Monolingual Corpora Only”, Lample et al 2017
Unsupervised Machine Translation Using Monolingual Corpora Only
“Word Translation Without Parallel Data”, Conneau et al 2017
“Linguistically Deprived Children: Meta-Analysis of Published Research Underlines the Importance of Early Syntactic Language Use for Normal Brain Development”, Vyshedskiy et al 2017
“Language and Thought Are Not the Same Thing: Evidence from Neuroimaging and Neurological Patients”, Fedorenko & Varley 2016
Language and thought are not the same thing: evidence from neuroimaging and neurological patients
“The Failure of Analysis and the Nature of Concepts”, Huemer 2015
“The Idiom Principle Revisited”, Siyanova-Chanturia & Martinez 2014
“Ontogeny and Phylogeny of Language”, Yang 2013
“International Art English”, Rule & Levine 2012
View PDF:
“A Cross-Language Perspective On Speech Information Rate”, Pellegrino et al 2011
“Universal Entropy of Word Ordering Across Linguistic Families”, Montemurro & Zanette 2011
Universal Entropy of Word Ordering Across Linguistic Families
“Précis of How Children Learn the Meanings of Words”, Bloom 2010
“Language Evolution in the Laboratory”, Scott-Phillips & Kirby 2010
“Franz Boas and Inuktitut Terminology for Ice and Snow: From the Emergence of the Field to the ‘Great Eskimo Vocabulary Hoax’”, Krupnik & Müller-Wille 2010
“The Structure of Individual Differences in the Cognitive Abilities of Children and Chimpanzees § Table 1. Primate Cognition Test Battery: Description of Tasks and Mean Proportion (With Standard Deviation) of Correct Responses by Chimpanzees and Human Children”, Herrmann et al 2010 (page 4)
“Diversity in Saami Terminology for Reindeer, Snow, and Ice”, Magga 2006
“Least Effort and the Origins of Scaling in Human Language”, Cancho & Sole 2003
“Data Compression and Entropy Estimates by Non-Sequential Recursive Pair Substitution”, Grassberger 2002
Data Compression and Entropy Estimates by Non-sequential Recursive Pair Substitution
“Estimating and Comparing Entropy across Written Natural Languages Using PPM Compression”, Behr et al 2002
Estimating and Comparing Entropy across Written Natural Languages Using PPM Compression
“What Do You Care What Other People Think § It’s As Simple As One, Two, Three”, Feynman 2001
What Do You Care What Other People Think § It’s as Simple as One, Two, Three
“The Distribution of N-Grams”, Egghe 2000
“Text Compression As a Test for Artificial Intelligence”, Mahoney 1999
“THE ENTROPY OF ENGLISH USING PPM-BASED MODELS—Data Compression Conference, 1996. DCC '96. Proceedings”
“Processing Linguistic Probabilities: General Principles and Empirical Evidence”, Budescu & Wallsten 1995
Processing Linguistic Probabilities: General Principles and Empirical Evidence
“Entropy of Natural Languages: Theory and Experiment”, Levitin & Reingold 1994
“The Perception of Rhythm in Language”, Cutler 1994
“When It Is Better to Receive Than to Give: Syntactic and Conceptual Constraints on Vocabulary Growth”, Fisher et al 1994
When it is better to receive than to give: Syntactic and conceptual constraints on vocabulary growth
“How Could a Child Use Verb Syntax to Learn Verb Semantics?”, Pinker 1994
“Building a Large Annotated Corpus of English: The Penn Treebank”, Marcus et al 1993
Building a Large Annotated Corpus of English: The Penn Treebank
“A Silent Childhood”, Rymer 1992
“Writing Is a Technology That Restructures Thought”, Ong 1992
“Epistemology and Cognition”, Fetzer 1991
View PDF (31MB):
/doc/psychology/linguistics/1991-fetzer-epistemologyandcognition.pdf
“A Dynamic Systems Model of Cognitive and Language Growth”, Geert 1991
“An Extended Family With a Dominantly Inherited Speech Disorder”, Hurst et al 1990
An extended Family with a Dominantly Inherited Speech Disorder
“Literacy Theory and Research: Analyses from Multiple Paradigms. Proceedings of the Annual Meeting of the National Reading Conference (39th, AUstin, Texas, November 28–December 2, 1989)”, Zutell & McCormick 1990
“How Much Do People Remember? Some Estimates of the Quantity of Learned Information in Long-Term Memory”, Landauer 1986
“Comparative Patterns of Reading Eye Movement in Chinese and English”
Comparative patterns of reading eye movement in Chinese and English:
View PDF:
“A Convergent Gambling Estimate Of The Entropy Of English”, Cover & King 1978
“Interstellar Communication: Scientific Perspectives”, Ponnamperuma & Cameron 1974
“Distributional Structure”, Harris 1954
“Prediction and Entropy of Printed English”, Shannon 1951
“Chance Remarks”, Pierce 1949
“The World I Live In § XI. Before The Soul Dawn”, Keller 1904
“An Estimate of an Upper Bound for the Entropy of English”
Wikipedia
Miscellaneous
-
http://sparkplugged.net/2007/10/outsider-speed-rap-extraordinaire/: -
https://academic.oup.com/pnasnexus/article/2/12/pgad384/7457938 -
https://antigonejournal.com/2024/01/decipherment-linear-b/: -
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.5610&rep=rep1&type=pdf -
https://english.stackexchange.com/questions/9780/did-english-ever-have-a-formal-version-of-you -
https://hedgehogreview.com/issues/markets-and-the-good/articles/language-machinery -
https://nautil.us/the-man-who-invented-modern-probability-234497/: -
https://plato.stanford.edu/entries/generics/:View External Link:
-
https://publicdomainreview.org/collection/dictionary-of-modern-slang/ -
https://publicdomainreview.org/essay/early-androids-and-artificial-speech/ -
https://www.fhi.ox.ac.uk/wp-content/uploads/2021/08/QNRs_FHI-TR-2021-3.0.pdf -
https://www.lesswrong.com/posts/soQX8yXLbKy7cFvy8/entropy-and-short-codes:View External Link:
https://www.lesswrong.com/posts/soQX8yXLbKy7cFvy8/entropy-and-short-codes -
https://www.nytimes.com/2023/09/20/magazine/animal-communication.html -
https://www.nytimes.com/2024/05/07/science/whale-song-alphabet.html -
https://www.scientificamerican.com/article/dog-language-geniuses-are-rare-but-apparently-real/
Link Bibliography
-
2024-suzuki.pdf: “The ‘After You’ Gesture in a Bird”, -
2024-vong.pdf: “Grounded Language Acquisition through the Eyes and Ears of a Single Child”, -
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936766/: “Large Language Models Are Able to Downplay Their Cognitive Abilities to Fit the Persona They Simulate”, -
https://arxiv.org/abs/2305.07759#microsoft: “TinyStories: How Small Can Language Models Be and Still Speak Coherent English?”, -
https://arxiv.org/abs/2303.10931: “Approaching an Unknown Communication System by Latent Space Exploration and Causal Inference”, -
2022-grand.pdf: “Semantic Projection Recovers Rich Human Knowledge of Multiple Object Features from Word Embeddings”, -
https://www.frontiersin.org/articles/10.3389/fpsyg.2020.574789/full: “The Impact of Bilingualism on Executive Functions in Children and Adolescents: A Systematic Review Based on the PRISMA Method”, -
2019-brysbaert.pdf: “How Many Words Do We Read per Minute? A Review and Meta-Analysis of Reading Rate”, -
1983-wolfe-thecitadeloftheautarch-thejustman: “Loyal to the Group of 17’s Story—The Just Man”, -
1992-ong.pdf: “Writing Is a Technology That Restructures Thought”, -
1991-vangeert.pdf: “A Dynamic Systems Model of Cognitive and Language Growth”, -
1986-landauer.pdf: “How Much Do People Remember? Some Estimates of the Quantity of Learned Information in Long-Term Memory”,