- See Also
- Gwern
-
Links
- “A Rotation and a Translation Suffice: Fooling CNNs With Simple Transformations”, Engstrom et al 2024
- “Machine Learning Reveals the Control Mechanics of an Insect Wing Hinge”, Melis et al 2024
- “Supplementary Materials for Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong 2024
- “Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong et al 2024
- “Machine Learning As a Tool for Hypothesis Generation”, Jens & Sendhil 2024
- “Multi Visual Feature Fusion Based Fog Visibility Estimation for Expressway Surveillance Using Deep Learning Network”, Yang et al 2023
- “Auditing the Inference Processes of Medical-Image Classifiers by Leveraging Generative AI and the Expertise of Physicians”, DeGrave et al 2023
- “Development of Deep Ensembles to Screen for Autism and Symptom Severity Using Retinal Photographs”, Kim et al 2023
- “May the Noise Be With You: Adversarial Training without Adversarial Examples”, Arous et al 2023
- “Are Vision Transformers More Data Hungry Than Newborn Visual Systems?”, Pandey et al 2023
- “UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition”, Ding et al 2023
- “The Possibility of Making $138,000 from Shredded Banknote Pieces Using Computer Vision”, Kong 2023
- “ConvNets Match Vision Transformers at Scale”, Smith et al 2023
- “Interpret Vision Transformers As ConvNets With Dynamic Convolutions”, Zhou et al 2023
- “Artificial Intelligence-Supported Screen Reading versus Standard Double Reading in the Mammography Screening With Artificial Intelligence Trial (MASAI): a Clinical Safety Analysis of a Randomised, Controlled, Non-Inferiority, Single-Blinded, Screening Accuracy Study”, Lång et al 2023
- “Hand-Drawn Anime Line Drawing Colorization of Faces With Texture Details”, Akita et al 2023
- “High-Quality Synthetic Character Image Extraction via Distortion Recognition”, Sawada et al 2023
- “Loss of Plasticity in Deep Continual Learning”, Dohare et al 2023
- “Neural Networks Trained With SGD Learn Distributions of Increasing Complexity”, Refinetti et al 2023
- “Rosetta Neurons: Mining the Common Units in a Model Zoo”, Dravid et al 2023
- “Improving Neural Network Representations Using Human Similarity Judgments”, Muttenthaler et al 2023
- “U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning”, Hsu & Serrão 2023
- “VanillaNet: the Power of Minimalism in Deep Learning”, Chen et al 2023
- “Multi-Label Classification in Anime Illustrations Based on Hierarchical Attribute Relationships”, Lan et al 2023
- “ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification”, Taesiri et al 2023
- “Hierarchical Multi-Label Attribute Classification With Graph Convolutional Networks on Anime Illustration”, Lan et al 2023
- “Loss Landscapes Are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent”, Chiang et al 2023
- “Adding Conditional Control to Text-To-Image Diffusion Models”, Zhang et al 2023
- “Pruning Compact ConvNets for Efficient Inference”, Ghosh et al 2023
- “Does Progress on ImageNet Transfer to Real-World Datasets?”, Fang et al 2023
- “EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers”, Mahdad et al 2022
- “Pretraining Without Attention”, Wang et al 2022
- “What Do Vision Transformers Learn? A Visual Exploration”, Ghiasi et al 2022
- “A 64-Core Mixed-Signal In-Memory Compute Chip Based on Phase-Change Memory for Deep Neural Network Inference”, Gallo et al 2022
- “Simulated Automated Facial Recognition Systems As Decision-Aids in Forensic Face Matching Tasks”, Carragher & Hancock 2022
- “Interpreting Neural Networks through the Polytope Lens”, Black et al 2022
- “Predicting Sex, Age, General Cognition and Mental Health With Machine Learning on Brain Structural Connectomes”, Yeung et al 2022
- “The Power of Ensembles for Active Learning in Image Classification”, Beluch et al 2022
- “GCN-Based Multi-Modal Multi-Label Attribute Classification in Anime Illustration Using Domain-Specific Semantic Features”, Lan et al 2022
- “Deep Differentiable Logic Gate Networks”, Petersen et al 2022
- “The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes”, Kocsis et al 2022
- “Understanding the Covariance Structure of Convolutional Filters”, Trockman et al 2022
- “VICRegL: Self-Supervised Learning of Local Visual Features”, Bardes et al 2022
- “Omnigrok: Grokking Beyond Algorithmic Data”, Liu et al 2022
-
“
g.pt: Learning to Learn With Generative Models of Neural Network Checkpoints”, Peebles et al 2022 - “FastSiam: Resource-Efficient Self-Supervised Learning on a Single GPU”, Pototzky et al 2022
- “Evaluation of Transfer Learning Methods for Detecting Alzheimer’s Disease With Brain MRI”, Dhinagar et al 2022
- “Reassessing Hierarchical Correspondences between Brain and Deep Networks through Direct Interface”, Sexton & Love 2022
- “Timesweeper: Accurately Identifying Selective Sweeps Using Population Genomic Time Series”, Whitehouse & Schrider 2022
- “RHO-LOSS: Prioritized Training on Points That Are Learnable, Worth Learning, and Not Yet Learnt”, Mindermann et al 2022
- “BigVGAN: A Universal Neural Vocoder With Large-Scale Training”, Lee et al 2022
- “Continual Pre-Training Mitigates Forgetting in Language and Vision”, Cossu et al 2022
- “Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (RepLKNet)”, Ding et al 2022
- “Democratizing Contrastive Language-Image Pre-Training: A CLIP Benchmark of Data, Model, and Supervision”, Cui et al 2022
- “Variational Autoencoders Without the Variation”, Daly et al 2022
- “On the Effectiveness of Dataset Watermarking in Adversarial Settings”, Tekgul & Asokan 2022
- “Approximating CNNs With Bag-Of-Local-Features Models Works Surprisingly Well on ImageNet”, Wiel et al 2022
- “Variational Neural Cellular Automata”, Palm et al 2022
- “ConvMixer: Patches Are All You Need?”, Trockman & Kolter 2022
- “HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning”, Zhmoginov et al 2022
- “ConvNeXt: A ConvNet for the 2020s”, Liu et al 2022
- “An Empirical Investigation of the Role of Pre-Training in Lifelong Learning”, Mehta et al 2021
- “Noether Networks: Meta-Learning Useful Conserved Quantities”, Alet et al 2021
- “AugMax: Adversarial Composition of Random Augmentations for Robust Training”, Wang et al 2021
- “The Efficiency Misnomer”, Dehghani et al 2021
- “Logical Activation Functions: Logit-Space Equivalents of Probabilistic Boolean Operators”, Lowe et al 2021
- “Evaluating Loss Functions for Illustration Super-Resolution Neural Networks”, Nepomuceno & Silva 2021
- “TWIST: Self-Supervised Learning by Estimating Twin Class Distributions”, Wang et al 2021
- “Deep Learning Models of Cognitive Processes Constrained by Human Brain Connectomes”, Zhang et al 2021
- “Decoupled Contrastive Learning”, Yeh et al 2021
- “Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-Training Paradigm (DeCLIP)”, Li et al 2021
- “Mining for Strong Gravitational Lenses With Self-Supervised Learning”, Stein et al 2021
-
“
THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks”, Muttenthaler & Hebart 2021 - “A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP”, Zhao et al 2021
- “Predicting Phenotypes from Genetic, Environment, Management, and Historical Data Using CNNs”, Washburn et al 2021
- “Do Vision Transformers See Like Convolutional Neural Networks?”, Raghu et al 2021
- “Dataset Distillation With Infinitely Wide Convolutional Networks”, Nguyen et al 2021
- “Neuroprosthesis for Decoding Speech in a Paralyzed Person With Anarthria”, Moses et al 2021
- “Graph Jigsaw Learning for Cartoon Face Recognition”, Li et al 2021
- “Prediction Depth: Deep Learning Through the Lens of Example Difficulty”, Baldock et al 2021
- “Revisiting the Calibration of Modern Neural Networks”, Minderer et al 2021
- “Partial Success in Closing the Gap between Human and Machine Vision”, Geirhos et al 2021
- “CoAtNet: Marrying Convolution and Attention for All Data Sizes”, Dai et al 2021
- “Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images”, Cherti & Jitsev 2021
- “Embracing New Techniques in Deep Learning for Estimating Image Memorability”, Needell & Bainbridge 2021
- “Predicting Sex from Retinal Fundus Photographs Using Automated Deep Learning”, Korot et al 2021
- “Rethinking and Improving the Robustness of Image Style Transfer”, Wang et al 2021
- “Rip Van Winkle’s Razor, a Simple New Estimate for Adaptive Data Analysis”, Arora & Zhang 2021
- “The Surprising Impact of Mask-Head Architecture on Novel Class Segmentation”, Birodkar et al 2021
- “Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks”, Northcutt et al 2021
- “ConViT: Improving Vision Transformers With Soft Convolutional Inductive Biases”, d’Ascoli et al 2021
- “Learning from Videos to Understand the World”, Zweig et al 2021
- “Fast and Accurate Model Scaling”, Dollár et al 2021
- “Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants”, Soemers et al 2021
- “Momentum Residual Neural Networks”, Sander et al 2021
- “Hiding Data Hiding”, Wu et al 2021
- “Explaining Neural Scaling Laws”, Bahri et al 2021
- “NFNet: High-Performance Large-Scale Image Recognition Without Normalization”, Brock et al 2021
- “Brain2Pix: Fully Convolutional Naturalistic Video Reconstruction from Brain Activity”, Le et al 2021
- “Words As a Window: Using Word Embeddings to Explore the Learned Representations of Convolutional Neural Networks”, Dharmaretnam et al 2021
- “E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials”, Batzner et al 2021
- “Meta Pseudo Labels”, Pham et al 2021
- “Is MLP-Mixer a CNN in Disguise? As Part of This Blog Post, We Look at the MLP Mixer Architecture in Detail and Also Understand Why It Is Not Considered Convolution Free.”
- “Converting Tabular Data into Images for Deep Learning With Convolutional Neural Networks”, Zhu et al 2021
- “Taming Transformers for High-Resolution Image Synthesis”, Esser et al 2020
- “Ensemble Learning of Convolutional Neural Network, Support Vector Machine, and Best Linear Unbiased Predictor for Brain Age Prediction: ARAMIS Contribution to the Predictive Analytics Competition 2019 Challenge”, Couvy-Duchesne et al 2020
- “Scaling down Deep Learning”, Greydanus 2020
- “Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images”, Child 2020
- “Understanding RL Vision: With Diverse Environments, We Can Analyze, Diagnose and Edit Deep Reinforcement Learning Models Using Attribution”, Hilton et al 2020
- “Fourier Neural Operator for Parametric Partial Differential Equations”, Li et al 2020
- “Deep Learning-Based Classification of the Polar Emotions of ‘Moe’-Style Cartoon Pictures”, Cao et al 2020b
- “Demonstrating That Dataset Domains Are Largely Linearly Separable in the Feature Space of Common CNNs”, Dragan 2020
- “Optimal Peanut Butter and Banana Sandwiches”, Rosenthal 2020
- “Accuracy and Performance Comparison of Video Action Recognition Approaches”, Hutchinson et al 2020
- “A Digital Biomarker of Diabetes from Smartphone-Based Vascular Signals”, Avram et al 2020
- “SSD vs. YOLO for Detection of Outdoor Urban Advertising Panels under Multiple Variabilities”, Morera et al 2020
- “On Robustness and Transferability of Convolutional Neural Networks”, Djolonga et al 2020
- “NVAE: A Deep Hierarchical Variational Autoencoder”, Vahdat & Kautz 2020
- “CoCoNuT: Combining Context-Aware Neural Translation Models Using Ensemble for Program Repair”, Lutellier et al 2020
- “The Many Faces of Robustness: A Critical Analysis of Out-Of-Distribution Generalization”, Hendrycks et al 2020
- “SimCLRv2: Big Self-Supervised Models Are Strong Semi-Supervised Learners”, Chen et al 2020
- “FBNetV3: Joint Architecture-Recipe Search Using Predictor Pretraining”, Dai et al 2020
- “Danny Hernandez on Forecasting and the Drivers of AI Progress”, Koehler et al 2020
- “Measuring the Algorithmic Efficiency of Neural Networks”, Hernandez & Brown 2020
- “AI and Efficiency: We’re Releasing an Analysis Showing That Since 2012 the Amount of Compute Needed to Train a Neural Net to the Same Performance on ImageNet Classification Has Been Decreasing by a Factor of 2 Every 16 Months”, Hernandez & Brown 2020
- “Reinforcement Learning With Augmented Data”, Laskin et al 2020
- “YOLOv4: Optimal Speed and Accuracy of Object Detection”, Bochkovskiy et al 2020
- “Scaling Laws from the Data Manifold Dimension”, Sharma & Kaplan 2020
- “Shortcut Learning in Deep Neural Networks”, Geirhos et al 2020
- “Evolving Normalization-Activation Layers”, Liu et al 2020
- “Conditional Convolutions for Instance Segmentation”, Tian et al 2020
- “Train-By-Reconnect: Decoupling Locations of Weights from Their Values (LaPerm)”, Qiu & Suda 2020
- “Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited”, Maddox et al 2020
- “Do We Need Zero Training Loss After Achieving Zero Training Error?”, Ishida et al 2020
- “Bayesian Deep Learning and a Probabilistic Perspective of Generalization”, Wilson & Izmailov 2020
- “A Simple Framework for Contrastive Learning of Visual Representations”, Chen et al 2020
- “Growing Neural Cellular Automata: Differentiable Model of Morphogenesis”, Mordvintsev et al 2020
- “Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving”, Song et al 2020
- “First-In-Human Evaluation of a Hand-Held Automated Venipuncture Device for Rapid Venous Blood Draws”, Leipheimer et al 2020
- “ImageNet-A: Natural Adversarial Examples”, Hendrycks et al 2020
- “Deep-Eyes: Fully Automatic Anime Character Colorization With Painting of Details on Empty Pupils”, Akita et al 2020
- “CARN: Convolutional Anchored Regression Network for Fast and Accurate Single Image Super-Resolution”, Li 2020
- “Linear Mode Connectivity and the Lottery Ticket Hypothesis”, Frankle et al 2019
- “Deep Double Descent: We Show That the Double Descent Phenomenon Occurs in CNNs, ResNets, and Transformers: Performance First Improves, Then Gets Worse, and Then Improves Again With Increasing Model Size, Data Size, or Training Time. This Effect Is Often Avoided through Careful Regularization. While This Behavior Appears to Be Fairly Universal, We Don’t yet Fully Understand Why It Happens, and View Further Study of This Phenomenon As an Important Research Direction.”, Nakkiran et al 2019
- “Anonymous Market Product Classification Based on Deep Learning”, Yang et al 2019b
- “The Origins and Prevalence of Texture Bias in Convolutional Neural Networks”, Hermann et al 2019
- “How Machine Learning Can Help Unlock the World of Ancient Japan”, Lamb 2019
- “Self-Training With Noisy Student Improves ImageNet Classification”, Xie et al 2019
- “Taxonomy of Real Faults in Deep Learning Systems”, Humbatova et al 2019
- “On the Measure of Intelligence”, Chollet 2019
- “DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames”, Wijmans et al 2019
- “Accelerating Deep Learning by Focusing on the Biggest Losers”, Jiang et al 2019
- “ANIL: Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML”, Raghu et al 2019
- “ObjectNet: A Large-Scale Bias-Controlled Dataset for Pushing the Limits of Object Recognition Models”, Barbu et al 2019
- “CAR: Learned Image Downscaling for Upscaling Using Content Adaptive Resampler”, Sun & Chen 2019
- “Finding the Needle in the Haystack With Convolutions: on the Benefits of Architectural Bias”, d’Ascoli et al 2019
- “Intriguing Properties of Adversarial Training at Scale”, Xie & Yuille 2019
- “Adversarial Robustness As a Prior for Learned Representations”, Engstrom et al 2019
- “Human-Level Performance in 3D Multiplayer Games With Population-Based Reinforcement Learning”, Jaderberg et al 2019
- “ImageNet-Sketch: Learning Robust Global Representations by Penalizing Local Predictive Power”, Wang et al 2019
- “Improved Object Recognition Using Neural Networks Trained to Mimic the Brain’s Statistical Properties”, Federer et al 2019
- “Neural System Identification With Neural Information Flow”, Seeliger et al 2019
- “Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning”, Din et al 2019
- “CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features”, Yun et al 2019
- “Adversarial Examples Are Not Bugs, They Are Features”, Ilyas et al 2019
- “Searching for MobileNetV3”, Howard et al 2019
- “Billion-Scale Semi-Supervised Learning for Image Classification”, Yalniz et al 2019
- “NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection”, Ghiasi et al 2019
- “COCO-GAN: Generation by Parts via Conditional Coordinating”, Lin et al 2019
- “Benchmarking Neural Network Robustness to Common Corruptions and Perturbations”, Hendrycks & Dietterich 2019
- “Semantic Image Synthesis With Spatially-Adaptive Normalization”, Park et al 2019
- “The Bitter Lesson”, Sutton 2019
- “Learning To Follow Directions in Street View”, Hermann et al 2019
- “SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data”, Sun et al 2019
- “Real-Time Continuous Transcription With Live Transcribe”, Savla 2019
- “Do We Train on Test Data? Purging CIFAR of Near-Duplicates”, Barz & Denzler 2019
- “Pay Less Attention With Lightweight and Dynamic Convolutions”, Wu et al 2019
- “Association Between Surgical Skin Markings in Dermoscopic Images and Diagnostic Performance of a Deep Learning Convolutional Neural Network for Melanoma Recognition”, Winkler et al 2019
- “Detecting Advertising on Building Façades With Computer Vision”, Bochkarev & Smirnov 2019
- “Quantifying Generalization in Reinforcement Learning”, Cobbe et al 2018
- “ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware”, Cai et al 2018
- “ImageNet-Trained CNNs Are Biased towards Texture; Increasing Shape Bias Improves Accuracy and Robustness”, Geirhos et al 2018
- “Evolving Space-Time Neural Architectures for Videos”, Piergiovanni et al 2018
- “ADNet: A Deep Network for Detecting Adverts”, Hossari et al 2018
- “AdVersarial: Perceptual Ad Blocking Meets Adversarial Machine Learning”, Tramèr et al 2018
- “FloWaveNet: A Generative Flow for Raw Audio”, Kim et al 2018
- “StreetNet: Preference Learning With Convolutional Neural Network on Urban Crime Perception”, Fu et al 2018
- “Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation”, Wang et al 2018
- “Understanding and Correcting Pathologies in the Training of Learned Optimizers”, Metz et al 2018
- “Graph Convolutional Reinforcement Learning”, Jiang et al 2018
- “Cellular Automata As Convolutional Neural Networks”, Gilpin 2018
- “Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization”, Narayan et al 2018
- “Human-Like Playtesting With Deep Learning”, Gudmundsson et al 2018
- “CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images”, Guo et al 2018
- “MnasNet: Platform-Aware Neural Architecture Search for Mobile”, Tan et al 2018
- “LEO: Meta-Learning With Latent Embedding Optimization”, Rusu et al 2018
- “Glow: Generative Flow With Invertible 1×1 Convolutions”, Kingma & Dhariwal 2018
- “Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations”, Hendrycks & Dietterich 2018
- “Confounding Variables Can Degrade Generalization Performance of Radiological Deep Learning Models”, Zech et al 2018
- “Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks”, Xiao et al 2018
- “Resource-Efficient Neural Architect”, Zhou et al 2018
- “More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch”, Calandra et al 2018
- “Deep Learning Generalizes Because the Parameter-Function Map Is Biased towards Simple Functions”, Valle-Pérez et al 2018
- “Bidirectional Learning for Robust Neural Networks”, Pontes-Filho & Liwicki 2018
- “BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning”, Yu et al 2018
- “Self-Distillation: Born Again Neural Networks”, Furlanello et al 2018
- “Tile2Vec: Unsupervised Representation Learning for Spatially Distributed Data”, Jean et al 2018
- “Exploring the Limits of Weakly Supervised Pretraining”, Mahajan et al 2018
- “YOLOv3: An Incremental Improvement”, Redmon & Farhadi 2018
- “Reptile: On First-Order Meta-Learning Algorithms”, Nichol et al 2018
- “Essentially No Barriers in Neural Network Energy Landscape”, Draxler et al 2018
- “Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs”, Garipov et al 2018
- “Guess, Check and Fix: a Phenomenology of Improvisation in ‘Neural’ Painting”, Choi 2018
- “Sim-To-Real Optimization of Complex Real World Mobile Network With Imperfect Information via Deep Reinforcement Learning from Self-Play”, Tan et al 2018
- “Evolved Policy Gradients”, Houthooft et al 2018
- “Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-Of-The-Art Deep Artificial Neural Networks”, Rajalingham et al 2018
- “IMPALA: Scalable Distributed Deep-RL With Importance Weighted Actor-Learner Architectures”, Espeholt et al 2018
- “Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts”, Zhou et al 2018
- “ArcFace: Additive Angular Margin Loss for Deep Face Recognition”, Deng et al 2018
- “Man against Machine: Diagnostic Performance of a Deep Learning Convolutional Neural Network for Dermoscopic Melanoma Recognition in Comparison to 58 Dermatologists”, Haenssle et al 2018
- “DeepGS: Predicting Phenotypes from Genotypes Using Deep Learning”, Ma et al 2017
- “Deep Image Reconstruction from Human Brain Activity”, Shen et al 2017
- “SPP-Net: Deep Absolute Pose Regression With Synthetic Views”, Purkait et al 2017
- “China’s AI Advances Help Its Tech Industry, and State Security”, Mozur & Bradsher 2017
- “Measuring the Tendency of CNNs to Learn Surface Statistical Regularities”, Jo & Bengio 2017
- “3D Semantic Segmentation With Submanifold Sparse Convolutional Networks”, Graham et al 2017
- “BlockDrop: Dynamic Inference Paths in Residual Networks”, Wu et al 2017
- “Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN”, Gao et al 2017
- “The Signature of Robot Action Success in EEG Signals of a Human Observer: Decoding and Visualization Using Deep Convolutional Neural Networks”, Behncke et al 2017
- “11K Hands: Gender Recognition and Biometric Identification Using a Large Dataset of Hand Images”, Afifi 2017
- “Learning to Play Chess With Minimal Lookahead and Deep Value Neural Networks”, Sabatelli 2017 (page 3)
- “Learning to Generalize: Meta-Learning for Domain Generalization”, Li et al 2017
- “High-Precision Automated Reconstruction of Neurons With Flood-Filling Networks”, Januszewski et al 2017
- “Efficient K-Shot Learning With Regularized Deep Networks”, Yoo et al 2017
- “NIMA: Neural Image Assessment”, Talebi & Milanfar 2017
- “Squeeze-And-Excitation Networks”, Hu et al 2017
- “What Does a Convolutional Neural Network Recognize in the Moon?”, Shoji 2017
- “SMASH: One-Shot Model Architecture Search through HyperNetworks”, Brock et al 2017
- “A Deep Architecture for Unified Esthetic Prediction”, Murray & Gordo 2017
- “Learning With Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback”, Li et al 2017
- “WebVision Database: Visual Learning and Understanding from Web Data”, Li et al 2017
- “Focal Loss for Dense Object Detection”, Lin et al 2017
- “Active Learning for Convolutional Neural Networks: A Core-Set Approach”, Sener & Savarese 2017
- “Learning to Infer Graphics Programs from Hand-Drawn Images”, Ellis et al 2017
- “A Downsampled Variant of ImageNet As an Alternative to the CIFAR Datasets”, Chrabaszcz et al 2017
- “Learning Transferable Architectures for Scalable Image Recognition”, Zoph et al 2017
- “Efficient Architecture Search by Network Transformation”, Cai et al 2017
- “A Simple Neural Attentive Meta-Learner”, Mishra et al 2017
- “Towards Deep Learning Models Resistant to Adversarial Attacks”, Madry et al 2017
- “Gradient Diversity: a Key Ingredient for Scalable Distributed Learning”, Yin et al 2017
- “Device Placement Optimization With Reinforcement Learning”, Mirhoseini et al 2017
- “Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour”, Goyal et al 2017
- “Submanifold Sparse Convolutional Networks”, Graham & Maaten 2017
- “A Simple Neural Network Module for Relational Reasoning”, Santoro et al 2017
- “Deep Learning Is Robust to Massive Label Noise”, Rolnick et al 2017
- “What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features”, Zhang et al 2017
- “Picasso: A Modular Framework for Visualizing the Learning Process of Neural Network Image Classifiers”, Henderson & Rothe 2017
- “BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography”, Wilber et al 2017
- “Adversarial Neural Machine Translation”, Wu et al 2017
- “Multi-Scale Dense Networks for Resource Efficient Image Classification”, Huang et al 2017
- “Scaling the Scattering Transform: Deep Hybrid Networks”, Oyallon et al 2017
- “Mask R-CNN”, He et al 2017
- “Using Human Brain Activity to Guide Machine Learning”, Fong et al 2017
- “Learned Optimizers That Scale and Generalize”, Wichrowska et al 2017
- “Prediction and Control With Temporal Segment Models”, Mishra et al 2017
- “Parallel Multiscale Autoregressive Density Estimation”, Reed et al 2017
- “Convolution Aware Initialization”, Aghajanyan 2017
- “Gender-From-Iris or Gender-From-Mascara?”, Kuehlkamp et al 2017
- “BrainNetCNN: Convolutional Neural Networks for Brain Networks; towards Predicting Neurodevelopment”, Kawahara et al 2017
- “Universal Representations: The Missing Link between Faces, Text, Planktons, and Cat Breeds”, Bilen & Vedaldi 2017
- “PixelCNN++: Improving the PixelCNN With Discretized Logistic Mixture Likelihood and Other Modifications”, Salimans et al 2017
- “YOLO9000: Better, Faster, Stronger”, Redmon & Farhadi 2016
- “Language Modeling With Gated Convolutional Networks”, Dauphin et al 2016
- “Learning from Simulated and Unsupervised Images through Adversarial Training”, Shrivastava et al 2016
- “LipNet: End-To-End Sentence-Level Lipreading”, Assael et al 2016
- “Self-Critical Sequence Training for Image Captioning”, Rennie et al 2016
- “ResNeXt: Aggregated Residual Transformations for Deep Neural Networks”, Xie et al 2016
- “Responses to Critiques on Machine Learning of Criminality Perceptions (Addendum of ArXiv:1611.04135)”, Wu & Zhang 2016
- “Understanding Deep Learning Requires Rethinking Generalization”, Zhang et al 2016
- “Designing Neural Network Architectures Using Reinforcement Learning”, Baker et al 2016
- “VPN: Video Pixel Networks”, Kalchbrenner et al 2016
- “HyperNetworks”, Ha et al 2016
- “Neural Photo Editing With Introspective Adversarial Networks”, Brock et al 2016
- “On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima”, Keskar et al 2016
- “WaveNet: A Generative Model for Raw Audio”, Oord et al 2016
- “Direct Feedback Alignment Provides Learning in Deep Neural Networks”, Nøkland 2016
- “Deep Learning Human Mind for Automated Visual Classification”, Spampinato et al 2016
- “Temporal Convolutional Networks: A Unified Approach to Action Segmentation”, Lea et al 2016
- “DenseNet: Densely Connected Convolutional Networks”, Huang et al 2016
- “Clockwork Convnets for Video Semantic Segmentation”, Shelhamer et al 2016
- “Deep Learning the City: Quantifying Urban Perception At A Global Scale”, Dubey et al 2016
- “Convolutional Neural Fabrics”, Saxena & Verbeek 2016
- “Deep Neural Networks Are Robust to Weight Binarization and Other Non-Linear Distortions”, Merolla et al 2016
- “FractalNet: Ultra-Deep Neural Networks without Residuals”, Larsson et al 2016
- “Wide Residual Networks”, Zagoruyko & Komodakis 2016
- “Residual Networks Behave Like Ensembles of Relatively Shallow Networks”, Veit et al 2016
- “ViZDoom: A Doom-Based AI Research Platform for Visual Reinforcement Learning”, Kempka et al 2016
- “OHEM: Training Region-Based Object Detectors With Online Hard Example Mining”, Shrivastava et al 2016
- “Deep Networks With Stochastic Depth”, Huang et al 2016
- “Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Esser et al 2016
- “Do Deep Convolutional Nets Really Need to Be Deep and Convolutional?”, Urban et al 2016
- “XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks”, Rastegari et al 2016
- “Learning Hand-Eye Coordination for Robotic Grasping With Deep Learning and Large-Scale Data Collection”, Levine et al 2016
- “Network Morphism”, Wei et al 2016
- “Inception-V4, Inception-ResNet and the Impact of Residual Connections on Learning”, Szegedy et al 2016
- “PlaNet—Photo Geolocation With Convolutional Neural Networks”, Weyand et al 2016
- “Value Iteration Networks”, Tamar et al 2016
- “PixelRNN: Pixel Recurrent Neural Networks”, Oord et al 2016
-
“Image Synthesis from Yahoo’s
open_nsfw”, Goh 2016 - “Deep Residual Learning for Image Recognition”, He et al 2015
- “Microsoft Researchers Win ImageNet Computer Vision Challenge”, Linn 2015
- “Adding Gradient Noise Improves Learning for Very Deep Networks”, Neelakantan et al 2015
- “The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition”, Krause et al 2015
- “Learning Visual Features from Large Weakly Supervised Data”, Joulin et al 2015
-
“
Illustration2Vec: a Semantic Vector Representation of Illustrations”, Masaki & Matsui 2015 - “BinaryConnect: Training Deep Neural Networks With Binary Weights during Propagations”, Courbariaux et al 2015
- “Predicting and Understanding Urban Perception With Convolutional Neural Networks”, Porzi et al 2015
- “A Neural Attention Model for Abstractive Sentence Summarization”, Rush et al 2015
- “LSUN: Construction of a Large-Scale Image Dataset Using Deep Learning With Humans in the Loop”, Yu et al 2015
- “You Only Look Once: Unified, Real-Time Object Detection”, Redmon et al 2015
- “Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification”, Xiao et al 2015
- “Dropout As a Bayesian Approximation: Representing Model Uncertainty in Deep Learning”, Gal & Ghahramani 2015
- “Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks”, Ren et al 2015
- “Cyclical Learning Rates for Training Neural Networks”, Smith 2015
- “Deep Learning”, LeCun et al 2015
- “Fast R-CNN”, Girshick 2015
- “End-To-End Training of Deep Visuomotor Policies”, Levine et al 2015
- “FaceNet: A Unified Embedding for Face Recognition and Clustering”, Schroff et al 2015
- “DeepID3: Face Recognition With Very Deep Neural Networks”, Sun et al 2015
- “Understanding Image Representations by Measuring Their Equivariance and Equivalence”, Lenc & Vedaldi 2014
- “Going Deeper With Convolutions”, Szegedy et al 2014
- “Very Deep Convolutional Networks for Large-Scale Image Recognition”, Simonyan & Zisserman 2014
- “ImageNet Large Scale Visual Recognition Challenge”, Russakovsky et al 2014
- “Deep Learning Face Representation by Joint Identification-Verification”, Sun et al 2014
- “Network In Network”, Lin et al 2013
- “R-CNN: Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation”, Girshick et al 2013
- “ImageNet Classification With Deep Convolutional Neural Networks”, Krizhevsky et al 2012
- “Multi-Column Deep Neural Network for Traffic Sign Classification”, Cireşan et al 2012b
- “Multi-Column Deep Neural Networks for Image Classification”, Cireşan et al 2012
- “Building High-Level Features Using Large Scale Unsupervised Learning”, Le et al 2011
- “DanNet: Flexible, High Performance Convolutional Neural Networks for Image Classification”, Ciresan et al 2011
- “Hierarchical Object Detection With Deep Reinforcement Learning”
- “Now Anyone Can Train Imagenet in 18 Minutes”
- Wikipedia
- Miscellaneous
- Link Bibliography
See Also
Gwern
“InvertOrNot.com Proposal”, Gwern 2021
Links
“A Rotation and a Translation Suffice: Fooling CNNs With Simple Transformations”, Engstrom et al 2024
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations
“Machine Learning Reveals the Control Mechanics of an Insect Wing Hinge”, Melis et al 2024
Machine learning reveals the control mechanics of an insect wing hinge
“Supplementary Materials for Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong 2024
“Grounded Language Acquisition through the Eyes and Ears of a Single Child”, Vong et al 2024
Grounded language acquisition through the eyes and ears of a single child
“Machine Learning As a Tool for Hypothesis Generation”, Jens & Sendhil 2024
“Multi Visual Feature Fusion Based Fog Visibility Estimation for Expressway Surveillance Using Deep Learning Network”, Yang et al 2023
“Auditing the Inference Processes of Medical-Image Classifiers by Leveraging Generative AI and the Expertise of Physicians”, DeGrave et al 2023
“Development of Deep Ensembles to Screen for Autism and Symptom Severity Using Retinal Photographs”, Kim et al 2023
Development of Deep Ensembles to Screen for Autism and Symptom Severity Using Retinal Photographs
“May the Noise Be With You: Adversarial Training without Adversarial Examples”, Arous et al 2023
May the Noise be with you: Adversarial Training without Adversarial Examples
“Are Vision Transformers More Data Hungry Than Newborn Visual Systems?”, Pandey et al 2023
Are Vision Transformers More Data Hungry Than Newborn Visual Systems?
“UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition”, Ding et al 2023
“The Possibility of Making $138,000 from Shredded Banknote Pieces Using Computer Vision”, Kong 2023
The possibility of making $138,000 from shredded banknote pieces using computer vision
“ConvNets Match Vision Transformers at Scale”, Smith et al 2023
“Interpret Vision Transformers As ConvNets With Dynamic Convolutions”, Zhou et al 2023
Interpret Vision Transformers as ConvNets with Dynamic Convolutions
“Artificial Intelligence-Supported Screen Reading versus Standard Double Reading in the Mammography Screening With Artificial Intelligence Trial (MASAI): a Clinical Safety Analysis of a Randomised, Controlled, Non-Inferiority, Single-Blinded, Screening Accuracy Study”, Lång et al 2023
“Hand-Drawn Anime Line Drawing Colorization of Faces With Texture Details”, Akita et al 2023
Hand-drawn anime line drawing colorization of faces with texture details
“High-Quality Synthetic Character Image Extraction via Distortion Recognition”, Sawada et al 2023
High-Quality Synthetic Character Image Extraction via Distortion Recognition
“Loss of Plasticity in Deep Continual Learning”, Dohare et al 2023
“Neural Networks Trained With SGD Learn Distributions of Increasing Complexity”, Refinetti et al 2023
Neural networks trained with SGD learn distributions of increasing complexity
“Rosetta Neurons: Mining the Common Units in a Model Zoo”, Dravid et al 2023
“Improving Neural Network Representations Using Human Similarity Judgments”, Muttenthaler et al 2023
Improving neural network representations using human similarity judgments
“U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning”, Hsu & Serrão 2023
U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning
“VanillaNet: the Power of Minimalism in Deep Learning”, Chen et al 2023
“Multi-Label Classification in Anime Illustrations Based on Hierarchical Attribute Relationships”, Lan et al 2023
Multi-Label Classification in Anime Illustrations Based on Hierarchical Attribute Relationships
“ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification”, Taesiri et al 2023
“Hierarchical Multi-Label Attribute Classification With Graph Convolutional Networks on Anime Illustration”, Lan et al 2023
“Loss Landscapes Are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent”, Chiang et al 2023
“Adding Conditional Control to Text-To-Image Diffusion Models”, Zhang et al 2023
Adding Conditional Control to Text-to-Image Diffusion Models
“Pruning Compact ConvNets for Efficient Inference”, Ghosh et al 2023
“Does Progress on ImageNet Transfer to Real-World Datasets?”, Fang et al 2023
“EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers”, Mahdad et al 2022
EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers
“Pretraining Without Attention”, Wang et al 2022
“What Do Vision Transformers Learn? A Visual Exploration”, Ghiasi et al 2022
“A 64-Core Mixed-Signal In-Memory Compute Chip Based on Phase-Change Memory for Deep Neural Network Inference”, Gallo et al 2022
“Simulated Automated Facial Recognition Systems As Decision-Aids in Forensic Face Matching Tasks”, Carragher & Hancock 2022
Simulated automated facial recognition systems as decision-aids in forensic face matching tasks
“Interpreting Neural Networks through the Polytope Lens”, Black et al 2022
“Predicting Sex, Age, General Cognition and Mental Health With Machine Learning on Brain Structural Connectomes”, Yeung et al 2022
“The Power of Ensembles for Active Learning in Image Classification”, Beluch et al 2022
The Power of Ensembles for Active Learning in Image Classification
“GCN-Based Multi-Modal Multi-Label Attribute Classification in Anime Illustration Using Domain-Specific Semantic Features”, Lan et al 2022
“Deep Differentiable Logic Gate Networks”, Petersen et al 2022
“The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes”, Kocsis et al 2022
The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes
“Understanding the Covariance Structure of Convolutional Filters”, Trockman et al 2022
Understanding the Covariance Structure of Convolutional Filters
“VICRegL: Self-Supervised Learning of Local Visual Features”, Bardes et al 2022
“Omnigrok: Grokking Beyond Algorithmic Data”, Liu et al 2022
“g.pt: Learning to Learn With Generative Models of Neural Network Checkpoints”, Peebles et al 2022
g.pt: Learning to Learn with Generative Models of Neural Network Checkpoints
“FastSiam: Resource-Efficient Self-Supervised Learning on a Single GPU”, Pototzky et al 2022
FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU
“Evaluation of Transfer Learning Methods for Detecting Alzheimer’s Disease With Brain MRI”, Dhinagar et al 2022
Evaluation of Transfer Learning Methods for Detecting Alzheimer’s Disease with Brain MRI
“Reassessing Hierarchical Correspondences between Brain and Deep Networks through Direct Interface”, Sexton & Love 2022
Reassessing hierarchical correspondences between brain and deep networks through direct interface
“Timesweeper: Accurately Identifying Selective Sweeps Using Population Genomic Time Series”, Whitehouse & Schrider 2022
Timesweeper: Accurately Identifying Selective Sweeps Using Population Genomic Time Series
“RHO-LOSS: Prioritized Training on Points That Are Learnable, Worth Learning, and Not Yet Learnt”, Mindermann et al 2022
RHO-LOSS: Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt
“BigVGAN: A Universal Neural Vocoder With Large-Scale Training”, Lee et al 2022
BigVGAN: A Universal Neural Vocoder with Large-Scale Training
“Continual Pre-Training Mitigates Forgetting in Language and Vision”, Cossu et al 2022
Continual Pre-Training Mitigates Forgetting in Language and Vision
“Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (RepLKNet)”, Ding et al 2022
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (RepLKNet)
“Democratizing Contrastive Language-Image Pre-Training: A CLIP Benchmark of Data, Model, and Supervision”, Cui et al 2022
“Variational Autoencoders Without the Variation”, Daly et al 2022
“On the Effectiveness of Dataset Watermarking in Adversarial Settings”, Tekgul & Asokan 2022
On the Effectiveness of Dataset Watermarking in Adversarial Settings
“Approximating CNNs With Bag-Of-Local-Features Models Works Surprisingly Well on ImageNet”, Wiel et al 2022
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
“Variational Neural Cellular Automata”, Palm et al 2022
“ConvMixer: Patches Are All You Need?”, Trockman & Kolter 2022
“HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning”, Zhmoginov et al 2022
HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning
“ConvNeXt: A ConvNet for the 2020s”, Liu et al 2022
“An Empirical Investigation of the Role of Pre-Training in Lifelong Learning”, Mehta et al 2021
An Empirical Investigation of the Role of Pre-training in Lifelong Learning
“Noether Networks: Meta-Learning Useful Conserved Quantities”, Alet et al 2021
“AugMax: Adversarial Composition of Random Augmentations for Robust Training”, Wang et al 2021
AugMax: Adversarial Composition of Random Augmentations for Robust Training
“The Efficiency Misnomer”, Dehghani et al 2021
“Logical Activation Functions: Logit-Space Equivalents of Probabilistic Boolean Operators”, Lowe et al 2021
Logical Activation Functions: Logit-space equivalents of Probabilistic Boolean Operators
“Evaluating Loss Functions for Illustration Super-Resolution Neural Networks”, Nepomuceno & Silva 2021
Evaluating Loss Functions for Illustration Super-Resolution Neural Networks
“TWIST: Self-Supervised Learning by Estimating Twin Class Distributions”, Wang et al 2021
TWIST: Self-Supervised Learning by Estimating Twin Class Distributions
“Deep Learning Models of Cognitive Processes Constrained by Human Brain Connectomes”, Zhang et al 2021
Deep learning models of cognitive processes constrained by human brain connectomes
“Decoupled Contrastive Learning”, Yeh et al 2021
“Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-Training Paradigm (DeCLIP)”, Li et al 2021
“Mining for Strong Gravitational Lenses With Self-Supervised Learning”, Stein et al 2021
Mining for strong gravitational lenses with self-supervised learning
“THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks”, Muttenthaler & Hebart 2021
“A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP”, Zhao et al 2021
A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP
“Predicting Phenotypes from Genetic, Environment, Management, and Historical Data Using CNNs”, Washburn et al 2021
Predicting phenotypes from genetic, environment, management, and historical data using CNNs
“Do Vision Transformers See Like Convolutional Neural Networks?”, Raghu et al 2021
Do Vision Transformers See Like Convolutional Neural Networks?
“Dataset Distillation With Infinitely Wide Convolutional Networks”, Nguyen et al 2021
Dataset Distillation with Infinitely Wide Convolutional Networks
“Neuroprosthesis for Decoding Speech in a Paralyzed Person With Anarthria”, Moses et al 2021
Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
“Graph Jigsaw Learning for Cartoon Face Recognition”, Li et al 2021
“Prediction Depth: Deep Learning Through the Lens of Example Difficulty”, Baldock et al 2021
Prediction Depth: Deep Learning Through the Lens of Example Difficulty
“Revisiting the Calibration of Modern Neural Networks”, Minderer et al 2021
“Partial Success in Closing the Gap between Human and Machine Vision”, Geirhos et al 2021
Partial success in closing the gap between human and machine vision
“CoAtNet: Marrying Convolution and Attention for All Data Sizes”, Dai et al 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes
“Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images”, Cherti & Jitsev 2021
“Embracing New Techniques in Deep Learning for Estimating Image Memorability”, Needell & Bainbridge 2021
Embracing New Techniques in Deep Learning for Estimating Image Memorability
“Predicting Sex from Retinal Fundus Photographs Using Automated Deep Learning”, Korot et al 2021
Predicting sex from retinal fundus photographs using automated deep learning
“Rethinking and Improving the Robustness of Image Style Transfer”, Wang et al 2021
Rethinking and Improving the Robustness of Image Style Transfer
“Rip Van Winkle’s Razor, a Simple New Estimate for Adaptive Data Analysis”, Arora & Zhang 2021
Rip van Winkle’s Razor, a Simple New Estimate for Adaptive Data Analysis
“The Surprising Impact of Mask-Head Architecture on Novel Class Segmentation”, Birodkar et al 2021
The surprising impact of mask-head architecture on novel class segmentation
“Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks”, Northcutt et al 2021
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
“ConViT: Improving Vision Transformers With Soft Convolutional Inductive Biases”, d’Ascoli et al 2021
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
“Learning from Videos to Understand the World”, Zweig et al 2021
“Fast and Accurate Model Scaling”, Dollár et al 2021
“Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants”, Soemers et al 2021
Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants
“Momentum Residual Neural Networks”, Sander et al 2021
“Hiding Data Hiding”, Wu et al 2021
“Explaining Neural Scaling Laws”, Bahri et al 2021
“NFNet: High-Performance Large-Scale Image Recognition Without Normalization”, Brock et al 2021
NFNet: High-Performance Large-Scale Image Recognition Without Normalization
“Brain2Pix: Fully Convolutional Naturalistic Video Reconstruction from Brain Activity”, Le et al 2021
Brain2Pix: Fully convolutional naturalistic video reconstruction from brain activity
“Words As a Window: Using Word Embeddings to Explore the Learned Representations of Convolutional Neural Networks”, Dharmaretnam et al 2021
“E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials”, Batzner et al 2021
E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
“Meta Pseudo Labels”, Pham et al 2021
“Is MLP-Mixer a CNN in Disguise? As Part of This Blog Post, We Look at the MLP Mixer Architecture in Detail and Also Understand Why It Is Not Considered Convolution Free.”
View HTML:
“Converting Tabular Data into Images for Deep Learning With Convolutional Neural Networks”, Zhu et al 2021
Converting tabular data into images for deep learning with convolutional neural networks
“Taming Transformers for High-Resolution Image Synthesis”, Esser et al 2020
“Ensemble Learning of Convolutional Neural Network, Support Vector Machine, and Best Linear Unbiased Predictor for Brain Age Prediction: ARAMIS Contribution to the Predictive Analytics Competition 2019 Challenge”, Couvy-Duchesne et al 2020
“Scaling down Deep Learning”, Greydanus 2020
“Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images”, Child 2020
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
“Understanding RL Vision: With Diverse Environments, We Can Analyze, Diagnose and Edit Deep Reinforcement Learning Models Using Attribution”, Hilton et al 2020
“Fourier Neural Operator for Parametric Partial Differential Equations”, Li et al 2020
Fourier Neural Operator for Parametric Partial Differential Equations
“Deep Learning-Based Classification of the Polar Emotions of ‘Moe’-Style Cartoon Pictures”, Cao et al 2020b
Deep learning-based classification of the polar emotions of ‘moe’-style cartoon pictures
“Demonstrating That Dataset Domains Are Largely Linearly Separable in the Feature Space of Common CNNs”, Dragan 2020
“Optimal Peanut Butter and Banana Sandwiches”, Rosenthal 2020
“Accuracy and Performance Comparison of Video Action Recognition Approaches”, Hutchinson et al 2020
Accuracy and Performance Comparison of Video Action Recognition Approaches
“A Digital Biomarker of Diabetes from Smartphone-Based Vascular Signals”, Avram et al 2020
A digital biomarker of diabetes from smartphone-based vascular signals
“SSD vs. YOLO for Detection of Outdoor Urban Advertising Panels under Multiple Variabilities”, Morera et al 2020
SSD vs. YOLO for Detection of Outdoor Urban Advertising Panels under Multiple Variabilities
“On Robustness and Transferability of Convolutional Neural Networks”, Djolonga et al 2020
On Robustness and Transferability of Convolutional Neural Networks
“NVAE: A Deep Hierarchical Variational Autoencoder”, Vahdat & Kautz 2020
“CoCoNuT: Combining Context-Aware Neural Translation Models Using Ensemble for Program Repair”, Lutellier et al 2020
CoCoNuT: Combining Context-Aware Neural Translation Models using Ensemble for Program Repair
“The Many Faces of Robustness: A Critical Analysis of Out-Of-Distribution Generalization”, Hendrycks et al 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
“SimCLRv2: Big Self-Supervised Models Are Strong Semi-Supervised Learners”, Chen et al 2020
SimCLRv2: Big Self-Supervised Models are Strong Semi-Supervised Learners
“FBNetV3: Joint Architecture-Recipe Search Using Predictor Pretraining”, Dai et al 2020
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
“Danny Hernandez on Forecasting and the Drivers of AI Progress”, Koehler et al 2020
Danny Hernandez on forecasting and the drivers of AI progress
“Measuring the Algorithmic Efficiency of Neural Networks”, Hernandez & Brown 2020
“AI and Efficiency: We’re Releasing an Analysis Showing That Since 2012 the Amount of Compute Needed to Train a Neural Net to the Same Performance on ImageNet Classification Has Been Decreasing by a Factor of 2 Every 16 Months”, Hernandez & Brown 2020
“Reinforcement Learning With Augmented Data”, Laskin et al 2020
“YOLOv4: Optimal Speed and Accuracy of Object Detection”, Bochkovskiy et al 2020
“Shortcut Learning in Deep Neural Networks”, Geirhos et al 2020
“Evolving Normalization-Activation Layers”, Liu et al 2020
“Conditional Convolutions for Instance Segmentation”, Tian et al 2020
“Train-By-Reconnect: Decoupling Locations of Weights from Their Values (LaPerm)”, Qiu & Suda 2020
Train-by-Reconnect: Decoupling Locations of Weights from their Values (LaPerm)
“Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited”, Maddox et al 2020
Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited
“Do We Need Zero Training Loss After Achieving Zero Training Error?”, Ishida et al 2020
Do We Need Zero Training Loss After Achieving Zero Training Error?
“Bayesian Deep Learning and a Probabilistic Perspective of Generalization”, Wilson & Izmailov 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization
“A Simple Framework for Contrastive Learning of Visual Representations”, Chen et al 2020
A Simple Framework for Contrastive Learning of Visual Representations
“Growing Neural Cellular Automata: Differentiable Model of Morphogenesis”, Mordvintsev et al 2020
Growing Neural Cellular Automata: Differentiable Model of Morphogenesis
“Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving”, Song et al 2020
Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving
“First-In-Human Evaluation of a Hand-Held Automated Venipuncture Device for Rapid Venous Blood Draws”, Leipheimer et al 2020
First-in-human evaluation of a hand-held automated venipuncture device for rapid venous blood draws
“ImageNet-A: Natural Adversarial Examples”, Hendrycks et al 2020
“Deep-Eyes: Fully Automatic Anime Character Colorization With Painting of Details on Empty Pupils”, Akita et al 2020
Deep-Eyes: Fully Automatic Anime Character Colorization with Painting of Details on Empty Pupils
“CARN: Convolutional Anchored Regression Network for Fast and Accurate Single Image Super-Resolution”, Li 2020
CARN: Convolutional Anchored Regression Network for Fast and Accurate Single Image Super-Resolution:
“Linear Mode Connectivity and the Lottery Ticket Hypothesis”, Frankle et al 2019
“Deep Double Descent: We Show That the Double Descent Phenomenon Occurs in CNNs, ResNets, and Transformers: Performance First Improves, Then Gets Worse, and Then Improves Again With Increasing Model Size, Data Size, or Training Time. This Effect Is Often Avoided through Careful Regularization. While This Behavior Appears to Be Fairly Universal, We Don’t yet Fully Understand Why It Happens, and View Further Study of This Phenomenon As an Important Research Direction.”, Nakkiran et al 2019
“Anonymous Market Product Classification Based on Deep Learning”, Yang et al 2019b
Anonymous market product classification based on deep learning
“The Origins and Prevalence of Texture Bias in Convolutional Neural Networks”, Hermann et al 2019
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks
“How Machine Learning Can Help Unlock the World of Ancient Japan”, Lamb 2019
How Machine Learning Can Help Unlock the World of Ancient Japan
“Self-Training With Noisy Student Improves ImageNet Classification”, Xie et al 2019
Self-training with Noisy Student improves ImageNet classification
“Taxonomy of Real Faults in Deep Learning Systems”, Humbatova et al 2019
“On the Measure of Intelligence”, Chollet 2019
“DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames”, Wijmans et al 2019
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
“Accelerating Deep Learning by Focusing on the Biggest Losers”, Jiang et al 2019
Accelerating Deep Learning by Focusing on the Biggest Losers
“ANIL: Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML”, Raghu et al 2019
ANIL: Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
“ObjectNet: A Large-Scale Bias-Controlled Dataset for Pushing the Limits of Object Recognition Models”, Barbu et al 2019
ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models
“CAR: Learned Image Downscaling for Upscaling Using Content Adaptive Resampler”, Sun & Chen 2019
CAR: Learned Image Downscaling for Upscaling using Content Adaptive Resampler
“Finding the Needle in the Haystack With Convolutions: on the Benefits of Architectural Bias”, d’Ascoli et al 2019
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
“Intriguing Properties of Adversarial Training at Scale”, Xie & Yuille 2019
“Adversarial Robustness As a Prior for Learned Representations”, Engstrom et al 2019
Adversarial Robustness as a Prior for Learned Representations
“Human-Level Performance in 3D Multiplayer Games With Population-Based Reinforcement Learning”, Jaderberg et al 2019
Human-level performance in 3D multiplayer games with population-based reinforcement learning
“ImageNet-Sketch: Learning Robust Global Representations by Penalizing Local Predictive Power”, Wang et al 2019
ImageNet-Sketch: Learning Robust Global Representations by Penalizing Local Predictive Power
“Improved Object Recognition Using Neural Networks Trained to Mimic the Brain’s Statistical Properties”, Federer et al 2019
“Neural System Identification With Neural Information Flow”, Seeliger et al 2019
“Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning”, Din et al 2019
Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning
“CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features”, Yun et al 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
“Adversarial Examples Are Not Bugs, They Are Features”, Ilyas et al 2019
“Searching for MobileNetV3”, Howard et al 2019
“Billion-Scale Semi-Supervised Learning for Image Classification”, Yalniz et al 2019
Billion-scale semi-supervised learning for image classification
“NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection”, Ghiasi et al 2019
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
“COCO-GAN: Generation by Parts via Conditional Coordinating”, Lin et al 2019
“Benchmarking Neural Network Robustness to Common Corruptions and Perturbations”, Hendrycks & Dietterich 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
“Semantic Image Synthesis With Spatially-Adaptive Normalization”, Park et al 2019
Semantic Image Synthesis with Spatially-Adaptive Normalization
“The Bitter Lesson”, Sutton 2019
“Learning To Follow Directions in Street View”, Hermann et al 2019
“SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data”, Sun et al 2019
SuperTML: Two-Dimensional Word Embedding for the Precognition on Structured Tabular Data
“Real-Time Continuous Transcription With Live Transcribe”, Savla 2019
“Do We Train on Test Data? Purging CIFAR of Near-Duplicates”, Barz & Denzler 2019
“Pay Less Attention With Lightweight and Dynamic Convolutions”, Wu et al 2019
Pay Less Attention with Lightweight and Dynamic Convolutions
“Association Between Surgical Skin Markings in Dermoscopic Images and Diagnostic Performance of a Deep Learning Convolutional Neural Network for Melanoma Recognition”, Winkler et al 2019
View PDF:
“Detecting Advertising on Building Façades With Computer Vision”, Bochkarev & Smirnov 2019
Detecting advertising on building façades with computer vision
“Quantifying Generalization in Reinforcement Learning”, Cobbe et al 2018
“ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware”, Cai et al 2018
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
“ImageNet-Trained CNNs Are Biased towards Texture; Increasing Shape Bias Improves Accuracy and Robustness”, Geirhos et al 2018
“Evolving Space-Time Neural Architectures for Videos”, Piergiovanni et al 2018
“ADNet: A Deep Network for Detecting Adverts”, Hossari et al 2018
“AdVersarial: Perceptual Ad Blocking Meets Adversarial Machine Learning”, Tramèr et al 2018
AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learning
“FloWaveNet: A Generative Flow for Raw Audio”, Kim et al 2018
“StreetNet: Preference Learning With Convolutional Neural Network on Urban Crime Perception”, Fu et al 2018
StreetNet: Preference Learning with Convolutional Neural Network on Urban Crime Perception
“Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation”, Wang et al 2018
“Understanding and Correcting Pathologies in the Training of Learned Optimizers”, Metz et al 2018
Understanding and correcting pathologies in the training of learned optimizers
“Graph Convolutional Reinforcement Learning”, Jiang et al 2018
“Cellular Automata As Convolutional Neural Networks”, Gilpin 2018
“Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization”, Narayan et al 2018
“Human-Like Playtesting With Deep Learning”, Gudmundsson et al 2018
“CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images”, Guo et al 2018
CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images
“MnasNet: Platform-Aware Neural Architecture Search for Mobile”, Tan et al 2018
MnasNet: Platform-Aware Neural Architecture Search for Mobile
“LEO: Meta-Learning With Latent Embedding Optimization”, Rusu et al 2018
“Glow: Generative Flow With Invertible 1×1 Convolutions”, Kingma & Dhariwal 2018
“Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations”, Hendrycks & Dietterich 2018
Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
“Confounding Variables Can Degrade Generalization Performance of Radiological Deep Learning Models”, Zech et al 2018
Confounding variables can degrade generalization performance of radiological deep learning models
“Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks”, Xiao et al 2018
“Resource-Efficient Neural Architect”, Zhou et al 2018
“More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch”, Calandra et al 2018
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
“Deep Learning Generalizes Because the Parameter-Function Map Is Biased towards Simple Functions”, Valle-Pérez et al 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions
“Bidirectional Learning for Robust Neural Networks”, Pontes-Filho & Liwicki 2018
“BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning”, Yu et al 2018
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
“Self-Distillation: Born Again Neural Networks”, Furlanello et al 2018
“Tile2Vec: Unsupervised Representation Learning for Spatially Distributed Data”, Jean et al 2018
Tile2Vec: Unsupervised representation learning for spatially distributed data
“Exploring the Limits of Weakly Supervised Pretraining”, Mahajan et al 2018
“YOLOv3: An Incremental Improvement”, Redmon & Farhadi 2018
“Reptile: On First-Order Meta-Learning Algorithms”, Nichol et al 2018
“Essentially No Barriers in Neural Network Energy Landscape”, Draxler et al 2018
“Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs”, Garipov et al 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
“Guess, Check and Fix: a Phenomenology of Improvisation in ‘Neural’ Painting”, Choi 2018
Guess, check and fix: a phenomenology of improvisation in ‘neural’ painting
“Sim-To-Real Optimization of Complex Real World Mobile Network With Imperfect Information via Deep Reinforcement Learning from Self-Play”, Tan et al 2018
“Evolved Policy Gradients”, Houthooft et al 2018
“Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-Of-The-Art Deep Artificial Neural Networks”, Rajalingham et al 2018
“IMPALA: Scalable Distributed Deep-RL With Importance Weighted Actor-Learner Architectures”, Espeholt et al 2018
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
“Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts”, Zhou et al 2018
Active, Continual Fine Tuning of Convolutional Neural Networks for Reducing Annotation Efforts
“ArcFace: Additive Angular Margin Loss for Deep Face Recognition”, Deng et al 2018
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
“Man against Machine: Diagnostic Performance of a Deep Learning Convolutional Neural Network for Dermoscopic Melanoma Recognition in Comparison to 58 Dermatologists”, Haenssle et al 2018
View PDF:
“DeepGS: Predicting Phenotypes from Genotypes Using Deep Learning”, Ma et al 2017
DeepGS: Predicting phenotypes from genotypes using Deep Learning
“Deep Image Reconstruction from Human Brain Activity”, Shen et al 2017
“SPP-Net: Deep Absolute Pose Regression With Synthetic Views”, Purkait et al 2017
“China’s AI Advances Help Its Tech Industry, and State Security”, Mozur & Bradsher 2017
China’s AI Advances Help Its Tech Industry, and State Security
“Measuring the Tendency of CNNs to Learn Surface Statistical Regularities”, Jo & Bengio 2017
Measuring the tendency of CNNs to Learn Surface Statistical Regularities
“3D Semantic Segmentation With Submanifold Sparse Convolutional Networks”, Graham et al 2017
3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
“BlockDrop: Dynamic Inference Paths in Residual Networks”, Wu et al 2017
“Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN”, Gao et al 2017
Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN
“The Signature of Robot Action Success in EEG Signals of a Human Observer: Decoding and Visualization Using Deep Convolutional Neural Networks”, Behncke et al 2017
“11K Hands: Gender Recognition and Biometric Identification Using a Large Dataset of Hand Images”, Afifi 2017
11K Hands: Gender recognition and biometric identification using a large dataset of hand images
“Learning to Play Chess With Minimal Lookahead and Deep Value Neural Networks”, Sabatelli 2017 (page 3)
Learning to Play Chess with Minimal Lookahead and Deep Value Neural Networks
“Learning to Generalize: Meta-Learning for Domain Generalization”, Li et al 2017
Learning to Generalize: Meta-Learning for Domain Generalization
“High-Precision Automated Reconstruction of Neurons With Flood-Filling Networks”, Januszewski et al 2017
High-Precision Automated Reconstruction of Neurons with Flood-filling Networks
“Efficient K-Shot Learning With Regularized Deep Networks”, Yoo et al 2017
“NIMA: Neural Image Assessment”, Talebi & Milanfar 2017
“Squeeze-And-Excitation Networks”, Hu et al 2017
“What Does a Convolutional Neural Network Recognize in the Moon?”, Shoji 2017
What does a convolutional neural network recognize in the moon?
“SMASH: One-Shot Model Architecture Search through HyperNetworks”, Brock et al 2017
SMASH: One-Shot Model Architecture Search through HyperNetworks
“A Deep Architecture for Unified Esthetic Prediction”, Murray & Gordo 2017
“Learning With Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback”, Li et al 2017
Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback
“WebVision Database: Visual Learning and Understanding from Web Data”, Li et al 2017
WebVision Database: Visual Learning and Understanding from Web Data
“Focal Loss for Dense Object Detection”, Lin et al 2017
“Active Learning for Convolutional Neural Networks: A Core-Set Approach”, Sener & Savarese 2017
Active Learning for Convolutional Neural Networks: A Core-Set Approach
“Learning to Infer Graphics Programs from Hand-Drawn Images”, Ellis et al 2017
“A Downsampled Variant of ImageNet As an Alternative to the CIFAR Datasets”, Chrabaszcz et al 2017
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
“Learning Transferable Architectures for Scalable Image Recognition”, Zoph et al 2017
Learning Transferable Architectures for Scalable Image Recognition
“Efficient Architecture Search by Network Transformation”, Cai et al 2017
“A Simple Neural Attentive Meta-Learner”, Mishra et al 2017
“Towards Deep Learning Models Resistant to Adversarial Attacks”, Madry et al 2017
Towards Deep Learning Models Resistant to Adversarial Attacks
“Gradient Diversity: a Key Ingredient for Scalable Distributed Learning”, Yin et al 2017
Gradient Diversity: a Key Ingredient for Scalable Distributed Learning
“Device Placement Optimization With Reinforcement Learning”, Mirhoseini et al 2017
“Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour”, Goyal et al 2017
“Submanifold Sparse Convolutional Networks”, Graham & Maaten 2017
“A Simple Neural Network Module for Relational Reasoning”, Santoro et al 2017
“Deep Learning Is Robust to Massive Label Noise”, Rolnick et al 2017
“What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features”, Zhang et al 2017
What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features
“Picasso: A Modular Framework for Visualizing the Learning Process of Neural Network Image Classifiers”, Henderson & Rothe 2017
“BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography”, Wilber et al 2017
BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography
“Adversarial Neural Machine Translation”, Wu et al 2017
“Multi-Scale Dense Networks for Resource Efficient Image Classification”, Huang et al 2017
Multi-Scale Dense Networks for Resource Efficient Image Classification
“Scaling the Scattering Transform: Deep Hybrid Networks”, Oyallon et al 2017
“Mask R-CNN”, He et al 2017
“Using Human Brain Activity to Guide Machine Learning”, Fong et al 2017
“Learned Optimizers That Scale and Generalize”, Wichrowska et al 2017
“Prediction and Control With Temporal Segment Models”, Mishra et al 2017
“Parallel Multiscale Autoregressive Density Estimation”, Reed et al 2017
“Convolution Aware Initialization”, Aghajanyan 2017
“Gender-From-Iris or Gender-From-Mascara?”, Kuehlkamp et al 2017
“BrainNetCNN: Convolutional Neural Networks for Brain Networks; towards Predicting Neurodevelopment”, Kawahara et al 2017
BrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment
“Universal Representations: The Missing Link between Faces, Text, Planktons, and Cat Breeds”, Bilen & Vedaldi 2017
Universal representations: The missing link between faces, text, planktons, and cat breeds
“PixelCNN++: Improving the PixelCNN With Discretized Logistic Mixture Likelihood and Other Modifications”, Salimans et al 2017
“YOLO9000: Better, Faster, Stronger”, Redmon & Farhadi 2016
“Language Modeling With Gated Convolutional Networks”, Dauphin et al 2016
“Learning from Simulated and Unsupervised Images through Adversarial Training”, Shrivastava et al 2016
Learning from Simulated and Unsupervised Images through Adversarial Training
“LipNet: End-To-End Sentence-Level Lipreading”, Assael et al 2016
“Self-Critical Sequence Training for Image Captioning”, Rennie et al 2016
“ResNeXt: Aggregated Residual Transformations for Deep Neural Networks”, Xie et al 2016
ResNeXt: Aggregated Residual Transformations for Deep Neural Networks
“Responses to Critiques on Machine Learning of Criminality Perceptions (Addendum of ArXiv:1611.04135)”, Wu & Zhang 2016
Responses to Critiques on Machine Learning of Criminality Perceptions (Addendum of arXiv:1611.04135)
“Understanding Deep Learning Requires Rethinking Generalization”, Zhang et al 2016
Understanding deep learning requires rethinking generalization
“Designing Neural Network Architectures Using Reinforcement Learning”, Baker et al 2016
Designing Neural Network Architectures using Reinforcement Learning
“VPN: Video Pixel Networks”, Kalchbrenner et al 2016
“HyperNetworks”, Ha et al 2016
“Neural Photo Editing With Introspective Adversarial Networks”, Brock et al 2016
Neural Photo Editing with Introspective Adversarial Networks
“On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima”, Keskar et al 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
“WaveNet: A Generative Model for Raw Audio”, Oord et al 2016
“Direct Feedback Alignment Provides Learning in Deep Neural Networks”, Nøkland 2016
Direct Feedback Alignment Provides Learning in Deep Neural Networks
“Deep Learning Human Mind for Automated Visual Classification”, Spampinato et al 2016
Deep Learning Human Mind for Automated Visual Classification
“Temporal Convolutional Networks: A Unified Approach to Action Segmentation”, Lea et al 2016
Temporal Convolutional Networks: A Unified Approach to Action Segmentation
“DenseNet: Densely Connected Convolutional Networks”, Huang et al 2016
“Clockwork Convnets for Video Semantic Segmentation”, Shelhamer et al 2016
“Deep Learning the City: Quantifying Urban Perception At A Global Scale”, Dubey et al 2016
Deep Learning the City: Quantifying Urban Perception At A Global Scale
“Convolutional Neural Fabrics”, Saxena & Verbeek 2016
“Deep Neural Networks Are Robust to Weight Binarization and Other Non-Linear Distortions”, Merolla et al 2016
Deep neural networks are robust to weight binarization and other non-linear distortions
“FractalNet: Ultra-Deep Neural Networks without Residuals”, Larsson et al 2016
“Wide Residual Networks”, Zagoruyko & Komodakis 2016
“Residual Networks Behave Like Ensembles of Relatively Shallow Networks”, Veit et al 2016
Residual Networks Behave Like Ensembles of Relatively Shallow Networks
“ViZDoom: A Doom-Based AI Research Platform for Visual Reinforcement Learning”, Kempka et al 2016
ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning
“OHEM: Training Region-Based Object Detectors With Online Hard Example Mining”, Shrivastava et al 2016
OHEM: Training Region-based Object Detectors with Online Hard Example Mining
“Deep Networks With Stochastic Depth”, Huang et al 2016
“Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Esser et al 2016
Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
“Do Deep Convolutional Nets Really Need to Be Deep and Convolutional?”, Urban et al 2016
Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
“XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks”, Rastegari et al 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
“Learning Hand-Eye Coordination for Robotic Grasping With Deep Learning and Large-Scale Data Collection”, Levine et al 2016
“Network Morphism”, Wei et al 2016
“Inception-V4, Inception-ResNet and the Impact of Residual Connections on Learning”, Szegedy et al 2016
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
“PlaNet—Photo Geolocation With Convolutional Neural Networks”, Weyand et al 2016
“Value Iteration Networks”, Tamar et al 2016
“PixelRNN: Pixel Recurrent Neural Networks”, Oord et al 2016
“Image Synthesis from Yahoo’s open_nsfw”, Goh 2016
“Deep Residual Learning for Image Recognition”, He et al 2015
“Microsoft Researchers Win ImageNet Computer Vision Challenge”, Linn 2015
Microsoft researchers win ImageNet computer vision challenge
“Adding Gradient Noise Improves Learning for Very Deep Networks”, Neelakantan et al 2015
Adding Gradient Noise Improves Learning for Very Deep Networks
“The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition”, Krause et al 2015
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
“Learning Visual Features from Large Weakly Supervised Data”, Joulin et al 2015
“Illustration2Vec: a Semantic Vector Representation of Illustrations”, Masaki & Matsui 2015
Illustration2Vec: a semantic vector representation of illustrations
“BinaryConnect: Training Deep Neural Networks With Binary Weights during Propagations”, Courbariaux et al 2015
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
“Predicting and Understanding Urban Perception With Convolutional Neural Networks”, Porzi et al 2015
Predicting and Understanding Urban Perception with Convolutional Neural Networks
“A Neural Attention Model for Abstractive Sentence Summarization”, Rush et al 2015
A Neural Attention Model for Abstractive Sentence Summarization
“LSUN: Construction of a Large-Scale Image Dataset Using Deep Learning With Humans in the Loop”, Yu et al 2015
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
“You Only Look Once: Unified, Real-Time Object Detection”, Redmon et al 2015
“Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification”, Xiao et al 2015
Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification
“Dropout As a Bayesian Approximation: Representing Model Uncertainty in Deep Learning”, Gal & Ghahramani 2015
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
“Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks”, Ren et al 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
“Cyclical Learning Rates for Training Neural Networks”, Smith 2015
“Deep Learning”, LeCun et al 2015
“Fast R-CNN”, Girshick 2015
“End-To-End Training of Deep Visuomotor Policies”, Levine et al 2015
“FaceNet: A Unified Embedding for Face Recognition and Clustering”, Schroff et al 2015
FaceNet: A Unified Embedding for Face Recognition and Clustering
“DeepID3: Face Recognition With Very Deep Neural Networks”, Sun et al 2015
“Understanding Image Representations by Measuring Their Equivariance and Equivalence”, Lenc & Vedaldi 2014
Understanding image representations by measuring their equivariance and equivalence
“Going Deeper With Convolutions”, Szegedy et al 2014
“Very Deep Convolutional Networks for Large-Scale Image Recognition”, Simonyan & Zisserman 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition
“ImageNet Large Scale Visual Recognition Challenge”, Russakovsky et al 2014
“Deep Learning Face Representation by Joint Identification-Verification”, Sun et al 2014
Deep Learning Face Representation by Joint Identification-Verification
“Network In Network”, Lin et al 2013
“R-CNN: Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation”, Girshick et al 2013
R-CNN: Rich feature hierarchies for accurate object detection and semantic segmentation
“ImageNet Classification With Deep Convolutional Neural Networks”, Krizhevsky et al 2012
ImageNet Classification with Deep Convolutional Neural Networks
“Multi-Column Deep Neural Network for Traffic Sign Classification”, Cireşan et al 2012b
Multi-column deep neural network for traffic sign classification
“Multi-Column Deep Neural Networks for Image Classification”, Cireşan et al 2012
“Building High-Level Features Using Large Scale Unsupervised Learning”, Le et al 2011
Building high-level features using large scale unsupervised learning
“DanNet: Flexible, High Performance Convolutional Neural Networks for Image Classification”, Ciresan et al 2011
DanNet: Flexible, High Performance Convolutional Neural Networks for Image Classification
“Hierarchical Object Detection With Deep Reinforcement Learning”
Hierarchical Object Detection with Deep Reinforcement Learning:
“Now Anyone Can Train Imagenet in 18 Minutes”
Wikipedia
Miscellaneous
-
/doc/ai/nn/cnn/2023-hsu-appendix-acompleteunetnnimplementationinapl.png: -
/doc/ai/nn/cnn/2022-mindermann-figure1-18xspeedupfromactivelearningofclothing1mdataset.jpg: -
/doc/ai/nn/cnn/2020-07-24-gwern-meme-moneyprinter-bitterlesson-gpt3.png: -
/doc/ai/nn/cnn/2019-boazbarak-deepdoubledescent-expandedversion-degree1000spline-goodoverfitting.png: -
/doc/ai/nn/cnn/2019-humbatova-figure1-taxonomyofrealfaultsindeeplearningsystems.png: -
/doc/ai/nn/cnn/2017-rawat.pdf:View PDF:
-
/doc/ai/nn/cnn/2015-joulin-figure2-flickrpascalvoc2007precisionscalingwithflickr100mnscaling.png: -
https://animatedai.github.io/:View External Link:
-
https://frankzliu.com/blog/vision-transformers-are-overrated: -
https://github.com/ultralytics/yolov5/issues/6998#issue-1170533269 -
https://heritagesciencejournal.springeropen.com/articles/10.1186/s40494-023-01094-0: -
https://wandb.ai/wandb_fc/articles/reports/Image-to-LaTeX--Vmlldzo1NDQ0MTAx -
https://www.quantamagazine.org/sparse-neural-networks-point-physicists-to-useful-data-20230608/: -
https://www.wired.com/story/beauty-is-in-the-eye-of-the-beholder-but-memorability-may-be-universal/
Link Bibliography
-
2024-vong.pdf: “Grounded Language Acquisition through the Eyes and Ears of a Single Child”, -
https://arxiv.org/abs/2311.15599#tencent: “UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition”, -
https://arxiv.org/abs/2310.16764#deepmind: “ConvNets Match Vision Transformers at Scale”, -
https://arxiv.org/abs/2309.10713: “Interpret Vision Transformers As ConvNets With Dynamic Convolutions”, -
https://arxiv.org/abs/2306.09346: “Rosetta Neurons: Mining the Common Units in a Model Zoo”, -
https://dl.acm.org/doi/pdf/10.1145/3589246.3595371: “U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning”, -
https://arxiv.org/abs/2305.12972: “VanillaNet: the Power of Minimalism in Deep Learning”, -
https://arxiv.org/abs/2304.05538: “ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification”, -
https://ieeexplore.ieee.org/abstract/document/10097719: “Hierarchical Multi-Label Attribute Classification With Graph Convolutional Networks on Anime Illustration”, -
https://arxiv.org/abs/2212.10544: “Pretraining Without Attention”, -
https://arxiv.org/abs/2212.06727: “What Do Vision Transformers Learn? A Visual Exploration”, -
2022-carragher.pdf: “Simulated Automated Facial Recognition Systems As Decision-Aids in Forensic Face Matching Tasks”, -
2022-lan.pdf: “GCN-Based Multi-Modal Multi-Label Attribute Classification in Anime Illustration Using Domain-Specific Semantic Features”, -
https://arxiv.org/abs/2210.03651: “Understanding the Covariance Structure of Convolutional Filters”, -
https://arxiv.org/abs/2209.12892: “g.pt: Learning to Learn With Generative Models of Neural Network Checkpoints”, -
2022-pototzky.pdf: “FastSiam: Resource-Efficient Self-Supervised Learning on a Single GPU”, -
https://www.science.org/doi/10.1126/sciadv.abm2219: “Reassessing Hierarchical Correspondences between Brain and Deep Networks through Direct Interface”, -
https://arxiv.org/abs/2206.07137: “RHO-LOSS: Prioritized Training on Points That Are Learnable, Worth Learning, and Not Yet Learnt”, -
https://arxiv.org/abs/2206.04658#nvidia: “BigVGAN: A Universal Neural Vocoder With Large-Scale Training”, -
https://arxiv.org/abs/2203.06717: “Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (RepLKNet)”, -
https://openreview.net/forum?id=SkfMWhAqYQ: “Approximating CNNs With Bag-Of-Local-Features Models Works Surprisingly Well on ImageNet”, -
https://arxiv.org/abs/2201.09792: “ConvMixer: Patches Are All You Need?”, -
https://arxiv.org/abs/2201.03545#facebook: “ConvNeXt: A ConvNet for the 2020s”, -
https://arxiv.org/abs/2110.13771#nvidia: “AugMax: Adversarial Composition of Random Augmentations for Robust Training”, -
https://arxiv.org/abs/2110.07402#bytedance: “TWIST: Self-Supervised Learning by Estimating Twin Class Distributions”, -
https://arxiv.org/abs/2110.06848: “Decoupled Contrastive Learning”, -
https://arxiv.org/abs/2110.05208: “Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-Training Paradigm (DeCLIP)”, -
https://www.frontiersin.org/articles/10.3389/fninf.2021.679838/full: “THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks”, -
https://arxiv.org/abs/2108.13002#microsoft: “A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP”, -
https://arxiv.org/abs/2108.08810#google: “Do Vision Transformers See Like Convolutional Neural Networks?”, -
2021-moses.pdf: “Neuroprosthesis for Decoding Speech in a Paralyzed Person With Anarthria”, -
https://arxiv.org/abs/2106.07411: “Partial Success in Closing the Gap between Human and Machine Vision”, -
https://arxiv.org/abs/2106.04803#google: “CoAtNet: Marrying Convolution and Attention for All Data Sizes”, -
https://arxiv.org/abs/2106.00116: “Effect of Pre-Training Scale on Intra/Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images”, -
https://arxiv.org/abs/2104.05623: “Rethinking and Improving the Robustness of Image Style Transfer”, -
https://www.offconvex.org/2021/04/07/ripvanwinkle/: “Rip Van Winkle’s Razor, a Simple New Estimate for Adaptive Data Analysis”, -
https://arxiv.org/abs/2104.00613: “The Surprising Impact of Mask-Head Architecture on Novel Class Segmentation”, -
https://arxiv.org/abs/2103.14749: “Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks”, -
https://arxiv.org/abs/2103.10697#facebook: “ConViT: Improving Vision Transformers With Soft Convolutional Inductive Biases”, -
https://ai.facebook.com/blog/learning-from-videos-to-understand-the-world/: “Learning from Videos to Understand the World”, -
https://arxiv.org/abs/2102.06171#deepmind: “NFNet: High-Performance Large-Scale Image Recognition Without Normalization”, -
https://arxiv.org/abs/2003.10580#google: “Meta Pseudo Labels”, -
https://greydanus.github.io/2020/12/01/scaling-down/: “Scaling down Deep Learning”, -
https://arxiv.org/abs/2011.10650#openai: “Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images”, -
https://www.ethanrosenthal.com/2020/08/25/optimal-peanut-butter-and-banana-sandwiches/: “Optimal Peanut Butter and Banana Sandwiches”, -
https://arxiv.org/abs/2008.09037: “Accuracy and Performance Comparison of Video Action Recognition Approaches”, -
https://www.mdpi.com/1424-8220/20/16/4587: “SSD vs. YOLO for Detection of Outdoor Urban Advertising Panels under Multiple Variabilities”, -
https://arxiv.org/abs/2007.03898#nvidia: “NVAE: A Deep Hierarchical Variational Autoencoder”, -
https://arxiv.org/abs/2004.10934: “YOLOv4: Optimal Speed and Accuracy of Object Detection”, -
https://arxiv.org/abs/2004.10802: “Scaling Laws from the Data Manifold Dimension”, -
https://arxiv.org/abs/2002.05709#google: “A Simple Framework for Contrastive Learning of Visual Representations”, -
https://arxiv.org/abs/2002.03629: “Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving”, -
https://arxiv.org/abs/1911.04252#google: “Self-Training With Noisy Student Improves ImageNet Classification”, -
https://arxiv.org/abs/1911.00357#facebook: “DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames”, -
2019-jaderberg.pdf#deepmind: “Human-Level Performance in 3D Multiplayer Games With Population-Based Reinforcement Learning”, -
https://arxiv.org/abs/1905.04899: “CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features”, -
https://arxiv.org/abs/1905.00546#facebook: “Billion-Scale Semi-Supervised Learning for Image Classification”, -
https://www.sciencedirect.com/science/article/pii/S1877050919311299: “Detecting Advertising on Building Façades With Computer Vision”, -
https://arxiv.org/abs/1811.02155: “FloWaveNet: A Generative Flow for Raw Audio”, -
2018-fu.pdf: “StreetNet: Preference Learning With Convolutional Neural Network on Urban Crime Perception”, -
2018-gudmundsson.pdf: “Human-Like Playtesting With Deep Learning”, -
https://arxiv.org/abs/1808.01097: “CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images”, -
https://arxiv.org/abs/1805.00932#facebook: “Exploring the Limits of Weakly Supervised Pretraining”, -
https://arxiv.org/abs/1804.02767: “YOLOv3: An Incremental Improvement”, -
https://arxiv.org/abs/1803.02999#openai: “Reptile: On First-Order Meta-Learning Algorithms”, -
2018-choi.pdf: “Guess, Check and Fix: a Phenomenology of Improvisation in ‘Neural’ Painting”, -
https://www.nytimes.com/2017/12/03/business/china-artificial-intelligence.html: “China’s AI Advances Help Its Tech Industry, and State Security”, -
2017-sabatelli.pdf#page=3: “Learning to Play Chess With Minimal Lookahead and Deep Value Neural Networks”, -
https://arxiv.org/abs/1708.05344: “SMASH: One-Shot Model Architecture Search through HyperNetworks”, -
https://arxiv.org/abs/1707.08819: “A Downsampled Variant of ImageNet As an Alternative to the CIFAR Datasets”, -
https://arxiv.org/abs/1706.06083: “Towards Deep Learning Models Resistant to Adversarial Attacks”, -
https://arxiv.org/abs/1706.01427#deepmind: “A Simple Neural Network Module for Relational Reasoning”, -
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2976021: “What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features”, -
https://arxiv.org/abs/1703.08961: “Scaling the Scattering Transform: Deep Hybrid Networks”, -
https://arxiv.org/abs/1703.06870#facebook: “Mask R-CNN”, -
https://arxiv.org/abs/1702.06295: “Convolution Aware Initialization”, -
2017-kawahara.pdf: “BrainNetCNN: Convolutional Neural Networks for Brain Networks; towards Predicting Neurodevelopment”, -
https://arxiv.org/abs/1612.08242: “YOLO9000: Better, Faster, Stronger”, -
https://arxiv.org/abs/1612.00563: “Self-Critical Sequence Training for Image Captioning”, -
https://arxiv.org/abs/1611.05431#facebook: “ResNeXt: Aggregated Residual Transformations for Deep Neural Networks”, -
https://arxiv.org/abs/1608.03609: “Clockwork Convnets for Video Semantic Segmentation”, -
https://arxiv.org/abs/1605.07146: “Wide Residual Networks”, -
https://arxiv.org/abs/1603.05279: “XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks”, -
https://blogs.microsoft.com/ai/microsoft-researchers-win-imagenet-computer-vision-challenge/: “Microsoft Researchers Win ImageNet Computer Vision Challenge”, -
https://arxiv.org/abs/1511.06789#google: “The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition”, -
https://arxiv.org/abs/1511.02251#facebook: “Learning Visual Features from Large Weakly Supervised Data”, -
2015-porzi.pdf: “Predicting and Understanding Urban Perception With Convolutional Neural Networks”, -
https://arxiv.org/abs/1506.02640: “You Only Look Once: Unified, Real-Time Object Detection”, -
https://openaccess.thecvf.com/content_cvpr_2015/papers/Xiao_Learning_From_Massive_2015_CVPR_paper.pdf#baidu: “Clothing-1M: Learning from Massive Noisy Labeled Data for Image Classification”, -
https://arxiv.org/abs/1506.01497#microsoft: “Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks”, -
https://arxiv.org/abs/1506.01186: “Cyclical Learning Rates for Training Neural Networks”, -
https://arxiv.org/abs/1504.08083#microsoft: “Fast R-CNN”, -
https://arxiv.org/abs/1311.2524: “R-CNN: Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation”, -
https://arxiv.org/abs/1102.0183#schmidhuber: “DanNet: Flexible, High Performance Convolutional Neural Networks for Image Classification”,