Deepmind's Path to Neuro-Inspired General Intelligence

Category: Machine Intelligence

Read the original document

<!-- gdoc-inlined -->

By Jeremy Nixon [[email protected]]. Nov. 2017. Updated June 2018.

Overview

1. Deepmind Paper Framing
2. Deepmind Papers through Framing
3. Current Frontier
4. Examples of Systems Neuroscience Inspiration

Deepmind Papers

Categories of the path to date:

1. Transfer Learning

2. Multi-task Learning

3. Tools, Environment & Datasets

4. Intuitive Physics

5. Reinforcement Learning

   1. Model-based RL
   2. Exploration in RL

6. Applications

7. Safety

8. Deep Learning

   1. RNNs
   2. CNNs

9. Generative Models

10. Variational Inference

11. Unsupervised Learning

12. Representation Learning

13. Attention

14. Memory

15. Multi-Agent Systems

16. Imitation Learning

17. Metalearning

18. Neural Programming

19. Evolution

20. Game Theory

21. Natural Language Processing

22. Multi-Modal Learning

23. General Machine Learning

24. Theory

25. Miscellaneous

26. Neuroscience

27. Transfer Learning

    1. DARLA: Improving Zero-Shot Transfer In Reinforcement Learning
       1. https://arxiv.org/pdf/1707.08475.pdf
    2. PathNet: Evolution Channels Gradient Descent in Super Neural Networks
       1. https://arxiv.org/pdf/1701.08734.pdf
    3. Matching Networks for One Shot Learning
       1. https://arxiv.org/abs/1606.04080
    4. Progressive Neural Networks
       1. https://arxiv.org/pdf/1606.04671.pdf
    5. Sim-to-Real Robot Learning from Pixels with Progressive Nets
       1. https://arxiv.org/pdf/1610.04286.pdf
    6. Successor Features for Transfer in Reinforcement Learning
       1. https://arxiv.org/pdf/1606.05312.pdf

28. Multi-Task Learning

    1. Multi-task Self-Supervised Visual Learning
       1. https://arxiv.org/pdf/1708.07860.pdf
    2. The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously
       1. https://arxiv.org/pdf/1707.03300.pdf
    3. Distral: Robust Multitask Reinforcement Learning
       1. https://arxiv.org/pdf/1707.04175.pdf
    4. Emergence of Locomotion Behaviors in Rich Environments
       1. https://arxiv.org/pdf/1707.02286.pdf
    5. Reinforcement Learning with Unsupervised Auxiliary Tasks
       1. https://arxiv.org/pdf/1611.05397.pdf
    6. Learning to Navigate in Complex Environments
       1. https://arxiv.org/pdf/1611.03673.pdf
    7. Learning and Transfer of Modulated Locomotor Controllers
       1. https://arxiv.org/pdf/1610.05182.pdf
    8. Multi-Task Sequence to Sequence Learning
       1. https://arxiv.org/pdf/1511.06114v3.pdf
    9. Learning by Playing - Solving Sparse Reward Tasks from Scratch
       1. https://arxiv.org/abs/1802.10567
    10. Unicorn: Continual Learning with a Universal, Off-policy Agent
    11. https://arxiv.org/abs/1802.08294
    12. Progress & Compress: A Scalable Framework for Continual Learning
    13. https://arxiv.org/abs/1805.06370

29. Tools, Environments, Evaluation & Datasets

    1. Starcraft II: A New Challenge for Reinforcement Learning
       1. https://arxiv.org/pdf/1708.04782.pdf
    2. DeepMind Lab
       1. https://arxiv.org/pdf/1612.03801.pdf
    3. The Kinetics Human Action Video Dataset
       1. https://arxiv.org/pdf/1705.06950.pdf
    4. An approximation of the Universal Intelligence Measure
       1. https://arxiv.org/pdf/1109.5951v2.pdf
    5. Psychlab: A Psychology Laboratory for Deep Reinforcement Learning
       1. https://arxiv.org/abs/1801.08116
    6. Deepmind Control Suite
       1. https://arxiv.org/pdf/1801.00690v1.pdf
    7. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
       1. https://arxiv.org/pdf/1705.07750.pdf

30. Intuitive Physics

    1. Position-Velocity Encoders for Unsupervised Learning of Structured State Representations
       1. https://arxiv.org/pdf/1705.09805.pdf
    2. Learning to Perform Physics Experiments via Deep Reinforcement Learning
       1. https://arxiv.org/pdf/1611.01843.pdf
    3. Continuous Control with Deep Reinforcement Learning
       1. https://arxiv.org/pdf/1509.02971v2.pdf

31. Reinforcement Learning (Papers with a pure RL focus)

    1. Model-Based RL
       1. Learning Model-Based Planning from Scratch [Also, Planning]
          1. https://arxiv.org/pdf/1707.06170.pdf
       2. Recurrent Environment Simulators
          1. https://arxiv.org/pdf/1704.02254.pdf
       3. Structure Learning in Motor Control: A Deep Reinforcement Learning Model [Also Transfer, Intuitive Physics]
          1. https://arxiv.org/pdf/1706.06827.pdf
       4. Imagination-Augmented Agents for Deep Reinforcement Learning [Also, Planning]
          1. https://arxiv.org/abs/1707.06203
       5. Continuous Deep Q-Learning with Model-based Acceleration
          1. https://arxiv.org/abs/1603.00748
       6. Skip Context Tree Switching
          1. http://proceedings.mlr.press/v32/bellemare14.pdf
       7. Bayes-Adaptive Simulation-Based Search with Value Function Approximation
          1. http://www0.cs.ucl.ac.uk/staff/d.silver/web/Publications_files/bafa.pdf
       8. Learning and Querying Fast Generative Models for Reinforcement Learning
          1. https://arxiv.org/abs/1802.03006
    2. Exploration in RL
       1. Count-Based Exploration with Neural Density Models
          1. https://arxiv.org/pdf/1703.01310.pdf
       2. Unifying Count-Based Exploration and Intrinsic Motivation
          1. https://arxiv.org/abs/1606.01868
       3. Deep Exploration via Bootstrapped DQN
          1. https://arxiv.org/abs/1602.04621
       4. Variational Intrinsic Control
          1. https://arxiv.org/pdf/1611.07507.pdf
       5. Learning to Search with MCTSnets
          1. https://arxiv.org/abs/1802.04697v1
       6. Observe and Look Further: Achieving Consistent Performance on Atari
          1. https://arxiv.org/abs/1805.11593
    3. A Distributional Perspective on Reinforcement Learning
       1. https://arxiv.org/pdf/1707.06887.pdf
    4. FeUdal Networks for Hierarchical Reinforcement Learning [Also, Planning]
       1. https://arxiv.org/pdf/1703.01161.pdf
    5. Combining Policy Gradient and Q-Learning
       1. https://arxiv.org/pdf/1611.01626.pdf
    6. Strategic Attentive Writer for Learning Macro-Actions
       1. https://arxiv.org/pdf/1606.04695.pdf
    7. Safe and Efficient Off-Policy Reinforcement Learning
       1. https://arxiv.org/abs/1606.02647
    8. Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
       1. https://arxiv.org/pdf/1610.00633.pdf
    9. Thompson Sampling is Asymptotically Optimal in General Environments
       1. https://arxiv.org/pdf/1602.07905.pdf
    10. Asynchronous Methods for Deep Reinforcement Learning
    11. https://arxiv.org/abs/1602.01783
    12. Dueling Network Architectures for Deep Reinforcement Learning
    13. https://arxiv.org/abs/1511.06581
    14. Increasing the Action Gap: New Operators for Reinforcement Learning
    15. https://arxiv.org/abs/1512.04860
    16. Deep Reinforcement Learning with Double Q-Learning
    17. https://arxiv.org/abs/1509.06461
    18. Policy Distillation
    19. https://arxiv.org/pdf/1511.06295.pdf
    20. Universal Value Function Approximators
    21. http://proceedings.mlr.press/v37/schaul15.pdf
    22. Human-level Control through Deep Reinforcement Learning
    23. https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf
    24. Learning Continuous Control Policies by Stochastic Value Gradients
    25. https://arxiv.org/pdf/1510.09142v1.pdf
    26. Fictitious Self-Play in Extensive Form Games
    27. http://proceedings.mlr.press/v37/heinrich15.pdf
    28. Toward Minimax Off-policy Value Estimation
    29. http://proceedings.mlr.press/v38/li15b.html
    30. Massively Parallel Methods for Deep Reinforcement Learning
    31. https://arxiv.org/pdf/1507.04296.pdf
    32. Compress and Control
    33. https://arxiv.org/pdf/1411.5326v1.pdf
    34. Deterministic Policy Gradient Algorithms
    35. http://proceedings.mlr.press/v32/silver14.pdf
    36. Playing Atari with Deep Reinforcement Learning
    37. https://arxiv.org/pdf/1312.5602v1.pdf
    38. Reinforcement Learning, Efficient Coding, and the Statistics of Natural Tasks
    39. http://www.sciencedirect.com/science/article/pii/S2352154615001151
    40. Rainbow: Combining Improvements in Deep Reinforcement Learning
    41. https://arxiv.org/abs/1710.02298
    42. Path Consistency Learning in Tsallis Entropy Regularized MDPs
    43. https://arxiv.org/abs/1802.03501
    44. More Robust Doubly Robust Off-Policy Evaluation
    45. https://arxiv.org/abs/1802.03493
    46. IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
    47. https://arxiv.org/abs/1802.01561
    48. Mis&Match - Agent Curricula for Reinforcement Learning
    49. https://arxiv.org/abs/1806.01780
    50. Vector-based Navigation Using Grid-Like Representations in Artificial Agents
    51. https://www.nature.com/articles/s41586-018-0102-6.epdf?
    52. Kickstarting Deep Reinforcement Learning
    53. https://arxiv.org/abs/1803.03835

32. Applications

    1. Go
       1. Mastering the Game of Go with Deep Neural Networks and Tree Search
          1. https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf
       2. More Evaluation in Go using Deep Convolutional Neural Networks [Also, Convolutional Neural Networks]
          1. http://www0.cs.ucl.ac.uk/staff/d.silver/web/Publications_files/deepgo.pdf
       3. Mastering the Game of Go Without Human Knowledge
          1. https://www.nature.com/articles/nature24270.epdf?
    2. Poker
       1. Smooth UCT Search in Computer Poker
          1. http://www0.cs.ucl.ac.uk/staff/d.silver/web/Publications_files/smooth_uct.pdf
    3. Fairness
       1. Path-Specific Counterfactual Fairness
          1. https://arxiv.org/pdf/1802.08139.pdf

33. Safety / Security

    1. Reinforcement Learning with a Corrupted Reward Channel [Also, Safety]
       1. https://arxiv.org/pdf/1705.08417.pdf
    2. Safely Interruptible Agents [Also, Safety]
       1. https://intelligence.org/files/Interruptibility.pdf
    3. AI Safety Gridworlds
       1. https://arxiv.org/abs/1711.09883
    4. Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
       1. https://arxiv.org/abs/1802.05666
    5. Safe Exploration in Continuous Action Spaces
       1. https://arxiv.org/abs/1801.08757
    6. Measuring and Avoiding Side Effects Using Relative Reachability
       1. https://arxiv.org/abs/1806.01186

34. Deep Learning

    1. Recurrent Neural Networks
       1. Sequential Neural Models with Stochastic Layers [Also, Planning]
          1. https://arxiv.org/abs/1605.07571
       2. Memory-Efficient Backpropagation Through Time
          1. https://arxiv.org/abs/1606.03401
       3. Adaptive Computation Time for Recurrent Neural Networks
          1. https://arxiv.org/abs/1603.08983
       4. Grid Long-Short Term Memory
          1. https://arxiv.org/pdf/1507.01526v3.pdf
       5. Order Matters: Sequence to Sequence for Sets
          1. https://arxiv.org/pdf/1511.06391v3.pdf
    2. Convolutional Neural Networks
       1. Exploiting Cyclic Symmetry in Convolutional Neural Networks
          1. https://arxiv.org/abs/1602.02660
       2. Spatial Transformer Networks
          1. https://arxiv.org/pdf/1506.02025.pdf
       3. Very Deep Convolutional Networks for Large Scale Image Recognition
          1. https://arxiv.org/pdf/1409h.1556v6.pdf
       4. Pooling is Neither Necessary Nor Sufficient for Appropriate Deformation Stability in CNNs
          1. https://arxiv.org/abs/1804.04438
    3. Noisy Networks for Exploration
       1. https://arxiv.org/pdf/1706.10295.pdf
    4. Sobolev Training for Neural Networks
       1. https://arxiv.org/abs/1706.04859
    5. Decoupled Neural Interfaces using Synthetic Gradients
       1. https://arxiv.org/pdf/1608.05343.pdf
    6. Understanding Synthetic Gradients and Decoupled Neural Interfaces
       1. https://arxiv.org/pdf/1703.00522.pdf
    7. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
       1. https://arxiv.org/pdf/1612.01474.pdf
    8. Overcoming Catastrophic Forgetting in Neural Networks
       1. https://arxiv.org/pdf/1612.00796.pdf
    9. Local Minima in Training of Neural Networks
       1. https://arxiv.org/pdf/1611.06310.pdf
    10. Learning Values Across Many Orders of Magnitude
    11. https://arxiv.org/abs/1602.07714
    12. MuProp: Unbiased Backpropagation for Stochastic Neural Networks
    13. https://arxiv.org/pdf/1511.05176v2.pdf
    14. ACDC: A Structured Efficient Linear Layer
    15. https://arxiv.org/pdf/1511.05946v3.pdf
    16. Natural Neural Networks
    17. https://arxiv.org/pdf/1507.00210.pdf
    18. Gradient Estimation Using Stochastic Computation Graphs
    19. https://arxiv.org/pdf/1506.05254v1.pdf
    20. Weight Uncertainty in Neural Networks
    21. http://proceedings.mlr.press/v37/blundell15.pdf
    22. Stochastic Backpropagation and Approximate Inference in Deep Generative Models
    23. https://arxiv.org/abs/1401.4082
    24. On the Importance of Single Directions for Generalization
    25. https://arxiv.org/abs/1803.06959

35. Variational Inference

    1. Filtering Variational Objectives
       1. https://arxiv.org/pdf/1705.09279.pdf
    2. Variational Inference for Monte Carlo Objectives
       1. https://arxiv.org/abs/1602.06725
    3. Variational Inference with Normalizing Flows
       1. https://arxiv.org/pdf/1505.05770.pdf
    4. Variational Information Maximization for Intrinsically Motivated Reinforcement Learning [Also, Reinforcement Learning]
       1. https://arxiv.org/pdf/1509.08731v1.pdf
    5. Neural Variational Inference and Learning in Belief Networks
       1. https://arxiv.org/pdf/1402.0030v2.pdf
    6. Distribution Matching in Variational Inference [Also, Generative, Unsupervised Learning]
       1. https://arxiv.org/abs/1802.06847

36. Generative Models

37. The Cramer Distance as a Solution to Biased Wasserstein Gradients

    1. https://arxiv.org/pdf/1705.10743.pdf

38. Variational Approaches for Auto-Encoding Generative Adversarial Networks

    1. https://arxiv.org/pdf/1706.04987.pdf

39. Comparison of Maximum Likelihood and GAN-based training of Real NVPs

    1. https://arxiv.org/pdf/1705.05263.pdf

40. Parallel Multiscale Autoregressive Density Estimation

    1. https://arxiv.org/pdf/1703.03664.pdf

41. Conditional Image Generation with PixelCNN Decoders

    1. https://arxiv.org/pdf/1606.05328.pdf

42. WaveNet: A Generative Model for Raw Audio

    1. https://arxiv.org/pdf/1609.03499.pdf

43. Video Pixel Networks

    1. https://arxiv.org/pdf/1610.00527.pdf

44. Learning in Implicit Generative Models

    1. https://arxiv.org/pdf/1610.03483.pdf

45. Connecting Generative Adversarial Networks and Actor-Critic Methods [Also, Reinforcement Learning]

    1. https://arxiv.org/pdf/1610.01945.pdf

46. Pixel Recurrent Neural Networks 1. https://arxiv.org/abs/1601.06759

47. One-Shot Generalization in Deep Generative Models 1. https://arxiv.org/abs/1603.05106

48. A Test of Relative Similarity for Model Selection in Generative Models 1. https://arxiv.org/pdf/1511.04581.pdf

49. DRAW: A Recurrent Neural Network for Image Generation [Also, Attention] 1. http://proceedings.mlr.press/v37/gregor15.pdf

50. Semi-Supervised Learning with Deep Generative Models 1. https://arxiv.org/abs/1406.5298

51. Deep AutoRegressive Networks 1. https://arxiv.org/abs/1310.8499

52. A Note on the Evaluation of Generative Models 1. https://arxiv.org/pdf/1511.01844v2.pdf

53. Parallel WaveNet: Fast High-Fidelity Speech Synthesis (WaveRNN) 1. https://arxiv.org/abs/1711.10433

54. Efficient Neural Audio synthesis 1. https://arxiv.org/abs/1802.08435

55. Learning and Querying Fast Generative Models for Reinforcement Learning 1. https://arxiv.org/abs/1802.03006

56. Unsupervised Learning

57. Unsupervised Learning of 3D Structure from Images [Also, Computer Vision]

    1. https://arxiv.org/pdf/1607.00662.pdf

58. Early Visual Concept Learning with Unsupervised Deep Learning (beta-VAE)

    1. https://arxiv.org/pdf/1606.05579.pdf

59. Neural Scene Representation and Rendering

    1. http://science.sciencemag.org/content/360/6394/1204

60. Spectral Inference Networks: Unifying Spectral Methods with Deep Learning

    1. https://arxiv.org/abs/1806.02215

61. Representation Learning

62. SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

    1. https://arxiv.org/pdf/1707.03389.pdf

63. Towards Conceptual Compression

    1. https://arxiv.org/abs/1604.08772

64. Neural Discrete Representation Learning [Also, Unsupervised Learning]

    1. https://arxiv.org/abs/1711.00937

65. Disentangling by Factorising

    1. https://arxiv.org/abs/1802.05983

66. Associative Compression Networks for Representation Learning

    1. https://arxiv.org/abs/1804.02476

67. Attention

68. Attend, Infer, Repeat: Fast Scene Understanding with Generative Models

    1. https://arxiv.org/pdf/1603.08575.pdf

69. Reasoning about Entailment with Neural Attention [Also, Natural Language Processing]

    1. https://arxiv.org/pdf/1509.06664v2.pdf

70. Multiple Object Recognition with Visual Attention

    1. https://arxiv.org/pdf/1412.7755v2.pdf

71. Recurrent Models of Visual Attention

    1. https://arxiv.org/abs/1406.6247

72. Memory

73. Neural Episodic Control

    1. https://arxiv.org/pdf/1703.01988.pdf

74. Generative Temporal Models With Memory

    1. https://arxiv.org/pdf/1702.04649.pdf

75. Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes

    1. https://arxiv.org/pdf/1610.09027.pdf

76. Model-Free Episodic Control

    1. https://arxiv.org/abs/1606.04460

77. One-Shot Learning with Memory-Augmented Neural Networks

    1. https://arxiv.org/abs/1605.06065

78. Associative Long Short-Term Memory

    1. https://arxiv.org/abs/1602.03032

79. Prioritized Experience Replay

    1. https://arxiv.org/pdf/1511.05952v3.pdf

80. Sample Efficient Actor-Critic with Experience Replay

    1. https://arxiv.org/pdf/1611.01224.pdf

81. Learning Efficient Algorithms with Hierarchical Attentive Memory [Also, attention]

    1. https://arxiv.org/abs/1602.03218

82. Count-Based Frequency Estimation with Bounded Memory [Also, Natural Language Processing]
    1. http://www.ijcai.org/Proceedings/15/Papers/470.pdf

83. Memory-based Parameter Adaptation 1. https://arxiv.org/abs/1802.10542

84. Multi-Agent Systems

85. Value Decomposition Networks For Cooperative Multi-Agent Learning

    1. https://arxiv.org/pdf/1706.05296.pdf

86. Learning to Communicate with Deep Multi-Agent Reinforcement Learning [Also, Multi-Task RL]

    1. https://arxiv.org/pdf/1605.06676v2.pdf

87. A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning [Also, Game Theory]

    1. https://arxiv.org/abs/1711.00832

88. Machine Theory of Mind

    1. https://arxiv.org/abs/1802.07740

89. Imitation Learning

90. Robust Imitation of Diverse Behaviors

    1. https://arxiv.org/pdf/1707.02747.pdf

91. Learning Human Behaviors from Motion Capture by Adversarial Imitation

    1. https://arxiv.org/abs/1707.02201

92. Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

    1. https://arxiv.org/pdf/1707.08817.pdf

93. Reinforcement and Imitation Learning for Diverse Visuomotor Skills

    1. https://arxiv.org/abs/1802.09564

94. Playing Hard Exploration Games by Watching Youtube

    1. https://arxiv.org/abs/1805.11592

95. Metalearning

96. Neural Programming

    1. Hybrid Computing Using a Neural Network with Dynamic External Memory
       1. https://www.nature.com/articles/nature20101.epdf?author_access_token=ImTXBI8aWbYxYQ51Plys8NRgN0jAjWel9jnR3ZoTv0MggmpDmwljGswxVdeocYSurJ3hxupzWuRNeGvvXnoO8o4jTJcnAyhGuZzXJ1GEaD-Z7E6X_a9R-xqJ9TfJWBqz
    2. Programmable Agents [Also, Representation Learning]
       1. https://arxiv.org/pdf/1706.06383.pdf
    3. Neural Programmer-Interpreters
       1. https://arxiv.org/pdf/1511.06279v3.pdf
    4. Neural Random-Access Machines
       1. https://arxiv.org/pdf/1511.06392v3.pdf
    5. Neural Turing Machines
       1. https://arxiv.org/abs/1410.5401
    6. Learning Explanatory Rules from Noisy Data
       1. https://arxiv.org/abs/1711.04574
    7. Synthesizing Programs for Images using Reinforced Adversarial Learning (SPIRAL)
       1. https://arxiv.org/abs/1804.01118

97. Learning to learn by gradient descent by gradient descent

    1. https://arxiv.org/abs/1606.04474

98. Learning to Reinforcement Learn [Also, Reinforcement Learning]

    1. https://arxiv.org/pdf/1611.05763.pdf

99. Hierarchical Representations for Efficient Architecture Search

    1. https://arxiv.org/pdf/1711.00436.pdf

100. Population Based Training of Neural Networks

     1. https://arxiv.org/abs/1711.09846

101. Meta-Gradient Reinforcement Learning

     1. https://arxiv.org/abs/1805.09801

102. Evolution

103. Convolution by Evolution

     1. https://arxiv.org/pdf/1606.02580.pdf

104. Game Theory

105. Learning Nash Equilibrium for General-Sum Markov Games from Batch Data

     1. https://arxiv.org/pdf/1606.08718.pdf

106. The Mechanics of n-Player Differentiable Games [Also, Generative Models (GANs)]

     1. https://arxiv.org/abs/1802.05642

107. Symmetric Decomposition of Asymmetric Games

     1. https://www.nature.com/articles/s41598-018-19194-4

108. A Generalised Method for Empirical Game Theoretic Analysis

     1. https://arxiv.org/abs/1803.06376

109. Inequity Aversion Resolves Intertemporal Social Dilemmas

     1. https://arxiv.org/abs/1803.08884

110. Natural Language Processing

111. Generative and Discriminative Text Classification with Recurrent Neural Networks

     1. https://arxiv.org/pdf/1703.01898.pdf

112. Learning to Compose Words Into Sentences with Reinforcement Learning

     1. https://arxiv.org/pdf/1611.09100.pdf

113. Reference-Aware Language Models

     1. https://arxiv.org/pdf/1611.01628.pdf

114. The Neural Noisy Channel

     1. https://arxiv.org/pdf/1611.02554.pdf

115. Latent Predictor Networks for Code Generation

     1. https://arxiv.org/pdf/1603.06744.pdf

116. Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning

     1. https://arxiv.org/pdf/1605.03852.pdf

117. Semantic Parsing with Semi-Supervised Sequential Autoencoders

     1. https://arxiv.org/pdf/1609.09315.pdf

118. On the State of the Art of Evaluation in Neural Language Models

     1. https://arxiv.org/abs/1707.05589

119. Teaching Machines to Read and Comprehend

     1. https://arxiv.org/pdf/1506.03340v1.pdf

120. Learning to Transduce with Unbounded Memory [Also, Memory, Neural Programming] 1. https://arxiv.org/pdf/1506.02516v1.pdf

121. Dependency Recurrent Neural Language Models for Sentence Completion 1. http://cs.nyu.edu/~mirowski/pub/MirowskiVlachos_ACL2015_DependencyTreeRNN.pdf

122. Towards End-to-End Speech Recognition with Recurrent Neural Networks 1. http://proceedings.mlr.press/v32/graves14.pdf

123. Learning Word Embeddings Efficiently with Noise-Contrastive Estimation 1. http://papers.nips.cc/paper/5165-learning-word-embeddings-efficiently-with-noise-contrastive-estimation.pdf

124. The NarrativeQA Reading Comprehension Challenge 1. https://arxiv.org/abs/1712.07040v1

125. Learning to Follow Language Instructions with Adversarial Reward Induction [Also, Loss Function Learning] 1. https://arxiv.org/abs/1806.01946

126. Multi-Modal

127. Look, Listen and Learn

     1. https://arxiv.org/pdf/1705.08168.pdf

128. End-to-end Optimization of Goal-Driven and Visually Grounded Dialogue Systems

     1. https://arxiv.org/pdf/1703.05423.pdf

129. GuessWhat?! Visual Object Discovery through Multi-Modal Dialogue

     1. https://arxiv.org/pdf/1611.08481.pdf

130. Grounded Language Learning in a Simulated 3D World

     1. https://arxiv.org/pdf/1706.06551.pdf

131. Understanding Grounded Language Learning Agents [Also, Natural Language Processing]

     1. https://arxiv.org/abs/1710.09867

132. Objects that Sound

     1. https://arxiv.org/abs/1712.06651

133. General Machine Learning

134. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

     1. https://arxiv.org/pdf/1611.00712.pdf

135. Learning Deep Nearest Neighbor Representations Using Differentiable Boundary Trees

     1. https://arxiv.org/pdf/1702.08833.pdf

136. Unit Tests for Stochastic Optimization

     1. https://arxiv.org/pdf/1312.6055v3.pdf

137. Bayesian Hierarchical Community Discovery

     1. http://papers.nips.cc/paper/5048-bayesian-hierarchical-community-discovery.pdf

138. Implicit Reparameterization Gradients

     1. https://arxiv.org/abs/1805.08498

139. Cleaning up the Neighborhood: A Full Classification for Adversarial Partial Monitoring

     1. https://arxiv.org/abs/1805.09247

140. Theory

141. Online Learning with Gated Linear Networks

     1. https://arxiv.org/abs/1712.01897v1

142. Miscellaneous

143. Generalized Probability Smoothing

     1. https://arxiv.org/abs/1712.02151

144. Agents and Devices: A Relative Definition of Agency

     1. https://arxiv.org/abs/1805.12387

145. Neuroscience

146. The Successor representation in human reinforcement learning

     1. http://www.biorxiv.org/content/biorxiv/early/2017/07/04/083824.full.pdf

147. Dorsal Hippocampus Contributes to Model-Based Planning

     1. https://www.nature.com/articles/nn.4613.epdf?author_access_token=OfuqRzRgBmFKQdGE1Qw7FdRgN0jAjWel9jnR3ZoTv0N3IprbEH8EVdPgVTpPLVjgaMNMGW_KprBhEEIm7f1drNjI5FB2fds3h58n3XEtMJPC3kLK1Pp3J2_Qb45cy7uk

148. Neuroscience-Inspired Artificial Intelligence

     1. http://www.cell.com/neuron/fulltext/S0896-6273(17)30509-3

149. Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanism for Updating and Representing Self-Relevant Information

     1. http://www.cell.com/neuron/pdf/S0896-6273(16)30802-9.pdf

150. Dorsal Anterior Cingulate Cortex and the Value of Control

     1. https://www.nature.com/articles/nn.4384.epdf?author_access_token=dq-w7RyWLn3z-0m4nbyTW9RgN0jAjWel9jnR3ZoTv0N7RVyemANPvboWSepiJaSAsTFiGqyORVbog9B6IjN113kC9aqMAEoNVCCfdRA4gLVJXcy4e1klhW0KiKS5F1gp

151. Semantic Representations in the Temporal Pole Predict False Memories

     1. http://www.pnas.org/content/113/36/10180.abstract

152. Towards an Integration of Deep Learning and Neuroscience

     1. http://www.biorxiv.org/content/early/2016/06/13/058545

153. What Learning Systems do Intelligent Agents Need? Complementary Learning systems Theory Updated

     1. http://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(16)30043-2

154. Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network [Also, Planning]

     1. http://www.cell.com/neuron/abstract/S0896-6273(16)30057-5

155. Predictive Representations can Link Model-Based Reinforcement Learning to Model-Free Mechanisms 1. https://www.biorxiv.org/content/early/2016/10/27/083857

156. Hippocampal place cells construct reward related sequences through unexplored space 1. https://elifesciences.org/articles/06063

157. A Probabilistic Approach to Demixing Odors 1. http://www.nature.com/neuro/journal/v20/n1/full/nn.4444.html

158. Approximate Hubel-Wiesel Modules and the Data Structures of Neural Computation 1. https://arxiv.org/pdf/1512.08457v1.pdf

159. The Future of Memory: Remembering, Imagining, and the Brain 1. http://static1.1.sqspcdn.com/static/f/1096238/22043246/1361990370157/FutureMemory--Neuron12.pdf?token=b5gB3ycz3e%2BmKnQQCW3%2FvwZyHwE%3D

160. Is the Brain a Good Model for Machine Intelligence? 1. http://www.gatsby.ucl.ac.uk/~demis/TuringSpecialIssue(Nature2012).pdf

161. Evidence Integration in Model-Based Tree Search 1. http://www.pnas.org/content/112/37/11708.full.pdf

162. (Commentary on0 Building Machines that Learn and Think for Themselves 1. https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/building-machines-that-learn-and-think-for-themselves/E28DBFEC380D4189FB7754B50066A96F

163. Prefrontal Cortex as a Meta-Reinforcement Learning System 1. https://www.nature.com/articles/s41593-018-0147-8

Current Frontier:

1. Hierarchical planning
2. Imagination-based planning with generative models
3. Unsupervised Learning
4. Memory and one-shot learning
5. Abstract Concepts
6. Continual and Transfer Learning

Emphasis on systems neuroscience - using the brain as inspiration for the structure and function of algorithms.

Neuroscience Inspired Artificial Intelligence Examples of previous success of neuro-inspiration:

• Reinforcement Learning
  • Inspired by animal learning
  • TD Learning came out of animal behavior research.
  • Second-order conditioning (Conditional Stimulus) (Sutton and Barto, 1981)
• Deep Learning.
  • Convolutional Neural Networks. Visual Cortex (V1)
    • Uses hierarchical structure (successive processing layers)
    • Neurons in the early visual systems responds strongly to specific patterns of light (say, precisely oriented bars) but hardly responds to many other patterns.
    • Gabor functions describe the weights in V1 cells.
    • Nonlinear Transduction
    • Divisive Normalization
  • Word / Sentence Vectors - Distributed Embeddings
    • Parallel Distributed Processing in the brain for representation and computation
  • Dropout
    • Stochasticity in neurons that fire with` Poisson-like statistics (Hinton 2012)
• Attention
  • Applying attention to memory
  • Thought - it doesn’t make much sense to train an attention model over a static image, rather than over a time series. With a time series, bringing attention to changing aspects of the input makes sense.
• Multiple Memory Systems
  • Episodic Memory
    • Experience Replay
    • Especially for one shot experiences
  • Working Memory
    • LSTM - gating allows for conditioning on current state
  • Long-term Memory
    • External Memory
    • Gating in LSTM
• Continual Learning
  • Elastic weight consolidation for slowing down learning on weights that are important for previous tasks.

Examples of future success:

• Intuitive Understanding of Physics

  • Need to understand space, number, objectness
  • Need to disentangle representations for transfer. (Dude, I feel so stolen from)

• Efficient Learning (Learning from few examples)

• Transfer Learning

  • Transferring generalized knowledge gained in one context to novel domains
  • Concept representations for transfer
    • No direct evidence of concept representations in brains

• Imagination and Planning

  • Toward model-based RL
  • Internal model of the environment
    • Model needs to include compositional / disentangled representations for flexibility
  • Implementing a forecasted-based method of action selection
  • Monte-carlo Tree Search as simulation based planning
  • In rat brains, we observe ‘preplay’ where rats imagine the likely future experience - measured by comparing neural activations at preplay to activations during the activity
  • Generalization + Transfer in human planning
  • Hierarchical Planning

• Virtual Brain Analytics

  2.
  

Source: Original Google Doc