https://paperswithcode.com/sota
review/position paper on giving neural nets the right biases – https://arxiv.org/abs/1806.01261
Deep Learning State of the Art (2019) - MIT
Advances in reinforcement learning
Advances in NLP, Transformer network, BERT (see here) – https://blog.openai.com/better-language-models/#task6
2017 updates: http://www.wildml.com/2017/12/ai-and-deep-learning-in-2017-a-year-in-review/
DeepMind publications: https://deepmind.com/research/publications/
Conferences: ICLR, NIPS
Transformer networks. https://research.googleblog.com/2017/08/transformer-novel-neural-network.html – New: Universal Transformers networks
Generative adversarial networks, etc.
Generating video! Video-to-video synthesis, like Image-to-image synthesis (pix2pix, CycleGAN).
WaveNet, PixelRNN, SampleRNN
Neural Architecture Search with Reinforcement Learning
Avoiding catastrophic forgetting..
Capsules. Dynamic Routing Between Capsules
.. PoseNet
neural theorem proving: https://arxiv.org/abs/1807.08204
Deep learning with Elastic Averaging SGD
We study the problem of stochastic optimization for deep learning in the parallel computing environment under communication constraints. A new algorithm is proposed in this setting where the communication and coordination of work among concurrent processes (local workers), is based on an elastic force which links the parameters they compute with a center variable stored by the parameter server (master).
Solve vanishing gradients problem, allow be computations to depend on all/any of the existing computations, thus creating a DAG structure instead of a layered one: http://www.jonolick.com/home/dagnn-a-deeper-fully-connected-network
Deep Networks with Stochastic Depth Stochastic Depth Networks will Become the New Normal
https://en.m.wikipedia.org/wiki/Modular_neural_network STN?
DCGAN http://arxiv.org/abs/1511.06434
DRAW http://arxiv.org/abs/1502.04623
Soft/hard attention https://www.google.es/url?sa=t&source=web&rct=j&url=http://arxiv.org/pdf/1502.03044&ved=0ahUKEwi4yof-jPjLAhVC5xoKHcTjDM4QFgggMAA&usg=AFQjCNEs1Yw8fZF9oaqo73cwbHJqKwQHTw
NeuralStyle https://github.com/jcjohnson/neural-style
"Take a look at @karpathy's Tweet: https://twitter.com/karpathy/status/709465955223543808?s=09"
http://arxiv.org/abs/1604.00790 bidirectional LSTM
http://www.computervisionblog.com/2016/06/deep-learning-trends-iclr-2016.html
Adversarial networks
Neural Turing machines, neural programmers-interpreters
http://www.kdnuggets.com/2016/09/9-key-deep-learning-papers-explained.html
A couple of cool recent neural network developments! * Lip reading * Neural enhance! Super-resolution of images (like in movies but real ;) )
LipNet. Lip reading NN: http://prostheticknowledge.tumblr.com/post/152735696866/lipnet-deep-learning-research-from-the-university
image super-resolution using deep convolutional networks! try here: http://waifu2x.udp.jp/ NeuralEnhance lets you apply 4x super-resolution to your photos CSI-style in only 340 lines of code! https://github.com/alexjc/neural-enhance