Neural networks with memory

cosmos 4th November 2016 at 2:43pm

aka Memory-augmented neural networks

Memory is good for recognizing time sequence data.

Memory networks. Apply max-margin. Actual drescription. Paper

Time constraints for facts

Recurrent neural nets. Vanishing gradient problem, naively, RNNs don't give you long term memory.. RNNs

Long Short-Term Memory (LSTM) was introduced to solve this problem.

See also Content-addressable memory

Associative Long Short-Term Memory

Active Long Term Memory Networks

Attention and Augmented Recurrent Neural Networks

Neural Turing machines

Integrating symbols into deep learning