Differentiable neural computer

cosmos 15th March 2017 at 8:43pm
Augmented RNN

A new variation of the Neural Turing machine, with extra/improved Attention mechanisms for addressing memory. In particular:

  • The location-based addressing is generalized from being just an index of relative position, to a temporal memory link matrix, which allows the network to (approximately) retrace previously traced paths on the memory.
  • There is a Memory allocation system which associates a number between 0 and 1 to each memory location, indicating how "free" it is.

Hybrid computing using a neural network with dynamic external memory

Differentiable neural computers