See Topics in Nonlinear Dynamics by Balakrishnan, and another lectures by him
Nonlinear dynamical systems (often abreviated to Nonlinear systems) are Dynamical systems where the O.D.Es or the mapping functions that describe the dynamics are nonlinear. They offer much richer behavior like bifurcations and chaos. Thus, while, locally, they can be linearized and analyzed by the same linear Jacobian techniques, they require more variety in analysis techniques, such as bifurcation theory, Lyanpunov functions, trapping regions, attractors, and chaos theory. Make subsections of these and organize better See Wiggins book, and Strogatz.
The theory of discrete systems has many analogies to the theory of continuous systems.
Invariant manifolds in dynamical systems
Books
Strogatz Nonlinear systems dynamics and chaos
Deterministic Nonlinear Systems: A Short Course Vadim S. Anishchenko, Tatyana E. Vadivasova, Galina I. Strelkova (auth.)
See books in oxford course website
Other lecture notes: http://www.jpoffline.com/physics_docs/y3s5/nlp_lecture_notes.pdf
More LNs: http://14.139.172.204/nptel/CSE/Web/108106024/Module5.pdf