Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity

Details Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity

2009-11-17

arxiv.org/abs/0911.3298v1

Alippi, C., Polycarpou, M. Panayiotou, C., Ellinas, G. (Eds.) ICANN 2009, LNCS, vol. 5768 pp. 952-963, Springer, Heidelberg (2

Computer Science

Neural and Evolutionary Computing

11 pages, 4 figures, Recurrent Networks session ICANN 2009

Scientific paper

Recursive Neural Networks are non-linear adaptive models that are able to learn deep structured information. However, these models have not yet been broadly accepted. This fact is mainly due to its inherent complexity. In particular, not only for being extremely complex information processing models, but also because of a computational expensive learning phase. The most popular training method for these models is back-propagation through the structure. This algorithm has been revealed not to be the most appropriate for structured processing due to problems of convergence, while more sophisticated training methods enhance the speed of convergence at the expense of increasing significantly the computational cost. In this paper, we firstly perform an analysis of the underlying principles behind these models aimed at understanding their computational power. Secondly, we propose an approximate second order stochastic learning algorithm. The proposed algorithm dynamically adapts the learning rate throughout the training phase of the network without incurring excessively expensive computational effort. The algorithm operates in both on-line and batch modes. Furthermore, the resulting learning scheme is robust against the vanishing gradients problem. The advantages of the proposed algorithm are demonstrated with a real-world application example.

Affiliated with

Also associated with

No associations

Rating

Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-650648