LOGIN TO YOUR ACCOUNT

Username
Password
Remember Me
Or use your Academic/Social account:

CREATE AN ACCOUNT

Or use your Academic/Social account:

Congratulations!

You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.

Important!

Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message

CREATE AN ACCOUNT

Name:
Username:
Password:
Verify Password:
E-mail:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Scarpetta, Silvia; Rattray, Magnus; Saad, David
Publisher: IEEE
Languages: English
Types: Part of book or chapter of book
Subjects:

Classified by OpenAIRE into

ACM Ref: MathematicsofComputing_NUMERICALANALYSIS
Natural gradient learning is an efficient and principled method for improving on-line learning. In practical applications there will be an increased cost required in estimating and inverting the Fisher information matrix. We propose to use the matrix momentum algorithm in order to carry out efficient inversion and study the efficacy of a single step estimation of the Fisher information matrix. We analyse the proposed algorithm in a two-layer network, using a statistical mechanics framework which allows us to describe analytically the learning dynamics, and compare performance with true natural gradient learning and standard gradient descent.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • [l] Saad D (ed) 1998 On-Line Learning in Neural Networks, Publications of the Newton Institute, Cambridge University Press, Cambridge.
    • [2] Amari S 1998 'Natural gradient works efficiently in learning' Neural Comp. 10 251.
    • [3] Rattray M and Saad D 1998 'Transient and Asymptotics od Natural Gradient Learning', Proc. of the Int. Conf. on Artificial Neural Networks, ed Niklasson, Bodbden and Ziemke (London, UK: Springer-Verlag) 165; Rattray M Saad D and Amari S 1998, Phys. Rev. Lett.,
    • [4] Orr G B and Leen T K 1997 Advances in Neural Information Processing Systems, vol 9 , ed Mozer, Jordan and Petsche (Cambridge, MA: MIT Press) 606.
    • (5) Rattray M and Saad D 1998 'The Dynamics of Matrix Momentum', Proc. of the Int. Conf. on Artificial Neural Networks, ed Niklasson, Bodbden and Ziemke (London, UK: Springer-Verlag) 183.
    • [6] Saad D and Solla S A 1995 Phys. Rev. Lett. 74 4337, Phys. Rev. E 5 2 4225.
    • [7] Yang H Y and Amari S 1997 Advances in Neural Information Processing Systems, vol 10, ed Mozer, Jordan and Petsche (Cambridge, MA: MIT Press) 385.
    • [8] Cybenko Math. Control Signals and Systems 2 303.
    • [9] Prugel-Bennett A 1996, unpublished notes.
    • [lo] Saad D and Rattray M 1997 Phys. Rev. Lett. 79 2578; Rattray M and Saad D 1998 Phys. Rev. E 58 6379.
    • [ll]Amari S and Murata N 1993, Neural Comp. 5 140.
    • [12] Leen T K, Schottky'B and Saad D 1998 Advanced in Neural Information Systems Vol. 10, 301, ed. by Jordan, Kearns, Solla (Cambridge, MA: MIT Press); Leen T K , Schottky B and Saad D 1999 Phys. Rev. E 59 985.
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article