You have just completed your registration at OpenAire.
Before you can login to the site, you will need to activate your account.
An e-mail will be sent to you with the proper instructions.
Important!
Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version
of the site upon release.
In this paper, we analyze the recurrent correlation associative memory (RCAM) model of Chiueh and Goodman. This is an associative memory in which stored binary memory patterns are recalled via an iterative update rule. The update of the individual pattern-bits is controlled by an excitation function, which takes as its arguement the inner product between the stored memory patterns and the input patterns. Our contribution is to analyze the dynamics of pattern recall when the input patterns are corrupted by noise of a relatively unrestricted class. We make three contributions. First, we show how to identify the excitation function which maximizes the separation (the Fisher discriminant) between the uncorrupted realization of the noisy input pattern and the remaining patterns residing in the memory. Moreover, we show that the excitation function which gives maximum separation is exponential when the input bit-errors follow a binomial distribution. Our second contribution is to develop an expression for the expectation value of bit-error probability on the input pattern after one iteration. We show how to identify the excitation function which minimizes the bit-error probability. However, there is no closed-form solution and the excitation function must be recovered numerically. The relationship between the excitation functions which result from the two different approaches is examined for a binomial distribution of bit-errors. The final contribution is to develop a semiempirical approach to the modeling of the dynamics of the RCAM. This provides us with a numerical means of predicting the recall error rate of the memory. It also allows us to develop an expression for the storage capacity for a given recall error rate.
[4] E. Gardner, “Structure of metastable states in the Hopfield model,” J. Phys. A, vol. 19, no. 16, pp. 1047-1052, 1986.
[5] T. D. Chiueh and R. M. Goodman, “Recurrent correlation associative memories,” IEEE Trans. Neural Networks, vol. 2, pp. 275-284, Mar. 1991.
[6] , “VLSI implementation of a high-capacity neural network associative memory,” in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 793-800.
[7] C. C. Wang and H. S. Don, “An analysis of high-capacity discrete exponential BAM,” IEEE Trans. Neural Networks, vol. 6, pp. 492-496, Mar. 1995.
[8] T. D. Chiueh and H. K. Tsai, “Multi-valued associative memories based on recurrent networks,” IEEE Trans. Neural Networks, vol. 4, pp. 364-366, Mar. 1993.
[9] P. Kanerva, Sparse Distributed Memory. Cambridge, MA: MIT Press, 1988.
[10] D. Milun and D. Sher, “Improving sampled probability distributions for Markov random fields,” Pattern Recognition Lett., vol. 14, pp. 781-788, 1993.
[11] P. A. Chou, “The capacity of the Kanerva associative memory is exponential,” in Neural Information Processing Systems, D. Z. Anderson, Ed. New York: AIP, 1988, pp. 184-191.
[12] P. Whittle, “Artificial memories: Capacity, basis rate and inference,” Neural Networks, vol. 10, pp. 1619-1626, 1997.
[13] C. C. Wang, S. M. Hwang, and J. P. Lee, “Capacity analysis of the asymptotically stable multi-valued exponential directional associative memory,” IEEE Trans. Syst., Man, Cybern., vol. 26, pp. 733-743, June 1996.
[14] Z. B. Xu, G. Q. Huy, and C. P. Kwong, “Asymmetric Hopfield-type networks; Theory and applications,” Neural Natworks, vol. 9, pp. 483-501, 1996.
[15] Z. Y. Chen, C. P. Kwong, and Z. B. Xu, “Multiple valued feedback and recurrent correlation neural networks,” Neural Comput. Applicat., vol. 3, pp. 242-250, 1995.
[16] C. C. Wang and J. P. Lee, “The decision making properties of discrete multiple exponential bi-directional associative memories,” IEEE Trans. Neural Networks, vol. 6, pp. 993-999, July 1995.
[17] H. C. Shi, Y. X. Zhao, and X. H. Zhuang, “A general model for bidirectional associative memories,” IEEE Trans. Syst., Man, Cybern. B, vol. 128, pp. 511-519, June 1998.
[18] C. C. Wang, C. F. Tsai, and J. P. Lee, “Analysis of radix searching of exponential bidirectional associative memory,” Proc. Inst. Elect. Eng. Comput. Digital Techniques, vol. 145, pp. 279-285, 1998.
[19] C. C. Wang, “The majority theorem of centralized multiple BAM's networks,” Inform. Sci., vol. 110, pp. 179-193, 1998.
[20] E. R. Hancock and M. Pelillo, “The Bayesian interpretation of the exponential correlation associative memory,” Pattern Recognition Lett., vol. 19, pp. 149-159, 1998.
[21] E. R. Hancock and J. Kittler, “Discrete relaxation,” Pattern Recognition, vol. 23, pp. 711-733, 1990.
[22] C. Mazza, “On the storage capacity of nonlinear neural networks,” Neural Networks, vol. 10, pp. 593-597, 1997.
[24] F. T. Sommer and P. Dayan, “Bayesian retrieval in associative memories with storage errors,” IEEE Trans. Neural Networks, vol. 9, pp. 705-713, July 1998.
[25] A. Jagota, G. Narasimhan, and K. W. Regan, “Information capacity of binary weights associative memories,” Neurocomputing, vol. 19, pp. 35-58, 1998.
[26] D. Burshtein, “Typical error pattern recovery of the Hopfield memory under error tolerant conditions,” IEEE Trans. Inform. Theory, vol. 44, pp. 861-865, Apr. 1998.
[27] D. Casasent and B. Telfer, “High-capacity pattern-recognition associative-processors,” Neural Networks, vol. 5, no. 4, pp. 687-698, 1992.
[28] M. Lowe, “On the storage capacity of the Hopfield model with biased patterns,” IEEE Trans. Inform. Theory, vol. 45, pp. 314-318, Feb. 1999.
[29] W. Feller, An Introduction to Probability Theory and Its Applications, 3rd ed. New York: Wiley, 1971, vol. 2, pp. 257-258. Richard C. Wilson received the B.A. degree in physics from the University of Oxford, Oxford, U.K., in 1992 and the D.Phil. degree from the University of York, York, U.K., for his thesis “inexact graph matching using symbolic constraints” in 1996. From 1996 to 1998, he was a Research Associate at the University of York. After a period of postdoctoral research, he was awarded an Advanced Research Fellowship in 1998, a position which he currently holds in the Department of Computer Science at the University of York. He has published approximately 70 papers in journals, edited books, and refereed conferences. He is currently an Associate Editor of the journal Pattern Recognition. His research interests are in statistical and structural pattern recognition, graph methods for computer vision, high-level vision, and scene understanding. Dr. Wilson received an outstanding paper award in the 1997 Pattern Recognition Society awards and has won the best paper prize in ACCV 2002. He is a member of the IEEE computer society.