Theory of Intelligence with Forgetting: Mathematical Theorems Explaining Human Universal Forgetting using “Forgetting Neural Networks”
Author(s)Cano-Córdoba, Felipe; Sarma, Sanjay; Subirana, Brian
MetadataShow full item record
In  we suggested that any memory stored in the human/animal brain is forgotten following the Ebingghaus curve – in this follow-on paper, we define a novel algebraic structure, a Forgetting Neural Network, as a simple mathematical model based on assuming parameters of a neuron in a neural network are forgotten using the Ebbinghaus forgetting curve. We model neural networks in Sobolev spaces using  as our departure point and demonstrate four novel theorems of Forgetting Neural Networks: theorem of non-instantaneous forgetting, theorem of universal forgetting, curse of forgetting theorem, and center of mass theorem. We also proof the novel decreasing inference theorem which we feel is relevant beyond Ebbinghaus forgetting: compositional deep neural networks cannot arbitrarily combine low level “features” – meaning only certain arrangements of features calculated in intermediate levels can show up in higher levels. This proof leads us to present the possibly most efficient representation of neural networks’ “minimal polynomial basis layer” (MPBL) since our basis construct can generate n polynomials of order m using only 2m + 1 + n neurons. As we briefly discuss in the conclusion, there are about 10 similarities between forgetting neural networks and human forgetting and our research elicits more questions than it answers and may have implications for neuroscience research including our understanding of how babies learn (or, perhaps, forget), including what we call the baby forgetting conjecture.
Center for Brains, Minds and Machines (CBMM)
CBMM Memo Series;071
Forgetting Neural Network, decreasing inference theorem, theorem of non-instantaneous forgetting, theorem of universal forgetting, curse of forgetting theorem, center of mass theorem
The following license files are associated with this item:
Showing items related by title, author, creator and subject.
Gibson, Nathan A.; Forget, Benoit; Gibson, Nathan Andrew; Forget, Benoit Robert Yves (Elsevier, 2014-01)This paper investigates the stability of the recondensation procedure of the Discrete Generalized Multigroup method and proposes alternatives to improve stability of the original formulation. Instabilities are shown to ...
Estimate of Radiation Release from MIT Reactor with Low Enriched Uranium (LEU) Core During Maximum Hypothetical Accident Plumer, Kevin E.; Newton, Thomas H., Jr.; Forget, Benoit (Massachusetts Institute of Technology. Center for Advanced Nuclear Energy Systems. MIT Reactor Redesign Program, 2011-06-01)In accordance with a 1986 NRC ruling, the MIT Research Reactor (MITR) is planning on converting from the use of highly enriched uranium (HEU) to low enriched uranium (LEU) for fuel. A component of the conversion analysis ...
Kempf, Stephanie A.; Hu, Lin-Wen; Forget, Benoit (Massachusetts Institute of Technology. Center for Advanced Nuclear Energy Systems. MIT Reactor Redesign Program, 2011-06-01)In response to increasing demands for the services of research reactors, a 5 MW LEUfueled research reactor core is developed and optimized to provide high thermal flux within specified limits upon thermal hydraulic ...