Disambiguating words with self-organizing maps

Abstract

Today, powerful programs readily parse English text; understanding, however, is another matter. In this thesis, I take a step toward understanding by introducing CLARIFY, a program that disambiguates words. CLARIFY identifies patterns in observed word contexts, and uses these patterns to select the optimal word sense for any specific situation. CLARIFY learns successful patterns by manipulating an accelerated Self-Organizing Map to save these example contexts and then references them to perform further context based disambiguation within the language. Through this process and after training on 125 examples, CLARIFY can now decipher that shrimp in the sentence "The shrimp goes to the store. " is a small-person, not relying on a literal definition of each word as a separate element but looking at the sentence as a fluid solution of many elements, thereby making the inference crustacean absurd. CLARIFY is implemented in 1500 lines of Java.