Building a similarity engine

Abstract

In the seventeenth century, Philosophers such as Leibniz and Descartes put forward proposal for codes to relate words between languages. The first patents for "translating machines" were applied for in the mid-1930s. Up to the 1980s, most Natural Language Processing (NLP) systems were based on complex sets of hand-written rules. At that time however, the introduction of machine learning algorithms for language processing revolutionized NLP.[5] In 2008, Collobert and Weston exhibited the power of pre-trained word embed- dings in a paper called A unified architecture for natural language processing. Here, word embeddings is highlight for its ability in downstream tasks. They also discuss a neural network architecture that many of todays approaches are built upon. In 2013, Mikolov created word2vec, a toolkit that enabled the training and use of pre-trained embeddings. In 2014, Pennington introduced GloVe, a competitive set of pre-trained embeddings. Starting off, a single word or group of words can be converted into a vector. This vector can be created using the Skip gram method, which predicts the possible words nearby, the LSTM-RNN method, which forms semantic representations of sentences by learning more about the sentence as it iterates through a sentence, using single convolution neural networks, and several other methods. Using these theories, we are trying to build a Similarity Engine which provides machine learning based content search and classification of data.