From Linguistics to Statistics and AI

1980s marked a shift towards probabilistic statistical models. Moore’s law allowed for complex computations at scale. Major industry players (IBM etc.) successfully adopted large statistical models. A steady increase of computational power helped the evolution of machine learning algorithms.

The new era marked the dominance of statistical models and saw major improvements in text and speech recognition. Machine learning have become the mainstay of natural language processing.

In 1992 AT&T Bell Laboratories automated phone call routing. They did so by the means of voice recognition. Their system was able to interpret common requests such as person-to-person call or a collect call. Up until then these simple operations required human assistance. With voice recognition in place the phone calls could be completed without any manual interference.

In the same year researchers at Carnegie Mellon university achieved a significant milestone in their speech recognition project, Sphinx II. Check the PDF in the link, it has many interesting details. The researches were able to push the error rate down to 5% thanks to “large amounts of training data”. Courtesy of Wall Street Journal who contributed tens of millions of words and a few thousand utterances into the training data set.

In 1997 Sepp Hochreiter and Jürgen Schmidhuber came up with a new concept of neural network. Long short-term memory (LSTM) RNN network allowed for processing of large sequences of data, such in case of speech recognition. The invention accelerated further advancements in NLP.

Thanks for reading and if you like this post and want to learn more about pioneers in the space of NLP, go check the related posts in the section below.