History of natural language processing


The history of natural language processing describes the advances of natural language processing . There is some overlap with the history of machine translation, the history of speech recognition, and the history of artificial intelligence.

Research">Research on medit">Research and development

The history of machine translation dates back to the seventeenth century, when philosophers such as Leibniz and Descartes put forward proposals for codes which would relate words between languages. All of these proposals remained theoretical, and none resulted in the development of an actual machine.
The first patents for "translating machines" were applied for in the mid-1930s. One proposal, by Georges Artsrouni was simply an automatic bilingual dictionary using paper tape. The other proposal, by Peter Troyanskii, a Russian, was more detailed. It included both the bilingual dictionary, and a method for dealing with grammatical roles between languages, based on Esperanto.
In 1950, Alan Turing published his famous article "Computing Machinery and Intelligence" which proposed what is now called the Turing test as a criterion of intelligence. This criterion depends on the ability of a computer program to impersonate a human in a real-time written conversation with a human judge, sufficiently well that the judge is unable to distinguish reliably — on the basis of the conversational content alone — between the program and a real human.
In 1957, Noam Chomsky’s Syntactic Structures revolutionized Linguistics with 'universal grammar', a rule based system of syntactic structures.
The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English. The authors claimed that within three or five years, machine translation would be a solved problem. However, real progress was much slower, and after the ALPAC report in 1966, which found that ten years long research had failed to fulfill the expectations, funding for machine translation was dramatically reduced. Little further research in machine translation was conducted until the late 1980s, when the first statistical machine translation systems were developed.
Some notably successful NLP systems developed in the 1960s were SHRDLU, a natural language system working in restricted "blocks worlds" with restricted vocabularies.
In 1969 Roger Schank introduced the conceptual dependency theory for natural language understanding. This model, partially influenced by the work of Sydney Lamb, was extensively used by Schank's students at Yale University, such as Robert Wilensky, Wendy Lehnert, and Janet Kolodner.
In 1970, William A. Woods introduced the augmented transition network to represent natural language input. Instead of phrase structure rules ATNs used an equivalent set of finite state automata that were called recursively. ATNs and their more general format called "generalized ATNs" continued to be used for a number of years.During the 70's many programmers began to write 'conceptual ontologies', which structured real-world information into computer-understandable data. Examples are MARGIE, SAM, PAM, TaleSpin, QUALM, Politics, and Plot Units. During this time, many chatterbots were written including PARRY, Racter, and Jabberwacky.
Up to the 1980s, most NLP systems were based on complex sets of hand-written rules. Starting in the late 1980s, however, there was a revolution in NLP with the introduction of machine learning algorithms for language processing. This was due both to the steady increase in computational power resulting from Moore's Law and the gradual lessening of the dominance of Chomskyan theories of linguistics, whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the machine-learning approach to language processing. Some of the earliest-used machine learning algorithms, such as decision trees, produced systems of hard if-then rules similar to existing hand-written rules. Increasingly, however, research has focused on statistical models, which make soft, probabilistic decisions based on attaching real-valued weights to the features making up the input data. The cache language models upon which many speech recognition systems now rely are examples of such statistical models. Such models are generally more robust when given unfamiliar input, especially input that contains errors, and produce more reliable results when integrated into a larger system comprising multiple subtasks.
Many of the notable early successes occurred in the field of machine translation, due especially to work at IBM Research, where successively more complicated statistical models were developed. These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government. However, most other systems depended on corpora specifically developed for the tasks implemented by these systems, which was a major limitation in the success of these systems. As a result, a great deal of research has gone into methods of more effectively learning from limited amounts of data.
Recent research has increasingly focused on unsupervised and semi-supervised learning algorithms. Such algorithms are able to learn from data that has not been hand-annotated with the desired answers, or using a combination of annotated and non-annotated data. Generally, this task is much more difficult than supervised learning, and typically produces less accurate results for a given amount of input data. However, there is an enormous amount of non-annotated data available, which can often make up for the inferior results.

Software

SoftwareYearCreatorDescriptionReference
Georgetown experiment 1954Georgetown University and IBMinvolved fully automatic translation of more than sixty Russian sentences into English.
STUDENT 1964Daniel Bobrowcould solve high school algebra word problems.
ELIZA 1964Joseph Weizenbauma simulation of a Rogerian psychotherapist, rephrasing her response with a few grammar rules.
SHRDLU 1970Terry Winograda natural language system working in restricted "blocks worlds" with restricted vocabularies, worked extremely well
PARRY 1972Kenneth ColbyA chatterbot
KL-ONE 1974Sondheimer et al.a knowledge representation system in the tradition of semantic networks and frames; it is a frame language.
MARGIE 1975Roger Schank-
TaleSpin 1976Meehan-
QUALM Lehnert-
LIFER/LADDER 1978Hendrixa natural language interface to a database of information about US Navy ships.
SAM 1978Cullingford-
PAM 1978Robert Wilensky-
Politics 1979Carbonell-
Plot Units 1981Lehnert-
Jabberwacky 1982Rollo Carpenterchatterbot with stated aim to "simulate natural human chat in an interesting, entertaining and humorous manner".
MUMBLE 1982McDonald-
Racter 1983William Chamberlain and Thomas Etterchatterbot that generated English language prose at random.
MOPTRANS 1984Lytinen-
KODIAK 1986Wilensky-
Absity 1987Hirst-
Dr. Sbaitso 1991Creative Labs-
Watson 2006IBMA question answering system that won the Jeopardy! contest, defeating the best human players in February 2011.-
Siri 2011AppleA virtual assistant developed by Apple.-
Amazon Alexa 2014AmazonA virtual assistant developed by Amazon.-
Google Assistant 2016GoogleA virtual assistant developed by Google.

Com