History of software engineering


From its beginnings in the 1960s, writing software has evolved into a profession concerned with how best to maximize the quality of software and of how to create it. Quality can refer to how maintainable software is, to its stability, speed, usability, testability, readability, size, cost, security, and number of flaws or "bugs", as well as to less measurable qualities like elegance, conciseness, and customer satisfaction, among many other attributes. How best to create high quality software is a separate and controversial problem covering software design principles, so-called "best practices" for writing code, as well as broader management issues such as optimal team size, process, how best to deliver software on time and as quickly as possible, work-place "culture", hiring practices, and so forth. All this falls under the broad rubric of software engineering.

Overview

The evolution of software engineering is notable in a number of areas:
Today, fewer women work in software engineering than in other professions, a situation whose cause is not clearly identified. Many academic and professional organizations consider this situation unbalanced and are trying hard to solve it.
Putative origins for the term software engineering include a 1965 letter from ACM president Anthony Oettinger, lectures by Douglas T. Ross at MIT in the 1950s. Margaret H. Hamilton "is the person who came up with the idea of naming the discipline, software engineering, as a way of giving it legitimacy."
The NATO Science Committee sponsored two conferences on software engineering in 1968 and 1969, which gave the field its initial boost. Many believe these conferences marked the official start of the profession of software engineering.

1965 to 1985: The software crisis

Software engineering was spurred by the so-called software crisis of the 1960s, 1970s, and 1980s, which identified many of the problems of software development. Many projects ran over budget and schedule. Some projects caused property damage. A few projects caused loss of life. The software crisis was originally defined in terms of productivity, but evolved to emphasize quality. Some used the term software crisis to refer to their inability to hire enough qualified programmers.
Peter G. Neumann has kept a contemporary list of software problems and disasters. The software crisis has been fading from view, because it is psychologically extremely difficult to remain in crisis mode for a protracted period. Nevertheless, software – especially real-time embedded software – remains risky and is pervasive, and it is crucial not to give in to complacency. Over the last 10–15 years Michael A. Jackson has written extensively about the nature of software engineering, has identified the main source of its difficulties as lack of specialization, and has suggested that his problem frames provide the basis for a "normal practice" of software engineering, a prerequisite if software engineering is to become an engineering science.

1985 to 1989: "''No Silver Bullet''"

For decades, solving the software crisis was paramount to researchers and companies producing software tools.
The cost of owning and maintaining software in the 1980s was twice as expensive as developing the software.
Seemingly, every new technology and practice from the 1970s through the 1990s was trumpeted as a silver bullet to solve the software crisis. Tools, discipline, formal methods, process, and professionalism were touted as silver bullets:
In 1986, Fred Brooks published his No Silver Bullet article, arguing that no individual technology or practice would ever make a 10-fold improvement in productivity within 10 years.
Debate about silver bullets raged over the following decade. Advocates for Ada, components, and processes continued arguing for years that their favorite technology would be a silver bullet. Skeptics disagreed. Eventually, almost everyone accepted that no silver bullet would ever be found. Yet, claims about silver bullets pop up now and again, even today.
Some interpret no silver bullet to mean that software engineering failed. However, with further reading, Brooks goes on to say: "We will surely make substantial progress over the next 40 years; an order of magnitude over 40 years is hardly magical..."
The search for a single key to success never worked. All known technologies and practices have only made incremental improvements to productivity and quality. Yet, there are no silver bullets for any other profession, either. Others interpret no silver bullet as proof that software engineering has finally matured and recognized that projects succeed due to hard work.
However, it could also be said that there are, in fact, a range of silver bullets today, including lightweight methodologies, spreadsheet calculators, customized browsers, in-site search engines, database report generators, integrated design-test coding-editors with memory/differences/undo, and specialty shops that generate niche software, such as information web sites, at a fraction of the cost of totally customized web site development. Nevertheless, the field of software engineering appears too complex and diverse for a single "silver bullet" to improve most issues, and each issue accounts for only a small portion of all software problems.

1990 to 1999: Prominence of the Internet

The rise of the Internet led to very rapid growth in the demand for international information display/e-mail systems on the World Wide Web. Programmers were required to handle illustrations, maps, photographs, and other images, plus simple animation, at a rate never before seen, with few well-known methods to optimize image display/storage.
The growth of browser usage, running on the HyperText Markup Language, changed the way in which information-display and retrieval was organized. The widespread network connections led to the growth and prevention of international computer viruses on MS Windows computers, and the vast proliferation of spam e-mail became a major design issue in e-mail systems, flooding communication channels and requiring semi-automated pre-screening. Keyword-search systems evolved into web-based search engines, and many software systems had to be re-designed, for international searching, depending on search engine optimization techniques. Human natural-language translation systems were needed to attempt to translate the information flow in multiple foreign languages, with many software systems being designed for multi-language usage, based on design concepts from human translators. Typical computer-user bases went from hundreds, or thousands of users, to, often, many-millions of international users.

2000 to 2015: Lightweight methodologies

With the expanding demand for software in many smaller organizations, the need for inexpensive software solutions led to the growth of simpler, faster methodologies that developed running software, from requirements to deployment, quicker & easier. The use of rapid-prototyping evolved to entire lightweight methodologies, such as Extreme Programming, which attempted to simplify many areas of software engineering, including requirements gathering and reliability testing for the growing, vast number of small software systems. Very large software systems still used heavily documented methodologies, with many volumes in the documentation set; however, smaller systems had a simpler, faster alternative approach to managing the development and maintenance of software calculations and algorithms, information storage/retrieval and display.

Current trends in software engineering

Software engineering is a young discipline, and is still developing. The directions in which software engineering is developing include:

Aspects

help software engineers deal with quality attributes by providing tools to add or remove boilerplate code from many areas in the source code. Aspects describe how all objects or functions should behave in particular circumstances. For example, aspects can add debugging, logging, or locking control into all objects of particular types. Researchers are currently working to understand how to use aspects to design general-purpose code. Related concepts include generative programming and templates.

Experimental

is a branch of software engineering interested in devising experiments on software, in collecting data from the experiments, and in devising laws and theories from this data. Proponents of this method advocate that the nature of software is such that we can advance the knowledge on software through experiments only.

Software product lines

, is a systematic way to produce families of software systems, instead of creating a succession of completely individual products. This method emphasizes extensive, systematic, formal code reuse, to try to industrialize the software development process.
The Future of Software Engineering conference, held at ICSE 2000, documented the state of the art of SE in 2000 and listed many problems to be solved over the next decade. The FOSE tracks at the ICSE 2000 and the ICSE 2007 conferences also help identify the state of the art in software engineering.

Software engineering today

The profession is trying to define its boundary and content. The Software Engineering Body of Knowledge SWEBOK has been tabled as an ISO standard during 2006.
In 2006, Money Magazine and Salary.com rated software engineering as the best job in America in terms of growth, pay, stress levels, flexibility in hours and working environment, creativity, and how easy it is to enter and advance in the field.

Sub-disciplines

Artificial intelligence

A wide variety of platforms has allowed different aspects of AI to develop, ranging from expert systems such as Cyc to deep-learning frameworks to robot platforms such as the Roomba with open interface. Recent advances in deep artificial neural networks and distributed computing have led to a proliferation of software libraries, including Deeplearning4j, TensorFlow, Theano and Torch.
A 2011 McKinsey Global Institute study found a shortage of 1.5 million highly trained data and AI professionals and managers and a number of private bootcamps have developed programs to meet that demand, including free programs like The Data Incubator or paid programs like General Assembly.

Languages

Early symbolic AI inspired Lisp and Prolog, which dominated early AI programming. Modern AI development often uses mainstream languages such as Python or C++, or niche languages such as Wolfram Language.

Prominent figures in the history of software engineering