TeX


TeX, stylized within the system as TeX, is a typesetting system which was designed and mostly written by Donald Knuth and released in 1978. TeX is a popular means of typesetting complex mathematical formulae; it has been noted as one of the most sophisticated digital typographical systems.
TeX is popular in academia, especially in mathematics, computer science, economics, engineering, linguistics, physics, statistics, and quantitative psychology. It has largely displaced Unix troff, the other favored formatting system, in many Unix installations which use both for different purposes. It is also used for many other typesetting tasks, especially in the form of LaTeX, ConTeXt, and other macro packages.
TeX was designed with two main goals in mind: to allow anybody to produce high-quality books with minimal effort, and to provide a system that would give exactly the same results on all computers, at any point in time. TeX is free software, which made it accessible to a wide range of users.

History

When the first paper volume of Donald Knuth's The Art of Computer Programming was published in 1968, it was typeset using hot metal typesetting set by a Monotype machine. This method, dating back to the 19th century, produced a "classic style" appreciated by Knuth. When the second edition was published, in 1976, the whole book had to be typeset again because the Monotype technology had been largely replaced by phototypesetting, and the original fonts were no longer available. When Knuth received the galley proofs of the new book on 30 March 1977, he found them inferior.
Disappointed by the galley proofs of the second edition of the second volume, he was motivated to design his own typesetting system. Knuth saw for the first time the output of a high-quality digital typesetting system, and became interested in digital typography. On 13 May 1977, he wrote a memo to himself describing the basic features of TeX.
He planned to finish it on his sabbatical in 1978, but as it happened, the language was not "frozen" until 1989, more than ten years later. Guy Steele happened to be at Stanford during the summer of 1978, when Knuth was developing his first version of TeX. When Steele returned to the Massachusetts Institute of Technology that autumn, he rewrote TeX's input/output to run under the Incompatible Timesharing System operating system. The first version of TeX, called TeX78, was written in the SAIL programming language to run on a PDP-10 under Stanford's WAITS operating system.

WEB and literate programming

For later versions of TeX, Knuth invented the concept of literate programming, a way of producing compilable source code and cross-linked documentation typeset in TeX from the same original file. The language used is called WEB and produces programs in DEC PDP-10 Pascal.

TeX82

TeX82, a new version of TeX which is rewritten from scratch, was published in 1982. Among other changes, the original hyphenation algorithm was replaced by a new algorithm written by Frank Liang. TeX82 also uses fixed-point arithmetic instead of floating-point, to ensure reproducibility of the results across different computer hardware, and includes a real, Turing-complete programming language, following intense lobbying by Guy Steele. In 1989, Donald Knuth released new versions of TeX and Metafont. Despite his desire to keep the program stable, Knuth realised that 128 different characters for the text input were not enough to accommodate foreign languages; the main change in version 3.0 of TeX is thus the ability to work with 8-bit inputs, allowing 256 different characters in the text input.
Since version 3, TeX has used an idiosyncratic version numbering system, where updates have been indicated by adding an extra digit at the end of the decimal, so that the version number asymptotically approaches Pi|. This is a reflection of the fact that TeX is now very stable, and only minor updates are anticipated. The current version of TeX is 3.14159265; it was last updated on January 12, 2014. The design was frozen after version 3.0, and no new feature or fundamental change will be added, so all newer versions will contain only bug fixes. Even though Donald Knuth himself has suggested a few areas in which TeX could have been improved, he indicated that he firmly believes that having an unchanged system that will produce the same output now and in the future is more important than introducing new features. For this reason, he has stated that the "absolutely final change " will be to change the version number to, at which point all remaining bugs will become features. Likewise, versions of Metafont after 2.0 asymptotically approach E |, and a similar change will be applied after Knuth's death.

Public domain

Since the source code of TeX is essentially in the public domain, other programmers are allowed to improve the system, but are required to use another name to distribute the modified TeX, meaning that the source code can still evolve. For example, the Omega project was developed after 1991, primarily to enhance TeX's multilingual typesetting abilities. Knuth created "unofficial" modified versions, such as TeX-XeT, which allows a user to mix texts written in left-to-right and right-to-left writing systems in the same document.

Use of TeX

In several technical fields such as computer science, mathematics, engineering and physics, TeX has become a de facto standard. Many thousands of books have been published using TeX, including books published by Addison-Wesley, Cambridge University Press, Elsevier, Oxford University Press and Springer. Numerous journals in these fields are produced using TeX or LaTeX, allowing authors to submit their raw manuscript written in TeX. While many publications in other fields, including dictionaries and legal publications, have been produced using TeX, it has not been as successful as in the more technical fields, as TeX was primarily designed to typeset mathematics.
When he designed TeX, Donald Knuth did not believe that a single typesetting system would fit everyone's needs; instead, he designed many hooks inside the program so that it would be possible to write extensions, and released the source code, hoping that the publishers would design versions tailoring to their own needs. While such extensions have been created, most people have extended TeX only using macros and it has remained a system associated with technical typesetting.

Typesetting system

TeX commands commonly start with a backslash and are grouped with curly braces. Almost all of TeX's syntactic properties can be changed on the fly, which makes TeX input hard to parse by anything but TeX itself. TeX is a macro- and token-based language: many commands, including most user-defined ones, are expanded on the fly until only unexpandable tokens remain, which are then executed. Expansion itself is practically free from side effects. Tail recursion of macros takes no memory, and if-then-else constructs are available. This makes TeX a Turing-complete language even at the expansion level. The system can be divided into four levels: in the first, characters are read from the input file and assigned a category code. Combinations of a backslash followed by letters or a single other character are replaced by a control-sequence token. In this sense, this stage is like lexical analysis, although it does not form numbers from digits. In the next stage, expandable control sequences are replaced by their replacement text. The input for the third stage is then a stream of characters and unexpandable control sequences. Here, the characters get assembled into a paragraph, and TeX's paragraph breaking algorithm works by optimizing breakpoints over the whole paragraph. The fourth stage breaks the vertical list of lines and other material into pages.
The TeX system has precise knowledge of the sizes of all characters and symbols, and using this information, it computes the optimal arrangement of letters per line and lines per page. It then produces a DVI file containing the final locations of all characters. This dvi file can then be printed directly given an appropriate printer driver, or it can be converted to other formats. Nowadays, pdfTeX is often used, which bypasses DVI generation altogether. The base TeX system understands about 300 commands, called primitives. These low-level commands are rarely used directly by users, and most functionality is provided by format files. Knuth's original default format, which adds about 600 commands, is Plain TeX. The most widely used format is LaTeX, originally developed by Leslie Lamport, which incorporates document styles for books, letters, slides, etc., and adds support for referencing and automatic numbering of sections and equations. Another widely used format, AMS-TeX, is produced by the American Mathematical Society and provides many more user-friendly commands, which can be altered by journals to fit with their house style. Most of the features of AMS-TeX can be used in LaTeX by using the "AMS packages" and the "AMS document classes". This is then referred to as AMS-LaTeX. Other formats include ConTeXt, used primarily for desktop publishing and written mostly by Hans Hagen at Pragma.

How it is run

A sample Hello world program in plain TeX is:

Hello, World
\bye % marks the end of the file; not shown in the final output

This might be in a file myfile.tex, as .tex is a common file extension for plain TeX files. By default, everything that follows a percent sign on a line is a comment, ignored by TeX. Running TeX on this file will create an output file called myfile.dvi, representing the content of the page in a device independent format. A DVI file could then either be viewed on screen or converted to a suitable format for any of the various printers for which a device driver existed. Knuth has said that there is nothing inherent in TeX that requires DVI as the output format, and later versions of TeX, notably pdfTeX, XeTeX and LuaTeX, all support output directly to PDF.

Mathematical example

TeX provides a different text syntax specifically for mathematical formulas. For example, the quadratic formula appears as:
The formula is printed in a way a person would write by hand, or typeset the equation. In a document, entering mathematics mode is done by starting with a $ symbol, then entering a formula in TeX syntax, and closing again with another of the same symbol. Knuth explained in jest that he chose the dollar sign to indicate the beginning and end of mathematical mode in plain TeX because typesetting mathematics was traditionally supposed to be expensive. Display mathematics is similar but uses $$ instead of a single $ symbol. For example, the above with the quadratic formula in display math:

Aspects

The TeX software incorporates several aspects that were not available, or were of lower quality, in other typesetting programs at the time when TeX was released. Some of the innovations are based on interesting algorithms, and have led to several theses for Knuth's students. While some of these discoveries have now been incorporated into other typesetting programs, others, such as the rules for mathematical spacing, are still unique.

Mathematical spacing

Since the primary goal of the TeX language is high-quality typesetting for publishers of books, Knuth gave a lot of attention to the spacing rules for mathematical formulae. He took three bodies of work that he considered to be standards of excellence for mathematical typography: the books typeset by Addison-Wesley Publishing's house, in particular the work of Hans Wolf and Joseph Louis Lagrange ; editions of the mathematical journal Acta Mathematica dating from around 1910; and a copy of Indagationes Mathematicae, a Dutch mathematics journal. Knuth looked closely at these printed papers to sort out and look for a set of rules for spacing. While TeX provides some basic rules and the tools needed to specify proper spacing, the exact parameters depend on the font used to typeset the formula. For example, the spacing for Knuth's Computer Modern fonts has been precisely fine-tuned over the years and is now set; but when other fonts, such as AMS Euler, were used by Knuth for the first time, new spacing parameters had to be defined.
The typesetting of Math in TeX is not without criticism, particularly with respect to technical details of the font metrics, which were designed in an era when significant attention was paid to storage requirements. This resulted in some "hacks" overloading some fields, which in turn required other "hacks". On an aesthetics level, the rendering of radicals has also been criticized. The OpenType math font specification largely borrows from TeX, but has some new features/enhancements.

Hyphenation and justification

In comparison with manual typesetting, the problem of justification is easy to solve with a digital system such as TeX, which, provided that good points for line breaking have been defined, can automatically spread the spaces between words to fill in the line. The problem is thus to find the set of breakpoints that will give the most visually pleasing result. Many line breaking algorithms use a first-fit approach, where the breakpoints for each line are determined one after the other, and no breakpoint is changed after it has been chosen. Such a system is not able to define a breakpoint depending on the effect that it will have on the following lines. In comparison, the total-fit line breaking algorithm used by TeX and developed by Donald Knuth and Michael Plass considers all the possible breakpoints in a paragraph, and finds the combination of line breaks that will produce the most globally pleasing arrangement.
Formally, the algorithm defines a value called badness associated with each possible line break; the badness is increased if the spaces on the line must stretch or shrink too much to make the line the correct width. Penalties are added if a breakpoint is particularly undesirable: for example, if a word must be hyphenated, if two lines in a row are hyphenated, or if a very loose line is immediately followed by a very tight line. The algorithm will then find the breakpoints that will minimize the sum of squares of the badness of the resulting lines. If the paragraph contains possible breakpoints, the number of situations that must be evaluated naively is. However, by using the method of dynamic programming, the complexity of the algorithm can be brought down to . Further simplifications lead to an efficient algorithm whose running time is, where is the width of a line. A similar algorithm is used to determine the best way to break paragraphs across two pages, in order to avoid widows or orphans. However, in general, a thesis by Michael Plass shows how the page breaking problem can be NP-complete because of the added complication of placing figures. TeX's line breaking algorithm has been adopted by several other programs, such as Adobe InDesign and the GNU fmt Unix command line utility.
If no suitable line break can be found for a line, the system will try to hyphenate a word. The original version of TeX used a hyphenation algorithm based on a set of rules for the removal of prefixes and suffixes of words, and for deciding if it should insert a break between the two consonants in a pattern of the form vowel–consonant–consonant–vowel. TeX82 introduced a new hyphenation algorithm, designed by Frank Liang in 1983, to assign priorities to breakpoints in letter groups. A list of hyphenation patterns is first generated automatically from a corpus of hyphenated words. If TeX must find the acceptable hyphenation positions in the word encyclopedia, for example, it will consider all the subwords of the extended word .encyclopedia., where . is a special marker to indicate the beginning or end of the word. The list of subwords includes all the subwords of length 1, of length 2, etc., up to the subword of length 14, which is the word itself, including the markers. TeX will then look into its list of hyphenation patterns, and find subwords for which it has calculated the desirability of hyphenation at each position. In the case of our word, 11 such patterns can be matched, namely 1c4l4, 1cy, 1d4i3a, 4edi, e3dia, 2i1a, ope5d, 2p2ed, 3pedi, pedia4, y1c. For each position in the word, TeX will calculate the maximum value obtained among all matching patterns, yielding en1cy1c4l4o3p4e5d4i3a4. Finally, the acceptable positions are those indicated by an odd number, yielding the acceptable hyphenations en-cy-clo-pe-di-a. This system based on subwords allows the definition of very general patterns, with low indicative numbers, which can then be superseded by more specific patterns if necessary. These patterns find about 90% of the hyphens in the original dictionary; more importantly, they do not insert any spurious hyphen. In addition, a list of exceptions are included with the Plain TeX format; additional ones can be specified by the user.

Metafont

Metafont, not strictly part of TeX, is a font description system which allows the designer to describe characters algorithmically. It uses Bézier curves in a fairly standard way to generate the actual characters to be displayed, but Knuth devotes substantial attention to the rasterizing problem on bitmapped displays. Another thesis, by John Hobby, further explores this problem of digitizing "brush trajectories". This term derives from the fact that Metafont describes characters as having been drawn by abstract brushes. It is commonly believed that TeX is based on bitmap fonts but, in fact, these programs "know" nothing about the fonts that they are using other than their dimensions. It is the responsibility of the device driver to appropriately handle fonts of other types, including PostScript Type 1 and TrueType. Computer Modern is freely available in Type 1 format, as are the AMS math fonts. Users of TeX systems that output directly to PDF, such as pdfTeX, XeTeX, or LuaTeX, generally never use Metafont output at all.

Macro language

TeX documents are written and programmed using an unusual macro language. Broadly speaking, the running of this macro language involves expansion and execution stages which do not interact directly. Expansion includes both literal expansion of macro definitions as well as conditional branching, and execution involves such tasks as setting variables/registers and the actual typesetting process of adding glyphs to boxes.
The definition of a macro not only includes a list of commands but also the syntax of the call. It differs with most widely used lexical preprocessors like M4, in that the body of a macro gets tokenized at definition time.
The TeX macro language has been used to write larger document production systems, most notably including LaTeX and ConTeXt.

Development

The original source code for the current TeX software is written in WEB, a mixture of documentation written in TeX and a Pascal subset in order to ensure portability. For example, TeX does all of its dynamic allocation itself from fixed-size arrays and uses only fixed-point arithmetic for its internal calculations. As a result, TeX has been ported to almost all operating systems, usually by using the web2c program to convert the source code into C instead of directly compiling the Pascal code. Knuth has kept a very detailed log of all the bugs he has corrected and changes he has made in the program since 1982;, the list contains 427 entries, not including the version modification that should be done after his death as the final change in TeX. Knuth offers monetary awards to people who find and report a bug in TeX. The award per bug started at US$2.56 and doubled every year until it was frozen at its current value of $327.68. Knuth has lost relatively little money as there have been very few bugs claimed. In addition, recipients have been known to frame their check as proof that they found a bug in TeX rather than cashing it.
Due to scammers finding scanned copies of his checks on the internet and using them to try to drain his bank account, Knuth no longer sends out real checks, but those who submit bug reports can get credit at The Bank of San Serriffe instead.

Distributions and extensions

TeX is usually provided in the form of an easy-to-install bundle of TeX itself along with Metafont and all the necessary fonts, documents formats, and utilities needed to use the typesetting system. On UNIX-compatible systems, including Linux and Apple macOS, TeX is distributed as part of the larger TeX Live distribution. On Microsoft Windows, there is the MiKTeX distribution and the Microsoft Windows version of TeX Live.
Several document processing systems are based on TeX, notably jadeTeX, which uses TeX as a backend for printing from James Clark's DSSSL Engine, the Arbortext publishing system, and Texinfo, the GNU documentation processing system. TeX has been the official typesetting package for the GNU operating system since 1984.
Numerous extensions and companion programs for TeX exist, among them BibTeX for bibliographies, pdfTeX, a TeX-compatible engine which bypasses DVI and produces output in PDF, XeTeX, a TeX-compatible engine that supports Unicode and OpenType, and LuaTeX, a Unicode-aware extension to TeX that includes a Lua runtime with extensive hooks into the underlying TeX routines and algorithms. Most TeX extensions are available for free from CTAN, the Comprehensive TeX Archive Network.

Editors

There are a variety of :Category:TeX editors|editors designed to work with TeX:
  1. The TeXmacs text editor is a WYSIWYG scientific text editor, inspired by both TeX and Emacs. It uses Knuth's fonts and can generate TeX output.
  2. BaKoMa-TeX is a Windows/Mac/Linux-based WYSIWYG editor that allows for editing the document by directly altering the source code, or by directly altering the preview of the source code.
  3. Overleaf is a partial-WYSIWYG, online editor that provides a cloud-based solution to TeX along with additional features in real-time collaborative editing.
  4. LyX is a WYSIWYM document processor which runs on a variety of platforms including:
  5. # Linux,
  6. # Microsoft Windows
  7. # Apple Mac OS X.
  8. TeXShop, TeXworks and WinShell are similar tools and provide an integrated development environment for working with LaTeX or TeX. For KDE/Qt, Kile provides such an IDE.
  9. Texmaker is the Pure Qt equivalent of Kile, with a user interface that is nearly the same as Kile's.
  10. TeXstudio is an open-source fork of Texmaker that offers a different approach to configurability and features. Free downloadable binaries are provided for Windows, Linux, Mac OS X, OS/2, and FreeBSD.
  11. GNU Emacs has various built-in and third-party packages with support for TeX, the major one being AUCTeX.
  12. For Vim, possible plugins include Vim-LaTeX Suite, Automatic TeX and TeX-9.
  13. For Google Docs, Auto-Latex Equations is a Google Docs add-on that provides mathematical TeX typesetting.
  14. For Apache OpenOffice and LibreOffice, iMath and TexMaths extensions can provides mathematical TeX typesetting.
  15. For MediaWiki, the provides mathematical TeX typesetting, but the code need to be surrounded by tag.

    License

Donald Knuth has indicated several times that the source code of TeX has been placed into the "public domain", and he strongly encourages modifications or experimentations with this source code. In particular, since Knuth highly values the reproducibility of the output of all versions of TeX, any changed version must not be called TeX, or anything confusingly similar. To enforce this rule, any implementation of the system must pass a test suite called the TRIP test before being allowed to be called TeX. The question of license is somewhat confused by the statements included at the beginning of the TeX source code, which indicate that "all rights are reserved. Copying of this file is authorized only if ... you make absolutely no changes to your copy". This restriction should be interpreted as a prohibition to change the source code as long as the file is called tex.web. This interpretation is confirmed later in the source code when the TRIP test is mentioned. The American Mathematical Society tried in the early 1980s to claim a trademark for TeX. This was rejected because at the time "TEX" was registered by Honeywell for the "Text EXecutive" text processing system.

XML publication

It is possible to use TeX for automatic generation of sophisticated layout for XML data. The differences in syntax between the two description languages can be overcome with the help of TeXML. In the context of XML publication, TeX can thus be considered an alternative to XSL-FO. TeX allowed scientific papers in mathematical disciplines to be reduced to relatively small files that could be rendered client-side, allowing fully typeset scientific papers to be exchanged over the early Internet and emerging World Wide Web, even when sending large files was difficult. This paved the way for the creation of repositories of scientific papers such as arXiv, through which papers could be 'published' without an intermediary publisher.

Pronunciation and spelling

The name TeX is intended by its developer to be, with the final consonant of loch or Bach. The letters of the name are meant to represent the capital Greek letters tau, epsilon, and chi, as TeX is an abbreviation of τέχνη, Greek for both "art" and "craft", which is also the root word of technical. English speakers often pronounce it, like the first syllable of technical. Knuth instructs that it be typeset with the "E" [|below] the baseline and reduced spacing between the letters. This is done, as Knuth mentions in his TeXbook, to distinguish TeX from other system names such as TEX, the Text EXecutive processor. Fans like to proliferate names from the word "TeX"—such as TeXnician, TeXhacker, TeXmaster, TeXhax, and TeXnique.

Community

Notable entities in the TeX community include the TeX Users Group, which publishes TUGboat and The PracTeX Journal, covering a wide range of topics in digital typography relevant to TeX. The Deutschsprachige Anwendervereinigung TeX is a large user group in Germany. The TeX Users Group was founded in 1980 for educational and scientific purposes, provides an organization for those who have an interest in typography and font design, and are users of the TeX typesetting system invented by Knuth. The TeX Users Group represents the interests of TeX users worldwide. The TeX Users Group publishes the journal TUGboat three times per year; DANTE publishes Die TeXnische Komödie four times per year. Other user groups include DK-TUG in Denmark, :fr:GUTenberg|GUTenberg in France, GuIT in Italy, and UK-TUG in the United Kingdom; TUG maintains a complete list.

Extensions