Standard streams


In computer programming, standard streams are interconnected input and output communication channels between a computer program and its environment when it begins execution. The three input/output connections are called standard input, standard output and standard error. Originally I/O happened via a physically connected system console, but standard streams abstract this. When a command is executed via an interactive shell, the streams are typically connected to the text terminal on which the shell is running, but can be changed with redirection or a pipeline. More generally, a child process inherits the standard streams of its parent process.

Application

Users generally know standard streams as input and output channels that handle data coming from an input device, or that write data from the application. The data may be text with any encoding, or binary data.
In many modern systems, the standard error stream of a program is redirected into a log file, typically for error analysis purposes.
Streams may be used to chain applications, meaning that the output stream of one program can be redirected to be the input stream to another application. In many operating systems this is expressed by listing the application names, separated by the vertical bar character, for this reason often called the pipeline character. A well-known example is the use of a pagination application, such as more, providing the user control over the display of the output stream on the display.

Background

In most operating systems predating Unix, programs had to explicitly connect to the appropriate input and output devices. OS-specific intricacies caused this to be a tedious programming task. On many systems it was necessary to obtain control of environment settings, access a local file table, determine the intended data set, and handle hardware correctly in the case of a punch card reader, magnetic tape drive, disk drive, line printer, card punch, or interactive terminal.
One of Unix's several groundbreaking advances was abstract devices, which removed the need for a program to know or care what kind of devices it was communicating with. Older operating systems forced upon the programmer a record structure and frequently non-orthogonal data semantics and device control. Unix eliminated this complexity with the concept of a data stream: an ordered sequence of data bytes which can be read until the end of file. A program may also write bytes as desired and need not, and cannot easily declare their count or grouping.
Another Unix breakthrough was to automatically associate input and output to terminal keyboard and terminal display, respectively, by default — the program did absolutely nothing to establish input and output for a typical input-process-output program. In contrast, previous operating systems usually required some—often complex—job control language to establish connections, or the equivalent burden had to be orchestrated by the program.
Since Unix provided standard streams, the Unix C runtime environment was obliged to support it as well. As a result, most C runtime environments, regardless of the operating system, provide equivalent functionality.

Standard input (stdin)

Standard input is a stream from which a program reads its input data. The program requests data transfers by use of the read operation. Not all programs require stream input. For example, the dir and ls programs may take command-line arguments, but perform their operations without any stream data input.
Unless redirected, standard input is inherited from the parent process. In the case of an interactive shell, that is usually associated with the keyboard.
The file descriptor for standard input is 0 ; the POSIX definition is STDIN_FILENO; the corresponding C variable is FILE* stdin; similarly, the C++ variable is std::cin.

Standard output (stdout)

Standard output is a stream to which a program writes its output data. The program requests data transfer with the write operation. Not all programs generate output. For example, the file rename command is silent on success.
Unless redirected, standard output is inherited from the parent process. In the case of an interactive shell, that is usually the text terminal which initiated the program.
The file descriptor for standard output is 1 ; the POSIX definition is STDOUT_FILENO; the corresponding C variable is FILE* stdout; similarly, the C++ variable is std::cout.

Standard error (stderr)

Standard error is another output stream typically used by programs to output error messages or diagnostics. It is a stream independent of standard output and can be redirected separately. This solves the semi-predicate problem, allowing output and errors to be distinguished, and is analogous to a function returning a pair of values – see Semi-predicate problem: Multi valued return. The usual destination is the text terminal which started the program to provide the best chance of being seen even if standard output is redirected. For example, output of a program in a pipeline is redirected to input of the next program, but errors from each program still go directly to the text terminal.
It is acceptable and normal to direct standard output and standard error to the same destination, such as the text terminal. Messages appear in the same order as the program writes them, unless buffering is involved. For example, in common situations the standard error stream is unbuffered but the standard output stream is line-buffered; in this case, text written to standard error later may appear on the terminal earlier, if the standard output stream buffer is not yet full.
The file descriptor for standard error is defined by POSIX as 2 ; the header file provides the symbol STDERR_FILENO; the corresponding C variable is FILE* stderr. The C++ standard header provides two variables associated with this stream: std::cerr and std::clog, the former being unbuffered and the latter using the same buffering mechanism as all other C++ streams.
Bourne-style shells allow standard error to be redirected to the same destination that standard output is directed to using
2>&1
csh-style shells allow standard error to be redirected to the same destination that standard output is directed to using
>&
Standard error was added to Unix in the 1970s after several wasted phototypesetting runs ended with error messages being typeset instead of displayed on the user's terminal.

Timeline

1950s: Fortran

Fortran has the equivalent of Unix file descriptors: By convention, many Fortran implementations use unit numbers UNIT=5 for stdin, UNIT=6 for stdout and UNIT=0 for stderr. In Fortran-2003, the intrinsic ISO_FORTRAN_ENV module was standardized to include the named constants INPUT_UNIT, OUTPUT_UNIT, and ERROR_UNIT to portably specify the unit numbers.

! FORTRAN 77 example
PROGRAM MAIN
INTEGER NUMBER
READ NUMBER
WRITE ' NUMBER IS: ',NUMBER
END


! Fortran 2003 example
program main
use iso_fortran_env
implicit none
integer :: number
read number
write 'Number is: ', number
end program

1960: ALGOL 60

was criticized for having no standard file access.

1968: ALGOL 68

's input and output facilities were collectively referred to as the transput. Koster coordinated the definition of the transput standard. The model included three standard channels: stand in, stand out, and stand back.

1970s: C and Unix

In the C programming language, the standard input, output, and error streams are attached to the existing Unix file descriptors 0, 1 and 2 respectively. In a POSIX environment the definitions STDIN_FILENO, STDOUT_FILENO or STDERR_FILENO should be used instead rather than magic numbers. File pointers stdin, stdout, and stderr are also provided.
Ken Thompson modified sort in Version 5 Unix to accept "-" as representing standard input, which spread to other utilities and became a part of the operating system as a special file in Version 8. Diagnostics were part of standard output through Version 6, after which Dennis M. Ritchie created the concept of standard error.

1995: Java

In Java, the standard streams are referred to by , , and .

public static void main

2000s: .NET

In C# and other.NET languages, the standard streams are referred to by System.Console.In, System.Console.Out and System.Console.Error. Basic read and write capabilities for the stdin and stdout streams are also accessible directly through the class System.Console can be used instead of System.Console.Out.WriteLine).
System.Console.In, System.Console.Out and System.Console.Error are System.IO.TextReader and System.IO.TextWriter objects, which only allow access to the underlying standard streams on a text basis. Full binary access to the standard streams must be performed through the System.IO.Stream objects returned by System.Console.OpenStandardInput, System.Console.OpenStandardOutput and System.Console.OpenStandardError respectively.

// C# example
public static int Main


' Visual Basic.NET example
Public Function Main As Integer
Try
Dim s As String = System.Console..ReadLine
Dim number As Double = Double.Parse
System.Console.Out.WriteLine
Return 0
' If Parse threw an exception
Catch ex As System.ArgumentNullException
System.Console..WriteLine
Catch ex2 As System.FormatException
System.Console..WriteLine
Catch ex3 As System.OverflowException
System.Console..WriteLine
End Try
Return -1
End Function

When applying the System.Diagnostics.Process class one can use the instance properties StandardInput, StandardOutput, and StandardError of that class to access the standard streams of the process.

Python

Python provides file-like objects that represent stdin, stdout, and stderr. We can show how we could use these objects to work with the input and output of our program.

sys.stdin

Python’s sys module provides us with all three file objects for stdin, stdout, and stderr. For the input file object, we use sys.stdin. This is similar to a file, where you can open and close it, just like any other file.
here a basic example:
------------------------------------
program stdin.py
------------------------------------

import sys
stdin_fileno = sys.stdin
  1. Keeps reading from stdin and quits only if the word 'exit' is there
  2. This loop, by default does not terminate, since stdin is open
for line in stdin_fileno:
# Remove trailing newline characters using strip
if 'exit' line.strip:
print
exit
else:
print

------------------------------------
Usage:
printf "these \n are different \n words" | python stdin.py
Message from sys.stdin: ---> these <---
Message from sys.stdin: ---> are different <---
Message from sys.stdin: ---> words <---
The above snippet keeps reading input from stdin and prints the message to the Console until the word exit is encountered.
NOTE: We do not normally close the default stdin file object, although it is allowed. So stdin_fileno.close is valid Python code.
Now that we know a little bit about stdin, let us move to stdout.

sys.stdout

For the output file object, we use sys.stdout. It is similar to sys.stdin, but it directly displays anything written to it to the Console.
The below snippet shows that we get the Output to the Console if we write to sys.stdout.
------------------------------------
program stdout.py
------------------------------------

import sys
stdout_fileno = sys.stdout
sample_input =
for ip in sample_input:
# Prints to stdout
stdout_fileno.write

To execute:
python stdout.py
Output:
Hi
Hello from Python
exit

sys.stderr

This is similar to sys.stdout because it also prints directly to the Console. But the difference is that can be used to print Exceptions and Error messages and also info/debugging comments.. This can be very useful when the stdout is used to write data instead.
Here an example.
------------------------------------
program stderr.py
------------------------------------

import sys
stdout_fileno = sys.stdout
stderr_fileno = sys.stderr
sample_input =
for ip in sample_input:
# Prints to stdout
# Tries to add an Integer with string. Raises an exception
try:
stdout_fileno.write
# Catch exceptions
except:
stderr_fileno.write
stderr_fileno.write
try:
ip = ip + 100
# Catch all exceptions
except:
stderr_fileno.write

To execute:
python stderr.py
Output:

Hi

Exception Occurred! ip = Hi

Hello from Python

Exception Occurred! ip = Hello from Python

exit

Exception Occurred! ip = exit
As you can observe, for all the input strings, we try to add to an Integer, which will raise an Exception. We catch all such exceptions and print another debug message using sys.stderr.
------------------------------------
Redirection to a file
We can redirect the stdout and also stderr file handles to any other file. This may be useful if you want to log events to a file without using any other module such as Logging.
The below snippet redirects output to a file called Output.txt.
So, we will not see anything printed to the Console, because it is now being printed to the file itself! This is the essence of Output redirection. You ‘redirect’ the Output to some other place.

import sys
  1. Save the current stdout so that we can revert sys.stdout after we complete
  2. our redirection
stdout_fileno = sys.stdout
sample_input =
  1. Redirect sys.stdout to the file
sys.stdout = open
for ip in sample_input:
# Prints to the redirected stdout
sys.stdout.write
# Prints to the actual saved stdout handler
stdout_fileno.write
  1. Close the file
sys.stdout.close
  1. Restore sys.stdout to our old saved file handler
sys.stdout = stdout_fileno

----------------------------------------
Output:
root@ubuntu:~# python3 output_redirection.py
Hi
Hello from Python
exit
root@ubuntu:~# cat Output.txt
Hi
Hello from Python
exit
As you can see, we have printed the output to both the Console, as well as to Output.txt.
We first save the original sys.stdout file handler object to another Variable. We not only need this to restore sys.stdout to the old handler, but we can also print to the console using this Variable!
Note that after writing to the file, we close it, similar to how we close a file because that file was still open.
We finally restore the handler of sys.stdout to the Console, using the variable stdout_fileno.
A similar process can be followed for Input and Error redirection, replacing sys.stdout with sys.stdin or sys.stderr and working with Inputs and Exceptions instead of Output.
Here was described about using stdin, stdout and stderr in Python, using the sys module. And also how to manipulate the corresponding file handlers for redirection to/from a file.

GUIs

s not always make use of the standard streams, they do when GUIs are wrappers of underlying scripts and/or console programs, for instance the git-cola git GUI https://git-cola.github.io or the synaptic package manager GUI, that wraps apt commands in debian and/or ubuntu. Redirecting natively GUI programs not always is practical and useful. The GUIs created with scripting tools like zenity and kdialog by KDE project or instead make use of stdin and stdout and stderr, such kind of GUIs are more simple to implement and use also because of the scripting nature, instead of a complete GUI programmed and compiled in C/C++ using qt or gtk or other equivalent proprietary widget framework.
There is some analogy with the standard streams communication in pure GUI programs probably in cutting from text pads of one application and pasting into another, but since manual user operations are required, moving large numbers of pastes is not efficient, that's why the stdin/stdout communication is so important in pipeline scripts, that can be also commanded by GUI programs and ended by graphical visualisers programs. The Services menu, as implemented on NeXTSTEP and Mac OS X, is also analogous to standard streams. On these operating systems, graphical applications can provide functionality through a systemwide menu that operates on the current in the GUI, no matter in what application.
Some GUI programs, primarily on Unix, still write debug information to standard error. Others may read files from standard input. Popular Windows programs that open a separate console window in addition to their GUI windows are the emulators pSX and DOSBox.
GTK-server can use stdin as a communication interface with an interpreted program to realize a GUI.
The Common Lisp Interface Manager paradigm "presents" GUI elements sent to an extended output stream.