TCP congestion control
uses a network congestion-avoidance algorithm that includes various aspects of an additive increase/multiplicative decrease scheme, along with other schemes including slow start and congestion window, to achieve congestion avoidance. The TCP congestion-avoidance algorithm is the primary basis for congestion control in the Internet. Per the end-to-end principle, congestion control is largely a function of internet hosts, not the network itself. There are several variations and versions of the algorithm implemented in protocol stacks of operating systems of computers that connect to the Internet.
Operation
To avoid congestive collapse, TCP uses a multi-faceted congestion-control strategy. For each connection, TCP maintains a congestion window, limiting the total number of unacknowledged packets that may be in transit end-to-end. This is somewhat analogous to TCP's sliding window used for flow control. TCP uses a mechanism called slow start to increase the congestion window after a connection is initialized or after a timeout. It starts with a window, a small multiple of the maximum segment size in size. Although the initial rate is low, the rate of increase is very rapid; for every packet acknowledged, the congestion window increases by 1 MSS so that the congestion window effectively doubles for every round-trip time.When the congestion window exceeds the slow-start threshold, ssthresh, the algorithm enters a new state, called congestion avoidance. In congestion avoidance state, as long as non-duplicate ACKs are received the congestion window is additively increased by one MSS every round-trip time.
Congestion window
In TCP, the congestion window is one of the factors that determines the number of bytes that can be sent out at any time. The congestion window is maintained by the sender and is a means of stopping a link between the sender and the receiver from becoming overloaded with too much traffic. This should not to be confused with the sliding window maintained by the receiver which exists to prevent the receiver from becoming overloaded. The congestion window is calculated by estimating how much congestion there is on the link.When a connection is set up, the congestion window, a value maintained independently at each host, is set to a small multiple of the MSS allowed on that connection. Further variance in the congestion window is dictated by an AIMD approach. This means that if all segments are received and the acknowledgments reach the sender on time, some constant is added to the window size. When the window reaches ssthresh, the congestion window increases linearly at the rate of 1/ segment on each new acknowledgement received. The window keeps growing until a timeout occurs. On timeout:
- Congestion window is reset to 1 MSS.
- ssthresh is set to half the congestion window size before the timeout.
- slow start is initiated.
The flow of data over a TCP connection is also controlled by the use of the receive window advertised by the receiver. By comparing its own congestion window with the receive window, a sender can determine how much data it may send at any given time.
Slow start
Slow start is part of the congestion control strategy used by TCP in conjunction with other algorithms to avoid sending more data than the network is capable of forwarding, that is, to avoid causing network congestion. The algorithm is specified by RFC 5681.Although the strategy is referred to as slow start, its congestion window growth is quite aggressive, more aggressive than the congestion avoidance phase. Before slow start was introduced in TCP, the initial pre-congestion avoidance phase was even faster.
Slow start begins initially with a congestion window size of 1, 2, 4 or 10 MSS. The value for the congestion window size will be increased by one with each acknowledgement received, effectively doubling the window size each round-trip time. The transmission rate will be increased by the slow-start algorithm until either a loss is detected, or the receiver's advertised window is the limiting factor, or ssthresh is reached. If a loss event occurs, TCP assumes that it is due to network congestion and takes steps to reduce the offered load on the network. These measurements depend on the exact TCP congestion avoidance algorithm used.
; TCP Tahoe
; TCP Reno
Once ssthresh is reached, TCP changes from slow-start algorithm to the linear growth algorithm. At this point, the window is increased by 1 segment for each round-trip delay time.
Slow start assumes that unacknowledged segments are due to network congestion. While this is an acceptable assumption for many networks, segments may be lost for other reasons, such as poor data link layer transmission quality. Thus, slow start can perform poorly in situations with poor reception, such as wireless networks.
The slow start protocol also performs badly for short-lived connections. Older web browsers would create many consecutive short-lived connections to the web server, and would open and close the connection for each file requested. This kept most connections in the slow start mode, which resulted in poor response time. To avoid this problem, modern browsers either open multiple connections simultaneously or reuse one connection for all files requested from a particular web server. Connections, however, cannot be reused for the multiple third-party servers used by web sites to implement web advertising, sharing features of social networking services, and counter scripts of web analytics.
Additive increase/multiplicative decrease
The additive increase/multiplicative decrease algorithm is a closed-loop control algorithm. AIMD combines linear growth of the congestion window with an exponential reduction when a congestion takes place. Multiple flows using AIMD congestion control will eventually converge to use equal amounts of a contended link.Fast retransmit
Fast retransmit is an enhancement to TCP that reduces the time a sender waits before retransmitting a lost segment. A TCP sender normally uses a simple timer to recognize lost segments. If an acknowledgement is not received for a particular segment within a specified time, the sender will assume the segment was lost in the network, and will retransmit the segment.Duplicate acknowledgement is the basis for the fast retransmit mechanism. After receiving a packet an acknowledgement is sent for the last in order byte of data received. For an in order packet, this is effectively the last packet's sequence number plus its payload length. If the next packet in the sequence is lost but a third packet in the sequence is received, then the receiver can only acknowledge the last in-order byte of data, which is the same value as was acknowledged for the first packet. The second packet is lost and the third packet is not in order, so the last in order byte of data remains the same as before. Thus a Duplicate acknowledgement occurs. The sender continues to send packets, and a fourth and fifth packet are received by the receiver. Again, the second packet is missing from the sequence, so the last in order byte has not changed. Duplicate acknowledgements are sent to both these packets.
When a sender receives three duplicate acknowledgements, it can be reasonably confident that the segment carrying the data that followed the last in order byte specified in the acknowledgment was lost. A sender with fast retransmit will then retransmit this packet immediately without waiting for its timeout. On receipt of the re-transmitted segment, the receiver can acknowledge the last in order byte of data received. In the above example, this would acknowledge to the end of the payload of the fifth packet. There is no need to acknowledge intermediate packets.
Algorithms
The naming convention for congestion control algorithms may have originated in a 1996 paper by Kevin Fall and Sally Floyd.The following is one possible classification according to the following properties:
- the type and amount of feedback received from the network
- incremental deployability on the current Internet
- the aspect of performance it aims to improve: high bandwidth-delay product networks ; lossy links ; fairness ; advantage to short flows ; variable-rate links ; speed of convergence
- the fairness criterion it uses
| Variant | Feedback | Required changes | Benefits | Fairness |
| Reno | Loss | Delay | ||
| Vegas | Delay | Sender | Less loss | Proportional |
| High Speed | Loss | Sender | High bandwidth | |
| BIC | Loss | Sender | High bandwidth | |
| CUBIC | Loss | Sender | High bandwidth | |
| C2TCP | Loss/Delay | Sender | Ultra-low latency and high bandwidth | |
| NATCP | Multi-bit signal | Sender | Near Optimal Performance | |
| Elastic-TCP | Loss/Delay | Sender | High bandwidth/short & long-distance | |
| Agile-TCP | Loss | Sender | High bandwidth/short-distance | |
| H-TCP | Loss | Sender | High bandwidth | |
| FAST | Delay | Sender | High bandwidth | Proportional |
| Compound TCP | Loss/Delay | Sender | High bandwidth | Proportional |
| Westwood | Loss/Delay | Sender | L | |
| Jersey | Loss/Delay | Sender | L | |
| BBR | Delay | Sender | BLVC, Bufferbloat | |
| CLAMP | Multi-bit signal | Receiver, Router | V | Max-min |
| TFRC | Loss | Sender, Receiver | No Retransmission | Minimum delay |
| XCP | Multi-bit signal | Sender, Receiver, Router | BLFC | Max-min |
| VCP | 2-bit signal | Sender, Receiver, Router | BLF | Proportional |
| MaxNet | Multi-bit signal | Sender, Receiver, Router | BLFSC | Max-min |
| JetMax | Multi-bit signal | Sender, Receiver, Router | High bandwidth | Max-min |
| RED | Loss | Router | Reduced delay | |
| ECN | Single-bit signal | Sender, Receiver, Router | Reduced loss |
TCP Tahoe and Reno
TCP Tahoe and Reno algorithms were retrospectively named after the versions or flavours of the 4.3BSD operating system in which each first appeared. The Tahoe algorithm first appeared in 4.3BSD-Tahoe, and was later made available to non-AT&T licensees as part of the 4.3BSD Networking Release 1; this ensured its wide distribution and implementation. Improvements were made in 4.3BSD-Reno and subsequently released to the public as Networking Release 2 and later 4.4BSD-Lite.While both consider retransmission timeout and duplicate ACKs as packet loss events, the behavior of Tahoe and Reno differ primarily in how they react to duplicate ACKs:
- Tahoe: if three duplicate ACKs are received, Tahoe performs a fast retransmit, sets the slow start threshold to half of the current congestion window, reduces the congestion window to 1 MSS, and resets to slow start state.
- Reno: if three duplicate ACKs are received, Reno will perform a fast retransmit and skip the slow start phase by instead halving the congestion window, setting the slow start threshold equal to the new congestion window, and enter a phase called fast recovery.
TCP Vegas
Until the mid-1990s, all of TCP's set timeouts and measured round-trip delays were based upon only the last transmitted packet in the transmit buffer. University of Arizona researchers Larry Peterson and Lawrence Brakmo introduced TCP Vegas in which timeouts were set and round-trip delays were measured for every packet in the transmit buffer. In addition, TCP Vegas uses additive increases in the congestion window. In a comparison study of various TCP congestion control algorithms, TCP Vegas appeared to be the smoothest followed by TCP CUBIC.TCP Vegas was not widely deployed outside Peterson's laboratory but was selected as the default congestion control method for DD-WRT firmware v24 SP2.
TCP New Reno
TCP New Reno, defined by , improves retransmission during the fast-recovery phase of TCP Reno. During fast recovery, to keep the transmit window full, for every duplicate ACK that is returned, a new unsent packet from the end of the congestion window is sent. For every ACK that makes partial progress in the sequence space, the sender assumes that the ACK points to a new hole, and the next packet beyond the ACKed sequence number is sent.Because the timeout is reset whenever there is progress in the transmit buffer, New Reno can fill large holes, or multiple holes, in the sequence spacemuch like TCP SACK. Because New Reno can send new packets at the end of the congestion window during fast recovery, high throughput is maintained during the hole-filling process, even when there are multiple holes, of multiple packets each. When TCP enters fast recovery it records the highest outstanding unacknowledged packet sequence number. When this sequence number is acknowledged, TCP returns to the congestion avoidance state.
A problem occurs with New Reno when there are no packet losses but instead, packets are reordered by more than 3 packet sequence numbers. In this case, New Reno mistakenly enters fast recovery. When the reordered packet is delivered, ACK sequence-number progress occurs and from there until the end of fast recovery, all sequence-number progress produces a duplicate and needless retransmission that is immediately ACKed.
New Reno performs as well as SACK at low packet error rates, and substantially outperforms Reno at high error rates.
TCP Hybla
TCP Hybla aims to eliminate penalties to TCP connections that incorporate a high-latency terrestrial or satellite radio links. Hybla improvements are based on analytical evaluation of the congestion window dynamics.TCP BIC
Binary Increase Congestion control is a TCP implementation with an optimized CCA for high speed networks with high latency, known as long fat networks. BIC is used by default in Linux kernels 2.6.8 through 2.6.18.TCP CUBIC
CUBIC is a less aggressive and more systematic derivative of BIC, in which the window is a cubic function of time since the last congestion event, with the inflection point set to the window prior to the event. CUBIC is used by default in Linux kernels between versions 2.6.19 and 3.2.Agile-SD TCP
Agile-SD is a Linux-based CCA which is designed for the real Linux kernel. It is a receiver-side algorithm that employs a loss-based approach using a novel mechanism, called agility factor. to increase the bandwidth utilization over high-speed and short-distance networks such as local area networks or fiber-optic network, especially when the applied buffer size is small. It has been evaluated by comparing its performance to Compound-TCP and CUBIC using NS-2 simulator. It improves the total performance up to 55% in term of average throughput.TCP Westwood+
Westwood+ is a sender-side only modification of the TCP Reno protocol stack that optimizes the performance of TCP congestion control over both wireline and wireless networks. TCP Westwood+ is based on end-to-end bandwidth estimation to set congestion window and slow start threshold after a congestion episode, that is, after three duplicate acknowledgments or a timeout. The bandwidth is estimated by properly low-pass filtering the rate of returning acknowledgment packets. The rationale of this strategy is simple: in contrast with TCP Reno, which blindly halves the congestion window after three duplicate ACKs, TCP Westwood+ adaptively sets a slow start threshold and a congestion window which takes into account the bandwidth used at the time congestion is experienced. TCP Westwood+ significantly increases throughput over wireless links and fairness compared to TCP Reno/New Reno in wired networks.Compound TCP
Compound TCP is a Microsoft implementation of TCP which maintains two different congestion windows simultaneously, with the goal of achieving good performance on LFNs while not impairing fairness. It has been widely deployed in Windows versions since Microsoft Windows Vista and Windows Server 2008 and has been ported to older Microsoft Windows versions as well as Linux.TCP Proportional Rate Reduction
TCP Proportional Rate Reduction is an algorithm designed to improve the accuracy of data sent during recovery. The algorithm ensures that the window size after recovery is as close as possible to the slow start threshold. In tests performed by Google, PRR resulted in a 3–10% reduction in average latency and recovery timeouts reduced by 5%. PRR is available in Linux kernels since version 3.2.TCP BBR
Bottleneck Bandwidth and Round-trip propagation time is a CCA developed at Google in 2016. While most congestion control algorithms are loss-based, in that they rely on packet loss as a signal to lower rates of transmission, BBR, like Vegas, is model-based. The algorithm uses the maximum bandwidth and round-trip time at which the network delivered the most recent flight of outbound data packets to build an explicit model of the network. Each cumulative or selective acknowledgment of packet delivery produces a rate sample which records the amount of data delivered over the time interval between the transmission of a data packet and the acknowledgment of that packet. As network interface controllers evolve from megabit per second to gigabit per second performance, latency associated with bufferbloat instead of packet loss becomes a more reliable marker of the maximum throughput, making latency/model-based congestion control algorithms which provide higher throughput and lower latency, such as BBR, a more reliable alternative to more popular loss-based algorithms like CUBIC.When implemented within YouTube, BBR yielded an average of 4% higher network throughput and up to 14% in some countries. BBR is also available for QUIC. It is available for Linux TCP since Linux 4.9.
BBR is efficient and fast, but its fairness to non-BBR streams is disputed. While Google's presentation shows BBR co-existing well with CUBIC, researchers like Geoff Huston and Hock, Bless and Zitterbart finds it unfair to other streams and not scalable. Hock et al also found "some severe inherent issues such as increased queuing delays, unfairness, and massive packet loss" in the BBR implementation of Linux 4.9.
Soheil Abbasloo et al. show that BBR doesn't perform well in dynamic environments such as cellular networks. They have also shown that BBR has an unfairness issue. For instance, when a CUBIC flow coexists with a BBR flow in the network, the BBR flow can dominate the CUBIC flow and get the whole link bandwidth from it.
C2TCP
Cellular Controlled Delay TCP was motivated by the lack of a flexible end-to-end TCP approach that can satisfy various QoS requirements of different applications without requiring any changes in the network devices. C2TCP aims to satisfy ultra-low latency and high bandwidth requirements of applications such as virtual reality, video conferencing, online gaming, vehicular communication systems, etc. in a highly dynamic environment such as current LTE and future 5G cellular networks. C2TCP works as an add-on on top of loss-based TCP and makes the average delay of packets bounded to the desired delays set by the applications.Researchers at NYU showed that C2TCP outperforms the delay/Jitter performance of various state-of-the-art TCP schemes. For instance, they showed that compared to BBR, CUBIC, and Westwood on average, C2TCP decreases the average delay of packets by about 250%, 900%, and 700% respectively on various cellular network environments.
C2TCP is only required to be installed on the server-side.
Elastic-TCP
Elastic-TCP has been proposed in February 2019 by Mohamed A. Alrshah et al. to increase the bandwidth utilization over high-BDP networks to support recent applications such as cloud computing, big data transfer, IoT, etc. It is a Linux-based CCA which is designed for the Linux kernel. It is a receiver-side algorithm that employs a Loss-delay-based approach using a novel mechanism, called Window-correlated Weighting Function. It has a high level of elasticity to deal with different network characteristics without the need for human tuning. It has been evaluated by comparing its performance to Compound-TCP, CUBIC and TCP-BBR using NS-2 simulator and testbed. Elastic-TCP significantly improves the total performance in term of average throughput, loss ratio, and delay.NATCP/NACubic
Recently, Soheil Abbasloo et. al. proposed NATCP a controversial TCP design targeting Mobile Edge networks such as MEC. The key idea of NATCP is that if the characteristics of the network were known beforehand, TCP would have been designed in a better way. Therefore, NATCP employs the available features and properties in the current MEC-based cellular architectures to push the performance of TCP close to the optimal performance. NATCP uses an out-of-band feedback from the network to the servers located nearby. The feedback from the network, which includes the capacity of cellular access link and the minimum RTT of the network, guides the servers to adjust their sending rates. As preliminary results show, NATCP outperforms the state-of-the-art TCP schemes by at least achieving 2x higher Power. NATCP replaces the traditional TCP scheme at the sender.To deal with backward compatibility issue, they proposed another version called NACubic. NACubic is a backward compatible design, requiring no change in TCP on the connected nodes. NACubic employs the received feedback and enforces a cap on the congestion window and the pacing rate as required.
Other TCP congestion avoidance algorithms
- FAST TCP
- Generalized FAST TCP
- H-TCP
- Data Center TCP
- High Speed TCP
- HSTCP-LP
- TCP-Illinois
- TCP-LP
- TCP SACK
- Scalable TCP
- TCP Veno
- Westwood
- XCP
- YeAH-TCP
- TCP-FIT
- Congestion Avoidance with Normalized Interval of Time
- Non-linear neural network congestion control based on genetic algorithm for TCP/IP networks
When the per-flow product of bandwidth and latency increases, regardless of the queuing scheme, TCP becomes inefficient and prone to instability. This becomes increasingly important as the Internet evolves to incorporate very high-bandwidth optical links.
TCP Interactive allows applications to subscribe to TCP events and respond accordingly enabling various functional extensions to TCP from outside TCP layer. Most TCP congestion schemes work internally. iTCP additionally enables advanced applications to directly participate in congestion control such as to control the source generation rate.
Zeta-TCP detects the congestions from both the latency and loss rate measures, and applies different congestion window backoff strategies based on the likelihood of the congestions to maximize the goodput. It also has a couple of other improvements to accurately detect the packet losses, avoiding retransmission timeout retransmission; and accelerate/control the inbound traffic.
Classification by network awareness
Congestion control algorithms are classified in relation to network awareness, meaning the extent to which these algorithms are aware of the state of the network, and consist of three primary categories: black box, grey box, and green box.Black box algorithms offer blind methods of congestion control. They operate only on the binary feedback received upon congestion and do not assume any knowledge concerning the state of the networks which they manage.
Grey box algorithms use time-instances in order to obtain measurements and estimations of bandwidth, flow contention, and other knowledge of network conditions.
Green box algorithms offer bimodal methods of congestion control which measures the fair-share of total bandwidth which should be allocated for each flow, at any point, during the system's execution.
Black box
- Highspeed-TCP
- BIC TCP uses a concave increase of the sources rate after each congestion event until the window is equal to that before the event, in order to maximise the time that the network is fully utilised. After that, it probes aggressively.
- CUBIC TCP – a less aggressive and more systematic derivative of BIC, in which the window is a cubic function of time since the last congestion event, with the inflection point set to the window prior to the event.
- AIMD-FC, an improvement of AIMD.
- Binomial Mechanisms
- SIMD Protocol
- GAIMD
Grey box
- TCP Vegas – estimates the queuing delay, and linearly increases or decreases the window so that a constant number of packets per flow are queued in the network. Vegas implements proportional fairness.
- FAST TCP – achieves the same equilibrium as Vegas, but uses proportional control instead of linear increase, and intentionally scales the gain down as the bandwidth increases with the aim of ensuring stability.
- TCP BBR – estimates the queuing delay, but uses exponential increase. Intentionally slows down periodically for fairness and decreased delay.
- TCP-Westwood – a loss causes the window to be reset to the sender's estimate of the bandwidth-delay product, which is the smallest measured RTT times the observed rate of receiving ACKs.
- C2TCP
- TFRC
- TCP-Real
- TCP-Jersey
Green box
- Bimodal Mechanism – Bimodal Congestion Avoidance and Control mechanism.
- Signalling methods implemented by routers
- * Random Early Detection randomly drops packets in proportion to the router's queue size, triggering multiplicative decrease in some flows.
- * Explicit Congestion Notification
- Network-Assisted Congestion Control
- * NATCP - Network-Assisted TCP uses out-of-band explicit feedback indicating minimum RTT of the network and capacity of the cellular access link.
- *VCP – The variable-structure congestion control protocol uses two ECN bits to explicitly feedback the network state of congestion. It includes an end host side algorithm as well.
- Explicit Control Protocol – XCP routers signal explicit increase and decreases in the senders' congestion windows.
- MaxNet – MaxNet uses a single header field, which carries the maximum congestion level of any router on a flow's path. The rate is set as a function of this maximum congestion, resulting in max-min fairness.
- JetMax – JetMax, like MaxNet, also responds only to the maximum congestion signal, but also carries other overhead fields
Usage
- BIC is used by default in Linux kernels 2.6.8 through 2.6.18.
- CUBIC is used by default in Linux kernels since version 2.6.19.
- PRR is incorporated in Linux kernels to improve loss recovery since version 3.2.
- BBR is incorporated in Linux kernels to enable model-based congestion control since version 4.9.