TCP NewReno: The NewReno Modification to TCP's Fast Recovery Algorithm
How a TCP sender without SACK repairs several lost packets in one window without timing out or shrinking its window again and again.
Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 6582, DOI 10.17487/RFC6582, April 2012, <https://www.rfc-editor.org/info/rfc6582>.
@misc{rfc6582,
series = {Request for Comments},
number = 6582,
howpublished = {RFC 6582},
publisher = {RFC Editor},
doi = {10.17487/RFC6582},
url = {https://www.rfc-editor.org/info/rfc6582},
author = {T. Henderson and S. Floyd and A. Gurtov and Y. Nishida},
title = {{The NewReno Modification to TCP's Fast Recovery Algorithm}},
pagetotal = 16,
year = 2012,
month = apr,
} RFC text
Contents
Internet Engineering Task Force (IETF) T. Henderson
Request for Comments: 6582 Boeing
Obsoletes: 3782 S. Floyd
Category: Standards Track ICSI
ISSN: 2070-1721 A. Gurtov
University of Oulu
Y. Nishida
WIDE Project
April 2012
The NewReno Modification to TCP's Fast Recovery Algorithm
Abstract
RFC 5681 documents the following four intertwined TCP congestion
control algorithms: slow start, congestion avoidance, fast
retransmit, and fast recovery. RFC 5681 explicitly allows certain
modifications of these algorithms, including modifications that use
the TCP Selective Acknowledgment (SACK) option (RFC 2883), and
modifications that respond to "partial acknowledgments" (ACKs that
cover new data, but not all the data outstanding when loss was
detected) in the absence of SACK. This document describes a specific
algorithm for responding to partial acknowledgments, referred to as
"NewReno". This response to partial acknowledgments was first
proposed by Janey Hoe. This document obsoletes RFC 3782.
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 5741.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc6582.
Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
1. Introduction
For the typical implementation of the TCP fast recovery algorithm
described in [RFC5681] (first implemented in the 1990 BSD Reno
release, and referred to as the "Reno algorithm" in [FF96]), the TCP
data sender only retransmits a packet after a retransmit timeout has
occurred, or after three duplicate acknowledgments have arrived
triggering the fast retransmit algorithm. A single retransmit
timeout might result in the retransmission of several data packets,
but each invocation of the fast retransmit algorithm in RFC 5681
leads to the retransmission of only a single data packet.
Two problems arise with Reno TCP when multiple packet losses occur in
a single window. First, Reno will often take a timeout, as has been
documented in [Hoe95]. Second, even if a retransmission timeout is
avoided, multiple fast retransmits and window reductions can occur,
as documented in [F94]. When multiple packet losses occur, if the
SACK option [RFC2883] is available, the TCP sender has the
information to make intelligent decisions about which packets to
retransmit and which packets not to retransmit during fast recovery.
This document applies to TCP connections that are unable to use the
TCP Selective Acknowledgment (SACK) option, either because the option
is not locally supported or because the TCP peer did not indicate a
willingness to use SACK.
In the absence of SACK, there is little information available to the
TCP sender in making retransmission decisions during fast recovery.
From the three duplicate acknowledgments, the sender infers a packet
loss, and retransmits the indicated packet. After this, the data
sender could receive additional duplicate acknowledgments, as the
data receiver acknowledges additional data packets that were already
in flight when the sender entered fast retransmit.
In the case of multiple packets dropped from a single window of data,
the first new information available to the sender comes when the
sender receives an acknowledgment for the retransmitted packet (that
is, the packet retransmitted when fast retransmit was first entered).
If there is a single packet drop and no reordering, then the
acknowledgment for this packet will acknowledge all of the packets
transmitted before fast retransmit was entered. However, if there
are multiple packet drops, then the acknowledgment for the
retransmitted packet will acknowledge some but not all of the packets
transmitted before the fast retransmit. We call this acknowledgment
a partial acknowledgment.
Along with several other suggestions, [Hoe95] suggested that during
fast recovery the TCP data sender respond to a partial acknowledgment
by inferring that the next in-sequence packet has been lost and
retransmitting that packet. This document describes a modification
to the fast recovery algorithm in RFC 5681 that incorporates a
response to partial acknowledgments received during fast recovery.
We call this modified fast recovery algorithm NewReno, because it is
a slight but significant variation of the behavior that has been
historically referred to as Reno. This document does not discuss the
other suggestions in [Hoe95] and [Hoe96], such as a change to the
ssthresh parameter during slow start, or the proposal to send a new
packet for every two duplicate acknowledgments during fast recovery.
The version of NewReno in this document also draws on other
discussions of NewReno in the literature [LM97] [Hen98].
We do not claim that the NewReno version of fast recovery described
here is an optimal modification of fast recovery for responding to
partial acknowledgments, for TCP connections that are unable to use
SACK. Based on our experiences with the NewReno modification in the
network simulator known as ns-2 [NS] and with numerous
implementations of NewReno, we believe that this modification
improves the performance of the fast retransmit and fast recovery
algorithms in a wide variety of scenarios. Previous versions of this
RFC [RFC2582] [RFC3782] provide simulation-based evidence of the
possible performance gains.
2. Terminology and Definitions
This document assumes that the reader is familiar with the terms
SENDER MAXIMUM SEGMENT SIZE (SMSS), CONGESTION WINDOW (cwnd), and
FLIGHT SIZE (FlightSize) defined in [RFC5681].
This document defines an additional sender-side state variable called
"recover":
recover:
When in fast recovery, this variable records the send sequence
number that must be acknowledged before the fast recovery
procedure is declared to be over.
3. The Fast Retransmit and Fast Recovery Algorithms in NewReno
3.1. Protocol Overview
The basic idea of these extensions to the fast retransmit and fast
recovery algorithms described in Section 3.2 of [RFC5681] is as
follows. The TCP sender can infer, from the arrival of duplicate
acknowledgments, whether multiple losses in the same window of data
have most likely occurred, and avoid taking a retransmit timeout or
making multiple congestion window reductions due to such an event.
The NewReno modification applies to the fast recovery procedure that
begins when three duplicate ACKs are received and ends when either a
retransmission timeout occurs or an ACK arrives that acknowledges all
of the data up to and including the data that was outstanding when
the fast recovery procedure began.
3.2. Specification
The procedures specified in Section 3.2 of [RFC5681] are followed,
with the modifications listed below. Note that this specification
avoids the use of the key words defined in RFC 2119 [RFC2119], since
it mainly provides sender-side implementation guidance for
performance improvement, and does not affect interoperability.
1) Initialization of TCP protocol control block:
When the TCP protocol control block is initialized, recover is
set to the initial send sequence number.
2) Three duplicate ACKs:
When the third duplicate ACK is received, the TCP sender first
checks the value of recover to see if the Cumulative
Acknowledgment field covers more than recover. If so, the value
of recover is incremented to the value of the highest sequence
number transmitted by the TCP so far. The TCP then enters fast
retransmit (step 2 of Section 3.2 of [RFC5681]). If not, the TCP
does not enter fast retransmit and does not reset ssthresh.
3) Response to newly acknowledged data:
Step 6 of [RFC5681] specifies the response to the next ACK that
acknowledges previously unacknowledged data. When an ACK arrives
that acknowledges new data, this ACK could be the acknowledgment
elicited by the initial retransmission from fast retransmit, or
elicited by a later retransmission. There are two cases:
Full acknowledgments:
If this ACK acknowledges all of the data up to and including
recover, then the ACK acknowledges all the intermediate segments
sent between the original transmission of the lost segment and
the receipt of the third duplicate ACK. Set cwnd to either (1)
min (ssthresh, max(FlightSize, SMSS) + SMSS) or (2) ssthresh,
where ssthresh is the value set when fast retransmit was entered,
and where FlightSize in (1) is the amount of data presently
outstanding. This is termed "deflating" the window. If the
second option is selected, the implementation is encouraged to
take measures to avoid a possible burst of data, in case the
amount of data outstanding in the network is much less than the
new congestion window allows. A simple mechanism is to limit the
number of data packets that can be sent in response to a single
acknowledgment. Exit the fast recovery procedure.
Partial acknowledgments:
If this ACK does *not* acknowledge all of the data up to and
including recover, then this is a partial ACK. In this case,
retransmit the first unacknowledged segment. Deflate the
congestion window by the amount of new data acknowledged by the
Cumulative Acknowledgment field. If the partial ACK acknowledges
at least one SMSS of new data, then add back SMSS bytes to the
congestion window. This artificially inflates the congestion
window in order to reflect the additional segment that has left
the network. Send a new segment if permitted by the new value of
cwnd. This "partial window deflation" attempts to ensure that,
when fast recovery eventually ends, approximately ssthresh amount
of data will be outstanding in the network. Do not exit the fast
recovery procedure (i.e., if any duplicate ACKs subsequently
arrive, execute step 4 of Section 3.2 of [RFC5681]).
For the first partial ACK that arrives during fast recovery, also
reset the retransmit timer. Timer management is discussed in
more detail in Section 4.
4) Retransmit timeouts:
After a retransmit timeout, record the highest sequence number
transmitted in the variable recover, and exit the fast recovery
procedure if applicable.
Step 2 above specifies a check that the Cumulative Acknowledgment
field covers more than recover. Because the acknowledgment field
contains the sequence number that the sender next expects to receive,
the acknowledgment "ack_number" covers more than recover when
ack_number - 1 > recover;
i.e., at least one byte more of data is acknowledged beyond the
highest byte that was outstanding when fast retransmit was last
entered.
Note that in step 3 above, the congestion window is deflated after a
partial acknowledgment is received. The congestion window was likely
to have been inflated considerably when the partial acknowledgment
was received. In addition, depending on the original pattern of
packet losses, the partial acknowledgment might acknowledge nearly a
window of data. In this case, if the congestion window was not
deflated, the data sender might be able to send nearly a window of
data back-to-back.
This document does not specify the sender's response to duplicate
ACKs when the fast retransmit/fast recovery algorithm is not invoked.
This is addressed in other documents, such as those describing the
Limited Transmit procedure [RFC3042]. This document also does not
address issues of adjusting the duplicate acknowledgment threshold,
but assumes the threshold specified in the IETF standards; the
current standard is [RFC5681], which specifies a threshold of three
duplicate acknowledgments.
As a final note, we would observe that in the absence of the SACK
option, the data sender is working from limited information. When
the issue of recovery from multiple dropped packets from a single
window of data is of particular importance, the best alternative
would be to use the SACK option.
4. Handling Duplicate Acknowledgments after a Timeout
After each retransmit timeout, the highest sequence number
transmitted so far is recorded in the variable recover. If, after a
retransmit timeout, the TCP data sender retransmits three consecutive
packets that have already been received by the data receiver, then
the TCP data sender will receive three duplicate acknowledgments that
do not cover more than recover. In this case, the duplicate
acknowledgments are not an indication of a new instance of
congestion. They are simply an indication that the sender has
unnecessarily retransmitted at least three packets.
However, when a retransmitted packet is itself dropped, the sender
can also receive three duplicate acknowledgments that do not cover
more than recover. In this case, the sender would have been better
off if it had initiated fast retransmit. For a TCP sender that
implements the algorithm specified in Section 3.2 of this document,
the sender does not infer a packet drop from duplicate
acknowledgments in this scenario. As always, the retransmit timer is
the backup mechanism for inferring packet loss in this case.
There are several heuristics, based on timestamps or on the amount of
advancement of the Cumulative Acknowledgment field, that allow the
sender to distinguish, in some cases, between three duplicate
acknowledgments following a retransmitted packet that was dropped,
and three duplicate acknowledgments from the unnecessary
retransmission of three packets [Gur03] [GF04]. The TCP sender may
use such a heuristic to decide to invoke a fast retransmit in some
cases, even when the three duplicate acknowledgments do not cover
more than recover.
For example, when three duplicate acknowledgments are caused by the
unnecessary retransmission of three packets, this is likely to be
accompanied by the Cumulative Acknowledgment field advancing by at
least four segments. Similarly, a heuristic based on timestamps uses
the fact that when there is a hole in the sequence space, the
timestamp echoed in the duplicate acknowledgment is the timestamp of
the most recent data packet that advanced the Cumulative
Acknowledgment field [RFC1323]. If timestamps are used, and the
sender stores the timestamp of the last acknowledged segment, then
the timestamp echoed by duplicate acknowledgments can be used to
distinguish between a retransmitted packet that was dropped and three
duplicate acknowledgments from the unnecessary retransmission of
three packets.
4.1. ACK Heuristic
If the ACK-based heuristic is used, then following the advancement of
the Cumulative Acknowledgment field, the sender stores the value of
the previous cumulative acknowledgment as prev_highest_ack, and
stores the latest cumulative ACK as highest_ack. In addition, the
following check is performed if, in step 2 of Section 3.2, the
Cumulative Acknowledgment field does not cover more than recover.
2*) If the Cumulative Acknowledgment field didn't cover more than
recover, check to see if the congestion window is greater than
SMSS bytes and the difference between highest_ack and
prev_highest_ack is at most 4*SMSS bytes. If true, duplicate
ACKs indicate a lost segment (enter fast retransmit).
Otherwise, duplicate ACKs likely result from unnecessary
retransmissions (do not enter fast retransmit).
The congestion window check serves to protect against fast retransmit
immediately after a retransmit timeout.
If several ACKs are lost, the sender can see a jump in the cumulative
ACK of more than three segments, and the heuristic can fail.
[RFC5681] recommends that a receiver should send duplicate ACKs for
every out-of-order data packet, such as a data packet received during
fast recovery. The ACK heuristic is more likely to fail if the
receiver does not follow this advice, because then a smaller number
of ACK losses are needed to produce a sufficient jump in the
cumulative ACK.
4.2. Timestamp Heuristic
If this heuristic is used, the sender stores the timestamp of the
last acknowledged segment. In addition, the last sentence of step 2
in Section 3.2 of this document is replaced as follows:
2**) If the Cumulative Acknowledgment field didn't cover more than
recover, check to see if the echoed timestamp in the last
non-duplicate acknowledgment equals the stored timestamp. If
true, duplicate ACKs indicate a lost segment (enter fast
retransmit). Otherwise, duplicate ACKs likely result from
unnecessary retransmissions (do not enter fast retransmit).
The timestamp heuristic works correctly, both when the receiver
echoes timestamps, as specified by [RFC1323], and by its revision
attempts. However, if the receiver arbitrarily echoes timestamps,
the heuristic can fail. The heuristic can also fail if a timeout was
spurious and returning ACKs are not from retransmitted segments.
This can be prevented by detection algorithms such as the Eifel
detection algorithm [RFC3522].
5. Implementation Issues for the Data Receiver
[RFC5681] specifies that "Out-of-order data segments SHOULD be
acknowledged immediately, in order to accelerate loss recovery".
Neal Cardwell has noted that some data receivers do not send an
immediate acknowledgment when they send a partial acknowledgment, but
instead wait first for their delayed acknowledgment timer to expire
[C98]. As [C98] notes, this severely limits the potential benefit of
NewReno by delaying the receipt of the partial acknowledgment at the
data sender. Echoing [RFC5681], our recommendation is that the data
receiver send an immediate acknowledgment for an out-of-order
segment, even when that out-of-order segment fills a hole in the
buffer.
6. Implementation Issues for the Data Sender
In Section 3.2, step 3 above, it is noted that implementations should
take measures to avoid a possible burst of data when leaving fast
recovery, in case the amount of new data that the sender is eligible
to send due to the new value of the congestion window is large. This
can arise during NewReno when ACKs are lost or treated as pure window
updates, thereby causing the sender to underestimate the number of
new segments that can be sent during the recovery procedure.
Specifically, bursts can occur when the FlightSize is much less than
the new congestion window when exiting from fast recovery. One
simple mechanism to avoid a burst of data when leaving fast recovery
is to limit the number of data packets that can be sent in response
to a single acknowledgment. (This is known as "maxburst_" in ns-2
[NS].) Other possible mechanisms for avoiding bursts include rate-
based pacing, or setting the slow start threshold to the resultant
congestion window and then resetting the congestion window to
FlightSize. A recommendation on the general mechanism to avoid
excessively bursty sending patterns is outside the scope of this
document.
An implementation may want to use a separate flag to record whether
or not it is presently in the fast recovery procedure. The use of
the value of the duplicate acknowledgment counter for this purpose is
not reliable, because it can be reset upon window updates and out-of-
order acknowledgments.
When updating the Cumulative Acknowledgment field outside of fast
recovery, the state variable recover may also need to be updated in
order to continue to permit possible entry into fast recovery
(Section 3.2, step 2). This issue arises when an update of the
Cumulative Acknowledgment field results in a sequence wraparound that
affects the ordering between the Cumulative Acknowledgment field and
the state variable recover. Entry into fast recovery is only
possible when the Cumulative Acknowledgment field covers more than
the state variable recover.
It is important for the sender to respond correctly to duplicate ACKs
received when the sender is no longer in fast recovery (e.g., because
of a retransmit timeout). The Limited Transmit procedure [RFC3042]
describes possible responses to the first and second duplicate
acknowledgments. When three or more duplicate acknowledgments are
received, the Cumulative Acknowledgment field doesn't cover more than
recover, and a new fast recovery is not invoked, the sender should
follow the guidance in Section 4. Otherwise, the sender could end up
in a chain of spurious timeouts. We mention this only because
several NewReno implementations had this bug, including the
implementation in ns-2 [NS].
It has been observed that some TCP implementations enter a slow start
or congestion avoidance window updating algorithm immediately after
the cwnd is set by the equation found in Section 3.2, step 3, even
without a new external event generating the cwnd change. Note that
after cwnd is set based on the procedure for exiting fast recovery
(Section 3.2, step 3), cwnd should not be updated until a further
event occurs (e.g., arrival of an ack, or timeout) after this
adjustment.
7. Security Considerations
[RFC5681] discusses general security considerations concerning TCP
congestion control. This document describes a specific algorithm
that conforms with the congestion control requirements of [RFC5681],
and so those considerations apply to this algorithm, too. There are
no known additional security concerns for this specific algorithm.
8. Conclusions
This document specifies the NewReno fast retransmit and fast recovery
algorithms for TCP. This NewReno modification to TCP can even be
important for TCP implementations that support the SACK option,
because the SACK option can only be used for TCP connections when
both TCP end-nodes support the SACK option. NewReno performs better
than Reno in a number of scenarios discussed in previous versions of
this RFC ([RFC2582] [RFC3782]).
A number of options for the basic algorithms presented in Section 3
are also referenced in Appendix A of this document. These include
the handling of the retransmission timer, the response to partial
acknowledgments, and whether or not the sender must maintain a state
variable called recover. Our belief is that the differences between
these variants of NewReno are small compared to the differences
between Reno and NewReno. That is, the important thing is to
implement NewReno instead of Reno for a TCP connection without SACK;
it is less important exactly which variant of NewReno is implemented.
9. Acknowledgments
Many thanks to Anil Agarwal, Mark Allman, Armando Caro, Jeffrey Hsu,
Vern Paxson, Kacheong Poon, Keyur Shah, and Bernie Volz for detailed
feedback on the precursor RFCs 2582 and 3782. Jeffrey Hsu provided
clarifications on the handling of the variable recover; these
clarifications were applied to RFC 3782 via an erratum and are
incorporated into the text of Section 6 of this document. Yoshifumi
Nishida contributed a modification to the fast recovery algorithm to
account for the case in which FlightSize is 0 when the TCP sender
leaves fast recovery and the TCP receiver uses delayed
acknowledgments. Alexander Zimmermann provided several suggestions
to improve the clarity of the document.
10. References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
Control", RFC 5681, September 2009.
10.2. Informative References
[C98] Cardwell, N., "delayed ACKs for retransmitted packets:
ouch!". November 1998, Email to the tcpimpl mailing list,
archived at
<http://groups.yahoo.com/group/tcp-impl/message/1428>.
[F94] Floyd, S., "TCP and Successive Fast Retransmits", Technical
report, May 1995.
<ftp://ftp.ee.lbl.gov/papers/fastretrans.ps>.
[FF96] Fall, K. and S. Floyd, "Simulation-based Comparisons of
Tahoe, Reno and SACK TCP", Computer Communication Review,
July 1996. <ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z>.
[GF04] Gurtov, A. and S. Floyd, "Resolving Acknowledgment
Ambiguity in non-SACK TCP", NExt Generation Teletraffic and
Wired/Wireless Advanced Networking (NEW2AN'04),
February 2004. <http://www.cs.helsinki.fi/u/gurtov/
papers/heuristics.html>.
[Gur03] Gurtov, A., "[Tsvwg] resolving the problem of unnecessary
fast retransmits in go-back-N", email to the tsvwg mailing
list, July 28, 2003. <http://www.ietf.org/mail-archive/
web/tsvwg/current/msg04334.html>.
[Hen98] Henderson, T., "Re: NewReno and the 2001 Revision",
September 1998. Email to the tcpimpl mailing list,
archived at
<http://groups.yahoo.com/group/tcp-impl/message/1321>.
[Hoe95] Hoe, J., "Startup Dynamics of TCP's Congestion Control and
Avoidance Schemes", Master's Thesis, MIT, June 1995.
[Hoe96] Hoe, J., "Improving the Start-up Behavior of a Congestion
Control Scheme for TCP", ACM SIGCOMM, August 1996.
<http://ccr.sigcomm.org/archive/1996/conf/hoe.pdf>.
[LM97] Lin, D. and R. Morris, "Dynamics of Random Early
Detection", SIGCOMM 97, October 1997.
[NS] "The Network Simulator version 2 (ns-2)",
<http://www.isi.edu/nsnam/ns/>.
[RFC1323] Jacobson, V., Braden, R., and D. Borman, "TCP Extensions
for High Performance", RFC 1323, May 1992.
[RFC2582] Floyd, S. and T. Henderson, "The NewReno Modification to
TCP's Fast Recovery Algorithm", RFC 2582, April 1999.
[RFC2883] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An
Extension to the Selective Acknowledgement (SACK) Option
for TCP", RFC 2883, July 2000.
[RFC3042] Allman, M., Balakrishnan, H., and S. Floyd, "Enhancing
TCP's Loss Recovery Using Limited Transmit", RFC 3042,
January 2001.
[RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for
TCP", RFC 3522, April 2003.
[RFC3782] Floyd, S., Henderson, T., and A. Gurtov, "The NewReno
Modification to TCP's Fast Recovery Algorithm", RFC 3782,
April 2004.
Appendix A. Additional Information
Previous versions of this RFC ([RFC2582] [RFC3782]) contained
additional informative material on the following subjects, and may be
consulted by readers who may want more information about possible
variants to the algorithms and who may want references to specific
[NS] simulations that provide NewReno test cases.
Section 4 of [RFC3782] discusses some alternative behaviors for
resetting the retransmit timer after a partial acknowledgment.
Section 5 of [RFC3782] discusses some alternative behaviors for
performing retransmission after a partial acknowledgment.
Section 6 of [RFC3782] describes more information about the
motivation for the sender's state variable recover.
Section 9 of [RFC3782] introduces some NS simulation test suites for
NewReno. In addition, references to simulation results can be found
throughout [RFC3782].
Section 10 of [RFC3782] provides a comparison of Reno and
NewReno TCP.
Section 11 of [RFC3782] lists changes relative to [RFC2582].
Appendix B. Changes Relative to RFC 3782
In [RFC3782], the cwnd after Full ACK reception will be set to
(1) min (ssthresh, FlightSize + SMSS) or (2) ssthresh. However, the
first option carries a risk of performance degradation: With the
first option, if FlightSize is zero, the result will be 1 SMSS. This
means TCP can transmit only 1 segment at that moment, which can cause
a delay in ACK transmission at the receiver due to a delayed ACK
algorithm.
The FlightSize on Full ACK reception can be zero in some situations.
A typical example is where the sending window size during fast
recovery is small. In this case, the retransmitted packet and new
data packets can be transmitted within a short interval. If all
these packets successfully arrive, the receiver may generate a Full
ACK that acknowledges all outstanding data. Even if the window size
is not small, loss of ACK packets or a receive buffer shortage during
fast recovery can also increase the possibility of falling into this
situation.
The proposed fix in this document, which sets cwnd to at least 2*SMSS
if the implementation uses option 1 in the Full ACK case
(Section 3.2, step 3, option 1), ensures that the sender TCP
transmits at least two segments on Full ACK reception.
In addition, an erratum was reported for RFC 3782 (an editorial
clarification to Section 8); this erratum has been addressed in
Section 6 of this document.
The specification text (Section 3.2 herein) was rewritten to more
closely track Section 3.2 of [RFC5681].
Sections 4, 5, and 9-11 of [RFC3782] were removed, and instead
Appendix A of this document was added to back-reference this
informative material. A few references that have no citation in the
main body of the document have been removed.
Authors' Addresses
Tom Henderson
The Boeing Company
EMail: thomas.r.henderson@boeing.com
Sally Floyd
International Computer Science Institute
Phone: +1 (510) 666-2989
EMail: floyd@acm.org
URL: http://www.icir.org/floyd/
Andrei Gurtov
University of Oulu
Centre for Wireless Communications CWC
P.O. Box 4500
FI-90014 University of Oulu
Finland
EMail: gurtov@ee.oulu.fi
Yoshifumi Nishida
WIDE Project
Endo 5322
Fujisawa, Kanagawa 252-8520
Japan
EMail: nishida@wide.ad.jp
ISO Plain Language
On this page
§1Introduction
RFC 5681 documents four intertwined TCP congestion control algorithms: slow start, congestion avoidance, fast retransmit and fast recovery. It explicitly allows modifications of these algorithms, including ones that use the TCP Selective Acknowledgment option of RFC 2883, and ones that respond to partial acknowledgments, which are acknowledgments that cover new data but not all the data outstanding when loss was detected, in the absence of SACK. This document describes a specific algorithm for responding to partial acknowledgments, called NewReno, first proposed by Janey Hoe. It obsoletes RFC 3782.
In the typical implementation of the fast recovery algorithm of RFC 5681, first implemented in the 1990 BSD Reno release, the sender only retransmits a packet after a retransmit timeout or after three duplicate acknowledgments have arrived and triggered fast retransmit. A single timeout might cause several data packets to be retransmitted, but each invocation of fast retransmit leads to the retransmission of only a single data packet.
Two problems arise with Reno when multiple packet losses occur in a single window. Reno will often take a timeout, and even when a timeout is avoided, multiple fast retransmits and window reductions can occur. When multiple packets are lost and the SACK option is available, the sender has the information to make intelligent decisions about which packets to retransmit and which not to.
This document applies to TCP connections unable to use SACK, either because the option is not supported locally or because the peer did not indicate a willingness to use it. In the absence of SACK, little information is available to the sender during fast recovery: from the three duplicate acknowledgments it infers a packet loss and retransmits the indicated packet. In the case of multiple packets dropped from a single window, the first new information arrives when the sender receives an acknowledgment for the retransmitted packet. With a single drop and no reordering, that acknowledgment covers everything transmitted before fast retransmit was entered; with multiple drops it covers only some of it. That acknowledgment is called a partial acknowledgment.
The response described here is to infer that the next in-sequence packet has been lost and to retransmit it. The RFC does not claim that this is the optimal modification for connections that cannot use SACK, but based on experience with the network simulator ns-2 and with numerous implementations, the authors believe it improves the performance of fast retransmit and fast recovery in a wide variety of scenarios. Earlier versions of this document, RFC 2582 and RFC 3782, provide simulation-based evidence of the possible gains.
TCP normally resends one lost segment at a time and cuts its sending rate each time. If several segments from the same batch of data are lost, that means several cuts or a long wait. The rule here lets the sender keep repairing the same window until all the missing data has been acknowledged.
§2Terminology and the recover Variable
The document assumes the reader is familiar with the terms SENDER MAXIMUM SEGMENT SIZE (SMSS), CONGESTION WINDOW (cwnd) and FLIGHT SIZE (FlightSize) defined in RFC 5681. SMSS is the largest segment the sender may transmit, cwnd is the limit the sender places on how much data may be outstanding, and FlightSize is the amount of data presently outstanding.
The document defines one additional sender-side state variable, recover. When the sender is in fast recovery, this variable records the send sequence number that must be acknowledged before the fast recovery procedure is declared to be over. When the TCP protocol control block is initialized, recover is set to the initial send sequence number.
The variable recover marks the point in the byte stream that the sender must reach before it can say the loss episode is over. It is set when fast recovery starts and updated when the sender times out.
§3The NewReno Fast Retransmit and Fast Recovery
The basic idea is that the sender can infer, from the arrival of duplicate acknowledgments, whether multiple losses in the same window of data have most likely occurred, and so avoid taking a retransmit timeout or making multiple congestion window reductions. The NewReno modification applies to the fast recovery procedure that begins when three duplicate acknowledgments are received and ends when either a retransmission timeout occurs or an acknowledgment arrives that acknowledges all of the data up to and including the data that was outstanding when the procedure began.
The procedures of Section 3.2 of RFC 5681 are followed with these modifications. On initialization, recover is set to the initial send sequence number. When the third duplicate acknowledgment is received, the sender first checks whether the Cumulative Acknowledgment field covers more than recover. If it does, the value of recover is incremented to the highest sequence number transmitted so far, and the TCP enters fast retransmit. If it does not, the TCP does not enter fast retransmit and does not reset ssthresh.
When an acknowledgment arrives that acknowledges new data, there are two cases. For a full acknowledgment, one that acknowledges all of the data up to and including recover, the sender sets cwnd to either min(ssthresh, max(FlightSize, SMSS) + SMSS) or to ssthresh, where ssthresh is the value set when fast retransmit was entered and FlightSize is the amount of data presently outstanding. This is called deflating the window. If the second option is chosen, the implementation is encouraged to avoid a possible burst of data, in case the amount of data outstanding in the network is much less than the new congestion window allows; a simple mechanism is to limit the number of data packets that can be sent in response to a single acknowledgment. The sender then exits fast recovery.
For a partial acknowledgment, one that does not acknowledge all of the data up to and including recover, the sender retransmits the first unacknowledged segment, deflates the congestion window by the amount of new data acknowledged by the Cumulative Acknowledgment field, and, if the partial acknowledgment acknowledges at least one SMSS of new data, adds back SMSS bytes to the congestion window. This artificially inflates the window to reflect the additional segment that has left the network. The sender sends a new segment if the new value of cwnd permits it. This partial window deflation attempts to ensure that when fast recovery eventually ends, approximately the ssthresh amount of data will be outstanding in the network. The sender does not exit fast recovery, so if any duplicate acknowledgments subsequently arrive, step 4 of Section 3.2 of RFC 5681 is executed. For the first partial acknowledgment that arrives during fast recovery, the retransmit timer is also reset.
After a retransmit timeout, the sender records the highest sequence number transmitted in recover and exits the fast recovery procedure if applicable. Because the acknowledgment field contains the sequence number that the sender next expects to receive, the acknowledgment covers more than recover when ack_number - 1 > recover, that is, when at least one byte more of data is acknowledged beyond the highest byte that was outstanding when fast retransmit was last entered.
When a full acknowledgment finally arrives, the sender puts its sending limit back to where it was before the loss, and leaves the recovery state. When only part of the missing data has been acknowledged, the sender resends the next missing piece, reduces its limit by the amount that was just acknowledged, adds one segment back, and stays in the recovery state instead of starting again.
- Normal sending
- Fast recovery
| From | To | Event |
|---|---|---|
| Normal sending | Fast recovery | Three duplicate ACKs and ack_number - 1 > recover |
| Fast recovery | Fast recovery | Partial ACK: retransmit the first unacknowledged segment, deflate the window, stay in recovery |
| Fast recovery | Normal sending | Full ACK: deflate the window and exit fast recovery |
| Fast recovery | Normal sending | Retransmit timeout |
| Acknowledgment | What the sender does |
|---|---|
| Full ACK THIS RFC | Acknowledges all data up to and including recover. Set cwnd to min(ssthresh, max(FlightSize, SMSS) + SMSS) or to ssthresh, then leave fast recovery. |
| Partial ACK | Does not acknowledge all data up to recover. Retransmit the first unacknowledged segment, deflate cwnd by the new data acknowledged, add back one SMSS if at least one SMSS was acknowledged, and stay in fast recovery. |
ack_number - 1 > recover ; does this ACK cover more than recover?
cwnd = min(ssthresh, max(FlightSize, SMSS) + SMSS) ; or cwnd = ssthresh
cwnd = cwnd - (new data acknowledged) + SMSS ; on a partial ACK§4Duplicate Acknowledgments after a Timeout
After each retransmit timeout, the highest sequence number transmitted so far is recorded in recover. If, after a timeout, the sender retransmits three consecutive packets that the receiver has already received, it will receive three duplicate acknowledgments that do not cover more than recover. In that case the duplicate acknowledgments are not an indication of a new instance of congestion; they simply show that the sender has unnecessarily retransmitted at least three packets.
However, when a retransmitted packet is itself dropped, the sender can also receive three duplicate acknowledgments that do not cover more than recover, and in that case it would have been better off initiating fast retransmit. For a sender that implements this document's algorithm, the sender does not infer a packet drop from duplicate acknowledgments in this scenario, and the retransmit timer remains the backup mechanism for inferring packet loss.
Several heuristics, based on timestamps or on the amount of advancement of the Cumulative Acknowledgment field, allow the sender to distinguish in some cases between the two situations, and the TCP sender may use such a heuristic to decide to invoke a fast retransmit even when the three duplicate acknowledgments do not cover more than recover.
With the ACK heuristic, the sender stores the previous cumulative acknowledgment as prev_highest_ack and the latest as highest_ack, and performs an extra check when the Cumulative Acknowledgment field does not cover more than recover. If the congestion window is greater than SMSS bytes and the difference between highest_ack and prev_highest_ack is at most 4*SMSS bytes, the duplicate acknowledgments indicate a lost segment and the sender enters fast retransmit; otherwise they likely result from unnecessary retransmissions and it does not. The congestion window check protects against fast retransmit immediately after a retransmit timeout. The heuristic can fail if several acknowledgments are lost, because then the cumulative acknowledgment jumps by more than three segments.
With the timestamp heuristic, the sender stores the timestamp of the last acknowledged segment, and the check becomes: if the Cumulative Acknowledgment field did not cover more than recover, compare the echoed timestamp in the last non-duplicate acknowledgment with the stored timestamp. If they are equal, the duplicate acknowledgments indicate a lost segment and the sender enters fast retransmit; otherwise they likely result from unnecessary retransmissions. The heuristic works correctly when the receiver echoes timestamps as specified in RFC 1323 and by its revision attempts, but it can fail if the receiver arbitrarily echoes timestamps, or if a timeout was spurious and the returning acknowledgments are not from retransmitted segments, which can be prevented by detection algorithms such as the Eifel detection algorithm of RFC 3522.
After a timeout, three duplicate acknowledgments can mean two different things: that packets were genuinely lost, or that the sender resent packets the receiver already had. The RFC gives two ways to tell them apart, one by how far the acknowledgment number moved and one by comparing timestamps.
- ack_number - 1 > recoverThe test that decides whether the third duplicate ACK really covers new data.
- recoverThe sequence number that must be acknowledged before fast recovery is over. Raised when fast retransmit is entered and after a timeout.
- ACK heuristicAfter a timeout, treat duplicate ACKs as a loss if the cumulative ACK moved by at most 4*SMSS and the window is above SMSS.
- Timestamp heuristicAfter a timeout, treat duplicate ACKs as a loss if the echoed timestamp equals the stored timestamp of the last acknowledged segment.
- Retransmit timerReset on the first partial ACK, and it stays the backup mechanism when the sender cannot tell a loss from an unnecessary retransmission.
- maxburstLimit how many segments may be sent for one acknowledgment, so leaving fast recovery does not produce a burst.
§6Implementation Notes for Senders and Receivers
For the data receiver, RFC 5681 specifies that out-of-order data segments SHOULD be acknowledged immediately in order to accelerate loss recovery. Some receivers instead wait for their delayed acknowledgment timer to expire when they send a partial acknowledgment, which severely limits the potential benefit of NewReno by delaying the receipt of the partial acknowledgment at the sender. The recommendation is that the receiver send an immediate acknowledgment for an out-of-order segment, even when that out-of-order segment fills a hole in the buffer.
For the data sender, implementations are told to take measures to avoid a possible burst of data when leaving fast recovery, in case the amount of new data the sender may send under the new congestion window is large. Bursts can occur when the FlightSize is much less than the new congestion window on exit from fast recovery, which can arise during NewReno when acknowledgments are lost or treated as pure window updates. One simple mechanism is to limit the number of data packets that can be sent in response to a single acknowledgment, known as maxburst in ns-2. Other possibilities are rate-based pacing, or setting the slow start threshold to the resultant congestion window and then resetting the congestion window to FlightSize.
An implementation may want a separate flag to record whether it is presently in the fast recovery procedure; the duplicate acknowledgment counter is not reliable for that, because it can be reset on window updates and out-of-order acknowledgments. When updating the Cumulative Acknowledgment field outside fast recovery, the state variable recover may also need updating so that entry into fast recovery remains possible; this arises when an update results in a sequence wraparound that affects the ordering between the acknowledgment field and recover.
It is important for the sender to respond correctly to duplicate acknowledgments received when it is no longer in fast recovery, for example after a retransmit timeout. The Limited Transmit procedure of RFC 3042 describes possible responses to the first and second duplicate acknowledgments. When three or more duplicate acknowledgments are received, the Cumulative Acknowledgment field does not cover more than recover and a new fast recovery is not invoked, the sender should follow the guidance of Section 4; otherwise it could end up in a chain of spurious timeouts. The RFC mentions this only because several NewReno implementations had this bug, including the implementation in ns-2.
It has been observed that some TCP implementations enter a slow start or congestion avoidance window updating algorithm immediately after cwnd is set by the equation of Section 3.2, step 3, even without a new external event generating the change. After cwnd is set on the way out of fast recovery, cwnd should not be updated until a further event occurs, such as the arrival of an acknowledgment or a timeout.
Two things make the rule work. The receiver should acknowledge out-of-order data at once rather than waiting, so the sender learns quickly what is missing. The sender should avoid sending a burst of data when it leaves the recovery state, and it must not mistake duplicate acknowledgments left over from a timeout for fresh congestion.
§7Security Considerations
RFC 5681 discusses the general security considerations concerning TCP congestion control. This document describes a specific algorithm that conforms with the congestion control requirements of RFC 5681, so those considerations apply to it as well. There are no known additional security concerns for this specific algorithm.
The rule changes how the sender reacts to loss, not what is sent, so it adds no security risk of its own. The usual cautions about TCP congestion control apply.
§8Conclusions and Changes since RFC 3782
This document specifies the NewReno fast retransmit and fast recovery algorithms for TCP. The modification can even be important for TCP implementations that support the SACK option, because the SACK option can only be used when both end nodes support it. NewReno performs better than Reno in a number of scenarios discussed in previous versions of the document.
The document says the important thing is to implement NewReno instead of Reno for a connection without SACK, and that it is less important exactly which variant of NewReno is implemented, because the differences between the variants are small compared with the difference between Reno and NewReno. Appendix A points back to the informative material in RFC 3782 on alternative behaviors for resetting the retransmit timer after a partial acknowledgment, for performing retransmission after a partial acknowledgment, for the motivation for the recover variable, for simulation test suites, for a comparison of Reno and NewReno, and for the changes relative to RFC 2582.
The change from RFC 3782 is in the value of cwnd on receipt of a full acknowledgment. In RFC 3782, cwnd after a full acknowledgment was set to min(ssthresh, FlightSize + SMSS) or to ssthresh. The first option carries a risk of performance degradation: if FlightSize is zero, the result is 1 SMSS, so TCP can transmit only one segment at that moment, which can cause a delay in acknowledgment transmission at the receiver because of a delayed acknowledgment algorithm. FlightSize on full acknowledgment reception can be zero in some situations, a typical example being a small sending window during fast recovery, where the retransmitted packet and new data packets can be transmitted within a short interval and the receiver may generate a full acknowledgment acknowledging all outstanding data; loss of acknowledgment packets or a receive buffer shortage during fast recovery can also increase the likelihood.
The fix in this document sets cwnd to at least 2*SMSS if the implementation uses the first option in the full acknowledgment case, so the sender transmits at least two segments on full acknowledgment reception. An erratum reported for RFC 3782, an editorial clarification to its Section 8, has been addressed in Section 6 of this document. The specification text of Section 3.2 was rewritten to track Section 3.2 of RFC 5681 more closely, and Sections 4, 5 and 9 to 11 of RFC 3782 were removed in favour of the new Appendix A.
The one technical difference from the previous version is that the sender now keeps enough of a window to send at least two segments when the loss episode ends, so that the receiver is not left waiting for an acknowledgment.