This document is obsolete. Please
refer to RFC 4960.
Network Working Group J. Stone Request for Comments: 3309 Stanford Updates: 2960 R. Stewart Category: Standards Cisco Systems D. Otis SANlight September 2002
Stream Control Transmission Protocol (SCTP) Checksum Change
Status of this Memo
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2002). All Rights Reserved.
Abstract
Stream Control Transmission Protocol (SCTP) currently uses an Adler- 32 checksum. For small packets Adler-32 provides weak detection of errors. This document changes that checksum and updates SCTP to use a 32 bit CRC checksum.
A fundamental weakness has been detected in SCTP's current Adler-32 checksum algorithm [STONE]. This document updates and replaces the Adler-32 checksum definition in [RFC 2960]. Note that there is no graceful transition mechanism for migrating to the new checksum. Implementations are expected to immediately switch to the new algorithm; use of the old algorithm is deprecated.
One requirement of an effective checksum is that it evenly and smoothly spreads its input packets over the available check bits.
From an email from Jonathan Stone, who analyzed the Adler-32 as part of his doctoral thesis:
"Briefly, the problem is that, for very short packets, Adler32 is guaranteed to give poor coverage of the available bits. Don't take my word for it, ask Mark Adler. :-)
Adler-32 uses two 16-bit counters, s1 and s2. s1 is the sum of the input, taken as 8-bit bytes. s2 is a running sum of each value of s1. Both s1 and s2 are computed mod-65521 (the largest prime less than 2^16). Consider a packet of 128 bytes. The *most* that each byte can be is 255. There are only 128 bytes of input, so the greatest value which the s1 accumulator can have is 255 * 128 = 32640. So, for 128-byte packets, s1 never wraps. That is critical. Why?
The key is to consider the distribution of the s1 values, over some distribution of the values of the individual input bytes in each packet. Because s1 never wraps, s1 is simply the sum of the individual input bytes. (Even Doug's trick of adding 0x5555 doesn't help here, and an even larger value doesn't really help: we can get at most one mod-65521 reduction.)
Given the further assumption that the input bytes are drawn independently from some distribution (they probably aren't: for file system data, it's even worse than that!), the Central Limit Theorem tells us that that s1 will tend to have a normal distribution. That's bad: it tells us that the value of s1 will have hot-spots at around 128 times the mean of the input distribution: around 16k, assuming a uniform distribution. That's bad. We want the accumulator to wrap as many times as possible, so that the resulting sum has as close to a uniform distribution as possible. (I call this "fairness".)
Stone, et. al. Standards Track [Page 2]
RFC 3309 SCTP Checksum Change September 2002
So, for short packets, the Adler-32 s1 sum is guaranteed to be unfair. Why is that bad? It's bad because the space of valid packets -- input data, plus checksum values -- is also small. If all packets have checksum values very close to 32640, then the likelihood of even a 'small' error leaving a damaged packet with a valid checksum is higher than if all checksum values are equally likely."
Due to this inherent weakness, exacerbated by the fact that SCTP will first be used as a signaling transport protocol where signaling messages are usually less than 128 bytes, a new checksum algorithm is specified by this document, replacing the current Adler-32 algorithm with CRC-32c.
The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD,SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when they appear in this document, are to be interpreted as described in [RFC2119].
The procedures described in section 2.1 of this document MUST be followed, replacing the current checksum defined in [RFC2960].
Furthermore any references within [RFC2960] to Adler-32 MUST be treated as a reference to CRC-32c. Section 2.1 of this document describes the new calculation and verification procedures that MUST be followed.
When sending an SCTP packet, the endpoint MUST strengthen the data integrity of the transmission by including the CRC-32c checksum value calculated on the packet, as described below.
After the packet is constructed (containing the SCTP common header and one or more control or DATA chunks), the transmitter shall:
1) Fill in the proper Verification Tag in the SCTP common header and initialize the Checksum field to 0's.
2) Calculate the CRC-32c of the whole packet, including the SCTP common header and all the chunks.
Stone, et. al. Standards Track [Page 3]
RFC 3309 SCTP Checksum Change September 2002
3) Put the resulting value into the Checksum field in the common header, and leave the rest of the bits unchanged.
When an SCTP packet is received, the receiver MUST first check the CRC-32c checksum:
1) Store the received CRC-32c value,
2) Replace the 32 bits of the Checksum field in the received SCTP packet with all '0's and calculate a CRC-32c value of the whole received packet. And,
3) Verify that the calculated CRC-32c value is the same as the received CRC-32c value. If not, the receiver MUST treat the packet as an invalid SCTP packet.
The default procedure for handling invalid SCTP packets is to silently discard them.
Any hardware implementation SHOULD be done in a way that is verifiable by the software.
We define a 'reflected value' as one that is the opposite of the normal bit order of the machine. The 32 bit CRC is calculated as described for CRC-32c and uses the polynomial code 0x11EDC6F41 (Castagnoli93) or x^32+x^28+x^27+x^26+x^25 +x^23+x^22+x^20+x^19+x^18+x^14+x^13+x^11+x^10+x^9+x^8+x^6+x^0. The CRC is computed using a procedure similar to ETHERNET CRC [ITU32], modified to reflect transport level usage.
CRC computation uses polynomial division. A message bit-string M is transformed to a polynomial, M(X), and the CRC is calculated from M(X) using polynomial arithmetic [Peterson 72].
When CRCs are used at the link layer, the polynomial is derived from on-the-wire bit ordering: the first bit 'on the wire' is the high- order coefficient. Since SCTP is a transport-level protocol, it cannot know the actual serial-media bit ordering. Moreover, different links in the path between SCTP endpoints may use different link-level bit orders.
A convention must therefore be established for mapping SCTP transport messages to polynomials for purposes of CRC computation. The bit- ordering for mapping SCTP messages to polynomials is that bytes are taken most-significant first; but within each byte, bits are taken least-significant first. The first byte of the message provides the eight highest coefficients. Within each byte, the least-significant SCTP bit gives the most significant polynomial coefficient within
Stone, et. al. Standards Track [Page 4]
RFC 3309 SCTP Checksum Change September 2002
that byte, and the most-significant SCTP bit is the least significant polynomial coefficient in that byte. (This bit ordering is sometimes called 'mirrored' or 'reflected' [Williams93].) CRC polynomials are to be transformed back into SCTP transport-level byte values, using a consistent mapping.
The SCTP transport-level CRC value should be calculated as follows:
- CRC input data are assigned to a byte stream, numbered from 0 to N-1.
- the transport-level byte-stream is mapped to a polynomial value. An N-byte PDU with j bytes numbered 0 to N-1, is considered as coefficients of a polynomial M(x) of order 8N-1, with bit 0 of byte j being coefficient x^(8(N-j)-8), bit 7 of byte j being coefficient x^(8(N-j)-1).
- the CRC remainder register is initialized with all 1s and the CRC is computed with an algorithm that simultaneously multiplies by x^32 and divides by the CRC polynomial.
- the polynomial is multiplied by x^32 and divided by G(x), the generator polynomial, producing a remainder R(x) of degree less than or equal to 31.
- the coefficients of R(x) are considered a 32 bit sequence.
- the bit sequence is complemented. The result is the CRC polynomial.
- The CRC polynomial is mapped back into SCTP transport-level bytes. Coefficient of x^31 gives the value of bit 7 of SCTP byte 0, the coefficient of x^24 gives the value of bit 0 of byte 0. The coefficient of x^7 gives bit 7 of byte 3 and the coefficient of x^0 gives bit 0 of byte 3. The resulting four- byte transport-level sequence is the 32-bit SCTP checksum value.
IMPLEMENTATION NOTE: Standards documents, textbooks, and vendor literature on CRCs often follow an alternative formulation, in which the register used to hold the remainder of the long-division algorithm is initialized to zero rather than all-1s, and instead the first 32 bits of the message are complemented. The long-division algorithm used in our formulation is specified, such that the the initial multiplication by 2^32 and the long-division are combined into one simultaneous operation. For such algorithms, and for messages longer than 64 bits, the two specifications are precisely equivalent. That equivalence is the intent of this document.
Stone, et. al. Standards Track [Page 5]
RFC 3309 SCTP Checksum Change September 2002
Implementors of SCTP are warned that both specifications are to be found in the literature, sometimes with no restriction on the long- division algorithm. The choice of formulation in this document is to permit non-SCTP usage, where the same CRC algorithm may be used to protect messages shorter than 64 bits.
If SCTP could follow link level CRC use, the CRC would be computed over the link-level bit-stream. The first bit on the link mapping to the highest-order coefficient, and so on, down to the last link-level bit as the lowest-order coefficient. The CRC value would be transmitted immediately after the input message as a link-level 'trailer'. The resulting link-level bit-stream would be (M(X)x) * x^32) + (M(X)*x^32))/ G(x), which is divisible by G(X). There would thus be a constant CRC remainder for 'good' packets. However, given that implementations of RFC 2960 have already proliferated, the IETF discussions considered that the benefit of a 'trailer' CRC did not outweigh the cost of making a very large change in the protocol processing. Further, packets accepted by the SCTP 'header' CRC are in one-to-one correspondence with packets accepted by a modified procedure using a 'trailer' CRC value, and where the SCTP common checksum header is set to zero on transmission and is received as zero.
There may be a computational advantage in validating the Association against the Verification Tag, prior to performing a checksum, as invalid tags will result in the same action as a bad checksum in most cases. The exceptions for this technique would be INIT and some SHUTDOWN-COMPLETE exchanges, as well as a stale COOKIE-ECHO. These special case exchanges must represent small packets and will minimize the effect of the checksum calculation.
The authors would like to thank the following people that have provided comments and input on the checksum issue:
Mark Adler, Ran Atkinson, Stephen Bailey, David Black, Scott Bradner, Mikael Degermark, Laurent Glaude, Klaus Gradischnig, Alf Heidermark, Jacob Heitz, Gareth Kiely, David Lehmann, Allision Mankin, Lyndon Ong, Craig Partridge, Vern Paxson, Kacheong Poon, Michael Ramalho, David Reed, Ian Rytina, Hanns Juergen Schwarzbauer, Chip Sharp, Bill Sommerfeld, Michael Tuexen, Jim Williams, Jim Wendt, Michael Welzl, Jonathan Wood, Lloyd Wood, Qiaobing Xie, La Monte Yarroll.
Special thanks to Dafna Scheinwald, Julian Satran, Pat Thaler, Matt Wakeley, and Vince Cavanna, for selection criteria of polynomials and examination of CRC polynomials, particularly CRC-32c [Castagnoli93].
Special thanks to Mr. Ross Williams and his document [Williams93]. This non-formal perspective on software aspects of CRCs furthered understanding of authors previously unfamiliar with CRC computation. More formal treatments of [Blahut 94] or [Peterson 72], was also essential.
[Castagnoli93] G. Castagnoli, S. Braeuer and M. Herrman, "Optimization of Cyclic Redundancy-Check Codes with 24 and 32 Parity Bits", IEEE Transactions on Communications, Vol. 41, No. 6, June 1993
[McKee75] H. McKee, "Improved {CRC} techniques detects erroneous leading and trailing 0's in transmitted data blocks", Computer Design Volume 14 Number 10 Pages 102-4,106, October 1975
[RFC1700] Reynolds, J. and J. Postel, "ASSIGNED NUMBERS", RFC 1700, October 1994.
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L. and V. Paxson, "Stream Control Transmission Protocol," RFC 2960, October 2000.
Stone, et. al. Standards Track [Page 7]
RFC 3309 SCTP Checksum Change September 2002
[ITU32] ITU-T Recommendation V.42, "Error-correcting procedures for DCEs using asynchronous-to-synchronous conversion", section 8.1.1.6.2, October 1996.
[Feldmeier 95] David C. Feldmeier, Fast software implementation of error detection codes, IEEE Transactions on Networking, vol 3 no 6, pp 640-651, December, 1995.
[Glaise 1997] R. J. Glaise, A two-step computation of cyclic redundancy code CRC-32 for ATM networks, IBM Journal of Research and Development} vol 41 no 6, 1997. http://www.research.ibm.com/journal/rd/416/ glaise.html.
[Prange 1957] E. Prange, Cyclic Error-Correcting codes in two symbols, Technical report AFCRC-TN-57-103, Air Force Cambridge Research Center, Cambridge, Mass. 1957.
[Peterson 1972] W. W. Peterson and E.J Weldon, Error Correcting Codes, 2nd. edition, MIT Press, Cambridge, Massachusetts.
[Shie2001] Ming-Der Shieh et. al, A Systematic Approach for Parallel CRC Computations. Journal of Information Science and Engineering, Vol.17 No.3, pp.445-461
[Sprachman2001] Michael Sprachman, Automatic Generation of Parallel CRC Circuits, IEEE Design & Test May-June 2001
Stone, et. al. Standards Track [Page 8]
RFC 3309 SCTP Checksum Change September 2002
Appendix
This appendix is for information only and is NOT part of the standard.
The anticipated deployment of SCTP ranges over several orders of magnitude of link speed: from cellular-power telephony devices at tens of kilobits, to local links at tens of gigabits. Implementors of SCTP should consider their link speed and choose, from the wide range of CRC implementations, one which matches their own design point for size, cost, and throughput. Many techniques for computing CRCs are known. This Appendix surveys just a few, to give a feel for the range of techniques available.
CRCs are derived from early work by Prange in the 1950s [Prange 57]. The theory underlying CRCs and choice of generator polynomial can be introduced by either the theory of Galois fields [Blahut 94] or as ideals of an algebra over cyclic codes [cite Peterson 72].
One of the simplest techniques is direct bit-serial hardware implementations, using the generator polynomial as the taps of a linear feedback shift register (LSFR). LSFR computation follows directly from the mathematics, and is generally attributed to Prange. Tools exist which, a CRC generator polynomial, will produce synthesizable Verilog code for CRC hardware [Easics 2001].
Since LSFRs do not scale well in speed, a variety of other techniques have been explored. One technique exploits the fact that the divisor of the polynomial long-division, G, is known in advance. It is thus possible to pre-compute lookup tables giving the polynomial remainder of multiple input bits --- typically 2, 4, or 8 bits of input at a time. This technique can be used either in software or in hardware. Software to compute lookup tables yielding 2, 4, or 8 bits of result is freely available. [Williams93]
For multi-gigabit links, the above techniques may still not be fast enough. One technique for computing CRCS at OC-48 rates is 'two- stage' CRC computation [Glaise 1997]. Here, some multiple of G(x), G(x)H(x), is chosen so as to minimize the number of nonzero coefficients, or weight, of the product G(x)H(x). The low weight of the product polynomial makes it susceptible to efficient hardware divide-by-constant implementations. This first stage gives M(x)/ (G(x)H(x)), as its result. The second stage then divides the result of the first stage by H(x), yielding (M(x)/(G(x)H(x)))/H(x). If H(x) is also relatively prime to G(x), this gives M(x)/G(x). Further developments on this approach can be found in [Shie2001] and [Sprachman2001].
Stone, et. al. Standards Track [Page 9]
RFC 3309 SCTP Checksum Change September 2002
The literature also includes a variety of software CRC implementations. One approach is to use a carefully-tuned assembly code for direct polynomial division. [Feldmeier 95] reports that for low-weight polynomials, tuned polynomial arithmetic gives higher throughput than table-lookup algorithms. Even within table-lookup algorithms, the size of the table can be tuned, either for total cache footprint, or (for space-restricted environments) to minimize total size.
Implementors should keep in mind, the bit ordering described in Section 2: the ordering of bits within bytes for computing CRCs in SCTP is the least significant bit of each byte is the most- significant polynomial coefficient(and vice-versa). This 'reflected' SCTP CRC bit ordering matches on-the-wire bit order for Ethernet and other serial media, but is the reverse of traditional Internet bit ordering.
One technique to accommodate this bit-reversal can be explained as follows: sketch out a hardware implementation, assuming the bits are in CRC bit order; then perform a left-to-right inversion (mirror image) on the entire algorithm. (We defer, for a moment, the issue of byte order within words.) Then compute that "mirror image" in software. The CRC from the "mirror image" algorithm will be the bit-reversal of a correct hardware implementation. When the link- level media sends each byte, the byte is sent in the reverse of the host CPU bit-order. Serialization of each byte of the "reflected" CRC value re-reverses the bit order, so in the end, each byte will be transmitted on-the-wire in the specified bit order.
The following non-normative sample code is taken from an open-source CRC generator [Williams93], using the "mirroring" technique and yielding a lookup table for SCTP CRC32-c with 256 entries, each 32 bits wide. While neither especially slow nor especially fast, as software table-lookup CRCs go, it has the advantage of working on both big-endian and little-endian CPUs, using the same (host-order) lookup tables, and using only the pre-defined ntohl() and htonl() operations. The code is somewhat modified from [Williams93], to ensure portability between big-endian and little-endian architectures. (Note that if the byte endian-ness of the target architecture is known to be little-endian the final bit-reversal and byte-reversal steps can be folded into a single operation.)
Stone, et. al. Standards Track [Page 10]
RFC 3309 SCTP Checksum Change September 2002
/*************************************************************/ /* Note Definition for Ross Williams table generator would */ /* be: TB_WIDTH=4, TB_POLLY=0x1EDC6F41, TB_REVER=TRUE */ /* For Mr. Williams direct calculation code use the settings */ /* cm_width=32, cm_poly=0x1EDC6F41, cm_init=0xFFFFFFFF, */ /* cm_refin=TRUE, cm_refot=TRUE, cm_xorort=0x00000000 */ /*************************************************************/
/* Example of the crc table file */ #ifndef __crc32cr_table_h__ #define __crc32cr_table_h__
if (fclose (tf) != 0) printf ("Unable to close <%s>." OUTPUT_FILE); else printf ("\nThe CRC-32c table has been written to <%s>.\n", OUTPUT_FILE); }
/* Example of crc insertion */
#include "crc32cr.h"
unsigned long generate_crc32c(unsigned char *buffer, unsigned int length) { unsigned int i; unsigned long crc32 = ~0L; unsigned long result; unsigned char byte0,byte1,byte2,byte3;
for (i = 0; i < length; i++){ CRC32C(crc32, buffer[i]); } result = ~crc32;
/* result now holds the negated polynomial remainder; * since the table and algorithm is "reflected" [williams95]. * That is, result has the same value as if we mapped the message * to a polynomial, computed the host-bit-order polynomial * remainder, performed final negation, then did an end-for-end * bit-reversal. * Note that a 32-bit bit-reversal is identical to four inplace * 8-bit reversals followed by an end-for-end byteswap. * In other words, the bytes of each bit are in the right order, * but the bytes have been byteswapped. So we now do an explicit * byteswap. On a little-endian machine, this byteswap and * the final ntohl cancel out and could be elided. */
int insert_crc32(unsigned char *buffer, unsigned int length) { SCTP_message *message; unsigned long crc32; message = (SCTP_message *) buffer; message->common_header.checksum = 0L; crc32 = generate_crc32c(buffer,length); /* and insert it into the message */ message->common_header.checksum = htonl(crc32); return 1; }
int validate_crc32(unsigned char *buffer, unsigned int length) { SCTP_message *message; unsigned int i; unsigned long original_crc32; unsigned long crc32 = ~0L;
/* save and zero checksum */ message = (SCTP_message *) buffer; original_crc32 = ntohl(message->common_header.checksum); message->common_header.checksum = 0L; crc32 = generate_crc32c(buffer,length); return ((original_crc32 == crc32)? 1 : -1); }
Stone, et. al. Standards Track [Page 15]
RFC 3309 SCTP Checksum Change September 2002
Authors' Addresses
Jonathan Stone Room 446, Mail code 9040 Gates building 4A Stanford, Ca 94305
EMail: jonathan@dsg.stanford.edu
Randall R. Stewart 24 Burning Bush Trail. Crystal Lake, IL 60012 USA
EMail: rrs@cisco.com
Douglas Otis 800 E. Middlefield Mountain View, CA 94043 USA
EMail: dotis@sanlight.net
Stone, et. al. Standards Track [Page 16]
RFC 3309 SCTP Checksum Change September 2002
Full Copyright Statement
Copyright (C) The Internet Society (2002). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the Internet Society.