Network Working Group E. Boschi Request for Comments: 5610 B. Trammell Category: Standards Track Hitachi Europe L. Mark Fraunhofer IFAM T. Zseby Fraunhofer FOKUS July 2009
Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements
Abstract
This document describes an extension to the IP Flow Information Export (IPFIX) protocol, which is used to represent and transmit data from IP flow measurement devices for collection, storage, and analysis, to allow the encoding of IPFIX Information Model properties within an IPFIX Message stream. This enables the export of extended type information for enterprise-specific Information Elements and the storage of such information within IPFIX Files, facilitating interoperability and reusability among a wide variety of applications and tools.
Status of This Memo
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.
IP Flow Information Export (IPFIX) provides a template mechanism for the flexible description of Data Records, by defining a record as a collection of Information Elements defined in an IANA registry, However, these Templates provide limited information about the type of described data; indeed, they encode only the size of the fields defined by these Information Elements. There presently exists no mechanism to provide full type information for these Information Elements, as is defined for the Information Elements in the IPFIX Information Model [RFC5102].
This especially limits the interoperability of enterprise-specific Information Elements. It is not possible to use analysis tools on IPFIX records containing these partially defined Information Elements that have not been developed with a priori knowledge of their types, since such tools will not be able to decode them; these tools can only treat and store them as opaque octet arrays. However, if richer information is available, additional operations such as efficient storage, display, and limited analysis of records containing enterprise-specific Information Elements become possible, even for Collecting Processes that have not been specifically developed to understand them.
This document defines a general mechanism to encode the full set of properties available for the definition of Information Elements within the IPFIX Information Model inline within an IPFIX Message stream using IPFIX Options. This mechanism may be used to fully define type information for Information Elements used within a message stream, without resorting to an external reference or reliance on out-of-band configuration, thereby improving the interoperability of enterprise-specific Information Elements.
Note that the solution described in this document is not intended as a replacement for registration with IANA of generally useful Information Elements. It introduces overhead and does not lead to real interoperability as provided by standardization. Therefore, we highly recommend standardizing all new generally useful Information Elements by registering them with IANA. Standardization is straightforward, and the type information that needs to be specified in order to support the proposed solution provides a perfect basis for the description required for standardizing the Information Element.
It might happen that an Information Element previously described by the mechanism in this document later becomes an IANA-registered, standard Information Element. In such environments, old and new versions of the Information Element can coexist. A translation
Boschi, et al. Standards Track [Page 3]
RFC 5610 IPFIX Type Information July 2009
between Information Elements expressed by the described solution and standardized Information Elements is therefore not necessary and is out of scope for this document.
"Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information" [RFC5101] (informally, the IPFIX Protocol document) and its associated documents define the IPFIX Protocol, which provides network engineers and administrators with access to IP traffic flow information.
"Architecture for IP Flow Information Export" [RFC5470] (the IPFIX Architecture document) defines the architecture for the export of measured IP flow information out of an IPFIX Exporting Process to an IPFIX Collecting Process, and the basic terminology used to describe the elements of this architecture, per the requirements defined in "Requirements for IP Flow Information Export" [RFC3917]. The IPFIX Protocol document [RFC5101] then covers the details of the method for transporting IPFIX Data Records and Templates via a congestion-aware transport protocol from an IPFIX Exporting Process to an IPFIX Collecting Process.
"Information Model for IP Flow Information Export" [RFC5102] (informally, the IPFIX Information Model document) describes the Information Elements used by IPFIX, including details on Information Element naming, numbering, and data type encoding.
This document references the Protocol and Architecture documents for terminology and extends the IPFIX Information Model to provide new Information Elements for the representation of Information Element properties. It draws data type definitions and data type semantics definitions from the Information Model; the encodings of these data types are defined in [RFC5101].
Terms used in this document that are defined in the Terminology section of the IPFIX Protocol [RFC5101] document are to be interpreted as defined there.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
This section describes the mechanism used to encode Information Element type information within an IPFIX Message stream. This mechanism consists of an Options Template Record used to define Information Element type records, and a set of Information Elements required by these type records. We first specify the necessary Information Elements, followed by the structure of the Options Template describing the type records.
Note that Information Element type records require one Information Element, informationElementId, that is defined in the Packet Sampling (PSAMP) Information Model [RFC5477]. This Information Element supports references only to IANA-defined Information Elements; the privateEnterpriseNumber Information Element is required alongside informationElementId to describe enterprise-specific Information Elements.
Description: A description of the abstract data type of an IPFIX information element. These are taken from the abstract data types defined in Section 3.1 of the IPFIX Information Model [RFC5102]; see that section for more information on the types described below. This field may take the values defined in Table 1 below.
These types are registered in the IANA IPFIX Information Element Data Type subregistry. This subregistry is intended to assign numbers for type names, not to provide a mechanism for adding data types to the IPFIX Protocol, and as such requires a Standards Action [RFC5226] to modify.
Description: A UTF-8 [RFC3629] encoded Unicode string containing a human-readable description of an Information Element. The content of the informationElementDescription MAY be annotated with one or more language tags [RFC4646], encoded in-line [RFC2482] within the UTF-8 string, in order to specify the language in which the
Boschi, et al. Standards Track [Page 6]
RFC 5610 IPFIX Type Information July 2009
description is written. Description text in multiple languages MAY tag each section with its own language tag; in this case, the description information in each language SHOULD have equivalent meaning. In the absence of any language tag, the "i-default" [RFC2277] language SHOULD be assumed. See the Security Considerations (Section 4) for notes on string handling for Information Element type records.
Description: A UTF-8 [RFC3629] encoded Unicode string containing the name of an Information Element, intended as a simple identifier. See the Security Considerations (Section 4) for notes on string handling for Information Element type records.
Description: A description of the semantics of an IPFIX Information Element. These are taken from the data type semantics defined in Section 3.2 of the IPFIX Information Model [RFC5102]; see that section for more information on the types described below. This field may take the values in Table 2 below. The special value 0x00 (default) is used to note that no semantics apply to the field; it cannot be manipulated by a Collecting Process or File Reader that does not understand it a priori.
These semantics are registered in the IANA IPFIX Information Element Semantics subregistry. This subregistry is intended to assign numbers for semantics names, not to provide a mechanism for adding semantics to the IPFIX Protocol, and as such requires a Standards Action [RFC5226] to modify.
Description: A description of the units of an IPFIX Information Element. These correspond to the units implicitly defined in the Information Element definitions in Section 5 of the IPFIX Information Model [RFC5102]; see that section for more information on the types described below. This field may take the values in Table 3 below; the special value 0x00 (none) is used to note that the field is unitless.
These types are registered in the IANA IPFIX Information Element Units subregistry; new types may be added on a First Come First Served [RFC5226] basis.
Description: A private enterprise number, as assigned by IANA. Within the context of an Information Element Type record, this element can be used along with the informationElementId element to scope properties to a specific Information Element. To export
Boschi, et al. Standards Track [Page 9]
RFC 5610 IPFIX Type Information July 2009
type information about an IANA-assigned Information Element, set the privateEnterpriseNumber to 0, or do not export the privateEnterpriseNumber in the type record. To export type information about an enterprise-specific Information Element, export the enterprise number in privateEnterpriseNumber, and export the Information Element number with the Enterprise bit cleared in informationElementId. The Enterprise bit in the associated informationElementId Information Element MUST be ignored by the Collecting Process.
Abstract Data Type: unsigned32
Data Type Semantics: identifier
ElementId: 346
Status: current
Reference: Sections 3.2 and 3.4.1 of the IPFIX Protocol [RFC5101]; Section 8.2.3 of the PSAMP Information Model [RFC5477].
The Information Element Type Options Template attaches type information to Information Elements used within Template Records, as scoped to an Observation Domain within a Transport Session. This provides a mechanism for representing an IPFIX Information Model inline within an IPFIX Message stream. Data Records described by this template are referred to as Information Element type records.
In deployments in which interoperability across vendor implementations of IPFIX is important, an Exporting Process exporting data using Templates containing enterprise-specific Information Elements SHOULD export an Information Element type record for each enterprise-specific Information Element it exports. Collecting Processes MAY use these type records to improve handling of unknown enterprise-specific Information Elements. Exporting Processes using enterprise-specific Information Elements to implement proprietary features MAY omit type records for those Information Elements.
Information Element type records MUST be handled by Collecting Processes as scoped to the Transport Session in which they are sent; this facility is not intended to provide a method for the permanent definition of Information Elements.
Similarly, for security reasons, type information for a given Information Element MUST NOT be redefined by Information Element type records, and a Collecting Process MUST NOT allow an Information
Boschi, et al. Standards Track [Page 10]
RFC 5610 IPFIX Type Information July 2009
Element type record to replace its own internal definition of an Information Element. Information Element type records SHOULD NOT be duplicated in a given Observation Domain within a Transport Session. Once an Information Element type record has been exported for a given Information Element within a given Transport Session, all subsequent type records for that Information Element MUST be identical. Information Elements for which a Collecting Process receives conflicting semantic or type information MUST be ignored.
Note that while this template MAY be used to export information about any Information Element, including those registered with IANA, Exporting Processes SHOULD NOT export any type records that could be reasonably assumed to duplicate type information available at the Collecting Process. This mechanism is not intended as a replacement for Exporting and Collecting Processes keeping up to date with changes to the IANA registry; such an update mechanism is out of scope for this document.
The template SHOULD contain the Information Elements in Table 4, below, as defined in the PSAMP Information Model [RFC5477] and in this document, above.
+-------------------------------+-----------------------------------+ | IE | Description | +-------------------------------+-----------------------------------+ | informationElementId [scope] | The Information Element | | | identifier of the Information | | | Element described by this type | | | record. This Information Element | | | MUST be defined as a Scope Field. | | | See the PSAMP Information Model | | | [RFC5477] for a definition of | | | this field. | | privateEnterpriseNumber | The Private Enterprise number of | | [scope] | the Information Element described | | | by this type record. This | | | Information Element MUST be | | | defined as a Scope Field. | | informationElementDataType | The storage type of the specified | | | Information Element. | | informationElementSemantics | The semantic type of the | | | specified Information Element. | | informationElementUnits | The units of the specified | | | Information Element. This | | | element SHOULD be omitted if the | | | Information Element is a unitless | | | quantity, or a not a quantity or | | | counter. |
Boschi, et al. Standards Track [Page 11]
RFC 5610 IPFIX Type Information July 2009
+-------------------------------+-----------------------------------+ | IE (Continued) | Description (Continued) | +-------------------------------+-----------------------------------+ | informationElementRangeBegin | The low end of the range of | | | acceptable values for the | | | specified Information Element. | | | This element SHOULD be omitted if | | | the beginning of the Information | | | Element's acceptable range is | | | defined by its data type. | | informationElementRangeEnd | The high end of the range of | | | acceptable values for the | | | specified Information Element. | | | This element SHOULD be omitted if | | | the end Information Element's | | | acceptable range is defined by | | | its data type. | | informationElementName | The name of the specified | | | Information Element. | | informationElementDescription | A human-readable description of | | | the specified Information | | | Element. This element MAY be | | | omitted in the interest of export | | | efficiency. | +-------------------------------+-----------------------------------+
Note that the informationElementSemantics values defined in Section 3.2 of [RFC5102] are primarily intended to differentiate semantic interpretation of numeric values, and that not all combinations of the informationElementDataType and informationElementSemantics Information Elements are valid; e.g., a counter cannot be encoded as an IPv4 address. The following are acceptable values of informationElementSemantics:
o Any value is valid for unsigned informationElementDataType values ("unsigned8", "unsigned16", "unsigned32", or "unsigned64").
o Any value except "flags" is valid for signed informationElementDataType values ("signed8", "signed16", "signed32", or "signed64").
o Any value except "identifier" or "flags" is valid for floating- point informationElementDataType values ("float32" or "float64").
Boschi, et al. Standards Track [Page 12]
RFC 5610 IPFIX Type Information July 2009
o Only "default" is valid for all other informationElementDataType values ("octetArray", "boolean", "macAddress", "string", "dateTimeSeconds", "dateTimeMilliseconds", "dateTimeMicroseconds", "dateTimeNanoseconds", "ipv4Address", or "ipv6Address").
Information Element type records containing invalid combinations of informationElementSemantics and informationElementDataType MUST NOT be sent by Exporting Processes, and MUST be ignored by Collecting Processes.
Future Standards Actions that modify the Information Element Data Type subregistry or the Information Element Semantics subregistry should contain a Data Type and Semantics Restrictions section such as this one to define allowable combinations of type and semantics information.
The same security considerations as for the IPFIX Protocol [RFC5101] apply.
In addition, attention must be paid to the handling of Information Element type records at the Collecting Process. Type information precedence rules defined above (a Collecting Process' current knowledge overrides type records; types are not redefinable during a session) are designed to minimize the opportunity for an attacker to maliciously redefine the data model.
Note that Information Element type records may contain two strings describing Information Elements: informationElementName and informationElementDescription. IPFIX strings on the wire are length- prefixed and UTF-8 [RFC3629] encoded, most often within an IPFIX variable-length Information Element, which mitigates the risk of unterminated-string attacks against IPFIX Collecting Processes. However, care should still be taken when handling strings within the type system of the Collecting Process.
First, Collecting Processes should pay particular attention to buffer sizes converting between length-prefixed and null-terminated strings. Exporting Processes MUST NOT export, and Collecting Processes MUST ignore, any informationElementName or informationElementDescription content that contains null characters (U+0000) in order to ensure buffer and string lengths are consistent.
Also, note that there is no limit to IPFIX string length beyond that inherent in the protocol. The maximum IPFIX string length is 65512 octets (maximum message length (65535), minus message header (16), minus set header (4), minus long variable length field (3)).
Boschi, et al. Standards Track [Page 13]
RFC 5610 IPFIX Type Information July 2009
Specifically, although the informationElementName of all IANA Information Elements at the time of this writing is less than about 40 octets, and the informationElementDescription is less than 4096 octets, either of these Information Elements may contain strings up to 65512 octets long.
This document specifies several new IPFIX Information Elements in the IPFIX Information Element registry as defined in Section 3 above. IANA has assigned the following Information Element numbers for their respective Information Elements as specified below:
o Information Element Number 339 for the informationElementDataType Information Element
o Information Element Number 340 for the informationElementDescription Information Element
o Information Element Number 341 for the informationElementName Information Element
o Information Element Number 342 for the informationElementRangeBegin Information Element
o Information Element Number 343 for the informationElementRangeEnd Information Element
o Information Element Number 344 for the informationElementSemantics Information Element
o Information Element Number 345 for the informationElementUnits Information Element
o Information Element Number 346 for the privateEnterpriseNumber Information Element
IANA has created an Information Element Data Type subregistry for the values defined for the informationElementDataType Information Element. Entries may be added to this subregistry subject to a Standards Action [RFC5226].
IANA has created an Information Element Semantics subregistry for the values defined for the informationElementSemantics Information Element. Entries may be added to this subregistry subject to a Standards Action [RFC5226].
Boschi, et al. Standards Track [Page 14]
RFC 5610 IPFIX Type Information July 2009
IANA has created an Information Element Units subregistry for the values defined for the informationElementUnits Information Element. Entries may be added to this subregistry on an Expert Review [RFC5226] basis.
Thanks to Paul Aitken and Gerhard Muenz for the detailed reviews, and to David Moore for first raising this issue to the IPFIX mailing list. Thanks to the PRISM project for its support of this work.
[RFC5101] Claise, B., "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information", RFC 5101, January 2008.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. Meyer, "Information Model for IP Flow Information Export", RFC 5102, January 2008.
[RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. Carle, "Information Model for Packet Sampling Exports", RFC 5477, March 2009.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.
[RFC2277] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998.
[RFC2482] Whistler, K. and G. Adams, "Language Tagging in Unicode Plain Text", RFC 2482, January 1999.
[RFC4646] Phillips, A. and M. Davis, "Tags for Identifying Languages", BCP 47, RFC 4646, September 2006.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
The following example illustrates how the type information extension mechanism defined in this document may be used to describe the semantics of enterprise-specific Information Elements. The Information Elements used in this example are as follows:
o initialTCPFlags, an example private IE 14, 1 octet, the TCP flags on the first TCP packet in the flow.
o unionTCPFlags, an example private IE 15, 1 octet, the union of the TCP flags on all packets after the first TCP packet in the flow.
An Exporting Process exporting flows containing these Information Elements might use a Template like the following:
However, a Collecting Process receiving Data Sets described by this Template can only treat the enterprise-specific Information Elements as opaque octets; specifically, there is no hint to the collector that they contain flag information. To use the type information extension mechanism to address this problem, the Exporting Process would first export the Information Element Type Options Template described in Section 3.9 above:
Figure 2: Example Information Element Type Options Template
Then, the Exporting Process would export two records described by the Example Information Element Type Options Template to describe the enterprise-specific Information Elements: