Netnews Article Format
The header fields a Usenet article carries, and the rules that make one article readable by every server.
Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews Article Format", RFC 5536, DOI 10.17487/RFC5536, November 2009, <https://www.rfc-editor.org/info/rfc5536>.
@misc{rfc5536,
series = {Request for Comments},
number = 5536,
howpublished = {RFC 5536},
publisher = {RFC Editor},
doi = {10.17487/RFC5536},
url = {https://www.rfc-editor.org/info/rfc5536},
author = {K. Murchison and C. Lindsey and D. Kohn},
title = {{Netnews Article Format}},
pagetotal = 36,
year = 2009,
month = nov,
} RFC text
Contents
Network Working Group K. Murchison, Ed.
Request for Comments: 5536 Carnegie Mellon University
Obsoletes: 1036 C. Lindsey
Category: Standards Track University of Manchester
D. Kohn
Healing Thresholds
November 2009
Netnews Article Format
Abstract
This document specifies the syntax of Netnews articles in the context
of the Internet Message Format (RFC 5322) and Multipurpose Internet
Mail Extensions (MIME) (RFC 2045). This document obsoletes RFC 1036,
providing an updated specification to reflect current practice and
incorporating incremental changes specified in other documents.
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Table of Contents
1. Introduction ....................................................4
1.1. Basic Concepts .............................................4
1.2. Scope ......................................................4
1.3. Requirements Notation ......................................4
1.4. Syntax Notation ............................................5
1.5. Definitions ................................................5
1.6. Structure of This Document .................................7
2. Format ..........................................................7
2.1. Base .......................................................7
2.2. Header Fields ..............................................8
2.3. MIME Conformance ...........................................9
3. News Header Fields ..............................................9
3.1. Mandatory Header Fields ...................................10
3.1.1. Date ...............................................11
3.1.2. From ...............................................11
3.1.3. Message-ID .........................................11
3.1.4. Newsgroups .........................................13
3.1.5. Path ...............................................14
3.1.6. Subject ............................................16
3.2. Optional Header Fields ....................................16
3.2.1. Approved ...........................................17
3.2.2. Archive ............................................17
3.2.3. Control ............................................17
3.2.4. Distribution .......................................18
3.2.5. Expires ............................................19
3.2.6. Followup-To ........................................19
3.2.7. Injection-Date .....................................20
3.2.8. Injection-Info .....................................20
3.2.9. Organization .......................................22
3.2.10. References ........................................22
3.2.11. Summary ...........................................23
3.2.12. Supersedes ........................................23
3.2.13. User-Agent ........................................23
3.2.14. Xref ..............................................24
3.3. Obsolete Header Fields ....................................25
3.3.1. Lines ..............................................25
4. Internationalization Considerations ............................25
5. Security Considerations ........................................25
6. IANA Considerations ............................................26
7. References .....................................................31
7.1. Normative References ......................................31
7.2. Informative References ....................................32
Appendix A. Acknowledgments ......................................34
Appendix B. Differences from RFC 1036 and Its Derivatives ........34
Appendix C. Differences from RFC 5322 ............................35
1. Introduction
1.1. Basic Concepts
"Netnews" is a set of protocols for generating, storing, and
retrieving news "articles" (whose format is a subset of that for
Email messages), and for exchanging them amongst a readership that is
potentially widely distributed. It is organized around "newsgroups",
with the expectation that each reader will be able to see all
articles posted to each newsgroup in which he participates. These
protocols most commonly use a flooding algorithm, which propagates
copies throughout a network of participating servers. Typically,
only one copy is stored per server, and each server makes it
available on demand to readers who are able to access that server.
1.2. Scope
This document specifies the syntax of Netnews articles in the context
of the Internet Message Format [RFC5322] and Multipurpose Internet
Mail Extensions (MIME) [RFC2045]. This document obsoletes [RFC1036],
updating the syntax of Netnews articles to reflect current practice
and incorporating changes and clarifications specified in other
documents such as [Son-of-1036].
This is the first in a set of documents that obsolete [RFC1036].
This document focuses on the syntax and semantics of Netnews
articles. [RFC5537] is also a Standards Track document and describes
the protocol issues of Netnews articles, independent of transport
protocols such as the Network News Transfer Protocol (NNTP)
[RFC3977]. [USEAGE], "Usenet Best Practice", describes
implementation recommendations to improve interoperability and
usability.
This specification is intended as a definition of what article
content format is to be passed between systems. Although many news
systems locally store articles in this format (which eliminates the
need for translation between formats), local storage is outside of
the scope of this standard.
1.3. Requirements Notation
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
1.4. Syntax Notation
Header fields defined in this specification use the Augmented Backus-
Naur Form (ABNF) notation (including the Core Rules) specified in
[RFC5234] as well as many constructs defined in [RFC5322], [RFC2045]
as updated by [RFC2231], and [RFC3986]. Specifically:
token = <see RFC 2045 Section 5.1>
value = <see RFC 2045 Section 5.1>
parameter = <see RFC 2231 Section 7>
attribute = <see RFC 2231 Section 7>
FWS = <see RFC 5322 Section 3.2.2>
comment = <see RFC 5322 Section 3.2.2>
CFWS = <see RFC 5322 Section 3.2.2>
atext = <see RFC 5322 Section 3.2.3>
dot-atom-text = <see RFC 5322 Section 3.2.3>
phrase = <see RFC 5322 Section 3.2.5>
date-time = <see RFC 5322 Section 3.3>
mailbox = <see RFC 5322 Section 3.4>
mailbox-list = <see RFC 5322 Section 3.4>
address-list = <see RFC 5322 Section 3.4>
body = <see RFC 5322 Section 3.5>
fields = <see RFC 5322 Section 3.6>
IPv6address = <see RFC 3986 Section 3.2.2>
IPv4address = <see RFC 3986 Section 3.2.2>
ALPHA = <see RFC 5234 Appendix B.1>
CRLF = <see RFC 5234 Appendix B.1>
DIGIT = <see RFC 5234 Appendix B.1>
DQUOTE = <see RFC 5234 Appendix B.1>
SP = <see RFC 5234 Appendix B.1>
VCHAR = <see RFC 5234 Appendix B.1>
Additionally, Section 3.1.3 specifies a stricter definition of
<msg-id> than the syntax in Section 3.6.4 of [RFC5322].
1.5. Definitions
An "article" is the unit of Netnews, analogous to an [RFC5322]
"message". A "proto-article" is one that has not yet been injected
into the news system. In contrast to an article, a proto-article may
lack some mandatory header fields.
A "message identifier" (Section 3.1.3) is a unique identifier for an
article, usually supplied by the user agent that posted it or,
failing that, by the "news server". It distinguishes the article
from every other article ever posted anywhere. Articles with the
same message identifier are treated as if they are the same article
regardless of any differences in the body or header fields.
A "newsgroup" is a forum having a name and that is intended for
articles on a specific topic. An article is "posted to" a single
newsgroup or several newsgroups. When an article is posted to more
than one newsgroup, it is said to be "crossposted"; note that this
differs from posting the same text as part of each of several
articles, one per newsgroup.
A newsgroup may be "moderated", in which case submissions are not
posted directly, but mailed to a "moderator" for consideration and
possible posting. Moderators are typically human but may be
implemented partially or entirely in software.
A "poster" is the person or software that composes and submits a
potentially compliant article to a user agent.
A "reader" is the person or software reading Netnews articles.
A "followup" is an article containing a response to the contents of
an earlier article, its "precursor". Every followup includes a
"References" header field identifying that precursor (but note that
non-followup articles may also use a References header field).
A "control message" is an article that is marked as containing
control information; a news server receiving such an article may
(subject to the policies observed at that site) take actions beyond
just filing and passing on the article.
A news server is software that may accept articles from a user agent,
and/or make articles available to user agents, and/or exchange
articles with other news servers.
A "user agent" is software that may help posters submit proto-
articles to a news server, and/or fetch articles from a news server
and present them to a reader, and/or assist the reader in creating
articles and followups.
The generic term "agent" is used when describing requirements that
apply to both user agents and news servers.
An agent is said to "generate" a construct if it did not exist before
the agent created it. Examples are when a user agent creates a
message from text and addressing information supplied by a user, or
when a news server creates an "Injection-Info" header field for a
newly posted message.
An agent is said to "accept" a construct if some other entity
generates it and passes it to the agent in question, and the agent
processes it without treating it as a format or protocol error.
1.6. Structure of This Document
This document uses a cite-by-reference methodology, rather than
repeating the contents of other standards, which could otherwise
result in subtle differences and interoperability challenges.
Although this document is as a result rather short, it requires
complete understanding and implementation of the normative references
to be compliant.
Section 2 defines the format of Netnews articles. Section 3 details
the header fields necessary to make an article suitable for the
Netnews environment.
2. Format
2.1. Base
An article is said to be conformant to this specification if it
conforms to the format specified in Section 3 of [RFC5322] and to the
additional requirements of this specification.
An article that uses the obsolete syntax specified in Section 4 of
[RFC5322] is NOT conformant to this specification, except for the
following two cases:
o Articles are conformant if they use the <obs-phrase> construct
(use of a phrase like "John Q. Public" without the use of quotes,
see Section 4.1 of [RFC5322]), but agents MUST NOT generate
productions of such syntax.
o Articles are conformant if they use the "GMT" <zone>, as specified
in Section 3.1.1.
This document, and specifications that build upon it, specify how to
handle conformant articles. Handling of non-conformant articles is
outside the scope of this specification.
Agents conforming to this specification MUST generate only conformant
articles.
The text below uses ABNF to specify restrictions on the syntax
specified in [RFC5322]; this grammar is intended to be more
restrictive than the [RFC5322] grammar. Articles must conform to the
ABNF specified in [RFC5322] and also to the restrictions specified
here, both those that are expressed as text and those that are
expressed as ABNF.
NOTE: Other specifications use the term "header" as a synonym for
what [RFC5322] calls "header field". This document follows the
terminology in Section 2 of [RFC5322] in using the terms "line",
"header field", "header field name", "header field body", and
"folding", based on a belief that consistent terminology among
specifications that depend on each other makes the specifications
easier to use in the long run.
2.2. Header Fields
All header fields in a Netnews article are compliant with [RFC5322];
this specification, however, is less permissive in what can be
generated and accepted by agents. The syntax allowed for Netnews
article headers is a strict subset of the Internet Message Format
headers, making all headers compliant with this specification
inherently compliant with [RFC5322]. Note however that the converse
is not guaranteed to be true in all cases.
General rules that apply to all header fields (even those documented
in [RFC5322] and [RFC2045]) are listed below, and those that apply to
specific header fields are described in the relevant sections of this
document.
o All agents MUST generate header fields so that at least one space
immediately follows the ':' separating the header field name and
the header field body (for compatibility with deployed software,
including NNTP [RFC3977] servers). News agents MAY accept header
fields that do not contain the required space.
o Every line of a header field body (including the first and any
that are subsequently folded) MUST contain at least one non-
whitespace character.
NOTE: This means that no header field body defined by or
referenced by this document can be empty. As a result, rather
than using the <unstructured> syntax from Section 3.2.5 of
[RFC5322], this document uses a stricter definition:
unstructured = *WSP VCHAR *( [FWS] VCHAR ) *WSP
NOTE: The [RFC5322] specification sometimes uses [FWS] at the
beginning or end of ABNF describing header field content. This
specification uses *WSP in such cases, also in cases where this
specification redefines constructs from [RFC5322]. This is
done for consistency with the restriction described here, but
the restriction applies to all header fields, not just those
where ABNF is defined in this document.
o Compliant software MUST NOT generate (but MAY accept) header field
lines of more than 998 octets. This is the only limit on the
length of a header field line prescribed by this standard.
However, specific rules to the contrary may apply in particular
cases (for example, according to [RFC2047], lines of a header
field containing encoded words are limited to 76 octets).
[USEAGE] includes suggested limits for convenience of display by
user agents.
NOTE: As stated in [RFC5322], there is NO restriction on the
number of lines into which a header field may be split, and
hence there is NO restriction on the total length of a header
field (in particular it may, by suitable folding, be made to
exceed the 998-octet restriction pertaining to a single header
field line).
o The character set for header fields is US-ASCII. Where the use of
non-ASCII characters is required, they MUST be encoded using the
MIME mechanisms defined in [RFC2047] and [RFC2231].
2.3. MIME Conformance
User agents MUST meet the definition of MIME conformance in [RFC2049]
and MUST also support [RFC2231]. This level of MIME conformance
provides support for internationalization and multimedia in message
bodies [RFC2045], [RFC2046], and [RFC2231], and support for
internationalization of header fields [RFC2047] and [RFC2231]. Note
that [Errata] currently exist for [RFC2045], [RFC2046], [RFC2047] and
[RFC2231].
For the purposes of Section 5 of [RFC2047], all header fields defined
in Section 3 of this standard are to be considered as "extension
message header fields", permitting the use of [RFC2047] encodings
within any <unstructured> header field, or within any <comment> or
<phrase> permitted within any structured header field.
User agents MAY accept and generate other MIME extension header
fields, and in particular SHOULD accept Content-Disposition [RFC2183]
and Content-Language [RFC3282].
3. News Header Fields
The following news header fields extend those defined in Section 3.6
of [RFC5322]:
fields =/ *( approved /
archive /
control /
distribution /
expires /
followup-to /
injection-date /
injection-info /
lines /
newsgroups /
organization /
path /
summary /
supersedes /
user-agent /
xref )
Each of these header fields MUST NOT occur more than once in a news
article.
The following header fields defined in this document do not allow
<comment>s (i.e., they use FWS rather than CFWS).
Control
Distribution
Followup-To
Lines
Newsgroups
Path
Supersedes
Xref
This also applies to the following header field defined in [RFC5322]:
Message-ID
Most of these header fields are mainly of interest to news servers,
and news servers often need to process these fields very rapidly.
Thus, some header fields prohibit <comment>s.
3.1. Mandatory Header Fields
Each Netnews article conformant with this specification MUST have
exactly one of each of the following header fields: Date, From,
Message-ID, Newsgroups, Path, and Subject.
3.1.1. Date
The Date header field is the same as that specified in Sections 3.3
and 3.6.1 of [RFC5322], with the added restrictions detailed above in
Section 2.2. However, the use of "GMT" as a time zone (part of
<obs-zone>), although deprecated, is widespread in Netnews articles
today. Therefore, agents MUST accept <date-time> constructs that use
the "GMT" zone.
orig-date = "Date:" SP date-time CRLF
NOTE: This specification does not change [RFC5322], which says
that agents MUST NOT generate <date-time> constructs that include
any zone names defined by <obs-zone>.
Software that accepts dates with unknown timezones SHOULD treat such
timezones as equivalent to "-0000" when comparing dates, as specified
in Section 4.3 of [RFC5322].
Also note that these requirements apply wherever <date-time> is used,
including Injection-Date and Expires (Sections 3.2.7 and 3.2.5,
respectively).
3.1.2. From
The From header field is the same as that specified in Section 3.6.2
of [RFC5322], with the added restrictions detailed above in
Section 2.2.
from = "From:" SP mailbox-list CRLF
3.1.3. Message-ID
The Message-ID header field contains a unique message identifier.
Netnews is more dependent on message identifier uniqueness and fast
comparison than Email is, and some news software and standards
[RFC3977] might have trouble with the full range of possible
<msg-id>s permitted by [RFC5322]. This section therefore restricts
the syntax of <msg-id> as compared to Section 3.6.4 of [RFC5322].
The global uniqueness requirement for <msg-id> in [RFC5322] is to be
understood as applying across all protocols using such message
identifiers, and across both Email and Netnews in particular.
message-id = "Message-ID:" SP *WSP msg-id *WSP CRLF
msg-id = "<" msg-id-core ">"
; maximum length is 250 octets
msg-id-core = id-left "@" id-right
id-left = dot-atom-text
id-right = dot-atom-text / no-fold-literal
no-fold-literal = "[" *mdtext "]"
mdtext = %d33-61 / ; The rest of the US-ASCII
%d63-90 / ; characters not including
%d94-126 ; ">", "[", "]", or "\"
The <msg-id> MUST NOT be more than 250 octets in length.
NOTE: The length restriction ensures that systems that accept
message identifiers as a parameter when referencing an article
(e.g., [RFC3977]) can rely on a bounded length.
Observe that <msg-id> includes the < and >.
Observe also that in contrast to the corresponding header field in
[RFC5322]:
o The syntax does not allow comments within the Message-ID header
field.
o There is no possibility for ">" or WSP to occur inside a <msg-id>.
o Even though commonly derived from <domain>s, <id-rights>s are
case-sensitive (and thus, once created, are not to be altered
during subsequent transmission or copying)
This is to simplify processing by news servers and to ensure
interoperability with existing implementations and compliance with
[RFC3977]. A simple comparison of octets will always suffice to
determine the identity of two <msg-id>s.
Also note that this updated ABNF applies wherever <msg-id> is used,
including the References header field discussed in Section 3.2.10 and
the Supersedes header field discussed in Section 3.2.12.
Some software will try to match the <id-right> of a <msg-id> in a
case-insensitive fashion; some will match it in a case-sensitive
fashion. Implementations MUST NOT generate a Message-ID where the
only difference from another Message-ID is the case of characters in
the <id-right> part.
When generating a <msg-id>, implementations SHOULD use a domain name
as the <id-right>.
NOTE: Section 3.6.4 of [RFC5322] recommends that the <id-right>
should be a domain name or a domain literal. Domain literals are
troublesome since many IP addresses are not globally unique;
domain names are more likely to generate unique Message-IDs.
3.1.4. Newsgroups
The Newsgroups header field specifies the newsgroup(s) to which the
article is posted.
newsgroups = "Newsgroups:" SP newsgroup-list CRLF
newsgroup-list = *WSP newsgroup-name
*( [FWS] "," [FWS] newsgroup-name ) *WSP
newsgroup-name = component *( "." component )
component = 1*component-char
component-char = ALPHA / DIGIT / "+" / "-" / "_"
Not all servers support optional FWS in the list of newsgroups. In
particular, folding the Newsgroups header field over several lines
has been shown to harm propagation significantly. Optional FWS in
the <newsgroup-list> SHOULD NOT be generated, but MUST be accepted.
A <component> SHOULD NOT consist solely of digits and SHOULD NOT
contain uppercase letters. Such <component>s MAY be used only to
refer to existing groups that do not conform to this naming scheme,
but MUST NOT be used otherwise.
NOTE: All-digit <component>s conflict with one widely used storage
scheme for articles. Mixed-case groups cause confusion between
systems with case-sensitive matching and systems with case-
insensitive matching of <newsgroup-name>s.
<component>s beginning with underline ("_") are reserved for use by
future versions of this standard and SHOULD NOT be generated by user
agents (whether in header fields or in newgroup control messages as
defined by [RFC5537]). However, such names MUST be accepted by news
servers.
<component>s beginning with "+" and "-" are reserved for private use
and SHOULD NOT be generated by user agents (whether in header fields
or in newgroup control messages [RFC5537]) without a private prior
agreement to do so. However, such names MUST be accepted by news
servers.
The following <newsgroup-name>s are reserved and MUST NOT be used as
the name of a newsgroup:
o Groups whose first (or only) <component> is "example"
o The group "poster"
The following <newsgroup-name>s have been used for specific purposes
in various implementations and protocols and therefore MUST NOT be
used for the names of normal newsgroups. They MAY be used for their
specific purpose or by local agreement.
o Groups whose first (or only) component is "to"
o Groups whose first (or only) component is "control"
o Groups that contain (or consist only of) the component "all"
o Groups that contain (or consist only of) the component "ctl"
o The group "junk"
NOTE: "example.*" is reserved for examples in this and other
standards; "poster" has a special meaning in the Followup-To
header field; "to.*" is reserved for certain point-to-point
communications in conjunction with the "ihave" control message as
defined in [RFC5537]; "control.*" and "junk" have special meanings
in some news servers; "all" is used as a wildcard in some
implementations; and "ctl" was formerly used to indicate a
<control-command> within the Newsgroups header field.
3.1.5. Path
The Path header field indicates the route taken by an article since
its injection into the Netnews system. Each agent that processes an
article is required to prepend at least one <path-identity> to this
header field body. This is primarily so that news servers are able
to avoid sending articles to sites already known to have them, in
particular the site they came from. Additionally, it permits
gathering statistics and tracing the route articles take in moving
over the network.
path = "Path:" SP *WSP path-list tail-entry *WSP CRLF
path-list = *( path-identity [FWS] [path-diagnostic] "!" )
path-diagnostic = diag-match / diag-other / diag-deprecated
diag-match = "!" ; another "!"
diag-other = "!." diag-keyword [ "." diag-identity ] [FWS]
diag-deprecated = "!" IPv4address [FWS]
diag-keyword = 1*ALPHA ; see [RFC5537]
diag-identity = path-identity / IPv4address / IPv6address
tail-entry = path-nodot
; may be the string "not-for-mail"
path-identity = ( 1*( label "." ) toplabel ) / path-nodot
path-nodot = 1*( alphanum / "-" / "_" ) ; legacy names
label = alphanum [ *( alphanum / "-" ) alphanum ]
toplabel = ( [ label *( "-" ) ] ALPHA *( "-" ) label ) /
( label *( "-" ) ALPHA [ *( "-" ) label ] ) /
( label 1*( "-" ) label )
alphanum = ALPHA / DIGIT ; compare [RFC3696]
A <path-identity> is a name identifying a site. It takes the form of
a domain name having two or more components separated by dots, or a
single name with no dots (<path-nodot>).
Each <path-identity> in the <path-list> (which does not include the
<tail-entry>) indicates, from right to left, the successive agents
through which the article has passed. The use of the <diag-match>,
which appears as "!!", indicates that the agent to its left verified
the identity of the agent to its right before accepting the article
(whereas the <path-delimiter> "!" implies no such claim).
NOTE: Historically, the <tail-entry> indicated the name of the
sender. If not used for this purpose, the string "not-for-mail"
is often used instead (since at one time the whole path could be
used as a mail address for the sender).
NOTE: Although case-insensitive, it is intended that the
<diag-keyword>s should be in uppercase, to distinguish them from
the <path-identity>s, which are traditionally in lowercase.
A <path-diagnostic> is an item inserted into the Path header field
for purposes other than to indicate the name of a site. The use of
these is described in [RFC5537].
NOTE: One usage of a <path-diagnostic> is to record an IP address.
The fact that <IPv6address>es are allowed means that the colon (:)
is permitted; note that this may cause interoperability problems
at older sites that regard ":" as a <path-delimiter> and have
neighbors whose names have 4 or fewer characters, and where all
the characters are valid HEX digits.
NOTE: Although <IPv4address>es have occasionally been used in the
past (usually with a diagnostic intent), their continued use is
deprecated (though it is still acceptable in the form of the
<diag-deprecated>).
3.1.6. Subject
The Subject header field is the same as that specified in Section
3.6.5 of [RFC5322], with the added restrictions detailed above in
Section 2.2. Further discussion of the content of the Subject header
field appears in [RFC5537] and [USEAGE].
subject = "Subject:" SP unstructured CRLF
3.2. Optional Header Fields
None of the header fields appearing in this section are required to
appear in every article, but some of them may be required in certain
types of articles. Further discussion of these requirements appears
in [RFC5537] and [USEAGE].
The header fields Comments, Keywords, Reply-To, and Sender are used
in Netnews articles in the same circumstances and with the same
meanings as those specified in [RFC5322], with the added restrictions
detailed above in Section 2.2. Multiple occurrences of the Keywords
header field are not permitted.
comments = "Comments:" SP unstructured CRLF
keywords = "Keywords:" SP phrase *("," phrase) CRLF
reply-to = "Reply-To:" SP address-list CRLF
sender = "Sender:" SP mailbox CRLF
The MIME header fields MIME-Version, Content-Type, Content-Transfer-
Encoding, Content-Disposition, and Content-Language are used in
Netnews articles in the same circumstances and with the same meanings
as those specified in [RFC2045], [RFC2183], and [RFC3282], with the
added restrictions detailed above in Section 2.2.
All remaining news header fields are described below.
3.2.1. Approved
The Approved header field indicates the mailing addresses (and
possibly the full names) of the persons or entities approving the
article for posting. Its principal uses are in moderated articles
and in group control messages; see [RFC5537].
approved = "Approved:" SP mailbox-list CRLF
3.2.2. Archive
The Archive header field provides an indication of the poster's
intent regarding preservation of the article in publicly accessible
long-term or permanent storage.
archive = "Archive:" SP [CFWS] ("no" / "yes")
*( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF
archive-param = parameter
The presence of an Archive header field in an article with a field
body of "no" indicates that the poster does not permit redistribution
from publicly accessible long-term or permanent archives. A field
body of "yes" indicates that the poster permits such redistribution.
No <parameter>s are currently defined; if present, they can be
ignored. Further discussion of the use of the Archive header field
appears in [USEAGE].
3.2.3. Control
The Control header field marks the article as a control message and
specifies the desired actions (in addition to the usual actions of
storing and/or relaying the article).
control = "Control:" SP *WSP control-command *WSP CRLF
control-command = verb *( 1*WSP argument )
verb = token
argument = 1*( %x21-7E )
The verb indicates what action should be taken, and the argument(s)
(if any) supply details. In some cases, the <body> (as defined in
[RFC5322]) of the article may also contain details. The legal verbs
and respective arguments are discussed in the companion document,
[RFC5537].
An article with a Control header field MUST NOT also have a
Supersedes header field.
3.2.4. Distribution
The Distribution header field specifies geographic or organizational
limits on an article's propagation.
distribution = "Distribution:" SP dist-list CRLF
dist-list = *WSP dist-name
*( [FWS] "," [FWS] dist-name ) *WSP
dist-name = ALPHA / DIGIT
*( ALPHA / DIGIT / "+" / "-" / "_" )
The <dist-name>s "world" and "local" are reserved. "world" indicates
unlimited distribution and SHOULD NOT be used explicitly, since it is
the default when the Distribution header field is absent entirely.
"local" is reserved for indicating distribution only to the local
site, as defined by local software configuration.
"All" MUST NOT be used as a <dist-name>. <dist-name>s SHOULD contain
at least three characters, except when they are two-letter country
codes drawn from [ISO3166-1]. <dist-name>s are case-insensitive
(i.e., "US", "Us", "uS", and "us" all specify the same distribution).
Optional FWS in the <dist-list> SHOULD NOT be generated, but MUST be
accepted.
3.2.5. Expires
The Expires header field specifies a date and time when the poster
deems the article to be no longer relevant and could usefully be
removed ("expired").
NOTE: This header field is useful when the poster desires an
unusually long or an unusually short expiry time.
expires = "Expires:" SP date-time CRLF
See the remarks under Section 3.1.1 regarding the syntax of
<date-time> and the requirements and recommendations to which it is
subject.
NOTE: The Expires header field is also sometimes used in Email
with a similar meaning; see [RFC2156].
3.2.6. Followup-To
The Followup-To header field specifies to which newsgroup(s) the
poster has requested that followups are to be posted. The
Followup-To header field SHOULD NOT appear in a message, unless its
content is different from the content of the Newsgroups header field.
followup-to = "Followup-To:" SP ( newsgroup-list / poster-text )
CRLF
poster-text = *WSP %d112.111.115.116.101.114 *WSP
; "poster" in lowercase
The syntax is the same as that of the Newsgroups (Section 3.1.4)
header field, with the exception that the keyword "poster" requests
that followups should be emailed directly to the article's poster
(using the addresses contained in the Reply-To header field if one
exists, otherwise using the addresses contained in the From header
field) rather than posted to any newsgroups. Agents MUST generate
the keyword "poster" in lowercase, but MAY choose to recognize case-
insensitive forms such as "Poster".
As in the Newsgroups (Section 3.1.4) header field, optional FWS in
the <newsgroup-list> SHOULD NOT be generated, but MUST be accepted.
3.2.7. Injection-Date
The Injection-Date header field contains the date and time that the
article was injected into the network. Its purpose is to enable news
servers, when checking for "stale" articles, to use a <date-time>
that was added by a news server at injection time rather than one
added by the user agent at message composition time.
This header field MUST be inserted whenever an article is injected.
However, software that predates this standard does not use this
header, and therefore agents MUST accept articles without the
Injection-Date header field.
injection-date = "Injection-Date:" SP date-time CRLF
See the remarks under Section 3.1.1 regarding the syntax of
<date-time> and the requirements and recommendations to which it is
subject.
NOTE: Since clocks on various agents are not necessarily
synchronized, the <date-time> in this header field might not be a
later value than that in the Date header field. Agents MUST NOT
alter a pre-existing Date header field when adding an Injection-
Date header field.
This header field is intended to replace the currently used but
undocumented "NNTP-Posting-Date" header field, whose use is now
deprecated.
3.2.8. Injection-Info
The Injection-Info header field contains information provided by the
injecting news server as to how an article entered the Netnews
system; it assists in tracing the article's true origin. It can also
specify one or more addresses where complaints concerning the poster
of the article may be sent.
injection-info = "Injection-Info:" SP [CFWS] path-identity
[CFWS] *( ";" [CFWS] parameter ) [CFWS] CRLF
NOTE: The syntax of <parameter> (Section 5.1 of [RFC2045], as
amended by [RFC2231]), taken in conjunction with the folding rules
of [RFC0822] (note: not [RFC2822] or [RFC5322]), effectively
allows [CFWS] to occur on either side of the "=" inside a
<parameter>.
The following table gives the <attribute> and the format of the
<value> for each <parameter> defined for use with this header field.
At most, one occurrence of each such <parameter> is allowed.
<attribute> format of <value>
-------------------- -----------------
"posting-host" a <host-value>
"posting-account" any <value>
"logging-data" any <value>
"mail-complaints-to" an <address-list>
where
host-value = dot-atom-text / IPv4address / IPv6address /
(dot-atom-text ":" ( IPv4address / IPv6address ))
NOTE: Since any such <host-value> or <address-list> also has to be
a syntactically correct <value>, it will usually be necessary to
encapsulate it as a <quoted-string>, for example:
posting-host = "posting.example.com:192.0.2.1"
Other <attribute>s SHOULD NOT be used unless defined in extensions to
this standard. If non-standards-based <attribute>s are used, they
MUST begin with an "x-".
Although comments and folding of whitespace are permitted throughout
the Injection-Info header field, folding SHOULD NOT be used within
any <parameter>. Folding SHOULD only occur before or after the ";"
separating <parameter>s, and comments SHOULD only be used following
the last <parameter>.
NOTE: Some of this information has previously been sent in non-
standardized header fields such as NNTP-Posting-Host, X-Trace,
X-Complaints-To, and others. Once a news server generates an
Injection-Info header field, it should have no need to send these
non-standard header fields.
The "posting-host" <parameter> specifies the Fully Qualified Domain
Name (FQDN) and/or IP address (IPv4address or IPv6address) of the
host from which the news server received the article.
NOTE: If the "posting-host" <parameter> fails to deterministically
identify the host (e.g., dynamic IP address allocation), the
"posting-account" or "logging-data" <parameter> may provide
additional information about the true origin of the article.
The "posting-account" <parameter> identifies the source from which
that news server received the article, in a notation that can be
interpreted by the news server administrator. This notation can
include any information the administrator deems pertinent. In order
to limit the exposure of personal data, it SHOULD be given in a form
that cannot be interpreted by other sites. However, to make it
useful for rate limiting and abuse detection, two messages posted
from the same source SHOULD have the same value of "posting-account",
and two messages from different sources SHOULD have differing values
of "posting-account". The exact definition of "source" is left to
the discretion of the news server administrator.
The "logging-data" <parameter> contains information (typically a
session number or other non-persistent means of identifying a posting
account) that will enable the true origin of the article to be
determined by reference to logging information kept by the news
server.
The "mail-complaints-to" <parameter> specifies one or more mailboxes
for sending complaints concerning the behavior of the poster of the
article.
It is a matter of local policy which of the above <parameter>s to
include. Some pieces of information have privacy implications; this
is discussed in [USEAGE].
3.2.9. Organization
The Organization header field is a short phrase identifying the
poster's organization.
organization = "Organization:" SP unstructured CRLF
NOTE: There is no "s" in Organization.
3.2.10. References
The References header field is the same as that specified in Section
3.6.4 of [RFC5322], with the added restrictions detailed above in
Section 2.2 and those listed below:
o The updated <msg-id> construct defined in Section 3.1.3 MUST be
used.
o Message identifiers MUST be separated with CFWS.
o Comments in CFWS between message identifiers can cause
interoperability problems, so comments SHOULD NOT be generated but
MUST be accepted.
references = "References:" SP [CFWS] msg-id *(CFWS msg-id)
[CFWS] CRLF
3.2.11. Summary
The Summary header field is a short phrase summarizing the article's
content.
summary = "Summary:" SP unstructured CRLF
3.2.12. Supersedes
The Supersedes header field contains a message identifier specifying
an article to be superseded upon the arrival of this one. An article
containing a Supersedes header field is equivalent to a "cancel"
[RFC5537] control message for the specified article, followed
immediately by the new article without the Supersedes header field.
supersedes = "Supersedes:" SP *WSP msg-id *WSP CRLF
NOTE: There is no "c" in Supersedes.
NOTE: The Supersedes header field defined here has no connection
with the Supersedes header field that sometimes appears in Email
messages converted from X.400 according to [RFC2156]; in
particular, the syntax here permits only one <msg-id> in contrast
to the multiple <msg-id>s in that Email version.
3.2.13. User-Agent
The User-Agent header field contains information about the user agent
(typically a newsreader) generating the article, for statistical
purposes and tracing of standards violations to specific software in
need of correction. It is intended that this header field be
suitable for use in Email.
user-agent = "User-Agent:" SP 1*product [CFWS] CRLF
product = [CFWS] token [ [CFWS] "/" product-version ]
product-version = [CFWS] token
This header field MAY contain multiple <product> tokens identifying
the user agent and any subproducts that form a significant part of
it, listed in order of their significance for identifying the
application.
NOTE: Some of this information has previously been sent in non-
standardized header fields such as X-Newsreader, X-Mailer,
X-Posting-Agent, X-Http-User-Agent, and others. Once a user agent
generates a User-Agent header field, it should have no need to
send these non-standard header fields.
NOTE: [RFC2616] describes a similar facility for the HTTP
protocol. The Netnews article format differs in that "{" and "}"
are allowed in tokens (<product> and <product-version>) and
comments are permitted wherever white space is allowed.
3.2.14. Xref
The Xref header field indicates where an article was filed by the
last news server to process it. User agents often use the
information in the Xref header field to avoid multiple processing of
crossposted articles.
xref = "Xref:" SP *WSP server-name
1*( FWS location ) *WSP CRLF
server-name = path-identity
location = newsgroup-name ":" article-locator
article-locator = 1*( %x21-27 / %x29-3A / %x3C-7E )
; US-ASCII printable characters
; except '(' and ';'
The <server-name> is included so that software can determine which
news server generated the header field. The locations specify where
the article is filed -- i.e., under which newsgroups (which may
differ from those in the Newsgroups header field), and where under
those newsgroups. The exact form of an <article-locator> is
implementation-specific.
NOTE: The traditional form of an <article-locator> (as required by
[RFC3977]) is a decimal number, with articles in each newsgroup
numbered consecutively starting from 1.
3.3. Obsolete Header Fields
The header fields Date-Received, Posting-Version, and Relay-Version
defined in [RFC0850], as well as Also-Control, Article-Names,
Article-Updates, and See-Also defined in [Son-of-1036] are declared
obsolete. See the cited specification documents for further
information on their original use.
These header fields MUST NOT be generated and SHOULD be ignored.
3.3.1. Lines
The Lines header field indicates the number of lines in the <body>
(as defined in [RFC5322]) of the article.
lines = "Lines:" SP *WSP 1*DIGIT *WSP CRLF
The line count is the number of CRLF separators in the <body>.
Historically, this header field was used by the NNTP [RFC3977]
overview facility, but its use for this purpose is now deprecated.
As a result, this header field is to be regarded as obsolescent, and
it is likely to be removed entirely in a future version of this
standard. All agents SHOULD ignore it and SHOULD NOT generate it.
4. Internationalization Considerations
Internationalization of Netnews article header fields and bodies is
provided using the MIME mechanisms discussed in Section 2.3. Note
that the generation of internationalized <newsgroup-name>s for use in
header fields is not addressed in this document.
5. Security Considerations
The Netnews article format specified in this document does not
provide any security services, such as confidentiality,
authentication of sender, or non-repudiation. Instead, such services
need to be layered above, using such protocols as S/MIME [RFC3851] or
PGP/MIME (Pretty Good Privacy / MIME) [RFC3156], or below, using
secure versions of news transport protocols. Additionally, several
currently non-standardized protocols such as [PGPVERIFY] may be
standardized in the near future.
Message identifiers (Section 3.1.3) in Netnews articles are required
to be unique; articles may be refused (in server-to-server transfer)
if the identifier has already been seen. If a malicious agent can
predict the identifier of an article, it can preempt the article by
posting its own article (possibly to a quite different group) with
the same message identifier, thereby preventing the target article
from propagating. Therefore, agents that generate message
identifiers for Netnews articles SHOULD ensure that they are
unpredictable.
MIME security considerations are discussed in [RFC2046]. Note that
the full range of encodings allowed for parameters in [RFC2046] and
[RFC2231] permits constructs that simple parsers may fail to parse
correctly; examples of hard-to-parse constructs are:
Content-Type: multipart/mixed
(; boundary=foo ; xyz=");bOuNdArY*=''next%20part(")
Content-Type: multipart/digest;
boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy
Such deficiencies in parsing may be used as part of an attack.
Further security considerations are discussed in [RFC5537].
6. IANA Considerations
IANA has registered the following header fields in the Permanent
Message Header Field Repository, in accordance with the procedures
set out in [RFC3864].
Header field name: Also-Control
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [Son-of-1036] (Section 6.15)
Header field name: Approved
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.1)
Header field name: Archive
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.2)
Header field name: Article-Names
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [Son-of-1036] (Section 6.17)
Header field name: Article-Updates
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [Son-of-1036] (Section 6.18)
Header field name: Comments
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2),
[RFC5322] (Section 3.6.5)
Header field name: Control
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.3)
Header field name: Date
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.1),
[RFC5322] (Section 3.6.1)
Header field name: Date-Received
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [RFC0850] (Section 2.2.4)
Header field name: Distribution
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.4)
Header field name: Expires
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.5)
Header field name: Followup-To
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.6)
Header field name: From
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.2),
[RFC5322] (Section 3.6.2)
Header field name: Injection-Date
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.7)
Header field name: Injection-Info
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.8)
Header field name: Keywords
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2),
[RFC5322] (Section 3.6.5)
Header field name: Lines
Applicable protocol: netnews
Status: deprecated
Author/change controller: IETF
Specification document(s): This document (Section 3.3.1)
Related information: [RFC3977] (Section 8.1)
Header field name: Message-ID
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.3)
Related information: [RFC5322] (Section 3.6.4)
Header field name: Newsgroups
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.4)
Header field name: NNTP-Posting-Date
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): none
Header field name: NNTP-Posting-Host
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [RFC2980] (Section 3.4.1)
Header field name: Organization
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.9)
Header field name: Path
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.5)
Header field name: Posting-Version
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [RFC0850] (Section 2.1.2)
Header field name: References
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.10),
[RFC5322] (Section 3.6.4)
Header field name: Relay-Version
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [RFC0850] (Section 2.1.1)
Header field name: Reply-To
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2),
[RFC5322] (Section 3.6.2)
Header field name: See-Also
Applicable protocol: netnews
Status: obsoleted
Author/change controller: IETF
Specification document(s): [Son-of-1036] (Section 6.16)
Header field name: Sender
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2),
[RFC5322] (Section 3.6.2)
Header field name: Subject
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.1.6),
[RFC5322] (Section 3.6.5)
Header field name: Summary
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.11)
Header field name: Supersedes
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.12)
Header field name: User-Agent
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.13)
Related information: [RFC2616] (Section 14.43)
Header field name: Xref
Applicable protocol: netnews
Status: standard
Author/change controller: IETF
Specification document(s): This document (Section 3.2.14)
7. References
7.1. Normative References
[RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part One: Format of Internet
Message Bodies", RFC 2045, November 1996.
[RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part Two: Media Types",
RFC 2046, November 1996.
[RFC2047] Moore, K., "MIME (Multipurpose Internet Mail
Extensions) Part Three: Message Header Extensions for
Non-ASCII Text", RFC 2047, November 1996.
[RFC2049] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part Five: Conformance Criteria
and Examples", RFC 2049, November 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2183] Troost, R., Dorner, S., and K. Moore, "Communicating
Presentation Information in Internet Messages: The
Content-Disposition Header Field", RFC 2183,
August 1997.
[RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and
Encoded Word Extensions: Character Sets, Languages,
and Continuations", RFC 2231, November 1997.
[RFC3282] Alvestrand, H., "Content Language Headers", RFC 3282,
May 2002.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter,
"Uniform Resource Identifier (URI): Generic Syntax",
STD 66, RFC 3986, January 2005.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322,
October 2008.
[RFC5537] Allbery, R., Ed. and C. Lindsey, "Netnews Architecture
and Protocols", RFC 5537, November 2009.
7.2. Informative References
[Errata] "RFC Editor Errata",
<http://www.rfc-editor.org/errata.php>.
[ISO3166-1] International Organization for Standardization, "ISO
3166-1:1997. Codes for the representation of names of
countries and their subdivisions -- Part 1: Country
codes", 1997.
[PGPVERIFY] Lawrence, D., "Authentication of Usenet Group Changes
(pgpverify)", June 1999,
<ftp://ftp.isc.org/pub/pgpcontrol/README.html>.
[RFC0822] Crocker, D., "Standard for the format of ARPA Internet
text messages", STD 11, RFC 822, August 1982.
[RFC0850] Horton, M., "Standard for interchange of USENET
messages", RFC 850, June 1983.
[RFC1036] Horton, M. and R. Adams, "Standard for interchange of
USENET messages", RFC 1036, December 1987.
[RFC2156] Kille, S., "MIXER (Mime Internet X.400 Enhanced
Relay): Mapping between X.400 and RFC 822/MIME",
RFC 2156, January 1998.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee,
"Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616,
June 1999.
[RFC2822] Resnick, P., "Internet Message Format", RFC 2822,
April 2001.
[RFC2980] Barber, S., "Common NNTP Extensions", RFC 2980,
October 2000.
[RFC3156] Elkins, M., Del Torto, D., Levien, R., and T.
Roessler, "MIME Security with OpenPGP", RFC 3156,
August 2001.
[RFC3696] Klensin, J., "Application Techniques for Checking and
Transformation of Names", RFC 3696, February 2004.
[RFC3851] Ramsdell, B., "Secure/Multipurpose Internet Mail
Extensions (S/MIME) Version 3.1 Message
Specification", RFC 3851, July 2004.
[RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration
Procedures for Message Header Fields", BCP 90,
RFC 3864, September 2004.
[RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)",
RFC 3977, October 2006.
[Son-of-1036] Spencer, H., "Son of 1036: News Article Format and
Transmission", Work in Progress, May 2009.
[USEAGE] Lindsey, C., "Usenet Best Practice", Work in Progress,
March 2005.
Appendix A. Acknowledgments
As this document is the result of an eight-year effort, the number of
people that have contributed to its content are too numerous to
mention individually. Many thanks go out to all past and present
members of the USEFOR Working Group of the Internet Engineering Task
Force (IETF) and its accompanying mailing list.
Appendix B. Differences from RFC 1036 and Its Derivatives
This appendix contains a list of changes that have been made in the
Netnews article format from earlier standards, specifically
[RFC1036].
o The [RFC5322] conventions for parenthesis-enclosed <comment>s in
header fields are supported in all newly defined header fields and
in header fields inherited from [RFC5322]. They are, however,
still disallowed for performance and/or compatibility reasons in
the Control, Distribution, Followup-To, Lines, Message-ID,
Newsgroups, Path, Supersedes, and Xref header fields.
o Multiple addresses are allowed in the From header field.
o [FWS] is permitted in Newsgroups header fields.
o An enhanced syntax for the Path header field enables the injection
point of, and the route taken by, an article to be determined with
more precision.
o Only one (1) message identifier is allowed in the Supersedes
header field.
o MIME is recognized as an integral part of Netnews.
o There is a new Injection-Date header field to make the rejection
of stale articles more precise and to minimize spurious
rejections.
o There are several new optional header fields defined, notably
Archive, Injection-Info, and User-Agent, leading to increased
functionality.
o Certain header fields, notably Lines, have been deprecated or made
obsolete (Section 3.3).
o The convention to interpret subjects starting with the word "cmsg"
as a control message was removed.
o There are numerous other small changes, clarifications, and
enhancements.
Appendix C. Differences from RFC 5322
This appendix lists the differences between the syntax allowed by the
Netnews article format (this document) as compared to the Internet
Message Format, as specified in [RFC5322].
The Netnews article format is a strict subset of the Internet Message
Format; all Netnews articles conform to the syntax of [RFC5322].
The following restrictions are important:
o A SP (space) is REQUIRED after the colon (':') following a header
field name.
o A slightly restricted syntax of <msg-id> (to be used by the
Message-ID, References, and Supersedes header fields) is defined.
o The length of a <msg-id> MUST NOT exceed 250 octets.
o Comments are not allowed in the Message-ID header field.
o The CFWS between <msg-id>s in the References header field is not
optional.
o It is legal for a parser to reject obsolete syntax, except that:
* The <obs-phrase> construct MUST be accepted.
* The obsolete <zone> "GMT" MUST be accepted within a
<date-time>.
o Every line of a header field body (including the first and any
that are subsequently folded) MUST contain at least one non-
whitespace character. This means that an empty header field body
is illegal.
Authors' Addresses
Kenneth Murchison (editor)
Carnegie Mellon University
5000 Forbes Avenue
Cyert Hall 285
Pittsburgh, PA 15213
U.S.A.
Phone: +1 412 268 2638
EMail: murch@andrew.cmu.edu
Charles H. Lindsey
University of Manchester
5 Clerewood Avenue
Heald Green
Cheadle
Cheshire SK8 3JU
U.K.
Phone: +44 161 436 6131
EMail: chl@clerew.man.ac.uk
Dan Kohn
Healing Thresholds
211 N End Ave Apt 22E
New York, NY 10282
U.S.A.
Phone: +1 415 233 1000
EMail: dan@dankohn.com
ISO Plain Language
On this page
§1Introduction
Netnews is a set of protocols for generating, storing and retrieving news articles, whose format is a subset of that used for email messages. It is organised around newsgroups, each a forum with a name and a topic, with the expectation that every reader can see all the articles posted to the groups they take part in.
The protocols most commonly use a flooding algorithm: copies of an article propagate through a network of participating servers. Typically each server keeps only one copy of an article and makes it available on demand to the readers that can reach it.
This document specifies the syntax of Netnews articles in the context of the Internet Message Format of RFC 5322 and the MIME extensions of RFC 2045. It obsoletes RFC 1036, updating the syntax to reflect current practice and to incorporate changes specified in other documents. It is the first of a set of documents that obsolete RFC 1036: this one covers the syntax and semantics of articles, while RFC 5537 covers the protocol issues of articles independently of transport protocols such as the Network News Transfer Protocol in RFC 3977.
The document is deliberately short, because it uses a cite-by-reference method: rather than repeating the contents of other standards, it points at them and adds only the restrictions that Netnews needs. Appendix B lists the changes from RFC 1036, among them multiple addresses in the From field, MIME as an integral part of Netnews, new optional fields such as Archive, Injection-Info and User-Agent, the deprecation of the Lines field, and the removal of the convention that treated a subject starting with cmsg as a control message.
Netnews is the system behind Usenet. Articles are copied from server to server, and each server keeps one copy and hands it out to readers on request. This document defines the text format of an article. It is the first of two documents replacing the 1987 specification: this one covers the format, and another covers the protocol that moves articles around.
- ARTICLEThe poster supplies the text and the addressing.
- PROTO-ARTICLEThe user agent submits the article, which may still lack mandatory fields.
- ARTICLEThe server stores the article and floods a copy to other servers.
- ARTICLEThe server hands stored articles to the user agent on request.
- ARTICLEThe user agent presents the article to the reader.
§1.5The Vocabulary of Netnews
The document defines the terms the rest of the specification depends on.
- An article is the unit of Netnews, the counterpart of a message in email. A proto-article is one that has not yet been injected into the news system and may still lack some mandatory header fields.
- A message identifier is a unique identifier for an article, usually supplied by the user agent that posted it or, failing that, by the news server. It distinguishes the article from every other article ever posted anywhere, and articles with the same identifier are treated as the same article whatever differences their bodies or header fields hold.
- A newsgroup is a forum with a name for articles on a specific topic. An article is posted to one group or several, and posting to several is called crossposting, which differs from posting the same text as separate articles.
- A group may be moderated, in which case submissions are not posted directly but mailed to a moderator for consideration and possible posting. The moderator is typically a person but may be implemented partly or entirely in software.
- A poster composes and submits a potentially compliant article to a user agent; a reader is the person or software that reads articles. A followup is an article that responds to an earlier article, its precursor, and every followup carries a References header field naming that precursor.
- A control message is an article marked as carrying control information, and a news server receiving one may take actions beyond filing and passing it on, subject to the policies of that site.
- A news server accepts articles from a user agent, makes articles available to user agents, and exchanges articles with other news servers. A user agent helps posters submit proto-articles, fetches articles and presents them to a reader, and helps the reader write articles and followups. The generic term agent covers both.
An agent is said to generate a construct if it did not exist before the agent created it, for example a message made from text supplied by a user, or an Injection-Info header field created by a server for a newly posted message. An agent is said to accept a construct if another entity generated it and passed it over, and the agent processed it without treating it as a format or protocol error.
The document separates what a reader sees from what the servers need to know. An article before it reaches the network is a proto-article and may still be missing fields. A message identifier is what makes an article unique, so servers can recognise a copy they already have. A control message is an article that tells servers to do something rather than just file it.
§2The Article Format
An article is conformant to this specification if it conforms to the format of RFC 5322 and to the additional requirements of this document. An article that uses the obsolete syntax of Section 4 of RFC 5322 is NOT conformant, with two exceptions: articles are conformant if they use the obs-phrase construct, a phrase such as a name without quotes, but agents MUST NOT generate such syntax; and articles are conformant if they use GMT as the time zone. Agents conforming to this specification MUST generate only conformant articles.
The grammar in this document restricts the syntax of RFC 5322 and is intended to be more restrictive, so an article must satisfy both the ABNF of RFC 5322 and the restrictions given here, whether those restrictions are written as text or as ABNF. The header fields are written in Augmented Backus-Naur Form, as in RFC 5234, reusing constructs from RFC 5322, RFC 2045 as updated by RFC 2231, and RFC 3986. The message identifier is defined more strictly than in Section 3.6.4 of RFC 5322.
Four general rules apply to every header field. Agents MUST generate header fields so that at least one space follows the colon separating the name from the body, for compatibility with deployed software including NNTP servers, and MAY accept fields that lack the space. Every line of a header field body, including the first and any folded lines, MUST contain at least one non-whitespace character, so no header field body can be empty; that is why the document uses a stricter definition of unstructured than RFC 5322 does. Compliant software MUST NOT generate header field lines of more than 998 octets, though it MAY accept them, and there is no limit on the number of lines a field may be folded into. The character set for header fields is US-ASCII, and any non-ASCII character MUST be encoded with the MIME mechanisms of RFC 2047 and RFC 2231.
User agents MUST meet the definition of MIME conformance in RFC 2049 and MUST also support RFC 2231. That level of conformance covers internationalisation and multimedia in message bodies and in header fields. For the purposes of Section 5 of RFC 2047, all the header fields defined in Section 3 of this standard count as extension message header fields, so encoded words may be used in any unstructured field or in any comment or phrase inside a structured field. User agents MAY accept and generate other MIME extension header fields, and SHOULD in particular accept Content-Disposition and Content-Language.
An article is an email message with extra rules, and the rules are mostly about being strict. A space must follow the colon in every header field, no field may be empty, a single line of a field may not be longer than 998 characters, and anything outside US-ASCII has to be encoded the MIME way. Two old-fashioned things are still allowed: a name written without quotes in the From field, and the GMT time zone.
§3News Header Fields
The news header fields extend those defined in Section 3.6 of RFC 5322. Each of them MUST NOT occur more than once in an article. The fields Control, Distribution, Followup-To, Lines, Newsgroups, Path, Supersedes and Xref, and the Message-ID field from RFC 5322, do not allow comments, because news servers often have to process these fields very quickly.
Every conformant article MUST have exactly one of each of six fields: Date, From, Message-ID, Newsgroups, Path and Subject. The Date field is the one of RFC 5322, except that agents MUST accept date-time constructs using the GMT zone because that practice is widespread. The From field carries a mailbox list. The Newsgroups field names the groups the article is posted to, written as dot-separated components of letters, digits and the characters plus, minus and underscore; a component SHOULD NOT consist solely of digits and SHOULD NOT contain uppercase letters, and components starting with an underscore are reserved for future versions of the standard. Components starting with plus or minus are reserved for private use. Groups whose first component is example, the group poster, and several other names are reserved and MUST NOT be used for ordinary newsgroups.
The optional fields add the rest of the news vocabulary. Approved names the people or entities approving the article, which matters for moderated articles and group control messages. Archive states whether the poster permits redistribution from public long-term archives, with a body of no or yes. Control marks the article as a control message and gives the action to take. Distribution limits where the article may be distributed, and Expires gives a date after which it should be removed. Followup-To names where a reply to the article should be sent. Injection-Date records when the article entered the network, and Injection-Info records the injecting server, with parameters for the posting host, the posting account, logging data and where complaints about the poster should be sent. Organization gives a short phrase for the poster's organisation. References names the precursor of a followup, Summary holds a one-line summary, Supersedes names an article that this one replaces, and Xref names the other groups the article was crossposted to. The Lines field is obsolete.
Six header fields must appear exactly once in every article: the date, the sender, the message identifier, the groups, the path and the subject. Everything else is optional. Among the optional ones, Injection-Info records where the article entered the network and where to complain about it, and Control tells a server to do something other than just file the article.
- DateWhen the article was written, as a date-time with a numeric zone.
- FromThe mailbox of the poster, or a list of mailboxes.
- Message-IDThe identifier that makes the article unique, at most 250 octets.
- NewsgroupsThe groups the article is posted to, as dot-separated names.
- PathThe trail of servers the article has passed through.
- SubjectA short description of the article.
- ApprovedThe people or entities who approved the article, used for moderated groups.
- ArchiveWhether the poster permits redistribution from public archives.
- ControlMarks the article as a control message and gives the action to take.
- DistributionLimits the areas in which the article may be distributed.
- ExpiresThe date after which the article should be removed.
- Followup-ToWhere a reply to the article should be sent.
- Injection-InfoHow the article entered the network, and where to complain about it.
- SupersedesThe article that this one replaces.
fields =/ *( approved /
archive /
control /
distribution /
expires /
followup-to /
injection-date /
injection-info /
lines /
newsgroups /
organization /
path /
summary /
supersedes /
user-agent /
xref )§3.1.3Message Identifiers
The Message-ID header field holds a unique message identifier. Netnews depends on identifier uniqueness and fast comparison more than email does, and some news software and standards might have trouble with the full range of message identifiers RFC 5322 allows, so this document restricts the syntax.
The identifier is written as the left part, an at sign and the right part, wrapped in angle brackets, and the whole thing MUST NOT be more than 250 octets long. The length limit exists so that systems that accept message identifiers as a parameter when referencing an article, such as NNTP, can rely on a bounded length. The left part is a dot-atom-text and the right part is either a dot-atom-text or a literal in square brackets that contains none of the characters greater-than, left or right square bracket or backslash.
Four differences from RFC 5322 matter. Comments are not allowed inside the Message-ID field. Neither a greater-than sign nor whitespace can occur inside the identifier, so a simple comparison of octets always settles whether two identifiers are the same. The right part is case-sensitive, even though it is usually derived from a domain name, so it must not be altered during later transmission or copying, and an implementation MUST NOT generate a Message-ID whose only difference from another is the case of characters in the right part. And when generating an identifier, implementations SHOULD use a domain name as the right part, because domain literals are troublesome, as many IP addresses are not globally unique, while domain names are more likely to produce unique identifiers.
The message identifier is the part of an article that says which article it is. It is short, at most 250 characters, and it contains nothing that could be read two ways, so a server can compare two identifiers byte for byte and be sure of the answer. The part after the at sign is treated as case-sensitive and must not be changed once the article is on its way.
message-id = "Message-ID:" SP *WSP msg-id *WSP CRLF
msg-id = "<" msg-id-core ">"
; maximum length is 250 octets
msg-id-core = id-left "@" id-right
id-left = dot-atom-text
id-right = dot-atom-text / no-fold-literal
no-fold-literal = "[" *mdtext "]"§3.1.5The Path Header Field
The Path field records the route an article has taken since it was injected into the Netnews system. Every agent that processes an article must prepend at least one path identity to the field body. The main reason is so that news servers avoid sending an article to sites already known to have it, in particular the site it came from; it also allows statistics to be gathered and the route of an article to be traced.
A path identity is a name identifying a site. It takes the form of a domain name with two or more dot-separated components, or a single name with no dots. The identities in the path list, read from right to left, give the successive agents through which the article has passed. A double exclamation mark means that the agent to its left verified the identity of the agent to its right before accepting the article, whereas a single exclamation mark implies no such claim. The last entry, the tail entry, may be the string not-for-mail, which is often used because the path was once used as a mail address for the sender.
The Path field may also carry a path diagnostic, an item inserted for purposes other than naming a site; the keywords used for this are described in RFC 5537. One use of a diagnostic is to record an IP address, and IPv6 addresses are allowed, which permits the colon character; the document notes that this may cause interoperability problems at older sites that treat a colon as a delimiter and have neighbours whose names are four or fewer characters of hexadecimal digits. The use of IPv4 addresses in the field is deprecated, though still acceptable in the deprecated diagnostic form.
The Path field is the trail an article leaves as it moves from server to server. Each server that handles the article adds its own name at the front, so the field reads as the list of servers, most recent first. That lets a server avoid sending an article back where it came from, and lets an operator trace the route an article took.
§5Security Considerations
The Netnews article format provides no security services of its own: no confidentiality, no authentication of the sender and no non-repudiation. Those have to be layered above the format, with protocols such as S/MIME in RFC 3851 or PGP/MIME in RFC 3156, or below it, with secure versions of the news transport protocols.
The document singles out one risk that the format itself creates. Message identifiers must be unique, and an article may be refused in server-to-server transfer if the identifier has already been seen. If a malicious agent can predict the identifier of an article, it can post its own article with the same identifier, possibly to a quite different group, and stop the real article from propagating. Agents that generate message identifiers SHOULD therefore ensure that they are unpredictable.
On MIME, the document points out that the range of encodings allowed for parameters permits constructs that simple parsers may fail to parse correctly, and gives two examples of hard-to-parse Content-Type headers. Such parsing deficiencies may be used as part of an attack. Further security considerations are discussed in RFC 5537.
The article format itself offers no protection: anyone can write an article that claims to be from anyone else. Sending a service such as S/MIME is what adds confidentiality and a signature. The document also warns that if the message identifier can be guessed, an attacker can post a false article under the same identifier and stop the real one from being delivered, so the identifier should be hard to predict.
§6Registered Header Fields
IANA has registered the header fields of this document in the Permanent Message Header Field Repository, following the procedures of RFC 3864. Each registration records the field name, the protocol it applies to, its status, the change controller and the documents that specify it.
The fields registered as standard by this document are Approved, Archive, Comments, Control, Date, Distribution, Expires, Followup-To, From, Injection-Date, Injection-Info, Keywords, Message-ID, Newsgroups, Organization, Path, References, Reply-To, Sender, Subject, Summary, Supersedes, User-Agent and Xref. The Lines field is registered as deprecated, and fields carried over from the earlier Netnews specifications, such as Also-Control, Article-Names, Article-Updates, Date-Received, NNTP-Posting-Date, NNTP-Posting-Host, Posting-Version, Relay-Version and See-Also, are registered with the status obsoleted and point at the document that replaced them.
Every field this document defines has been registered with IANA, which keeps a list of the header fields that exist and which document each one belongs to. Fields that earlier Netnews specifications defined are marked as obsolete on that list, along with the document that replaced them.