TOC 
Network Working GroupM. Wong
Internet-DraftW. Schlitt
Expires: June 27, 2005December 27, 2004

Sender Policy Framework: Authorizing Use of Domains in E-MAIL

draft-schlitt-spf-02

Status of this Memo

This document is an Internet-Draft and is subject to all provisions of section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she become aware will be disclosed, in accordance with RFC 3668.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on June 27, 2005.

Copyright Notice

Copyright (C) The Internet Society (2004).

Abstract

E-mail on the Internet can be forged in a number of ways. In particular, existing protocols place no restriction in what a sending host can use as the reverse-path of a message. This document describes a protocol whereby a domain can explicitly authorize the hosts that are allowed to use its domain name in a reverse-path, and a way for receiving hosts to check such authorization.



Table of Contents

1.  Introduction
    1.1  Protocol Status
    1.2  Terminology
2.  Operation
    2.1  The HELO Identity
    2.2  The MAIL FROM Identity
    2.3  Publishing Authorization
    2.4  Checking Authorization
    2.5  Interpreting the Result
        2.5.1  None
        2.5.2  Neutral
        2.5.3  Pass
        2.5.4  Fail
        2.5.5  SoftFail
        2.5.6  TempError
        2.5.7  PermError
3.  SPF Records
    3.1  Publishing
        3.1.1  DNS Resource Record Types
        3.1.2  Multiple Records
        3.1.3  Multiple Strings
        3.1.4  Record Size
        3.1.5  Wildcard Records
4.  The check_host() Function
    4.1  Arguments
    4.2  Results
    4.3  Initial Processing
    4.4  Record Lookup
    4.5  Selecting Records
    4.6  Record Evaluation
        4.6.1  Term Evaluation
        4.6.2  Mechanisms
        4.6.3  Modifiers
    4.7  Default Result
    4.8  Domain Specification
5.  Mechanism Definitions
    5.1  "all"
    5.2  "include"
    5.3  "a"
    5.4  "mx"
    5.5  "ptr"
    5.6  "ip4" and "ip6"
    5.7  "exists"
6.  Modifier Definitions
    6.1  redirect: Redirected Query
    6.2  exp: Explanation
7.  Miscellaneous
    7.1  Processing Limits
    7.2  The Received-SPF header
8.  Macros
    8.1  Macro definitions
    8.2  Expansion Examples
9.  Implications
    9.1  Sending Domains
    9.2  Mailing Lists
    9.3  Forwarding Services and Aliases
    9.4  Mail Services
    9.5  MTA Relays
10.  Security Considerations
11.  IANA Considerations
12.  Contributors and Acknowledgements
13.  References
13.1  Normative References
13.2  Informative References
§  Authors' Addresses
A.  Collected ABNF
B.  Extended Examples
    B.1  Simple Examples
    B.2  Multiple Domain Example
    B.3  DNSBL Style Example
§  Intellectual Property and Copyright Statements




 TOC 

1. Introduction

The current e-mail infrastructure has the property that any host injecting mail into the mail system can identify itself as any domain name it wants. Hosts can do this at a variety of levels: in particular, the session, the envelope, and the mail headers. While this feature is desirable in some circumstances, it is a major obstacle to reducing end-user unwanted e-mail (or "spam"). Furthermore, many domain name holders are understandably concerned about the ease with which other entities may make use of their domain names, often with intent to impersonate.

This document defines a protocol by which domain owners may authorize hosts to use their domain name in the "MAIL FROM" or "HELO" identity. Compliant domain holders publish SPF records about which hosts are permitted to use their names, and compliant mail receivers use the published SPF records to test the authorization of hosts using a given "HELO" or "MAIL FROM" identity during a mail transaction.

An additional benefit to mail receivers is that when the use of an identity is verified, then local policy decisions about the mail can be made on the basis of the domain, rather than the host's IP address. This is advantageous because reputation of domain names is likely to be more accurate than reputation of host IP addresses. Furthermore, if a claimed identity fails verification, then local policy can take stronger action against such e-mail, such as rejecting it.

1.1 Protocol Status

SPF has been in development since the Summer of 2003, and has seen deployment beyond the developers beginning in December, 2003. The design of SPF slowly evolved until the spring of 2004 and has since stabilized. There have been quite a number of forms of SPF, some written up as documents, some submitted as Internet Drafts, and many discussed and debated in development forums.

The goal of this document is to clearly document the protocol defined by earlier drafts specifications of SPF as used in existing implementations. This conception of SPF is sometimes called "SPF Classic". It is understood that particular implementations and deployments may differ from, and build upon, this work. It is hoped that we have nonetheless captured the common understanding of SPF version 1.

1.2 Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]Bradner, S., Key words for use in RFCs to Indicate Requirement Levels, March 1997..

This document is concerned with a portion of a mail message commonly called "envelope sender", "return path", "reverse path", "bounce address", "2821 FROM", or "MAIL FROM". Since these terms are either not well defined, or often used casually, this document defines the "MAIL FROM" identity in Section 2.2The MAIL FROM Identity. Note that other terms, that may superficially look like the common terms, such as "reverse-path" or "Return-Path" are used only with the defined meanings from normative documents.



 TOC 

2. Operation

2.1 The HELO Identity

The "HELO" identity derives from either the SMTP HELO or EHLO command (see [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001..) These commands supply the SMTP client (sender) for the STMP session. Note that requirements for the domain presented in the EHLO or HELO command are not always clear to the sending party, and receiving software must be prepared for the "HELO" identity to be malformed.

SPF clients MAY check the "HELO" identity by calling the check_host() function (Section 4The check_host() Function) with the "HELO" identity as the <sender>. If the HELO test returns a "fail", the overall result for the SMTP session is "fail", and there is no need to test the "MAIL FROM" identity.

2.2 The MAIL FROM Identity

The "MAIL FROM" identity derives from the SMTP MAIL command (see [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001..) This command supplies the "reverse-path" for a message, which generally consists of the sender mailbox, and is the mailbox to which notification messages are sent if there are problems delivering the message.

[RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001. allows the reverse-path to be null (see Section 4.5.5.) In this case, there is no explicit sender mailbox, and such a message can be assumed to be a notification message from the mail system itself. When the reverse-path is null, this document defines the "MAIL FROM" identity to be the mailbox composed of the localpart "postmaster" and the "HELO" identity

SPF clients MUST check the "MAIL FROM" identity unless HELO testing produced a "fail". SPF clients check the "MAIL FROM" identity by calling the check_host() function with the "MAIL FROM" identity as the <sender>.

2.3 Publishing Authorization

An SPF compliant domain MUST publish a valid SPF record as described in Section 3SPF Records. This record authorizes the use of the domain name in the "HELO" and/or "MAIL FROM" identity, by some sending MTAs, and not by others.

It is RECOMMENDED that domains publish SPF records that end in "-all", or redirect to other records that do, so that a definitive determination of authorization can be made.

Domain holders may publish SPF records that explicitly authorize no hosts for domain names that shouldn't be used in sender mailboxes.

2.4 Checking Authorization

A mail receiver can perform an SPF compliant check for each mail message it receives. This check tests the authorization of a client host to inject mail with a given "MAIL FROM" identity. This check MAY also be applied to the "HELO" identity. Typically, such checks are done by a receiving MTA, but can be performed elsewhere in the mail processing chain so long as the required information is available. Checking other identities against SPF records is NOT RECOMMENDED because there are cases that are known to give incorrect results.

It is expected that mail receivers will use the SPF check as part of a larger set of tests on incoming mail. The results of other tests may influence whether or not a particular SPF check is performed. For example, finding the sending host on a local white list may cause all other tests to be skipped and all mail from that host to be accepted.

When a mail receiver decides to perform an SPF check, it MUST implement and evaluate the check_host() function (Section 4The check_host() Function) correctly. While the test as a whole is optional, once it has been decided to perform a test it must be performed as specified so that the correct semantics are preserved between publisher and receiver.

To make the test, the mail receiver MUST evaluate the check_host() with the arguments set as follows:

<ip>
- the IP address of the SMTP client that is injecting the mail, either IPv4 or IPv6.
<domain>
- the domain portion of the "MAIL FROM" or "HELO" identity.
<sender>
- the "MAIL FROM" or "HELO" identity.

Note that the <domain> argument may not be a well formed domain name. For example, if the reverse-path was null, then the EHLO or HELO domain is used. In a valid SMTP session, this can be an address literal or entirely malformed. In these cases, check_host() is defined in Section 4.3Initial Processing to return a "None" result.

Care must be taken to correctly extract the <domain> from the <sender> as many MTAs will still accept such things as source routes (see [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001. appendix C), the percent hack (see [RFC2162]Allocchio, C., MaXIM-11 - Mapping between X.400 / Internet mail and Mail-11 mail, January 1998.) and bang paths (see [RFC1983]Malkin, G., Internet Users' Glossary, August 1996.). Spammers often use of such archaic features to try and trick MTAs into being open relays.

Software SHOULD perform this authorization check during the processing of the SMTP transaction that injects the mail. This allows errors to be returned directly to the injecting server by way of SMTP replies. Software can perform the check as early as the MAIL command, though it may be easier to delay the check to some later stage of the transaction.

Software can perform the authorization after the corresponding SMTP transaction has completed. There are two problems with this approach: 1) It may be difficult to accurately extract all the required information such as client IP address and HELO domain name. 2) If the authorization fails, then generating a non-delivery notification to the alleged sender is problematic due to the large number of forged emails on the Internet today. Such an action would go against the explicit wishes of the alleged sender.

2.5 Interpreting the Result

The check_host() function returns one of seven results. This section describes how software that performs the authorization must interpret the results. If the check is being performed during the SMTP mail transaction, it also describes how to respond.

2.5.1 None

A result of None means that no records were published by the domain. The checking software cannot ascertain if the client host is authorized or not.

2.5.2 Neutral

The domain owner has explicitly stated that doesn't know whether the IP is authorized or not. A Neutral result MUST be treated exactly like the None result.

2.5.3 Pass

A Pass result means that the client is authorized to inject mail with the given identity. Further policy checks, such as reputation, or black and/or white listing, can now proceed with confidence in the identity.

2.5.4 Fail

A Fail result is an explicit statement that the client is not authorized to use the domain in the given identity. The checking software can choose to mark the mail based on this, or to reject the mail outright.

If the checking software chooses to reject the mail during the SMTP transaction, then it SHOULD use an SMTP reply code of 550 (see [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001.) and the 5.7.1 DSN code (see [RFC2034]Freed, N., SMTP Service Extension for Returning Enhanced Error Codes, October 1996.), in addition to an appropriate message. The check_host() function may return either a default explanation string, or one from the domain that published the SPF records (see Section 6.2exp: Explanation). If the information doesn't originate with the checking software, it should be made clear that text is not trusted. For example:

    550-5.7.1 SPF MAIL FROM check failed:
    550-5.7.1 The domain example.com explains:
    550 5.7.1 Please see http://www.example.com/mailpolicy.html

2.5.5 SoftFail

A SoftFail result should be treated as somewhere between a Fail and a Neutral. This value is used by domains as an intermediate state during roll-out of publishing records. The domain believes the host isn't authorized but isn't willing to make that strong of a statement. Receiving software SHOULD NOT reject the message based on this result, but MAY subject the message to closer scrutiny.

Because the domain has discouraged any legitimate use of this IP address, receivers MAY try to inform either the sender or the recipient of the e-mail. For example, the MUA could highlight the SoftFail status for the receiver, or the MTA could issue an SMTP reply code of 451 and the 4.3.0 DSN code, in addition to an with a note the first time the message was received, but accept it the second time.

2.5.6 TempError

A TempError result means that the receiving server encountered a transient error when performing the check. Checking software can choose to accept or temporarily reject the message. If the message is rejected during the SMTP transaction for this reason, the software SHOULD use an SMTP reply code of 451 and the 4.4.3 DSN code.

2.5.7 PermError

A PermError result means that the domain's published records couldn't be correctly interpreted. Checking software SHOULD reject the message. If rejecting during SMTP transaction time, an SHOULD use an SMTP reply code of 550 and the 5.5.2 DSN code.



 TOC 

3. SPF Records

An SPF record declares which hosts are, and are not, authorized to use a domain name for the "HELO" or "MAIL FROM" identity. Loosely, the record partitions all hosts into permitted and not-permitted sets. (Though some hosts might fall into other categories.)

The SPF record is a single string of text. An example record is:

v=spf1 +mx a:colo.example.com/28 -all

This record has a version of "v=spf1" and three directives: "+mx", "a:colo.example.com/28" (the + is implied), and "-all".

3.1 Publishing

Domain owners wishing to be SPF compliant must publish SPF records for the hosts that are used in both the MAIL FROM and HELO identities. The SPF records are placed in the DNS tree at the host name it partains to, not a subdomain under it, such as is done with SRV records. This is the same whether TXT RRs or SPF RRs are used.

The example above in Section 3SPF Records might be published easily via this lines in a domain zone file:

example.com. IN TXT "v=spf1 +mx a:colo.example.com/28 -all"

smtp-out.example.com. IN TXT "v=spf1 a -all"

When published via TXT records, there may be other TXT records for other purposes published there and this may cause problems with size limits (see Section 3.1.4Record Size.)

An SPF record published at the zone cut for the domain will be used as a default for all other domains and subdomains within the zone. See Section 4.5Selecting Records for details. Domain owners SHOULD publish SPF records for hosts used for the HELO and MAIL FROM identities instead of using the zone cut default because the fallback requires additional DNS lookups. The zone cut default does reduce the need to publish SPF records for non-email related hosts, such as www.example.com.

3.1.1 DNS Resource Record Types

This document defines a new DNS Resource Record (RR) of type SPF, type code to be determined. The format of this type is identical to the TXT RR [RFC1035]Mockapetris, P., Domain names - implementation and specification, November 1987.. For either type, the character content of the record is encoded as US-ASCII.

It is recognized that the current practice (using a TXT record) is not optimal, but it is necessary because there are a number of DNS server and resolver implementations in common use that cannot handle the new RR type. The two record type scheme provides a forward path to the better solution of using a RR type reserved for this purpose.

An SPF compliant domain name SHOULD have SPF records of both RR types. A compliant domain name MUST have a record of at least one type. If a domain has records of both types, they MUST have identical content. For example, instead of just publishing one record as in Section 3.1Publishing above, it is better to publish:

   example.com. IN TXT "v=spf1 +mx a:colo.example.com/28 -all"
   example.com. IN SPF "v=spf1 +mx a:colo.example.com/28 -all"

An SPF compliant check SHOULD lookup both types. Lookups can be performed serially or in parallel. If both types of records are obtained for a domain, the SPF type MUST be used.

Example RRs in this document are shown with the TXT record type, however they could also be published with both RR types.

3.1.2 Multiple Records

A domain name MUST NOT have multiple records that would cause an authorization check to select more than one record. See Section 4.5Selecting Records for the selection rules.

3.1.3 Multiple Strings

A text DNS record (either TXT and SPF RR types) can be composed of more than one string. If a published record contains multiple strings, then the record MUST be treated as if those strings are concatenated together without adding spaces. For example:

IN TXT "v=spf1 .... first" "second string..."

MUST be treated as equivalent to

IN TXT "v=spf1 .... firstsecond string..."

SPF or TXT records containing multiple strings are useful in order to construct longer records which would otherwise exceed the maximum length of a string within a TXT or SPF RR record.

3.1.4 Record Size

The published SPF record for a given domain name SHOULD remain small enough that the results of a query for it will fit within 512 octets. This will keep even older DNS implementations from falling over to TCP. Since the answer size is dependent on many things outside the scope of this document, it is only possible to give this guideline: If the combined length of the DNS name and the text of all the records of a given type (TXT or SPF) is under 450 characters, then DNS answers should fit in UDP packets. Note that when computing the sizes for queries of the TXT format, one must take into account any other TXT records published at the domain name. Records that are too long to fit in a single UDP packet MAY be silently ignored.

3.1.5 Wildcard Records

Use of wildcard records for publishing is not recommended. Care must be taken if wildcard records are used. If a domain publishes wildcard MX records, it may want to publish wildcard declarations, subject to the same requirements and problems. In particular, the declaration must be repeated for any host that has any RR records at all, and for subdomains thereof. For example, the example given in [RFC1034]Mockapetris, P., Domain names - concepts and facilities, November 1987., Section 4.3.3, could be extended with:

    X.COM           MX      10      A.X.COM
    X.COM           TXT     "v=spf1 a:A.X.COM -all"

    *.X.COM         MX      10      A.X.COM
    *.X.COM         TXT     "v=spf1 a:A.X.COM -all"

    A.X.COM         A       1.2.3.4
    A.X.COM         MX      10      A.X.COM
    A.X.COM         TXT     "v=spf1 a:A.X.COM -all"

    *.A.X.COM       MX      10      A.X.COM
    *.A.X.COM       TXT     "v=spf1 a:A.X.COM -all"

Notice that SPF records must be repeated twice for every name within the domain: Once for the name, and once with a wildcard to cover the tree under the name.

Use of wildcards is discouraged in general as they cause every name under the domain to exist and queries against arbitrary names will never return RCODE 3 (Name Error).



 TOC 

4. The check_host() Function

The check_host() function fetches SPF records, parses them, and interprets them to evaluate if a particular host is or is not permitted to send mail with a given identity. Mail receivers that perform this check MUST correctly evaluate the check_host() function as described here.

Implementations MAY use a different algorithm than the canonical algorithm defined here, so long as the results are the same.

4.1 Arguments

The function check_host() takes these arguments:

<ip>
- the IP address of the SMTP client that is injecting the mail, either IPv4 or IPv6.
<domain>
- the domain portion of the "MAIL FROM" or "HELO" identity.
<sender>
- the "MAIL FROM" or "HELO" identity.

The domain portion of <sender> will usually be the same as the <domain> argument when check_host() is initially evaluated. However, it will generally not be true for recursive evaluations (see Section 5.2"include" below).

Actual implementations of the check_host() function will likely need additional arguments.

4.2 Results

The function check_host() can result in one of seven results described in Section 2.5Interpreting the Result. Based on the result, the action to be taken is determined by the local policies of the receiver.

4.3 Initial Processing

If the <domain> is malformed or is not a fully qualified domain name, check_host() immediately returns the result "None".

If the <sender> has no localpart, substitute the string "postmaster" for the localpart.

4.4 Record Lookup

In accordance with how the records are published, see Section 3.1Publishing above, a DNS query needs to be made for the <domain> name, querying for either RR type TXT, SPF or both.

If the DNS lookup returns a server failure (RCODE 2), or other error (RCODE other than 0 or 3), or the query times out, check_host() exits immediately with the result "TempError"

4.5 Selecting Records

Records begin with a version section:

record         = version terms *SP
version        = "v=spf1"

Starting with the set of records that were returned by the lookup, record selection proceeds in two steps:

  1. If any records of type SPF are in the set, then all records of type TXT are discarded.
  2. Records that do not begin with a version section of exactly "v=spf1" are discarded. Note that the version section is terminated either by a SP character or the end of the record. A record with a version section of "v=spf10" does not match and must be discarded.

After the above steps, there should be exactly one record remaining and evaluation can proceed. If there are two or more records remaining, then check_host() exits immediately with the result of "PermError".

If no matching records are returned for the <domain;>, the SPF client MUST find the Zone Cut as defined in [RFC2181]Elz, R. and R. Bush, Clarifications to the DNS Specification, July 1997. section 6 and repeat the above steps. The <domain>'s zone origin is then searched for SPF records. If an SPF record is found at the zone origin, the <domain> is set to the zone origin as if a "redirect" modifier was executed.

If no matching records are returned for either search, an SPF client MUST assume that the domain makes no SPF declarations. SPF processing MUST abort and return "None".

4.6 Record Evaluation

After one SPF record has been selected, the check_host() function parses and interprets it to find a result for the current test. If there are any syntax errors, check_host() returns immediately with the result "PermError".

Implementations MAY choose to parse the entire record first and return "PermError" if the record is not syntactically well formed. However, in all cases, any syntax errors anywhere in the record MUST be detected.

4.6.1 Term Evaluation

There are two types of terms: mechanisms and modifiers. A record contains an ordered list of these mechanisms and modifiers:

terms          = *( 1*SP ( directive / modifier ) )

directive      = [ prefix ] mechanism
prefix         = "+" / "-" / "?" / "~"
mechanism      = ( all / include
                 / A / MX / PTR / IP4 / IP6 / exists )

modifier       = redirect / explanation / unknown-modifier
unknown-modifier = name "=" macro-string

name           = ALPHA *( ALPHA / DIGIT / "-" / "_" / "." )

Most mechanisms allow a ":" or "/" character after the name.

Modifiers always contain an equals ('=') character immediately after the name, and before any ":" or "/" characters that may be part of the macro-string.

Terms that do not contain any of "=", ":" or "/" are mechanisms.

As per the definition of the ABNF notation in [RFC2234]Crocker, D. and P. Overell, Augmented BNF for Syntax Specifications: ABNF, November 1997., mechanism and modifier names are case-insensitive.

4.6.2 Mechanisms

Each mechanism is considered in turn from left to right. If there are no more mechanisms, the result is specified in Section 4.7Default Result.

When a mechanism is evaluated, one of three things can happen: it can match, it can not match, or it can throw an exception.

If it matches, processing ends and the prefix value is returned as the result of that record. If it does not match, processing continues with the next mechanism. If it throws an exception, mechanism processing ends and the exception value is returned.

The possible prefixes, and the results they return are:

"+"
Pass
"-"
Fail
"~"
SoftFail
"?"
Neutral

The prefix is optional and defaults to "+".

When a mechanism matches, and the prefix is "-" so that a "Fail" result is returned and the explanation string is computed as described in Section 6.2exp: Explanation.

Specific mechanisms are described in Section 5Mechanism Definitions.

4.6.3 Modifiers

Modifiers are not mechanisms: they do not return match or not-match. Instead they provide additional information. While modifiers do not directly effect the evaluation of the record, the "redirect" modifier has an effect after all the mechanisms have been evaluated.

4.7 Default Result

If none of the mechanisms match and there is no "redirect" modifier, then the check_host() returns a result of "Neutral". If there is a "redirect" modifier, check_host() proceeds as defined in Section 6.1redirect: Redirected Query.

Note that records SHOULD always either use a "redirect" modifier or an "all" mechanism to explicitly terminate processing.

For example:

v=spf1 +mx -all

or

v=spf1 +mx redirect=_spf.example.com

4.8 Domain Specification

Several of these mechanisms and modifiers have a <domain-spec> section. The <domain-spec> string is macro expanded (see Section 8Macros). The resulting string is the common presentation form of a fully qualified DNS name: A series of labels separated by periods. This domain is called the <target-name> in the rest of this document.

Note: The result of the macro expansion is not subject to any further escaping. Hence, this facility cannot produce all characters that are legal in a DNS label (e.g. the control characters). However, this facility is powerful enough to express legal host names, and common utility labels (such as "_spf") that are used in DNS.

For several mechanisms, the <domain-spec> is optional. If it is not provided, the <domain> is used as the <target-name>.



 TOC 

5. Mechanism Definitions

This section defines two types of mechanisms.

Basic mechanisms contribute to the language framework. They do not specify a particular type of authorization scheme.

all

include

Designated sender mechanisms are used to designate a set of <ip> addresses as being permitted or not to use the <domain> for sending mail.

a

mx

ptr

ip4

ip6

exists

The following conventions apply to all mechanisms that perform a comparison between <ip> and an IP address at any point:

If no CIDR-length is given in the directive, then <ip> and the IP address are compared for equality.

If a CIDR-length is specified, then only the specified number of high-order bits of <ip> and the IP address are compared for equality.

When any mechanism fetches host addresses to compare with <ip>, when <ip> is an IPv4 address, A records are fetched, when <ip> is an IPv6 address, AAAA records are fetched. Even if the SMTP connection is via IPv6, an IPv4-mapped IPv6 IP address (see [RFC3513]Hinden, R. and S. Deering, Internet Protocol Version 6 (IPv6) Addressing Architecture, April 2003. section 2.5.5) MUST still be considered an IPv4 address.

Several mechanisms rely on information fetched from DNS. For these DNS queries, except where noted, if the DNS server returns an error (RCODE other than 0 or 3) or the query times out, the mechanism throws the exception "TempError". If the server returns "domain does not exist" (RCODE 3), then evaluation of the mechanism continues as if the server returned no error (RCODE 0) and zero answer records.

5.1 "all"

all            = "all"

The "all" mechanism is a test that always matches. It is used as the rightmost mechanism in a record to provide an explicit default.

For example:

v=spf1 a mx -all

Mechanisms after "all" will never be tested. Any "redirect" modifier (Section 6.1redirect: Redirected Query) has no effect when there is an "all" mechanism.

5.2 "include"

include        = "include"  ":" domain-spec

The "include" mechanism triggers a recursive evaluation of check_host(). The domain-spec is expanded as per Section 8Macros. Then check_host() is evaluated with the resulting string as the <domain>. The <ip> and <sender> arguments remain the same as in the current evaluation of check_host().

In hind sight, the name "include" was poorly chosen. Only the evaluated results of the referenced SPF record is used, rather than acting as if the referenced SPF record was literally included in the first. Better names for this mechanism would have been something like "on-pass" or "if-pass".

The "include" mechanism makes it possible for one domain to designate multiple administratively independent domains. For example, a vanity domain "example.net" might send mail using the servers of administratively independent domains example.com and example.org.

Example.net could say

"v=spf1 include:example.com include:example.org -all".

That would direct check_host() to, in effect, check the records of example.com and example.org for a "pass" result. Only if the host were not permitted for either of those domains would the result be "Fail".

Whether this mechanism matches or not, or throws an error depends on the result of the recursive evaluation of check_host():

A recursive check_host() result of: Causes the "include" mechanism to:
Pass match
Fail not match
SoftFail not match
Neutral not match
TempError throw TempError
PermError throw PermError
None throw PermError

The "include" mechanism is intended for crossing administrative boundaries. While it is possible to use includes to consolidate multiple domains that share the same set of designated hosts, domains are encouraged to use redirects where possible, and to minimize the number of includes within a single administrative domain. For example, if example.com and example.org were managed by the same entity, and if the permitted set of hosts for both domains were "mx:example.com", it would be possible for example.org to specify "include:example.com", but it would be preferable to specify "redirect=example.com" or even "mx:example.com".

5.3 "a"

This mechanism matches if <ip> is one of the <target-name>'s IP addresses.

A              = "a"      [ ":" domain-spec ] [ dual-cidr-length ]

An address lookup is done on the <target-name>. The <ip> is compared to the returned address(es). If any address matches, the mechanism matches.

5.4 "mx"

This mechanism matches if <ip> is one of the MX hosts for a domain name.

MX             = "mx"     [ ":" domain-spec ] [ dual-cidr-length ]

check_host() first performs an MX lookup on the <target-name>. Then it performs an address lookup on each MX name returned. The <ip> is compared to each returned IP address. To prevent DoS attacks, a limit of 10 MX names MUST be enforced (see Section 10Security Considerations). If any address matches, the mechanism matches.

Note regarding implicit MXes: If the <target-name> has no MX records, check_host() MUST NOT pretend the target is its single MX, and MUST NOT default to an A lookup on the <target-name> directly. This behavior breaks with the legacy "implicit MX" rule. See [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001. Section 5. If such behavior is desired, the publisher should specify an "a" directive.

5.5 "ptr"

This mechanism tests if the DNS reverse mapping for <ip> exists and validly points to a domain name within a particular domain.

PTR            = "ptr"    [ ":" domain-spec ]

First the <ip>'s name is looked up using this procedure: perform a DNS reverse-mapping for <ip>, looking up the corresponding PTR record in "in-addr.arpa." if the address is an IPv4 one and "ip6.arpa." if it is an IPv6 address. For each record returned, validate the domain name by looking up its IP address. To prevent DoS attacks, a limit of 10 PTR names MUST be enforced (see Section 10Security Considerations). If <ip> is among the returned IP addresses, then that domain name is validated. In pseudocode:

sending-domain_names := ptr_lookup(sending-host_IP);
if more than 10 sending-domain_names are found, use at most 10.
for each name in (sending-domain_names) {
  IP_addresses := a_lookup(name);
  if the sending-domain_IP is one of the IP_addresses {
    validated-sending-domain_names += name;
  }
}

Check all validated domain names to see if they end in the <target-name> domain. If any do, this mechanism matches. If no validated domain name can be found, or if none of the validated domain names end in the <target-name>, this mechanism fails to match. If a DNS error occurs while doing the PTR RR lookup, then this mechanism fails to match. If a DNS error occurs while doing an A RR lookup, then that domain name is skipped and the search continues.

Pseudocode:

for each name in (validated-sending-domain_names) {
  if name ends in <domain-spec>, return match.
  if name is <domain-spec>, return match.
}
return no-match.

This mechanism matches if the <target-name> is either an ancestor of a validated domain name, or if the <target-name> and a validated domain name are the same. For example: "mail.example.com" is within the domain "example.com", but "mail.bad-example.com" is not.

Note: Use of this mechanism is discouraged because it is slow, is not as reliable as other mechanisms in cases of DNS errors and it places a large burden on the arpa name servers. If used, proper PTR records must be in place for the domain's hosts and the "ptr" mechanism should be one of the last mechanisms checked.

5.6 "ip4" and "ip6"

These mechanisms test if <ip> is contained within a given IP network.

IP4            = "ip4"      ":" ip4-network   [ ip4-cidr-length ]
IP6            = "ip6"      ":" ip6-network   [ ip6-cidr-length ]

ip4-cidr-length  = "/" 1*DIGIT
ip6-cidr-length  = "/" 1*DIGIT
dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ]

ip4-network    = ; as per conventional dotted quad notation,
          ; e.g. 192.0.2.0
ip6-network    = ; as per [RFC 3513], section 2.2,
          ; e.g. 2001:DB8::CD30

The <ip> is compared to the given network. If CIDR-length high-order bits match, the mechanism matches.

If ip4-cidr-length is omitted it is taken to be "/32". If ip6-cidr-length is omitted it is taken to be "/128". It is not permitted to omit parts of the IP address instead of using CIDR notations. That is, use 10.23.45.0/24 instead of 10.23.45.

5.7 "exists"

This mechanism is used to construct an arbitrary domain name that is used for a DNS A record query. It allows for complicated schemes involving arbitrary parts of the mail envelope to determine what is permitted.

exists         = "exists"   ":" domain-spec

The domain-spec is expanded as per Section 8Macros. The resulting domain name is used for a DNS A RR lookup. If any A record is returned, this mechanism matches. The lookup type is 'A' even when the connection type is IPv6.

Domains can use this mechanism to specify arbitrarily complex queries. For example, suppose example.com publishes the record:

v=spf1 exists:%{ir}.%{l1r+-}._spf.%{d} -all

The <target-name> might expand to "1.2.0.192.someuser._spf.example.com". This makes fine-grained decisions possible at the level of the user and client IP address.

This mechanism enables queries that mimic the style of tests that existing DNSBL lists use.



 TOC 

6. Modifier Definitions

Modifiers are name/value pairs that provide additional information. Modifiers always have an "=" separating the name and the value.

The modifiers defined in this document ("redirect" and "exp") MAY appear anywhere in the record, but SHOULD appear at the end, after all mechanisms. Ordering of these two modifiers does not matter. These modifiers MUST NOT appear in a record more than once each. If they do, then check_host() exits with a result of "PermError".

Unrecognized modifiers SHOULD be ignored no matter where in a record, nor how often. This allows implementations of this document to handle records with modifiers that are defined in other specifications.

6.1 redirect: Redirected Query

If all mechanisms fail to match, and a "redirect" modifier is present, then processing proceeds as follows.

redirect       = "redirect" "=" domain-spec

The domain-spec portion of the redirect section is expanded as per the macro rules in Section 8Macros. Then check_host() is evaluated with the resulting string as the <domain>. The <ip> and <sender> arguments remain the same as current evaluation of check_host().

The result of this new evaluation of check_host() is then considered the result of the current evaluation.

Note that the newly queried domain may itself specify redirect processing.

This facility is intended for use by organizations that wish to apply the same record to multiple domains. For example:

  la.example.com. TXT "v=spf1 redirect=_spf.example.com"
  ny.example.com. TXT "v=spf1 redirect=_spf.example.com"
  sf.example.com. TXT "v=spf1 redirect=_spf.example.com"
_spf.example.com. TXT "v=spf1 mx:example.com -all"

In this example, mail from any of the three domains is described by the same record. This can be an administrative advantage.

Note: In general, the domain "A" cannot reliably use a redirect to another domain "B" not under the same administrative control. Since the <sender> stays the same, there is no guarantee that the record at domain "B" will correctly work for addresses in domain "A", especially if domain "B" uses mechanisms involving localparts. An "include" directive may be more appropriate.

For clarity it is RECOMMENDED that any "redirect" modifier appear as the very last term in a record.

6.2 exp: Explanation

explanation    = "exp" "=" domain-spec

If check_host() results in a "Fail" due to a mechanism match (such as "-all"), and the "exp" modifier is present, then the explanation string returned is computed as described below. If no "exp" modifier is present, then either a default explanation string or an empty explanation string may be returned.

The <domain-spec> is macro expanded (see Section 8Macros) and becomes the <target-name>. The DNS TXT record for the <target-name> is fetched.

If <domain-spec> is empty, or there are any processing errors (any RCODE other than 0), or if no records are returned, or if more than one record is returned, then proceed as if no exp modifier was given.

The fetched TXT record's strings are concatenated with no spaces, and then treated as an <explain-string> which is macro-expanded. This final result is the explanation string.

Software evaluating check_host() can use this string when to communicate information from the publishing domain in the form of a short message or URL. Software should make it clear that the explanation string comes from a third party. For example, it can prepend the macro string "%{o} explains: " to the explanation.

Implementations MAY limit the length of the resulting explanation string to allow for other protocol constraints and/or reasonable processing limits. The SPF client SHOULD make it clear when an explanation string is coming from a third party, such as shown in Section 2.5.4Fail.

Suppose example.com has this record

v=spf1 mx -all exp=explain._spf.%{d}

Here are some examples of possible explanation TXT records at explain._spf.example.com:

Example.com mail should only be sent by its own servers.

--
a simple, constant message

%{i} is not one of %{d}'s designated mail servers.

--
a message with a little more info, including the IP address that failed the check

See http://%{d}/why.html?s=%{S}&i=%{I}

--
a complicated example that constructs a URL with the arguments to check_host() so that a web page can be generated with detailed, custom instructions

Note: During recursion into an "include" mechanism, exp= modifiers do not propagate out. In contrast, during execution of a "redirect" modifier, the explanation string from the target of the redirect is used.



 TOC 

7. Miscellaneous

7.1 Processing Limits

During processing, an evaluation of check_host() may require additional evaluations of check_host() due to the "include" mechanism and/or the "redirect" modifier.

In order to prevent Denial-of-Service (DoS) attacks, the total number of DNS lookups must be limited. The subject of a DoS attack can be either the SPF client directly, the domain owner of the claimed sender, or some third party domain that is referenced in the SPF record.

Of these, the case of a third party referenced in the SPF record is the easiest for a DoS attack to effectively exploit. For example, a malicious person could create an SPF record with many references to a victim domain, send many e-mails to different SPF clients and the SPF clients would create a DoS attack. In effect, the SPF clients are being used to amplify the attacker's bandwidth by using fewer bytes in the SMTP session than is generated by the DNS queries. Using SPF clients also allows the attacker to hide the true source of the attack.

As a result, limits that may seem reasonable for an individual mail server can still allow an unreasonable amount of bandwidth amplification. Therefore the processing limits need to be quite small.

SPF implementations MUST limit the number of mechanisms that do DNS lookups to at most 10. For example, the "mx" and "include" mechanisms requires DNS lookups, so will count against this limit, while the "ip4" mechanism does not require any DNS lookups.

When evaluating the "mx" mechanism, there MUST be a limit of no more than 10 MXes looked up and checked for matching IP addresses.

When evaluating the "ptr" mechanism or the %{p} macro, there MUST be a limit of at most 10 PTR DNS records looked up and checked for a validated domain name.

SPF implementation SHOULD limit the total amount of data obtained from the DNS queries. For example, when DNS over TCP or EDNS0 are available, there may need to be an explicit limit to how much data will be accepted to prevent excessive bandwidth usage or memory usage, and DoS attacks.

Implementations must be prepared to handle records that are set up incorrectly or maliciously.

MTAs or other processors MAY also impose a limit on the maximum amount of elapsed time to evaluate check_host(). Such a limit SHOULD allow at least 20 seconds. If such a limit is exceeded, the result of authentication SHOULD be "TempError".

Domains publishing records SHOULD try to keep the number of "include" mechanisms and chained "redirect" modifiers to a minimum. Domains SHOULD also try to minimize the amount of other DNS information needed to evaluate a record. This can be done by choosing directives that require less DNS information and placing lower cost mechanisms earlier in the SPF record.

For example, consider a domain set up as:

example.com.      IN MX   10 mx.example.com.
mx.example.com.   IN A    192.0.2.1
a.example.com.    IN TXT  "v=spf1 mx:example.com -all"
b.example.com.    IN TXT  "v=spf1 a:mx.example.com -all"
c.example.com.    IN TXT  "v=spf1 ip4:192.0.2.1 -all"

Evaluating check_host() for the domain "a.example.com" requires the MX records for "example.com", and then the A records for the listed hosts. Evaluating for "b.example.com" only requires the A records. Evaluating for "c.example.com" requires none.

However, there may be administrative considerations: Using "a" over "ip4" allows hosts to be renumbered easily. Using "mx" over "a" allows the set of mail hosts to be changed easily.

7.2 The Received-SPF header

It is RECOMMENDED that SMTP receivers record the result of SPF processing in the message headers. If an SMTP receiver chooses to do so, it SHOULD use the "Received-SPF" header defined here. This information is intended for the recipient. (Information intended for the sender described in Section 6.2exp: Explanation, Explanation.)

The header SHOULD be prepended to existing headers, above the Received: header that is generated by the SMTP receiver. It MUST appear above any other Received-SPF headers in the message. The header has the format:

header         = "Received-SPF:" [CFWS] result [CFWS]
                 [ key-value-list ]

result         = "Pass" / "Fail" / "TempError" / "SoftFail" /
                 "Neutral" / "None" / "PermError"

key-value-list = key-value-pair *( ";" [CFWS] key-value-pair ) [";"]

key-value-pair = name [CFWS] "=" ( dot-atom / quoted-string )

dot-atom       = ; unquoted word as per RFC2822

quoted-string  = ; quoted string as per RFC2822

CFWS           = ; comment or folding white space as per RFC2822

The <comment-string> should convey supporting information for the result, such as <ip>, <sender> and <domain>.

SPF clients may append zero or more of the following key-value-pairs at their discretion:

receiver
the host name of the SPF client
client-ip
the IP address of the SMTP client
envelope-from
the envelope sender address
helo
the host name given in the HELO or EHLO command
mechanism
the mechanism that matched (if no mechanisms matched, substitute the word "default".)
problem
if an error was returned, details about the error

Other key-value pairs may be defined by SPF clients. Until a new key name becomes widely accepted, new key names should start with "x-".

SPF clients MUST make sure that the Received-SPF header does not contain invalid characters, is excessively long, or contain malicious data that has been provided by the sender.

Examples of various header styles that could be generated:

Received-SPF: Pass (mybox.example.org: domain of
   myname@example.com designates 192.0.2.1 as permitted sender)
      receiver=mybox.example.org; client-ip=192.0.2.1;
      envelope-from=<myname@example.com>; helo=foo.example.com;

Received-SPF: Fail (mybox.example.org: domain of
                    myname@example.com does not designate
                    192.0.2.1 as permitted sender)
                    receiver=mybox.example.org;
                    client-ip=192.0.2.1;
                    envelope-from=<myname@example.com>;
                    helo=foo.example.com;

Received-SPF: SoftFail (mybox.example.org: domain of
                        transitioning myname@example.com discourages
                        use of 192.0.2.1 as permitted sender)

Received-SPF: Neutral (mybox.example.org: 192.0.2.1 is neither
                       permitted nor denied by domain of
                       myname@example.com)

Received-SPF: None (mybox.example.org: myname@example.com does
                    not designate permitted sender hosts)

Received-SPF: PermError (mybox.example.org: domain
                         of myname@example.com used an invalid
                         SPF mechanism)

Received-SPF: TempError (mybox.example.org: error in processing
                     during lookup of myname@example.com: DNS
                     timeout)



 TOC 

8. Macros

8.1 Macro definitions

Many mechanisms and modifiers perform macro interpolation on part of the term.

domain-spec    = macro-string domain-end
domain-end     = ( "." toplabel ) / macro-expand
toplabel       = ALPHA / ALPHA *[ alphanum / "-" ] alphanum
                 ; LDH rule (See RFC3696)
alphanum       = ALPHA / DIGIT
macro-string   = *( macro-expand / macro-literal )
explain-string = *( macro-string / SP )
macro-expand   = ( "%{" macro-letter transformer *delimiter "}" )
                 / "%%" / "%_" / "%-"
macro-literal  = %x21-24 / %x26-7E
               ; visible characters except "%"
macro-letter   = "s" / "l" / "o" / "d" / "i" / "p" / "h" /
                 "c" / "r" / "t"
transformer    = *DIGIT [ "r" ]
delimiter      = "." / "-" / "+" / "," / "/" / "_" / "="

A literal "%" is expressed by "%%".

"%_" expands to a single " " space.

"%-" expands to a URL-encoded space, viz. "%20".

The following macro letters are expanded in term arguments:

s
= <sender>
l
= local-part of <sender>
o
= domain of <sender>
d
= <domain>
i
= <ip>
p
= the validated domain name of <ip>
v
= the string "in-addr" if <ip> is ipv4, or "ip6" if <ip> is ipv6
h
= HELO/EHLO domain

The following macro letters are only allowed in "exp" text:

c
= SMTP client IP (easily readable format)
r
= domain name of host performing the check
t
= current timestamp

A '%' character not followed by a '{', '%', '-', or '_' character is a syntax error. So,

-exists:%(ir).sbl.spamhaus.org

is incorrect and will cause check_host() to return a "PermError". Instead, say

-exists:%{ir}.sbl.spamhaus.org

Optional transformers are:

*DIGIT
: zero or more digits
'r'
: reverse value, splitting on dots by default

If transformers or delimiters are provided, the replacement value for a macro letter is split into parts. After performing any reversal operation and/or removal of left-hand parts, the parts are rejoined using "." and not the original splitting characters.

By default, strings are split on "." (dots). Note that no special treatment is given to leading, trailing or consecutive delimiters, and so the list of parts may contain empty strings. Macros may specify delimiter characters which are used instead of ".".

The 'r' transformer indicates a reversal operation: if the client IP address were 192.0.2.1, the macro %{i} would expand to "192.0.2.1" and the macro %{ir} would expand to "1.2.0.192".

The DIGIT transformer indicates the number of right-hand parts to use, after optional reversal. If a DIGIT is specified, the value MUST be nonzero. If no DIGITs are specified, or if the value specifies more parts than are available, all the available parts are used. If the DIGIT was 5, and only 3 parts were available, the macro interpreter would pretend the DIGIT was 3. Implementations MUST support at least a value of 128, as that is the maximum number of labels in a domain name.

The "s" macro expands to the <sender> argument. It is an e-mail address with a localpart, an "@" character, and a domain. The "l" macro expands to just the localpart. The "o" macro expands to just the domain part. Note that these values remain the same during recursive and chained evaluations due to "include" and/or "redirect". Note also that if the original <sender> had no localpart, the localpart was set to "postmaster" in initial processing (see Section 4.3Initial Processing).

For IPv4 addresses, both the "i" and "c" macros expand to the standard dotted-quad format.

For IPv6 addresses, the "i" macro expands to a dot-format address; it is intended for use in %{ir}. The "c" macro may expand to any of the hexadecimal colon-format addresses specified in [RFC3513]Hinden, R. and S. Deering, Internet Protocol Version 6 (IPv6) Addressing Architecture, April 2003. section 2.2. It is intended for humans to read.

The "p" macro expands to the validated domain name of <ip>. The procedure for finding the validated domain name is defined in Section 5.5"ptr". If the <domain> is present in the list of validated domains, it SHOULD be used. Otherwise, if a subdomain of the <domain> is present, it SHOULD be used. Otherwise, any name from the list may be used. If there are no validated domain name or if a DNS error occurs, the string "unknown" is used.

The "r" macro expands to the name of the receiving MTA. This SHOULD be a fully qualified domain name, but if one does not exist (as when the checking is done by a script) or if policy restrictions dictate otherwise, the word "unknown" should be substituted. The domain name may be different than the name found in the MX record that the client MTA used to locate the receiving MTA.

The "t" macro expands to the decimal representation of the approximate number of seconds since the Epoch (Midnight, January 1st, 1970, UTC). This is the same value as returned by the POSIX time() function in most standards compliant libraries.

When the result of macro expansion is used in a domain name query, if the expanded domain name exceeds 253 characters (the maximum length of a domain name), the left side is truncated to fit, by removing successive subdomains until the total length does not exceed 253 characters.

Uppercased macros expand exactly as their lower case equivalents, and are then URL escaped. URL escaping for the non-uric characters is described in [RFC2396]Berners-Lee, T., Fielding, R. and L. Masinter, Uniform Resource Identifiers (URI): Generic Syntax, August 1998..

Note: Domains should avoid using the "s", "l", "o" or "h" macros in conjunction with any mechanism directive. While these macros are powerful and allow per-user records to be published, they severely limit the ability of implementations to cache results of check_host() and they reduce the effectiveness of DNS caches.

Implementations should be aware that if no directive processed during the evaluation of check_host() contains an "s", "l", "o" or "h" macro, then the results of the evaluation can be cached on the basis of <domain> and <ip> alone for as long as the shortest TTL of all the DNS records involved.

8.2 Expansion Examples

The <sender> is strong-bad@email.example.com.

The IPv4 SMTP client IP is 192.0.2.3.

The IPv6 SMTP client IP is 5f05:2000:80ad:5800::1.

The PTR domain name of the client IP is mx.example.org.

macro                       expansion
-------  ----------------------------
%{s}     strong-bad@email.example.com
%{o}                email.example.com
%{d}                email.example.com
%{d4}               email.example.com
%{d3}               email.example.com
%{d2}                     example.com
%{d1}                             com
%{dr}               com.example.email
%{d2r}                  example.email
%{l}                       strong-bad
%{l-}                      strong.bad
%{lr}                      strong-bad
%{lr-}                     bad.strong
%{l1r-}                        strong
macro-string                                               expansion
--------------------------------------------------------------------
%{ir}.%{v}._spf.%{d2}             3.2.0.192.in-addr._spf.example.com
%{lr-}.lp._spf.%{d2}                  bad.strong.lp._spf.example.com

%{lr-}.lp.%{ir}.%{v}._spf.%{d2}
bad.strong.lp.3.2.0.192.in-addr._spf.example.com

%{ir}.%{v}.%{l1r-}.lp._spf.%{d2}
3.2.0.192.in-addr.strong.lp._spf.example.com

%{d2}.trusted-domains.example.net
example.com.trusted-domains.example.net

IPv6:
%{ir}.%{v}._spf.%{d2}         1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.
5.d.a.0.8.0.0.0.2.5.0.f.5.ip6._spf.example.com


 TOC 

9. Implications

This section outlines the major implications that adoption of this document will have on various entities involved in Internet e-mail. It is intended to make clear to the reader where this document knowingly affects the operation of such entities. This section is not a "how-to" manual, nor a "best practices" document, and is not a comprehensive list of what such entities should do in light of this document.

This section is non-normative.

9.1 Sending Domains

Domains that wish to be compliant with this specification will need to determine the list of hosts that they allow to use their domain name in the "HELO" and "MAIL FROM" identities. It is recognized that forming such a list is not just a simple technical exercise, but involves policy decisions with both technical and administrative considerations.

It can be helpful to publish records that include a "tracking exists:" mechanism. By looking at the name server logs, an incompletely list may be generated. For example:

v=spf1 exists:CL.%{i}.FR.%{s}.HE.%{h}._spf.%{d} ?all"

9.2 Mailing Lists

Mailing lists must be aware of how they re-inject mail that is sent to the list. Mailing lists MUST comply with the requirement in [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001. Section 3.10 and [RFC1123]Braden, R., Requirements for Internet Hosts - Application and Support, October 1989. Section 5.3.6 that say that the reverse-path MUST be changed to be the address of a person or other entity who administers the list. While the reasons for changing the reverse-path are many and long standing, SPF adds enforcement to this requirement.

In practice, almost all mailing list software in use already complies with this requirement. Mailing lists that do not comply, may or may not encounter problems depending on how access to the list is restricted. Such lists that are entirely internal to a domain (only people in the domain can send to or receive from the list) are not affected.

9.3 Forwarding Services and Aliases

Forwarding services take mail that is received at a mailbox and direct it to some external mailbox. At the time of this writing, the near-universal practice of such services is to use the original reverse-path of a message when re-injecting it for delivery to the external mailbox. [RFC1123]Braden, R., Requirements for Internet Hosts - Application and Support, October 1989. and [RFC2821]Klensin, J., Simple Mail Transfer Protocol, April 2001. describe this action as an "alias" rather than a "mail list". This means the external mailbox's MTA sees all such mail in a connection from a host of the forwarding service, and so the "MAIL FROM" identity will not in general pass authorization.

There are several possible ways that this authorization failure can be ameliorated. If the owner of the external mailbox wishes to trust the forwarding service, they can direct the external mailbox's MTA to skip such tests when the client host belongs to the forwarding service. Tests against some other identity may also be used to override the test against the "MAIL FROM" identity.

For larger domains, it may not be possible to have a complete or accurate list of forwarding services used by the owners of the domain's mailboxes. In such cases, white lists of generally recognized forwarding services could be employed.

Forwarding services can also solve the problem by using reverse-paths that contain their own domain. This means that mail bounced from the external mailbox will have to be re-bounced by the forwarding service. Various schemes to do this exist though they vary widely in complexity and resource requirements on the part of the forwarding service. Several polular MTAs can change "alias" semantics to "mailing list" semantics by including an adding another alias with "owner-" added to the beginning of the alias name. (e.g. an alias of "friends: george@example.com, fred@example.org" would need another alias of the form "owner-friends: localowner")

9.4 Mail Services

Entities that offer mail services to other domains such as sending of bulk mail will may have to alter their mail in light of the authorization check in this document. If the reverse-path used for such e-mail uses the domain of the mail service provider, then the provider needs only to ensure that their sending host is authorized by their own SPF record, if any.

If the reverse-path does not use the mail service provider's domain, then extra care must be taken. The SPF record format has several options for authorizing the sending MTAs of another domain (the service provider's)

9.5 MTA Relays

The authorization check generally precludes the use of arbitrary MTA relays between sender and receiver of an e-mail message.

Within an organization, MTA relays can be effectively deployed. However, for purposes of this document, such relays are effectively invisible. The "MAIL FROM" identity authorization check is a check between border MTAs.

For mail senders, this means that published SPF records must authorize any MTAs that actually send across the Internet. Usually, these are just the border MTAs as internal MTAs simply forward mail to these MTAs for delivery.

Mail receivers will generally want to perform the authorization check at the border MTAs. This allows mail that fails to be rejected during the SMTP session rather than bounced. Internal MTAs then do not perform the authorization test. To perform the authorization test other than at the border, the host that first transferred the message to the organization must be determined, which can be difficult to extract from headers. Testing other than at the border is not recommended.



 TOC 

10. Security Considerations

The "MAIL FROM" and "HELO" identity authorizations must not be construed to provide more assurance than it does. It is entirely possible for a malicious sender to inject a message using their own domain in the identities used by SPF, to have that domain's SPF record authorize the sending host, and yet the message content can easily claim other identities in the headers. Unless the user, or the MUA takes care to note that the authorized identity does not match the other, more commonly presented identities (such as the From: header), the user may be lulled into a false sense of security.

There are two aspects of this protocol that malicious parties could exploit to undermine the validity of the check_host() function:

The evaluation of check_host() relies heavily on DNS. A malicious attacker could attack the DNS infrastructure and cause check_host() to see spoofed DNS data, and then return incorrect results. This could include returning "Pass" for an <ip> value where the actual domain's record would evaluate to "Fail". See [RFC3833]Atkins, D. and R. Austein, Threat Analysis of the Domain Name System (DNS), August 2004. for a description of the DNS weaknesses.

The client IP address, <ip>, is assumed to be correct. A malicious attacker could spoof TCP sequence numbers to make mail appear to come from a permitted host for a domain that the attacker is impersonating.

As with most aspects of mail, there are a number of ways that malicious parties could use the protocol as an avenue of a Denial-of-Service (DoS) attack. The processing limits outlined in Section 7.1Processing Limits are designed to prevent attacks such as:

Malicious parties could create SPF records that make many references to the target's domain and then send large volume mail to other SPF clients that use this SPF record. These legitimate machines would then present a DNS load on the target as they fetched the relevant DNS references.

While implementations of check_host() need to limit the number of DNS lookups, malicious domains could publish records that exercise or exceed these limits in an attempt to waste computation effort at their targets when they send them mail. Malicous domains could also design SPF records that cause excessive memory or CPU usage.

Malicious parties could send large volume mail purporting to come from the intended target to a wide variety of legitimate mail hosts. These legitimate machines would then present a DNS load on the target as they fetched the relevant records.

When the authorization check fails, an explanation string may be included in the reject response. Both the sender and the rejecting receiver need to be aware that the explanation was determined by the publisher of the SPF record checked and, in general, not the receiver. The explanation may contain URLs that may be malicious, offensive and/or have misleading text. This is probably less of a concern than it may initially seem since such messages are returned to the sender, and the source is the SPF record published by the domain in the identity claimed by that very sender. To put it another way, the only people who see malicious explanation strings are people whose messages claim to be from domains that publish such strings in their SPF records.

SPF uses information supplied by third parties, such as the HELO domain name, the return-path and SPF records. This information is then sent to the receiver in the Received-SPF: mail headers and possibly returned to the client MTA in the form of an SMTP rejection message. This information must be checked for invalid characters and excessively long lines.



 TOC 

11. IANA Considerations

The IANA needs to assign a new Resource Record Type and Qtype from the DNS Parameters Registry for the SPF RR type.



 TOC 

12. Contributors and Acknowledgements

This document is largely based on the work of Meng Weng Wong and Mark Lentczner. Mark is not listed as an author by his request. While, as this section acknowledges, many people have contributed to this document, a very large portion of the writing and editing are due to Meng and Mark.

This design owes a debt of parentage to [RMX]Danish, H., The RMX DNS RR Type for light weight sender authentication, October 2003. by Hadmut Danisch and to [DMP]Fecyk, G., Designated Mailers Protocol, December 2003. by Gordon Fecyk. The idea of using a DNS record to check the legitimacy of an e-mail address traces its ancestry farther back through messages on the namedroppers mailing list by Paul Vixie [Vixie]Vixie, P., Repudiating MAIL FROM, 2002. (based on suggestion by Jim Miller) and by David Green [Green]Green, D., Domain-Authorized SMTP Mail, 2002..

Philip Gladstone contributed macros to the specification, multiplying the expressiveness of the language and making per-user and per-IP lookups possible.

The authors would also like to thank the literally hundreds of individuals who have participated in the development of this design. There are far too numerous to name, but they include:

The folks on the spf-discuss mailing list.

The folks on the SPAM-L mailing list.

The folks on the IRTF ASRG mailing list.

The folks on the IETF MARID mailing list.

The folks on #perl.



 TOC 

13. References



 TOC 

13.1 Normative References

[RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987.
[RFC1123] Braden, R., "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, October 1989.
[RFC2034] Freed, N., "SMTP Service Extension for Returning Enhanced Error Codes", RFC 2034, October 1996 (TXT, HTML, XML).
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 (TXT, HTML, XML).
[RFC2181] Elz, R. and R. Bush, "Clarifications to the DNS Specification", RFC 2181, July 1997 (TXT, HTML, XML).
[RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997.
[RFC2396] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource Identifiers (URI): Generic Syntax", RFC 2396, August 1998 (TXT, HTML, XML).
[RFC2821] Klensin, J., "Simple Mail Transfer Protocol", RFC 2821, April 2001.
[RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April 2001.
[RFC3513] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6) Addressing Architecture", RFC 3513, April 2003.


 TOC 

13.2 Informative References

[RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, November 1987.
[RFC1983] Malkin, G., "Internet Users' Glossary", RFC 1983, August 1996.
[RFC2162] Allocchio, C., "MaXIM-11 - Mapping between X.400 / Internet mail and Mail-11 mail", RFC 2162, January 1998.
[RFC3696] Klensin, J., "Application Techniques for Checking and Transformation of Names", RFC 3696, February 2004.
[RFC3833] Atkins, D. and R. Austein, "Threat Analysis of the Domain Name System (DNS)", RFC 3833, August 2004.
[RMX] Danish, H., "The RMX DNS RR Type for light weight sender authentication", October 2003.

Work In Progress

[DMP] Fecyk, G., "Designated Mailers Protocol", December 2003.

Work In Progress

[Vixie] Vixie, P., "Repudiating MAIL FROM", 2002.
[Green] Green, D., "Domain-Authorized SMTP Mail", 2002.


 TOC 

Authors' Addresses

  Meng Weng Wong
  Singapore
EMail:  mengwong+spf@pobox.com
URI:  http://spf.pobox.com/
  
  Wayne Schlitt
  4615 Meredeth #9
  Lincoln Nebraska, NE 68506
  United States of America
EMail:  wayne@schlitt.net
URI:  http://www.schlitt.net/spf/


 TOC 

Appendix A. Collected ABNF

This section is normative and any discrepancies with the ABNF fragments in the preceding text are to be resolved in favor of this grammar.

See [RFC2234]Crocker, D. and P. Overell, Augmented BNF for Syntax Specifications: ABNF, November 1997. for ABNF notation. Please note that as per this ABNF definition, literal text strings (those in quotes) are case-insensitive. Hence, "mx" matches "mx", "MX", "mX" and "Mx".

record         = version terms *SP
version        = "v=spf1"

terms          = *( 1*SP ( directive / modifier ) )

directive      = [ prefix ] mechanism
prefix         = "+" / "-" / "?" / "~"
mechanism      = ( all / include
                 / A / MX / PTR / IP4 / IP6 / exists )

all            = "all"
include        = "include"  ":" domain-spec
A              = "a"      [ ":" domain-spec ] [ dual-cidr-length ]
MX             = "mx"     [ ":" domain-spec ] [ dual-cidr-length ]
PTR            = "ptr"    [ ":" domain-spec ]
IP4            = "ip4"      ":" ip4-network   [ ip4-cidr-length ]
IP6            = "ip6"      ":" ip6-network   [ ip6-cidr-length ]
exists         = "exists"   ":" domain-spec

modifier       = redirect / explanation / unknown-modifier
redirect       = "redirect" "=" domain-spec
explanation    = "exp" "=" domain-spec
unknown-modifier = name "=" macro-string

ip4-cidr-length  = "/" 1*DIGIT
ip6-cidr-length  = "/" 1*DIGIT
dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ]

ip4-network    = ; as per conventional dotted quad notation,
          ; e.g. 192.0.2.0
ip6-network    = ; as per [RFC 3513], section 2.2,
          ; e.g. 2001:DB8::CD30

domain-spec    = macro-string domain-end
domain-end     = ( "." toplabel ) / macro-expand
toplabel       = ALPHA / ALPHA *[ alphanum / "-" ] alphanum
                 ; LDH rule (See RFC3696)
alphanum       = ALPHA / DIGIT
macro-string   = *( macro-expand / macro-literal )
explain-string = *( macro-string / SP )
macro-expand   = ( "%{" macro-letter transformer *delimiter "}" )
                 / "%%" / "%_" / "%-"
macro-literal  = %x21-24 / %x26-7E
               ; visible characters except "%"
macro-letter   = "s" / "l" / "o" / "d" / "i" / "p" / "h" /
                 "c" / "r" / "t"
transformer    = *DIGIT [ "r" ]
delimiter      = "." / "-" / "+" / "," / "/" / "_" / "="

name           = ALPHA *( ALPHA / DIGIT / "-" / "_" / "." )

header         = "Received-SPF:" [CFWS] result [CFWS]
                 [ key-value-list ]

result         = "Pass" / "Fail" / "TempError" / "SoftFail" /
                 "Neutral" / "None" / "PermError"

key-value-list = key-value-pair *( ";" [CFWS] key-value-pair ) [";"]

key-value-pair = name [CFWS] "=" ( dot-atom / quoted-string )

dot-atom       = ; unquoted word as per RFC2822

quoted-string  = ; quoted string as per RFC2822

CFWS           = ; comment or folding white space as per RFC2822


 TOC 

Appendix B. Extended Examples

These examples are based on the following DNS setup:

; A domain with two mail servers, two hosts
; and two servers at the domain name

$ORIGIN example.com.
@           MX  10 mail-a
MX  20 mail-b
A   192.0.2.10
A   192.0.2.11
amy         A   192.0.2.65
bob         A   192.0.2.66
mail-a      A   192.0.2.129
mail-b      A   192.0.2.130
www         CNAME example.com.

; A related domain

$ORIGIN example.org
@           MX  10 mail-c
mail-c      A   192.0.2.140

; The reverse IP for those addresses

$ORIGIN 2.0.192.in-addr.arpa.
10          PTR example.com.
11          PTR example.com.
65          PTR amy.example.com.
66          PTR bob.example.com.
129         PTR mail-a.example.com.
130         PTR mail-b.example.com.
140         PTR mail-c.example.org.

; A rogue reverse IP domain that claims to be
; something it's not

$ORIGIN 0.0.10.in-addr.arpa.
4           PTR bob.example.com.

B.1 Simple Examples

These examples show various possible published records for example.com and which values if <ip> would cause check_host() to return "Pass". Note that <domain> is "example.com".

v=spf1 +all

-- any <ip> passes

v=spf1 a -all

-- hosts 192.0.2.10 and 192.0.2.11 pass

v=spf1 a:example.org -all

-- no sending hosts pass since example.org has no A records

v=spf1 mx -all

-- sending hosts 192.0.2.129 and 192.0.2.130 pass

v=spf1 mx:example.org -all

-- sending host 192.0.2.140 passes

v=spf1 mx mx:example.org -all

-- sending hosts 192.0.2.129, 192.0.2.130, and 192.0.2.140 pass

v=spf1 mx/30 mx:example.org/30 -all

-- any sending host in 192.0.2.128/30 or 192.0.2.140/30 passes

v=spf1 ptr -all

-- sending host 192.0.2.65 passes (reverse IP is valid and in example.com)

-- sending host 192.0.2.140 fails (reverse IP is valid, but not in example.com)

-- sending host 10.0.0.4 fails (reverse IP is not valid)

v=spf1 ip4:192.0.2.128/28 -all

-- sending host 192.0.2.65 fails

-- sending host 192.0.2.129 passes

B.2 Multiple Domain Example

These examples show the effect of related records:

example.org: "v=spf1 include:example.com include:example.net -all"

This record would be used if mail from example.org actually came through servers at example.com and example.net. Example.org's designated servers are the union of example.com and example.net's designated servers.

la.example.org: "v=spf1 redirect=example.org"

ny.example.org: "v=spf1 redirect=example.org"

sf.example.org: "v=spf1 redirect=example.org"

These records allow a set of domains that all use the same mail system to make use of that mail system's record. In this way, only the mail system's record needs to updated when the mail setup changes. These domains' records never have to change.

B.3 DNSBL Style Example

Imagine that, in addition to the domain records listed above, there are these:

$Origin _spf.example.com.
mary.mobile-users                   A 127.0.0.2
fred.mobile-users                   A 127.0.0.2
15.15.168.192.joel.remote-users     A 127.0.0.2
16.15.168.192.joel.remote-users     A 127.0.0.2

The following records describe users at example.com who mail from arbitrary servers, or who mail from personal servers.

example.com:

v=spf1 mx
       include:mobile-users._spf.%{d}
       include:remote-users._spf.%{d}
       -all

mobile-users._spf.example.com:

v=spf1 exists:%{l1r+}.%{d}

remote-users._spf.example.com:

v=spf1 exists:%{ir}.%{l1r+}.%{d}



 TOC 

Intellectual Property Statement

Disclaimer of Validity

Copyright Statement

Acknowledgment