--- ../rfc/draft-lentczner-spf-00.clean.txt 2004-11-02 10:41:20.000000000 -0600 +++ draft-schlitt-spf-classic-01pre3.clean.txt 2005-04-27 17:02:32.000000000 -0500 @@ -1,12 +1,12 @@ -Network Working Group M. Lentczner -Internet-Draft M. Wong -Expires: April 12, 2005 October 12, 2004 +Network Working Group M. Wong +Internet-Draft W. Schlitt +Expires: October 26, 2005 April 27, 2005 - Sender Policy Framework: Authorizing Use of Domains in MAIL FROM - draft-lentczner-spf-00 + Sender Policy Framework: Authorizing Use of Domains in E-MAIL + draft-schlitt-spf-classic-01 Status of this Memo @@ -33,93 +33,99 @@ The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on April 12, 2005. + This Internet-Draft will expire on October 26, 2005. Copyright Notice - Copyright (C) The Internet Society (2004). + Copyright (C) The Internet Society (2005). Abstract - Mail on the Internet can be forged in a number of ways. In + E-mail on the Internet can be forged in a number of ways. In particular, existing protocols place no restriction in what a sending host can use as the reverse-path of a message. This document - describes a protocol whereby a domain can explicitly authorize the - hosts that are allowed to use its domain name in a reverse-path, and - a way for receiving hosts to check such authorization. + describes version 1 of the SPF protocol, whereby a domain can + explicitly authorize the hosts that are allowed to use its domain + name in a reverse-path, and a way for receiving hosts to check such + authorization. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1 Protocol Status . . . . . . . . . . . . . . . . . . . . . 4 - 1.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 + 1.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 2. Operation . . . . . . . . . . . . . . . . . . . . . . . . . . 6 - 2.1 The Mail From Identity . . . . . . . . . . . . . . . . . . 6 - 2.2 Publishing Authorization . . . . . . . . . . . . . . . . . 6 - 2.3 Checking Authorization . . . . . . . . . . . . . . . . . . 6 - 2.4 Interpreting the Result . . . . . . . . . . . . . . . . . 7 - 2.4.1 Neutral . . . . . . . . . . . . . . . . . . . . . . . 8 - 2.4.2 Pass . . . . . . . . . . . . . . . . . . . . . . . . . 8 - 2.4.3 Fail . . . . . . . . . . . . . . . . . . . . . . . . . 8 - 2.4.4 SoftFail . . . . . . . . . . . . . . . . . . . . . . . 8 - 2.4.5 None . . . . . . . . . . . . . . . . . . . . . . . . . 9 - 2.4.6 TempError . . . . . . . . . . . . . . . . . . . . . . 9 - 2.4.7 PermError . . . . . . . . . . . . . . . . . . . . . . 9 - 3. SPF Records . . . . . . . . . . . . . . . . . . . . . . . . . 10 - 3.1 Publishing . . . . . . . . . . . . . . . . . . . . . . . . 10 - 3.1.1 RR Types . . . . . . . . . . . . . . . . . . . . . . . 10 - 3.1.2 Multiple Records . . . . . . . . . . . . . . . . . . . 11 - 3.1.3 Additional Records . . . . . . . . . . . . . . . . . . 11 - 3.1.4 Multiple Strings . . . . . . . . . . . . . . . . . . . 11 - 3.1.5 Record Size . . . . . . . . . . . . . . . . . . . . . 12 - 3.1.6 Wildcard Records . . . . . . . . . . . . . . . . . . . 12 - 4. The check_host() Function . . . . . . . . . . . . . . . . . . 13 - 4.1 Arguments . . . . . . . . . . . . . . . . . . . . . . . . 13 - 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . 13 + 2.1 The HELO Identity . . . . . . . . . . . . . . . . . . . . 6 + 2.2 The MAIL FROM Identity . . . . . . . . . . . . . . . . . . 6 + 2.3 Publishing Authorization . . . . . . . . . . . . . . . . . 6 + 2.4 Checking Authorization . . . . . . . . . . . . . . . . . . 7 + 2.5 Interpreting the Result . . . . . . . . . . . . . . . . . 8 + 2.5.1 None . . . . . . . . . . . . . . . . . . . . . . . . . 8 + 2.5.2 Neutral . . . . . . . . . . . . . . . . . . . . . . . 8 + 2.5.3 Pass . . . . . . . . . . . . . . . . . . . . . . . . . 9 + 2.5.4 Fail . . . . . . . . . . . . . . . . . . . . . . . . . 9 + 2.5.5 SoftFail . . . . . . . . . . . . . . . . . . . . . . . 9 + 2.5.6 TempError . . . . . . . . . . . . . . . . . . . . . . 9 + 2.5.7 PermError . . . . . . . . . . . . . . . . . . . . . . 10 + 3. SPF Records . . . . . . . . . . . . . . . . . . . . . . . . . 11 + 3.1 Publishing . . . . . . . . . . . . . . . . . . . . . . . . 11 + 3.1.1 DNS Resource Record Types . . . . . . . . . . . . . . 11 + 3.1.2 Multiple Records . . . . . . . . . . . . . . . . . . . 12 + 3.1.3 Multiple Strings . . . . . . . . . . . . . . . . . . . 12 + 3.1.4 Record Size . . . . . . . . . . . . . . . . . . . . . 12 + 3.1.5 Wildcard Records . . . . . . . . . . . . . . . . . . . 13 + 4. The check_host() Function . . . . . . . . . . . . . . . . . . 14 + 4.1 Arguments . . . . . . . . . . . . . . . . . . . . . . . . 14 + 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.3 Initial Processing . . . . . . . . . . . . . . . . . . . . 14 4.4 Record Lookup . . . . . . . . . . . . . . . . . . . . . . 14 - 4.5 Selecting Records . . . . . . . . . . . . . . . . . . . . 14 + 4.5 Selecting Records . . . . . . . . . . . . . . . . . . . . 15 4.6 Record Evaluation . . . . . . . . . . . . . . . . . . . . 15 - 4.6.1 Term Evaluation . . . . . . . . . . . . . . . . . . . 15 + 4.6.1 Term Evaluation . . . . . . . . . . . . . . . . . . . 16 4.6.2 Mechanisms . . . . . . . . . . . . . . . . . . . . . . 16 - 4.6.3 Modifiers . . . . . . . . . . . . . . . . . . . . . . 16 - 4.7 Default result . . . . . . . . . . . . . . . . . . . . . . 17 - 4.8 Domain Spec . . . . . . . . . . . . . . . . . . . . . . . 17 - 5. Mechanism Definitions . . . . . . . . . . . . . . . . . . . . 18 - 5.1 "all" . . . . . . . . . . . . . . . . . . . . . . . . . . 18 - 5.2 "include" . . . . . . . . . . . . . . . . . . . . . . . . 19 - 5.3 "a" . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 - 5.4 "mx" . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 - 5.5 "ptr" . . . . . . . . . . . . . . . . . . . . . . . . . . 21 - 5.6 "ip4" and "ip6" . . . . . . . . . . . . . . . . . . . . . 21 - 5.7 "exists" . . . . . . . . . . . . . . . . . . . . . . . . . 22 - 6. Modifier Definitions . . . . . . . . . . . . . . . . . . . . . 23 - 6.1 redirect: Redirected Query . . . . . . . . . . . . . . . . 23 - 6.2 exp: Explanation . . . . . . . . . . . . . . . . . . . . . 24 - 7. Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . 26 - 7.1 Unrecognized Mechanisms and Modifiers . . . . . . . . . . 26 - 7.2 Processing Limits . . . . . . . . . . . . . . . . . . . . 26 - 8. Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 - 8.1 Macro definitions . . . . . . . . . . . . . . . . . . . . 28 - 8.2 Expansion Examples . . . . . . . . . . . . . . . . . . . . 31 - 9. Implications . . . . . . . . . . . . . . . . . . . . . . . . . 32 - 9.1 Sending Domains . . . . . . . . . . . . . . . . . . . . . 32 - 9.2 Mailing Lists . . . . . . . . . . . . . . . . . . . . . . 32 - 9.3 Forwarding Services . . . . . . . . . . . . . . . . . . . 32 - 9.4 Mail Services . . . . . . . . . . . . . . . . . . . . . . 33 - 9.5 MTA Relays . . . . . . . . . . . . . . . . . . . . . . . . 33 - 10. Security Considerations . . . . . . . . . . . . . . . . . . 35 - 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 37 - 12. Contributors and Acknowledgements . . . . . . . . . . . . . 38 - 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 - 13.1 Normative References . . . . . . . . . . . . . . . . . . . . 39 - 13.2 Informative References . . . . . . . . . . . . . . . . . . . 39 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 40 - A. Collected ABNF . . . . . . . . . . . . . . . . . . . . . . . . 41 - B. Extended Examples . . . . . . . . . . . . . . . . . . . . . . 43 - B.1 Simple Examples . . . . . . . . . . . . . . . . . . . . . 43 - B.2 Multiple Domain Example . . . . . . . . . . . . . . . . . 44 - B.3 RBL Style Example . . . . . . . . . . . . . . . . . . . . 45 - Intellectual Property and Copyright Statements . . . . . . . . 46 + 4.6.3 Modifiers . . . . . . . . . . . . . . . . . . . . . . 17 + 4.7 Default Result . . . . . . . . . . . . . . . . . . . . . . 17 + 4.8 Domain Specification . . . . . . . . . . . . . . . . . . . 17 + 5. Mechanism Definitions . . . . . . . . . . . . . . . . . . . . 19 + 5.1 "all" . . . . . . . . . . . . . . . . . . . . . . . . . . 19 + 5.2 "include" . . . . . . . . . . . . . . . . . . . . . . . . 20 + 5.3 "a" . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 + 5.4 "mx" . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 + 5.5 "ptr" . . . . . . . . . . . . . . . . . . . . . . . . . . 22 + 5.6 "ip4" and "ip6" . . . . . . . . . . . . . . . . . . . . . 23 + 5.7 "exists" . . . . . . . . . . . . . . . . . . . . . . . . . 23 + 6. Modifier Definitions . . . . . . . . . . . . . . . . . . . . . 25 + 6.1 redirect: Redirected Query . . . . . . . . . . . . . . . . 25 + 6.2 exp: Explanation . . . . . . . . . . . . . . . . . . . . . 26 + 7. The Received-SPF header . . . . . . . . . . . . . . . . . . . 28 + 8. Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 + 8.1 Macro definitions . . . . . . . . . . . . . . . . . . . . 30 + 8.2 Expansion Examples . . . . . . . . . . . . . . . . . . . . 33 + 9. Implications . . . . . . . . . . . . . . . . . . . . . . . . . 34 + 9.1 Sending Domains . . . . . . . . . . . . . . . . . . . . . 34 + 9.2 Mailing Lists . . . . . . . . . . . . . . . . . . . . . . 34 + 9.3 Forwarding Services and Aliases . . . . . . . . . . . . . 34 + 9.4 Mail Services . . . . . . . . . . . . . . . . . . . . . . 36 + 9.5 MTA Relays . . . . . . . . . . . . . . . . . . . . . . . . 36 + 10. Security Considerations . . . . . . . . . . . . . . . . . . 38 + 10.1 SPF-Authorized E-Mail May Still Be UBE . . . . . . . . . . 38 + 10.2 Spoofed DNS and IP Data . . . . . . . . . . . . . . . . . 38 + 10.3 Processing Limits . . . . . . . . . . . . . . . . . . . . 38 + 10.4 Untrusted Information Sources . . . . . . . . . . . . . . 40 + 10.5 Privacy Exposure . . . . . . . . . . . . . . . . . . . . . 40 + 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 41 + 11.1 The SPF DNS Record Type . . . . . . . . . . . . . . . . . 41 + 11.2 The Received-SPF mail header . . . . . . . . . . . . . . . 41 + 12. Contributors and Acknowledgements . . . . . . . . . . . . . 42 + 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 43 + 13.1 Normative References . . . . . . . . . . . . . . . . . . . . 43 + 13.2 Informative References . . . . . . . . . . . . . . . . . . . 43 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 44 + A. Collected ABNF . . . . . . . . . . . . . . . . . . . . . . . . 45 + B. Extended Examples . . . . . . . . . . . . . . . . . . . . . . 47 + B.1 Simple Examples . . . . . . . . . . . . . . . . . . . . . 47 + B.2 Multiple Domain Example . . . . . . . . . . . . . . . . . 48 + B.3 DNSBL Style Example . . . . . . . . . . . . . . . . . . . 49 + Intellectual Property and Copyright Statements . . . . . . . . 50 1. Introduction The current e-mail infrastructure has the property that any host @@ -127,17 +133,17 @@ name it wants. Hosts can do this at a variety of levels: in particular, the session, the envelope, and the mail headers. While this feature is desirable in some circumstances, it is a major - obstacle to reducing end-user unwanted e-mail (or "spam"). + obstacle to reducing Unsolicited Bulk E-mail (UBE, aka "spam"). Furthermore, many domain name holders are understandably concerned about the ease with which other entities may make use of their domain - names, often with intent to impersonate. + names, often with the intent to impersonate. - This document defines a protocol by which hosts may be authorized by - domains to use the domain name in the envelope "Mail From" identity. - Compliant domain name holders publish SPF records about which hosts - are permitted to use their names, and compliant mail receivers use - the published SPF records to test the authorization of hosts using a - given "Mail From" identity during a mail transaction. + This document defines a protocol by which domain owners may authorize + hosts to use their domain name in the "MAIL FROM" or "HELO" identity. + Compliant domain holders publish SPF records about which hosts are + permitted to use their names, and compliant mail receivers use the + published SPF records to test the authorization of hosts using a + given "HELO" or "MAIL FROM" identity during a mail transaction. An additional benefit to mail receivers is that when the use of an identity is verified, then local policy decisions about the mail can @@ -152,20 +158,19 @@ SPF has been in development since the Summer of 2003, and has seen deployment beyond the developers beginning in December, 2003. The - design of SPF has continuously evolved from them and is under active - development today. There have been quite a number of forms of SPF, - some written up as documents, some submitted as Internet Drafts, and - many discussed and debated in development forums. - - This document attempts to set down the common core of the SPF version - 1 protocol, as implemented and deployed since about December, 2003. - This conception of SPF is sometimes called "SPF Classic". The goal - of this document is to be a stable reference that can be used for - experimenting with existing implementations and developing SPF - further. It is understood that particular implementations and + design of SPF slowly evolved until the spring of 2004 and has since + stabilized. There have been quite a number of forms of SPF, some + written up as documents, some submitted as Internet Drafts, and many + discussed and debated in development forums. + + The goal of this document is to clearly document the protocol defined + by earlier drafts specifications of SPF as used in existing + implementations. This conception of SPF is sometimes called "SPF + Classic". It is understood that particular implementations and deployments may differ from, and build upon, this work. It is hoped that we have nonetheless captured the common understanding of SPF version 1. + 1.2 Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", @@ -174,69 +179,84 @@ This document is concerned with a portion of a mail message commonly called "envelope sender", "return path", "reverse path", "bounce - address", "2821 from", or "mail from". Since these terms are either + address", "2821 FROM", or "MAIL FROM". Since these terms are either not well defined, or often used casually, this document defines the - "Mail From" identity in Section 2.1. Note that other terms, that may - superficially look like the common terms, such as "reverse-path" or - "Return-Path" are used only with the defined meanings from normative - documents. + "MAIL FROM" identity in Section 2.2. Note that other terms, that may + superficially look like the common terms, such as "reverse-path", are + used only with the defined meanings from normative documents. 2. Operation -2.1 The Mail From Identity +2.1 The HELO Identity - The "Mail From" identity derives from the SMTP MAIL command (see - [RFC2821].) This command supplies the "reverse-path" for a message, - which generally consists of the sender mailbox, and is the mailbox to - which notification messages are sent if there are problems delivering - the message. + The "HELO" identity derives from either the SMTP HELO or EHLO command + (see [RFC2821]). These commands supply the SMTP client (sending + host) for the SMTP session. Note that requirements for the domain + presented in the EHLO or HELO command are not always clear to the + sending party, and SPF clients must be prepared for the "HELO" + identity to be malformed or an IP address literal. At the time of + this writting, many legitimate e-mails are delivered with invalid + HELO domains. + + It is RECOMMENDED that SPF clients check not only the "MAIL FROM" + identity, but also the "HELO" identity by applying the check_host() + function (Section 4) to the "HELO" identity as the . If the + HELO test is performed, and results in a "Fail", the overall result + for the SMTP session is "Fail", and there is no need to test the + "MAIL FROM" identity. - This document defines the "Mail From" identity to be mailbox portion - of the path of the reverse-path as defined in [RFC2821], Section - 4.1.2. when it is non-null. +2.2 The MAIL FROM Identity - [RFC2821] allows the reverse-path to be null (see Section 4.5.5.) In + The "MAIL FROM" identity derives from the SMTP MAIL command (see + [RFC2821]). This command supplies the "reverse-path" for a message, + which generally consists of the sender mailbox, and is the mailbox to + which notification messages are to be sent if there are problems + delivering the message. + + [RFC2821] allows the reverse-path to be null (see Section 4.5.5). In this case, there is no explicit sender mailbox, and such a message can be assumed to be a notification message from the mail system itself. When the reverse-path is null, this document defines the - "Mail From" identity to be the mailbox composed of the localpart - "postmaster" and the domain supplied with the SMTP EHLO or HELO - command. Note that requirements for the domain presented in the EHLO - and HELO commands are not strict, and software must be prepared for a - "Mail From" identity so constructed to be ill formed. - - Generally, software that performs the authorization checks described - below does so during a SMTP transaction, and so readily has the - information required at hand. - -2.2 Publishing Authorization - - An SPF compliant domain name MUST publish a valid SPF record as - described in Section 3. This record authorizes the use of the domain - name in the envelope "Mail From" identity, by some sending MTAs, and - not by others. - - Domains SHOULD publish SPF records that end in "-all", or redirect to - other records that do, so that a definitive determination of - authorization can be made. + "MAIL FROM" identity to be the mailbox composed of the localpart + "postmaster" and the "HELO" identity (which may or may not have been + checked separately before). + + SPF clients MUST check the "MAIL FROM" identity unless HELO testing + produced a "Fail". SPF clients check the "MAIL FROM" identity by + applying the check_host() function to the "MAIL FROM" identity as the + . + +2.3 Publishing Authorization + + An SPF compliant domain MUST publish a valid SPF record as described + in Section 3. This record authorizes the use of the domain name in + the "HELO" and "MAIL FROM" identity, by some sending MTAs, and not by + others. + It is RECOMMENDED that domains publish SPF records that end in + "-all", or redirect to other records that do, so that a definitive + determination of authorization can be made. Domain holders may publish SPF records that explicitly authorize no hosts for domain names that shouldn't be used in sender mailboxes. -2.3 Checking Authorization +2.4 Checking Authorization - A mail receiver can perform an SPF compliant check for each mail - message it receives. This check tests the authorization of a client - host to inject mail with a given "Mail From" identity. Typically, - such checks are done by a receiving MTA, but can be performed - elsewhere in the mail processing chain so long as the required - information is available. + A mail receiver can perform a set of SPF checks for each mail message + it receives. An SPF check tests the authorization of a client host + to emit mail with a given identity. Typically, such checks are done + by a receiving MTA, but can be performed elsewhere in the mail + processing chain so long as the required information is available and + reliable. At least the "MAIL FROM" identity MUST be checked, but it + is RECOMMENDED that the "HELO" identity also be checked beforehand. + Checking other identities against SPF records is NOT RECOMMENDED + because there are cases (e.g. Section 9.3) that are known to give + incorrect results. - It is expected that mail receivers will use the SPF check as part of + It is possible that mail receivers will use the SPF check as part of a larger set of tests on incoming mail. The results of other tests may influence whether or not a particular SPF check is performed. - For example, finding the sending host on a local white list may cause - all other tests to be skipped and all mail from that host to be - accepted. + For example, finding the sending host's IP address on a local white + list may cause all other tests to be skipped and all mail from that + host to be accepted. When a mail receiver decides to perform an SPF check, it MUST implement and evaluate the check_host() function (Section 4) @@ -245,170 +265,187 @@ the correct semantics are preserved between publisher and receiver. To make the test, the mail receiver MUST evaluate the check_host() - with the arguments set as follows: + function with the arguments set as follows: - - the IP address of the client host that is injecting the - mail - - the domain portion of the "Mail From" identity - - the "Mail From" identity + - the IP address of the SMTP client that is emitting the + mail, either IPv4 or IPv6. + - the domain portion of the "MAIL FROM" or "HELO" identity. + - the "MAIL FROM" or "HELO" identity. Note that the argument may not be a well formed domain name. - For example, if the reverse-path was null, then the EHLO or HELO - domain is used, and that can be an address literal or entirely - malformed in a valid SMTP transaction. In these cases, check_host() - is defined in Section 4.3 to return a Fail result. - - Software SHOULD perform this authorization check during the - processing of the SMTP transaction that injects the mail. This - allows errors to be returned directly to the injecting server by way - of SMTP replies. Software can perform the check as early as the MAIL - command, though it may be easier to delay the check to some later - stage of the transaction. - - Software can perform the authorization after the corresponding SMTP - transaction has completed. There are two problems with this - approach: 1) It may be difficult to accurately extract all the + For example, if the reverse-path was null, then the EHLO/HELO domain + is used, with its associated problems. (see Section 2.1) In these + cases, check_host() is defined in Section 4.3 to return a "None" + result. + + While invalid, malformed, or non-existent domains cause SPF checks to + return "none" because no SPF record can be found, it has long been + the policy of many MTAs to reject e-mail from such domains, + especially in the MAIL FROM. In order to prevent the circumvention + of SPF records, rejecting e-mail from invalid domains should be + considered. + + Care must be taken to correctly extract the from the + as many MTAs will still accept such things as source routes + (see [RFC2821] appendix C), the %-hack (see [RFC1123]) and bang paths + (see [RFC1983]). These archaic features have been maliciously used + to bypass security systems. + + This authorization check SHOULD be performed during the processing of + the SMTP transaction that sends the mail. This allows errors to be + returned directly to the sending server by way of SMTP replies. + + Software can also perform the authorization after the corresponding + SMTP transaction has completed. However there are two problems with + this approach: 1) It may be difficult to accurately extract all the required information such as client IP address and HELO domain name. 2) If the authorization fails, then generating a non-delivery - notification to the alleged sender is problematic as such an action - would go against the explicit wishes of the alleged sender. - -2.4 Interpreting the Result - - The check_host() function returns one of seven results, some with - additional information. This section describes how software that - performs the authorization must interpret the results. If the check - is being performed during the SMTP mail transaction, it also - describes how to respond. - -2.4.1 Neutral - - A Neutral result MUST be treated exactly like a None result. - -2.4.2 Pass - - A Pass result means that the client is authorized to inject mail with - the given "Mail From" identity. Further policy checks, such as - reputation, or black and/or white listing, can now proceed with - confidence based on the "Mail From" identity. - -2.4.3 Fail - - A Fail result is an explicit statement that the client is not - authorized to use the domain in the "Mail From" identity. The - checking software can choose to mark the mail based on this, or to - reject the mail outright. + notification to the alleged sender is problematic due to the large + number of forged e-mails on the Internet today. Such an action would + go against the explicit wishes of the alleged sender. + +2.5 Interpreting the Result + + The check_host() function returns one of several result codes. This + section describes how software that performs the authorization must + interpret the results. If the check is being performed during the + SMTP mail transaction, it also describes how to respond. + +2.5.1 None + + A result of "None" means that no records were published by the + domain, or that no checkable sender domain could be determined from + the given identity. The checking software cannot ascertain if the + client host is authorized or not. + +2.5.2 Neutral + + The domain owner has explicitly stated that they don't know whether + the IP address is authorized or not. A "Neutral" result MUST be + treated exactly like the "None" result; the distinction exists only + for informational purposes. +2.5.3 Pass + + A "Pass" result means that the client is authorized to inject mail + with the given identity. Further policy checks, such as reputation, + or black and/or white listing, can now proceed with confidence in the + identity. + +2.5.4 Fail + + A "Fail" result is an explicit statement that the client is not + authorized to use the domain in the given identity. The checking + software can choose to mark the mail based on this, or to reject the + mail outright. If the checking software chooses to reject the mail during the SMTP - transaction, then it MUST use a 550 reply code with an appropriate - message. The Fail result includes a reason. The reason can be used - to construct an appropriate message. If the reason is "Not - Permitted", then an explanation string is also returned. This - explanation string comes from the domain that published the SPF - records and may contain a URL. Since that information doesn't - originate with the checking software, the checking software will want - to make it clear that text is not trusted. Example reply messages - for rejecting are: - - 550 SPF Mail From check failed: Malformed Domain - - 550 SPF Mail From check failed: Domain Does Not Exist - - 550-SPF Mail From check failed: Not Permitted - 550-The domain example.com said: - 550 Please see http://www.example.com/mailpolicy.html - - -2.4.4 SoftFail - - A SoftFail result should be treated as somewhere between a Fail and a - Neutral. This value is used by domains as an intermediate state - during roll-out of publishing records. The domain believes the host - isn't authorized but isn't willing to make that strong of a - statement. Receiving software SHOULD NOT reject the message based on - this result, but MAY subject the message to closer scrutiny. - -2.4.5 None - - A result of None means that no records were published by the domain. - The checking software cannot ascertain if the client host is - authorized or not. + transaction, then it SHOULD use an SMTP reply code of 550 (see + [RFC2821]) and, if supported, the 5.7.1 DSN code (see [RFC2034]), in + addition to an appropriate reply text. The check_host() function may + return either a default explanation string, or one from the domain + that published the SPF records (see Section 6.2). If the information + doesn't originate with the checking software, it should be made clear + that text is not trusted. For example: + + 550-5.7.1 SPF MAIL FROM check failed: + 550-5.7.1 The domain example.com explains: + 550 5.7.1 Please see http://www.example.com/mailpolicy.html + + +2.5.5 SoftFail + + A "SoftFail" result should be treated as somewhere between a "Fail" + and a "Neutral". The domain believes the host isn't authorized but + isn't willing to make that strong of a statement. Receiving software + SHOULD NOT reject the message based on this result, but MAY subject + the message to closer scrutiny. + + Since the domain has discouraged the use of this host, receivers MAY + try to inform either the sender or the recipient of the e-mail. For + example, the recipient's MUA could highlight the "SoftFail" status. + Or the MTA could give the sender a message using a technique called + "greylisting" where by the MTA can issue an SMTP reply code of 451 + (4.3.0 DSN code) with a note the first time the message was received, + but accept it the second time. -2.4.6 TempError +2.5.6 TempError - A TempError result means that the receiving server encountered a + A "TempError" result means that the SPF client encountered a transient error when performing the check. Checking software can choose to accept or temporarily reject the message. If the message is rejected during the SMTP transaction for this reason, the software - MUST use a 450 reply code. + SHOULD use an SMTP reply code of 451 and, if supported, the 4.4.3 DSN + code. -2.4.7 PermError +2.5.7 PermError - A PermError result means that the domain's published records couldn't - be correctly interpreted for this "Mail From" identity. Checking - software SHOULD reject the message. If rejecting during SMTP - transaction time, a 550 reply MUST be used. + A "PermError" result means that the domain's published records + couldn't be correctly interpreted. Checking software SHOULD reject + the message with an SMTP reply code of 550 and, if supported, the + 5.5.2 DSN code. 3. SPF Records - An SPF record declares which hosts are, and are not, authorized to - use a domain name for the "Mail From" identity. Loosely, the record - partitions all hosts into permitted and not-permitted sets. (Though - some hosts might fall into other categories.) + An SPF record is a DNS Resource Record (RR) that declares which hosts + are, and are not, authorized to use a domain name for the "HELO" and + "MAIL FROM" identities. Loosely, the record partitions all hosts + into permitted and not-permitted sets. (Though some hosts might fall + into neither category.) The SPF record is a single string of text. An example record is: - v=spf1 +mx +a:colo.example.com/28 -all + v=spf1 +mx a:colo.example.com/28 -all - This record has a version of "v=spf1" and three directives: "+mx", - "+a:colo.example.com/28", and "-all". + This record has a version of "spf1" and three directives: "+mx", + "a:colo.example.com/28" (the + is implied), and "-all". 3.1 Publishing - A domain name's SPF record is published in DNS. The record is placed - in the DNS tree at the domain name it pertains to. + Domain owners wishing to be SPF compliant must publish SPF records + for the hosts that are used in the "MAIL FROM" and "HELO" identities. + The SPF records are placed in the DNS tree at the host name it + pertains to, not a subdomain under it, such as is done with SRV + records. This is the same whether the TXT or SPF RR type is used. - The previous example might be published easily via this line in a - domain zone file: + The example above in Section 3 might be published easily via this + lines in a domain zone file: - example.com. IN SPF "v=spf1 +mx +a:colo.example.com/28 -all" + example.com. IN TXT "v=spf1 +mx a:colo.example.com/28 -all" + smtp-out.example.com. IN TXT "v=spf1 a -all" - Note: The record is published at the domain name to which it - pertains, not a name within the domain (such as is done with SRV - records.) When published using the SPF RR type (see below), this - poses no problems and was chosen as the clearest way to express the - declaration. When published via TXT records it is still published - directly at the domain name, even though other TXT records, for other - purposes may be published there. + When publishing via TXT records, beware of other TXT records + published there for other purposes. They may cause problems with + size limits (see Section 3.1.4). -3.1.1 RR Types +3.1.1 DNS Resource Record Types - This document defines a new DNS RR type SPF, type code to be + This document defines a new DNS RR of type SPF, type code to be determined. The format of this type is identical to the TXT RR - [RFC1035]. + [RFC1035]. For either type, the character content of the record is + encoded as US-ASCII. - However, because there are a number of DNS server and resolver - implementations in common use that cannot handle new RR types, a - record can be published with type TXT. + It is recognized that the current practice (using a TXT record) is + not optimal, but it is necessary because there are a number of DNS + server and resolver implementations in common use that cannot handle + the new RR type. The two record type scheme provides a forward path + to the better solution of using a RR type reserved for this purpose. An SPF compliant domain name SHOULD have SPF records of both RR types. A compliant domain name MUST have a record of at least one type. If a domain has records of both types, they MUST have - identical content. + identical content. For example, instead of just publishing one + record as in Section 3.1 above, it is better to publish: - An SPF compliant check SHOULD lookup both types. Lookups can be - performed serially or in parallel. If both types of records are - obtained for a domain, the SPF type MUST be used. - - It is recognized that the current practice (using a TXT type record), - is not optimal, but a practical reality due to the state of deployed - software. The two record type scheme provides a forward path to the - better solution of using a RR type reserved for this purpose. + example.com. IN TXT "v=spf1 +mx a:colo.example.com/28 -all" + example.com. IN SPF "v=spf1 +mx a:colo.example.com/28 -all" - For either type, the character content of the record is encoded as - US-ASCII. + An SPF compliant check SHOULD try to look up and use a record of the + SPF type first, before falling back to the TXT type. However, the + client MAY also perform lookup of both types in parallel. If for a + domain both types are obtained but their contents do not match, the + SPF client SHOULD return a "PermError" result. - Example RRs in this document are shown with the SPF record type, - however they could also be published with a TXT type. + Example RRs in this document are shown with the TXT record type, + however they could be published with the SPF type or with both types. 3.1.2 Multiple Records @@ -416,32 +453,24 @@ authorization check to select more than one record. See Section 4.5 for the selection rules. -3.1.3 Additional Records +3.1.3 Multiple Strings - Some records contain directives that require additional SPF records. - It is suggested that those records be placed under an "_spf" - subdomain. See Appendix B for examples. - -3.1.4 Multiple Strings - - A Text DNS record (either TXT and SPF RR types) can be composed of + A text DNS record (either TXT and SPF RR types) can be composed of more than one string. If a published record contains multiple strings, then the record MUST be treated as if those strings are concatenated together without adding spaces. For example: - SPF "v=spf1 .... first" "second string..." + IN TXT "v=spf1 .... first" "second string..." MUST be treated as equivalent to - SPF "v=spf1 .... firstsecond string..." + IN TXT "v=spf1 .... firstsecond string..." SPF or TXT records containing multiple strings are useful in order to construct longer records which would otherwise exceed the maximum length of a string within a TXT or SPF RR record. - Note: Some nameserver implementations will silently split long - strings in TXT records into several shorter strings. -3.1.5 Record Size +3.1.4 Record Size The published SPF record for a given domain name SHOULD remain small enough that the results of a query for it will fit within 512 octets. @@ -449,12 +478,14 @@ TCP. Since the answer size is dependent on many things outside the scope of this document, it is only possible to give this guideline: If the combined length of the DNS name and the text of all the - records of a given type (TXT or SPF) is under 480 characters, then + records of a given type (TXT or SPF) is under 450 characters, then DNS answers should fit in UDP packets. Note that when computing the sizes for queries of the TXT format, one must take into account any - other TXT records published at the domain name. + other TXT records published at the domain name. Records that are too + long to fit in a single UDP packet MAY be silently ignored by SPF + clients. -3.1.6 Wildcard Records +3.1.5 Wildcard Records Use of wildcard records for publishing is not recommended. Care must be taken if wildcard records are used. If a domain publishes @@ -464,18 +495,18 @@ all, and for subdomains thereof. For example, the example given in [RFC1034], Section 4.3.3, could be extended with: - X.COM MX 10 A.X.COM - X.COM SPF "v=spf1 +a:A.X.COM -all" + X.COM. MX 10 A.X.COM + X.COM. TXT "v=spf1 a:A.X.COM -all" - *.X.COM MX 10 A.X.COM - *.X.COM SPF "v=spf1 +a:A.X.COM -all" + *.X.COM. MX 10 A.X.COM + *.X.COM. TXT "v=spf1 a:A.X.COM -all" - A.X.COM A 1.2.3.4 - A.X.COM MX 10 A.X.COM - A.X.COM SPF "v=spf1 +a:A.X.COM -all" + A.X.COM. A 1.2.3.4 + A.X.COM. MX 10 A.X.COM + A.X.COM. TXT "v=spf1 a:A.X.COM -all" - *.A.X.COM MX 10 A.X.COM - *.A.X.COM SPF "v=spf1 +a:A.X.COM -all" + *.A.X.COM. MX 10 A.X.COM + *.A.X.COM. TXT "v=spf1 a:A.X.COM -all" Notice that SPF records must be repeated twice for every name within the domain: Once for the name, and once with a wildcard to cover the @@ -487,88 +518,57 @@ 4. The check_host() Function The check_host() function fetches SPF records, parses them, and - interprets them to evaluate if a particular host is or is not - permitted to send mail with a given "Mail From" identity. Mail - receivers that perform this check MUST correctly evaluate the - check_host() function as described here. + interprets them to determine if a particular host is or is not + permitted to send mail with a given identity. Mail receivers that + perform this check MUST correctly evaluate the check_host() function + as described here. Implementations MAY use a different algorithm than the canonical - algorithm defined here, so long as the results are the same. + algorithm defined here, so long as the results are the same in all + cases. 4.1 Arguments - The function check_host() takes three arguments: + The function check_host() takes these arguments: - - the IP address of the host under test - - the domain to check - - the full sending mailbox address + - the IP address of the SMTP client that is emitting the + mail, either IPv4 or IPv6. + - the domain that provides the sought-after authorization + information; initially the domain portion of the "MAIL FROM" + or "HELO" identity. + - the "MAIL FROM" or "HELO" identity. The domain portion of will usually be the same as the argument when check_host() is initially evaluated. However, - it will generally not be true for recursive evaluations (see Section - 5.2 below). + this will generally not be true for recursive evaluations (see + Section 5.2 below). - Note: The IP address may be either IPv4 or IPv6. + Actual implementations of the check_host() function may need + additional arguments. 4.2 Results - The function check_host() can result in one of seven results - described here. Based on the result, the action to be taken is - determined by and the local policies of the receiver. (see Section - 2.4) - - Results from interpreting valid records: - - Neutral (?): published data is explicitly inconclusive - Pass (+): the is in the permitted set - Fail (-): the is in the not permitted set - SoftFail (~): the may be in the not permitted set, its use is - discouraged and the domain owner may move it to the not - permitted set in the future - - Results from error conditions: - - None - no published data - TempError - transient error during DNS lookup or other processing - PermError - unrecoverable error during processing, such as an - error in the record format - - If the result is "Fail", then an additional reason is returned. The - reason may be one of: - - Not Permitted - Malformed Domain - Domain Does Not Exist - - If the reason is "Not Permitted", then an explanation string is also - returned. The explanation string may be empty. + The function check_host() can return one of several results described + in Section 2.5. Based on the result, the action to be taken is + determined by the local policies of the receiver. 4.3 Initial Processing - If the is not an fully qualified domain name, check_host() - immediately returns the result "Fail" and a reason of "Malformed - Domain". + If the is malformed or is not a fully qualified domain name, + check_host() immediately returns the result "None". If the has no localpart, substitute the string "postmaster" for the localpart. 4.4 Record Lookup - The records for are fetched. If the records are in a cache, - and have not expired, then they may simply be used. Otherwise, the - records must be fetched from DNS as follows: - - In accordance with how the records are published, (see Section 3.1 - above), a DNS query needs to be made for the name, querying - for either RR type TXT, SPF or both. - - If the domain does not exist (RCODE 3), check_host() exits - immediately with the result "Fail" and a reason of "Domain Does Not - Exist" + In accordance with how the records are published, see Section 3.1 + above, a DNS query needs to be made for the name, querying + for either RR type TXT, SPF, or both. If the DNS lookup returns a server failure (RCODE 2), or other error (RCODE other than 0 or 3), or the query times out, check_host() exits - immediately with the result "TempError" + immediately with the result "TempError". 4.5 Selecting Records @@ -576,6 +576,7 @@ record = version terms *SP version = "v=spf1" + Starting with the set of records that were returned by the lookup, record selection proceeds in two steps: @@ -588,42 +589,42 @@ must be discarded. After the above steps, there should be exactly one record remaining - and evaluation can proceed. If there are no records remaining, - check_host() exits immediately with the result "None". If there are - two or more records remaining, then check_host() exits immediately - with the error "PermError". + and evaluation can proceed. If there are two or more records + remaining, then check_host() exits immediately with the result of + "PermError". + + If no matching records are returned, an SPF client MUST assume that + the domain makes no SPF declarations. SPF processing MUST abort and + return "None". 4.6 Record Evaluation After one SPF record has been selected, the check_host() function parses and interprets it to find a result for the current test. If - at any point a syntax error is encountered, check_host() returns - immediately with the result "PermError". + there are any syntax errors, check_host() returns immediately with + the result "PermError". Implementations MAY choose to parse the entire record first and return "PermError" if the record is not syntactically well formed. - Note: Unrecognized mechanisms are still syntactically well formed. - See Section 7.1. - + However, in all cases, any syntax errors anywhere in the record MUST + be detected. 4.6.1 Term Evaluation - There are two types of terms: mechanisms and modifiers. A given - mechanism type may appear multiple times in a record. A given - modifier may appear at most once per record. Unknown mechanisms - cause processing to abort with the result "PermError". Unknown - modifiers are ignored. - - A record contains an ordered list of mechanisms and modifiers: + There are two types of terms: mechanisms and modifiers. A record + contains an ordered list of these as specified in the following ABNF. terms = *( 1*SP ( directive / modifier ) ) directive = [ prefix ] mechanism prefix = "+" / "-" / "?" / "~" mechanism = ( all / include - / A / MX / PTR / IP4 / IP6 / exists - / unknown-mechanism ) - modifier = name "=" macro-string - name = alpha *( alpha / digit / "-" / "_" / "." ) + / A / MX / PTR / IP4 / IP6 / exists ) + + modifier = redirect / explanation / unknown-modifier + unknown-modifier = name "=" macro-string + + name = ALPHA *( ALPHA / DIGIT / "-" / "_" / "." ) + Most mechanisms allow a ":" or "/" character after the name. Modifiers always contain an equals ('=') character immediately after @@ -632,25 +633,21 @@ Terms that do not contain any of "=", ":" or "/" are mechanisms. - Mechanism and modifier names are case-insensitive. A mechanism - "INCLUDE" is equivalent to "include". + As per the definition of the ABNF notation in [RFC2234], mechanism + and modifier names are case-insensitive. 4.6.2 Mechanisms - Each mechanism is considered in turn from left to right. + Each mechanism is considered in turn from left to right. If there + are no more mechanisms, the result is specified in Section 4.7. When a mechanism is evaluated, one of three things can happen: it can match, it can not match, or it can throw an exception. If it matches, processing ends and the prefix value is returned as - the result of that record. (The default prefix value is "+".) - - If it does not match, processing continues with the next mechanism. - If no mechanisms remain, the default result is specified in Section - 4.7. - - If it throws an exception, mechanism processing ends and the - exception value is returned. + the result of that record. If it does not match, processing + continues with the next mechanism. If it throws an exception, + mechanism processing ends and the exception value is returned. The possible prefixes, and the results they return are: "+" Pass @@ -658,35 +655,27 @@ "~" SoftFail "?" Neutral - A missing prefix for a mechanism is the same as a prefix of "+". + The prefix is optional and defaults to "+". - When a mechanism matches, and the prefix is "-" so that a "Fail" - result is returned, the reason is Not Permitted, and the explanation - string is computed as described in Section 6.2. + When a mechanism matches and the prefix is "-", then a "Fail" result + is returned and the explanation string is computed as described in + Section 6.2. - Specific mechanisms are described in Section 5. + The specific mechanisms are described in Section 5. 4.6.3 Modifiers - Modifiers are key/value pairs that affect the evaluation of the - check_host() function. + Modifiers are not mechanisms: they do not return match or not-match. + Instead they provide additional information. While modifiers do not + directly affect the evaluation of the record, the "redirect" modifier + has an effect after all the mechanisms have been evaluated. - The modifiers defined in this document ("redirect" and "exp") MAY - appear anywhere in the record, but SHOULD appear at the end, after - all mechanisms. Ordering of these modifiers does not matter. These - modifiers MUST NOT appear in a record more than once each. If they - do, then check_host() exits with a result of "PermFail". - - Unrecognized modifiers MUST be ignored no matter where in a record, - or how often. This allows implementations of this document to handle - records with modifiers that are defined in later versions. - -4.7 Default result +4.7 Default Result If none of the mechanisms match and there is no "redirect" modifier, - then the check_host() exits with a result of "Neutral". If there is - a "redirect" modifier, check_host() proceeds as defined in Section - 6.1. + then the check_host() returns a result of "Neutral", just as if + "?all" were specified as the last directive. If there is a + "redirect" modifier, check_host() proceeds as defined in Section 6.1. Note that records SHOULD always either use a "redirect" modifier or an "all" mechanism to explicitly terminate processing. @@ -697,7 +686,7 @@ or v=spf1 +mx redirect=_spf.example.com -4.8 Domain Spec +4.8 Domain Specification Several of these mechanisms and modifiers have a section. The string is macro expanded (see Section 8). @@ -707,10 +696,9 @@ Note: The result of the macro expansion is not subject to any further escaping. Hence, this facility cannot produce all characters that - are legal in a DNS label, for example, the space or control - characters. However, this facility is powerful enough to express - legal host names, and common utility labels (such as "_spf") that are - used in DNS. + are legal in a DNS label (e.g. the control characters). However, + this facility is powerful enough to express legal host names, and + common utility labels (such as "_spf") that are used in DNS. For several mechanisms, the is optional. If it is not provided, the is used as the . @@ -735,8 +723,6 @@ ip6 exists - Other mechanisms may be defined in the future. - The following conventions apply to all mechanisms that perform a comparison between and an IP address at any point: @@ -748,14 +734,16 @@ When any mechanism fetches host addresses to compare with , when is an IPv4 address, A records are fetched, when is an IPv6 - address, AAAA records are fetched. + address, AAAA records are fetched. Even if the SMTP connection is + via IPv6, an IPv4-mapped IPv6 IP address (see [RFC3513] section + 2.5.5) MUST still be considered an IPv4 address. Several mechanisms rely on information fetched from DNS. For these - DNS queries, if the DNS server returns an error (RCODE other than 0 - or 3) or the query times out, the mechanism throws the exception - "TempError". If the server returns "domain does not exist" (RCODE - 3), then evaluation of the mechanism continues as if the server - returned no error (RCODE 0) and zero answer records. + DNS queries, except where noted, if the DNS server returns an error + (RCODE other than 0 or 3) or the query times out, the mechanism + throws the exception "TempError". If the server returns "domain does + not exist" (RCODE 3), then evaluation of the mechanism continues as + if the server returned no error (RCODE 0) and zero answer records. 5.1 "all" @@ -765,10 +753,10 @@ rightmost mechanism in a record to provide an explicit default. For example: - v=spf1 +mx +a -all + v=spf1 a mx -all Mechanisms after "all" will never be tested. Any "redirect" modifier - (Section 6.1) has no effect when there is an "all" directive. + (Section 6.1) has no effect when there is an "all" mechanism. 5.2 "include" @@ -780,25 +768,30 @@ The and arguments remain the same as in the current evaluation of check_host(). - "include" makes it possible for one domain to designate multiple - administratively independent domains. - - For example, a vanity domain "example.net" might send mail using the - servers of administratively independent domains example.com and - example.org. + In hind sight, the name "include" was poorly chosen. Only the + evaluated result of the referenced SPF record is used, rather than + acting as if the referenced SPF record was literally included in the + first. For example, evaluating a "-all" directive in the referenced + record does not terminate the overall processing and does not + necessarily result in a overall "Fail". (Better names for this + mechanism would have been "if-pass", "on-pass", "call", "eval, etc.) + + The "include" mechanism makes it possible for one domain to designate + multiple administratively independent domains. For example, a vanity + domain "example.net" might send mail using the servers of + administratively independent domains example.com and example.org. Example.net could say "v=spf1 include:example.com include:example.org -all". That would direct check_host() to, in effect, check the records of - example.com and example.org for a "pass" result. Only if the host + example.com and example.org for a "Pass" result. Only if the host were not permitted for either of those domains would the result be "Fail". Whether this mechanism matches or not, or throws an error depends on the result of the recursive evaluation of check_host(): - +---------------------------------+---------------------------------+ | A recursive check_host() result | Causes the "include" mechanism | | of: | to: | @@ -846,124 +839,135 @@ name. MX = "mx" [ ":" domain-spec ] [ dual-cidr-length ] - check_host() first performs an MX lookup on the . Then it performs an address lookup on each MX name returned. The is - compared to each returned IP address. If any address matches, the + compared to each returned IP address. To prevent DoS attacks, more + than 10 MX names MUST NOT be looked up during the evaluation of a + "mx" mechanism (see Section 10). If any address matches, the mechanism matches. - Note Regarding Implicit MXes: If the has no MX records, + Note regarding implicit MXes: If the has no MX records, check_host() MUST NOT pretend the target is its single MX, and MUST NOT default to an A lookup on the directly. This behavior breaks with the legacy "implicit MX" rule. See [RFC2821] Section 5. If such behavior is desired, the publisher should specify an "a" directive. + 5.5 "ptr" This mechanism tests if the DNS reverse mapping for exists and - validly points to a domain name within a particular domain. + correctly points to a domain name within a particular domain. PTR = "ptr" [ ":" domain-spec ] First the 's name is looked up using this procedure: perform a DNS reverse-mapping for , looking up the corresponding PTR record - in "in-addr.apra." if the address is an IPv4 one and "ip6.arpa." if - it is an IPv6 address. For each record returned, validate the host - name by looking up its IP address. If is among the returned IP - addresses, then that host name is validated. In pseudocode: - - sending-host_names := ptr_lookup(sending-host_IP); - for each name in (sending-host_names) { + in "in-addr.arpa." if the address is an IPv4 one and in "ip6.arpa." + if it is an IPv6 address. For each record returned, validate the + domain name by looking up its IP address. To prevent DoS attacks, + more than 10 PTR names MUST NOT be looked up during the evaluation of + a "ptr" mechanism (see Section 10). If is among the returned IP + addresses, then that domain name is validated. In pseudocode: + + sending-domain_names := ptr_lookup(sending-host_IP); + if more than 10 sending-domain_names are found, use at most 10. + for each name in (sending-domain_names) { IP_addresses := a_lookup(name); - if the sending-host_IP is one of the IP_addresses { - validated_sending-host_names += name; + if the sending-domain_IP is one of the IP_addresses { + validated-sending-domain_names += name; } } - Check all validated hostnames to see if they end in the - domain. If any do, this mechanism matches. If no validated hostname - can be found, or if none of the validated hostnames end in the - , this mechanism fails to match. - + Check all validated domain names to see if they end in the + domain. If any do, this mechanism matches. If no + validated domain name can be found, or if none of the validated + domain names end in the , this mechanism fails to match. + If a DNS error occurs while doing the PTR RR lookup, then this + mechanism fails to match. If a DNS error occurs while doing an A RR + lookup, then that domain name is skipped and the search continues. Pseudocode: - for each name in (validated_sending-host_names) { + for each name in (validated-sending-domain_names) { if name ends in , return match. if name is , return match. } return no-match. - This mechanism matches if the is an ancestor of a - validated hostname, or if the and a validated hostname - are the same. For example: "mail.example.com" is within the domain - "example.com", but "mail.bad-example.com" is not. If a validated - hostname is the , a match results. - - Note: This mechanism is not recommended. If a domain decides to use - it, it should make sure is has the proper PTR records in place for - its hosts. + This mechanism matches if the is either an ancestor of + a validated domain name, or if the and a validated + domain name are the same. For example: "mail.example.com" is within + the domain "example.com", but "mail.bad-example.com" is not. + + Note: Use of this mechanism is discouraged because it is slow, is not + as reliable as other mechanisms in cases of DNS errors and it places + a large burden on the arpa name servers. If used, proper PTR records + must be in place for the domain's hosts and the "ptr" mechanism + should be one of the last mechanisms checked. 5.6 "ip4" and "ip6" These mechanisms test if is contained within a given IP network. + IP4 = "ip4" ":" ip4-network [ ip4-cidr-length ] IP6 = "ip6" ":" ip6-network [ ip6-cidr-length ] + ip4-cidr-length = "/" 1*DIGIT ip6-cidr-length = "/" 1*DIGIT + dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ] - ip4-network = as per conventional dotted quad notation, - e.g. 192.0.2.0 - ip6-network = as per [RFC 3513], section 2.2, - e.g. 2001:DB8::CD30 + ip4-network = ; as per conventional dotted quad notation, + ; e.g. 192.0.2.0 + ip6-network = ; as per [RFC 3513], section 2.2, + ; e.g. 2001:DB8::CD30 The is compared to the given network. If CIDR-length high-order bits match, the mechanism matches. If ip4-cidr-length is omitted it is taken to be "/32". If - ip6-cidr-length is omitted it is taken to be "/128". + ip6-cidr-length is omitted it is taken to be "/128". It is not + permitted to omit parts of the IP address instead of using CIDR + notations. That is, use 10.23.45.0/24 instead of 10.23.45. 5.7 "exists" - This mechanism is used to construct an arbitrary host name that is + This mechanism is used to construct an arbitrary domain name that is used for a DNS A record query. It allows for complicated schemes involving arbitrary parts of the mail envelope to determine what is - legal. + permitted. exists = "exists" ":" domain-spec The domain-spec is expanded as per Section 8. The resulting domain - name is used for a DNS A lookup. If any A record is returned, this - mechanism matches. The lookup type is 'A' even when the connection - type is IPv6. + name is used for a DNS A RR lookup. If any A record is returned, + this mechanism matches. The lookup type is 'A' even when the + connection type is IPv6. Domains can use this mechanism to specify arbitrarily complex queries. For example, suppose example.com publishes the record: v=spf1 exists:%{ir}.%{l1r+-}._spf.%{d} -all - The target-name might expand to + The might expand to "1.2.0.192.someuser._spf.example.com". This makes fine-grained decisions possible at the level of the user and client IP address. This mechanism enables queries that mimic the style of tests that - existing RBL lists use. + existing DNSBL lists use. 6. Modifier Definitions - Modifiers are not mechanisms: they do not return match or no-match. - Instead they provide additional information or alter check_host() - processing. + Modifiers are name/value pairs that provide additional information. + Modifiers always have an "=" separating the name and the value. + + The modifiers defined in this document ("redirect" and "exp") MAY + appear anywhere in the record, but SHOULD appear at the end, after + all mechanisms. Ordering of these two modifiers does not matter. + These modifiers MUST NOT appear in a record more than once each. If + they do, then check_host() exits with a result of "PermError". - While unrecognized mechanisms cause an immediate "PermError" abort, - unrecognized modifiers MUST be simply ignored. Modifiers therefore - provide a way to extend the record format in the future with backward - compatibility. - - Only two modifiers are currently defined: "redirect" and "exp". - Implementations of check_host() MUST support them both. - - There is one deprecated modifier: "default", which cannot be defined - by any future version of this document. Implementations MUST ignore - it. + Unrecognized modifiers MUST be ignored no matter where in a record, + nor how often. This allows implementations of this document to + gracefully handle records with modifiers that are defined in other + specifications. 6.1 redirect: Redirected Query @@ -986,20 +990,20 @@ This facility is intended for use by organizations that wish to apply the same record to multiple domains. For example: - la.example.com. SPF "v=spf1 redirect=_spf.example.com" - ny.example.com. SPF "v=spf1 redirect=_spf.example.com" - sf.example.com. SPF "v=spf1 redirect=_spf.example.com" - _spf.example.com. SPF "v=spf1 mx:example.com -all" + la.example.com. TXT "v=spf1 redirect=_spf.example.com" + ny.example.com. TXT "v=spf1 redirect=_spf.example.com" + sf.example.com. TXT "v=spf1 redirect=_spf.example.com" + _spf.example.com. TXT "v=spf1 mx:example.com -all" In this example, mail from any of the three domains is described by the same record. This can be an administrative advantage. - Note: In general, a domain A cannot reliably use a redirect to - another domain B not under the same administrative control. Since + Note: In general, the domain "A" cannot reliably use a redirect to + another domain "B" not under the same administrative control. Since the stays the same, there is no guarantee that the record at - domain B will correctly work for addresses in domain A, especially if - domain B uses mechanisms involving localparts. An "include" - directive may be more appropriate. + domain "B" will correctly work for mailboxes in domain "A", + especially if domain "B" uses mechanisms involving localparts. An + "include" directive may be more appropriate. For clarity it is RECOMMENDED that any "redirect" modifier appear as the very last term in a record. @@ -1011,29 +1015,31 @@ If check_host() results in a "Fail" due to a mechanism match (such as "-all"), and the "exp" modifier is present, then the explanation string returned is computed as described below. If no "exp" modifier - is present, then an empty explanation string is returned. + is present, then either a default explanation string or an empty + explanation string may be returned. The is macro expanded (see Section 8) and becomes the . The DNS TXT record for the is fetched. If is empty, or there are any processing errors (any RCODE other than 0), or if no records are returned, or if more than - one record is returned, then an empty explanation string is returned. + one record is returned, then proceed as if no exp modifier was given. The fetched TXT record's strings are concatenated with no spaces, and - then treated as a new macro-string which is macro-expanded. This + then treated as an which is macro-expanded. This final result is the explanation string. - Software evaluating check_host() can use this string when the result - is "Fail" with a reason of "Not Permitted", to communicate + Software evaluating check_host() can use this string to communicate information from the publishing domain in the form of a short message or URL. Software should make it clear that the explanation string comes from a third party. For example, it can prepend the macro - string "%{d} explains: " to the explanation. + string "%{o} explains: " to the explanation. Implementations MAY limit the length of the resulting explanation string to allow for other protocol constraints and/or reasonable - processing limits. + processing limits. The SPF client SHOULD make it clear when an + explanation string is coming from a third party, such as shown in + Section 2.5.4. Suppose example.com has this record @@ -1056,97 +1062,106 @@ arguments to check_host() so that a web page can be generated with detailed, custom instructions - Note: During recursion into an "include" mechanism, explanations do - not propagate out. But during execution of a "redirect" modifier, - the explanation string from the target of the redirect is used. -7. Miscellaneous - -7.1 Unrecognized Mechanisms and Modifiers - - New mechanisms can only be introduced by new versions of this - document. - - Unrecognized mechanisms cause processing to abort: If, during - evaluation of a record, check_host() encounters a mechanism which it - does not understand, it terminates processing and returns - "PermError", without evaluating any further mechanisms. Mechanisms - listed before the unknown mechanism MUST, however, be evaluated. - - For example, consider the record: - - v=spf1 a mx ptr foo:_foo.%{d} -all - - If during the evaluation of check_host(), any of the "a", "mx", or - "ptr" directives match, then check_host() would return a "Pass" - result. If none of those directives resulted in a match, then an - implementation that did not recognize the "foo" mechanism would - return "PermError". An implementation that did recognize the "foo" - mechanism would be able to perform an extended evaluation. - - Note: "foo" is an example of an unknown extension mechanism that - could be defined in the future. It is NOT defined by this proposal. - - Unrecognized modifiers are ignored: if an implementation encounters - modifiers which it does not recognize, it MUST ignore them. - -7.2 Processing Limits - - During processing, an evaluation of check_host() may require - additional evaluations of check_host() due to the "include" mechanism - and/or the "redirect" modifier. - - Implementations must be prepared to handle records that are set up - incorrectly or maliciously. Implementations MUST perform loop - detection, limit additional evaluations, or both. If an - implementation chooses to limit additional evaluations, then at least - a total of 10 evaluations of check_host() for a single query MUST be - supported. (This number should be enough for even the most - complicated configurations.) - - If a loop is detected, or the evaluation limit of an implementation - is reached, check_host() MUST abort processing and return the result - "PermError". - MTAs or other processors MAY also impose a limit on the maximum - amount of elapsed time to evaluate check_host(). Such a limit SHOULD - allow at least 20 seconds. If such a limit is exceeded, the result - of authentication SHOULD be "TempError". - - Domains publishing records SHOULD try to keep the number of "include" - directives and chained "redirect" modifiers to a minimum. Domains - SHOULD also try to minimize the amount of other DNS information - needed to evaluate a record. This can be done by choosing directives - that require less DNS information. - - For example, consider a domain set up as: + Note: During recursion into an "include" mechanism, an exp= modifier + from the target domain MUST NOT be used. In contrast, when executing + a "redirect" modifier, an exp= modifier from the original domain MUST + NOT be used. +7. The Received-SPF header + + It is RECOMMENDED that SMTP receivers record the result of SPF + processing in the message headers. If an SMTP receiver chooses to do + so, it SHOULD use the "Received-SPF" header defined here. This + information is intended for the recipient. (Information intended for + the sender is described in Section 6.2, Explanation.) + + The Received-SPF header is a trace field (see [RFC2822] section + 3.6.7) and SHOULD be prepended to existing headers, above the + Received: header that is generated by the SMTP receiver. It MUST + appear above any other Received-SPF headers in the message. The + header has the format: + + + header = "Received-SPF:" [CFWS] result [FWS [comment]] + [ key-value-list ] + + result = "Pass" / "Fail" / "SoftFail" / "Neutral" / + "None" / "TempError" / "PermError" + + key-value-list = key-value-pair *( ";" [CFWS] key-value-pair ) + [";"] + + key-value-pair = key [CFWS] "=" ( dot-atom / quoted-string ) + + key = "client-ip" / "envelope-from" / "helo" / + "problem" / "receiver" / + mechanism / "x-" name / name + + dot-atom = ; unquoted word as per [RFC2822] + quoted-string = ; quoted string as per [RFC2822] + comment = ; comment string as per [RFC2822] + CFWS = ; comment or folding white space as per [RFC2822] + FWS = ; folding white space as per [RFC2822] + + The header SHOULD include a "(...)" style after the result, + conveying supporting information for the result, such as , + and . + + The following key-value-pairs are designed for later machine parsing. + SPF clients SHOULD give at least the "client-ip" and either the + "envelope-from" or "helo" information so that the SPF results can be + verified. + client-ip the IP address of the SMTP client + envelope-from the envelope sender mailbox + helo the host name given in the HELO or EHLO command + mechanism the mechanism that matched (if no mechanisms matched, + substitute the word "default".) + problem if an error was returned, details about the error + receiver the host name of the SPF client + + Other keys may be defined by SPF clients. Until a new key name + becomes widely accepted, new key names should start with "x-". + + SPF clients MUST make sure that the Received-SPF header does not + contain invalid characters, is excessively long, or contain malicious + data that has been provided by the sender. + + Examples of various header styles that could be generated: + + Received-SPF: Pass (mybox.example.org: domain of + myname@example.com designates 192.0.2.1 as permitted sender) + receiver=mybox.example.org; client-ip=192.0.2.1; + envelope-from=; helo=foo.example.com; + + + Received-SPF: Fail (mybox.example.org: domain of + myname@example.com does not designate + 192.0.2.1 as permitted sender) + client-ip=192.0.2.1; + envelope-from=; +8. Macros - example.com. IN MX 10 mx.example.com. - mx.example.com. IN A 192.0.2.1 - a.example.com. IN SPF "v=spf1 +mx:example.com -all" - b.example.com. IN SPF "v=spf1 +a:mx.example.com -all" - c.example.com. IN SPF "v=spf1 +ip4:192.0.2.1 -all" +8.1 Macro definitions - Evaluating check_host() for the domain "a.example.com" requires the - MX records for "example.com", and then the A records for the listed - hosts. Evaluating for "b.example.com" only requires the A records. - Evaluating for "c.example.com" requires none. + Many mechanisms and modifiers perform macro expansion on part of the + term. - However, there may be administrative considerations: Using "a" over - "ip4" allows hosts to be renumbered easily. Using "mx" over "a" - allows the set of mail hosts to be changed easily. -8. Macros + domain-spec = macro-string domain-end + domain-end = ( "." toplabel ) / macro-expand -8.1 Macro definitions + toplabel = ALPHA / ALPHA *[ alphanum / "-" ] alphanum + ; LDH rule (See [RFC3696]) + alphanum = ALPHA / DIGIT - Many mechanisms and modifiers perform macro interpolation on part of - the term. + explain-string = *( macro-string / SP ) - domain-spec = *( macro-expand / macro-literal ) - macro-string = *( macro-expand / macro-literal / "/" ) - macro-expand = ( "%{" ALPHA transformer *delimiter "}" ) + macro-string = *( macro-expand / macro-literal ) + macro-expand = ( "%{" macro-letter transformers *delimiter "}" ) / "%%" / "%_" / "%-" - macro-literal = %x21-24 / %x26-2E / %x30-7E - ; visible characters except "%" and "/" - transformer = *DIGIT [ "r" ] + macro-literal = %x21-24 / %x26-7E + ; visible characters except "%" + macro-letter = "s" / "l" / "o" / "d" / "i" / "p" / "h" / + "c" / "r" / "t" + transformers = *DIGIT [ "r" ] delimiter = "." / "-" / "+" / "," / "/" / "_" / "=" A literal "%" is expressed by "%%". @@ -1161,34 +1176,24 @@ o = domain of d = i = - p = the validated host name of + p = the validated domain name of v = the string "in-addr" if is ipv4, or "ip6" if is ipv6 + h = HELO/EHLO domain The following macro letters are only allowed in "exp" text: - - c = SMTP client IP (easily readable format) r = domain name of host performing the check t = current timestamp - The uppercase versions of all these macros are URL-encoded. - - A '%' character not followed by a '{', '%', '-', or '_' character - MUST be interpreted as a literal. Domains SHOULD NOT rely on this - feature; they MUST escape % literals. For example, an explanation - TXT record - Your spam volume has increased by 581% - is incorrect. Instead, say - Your spam volume has increased by 581%% - - All other legal visible characters are simply expanded to themselves. - Note that the two different macro contexts, domain-spec, and - macro-string allow slightly different sets of legal visible - characters. In particular, macro-string allows the slash character. - - Legal optional transformers are: + A '%' character not followed by a '{', '%', '-', or '_' character is + a syntax error. So, + -exists:%(ir).sbl.spamhaus.org + is incorrect and will cause check_host() to return a "PermError". + Instead, say + -exists:%{ir}.sbl.spamhaus.org + Optional transformers are: *DIGIT : zero or more digits 'r' : reverse value, splitting on dots by default @@ -1202,9 +1207,6 @@ treatment is given to leading, trailing or consecutive delimiters, and so the list of parts may contain empty strings. Macros may specify delimiter characters which are used instead of ".". - Delimiters MUST be one or more of the characters: - - "." / "-" / "+" / "," / "/" / "_" / "=" The 'r' transformer indicates a reversal operation: if the client IP address were 192.0.2.1, the macro %{i} would expand to "192.0.2.1" @@ -1235,47 +1237,50 @@ hexadecimal colon-format addresses specified in [RFC3513] section 2.2. It is intended for humans to read. - The "p" macro expands to the validated host name of . The - procedure for finding the validated host names is defined in Section - 5.5. If that procedure produces more than one validated host name, - any name from the list may be used. If that procedure produces no - validate host name the string "unknown" is used. + The "p" macro expands to the validated domain name of . The + procedure for finding the validated domain name is defined in Section + 5.5. If the is present in the list of validated domains, it + SHOULD be used. Otherwise, if a subdomain of the is + present, it SHOULD be used. Otherwise, any name from the list may be + used. If there are no validated domain names or if a DNS error + occurs, the string "unknown" is used. The "r" macro expands to the name of the receiving MTA. This SHOULD be a fully qualified domain name, but if one does not exist (as when - the checking is done by a script) or if policy restrictions dictate + the checking is done by a MUA) or if policy restrictions dictate otherwise, the word "unknown" SHOULD be substituted. The domain name may be different than the name found in the MX record that the client MTA used to locate the receiving MTA. - The "t" macro expands to the decimal representation of the number of - seconds since the Epoch (Midnight, January 1st, 1970, UTC). This is - the same value as returned by the time() function in most standards - compliant libraries. - - Any unrecognized macro letters are expanded as the string "unknown". - There is one deprecated macro letter: "h". It is expanded as the - string "deprecated". + The "t" macro expands to the decimal representation of the + approximate number of seconds since the Epoch (Midnight, January 1st, + 1970, UTC). This is the same value as returned by the POSIX time() + function in most standards compliant libraries. When the result of macro expansion is used in a domain name query, if - the expanded domain name exceeds 255 characters (the maximum length + the expanded domain name exceeds 253 characters (the maximum length of a domain name), the left side is truncated to fit, by removing - successive subdomains until the total length falls below 255 + successive domain labels until the total length does not exceed 253 characters. Uppercased macros expand exactly as their lower case equivalents, and - are then URL escaped. URL escaping is described in [RFC2396]. + are then URL escaped. URL escaping must be performed for characters + not in the "uric" set, which is defined in [RFC3986]. + + Note: Care must be taken so that macro expansion for legitimate + e-mail does not exceed the 63 character limit on DNS labels. The + localpart of e-mail addresses, in particular, can have more than 63 + characters between dots. - Note: Domains should avoid using the "s", "l" or "o" macros in + Note: Domains should avoid using the "s", "l", "o", or "h" macros in conjunction with any mechanism directive. While these macros are powerful and allow per-user records to be published, they severely limit the ability of implementations to cache results of check_host() and they reduce the effectiveness of DNS caches. - Implementations should be aware that if no directive processed during - the evaluation of check_host() contains an "s", "l", or "o" macro, - then the results of the evaluation can be cached on the basis of - and alone for as long as the shortest TTL of all the + the evaluation of check_host() contains an "s", "l", "o" or "h" + macro, then the results of the evaluation can be cached on the basis + of and alone for as long as the shortest TTL of all the DNS records involved. 8.2 Expansion Examples @@ -1318,8 +1323,8 @@ example.com.trusted-domains.example.net IPv6: - %{ir}.%{v}._spf.%{d2} 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8. - 5.d.a.0.8.0.0.0.2.5.0.f.5.ip6._spf.example.com + %{ir}.%{v}._spf.%{d2} 1.0.0.0.0.0.0.0. + 0.0.0.0.0.0.0.0.0.0.8.5.d.a.0.8.0.0.0.2.5.0.f.5.ip6._spf.example.com 9. Implications This section outlines the major implications that adoption of this @@ -1336,77 +1341,112 @@ Domains that wish to be compliant with this specification will need to determine the list of hosts that they allow to use their domain - name in the "Mail From" identity. It is recognized that forming such - a list is not just a simple technical exercise, but involves policy - decisions with both technical and administrative considerations. + name in the "HELO" and "MAIL FROM" identities. It is recognized that + forming such a list is not just a simple technical exercise, but + involves policy decisions with both technical and administrative + considerations. + + It can be helpful to publish records that include a "tracking + exists:" mechanism. By looking at the name server logs, a rough list + may then be generated. For example: + v=spf1 exists:_h.%{h}._l.%{l}._o.%{o}._i.%{i}._spf.%{d} ?all 9.2 Mailing Lists Mailing lists must be aware of how they re-inject mail that is sent - to the list. If the list re-injects mail with the same reverse-path - that the mail had when it was received, then that mail may fail the - authorization tests defined in this document. In particular, they - will fail when the domain of the reverse-path publishes SPF records - for the "Mail From" identity, those records do not authorize the - mailing list host, and a receiver of the mailing list performs the - authorization test. - - Almost all mailing list software in use for public mailing lists uses - a reverse-path with the mailing list's own domain so that the - software can receive mail bounces and assist in the administration of - the list. Lists that use such software, configured to operate this - way will require only one modest change in light of this document: - The mailing list host needs to be authorized by the mailing list - domain's own SPF record, if the domain publishes one. - - Mailing lists based on simple alias expansion, or other software that - doesn't manage bounces directly, may or may not encounter problems - depending on how access to the list is restricted. Such lists that - are entirely internal to a domain (only people in the domain can send - to or receive from the list) are not affected. + to the list. Mailing lists MUST comply with the requirement in + [RFC2821] Section 3.10 and [RFC1123] Section 5.3.6 that say that the + reverse-path MUST be changed to be the mailbox of a person or other + entity who administers the list. While the reasons for changing the + reverse-path are many and long standing, SPF adds enforcement to this + requirement. + + In practice, almost all mailing list software in use already complies + with this requirement. Mailing lists that do not comply may or may + not encounter problems depending on how access to the list is + restricted. Such lists that are entirely internal to a domain (only + people in the domain can send to or receive from the list) are not + affected. -9.3 Forwarding Services +9.3 Forwarding Services and Aliases Forwarding services take mail that is received at a mailbox and - direct it to some external mailbox. At the time of this writing, the - near-universal practice of such services is to use the original - reverse-path of a message when re-injecting it for delivery to the - external mailbox. This means the external mailbox's MTA sees all - such mail in a connection from a host of the forwarding service, and - so the "Mail From" identity will not in general pass authorization. - - There are several possible ways that this authorization failure can - be ameliorated. If the owner of the external mailbox wishes to trust - the forwarding service, they can direct the external mailbox's MTA to - skip such tests when the client host belongs to the forwarding - service. Tests against some other identity may also be used to - override the test against the "Mail From" identity. - - For larger domains, it may not be possible to have a complete or - accurate list of forwarding services used by the owners of the - domain's mailboxes. In such cases, white lists of generally - recognized forwarding services could be employed. - - Forwarding services could also skirt the issue by using reverse-paths - that contain their own domain. This means that mail bounced from the - external mailbox will have to be re-bounced by the forwarding - service. Various schemes to do this exist though they vary widely in - complexity and resource requirements on the part of the forwarding - service. + directs it to some external mailbox. At the time of this writing, + the near-universal practice of such services is to use the original + "MAIL FROM" of a message when re-injecting it for delivery to the + external mailbox. [RFC1123] and [RFC2821] describe this action as an + "alias" rather than a "mail list". This means the external mailbox's + MTA sees all such mail in a connection from a host of the forwarding + service, and so the "MAIL FROM" identity will not, in general, pass + authorization. + + There are three places that techniques can be used to ameliorate this + problem. + 1. The beginning, when e-mail is first sent. + * "Neutral" results could be given for IP addresses that may be + forwarders, instead of "Fail" results. For example: + "v=spf1 mx -exists:%{ir}.sbl.spamhaus.org ?all" + This would cause a lookup on an anti-spam DNS blocklist + (DNSBL) and cause a result of "Fail" only for e-mail coming + from listed sources. All other e-mail, including e-mail sent + through forwarders, would receive a "Neutral" result. By + checking the DNSBL after the known good sources, problems with + incorrect listing on the DNSBL are greatly reduced. + * The "MAIL FROM" identity could have additional information in + the localpart that cryptographically identifies the mail as + coming from an authorized source. In this case, such an SPF + record could be used: + "v=spf1 mx exists:%{l}._spf_verify.%{d} -all" + Then, a specialized DNS server can be set up to serve the + _spf_verify subdomain which validates the localpart. While + this requires an extra DNS lookup, this only happens when the + e-mail would otherwise be rejected as not coming from a known, + good source. + * Similarly, a specialized DNS server could be set up that will + rate-limit the e-mail coming from unexpected IP addresses. + "v=spf1 mx exists:%{ir}._spf_rate.%{d} -all" + * SPF allows the creation of per-user policies for special + cases. For example, the following SPF record and appropriate + wildcard DNS records can be used: + "v=spf1 mx redirect=%{l1r+}._at_.%{o}._spf.%{d}" + 2. The middle, when e-mail is forwarded. + * Forwarding services can solve the problem by rewriting the + "MAIL FROM" to be in their own domain. This means that mail + bounced from the external mailbox will have to be re-bounced + by the forwarding service. Various schemes to do this exist + though they vary widely in complexity and resource + requirements on the part of the forwarding service. + * Several popular MTAs can be forced from "alias" semantics to + "mailing list" semantics by configuring an additional alias + with "owner-" prepended to the original alias name (e.g. an + alias of "friends: george@example.com, fred@example.org" would + need another alias of the form "owner-friends: localowner"). + 3. The end, when e-mail is received. + * If the owner of the external mailbox wishes to trust the + forwarding service, they can direct the external mailbox's MTA + to skip SPF tests when the client host belongs to the + forwarding service. + * Tests against other identities, such as the "HELO" identity, + may be used to override a failed test against the "MAIL FROM" + identity. + * For larger domains, it may not be possible to have a complete + or accurate list of forwarding services used by the owners of + the domain's mailboxes. In such cases, whitelists of + generally recognized forwarding services could be employed. 9.4 Mail Services - Entities that offer mail services to other domains such as sending of - bulk mail will may have to alter their mail in light of the - authorization check in this document. If the reverse-path used for - such e-mail uses the domain of the mail service provider, then the - provider needs only to ensure that their sending host is authorized - by their own SPF record, if any. + Service providers that offer mail services to third party domains, + such as sending of bulk mail, may have to adjust their setup in light + of the authorization check described in this document. If the "MAIL + FROM" used for such e-mail uses the domain of the service provider, + then the provider needs only to ensure that their sending host is + authorized by their own SPF record, if any. - If the reverse-path does not use the mail service provider's domain, + If the "MAIL FROM" does not use the mail service provider's domain, then extra care must be taken. The SPF record format has several - options for authorizing the sending MTAs of another domain (the - service provider's) + options for the third party domain to authorize the service + provider's MTAs to send mail on its behalf. 9.5 MTA Relays @@ -1415,8 +1455,8 @@ Within an organization, MTA relays can be effectively deployed. However, for purposes of this document, such relays are effectively - invisible. The "Mail From" identity authorization check is a check - between border MTAs. + transparent. The "MAIL FROM" identity authorization check is a check + between border MTAs of different domains. For mail senders, this means that published SPF records must authorize any MTAs that actually send across the Internet. Usually, @@ -1424,24 +1464,28 @@ to these MTAs for delivery. Mail receivers will generally want to perform the authorization check - at the border MTAs. This allows mail that fails to be rejected - during the SMTP session rather than bounced. Internal MTAs then do - not perform the authorization test. To perform the authorization - test other than at the border, the host that first transferred the - message to the organization must be determined, which can be - difficult to extract from headers. Testing other than at the border - is not recommended. + at the border MTAs, specifically including all secondary MXes. This + allows mail that fails to be rejected during the SMTP session rather + than bounced. Internal MTAs then do not perform the authorization + test. To perform the authorization test other than at the border, + the host that first transferred the message to the organization must + be determined, which can be difficult to extract from headers. + Testing other than at the border is not recommended. 10. Security Considerations - The "Mail From" identity authorization must not be construed to - provide more assurance than it does. It is entirely possible for a - malicious sender to inject a message with a reverse-path that uses - their own domain, to have that domain's SPF record authorize the - sending host, and yet the message content can easily claim other - identities in the headers. Unless the user, or the MUA takes care to - note that the authorized "Mail From" identity does not match the - other, more commonly presented identities (such as the From: header), - the user may be lulled into a false sense of security. +10.1 SPF-Authorized E-Mail May Still Be UBE + + The "MAIL FROM" and "HELO" identity authorizations must not be + construed to provide more assurance than it does. It is entirely + possible for a malicious sender to inject a message using their own + domain in the identities used by SPF, to have that domain's SPF + record authorize the sending host, and yet the message content can + easily claim other identities in the headers. Unless the user, or + the MUA takes care to note that the authorized identity does not + match the other, more commonly presented identities (such as the + From: header), the user may be lulled into a false sense of security. + +10.2 Spoofed DNS and IP Data There are two aspects of this protocol that malicious parties could exploit to undermine the validity of the check_host() function: @@ -1453,66 +1497,166 @@ where the actual domain's record would evaluate to "Fail". See [RFC3833] for a description of the DNS weaknesses. The client IP address, , is assumed to be correct. A - malicious attacker could spoof TCP sequences to make mail appear - to come from a permitted host for a domain that the attacker is - impersonating. + malicious attacker could spoof TCP sequence numbers to make mail + appear to come from a permitted host for a domain that the + attacker is impersonating. - As with most aspects of mail, there are a number of ways that +10.3 Processing Limits + + As with most aspects of e-mail, there are a number of ways that malicious parties could use the protocol as an avenue of a - distributed denial of service attack: + Denial-of-Service (DoS) attack. The processing limits outlined here + are designed to prevent attacks such as: - While implementations of check_host() need to limit the number of - "include" and "redirect" terms and/or check for loops, malicious - domains could publish records that exercise or exceed these limits - in an attempt to waste computation effort at their targets when - they send them mail. - Malicious parties could send large volume mail purporting to come - from the intended target to a wide variety of legitimate mail + o A malicious party could create an SPF record with many references + to a victim's domain, send many e-mails to different SPF clients + and those SPF clients would then create a DoS attack. In effect, + the SPF clients are being used to amplify the attacker's bandwidth + by using fewer bytes in the SMTP session than is generated by the + DNS queries. Using SPF clients also allows the attacker to hide + the true source of the attack. + o While implementations of check_host() are supposed to limit the + number of DNS lookups, malicious domains could publish records + that exceed these limits in an attempt to waste computation effort + at their targets when they send them mail. Malicious domains + could also design SPF records that cause particular + implementations to use excessive memory or CPU usage, or to + trigger bugs. + o Malicious parties could send a large volume mail purporting to + come from the intended target to a wide variety of legitimate mail hosts. These legitimate machines would then present a DNS load on the target as they fetched the relevant records. - While these distributed denial of service attacks are possible, - they seem more convoluted to mount, and have less of an impact, - than other simpler attacks. - - When the authorization check fails with the code "Not Permitted", an - explanation string may be included in the reject response. Both the - sender and the rejecting receiver need to be aware that the - explanation was determined by the publisher of the SPF record - checked, and is in general not the receiver. The explanation may - contain URLs that may be malicious, and/or offensive or misleading - text. This is probably less of a concern than it may seem since such - messages are returned to the sender, and their source is the SPF - record published by the domain in the "Mail From" identity claimed by - that very sender. To put it another way, the only people who see - malicious explanation strings are people whose messages claim to be - from domains that publish such strings in their SPF records. + + Of these, the case of a third party referenced in the SPF record is + the easiest for a DoS attack to effectively exploit. As a result, + limits that may seem reasonable for an individual mail server can + still allow an unreasonable amount of bandwidth amplification. + Therefore the processing limits need to be quite low. + + SPF implementations MUST limit the number of mechanisms and modifiers + that do DNS lookups to at most 10 per SPF check. If this number is + exceeded during a check, a PermError MUST be returned. The + "include", "a", "mx", "ptr", and "exists" mechanisms as well as the + "redirect" and "exp" modifiers do count against this limit. The + "all", "ip4" and "ip6" mechanisms do not require DNS lookups and + therefore do not count against this limit. + + When evaluating the "mx" and "ptr" mechanisms, or the %{p} macro, + there MUST be a limit of no more than 10 MX or PTR RRs looked up and + checked. + + SPF implementation SHOULD limit the total amount of data obtained + from the DNS queries. For example, when DNS over TCP or EDNS0 are + available, there may need to be an explicit limit to how much data + will be accepted to prevent excessive bandwidth usage or memory + usage, and DoS attacks. + + MTAs or other processors MAY also impose a limit on the maximum + amount of elapsed time to evaluate check_host(). Such a limit SHOULD + allow at least 20 seconds. If such a limit is exceeded, the result + of authentication SHOULD be "TempError". + + Domains publishing records SHOULD try to keep the number of "include" + mechanisms and chained "redirect" modifiers to a minimum. Domains + SHOULD also try to minimize the amount of other DNS information + needed to evaluate a record. This can be done by choosing directives + that require less DNS information and placing lower cost mechanisms + earlier in the SPF record. + + For example, consider a domain set up as: + example.com. IN MX 10 mx.example.com. + mx.example.com. IN A 192.0.2.1 + a.example.com. IN TXT "v=spf1 mx:example.com -all" + b.example.com. IN TXT "v=spf1 a:mx.example.com -all" + c.example.com. IN TXT "v=spf1 ip4:192.0.2.1 -all" + + Evaluating check_host() for the domain "a.example.com" requires the + MX records for "example.com", and then the A records for the listed + hosts. Evaluating for "b.example.com" only requires the A records. + Evaluating for "c.example.com" requires none. + + However, there may be administrative considerations: Using "a" over + "ip4" allows hosts to be renumbered easily. Using "mx" over "a" + allows the set of mail hosts to be changed easily. + +10.4 Untrusted Information Sources + + When the authorization check fails, an explanation string may be + included in the reject response. Both the sender and the rejecting + receiver need to be aware that the explanation was determined by the + publisher of the SPF record checked and, in general, not the + receiver. The explanation may contain URLs that may be malicious, + offensive and/or have misleading text. This is probably less of a + concern than it may initially seem since such messages are returned + to the sender, and the source is the SPF record published by the + domain in the identity claimed by that very sender. To put it + another way, the only people who see malicious explanation strings + are people whose messages claim to be from domains that publish such + strings in their SPF records. + + SPF uses information supplied by third parties, such as the "HELO" + domain name, the "MAIL FROM" and SPF records. This information is + then sent to the receiver in the Received-SPF: mail headers and + possibly returned to the client MTA in the form of an SMTP rejection + message. This information must be checked for invalid characters and + excessively long lines. + +10.5 Privacy Exposure + + Checking SPF records causes DNS queries to be sent to the domain + owner. These DNS queries, especially if they are caused by the + "exists" mechanism, can contain information about who is sending + e-mail and likely to which MTA the e-mail is being sent to. This can + introduce some privacy issues, which may be more or less of an issue + depending on local laws and the relationships between the domain + owner and the persons sending the e-mail. 11. IANA Considerations +11.1 The SPF DNS Record Type + The IANA needs to assign a new Resource Record Type and Qtype from the DNS Parameters Registry for the SPF RR type. + +11.2 The Received-SPF mail header + + Per [RFC3864], the "Received-SPF:" header field is added to the IANA + Permanent Message Header Field Registry. The following is the + registration template: + + o Header field name: Received-SPF + o Applicable protocol: mail + o Status: eperimental + o Author/Change controller: wayne@schlitt.net + o Specification document(s): this memo + o Related information: http://spf.mehnle.net/ 12. Contributors and Acknowledgements + This document is largely based on the work of Meng Weng Wong and Mark + Lentczner. Mark is not listed as an author by his request. While, + as this section acknowledges, many people have contributed to this + document, a very large portion of the writing and editing are due to + Meng and Mark. + This design owes a debt of parentage to [RMX] by Hadmut Danisch and to [DMP] by Gordon Fecyk. The idea of using a DNS record to check - the legitimacy of an email address traces its ancestry farther back + the legitimacy of an e-mail address traces its ancestry farther back through messages on the namedroppers mailing list by Paul Vixie [Vixie] (based on suggestion by Jim Miller) and by David Green [Green]. - Philip Gladstone contributed macros to the specification, multiplying - the expressiveness of the language and making per-user and per-IP - lookups possible. + Philip Gladstone contributed the concept of macros to the + specification, multiplying the expressiveness of the language and + making per-user and per-IP lookups possible. The authors would also like to thank the literally hundreds of individuals who have participated in the development of this design. - There are far too numerous to name, but they include: + They are far too numerous to name, but they include: - The folks on the SPAM-L mailing list. - The folks on the ASRG mailing list. The folks on the spf-discuss mailing list. + The folks on the SPAM-L mailing list. + The folks on the IRTF ASRG mailing list. + The folks on the IETF MARID mailing list. The folks on #perl. - The folks in the MARID working group and on the MXCOMP mailing - list. 13. References 13.1 Normative References @@ -1520,27 +1664,49 @@ [RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987. + [RFC1123] Braden, R., "Requirements for Internet Hosts - Application + and Support", STD 3, RFC 1123, October 1989. + + [RFC2034] Freed, N., "SMTP Service Extension for Returning Enhanced + Error Codes", RFC 2034, October 1996. + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC2181] Elz, R. and R. Bush, "Clarifications to the DNS + Specification", RFC 2181, July 1997. + [RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. - [RFC2396] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform - Resource Identifiers (URI): Generic Syntax", RFC 2396, - August 1998. - [RFC2821] Klensin, J., "Simple Mail Transfer Protocol", RFC 2821, April 2001. + [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April + 2001. + [RFC3513] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6) Addressing Architecture", RFC 3513, April 2003. + [RFC3864] Klyne, G., Nottingham, M. and J. Mogul, "Registration + Procedures for Message Header Fields", BCP 90, RFC 3864, + September 2004. + + [RFC3986] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform + Resource Identifier (URI): Generic Syntax", STD 66, RFC + 3986, January 2005. + 13.2 Informative References [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, November 1987. + [RFC1983] Malkin, G., "Internet Users' Glossary", RFC 1983, August + 1996. + + [RFC3696] Klensin, J., "Application Techniques for Checking and + Transformation of Names", RFC 3696, February 2004. + [RFC3833] Atkins, D. and R. Austein, "Threat Analysis of the Domain Name System (DNS)", RFC 3833, August 2004. @@ -1553,25 +1719,27 @@ Work In Progress - [Vixie] Vixie, P., "Repudiating Mail-From", 2002. + [Vixie] Vixie, P., "Repudiating MAIL FROM", 2002. [Green] Green, D., "Domain-Authorized SMTP Mail", 2002. -Authors' Addresses - - Mark Lentczner - 1209 Villa Street - Mountain View, CA 94041 - United States of America - EMail: markl@glyphic.com - URI: http://www.ozonehouse.com/mark/ +Authors' Addresses Meng Weng Wong Singapore EMail: mengwong+spf@pobox.com URI: http://spf.pobox.com/ + + + Wayne Schlitt + 4615 Meredeth #9 + Lincoln Nebraska, NE 68506 + United States of America + + EMail: wayne@schlitt.net + URI: http://www.schlitt.net/spf/ Appendix A. Collected ABNF This section is normative and any discrepancies with the ABNF @@ -1579,8 +1747,8 @@ grammar. See [RFC2234] for ABNF notation. Please note that as per this ABNF - definition, literal text strings (those in quotes) are case - insensitive. Hence, "mx" matches "mx", "MX", "mX" and "Mx". + definition, literal text strings (those in quotes) are + case-insensitive. Hence, "mx" matches "mx", "MX", "mX" and "Mx". record = version terms *SP version = "v=spf1" @@ -1590,8 +1758,7 @@ directive = [ prefix ] mechanism prefix = "+" / "-" / "?" / "~" mechanism = ( all / include - / A / MX / PTR / IP4 / IP6 / exists - / unknown-mechanism ) + / A / MX / PTR / IP4 / IP6 / exists ) all = "all" include = "include" ":" domain-spec @@ -1602,40 +1769,65 @@ IP6 = "ip6" ":" ip6-network [ ip6-cidr-length ] exists = "exists" ":" domain-spec - unknown-mechanism = name [ ":" macro-string ] - modifier = redirect / explanation / unknown-modifier redirect = "redirect" "=" domain-spec explanation = "exp" "=" domain-spec unknown-modifier = name "=" macro-string - - ip4-network = as per conventional dotted quad notation, - e.g. 192.0.2.0 - ip6-network = as per [RFC 3513], section 2.2, - e.g. 2001:DB8::CD30 - - dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ] ip4-cidr-length = "/" 1*DIGIT ip6-cidr-length = "/" 1*DIGIT + dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ] - domain-spec = *( macro-expand / macro-literal ) - macro-string = *( macro-expand / macro-literal / "/" ) - macro-expand = ( "%{" ALPHA transformer *delimiter "}" ) + ip4-network = ; as per conventional dotted quad notation, + ; e.g. 192.0.2.0 + ip6-network = ; as per [RFC 3513], section 2.2, + ; e.g. 2001:DB8::CD30 + + domain-spec = macro-string domain-end + domain-end = ( "." toplabel ) / macro-expand + toplabel = ALPHA / ALPHA *[ alphanum / "-" ] alphanum + ; LDH rule (See [RFC3696]) + alphanum = ALPHA / DIGIT + explain-string = *( macro-string / SP ) + + macro-string = *( macro-expand / macro-literal ) + macro-expand = ( "%{" macro-letter transformers *delimiter "}" ) / "%%" / "%_" / "%-" - macro-literal = %x21-24 / %x26-2E / %x30-7E - ; visible characters except "%" and "/" - transformer = *DIGIT [ "r" ] + macro-literal = %x21-24 / %x26-7E + ; visible characters except "%" + macro-letter = "s" / "l" / "o" / "d" / "i" / "p" / "h" / + "c" / "r" / "t" + transformers = *DIGIT [ "r" ] delimiter = "." / "-" / "+" / "," / "/" / "_" / "=" name = ALPHA *( ALPHA / DIGIT / "-" / "_" / "." ) + + header = "Received-SPF:" [CFWS] result [FWS [comment]] + [ key-value-list ] + + result = "Pass" / "Fail" / "SoftFail" / "Neutral" / + "None" / "TempError" / "PermError" + + key-value-list = key-value-pair *( ";" [CFWS] key-value-pair ) + [";"] + + key-value-pair = key [CFWS] "=" ( dot-atom / quoted-string ) + + key = "client-ip" / "envelope-from" / "helo" / + "problem" / "receiver" / + mechanism / "x-" name / name + + dot-atom = ; unquoted word as per [RFC2822] + quoted-string = ; quoted string as per [RFC2822] + comment = ; comment string as per [RFC2822] + CFWS = ; comment or folding white space as per [RFC2822] + FWS = ; folding white space as per [RFC2822] Appendix B. Extended Examples These examples are based on the following DNS setup: ; A domain with two mail servers, two hosts ; and two servers at the domain name - $ORIGIN example.com. @ MX 10 mail-a MX 20 mail-b @@ -1648,13 +1840,11 @@ www CNAME example.com. ; A related domain - - $ORIGIN example.org + $ORIGIN example.org. @ MX 10 mail-c mail-c A 192.0.2.140 ; The reverse IP for those addresses - $ORIGIN 2.0.192.in-addr.arpa. 10 PTR example.com. 11 PTR example.com. @@ -1666,7 +1856,6 @@ ; A rogue reverse IP domain that claims to be ; something it's not - $ORIGIN 0.0.10.in-addr.arpa. 4 PTR bob.example.com. @@ -1676,6 +1865,7 @@ These examples show various possible published records for example.com and which values if would cause check_host() to return "Pass". Note that is "example.com". + v=spf1 +all -- any passes @@ -1716,7 +1906,7 @@ This record would be used if mail from example.org actually came through servers at example.com and example.net. Example.org's - designated servers are the union of example.com and example.net's + designated servers are the union of example.com's and example.net's designated servers. la.example.org: "v=spf1 redirect=example.org" @@ -1725,15 +1915,14 @@ These records allow a set of domains that all use the same mail system to make use of that mail system's record. In this way, only - the mail system's record needs to updated when the mail setup + the mail system's record needs to be updated when the mail setup changes. These domains' records never have to change. - -B.3 RBL Style Example +B.3 DNSBL Style Example Imagine that, in addition to the domain records listed above, there are these: - $Origin _spf.example.com. + $ORIGIN _spf.example.com. mary.mobile-users A 127.0.0.2 fred.mobile-users A 127.0.0.2 15.15.168.192.joel.remote-users A 127.0.0.2 @@ -1794,7 +1983,7 @@ Copyright Statement - Copyright (C) The Internet Society (2004). This document is subject + Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. @@ -1807,5 +1996,5 @@ -Lentczner & Wong Expires April 12, 2005 [Page 46] +Wong & Schlitt Expires October 26, 2005 [Page 50]