The Problem and the ApproachUnsolicited commercial email, email fraud attempts, and email viruses, collectively known as “junk email” and “spam”, have increasingly become a burden on the Internet. Junk email takes bandwidth to transmit, storage space to receive, and time and resources to either manually discard or programmatically block. Viruses in particular wreak havoc upon their victims and their associates. And blocking mechanisms invariably make mistakes, capturing legitimate email as well. End users, especially home users, are increasingly turned off by the barrage of offensive advertising; the Internet experience is soured by junk email.
Some of the most deceptive senders of junk messages attempt to evade the consequences of their actions by forging the source addresses of the messages they send.
Problem:This is possible because SMTP, the standard method of delivering email messages across the Internet, has no way to verify the sender’s address; SMTP was designed in a more innocent age of the Internet, when it was assumed that all participants were trustworthy.
SPF is an attempt to address this oversight by requiring that all email from a certain domain name comes from an authorized computer. Each domain name, in its DNS records, can specify what hosts are allowed to send mail for that domain name. Any SMTP server that receives a message can use the SPF mechanism to try to filter out messages with forged sender addresses.
How SPF worksDuring an SMTP connection in which message delivery is attempted, the sending host must identify the sender’s email address (the envelope sender address). The receiving server can either accept the sender’s address or reject it; the receiving server can perform any of a number of tests on the sender address before making its decision. For example, it’s common to verify that the envelope sender’s domain name actually exists in the DNS, and that a DNS record exists that can be used to attempt return delivery. SPF adds another test to the arsenal.
To perform an SPF check, the receiving mail server performs a DNS query looking for TXT records with the same name as the sender’s domain name. If any are returned, it looks through the result set for TXT records whose data starts with “v=spf1”; this is a special identifier declaring that the record is an SPF record. If an SPF record is found, the record is parsed to see if it authorizes the sending host to deliver mail for this domain.
An SPF record contains one or more rules, called “mechanisms”. A mechanism can specify, for example, that the SMTP servers that receive mail for a particular domain are also allowed to send mail for the domain. Or a mechanism can specify that a particular host, or any host in a particular subnet, is allowed to deliver mail for a domain.
Typically, an SPF record ends with a mechanism that declares what to do if a sending host fails to match any previous mechanism. The possible outcomes of an SPF test include “pass”, “fail”, “soft fail”, and a few others. If an appropriate SPF record can’t be found, for whatever reason, or if it’s malformed, it’s recommended that a default value be applied. The default value is:
“v=spf1 a/24 mx/24 ptr -all”At the time of this writing, the details of how an SPF mechanism is constructed are available here:
As stated before, an SPF record is a TXT record with the same name as the domain name of the envelope sender address. The record’s data field begins with “v=spf1”. After that, the data field continues with mechanism definitions, separated by spaces. For example:
example.com. TXT “v=spf1 a/24 mx ~all”Mechanisms are tested from left to right. So the mechanisms above amount to the following rule set:
- If the sending host has an IP address in the same class C subnet (24-bit subnet) as the host named “example.com”, it’s allowed to send mail for “example.com”.
- Look for MX records named “example.com”, and then resolve the list of mailhost names to a list of A records. If the sender’s address is in this list, it’s allowed to send mail for “example.com”.
- Soft fail. If the sender’s address hasn’t matched either of the previous two mechanisms, it fails softly.
So, what should an MTA do with a message based on the result of the SPF test? There are seven different possible results. It’s up to the MTA’s administrator to decide how to handle each case. For example, an administrator may decide to assign each result type a value, for use by an additive
Acting on the result
spam evaluator such as SpamAssassin -messages that pass may get a negative value, while messages that fail may get a higher value than messages that soft-fail. Or failed messages could be bounced, while soft-failed messages could have a warning attached to the headers.
How is SPF implemented?
A typical domain will need two types of SPF records, authorization and denial. The authorization record (and there’s usually just one or a few) describes a list of hosts that are permitted to send mail for the domain (and sometimes its subdomains). A denial record (and there will generally be many of these) simply says that a subdomain name is not allowed to send mail -period.
Generating your own SPF records
The easiest way to generate an authorization SPF record is to use a wizard, which can be found at a variety of places around the web. At the time of this writing, one place to find an SPF wizard is here:
Otherwise, you can create one manually out of the mechanisms described above. For example, to indicate that your inbound mail exchangers are also your outbound MTA’s, and that no other server should be sending mail for your domain, use this record:
example.com. TXT “v=spf1 mx -all”When evaluating this, an SMTP server running an SPF test would retrieve all MX records named “example.com.” and construct a list of mail exchanger hostnames. It would then perform A record queries (if necessary) to look up the IP addresses of these hostnames. If the source IP of the SMTP connection is in the resulting list, then the test is passed.
Otherwise, because of the “-all”, the result is failure. (If the last mechanism listed were “~all” instead, the result would be a soft fail.) As another example, to indicate a specific hostname, you could use this syntax:
example.com. TXT “v=spf1 a:spf.example.com -all”Then create an A record named “spf.example.com” for each authorized outbound MTA -if there are 12 of them, create 12 A records. Obviously, this method is limited to about 10 to 15 IP addresses, due to the size limit of the UDP packet used in DNS.
As a final example, if you have an entire subnet of outbound MTA’s, you could use this syntax:
example.com. TXT “v=spf1 ip4:192.168.14/24 -all”This should be self-explanatory. The reason for denial SPF records is, it may be undesirable to have some spammer fake the address “firstname.lastname@example.org” -it’s likely that there are no legitimate email addresses ending in “www.example.com”, but that doesn’t stop someone from forging such an address.
This SPF record does, though:
www.example.com. TXT “v=spf1 -all”Any domain name (including subdomain names) that has either an A record or an MX record can reasonably be given an SPF record.
The problem that can occur with SPF is, if your outbound MTA is configured to use SPF (which is reasonable and prudent to do), if no special consideration is made for client machines sending email, the SPF test will prevent everyone from sending mail to the server.
Configuring for in-house interaction
This is because clients usually connect to their outbound MTA using the same mechanism as is used to send mail between MTA -the SMTP protocol. The solution varies by implementation, but typically involves either a special SPF record for use by the MTA only, or a whitelist, or something else along those lines.
Similarly, if client machines send mail through a relay which then connects to the main outbound SMTP server, special consideration must be made so that the SPF test doesn’t prevent normal, legitimate outgoing mail from being delivered as intended.
The Men & Mice Suite includes a Management Console for DNS server configuration. In the Management Console, SPF records must be entered without the quotes normally surrounding the SPF string.
SPF records and the Men & Mice Suite
An example zone with SPF records might look like the following picture:
Is SPF an effective way to stop junk email?
Email worms, which take advantage of the ubiquity and scriptability of Microsoft’s Outlook family of email clients, often now use their own SMTP engines in order to bypass filtering implemented at the local SMTP relay. SPF records will cut down on the viability of this approach, and this may turn out to be the largest contribution to general Internet usability made by the SPF mechanism. However, this requires that publicly advertised SPF records do not authorize entire subnets -care should be used to ensure that only certain gateway hosts can send mail out to the Internet.
Junk mail involving fraud often uses forged source addresses. As SPF becomes more and more widely implemented, it will become harder to forge an address and get away with it. A forged address will be stopped by any domain that has an SPF record, and a domain that doesn’t have an SPF record
will become increasingly suspicious.