Sender Policy Framework

The Sender Policy Framework is one of the big three email security functions commonly used to provide signing, non repudiation, allow listing and environmental controls to the aging email protocol. The other two functions are of cause DKIM and DMARC.

Sender Policy Framework, hereby called SPF for short, works very simply through a text record published to the public DNS zone of a domain. Here’s mine from “fabricius.no”:

v=spf1 include:_spf.domeneshop.no -all

So what does this mean?
v=spf1
This is the pretext of a SPF record. You will always have this in the SPF record. Without it, the information stored in the SPF would not be computer readable, rendering it useless. It refers to the SPF version, where the currently used one is version 1. v, version = SPF (version) 1.

Then you’ll have operators:
include:
redirect=
a:
mx:
ip4:
ip6:

These define how your record will work. the include: structure allow you to do DNS checks for services who maintain their own records, or you can use ip4: or ip6: to authorize a single IP address (or address space) to send your emails. The redirect= tag performs the same function as a HTTP 301 redirect and asks the DNS server to perform an other request to another domain, substituting it for the first domain. You can see this in action by making the following DNS lookup in terminal/cmd:

┌──(james㉿tux)-[~]
└─$ nslookup

set q=txt
gmail.com
Server: 192.168.20.5
Address: 192.168.20.5#53

Non-authoritative answer:
gmail.com text = “v=spf1 redirect=_spf.google.com”

The a: tag refers to the DNS A-record, and checks the A record of a given domain. This is useful if you’re running an email server from the same IP address as your webpage. Meaning you have configured the system to use PAT to determine the routing of your network, instead of using Server Name Identification to route the traffic. The mx: tag is identical, just referring to the mail exchanger IP address. instead.

At the end of the record, you’ll find one of the following:
-all
~all
+all

The end means different things and allow for different uses. When I talk to customers I often tell them it sends a signal to how well you control your email system. The -all tag means “everything else fails”. The emails from servers not listed are not authorized and should be rejected. The ~all means “softfail”, and means the email will be accepted, but marked. +all should really not be used, as it negates the entirety of your SPF record, allowing all servers to send using your domain.

Immediate pitfalls

The first pitfalls of creating a SPF record is to either allow too much, too little, or simply the wrong thing. We’ll go through these in turn.

Allowing too much
Allowing too much creates several issues, depending on what you are allowing. The main components are:

Allowing too many DNS records
Remember the include: tag? There’s a limit to how many DNS requests a SPF record can require a receiving email server to perform. It’s mainly a restriction to stop SPF from becoming a vector for DOS attacks. Imagine the strain on the receiving computer if it was supposed to perform a hundred DNS queries per received mail. The limit is arbitrary, but is set to 10. This means a SPF record like this:

v=spf1 include:spf.protection.outlook.com include:_spf.questback.com include:spf.altibox.no include:_spf.anpdm.com include:_spf.google.com include:spf.eu.exclaimer.net include:email.freshservice.com -all

Will automatically fail, as it has way too many DNS queries to perform. The reasons for these problems are usually because one or more of the services the SPF record refers to has poorly constructed records. In the last one it’s the “email.freshservice.com” record. Checking it, you’ll find it has several DNS lookups:

This means the receiving email server will have to first check the email.freshservice.com domain, then sendgrid.net, emailus.freshservice.com, emaileuc.freshservice.com and so on. One such poorly constructed record is usually why companies use the ip4: tags instead. However that has a different problem..

Allowing too many IP addresses
Because creating a good SPF record is difficult, a lot of companies use large IP chunks instead to “just give them the space the need” to services used. Checking the spf record for _spf..mailgun.org serves as an example of a large chunk spf record:

_spf.mailgun.org TXT = “v=spf1 ip4:209.61.151.0/24 ip4:166.78.68.0/22 ip4:198.61.254.0/23 ip4:192.237.158.0/23 ip4:23.253.182.0/23 ip4:104.130.96.0/28 ip4:146.20.113.0/24 ip4:146.20.191.0/24 ip4:159.135.224.0/20 ip4:69.72.32.0/20″ ” ip4:104.130.122.0/23 ip4:146.20.112.0/26 ip4:161.38.192.0/20 ip4:143.55.224.0/21 ip4:143.55.232.0/22 ip4:159.112.240.0/20 ~all”

Here you see quite massive chunks of IP addresses. For non-network personnel, here’s a rough guide:
/20 = 4096
/21 = 2048
/22 = 1024
/23 = 512
/24 = 256
/25 = 128
/26 = 64
/27 = 32
/28 = 16

And we have:
/20 x 3 = 12288
/21 x 1 = 2048
/22 x 2 = 2048
/23 x 4 = 1536
/24 x 3 = 768
/26 x 1 = 64
/28 x 1 = 16
= 18768 IP addresses who’re authorized to send emails on behalf of your domain if you add the mailgun.org spf record. This is compounded by other additions like microsoft’s spf.protection.outlook.com who add several thousand more.

Allowing too many services
Allowing too many services are very alike the problem above, but works slightly different. Services like sendgrid, mailgun and other SMTP HELO services (email sending through command line or other programmatic API services who are cloud native, often rely on each other to provide offloading. This means you’re often ending up adding the same IP addresses several times over. This isn’t really an issue as often as it becomes an high upkeep solution. By running a few test-emails and checking the headers, you could short down the spf record of services like freshservice.com to only the area record you use.

Allowing too little
This one should make sense on it’s own. If you’re experiencing issues with emails coming from one service, and not the ones coming from your core service (like Gsuite, exchange or your own email server) – you’ve probably forgotten to add the service to your SPF record. You can always have a look at the X-Authentication-Results header of your email and see if it has the Spf=fail notifier. If it does – your SPF record is wrong or missing service notations.

Allowing the wrong things
If you’re a consultant maintaining SPF records for a customer, you may come across the issue I found in an customer environment in February 2022. The customer had added a new servicedesk solution to maintain and answer internal tickets out of. The solution was implemented, but ticket update emails were being delivered to the spam box. I asked them if they had updated the SPF record, and whether they had configured DKIM signing for the service.

Tracking the headers, I found the X-Spam-Status: SCL-7 note from SpamAssassin. SCL is short for Spam Confidence Level, and denotes how certain the server about the email element is to be spam. Checking further in the headers, I found the Authentication-Results saying spf=fail. This indicates the SPF record doesn’t validate. I checked it, and they had added the wrong service. The customer had bought freshdesk.com and they had added zendesk.com’s SPF records using an include: tag. This of cause does not validate, leading to the emails being redirected to junk.

You can check any email’s headers by opening an email and pressing “File”:

Press the “Properties” button:

And then copy the information in “internet headers”:

You can then read it using notepad, or you can parse it automatically through a service like: Mail Header Analyzer

Allowing too much (not knowing what you’re allowing)
All services are equal, but some services are more equal than others. One of my favorite things to point out to Norwegian customers are when they’re allowing more than they know. A common service to configure on printers is the scan-to-email function. It’s beautiful. Scan a document and it arrives in your email or the recipient directly.

There are two common ways of doing this, or three if you’re a Microsoft 365 customer.
1. Use a dedicated user account, and let the printer authenticate the account upon sending the email with the attachment.
2. Send the email unauthenticated and unencrypted.
3. Send the email directly to the MX endpoint provided by Microsoft to send emails to your organization only.

The most common solution in the SMB market is to send the email unauthenticated/unencrypted. Whenever you see an email configuration setting sending the email to port 25 anywhere, you know you’ve struck gold. A common configuration found almost everywhere is something akin to this:

Server: smtp.altibox.no (local ISP)
Port: 25
SSL: No
Email: Scanner@exampledomain.com

This will flick the email to the server, which will accept the email and forward it to the email recipient in the “To” field of the email. This is often considered a SMTP relay, or otherwise known as a “smarthost”.

The great thing for security people, smart hosts are incredibly dumb.

When companies add these “smarthosts” to their SPF record, you’ll be able to send emails using their domain without ever authenticating with it. Especially if it is using the ~all ending tag. Take for instance a small to medium business working in the oil sector in Stavanger:

v=spf1 ip4:23.21.xxx.197 ip4:23.21.xxx.212 ip4:147.xxx.167.0/26 include:spf.protection.outlook.com include:_spf.psm.knowbe4.com include:spf.altibox.no -all

It’s great to see they care about phishing emails, but their poor SPF configuration puts them at risk from phishing from any customer who can reach the altibox smtp “smarthost”. You can easily check if you can by running the following command:

┌──(james㉿tux)-[~]
└─$ telnet smtp.altibox.no 25
Trying 109.247.116.10…
Connected to smtp.altibox.no.
Escape character is ‘^]’.
220 asav21.altibox.net ESMTP Postfix

As long as you’re not getting stuck on the “Trying…” part, you’ve got contact. And congratulations! You can now phish this company without ever breaching a single account, and use their internal users accounts to do it! The best part is if they post the users names and titles online. The rest is just a little bit of refinement and HTML fun.

So, how to build a good SPF record?
There are a few ways to make great SPF records. My personal favorite is to build service specific subdomains. Let me take you through a use case I had from the beginning of the year 2022:

Company McFinnigan produce fittings for pipes in the oil sector. They run Microsoft 365 for their internal users with Exchange online, with exclaimer to provide organization-wide signature configuration. In addition, they send out marketing emails using a slew of different services, as their business is world-wide. They also use freshservice.com to structure tickets and handle issues in the local IT servicedesk. But as we’ve shown, the freshservice.com SPF record is poorly configured, making it difficult to mix.

My favourite way to handle it is to make it easier to structure.
1. Internal users use the root domain: McFinnigan.com
v=spf1 include:spf.protection.outlook.com include:spf.EU.exclaimer.net -all

2. The newsletters and marketing emails use: marketing.McFinnigan.com
v=spf1 include:_vccnets.8×8.com include:_netblocks.eloqua.com include:spf.mandrillapp.com include:_spf.freshmail.pl -all

3. Support uses: support.McFinnigan.com
v=spf1 include:email.freshservice.com -all

This will make it much simpler to care for, and you’ll find it’s much prettier to look at when you’re receiving a marketing email. As you have to sign up for marketing emails today – if you’re sending out marketing emails like newsletters, your customers will know what it is when they receive them. You may as well be truthful. Let your users have the root-domain.

What does failure look like?
I will leave you with this monstrosity of an SPF record to show you how NOT to do things. I have hidden the real name of the company to be nice.

v=spf1 ip4:8.28.3.177 ip4:52.172.38.8 ip4:198.21.4.52 ip4:8.28.3.0/24 ip4:52.169.0.179 ip4:64.75.5.0/24 ip4:64.75.4.0/24 ip4:51.141.5.228 ip4:52.242.32.10 ip4:40.92.0.0/15 ip4:51.4.80.0/27 ip4:51.5.72.0/24 ip4:167.89.31.27 ip4:51.5.80.0/27 ip4:51.4.72.0/2″ “4 ip4:3.222.0.24/29 ip4:13.70.157.244 ip4:198.21.0.0/21 ip4:104.210.80.79 ip4:149.72.0.0/16 ip4:40.107.0.0/16 ip4:104.209.35.28 ip4:191.237.4.149 ip4:91.213.250.91 include:_s0.McFinnigan.com -all

But wait, what’s that hiding under the _s0.McFinnigan.com

v=spf1 ip4:8.21.164.0/24 ip4:104.47.0.0/17 ip4:52.100.0.0/14 ip4:51.140.37.132 ip4:50.31.32.0/19 ip4:167.89.0.0/17 ip4:52.233.37.155 ip4:52.172.222.27 ip4:38.99.96.0/24 ip4:198.2.178.0/24 ip4:198.2.128.0/24 ip4:109.70.58.0/24 ip4:104.40.229.156 ip4:200.98″ “.232.161 ip4:64.56.204.0/24 ip4:200.98.232.148 ip4:167.89.127.244 ip4:50.100.15.0/24 ip4:198.2.132.0/22 ip4:198.2.179.0/24 ip4:198.2.136.0/23 ip4:185.64.72.0/24 include:_s1.McFinnigan.com -all

Wait… there’s another one… _s1.McFinnigan.com…

v=spf1 ip4:198.2.177.0/24 ip4:198.2.180.0/24 ip4:168.245.0.0/17 ip4:198.2.186.0/23 ip4:217.163.57.0/24 ip4:217.18.206.0/24 ip4:91.227.208.0/24 ip4:141.145.10.0/24 ip4:141.145.12.0/24 ip4:194.71.205.0/24 ip4:141.145.11.0/24 ip4:198.37.144.0/20 ip4:208.117.” “48.0/20 ip4:205.201.136.0/23 ip4:205.201.139.0/24 ip4:142.165.219.0/24 ip4:216.136.148.0/24 ip4:192.254.112.0/20 ip4:103.239.164.0/24 ip4:103.252.162.0/24 include:_s2.McFinnigan.com -all

Nonononono

v=spf1 ip4:20.47.149.138/32 ip4:3.120.181.200/29 ip4:205.201.134.128/25 ip4:205.201.131.128/25 ip6:2a01:111:f403::/48 ip6:2a01:111:f400::/48 ip6:2a01:4180:4050:0400::/64 ip6:2a01:4180:4051:0400::/64 ip6:2a01:4180:4050:0800::/64 ip6:2a01:4180:4051:0800::/6″ “4 exists:%{i}._spf.mta.salesforce.com -all

This one does not even compute…

Here’s the kitterman response to the record:
Input accepted, querying now…
evaluating v=spf1 ip4:20.47.149.138/32 ip4:3.120.181.200/29 ip4:205.201.134.128/25 ip4:205.201.131.128/25 ip6:2a01:111:f403::/48 ip6:2a01:111:f400::/48 ip6:2a01:4180:4050:0400::/64 ip6:2a01:4180:4051:0400::/64 ip6:2a01:4180:4050:0800::/64 ip6:2a01:4180:4051:0800::/6″ “4 exists:%{i}._spf.mta.salesforce.com -all …
Results – PermError SPF Permanent Error: Invalid IP6 address: ip6:2a01:4180:4051:0800::/6″

An IPv6 subnet of size /8 (which is the largest you can have) is:
1,329,227,995,784,915,872,903,807,060,280,344,576 IP addresses.

And that’s more than people on earth. Significantly more. Please don’t do this, you’re making James cry.

I hope this helps someone. If it does, send someone else this page too.

Cordially,
James