SPF, DKIM, and DMARC Explained

-

 

I found myself reading RFC 733 today as I had no idea how long BCC emails had been around. Turns out, it's a long time (November 21, 1977). In fact, that is when the "Standard for the Format of ARPA Network Text Messages" was ratified so a whole lot of other things were included in that RFC that are still used in email today, not just BCC emails.

Some things that weren't proposed in 1977 for obvious reasons are SPF, DKIM, and DMARC. We'll be talking about those today! (If you're curious like I was what those RFCs are anyway, I linked them so you can check those out too). 

SPF - Sender Policy Framework

SPF is an authentication mechanism that allows an email server to verify that an email it receives was sent from an authorized server. SPF does this by checking the domain contained in the email header (specifically the envelope sender/MAILFROM), queries for an SPF TXT record that should be contained in that domain's DNS records, and depending on what that record says the email server can take appropriate action. 

Let's run through an example:
  1. Apple sends an email to your Gmail inbox. This email leaves Apple's email server bound for one of Google's email servers.
  2. Upon reaching Google's email server, Google's server checks the domain contained in the envelope sender (in the header of the email).
  3. Google's server checks the DNS records of email.apple.com and determines that the sending IP address (sending email server) is authorized to send email from email.apple.com. Hence we can see in the email header spf=pass along with the IP address of the sending email server, and the statement that the IP address is a permitted sender.


  4. As a result of SPF passing, Google's email server will not mark the email as spam ultimately helping it reach your inbox. This isn't the only check it will perform but we'll get to the rest later.
So what exactly is the SPF record that Google checks to ensure that the specified IP address / domain is a permitted sender? 

v=spf1 include:_spf-txn.apple.com include:_spf-mkt.apple.com include:_spf.apple.com ~all
SPF records can be checked a number of ways, but MXToolbox's SuperTool is my favorite (and where the above output came from). 

A quick breakdown on each of the components included above, also courtesy of MXToolbox's output.

PrefixTypeValuePrefixDescDescription
vspf1The SPF record version
+include_spf-txn.apple.comPassThe specified domain is searched for an 'allow'.
+include_spf-mkt.apple.comPassThe specified domain is searched for an 'allow'.
+include_spf.apple.comPassThe specified domain is searched for an 'allow'.
~allSoftFailAlways matches. It goes at the end of your record.
A few notes:
  • "include" statements can have hostname or IP address values (or both!)
  • The "~all" in Apple's case could just have easily have been "-all" if they wanted (or not included it at all). The difference being that "~" tells the receiving email server that if SPF fails, the email should be marked as suspicious. "-" tells the receiving email server that if SPF fails, the email should be rejected.
  • Instead of (or in addition to) "include" SPF records can contain other ways of designating permitted senders such as "a" and "mx".
In the example above you will notice that the IP address in the header (17.111.110.120) isn't specifically listed in the SPF record, nor is email.apple.com. Yet, SPF passes.

The reason for this is _spf-txn.apple.com which does specifically list 17.111.110.0/23 as a permitted sender (among other subnets). In this way the SPF record is nested with permitted senders! Not something I personally see very often, but for a company the size of Apple it makes sense.

SPF is not without its pitfalls though. The biggest being that SPF looks at the envelope sender (MAILFROM) -- the part of the email that most users don't see. So attackers can use a domain they control, implement the correct TXT record for SPF to pass, but then spoof the the part of the email that users do see (letter sender/FROM) tricking them into thinking it has passed a security check it really hasn't.


DKIM - DomainKeys Identified Mail

DKIM is an authentication mechanism with many similarities SPF. It verifies that an email server did indeed send a particular email, does this by performing a lookup of DNS records, and looks at the envelope sender/MAILFROM of an email to know what DNS records to lookup. However, there is a bit more under the hood. Let's go back to our Apple example we used in SPF:
  1. Apple sends an email to your Gmail inbox.
  2. Before the email leaves the Apple email server, the email body is hashed, and the body plus select email headers are digitally signed (hashed then encrypted with a private key).
  3. Upon reaching Google's email server, Google's server checks the domain contained in the envelope sender/MAILFROM (in the header of the email).
  4. Google's email server, upon checking the DNS records of Apple, sees that Apple has published a public key as a TXT record. 
  5. Google's email server takes this public key, decrypts the digital signature in Apple's email, and verifies that the hash Apple provided matches the value Google gets when they hash the received email. If the values match, DKIM passes!
This authentication design provides not just authenticity but integrity of the email as the hashes would not match if the email was altered in transit.


Below we can see a breakdown of what some of the values in the DKIM signature are:

v=DKIM1; k=rsa; h=sha256; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAy4V3QZgjpUK3HSijzU+uGQlfBx8N40Ese/TaM0P8BES/kTKJS7OvtEfmA+ihMYcwh+vpvjK7aBIpV2CT49QnC/lXm8KwXad64VF18pgaCiuOW4rxC2L687Gn2BHEekcbl5FozRa716DLXpo07j5IX5sdvKPi6KylnmyOmjD/NqfvbLZf/lRlTb9pXf1N3fLu+W25vaotR4ZQpbrMQkKIANDafdL4KvPmfFOfYuZYiLpQfHfuJYok0aROsS5as1cEthN9MnkdSBAJHLG/f63+jNLgSC9x77YBWH2gPIDIqEanPVPFPJjs0yNh1zCWJUma9ihNXNwBP9GDclt042thAwIDAQAB
TagTagValueNameDescription
vDKIM1VersionIdentifies the record retrieved as a DKIM record. It must be the first tag in the record.
krsa (Length: 2048 bits)Key TypeThe type of the key used by tag (p).
hsha256Hash AlgorithmsA colon-separated list of hash algorithms that might be used.
pMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAy4V3QZgjpUK3HSijzU+uGQlfBx8N40Ese/TaM0P8BES/kTKJS7OvtEfmA+ihMYcwh+vpvjK7aBIpV2CT49QnC/lXm8KwXad64VF18pgaCiuOW4rxC2L687Gn2BHEekcbl5FozRa716DLXpo07j5IX5sdvKPi6KylnmyOmjD/NqfvbLZf/lRlTb9pXf1N3fLu+W25vaotR4ZQpbrMQkKIANDafdL4KvPmfFOfYuZYiLpQfHfuJYok0aROsS5as1cEthN9MnkdSBAJHLG/f63+jNLgSC9x77YBWH2gPIDIqEanPVPFPJjs0yNh1zCWJUma9ihNXNwBP9GDclt042thAwIDAQABPublic KeyThe syntax and semantics of this tag value before being encoded in base64 are defined by the (k) tag.
One not mentioned in the table above is the d tag. d indicates the domain that is checked for record of the public key (in our example, email.apple.com).

Another piece not mentioned is the s tag. s indicates the selector that is used in combination with tag d (remember, the domain) to locate the public key. In or example it is email0517. This can be changed and made almost any custom value an administrator wants. Selectors are beneficial in the event multiple public/private key pairs are used and you want some emails to be sent using key1 versus key2. Another benefit is easier key rotation -- in the event of compromise of key1, key2 can be used for all emails while key1 is decommissioned and replaced with another key.

DKIM is great for what it does and as we've seen provides additional benefits over just using SPF alone. However, DKIM also just looks at the envelope sender/MAILFROM address which isn't presented to the user. This means that someone can create a phishing email that passes both SPF and DKIM perfectly fine, but also spoofs your company's domain in the letter sender/FROM field (the FROM field you see when you open an email).

Thankfully those that send phishing emails usually don't go to the trouble of setting all that up. If they do, that's where DMARC comes in.


DMARC - Domain-based Message Authentication, Reporting & Conformance

While not an email authentication protocol, DMARC builds on top of SPF and DKIM. Without enabling at least one of them first you cannot use DMARC. DMARC also has its own policy to be published in DNS which instructs email servers on what to do if SPF or DKIM fail. An even better feature of DMARC in my opinion that often goes unmentioned, is that it checks against the FROM sender unlike SPF and DKIM.

Let's run through our example from email.apple.com to see what DMARC does:
  1. Apple sends their email to Google's email server. Apple has all the necessary SPF, DKIM, and DMARC DNS records published.
  2. Google's email server receives the email from Apple and runs through the SPF and DKIM checks.
  3. Google's email server now checks DMARC. It does this by:
    • Looks up the DMARC policy of the letter sender / FROM domain (remember this is the one users can see in the email).
    • Verifies if DKIM passed.
    • Verifies if SPF passed.
    • Verifies if the domain in the letter sender / FROM field "aligns" with the domains that SPF and DKIM check. In other words, does the sending email address that's visible to the user match the email address in DKIM's (d) parameter and SPF's envelope sender/MAIL FROM.
  4. Apple's email passes SPF, DKIM, and DMARC checks so Google's email server passes the email along to the end user.
This is how email should flow in a relatively simple case. We'll look at what happens if things go wrong in just a bit.


Just like SPF and DKIM, you can view the header of an email to see if DMARC passed or failed, along with additional detail about the domain's DMARC policy.

Details that aren't available in the email header but are available using MxToolbox's DMARC tool are shown in the table above.

v=DMARC1; p=reject; rua=mailto:d@rua.agari.com; ruf=mailto:d@ruf.agari.com;
TagTagValueNameDescription
vDMARC1VersionIdentifies the record retrieved as a DMARC record. It must be the first tag in the list.
prejectPolicyPolicy to apply to email that fails the DMARC test. Valid values can be 'none', 'quarantine', or 'reject'.
ruamailto:d@rua.agari.comReceiversAddresses to which aggregate feedback is to be sent. Comma separated plain-text list of DMARC URIs.
rufmailto:d@ruf.agari.comForensic ReceiversAddresses to which message-specific failure information is to be reported. Comma separated plain-text list of DMARC URIs.
Looking again at our example of email.apple.com we can see that Apple has set a policy (p) of REJECT. They have also set their subdomain policy (sp) to REJECT. When a subdomain policy is present it take precedence over the domain policy. Other possible values for these policies are NONE (do nothing) and QUARANTINE (spam folder of the email server or end user, depending on email server).
If Google's email server determined that SPF, DKIM, or DMARC validation failed it would look at the policy published in Apple's DNS server and take that action. An important note on the logic of this policy evaluation:
DMARC only cares if SPF OR DKIM pass. If one fails and the other passes, DMARC considers this a pass. That being said, DMARC can still fail if the domain fails "alignment".

Summary


SPF, DKIM, and DMARC are great additions to RFC 733 and are extremely useful tools in verifying the legitimacy of emails. They're one of the first things I check when detecting phishing emails. 

If you have any questions or want me to cover any specifics of SPF, DKIM, or DMARC let me know!

Comments

Post a Comment

Popular Posts

Hashcat - Cracking MD5 and NTLM Hashes

-
Image
Today we'll be exploring Hashcat -- "the world’s fastest and most advanced password recovery utility". This, along with similar tools, should be used only for password recovery, pentest engagements, CTFs, etc and never for illegal purposes. I could write an entire series about the capabilities Hashcat provides, but we will cover a few basic examples for now ( besides, Hashcat does a pretty good job of that themselves ): Crack MD5 hashes using the rockyou.txt wordlist. Crack NTLM hashes using the rockyou.txt wordlist. Crack NTLM hashes using a mask attack (modified brute force). I'll be using Kali Linux as Hashcat comes pre-installed, but Hashcat can run on Windows, macOS, and other Linux distributions as well. As you'll see, I'll be using some lists of hashes I made previously. Tools to generate hashes using dozens of algorithms are available online -- just save the output as a text file if you want to follow along. Cracking MD5 Hashes Using rockyou.txt Wordl...
Read more

Configuring Secure Ciphers Suites and TLS

-
Image
Hey everyone, today we're back on cipher suites. If you want a refresher of TLS and secure cipher suites overall, check out my previous post . There are many instances in which you'll need to edit cipher suites on a system -- compliance efforts, CIS benchmarks, or simply ensuring your system doesn't use insecure suites. There are a few ways to go about this and I'll detail two of them now: IIS Crypto and the Windows registry. IIS Crypto My favorite way of editing TLS versions and cipher suites is using IIS Crypto . IIS Crypto allows you to select your desired TLS/SSL version, cipher suites, and backup the registry, all with a few mouse clicks. Only downside is that it's Windows only (even the command line version). This is what IIS Crypto will look like on an unmodified Windows 10 system. Note the separate  Server Protocols  and  Client Protocols  sections. These are important to keep straight depending on what system is listening for connections and what system is ...
Read more

Configuring SSH Key Based Authentication

-
Image
Overview Most personal computers and many organizations running Linux servers utilize password-based authentication. This is what most of us are used to -- you enter a username and password in order to authenticate your account and gain access to a system. While straightforward and commonplace, this isn't the best solution in most cases. Today I'll be explaining why key based authentication is more convenient, more secure, and allows for automation of tasks without saving credentials in plain-text in a text file. Key based authentication involves the use of private and public keys ( public-key cryptography ). The private key is stored on the client and the public key is stored on the server in a special file ( ~/.ssh/authorized_keys ). This allows any system with the corresponding private key to login to the server where we just stored the public key. Our steps are: Generate key pair (public / private key). Transfer public key and add it to the authorized_keys file on the remot...
Read more