Email shouldn't feel like a dark art, but to a lot of people it does. Everyone should have DMARC setup by this point, but they don't. Here's the first piece of a 3 part guide covering why it works and how to set it up.
Since writing about how to reverse account takeovers last week I've decided to write a security series covering all the weird things I encountered back in 2012, when I accidentally ended up combating phishing and fraud for a year. In the last article, the first recommendation was to setup DMARC. So let's take a deeper look at why, how and what's involved in long term management once it's setup.
// Share.on([hacker news,
linkedin,
twitter,
facebook,
reddit])
This guide is exhaustive and is broken into three parts: Education, Deployment and Enterprise. In this post we'll cover the problems of phishing, why they exist, what DMARC is how it works, why it works and why everyone needs it. In Deployment next week, we go into detail on how to deploy DMARC from a domain with no email to a startup with a few email sources all the way to an enterprise with an entirely unknown number of email sources.
ANNOUNCEMENT: At BSides on October 28th, 2023 I announced the upcoming 2024 private beta of dmarcSTAR, a service that approaches DMARC in what I believe is the ideal manner after over a decade of experience with the specification. Read more at dmarcSTAR.com.
I walked into a weird situation in 2012. I took over everything for a 14 year old high end marketplace that was in the middle of a rewrite from Perl to Ruby, in which the prior developers and support staff were gone. Because of the transition from old site to new site, old design to new design and the mixing of "some new" and "some old" in the look of things, there was a lot of confusion among the site's 300,000+ users already. When the phishers realize they could capitalize on this confusion, things got crazy fast.
In my first week of trying to figure things out, I was presented with an email that had been sent to a large number of our users. It looked exactly like the sites new emails, instructing them that they needed to visit a particular page to login. The email looked sharp, professional and didn't include choppy language. The reason that we given for the users to login was even believable and somehow related to the site migration. The link to login, of course, went to a fake login page made up to look just like ours. The kicker though, was that this email had been sent From: [email protected] and that made it completely believable.
I didn't handle it well. My first instinct was to reset EVERYONE's passwords because so many people were going to fall for this and initially I did exactly that before I had to reverse it. One of the other lessons that I learned from this site was that the people paying a lot of money to use the site as vendors would tolerate almost no change, including ever having to change their password from the...6 character minimum requirement. Yes, six, that wasn't a typo.
While that was certainly hard to hear, it set the requirements for everything we would build over the next year to try to protect all of these users. We had to protect them secretly and silently, without asking them to change how they did anything at all.
That was not the last phishing email. They were constant, well engineered and intentionally timed.
At the time, I had no idea how to solve this problem because it wasn't something I'd ever had to deal with before. As a developer, all I ever cared about was that my application emails were being delivered. The thought of trying to prevent other emails that said they were mine hadn't ever really been a concern. Isn't that what spam filters were for? I'm not supposed to have to worry about this, am I?
So to combat the problem, we decided to look at the structure of the emails to see what we could do. All of the messages were linking back to the same fake login page, so we would find out where it was hosted and reach out to the abuse team to ask them to take it down. The first time, the team responded quickly. The 50th time...we ended up in a queue. Our phish were usually taking advantage of some free site generator or promo plan. Eventually we developed an internal process for finding the site, contacting whatever abuse team was involved and trying to get the phishing site itself taken down.
This strategy actually worked pretty well. Our phish would keep creating new sites of course, but we knew we were frustrating them because their behavior changed. They started waiting to send out the phishing emails after east coast work hours so that we would be out of the office. One time they even used a host in South Africa whose abuse team only accepted phone calls and a number of hoops had to be jumped through to even find that number. Thank goodness for Skype back then too.
These delay tactics worked for them though. IBM research found that 50% of credentials captured by a phishing email were harvested within the first 60 minutes of it being received, which they called The Golden Hour. We needed to make sure the phishing site came down as fast as possible to protect our users, but this was not a winning strategy and involved constant work on our part. We had to find a way to stop the emails themselves.
We even reported the phishing sites to multiple different outlets to track if it helped with take downs. One thing that we did find was that phishing sites reported to PhishTank were usually blocked by OpenDNS within a couple of hours. We didn't have as fast of a turnaround with the other reporting outlets though. I recommend reporting phishing to all of the following...
As far as I can remember, DMARC, which stands for Domain-based Message Authentication, Reporting, and Conformance, first publicly debuted in January of 2012 just as I was starting this position. Google had published a guide to rolling out DMARC to help prevent spoofing, phishing and spam. That sounded like exactly what I needed, so I looked into it. I had no idea what I was doing. The only thing I knew was that I had a problem and this thing sounded like it would fix it. I was following the Google guide, reading the RFC specification, bothering the people at Sendgrid support to see if they could help.
I'm telling you all of that to tell you this. I got this up and running successfully with no clue what I was doing during a time when there were no resources, online tools, tutorials or guides that could help. I was looking for all of that stuff because I assumed this was going to be a lot harder than it really was. It's not. DMARC is easy, which is exactly why everybody should have it setup.
In order to explain that, I need to give you a brief introduction into how email works for your domain name.
Let's say you have a great business idea, so you go and buy a domain name. I've always dreamed of opening a pirate themed gym called "Slimmer Ye Timbers!" where people can work on their Arrrms and walk the Planks. The marketing kinda writes itself and be honest, you're a little curious right now.
Anyway, say I go out and buy the domain slimmeryetimbers.com for my website. Then I have to get a hosting company and setup a DNS record to point slimmeryetimbers.com to that web host to so people go to the right place when they visit the domain.
Email doesn't work like that. If I setup a mail server and point to it with an MX DNS record so that people can send messages to my new [email protected] address, it works great. Exactly like running a website and pointing the domain to the right place.
The difference is that anyone, anywhere can send an email to somebody else claiming to be from my fancy new domain. This is why we have spam. This is why we have phishing. By default, anyone can send email claiming to be from any domain that exists.
Arrgh...
DMARC lets you turn it off and control exactly what's allowed to get through, so the only email saying it's from your domain is actually from your domain.
DMARC isn't the first specification to try to fix the spam problem. Multiple others have come along in the past with differing degrees of adoption. DMARC builds on two of them, called SPF and DKIM. Both of these protocols have gaps that are addressed by the other. With DMARC setup, you're telling any mail server that get an email claiming to be from your domain "If this email doesn't pass either SPF or DKIM for my domain, it's not from me and you can discard it (or send it to spam)."
It's important to keep this in mind too. Without DMARC, every email server that gets a message claiming to be from your domain has to make it's own rules to figure out if the message is legit or not. That means running blacklists, IP reputation scores, spam algorithms, machine learning, checking the age of the domain, doing reverse DNS look ups on the IP address where it came from, among other things. Ever been asked to add a business to your contact list? That's why.
Even if you do have SPF and DKIM setup, receiving mail servers have no way of knowing if you have it everywhere so they still have to guess. With DMARC, you're removing the guesswork for them and telling them exactly what to do.
The Sender Policy Framework works by publishing a DNS record(s) that lists the IP addresses or domains that are allowed to send email from your domain. All that you need to publish or modify an SPF record is access to your domain's DNS, so it's fairly straightforward to setup. The biggest challenge is identifying everywhere that you're sending email, which DMARC will make easier for you.
Here's an example of an SPF record for your domain, which is just a simple TXT record.
v=spf1 mx ip4:64.18.0.0/20 include:_spf.google.com ~all
This simple record says these IP addresses are allowed to send email from our domain...
mx allows the servers referenced by our DNS MX records, which points to the email servers that receive email since many will also send email.ip4:64.18.0.0/20 allows any IP address in the 64.18.0.0/20 range to send email. We can also specify individual IP addresses. ip6 can be used as well.include:_spf.google.com allows us to delegate management of some of our IP addresses to Google, who can update this record whenever things change without us having to worry about it.The v=spf1 indicates that this is an SPF record since DNS TXT records can be a lot of different things. The ~all at the end covers how strictly to enforce the rule, which we'll talk about in deployment.
When a mail server receives an email from us it will be able to see the IP address of the server that sent it and the domain name it claims to be sent from. That mail server will then check to see if there is a corresponding SPF record and verify that the sending IP addresses exists in this approved range.
It's so simple that it makes you wonder why anything else is even necessary, but email is a big world and SPF doesn't cover all of the use cases. If you send an email to a mailing list like a LISTSERV, your email is sent to the server of the list serve itself and then the list serve forwards the message from you out to every address on the list. In this case, the email will come from you but be sent from a server that you have no control over at all which will fail an SPF check.
Another situation is if you send an email to someone who is using a forwarding address, like a collegiate @alumni.university.edu address. The SPF check will fail after forwarding and if you're enforcing SPF too strictly legitimate messages will be entirely lost.
Lastly, an email server which is used by multiple domain names may allow other people to send on your behalf if the owner of the server hasn't taken steps to prevent it.
DomainKeys Identified Mail (DKIM) Signatures works by attaching an encrypted email signature to each outgoing message. A private key on an outgoing mail server will reference a public key published in your domain's DNS that can be used by the receiving mail server to verify the signature. Because the signature is based on the body of the message, a failing DKIM check can also indicate that they message may have been modified in transit.
It's more complicated to implement since it has to be setup individually for each sending mail source, but it's more reliable because the signature survives forwarding which fills the gap in SPF. Most 3rd party email senders today will require you to setup DKIM before even allowing you to send an email, but go out of their way to make this process easy for you.
But DKIM has flaws too.
The biggest flaw with DKIM is that if a mail server receives a message that doesn't include a DKIM signature, it has no way of knowing that there was supposed to be one present. This will lead to treating DKIM as a trust modifier if it's present, but not a dis-qualifier if it's not.
You can see this if you look at the scoring system for a tool like SpamAssassin, which is a popular open source library used to help filter potential spam messages. It uses a scoring system to judge messages based on a lot of different criteria. If a message scores higher than a certain score threshold, it will be classified as spam. Valid DKIM signatures will lower your score, but lack of a DKIM signature doesn't really hurt you.
There are a number of corporate email systems and vendors who provide services that will inspect incoming emails. Sometimes these just scan for viruses in attachments, but others will wrap the links in an email message with a new URL that will inspect the contents and warn you of hazards when you try to click them. These link replacements will invalidate the DKIM signature since the message was modified. Luckily, these systems are usually setup by the entity with the receiving mail server itself so steps can be taken to verify the DKIM signature before the message is modified.
When we talk about using public/private key encryption there's always an associated bit strength to the encryption. 256, 512, 1024, 2048 or even 4096 are what's commonly circulated as of this article. The lower the strength, the easier it will be to crack the private key. This is important because if anybody finds out the private key, email can be sent impersonating the domain and fully pass the authentication checks.
The most famous DKIM cracking incident actually happened at Google thanks to Zachary Harris, Math Pirate. He received a headhunters email coming from Google and thought that the message might have been spoofed, so he looked at the DKIM signature and realized it had a weak 512-bit key. The DKIM standard calls for 1024-bit minimum. He didn't believe that Google would be so careless, so he thought it was a recruiting test and proceeded to crack the key before sending an email to Larry Page from Sergey Brin, Google's founders. 2 days later Google's DKIM key was using a 2048-bit key, but Zachary never got a response.
When you setup a secure certificate on a website, it comes with an expiration date that will require it to be renewed. DKIM keys have the same need, but it's not required so it's often neglected. Protecting your DKIM private keys is extremely important, otherwise none of this matters. If a DKIM key is cracked in the wild and you don't know about it, the cracked key will keep working until you rotate yours. That means periodically rotating your keys on every outgoing email server that you have, which is going to be tedious if it's not automated.
Luckily, more and more email providers are automating this process for you by providing you with two CNAME DNS records that they can manage for you. This will allow them to change the key on one record while the other is still in use to avoid any disruption to recently sent mail. I know that several already do this, but I'm not aware of a comprehensive list of providers who offer automatic DKIM rotation. Personally, I would go out of my way to avoid providers who don't.
Even if you have a very high strength key, the knowledge of how many years it would take to crack doesn't protect you from everything. A departing employee with privileged access could take the key or leak it. A compromised or unpatched server could allow the key to be stolen. And let's be honest here, if you have IT staff who aren't willing to take the time to rotate those keys how likely is it that the mail server is being properly secured anyway?
DMARC builds on both SPF and DKIM, providing clearer rules about their usage while filling their weak spots. Here's how it accomplishes it.
First, the requirement for a message to pass a DMARC check is that it must pass a domain-aligned SPF or DKIM check, not both. What does that mean?
DMARC also introduces domain alignment to both SPF and DKIM, because it's possible for SPF and DKIM validation to both pass using domains that are not visible to the user at all which have nothing to do with the From address that you see in your email client. In order for SPF or DKIM to pass with DMARC, these checks must match up with the domain name in the From address so users can see it too. That's alignment. There are several simple examples present at the end of the DMARC specification to help alignment make more sense.
Here's an example of aligned DKIM...
DKIM-Signature: v=1; ...; d=example.com; ... From: [email protected] Date: Fri, Feb 15 2002 16:54:30 -0800 To: [email protected] Subject: here's a sample
The best part though, is that DMARC is extremely simple to setup to provide all of those benefits. It's just a single DNS TXT record. Most of the work comes from trying to figure out all the places you're sending email...but DMARC even helps with that.
Once we had DMARC setup, the phishing emails dropped off a cliff. We went from multiple daily reports to infrequent monthly reports. When a phishing email did still get sent, it was always from some other domain that our users could more easily spot. They would switch interior letters of the domain name or try to send from similar domains. Users got better at spotting this, but many email systems will even detect "similar" domains as part of their spam filters now.
Stay tuned for part two next week, Deploying DMARC Without Breaking Everything.
// Share.on([hacker news,
linkedin,
twitter,
facebook,
reddit])