Email Deliverability Explained

Getting your message through can make or break your business. Now, new rules for email security are making that harder. But you can fix it once you know what to ask for...

First, let me explain a little big about how the email system works for we dive into how to fix the problem...

Email was created before security!

The email messaging system was created in a time before the internet became a thing. In these before times, everyone trusted each other and so security wasn't really a consideration. Passwords were still a thing, but that was more for basic privacy. After many iterations, the early creators of the systems that later become the internet, Simple Mail Transfer Protocol - Wikipedia (SMTP) was published in November 1981 as a standard mechanism for delivering messages.

All the messages were, and still are, sent in plain-text. That means that anyone can read any message, all they need to do is look in the right place.

Email is like walking out your front door. Anyone who is looking will see everything. Make sure you have pants on.

The other problem is that SMTP had no mechanism that force you to verify you own a particular address. It is assumed that you will only use your email address as the sender. That is still true today.

The rise of SPAM

In the early days, all was good with SMTP. People were behaving nicely. You could easily send and receive emails. There were mechanisms to ensure delivery, even if your mail server was offline. And system administrators could easily monitor messages for issues or undesirable content. This later feature has proven useful in dealing with the rising scourge of Email spam - Wikipedia (SPAM) as tools were quickly created to help filter out this undesirable content based on keywords.

Sometimes a legitimate email would be flagged as SPAM, so we started using whitelists to allow certain email addresses to bypass these filters. The spammers quickly worked out how to get around this: impersonate whitelisted email addresses.

Remember how I said that SMTP does not have a way to verify you own a certain address? Well, the spammers soon worked out how to exploit that feature for their benefit...

Combatting email impersonation

Now, there was a problem. Retrofitting a scheme that required verification of ownership of an email address would require significant changes to how the email system worked. It would involve not just updating the server infrastructure, but the email apps themselves. Not to mention the millions of existing email systems already out there. Instead, the internet engineers decided that a list of trusted servers and allowing the recipient system to use this as an optional verification step would be easier.

An email filtering system is like a bouncer on the door of a nightclub. They need to verify your identity and assess if you are a threat before letting you in.

Thus, was created Sender Policy Framework - Wikipedia (SPF) to allow the owners of a domain (ie, you) to create list of servers authorised to send emails on your behalf. The recipient email system can use the existing information in your email message to then look up this list and check for a match. If everything matches, it's probably not SPAM. Otherwise, it probably is SPAM.

I say probably because it's not foolproof and you may have forgotten to add that new newsletter email system into the list. The other issue is that this list resolves to an IP address that could be shared with others (bit like a shared or service office address) and they could be using that fact to trick others that the messages were coming from your office.

The rise of Phishing

Having your emails tagged as spam because someone else has gone to the efforts of sharing your email server address is not good for your brand. However, another scourge of the internet started appearing—Phishing - Wikipedia. And these messages quickly went from annoying, but mostly harmless, to downright dangerous with the capability to open a beachhead into your business that results in a Ransomware - Wikipedia attack or worse.

Imagine walking into your house and everything to opens & shuts has been padlocked. Including the toilet seat (and you really need to go). Then finding a note saying "Send money and we might give you the key" — that's ransomware.

Enter DomainKeys Identified Mail - Wikipedia (DKIM) as new set of controls that adds digital signatures to your emails to allow recipients (the Bouncer) to determine your messages are authorised and have not been tampered with. These keys are like the anti-forgery mechanisms used on government identification documents like drivers' licenses and passports.

Is this thing on...

These new security controls are good and help you protect your business reputation yet how do you know if they are working or if one of your systems is not setup properly or that your business is the target of SPAM or Phishing campaign? Also, it would be nice to have some way to ask the recipient what to do with emails that fail the SPF & DKIM tests.

Rules without consequences tend to get ignored

Enter DMARC - Wikipedia (Domain-based Message Authentication, Reporting and Conformance) a set of policies to inform the recipient (bouncer) systems what to do with failed messages and who to send a report to. There are a few other times DMARC can do, but those are the most common use cases.

What does this mean for you?

Quite simply, if you want your emails delivered, you need to make sure you have setup all the email security controls. The ones that used to be optional, but now are being enforced by Google and soon everyone else. Yay...

The basic premise is that you will need to enable email authentication on all your email systems and add some records to your Domain Name System - Wikipedia (DNS).

Your domain is the most important part of your brand and is vital for proving your identity. If you do not have control of it, you are putting your business at risk!

The first step is to compile a list of all the systems that send emails for your business. Yes, ALL of them! Whilst doing this, you may find some you no longer need.

The next step will be to work through each system to ensure email authentication (SPF, DKIM, and DMARC) have been configured for that system. Every system is different, so you will need to consult the support guides for each. You will need to add several records to your DNS, so make sure you have the credentials (username & passwords) for your domain.

If this all sounds too technical, contact your IT Support team or contact The Executive Technologists (theexectechs.com). We would love to help you be awesome and get your messages delivered. 😎

Summary

Wow, that was a bit longer than I planned. There is so much going on in the space and getting your emails delivered is becoming harder because the Bad Guys™ are increasing their efforts to mess things up.

To combat the scourge of SPAM & Phishing we need to add more protections to our email systems. The problem is that the old SMTP system we use was never designed for the threats we face today. As such we are bolting on security controls to a system never designed for it.

To make our email systems work today we need four (4) key records in our DNS;

1. MX - this tells others where to send emails (your address)
2. SPF - list of systems sending emails on our behalf
3. DKIM - keys to check the system is allowed to send emails and if the message has been tampered with
4. DMARC - what to do if the checks above fail and who to tell about it

Thanks for reading.

Andy Prosser