The Importance of Reputation Scoring for Deliverability
You may have recently read the blog post from Oracle Marketing Cloud about the demise of the deliverability reputation score. There seem to be several misconceptions and misunderstandings about reputation and reputation scores that any organization sending emails should know, or risk the demise of their email marketing program.
Why a sender reputation is important to combat spam
In the 1990s, email spam was filtered from users’ inboxes primarily using Bayesian content filters. Over time, spam filters learned keywords most associated with spam and would filter or block emails that contained these keywords. You probably can still find some experts recommending senders avoid things like the word “free” or avoid words in all caps. Spammers found these spam rules easy to bypass.
In the early 2000s, anti-spam filters pursued a better way to identify spam from permission-based email marketing messages. By creating a profile of what a spammer looks like based on sending and user behaviors, mailbox providers and anti-spam solution providers were able to do a much better job of keeping spam out of their users’ inboxes.
This also provided a more predictable framework for senders on what to do, and what not to do to get their emails delivered to their users’ inboxes. This concept is known as sender reputation. Sender reputation is still used to filter spam today with much success. It’s a rarity to hear people say they have too much spam in their inboxes today, because most of it is filtered out of the inbox and never seen by the recipient. On the other hand, organizations worldwide still see about 1 in 5 messages delivered to the spam folder or outright blocked. As a result, many organizations sending emails pay close attention to the sending reputations of their IP addresses and sending domains.
The sender reputation score
For non-spammers, discovering the reputation reasons as to why an email message was blocked or filtered was difficult, confusing, and sometimes impossible. With hundreds of mailbox providers around the world, many used their reputation metrics to determine deliverability, while other organizations used third party filters that relied on their reputation metrics. Organizations became frustrated due to the lack of transparency of the reasons surrounding filtering issues and the industry responded by creating sender reputation scores. Sender reputations are calculated by looking at hundreds of signals. Here are a just few of the most important:
- Sending history: How long a sender has been sending from an IP address and domain and at what volume levels. New traffic on an IP address is considered suspicious, for example, and will see more deliverability issues until a trustworthy reputation has been established. Unusual spikes in volume are also suspicious.
- User-based spam complaints: Spam complaints from users provide very strong signals of what is spam.
- User-based “not spam” feedback: Opposite of the “report spam” option, the “not spam” option also provides a very strong signal that messages are not spam and should be delivered to the inbox.
- Unknown user rate: The number of attempted deliveries to email addresses that no longer exist.
- Spam traps: “Honeypot” addresses used to catch spammers.
- Mailing infrastructure: Spammers don’t care about setting up a well-configured mail server and often hijack PCs to send spam. Reputation systems look at things like rDNS, authentication (SPF, DKIM, DMARC), abuse@ address, up-to-date WHOIS record, etc.
Here are a few of the sender reputation score services available to marketers today:
Outlook.com Smart Network Data Services: Microsoft is clear on their postmaster page that “deliverability to Outlook.com is based on your reputation.” To help organizations troubleshoot deliverability issues, Microsoft provides Sender Network Reputation Data Services to give you “data you need to understand and improve your reputation at Outlook.com.” The reputation scores within SNDS via the “filter result” section are color-coded Green, Yellow, and Red based on filtering results.
Postmaster Tools by Gmail: Gmail recently launched their postmaster site to help organizations understand why their emails were sent to spam. Gmail’s reputation scores are broken out by both domain and IP address. It assigns a reputation score of Bad, Low, Medium, or High.
AOL Postmaster: Senders can enter their sending IP address to see their reputation score at AOL. AOL bases their reputation score on “a holistic view of your IP, and takes into account a wide variety of factors including—but not limited to—spam complaints, this-is-not-spam reports, spam folder deliveries, and invalid recipients.”
TrustedSource by McAfee: Provides a reputation score based on risk level—minimal risk to high risk.
Senderbase by Cisco: Provides a reputation score of Good, Neutral, or Poor. Senderbase provides additional data on email volume and volume history.
ReputationAuthority by WatchGuard: ReputationAuthority provides an overall reputation score on a scale between Good and Bad. It also provides the ratio of messages seen that were clean, malformed, or suspicious. Additionally, it provides a ratio of good and bad recipients.
Sender Score by Return Path: Sender Score is a free service offered by Return Path that scores IP addresses from 0 – 100 based on data from participating mailbox providers like blacklists, complaints, infrastructure, blacklists, spam folder rate, rejected rate, spam traps, and unknown users.
Sender reputation obsolete? Hardly.
Oracle’s recent blog post made a lot of strong statements about sender reputation scores, and specifically Sender Score. To avoid any businesses or organizations making mistakes based on bad advice, here’s what you need to know about reputation scores and Sender Score.
Myth #1: Neither Gmail or Hotmail would stop delivering email to a highly engaged audience.
False. There are a number of reasons that Gmail or Outlook.com would stop delivering email to a highly engaged audience. Here are just a few examples:
- No sending history: Gmail or Outlook.com would stop delivering to an engaged audience if the sender started mailing from a new IP address or domain. With no sending history, mailbox providers don’t know who you are, and reputation scores can be an important way to track reputation status when establishing a new IP address.
- Spam traps: Mailing to a high number of spam traps either acquired through list poisoning, accidentally mailing to a wrong list, or by renting lists. There are many reasons why a legitimate organization may send to these spam trap addresses. Gmail and Outlook.com would stop delivering mail from that IP address, despite high engagement, to prevent further spam abuse from that IP address until the IP address owner can resolve the problem. A reputation score like Sender Score, or any of the ones listed above, are a good way to evaluate how close you are to resolving spam trap issues.
- High complaint rates: Outlook.com uses two measures of subscriber displeasure with a sender in addition to low engagement: 1) User-based spam reports (as mentioned above), and 2) Feedback from a panel of subscribers that are asked if a message is spam or not (sometimes called Sender Reputation Data). These two factors explain 60 – 70% of delivery problems.
Myth #2: No mailbox provider uses Sender Score to determine delivery of email from a sender.
False. Comcast uses Sender Score to rate limit incoming traffic. The higher your Sender Score, the more recipients you can deliver to per hour. Road Runner uses Sender Score in a similar fashion according to their postmaster page. Other mailbox providers also reference Sender Score but have a practice of not publicly providing details about their filtering process.
Myth #3: The score doesn’t reflect the reality of deliverability success or failure.
False. Sender Score and the other reputation scoring services use actual data that determines deliverability success and failure. It’s more accurate to say deliverability at individual mailbox providers will vary. However, Sender Score uses global data aggregated from hundreds of mailbox providers and provides strong directional correlation (r=.91) on your email’s likelihood of inbox deliverability. For Gmail and Yahoo!, the correlation is also strong (r=.60), but slightly less than the other mailbox providers. Why? Engagement informs some filtering decisions at these mailbox providers, and engagement isn’t currently a variable included in Sender Score, although these metrics are available elsewhere for Return Path customers. Additionally, there are some issues with content and server configurations that cause deliverability problems not reflected in Sender Score.
Sender Score shouldn’t be used to determine inbox placement rate, nor should it be a replacement for monitoring your deliverability of each campaign. Sender Score is intended to show the trustworthiness of an IP address over a rolling 30-day window. The lower a Sender Score, for example, the more likely it will be to see deliverability problems. For senders looking for why a specific campaign wasn’t delivered to the inbox, Return Path offers Inbox Optimizer, which can determine if a campaign was delivered due to lack of engagement, or some other reputation issue.
Myth #4: The scoring only uses a couple of “secret” sources, which aren’t relevant.
False. Sender Score uses hundreds of mailbox provider and security sources. These worldwide mailbox providers provide global data that powers accurate scores for global and non-US senders. The sources are kept secret for valid reasons. Since Sender Score is used to determine placement at some mailbox providers (see above), disclosing the sources would allow senders to selectively clean lists, reducing Sender Score’s effectiveness for filtering email.
Myth #5: There is no explanation of what moves the score up or down.
False. The Sender Score FAQ site lists all reasons scores move up or down. Additionally, if that’s not enough detail, Return Path provides greater details of what moves the score up and down in its Reputation Monitor product.
Myth #6: Reputation and reputation scores are obsolete.
False. Mailbox providers aren’t decreasing their use of reputation metrics and scoring systems, like Sender Score. In fact, they are investing in them more heavily. Many mailbox providers are investing in domain reputation systems to increase the granularity of their reputation systems. Reputation-based filtering is here to stay.
Myth #7: Sending relevant messages to people who asked to receive them is the way to the inbox.
True and False. I completely agree that sending the right message to the right person is a cornerstone of marketing. But as described above, there’s more to deliverability than just engagement. The advice of “just send relevant content” makes for a good line in a blog post, but isn’t helpful when trying to solve deliverability issues—especially when you’re having issues right now.
Where to get reputation data
It’s understandable that troubleshooting issues around reputation is difficult, especially if your email service provider doesn’t have the right data. If you’re a customer of any email service provider that lacks Sender Score reputation data, please contact us to see how you can gain access to reputation data that you were used to having in Reputation Monitor. If you’re looking for a new email service provider that can help with deliverability and reputation issues, visit our partner page.
About Tom Sather
Email data and deliverability expert Tom Sather has worked with top-tier brands to diagnose and solve inbox placement and sender reputation issues as a strategic consultant with Return Path. As the company’s senior director of research, Tom is a frequent speaker and writer on email marketing trends and technology. His most recent analysis of new inbox applications’ effects on consumer behavior was widely cited across leading business media outlets including the Financial Times, Ad Age, and Media Post.