What is Aggregate Complaint Data and How Should You Use It?
Some mailbox providers share user complaint (This Is Spam) reports with senders, in order to help them pinpoint problems with their mail program and improve their list hygiene. There are different formats for providing these reports. Some mailbox providers share with the sender the full message and user who complained, while others provide only aggregate statistics. I co-moderated a session at the recent M3AAWG in Boston, MA about the latter. Aggregate complaint reports have received a lot of attention in the industry this year with Gmail’s release of their feedback loop in February, but other organizations have been providing feedback in this format for some time. Why do mailbox providers choose to provide the data in this format, and what should senders do with it? Let’s take a look…
What is aggregate complaint data?
Abuse Reporting Format (ARF) is the standard used by ISPs to share individual user complaint reports with senders via a domain or IP based feedback loop. Some mailbox providers are not comfortable providing ARF reports and instead choose to aggregate data over a period of time to report back to the sender. They do not include the user’s email address or a copy of the message which the user complained about. The mode of offering this data, the amount of data provided, and the criteria for access vary by provider.
Who provides it and why?
Signal Spam (on behalf of Orange and SFR), Mail.ru, Microsoft (via SNDS), and Gmail (to name a few) all offer aggregated complaint data. These providers have chosen the aggregated format for a few reasons.
The first and main reason is to protect the user’s Personally Identifiable Information (PII). Since ARF provides the sender with a copy of the original message and an identifier tied to the recipient, complaints can be traced back to the user. However, with aggregate feedback, the complaint is not forwarded to the sender. Since the complaint cannot be traced back to the user who marked the message as spam, no PII is being shared with senders. This reduces the provider’s liability, and more importantly, allows them to protect their users – their top priority! This is a big factor for European providers, like Signal Spam, for whom the laws about sharing PII are far more strict than those in the US.
The second reason ties in with the first – ease of release to the market. By providing only aggregate counts of complaints, and protecting the user’s address, providers are able to share the data more easily. This was a driving factor for Gmail in their release of the FBL.
Third, some providers – like Microsoft – have chosen to offer the data as a supplement to other reputation metrics which they share with senders, in order to help them manage issues with their mail programs.
There is one final reason, which applies mainly to Gmail: noise reduction. With most traditional (ARF) feedback loops, senders receive every complaint. There can be many complaints, or just a few, so determining whether or not you have a problem with your mail program involves comparing the number of complaints to the amount of mail sent and calculating a rate. Even then, senders can be at a loss if they don’t know what complaint rate mailbox providers require for a good reputation and inbox delivery. By only sending aggregate complaint reports to FBL subscribers once they have exceeded a certain threshold, Gmail provides the sender with a clearer message. If you get a report, you have a problem.
What should marketers do with this data?
Without the user’s email address and a copy of the message, it can be difficult to take action to improve your mail program (or that of your client, if you’re an ESP). Aggregate complaint data should be used as an additional signal for detecting problems with your mail program.
Since senders are able to identify the complainer through an ARF FBL, many unsubscribe that user without doing any additional analysis of why the recipients are complaining. Use both ARF-based and aggregate FBLs to dig into the data and identify the source of the issue – why are users complaining and how can you make sure you’re are sending compelling, timely, targeted mail that users will respond well to.
Again, look at the aggregate data in combination with the other sources of data available to you as a sender: traditional (ARF) FBLs, logs, IP reputation scores, user engagement (opens and clicks). Chances are, if you have problems with your program, the aggregate complaint data won’t be the only indicator.