Why do complaint rates vary across ISPs?

Posted by George Bilbrey 

George Bilbrey
By George Bilbrey

Complaint rates are a major driver of anti-spam systems. A high complaint rate (number of “this is spam” reports relative to messages in the inbox) is known to drive bad delivery rates for commercial mailers.

One of the more interesting problems that I’ve run across at Return Path is trying to figure out why complaint rates for the same IP address (or domain) vary so widely across different ISPs. Here is a scatter chart of complaint rates for ~2,500 commercial email marketing IPs across two different ISPs.  In this graph 0.05 = 5% complaint rate (5 messages per 100 places in the inbox).

What is apparent is:

  1. Most IPs in this group have “lowish” complaint rates (<1%)
  2. There isn’t a ton of correlation outside of that range.

So what might cause the variation in complaint rates? A partial list would include:

  • Differences in webmail user interface. Some ISP make it easier than others to hit “this is spam.” Some ISPs allow users to report all messages in the junk mail folder as spam at the same time. However, this would seem to be more of a systematic difference that would cause the average complaint rates to be different at the two ISPs. I wouldn’t necessarily expect that to drive a poor correlation.
  • Users at different ISPs have a different propensity to hit the “report as spam” button for different classes of mail.
  • Senders of email messages behave differently at different receiving domains – there appears to be a few cases of this in this sample.
  • Availability and use of feedback loops at different ISPs. In our analysis of the above IP addresses, this drives a large part of the variance for these IP addresses.

What does this mean practically for mailers and designers of anti-spam systems?

For anti-spam systems, there may be some value for gathering complaint data from variety of sources. This would allow you to catch IPs that are behaving well locally (in your network) but not behaving well outside of your network.

For commercial mailers, there are a few key “so whats”:

  • Sign up for feedback loops where available. These are useful for diagnosing the cause of complaint issues.
  • Monitor complaint rates at each ISP. You can have a problem at one ISP and be doing well at another.

What do you think?

Author Image

About George Bilbrey

George Bilbrey is the founder of the industry’s first deliverability service provider, Assurance Systems, which merged with Return Path in 2003. He is a recognized expert on the subjects of email reputation and deliverability and is active in many industry organizations, including the Messaging Anti-Abuse Working Group (MAAWG) and the Online Trust Alliance (OTA). In his role as president of Return Path George is the driving force behind the ongoing innovation of our products and services. Prior to Return Path, George served as Director of Product Management at Worldprints.com and as a partner in the telecommunications group at Mercer Management Consulting. He holds a B.A. in economics from Duke University, and an MBA from the Kenan-Flagler School of Business, University of North Carolina.

Author Archive