Click the comments link on any story to see comments or add your own.
Subscribe to this blog
30 Sep 2016
Unsubscribing from mailing lists is hard. How many times have you seen a message "please remove me from this list," followed by two or three more pointing out that the instructions are in the footer of every message, followed by three or four more asking people to not send their replies to the whole list (all sent to the whole list, of course,) perhaps with a final message by the list manager saying she's dealt with it?
For marketing broadcast lists, it's even worse because there's no list to write to. Messages are supposed to have an unsubscribe link (required by law in most places) which usually works except when it doesn't, or it leads to a web page making incomprehensible demands ("click here unless you want not to be removed only from this sender's mail") so for a lot of users it's easier just to click the junk button until the messages go away.
Mail system managers know that users aren't very good at unsubscribing, so they've invented some ad-hoc ways of dealing with it. Many large mail systems have feedback loops (FBLs) which let mail senders register their ranges of IP addresses or in Yahoo's case DKIM signatures, so the sender or perhaps sender's network gets a report when a recipient marks a message as junk. When the sender is a bulk mailer, they generally try to handle the report as an unsubscribe request.
While FBLs are great for finding when an ISP customer is compromised and starts spamming, they're not so great as a substitute for unsubscriptions. One reason is that even though there's a standard format called ARF (RFC 5965) for sending FBL reports, each mail system includes slightly different details, so the original mail sender needs to try and parse out enough from the report to identify the list and the subscriber. Many mail systems redact their ARF reports on advice of their lawyers, and the redaction is often so severe that it can be impossible to tell who to unsubscribe from what. AOL's reports are so redacted that the only way I can figure out who to unsubscribe is to take the transaction ID in a Received: header of the reported message and manually match it up with my outgoing mail logs. And Gmail doesn't provide individual FBL reports at all, only aggregate data.
The obvious solution to this problem is the
The problem with the click is that a lot of anti-spam systems automatically follow all the URLs in the message to see if they lead to malicious sites, and there's no way for the target of the URL to mechanically tell a request from a spam filter from a click by a live user. It's quite reasonable for spam filters to do this: Imagine a bad guy sending deliberately uninteresting spam with a fake unsubscribe link leading to his malware site.
As a result, the unsubscribe link usually leads to a web page with a confirmation button that the malware checkers won't click but a live person will. The confirmation page may also ask what address to remove. While there have been attempts to parse the web pages and figure out what to fill out and what to click next, they don't work very well since the confirmation buttons vary all over the place. Unsubscribing by mail works at small scale, but operators of large mail systems like Gmail and Yahoo have told me that they are so big compared to most other mail systems that what seems to them like a moderate amount of automated mail can easily overwhelm recipient systems.
To solve this problem, a few people at Gmail, AOL, Optivo (the bulk e-mail part of the German post office) and I have come up with an automatic one-click unsubscribe scheme. The goal is to allow automatic unsubscribes as an option for the junk button -- when the user clicks junk, a little window asks whether to unsubscribe too.
One-click unsubscribe uses an https POST action rather than the simpler GET. POST is intended for actions that change something, as opposed to GET which is just supposed to retrieve data. Anti-spam and malware checkers do GETs, not POSTs. (We know not everyone follows these rules, but they're how the web is supposed to work and usually does.)
We've defined a new message header
This one-click design avoids the redaction issue, since the user asked for the unsubscription, and the request goes directly to the link in the message, not an address intuited from IP addresses or DKIM signatures. The point of the FBL ARF redaction is in case the intuiting guessed wrong and the message went back to someone other than the sender, but there's no guessing here.
One-click should be useful in some other situations, too, notably when a mailbox has been closed or abandoned, so the recipient system wants to unsubscribe it from everything. Several large mail systems have said they plan to implement one-click as part of their junk buttons, so with any luck, it'll soon be helping senders send less mail the recipients don't want.
The current draft spec is here.
My other sites
© 2005-2018 John R. Levine.
CAN SPAM address harvesting notice: the operator of this website will not give, sell, or otherwise transfer addresses maintained by this website to any other party for the purposes of initiating, or enabling others to initiate, electronic mail messages.