Internet and e-mail policy and practice
including Notes on Internet E-mail


Click the comments link on any story to see comments or add your own.

Subscribe to this blog

RSS feed

Home :: Email

23 Dec 2013

The naive arrogance of FUSSPs Email

Everyone who's been in the e-mail biz long enough knows the term FUSSP, Final Ultimate Solution to the Spam Problem, as described in a checklist from Vern Schryver and a form response that's been floating around the net for a decade.

FUSSPs fall into two general categories, bad ideas that won't go away, and reasonable ideas that are oversold. The classic bad idea is e-postage, charging for e-mail. I wrote a white paper about it a decade ago, and nothing of importance has changed.

Reasonable ideas that are oversold are much more frustrating, and there are a lot of them in the e-mail authentication field. One of the bad ideas is that if we just knew who was sending every mail message, we'd have no spam, which is silly for many reasons.

It's not at all silly in weaker versions. There are plenty of partial authentication schemes that let us identify interesting parts of our mail stream so we can whitelist mail from some senders we know. Attempts to give an identity to each user, PGP and S/MIME, have never been widely adopted because it's too hard to distribute the keys into each user's mail program. Less ambitious schemes where the granularity is the domain name are quite workable for parts of the mail stream, but suffer when they're oversold.

A sure fire way to tell when an authentication scheme is turning into a FUSSP is when naive enthusiasts demand that other parts of the mail system that work perfectly well have to change what they do to deal with limitations of the authentication scheme. Invariably, the mail system keeps doing what it's always done, and people learn to live with the limitations of the authentication scheme, but teaching the enthusiasts to curb their enthusiasm is very frustrating.

About a decade ago, the new authentication scheme was SPF. For some kinds of mail, particularly bulk mail all sent from a known set of servers, it works quite well. For others, such as mail systems with human users who send mail in a variety of hard to predict ways, it works much less well. SPF has ways to deal with this limitation, by allowing various intermediate results between pass and fail, but by golly the enthusiasts wanted to fix the e-mail world so that all legitimate mail would pass. (Presumably they would reject all mail that didn't pass, and solve the spam problem.) To try to force the issue, some of them had their mail systems reject any mail that didn't pass, thereby losing lots of perfectly ordinary mail that didn't happen to be sent in a way that SPF can describe. Add-ons to SPF with names like SES were supposed to let the rest of us "upgrade" our mail systems so that forwarded mail, a particular sore point, could be SPF compliant, but in practice what happened was that people adjusted their SPF to avoid needless damage and the mail system works the same as always.

The next round was with DKIM. While SPF tries to validate the path a message takes, DKIM puts a cryptographic signature in the message. That doesn't depend on the path the message takes, so forwarding isn't usually a problem. For DKIM, the FUSSP issue arrives with mailing lists. Most mailing lists make small changes to the messages for the convenience of the list subscribers, such as adding a list tag to the subject line, or a message footer that says what the list is and how to subscribe or unsubscribe. If the incoming message has a DKIM signature, these changes invalidate it. The mail system running the list software can add its own DKIM signature, recipients use that signature to recognize list mail, and everything works.

Some DKIM enthusiasts insist that mailing lists need to preserve DKIM signatures from incoming mail by turning off the convenience features that break signatures, even though that would solve no problem whatsoever. (I can't wait to see the comments on that one.) List operators, of course, have ignored them, since the subject tags and message footers are useful, and lists work just fine as is.

This year's authentication hack is DMARC. It sits on top of DKIM and SPF and allows senders to describe their sending policies, and offer advice to quarantine or reject mail that doesn't match the policy. For domains like Paypal that are heavily forged and send all their mail from a small set of known hosts, DMARC is very useful to tell people how to detect forged mail. For domains like mine, who are forged only by the occasional random spammer who picks fake return addresses out of the spam lists, and whose mail goes through forwarders, mailing lists, newspaper send-to-a-friend, and so forth, DMARC has some interesting ways to gather statistics, but the policy parts are not useful.

Predictably, there are DMARC enthusiasts who want every message ever sent by anyone to have a "reject" policy, which will solve the spam problem by rejecting all spam. (Perhaps they are too young to remember the past two iterations.) This again brings us back to mailing lists. DMARC policy assertions can't describe list mail for the same reasons that list mail fails DKIM, and it doesn't matter for the same reason, receivers can recognize mail from lists their users subscribe to. But DMARC adds an extra level of excitement: overzealous use of DMARC by list contributors can make receivers reject valid list mail, which list software interprets as bad recipient addresses, and removes entirely valid subscribers from the list.

You will probably not be surprised to hear that the enthusiasts blame the problem on the mailing lists, which should turn off features that break DKIM signatures, or need to be rewritten with complicated heuristics to recognize and ignore DMARC bounces, or something, rather than the actual problem, which is that they're publishing the wrong policy. List software will of course not change what it does; at most it will recognize and reject incoming mail from subscribers that have strict DMARC policies. There are already patches to GNU mailman to do this, and I've found I can adjust my majordomo2 lists do to so with a one line configuration change.

I suppose it's nice to know that we're still getting new enthusiasts in the anti-spam biz, but it sure would be nice if they'd read the history before making the same mistakes over again.

posted at: 00:23 :: permanent link to this entry :: 4 comments
posted at: 00:23 :: permanent link to this entry :: 4 comments

comments...        (Jump to the end to add your own comment)

Apart from all the things you mention, my personal pet peeve when it comes to FUSSP is that they tend to ignore the fact that just about everyone uses some kind of spam filter. And that they tend to do a pretty good job. All FUSSPs I have seen break more than that they fix the small improvements that can be made in spam filters.

(by Martijn 23 Dec 2013 06:50)

Some of us who disagree with you are rational people who aren't "new enthusiasts" in the email space by this point. My hair may not be as grey as some, but I didn't just show up yesterday. Turning to insults in your frustration is not something that helps to move the conversation forward, nor is it kind or fair.

(by Al 02 Jan 2014 15:42)

industry observer, and serial napper
Email authentication schemes are reputational data points, that help receivers make decisions about messaging to determine if it is spam. It isn't, in and of itself, a binary upon which such determination can be made.

The 'it doesn't work for discussion list mail' point is valid, although at this point in time an edge-case, i'd hazard so tiny in amount of actual traffic as to be irrelevant as to the efficacy of authentication.

SPF is great, except when it is broken in implementation, which, from my experience monitoring hundreds of large senders, is often.

I've little experience with DKIM so I'll refrain from comment as to the current quality of deployment.

DMARC is fine and wonderful except it has now been co-opted to 'solve phishing and brand abuse' which is laughable, because most phishing and brand abuse, up to 90% of it, perhaps more, is done by way of look-alike 'cousin' domains. It cures, possibly, part of those problems when miscreants use the actual domain of the brand they are screwing with.

For me the bottom line is all of these schemes should be deployed by all legitimate senders of email, because they are good practices and add to transparency and a sense of assuredness for the messaging, but any expectation of them helping solve spam should be curbed in the face of reality at the receiving side.

(by Neil 02 Jan 2014 16:21)

Looseness brings no assuredness
What is that neural mechanism which, when someone answers the door phone asking who is there, makes us reply "It's me" --me who?

Similarly, the From: field, like caller ID and fax own number, is designed to be easily spoofable. It is up to the sender to set From: correctly, otherwise replies won't work. Replying to spam is bad anyway, so it doesn't hurt if it has a fake sender address. Yes, spam is annoying, but so are TV ads. Depriving spammees of the ability to complain helps keeping the rationale of marketing well hidden.

One may think that, given that we take the time to secure our systems, identify senders on submission, and deploy a bunch of crypto stuff to authenticate messages, once From: is recognized as a key field, we'd go ahead and secure it. Well, nope. In fact, there is a tradition to implement mailing lists as a continuation in email transit, although modifying the message while doing so is a protocol violation. Mailing lists are a small niche, compared to the rest of email traffic, but that's not a good reason to change them, especially since we just said securing From: is useless.

(by Ale 07 Oct 2014 07:45)

Add your comment...

Note: all comments require an email address to send a confirmation to verify that it was posted by a person and not a spambot. The comment won't be visible until you click the link in the confirmation. Unless you check the box below, which almost nobody does, your email won't be displayed, and I won't use it for other purposes.

Email: you@wherever (required, for confirmation)
Title: (optional)
Show my Email address
Save my Name and Email for next time


My other sites

Who is this guy?

Airline ticket info

Taughannock Networks

Other blogs

Remembering JD Falk - 10 years later
222 days ago

A keen grasp of the obvious
New Hope for the Dead
464 days ago

Related sites

Coalition Against Unsolicited Commercial E-mail

Network Abuse Clearinghouse

© 2005-2020 John R. Levine.
CAN SPAM address harvesting notice: the operator of this website will not give, sell, or otherwise transfer addresses maintained by this website to any other party for the purposes of initiating, or enabling others to initiate, electronic mail messages.