Internet and e-mail policy and practice
including Notes on Internet E-mail


2014
Months
Apr

Click the comments link on any story to see comments or add your own.


Subscribe to this blog


RSS feed


Home :: Email


07 Apr 2014

Yahoo addresses a security problem by breaking every mailing list in the world Email

DMARC is what one might call an emerging e-mail security scheme. It's emerging pretty fast, since many of the largest mail systems in the world have already implemented it, including Gmail, Hotmail/MSN/Outlook, Comcast, and Yahoo.

DMARC lets a domain owner make assertions about mail that has their domain in the address on the From: line. It lets the owner assert that mail will have a DKIM signature with the same domain, or an envelope return (bounce) address in the same domain that will pass SPF validation. The domain owner can also offer policy advice about what to do with mail that doesn't have matching DKIM or SPF, ranging from nothing to reject the mail in the SMTP session. The assertions are in the DNS, in a TXT record at _dmarc.domain. You can see mine at _dmarc.taugh.com.

For a lot of mail, notably bulk mail sent by companies, DMARC works great. For other kinds of mail it works less great, because like every mail security system, it has an implicit model of the way mail is delivered that is similar but not identical to the way mail is actually delivered.

Mailing lists are a particular weak spot for DMARC. Lists invarably use their own bounce address in their own domain so they can collect the error reports from list mail, so the SPF doesn't match. Lists generally modify messages via subject tags, body footers, attachment stripping, and other useful features that break the DKIM signature. So on even the most legitimate list mail, most of the mail fails the DMARC assertions, not due to the lists doing anything "wrong".

The reason this matters is that over the weekend Yahoo published a DMARC record with a policy saying to reject all yahoo.com mail that fails DMARC. I noticed this because I got a blizzard of bounces from my church mailing list, when a subscriber sent a message from her yahoo.com account, and the list got a whole bunch of rejections from gmail, Hotmail, Comcast, and Yahoo itself. This is definitely a DMARC problem, the bounces say so.

The problem for mailing lists isn't limited to the Yahoo subscribers. Since Yahoo mail provokes bounces from lots of other mail systems, innocent subscribers at Gmail, Hotmail, etc. not only won't get Yahoo subscribers' messages, but all those bounces are likely to bounce them off the lists. A few years back we had a similar problem due to an overstrict implementation of DKIM ADSP, but in this case, DMARC is doing what Yahoo is telling it to do.

The DMARC mailing list issue has been argued at length among us nerds, and one of the counter arguments has been that mail systems know who is sending mailing list mail, so they will whitelist those lists or otherwise avoid the problem. We now know that is not true. I've been running lists for years, no spam at all (they're all noncommercial stuff like my church, CAUCE's newsletter, and a group of folk dancers), and every possible technical feature including DKIM, SPF, correct forward and reverse DNS, you name it. As noted above it didn't help, and I have heard from many other list managers with the same problem, thanking me for explaining what happened.

I understand, from the always interesting Word to the Wise blog, that Yahoo has severe phishing problems, with crooks sending mail to Yahoo users, pretending to be yahoo.com administrators. Yahoo chased the crooks off their own servers, so now the crooks are (as I understand it) sending mail to Yahoo from the outside, pretending to be Yahoo. While I sympathize with their problems, and this is not exactly swatting a fly with a sledgehammer, it's a nail that needs a regular hammer, and the sledgehammer is demolishing the surrounding plaster every time it whacks the nail. Concretely, Yahoo should be able to figure out ways to reject non-Yahoo mail going into their own servers without abusing DMARC to screw up everyone else.

I hope they get their act together, but in the meantime here are some suggestions for people who run mailing lists or other mail software that might legitimately pass on a yahoo.com message:

  • Suspend posting permission of all yahoo.com addresses on any mailing lists you run, to limit the damage.
  • Tell Yahoo users to get a new mail account somewhere else, pronto, if they want to continue using mailing lists.
  • If you have source code for your list software, as a band-aid, see if you can add a hack to check for yahoo.com From: addresses and change them to something like "Address redacted", which will avoid triggering DMARC. I did that on my lists.
  • If you know people at Yahoo, ask if perhaps this wasn't such a good idea.

posted at: 21:21 :: permanent link to this entry :: 9 comments
posted at: 21:21 :: permanent link to this entry :: 9 comments

comments...        (Jump to the end to add your own comment)

Yahoo
Just so happened I got quite a few of the same emails this morning claiming I needed to update my Apple ID. 7 of the same emails stacked on top of one another caught my attention. Yesterday it was to update my twitter email. I don't have a Twitter acct. or anything Apple associated with that email. After doing a little common sense snooping it was obvious they were phishing. The reason I ended up here reading this article. Ranked high in the search results.

(by Drew F 08 Apr 2014 10:03)



Why is "every mailing list in the world" broken in the first place by expecting to be able to send "MAIL FROM" a domain that they don't operate? Mailing lists already are DMARC compliant when they send Digests to subscribers.

Why are all of the spam and phish fighting RFCs made ineffective in one way or another in implementation due to the poor "best practice" policies put forth by mailing lists? Why can't mailing list software be updated to support taking direct responsibility for forwarding mail on behalf of the members of the email on the mailing list?

(by Steven Bytnar 08 Apr 2014 10:08)


Mailing lists work fine
Mailing lists have done what they do, including keeping the author's address in the From: line where it belongs for at least 30 years. Anyone who finds that surprising hasn't been paying attention.

My mailing lists take full direct responsibility for the mail they send by adding valid DKIM signatures on every message with the list's domain, to make them easy to recognize.

(by John L 08 Apr 2014 10:47)



Sorry. If I replaced "MAIL FROM" with RFC5322:From, doesn't the argument still stand? Does it really make mailing lists less useful or break how mailing lists operate if they were to always rewrite RFC5322:From as From: "ListName on behalf of " ? Mailing list servers already do it for "Digest" mode subscribers. Can we make DMARC more valuable by changing mailing list best practices instead of limiting DMARCs deployment/usefulness (p=reject) based on how mailing lists want to operate? Do we need to define another protocol to achieve what Yahoo wants to achieve with their deployment of DMARC p=reject?

(by Steven Bytnar 08 Apr 2014 12:53)


Recommended error code?
What SMTP error code do you think for rejecting yahoo senders? I am thinking 5xx so that it's a hard bounce and the sender gets immediate notice. I am thinking something like

"550 5.7.1 yahoo senders not allowed due to mailing list incompatible DMARC policy, see http://jrl.guru/Email/yahoobomb.html"

(by Matt Taggart 08 Apr 2014 15:44)



This is a show-stopper, the worst problem I've encountered in my 20 years of using email. I'm not enough of a nerd to understand all this, but I think this is mainly yahoo's problem, not riseup's, and yahoo should be flooded with requests to find a way to block spam without blocking list serves.

(by Kim McCoy 16 Apr 2014 07:33)


trouble in sending my e-mails
MY CHURCH FRIENDS ENJOY THE BLESSINGS AND THINGS I SEND THEM FROM MISTYLIGHTHOUSE WEBSITE BUT FOR WEEKS NOW ALL MY E-MALIS HAVE BEEN BLOCKED. I HAVE BEEN USING YAHOO MAIL FOR A NUMBER OF YEARS AND I DON'T APPRECIETE ALL MY CHRISTIAN MAIL BEING BLOCKED

(by alma lee 29 Apr 2014 09:46)


Grizzled mailserver op
Can anyone share a good bounce hack for GNU Mailman 2.1.12?

(by Mark Turner 20 May 2014 22:44)


yahoo customer service phone
Customers using "free" services do give Yahoo something. Yahoo is raking in money from advertisers who believe we (the customers) are worth paying Yahoo for, so they can hawk their goods.

(by Alexen 22 Sep 2014 17:34)


Add your comment...

Note: all comments require an email address to send a confirmation to verify that it was posted by a person and not a spambot. The comment won't be visible until you click the link in the confirmation. Unless you check the box below, which almost nobody does, your email won't be displayed, and I won't use it for other purposes.

 
Name:
Email: you@wherever (required, for confirmation)
Title: (optional)
Comments:
Show my Email address
Save my Name and Email for next time

Topics


My other sites

Who is this guy?

Airline ticket info

Taughannock Networks

Other blogs

CAUCE
CSA recap: CAUCE discusses international email and security
80 days ago

A keen grasp of the obvious
My high security debit card
210 days ago

Related sites

Coalition Against Unsolicited Commercial E-mail

Network Abuse Clearinghouse



© 2005-2018 John R. Levine.
CAN SPAM address harvesting notice: the operator of this website will not give, sell, or otherwise transfer addresses maintained by this website to any other party for the purposes of initiating, or enabling others to initiate, electronic mail messages.