Internet and e-mail policy and practice
including Notes on Internet E-mail


Click the comments link on any story to see comments or add your own.

Subscribe to this blog

RSS feed

Home :: Internet

19 Apr 2014

Open Source software is the worst kind except for all of the others Internet

Heartbleed, for anyone who doesn't read the papers, is a serious bug in the popular OpenSSL security library. Its effects are particularly bad, because OpenSSL is so popular, used to implement the secure bit of https: secure web sites on many of the most popular web servers such as apache, nginx, and lighttpd.

A few people have suggested that the problem is that OpenSSL is open source, and code this important should be left to trained professionals. They're wrong. The problem is that writing and testing cryptographic software is really, really hard.

Writing and testing any sort of security software is hard, because the goals are more or less the opposite of normal software. For normal software, the main goal is to do the right thing with correct input. If it's a word processor or spreadsheet, you want it to compute and display the right results with reasonable input, but you don't much care what happens with unreasonable input. If you tell a spreadsheet to open a file full of garbage, and you get a strange screen display or the program crashes, that is at worst mildly annoying.

With security software, though, the entire value is to make sure that it rejects every incorrect input, and doesn't erroneously reveal secure material. This distinction is not one that is well understood in the computer industry. I can recall far too many reviews of desktop file encryption programs where the reviewer went on at great length about the speed of encryption and decryption, the ease of use of the various screen displays, but never bothered to check that it rejected attempts to decode data with the wrong password. Since the number of possible invalid inputs is stupendously greater than the number of valid inputs, ensuring that security software does what it is supposed to do presents a severe debugging and testing problem.

Public key (PK) cryptography, the core functions for which everyone uses OpenSSL, is doubly difficult because the programming is really tricky. All PK algorithms depend on mathematical operations which are relatively easy to do, but very difficult to reverse. A well-known example is multiplying two prime numbers to get their product vs. finding the two primes if you only know the product. All current algorithms involve arithmetic on very large numbers, much larger than any computer can handle without using special arithmetic libraries.

Even the relatively easy PK operations are still pretty slow, so the practical way to use PK is to use conventional shared key cryptography to protect the web page or mail message or whatever, and only use the PK for one end to tell the other the shared key to use. Cryptography has advanced a lot in the decade that OpenSSL has been in use, with both new PK cryptographic algorithms and shared key algorithms, so there are now about two dozen combinations of initial PK and session shared key schemes that OpenSSL has to support.

People have added extra complication to try to make things faster; one trick is to note that if you fetch a secure web page from a server, you'll probably fetch other stuff from the same server, so with the agreement of both ends they can leave the session open after the page is complete, and reuse the same session and same shared key on subsequent requests. The heartbleed bug is in "heartbeat" code that periodically checks to see if the other end of an open session is still there. (One way to fix heartbleed is just to turn off the session saving feature, in which case everything will still work, just slower since there will be more sessions to create.)

As if all this weren't complex enough, public keys by themselves are just large pseudo-random numbers, which have to be securely associated with domain names for web servers, or e-mail addresses for S/MIME mail, and there's a whole Public Key Infrastructure (PKI) in which well known entities can assert that a particular key belongs to a particular name using digital signatures, essentially PK encryption run backwards. The package of key, name, and a bunch of other stuff is known as a certificate, which is encoded using a system called ASN.1. The encodings and options in ASN.1 are so complicated that it is notoriously difficult to implement correctly, and has led to multiple security issues just from encoding errors.

Any software package that does what OpenSSL does has to do all the stuff I described above. OpenSSL has a few issues of its own. One is its history; it evolved from an earlier package in the 1990s called SSLeay which was apparently originally an experimental implementation of the large number arithmetic needed for PK cryptography. SSLeay turned into OpenSSL in 1998, so there is now close to 20 years of evolutionary cruft in the half million lines of OpenSSL code. It is written in the C programming language, which remains the lingua franca of the software community, in that no matter what language your application is written in, there's always a way for that language to connect to and use C libraries.

C grew up in the 1970s on small computers, notably 16-bit PDP-11s, where every bit of code and data space was precious, so it doesn't have much in the way of defensive programming features to detect and prevent buffer overruns and other bugs. Modern C applications can use libraries that provide much of this defensive programming, but rewriting old C code to be defensive is tedious, and doing so without introducing new bugs is hard, so people rarely do.

So all in all, it's a miracle that OpenSSL works at all, and a tribute to the skill and diligence of the handful of people who've been working on it all these years. Experience with heartbleed shows that while open source doesn't prevent bugs, it makes it very fast to fix them. It's been little more than a week since we learned about heartbleed, and surveys show that the majority of vulnerable systems are already fixed. (I fixed mine last Tuesday.)

With all these problems, why do people use OpenSSL?

  • It's free.
  • It's written in C and is fairly fast, so you can use it in all sorts of projects.
  • Its license allows commercial binary-only applications (as opposed to the GPL.)
  • It's free.

The key to the solution to OpenSSL's fragility is easy: it's MONEY, which will allow them to pay more people to work on it, and do testing and certification and improve the code and everything else that would make it more reliable. None of this is incompatible with open source, except for the fact that nearly all open source projects are starved for resources, because people's willingness to use free software vastly exceeds their willingness to send back even a little money to ensure that it remains available. (The exceptions are special cases, like the Mozilla Foundation, which gets a large revenue stream from commissions on ads shown when users do Google searches from the default home page, and the Apache Software Foundation which has a high enough profile to attract multiple sponsors at $40K and $100K/year.)

If we agree that OpenSSL is important, which it surely is if the panic about one bug in it is any indication, the question is how to persuade the people who depend on it to voluntarily pay for it. Any suggestions?

posted at: 09:45 :: permanent link to this entry :: 3 comments
posted at: 09:45 :: permanent link to this entry :: 3 comments

comments...        (Jump to the end to add your own comment)

My 49 years of experience with open and closed-source programs leads me to agree. Closed-source programs may have all manner of hidden bugs and back doors, but you only find out about the bugs that bite _YOU_ or your friends and correspondents. At least with open-source programs you have the option of looking into the code, even if you don't avail yourself of it.

(by Mike Andrews 13 Apr 2014 13:41)

It's worth pointing out two silver linings to this debacle. 1) We found it because Google is devoting time, money and energy into attempting to break key libraries. specifically includes this. The Finnish company that found it may have looked because of this and Google itself has been fuzzing like crazy (see also the 1K FFmpeg bugs they found). 2) OpenSSL has received more donations in the last week than a typical year 3) There's now going to be a whole lot more work we will try and do to prevent this. Ideas already include changing the malloc behaviour in OpenSSL, but some other possibilities could include IDS detection of general badly behaved protocols. On the down-side, it's also blatantly obvious we can't manage this kind of issue. A multitude of sites inside the Alexa 1000 were vulnerable for some time after. Government response was late. Advice was confusing. Responsible disclosure helped CloudFlare and Akamai. It's not obvious how that list was determined.

(by Philip Whitehouse 13 Apr 2014 17:50)

I gather that two of the major openssl developers work for Google but spend all their time on SSL. That's great, but it would be nice if the entire Internet weren't dependent on Google's enlightened self-interest.

(by John L 13 Apr 2014 18:02)

Add your comment...

Note: all comments require an email address to send a confirmation to verify that it was posted by a person and not a spambot. The comment won't be visible until you click the link in the confirmation. Unless you check the box below, which almost nobody does, your email won't be displayed, and I won't use it for other purposes.

Email: you@wherever (required, for confirmation)
Title: (optional)
Show my Email address
Save my Name and Email for next time


My other sites

Who is this guy?

Airline ticket info

Taughannock Networks

Other blogs

Remembering JD Falk - 10 years later
223 days ago

A keen grasp of the obvious
New Hope for the Dead
465 days ago

Related sites

Coalition Against Unsolicited Commercial E-mail

Network Abuse Clearinghouse

© 2005-2020 John R. Levine.
CAN SPAM address harvesting notice: the operator of this website will not give, sell, or otherwise transfer addresses maintained by this website to any other party for the purposes of initiating, or enabling others to initiate, electronic mail messages.