Archive for the ‘Policy’ Category

Lastpass, Risk, and Security Expectations

Wednesday, March 29th, 2017

Last week was a rough week for LastPass. In his continuing work of scrutinizing security products in general, and recently password managers in particular, Tavis Ormandy has released a series of critical bugs against LastPass (Tweet, Writeups). They’re exactly the sort of nightmare scenario that scares the crap out of users adopting password managers, and especially cloud-connected ones. However, it’s too easy and too simplistic to look at this week and conclude anything sweeping about password managers.

Don’t Lose Sight of the Big Picture

Password managers are a hugely important measure for most people, and something that all information security professionals should be advocating for. It’s not necessarily an easy sell, so it really pisses me off when I see self-professed experts muddying the waters with security absolutism (phrases like “well a hacker could still do…”, or advocating pet approaches that are infeasible for most people). When a vulnerability like this comes around, it provides another opportunity for those sorts of folks to shout and snark and win some personal points, at the expense of laymen who don’t have the ability to put these histrionics in context. So, even if these issues were a death knell for LastPass (I don’t believe they are, but more on that below), then password managers would *still* be a good idea for nearly everyone.

One important-but-often-overlooked benefit of password managers is that it conditions users to emphasize specific passwords less and be more comfortable with the abstract idea of authenticating to a service using some intermediary. That forms one part of the bridge (along with the spread of federated auth and mobile push authentication, among others) toward a post-password future.  

So, 3 bad vulns in LastPass, but the concept of password managers is still a net positive for security. How do we reconcile those?

Let’s put two points together:

  • All software has bugs, and (essentially) all important software has had critical vulns at some point (MS08-067, Stagefright, Heartbleed, Dirty CoW on Linux, iOS Trident, and various IE vulns)
  • Risk is more than just the count or severity of known vulns

I would argue that although the severity of the vulnerabilities were “critical”, the actual risk was relatively low for most users. By the time most people became aware of the issues, they had already been fixed or patched and effective for anyone who was online to receive the update. I don’t have metrics on how long it too the patch to be actually deployed to users, but my browser picked it up before I even got finished reading the published disclosure. For highly-targeted users, that might be enough time to put together an attack, but for the vast majority of the population they had a fix before they needed to care. A non-browser based manager is an option that avoids some of these issues, but I rarely meet a person using one who’s not in infosec already.

For the rest of users, they continued to realize all of the risk-reduction benefits of a password manager (in addition to convenience, etc), and never actually realized significant risk from the vulnerability.

Planning for Failure

There’s a longer blogpost here about the significant attack surface of a browser, defense in depth, and security configuration decisions around that, but that’ll have to be another time.

At the moment, I just want to point out LastPass’s ability to respond to a bug submission, triage it, then develop and deploy and appropriate fix in in time to limit user impact is not luck or accident; I would argue that it speaks to internal engineering values, and that it’s table stakes for a modern software shop, *especially* in security software. Even while he was hammering on them, Tavis pointed out that their responsiveness is a better experience than he’s used to having with vendors. One of the most controversial points in my BlackHat talk from a few years ago was congratulating WordPress on their approach to automatic updates; while it’s easy to dump on the project as a poster child for security vulns, in practice their effort in automatic updates actually do more to keep their users safe than some other projects with fewer bugs.

Final Thoughts

They got beat up pretty bad, but I’ll continue to use LastPass. We don’t often get to see in a very public way how a company handles a security issue, but when the response shows us that they’re thinking and doing the right things both before and during (Lastpass 2015, Kickstarter 2014), then it helps with the decision about whether to continue using them.

What Kickstarter Did Right

Monday, February 17th, 2014

Only a few details have emerged about the recent breach at Kickstarter, but it appears that this one will be a case study in doing things right both before and after the breach.

What Kickstarter has done right:

  • Timely notification
  • Clear messaging
  • Limited sensitive data retention
  • Proper password handling

Timely notification

The hours and days after a breach is discovered are incredibly hectic, and there will be powerful voices both attempting to delay public announcement and attempting to rush it. When users’ information may be at risk beyond the immediate breach, organizations should strive to make an announcement as soon as it will do more good than harm. An initial public announcement doesn’t have to have all the answers, it just needs to be able to give users an idea of how they are affected, and what they can do about it. While it may be tempting to wait for full details, an organization that shows transparency in the early stages of a developing story is going to have more credibility as it goes on.

Clear messaging

Kickstarter explained in clear terms what was and was not affected, and gave straightforward actions for users to follow as a result. The logging and access control groundwork for making these strong, clear statements at the time of a breach needs to be laid far in advance and thoroughly tested. Live penetration testing exercises with detailed post mortems can help companies decide if their systems will be able to capture this critical data.

Limited sensitive data retention

One of the first questions in any breach is “what did they get?”, and data handling policies in place before a breach are going to have a huge impact on the answer. Thinking far in advance about how we would like to be able to answer that question can be a driver for getting those policies in place. Kickstarter reported that they do not store full credit card numbers, a choice that is certainly saving them some headaches right now. Not all businesses have quite that luxury, but thinking in general about how to reduce the retention of sensitive data that’s not actively used can reduce costs in protecting it and chances of exposure over the long term.

Proper password handling (mostly)

Kickstarter appears to have done a pretty good job in handling user passwords, though not perfect. Password reuse across different websites continues to be one of the most significant threats to users, and a breach like this can often lead to ripple effects against users if attackers are able to obtain account passwords.

In order to protect against this, user passwords should always be stored in a hashed form, a representation that allows a server to verify that a correct password has been provided without ever actually storing the plaintext password. Kickstarter reported that their “passwords were uniquely salted and digested with SHA-1 multiple times. More recent passwords are hashed with bcrypt.” When reading breach reports, the level of detail shared by the organization is often telling and these details show that Kickstarter did their homework beforehand.

A strong password hashing scheme must protect against the two main approaches that attackers can use: hash cracking, and rainbow tables. The details of these approaches have been well-covered elsewhere, so we can focus on what Kickstarter used to make their users’ hashes more resistant to these attacks.

To resist hash cracking, defenders want to massively increase the amount of work an attacker has to do to check each possible password. The problem with hash algorithms like SHA1 and MD5 is that they are too efficient; they were designed to be completed in as few CPU cycles as possible. We want the opposite from a password hash function, so that it is reasonable to check a few possible passwords in normal use but computationally ridiculous to try out large numbers of possible passwords during cracking. Kickstarter indicated that they used “multiple” iterations of the SHA1 hash, which multiplies the attacker effort required for each guess (so 5 iterations of hashing means 5 times more effort). Ideally we like to see a hashing attempt take at least 100 ms, which is a trivial delay during a legitimate login but makes large scale hash cracking essentially infeasible. Unfortunately, SHA1 is so efficient that it would take more than 100,000 iterations to raise the effort to that level. While Kickstarter probably didn’t get to that level (it’s safe to assume they would have said so if they did), their use of multiple iterations of SHA1 is an improvement over many practices we see.

To resist rainbow tables, it is important to use a long, random, unique salt for each password. Salting passwords removes the ability of attackers to simply look up hashes in a precomputed rainbow tables. Using a random, unique salt on each password also means that an attacker has to perform cracking on each password individually; even if two users have an identical password, it would be impossible to tell from the hashes. There’s no word yet on the length of the salt, but Kickstarter appears to have gotten the random and unique parts right.

Finally, Kickstarter’s move to bcrypt for more recent passwords is particularly encouraging. Bcrypt is a modern key derivation function specifically designed for storing password representations. It builds in the idea of strong unique salts and a scalable work factor, so that defenders can easily dial up the amount computation required to try out a hash as computers get faster. Bcrypt and similar functions such as PBKDF2 and the newer scrypt (which adds memory requirements) are purpose built make it easy to get password handling right; they should be the go-to approach for all new development, and a high-priority change for any codebases still using MD5 or SHA1.

Humble Helps

Sunday, May 23rd, 2010

I just ran across a post at PreachSecurity about a recent CSRF discovered in OpenCart, and a blog post by the discoverer about his interactions with the maintainer. I share Rafal’s (and Ben’s) frustration with the interaction, but I think there’s an additional lesson to be learned here. Clearly this was a loss for the cause of security, but I think it was a loss in more ways than many of the participants see.

Ben didn’t just lose the battle to get one bug fixed, he also lost an ally in the fight. Daniel (of OpenCart) was and still is in the best position to make small changes to vastly improve the security of OpenCart for all users, but after a public browbeating what are the chances he’ll be willing to get help from security folks? That’s the critical part.

The bug by bug approach to information security is trench warfare: it will take us too many years and too many lives (or at least careers) to gain ten feet of mutilated ground. We need people and practices on our side, and “wins” in security should be measured in those terms, not just in bug counts.

Yes, we as security people often have to tell people that some code or process or idea is absolutely wrong. But to say those things without destroying the human capital we desperately need, we must do it with an overabundance of patience, humility, goodwill, and tact.

I think Ben did a reasonable job of trying to be respectful and helpful, but there was something jarring in it for me.

From: “Ben”
Sent: Friday, January 22, 2010 8:06 PM
To: < *******>
Subject: OpenCart – Enquiry


I recently installed OpenCart and I noticed that it is vulnerable to CSRF attacks. I have created a sample page that is capable of inserting a rouge user (the page currently prompts you but could be done silently if the attacker knows the url of the site).*********.html

Please let know that you are looking into the security issue and are going to release an update with a fix otherwise I will make the issue public.

If you need any help fixing the problem please let me know.


See it? “…otherwise I will make the issue public.” Now, as a security person that may not seem hugely disconcerting; we know that when the carrot of coordinated disclosure fails, the stick of public disclosure can get results.  However, put yourself in Daniel’s position: he just got threatened. Reread the email, and think honestly about whether anything else constructive that Ben said will have as much of an influence on Daniel as the mere feeling that he’s being bullied.

So, Daniel gets a lot of emails from people that misunderstand some code, and he sends back a quick response (on a Friday evening no less):

On 2010-01-22, at 4:50 PM, Daniel Kerr wrote:
Ben you seem to be very clever to come up with this. But! you need to be logged in for this to happen.

So, clearly he didn’t get it. Amazingly, he still seems to be receptive to Ben. On the technical side, he may not even have checked out the PoC Ben provided. Ben responds with a lot of good information. It’s a technically accurate and helpful response for someone who is ready to learn about CSRF, but this is how he leads off:

HI Daniel,
That is the whole point of a CSRF attack. Please read for an explanation on the attack.

Ouch. I’m sure it wasn’t Ben’s intent (or maybe he was just frustrated; understandable), but that line right there is going to put Daniel in defense mode. It’s subtle, but it’s an “I’m right, you’re wrong” moment. Even if Ben is right (he is), anyone’s ego would step in and interrupt rational thought right there.

Imagine if the second email had been even more patient and humble.

Hi Daniel,
Yes, you’re right that it requires the OpenCart to be logged in, but CSRF really is a commonly used attack, and it can be very dangerous <<insert Ben’s other paragrahs here; they’re a great description of how CSRF could be exploited.>>

There’s more good information on wikipedia [link], and there’s actually a pretty straightforward fix that can eliminate CSRF vulnerabilities  [link to owasp CSRF page, or whatever you like]. I’ve attached some files that fix these vulns, and added some anti-CSRF functionality to the URL class to make it easy to clean up any other cases.

Instead, things kinda spiral downward. It ends with:

On 2010-01-22, at 8:05 PM, Daniel Kerr wrote:
what protection do you recommend?

followed immediately by:

On 2010-01-22, at 8:05 PM, Daniel Kerr wrote:
“…your [sic] just wasting my time.”

Communication is no longer occurring. We as security people must take the responsibility to prevent that. What if, at the moment that things were spiraling downhill, Ben had sent an email like.

Hi Daniel,
I hear your frustration with trying to protect users from doing dumb stuff, and I agree there’s no way to fix all the stupid things they could do, but at least CSRF is one type of attack that we can stop cold. If you have ten minutes, I’d love to talk about why I think it’s really important, and how some protections could be added to OpenCart without too much effort.

My phone number is 555-555-1212: call me any time, or let me know if there’s a good time to call you.

Yes, I know it’s crazy. It’s over the top, it’s above and beyond the call of duty, and it’s kinda weird: use the telephone… really? But that might, just might, have helped Ben win not just the battle over a single bug, but might have won an ally in the security of an entire application, and gained the cause of security goodwill in the bargain.

I’m not saying that there aren’t complete, incorrigible, asshat people out there, or that killing them with kindness is any kind of panacea. Ultimately, Ben may not have been able to make progress even if he was a genetic hybrid of Mother Teresa and Bruce Schneier. What I am saying is that we too often forget that real security comes from the people, not just the code.

So, when you find yourself in a situation like Ben, please consider digging deep into your well of patience and giving a bit more when you’re tempted to give less.