Your Vulnerability Management Sucks

On March 16th I had the pleasure of speaking at the Yorkshire Cyber Security Cluster about Vulnerability Management. I’ve included my slides from the presentation and some speaker notes on the content covered here:

In this presentation, I attempt a tool-agnostic look into how organisations should approach vulnerability management. Whilst I definitely share some opinions about PDFs and Microsoft Excel not being the best tools to manage your vulnerability information and I do show some screenshots of Akimbo’s Vulnerability Management Platform – I try to highlight features to look out for if you’re looking at buying or building vulnerability management tooling, rather than recommending a specific tool.

I opened the presentation by discussing what vulnerability management is not – and that it is not just running vulnerability scans and performing penetration tests. I instead try to drive the conversation towards a system that allows you to manage and share information about vulnerabilities, over time. Of course, running a penetration test is useless if you’re not managing the risks that it finds, but more than that – I try to look at the different levels of maturity around vulnerability management, with the aim of making the data more actionable.

I also bring in the idea of tying your systems together, so of course if you’re managing your penetration testing data with one system, you should look to bring in vulnerability scanning data into the same system – but then I build on this idea to talk about bringing in asset management and risk management information into the same system. The example that I give is a common one found during penetration testing – that a system vulnerable to a critical issue is marked for decommissioning, but for some reason that decommissioning is delayed and so the vulnerability goes unintentionally unactioned. If you can bring in asset lifecycle information into your vulnerability management system this will allow you to adjust the risk based on the criticality of an asset, but also automatically highlight vulnerabilities that are no longer a concern once the assets that are affected are taken out of use.

Two running themes throughout the presentation were: securely communicating known issues to third parties, and centralising vulnerability discussions. For example, I highlight the problem of discussing a vulnerability with a third party such as an auditor or external penetration tester – you likely want to ensure that the discussion is secure, and you likely want to hold on to the decisions that were made during that discussion. I think most people will agree that email is neither the best tool for secure communication with third parties, nor is it the best tool for discussion involving multiple parties that you may want to archive.

When you build or buy a vulnerability management platform, you should make sure that it has user management that allows you to add third parties such as auditors and external testers – and you likely want their access to be: temporary, auditable, and secure. You should also ensure that if a user is removed from the platform that you still retain their contributions, such as input into discussions and disclosed vulnerabilities. “Secure” here is likely going to mean that access is controlled granularly (role-based access control, for example), that information is encrypted in transport, and that access control supports things such as multi-factor authentication.

Now thinking about the lifecycle of a vulnerability, from discovery, to remediation (potentially partial at first before a full fix at a later date), followed by archiving. It’s likely going to be important that you can assign vulnerabilities to different team members. For example an external tester should be able to pass a vulnerability to a security manager, who can then assign it to a developer for remediation, who can then pass it back to a tester (potentially a different one) who can confirm the issue is remediated. In short, the ability to assign issues is an important feature of a good vulnerability management system. Even if this is just something simple like a ticket management system that allows you to assign an issue to a user for action; it’s certainly better than passing around a Penetration Test Report as a PDF and playing vulnerability tag over email.

Finally, since many organisation’s either prefer, or are required to by compliance, have a “line in the sand” periodically regarding penetration testing – rather than my personal preference of using a continuous testing approach to security – it’s important to be able to highlight when the last full penetration test was; so the ability to archive a point-in-time-assessment (even if that’s still a report PDF) is important too.

In summary, if you’re using Report PDFs or an Excel spreadsheet to track your vulnerabilities, you’re probably doing things the hard way – and a well-designed platform will allow you to optimise your approach. Your vulnerability management system should allow you to:

• Share vulnerability information quickly and securely, including with third parties such as giving read-only access to auditors or allowing temporary access to external penetration testers.
• Discuss vulnerabilities and keep those discussions tied to the vulnerability or asset, so that you don’t lose data in long email threads or in Slack chats.
• Mark risk on a per-asset level. Different assets have different criticalities to the business and hold different information, you should be able to group vulnerabilities (e.g. show me all of the SQL injection) but still mark the risk at a per-asset level. (e.g. this system is critical to the business, so the risk is higher).
• Assign vulnerabilities for remediation, to make it clear who is currently responsible and to make actioning and retesting issues more efficient.
• Track a vulnerability throughout its lifecycle, even if that lifecycle is awkward such as an issue being found, partially fixed, fixed, reintroduced, and fixed again. Whilst we hope for a simple life sometimes remediation is only partial, or patch regression happens, or systems are decommissioned and then brought back online – your vulnerability management platform should be able to handle all of these awkward edge cases.

Thanks again to the YCSC for letting me rant about vulnerability management!

If you want to find out more about the platform that we offer along-side our Penetration Testing or Asset Hardening services, then please get in touch as I’ve love to show you around. Otherwise, just promise me you won’t track and track all of your important security data in a giant spreadsheet!