Best Practices for Verifying Vuln Fixes

The pentest lifecycle is coming to a close. The previous posts have weighed heavily on getting the process started and running smoothly. After all, it’s important to identify vulns within your apps. But most important is fixing them so the app’s users and data can remain well-protected.

Part of keeping a pentest successful is ensuring that the pentesters submit findings with clear risk and reproduction steps. This helps developers understand and address the problem.

A pentest isn’t over once the final report has been delivered. Ideally, the pentesters will remain available to confirm the fixes for each of their findings. Here are a few things to keep in mind when going through the results of a pentest.

1. Resolve Findings Promptly

The time it takes to release fixes is affected both by the complexity of the vuln as well as the maturity of the DevOps team. An efficient team can release a fix within a few days. Unfortunately, sometimes a fix can take a few months.

Part of the process for resolving findings is considering their risk. Critical vulns should be addressed quickly. Other vulns may not need to be addressed at all. These latter types of vulns fall into an “Accepted Risk” category.

Accepted Risk isn’t a loophole to avoid fixing issues. It’s an acknowledgement that some design decisions trade off different levels of security. Some decisions may be choose between two competing approaches to usability without sacrificing security significantly for either one.

2. Address the Underlying Cause

One reason to have pentesters confirm a fix is to ensure the vuln is correctly resolved. In the worst case, a developer may have simply blacklisted an attack payload instead of applying a proper countermeasure. As an egregious example, this would be like blacklisting “alert” or “javascript” to prevent a cross-site scripting vuln instead of correctly applying the relevant output encoding for the payload.

3. Look for Vuln Patterns Elsewhere

The nature of a blackbox pentest means that the pentesters don’t have full insight or visibility into the app. As a developer, you do.

Certain types of vulns have patterns that lends themselves to quick searches for similar problems elsewhere in the code. This might be a misused function that lead to an injection attack, a missing CSRF token, or a missing access control check. Take the time to review similar code paths for this kind of vuln.

Conduct a postmortem for high and critical vulns. This exercise brings a team together to discuss what went wrong, whether a proposed fix will be effective, and create action items for additional countermeasures. The goal of a postmortem is to collect and share knowledge about important security vulns. It’s not an exercise in placing blame. While the presence of a vuln implies that someone did make a mistake, it’s important to understand how the mistake was made and how it could be prevented in the future.

4. Create a Regression Test

When possible, create a regression test that can reproduce and catch the vuln in the future. A regression test is also a way to discover other areas where a vuln may be lurking. (This ties into step three above.) Tests should be an integral part of a DevOps pipeline. They ensure code quality as well as build confidence in an app’s stability under constant change.

Always prefer creating a regression test over making a comment in the code. Comments go stale quickly, may be ambiguous, and may often be incorrect. Code that is self-documenting (e.g. informative function and variable names) and whose functions are short enough to fit within a page of text is far easier to maintain.

Going through the effort of creating regression tests also helps reinforce an understanding of the nature of a vuln. By reproducing it, a developer can get a better appreciation for the underlying problem and consider more effective countermeasures. It’s also a way of determining whether your test infrastructure is sufficiently flexible enough to handle certain types of error cases. For example, if you have no way of rendering a web page or inspecting the DOM produced by a resource call, then you may not be able to effectively test for cross-site scripting.

It may not be possible to create a regression test for every vuln, especially complex ones that require multiple steps or exercise a series of flaws.

A pentest should equip you with the knowledge about the current risk exposed by your app and provide recommendations for reducing the risk of the app’s data or users being compromised. The pentest may be a point in time snapshot of the app’s flaws, but how you address the vulns will have a lasting improvement to its security.

Make it harder for vulns to appear. Refactor code so that developers can more easily do the right thing by default. Check out this blog post for additional ideas on making vulns a rare occurrence.

Now that you’ve completed a pentest, it’s time to start planning for the next one. The frequency of testing can align with a calendar, e.g. semi-annual or quarterly, or align with major code releases. In any case, the speed with which you’re able to resolve vulns will also be a signal for how well you’ll be able to manage more testing.

At the completion of the pentest, Cobalt takes an extra, important step to collect feedback about how its platform and researchers worked throughout the pentest lifecycle. This feedback helps ensure the entire process delivers quality results. The next post in this series shows how Cobalt uses feedback from customers and researchers alike to make pentesting an effective part of your security program.

In case you missed the previous step in the pentesting process. Here are 4 Tips for Making the Most of a Pentest Report.