The PtaaS Book Sneak Peek: Chapter 5 —Agile Pentesting and DevOps

Back in January, I launched The PtaaS Book: The A-Z of Pentest as a Service, aiming to put into words how the penetration testing services are changing to be faster, more agile, and more valuable.

I want to share with you my favorite chapter, Agile Pentesting and DevOps. For applications to truly stay safe without Development and Security teams burning out, we have to step up as leaders and rethink how we do our work. This chapter explores how small changes in tools and processes can lead to huge improvements for our teams, fewer vulnerabilities, and faster patches. Enjoy!

Chapter 5: Agile Pentesting and DevOps

Agile, lean, fast: these are all words swirling around in the technology space, and security needs to keep up with them. Between shortened development cycles, increasing customer requests, and automation evolving at warp speed, there’s a lot happening quickly in the world of tech. Traditional pentesting models haven’t been able to match that need for speed. Let’s look into why.

Inconsistent cycle times

The first issue is cycle time. For example, suppose an organization is using Scrum techniques with two-week sprints (that is, they plan and code a working release every two weeks). If a pentest on Sprint 1 takes 10 weeks from scheduling to final report, its findings won’t be available until the start of Sprint 7 (Figure 5-1). This disconnect is even bigger for teams following continuous integration and continuous delivery (CI/CD) methodology.

Figure 5-1: Traditional pentesting projects don’t fit well with agile practices

The gatekeeper mindset

A second issue is the traditional mindset of security as a gatekeeper, or the “house of no.” In the old (pre-agile) days of infrequent releases and waterfall development, developers would hand over finished code to the quality assurance (QA) group, which would take several weeks to test the release. The developers would then have to fix any problems until QA was satisfied and agreed the release could go into production. This approach led to even longer release cycles, stifled innovation,
and caused enterprises to sacrifice market share to their nimbler competitors.

A core principle of DevSecOps is that developers test their code for quality and security. Rather than being gatekeepers, QA and security teams play a consultative role, helping developers understand how to detect and fix issues as early as possible in the development cycle, when the cost of repairing bugs is lowest. Traditional pentesting timelines, where pentesters work in a silo for weeks and email a report, do not fit with an agile or DevSecOps mindset.

That’s not to say pentesting is dead to the agile team. It can bring tremendous value, if you:

• Make pentesting faster and more flexible.
• Demonstrate how it enables developers to continuously improve the quality of their code.
• Tie it all back to how your combined work creates value for the organization.
  
  

Shortening the Pentest Cycle

Pentest as a Service (PtaaS) can dramatically shorten the cycle time of pentests, making them faster and more flexible. In earlier chapters I described the features that make it happen, such as:

• Automated workflows for scheduling, scoping, and preparing pentests 
• Continuous collaboration between pentesters and security and DevOps teams
• Real-time findings and context, instead of waiting for a final report a week or more after testing is done
• Automatic ticket generation, rather than a manual process
• Faster remediation, clearer tracking, and more robust retesting 

These points lead to major changes in how pentesting and sprints function in parallel, which I’ve pulled together in Figure 5-2.

Figure 5-2: With PtaaS, pentesting becomes fast and agile enough to add value to more sprints

Using the example in Figure 5-1, suppose that you schedule a pentest and receive its final report within three weeks, instead of 10. Even better, suppose pentesters share findings with you as soon as they come up, which happens as early as two to three days from the end of Sprint 1.

These findings can help guide developers during Sprints 2 and 3. Your first pentest is over, but you schedule another one to check a critical release from Sprint 3. Its findings start appearing within days, helping your developers avoid the same mistakes during Sprints 4 and 5, and so on. Not every team will pentest as often as this at all times, but the point stays the same: if you need to, the opportunity is there.

Let’s pause for a moment here. PtaaS alone won’t make pentesting agile. You’ll also need to adjust your planning and work toward breaking large projects into smaller, faster, more frequent pentests. This is similar to the agile practice of dividing large software releases into bite-size “stories” that can be implemented in a short time.

You’ll also want to have each testing cycle focus on changes from the most recent code releases, since new code is most likely to introduce vulnerabilities. To keep track, you’ll need to monitor what developers are working on. How about attending some of those standup meetings or sprint planning sessions?

Pentesting Alongside the DevOps Pipeline

Can you insert pentests into the DevOps pipeline? Even with PtaaS, I don’t think so. Even with faster setup and earlier updates, the full process still takes two to three weeks, and that’s too long to ask developers to wait before each code commit. But pentesting can definitely work as a control that runs alongside software releases, like in Figure 5-3.

Figure 5-3: Pentests can run alongside software releases, giving developers the feedback they need to continuously improve

In this scenario, pentest results are still used to fix security issues in the code base. But equally important, development teams use the findings to determine the root causes of those issues and make larger improvements to prevent them in the future. In fact, you can achieve part of this either with or without PtaaS by changing how you position your findings. Instead of saying “Here’s a list of defects from release XX.X.X — go fix them,” try this wording: “Here are security issues that found their way into our code — let’s talk about how we can keep that from happening in later releases.”

This approach does two things: it prevents shifting of responsibility, and it changes the focus from making hot fixes to supporting developers in writing better code. In fact, the more the security team, development team, and pentesters collaborate, the more value you derive from each pentest. For example:

• Pentesters are more efficient because they can ask developers about application use cases and other technical details.
• Developers learn directly from pentesters about security issues and how to fix them.
• Pentesters help security teams and developers analyze root causes and determine what changes to training, tools, and processes will prevent them.

This is where the real-time collaboration features of PtaaS (check out Chapter 3 for more) can really make a difference.