The Third Wave of Application Security

The emergence of crowdsourced application security platforms, fueled by data and technology, has incited a new wave of change in the application security industry.

Overall, the application security industry has undergone three major developments. First, we saw the formation of the industry in the 1990s. This era was driven by people and the birth of organizations dedicated to application security. The second wave came around with the new millennium, where we saw automation being applied in application security. Now we are transitioning into the third wave, where crowdsourced security platforms are emerging. In this third wave, organizations gain access to a global on-demand security workforce via a platform with built-in processes and data insights.

This third wave will fundamentally alter the face of application security; it will impact the professionals, companies, and organizations in the industry. So it is paramount that we understand the underlying trends. In this post, we will look deeper into the transformation and explore how the industry has evolved over the past decades.

First Wave — People

Since the early days, the need for security testing has been driven by the sharing of IT resources. For example, every time there are multiple users using a system, there is also the risk of someone maliciously attacking other users of the system. Therefore, as time-sharing systems became a common computing paradigm in the 1970s, we also saw the first penetration tests coming along. These early tests were done by Tiger Teams to probe the security of the time-sharing systems.

Yet, it wasn’t until the 1990s that we saw a significant uptake in companies dedicated to security testing of systems and applications. One major driver for this was the launch of the Mosaic browser in 1993 which resulted in an increase of personal computing and e-commerce.

This first wave was all about the people. We saw how early hackers organized and launched non-profits, companies, and conferences. With this came some of the first security companies such as ISS X-Force, FishNet, atStake, and FoundStone. This early era of the security and hacker community is elegantly captured by Lopht Heavy Industries and their senate testimonial in 1998.

With people followed training and certifications. For example, the SANS Institute for security training was founded in 1989. A year prior to that, ISC had been established, which later in 1994 became the first organization to administer CISSP exams. Today, more than 100,000 professionals have passed the CISSP certification exam.

Second Wave — Scanners

At the beginning of the new millennium, the second wave came. This wave was centered around automation of the manual security testing that the industry had seen in the preceding years. One of the pioneering companies in this era was SPI Dynamics, who developed WebInspect. SPI Dynamics was eventually acquired by HP. Another early pioneer was the Israeli company Sanctum, who launched AppScan in 2000, and which at a later time ended up as a part of IBM.

Companies such as Qualys and Rapid7 grew up within network security and became vulnerability management companies. Testing a network for missing patches and known vulnerabilities is a different problem than testing application code for to-be-discovered vulnerabilities. However, over the years, companies like Qualys and Rapid7 also moved up the stack and added application security capabilities.

Over the decade the application security scanners grouped into two categories. One type of scanners focused on automatically testing of live web applications and are known as DAST scanners (Dynamic Application Security Testing). Other companies such as Fortify and Veracode focused more on the source code level and created the category of SAST (Static Application Security Testing). Whether testing the application statically or dynamically both types of scanning approaches faced challenges as we will discuss later.

People and automation working hand-in-hand

One can think of the first wave in the 1990s as the formation of the industry, in which people established a common framework and language. The second wave was focused on putting scanners to work and automating some of the more simple and tedious manual processes. In addition, the automation introduced a standardized approach to security testing which also brought consistency and predictability.

The power of automation is evident when searching for technical vulnerabilities. A human can manually test for a SQL injection vulnerability by putting a single quote in a search box and looking for a database error. By building a software program to simulate this behaviour, we can automatically find the same vulnerability at scale.

However, automation doesn’t make humans obsolete. On the contrary, automation actually enhances human capabilities. As an analogy, the smartest chess player in the world is neither a human nor a machine but a centaur.

A centaur chess player is a human that listens to the moves suggested by a machine but occasionally overrules and directs the machine in a different direction. Similarly, the smartest security professional is one who directs security tools.

Think of a Burb Suite and ZAP proxy as enhancements of security professionals. The human provides creativity and business logic awareness whereas the machine provides brute force and automation capability.

Changing application landscape

Web applications became increasingly complex during the 2000s. By 2010, a web application was no longer just a simple HTML page. Today, applications encompass even more business logic. From a security perspective, this means that additional business understanding is required to test the security of the application.

Furthermore, applications are moving to the cloud and are increasingly API-driven. From a security scanning point of view, this creates new challenges such as figuring out how a scanner can interact with an API. Often an API ‘speaks’ the business language of the organization. This language is custom to the organization and it’s therefore difficult for a standardized scanner to understand this and interact with the API to do fuzzing and test the security of the API.

Lastly, due to agile development methods, code is being deployed faster and faster. Scanners can keep up with this speed; however, from a manual perspective this is a new demand. For example, a yearly manual application security review just isn’t enough anymore. Instead, a higher testing frequency such as quarterly or on-demand is needed to ensure new features are tested.** **In the second wave, the changing application landscape therefore challenged traditional approaches to application security in a couple of different ways:

• Scanners are not a complete solution: Even though scanners bring great scale and consistency they are faced with several technical challenges. For example scanners can take a long time to run due to outdated crawler approach which fall prey to endless loops. Also, it’s a challenge that they often report many false positives and false negatives. Lastly, they fail when it comes to prioritization of the issues identified and hand the organization a long report with thousands of issues but no real way to distinguish what is important or not. Furthermore application security scanners do not understand the business logic of modern applications. As a result, application security scanners are not a complete solution and today’s most devastating attacks are still being identified by people.

• People do not scale: Organizations still need human-powered security testing. A company looking to engage people traditionally had two alternatives: hire full-time employees or engage a consultancy. Unfortunately, the industry skill shortage has made it difficult to hire on-staff and consultants are fairly costly.

Third Wave — Crowdsourced Platforms

Around 2010, we saw the third wave of application security taking off, ignited by the emergence of bug bounty programs. The first bug bounty program dates back to 1995, when Netscape offered cash for vulnerability reports against its browser.

However, it wasn’t until about fifteen years later that we saw a considerable increase in organizations starting bug bounty programs. In 2010, Google expanded its bug bounty program to include its web properties. Shortly thereafter, Facebook and Paypal followed suit.

These bounty programs popularized crowdsourced application security and between 2012 and 2013, we saw the first startups such as HackerOne, Synack and of course Cobalt, leverage the crowdsourced model.

On-demand talent, platforms, and data

Bug bounty programs not only delivered global talent sourcing to application security, but also popularized the notion to use a software platform when doing manual vulnerability discovery. These platforms are evolving from public bug bounties into other domains of application security such as crowdsourced penetration tests.

Overall the third wave has three defining characteristics:

1. A global on-demand security workforce

2. Platform and processes integrated

3. Data and KPIs

Global on-demand security workforce

The launch of bug bounty programs kickstarted a new and better approach for organizations to engage security talent. For decades, security professionals have been organized around hierarchical organizations where interactions are aligned according to who has authority. These closed, local organizations are now being transformed into a more open, liquid workforce where information and access flow freely. In short, the application security workforce is moving from an institution to a platform.

One might say that a crowdsourced penetration test draws on the same talent pool as traditional consultancies. However, the industry skill shortage is an incentive and matching problem, which to a large extent can be solved by global talent sourcing and better skill and availability matching. It’s a matter of local sourcing vs. global access. The talent pool is expanded as security practitioners are able to pick up extra work on weekends and evenings and thereby unleash previously untapped resources. This makes the security workforce more liquid and scalable.

Platform and processes

In the previous decades, most manual application security workflows such as vulnerability disclosure or penetration testing were performed without the use of a platform to effectively manage the workflow and integrate with developer bug tracking systems. For example, consultancies would engage with clients by sending PDF reports in email, and did not leverage technology to ensure a consistent process and secure sharing of data.

The current pen testing industry can in some ways be compared to the accounting industry before accounting software. Many workflows were manual, prone to error, and difficult to track. There have been some attempts by consultancies to build a platform, but with limited adoption. Other attempts to build platforms around application security process and collaboration include the Dradis framework or Vulnreport.

In the 2000s, we saw how scanners brought both automation and consistency to the testing. Similarly, the crowdsourced application security platforms will bring consistency to the manual processes and workflows. This will make the processes more smooth and accountable.

Data and Insights

For a long time, organizations have looked at security metrics such as cost-per-vuln, time-to-fix, and number-of-open-issues. However, because there didn’t exist a common system of records and standardized processes, these metrics and KPIs have often been an after-the-fact exercise. For example, organizations often need to import findings from pen test engagements into internal issue tracking systems such as Jira. This exercise is both time consuming and prone to error.

With the introduction of a crowdsourced security platform, the process is optimized as data and KPIs become the default and are seamlessly integrated. This makes application security more data-driven and allows organizations to more easily benchmark and share their KPIs.

From a talent perspective, the conventional service industry has been lacking feedback loops in the form of a quality and review system. This changes in the third wave as we now have elaborate reputation systems. As security professionals need to build and retain a strong, positive reputation to stay attractive on the platforms, it also improves the feedback loops and ensures real-time alignment of results with rewards.

Organizations are migrating to the Third Wave

Because the application security scanners don’t solve the challenges of modern organizations and the traditional way for organizations to hire security talent is outdated we see increasingly organizations leverage the flexibility of the Third Wave.

There is growth both from medium sized organizations migrating to crowdsourced pen testing platforms as well as government clients leveraging bug bounty solutions for vulnerability discovery. This trend will continue as the Third Wave presents real solutions to a better application security setup. The Third Wave provides access to quality security talent at scale at a lower cost than traditional approaches. Also the third wave provides transparency and a system of record to manage manual application security testing. This accountability will push more organizations toward a platform approach and it will soon be seen as irresponsible to not engage and manage your application security testing via a platform.

Is application security turning into a Chimera?

The first wave was all about the people and the foundation of the industry. In the second wave, we saw scanners and automation coming along. While the second wave had man and machine working hand in hand as a Centaur, the third wave might be better characterized as a Chimera. In ancient mythology, a Chimera is a hybrid creature, comprised of parts from three different animals. Similarly, the third wave is a dazzling hybrid of people, security automation and a global platform. Together, they work in concert to empower the creativity of humans and make the workforce smarter.

Three Waves of Application Security

Companies today now have access to an on-demand security workforce on a platform combined with built-in processes and readily accessible data. We’ve seen significant progress over the decades — and the symbiosis of the human, machine and platform is where application security is going next.

Thanks to Mike Shema, Caroline Wong and Caleb Sima for having read drafts of this post and providing valuable feedback and comments.