White box, black box, and gray box vulnerability testing: What's the difference and why does it matter?
2017 saw a record number of security vulnerabilities, with more than 16,000 vulnerabilities reported by the end of Q3. That is more than in all of 2016 combined. While some organizations are addressing these application security risks quickly, others are failing to do so. A recent SANS survey found that 10% of enterprises are not testing their applications for security at all, with another 24% testing only once per year or less. A mere 12% of organizations are performing application security testing on a regular basis.
Why are businesses neglecting application security testing? Rapid development cycles are a major contributing factor, with application security testing viewed as an obstacle that hinders development and deployment. A 2017 report on mobile and IoT application security testing found that up to 75% of application security professionals felt pressure to rush a release, at the expense of security.
The dichotomy between these statistics is alarming. Enterprises need to focus more attention on application security testing, even if it means slowing down. Fixing vulnerabilities after application deployment costs businesses more time and resources, not to mention negative publicity, lost revenue, lost customer data and trust, a tarnished reputation, and even possible legal action should those vulnerabilities get exploited.
Knowing the different approaches that can be taken to application security testing can help you develop a strategy that will deliver the biggest ROI, balancing improvement in security with the extra time and resources required.
There are three flavors of application security testing, generally referred to as black box security testing, white box security testing, and gray box security testing. Understanding the differences between each type will help you form a strong plan for application security testing that will decrease your chances of being exposed to potential threats and maximize resources, funds, and time.
Black box security testing mimics the actions of a hacker, attacking the application from the outside. In doing so, it tests the functionality of the application. It is performed based on the assumption that the attacker has no knowledge of the inner workings of the application: it treats the application like a "black box" whose contents you can't see, just as a hacker can't see into your application.
Script kiddies (skids) – Untrained individuals ("kiddies") who use programs or scripts developed by other people to attack an application, and scan for known vulnerabilities.
Mid-level hackers – Individuals who possess more knowledge about development, systems, and networking, and can therefore wreak more havoc.
Elite hackers – Individuals who are more likely to be sponsored by a nation-state. They have more time and resources to devote to hacking.
This testing looks for known vulnerabilities, weak access controls, and weak defense mechanisms in the application. To be thorough, it is important to test for all variables and scenarios that may come into play.
This testing can be done faster than white box vulnerability testing because it is more limited in scope, only attacking from outside the application.
It can be less costly, although this does depend on the specific application and the scope of testing.
It is not comprehensive, as it is only testing from the outside.
Test cases can be difficult to design without clear guidance on what areas and/or vulnerabilities to specifically test.
If you want to be thorough, it can take a lot of time to try different scenarios and use cases. Hackers often have more time on their hands than you do, so you can't always anticipate everything they might try.
White box security testing is performed based on having all knowledge of the application, testing the application's internal workings. It is frequently performed with access to the full source code, so source code scans and reviews are often included as part of the testing process. Full access internally (locally) to the application is provided, including log-in credentials and full authentication. White box security testing should be done before an application is released into production, so vulnerabilities can be identified and corrected prior to deployment.
Any potential vulnerabilities that are found need to be further tested via black box testing to see if they are actually exploitable, which requires additional time. Taking this step helps you prioritize your remediation work.
This type of testing is lengthier, slowing down the development process and conflicting with rapid development lifecycles.
It is more expensive due to the time and highly-skilled resources required. White box security testing requires knowledge of what makes applications insecure, ability to read code, and to see security exposures in software architecture.
As its name would imply, gray box security testing is a combination of white box and black box testing. It is the most ideal approach to take when you need to balance time, cost, and impact.
Gray box security testing can be employed in various ways, depending on the application and the testing goals. It may, for example, be performed by testing the exploitability of potential vulnerabilities identified in white box testing.
The hacker has a high-level understanding of the application's dataflow and architecture.
The hacker may have access to parts of the source code or compiled binaries, and may know the languages and specific technologies the application is using.
The hacker may have user or admin accounts with which he/she can login.
These are not unreasonable assumptions: seasoned attackers have ways of discovering things like this.
Provides a better representation of a determined hacker who is likely to have at least some knowledge of the internal workings of the application, and more time to focus on an attack, possibly researching the company, developers, and the application.
It will not cover all aspects of the application, as that would be too time-consuming.
It lacks the full coverage of the source code that white box testing provides.
Test cases can be difficult to design.
It is important to understand the basic differences between the three types of application security testing, and what each approach does to improve application security. Having multiple options helps in formulating an approach that best fits your project's budget and timeline.
In an ideal world, businesses will employ a blended approach, employing some of each type of testing based on the specific application and the resources available.
Black box security testing – Emulates short-term hack attempts of script "kiddies" (attackers without a lot of time, resources, or skills).
Understanding all three flavors and leveraging the strengths of each helps you create a more resilient application security profile. Taking the time up front to develop an application security testing plan, and incorporating it into the development process, can keep your application from contributing to the rise of security vulnerabilities in today's market.