Intelligence in Offensive Security: How to Level Up
Echelon cyber definition problem

Cyber Definition Problems: Red Teaming vs. Penetration Testing

The cybersecurity industry has a definition problem. Throughout my career, I’ve heard clients, partners, and even other leaders in the field use the terms "red teaming" and "penetration testing" incorrectly and interchangeably to describe multiple types of cyber assessments, products, or even concepts.

Sometimes this is done purposefully to make something sound much cooler than it is. But many times, it’s just because these incorrect definitions are taught and passed along and discussed, so everyone just assumes they are true.

Though either by blissful ignorance or willing misleadingness, this basic definition problem is only hurting our industry.

Pen Testing vs. Red Teaming

When you hear the term red team, what do you think about? Do you think about a penetration testing team doing offensive security work, or do you think of an objective-focused assessment that uses all realms of security (physical, logical, social) to gain unauthorized access to an organization over a long period of time by emulating real-world criminals?

Depending on whom you ask, the terms are used interchangeably, and depending on the context in which the term is used, it CAN be either one, but when used in the description of an assessment, a red team assessment is not the same as a penetration test.

By using the term interchangeably, we actually hurt the organization more than you may think. To understand why we must first understand the terminology.

Penetration Testing: Definition and Goal

A penetration test is a simulated attack on a computer system, network, web/mobile application, and/or an organization’s physical presence.

The goal of penetration testing assessments is to identify vulnerabilities in a particular focus area that could be exploited by a threat actor.

These simulations are usually a relatively short duration - a penetration test can be anywhere from one to six weeks (sometimes longer if it’s done on an extremely large scale), but they are almost always short assessments.

This is because the broad view of penetration testing is scaled to only certain focus area(s) and almost always done from a “gray box” perspective, which means the assessment team has at least some knowledge of the internal workings of the focus area(s) being tested. That doesn’t mean that penetration testing can’t be done from a no-knowledge perspective (also sometimes called black box testing), but these types of tests almost always start with partial knowledge to ensure that the focus area is fully vetted.

During a penetration test, there is almost always continual communication between the assessment team members and the organization’s security team/trusted agents. It’s not meant to be a secretive engagement, even though some sections of the organization may be unaware of the assessment.

I like to call these types of engagements a shotgun blast to discover where the gaps are. Overall, the result is a more secure system, network, web/mobile application, or physical security presence, but only within the focus areas set out in the statement of work.

The result of a penetration test is a list of vulnerabilities that the penetration testing team was able to exploit to gain access or escalate privileges.

Red Team Assessment: Definition and Goal

Now let’s transition into defining a red team assessment and its goal. A red team assessment (now sometimes called an adversarial simulation thanks to our definition problem) is a type of security assessment in which a team of experienced offensive security professionals simulate a coordinated attack on an organization’s systems, networks, physical locations, AND people to test the organization’s defenses and resiliencies.

The assessment team will use tactics, techniques, and procedures (TTPS) similar to - if not exactly like - real-world threat actors that would be relevant to that organization.

This is where I think many people start to mix up the two terms between red teaming and pen testing. The difference is that the purpose of a red team is to simulate or imitate a real-world attack. This means that the assessment team isn’t looking to find any vulnerability that they can within a limited focus area.

Instead, their focus is to accomplish a set of objectives that, if achieved, would result in catastrophic consequences for the organization if the attack was real.

A red team assessment might include simulating the deployment of ransomware on multiple devices, exfiltrating financial data, establishing undetectable persistence, and so on. The goal of a red team assessment is to identify weaknesses and vulnerabilities in an organization’s overall security posture AND its incident response plan.

Red team assessments are always done from a no-knowledge perspective and are not short by any means. Since the assessment team has zero knowledge going into these engagements, all phases are conducted fully. This means that more time must be spent to reach a level of knowledge in which the emulated threat actor can covertly carry out their plan to capture the objectives set in the pre-planning meeting.

So where a penetration test is anywhere from one to six weeks on average, a red team is anywhere from three to six months, depending on how mature the organization is and how many objectives are set out for the assessment team to accomplish. This is to ensure that the engagement is as realistic as possible.

Red team assessments result in a more robust security program that focuses on the strategic level gaps that could lead to a breach. Because the focus of a red team isn’t just to find vulnerabilities, the results are more of a story around how the team was able to gain access to the internal network and reach its series of objectives.

Sometimes this includes vulnerabilities, but it can also include, “No one stopped us at the door, so we walked in and plugged in a USB which called back to our infrastructure.” (If you think that isn’t a realistic scenario, this attack vector is actually on the rise according to researchers. Even cybersecurity giant Mandiant is seeing bigger groups taking part in this as well.)

The Danger of Confusing Red Teaming and Pen Testing

Now that we have defined the difference between the two, it’s easy to understand how referring to a pen test as red teaming can be hurtful to the organization.

Essentially, it sets up the false assumption that an organization is already performing the most complex style of testing within the offensive security space.

This false sense of security can lead organizations to accept a higher risk profile, which in today’s age of rising cybercrime only changes the question from “if” and breach will happen, to “when.”

At the same time, cyber insurance premiums are only increasing, so this isn’t a question that many organizations want to face.

The Bottom Line

While the definition problem is significant in the offensive security services world, it is not unique to it. Many other terms like Zero-Trust and Vulnerability Management have started to find their definitions changed for the sake of sales as well.

Ultimately, the lack of common definitions will only get worse until we stop letting definition change happen, and it’s up to us as industry professionals to be true to the services we offer.

At the end of the day, if you aren’t sure about what you’re getting from a pen test or red team service provider, it’s ok to ask a fellow professional for a gut check.

Asking smart questions and holding the line on how we define assessments are everyday steps we can all take to start improving our field and services, ultimately making everyone more secure.

Sign up to get Cyber Intelligence Weekly in your inbox.
Latest Intelligence