Black box, grey box and white box pentests compared.

What the three terms mean

The colour describes the box the tester looks at, and how much of its inside they can see before the test starts. The card industry’s penetration testing guidance gives the standard definitions: in a black-box assessment the client provides no information before testing, in a white-box assessment it may provide full and complete details of the network and applications, and a grey-box assessment sits in between, with partial details.1

The UK’s National Cyber Security Centre calls the two ends closed box and open box,2 which is the better metaphor. The choice is about what you hand over, not about colour or secrecy.

Why knowledge buys coverage

A penetration test is time-boxed: days or weeks, agreed up front.3 An attacker has no such limit. Every day your testers spend rediscovering what you could have told them is a day not spent on the questions you are paying to have answered.

That is why the card industry’s guidance says its own tests are typically white or grey box: they give more accurate results and a more comprehensive test, while a black-box assessment may need more time, money and resources to reach the same ground.4 The UK’s National Cyber Security Centre makes the same point from the other side: prepare test accounts for the testers, and share your current vulnerability assessment with them.5

Realism is the usual argument for black box, and it deserves a straight answer. A real attacker is not limited to what your login page reveals. They can take weeks, buy leaked credentials, and read your public API documentation and your job adverts. A test that withholds all of that is not more realistic; it is a shorter version of the attacker’s first week.

When each one fits

Black box

Choose it when the question is about the outside: what someone with no account can find and reach from the internet. Forgotten subdomains, exposed admin panels, services that were never meant to be public. Keep it short and focused; a long black-box engagement mostly buys reconnaissance.

Grey box

The default for a multi-tenant SaaS product, because the most damaging application flaws are about who may do what, and you cannot test that without accounts. Broken object level authorization, the first risk on OWASP’s API list, is one user reaching another user’s objects by changing an identifier;6 finding it takes at least two users, ideally in two tenants.

Add the API description (an OpenAPI file, for instance) and a staging environment with realistic, fictional data. The card industry’s guidance recommends handing over interface documentation and data flow diagrams for the same reason: testers who know how things should work can tell when they don’t.7

White box

Choose it when depth matters more than realism: a new authentication flow, tenant isolation, a payment or refund path, cryptography, or a review before a major release. With the source code a tester can follow a suspicion straight to the line that causes it, then confirm it against the running system instead of probing from outside. It sits naturally beside the security testing in development and acceptance that ISO/IEC 27001:2022 covers in control 8.29.8

A different question: who knows the test is on?

Box colour is about what the tester knows. A second choice is about what your own team knows. NIST calls it overt and covert testing: overt testing happens with the knowledge and consent of your IT staff, covert testing without it, with management’s permission. Overt testing is cheaper, carries less risk and is more common; covert testing shows how your people and your monitoring respond.9

The two choices combine freely. A grey-box test can run covertly, to see whether anyone notices a tester working through a valid account. A black-box test can be fully announced. Decide each one on purpose, and if you are not sure, start with an announced grey-box test.

What the box means for an AI feature

The same scale applies to a chatbot, a RAG application or an agent, with different contents. Black box is the chat window and nothing else. Grey box adds the system prompt, the tool and function definitions, the retrieval sources and test accounts in more than one role. White box adds the orchestration code, the retrieval configuration and the permissions each tool runs with.

Teams often hesitate to share the system prompt, as if it were a secret. OWASP’s advice is the opposite: the system prompt should not be treated as a secret nor used as a security control, and credentials do not belong in it.10 Treating it as public costs you nothing against an attacker who extracts it, and handing it over lets the testers spend their days on what the model can be made to do, which is where prompt injection lives. The 2026 edition of the OWASP list has a related entry, hidden context exposure.11

How to choose

Start from the question you need answered, then choose the least knowledge that can still answer it in the days you are buying:

• “What can a stranger on the internet reach?” Black box, external, short.
• “Can one customer see or change another customer’s data?” Grey box, with accounts in at least two roles and two tenants.
• “Is this new login, tenancy or payment code sound?” White box, with the developers reachable during the test.
• “Would we notice an attacker?” Any box, run covertly, with management’s permission and a named contact who can stop it.
• “Can our AI feature be talked into misusing its tools?” Grey box: the system prompt, the tool definitions and two test users.

Whatever you choose, write it into the scope, so that anyone who reads the report later knows what the testers started with. Our guide to preparing for a penetration test lists what to hand over for each.