How to prepare for a penetration test.

Before you start: the question and the scope

Nine steps, in the order they usually happen. Most of them are decisions and signatures rather than technical work, and most of the delays in a test come from them, so start with the slow ones.

Step 1 Write down the question the test must answer

Name the reason for the test, such as a customer questionnaire, an audit, a launch or a new AI feature, and the decision its report has to support.

It sounds like paperwork, and it decides everything else. “Our largest customer wants a recent pentest report for their supplier review” leads to a different scope than “we are launching an agent that can issue refunds”. The UK’s National Cyber Security Centre recommends scoping with all the relevant risk owners and technical staff, together with someone from the test team,1 because each of them knows a different part of what matters.

Step 2 Define the scope and what is out of scope

List the domains, APIs, cloud accounts, AI features and third-party services involved, the environment to test, and everything that is explicitly out of scope.

Be concrete: hostnames and URLs, API base paths and versions, cloud account or subscription identifiers, the AI features and the models behind them. Name the environment too. Staging is usually the better target, as long as it runs the same code and configuration as production. Production is possible when the rules of engagement fix the test windows and the limits (step 6).

Write down what is out of scope just as clearly: the payment provider, the identity provider, the marketing site that another agency hosts. The card industry’s guidance notes that a third party’s own infrastructure may fall outside your scope while the systems you manage on it fall inside.2

Access: box, accounts and documentation

Step 3 Decide how much the testers will know

Choose black, grey or white box; for most SaaS products, grey box with test accounts and API documentation gives the most coverage per testing day.

The box decides how the testing days are spent. Our guide to black, grey and white box testing compares the three; the card industry’s guidance notes that its own penetration tests are typically white or grey box, because they give more accurate and more comprehensive results.3

Step 4 Prepare test accounts and test data

Create two accounts for every role in scope, in two separate tenants, with realistic fictional data and no real customer data.

Two accounts per role is what lets a tester ask the question that matters most in a multi-tenant product: can one customer reach another customer’s data? Broken object level authorization, first on OWASP’s API list, is exactly that failure,4 and one account cannot reveal it. Create the accounts before the test starts, check that multi-factor sign-in works for the testers, and give an AI feature’s test users the same tools and permissions real users get.

Use fictional data. If the test environment holds real personal data, the testers may see it, and that brings a processing agreement into the paperwork (step 6).

Step 5 Share documentation and known issues

Hand over the API description, an architecture or data flow diagram, earlier pentest reports, your latest vulnerability scan and the issues you already know about.

The card industry’s guidance recommends handing over interface documentation and diagrams, and reviewing past vulnerabilities, earlier reports and current scan results with the tester;5 the UK’s NCSC likewise advises sharing your current vulnerability assessment.1 Known issues you have not fixed yet belong on the list: the testers can confirm them quickly and move on, instead of spending a day rediscovering them.

Permission: authorization, rules and providers

Step 6 Sign the authorization and the rules of engagement

Sign a written authorization that names the systems, the test window and the testers’ source IP addresses, agree the rules of engagement, and sign a processing agreement if testers may see personal data.

This is the step that makes the test legal. In the Netherlands, entering a computer system without permission is computervredebreuk under article 138ab of the Wetboek van Strafrecht;6 the signed authorization, often called a vrijwaringsverklaring, is what turns an attack into a test. It should be signed by someone entitled to authorize testing of those systems, and it lists the systems, the window and the addresses the testers will work from.

The rules of engagement then fix how the test runs. The card industry’s guidance lists what they should settle before testing starts: the time window, fragile or legacy systems, how you will communicate, whether you want updates during the test, security controls that may interfere, how sensitive data found during the test is handled, and what happens if the testers find signs of an earlier or ongoing compromise.7 Our own rules of engagement follow the same outline.

If the testers may see personal data, the Dutch Data Protection Authority is clear that controllers and processors need a processing agreement under Article 28(3) of the GDPR.8 Ask for the tester’s template together with the quote, so it is not the last document waiting for a signature.

Step 7 Check your providers’ testing rules

Confirm what your hosting and cloud providers allow, and get written permission from every vendor whose systems you do not own.

The large cloud providers do not ask for advance notice for ordinary tests of your own resources, but each sets limits. AWS allows tests of the services it lists without prior approval, prohibits denial of service and flooding, and runs a separate process for simulated events such as red team exercises.9 Microsoft’s rules of engagement keep tests inside your own tenant and rule out denial-of-service testing.10 Google Cloud does not need to be told, as long as you follow its acceptable use policy and only affect your own projects.11

Everything you do not own needs its own permission: the SaaS tools in your stack, a chatbot platform you license, a managed service your hosting provider runs for you. Where an agreement requires a provider’s approval, get it before the test.2 You cannot authorize a test of someone else’s system, however much of your data lives there.

The test week: people and environment

Step 8 Prepare your people and your environment

Name a technical contact and a stop contact for the whole test window, tell whoever watches your monitoring, take backups, and decide how protective controls such as a web application firewall treat the testers.

Name two people and make sure they can be reached for the whole window: a technical contact who answers questions and unblocks a locked account, and a stop contact who can halt testing if something behaves unexpectedly. Tell whoever watches your monitoring, in house or at a managed provider, when the test runs and from which addresses. If the point is to see whether they notice, management decides that, and still knows.

Take a backup or snapshot of anything stateful the test touches, and freeze deployments to the test environment if you can, so a finding does not move while someone confirms it. Decide in advance how protective controls such as a web application firewall treat the testers. The card industry’s guidance advises keeping them from interfering, because the test should measure the application itself, not the filter in front of it;12 test the filter separately if you want to know how it holds up.

After the test: report, fixes, retest

Step 9 Plan the report, the fixes and the retest

Before the test starts, decide who receives the findings, how critical ones reach you, who owns each fix and when the retest happens.

Findings arrive with a severity, and the critical ones should not wait for the final report, so agree how those reach you and who acts on them. Give each likely area of fixes an owner before the report lands, and book the retest when you book the test. PCI DSS, for one, expects exploitable findings to be corrected and the testing repeated to verify the corrections.13

A good report tells you what to fix first and how to check the fix. The sample report shows the format we use, with a fictional client.

The checklist on one page

Print it, or tick it off on screen. Nothing you tick here is saved or sent anywhere.

1. The question: the reason for the test, the decision the report supports, the deadline.
2. Scope: domains, APIs, cloud accounts and AI features in; the environment; what is out.
3. Box: black, grey or white, written into the scope.
4. Accounts: two per role in two tenants, sign-in working, fictional data only.
5. Documentation: API description, diagrams, earlier reports, latest scan, known issues.
6. Paperwork: signed authorization, rules of engagement, processing agreement if needed.
7. Providers: cloud rules checked, written permission from every vendor you do not own.
8. Test week: technical contact, stop contact, monitoring told, backups, firewall decided.
9. After: channel for critical findings, owners for fixes, retest booked.