Automated pentesting: what it finds, what it misses, and when it replaces a manual pentest

Introduction

In two years, "automated pentest" has become the most profitable label in offensive security. You will find it stuck on products that have next to nothing in common: a vulnerability scanner in new clothes, a platform that connects you with human pentesters, an agent that replays attack techniques inside your internal network, a service that watches your external surface. They all promise the same thing, the equivalent of a penetration test, continuously, for a fraction of the price.

We sell one of those products. That is not the best seat from which to write a neutral article, so we may as well write a useful one: say precisely what automation finds, what it will not find, when it really does replace a manual engagement, and when whoever tells you so is spinning a story.

The short answer, for those who will not read further: an automated pentest does not replace a manual pentest. It replaces the eleven months during which you do not have one.

Four products under one label

Before comparing anything, you need to know what you are looking at. Here are the families you will meet, with the vocabulary vendors use.

The only automated thing about PTaaS is the ordering. It is a delivery model for manual pentesting, often a good one, but it is not the subject of this article. BAS answers a different question: it does not look for your flaws, it tests your alarms.

That leaves the boundary that causes the most confusion, between scanning and automated pentesting. It comes down to one word: proof. A scanner writes "Apache 2.4.49 detected, vulnerable to such-and-such CVE". An automated pentest worthy of the name writes "here is the request we sent, here is the response we got, the flaw is exploitable on this asset". The former leaves the verification work to you, and there is a lot of it. We compared scanning approaches in more detail in which vulnerability scan to use.

A good way to sort an offer in thirty seconds: ask for a sample finding. If it contains neither a request nor a response, it is a scan.

What an automated pentest does better than a human

Not "as well". Better.

The perimeter. A pentester tests the list they are given, in the time they are paid for. An engine that starts by hunting for your subdomains, your related brands and your staging environments tests what exists, not what you thought you had. On the perimeters we see, a gap between the declared inventory and the observed one is the rule. The method to measure it yourself is in our article on inventorying your external attack surface.

Frequency. This is the central argument, and it gets its own section below.

Regression. A flaw fixed in March that comes back in June because a deployment overwrote the configuration: no annual audit will see it before the following year. An engine that compares each run with the previous one reports it the next day.

Reacting to a fresh CVE. When a critical vulnerability lands on a product you expose, the question "are we affected, and where?" needs an answer within hours. A human will spend that time opening a laptop and digging up the scope.

Boring exhaustiveness. Checking the same misconfiguration across 340 subdomains is exactly the work a consultant billed by the day will do by sampling, and rightly so. A machine does not get bored.

What it will not find

This is the section sales brochures leave out, and the one that will save you from a bad decision.

Business logic. Ordering an item at a negative price, approving your own expense report, skipping the payment step by replaying the confirmation request. To see that this is a flaw, you have to understand what the application is for. No engine understands that reliably today, whatever the product page says.

Authorization flaws in context. User A reading user B's invoice by changing an identifier. Technically detectable, and some tools have a go at it with several accounts. In practice, knowing that a given object should not be visible to a given role means knowing the business rules. And yet this is the first category of the OWASP Top 10.

Creative chains. An experienced pentester assembles three minor findings (an information leak, a forgotten subdomain, an overly chatty export feature) into a compromise. Engines chain what they have been taught to chain. The novel remains human.

Everything that is not technical. Phishing, the call to the IT help desk, the server room door. A red team tests an organization, a tool tests machines.

Whatever sits behind authentication it was not given. An external engine with no account sees what an anonymous visitor sees. That is already a lot, and it is exactly what an opportunistic attacker sees. But the customer area of a SaaS application remains, for the most part, grey-box manual testing territory.

If the application that carries your revenue has roles, workflows and money moving through it, it deserves a human application pentest. No tool, ours included, exempts you from that.

The real issue: the time between two audits

An annual pentest is a snapshot, and it is accurate on the day it is taken. The problem is what happens over the next 360 days.

Verizon's 2026 DBIR, which analyzes more than 22,000 breaches, puts vulnerability exploitation as the leading initial access vector, at 31%, ahead of stolen credentials. It stood at 20% a year earlier. The same report measures the median time to remediate a flaw listed in CISA's KEV catalog, so one under active exploitation: 43 days, up from 32 the year before. Attackers are speeding up while defenders slow down.

On the other side, the gap between a patch and the first mass exploitation is counted in days, sometimes hours. We documented it this month: for the GitLab flaw CVE-2026-85706, the patch shipped on September 10 and the first probes arrived on the morning of the 11th. A company audited in June had no chance of learning about it from its report.

Add the speed at which a perimeter changes. A new API route every week, a subdomain created by marketing, a contractor opening a port "temporarily". None of that exists in last year's report.

This is where automation wins, and the only place where it wins hands down. It does not go deeper than a human. It is there on the day it matters. And when a flaw shows up, you still need to know which one to handle first: that is the subject of our article on prioritizing with CVSS, EPSS and KEV.

Manual or automated: the honest comparison

For the amounts, we published a detailed pricing grid for security audits. Keep the order of magnitude in mind: a year of continuous coverage costs roughly the price of a short external pentest.

What about compliance?

A question we get at every demo: "does this count as a pentest for my auditor?". The answer depends on the text, and it is rarely a plain yes.

• ISO 27001
  . The standard does not require a pentest as such. Control A.8.8 asks for demonstrable management of technical vulnerabilities, and continuous monitoring with dated reports answers it very well. Many auditors like to see a periodic manual test too, without it being mandatory. Details in our guide on automated scanning and ISO 27001.
• NIS2. Article 21 asks for risk-management measures and an assessment of their effectiveness, without prescribing a tool. In France, the level of detail will come from the implementing texts. We take stock in the article on the Resilience law.
• PCI DSS 4.0. Both are required separately: quarterly scans (requirement 11.3) and a penetration test at least annually (requirement 11.4). One does not replace the other.
• DORA. The regulation imposes a testing program on financial entities and, for the most significant ones, threat-led penetration testing (TLPT) at least every three years. A TLPT is a supervised red team exercise. No tool seriously claims to be one.
• PASSI qualification (France). When a regime requires an audit by a provider qualified by ANSSI, the requirement bears on people and a method. Software cannot be PASSI-qualified.

To sum up: automation provides the evidence of monitoring between audits, and provides it better than a spreadsheet. When a text requires a penetration test, you need a penetration test.

Seven questions to evaluate an offer

To be asked during a demo, preferably to the technical person and not only to the salesperson.

1. Show me a real finding. Does it contain the request, the response and proof of exploitability, or only a version number and a score?
2. Is the perimeter discovered or declared? If I give you one domain, what do you find that I did not give you?
3. How do you measure your false positives? "Close to zero" is not a measurement. Validation by exploitation, or confrontation with a third party, is one.
4. What do you do in production that I should know about? Are the tests non-destructive? Are some payloads excluded by default? Can rate and time windows be limited?
5. What happens between two runs? Is there a diff (appeared, fixed, reappeared), or do I have to compare two PDFs by hand?
6. Where is my vulnerability data hosted, and who has access? The list of your exploitable flaws is one of the most sensitive documents your company owns.
7. What does your tool not find? A vendor with no precise answer to this question does not know their product, or would rather you did not.

Where Flawfence stands

Since we ask others question 7, let us answer it.

Flawfence is an external engine, with no agent and no configuration. Starting from a domain, it rebuilds the perimeter (related domains, subdomains, services), detects technologies, searches for vulnerabilities and tries to validate them before reporting them. The first report arrives in under 24 hours, then monitoring runs continuously with a diff from one run to the next. The engine is algorithmic. AI comes in for synthesis and writing, not for deciding to send a request to your production. The subscription starts at €5,000 excl. tax per year.

On false positives, our answer is public: we run the engine on bug bounty programs, where a flaw only counts if the targeted company validates it. The results can be checked on our YesWeHack profile and are shown on our home page. It is a harsher judge than a home-made benchmark, and it has made us fix more than one overly optimistic detection.

What Flawfence does not find: business logic, authorization flaws that require understanding your roles, anything living behind authentication, and your internal network. For that, keep budget for a targeted application pentest. The split we recommend to a small or mid-sized company stays the same: continuous coverage on one side, and human expertise on the application that matters most.

FAQ

Can an automated pentest replace a manual pentest?

No, not entirely. It covers known vulnerabilities, misconfigurations and forgotten assets better and more often. It does not find business logic flaws or most authorization defects, which require understanding the application. The most effective combination is continuous automated coverage, complemented by a manual pentest targeted at critical applications.

What is the difference between a vulnerability scan and an automated pentest?

A scan matches versions and configurations against a database of flaws and reports matches without verifying them. An automated pentest goes as far as attempting exploitation non-destructively, and provides the proof (request, response) that the flaw is exploitable on the asset in question. It generally includes perimeter discovery, which a scan does not.

Is an automated pentest dangerous for production?

A serious tool uses non-destructive payloads: it proves an injection is possible without modifying data, that code execution is reachable without installing anything. The risk is never zero, just as with a human pentester. Ask for the list of tests excluded by default, the rate limits and the ability to set time windows.

What is continuous pentesting?

It means testing your perimeter all the time rather than once a year. The term covers two realities: an automated engine that runs every day or every week, or a PTaaS subscription where human pentesters step in at each significant release. Gartner places these approaches within a broader practice, CTEM (Continuous Threat Exposure Management).

How much does an automated pentest cost?

External subscription offers start at a few thousand euros a year for a small or mid-sized company and grow with the number of assets. At Flawfence, the subscription starts at €5,000 excl. tax per year. Automated internal pentest tools, deployed inside the network, sit well above that.

Is it legal to run an automated pentest on a website?

Only on systems you are responsible for, or for which you hold written authorization. In France, testing a system without authorization falls under Article 323-1 of the Criminal Code, whether the test is manual or automated. Most jurisdictions have an equivalent. Remember to notify your hosting provider if its terms require it.

Conclusion

"Manual or automated?" is the wrong question, and those who answer it in one word have something to sell (so do we, but you have been warned). The two do not measure the same thing. A manual pentest measures how far an expert can go on a given perimeter, on a given day. An automated pentest measures what is exploitable across your whole perimeter, every day.

If you only have the budget for one of the two, look at where your incidents, and those of your industry, come from. For the vast majority of small and mid-sized companies, it is not a subtle business logic flaw. It is an exposed, forgotten, out-of-date service. That one, a machine finds very well.